* Support drivers that need special handling for literal string expressions
* Enable the :expression-literals feature for vertica
* Enable the :expression-literals feature for redshift
* Review feedback: lift comment about inlined literals from h2 to sql driver
- if you have bb on your path, then this would never be hit.
Testing by uninstalling bb will not uncover this issue if you have
repo/path/metabase/bin on your PATH!
* Rename True and False expressions in literal expression tests
Speculative fix for sqllite failing the tests.
* Update literal expression tests to work with drivers that treat empty string as null
* Add empty-string-literal-expression-test and remove empty string form standard-literal-expression-defs
* Fix logical merge conflicts in case-with-literal-expression-test
* Enable the :expression-literals feature for sqlite and starburst
* Review feedback: drop format-nil->empty-str
As pointed out by metamben in code review `(str nil)` is already the empty string. In fact, the culprit here was a
missing `format-nil-values?` arg to `format-rows-by`.
* Add route prop to disable command palette, Disable in setup routes
* increase wait timeout
* refresh user on setup
* Lets try a unit test instead
* removing refresh user from UserStep
We need to ensure that we consistently have entity-ids in metadata (and potentially elsewhere) even before we've finished backfilling them in. To do this, we check for entity-ids in the field/table/database/card after-select and run calculation/deduplication logic to get them when necessary. We then cache the results (because these calculations can be slow), and a separate job then drains the cache and writes the results back to the db.
* Migrations for database sandboxing (#53125)
Two changes here:
- add a self-referential foreign key,
`metabase_database.router_database_id`, to allow marking a mirror
database's router database. Databases with this set are mirror
databases, databases without it are primary databases.
- add a `db_sandbox` table with only three columns:
- `id`, primary ID,
- `db_id`, the `metabase_database.id` this sandbox is for, and
- `user_attribute`, the user attribute used to map users to a
particular mirror database.
* CRUD API for database routing (#54102)
* Migrations for database sandboxing
Two changes here:
- add a self-referential foreign key,
`metabase_database.primary_database_id`, to allow marking a mirror
database's primary database. Databases with this set are mirror
databases, databases without it are primary databases.
- add a `db_sandbox` table with only three columns:
- `id`, primary ID,
- `db_id`, the `metabase_database.id` this sandbox is for, and
- `user_attribute`, the user attribute used to map users to a
particular mirror database.
* Bulk endpoint for creating mirror dbs
The error case is a bit wonky right now, it just returns the errors for
the details that failed... without the details or name.
Co-authored-by: Ed Paget <ed@metabase.com>
* Add endpoint for marking a database as a RouterDB
Co-authored-by: Ed Paget <ed@metabase.com>
* Rebase fixes
* Exclude mirrordbs from regular db endpoints
* kondo fixes
* mark `DatabaseRouter` as not serialized
* serdes fix
* copy fix
* delete and update routers
Also reorganized things to be a bit less cattywompus.
Now we have:
- `POST api/ee/database-routing/mirror-database` to create a new mirror database
- `POST api/ee/database-routing/router` to create a new router using an
existing database
- `PUT api/ee/database-routing/router/:id` to update a router to use a
new user attribute
- `DELETE api/ee/database-routing/router/:id` to delete a router and all
of the associated mirror databases.
* GET endpoint for mirrors of a particular router db
* Don't test database conn for mirror dbs
---------
Co-authored-by: Ed Paget <ed@metabase.com>
* Modify the API - don't care about database routers (#55036)
* Modify the API - don't care about database routers
Add a `router_user_attribute` to the `GET /api/database/:id` API
endpoint.
Modifies the database routing API to address only by the database ID.
There's now just two endpoints:
- POST /ee/database-routing/mirror-database to create a mirror database
- PUT /ee/database-routing/router-database/:id to change the
`user_attribute` used for routing an existing database. Can be set to
`null` to turn of database routing for the database.
* fix test
* kondo
* DB Routing: Sloan's API improvements (#55438)
* router_user_attribute on `/database/`
* unique database names for mirror dbs
* Don't delete mirror databases when deleting router
* query param to check conn details
* no making a mirror a router
* Require `:manage-database` perms to get dest dbs
(rather than superuser.)
* set `visible-if`, NA on dest dbs
* Add conditions when making db router
* fix db migrations
* fix tests
* kondo
* reorder migrations for mysql/mariadb
* Implement database routing (#55053)
* Swap DB for DB routing
Caching should work, field values does not currently.
* Rename the metadata replacement dynamic var
for clarity. Since we're using it outside of tests now.
* Refactor Advanced Field Values
Previously we had 3 types of Advanced field values:
- sandbox,
- impersonation,
- and linked-filter
Each of these three had their own function, `hash-key-for-*`, and the
caller of this function was responsible for generating the correct hash
key for the field provided.
For example, `hash-key-for-advanced-field-values` took a `fv-type` and
called either `hash-key-for-linked-filters` or `hash-key-for-sandbox` or
`hash-key-for-impersonation` depending on the type.
The big problem here is that the hash keys aren't composable. You can
see this being an issue in `hash-key-for-linked-filters`, where the
enterprise version needs to also check whether the field is sandboxed,
and if so, include sandboxing data in the generated hash as well as the
linked filter data.
If we add more orthogonal things to the hash (like database routing) we
would end up with a rat's nest where each hashing function partially
implements a bunch of other hashing functions to make things work. E.g.
`hash-key-for-database-routing` might check if the field is sandboxed,
and if so include data about the sandbox, but also
`hash-key-for-linked-filters` should check if the field is
sandboxed *or* impersonated *or* subject to db-routing and include data
about those, and ...
Instead, I've replaced all these functions with `hash-input-for-*`
functions, which return self-contained maps that reflect
their *internal* knowledge of what should/should not be cached.
For example, the `hash-input-for-sandbox` function returns either
`nil` (if the field is not sandboxed for the current user) or
`{:sandbox-attributes {<the user attributes used for the sandbox>}}` (if
it is).
`hash-input-for-field-values` simply merges all of these together into
one lump.
To tell whether we're generating a `full` or `advanced` field value, we
can just check to see whether the hash input has any data other than the
field ID.
The one downside here is that we'll no longer be able to look at the DB
and distinguish between different types of FieldValues. As far as I can
tell, the code itself does not ever do this, so I think this is an
acceptable tradeoff.
* database routing for field vals
* move things around a bit
* fix tests
* Lint
* WIP: database routing
* fix test
* more tests, caching tests
* kondo
* stray #p
* Exclude mirror databases from search
* Exclude mirror databases from sync
* Do not create permissions for mirror databases
* fix more tests
* lint
* Add another e2e test for field values
* Don't grab database from query in h2 driver
* Fix my rebase error
* [DB Routing] Implement UI (#55132)
* save
* wip
* wip
* lots of progress
* iteration
* more progress, add delete mirror modal
* type clean up, bug fixes, code organization
* address lots of TODOs
* prevent configuring uploads to routed dbs, prevent db routing if db is upload destination db, replace all usages of routed database and mirror database in favor of destination database
* clean up url usage, add health check to destination database lists
* support hidden fields for database details
* fix types and TODOs
* render warning that model features can not be enabled when db routing is enabled, add unit tests for DestinationDatabasesList
* add e2e coverage for db routing CRUD
* move db routing e2e to a new file, make tests nicer, add test coverage for not showing destination databases outside of database settings
* test that db routing can not be enabled if other conflicting features are enabled
* finish e2e tests as much as possible, add feature token to session properties payload
* move more code into enterprise folder, add redirects and permissions checks for destination db routes, handle no user attributes on instance state, fix failing unit tests, make alerts tooltips when features cant be enabled based on other features being enabled
* test fixes, impl error states kyle provided
* split up tests by ee and oss
* error state updates, fix failing e2e tests
* fix failing oss e2e test, beef up db management perms test
* PR feedback
* impl more feedback
* Fix BE tests
* another be test
* pr feedback
* fix one of the failing e2e tests
* include_mirror_databases -> router_database_id
* skip test
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* fix deletion of router dbs with dest dbs (#55831)
* Don't require superuser perms to get dest db (#55754)
* [DB Routing] Database pages tech debt - RTKQuery + TS migrations (#55744)
* convert DatabaseList to typescript
* make DatabaseEditApp use rtkquery, remove old initialize approach, remove related redux code, port code to use api db rather than metabase lib Database class
* remove dismissDatabaseSyncSpinner redux code in favor of rtkquery
* fix some mistakes causes tests to fail
* port DatabaseList to typescript
* port adding sample database to rtkquery
* misc fixes
* fix failing unit test
* convert admin/databases/database.js and admin/databases/selectors.js to TS
* editParamsForUserControlledScheduling.js to editParamsForUserControlledScheduling.ts
* remove all usages of metabase lib Database class from Database pages
* fix type errors
* fix failing e2e
* prettier
* [DB Routing] Port Admin DB page components to Mantine (#55811)
* port components to mantine for admin db pages
* fix failing tests
* remove unused comment
* fix type issue
* Require opt in/out of database routing (#55841)
I was considering possible errors here and I think this will work well
as a safety harness.
In dev/testing (non-prod) environments, let's add an assertion to the
JDBC driver that if we're about to connect to a database. Either:
- we're *explicitly* trying to only connect to Router Databases. For
example, sync should *never* connect to mirror databases or go through
Database Routing. OR
- we've enabled Database Routing, and we *don't* want to access the
Router Database (unless of course DB routing has been configured to send
us there)
Another way of putting this is that there are two obvious ways for this
feature to go sideways: we accidentally sync a mirror database or do
something else that shouldn't work for mirror databases, OR we
accidentally skip the Database Routing process and connect to the router
database when we shouldn't.
Of course, we can't guarantee that all drivers will call
`(database-routing/check-allowed-access!)` before connecting! But we can
just call it from the JDBC router and that should be sufficient to catch
any issues in development in the vast vast majority of cases.
* [DB Routing] UI feedback part 1 (#55893)
* make dest db validate on creation, invalidate cache of user attributes after user is updated
* add test coverage for updates and uncomment e2e tests parts that were previously failing
* fix type issue
* Fix audit/query logging for DB routing (#56103)
[ADM-585: Handle audit logging and query logging](https://linear.app/metabase/issue/ADM-585/handle-audit-logging-and-query-logging)
* exclude destination dbs from serialization (#56076)
* Decrease the pool size for mirror DBs (#56104)
DB metadata seems like an odd thing - it's the Metabase Database
converted to a snake-hating map, so `router_database_id` becomes
`router-database-id`.
We can't write a function of `:model/Database` that decides whether
we're looking at a mirror database (because we're not looking at a
`:model/Database` here). So we're reduced to just checking the key.
Feels weird, but I think the cleanest way available here?
* Fix sandbox on a routed DB with question as filter (#56393)
This only affected turning on sandboxing, and was not an error for
sandboxing itself.
When we configure sandboxing, we want to make sure the column types for
the query we're using match the columns in the sandboxed table.
As part of this, we ran the query anonymously with
`(request/with-current-user nil ...)`. With Database Routing, however,
we added a new constraint: you can't run any queries against a database
with routing configured anonymously.
Instead just run the query as a superuser. This is effectively the same,
and it's consistent with what we do when actually *enforcing* the row
level restrictions anyway: we preprocess the source query as an admin
and then splice it into the user's query.
The one thing that gives me pause: I'm not really sure why we would have
done `(request/with-current-user nil ...)` in the first place here.
* Add database-router usage e2e tests (#56255)
* Improve error messages and UI for config (#56392)
* Improve error messages and UI for config
- improve error messages when a dest db can't be found
- improve the admin UI for configuring DB routing: https://metaboat.slack.com/archives/C07L6VCCYQL/p1744044752380479
* changes suggested by Jeff
* fix error msg tests
* disable linting for DB Routing description
* fix getting a destination db
db `can-read/write?` use router DB if necessary, since destination
databases don't actually have permissions saved.
* i18n for errors
* spelling 😁
* fix tests
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Fix options for Destination DBs (#56450)
* Fix options for Destination DBs
This may still require a bit of frontend work.
* Set destination-database=true for destination db modal
---------
Co-authored-by: Fraser McIntosh <fraser@metabase.com>
* Fix impersonation with Database Routing (#56503)
* Fix impersonation with Database Routing
Database routing was not working with impersonation, because by the time
the driver calls `set-role-if-supported!` the router database (which is
configured for impersonation) has already been swapped out for the
mirror database (which is not).
To fix this, have a new impersonation middleware set a dynamic var,
`*impersonation-role*`, which is preferred by `set-role-if-supported!`
if set. This way, in queries, `set-role-if-supported!` will use the
pre-calculated role (which seems better than computing it twice anyway).
Co-authored-by: Ed Paget <ed@metabase.com>
* Add impersonation tests
* Add sandbox question test
---------
Co-authored-by: Ed Paget <ed@metabase.com>
Co-authored-by: Fraser McIntosh <fraser@metabase.com>
* keep metadata store on metadata provider swap (#56502)
* keep metadata store on metadata provider swap
Before, we would blow away the existing QP store entirely when using
`*DANGER-allow-replacing-metadata-provider* to swap out the existing
metadata provider.
Instead we can only initialize the QP store when it was truly
uninitialized, and replace *only* the metadata provider when we're
trying to do that.
This is just in case something in QP post-processing needs data from the
original store, e.g. something that was stored with
`qp.store/store-miscellaneous-value!`.
Co-authored-by: Ed Paget <ed@metabase.com>
* only allow replacing md provider once
---------
Co-authored-by: Ed Paget <ed@metabase.com>
---------
Co-authored-by: Ed Paget <ed@metabase.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Fraser <fraser@metabase.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* feat: upgrade ring-jetty to 1.14.1
Moves us onto Jetty 12 which has some features (pthreading, QoSHanlder)
that we may want to take advantage of.
* fixup! customize jetty metrics
* fixup! license info for jetty ee9
* fixup! make a var private
* fixup! don't use deprecated method
* fixup! create custom statistics handler
Replicate the Jetty 11 statistics handler that is able to report on
asynchronously handled requests separately from synchronously handled
ones. Also ports it to directly report statistics to our prometheus
rather than needing to build a connector between the stats handler
object and our metrics reporting
Co-Authored-By: John Swanson <john.swanson@metabase.com>
* fixup! fix race condition in stats test
* fixup! missing private annotation
* fixup! use normal jetty release with logging fix
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* call e2e-component-tests-embedding-sdk-cross-version from
embedding-sdk.yml
* add e2e-component-tests-embedding-sdk-cross-version to the tests-results
so it needs to pass (or be skipped)
* Reset all errored triggers to `WAITING` on startup
Co-authored-by: Ed Paget <ed@metabase.com>
* Add explanatory comments
---------
Co-authored-by: Ed Paget <ed@metabase.com>
Metabase jars would crash when running on JVM 24 because the getReourse
method on the ClassLoader object was changed to return null if there is
no file at the provided path. This adds a ".keep-me" file to the
resources directory that Metabase can use to check if it is running in a
jar.
* fix: resolve snowflake sync cost issue
Using information_schema queries will cause snowflake to wake an
instance and thus incur costs. Using `show` queries will not do so.
In addition this adds a new multi-method for manipulating the results of
`describe-fields-sql`.
* Fix database-position and sorting
* Fix kondo
* Add order-by to mysql describe-fields
* Sort deps
* Use identity not map identity
* Fix databricks
* Address PR comments
* scaffold separate download options for dashboard static embeds
* parse the display options to hash options
* add toggles to the embed preview
* workaround for duplicate types
* cleanup the dashboard workaround
* cleanup the dashcard workaround
* downloads parameter should not show up on OSS
* replace the download button tests
* update existing tests
* fix test for Download buttons
* remove unnecessary fallback
* add a new snowplow schema
* update unit tests
* fix snowplow test assertions
* add snowplow tests
* use results instead of dashcard for consistency
* make the JSON schema backwards-compatible
* split download settings into two components
* use downloadsEnabled.results instead
* snowplow type must be results, not dashcard
* revert StaticEmbedSetupPane changes as it is unused
* create a new field for snowplow tracking
* Enable expression literals on drivers that pass tests
Closes QUE-780
* Update .devcontainer/Dockerfile for java 21 and node 22
This Dockerfile is apparently used by the "metabase assembler", which in turn is used by
.github/workflows/presto-kerberos-integration-test.yml to run the Presto Kerberos tests.
* Add case-with-literal-expression-test
* Review feedback: remove extra whitespace
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* feat: make cluster locking reusable
Builds a more generic interface for cluster locking using our existing
retry mechanism and jdbc level timeouts that will work with any database
rather than just postgresql.
* fixup! handle alt error text on mysql/mariadb
* fixup! don't lock on h2
* fixup! move malli annotations to retry
* Store all native editor selected ranges in redux
* Pass all selected ranges to redux
* Add repro for metabase#54799
* Fix unit tests
* Fix typo
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Do not intersperse space in when running selected text
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
This test was flaking if it was run close to the boundary of a UTC day
where the models could be created one day and then the clock ticked over
to a new day before the query ran.
* Don't update card metadata if it has not changed
* Store stored-metadata as a different field on info
* Fixed failing tests
* Update src/metabase/query_processor/middleware/results_metadata.clj
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
* Addressed review comments
* Update src/metabase/query_processor/middleware/results_metadata.clj
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Addressed review comments
* Update src/metabase/query_processor/middleware/results_metadata.clj
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Addressed review comments
* fixup! exclude info map from legacy normalization
* fixup! test that metadata equality skips update
* fixup! make adding the info map conditional
* fixup! use qp.store instead of adding to info map
* fixup! remove #p
* fixup! remove cyclic dep
* fixup! move load and store fns to same ns
* fixup! test name
---------
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
Co-authored-by: Braden Shepherdson <braden@metabase.com>
Co-authored-by: Edward Paget <ed.paget@gmail.com>
The new `:card_schema` field on `:model/Card` was wrongly excluded from Revisions.
This PR includes them, and adds a default value for old Revisions which predate `:card_schema`.
* fix: snowflake can-connect with ssh without touching other drivers
* Adjust host-and-port for can-connect without calling incorporate-ssh-tunnel-details
We were inadvertently logging db-details in the rarely used
`maybe-test-and-migrate-details`. This changes to more of a data
diff where we can still get important information on how it changed.
* Rename checkCompiledExpression to diagnoseExpression
* Add tests for arg count validation
* Add arg validator tests
* Add test for unsupported functions to diagnostics
* Add isCallExpression matcher
* Add visit helper
* Move unsupported function check to diagnostics
* Move arg validator to diagnostics
* Move arg count validation to diagnostics
* Remove EQUIVALENT_CLAUSES since we have them in config.js
* Remove database argument where no longer needed
* Add COMPARISON_OPERATORS to config
* Add LOGICAL_OPS to config
* Add NUMBER_OPERATORS to config
* Add FIELD_MARKERS to config
* Move comparison validator to diagnostics
* Simplify resolver
* Move checkers before Lib.diagnoseExpression
* Fix diagnoseExpression return type
* Move case or if operator argument check to diagnostics
* Remove unused tokenName
* Make expression types more specific
* Convert resolver to typescript
* Add test for multi-arg infix operators
* Add checker for unknown functions
* Add more tests for coalesce typing
* Add more tests for comparison operator resolving
* Add test for number operators
* Add test for logic operators
* Remove branches that are handled in the config already
* Allow mbql clause config to define the arg types
* Allow case/if to define argType too
* Move getNode and getToken to utils
* Make resolver fn required
* Validate field names in resolver
* Move trim into getMBQLName
* Fix top-level literals
* Handle expression refs
* Allow number operators to be multiple
* Add more tests for custom arg validators
* Test operator resolvers more
* Add more tests for multi arg functions with options
* Use better formatting in test
* Be more generic in getNode and getToken
* Use correct location for errors
* Make it work
* Use Mantine
* Update tests
* Display db type instead of mbql type
* Show db type instead of semantic type
* Revert "Show db type instead of semantic type"
This reverts commit 5a51c5abca.
* Update tests
* Prevent CLS
* Revert change
* Revert "Prevent CLS"
This reverts commit f74b6e80af.
This commit introduces empty states for visualizations
in order to help less experienced users create their first chart.
These new empty states come in two flavors:
1. With a "Summarize" button CTA that will open the summarize sidebar
2. With a link to our helpful documentation (for more complex charts)
We have been previously relying on the `placeholderSeries` dummy data to display a greyed out chart preview when we cannot render a chart. That functionality is now replaced with a new property called `hasEmptyState`.
`checkRenderable()` util will try to render a chart based on the series and settings. If it is unable to do so, it will throw a `ChartSettingsError`. This, in combination with hasEmptyState and a few other properties, is a cue to render a new, empty state for that visualization.
* Initial BE changes for custom expression literals (#55206)
* Empty
* Add `value` support in the expression editor (#55203)
* Literals in filters tests (#55283)
* E2E tests for literals in custom columns (#55288)
* BE changes for :value wrapped expression literals
* Extend BE test with more literal values
* Add float value to qp test for expression literals
* Add Float and Boolean cases to literal-expression-test
* Remove ^:internal from value clause
:value clauses are now generated by the frontend when creating a top-level expression literal and preserved when
converting legacy <-> pMBQL.
* Add some lib.convert tests for literal custom expressions
* Add legacy schema tests for literal custom expressions
* Add normalize / canonicalize tests for literal custom expressions
* Remove vestigial "date" expression from fields clause in query-with-literal-custom-expression-test
* Fix literal-expression-test for cljs
CLJS parses 1.00 as 1 and hence :type/Integer not :type/Float
* Fix js_test expression-clause-<->-legacy-expression-test for wrapped literals
* Move QP tests into expressions_test.clj and add more tests
* Remove commented out :value and update comment in stage-expressions->legacy-MBQL
* Restrict QP literal expressison tests to supported drivers
* Update boolish->bool to accept BigDecimal inputs
Apparently some DBs return BigDecimal results for boolean expressions.
* Temporarily remove literal-expression-case-test
Needs more debugging
* Move aggregations tests into expression_aggregations_test.clj
* Add some basic filter tests to QP expressions_test.clj
* E2E tests for literal custom columns in other clauses and fixes (#55295)
* E2E tests for literal custom columns in other clauses and fixes (#55295)
* Move changes from the FE branch
* Move changes from the FE branch
* Move changes from the FE branch
* Fix filter schema and false = false filter (#55417)
* Fix date/string literals effective type (#55427)
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix malformed order-by clauses
* Remove tests for offset expressions as non-aggregation expressions
These tests were initially added in #42302, but we disabled offset in custom columns in #43224.
Remove these tests so we can re-enabled nest-expressions when expressions appear in an order-by clause.
https://metaboat.slack.com/archives/C0645JP1W81/p1742317450874479
* Add order-by to the list of places to check in should-nest-expressions?
This is to support order-by of custom expressions, in particular literal custom expressions.
https://metaboat.slack.com/archives/C0645JP1W81/p1742317450874479
* Rename round-trip-expression-literal-test and move it next to existing literal tests in the same file
* Uncomment filter-literal-boolean-expression-with-no-operator-test
The schema was relaxed in bd82763beb, so now the test passes.
* Uncomment filter-literal-expression-with-and-or-test
* Add round-trip-filter-expression-test
* Add more cases to legacy schema filter-test
* Remove unused import
* Review feedback: set/difference -> disj
* Add :expression-literals driver feature (#55662)
* Add the :expression-literals database feature
And enabled it for postgres, mysql, bigquery, and snowflake.
These are the tier 1 and tier 2 DBs, minus redshift which is currently failing some tests.
* Add lib.metadata/database-supports? helper
* Check if :expression-literals feature is enabled in diagnose-expression
* Use standard-literal-expression-values in order-by test expectation
* Add comment about compiler warning to boolish->bool
* Add breakout-by-literal-expression-test
* Add expression-literals-test to nested-queries-test
* Add compound expressions involving literal custom columns to e2e filters tests
* Use lib.schema.common/is-clause? to detect :value expr in diagnose-expression
* Review feedback: fix order-by-integer-literal-expression-test
* Review feedback: conj -> vector
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Add explicit CAST when compiling string literal expressions in H2 driver (#55920)
* Add explicit CAST when compiling string literal expressions in H2 driver
Closes QUE-726
* Add e2e test for nested question with literal custom columns
* Add :expression-literals feature to various mock providers used in tests
* Update diagnose-expression tests
* Ensure legacy :value options are normalized in mbql.normalize
Values coming from the FE in expression-clause-for-legacy-expression call normalize and expect to have :value options
converted from strings to keywords.
* Add base_type to value literals in adjustTopLevelLiteral
* Consistent spelling of base_type
* Use lib.metadata/database-supports? to check for database :features (#56114)
* Use lib.metadata/database-supports? to check for database :features
Closes QUE-836
* Convert a couple more tests to use meta/updated-metadata-provider
* Add e2e test to ensure that sandboxing on literal custom column fails closed (#56190)
* Add e2e test to ensure that sandboxing on literal custom column fails closed
Closes QUE-844
* Review feedback: dedup CustomColumnType definition
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* scrape releases page to find latest major version
e.g.:
```
$./bin/mage jar-download 50
Warning: JARS not found in env.
Looking for latest version of 50 on release page 1
Downloading from: https://downloads.metabase.com/enterprise/v1.50.35/metabase.jar
to: /Users/bcm/dv/mb/metabase/jars/metabase_1.50.35.jar ...
Downloaded 1.50.35.jar to /Users/bcm/dv/mb/metabase/jars
```
* better feedback
* Make segment name clickable
* Upadte e2e test
* Convert SegmentItem to TS
* Modernize SegmentItem
* Add cells
* Add TableBreadcrumbs component
* Use TableBreadcrumbs in SegmentEditor
* Use real data
* Improve data handling
* Handle ellispis
* Add tests
* Add test for no schema
* Rename
* Prettier
* Update icon color
* Fix icon shrinking
* Use TableBreadcrumbs in SegmentItem
* Fix icon shrinking
* Update max width
* Update tests
* Sort
* Remove css file
* Add TableBreadcrumbs component
* Use TableBreadcrumbs in SegmentEditor
* Use real data
* Improve data handling
* Handle ellispis
* Add tests
* Add test for no schema
* Rename
* Prettier
* Update icon color
* Fix icon shrinking
* Remove css file
* scaffold setting up individual downloads
* use downloadsEnabled as boolean for result downloads
* allow SDK to use a single withDownloads props for compatibility
* sdk static dashboard should continue to support single withDownloads prop
* update downloads prop to be string compatible
* fix the areDownloadsEnabled matcher
* refactor download plugin matcher
* do not modify EmbedDisplayParams for now
* fix misleading types
* remove ConnectedDashboard default
* undo LookAndFeelSettings
* revert embed flow and analytics changes
* add test cases for public resource downloads
* add test cases for public question download
* do not support PDF downloads on questions
* remove redundant showDashboardCardActions
* create types for download results
* rename cardResult to dashcard
* replace cardResult with dashcard
* add downloadsEnabled to storybook
* update unit test for areDownloadsEnabled
* update unit test for no-token case
* update unit test for oss case
* try to disable downloads in storybook
* update story defaults
* make downloadsEnabled a component default
* add default to downloadsEnabled in PublicOrEmbeddedDashboardView for loki
* revert changes as loki diffs seems good now
* fix loki diff issue in stories
* Empty
* Date generalized tests (#55953)
* Centralize and generalize tests for `integer()`
Also make room for other casting functions to be similarly tested
* Remove unsued imports
* Move postgres date cast test into cast_test.clj
* Rename :cast to :expressions/date
* Remove old imports
* Remove unused features
* Remove more!
* Fix test files
* Centralize and generalize tests for `integer()` (#55950)
* Centralize and generalize tests for `integer()`
Also make room for other casting functions to be similarly tested
* Remove unsued imports
* Example tests for integer()
* Remove empty line
* Use field type
* Appease the Kondo
* Check for Number instead of Integer
* Cast to big-integer before comparing (for snowflake)
* Remove split-part-test. It doesn't belong in this branch.
* Rename :cast to :expressions/integer
* Fix parens after merge
* Unified text() tests
* Appease the kondo
* Appease the whitespace demons
* Oops, wasn't filtering for drivers
* Use literal string for redshift
* Add a repro for #55951
* Fix typos
* Refactor repro
* Fix#55951
* Remove unused components
* Use 'loading'
* Format code
* Fix test setup
* Rename isLoading back to isSearchLoading since it indicates Search.loadList usage
* Use 1 /search API call instead of 2
* Get rid of rawData
* Update unit tests
* fix: deadlock in last used at
This prevents bulk updates to last used at from deadlocking by creating
a new table in metabase's database that can be used by instances to take
cluster wide locks. It adds an initial implementation of a customizeable
db-based locking mechanism.
For the last_used_at update, a metabase instance will first set a
session variable to control locking timesout then take a cluster lock
using SELECT * FROM metabase_cluster_lock WHERE lock_name = 'name' then
execute the update_at.
If another table is holding this lock metabase will wait .5s for the
lock to be released then timeout its SQL transaction, sleep the Thread
for 1s then retry taking the lock. This makes sure we're not holding DB
connections waiting for locks.
* fixup! kondo
* Update resources/migrations/001_update_migrations.yaml
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update src/metabase/query_processor/middleware/update_used_cards.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* fixup! finish method rename
---------
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Fix formatter support for when both field ids and col names are used in viz settings
* Apply fix in qp common as well, which date formatter consumes
* Addressing PR feedback and breaking up large test
* Fixing or->merge reversion; adding datetime formatter tests
* Improving viz setting merges based on types of field_ref and column setting keys
* Addressing PR feedback
* Unified tests for `splitPart()`
* Remove postgres specific split-part-test
* Remove Redshift split-part-test
* Remove snowflake split-part-test
* Remove mysql split-part-test
* Remove bigquery-specific split-part test
* Fix MySQL implementation of splitPart()
* Only allow string literals for 2nd and 3rd args of replace()
* Only allow non-empty string for delimiter of splitPart()
* Better non-blank string
* Make position in splitPart() positive literal or numeric expression
* WIP new examples tests + corner cases
* Oops, go back to non-empty string
* Improve tests
* Improve test message
* Empty string on non-positive position in splitPart() for mysql and postgre
* Fix Redshift error
* Snowflake: return empty string with non-positive position
* Also test zero position
* Correct misplaced paren
* Fix `splitPart()` for snowflake
* Nicer comment on weird mysql implementation
* Rename positive-integer schema
* Clearer parameter comments
* dont skip mage tests when BE job is skipped
* checkout metabase for mage tests
* make mage tests its own thing
* add mage test workflow
* if it aint broke dont fix it
* remove dupe mage test from backend workflow
* fix: correct setting name check for static stats
also fix namespace for the defenterprise in token check to load the
embedding stats module
* fixup! use get-raw-value
* fixup! use-get-value-of-type
Fixes [QUE-838](https://linear.app/metabase/issue/QUE-838/linked-filters-are-not-working-on-public-dashboards).
Query params were getting parsed incorrectly to a map like `{:17fdc812 ...}`
with a keyword rather than a string. These failed to match any of the
dashboard's parameters (which are keyed by strings like `"17fdc812"`)
and returned no values (but did not throw any errors in prod).
Before, when you had an outer query referring to something in a nested field from the inner query, the query processor
would assume the field is still nested and generate invalid sql. Now, it will just refer to the alias.
* Move the lexifier to it's own file
* Remove duplicate type union member
* Remove unused typeinferencer
* Add ParseError type to errors
* Lexify straight in to pratt.Token
* Remove unused TOKEN type
* Use pratt.Token type everywhere
* Fix type
* Remove unused __support__/expressions helper file
* Ignore END_OF_INPUT when matching suggestions
Fixes#55929. (Requires a new export for 54 that also gets backported.)
Adds a new setting `install-analytics-database` that skips loading the
database at all, if set to false. That allows the dev instance that
exports the analytics content to have no real audit DB, and therefore to
generate the right `entity_id` hashes for the audit export!
Existing audit DBs are fine; the checksum will see these new entity_ids
in the export and overwrite any existing content.
* Implement splitPart() for BigQuery
* Always return string instead of null
* Better test failure message
It's running fine on my machine ;)
* Use coalesce instead
* fix: Ensure that local timestamps params work
Fixes#53371
By filtering a timestamp or date column on an OffsetDateTime it results
in an index being missed since the lhs needs to be upcast by the db engine
to a `timestamp with timezone`.
This ensures we only pass OffsetDateTime to `timestamp with timezone`
columns.
Context about decision here:
https://metaboat.slack.com/archives/C0645JP1W81/p1743111964032089
* Handle sending a datetime range rather than just a date
* Feature gate to dynamic dataset loading
* Fix kondo
* Remove comment
* Rename caseInsensitive to caseSensitivity
* Use literal string for case-insensitive template
* Update argument description to be more clear
* Support inline expression highlights
* Render help text description with Markdown
* Add more markdown to help text strings
* Use markdown code blocks to mark code in help text descriptions
* Highlight argument references in help text descriptions
* Fix arg alignment
* Fix HelpText unit tests
* Fix FunctionBrowser tests
* Render FunctionBrowser descriptions using Markdown too
* Mark true
* Fix e2e tests
* Fix e2e tests
* Remove commented-out code
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
---------
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Auto-categorize truncation units as datetime columns in insights code
* detect datetime units on the FE as well when rendering trend visualization
* prettier
* add unit test
* typo
* fix comment alignment
* Initial support for text wrapping in static viz
* Add text-align support
* Add row index support
* Fixing broken unit tests
* Adding new unit test coverage for Tanstack viz settings
* Fixing typo in whitespace; removing debug def
* Removing defs & outdated comment
* Addressing PR feedback
* Adding column width support to minibar columns; improving doc strings
* Adding minibar column width support; improving doc strings; moving consts to style namespace
* Fixing broken unit test after font size change
* Remove trailing whitespace
# Support Environment Variables for API Keys
### Overview
This PR adds support for using environment variables to define API keys in the configuration file, enhancing security and flexibility for deployment environments.
### Changes
- Modified API key validation to support environment variable substitution in the key field
- Added runtime validation for API keys from environment variables to ensure proper formatting
- Refactored tests to properly validate environment variable integration
- All API keys (including those from environment variables) still must follow the required format:
- Must be between 11-254 characters
- Must start with mb_
- Must match the pattern mb_[A-Za-z0-9+/=]+
### Usage
```
api-keys:
- name: "ENV API Key"
key: "{{env API_KEY_FROM_ENV}}"
creator: "admin@example.com"
group: "admin"
```
### Testing
- Creation of API keys from environment variables
- Validation of API key format from environment variables
- Error handling for improperly formatted environment variable values
# Skip keys if they exist
To guarantee that it is possible to keep the same yaml file on each instance initialization, if there is a key in the instance and the config file has a key with the same name, the creation is skipped.
* Minibar chart support in table static viz
* Fixing unit test; adding new minibar unit test
* Fixing clj kondo warnings
* Addressing PR feedback
* Removing dead minibar code
* Removing old minibar unit tests
* Empty
* Implement and test snowflake integer()
* Use card-with-source-metadata-for-query in snowflake tests
* Use lib/functions instead of lib/expression-clause in snowflake
* integer() for BigQuery
* Fix BigQuery tests for integer()
* Change to backticks
* kick ci
* Fix for BigQuery
* Capitalize name of field
* Use STRING type for casting in BigQuery
* Sometimes we get an integer back, and that's okay
* Use mt/card-with-source-metadata-for-query in bigquery tests
* use lib/functions instead of lib/expression-clause in bigquery
* feat: add remaining token check stats and settings
Adds the remaining five data points we want to report during the token
check request. Adds a new module to internal stats that does the
embedding counts and setting checks.
* fixup! log stats
* fixup! move enterprise settings into defenterprise
* fixup! remove row count
* fix: only remove vis_settings from dashboardcard
Fixes loading serialized dashboard cards where we would previously
delete and recreate them on each load.
In order to handle circular dependencies caused by links to other
dashboards in a dashboardcard's visualization_settings we were removing
the dashboardcards entirely from the dashboard model to break the cycle.
Later when we traversed back through the dependency path we would load
the original dashboard a second time with the restored dashboardcards.
However, on the load of the cycle breaking change we could entirely drop
dashcards belonging to the dashboard because the serialization code was
treating our cycle breaking changes as an edit to the dashboard to drop those
cards. This meant when we got back to the the second load of the
dashboard we would create the dashcards fresh instead of updating
existing instances.
The fix here is that we only remove the values from the dasboard card
model that can cause the cycle -- in this case just the
visualization_settings column. This seemed the most straightforward fix,
but there are other possibilities here.
One is that we could _just_ preserve the entity_id column in dashcards
which would eliminate the possibility of any other cycles here.
Another option would be to change how we handle nested models in serialization.
There is not really a foreign key dependency here between dashboard A
and dashboard B linked by the dashboard card. If we instead just created
those dashboards first _then_ went back and created their dashcards we
don't have any problems at all. We could move nested models up to the
same level of handling in serialization that we do for models, which
would let us create the dashboards first then create any dashboard cards
in a subsequent step.
* fixup! update docstring
Closes QUE-794 and QUE-705.
This will enable us to *know*, rather than *infer*, what version of
`report_card` we're looking at.
One example: in the near future, I'm going to add some values to
`result_metadata` on cards. If we later discovered a bug in that logic
and could no longer trust those fields written to AppDB in
`result_metadata`, the `card_schema` will tell us with certainty
whether the card was saved with the broken logic (`:card_schema N`)
or the new logic (`:card_schema (+ N k)`).
The upgrade steps could drop the invalid fields and, if desired, infer
the correct ones.
Note that there's nothing here about writing back! This is about
cleaning up old cards during `after-select`.
There are no upgrades yet; the current version is still the default.
```
2025-03-27 15:25:21,322 INFO sync.describe-database :: Checking "PUBLIC"."LARGE_NUMBERS"
2025-03-27 15:25:21,757 INFO sync.describe-database :: "PUBLIC"."LARGE_NUMBERS": SELECT privileges confirmed
2025-03-27 15:25:21,759 INFO sync.describe-database :: Checking "PUBLIC"."SINGLE_ROW_TABLE"
2025-03-27 15:25:21,975 INFO sync.describe-database :: "PUBLIC"."SINGLE_ROW_TABLE": SELECT privileges confirmed
2025-03-27 15:25:21,976 INFO sync.describe-database :: Checking "PUBLIC"."SLOW_QUERY_VIEW"
2025-03-27 15:25:22,219 INFO sync.describe-database :: "PUBLIC"."SLOW_QUERY_VIEW": SELECT privileges confirmed
```
This will list all tables it enumerates and then which ones it
removes. There are also two error handling states:
```
2025-03-27 15:29:25,418 INFO sync.describe-database :: "schema"."some_table": SELECT privileges confirmed
2025-03-27 15:29:25,419 INFO sync.describe-database :: "schema"."some_table": Assuming SELECT privileges: caught timeout exception
2025-03-27 15:29:25,419 INFO sync.describe-database :: "schema"."some_table": Assuming no SELECT privileges: caught exception
java.lang.Exception: Some exception
at metabase.driver.sql_jdbc.sync.describe_database$eval194718.invokeStatic(NO_SOURCE_FILE:33)
at metabase.driver.sql_jdbc.sync.describe_database$eval194718.invoke(NO_SOURCE_FILE:33)
[more stacktrace from the exception]
```
This is essentially just changing the log level. We can and perhaps
should change these back to debug or trace in the future but this is
useful for now.
* WIP Implement `integer()` for all :sql descendants
* Add test-data to tests
* Just do MySQL
* Undo change to query_processor_test
* Remove unused imports
* Remove extra blank line in query_processor
* Convert postgres tests to mysql
* Appease MySQL syntax
* Remove old tests for mysql integer()
* Remove unsupported postgres tests
* Fix kondo
* kick ci
* Fix up for MariaDB
* Fix queries in test
* Try bigger numbers for MariaDB
* Remove ability to cast integer to integer
* Check that boolean = 1 in mysql
* Use mt/card-with-source-metadata-for-query
* Use lib/functions instead of lib/expression-clause
* Use lib/functions instead of lib/expression-clause in postgres
* Use card-with-source-metadata-for-query for postgres tests
* Update some function names to match modern casing
* Fix test cases
* Update casing in docs for isEmpty, notEmpty, lTrim, rTrim, isNull, and notNull
* Fix isNull and isEmpty casing in tests
* Fix case in unit test
* Fix casing in e2e tests
* Fix couldn't find Slack channel
When the first `conversations.list` doesn't return the Slack channel we
want, we need to continue to paginate until we found it.
* [debug] for testing
* Revert "[debug] for testing"
This reverts commit 67dd2c0405.
* Simplify logic
Co-Authored-By: Timofey Kachalov <timofei@metabase.com>
---------
Co-authored-by: Timofey Kachalov <timofei@metabase.com>
* rename and reorganize
# Conflicts:
# enterprise/frontend/src/metabase-enterprise/google_drive/GdriveConnectionModal.tsx
# enterprise/frontend/src/metabase-enterprise/google_drive/GdriveSyncStatus.tsx
* resolve conflicts
* only show sync status to admin who created it
* Add connection info to disconnect modal (#55588)
* rename and reorganize
# Conflicts:
# enterprise/frontend/src/metabase-enterprise/google_drive/GdriveConnectionModal.tsx
# enterprise/frontend/src/metabase-enterprise/google_drive/GdriveSyncStatus.tsx
* resolve conflicts
* update disconnect modal
* better spacing
Improve the performance of the collection permission query by using a
semi-join instead of an inner join to the permission tables. This
ensures each collection only needs to be tested once for a route that
gives a user permission to access it rather than multiple times.
This reduces the query time for users that are members of lots of
collections since we dont need postgresql to create large join tables or
return the same collection id more than once from this query.
* test: make dashboard flaky test more robust and stable
* revert flaky
* add additional check
* fix that works locally
* revert prev version
* add missing line
* Add baseType prop to SemanticTypePicker
* Update unit tests
* Fix legacy v1/ Field['base_type'] typing, to make it same as Field['base_type'] from api/
* Use isa
* Add a hack for numerical types
* Fix typo
* Pass field
* Reuse type
* Use effective type
* First pass on the tech doc
* Handle booleans
* Use isa
* Refactor
* Update test
* Update test
* Assume no type can have 2 level-one ancestors
* Add comments
* Extract getCompatibleSemanticTypes
* Refactor isDeprecated
* Refactor
* Add fieldId param
* Add test for booleans
* Make initialValue required
* Do not assume a field cannot have multiple level-one ancestors
- these types have 2 level-one ancestors: type/JSON, type/XML, type/DruidJSON, type/OracleCLOB
* Extract FIELDS
* Add test
* Test Entity Name
* Remove '
* Add test
* Update test
* Add test for base_type fallback
* Update comment
* Refactor tests
* Add Field containing JSON tests
* Refactor
* Add a comment
* Improve naming
* Improve naming
* Use SemanticTypePicker in DatasetFieldMetadataSidebar
* Use Field from types/api
* Extract DatasetFieldMetadataSemanticTypePicker
* Fix typing
* Pass label, tabIndex, and onKeyDown
* Remove old picker
* Use SemanticTypePicker in FieldTypeDetail
* Remove unused prop
* Remove redundant margin
* Fix#55617
* Use SemanticTypePicker in Field
* Remove old SemanticTypePicker
* Reuse FkTargetPicker
* Introduce new top-level frontend directory: frontend/src/metabase/metadata
- Move semantic type picker + subcomponents in there
* Add readme
* Get rid of global CSS classes
* Remove old FKTargetPicker component
* Pass databaseId to components
* Don't check for text-like fields
* Merge branch 'SEM-97-limit-semantic-types' into SEM-176-new-semantic-type-picker-everywhere
* Update test
* Update tests
* Sort options by label
* Update test
* Fix docs links
* Change reproductions suite to TypeScript
* Repro #55618
* Repro #55617
* Make sure combobox flips if there is no space underneath
* fix: add index to field_usage query_execution_id
Supports faster deletes on the query_execution table. Without this index
deleting rows from that table leads to a seq scan on the field usage
table for each row deleted to handle the ON DELETE CASCADE trigger.
* fixup! drop empty preconditions
* Add baseType prop to SemanticTypePicker
* Update unit tests
* Fix legacy v1/ Field['base_type'] typing, to make it same as Field['base_type'] from api/
* Use isa
* Add a hack for numerical types
* Fix typo
* Pass field
* Reuse type
* Use effective type
* First pass on the tech doc
* Handle booleans
* Use isa
* Refactor
* Update test
* Update test
* Assume no type can have 2 level-one ancestors
* Add comments
* Extract getCompatibleSemanticTypes
* Refactor isDeprecated
* Refactor
* Add fieldId param
* Add test for booleans
* Make initialValue required
* Do not assume a field cannot have multiple level-one ancestors
- these types have 2 level-one ancestors: type/JSON, type/XML, type/DruidJSON, type/OracleCLOB
* Extract FIELDS
* Add test
* Test Entity Name
* Remove '
* Add test
* Update test
* Add test for base_type fallback
* Update comment
* Refactor tests
* Add Field containing JSON tests
* Refactor
* Add a comment
* Improve naming
* Improve naming
* Use Field from types/api
* Fix typing
* Don't check for text-like fields
Previously we looked at 1000 users and got the set of unique user
attribute names from them. So if 1000 users each had a `name` and user
1001 had `some-other-attribute`, we would give you back `name` as the
complete set of available attributes.
This modifies the magic number (to 5000 from 1000) and the logic. We
select the first 5000 unique *attributes*. So if 1000 users each have a
`name`, that only counts as 1, and we'll continue looking for other
unique attributes before returning to the client.
Implemented validation that ensures API keys when created with config.yml:
Start with the mb_ prefix
Have a minimum length of 12 characters and max 254
Prefix (first 7 characters) must be unique
String must be Base64
* Use field refs instead of field ids/names for tiles queries
* Fix up embedded + public endpoints
* use field_refs
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
* resetting qb state before running card on back navigation
* try something else
* adding e2e test for notebook navigation
* wait for fk call in test
* increase timeout
* e2e adjustment
* Control visible entities in multi-stage data picker (#54845)
* Add props to DataSelector, and keeps the same behaviors.
* Read entity_types from URL search params
* Fix type errors
* Fix SDK not rendering multi-stage data picker
* Add redux slice tests
* Add E2E tests
* WIP add integration tests
* Add all integration tests
* Put tests file in a proper folder
So the SDK build process doesn't check test file types
* Cover more E2E scenarios
* Cover the new code with a test
* Add more assertions to make sure more paths are tested
* Fix imports
* Make functions easier to understand
* Fix test case name typo
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Clarify embed state type
* Address review: Make code more concise
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Control visible entities in simple data picker (#55329)
* Filter simple data picker by "entity_types"
* Add a happy path for multi-stage data picker E2E test
* Fix loading a wrong entity
* Control visible entities on new item menu dropdown (#55331)
* Control visible entities on new item menu dropdown
* Hide action following product discussion
* Make a new menu item more accessible
* Add tests
* Control visible entities in Browse sidebar (#55332)
* Control visible entities in Browse sidebar
* Add relevant tests
* Fix test case typos
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Add comment to clarify magic value
Co-authored-by: Nicolò Pretto <info@npretto.com>
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Empty
* Add split-part backend (#55425)
* feat: prepare fe part of split-part (#55483)
* Add split-part backend
* feat: prepare fe part of split-part
* rename function and add test
* use postgres for testing
* add test for an error
* add db flag
* remove docs page
* update copy
---------
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
---------
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
* feat: prepare fe part of url-pathname
* fix dropdown styling
* fixup example
* rename function to path and update example
* Backend for url-pathname
* Rename to path()
* Backend tests for `path()`
* Frontend and Backend for url-pathname (#55353)
* feat: prepare fe part of url-pathname
* fix dropdown styling
* fixup example
* rename function to path and update example
* Backend for url-pathname
* Rename to path()
* Backend tests for `path()`
---------
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
* Fix kondo
* Fix kondo
* feat: prepare fe part of url-pathname (#55291)
* fix description
* fix styling
* cleanup
* fix rebase
* fix rebase
* add tests
* add tests
* update test
* fix test
* address review
* prevent url translations
* make tests less flaky
* update copy
* add category
* remove docs page and add feature flag
---------
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Convert typeinferencer to typescript
* Split up diagnostics
* Split of lib check from diagnostics function
* Move getDatabase to util file
* Move isErrorWithMessage to utils
* Add getExpressionMode helper
* Simplify branches
* Move resolveMBQLField to separate module
* Simplify expressions import
* Remove unused options
* Fix type of mock env
* Move compiler to separate file
* Remove unused arguments to compiler
* Move passes to separate file
* Return expression from diagnose
* Remove unused processSource helper
* Fix types in passes
* Make resolving optional in compiler
* Add fuzz testing for compiler
* Mock legacy mbql to clause in fuzz test
* Remove need for compare
* Use default parameters
* Remove compare and compareSeed
* Remove unused recursive-parser
* Remove fuzz testing for recursive parser
* Remove unused arguments
* Colocate helper with the tests that use it
* Use compileExpression in formatter tests
* Use compileExpression in typeinferencer tests
* Order compiler passes
* Move StartRule and ClauseType to metabase-lib/v1/expressions
* Remove useShorthands pass
* Fix token escaping in pratt compiler #54751
* Remove POPULAR_ functions and aggregations
* Move compiler passes out of pratt compiler
* Make getMBQLName optional
* Add missing adjustBooleans pass
* Type startRule in compiler
* Move resolverPass to field-resolver
* Add isCompilerPass helper
* Allow use of short-circuiting for disabling passes
* Only render friendly errors
* Mark ExpressionError and ResolverError as user-friendly
* Add check for missing comma in argument list
* Simplify diagnostic checks
* Always return error and expression
* Make compileExpression and diagnoseAndCompile generic to StartRule
* Remove diagnoseAndCompileExpression helper
* Remove unused completer utils
* Move enclosingFunction to expressions module
* Fix typeinferencer tests
* Fix e2e test
* Detect cycles
* Inline passes
* Flatten compiler
* Remove prevError
* Remove default from diagnoseAndCompile type definition
* Use try/catch to simplify diagnoseAndCompile
* Wrap diagnostic errors in their own type
* Remove unnecessary wrapping of diagnostic error
* Move errors to top-level expression module
* Remove duplicate error types in expressions
* Fix error format in tests
* Add comments to passes
* Add tests for passes
* Fix lint
* Fix type imports
* Flesh out references in test data with base-type
* Use fleshed out field reference in all test cases
* Add boolean expression to test data
* Port recursive-parser tests for compiler
* Fix support for unary +
* Add cast support to test database
* Move cast calls to separate test
* Add test for escaping identifiers
* Unskip test
* Fix compiler tests to support top-level literal
* Typo
* Use normal quotes
* Fix typeinferencer for value clause
* Use all passes by default
* Fix pratt compiler tests
* Unskip garbage test
* Use dedent from ts-dedent
* Remove ClauseType helper for expression and make everything use Lib.ExpressionClause
* Coerce clause to be expression clause in filter and aggregation pickers
* Fix diagnostic test
* Wrap diagnostic error in a DiagnosticError
* Use instanceof to check type of error
* Do not assume all diagnostic errors are human-friendly
* Update translation sources 2025-03-17
* Update translations
* don't add lithuanian yet
* language is always locale
* Fix the `ttag` i18n calls for pluralized strings
Pluralized strings *without* the number interpolated into them (eg.
"Month" and "Months") were not getting found by `i18n/trun` in CLJS,
which made them fall back to English.
This fixed the `ttag/msgid` call to be correctly formed in both cases.
* update some localized strings
---------
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
Co-authored-by: Braden Shepherdson <braden@metabase.com>
We are running a probe query on snowlfake to see if the current db user
can see views
```
select true from X where 1 <> 1 limit 0
```
Something like this.
But on large views, it seems snowflake does not quickly determine this
has no rows and it can take a while.
This leads to the connection being closed. Then in the error handler we
attempt to rollback. But if the connection is closed _this_ throws and
we have now thrown from the error handler and we break sync.
This does two things
1. set a timeout on sql statements. This does 10 seconds.
2. If a timeout occurs, assume that they _can_ see the object
The idea is that timeouts might happen but it's trying to run a query
that it would quickly reject if the permissions were lacking. So
timeouts mean you can see the data, it's just taking a while
* feat: add embedding stats to token request
Adds the embedding_users, internal_users, domains, and query_execution_*
stats to the token check request in order to track embedding usage.
This creates a new internal_stats module to share code between the
anonymously reported stats the stats for the token check request to
avoid some circular depedencies. This also seperates creation of a stat
and how it is reported, so we could move over other stats calculated for
the phone home metric.
* temp! add a logging statement to figure out why CI is different
* fixup! exclude api-key email
* fixup! exclude api-key/internal type users
* fix localhost:* in embedding app origins sdk
We need to call the setting's getter directly, and not use
`setting/get-value-of-type`. The setter it adds localhost:* which is required.
* add test
* test fix
* update module config
* improve test feedback
* fix ci
* use whitelabel colors in the static viz
* Fix backend whitelabel test for app colors
* update CI
---------
Co-authored-by: Tyler Plude <tsplude@gmail.com>
* Fix cond fmt of null vs empty str in table static viz
* Adding unit test
* Add TextWrapper support in json exports
* Updating unit tests with TextWrapper
* Empty
* Add `value` support in the expression editor (#55203)
* Literals in filters tests (#55283)
* E2E tests for literals in custom columns (#55288)
* E2E tests for literal custom columns in other clauses and fixes (#55295)
* Move changes from the FE branch
* Move changes from the FE branch
* Move changes from the FE branch
* Fix filter schema and false = false filter (#55417)
* Fix date/string literals effective type (#55427)
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Fix date/string literals effective type
* Forbid top-level constants until the BE is fixed (#55459)
* Use matches
* Make test flaky
---------
Co-authored-by: Mike Appleby <86076+appleby@users.noreply.github.com>
* ci(sdk): adds embedding-sdk-tests-result
* ci: should run the sdk tests as part of the embedding-sdk workflow
* let's try without concurrency
* rename "frontend" -> "embedding SDK"
* Integration
* Support for `text` MBQL function on the FE (#55274)
* Cast text backend on postgres (#55292)
* Add `text()` custom expression that casts to text
* Add :cast to list of all possible features
* Complete testing plan
* Fix kondo warnings
* Fix test
* Empty
* Add e2e tests (#55361)
* Fix failed tests for text() function (#55374)
* Add UI for integer() function, with e2e tests (#55366)
* Set category
* Set category
* Set category
* Set category
* Fix tests
* Cast integer backend (#55381)
* integer() custom expression with simple tests
(more tests coming)
* MOAR tests
* EVEN MOAR tests
* Change type to BigInteger
---------
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
This adds coercion support for serialization and adds a very limited partial schema for a Dashboard, in hopes that
this grows to be a full schema for our data models at some point.
fixes#54941
* sync translations with crowdin
* remove poeditor integration
* fix run name
* update to new project
* sync
* path
* move to config
* not master lol
* kick ci
* update crowdin config
* remove croatian
* roflcopter
* token
* maybe fetch harder
* test code - revert this
* test
* more tokens
* just use the cli
* line breaks
* le facepalm
* no verify
* token swap
* auto approve
* revert
* remove zh e2e test
* remove test code
* don't backport translations
* don't backport translations
* Add FunctionBrowser component
* Move header into CombineColumns for consistency
* Render the function browser
* Allow wrapping shortcuts
* Render toolbar buttons on top again
* Add missing Lib import
* Fix e2e test
* Use ellipsis in placeholder
* Add test-id to function browser
* Add e2e tests for function browser
* Add category to help text configs
* Only render supported functions in function browser
* Convert to helptext in getFilteredClauses
* Group functions by category
* More e2e tests
* Add empty state to function browser search
* Add more breakpoints for expression editor layout
* Move function browser item into its own component
* Increase width of function browser
* Add case to functions
* Remove unnecessary maw prop
* Sort functions
* Add tests for FunctionBrowser
* Remove nested component definitions
* Use standard breakpoints
* Add spacing between toolbar buttons
* Fix functions test
* Add category to distinct-if
* Fix tests that were broken by sorting
* Fix unit test examples
* Add more unit tests for filtering
* Do not enter parentheses when there are no arguments to the clause
* Sort by displayName
* Sort by structure
* Add e2e about clauses with no args
* Fix sorting test
* Fix font sizes for function browser
* Fix function browser spacing
* Allow replacing selection with function browser
* Context Driven Indicator Menu
* add unit tests
* unit test adjustments
* Ho
C time
* hasSeenAll
* cleanup, passing tests
* adjusting some tests, PR feedback
* more small adjustments. Starting to add docs
* hopfully final adjustments
* adjusting storybook path
Closes QUE-764.
Adds support for returning "remapped" columns, which have not been
supported in the library previously.
- Refactor `lib.field/remapped-field` to `lib.metadata`
- Add `lib.metadata.calculation/remapped-columns`
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* fix: improve startup by not scheduling all dbs
Improve long startup times by not scheduling tasks for all dbs on
startup. Due to the lack of bulk operations in quartz if an installation
has 500 dbs we need to load and scan through all associated db scanning
triggers 500 times for all dbs which means loading about 500k records in
1000 operations.
This is generally pretty fast on a per operation basis (about .5s
in local development), but on some postgresql installations this can
blow up to a few seconds per db, which makes the whole operation take 40
minutes plus. Additionally there maybe both lock contention and resource
contention between the quartz scheduler and the main app thread that
draws out this time.
Instead we want to init the schedule and put it into standby mode -- where
we can schedule tasks but are not executing them -- before we process
the configuration file. That we can schedule all necessary tasks as part
of loading the database from the config file and not need to then
iterate all database to schedule their tasks later in the startup.
Long term, it would be good to support rescheduling tasks using a bulk
operation in quartz (there are TODOs in their code base for supporting
this) rather than needing to update each trigger individually.
* fixup! doc string for check-health! function
* fix some mage installer logic
* make `cd bin && ./mage` work
get the absolute path, + cd to mb repo root in subshell
---------
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* do not setup sandboxing if using a sample database
* do not ask for tenancy columns if using a sample database
* alter copywriting when in sample database mode
* improve copywriting for scenarios with a sample database
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* fix misleading comments
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
The backfill job originally retried up to 20 times and then gave up. However, some instances apparently have ~150
duplicates or more, so 20 retries is insufficient. In addition, 2 db calls per conflict is sort of a problem when the
conflict count is O(duplicates^2). This bumps up the retry limit and reduces the db calls to 1 per 50 conflicts.
* Disallow navigating from model/{slug}/query and model/{slug}/metadata to model/{slug} when there are unsaved changes
* Reset state
* Simplify reset
* Use appropriate redux action when programmatically invoking navigation
* Disallow navigating from metric/{slug}/query and metric/{slug}/metadata to metric/{slug} when there are unsaved changes
* Remove mentions on inexisting /metric/metadata and /metric/{slug}/metadata routes
* Rename test
* Add repro for #32037 (models)
* Improve assertions
* Fix state cleanup
* Fix flaky test
* Deal with race conditions
* Add repro for metrics
* Correctly reset ui state when cancelling editing a model/metric
* Add repro for editing model's query
* Remove log
* Add a comment
* Revert "Skip tests that deal with the click outside and escape behaviour of the custom expression editor"
This reverts commit 2c05bc08c2.
* Revert "Remove CloseModal component"
This reverts commit ad1795e05a.
* Revert "Move usePreventClosePopover into utils"
This reverts commit de31b21282.
* Add PreventPopoverExit util
* Remove use of non-existant hook
* Prevent popover from closing on outside click or escape
* Rename argument to avoid confusion with double negatives
* Prevent custom expression editor from closing in aggregation sidebar
* Allow exiting the component when navigating away from custom expression editor
* Add a comment explaining why the useClickOutside does not work
* Add comment on how CloseModal works
* Remov empty file
* Add comment on what children should be
* Allow overriding cancel button text on ConfirmModal
* Allow passing more props to ConfirmModal
* Fix primary button on ConfirmModal
* Use ConfirmModal for custom expression editor
* Fix e2e test
* Remove second message in closemodal
* Update loki
If you run an uberjar with `MB_RUN_MODE=dev`, you get an exception
because you try to run the `watch-directory!` bit (which only works for
actual files) in an uberjar environment.
Let's just print a warning but continue along.
* Rename arg to caseInsensitive
* Add matcher for internal [:dimension name]
* Add formatting support for internal dimension clause
* Use raw MBQL in examples
* Format examples at certain width
* Remove unused suggest code
* Fix unit test for help text
* Add unit test for rendering examples on multiple lines in HelpText
* Improve spacing
* Make sure that all help texts can be formatted properly
* Add custom formatExample helper to format example expressions
* Use useAsync to format example
* fixup! Add custom formatExample helper to format example expressions
* Add HighlightExpression component
* Export all modules
* Fix `row` type to be `RowValue[]`
* Use richer `clicked` object in object detail view
* Fix column settings not available in detail view
* Add e2e test
* Rename Select to LegacySelect
* Convert frontend/src/metabase/lib/core.js to TypeScript
* Format code
* Extract SemanticTypePicker
* Fix type
* Fix styling
* Fix nulls
* Add nothingFoundMessage
* Extract DefaultSelectItem
* Add CurrencyPicker
* Optimize
* Add checked icon to CurrencyPicker
* Improve overflow in SelectItem
* Add FkTargetPicker
* Style font-weights to be the same as in other inputs around
* Fix colors
* Add a comment
* Fix imports
* Style dropdowns
* Make currency input full width
* Add Combobox component
* Show currency symbol next to value
* Modernize field settings button
* Modernize FieldVisibilityPicker
* Update tests
* Update test
* Update and merge tests
* Update tests
* Remove bold, fix padding and move components to their own dirs
* Update unit tests
* Update e2e tests
* Update tests
* Reduce flakiness
* Add comment
* Repro #53595
* Replace CSS modules overrides with props
* Fix symbol styling for selected option in CurrencyPicker
* POC - moving two pivot functions into CLJC
* Debug logs in frontend pivot impl
* POC continued
* setting isCollapsed consistently
* Working replication of valuesByKey map construction
* sorting logic
* hook up new code to the FE impl
* Fixing small potential bug in valuesByKey
* Fixing bug in clojure vs javascript string key for valuesByKey
* fix valueColumns key name
* Some performance improvements + cleanup
* A couple more attempts at perf improvements
* Fix bugs
* port format-values-in-tree
* WIP add-subtotals
* @tsp - Improved build-values-by-key perf; some debug perf lines in frontend land
* @tsp - in progress conversion of add-subtotals
* @tsp - working converted Pivot.add_subtotals
* @tsp - fixing bug in add-subtotal mapcat recursion
* @tsp - add-subtotal restoring map to mapcat (oops)
* @tsp - adding working Pivot.add_value_column_nodes to replace addValueColumnNodes
* @tsp - removing debug logs & perf timing logs
* Appease whitespace linter
* More formatting code ported, cleanup
* Fix bugs, cleanup
* Fix some tests
* Start hooking up the BE, fix some bugs
* Fix lint/formatting errors
* Fix comparator + i18n
* Fix another module issue
* Test using js-specific i18n function
* Revert "Test using js-specific i18n function"
This reverts commit 9a8604b583885f1c35f48507112e3a142ca0d3a0.
* Bump up timeout on cypress assertion
* WIP - migrate row section getter to cljc
* fix tests
* Fix e2e test
* Switch exports over to fully using CLJC code paths, delete now-unused code
* Delete outdated postprocess tests
* Bug fix
* Fix build
* cleanup, consolidation
* fix function call
* more cleanup
* empty test ns
* misc cleanup/docstrings
* WIP tests
* more tests and refactor
* small fix
* fix getting column title for value columns
* Fix some tests
* Fix another test
* Revert a couple changes
* Fix some more tests
* Fix more tests, fix default values for pivot viz settings
* Ensure headers respect format-rows? settings in exports, and fix more
tests
* Fix lint issues
* fix another test
* Fix a test, a little cleanup
* Try to fix another test by setting format-rows middleware to false for subscription attachments
* Revert "Try to fix another test by setting format-rows middleware to false for subscription attachments"
This reverts commit d860522a6d17c27e67afabccf119039508c96cca.
* Update formatting logic to fix another test
* Fix conditional formatting
* Fix last BE test
* remove test not needed anymore
* Fix drill through + more cleanup
* clean ns
* remove old code from .clj-kondo/config.edn
* add a comment + tweak keywords
* perf improvements and refactoring the CSV export path
* Performance improvements
* Fix bad rebase
* more perf improvements
* fix test
* lint
* oops, fix test
* more optimizations, memoize backend formatting
* fix range-based conditional formatting
* remove unused function
* add some new backend unit tests
* fix FE type checks - caused by an experiment that I didn't revert fully
* address PR review feedback
* fix up tests
---------
Co-authored-by: Tyler Plude <tsplude@gmail.com>
We are adding entity_ids to dbs, tables, and fields, but existing rows don't have entity_ids yet, and newly created dbs/tables/fields don't have entity_ids when created. This job adds those missing ids.
Ideally, we'd copy them too, but we need to decide if we should duplicate them or not (do a deep or shallow copy) is a
too hard to make immediately, and this at least will unblock people being unable to duplicate dashboard with actions
defined.
somewhat improves on #38224
* setup discouraged-tag linting for #p
- this is fine to use in dev, but shouldn't get comitted
so this will fail kondo linting.
* sample #p commit (check linting)
* Revert "sample #p commit (check linting)"
This reverts commit 171130c668.
I added it to double check the linter will complain correctly
API Key Configuration via config.yml
Overview
This PR adds support for configuring API keys through the config.yml file in Metabase Enterprise. This enhancement allows administrators to define and manage API keys declaratively through configuration, making it easier to set up and maintain API access across different environments.
Features
Adds support for defining API keys in config.yml under the api-keys section
API keys can be configured with:
name (required)
group ("admin" or "all-users")
creator (email of the user to assign it, user must exist in the instance or be created in the config file, must be superuser)
Automatically creates API keys on Metabase startup if they don't already exist
Skips creation of API keys that already exist (based on name)
Validates configuration including:
Required fields
Valid group values
Creator permissions
Example Configuration
version: 1
config:
api-keys:
- name: "My Admin API Key"
group: admin
description: "API key for admin operations"
creator: admin@metabase.com
- name: "My All Users API Key"
group: all-users
description: "API key for general operations"
creator: admin@metabase.com
Security Considerations
API keys are only created by admin users
Keys are properly hashed before storage
Existing keys are preserved to maintain continuity
Group validation ensures keys have appropriate permission levels
Use Cases
Automated deployment and configuration
Infrastructure as code
Consistent API key setup across environments
DevOps automation
Testing
Unit tests cover key creation, validation, and error cases
Tests for configuration parsing and application
Validation of group permissions
Error handling for invalid configurations
This enhancement is part of Metabase Enterprise's advanced configuration capabilities, providing more flexibility in managing API access through configuration files.
* Use timezoned timestamp for export filenames
* clean ns
* tweak docstring
* add unit test + make namespace aliases consistent
* use new function in xlsx exports
* feat: add malli schema, test converage to ldap api
Restores the malli schema that was causing e2e tests to fail on the
first attempt and updates those e2e tests to look for the new error
message returned when the api request is rejected due to schema
validation issues.
Expand test cases around ldap connection testing errors so those
messages can be caught on future api changes.
* fixup! error capitalization
* Add reproduction for #54638
* Add reproduction for #54722
* Add reproduction for #31964
* Correct user of metabase issue in test name
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Correct use of issue name in test title
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Use addCustomColumn helper
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Use helpers to open filter and summarize popovers
* Check for textbox role instead on cm-content class
* Just use enter
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
I enforced this by checking the return values of `visible-columns` and
`returned-columns`. I want to check that in (for dev and test only) but
it fails tests currently.
The trouble is that cards don't have `:ident` in their `result_metadata`.
That's coming later, so the enforcement flag is disabled for now.
* Truncate mrkdwn links
They could exceed block limits
* Truncate header block text
* Remove incorrect (testing) block in escaping test
* Make length consts under test public
* Add link truncation test
* Add header truncation tests
* Use lezer parser for custom expressions (#53540)
* Add @lezer/generator
* Add missing value on Token
* Add script to generate lezer grammar parser
* Add @lezer/lr
* Add more tests to tokenizer
* Add lezer grammar for custom expressions
* Generate lezer parser for custom expressions
* Convert TOKEN and OPERATOR to enums
* Make Token type more specific
* Add isReference to identifier token
* Typo
* Add more tests for tokenizer
* Add tokenize alternative that uses lezer grammar
* Remove old tokenizer
* Update location of grammer in generate rule
* Add test cases for tokens that share a prefix
* Merge tokens when necessary
* Only add character error when text is 1 character long
* Reduce the need to manually merge nodes
* Stop the iteration when node is found
* Update tests to be more specific
* Simplify tokenize
* Uncommit edge cases
* Generate parser as js
* Move previous tokenizer tests into expressions folder
* Add typing to unit test
* Allow escaping [] in reference
* Add test for escaping closing bracket
* Handle invalid parses manually
* Add support for signed expressions
* Document parser more
* Add custom tokenizer for Reference
* Make tests more explicit
* Fix e2e test
* Merge old test suite into tokenizer test suite
* Move @lezer/generator to devDependencies
* Custom expression suggestions (#53679)
* Add new suggesters for custom expressions
* Add test for joined fields
* Move complete into separate folder
* Split up completions into separate files
* Split up completion tests too
* Copy more tests from legacy completer
* Fix types
* Lezer highlight for custom expressions (#53651)
* Rename generated parser to lezer
* Add lezer highlighting rules to parser
* Add HighlightExpression component
* Use HighlightExpression in expression HelpText
* Add type annotation to highlighter
* Add test for highlight
* Simplify unit test
* Move syntax highlighting to shared module
* Rename util.tsx to util.ts
* Add monospace font variable
* Use monospace font from theme for syntax block
* Use relative font size in HighlightExpression
* Use mantine var for border-radius
* Add test id to example
* Fix tests that rely on example content
* Move highlighting code to metabase/ui/syntax
* Replace ACE with codemirror in custom expression editor (#53746)
* Rename generated parser to lezer
* Add lezer highlighting rules to parser
* Add HighlightExpression component
* Use HighlightExpression in expression HelpText
* Add type annotation to highlighter
* Add test for highlight
* Simplify unit test
* Move syntax highlighting to shared module
* Rename util.tsx to util.ts
* Add monospace font variable
* Use monospace font from theme for syntax block
* Use relative font size in HighlightExpression
* Use mantine var for border-radius
* Add test id to example
* Fix tests that rely on example content
* Move highlighting code to metabase/ui/syntax
* Replace custom expression editor with CodeMirror editor
* Add test mock for CodeMirror
* Colocate custom expression tests (#53812)
* Remove unused DISPLAY_QUOTES
* Colocate custom expressions diagnostic tests
* Colocate custom expressions process tests
* Colocate custom expressions recursive-parser tests
* Colocate custom expressions typeinferencer tests
* Colocate custom expressions completer tests
* Colocate custom expressions format tests
* Colocate custom expressions resolver tests
* Colocate custom expressions index tests
* Colocate custom expressions pratt compiler tests
* Colocate fuzz tests with custom expressions code
* Update paths for fuzz tests
* Fix types in fuzz test
* Fix types in process test
* Remove unused var
* Remove unused var in generator
* Merge types for custom expressions (#53902)
* Add missing expression types to metabase-types
* Add more explicit types to matchers
* Use Expression type from metabase-types everywhere instead of custom type
* Move matchers to separate file
* Further use Expression types for matchers
* Add types for Metric and Segment matchers
* Fix types in format
* Fix missing types
* Clean up imports
* Split string helpers into separate submodule
* Split identifier helpers into separate submodule
* Avoid importing from index file in expressions module
* Render suggestions and help text in new custom tooltip (#53760)
* Add new help text component for custom expression editor
* Add new suggestions listbox for custom expression editor
* Add helper to find help text by function name
* Add enclosingFunction helper that is more forgiving to broken syntax
* Add new tooltip component for custom expression editor
* Add extension to render custom react tooltips
* Render suggestions in custom tooltip
* Export custom types
* Remove old expression editor popover components
* Add typings to process
* Add missing shortcuts
* Fix example font size
* Fix type imports
* Fix click on helptext
* Stop click propagation
* Fix process types
* Remove superflouous optional chain
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix types whith optional expressionIndex
* Use imports instead of React namespace
* Use index for key
* Rename util to utils
* Remove unused box shadow
* Remove brackets around placeholder names
* Resize tooltip to avoid it overflowing the window
* Fix line height for args
* Fix missing key icons
* Fix chevron alignment
* Fix types
* Move Listbox resizing classes to Tooltip
* Fix completion template for functions with no arguments
* Add unit tests for enclosingFunction
* Add darker background to expression example
* Remove bold from function call
* Fix imports
* Fix HelpText tests
* Add custom color for highlighted syntax argument
* Use colors from design system
* Fix blurring when switching to a different tab
* Remove trailing comma
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Split up effect
* Remove background from template arguments
* Improve tooltip position
* Fix Popover position when selection changes
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix custom expression editor e2e tests (#53761)
* Fix e2e tests for custom expressions
* Fix unit test
* Fix e2e tests
* Fix types
* Fix moveToStart and moveToEnd by using Home/End
* Add test for popover position
* Remove JSDoc in favor of inline comments
* Add examples of side effects
* Remove shouldContain helper in favour of value()
* Add comment to wait
* Use CustomExpressionEditor helper instead of @formula
* Add wait to reduce flakiness
* Add back in the useful comment about popover position
* Remove superflouous cy.viewport call
* Add helper to accept completion
* Rename helper to verifyHelptextPosition
* Convert comments to cy.logs
* Make sure we don't avoid testing the regression
* Allow customising key with which to accept completion
* Use acceptCompletion helper
* Fix tooltip position bug
* Remove unnused wait
* Add selectCompletion helper
* Fix helper using H
* Add better emulation of copy/paste in reproduction
* Ignore ts error
* Multiline formatter for the custom expression editor (#53973)
* Make type name meaningful
* Add more tests for not isnull, etc
* Remove unreachable code
* Add isEmpty matcher
* Pass query to format tests
* Add stronger types to formatStringLiteral
* Fix CallOptions import
* Implement multiline formatter as a prettier plugin
* Reimplement legacy formatter as new formatter with infinite print width
* Use the new formatter in the expression editor
* Remove legacy expression format files
* Document utils
* Simplify Expression type further
* Fully type test cases
* Remove cast in test
* Add more tests and fix formatting
* Use recurse helper to recurse into path children
* Prefix logical operators
* Improve formatting of logical connectors
* Wait for expression editor to be ready
* Avoid passing unknown html props
* Use selectCompletion helper
* Remove unused @ts-expect-error
* Align UI more to design
* Remove shortcuts in suggestions popover
* Disable popular functions in completions
* Render shortcuts as separate component
* Move NameInput to separate component
* Fix button location when no name input is present
* Remove unused ExpressionWidgetInfo component
* Avoid flicker of shortcut buttons
* Tweak name input padding
* Deduplicate styleTags in Editor
* Do not format on blur and pass through all valid changes
* Split of diagnose and compile step
* Remove unused props
* Do not trigger completion on focus
* Show update button when no name is passed
* Expose expression types
* Use diagnoseAndCompileExpression in lint extension too
* Add toolbar with format button
* Only show popover when user has interacted with editor
* Throw ResolverError when types mismatch
* Fix missing highligh for boolean
* Render errors
* Disable format button when the expression contains an error
* Remove shortcut and popular function completion
* Move Shortcut type to component that uses it
* Fix blur for CustomExpression helper
* Add format helper
* Fix e2e tests
* Fix format helper
* Fix blur helper
* Use actual ellipsis in placeholder
* Add helper for nameInput
* Remove buttons from tab sequence once hidden
* Do not autofocus shortcuts
* Hide shortcuts when error occurs
* Allow reading multiline value
* Fix e2e tests
* Fix unit tests
* Add more tests for multiline custom expression editor
* Debounce error checking
* Add max height to tooltip
* Fix formatting
* Document changing reproduction
* Test wording
* Disable format button when expression is empty
* Do not try compiling when source is empty
* Cancel changes when source changes
* Autofocus editor
* Use Tooltip to mark button
* Remove default autoFocus on button
* Simplify shortcuts show and hide style
* Move shortcuts styles to parent element
* Fix line number alignment
* Add a gap between Cancel and Done buttons
* Always close helptext when suggestions are being shown
* Add usePreventClosePopover hook
* Prevent closing popover from the custom expression editor
* Add tests for not closing on click outside and escape
* Move toolbar down
* Fix e2e tests
* Add format_code icon
* Use correct icon for formatting code
* Use new format code icon in native editor too
* Update Loki Snapshots
* Merge useEffect cases into one
* Allow clicking the shortcuts
* Debounce validation
* Fix React update loop
* Prevent flicker in the listbox completions
* Tweak the linting delay
* Hide shortcuts when there is an error
* Only prevent closing popover when editor is rendered
* Fix width and height for editor widget
* Fix format button positioning
* Fix button alignment in expression editor
* Allow blurring tooltip by clicking nothing
* Add button role to header
* Show modal when clicking outside of editor
* Allow closing editor when it's empty
* Add tests for clicking outside of expression editor
* Fix missing onClose handler
* Add bigger timeout
* Rename Format to Auto-format
* Hide format button when there is an error or editor is empty
* Fix label names in test
* Fix format button test
* Deduplicate tests for exiting the editor
* Fix test
* Fix repro
* Only prevent closing popover when the source has changed
* Fix e2e test
* Hide lint point
* Fix types
* Fix tooltip helptext layout
* Update loki reference
* Only show format button when the expression was validated
* Scale the expression editor with the screen width
* Move usePreventClosePopover into utils
* Remove CloseModal component
* Remove custom PreventCloseContext from Popover
* Skip tests that deal with the click outside and escape behaviour of the custom expression editor
* Remove canPreventCloseHandlers from ClausePopover
* Add lineWrapping to editor
* Fix loki
* Fix loki screenshots
* Fix e2e test
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Apparently, a filter like `blah in (cast([uuid] as uuid))` is fine in postgres, but sqlserver doesn't like it very
much. This gives sql server the syntax it needs.
* first crack at linear pr link bot
* try it again
* bump actions versions
* bump
* try again
Added Node.js setup step
Added dependency installation step for node-fetch
Changed the ES module import to a CommonJS require statement
Changed variable name from fetch to nodeFetch for clarity
* extract script
* use the right graphql query
* add debug
* look at the right part of the linear api response
* lets see the issue numbers
* again
* again
* again
* again
* again
* again
* again
* again
* again
* fix formatting
* remove ???'s
* improve logging
* check all magic closing words
* refine checking logic
* test
* some review responses
* again
* make github object a thing
* install octokit
* install octokit again
* install octokit again
* running
* running
* running
* running
* running
* delimit message with spaces
* delimit message with spaces
* delimit message with spaces
* delimit message with spaces
* test
* only try to close metabase/metabase issues
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* support react 19 version range in embedding sdk
* sync supported react versions
* add oxford comma
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* revert docs and cli updates as we want to do a silent release
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* add use-toast-hook
* use the new toast hook
# Conflicts:
# frontend/src/metabase/admin/performance/components/ModelPersistenceConfiguration.tsx
* bonus change slipped in
* use those callbacks
* add new database header, make layout adjustments
* start building connection section
* reorganize connection info section content, migrate danger info section
* reorg
* flow rewired together mostly, connection details form is now in a modal
* make section content components the entire section for better hide/show logic, remove old code comments, show proper engine name in edit db header, hide model actions section if no actions can be taken
* lots of polishing
* fix BE punctuation
* test e2e fixes in ci
* more test fixes
* more test fixes
* fix unit tests
* fix last failing unit test
* fix last e2e test?
* final polish pass from self review
* fix model persistence section and its related tests, reorganize the tests
* fix model persistence section and its related tests, reorganize the tests
* impl feedback from arakaki around creation flow and date format
* clean up last commit
* make modal scroll container limited to the body, remove use of force to find inputs in cypress tests
* impl PR feedback
* fix: question stats are calculated in sql query
Previously instances could OOM when calculating these question stats if
they had too many cards because we load all these cards into memory.
We were doing this full calculation because we have some data we want to
calculate metrics over stored as json in a text column. This uses native
mysql/mariadb and postgresql json fuctions to calculate those metrics
inside the database prevent us from needing to load everything. That
also means we no longer calculate these json-based metrics for metabase
instances running h2.
* fixup! use older path_exists json function
* Add a native editor placeholder text
Resolves CLO-3674
* Keep the "data reference" panel open by default
Resolves CLO-3675
[ci skip]
* Add empty state for snippet sidebar
Resolves CLO-3684
* Empty state for query results panel
Resolves CLO-3676
* More refined way to initialize data reference
* Update copy in e2e tests
* Fix more tests
* Open data reference whenever native editor is open
* Do not alter the initializeQB flow
* Close the data reference sidebar when question settings are open
* Fix a failing Mongo test
* Address review comment
* Improve _.compose type + fix SDK types
* Use a simpler type overloading
* Fix Englishz
* Fix type errors
* Add a comment to clarify extrat type overload
* Clarify the source of code
* add fall-through for ignored embedding requests
* Gsheets mb api (#51307)
* send flag to show "connect gsheets" button to FE
* clean up docstring + add etl-connections condition
* metabase api WIP
* align APIs
* include site-url in the HM temp-url api call
* cleanup
* update HM<->MB api to be working (minus temp-url)
* cleanup
* remove some notes
* indentation + linting
* add tests
* fix typo
* remove illustrative (when true ...)
* add frontend for gsheets sync
* update frontend to use changed API
* Gsheets integration: more api alignment (#51492)
* add frontend for gsheets sync
* update frontend to use changed API
* wip
* cleanup
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* fix: declare gsheet setting fn before using in getter
* fix: delay u.secret instantiation to prevent build race condition
See https://metaboat.slack.com/archives/CKZEMT1MJ/p1734605793293989
* build: push ee-extra to dev repo, not ee
Cloud wants ee-extra, not just ee
* move settings out of auth middleware ns
- update kondo modules
- add metabase.api.auth ns
* More cleaning up
- update docstring
- remove declares + use {set,get}-value-of-type{!,}
- remove nil state from gsheets setting
* rely on more hm stubs
* save folder_owner to gsheets
* gsheets and hm http client => enterprise
+ move tests too
* require the right namespace
* add migration for is_gsheet to metabase_table
* Gsheets service account be flow (#51997)
* update setup flow to use service-account
* align GET api/ee/gsheets/service-account response
* align the HM service account endpoint
* fix migration yaml
* update test for service accounts
* indentation + migration quibble
* improve error messages + kondo housekeeping
* remove unreachable if-let branch + fix serdes test
* add is_gsheet to mock and test data
* add gsheets setting to serialization test
* fix fe-type-check
* typechecker
* [kondo] defendpoint -> defendpoint2
* Gsheets table management (#52145)
* include dwh ∈ GET /api/ee/upload-management/tables
* cljfmt
* simplify preliminary gsheet definition
* archive cards is a query param
* gsheets: update polling logic + better errors (#52162)
* update polling logic + better errors
* codify protocol for polling folder done-ness
* align on FE polling logic + the real sync complete
* [WIP] testing happy path
* status should be "active"
- remove a bunch of extra debugging
* remove outdated tests + cljfmt
* check admin + use the connection's id w/ delete
- cleanup
* fix :delete /folder + schema on :get /folder
* Gsheets frontend (#51946)
* non-oauth UI
* update api and add disconnect modal
* update illustration and add error displays
* add gsheets sync status
* better copy
* remove test code'
* fix bad api error
* updates to sync status
* Update sidebar entrypoint
* gate things behind the right features
- only show gsheets button
- http requests to /api/ee/gsheets/*
* model/Table has a db_id column not database_id
* add and use simple events
* differentiate failure modes
* + tests and error status codes
* make form submit on enter
* reset gsheet status when there is no gdrive conn
* add debug info
* FIXME FOR TESTING - disable feature checks
* fail on sync error + partially allow deletes
* add database page entrypoint
* FIXME FOR TESTING - set token-status-cache-ttl to 1 second (vs 12 hrs)
* only show _active_ dwh tables
* fix config.edn (howd it break tho?)
* handle multiple gdrive connections
- on delete, delete them all
- when looking at syncing, look at the most recent one
* add auto-cruft-tables to metabase_database.setting
- Here's the contract:
There should be an array of strings located at: db > settings >
"auto-cruft-tables". Each string will get fed into re-pattern. The table
names get lowercased before getting matched on. That list of regexes is
combined with our built in list of crufty regex patterns. We use re-find
to match them against the lower-cased table names. If there's a match we
will mark the table crufty (this is called a "Hidden Table" in the UI).
This way, if someone names their sheet `I'mUnique____dlt_loads` or
whatever they can at least fish it out of the hidden section.
Note: It will get set back to being crufty (aka what the ui calls
hidden) on the next sync... but this is configurable so we can fine-tune
the regexes to match the tables we want to hide more closely.
Examples:
If you put ["a"] in the config.yml: any table with A or a in it will be
marked hidden. So be sure to use anchors for the suffix or what-not. For
our thing, I think we want something like:
* add bunch of tests for auto-cruft
* mocked api tests
* add commment + mock hm responses
* documentation
* when multiple gdrive conns, sort by status, then created-at time
* cleanup
* joda time Instant/ofEpochSecond 0 -> t/instant 0
* if no gdrive conns are found, reset state
- add gdrive sorting tests
* indentation
* fix complete sync state and add error state
* update sync unit tests
* update sync unit tests
* add frontend analytics events
* don't show reconnect copy from DB page
* Add 10 minute timeout for polling connection
- return errors as json-encoded :error in the response body
- minor cleanup how we auto cruft tables (no change to behavior)
- add script for running the flow locally: from the repl
* better FE sync state and form validation
* refresh setting on error
* remove fake error state
* Delete connections on reset
- Change order of timeout checks in :get /folder handler
- add auto-cruft-columns setting
* update gsheets routes to use the new ns-handler
* loosen up gdrive link validation
* update regex
* add docstring to dynamic timeout var
* simplify api use via gdrive conn id
- move ->config into hm.client ns
* recover from suddenly-missing gdrive conn @ id
* cleanup hm.client
- indentation
fix [[make-request]] args
* fail to load invalid + present cruft patterns
* remove some extra comments + use hm-delete-conn
* fix update-table-metadata
- (no longer update nil -> nil)
* auto cruft columns
- pull the regex comparrison code into crufty ns
- reuse it on 2
- add defsetting so having the setting on the database doesn't complain
- pass database down into field syncing + grab vis types
* indent
* fix a few tests
* fix gsheets tests
* add harbormaster client test
* kondo + cljfmt
* migrate to underscores for auto_cruft_*
- add tests for field + fix schema for table syncs
* use kebab-case for settings but accept snake_cased
* cljfmt + kebab case stragglers
* fix update-field-metadata-if-needed! signature
* fixing tests
* rename macro double -> run-twice
* remove test code
* prettier
* cruft settings are db-local
* Fix search index update job restart issue (#53229)
* more idiomatic crufty db-local settings
* add auto cruft fields test
* bump gsheets sync timeout to 10 seconds
* handle going back to loading state
* kondo + add empty cruft tables test
* add get folder timeout test, update dox,
- remove regex folder check
* add cruft columns config.yml setup tests
* remove def + actually run the test
* test code for bruyan
* reset gsheets status on every error response
* save dbId and actually show it
* bust cache on gsheets setting
* solve initial-sync-status-table-only-test failure
* Update src/metabase/sync/sync_metadata/crufty.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update enterprise/backend/src/metabase_enterprise/advanced_config/file/databases.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update enterprise/backend/src/metabase_enterprise/advanced_config/file/databases.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update enterprise/backend/src/metabase_enterprise/gsheets.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* respond to Noah's review comments
* kondo
* update modal copy
* update error copy
* status click target update
* improve crufty/name? api
* modularize gsheets
* remove temp gsheets
- token status checks happen once per 12h
- remove hm/conn and mb/sync-info from :get /folder response
* gdrive-conn is now unused
* remove comment
* fix modules
* add docstring
* add harbormaster module
* kondo
* fix the tests
* module linter
* fix typo
* cljfmt
* setup api-key setting before reading it
* sort ns
* ::snowplow/simple_event -> :snowplow/simple_event
* hard code error message
* remove test code
* update frontend tests
* update module linter config
* add simple schema snowplow event schema
* fix schema name
* update some tests
* add a test
* better docstring
* better docstrings
* fix e2e tests
* fix fe token mock
* respond to review comments
+ fix timeout 50000 -> 5000
+ make internal regex function private
* build: push ee-extra to dev repo, not ee
Cloud wants ee-extra, not just ee
* fix strings
* remove resolved comment
* adds prometheus metrics to gsheets api
- :metabase-gsheets/sync-creation-begin
- :metabase-gsheets/sync-creation-ok
- :metabase-gsheets/sync-creation-error
- failure-reason is one of:
- "timeout" - gsheets.api/sync-complete? didn't return true after 10 minutes
- "status_error" - HM said the sync connection had status == error
- :metabase-gsheets/connection-deleted
* Refines prometheus events
They are now:
- :metabase-gsheets/connection-creation-began
- :metabase-gsheets/connection-creation-ok
- :metabase-gsheets/connection-creation-error {:reason "status_error"}
- :metabase-gsheets/connection-creation-error {:reason "timeout"}
- :metabase-gsheets/connection-deleted
* comment
* allow resolution of snake_cased setting names
* review responses
* add logging if any part of the batch-delete fails
* remove unused schema
* Revert "remove unused schema"
This schema is actually used! 🎉
This reverts commit f5dbd78051.
* update loki icon reference image
* switch off flaky test setup
* mark date filter picker tests flaky (#54866)
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Co-authored-by: Filipe Silva <filipematossilva@gmail.com>
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* WRK-114,WRK-115: Replace SchedulePicker with Schedule in Alerts.
Integrated a newer feature-rich component for Alerts scheduling.
Fixed bug related to incorrect handling of '*' values in minute and hour
alerts.
* Remove scaling transition from popInFromBottom animation.
This transition used for all modals led to a bug with sizing in some
Mantine components, when they were used inside a modal:
https://github.com/mantinedev/mantine/issues/6967#issuecomment-2417439117
* Reuse Personal Access Token on all tasks
* Fix permission when pushing to GitHub
* Use annotated tag for better information
* Remove steps for faster testing
* Revert "Remove steps for faster testing"
This reverts commit 8bd5d2e07f.
* Prefer slack_file for images, so metabase no longer requires a public files channel
This removes the requirement to have a public files channel so share
images or other files in slack bot posts. Simplifying the integration
and lessening requirements to get setup.
It enables the posting of images files directly to public or private
channels without sharing - improving privacy and security.
* Files channel deprecation note
* Drop slack-files-channel requirement from slack /settings API
* Remove remaining files-channel calls
* Remove files channel input from SlackSetupForm
* Remove slack-files-channel from SettingsManagerSettings
* Remove slack-files-channel from API docs
* Drop files channel configuration guidance
* Remove slack channel prompt/description
* Instead of polling files.read reuse the complete call
There was an accidental introduction of files.read.
The goal is to preserve the previous behaviour of reusing
completeUploadExternal.
This limits the oauth scope requirements of the bot token.
* If more than 50 slack blocks, send using multiple messages
* Escape mkdwn special-processing chars in card links
* Simplify slack attachment uploading
- Its easy to mapv
- (cond) over nested (if)s
- Fix out of date docstring
* Remove slack-files-channel from config-template.md
* Remove metabase_files check from e2e suite
* replace use of findDOMNode in core components
* do not wrap input in div
* always populate scrollElementRef
* remove scrollElement from TokenField
* do not export _TokenField
* remove scrollElementRef
* use appropriate ref container
* cleanup root container
* use the scrollingContainerRef instead
* pass the elementRef directly
* use full-width container
* remove full-width container
* use display contents to preserve layout
* attach listRootRef conditionally
* try to update Loki snapshots
* modify snapshots
* fallback to scrollingContainer instead of elementRef
* revert loki snapshot
* revert loki snapshot to master
* Do not use originalQuestion when changing the template tag's type
* Do not use originalQuestion when changing the template tag's type
* Fix test
* Fix
* Fix
* Fix
* Fix
* Fix
* build multi-arch arm image
* refactor artifact fetching and containerizing
* fixes after testing
* use old master for caution's sake
* enable multi-arch in publish workflow
* sync
Part of #54559
This adds `trace` level logging to:
- `metabase.driver.athena`
- `metabase.sync.sync-metadata.fields.sync-instances`
That should help us diagnose sync issues where describe-fields and
fingerprinting seem to be out of sync.
* Add a nitpick for postgres and impersonation
Something that a customer saw, and might be interesting for posterity
* Update docs/permissions/impersonation.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Update running-metabase-on-docker.md
Add a note since people think that this is the official way to run Metabase
* Update running-metabase-on-docker.md
* Update running-metabase-on-docker.md
---------
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
* use new endpoints to load tiles for pin map
* Fix public endpoints for unauthneticated users
* use new BE api for pin maps
---------
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* fix: update saml20 for full SAML SLO support
Update metabase to use a new version of the saml20-clj library that
correctly builds and signs HTTP Redirect requests and provides support
for checking LogoutResponses.
Note right now SAML SLO cannot be enabled just through the API it
requires the following settings to work correctly:
* saml-slo-enable - must be true
* saml-identity-provider-slo-uri - must be set to the IdP's SLO endpoint
* session-cookie-samesite - must be set to none
For the samesite none cookie to work metabase must be served over SSL.
Without this setting Chrome (and other browsers) will not send the
cookie when Okta redirects to our SLO endpoint which means we can't sign
out the user session.
* fixup! kondo lints
* booting up mage
* indent -> format
* formatter help
* Kondo commands
* Clear cache when running Kondo
* start-db command
* Add aliases in deps.edn
* comments + more
* subsume new commands into bb
* add mage_verbose env var
* add tests
* refine tests + check github action output
* run the tests
* add mage tests to backend workflow instead
* min version + rip out b/ask!
* Fix start-db latest
* actually call it with --help
* add readme entry for bb tasks
* move project-root-dir into util
* add mb download jar + cleanup bb.edn
* cleanup + fix jar downloader
* run tests
* finish removing bask
* add mage and jar downloader
* setup ./bin/mage entrypoint [responding to user research]
* uopdate examples
* update readme
* add todos
* add autotab + deprecate obsolete scripts
* migrate instructions for cljfmt over to mage
* respond to ./bin/mage kondo-updated master
* linting branding, clean up cli for fmt
* fix comment
* more tests
* better explainations of what mage is doing
* remove empty task -new-task!
fix start-db
* handle missing deps-edn-hash
* make start-db work with mysql and maria
* improve ux + simplify option and arg parsing
* add optional :usage-fn to task definitions
* cljfmt-* {-c}, and jar-download working
* last bit of cleanup
* docs
* add examples exit-codes test: ./bin/mage -test-examples
* add deprecation warns to old shell scripts + docs
* remove typo
* better errors
* typos
---------
Co-authored-by: Cam Saul <github@camsaul.com>
* Simple data picker UI (#52964)
* Add SimpleDataPicker base component
* Finish the flattened list version of the simple data picker
* Add selected entity state
* Fix the new simple data picker got run with loki accidentally
* Update frontend/src/metabase/embedding-sdk/components/SimpleDataPicker/SimpleDataPicker.tsx
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Fix selected data source icon color
* Add an empty state
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Real simple data picker (#52965)
* Connect data to the data picker
* Sort the entities alphabetically
* Make sure the data picker icon is always table
* Fix unit tests
* Fix interactive embedding E2E tests
* Force table icon on the DataPickerTarget for simple data picker
* Remove commented out code
* Remove tests that don't apply to the new simple data picker
* Fix passing wrong table ID
* Ensure we can only join data from same DB
* Make component name better
* todo
* better simple data picker UX
* Add multi stage embedding data picker back
* Able to force complex data picker in interactive embedding
* Fix tests for simple data picker
* Add old tests back
* Add simple data picker tests back
* Fix unit tests
* Fix SDK tests
* Remove ability to force multi stage data picker
* Add cross-db join tests for simple data picker
* Simplify code
* Fix popover not closing since Mantine prop changes
* Add simple data picker data source ordering tests
* Fix conflicts from merging master
* poc async plugins
* Remove SDK plugins poc code
* Fix unit tests after moving code to EE folder
* Mock search query explicitly
Co-Authored-By: Alexander Polyankin <alexander.polyankin@metabase.com>
* Don't modify cljs objects
Co-Authored-By: Alexander Polyankin <alexander.polyankin@metabase.>
* Reduce the ambiguity around NaN value
Co-Authored-By: Alexander Polyankin <alexander.polyankin@metabase>
* Remove dependency from an unrelated module
Co-Authored-By: Alexander Polyankin <alexander.polyankin@metabase>
* Fix SDK unit tests not running because EE plugins
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase>
* Fix embedding data picker not filtering by selected database (#54272)
* Fix embedding data picker not filtering by selected database
* Add an E2E test
* Fix SavedEntityPicker doesn't filtered out items in a selected DB
* Fix TypeScript types
* Address review, not passing unnecessary props
* Fix failed tests by using the same snippet in the past
* Hide metrics/saved questions from multi-stage data picker (#54409)
* Hide metrics from multi-stage data picker
* Hide saved questions from multi-stage data picker
* Modify saved questions tests
* Modify metrics tests
* Address review: create a new test suite and put tests there
* Adjust the data picker min-height according to review
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase>
Let's improve coverage of sandboxing in e2e tests.
These tests cover the following kinds of sandboxing policies:
* Filter by regular column
* Filter by custom column
* Use a custom view to filter a table
The tests verify that these sandboxing policies correctly affect:
* Saved questions
* Models
* Questions based on questions
* All of the above, when viewed in a dashboard
* Adhoc questions
* Filter values
* build multi-arch arm image
* refactor artifact fetching and containerizing
* fixes after testing
* backwards compatibility
* use old master for caution's sake
* update release code tests
* just bash it
* fix: Cast LocalDateTime to date in oracle
Fixes#53371
Remember that oracle Date is a timestamp type with seconds resolution.
This changes our inlining of LocalDateTime values to use a date rather
than timestamp if the LocalDateTime has no millisecond part.
We can see `timestamp` causes full scan:
```
SQL> explain plan for select * from test where d > timestamp '1970-01-01 00:00:00';
SQL> SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY());
--------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 35 | 2 (0)| 00:00:01 |
|* 1 | TABLE ACCESS FULL| TEST | 1 | 35 | 2 (0)| 00:00:01 |
--------------------------------------------------------------------------
```
We can see `to_date` causes idx scan for both date and timestamp LHS.
```
SQL> explain plan for select * from test where d > to_date('1970-01-01 00:00:00', 'YYYY-MM-DD HH24:MI:SS');
SQL> SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY());
PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 35 | 0 (0)| 00:00:01 |
| 1 | TABLE ACCESS BY INDEX ROWID BATCHED| TEST | 1 | 35 | 0 (0)| 00:00:01 |
|* 2 | INDEX RANGE SCAN | IDX_D | 1 | | 0 (0)| 00:00:01 |
SQL> explain plan for select * from test where ts > to_date('1970-01-01 00:00:00', 'YYYY-MM-DD HH24:MI:SS');
SQL> SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY());
PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 35 | 0 (0)| 00:00:01 |
| 1 | TABLE ACCESS BY INDEX ROWID BATCHED| TEST | 1 | 35 | 0 (0)| 00:00:01 |
|* 2 | INDEX RANGE SCAN | IDX_TS | 1 | | 0 (0)| 00:00:01 |
```
* Fix kondo
* Allow impersonated users to hit cache again
As part of this I refactored a bit: `impersonation` is now its own
module, used by the `advanced-permissions` module.
The real changes are in:
- `middleware.cache`, where we're no longer not caching impersonated users,
- `metabase.impersonation.middleware`, where we're adding a key to the
query if it'll be impersonated, and
- `metabase.query-processor.util/select-keys-for-hashing`, where we're
selecting that key as part of the query hash.
- a new test in `metabase-enterprise.impersonation.cache-test`, an e2e
test which verifies that impersonated users get a different cache than
non-impersonated users.
* Fix throwing when sandbox/gtap both enforceable
`enforce-impersonations?` was returning `false` because due to the
sandbox, we had "unrestricted" data access from another group (the
sandboxed group).
i refactored a bit:
- `enforced-impersonations-for-db?` now is responsible for throwing in the
sandbox conflict case, rather than `connection-impersonation-role`.
- this way, if there's a conflicting sandbox, we'll throw before we even
check to see whether those impersonations will be enforced (rather than
checking and deciding "nope, they have unrestricted access so we won't
enforce them.")
* kondo yay
* Recalculate metadata for derived models when updating Upload table
* Preserve modified metadata
* Test for display names being consistent
* Less ominous language
* Remove ace styles
* Remove ace-builds types
* Remove ace-builds alias from webpack and rspack config
* Remove ace mocks
* Remove ace packages
* Remove ace references in jest config
* Remove ace global in jest mock
* Use lezer parser for custom expressions (#53540)
* Add @lezer/generator
* Add missing value on Token
* Add script to generate lezer grammar parser
* Add @lezer/lr
* Add more tests to tokenizer
* Add lezer grammar for custom expressions
* Generate lezer parser for custom expressions
* Convert TOKEN and OPERATOR to enums
* Make Token type more specific
* Add isReference to identifier token
* Typo
* Add more tests for tokenizer
* Add tokenize alternative that uses lezer grammar
* Remove old tokenizer
* Update location of grammer in generate rule
* Add test cases for tokens that share a prefix
* Merge tokens when necessary
* Only add character error when text is 1 character long
* Reduce the need to manually merge nodes
* Stop the iteration when node is found
* Update tests to be more specific
* Simplify tokenize
* Uncommit edge cases
* Generate parser as js
* Move previous tokenizer tests into expressions folder
* Add typing to unit test
* Allow escaping [] in reference
* Add test for escaping closing bracket
* Handle invalid parses manually
* Add support for signed expressions
* Document parser more
* Add custom tokenizer for Reference
* Make tests more explicit
* Fix e2e test
* Merge old test suite into tokenizer test suite
* Move @lezer/generator to devDependencies
* Custom expression suggestions (#53679)
* Add new suggesters for custom expressions
* Add test for joined fields
* Move complete into separate folder
* Split up completions into separate files
* Split up completion tests too
* Copy more tests from legacy completer
* Fix types
* Lezer highlight for custom expressions (#53651)
* Rename generated parser to lezer
* Add lezer highlighting rules to parser
* Add HighlightExpression component
* Use HighlightExpression in expression HelpText
* Add type annotation to highlighter
* Add test for highlight
* Simplify unit test
* Move syntax highlighting to shared module
* Rename util.tsx to util.ts
* Add monospace font variable
* Use monospace font from theme for syntax block
* Use relative font size in HighlightExpression
* Use mantine var for border-radius
* Add test id to example
* Fix tests that rely on example content
* Move highlighting code to metabase/ui/syntax
* Replace ACE with codemirror in custom expression editor (#53746)
* Rename generated parser to lezer
* Add lezer highlighting rules to parser
* Add HighlightExpression component
* Use HighlightExpression in expression HelpText
* Add type annotation to highlighter
* Add test for highlight
* Simplify unit test
* Move syntax highlighting to shared module
* Rename util.tsx to util.ts
* Add monospace font variable
* Use monospace font from theme for syntax block
* Use relative font size in HighlightExpression
* Use mantine var for border-radius
* Add test id to example
* Fix tests that rely on example content
* Move highlighting code to metabase/ui/syntax
* Replace custom expression editor with CodeMirror editor
* Add test mock for CodeMirror
* Colocate custom expression tests (#53812)
* Remove unused DISPLAY_QUOTES
* Colocate custom expressions diagnostic tests
* Colocate custom expressions process tests
* Colocate custom expressions recursive-parser tests
* Colocate custom expressions typeinferencer tests
* Colocate custom expressions completer tests
* Colocate custom expressions format tests
* Colocate custom expressions resolver tests
* Colocate custom expressions index tests
* Colocate custom expressions pratt compiler tests
* Colocate fuzz tests with custom expressions code
* Update paths for fuzz tests
* Fix types in fuzz test
* Fix types in process test
* Remove unused var
* Remove unused var in generator
* Merge types for custom expressions (#53902)
* Add missing expression types to metabase-types
* Add more explicit types to matchers
* Use Expression type from metabase-types everywhere instead of custom type
* Move matchers to separate file
* Further use Expression types for matchers
* Add types for Metric and Segment matchers
* Fix types in format
* Fix missing types
* Clean up imports
* Split string helpers into separate submodule
* Split identifier helpers into separate submodule
* Avoid importing from index file in expressions module
* Render suggestions and help text in new custom tooltip (#53760)
* Add new help text component for custom expression editor
* Add new suggestions listbox for custom expression editor
* Add helper to find help text by function name
* Add enclosingFunction helper that is more forgiving to broken syntax
* Add new tooltip component for custom expression editor
* Add extension to render custom react tooltips
* Render suggestions in custom tooltip
* Export custom types
* Remove old expression editor popover components
* Add typings to process
* Add missing shortcuts
* Fix example font size
* Fix type imports
* Fix click on helptext
* Stop click propagation
* Fix process types
* Remove superflouous optional chain
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix types whith optional expressionIndex
* Use imports instead of React namespace
* Use index for key
* Rename util to utils
* Remove unused box shadow
* Remove brackets around placeholder names
* Resize tooltip to avoid it overflowing the window
* Fix line height for args
* Fix missing key icons
* Fix chevron alignment
* Fix types
* Move Listbox resizing classes to Tooltip
* Fix completion template for functions with no arguments
* Add unit tests for enclosingFunction
* Add darker background to expression example
* Remove bold from function call
* Fix imports
* Fix HelpText tests
* Add custom color for highlighted syntax argument
* Use colors from design system
* Fix blurring when switching to a different tab
* Remove trailing comma
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Split up effect
* Remove background from template arguments
* Improve tooltip position
* Fix Popover position when selection changes
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix custom expression editor e2e tests (#53761)
* Fix e2e tests for custom expressions
* Fix unit test
* Fix e2e tests
* Fix types
* Fix moveToStart and moveToEnd by using Home/End
* Add test for popover position
* Remove JSDoc in favor of inline comments
* Add examples of side effects
* Remove shouldContain helper in favour of value()
* Add comment to wait
* Use CustomExpressionEditor helper instead of @formula
* Add wait to reduce flakiness
* Add back in the useful comment about popover position
* Remove superflouous cy.viewport call
* Add helper to accept completion
* Rename helper to verifyHelptextPosition
* Convert comments to cy.logs
* Make sure we don't avoid testing the regression
* Allow customising key with which to accept completion
* Use acceptCompletion helper
* Fix tooltip position bug
* Remove unnused wait
* Add selectCompletion helper
* Fix helper using H
* Add better emulation of copy/paste in reproduction
* Ignore ts error
* Multiline formatter for the custom expression editor (#53973)
* Make type name meaningful
* Add more tests for not isnull, etc
* Remove unreachable code
* Add isEmpty matcher
* Pass query to format tests
* Add stronger types to formatStringLiteral
* Fix CallOptions import
* Implement multiline formatter as a prettier plugin
* Reimplement legacy formatter as new formatter with infinite print width
* Use the new formatter in the expression editor
* Remove legacy expression format files
* Document utils
* Simplify Expression type further
* Fully type test cases
* Remove cast in test
* Add more tests and fix formatting
* Use recurse helper to recurse into path children
* Prefix logical operators
* Improve formatting of logical connectors
* Wait for expression editor to be ready
* Avoid passing unknown html props
* Use selectCompletion helper
* Remove unused @ts-expect-error
* Always close helptext when suggestions are being shown
* Keep completion identity constant
* Fix unit test
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add condition for join alias to match source alias
* Add test for zoom order of joined fields
* Improve nil handling in test
* Remove unneeded dependency
* WIP Check not using generous? true
* Remove `:generous?` option
* Get rid of unused require
* fix: nested field refs from join-alias with spaces
Fixes: #37600
Join aliases get escaped by drivers (truncation and/or tr space to
underscore). However, when join fields are referenced in future stages,
the references were not being renamed and so the query was being created
with invalid columns from the previous stage.
* Fix tests, drop methods namespace
* Re-assoc info after conversion
* Just change query, none of the mw keys
* Add before/after mw for escape join alias to be able to use and capture converted pmbql queries
* Fix kondo errors
* Fix nest-breakouts
* Fix breakouts tests
* Field rename should happen in add-alias-info.
* Fix tests
* Don't use card name for collection name
* Format rows for sqlite
* Uncomment assertion
* lots of UI fixes
* adjusting focus color with theme
* updating z-index in tests
* only change focus color
* updating loki screenshots
* e2e adjustments, PR feedback
Rather than dragging the Custom column left by a fixed -100 px, drag the Total column right 10 px past the right edge
of the Custom column's bounding box. The idea is to make the test more flake resistant since (1) we drag based on the
size of the target column header rather than a fixed offset, (2) since there are no more columns to the right of
Custom, it's OK if we overshoot the distance slightly.
* wip - got the FE working w/o a real endpoint for DQs created from a dashboard
* progress
* wire in real endpoint with some ugly code
* get working for copy form and clean up code
* fix failing unit tests
* fix failing e2e tests
* fix tab select hidden when creating a question directly from a dashboard
* adds unit test coverage
* add e2e tests
* fix console warn when tab id field is not used in formik form
* small fixes
* fix flakes from two modals sometimes being opened at the same time
Follow up to #52990
When `future` is called clojure uses `Executors.newCachedThreadPool`
This means that the Threads remain in a pool for up to a minute.
That means that if `future` gets called anywhere before
`classloader/the-classloader` is called then that `Thread` can remain in
the pool with a bad classloader. Then when BigQuery's `future` tries to
`Class/forName` `BigQuery$JobOption` it will be calling it from the
classloader context of that initial `future` call and not be able to
find the class.
This adds a very early call to `classloader/the-classloader` in `init!`
and a safety call to it right in BigQuery. In the long term we should
probably be much more strict and explicit about the executors we use.
* Add @codemirror/legacy-modes package
* Add @codemirror/lang-html
* Add @codemirror/lang-javascript
* Add @xiechao/codemirror-lang-handlebars
* Add CodeBlock component
* Add custom metabase syntax highlighter for codemirror
* Use custom highlighing
* Render code sample using CodeBlock
* Add @codemirror/lang-python
* Add python support
* Restructure code
* Add classname to CodeBlock
* Deduplicate codemirror packages
* Add erb support
* Improve syntax highlighting
* Remove fold gutter
* Remove unused TextEditor
* Remove unused ace imports
* Add highlighted text to CodeBlock
* Fix tests
* Fix e2e test
* Disable current line highlighting
* Add the nonce plugin to Codemirror
* Sort languages
* Rename variable to be more descriptive
* Remove TODO
* Rename util to utils
* Find all occurences of highlighted text in source
* Use explicit classes in highlight
* Remove reliance on overriding based on language
* Remove null in signature
* Remove unused classnames
* Initialise highlighted text statefield
* Add e2e test for disappearing highlighted texts
* Replace all jade references with pug
* Remove erb support and replace it with HTML
* Add comment as to what is being tested
* Remove javascript as a code language
Nested fields (Mongo, Postgres JSON unfolding, etc.) can have the same
name within one table, if their parent paths are different.
Including the parent (if any) as part of a Field's `serdes/hash-fields` makes the hashes truly unique.
When you add a summary clause (aggregation or breakout), any `:fields`
clauses are removed from the stage and its joins.
However the reverse order was not correct: a join clause added after
summaries would keep its specified `:fields` clause, or get the default
`:fields :all`. This change fixes that quirk.
* use the last saved collection if the use saved to a collection more recently than saving to a dashboard
* make changes not affect models/metrics, fix e2e tests
* unit tests
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* Enable no-color-literals rule
* Fix all strings in the file being highlighted if only one of them violates the rule
* Silence violations
* Revert "Fix all strings in the file being highlighted if only one of them violates the rule"
This reverts commit 11a3c6aca7.
* Fix all strings in the file being highlighted if only one of them violates the rule
* revert the removal of shallow cloning a dashboard, prevent shallow copy if dashboard has a dashboard question
* load dashboard in copy modal to avoid cases where provided entity is incomplete and does not contain dashcard data
* type fix
* add e2e test coverage
* remove old e2e test
* address pr feedback and add test coverage for case
* fix lint
For i18n, we had a lot of things like
```
(tru "this user's string will be broken")
```
During i18n, we use `java.text.MessageFormat` to format messages. These
have a great idea: you can format messages with parameters like `{0}`, or
escape these cases with single-quotes like `'{0}'` when you actually
want the string "{0}". And of course, you can escape a single-quote
itself just by doubling it, so "''" becomes a "'".
All this makes sense. But then they had a problem. What if there's an
unescaped single-quote in the argument to MessageFormat?
A lesser program might have thrown an exception here, but not
MessageFormat! Nay! We must persist, lads, not give up at the first sign
of trouble!
Instead, let's just decide that the programmer probably *meant* to close
that single-quote! So, for example, "This user's message is totally
frickin broken" is interpreted AS IF you'd written "This user's message
is totally frickin broken'" (which ends up as "This users message is
totally frickin broken"), which - doubtless - is what you intended.
Unfortunately there are *some* few cases where - IT TURNS OUT - we don't
actually want this behavior! For example,
```
(deferred-tru "Key to retrieve the JWT user's email address.")
```
becomes "Key to retrieve the JWT users email address." Which is slightly
annoying.
So: let's have a kondo rule that will enforce statically that you don't
have unescaped single-quotes in your template strings for `tru`,
`deferred-tru`, `trs`, or `deferred-trs`.
* Fix underlying-records and pivot drills for mapped model metadata
* Fix zoom-in.timeseries drill for mapped model metadata
* Fix zoom-in.binning drill for mapped model metadata
* Fix zoom-in.geographic drill for mapped model metadata
* Replace breakout->filterable-column with breakout->resolved-column
* Fix sort drill for mapped model metadata
* Restrict fix to possible-model-mapped-breakout-column?
* Add e2e tests
* Move shared e2e helpers into outer scope
* Add tests for source-card-type and source-card-is-model?
* Fix indentation
* Remove unused require
* Rename local vars from filterable-column to resolved-column
The old name was a holdover from before the function got renamed.
* Cleanup some->> -> some->
This pipeline used to have more parts that required some->>. Since the only remaining func wants the resolved-column
threaded as the first arg, switch to some-> instead and drop the now useless lambda.
Fixes QUE-681.
Drops `metabase.lib.convert` from the "exported" namespaces under `lib` module, and refactors all usage to
prefer `lib/query` and friends. These require a `MetadataProvider` where `lib.convert/->pMBQL` does not, but
since most of the calls were in QP middleware there was already one handy.
A few API handlers were calling `lib.convert/->pMBQL` and some of those require new `MetadataProvider`s.
Note that currently `lib.convert/->pMBQL` and `->legacy-MBQL` are still exported from `metabase.lib.core`,
and used a few places in the QP! That's an unfortunate temporary measure, and I've started the ball rolling
on alternative ways to factor both of the main offenders (like `annotate/aggregation-name` called from some
drivers).
* PoC
* Base pass
* First pass
* First pass
* First pass
* First pass
* Basic test
* Basic test
* Basic test
* Basic test
* Basic test
* Basic test
* Basic test
* Basic test
* Basic test
* Basic test
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* reduce usage of legacy tooltip
* update Link component tooltip
* update e2e specs
* fix tooltip usages
* lint
* unit specs
* forward ref
* specs
* smart scalar tooltip color
* use tooltip secondary color
* drop flaky legacy tooltips and popovers from loki tests
* one more usage
The Product decision was to point all existing upsells
(that were previously leading to https://www.metabase.com/cloud)
to the existing, central Cloud upsell in the app.
This makes it easier to initiate the migration directly from within the app.
* Removing unused require
* Raising download-row-limit min to 1048576
* Removing unused require
* Raising download-row-limit min to 1048576
* Fix default to not be nil; remove useless unit test
* Change to 2**20 - 1 to account for header row
* Restore more efficient row total values in downloads row limits test
* Improve docstring for env var
* Testing if removing broken unit test fixes all of CI
* Adding new MB_ATTACHMENT_ROW_LIMIT env var; fixing unit test that dependend on old mb-row-download behavior
* Fixing nil support in min call by moving safe-min to u/safe-min and using that over clojure.core/min
* Fixing condition that tests if we're in a subscription email to include check for :pulse
* Removing default value for download-row-limit; updating limit logic; fixing test
* Fixing cljfmt
* Addressing PR feedback
* Fix circular dependency
* Fix imports for modules
* Fixing broken import
* Adding missing doc string
- allow not specifying a `collection_id` when creating a new
card (inferring it from the `dashboard_id` instead)
- correctly check perms on the *dashboard*'s `collection_id` when moving
a card to a dashboard
* Clean up the upsell exports
Resolves CLO-3582
* Get rid of styled components in upsells
Resolves CLO-3567
* Define urls in a single place
Resolves CLO-3566
* Tidy up the cloud upsell
* Tidy up `UpsellBanner`
* Nit: Format the sentence correctly
* Use safer approach when setting color for the `UpsellGem`
* sdk debug info component
* remove extra BUILD_TIME
* move the component to components/public so it gets automatically wrapped in the nextjs compat layer
```clojure
upload-test=> (driver/allowed-promotions :redshift)
{}
upload-test=> (mt/transitive {})
nil
```
just have to expect the allow-list of `{}` rather than nil
* Fixes the display by checking if the Mongo binary data is a UUID and converting it if so.
* Fixes the filtering by changing how we infer the base type so we can differentiate UUIDs and binData
* Fixes the existing UUID tests so that they use BSON UUIDs instead of strings.
* Add implicit actions schema change test
* For now, we define the column set as being determined by models
- this can break actions if new required columns are added
* Type changes however should be reflected in the derived parameters
* Drops, renames behaviour remains undefined for now
* Fix field :type change being ignored by implicit action params
* Add round trip test for implicit action params
* set up our own sort order, do some initial renaming
* consolidate viz stories into one folder
* Move app section specific components / patterns into app folder
* mark old components as deprecated in storybook
* group ui components under components
* add additional new components to proper folder
* move utils to components directory
* move data display to components
* move old core components into components directory w/ caveats
* move parameters stories into components
* move old segmented control to deprecated folder
* move deprecated radio to components folder
* move status components into components/feedback
* tweak order of pages
* run test-visual:loki-update
* rename colors_Default to Colors_Default
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* chore: restore no-restricted-imports rules for the sdk
* fix imports + add eslint-ignore for when we can't
* address feedback on PR: export BadgeProps + use types we already had
* also change enterprise/frontend/.eslintrc
* add comment to explain why we copy paste them
* "name": "@storybook/testing-library" -> @storybook/test
* add eslint ignore for an import used in a utils file for stories
* On csv upload allow column promotion from boolean to int / float
* Fix incorrect arglist
* Add postgres support for bool -> int, float upload migrations
* Improve compatibility with old drivers
* Rename alter-upload-columns to alter-table-columns
* Formatting, sparking joy
* Flip alter-column-using-hsql-expr arg order
* Docstring tweak alter-column-using-hsql-expr
* More symmetry with existing tests, redshift
* Add driver changelog note re `alter-table-columns!`
* Prefer ::driver over :default
* Allow drivers to specify the promo allowlist
* Prefer parser errors for redshift upload promos
* Ensure promo tests respect driver allowlist
* Make index-test/transitive available as a test util
* Add transitivity test
* Add allowlist note about transitivity requirement
* s/upload-promotion-allowlist/allowed-promotions/g
* Prefer case over condp
* Unlearn boolean/predicate habit
bikeshed: Personal style vs metabase style:
e.g I prefer flags/booleans to not have ?, reserving that only for
questions i.e predicates
* Fix alignment
* s/internal-allowed-promotions/translate-type-keywords/g
* Fix wierd comment line break
* Fix missing potemkin import
* add trigger button class with visibility
* add visibility clauses
* Remove empty file
* Colocate visibility logic
So we don't have to repeat this logic on every call site
* Fix E2E tests
* Remove leftover prop type
* Passing ref since we need it for popover
* Fix the rest of the failed e2e tests
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Convert back to original type when binning by day (mysql)
* Use h2x/->datetime instead of custom one in mysql
* Interpret base-type into database-type
* Remove println
* Undo using base-type to derive database-type
* Fix mysql date binning to maintain type
* Improve postgres bucketing test
* Fix test
* Refactor for clarity
* Whitespace
* Track type info when casting unix timestamps and date strings
* Track type when casting temporal strings (psql)
* Revert changing test (fix instead)
* Add very helpful comment
* Add very helpful and useful comment
* GH job to auto-upgrade backend deps weekly
* Run on push to this branch
* Run on this branch
* Disable reviewers
* Should open PR against master
* Exclude Kondo for now
* Try tagging backend dev with `reviewers`
* Run against `master`
* Set `master` as the base
* Add backend team as reviewers for deps.edn
* Enable auto-merge
* Fix antq config
* Use squash-merge
* Try creating PR using GH CLI
* Set commit username
* Set GH token
* Manually create branch
* Better branch name
* Checkout master before creating commit
* Set fetch-depth 0
* Leverage git stash
* Git diff
* Fix
* Unique branch name
* Push branch to origin
* Re-enable auto-merge
* Use GH Actions bot as author
* Open separate PRs for each dep bump
* Fix GH Action config
* Update Action name
* Fix script
* Final cleanup
* Bump clj-kondo/clj-kondo from 2024.11.14 to 2025.01.16
Inspect dependency changes here: https://github.com/clj-kondo/clj-kondo/blob/v2025.01.16/CHANGELOG.md
* Bump clj-kondo/clj-kondo from 2024.11.14 to 2025.01.16
Inspect dependency changes here: https://github.com/clj-kondo/clj-kondo/blob/v2025.01.16/CHANGELOG.md
* Bump Methodical & Toucan 2 for updated Kondo configs
* Update some Kondo configs
* Fix Kondo errors
---------
Co-authored-by: Cam Saul <github@camsaul.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Metabase bot <metabase-bot@metabase.com>
* feat: Reduce cost of initial field scanning
Setting a Field `has_field_values` to `:autolist` moves from classify
based on fingerprinting to the initial Field creation only based on
semantic and base types.
Querying for field values and receiving too many values or too many
bytes will no longer delete FieldValues or change Field
`has_field_values` but rather set the `has_more_values` to true.
During sync, if a `has_field_values` is `:autolist` or `:list` but
FieldValues is nil, we will not fetch a new cache of FieldValues. This
way, the only way to cache new FieldValues would be by triggering an
adhoc request to `GET /api/field/:id/values`.
* Unmark FKs as auto-list
* Fix tests to manually activate field values for param_values
* Add search to e2e helper when filter has more
* Fix e2e tests
* Fix helper
* Fixing more tests
* Fix e2e test
* Add upsell component for performance tools
* Make `tools` a regular route
When the "audit_app" feature flag is missing, we will show the upsell.
This now applies to all non-enterprise routes, which removes the need
for the route guard.
It also greatly simplifies the tools "plugin" system.
The logic is now as follows:
1. If the feature flag is enabled, the route shows enterprise component
(erroring questions page)
2. if the feature flag is not present, the route falls back to the upsell component
* Get rid of styled components
* Center the tools upsell
* Delete test case that's not valid anymore
* Add e2e coverage
* WIP: e2e querying test for preemptive caching
* WIP restructure BE tests for better coverage
* Misc cleanup
* revert some unneeded changes
* Fix tests
* Revert some typo fixes to avoid polluting this PR
* Refactor cypress test
* Try again to fix Cypress test in CI
* Address feedback
* fix tests
* fix tests again
* Fix#50266
* Add a boolean cast
* Add repro for #50266
* Improve test name
* Add RTK Query API middleware to storybook store
* Add the effect cleanup
* Fix "Promise was collected" error
* Fix other hooks
* Add missing RTK Query middleware in storybook
* Fix "Promise was collected" error
* Clean up the effect
* Fix tests
* Fix storybook setup correctly
`entity_id` columns on Databases, Tables and Fields are new in #53076.
This PR uses `entity_id` on Fields as their `:ident` in the metadata,
rather than the concatenated names previously used.
The names were a bit too fragile, since occasionally people rename
Fields or Tables in the appdb to retain the identity of something that
got renamed in place in their data warehouse. (That isn't exactly
supported, but it's also not blocked.)
With random (or hashed and saved) `entity_id`s for Fields used as their
`:ident`s instead, references to fields will not need to be updated if
fields or tables are renamed but their `entity_id`s preserved.
**On memoization**
Since the `entity_id` is `NULL` for existing fields and they're only
being populated in the background, we need to populate them on demand
when reading metadata. Since fields depend on their tables and tables on
their databases, this meant a lot of extra appdb reads at first.
This change includes memoization added to `serdes/hydrated-hash`, such
that any table or database only gets read once, and memoized forever.
There's a big comment about the memory impact of this; in summary it
should be safe since this is on demand, and holds only DBs/tables
and not fields.
A later PR will add a background job as mentioned, that will populate
all the `entity_id` columns with hashed values. After that, the cache
will always be empty.
Note that this memoization is not going to work if a table or field is
renamed before the background job catches up! I consider this an
acceptable loss - renaming fields is an admin action that requires
editing the appdb. The guide for doing that can include making sure all
`entity_id` fields are populated first.
* fix: use standard user order sort for group list
* fixup! remove alba from user fixtures
* fixup! better user model creation
* fixup! remove test-user? filter
* fixup! remove added test and use exisiting sort test
* fix: allow password reset if google sso disabled
* fixup! correct arg type to keyword in message test
* fixup! remove empty line
* fixup! add public docstring
* fixup! add temp google client id
* Use NativeEditor.type in test
* Use NativeEditor.type in all tests that aren't
* Stop using .as(editor) for native editor
* Do not return anything from startNewNativeQuestion to avoid confusion
* Fix more tests
* Fix movetostart/movetoend
- Ctrl + A is "select all" on non-apple machines, but on apple machines it's same as "home"
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Error explicitly when sandboxing/impersonation configs conflict
* Fix tests
* Fix kondo warning, use consistent ns aliases
* Fix another test
* Cleanup, consistent aliases
* Add an extra check of *current-user-id*
* Wrap east/west filterBounds to [-180,180] in LeafletMap.jsx
* Split filters that cross the antimerdian into two parts in update-lat-lon-filter
If the bounds cross the 180th meridian, split the filter into two parts ORed togehter. One part covering the
[west, 180] range and the other covering [-180, east].
* Add tests for bounds crossing the antimerdian
* Rename longDelta to lngDelta to conform to standard leaflet abbreviation
* Review feedback: use shorthand notation for east and west filterBounds
* Add e2e tests
* Fix existing e2e test
realMouseDown expects to have x and y coords passed in an options map.
* Use common pinMapSelectRegion test helper in new e2e tests
* wip logic to handle 'personal' for collectionId when creating dashboards
* unit test for useCreateDashboardApi supporting 'personal'
* add e2e test for createdashboardmodal
* collection browser support for 'personal'
* some actual decent logic
* change jest tests to use root collcetion
* fix type error in stories
* fix create dashboard modal support for root
* yeah i shouldn't have commited a .only
* self review, better docs + removed un-used type
* typo
* another step of self review: changed comments and removed useless
beforeEach
* updated docs
* Apply suggestions from code review
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* re-format file after suggestions were applied from gh
* refactor(e2e): use cy.wait().then instead of let =
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Enable pivot tables with only pivot columns
* Re-add one condition so that tests pass
* Tweak logic
* Fix logic again, and add cypress test
* Try again to fix conditional
* Enable results metadata during qp/process-query as it seems to also control insights which we need for slack alert trendlines
* Removing outdated alerts w/ models test
* fix: add logging during connection health checks
Trying to get ahead of #52458, add logs around startup health checks
* Add log for when health check is queued
* Use with-generator
* Replace for tu.rng
* Add ref to workflow, rename env
* Rename vars
* Remove former random ns
* Typo
* Adjust log level
* random-query-from
* Use random-query-from
* Adjust random exception handling
* Add with-gentest kondo hook
* Initial version of with-gentest
* Logging in test assertions
* Workflow, env var names, logging
* Remove redundant code
* Newline
* Seed for non exceptional queries
* Naming
* Cleanup
* Atom for seed
* Cleanup
* Whitespace
* Kondo
* Temporarily fix schema and suppress full-join, to be rebased out later
* Update .github/workflows/generative-query-test.yml
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* Update test/metabase/test/gentest.clj
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* Remove stale comment
* Use tu.rng
---------
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
* Improve ID test
* Improve ID test
* Improve ID test
* Improve ID test
* Add a test
* Between test
* Between test
* Between test
* Between test
* tests
* tests
* tests
* tests
* tests
* tests
* tests
* tests
* tests
* tests
* tests
* tests
* tests
* tests
* tests
* Make the UpsellBigCard more flexible
The product requirements for the Cloud upsell dictate that
the main CTA doesn't directly lead to Metabase website like in
all other cases. It opens a Migration modal instead.
UpsellBigCard had to be adjusted to enable that.
* Add Upsell Cloud component
* Resolve CLO-3576
* Use the cloud upsell instead of the previous CTA
* Adjust the UI
* Update stories
* Update related test
* Fix e2e test
* Add context for translators
* Adjust the vertical spacing
* Don't :return-drills-for-dimensions? for header-context-only drills
Dimensions can never have a "missing" value, i.e. (nil? value), and so none of these drills will ever apply when
called with a context built from dimension-contexts.
* Add comment with updated table for Pivot Cell clicks
#53076 added `:hook/entity-id` for databases, tables and fields, but this is incorrect behaviour.
The sync process periodically detects new tables and fields and adds them to the appdb. If multiple instances of Metabase are connected via serdes, and linked to the same data warehouses, then eg. a new table would get duplicated before the serdes process would be run.
Instead, the `entity_id`s should be based only on the hashes, the same as serdes for databases, tables and fields.
* Cast date binning back to date in postgres
* Remove unnecessary conversion to timestamp
* Fix existing test.
* Test that temporal columns don't change type when binned.
* Fix bucketing for postgres days
* Improve comment about casting interval addition
* Select single field in test for robustness
* Tag :to_timestamp to avoid other areas adding a type
* Whitespace
* Remove blank line at end of file
* Don't cast if type is nil.
* Add alias to native query in test
* Fix redshift
* Pass temporal unit when listing quick-filter options.
* Use with-temporal-bucket like a gentleman
* Use lib.temporal-bucket/temporal-bucket
* Whitespace
* check if the project is a nextjs project
* add nextjs imports and exports for nextjs projects
* add comments
* add notice on analytics.css and link to guide
* fix typo
* update snippet
* normalize path
* quote import path
* update generated directory
* use fileName for sampleComponents
* remove getImportSnippet
* use nextjs imports from the compatibility layer
* remove use client and export default from snippets
* fix checkIsInNextJsProject import
* add use client directive to pages with useReducer
* add suggestions on module resolution
* make import paths reasonable for next.js
* add example _app.tsx snippet
* trim snippets for Next.js
* show custom app snippets separately for next
* auto-generate custom app
* fix file extension
* do not show the react import step in nextjs
* add default root layout or app
* refine root layout generation logic
* add comments on root layout generation
* add next.js route to app and pages router
* add notes on API routes
* only generate AnalyticsPage when not using nextjs
* use writeFileSync
* create directory if it does not exist
* add messages on route created
* remove AppProps type from snippets
* inform users that a custom root layout has been added
* wait for root layout snippets to be ready
* use the correct layout file name in message
* cleanup snippet message
* highlight the layout file
* temporarily remove the moduleResolution warning
* cleanup messages helper
* cleanup exports
* cleanup analytics page snippet
* restore analytics.css
* add more info on the default provider
* fix whitespace in imports
* add a use-client directive to EmbeddingProvider
* highlight the analytics demo route
* add a sans-serif font by default
* do not show the Express server message if Express server is not there
* use glob to simplify path matching
* make analytics page snippet function pure
* make custom app snippet function pure
* add support for src directory
* add source prefix in router resolution
* trim layout snippet header and add eta
* strip the source prefix from import paths
* make src dir a constant
* add lightweight margin reset to sample analytics.css
* consistent source prefix
* parameterize the route name
* cleanup snippet whitespace
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* rename to getNextJsPagesWrapperOrAppWrapperSnippet
* consolidate snippets file
* inline hasNextJsCustomAppOrRootLayout
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Fix#50731
* Revert first fix as design team said there's a better way
* Remove HackyInvisibleTargetFiller
* Use max-width instead of width to prevent overflow if there's less than 300px available
* Add a repro for #50731
When wrapping literal values in :value clauses, fields with string-type ids (for example, fields after an aggregation)
were getting their type info from source-metadata. When the field in question was a postgres enum, the source
metadata was missing important data. Fetching info from both source-metadata and the type info of field-ref fixes
that.
These three entities have previously used their names as their serdes
keys. The names are still used for serdes but we want opaque, stable
`entity_id`s as well so that they can be used as `:ident`s in the field
refs overhaul. Then the name of a database, schema, table or field can
be changed without its `:ident` changing, and therefore without breaking
any questions that reference it.
* Updating the v2 serdes baseline export - it has `entity_id`s now
* Updating the exported audit database; it has entity_ids now
* Fix MySQL appdbs
Use lower case for all table/field names; hard-code the schema
"PUBLIC". (The actual schema for appdb is ignored.)
* run all smoke tests on OSS tests regularly
* skip certain smoke tests for oss v EE
* or not and
* union
* update prerelease tests
* run smoke tests on OSS except EE
* Add upsell for whitelabel/appearance
The most important change this PR introduces is a whitelabel route that's now present in the OSS and starter plans. How this works now:
- When the whitelabel token feature is not present, the route `/admin/settings/whitelabel` shows the upsell page
- When the whitelabel token feature gets activated, the route `/admin/settings/whitelabel` gets redirected to `/admin/settings/whitelabel/branding` and the upsell page is not visible anymore.
* Mantine v7-1 squash
update deps
update UI configs
easy fixes
happy little typescript
updates after rebase
test fixes
fix rebase issues
admin-2, collections, custom-columns, dashbaord and dashboard-cards suites
fix clause popovers
fixing rebase issue
more rebase sadness:
more e2e
autocomplete adjustment
fixing nested popovers
adjusting a couple more pickers
more e2e fixes
fix mantine provider, searchbox errors
unit test adjustments
theme colors, filter labels
Relative Date Picker
more unit test fixes
* e2e adjustments
* [MANTINE] ci: find css variables that are used but never defined (#52947)
* ci: find css variables that are used but never defined
* only search in frontend and enterprise/frontend
* whitelist mantine css variables
* temporarily allow some css variables
* read mantine variables directly from mantine css file
* UI Regressions
* lint and type fixes. green unit test
* Cherry pick updates to the never defined variables script (#53049)
* Update bin/find-never-defined-css-variables/find-never-defined-css-variables.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* test: checks that it doesn't confuse definitions with usages
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* more adjustments for e2e tests
* adjustments
* embedding sdk green, maybe unit tests?
* loki regressions
* test(sdk): migrate theme provider test to e2e to work with mantine v7 (#53290)
* test: migrate theme provider test to e2e to work with mantine v7
* re-put there things i removed by mistake
* preview and type fixes
* I think I'm making this better?
* fixing colors, unit tests
* scoping css vars and static classes, other things
* class prefix, scoping input css
* small adjustments
* shamelessly poach Nicolo's commit
* small fixes, should make Kamil happy :)
* finally green!
* type check
* chartsettingsinput fixes
* silly number types
* Adjustments for loki tests
* type fix
* undo Question View Download changes
* adjusting smart scalar
* updating loki reference
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* smooth out some rough edges for local e2e tests
* note for the normies
* remove premium embedding token
* add note about h2 app db
* update config file stuff
* Allow passing native extras to withDifferentDatabase
* Pass empty nativeExtras when switching databases
* Add reproduction for 53299
* Do not pass nativeExtras to withDifferentDatabase
* Do not update native-extras when chaning databases
* Remove with-extras overload on with-different-database
* Remove native-extras for with-different-database call sites
* Remove required throw from test
* Remove collection requirement from test
* convert pgarray to java array
* convert array to vector early
* add test and todo comment
* fix whitespace
* add tests for caching and emailing
* whitespace
* use feature and test other drivers
* update test names
* deduplicate native array methods
* remove vertica
* remove http from array agg test
* remove let
* fixes offset images when dnd-ing columns in dashcards
* we should be grateful for e2e tests
* -> mantine style prop
* fix this in the sortable list while I'm at it
* add clarification message to the cli
* update cli startup message
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* line-wrap the cli startup message
* use metaba.se/sdk for docs link
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Remove silent flag and re-enable multi-arch build
Check what's going on here
* test code
* fix test code
* Update Dockerfile_ubuntu
Fixes the multi-arch build
* Update Dockerfile_ubuntu
Have a feeling that the ca-certificates-java is making a mess here
* Update uberjar.yml
Add the logging to the process
* Update Dockerfile_ubuntu
* Update uberjar.yml
* Update run_metabase.sh
ensure the path lives with the user
* Update uberjar.yml
Alpine 21 is multi-arch, we should have that as well in the build script along ubuntu
* Update uberjar.yml
push explicitly all platforms
* Update uberjar.yml
Update actions to see if platform flag works
* Update uberjar.yml
This should push the multi-arch images without building again 🤞🏻
* Update uberjar.yml
return to old versions, just to keep stuff more consistent
* Update uberjar.yml
fix typo
* remove test code
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* feat: use liquibase session lock instead of transaction lock
Prevent the migration lock class of errors in non-h2 dbs by using a liquibase session lock instead of a transaction lock.
This type of lock would clear automatically when the connection closes for any reason.
Using this approach there is no need for the metabase release lock command except with h2.
The force migration command still is applicable though, as it does more things than just release the lock.
See https://github.com/blagerweij/liquibase-sessionlockFix#50212
* fix: mark migration 109 as none db
From Nathan Voxland:
From a liquibase-standpoint, deleting the changesets are just fine. As long as you don't accidentally reuse the id+author+path combo in the future. Databases that already ran it will not care that it's missing and new databases will just never see it to know. So it's more of an our-standards questions.
Alternately, you can set dbms="none" to keep it in the file but not have it actually apply anywhere. That doesn't change the checksum
* fix: error when force-releasing non-h2 locks
* test: add session lock test
* WIP - lots of streamlining
* revert dotenv addition
* ci fix
* some fixes
* make component testing work
* set some default ports
* fix component test ci
* fix all the e2e tests
* fix the other sdk tests
* update e2e docs
* fix embedding more
* fix tagged tests
* documentation
* split user and derived options
* better exit behavior
* more error handling
* revert saml change
* ❤️ typescript and booleans
* fix some stuff
* fixes
* stupid lockfile
* maybe keep the lockfile
* remove unnecessary diff
When fetching a param's possible values, we were originally looking for concrete columns and checking result metadata.
However, if the query is against a model and has an aggregation, the thing you are filtering on isn't a concrete field
and probably isn't in result metadata. Checking filterable-columns lets us find the appropriate field in that case. The practical result is that we actually find remapped values for a param when it is linked to an aggregated query against a model.
* Check cycles on PUT to /card/:id
* Remove extra protection from saving cycles.
* cljfmt
* kondo
* Better threading
* Cleaner tests
* Fix test
* Better naming in metabase.api.card/dataset-query->query
* Rename `dataset-query` arg to `dq-fragment`.
* Normalize query on backend.
* Throw exceptions with user messages.
Also rework `dataset-query->query`.
* Unshadow card-type in doseq
* Make consistent use of query/dataset-query
* Remove normalization from lib.query
* Don't shadow `query`.
* Rename `dq-fragment` to `query-fragment`
* `timelines` module
* Move some render-related timeline stuff into the timeline module
* Remove unused /api/card/:id/timelines endpoint
* PR feedback and test fixes 🔧
* Create dashboard questions for x-rays
* Don't make subcollections for autogened dashboards
Before, for an admin we'd create a structure like:
```
- Our analytics
- Automatically Generated Dashboards
- A look at Accounts
- Dashboard
- Card 1
- Card 2
- ...
```
Or, for regular users, a structure like:
```
- Your personal collection
- A look at Accounts
- Dashboard (A look at Accounts)
- Card 1
- ...
```
Now, we'll create the following structure:
```
- Our Analytics (OR Your personal collection, for non-admins)
- Automatically Generated Dashboards
- Dashboard (A look at Accounts)
```
Other autogenerated dashboards will be placed alongside.
* Fix question not found error showing after navigating from dashboards
* Add interactive dashboard tests
* Add editable dashboard tests
* Fix failed tests
* Add an assertion to ensure we display ad-hoc question
* update enable-embedding-sdk based on embedding-app-origins-sdk
|enable-embedding-sdk key exists? | then we:
|---------------------------------|---------
|false | no-op
|true | set enable-embedding-sdk to true when `embedding-app-origins-sdk` exists
* actually need to set it to false when embedding-app-origins-sdk dne
* reformat liquibase migration
* make the query run on h2, too
* per-db sql + update comment
* use preconditions to simplify update sql
* Allow private slack channels to be listed for alerts/subscriptions.
Requires the groups:read scope to see private channels
* manifest.yaml changed so new installations get it by default
* things are kept compatible if the token does not have the scope
* you can manually add the scope to the app in slack.
private channels will then up when the channels are refreshed
* Update the slack manifest.yaml inlined in the docs
* Add re-installation guidance to slack docs for `groups:read`
* Ensure slack token is set in conversations-list test
* Refactor conversations-list to take :private-channels opt
* Add refresh-channels-and-usernames! test to check groups:read behaviour
* prevent duplicated SET_EDITING_DASHBOARD dispatches from overriding edit dashboard state if the editing dashboard is the same dashboard
* small improvements
* add test coverage
* impl pr feedback
* unmark as flaky: should not reload the model for record in the same model
* unmark as flaky: scenarios > search should allow users to paginate results
* unmark as flaky: scenarios > model indexes should be able to see details of a record outside the first 2000
* unmark as flaky: should be able to search model index values and visit detail records
* Remove the custom permissions upsell component
* Implement upsell for advanced permissions
This applies to both:
- `/admin/permissions/data/group` and
- `/admin/permissions/data/database/` routes.
* Remove the old feature from the enterprise feature level permissions
* make dashboard dirty if tabs are edited and add test coverage for leaving dashboard after editing a dashboard or its tabs
* handle rename tab, add rename tab test coverage, simplify test coverage
* makes ci ts happy
* clean up test a bit
* rename helper
* Return column corresponding to aggregation value in SmartScalar getCurrentMetricData
Closes#46168
The returned clicked object had the `value` from the aggregation cell, but the `column` from the date dimension, which
breaks the quick filter drill (and probably others).
Instead, return the `column` that corresponds to the `value`.
* Add e2e test for smartscalar quick filter drill
* remove unsafe lifecycles from visualization component
* add e2e tests
* remove extra comma in test helpers
* only run tests under strict mode when a flag is passed
* extract console assertion to a test helper
* update tests for known console errors
* swap new and previous props
* combine old state for transform props to state
* combine initial state
* use getDerivedStateFromProps to fix failures
* only store a subset of lastProps to compare
* remove timeline events check from getDerivedStateFromProps
* reset state when timeline events changed
* get rid of mutation in strict mode
* remove existing state spread from state updates
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* remove deriveStateFromProps from constructor
* use sdkProviderProps to pass provider props
* replace UNSAFE_componentWillReceiveProps with getDerivedStateFromProps
* add unsafe lifecycle check for DashboardGrid
* lastProps is already part of DashboardGrid state
* optional chain state.initialCardSizes
* remove state comparison logic
* fix misleading hasVisibleDashcardsChanged condition
* fix editor issue
* remove unused dashcardData from lastProps
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Add generative test
* Use per test generator with adjustable seed
* Add workflow file
* Temporarily add trigger
* Make rand function work with default generator
* Update log4j2 test config
* Info log level
* Whitespace
* Add iteration count and env to make run on-demand
* cljfmt
* cljfmt
* remove unsafe lifecycles from visualization component
* add e2e tests
* remove extra comma in test helpers
* only run tests under strict mode when a flag is passed
* extract console assertion to a test helper
* update tests for known console errors
* swap new and previous props
* combine old state for transform props to state
* combine initial state
* use getDerivedStateFromProps to fix failures
* only store a subset of lastProps to compare
* remove timeline events check from getDerivedStateFromProps
* reset state when timeline events changed
* get rid of mutation in strict mode
* remove existing state spread from state updates
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* remove deriveStateFromProps from constructor
* use sdkProviderProps to pass provider props
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
When replacing a join, we were taking a valid join (with a correctly set up join alias), dropping the alias, creating
a new alias, and then inferring which fields belonged to the new alias. This caused issues when joining models using
a field that could plausibly belong to either model, because we inferred that the lhs join condition belonged to the
rhs model. This in turn confused the join naming code and caused us to refer to the lhs as "previous results" instead
of the correct table name.
With the new code, if we want to rename the new join, we explicitly replace the old alias with the new alias instead
of just removing the new alias and inferring what goes with what. This avoids potentially incorrect inferences and
fixes this issue (and possibly others?).
Last year we had a P1 where a high-visibility page in Brazilian Portuguese was broken. Let's add some basic e2e tests that check high-visibility admin/webapp pages (accessible in one click from the homepage) in our most popular locales.
* update ui to show trash icon, tooltip, and undo message if the dashcard being removed is the last dashcard for a dashboard question
* add e2e test, fix types
* Add pivot-drill-details
Similar to filter-drill-details and combine-columm-drill-details.
pivot-drill-details returns the info needed by FE to render the pivot drill popover UI.
* Merge pivot-types into pivot-drill-details
* Add e2e test
* Revert unintentional whitespace change
* Re-enable Breakout by e2e test
* Re-enable multi-stage pivot drill tests
* Add pivot drill test for query with explict join
* Add hostname support to BigQuery
* Fix formatting
---------
Co-authored-by: Case Nelson <snoe@snoe.ca>
Co-authored-by: Case Nelson <case@metabase.com>
Fixes#52843.
* Redshift: Adding an explicit dep on `aws-java-sdk-core`
This is listed as an optional dep, but it seems that at least one class
from it (`com.amazonaws.util.StringUtils`) is now dynamically required
by our driver - tests were throwing `NoClassDefFoundError`s without it.
Redshift tests are passing locally with this dependency added.
* fix(entity-id): show 404 for entity id that starts with digits
* simplify logic by removing support for sequential ids with slugs for the
tab
* clean up the tests and rename things
Year of era aggregations weren't working because our legacy malli schema didnt recognize it as a valid temporal unit.
Adding it to the schema makes it work.
* partition indexes and use separate queries
* test no exception thrown
* improve helper func name
* improve test
* remove test debugging
* address pr comments
This should ensure tasks will find classes in our plugin system. The
issue arose when `submit-task!` was called to sync an unhidden bigquery
table before any query was run on the main thread before hand.
This ensures that if either of these tasks runs before classloader has
been set, that their threads will inherit it.
`task.cache/pool` was found as an audit of other threads that may
exhibit this behavior.
Testing this from source is not possible right away since the
classloaders behave so differently between jars and sources.
* ci: find css variables that are used but never defined
* only search in frontend and enterprise/frontend
* Update bin/find-never-defined-css-variables/find-never-defined-css-variables.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* test: checks that it doesn't confuse definitions with usages
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
These date from the beginnings of the `metabase.lib` codebase,
and are unused outside of a couple of tests.
I deleted the tests for these functions specifically, and updated a few
other tests that used these functions to retrieve columns on a query.
Prework for #49182.
* Start refactoring Field to TS
* Revert "Start refactoring Field to TS"
This reverts commit 88af47fdac.
* Add description input
* Use translation
* Make placeholder match data model page UI
* Add repro for #37907
* Fixing xlsx pivot table exports to respect table column names
* Adding fallback to column title to use display_name if custom title is empty; fixing kondo error
* Fix cljfmt spacing
* Adding unit test for xlsx pivot exports respecting custom column name
* Improving logic to be more idiomatic
* OpenAPI dox generation part 2
* Fix docstring
* metabase.billing.billing => metabase.billing.api
* Fix dox generation code fr fr
* Revert changes to dox for now
* Remove define-routes
* Fix missing routes
* Fix even more missing routes
* Third time's a charm
* Remove extra newlines at end of file
Databricks has static test data, by creating views we run the risk of
concurrent test runs seeing the temporary views or those views not
getting cleaned up if a job gets killed.
This is part of ENG-14545
* Add upsell component for better support
* Implement upsell for better support on a troubleshooting page
* Snow better support upsell only for OSS
* Update import path
* Remove whitespace
* Account for smaller screens
* fix(entity-id): don't make call to the card endpoint if the entity id
starts with digits
* prevent `isXXXPath` to return true if the path is the one of entity ids
* Remove flaky tags in e2e/test/scenarios/native/native.cy.spec.js
* Remove flaky tags in e2e/test/scenarios/native/native_subquery.cy.spec.js
* Remove flaky tags in e2e/test/scenarios/native/native-reproductions.cy.spec.js
* Remove flaky tag from admin test that uses native editor
* Remove flaky tag from visualization test that uses native editor
* Remove flaky tag from dashboard test that uses native editor
* Add helper to clear browser cache
* Reduce flakiness in autocomplete test
* Remove flaky tag in e2e/test/scenarios/native-filters/native-filters-reproductions.cy.spec.js
* Fix SDK cross-version testing failure after test util refactor (#52900)
* Revert the code before the fix
* Revert "Revert the code before the fix"
This reverts commit be11146bf1.
* Another attempt by checking out the whole e2e dir
* Can't checkout twice it will override everything 😭
* Reset everything back to the start
* Add missing files
* Keep only the latest webpack config
* Clarify the temporary bandaid with a comment
* Ensure when we cut 54, we won't ran into SDK cross-version testing
Future work on the refs overhaul #49182 changes how queries are
represented. To enable testing that new behaviour at a per-test level,
we can't let global queries like `lib.tu/venues-query` be defined once
and reused forever: in that case, tests of the new refs would see global
queries defined under old refs, and vice versa.
This change transforms the global queries, cards and other helpers in
`lib.test-util` from static `def`s to functions, so they build the
queries etc. in the context of the current test.
(If this introduces a performance hit for the lib tests, we can update
the functions to cache the two versions up front and choose between them
at runtime. So far it does not seem to have made a significant impact.)
* Use CodeMirror 6 (#50484)
* Add @uiw/react-codemirror
* Add language plugins for codemirror
* Add @codemirror/autocomplete
* Add basic codemirror editor
* Add selection handler to CodeMirror editor
* Set height
* Add basic styling to codemirror editor
* Add custom selection background
* Add simple theme with font settings
* Move extensions to a separate file
* Implement right click on selection
* Add language support
* Add @codemirror/language
* Add @lezer/highlight
* Add custom syntax colors
* Hightlight tags
* Add highlight for matching bracket
* Add completions
* Add snippet completions
* Make the editor full height by default
* Remove duplicate dependencies
* Match autocomplete colors with ace editor
* Render completion items in subgrid
* Tweak completion margin
* Use nesting
* Add hover state on completions
* Make getTagAtCursor more sueful
* Move matchTagAtCursor to utils
* Avoid work when there's no need
* Implement onCursorMoveOnCardTag
* Move autocomplete into language helper
* Disable language completions when inside a tag
* Allow open ended tags to return without suffix
* Add testid to codemirror editor
* Update test helpers
* Fix native subquery tests
* Fix native reproductions
* Fix snippet tests
* Remove ensureAceEditorClosed helper
* Add nativeEditorType helper
* Fix action tests
* Do not use native editor helper for custom expression test
* Add more native editor helpers
* Fix model metadata tests
* Use better type helpers
* Add more TODO comments
* Fix sharing test
* Simplify e2e helper
* Use native editor type helper
* Fix clearNativeEditor helper
* Fix missing namespace
* Update Loki Snapshots
* Remove unused lang extension
* Be explicit about h2 engine
* Nest language plugin return
* Export useLazyGetCardQuery
* Add completions for referenced card columns
* Split up autocompleters
* Simplify completion info
* Use cached cards
* Move useMemoized to util
* Add TODO comment about engine type
* Fix subquery test
* Use helpers in native reproduction
* Fix e2e test meta helper
* Do not assume we've seen all suggestions when narrowing down
* Fix selectAll
* Unskip unbroken test
* Fix broken reproduction
* Increase the delay for autocompletions to kick in
* Fix nativeEditorSelectAll on non-Mac systems
* Rename completer
* Move completers to separate file and make them handle their own dependencies
* Add autocomplete endpoints to metabase/api
* Use autocompletion hooks from metabase/api
* Reuse AutocompleteMatchStyle type
* Simplify helper
* Whitespace formatting
* Test sub-limit completion
* Fix reproduction
* Add tests for matchTagAtCursor
* Remove commented out code
* Typos
* Clarify why we disable rtk-query's cache
* Simplify cx call
* Use Cypress.platform over navigator.platform
* Tweak fold caret position
* Pass font to cm-content too
* Add synteax highlighting for the name token
* Rename AutocompleteSuggestionsRequest to AutocompleteRequest
* Fix broken overflow on notebook preview
* Add more fonts to monospaceFontFamily
* Use the same line height as the ACE editor
* Autocomplete much faster
* Handle overflow in completion meta
* Use shared object
* Add padding to editor scroll pane
* Add more fonts to monospaceFontFamily
* Set line-height to normal
* Fold on parens, braces and brackets
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Namespace native editor e2e helpers (#51093)
* Add namespace for helpers
* Replace H.nativeEditor with namespaced version
* Replace H.focusNativeEditor with namespaced H.NativeEditor.focus()
* Replace H.nativeEditor completions helpers with namespaced versions
* Replace H.nativeEditorSelectAll with namespaced version
* Replace H.nativeEditorClear with namespaced version
* Replace H.nativeEditorType with namespaced version
* Use NativeEditor.type where possible
* Fix imports
* Replace H.focusNativeEditor with namespaced H.NativeEditor.focus()
* Use NativeEditor.type where possible
* Use NativeEditor.clear where possible
* Do not call NativeEditor
* Add missing backspace in helpers
* Fix case of selectall
* Remove use of focus().as(...)
* Fix NativeEditor.get call
* Fix e2e test
* Remove focus where it is wrong
* Fix text replacement test
* Add small delay
* Remove ACE version of the Native editor (#51437)
* Remove sql_behaviour
* Remove getEngineNativeAceMode
* Remove CARD_TAG_REGEX
* Do not re-export ACE_EDITOR_ID
* Remove AceEditor
* Remove ACE_EDITOR_ID from NativeEdiror mock
* Remove AceEditor reference in test
* Remove intermediate editor
* Update Loki Snapshots
* Fix formatting
* Fix broken test
* Various fixes for the native editor revamp (#52321)
* Deduplicate schema completions
* Align description to the right
* Add keyword description to comletions
* Render completion description in normal font
* Set indentation to use tabs
* Remove boost from completions
* Stop cmd+enter from entering a newline
* Use same indentation as previous line when pressing Enter
* Remove empty resize handler in CodeMirrorEditor
* Add defaults to NativeEditor.type
* Make config parameter optional
* Add custom keymaps for next and previous completions
* Add custom interpolations for previous and next completion
* Add helpers up and down arrows
* Add reproduction for #48712
* Add tests for suggestion issues
* Allow completing from inside a word
* Add unit test for useSchemaCompletions
* Add unit test for useSnippetCompletions
* Add unit test for useCardTagCompletion
* Add unit test for useReferencedCardCompletion
* Accept completion with tab
* Do not indent with tab, but just insert a tab
* Use Lib.Query in CodeMirrorEditor
* Simplif CodeMirror editor prop and ref names
* Add simple unit test for CodeMirrorEditor
* Use isNotNull helper
* Use Lib.Query in NotebookNativePreview
* Handle nil in engine
* Handle empty engine
* Use correct meta kay in test
* Reduce flakiness in dashboard test
* Reduce flakiness in native test
* Add completion for query locals
* Make database optional
* Add test for locals
* Add test for command enter
* Add tests for indenting with tab
* Handle non-darwin platform in test
* Allow typing {tab}
* Add test for indentation
* Handle quoted locals
* Add typing to source
* Improve syntax highlighting for mongo
* WIP: add custom keywords completion for mongo
* Make collection_id optional
* Remove test return
* Handle SQLDialct shape
* Handle keyword completion all in the completer
* Lift completers out of language plugin
* Move language helpers to separate file
* Simplify keywords extraction
* Add tests for keyword completion
* Add more mongo keywords
* Fix quoted local test
* Add suggestion tests for mongo
* Fix e2e test with NativeEditor
* Update Loki Snapshots
* Fix e2e test with NativeEditor
* Revert "Update Loki Snapshots"
This reverts commit 7d1ea82579.
* Revert "Revert "Update Loki Snapshots""
This reverts commit fd7b552770.
* update screenshot
* Fix autocomplete API calls in the new native query editor (#52688)
* Enable autoFocus
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Sync state in useStringFilter
* Sync state in useBooleanOperatorFilter
* Sync state in useBooleanOptionFilter
* Add FilterModalResult type
* Fix setTab type
* Make comments work as JSDocs
* Move FilterModalResult
* Add filter modal context
* Drop FilterModalHeader and use FilterSearchInput directly
* Start to use context in BooleanFilterEditor
* Start to use context in TimeFilterEditor
* Start to use context in DateFilterEditor
* Start to use context in CoordinateFilterEditor
* Start to use context in NumberFilterEditor
* Start to use context in StringFilterEditor
* Start to use context in DefaultFilterEditor
* Stop drilling isSearching
* Fix tests
* Stop drilling version prop
* Remove FilterModalBody and FilterModalFooter exports
* Make naming consistent
* Remove props drilling
* Remove props drilling
* Remove props drilling
* Inline FilterTabContent into FilterModalBody
* Revert "Inline FilterTabContent into FilterModalBody"
This reverts commit cfd5738e52.
* Remove props drilling
* Sort attributes
* Consolidate imports
* Fix a mistake
* Improve JSDoc
* Revert "Sync state in useBooleanOptionFilter"
This reverts commit 33a979dfdf.
* Update approach after merging refactoring branch
* Add a repro for string filters
* Go back and forth on search input
* Assert operator
* Move effect to concrete hook
* Add test for boolean filters
* Fix time filters
* Revert "Move effect to concrete hook"
This reverts commit 82b94f4345.
* Move effect outside use-time-filter
* Improve naming
* Remove TODO
* Extract setup48319
* Rename setup48319 to setupCustomColumns
* Rename handle* to on*
* Fix coordinate filters
* Fix default filters
* Fix number filters
* Revert 1st approach
* Use key prop
* Remove detailed tests and leave only one
* Use current value
* Add a test for "version" hack
* Use only version for the key
* Rename version to remountKey and add explanatory comments
* Rename forceDescendantsRemount to forceRemountDescendants
* Rename
* fix ultra wide tooltips (closes#52199)
* all tooltips wrap now, and better test
* sturdier test
* use setup() in test case
* removed unncessary code path
* use features instead of hard coding drivers in tests
* fix formatting
* fix details-name-is-db feature
* bad conn details helper instead of features
* address pr comments
* exclude vertica from materialized view tests
* fix whitespace
Update YAMLs for 53 release.
Move all questions into usage analytics dashboards. In some cases, questions had to be duplicated in order to be moved to dashboards.
* update copy and avoid showing two messages about moving when going between collections and dashboards in different collections
* pr feedback
* impl pr feedback
* Add dashboard empty states
* Separate empty states from the actual dashboard content
* Fix public dashboards view
* Add translation context
* Localize the `alt` tag
* Make sure the content doesn't get cut off
In sql server, language settings change what day weeks start on. The sql server driver assumes that weeks always
start on sunday (ie the US default), which means that if you change the db to french, our math is off and we start
calculating weeks and days of the week wrong. This adds back in an offset based on DATEFIRST to fix that.
* POC of entity id aware route
* wip e2e tests
* clean up logic
* changing approach as everything was broken
* revert unwanted changes
* Revert "revert unwanted changes"
This reverts commit f6ce10ac7b.
* reset routes to master
* /by-entity-id/ routes
* update tests to check for subpaths and search param preservation
* cleaning up un-needed code
* simplified even more the logic by merging the two config maps to one
array
* pass down router instead of single props
* split the tests to their files
* fix typo in test
* adds a test for a unlikely to be needed scenario
* fix import after merging master
* fix e2e copy
* /by-entity-id/ -> /entity/ as per product decision
* tag the test as flaky as it fails depending on the order of test files
being run
* Auto-rewrite defendpoint forms in dataset.clj
* Auto-rewrite defendpoint forms in dataset.clj
* Auto-rewrite defendpoint forms in dataset.clj
* Test fixes 🔧
* More test fixes 🔧
* A few more test fixes.
* Test fixes 🔧
* Test fixes 🔧
* Finally handle form-encoded bodies correctly
- Cleaned up session management
- Don't send "new device" email from embedded logins
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* Avoid cartesian products in pivot csv exports
* Get row and column sorting working properly
* Add a bunch of unit tests
* Test fixes, various tweaks to better match the FE output
* Fix another test
* address feedback
* Add FilterModalResult type
* Fix setTab type
* Make comments work as JSDocs
* Move FilterModalResult
* Add filter modal context
* Drop FilterModalHeader and use FilterSearchInput directly
* Start to use context in BooleanFilterEditor
* Start to use context in TimeFilterEditor
* Start to use context in DateFilterEditor
* Start to use context in CoordinateFilterEditor
* Start to use context in NumberFilterEditor
* Start to use context in StringFilterEditor
* Start to use context in DefaultFilterEditor
* Stop drilling isSearching
* Fix tests
* Stop drilling version prop
* Remove FilterModalBody and FilterModalFooter exports
* Make naming consistent
* Remove props drilling
* Remove props drilling
* Remove props drilling
* Inline FilterTabContent into FilterModalBody
* Revert "Inline FilterTabContent into FilterModalBody"
This reverts commit cfd5738e52.
* Remove props drilling
* Sort attributes
* Consolidate imports
* Fix a mistake
* Improve JSDoc
* Rename handle* to on*
* Auto-rewrite defendpoint forms in session.clj
* Auto-rewrite defendpoint forms in session.clj
* Remove &request anaphor and make it a normal arg
* Add test
* Defendpoint 2.1
* base-handler => handler
* Fix Kondo errors
* Make sure async exception handling is done correctly
* Remove defendpoint-async
* Rewrite all the rest of the remaining defendpoints except for those in public
* Remove card endpoints from this PR
* Update src/metabase/setup/api.clj
Co-authored-by: bryan <bryan.maass@gmail.com>
* cljfmt
* Test fixes 🔧
---------
Co-authored-by: bryan <bryan.maass@gmail.com>
* Drop Java 11/17 Support
* Update Dockerfile
Change the Java version of the builder as well
* mention that we are dropping support for earlier Java in docs
---------
Co-authored-by: Luis Paolini <paoliniluis@gmail.com>
Co-authored-by: Alexander Solovyov <alexander@solovyov.net>
* throw down ui
* pixel freaking perfect
* wire in real endpoint for fetching dashboard question candidates
* wire in move endpoint
* loading and error handling for mutation
* persist info step acknowledgement
* optionally show item in menu, fix flickering of info modal is acked previously, add a toast on successful bulk move
* fix some of the failing existing tests
* add unit tests for collection menu
* add unit tests for main migration modal
* add e2e test
* fix failing unit test
* fix another failing unit test
* fix candidates result being stale if dashcards added/removed from dashboards creates/removes candidates
* adjust url of feature
* copy feedback
* fix type-check
* scope feature to only admins
* use pagination to fetch count but not items when detecting if to show the migration tool menu item
* test + style improvements, impl pr feedback
* fix unit tests, add unit coverage for is admin
* Add lib.native/has-template-tag-variables?
* Disable drills for native queries with template tag variables
Closes#49051
* Re-export lib.native/has-template-tag-variables? from lib.core
* Add tests for disabled drills for native queries with variables
* Review feedback
Closes: https://github.com/metabase/metabase/issues/52116
Note that of icu4j was already a dependency, via graalvm.
We have just made it explicit now, ~~and updated to the latest version.~~ ([upgrading version breaks rendering](https://github.com/metabase/metabase/actions/runs/12926546646/job/36049844305?pr=52581))
```
org.graalvm.js/js 22.3.5
. org.graalvm.regex/regex 22.3.5
. org.graalvm.truffle/truffle-api 22.3.5
. com.ibm.icu/icu4j 71.1
. org.graalvm.truffle/truffle-api 22.3.5
. org.graalvm.sdk/graal-sdk 22.3.5
. org.graalvm.sdk/graal-sdk 22.3.5
. com.ibm.icu/icu4j 71.1
```
* Auto-rewrite defendpoint forms in src/metabase/api/api_key.clj
* Auto-rewrite defendpoint forms in src/metabase/api/api_key.clj
* Fix formatting
* Auto-rewrite defendpoint forms in api_key.clj
* Auto-rewrite defendpoint forms in api_key.clj
* updated some flaky screenshots
* simplified ci job
* updated types
* registered visualizations and added a way to register multiple visualizations in storybook
* replaced @storybook/testing-library with @storybook/test
* Paginate the DQ candidates endpoints
* Paginate in Clojure
I think this should be fine - the number of cards we're working with
should be relatively small, since this is working one collection at a time.
* CR response
* Auto-rewrite defendpoint forms in database.clj
* Some cleanup
* Update src/metabase/api/database.clj
Co-authored-by: bryan <bryan.maass@gmail.com>
* cljfmt
* Auto-rewrite defendpoint forms in database.clj
* Test fix?
---------
Co-authored-by: bryan <bryan.maass@gmail.com>
* Exclude DQs from DQ candidate endpoint
... oops
I also moved the `(= (:collection_id card) (:collection_id dashboard))`
check out of `card/sole-dashboard-id`, since it's not really part of
being a sole dashboard.
* Add `dashboard_id IS NULL` to the query
... instead of filtering them out in clojure.
This was already legal-ish, as [:* 1 [:sum ...]] was a valid argument to coalesce. However, you couldn't pass in the
sum directly. Under the hood, NumericExpressionArg allowed aggregations, but ExpressionArg didn't. Presumably, the
one is supposed to be a superset of the other, and this change made that actually be true.
* filter out collections and dashboards where can_write is false from recents tab in collection picker
* move modal filter out locations you can not save to from recents tab
* add e2e test
* pr feedback - make cleaner and add comment
* Auto-rewrite defendpoint forms in src/metabase/api/automagic_dashboards.clj
* Manual cleanup
* Auto-rewrite defendpoint forms in bookmark.clj
* Manual cleanup
* Auto-rewrite defendpoint forms in automagic_dashboards.clj
* Fix endpoints with no route params but no regexes
* Auto-rewrite defendpoint forms in automagic_dashboards.clj
* Auto-rewrite defendpoint forms in automagic_dashboards.clj
* Auto-rewrite defendpoint forms in automagic_dashboards.clj
* Don't include changes to dev namespace
* Revert changes to bookmark
* Some manual cleanup
* Show should be optional
* Auto-rewrite defendpoint forms in user_key_value.clj
* Auto-rewrite defendpoint forms in user_key_value.clj
* Auto-rewrite defendpoint forms in user_key_value.clj
* Cleanup + fixes 🔧
* Auto-rewrite defendpoint forms in cache.clj
* Auto-rewrite defendpoint forms in cache.clj
* Auto-rewrite defendpoint forms in cache.clj
* Auto-rewrite defendpoint forms in cache.clj
* Fix schema for PUT /api/cache
* Fix cache schema (part 2)
* Do not render "Make public" toggle in Create Action Modal
* Disable the "make public" toggle and add a tooltip explaining why it's disabled
- Use Mantine's Switch
- Use Mantine's Tooltip
* Add a repro for #51282
* Update test name
* Fix tests
* Update unit tests
* Add unit test repro for #51282
* Remove e2e repro for #51282
* Format code
* Format code
* fix unknown attribute error
* fix another error message about unknow HTML attribute
* fix invisible ellipsis when title is empty string
* added e2e test
* Update labels
* Revert "Update labels"
This reverts commit be632aec2e.
* Simplify the fix
* Update unit tests
* Update test
* Revert "Simplify the fix"
This reverts commit a9c267c4c0.
* Update labels
* Update test
* Make it work in native editor
* Revert "Update unit tests"
This reverts commit 3614a8ff2f.
* Update test
* Update test
* Update test
* Update test
* Using 1 as max value in minibar chart rendering logic if the column is pure percent
* Adding MiniBar unit spec
* Prettier --write
* fixing eslint errors
* polish specs
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
* Defendpoint 2 misc fixes
* More fixes 🔧
* Remove some requires for faster loading; always make all query params optional
* Fix comment
* Make :boolean default false as well
* Auto-rewrite defendpoint forms in activity.clj
* Auto-rewrite defendpoint forms in activity.clj
* Formatting
* Auto-rewrite defendpoint forms in activity.clj
* Auto-rewrite defendpoint forms in activity.clj
* Auto rewrite defendpoint ones
* My bird is typing
* Make sure response format matches old defendpoint format
* Cleanup
* Clean up alert tests
* Test fixes 🔧
* One last test fix 🔧
* Make actions execute body optional
As a follow-up to #52284, this PR adds lint rules banning the pattern that caused a P1 (#48144). These rules require us to use the browser default with `String.localeCompare` and related functions in the `Intl` library. Since we cannot pass these functions our own locale codes, since some of these are in the wrong format.
This is designed to be a good-enough, "80 percent of the value with 20 percent of the effort" solution, even though better solutions could be pursued such as aligning FE and BE around an international standard (BCP 47).
- [x] Tests have been added/updated to cover changes in this PR
This is a strange one. But in practice, this was still doing a seq scan
on postgres.
```sql
clean=# explain DELETE FROM query_execution WHERE id IN (SELECT id FROM query_execution WHERE started_at <= '2025-01-02 01:10:15.651839+00' LIMIT 50000);
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------
Delete on query_execution (cost=526084.17..1066675.96 rows=0 width=0)
-> Nested Loop (cost=526084.17..1066675.96 rows=50000 width=34)
-> Unique (cost=526076.73..526326.73 rows=50000 width=32)
-> Sort (cost=526076.73..526201.73 rows=50000 width=32)
Sort Key: "ANY_subquery".id
-> Subquery Scan on "ANY_subquery" (cost=0.00..522174.32 rows=50000 width=32)
-> Limit (cost=0.00..521674.32 rows=50000 width=4)
table scan >>>>> -> Seq Scan on query_execution query_execution_1 (cost=0.00..1704153.49 rows=163335 width=4)
Filter: (started_at <= '2025-01-01 19:10:15.651839-06'::timestamp with time zone)
-> Bitmap Heap Scan on query_execution (cost=7.44..11.46 rows=1 width=10)
Recheck Cond: (id = "ANY_subquery".id)
-> Bitmap Index Scan on query_execution_pkey (cost=0.00..7.44 rows=1 width=0)
Index Cond: (id = "ANY_subquery".id)
(13 rows)
```
The odd thing is that it is sensitive to what timestamp you use. Here's
the same query but with a timestamp in 2024:
```sql
clean=# explain DELETE FROM query_execution WHERE id IN (SELECT id FROM query_execution WHERE started_at <= '2024-12-19 01:10:15.651839+00' LIMIT 50000);
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------
Delete on query_execution (cost=225736.06..641819.90 rows=0 width=0)
-> Nested Loop (cost=225736.06..641819.90 rows=28490 width=34)
-> HashAggregate (cost=225724.95..226009.85 rows=28490 width=32)
Group Key: "ANY_subquery".id
-> Subquery Scan on "ANY_subquery" (cost=122917.34..225653.72 rows=28490 width=32)
-> Limit (cost=122917.34..225368.82 rows=28490 width=4)
-> Bitmap Heap Scan on query_execution query_execution_1 (cost=122917.34..225368.82 rows=28490 width=4)
Recheck Cond: (started_at <= '2024-12-18 19:10:15.651839-06'::timestamp with time zone)
-> Bitmap Index Scan on idx_query_execution_started_at (cost=0.00..122910.22 rows=28490 width=0)
Index Cond: (started_at <= '2024-12-18 19:10:15.651839-06'::timestamp with time zone)
-> Bitmap Heap Scan on query_execution (cost=11.11..15.12 rows=1 width=10)
Recheck Cond: (id = "ANY_subquery".id)
-> Bitmap Index Scan on query_execution_pkey (cost=0.00..11.11 rows=1 width=0)
Index Cond: (id = "ANY_subquery".id)
(14 rows)
```
And this is because the db engine knows that most of my data is around
2025 and will just hit the whole table rather than hit an index and then
go return most of the table.
The logic we want here is basically, "work up the index on created_at
until we get to a notion of recent".
If we add an order by, we're close to that and things go much faster
again
```sql
clean=# explain DELETE FROM query_execution WHERE id IN (SELECT id FROM query_execution WHERE started_at <= '2025-01-02 01:10:15.651839+00' order by id asc LIMIT 50000);
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------
Delete on query_execution (cost=851615.84..1392207.62 rows=0 width=0)
-> Nested Loop (cost=851615.84..1392207.62 rows=50000 width=34)
-> Unique (cost=851608.39..851858.39 rows=50000 width=32)
-> Sort (cost=851608.39..851733.39 rows=50000 width=32)
Sort Key: "ANY_subquery".id
-> Subquery Scan on "ANY_subquery" (cost=0.55..847705.98 rows=50000 width=32)
-> Limit (cost=0.55..847205.98 rows=50000 width=4)
-> Index Scan using query_execution_pkey on query_execution query_execution_1 (cost=0.55..2767566.54 rows=163335 width=4)
Filter: (started_at <= '2025-01-01 19:10:15.651839-06'::timestamp with time zone)
-> Bitmap Heap Scan on query_execution (cost=7.44..11.46 rows=1 width=10)
Recheck Cond: (id = "ANY_subquery".id)
-> Bitmap Index Scan on query_execution_pkey (cost=0.00..7.44 rows=1 width=0)
Index Cond: (id = "ANY_subquery".id)
(13 rows)
```
It caused problems as lint-staged stashes files and if developer is not familiar with the mechanism, he thinks that his work is lost. To avoid confusion we remove this git hook
* skips model popup
* fix types
* port to ts
* reorganize acknowledgement code
* improve ack api, add unit test, fix missing key prop console warning for question action menu
* move schema file to correct location, improve e2e
* Make sure we can load User KV schemas when in a JAR (#52246)
* Make sure we can load user KV schemas in a JAR
* A few minor tweaks
- removed some unused code
- added some more comments to clarify when each function is used
- move `update-user-key-value-schema!` outside the `doseq`
* Fix tests
* A bit more clarity
* actually fix tests
* move prod implementation below dev implementation; couple other tweaks
* fix fn name
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* remove only usage in test
* lint and unit fixes
* fix unit test for real this time
* but actually... unit tests are fixed this time
---------
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
When we replace a breakout, we also update any related order-by. However, we weren't updating all fields. This
meant that the order-by and breakout weren't identical. This was fine-ish on postgres, because they compiled to the
same thing. However, on google bigquery, it expects that group bys and order bys will refer to the AS alias instead
of just duplicating the code for the aliased thing. The differences between order-by and breakout meant that we
didn't realize that the one was supposed to match the other, and so we duplicated the code and bigquery barfed.
Syncing more fields fixes that.
**Joined columns**
- `join__${IDENT_OF_JOIN_CLAUSE}__${IDENT_OF_COLUMN}` for explicit
joins
- `implicit_via__${IDENT_OF_FK_COLUMN}__->__${IDENT_OF_TARGET_COLUMN}`
for implicit joins
These fulfill the contract for column `:ident`s:
- Unique identity within one stage of a query
- Globally unique
- Note that the same implicitly joined column (eg.
`Products via Orders.PRODUCT_ID`) might be referenced in
many queries, but it's the **same column** - same `display_name`,
types, etc.
**New test metadata**
In order to test these properly, I needed test metadata with more
permutations of FKs.
I've added three new tables to `metabase.lib.test-metadata`, which are a
simplified version of Github issues, users and comments. The tables are
`:gh/issues`, `:gh/users` and `:gh/comments`.
Some notable things:
- `:gh/users` has a `:type/Text` PK, ie. Github usernames
- `:gh/issues` and `:gh/comments` have `:type/UUID` PKs
- `:gh/issues` has **two** FKs to `:gh/users`: `:reporter-id` and
`:assignee-id`.
- `:gh/comments` has a **self FK**: `:reply-to`
- `:gh/issues :closed-at` is a `NULL`able `:type/DateTime` column, which
we lacked previously
* phase #1: remove all uses of openNativeEditor without a parameter
* phase #2: remove all uses with parameter
* remove useless test
* fix failing test cases
* fix failing tests
* add db id and native filter interaction
* fix flaky tests
* Support multiple operators in expectOp
* Refactor
* Show errors from parser
* Revert recursive-parser changes
* Pass human-friendly operator name to expectOp
* Fix typo
* Require comma if next token is not closing parenthesis
* Add a repro for #15892
* Update existing test
This drill thru was recently enabled for multi-stage queries in #51950, but there is a bug which causes the
QueryColumnPicker to fail if a join is present. Disable the drill thru for now while we work on a fix.
Closes#52221
Related to #52236
* Upgrade postgres app db version
The more the marrier
* Update docs/installation-and-operation/configuring-application-database.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Add repro for #50925
* Add a test
* Fix the case with whitespace following an incomplete token
* Add test case with bracket identifier following an incomplete token
* Add comments
* Thanks vscode
* Fix the case with bracket identifier following an incomplete token
* Simplify unit tests
* Fix the logic
* Update comment
* Fix it
* Refactor
* Throw different error when using a known name that is not a metric
* Drop checking for segments as they cannot be used in aggregation expressions anyway
* Add tests
* Remove redundant withName: true
* Wrap with describe
* Make naming consistent
* Improve copy
If a card with a default value doesn't actually require a value, letting people clear the default value can be useful.
However, if the card does require a value, letting people clear the default just throws an error. This is a
completely useless result, so we should use the default value in that case. See [here](https://metaboat.slack.com/archives/C0645JP1W81/p1719233020129649) for more info.
* Add a warning/disclaimer for H2 uploads
* Expand the component test
* Use the built in margin bottom
* Show the additional warning on hosted instances only
* Remove `mb/once` tag since we don't use it anymore
* Add `^:mb/old-migrations-tests` tag.
* Update GH Actions config
* Cleanup
* Keep the migrations test files in order
* Quote excluded tags
* Update running.md
Some little details about running Java apps on prod
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
* Update mysql.md
Add a note about vitess and planetscale
* Update docs/databases/connections/mysql.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* [Backend modularization] Remove `metabase.models`
* I guess we don't really need an ordered map
* Remove unneeded require of models
* Small fixes 🔧
* Test fix 🔧
* Fix metabase.api.embed.common
* Fix some insane code that was using symbols for raw table names
* We need to do model resolution for table names
* Don't test EE namespaces in OSS
* Secret handling in details
Secret handling in details was very confusing. This PR consolidates most
of the logic into models.secret around toucan2 lifecycles of
model/Database.
Some of the problems fixed here:
- path and value confusion. It was possible to swap between uploaded
files and file-paths and end up with the uploaded file being treated
as a file-path and leaking to users in logs, APIs, and UIs.
- Unclear management of secret properties in details. The
responsibility for taking in secrets in details, saving to a Secret,
and then ensuring those incoming secrets do not get saved in details
was spread out all over the place. This made it possible to save a
secret-value in details.
- Unclear hydration of secret values for connection testing. How a
driver was supposed to get a secret and which properties might be in
details was unclear and inconsistent. Often mangling strings and
files returned by Secret api was necessary.
- Encoding problems. Uploaded files were being mangled by taking bytes
into a string and back to bytes to save. Ensure that the bytes given
are the bytes saved.
* Fix merge
* Fix tests
* Fix db creation
* Fix db creation in other tests
* Manually add updated_at to database
* Fix tests
* Fix tests
* Fix after-select input
* Wrap in data-uri
* Fallback for -value if base64 encoding failed
* If not data-uri, then just turn string to utf8 bytes
* Poking at oracle failures
* Oracle wrap bytes in data-uri
* Don't clean secret properties from oracle
* Fix base64 typo and add test util for data-uri wrap
* Fix snowflake test
* Another snowflake test
* Update src/metabase/models/secret.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update src/metabase/models/secret.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update src/metabase/models/secret.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update test/metabase/test/util.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update src/metabase/util/jvm.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update src/metabase/util/jvm.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
---------
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update comment in automatic-insights drill to point to new github issue
* Move helpers from lib.drill-thru.common to lib.underlying
These funcs deal with "underlying" columns, so move them into lib.underlying.
aggregation-sourced?
breakout-sourced?
strictly-underlying-aggregation?
* Consistent naming for drill-thru multi-stage query tests
* Update multi-stage query drills tests to use new test helpers
* Homogenize apis for drill thru test helpers
Rename prepend-filter-to-stage to prepend-filter-to-test-expectation-stage and make the calling interface match
that of append-filter-stage-to-test-expectation.
* cljfmt
* Revise new table permissions logic to handle sandboxes
* tweak new perm logic
* make sure cond has a default
* fix comments
* tweak defenterprise conditions
* [Backend modularization] `query-analysis` cleanup
* Update Kondo config
* Add CODEOWNERS
* metabase.task.sweep-query-analysis => metabase.query-analysis.task.sweep-query-analysis
* Move test util
* Module linter should ignore my test-setup namespace
* Make sure query analysis init is loaded in tests that need it
* Deparallelize some of the tests
This change happened in peer
https://github.com/metabase/metabase/pull/51023 and is manifesting now
it seems.
The removal of the call to `(classloader/the-classloader)` is what is
hurting us.
Only clickhouse cloud instances can be set as a suitable upload spot so
i'm doing the following changes in the local jar repro to fix it up:
```clojure
(in-ns 'metabase.driver.clickhouse)
(defmethod driver/database-supports? [:clickhouse :uploads] [_driver _feature db]
true)
(in-ns 'metabase.public-settings)
(let [db_id 2, schema_name "default", table_prefix "upload_"]
(t2/update! :model/Database db_id {:uploads_enabled true
:uploads_schema_name schema_name
:uploads_table_prefix table_prefix}))
```
* [debug] add storybook helper
* Make error on InteractiveQuestion consistent
* Make error on StaticQuestion consistent
* [debug] static question stories
* [debug] InteractiveDashboard Stories
* Make error on InteractiveDashboard consistent
* [debug] StaticDashboard stories
* Make error on StaticDashboard consistent
* [debug] EditableDashboard stories
* Make error on EditableDashboard consistent
* Fix CreateQuestion not working after fixing entity ID states
* Add tests for InteractiveQuestion
* Add StaticQuestion tests
* Add InteractiveDashboard tests
* Add StaticDashboard tests
* Add EditableDashboard tests
* Make variable names consistent
* Extract resource not found error to a component
* Fix missing return
* Make storybook usable and reduce duplications
* Don't roll my own util
* Update error colors following the new design
* Update loki snapshots
* Fix unit test expectation to hex
* Make colors easier to understand with comments
* Update SDK error style
* Update error components copy
* Use Code from metabase/ui
* Remove change accidentally commited
* Fix InteractiveQuestion renders question not found briefly
metabase#51969
* WIP First attempt at passing start of week option to static viz bundle
* Cleaning debug lines
* set moment and dayjs start of week day setting in static viz
* reverting index.ts
* Adding backend test for checking first day of week setting is respected in static viz
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
* Update mysql.md
Add an additional flag as a note, since I've hit this one today with a customer
* Update docs/databases/connections/mysql.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Update activating-the-enterprise-edition.md
another little nitpick, since people tend to get creative with these things and then we end up debugging endless hours
* edit
---------
Co-authored-by: jeff bruemmer <jeff.bruemmer@gmail.com>
* initial impl
* better error handling, smooth scroll for adding existing cards, clean up of code in general
* add e2e test
* fix type error
* pr feedback
* reworks implementation to consume value from url then clear it, follows existing code patterns better
* clean up and test improvements
* fix e2e tests
* fix type issues
* pr feedback
* mostly working solution w/ tests
* add basic type impl, change interface to return obj from hook, update tests to account for both
* add types to setup mock endpoint fns
* hacks solution together to test if everything passes CI
* real implementation, removes hacked approach
* patch types
* add test coverage for patched package
* add one more unit test
* Update error colors following the new design
* Update loki snapshots
* Fix unit test expectation to hex
* Make colors easier to understand with comments
* Fix E2E tests
* Make setupModelPersistenceEndpoints work with multiple models
* Make test wait for data
* Improve tags invalidation
* Improve types
* Remove entity.ListLoader usage in UploadSettingsForm (#51909)
* Remove entity.ListLoader usage from ModelCacheRefreshJobs (#51910)
* Remove entity.ListLoader usage from SavedEntityList (#51914)
* Remove entity.ListLoader usage from SearchApp (#51915)
* Remove entity.ListLoader usage from QuestionList (#51912)
* Remove entity.ListLoader usage from SearchResults (#51913)
* Remove `entity.ListLoader` from `CollectionContentView` (#51916)
* Remove entity.ListLoader usage from CollectionContentView
* CI Failure notice improvements (#51885)
* don't notify slack on flaky job failing
also add more info to failure notice
* this is nicer
* add entity insertion library (#47895)
* add entity insertion library
- this lets us use `add-load/from-script`
* update add load examples
* Update year in the README (#51923)
* reset cache when starting cypress locally (#51921)
* Don't build FE for driver tests that don't need it; shave 2 minutes off of driver tests (#51884)
* Don't build FE for driver tests
* Try this again
* Consolidate MySQL job configs
* Oops fix conditional flags for test-driver
* More MySQL consolidation
* Oops
* Add a new job that waits for all other jobs (so we can require one single job going forward)
* Restore deleted code
* Try adding a Driver Tests (Summary) workflow
* How about a nice job name
* Don't love YAML inline array syntax
* Try fixing the build-static-viz configuration again
* Try fixing the build-static-viz configuration again
* Try this again
* Fix MySQL SSL config
* Try fixing the drivers summary workflow
* re-remove docs update block (#51928)
* upload title (#51931)
* Fix formatting of month-of-year when there are 53 weeks in the year (#51892)
* New deactivated_at field on users (#51765)
* New deactivated_at field on users
* fix migration syntax
* Setting alignment to top for icons in subscription and alert emails (#51886)
* Correct docstring for series (#51840)
Minor nitpicks on some endpoint docstrings
* Dont escape chars of header dashcards (#51880)
* fix rerun job
* feat(sdk): Interactive Question Chart Settings Dropdown #50976
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: bryan <bryan.maass@gmail.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Co-authored-by: Luis Paolini <paoliniluis@gmail.com>
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Remove `entity.ListLoader` from `CollectionItemsTable` - approach v3 (#51958)
* Break CollectionItemsTable down to avoid using Search.ListLoader
* ESLint
* Oops
* Fix auto-upload effect in ModelUploadModal
* Fix types
* Fix the effect running too often
* Remove redundant code
* Reduce diff
* Remove old EntityListLoader and rename new EntityListLoaderRtkQuery to EntityListLoader
* Remove PaginationState
* Remove EntityType
* Fix effect
* Fix effect
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: bryan <bryan.maass@gmail.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Co-authored-by: Luis Paolini <paoliniluis@gmail.com>
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Make setupModelPersistenceEndpoints work with multiple models
* Make test wait for data
* Improve tags invalidation
* Improve types
* Remove entity.ListLoader usage in UploadSettingsForm (#51909)
* Remove entity.ListLoader usage from ModelCacheRefreshJobs (#51910)
* Remove entity.ListLoader usage from SavedEntityList (#51914)
* Remove entity.ListLoader usage from SearchApp (#51915)
* Remove entity.ListLoader usage from QuestionList (#51912)
* Remove entity.ListLoader usage from SearchResults (#51913)
* Remove `entity.ListLoader` from `CollectionContentView` (#51916)
* Remove entity.ListLoader usage from CollectionContentView
* CI Failure notice improvements (#51885)
* don't notify slack on flaky job failing
also add more info to failure notice
* this is nicer
* add entity insertion library (#47895)
* add entity insertion library
- this lets us use `add-load/from-script`
* update add load examples
* Update year in the README (#51923)
* reset cache when starting cypress locally (#51921)
* Don't build FE for driver tests that don't need it; shave 2 minutes off of driver tests (#51884)
* Don't build FE for driver tests
* Try this again
* Consolidate MySQL job configs
* Oops fix conditional flags for test-driver
* More MySQL consolidation
* Oops
* Add a new job that waits for all other jobs (so we can require one single job going forward)
* Restore deleted code
* Try adding a Driver Tests (Summary) workflow
* How about a nice job name
* Don't love YAML inline array syntax
* Try fixing the build-static-viz configuration again
* Try fixing the build-static-viz configuration again
* Try this again
* Fix MySQL SSL config
* Try fixing the drivers summary workflow
* re-remove docs update block (#51928)
* upload title (#51931)
* Fix formatting of month-of-year when there are 53 weeks in the year (#51892)
* New deactivated_at field on users (#51765)
* New deactivated_at field on users
* fix migration syntax
* Setting alignment to top for icons in subscription and alert emails (#51886)
* Correct docstring for series (#51840)
Minor nitpicks on some endpoint docstrings
* Dont escape chars of header dashcards (#51880)
* fix rerun job
* feat(sdk): Interactive Question Chart Settings Dropdown #50976
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: bryan <bryan.maass@gmail.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Co-authored-by: Luis Paolini <paoliniluis@gmail.com>
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Remove `entity.ListLoader` from `CollectionItemsTable` - approach v3 (#51958)
* Break CollectionItemsTable down to avoid using Search.ListLoader
* ESLint
* Oops
* Fix auto-upload effect in ModelUploadModal
* Fix types
* Fix the effect running too often
* Remove redundant code
* Reduce diff
* Fix effect
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: bryan <bryan.maass@gmail.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Tyler Plude <tsplude@gmail.com>
Co-authored-by: Luis Paolini <paoliniluis@gmail.com>
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Don't select entire collections for permission checks
* Pull collection perm checks out as a CTE (wip)
* Cljfmt
* Refactor, fix tests
* Move comment, formatting fixes
* do a single query to fetch both the limited results and the total row count
* select more fields from collections
* inline values in the CTE to get it to work on H2
* fix e2e tests
* fix handling of limit = 0 edge case
* inline last ID
* Split Kondo lint fixes off into #51747
* Merge changes from #51660
* Reworked API documentation generation code
* Oops Markdown not YAML
* BIG documentation generation improvements
* Don't try to style tables ourselves
* Fix lint errors
* Fix tests 🔧
* Fix Kondo warnings
* Update OpenAPI dox generator to handle Defendpoint v2
* Update OpenAPI dox generator to handle Defendpoint v2
* Test fixes 🔧
* Fix uberjar compilation
* Static OpenAPI documentation generation
* change output to docs/api.html
* Update test
* PR feedback from @escherize
* add api.html
* Deprecate the old `defendpoint`
* Nuke the old version of the API documentation code (new version will be merged in #51872)
* Remove empty file
* Disable output validation in prod
* Fix the way we define defendpoint 2 handlers
* Fix deps.edn
---------
Co-authored-by: jeff bruemmer <jeff.bruemmer@gmail.com>
* Update docstring in zoom-in.timeseries
The comment is stale. Construction of dimensions from row data is now done in available-drill-thrus.
* Fix zoom-in.geographic drill thru for mutli-stage queries
* Fix return type schema for lib.underlying/top-level-column
* Add append-filter-stage-to-test-expectation to drill-thru test-utils
* Add tests for multi-stage queries to zoom-in.geographic tests
* add New Dashboard button and rework Create a new collection button to New Collection
* fix failing e2e tests
* fix unit tests
* allows users to create a new dashboard to save their dashboard question into
* remove usage of ref
* adds e2e test coverage for creating a dashboard on the go for a new question
* add unit tests
* fix type error
* clean up
* make test less flaky
* Fix quick-filter drill thru for multi-staged queries
* Add column-by-name and append-filter-stage to drill-thru test utils
* Add quick-filter tests for multi-stage queries
* Allow drill-thru tests to pass an update fn for :custom-{query,native,row}
The update fn will get passed the default value for the test case based on query-table and query-type. This allows
callers to easily modify or extend the default, rather than having to look it up or recreate it from scratch.
;; same as the standard test query, but with an extra filter stage
:custom-query #(lib.drill-thru.tu/append-filter-stage % "count")
* Use mu/defmethod and mu/defn for column-by-name and append-filter-stage
* Add lib.drill-thru.test-util/prepend-filter-to-stage
* Add lib.drill-thru.tu/test-drill-variants-with-merged-args
A helper function to make it easy to call test-returns-drill or test-drill-application with minor variations of some
base case (for example, drill behavior should remain unchanged when called with a query with filter stage after
aggregations).
* Use test-drill-variants-with-merged-args for quick-filter multi-stage tests
* Add multi-stage quick-filter tests for a couple of unaggregated queries
* Use lib.drill-thru.tu/column-by-name in metabase.lib.drill-thru-test
* Use lib.drill-thru.tu/column-by-name in lib.drill-thru.test-util.canned
* Support skipping tests in test-drill-variants-with-merged-args
* Allow variant-case to be a fn in test-drill-variants-with-merged-args
* Clean up test formatting
* Reduce diff
* feat(QB): combine chart type selector with viz settings sidebar
* fix typecheck
* fix unit tests
* chore(e2e): create helpers for viz settings and viz type buttons
* fix some tests
* fix most of the e2e tests
* fixup
* more fixes
* another test
* add a warning
* revert cross test changes
* chore: cleanup unused code
* render component directly
* Don't build FE for driver tests
* Try this again
* Consolidate MySQL job configs
* Oops fix conditional flags for test-driver
* More MySQL consolidation
* Oops
* Add a new job that waits for all other jobs (so we can require one single job going forward)
* Restore deleted code
* Try adding a Driver Tests (Summary) workflow
* How about a nice job name
* Don't love YAML inline array syntax
* Try fixing the build-static-viz configuration again
* Try fixing the build-static-viz configuration again
* Try this again
* Fix MySQL SSL config
* Try fixing the drivers summary workflow
* feat(QB): combine chart type selector with viz settings sidebar
* fix typecheck
* fix unit tests
* chore(e2e): create helpers for viz settings and viz type buttons
* fix some tests
* fix most of the e2e tests
* fixup
* more fixes
* another test
* add a warning
* revert cross test changes
* chore(QB): show small Join and CC icons for aggregation without breakout
* fix linting
* tests
* use compact prop and add another test
* revert provider
* fix memo
* Fix typo
* Fix broken memoization
* Use undefined instead of 0
* Introduce entityOptions
* Fix memoization
* Enable EntityListLoaderRtkQuery for alerts (#51501)
* Port bookmarks to use new `EntityListLoaderRtkQuery` (#51502)
* Fix types
* Enable EntityListLoaderRtkQuery for bookmarks
* Revert "Fix types"
This reverts commit f855c3a460.
* Enable EntityListLoaderRtkQuery for dashboards (#51503)
* Enable EntityListLoaderRtkQuery for persisted models (#51505)
* Enable EntityListLoaderRtkQuery for pulses (#51507)
* Enable EntityListLoaderRtkQuery for revisions (#51509)
* Enable EntityListLoaderRtkQuery for persisted segments (#51512)
* Enable EntityListLoaderRtkQuery for snippets (#51513)
* Enable EntityListLoaderRtkQuery for persisted questions (#51508)
* Enable EntityListLoaderRtkQuery for actions (#51515)
* Port tables to use new `EntityListLoaderRtkQuery` (#51518)
* Enable EntityListLoaderRtkQuery for tables
* Fix polling
* Port timelines to use new `EntityListLoaderRtkQuery` (#51516)
* Enable EntityListLoaderRtkQuery for timelines
* Add missing options
* Fix flaky test
* Port groups to use new `EntityListLoaderRtkQuery` (#51504)
* Enable EntityListLoaderRtkQuery for groups
* Bind action creators
* Fix flaky test
* Port users to use new `EntityListLoaderRtkQuery` (#51514)
* Enable EntityListLoaderRtkQuery for users
* Add missing options
* Fix crash
* Reimplement pagination
* Enable EntityListLoaderRtkQuery for search (#51568)
* Port schemas to use new `EntityListLoaderRtkQuery` (#51511)
* Enable EntityListLoaderRtkQuery for persisted schemas
* Add missing options
* Temporary commit
- just to save work at EOY
* Fix file upload
* Fix unit tests
* Fix data being set for different entityQuery
* Port collections to use new `EntityListLoaderRtkQuery` (#51519)
* Enable EntityListLoaderRtkQuery for collections
* Add missing options
* Improve typing
* Fix data being set for different entityQuery
* Fix destructuring crash
* Fix typing
* Pass options to RTK query where missing
* Use undefined instead of 0 for reloadInterval
* Port snippet collections to use new `EntityListLoaderRtkQuery` (#51847)
* Enable EntityListLoaderRtkQuery for snippet collections
* Introduce custom useListQuery
* Reduce diff
* Reduce diff
* Split Kondo lint fixes off into #51747
* Merge changes from #51660
* Reworked API documentation generation code
* Oops Markdown not YAML
* BIG documentation generation improvements
* Don't try to style tables ourselves
* Fix lint errors
* Fix tests 🔧
* Fix Kondo warnings
* Update OpenAPI dox generator to handle Defendpoint v2
* Update OpenAPI dox generator to handle Defendpoint v2
* Test fixes 🔧
* Fix uberjar compilation
* PR feedback from @escherize
* Deprecate the old `defendpoint`
* Nuke the old version of the API documentation code (new version will be merged in #51872)
* Remove empty file
* Disable output validation in prod
* Fix the way we define defendpoint 2 handlers
* start moving code around
* WIP fixes
* Test reorg, some more fixes
* Test fixes
* cljfmt, fix a test
* more cljfmt and linting
* cleanup, formatting fixes
* remove debugging code
* format again
* add some ns docstrings
* fix: Change pool acquireRetryAttemps to 1
Fixes: #51176
* Address PR review and only test dbs with username auth
* Fix model ref
* Exclude hive-like connections
* Ensure user and password
Updates JVM, JS and unit test `MetadataProvider` implementations to include `:ident` fields for all columns.
These idents need to be globally unique and serdes portable; therefore they are not randomized.
Instead they are built from the database, schema, table and field names, eg.
```clojure
{:ident "field__Sample Database__PUBLIC__Orders__CREATED_AT"
#_...}
```
**Note:** These `:ident` fields are not included on `/query_metadata` and other such API calls (currently).
They could be, but they can just as easily be synthesized from the DB, table and field in the JS metadata
implementation.
When you order by an aggregation, mbql v2 references the aggregation by uuid. However, that makes the query hash to a different value each time, because you get a different uuid each time you hash it. Converting the query to reference by index during the hashing process fixes this.
* Ensure database is fully encrypted if key is set, fail f database is encrypted and key is not set
* Added "remove-encryption" command as a more obvious alternative to rotate-encryption-key with an empty key
* fix broken link on BE
* Frontend
* update switch styles
* Fix type errors
* Fix BE tests
* fix malli schema
* logic tweak - don't show the switch if the target model is database
* cljfmt
* Add cypress test
* localize strings
* simplify form updates
* remove unneeded description style
* remove unused enabled? prop
* revert unintended change to cypress helper
* Make renaming options optional in lib.underlying/top-level-column
* Construct dimensions from row data in available-drill-thrus
Construct dimensions from row data once in available-drill-thrus, rather than doing it in each drill that needs it
separately. The dimensions must be constructed in available-drill-thrus since some drills are expected to
return-drills-for-dimensions?, but this also has the side benefit of reducing code duplication by lifting the
rows->dimensions conversion up into a single place.
* Fix the zoom-in.binning drill thru for multi-stage queries
* Add tests for zoom-in.binning with multi-stage queries
* cljfmt
* Explicitly disable the automatic-insights drill for multi-stage queries
Previously, this drill will was implicitly disabled for multi-stage queries because the FE would not pass dimensions
for queries with underlying breakouts / aggregations. Now that dimensions are reconstructed in available-drill-thrus,
automatic-insights drill is available again, but if you try to actually run the drill you get a big scary "Something
went wrong" error, so just explicitly disable the drill until we fix it.
* Update test/metabase/lib/drill_thru/zoom_in_bins_test.cljc
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Review feedback: not-empty -> seq
* Reivew feeback: shadow rather than using a single-quote suffix
* Review feedback: underyling -> underlying
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Fix date widgets dark theme
* Update more outdated loki snapshots
* Add a story to theme date filters missing dark theme
* Theme relative current widget and co.
* Theme date exclude widget
* Theme relative date interval widget and co.
* Update more snapshots from border color change
* Fix comment on the wrong line
* Fix light theme story switching + mock relative date stories
* Review: Remove unused function arg
* Update snapshots that change once a year 🤦
That is: aggregations, breakouts, expressions and joins.
Nothing is consuming these yet, but the BE populates them when reading cards.
MBQL changes
=============
These new `:ident`s have been present for new clauses for a few weeks, but now they'll be generated for existing queries on read.
The new idents are stable, including across serialization. They aren't written back until the card is saved again.
Backfilling
===========
Here's the constraints for generating idents for pre-existing clauses:
- If the same query exists across serdes instances, produce the same ident
- Globally unique, even if:
- two cards have identical queries
- two stages of the same card have identical `:count` aggregations.
- a single stage has multiple aggregations of the same type
- a single stage has duplicate breakouts with different time granularity
- Never change, even if the query does
- mbql-lib assumes these are (in theory) randomly generated, and never examines or updates them.
- If new clauses are added, they get random idents.
- If clauses are removed, that's no problem.
- If clauses are edited (eg. changing an expression name) then the BE *would* generate a new ident!
- But when the card gets saved, the original ident with the original expression name is enshrined forever as the ident - it just needs to be unique!
The backfilled idents have this form:
```clojure
"${kind}_${entity_id of the card}@${stage-number}__${name or index}"
;; eg.
"aggregation_ThisIsAValidNanoIdxxx@0__1"
"breakout_ThisIsAValidNanoIdxxx@1__1"
"expression_ThisIsAValidNanoIdxxx@0__exprname"
"join_ThisIsAValidNanoIdxxx@1__alias"
```
* Add M2M Oauth to Databricks Driver
* Add toggle for PAT/M2M Oauth
Introduces a 'Use Personal Access Token' toggle button which shows different UI components when setting up the connection. The implementation was inspired by @snoe in #51249
* Address feedback from @ibrdnk
Addresses the feedback related to #pullrequestreview-2515818008
* Default M2M, Add conditional Merge, Reduce whitespace
* Update docs/databases/connections/databricks.md
Co-authored-by: Jeff Bruemmer <jeffbruemmer@gmail.com>
* Reset FieldValuesWidget to v50 state
* Revert some changes in SingleSelectListField to v50
* Make onSearchChange optional
* Use RowValue instead of string
* Add test-ids to FieldValuesWidget inputs
* Add testing matrix for all parameter combinations
* Add filter matrix test to github workflow
* Fix unit test
* Render custom option value when it exists
* Fix e2e test
* Fix remove value helper
* Fix e2e test
* Render custom label on TokenField
* Update Loki Snapshots
* Use css colors for TokenFieldItem
* Make TokenField dark-mode aware
* Make TokenFieldItem dark-mode aware
* Update Loki Snapshots
* Fix e2e test after merge
* Add listbox role to tokenfield values
* Fix more e2e tests
* Fix e2e test for listbox
* Remove commented out code
* Fix e2e test
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* fixes showing admin links to not admins in the command palette
* updates calculation to take into account EE advanced permissions allowing certain users permissions to change settings who are not admins
* admin routes for non-admins
* fix unit tests
* actually fix unit tests
* adding checks for other permissions
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Azure Synapse doesn't like it when you give it a uuid parameter, even though synapse does support uuids in general.
Converting those uuids to strings beforehand fixes this error, and regular SQL Server is also fine with that.
* prevent dashboard from appearing in breadcrumbs when not on a DQ page
* updating unit test with location
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* fix: add supports for xx-YY locales in embed/public-links
* Generate locales.json
* [WIP] Read locales.json
* wip FE implementation
* refactor + unit and sdk component tests
* fixes after the merge, plus tests and cleand up logic
* fix file name in string
* change FE logic to use locales from the instance settings
* revert json generation
* remove leftover comment
* add comment about the code to keep in sync
* migrate e2e tests for static/public to use #locale=de-CH so we test that
it's using the fallback logic
* fix kondo error
Co-authored-by: dpsutton <dan@dpsutton.com>
* fix unit test
* fix more jest tests
* cy.request -> cy.intercept
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* fix: add supports for xx-YY locales in embed/public-links
* Generate locales.json
* [WIP] Read locales.json
* wip FE implementation
* refactor + unit and sdk component tests
* fixes after the merge, plus tests and cleand up logic
* fix file name in string
* change FE logic to use locales from the instance settings
* revert json generation
* remove leftover comment
* add comment about the code to keep in sync
* fix kondo error
Co-authored-by: dpsutton <dan@dpsutton.com>
* fix unit test
* fix more jest tests
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* chore: rework pickers in querying folder to css modules
* more css
* more css
* more css
* fix typecheck
* fix ui
* fix ui
* drop unused code
* fix ui
* fix css
* fix unit test
* revert color changes
* fix join color
* revert previous solution and move code into question question picker
* convert QuestionPicker to typescript
* port to css module
* refine button styles
* fix some e2e
* fix last failing e2e
* fe implementation for sessions
* fix styles if only one button is showing
* user css va for border colors
* fix failing unit tests and move new unit tests into existing file i had not noticed
* fix lint
* short circuit tracking for ignored requests
- add fall-through for ignored embedding requests
* add test for ignored status codes
* remove cruft
* add logging for unknown clients or statuses
* Fix some e2e tests in custom-column and view-data (#51431)
* warn on reflection!
---------
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
Using secrets.METABASE_AUTOMATION_USER_TOKEN causes the auto-approve PR step to fail with an error like
Can not approve your own pull request
Revert to using secrets.GITHUB_TOKEN instead, which matches what the auto-backport.yml workflow does and what this
workflow did previously.
* Add EntityListLoaderRtkQuery boilerplate
* Add loaded prop
* Add metadata prop to children
* Improve typing for entity selectors
* Add missing selector typing
* Adjust list loader
* Drop dispatchApiErrorEvent, entityAlias
* Add listName prop
* Conditionally use entityListLoaderRtkQuery
* Add transformResponse
* Enable new loader for database entity
* Update typing
* Add todo
* Remove actionCreators
* Update TODO
* Use loadListLegacy
* Fix condition
* Add dynamic reload* prop to children
* Handle onLoaded prop
* Reproduce isReloading behavior with isLoading
* Add support for reloadInterval prop
* Fix list state path when query is empty
* Migrate sample database endpoint
* Update type
* Simplify transformResponse
* Use modern syntax
* Type the wrapper
* Name the wrapper components
* Improve typing
* chore: rework mix components to css modules
* more css
* more css
* more css
* more css
* rework last pieces of emotion to css modules
* remove failing test
* address review
* fix view button styling
* fix view button styling
* drop not needed change
This PR builds on #50420 by adding an e2e test that the user can press the back button to return from the notebook editor to the homepage. Previously this was impossible due to a pop-under bug (#50971).
This PR introduces a simple way of avoiding pop-under bugs:
* First, we **give all overlays the same z-index.** It's 200, above everything else. SDK users can change this value.
* Second, we **prevent eager portal rendering**. On master, overlays are appended to the DOM inside an empty div, called a "portal". This portal is appended on mount. It stays hidden until the overlay is opened. In this branch, we stop portals from rendering eagerly. Instead, portals are only appended when the overlay is opened, and removed when the overlay is closed. This way, the most recently opened overlay is always last in the DOM.
Since the most recently opened overlay is last in the DOM, and since all overlays have the same z-index, the most recently opened overlay always appears on top.
This PR:
* closes#51256
* closes#45468
* closes#49466
* closes#50943
* closes#50971
* provides a more robust solution for two issues that previously received a quick, "minimum viable fix": #45469 and #48256
* improves the health of the codebase by removing hacks
* step1: add MetabaseReduxProvider and make our hooks + connect fn use it
step1: add MetabaseReduxProvider and make our hooks + connect fn use it
* step 2: run codemod to change the imports
step 2: run codemod to change the imports
* step 3: manual fixes
* step 4: e2e test for sdk
* adjusting webhook test response ui
* missed i18n, updating test to check for test response
* show response on error, not success
* parse the request as json response
---------
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Add EntityObjectLoaderRtkQuery boilerplate
* Incorporate EntityType logic into the new component
* Improve comment
* Add childrenProps
* Move component into its own directory, extract types to a separate file
* Add entityObjectLoaderRtkQuery and temporary mappings
* Hook up the useGetQuery hook (tables only)
* Pass all the props to children
* Add Playground
* Gather more info in the Playground
* Integrate RTK with redux store
* Adjust props passed down
* Improve typing
* Make the request hook dynamic
* Add missing id in the query
* Handle fetchMetadata
* Change requestType to fetchType
* Add rtk mappings for databases in entity framework
* Use requestType to get request state path
* Enable new component everywhere
* Use legacy loader for entities without rtk mappings
* Improve comment
* Handle fetchMetadataAndForeignTables
* Remove unused function
* Handle reload and remove
* Add TODO
* Add transformResponse option and use it for fetchIdFields
* Format code
* Remove Playground
* Remove unused getUseGetListQuery
* Add rtk mapping for alerts
* Add rtk mapping for dashboards
* Add rtk mapping for groups
* Add rtk mapping for pulses
* Remove options
* Fix permissions tests
* Fix notifications tests
* Add rtk mapping for segments
* Use params instead of body
- It's leftover because this code got reverted after body -> params refactoring for GET endpoint has been merged to master
* Add rtk mapping for snippet collections
* Add TODO
* ESLint
* ESLint
* Update TODO
* Deprecate entity HOCs
* Update TODO
* Revert other entities for now
* Revert other entities for now
* Bring tables back
* Improve todos
* Add comment
* Improve comment
* Remove TODO
* Use reload param
* Remove properties prop
This was used to short-circuit and avoid triggering api calls if we already had an object with given attributes in cache.
With RTK query now managing the cache, we don't want to reproduce this behavior.
* Remove reloadInterval
This is used in EntityListLoader, not EntityObjectLoader
* Don't use setRequestPromise action
* Fix deprecations
* Format code
* Fix flake :(
* Try to fix flake
* Revert "Try to fix flake"
This reverts commit 6408a22a37.
* Try to fix flake
* Fix UseQuery type
* Implement useGetMetadataAndForeignTables
* Fix useMemo and move things down
* Fix typo
* Bring back original hack
* Move file
* Enable RTK for alerts
* Undeprecate a test
* Enable RTK for dashboards
* Port Segments to use new EntityObjectLoaderRtkQuery (#50635)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* [WIP] Port Groups to use new EntityObjectLoaderRtkQuery (#50631)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Fix failing test
* Enable RTK for dashboards
* Port Snippet collections to use new EntityObjectLoaderRtkQuery (#50636)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* Revert "Enable RTK for segments"
This reverts commit ef65fe78f3.
* Enable RTK for snippet collections
* Fix skipToken passing
* Enable RTK for dashboards
* Port Pulses to use new EntityObjectLoaderRtkQuery (#50633)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Fix hook usage
* Enable RTK for dashboards
* Port Fields to use new EntityObjectLoaderRtkQuery (#50639)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* Revert "Enable RTK for segments"
This reverts commit ef65fe78f3.
* Enable RTK for snippet collections
* Revert "Enable RTK for snippet collections"
This reverts commit e1439cb28b.
* Enable RTK for fields
* Account for skipToken
* Fix fetchFieldValues
* Optimize hooks
* Format code
* Fix loading state not kicking in during first render
* Rename
* Add comment
* Use consistent reference name for revisions
* Port Questions to use new EntityObjectLoaderRtkQuery (#50634)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Rename import
* Fix weirdness
- Apparently what makes the difference here is the lack of children render pattern ({childProps => <ComposedComponent {...childProps} />})
* Refactor
* Use isFetching instead of isLoading in EntityObjectLoaderRtkQuery, replace useEffect+useState with useMemo in SavedQuestionLoader
* Change order of HOCs to prevent loading state from redundantly kicking in
* Rename import
* [WIP] Port Persisted Models to use new EntityObjectLoaderRtkQuery (#50853)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Rename import
* Fix weirdness
- Apparently what makes the difference here is the lack of children render pattern ({childProps => <ComposedComponent {...childProps} />})
* Refactor
* Use isFetching instead of isLoading in EntityObjectLoaderRtkQuery, replace useEffect+useState with useMemo in SavedQuestionLoader
* Change order of HOCs to prevent loading state from redundantly kicking in
* Port persisted models
* Port schemas (#50854)
* Port snippets (#50855)
* Port timelines (#50858)
* Do TODOs
* Port timeline events (#50857)
* Port users (#50866)
* Port Collections to use new EntityObjectLoaderRtkQuery (#50869)
* Port collections
* Fix imports
* Drop unused params
* Add missing property
* Fix collection not getting loaded
* Replace useCollectionQuery with useGetCollectionQuery
* Remove useCollectionQuery
* Format code
* Skip request on null id
* Revert "Remove useCollectionQuery"
This reverts commit 9b1068e611.
* Revert "Replace useCollectionQuery with useGetCollectionQuery"
This reverts commit 0c5ee660a3.
* Fix test
* Implement EntityName with RTK query
* git merge --squash 50323-port-other-entities
* Use entity getName selector
* Update comment
* Remove BaseEntity
* Improve typing
* git merge --squash 50323-port-other-entities
* Enable RTK for dashboards
* Deprecate the component
* Remove entityObjectLoader usage
* Remove old EntityObjectLoader and replace it with new one
* git merge --squash 51213-entity-loader
* Fix unexpected rounding in csv & json exports when column multiplier is non integral
* Fixing table unit test
* Adding WIP test
* Adding test case for validating new scaling logic is applied appropriately in exports
* Reverting id-or-name change and adjusting new scaling unit test
* Updating formatting to fix cljfmt in CI
* removing debug def
* Preserving integral behavior if scale is nil
`query=` logic was returning false negatives when two queries were identical except for their `:ident`s.
This broke Metrics UI, for example, since it makes up a query based on the "old" and "new" metrics, and
compares those queries.
Now `:ident`s are ignored by `query=`, so two queries of the same shape are equal.
* PopoverWithRef, plus some other stuff
* removing commented code
* I've done terrible things
* cleanup
* adding comments
* lets see what this breaks
* e2e adjustmnet
* removing unused component
* Add EntityObjectLoaderRtkQuery boilerplate
* Incorporate EntityType logic into the new component
* Improve comment
* Add childrenProps
* Move component into its own directory, extract types to a separate file
* Add entityObjectLoaderRtkQuery and temporary mappings
* Hook up the useGetQuery hook (tables only)
* Pass all the props to children
* Add Playground
* Gather more info in the Playground
* Integrate RTK with redux store
* Adjust props passed down
* Improve typing
* Make the request hook dynamic
* Add missing id in the query
* Handle fetchMetadata
* Change requestType to fetchType
* Add rtk mappings for databases in entity framework
* Use requestType to get request state path
* Enable new component everywhere
* Use legacy loader for entities without rtk mappings
* Improve comment
* Handle fetchMetadataAndForeignTables
* Remove unused function
* Handle reload and remove
* Add TODO
* Add transformResponse option and use it for fetchIdFields
* Format code
* Remove Playground
* Remove unused getUseGetListQuery
* Add rtk mapping for alerts
* Add rtk mapping for dashboards
* Add rtk mapping for groups
* Add rtk mapping for pulses
* Remove options
* Fix permissions tests
* Fix notifications tests
* Add rtk mapping for segments
* Use params instead of body
- It's leftover because this code got reverted after body -> params refactoring for GET endpoint has been merged to master
* Add rtk mapping for snippet collections
* Add TODO
* ESLint
* ESLint
* Update TODO
* Deprecate entity HOCs
* Update TODO
* Revert other entities for now
* Revert other entities for now
* Bring tables back
* Improve todos
* Add comment
* Improve comment
* Remove TODO
* Use reload param
* Remove properties prop
This was used to short-circuit and avoid triggering api calls if we already had an object with given attributes in cache.
With RTK query now managing the cache, we don't want to reproduce this behavior.
* Remove reloadInterval
This is used in EntityListLoader, not EntityObjectLoader
* Don't use setRequestPromise action
* Fix deprecations
* Format code
* Fix flake :(
* Try to fix flake
* Revert "Try to fix flake"
This reverts commit 6408a22a37.
* Try to fix flake
* Fix UseQuery type
* Implement useGetMetadataAndForeignTables
* Fix useMemo and move things down
* Fix typo
* Bring back original hack
* Move file
* Enable RTK for alerts
* Undeprecate a test
* Enable RTK for dashboards
* Port Segments to use new EntityObjectLoaderRtkQuery (#50635)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* [WIP] Port Groups to use new EntityObjectLoaderRtkQuery (#50631)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Fix failing test
* Enable RTK for dashboards
* Port Snippet collections to use new EntityObjectLoaderRtkQuery (#50636)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* Revert "Enable RTK for segments"
This reverts commit ef65fe78f3.
* Enable RTK for snippet collections
* Fix skipToken passing
* Enable RTK for dashboards
* Port Pulses to use new EntityObjectLoaderRtkQuery (#50633)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Fix hook usage
* Enable RTK for dashboards
* Port Fields to use new EntityObjectLoaderRtkQuery (#50639)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* Revert "Enable RTK for segments"
This reverts commit ef65fe78f3.
* Enable RTK for snippet collections
* Revert "Enable RTK for snippet collections"
This reverts commit e1439cb28b.
* Enable RTK for fields
* Account for skipToken
* Fix fetchFieldValues
* Optimize hooks
* Format code
* Fix loading state not kicking in during first render
* Rename
* Add comment
* Use consistent reference name for revisions
* Port Questions to use new EntityObjectLoaderRtkQuery (#50634)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Rename import
* Fix weirdness
- Apparently what makes the difference here is the lack of children render pattern ({childProps => <ComposedComponent {...childProps} />})
* Refactor
* Use isFetching instead of isLoading in EntityObjectLoaderRtkQuery, replace useEffect+useState with useMemo in SavedQuestionLoader
* Change order of HOCs to prevent loading state from redundantly kicking in
* Rename import
* [WIP] Port Persisted Models to use new EntityObjectLoaderRtkQuery (#50853)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Rename import
* Fix weirdness
- Apparently what makes the difference here is the lack of children render pattern ({childProps => <ComposedComponent {...childProps} />})
* Refactor
* Use isFetching instead of isLoading in EntityObjectLoaderRtkQuery, replace useEffect+useState with useMemo in SavedQuestionLoader
* Change order of HOCs to prevent loading state from redundantly kicking in
* Port persisted models
* Port schemas (#50854)
* Port snippets (#50855)
* Port timelines (#50858)
* Do TODOs
* Port timeline events (#50857)
* Port users (#50866)
* Port Collections to use new EntityObjectLoaderRtkQuery (#50869)
* Port collections
* Fix imports
* Drop unused params
* Add missing property
* Fix collection not getting loaded
* Replace useCollectionQuery with useGetCollectionQuery
* Remove useCollectionQuery
* Format code
* Skip request on null id
* Revert "Remove useCollectionQuery"
This reverts commit 9b1068e611.
* Revert "Replace useCollectionQuery with useGetCollectionQuery"
This reverts commit 0c5ee660a3.
* Fix test
* Implement EntityName with RTK query
* git merge --squash 50323-port-other-entities
* Use entity getName selector
* Update comment
* Remove BaseEntity
* Improve typing
* git merge --squash 50323-port-other-entities
* Enable RTK for dashboards
* Deprecate the component
* Fix flaky test
* Fix flake in all tests
* Use useGetActionQuery in ActionExecuteModal
* Remove use-action-query
* Remove Actions.load usage in ActionCreator
* Remove unused initialAction prop
* Remove redundant prop
* Use entityCompatibleQuery
* Update unit tests to account for RTK Query refetching
* Fix unit tests
* Fix unit tests
* Fix unit tests
* don't check for enterprise release tag anymore
* use oss tag for release channel tagging
* add error catcher
* don't trigger release log from tag job
* wip nextjs compat docs
* explain how to do manual wrapping
* doocs
* better explanation of the index file
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Update docs/embedding/sdk/next-js.md
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Get rid of enitity.Loader in PinnedQuestionLoader
* Set errorIcon
* Get rid of enitity.Loader in TargetClickMappings
* Remove duplication
* Get rid of enitity.Loader in LinkedField
* Convert metabase-lib/v1/metadata/utils/schema.js to TS
* Extract useGetSchemas
* Revert "Extract useGetSchemas"
This reverts commit 2271bd3aa7.
* Mark optional props as optional
* Properly handle skipToken
* Remove Schema.Loader from FieldMappingSelect
- This was added to fix metabase#19451 but seems no longer needed as the repro passes without it
- See 83f4097e87
* Update unit tests to use number as DatabaseId
* Extract DataSourceCrumbs and refactor it to TS
* Extract SourceDatasetBreadcrumbs and refactor it to TS
* Extract SourceQuestionBreadcrumbs and refactor it to TS
* Remove Questions.Loader from SourceQuestionBreadcrumbs
* Drop entity.Loader
* Update unit tests
* Drop checkNotNull usage
* Add missing -test suffix to zoom-in-timeseries test name
* Add missing -test suffix to underlying-records test name
* Fix distribution drill thru for multi-stage queries
Ensure the distribution drill thru is not returned by available-drill-thrus when a user clicks on an aggregation or
breakout column, even if there are additional query stages after the last aggregation / breakout stage.
* only create a single release for each release
* remove dead file
* little bug fixes
* update tests
* add link to metabase cloud in release notes
* remove checksums from release notes
Some funcs were added to lib.drill-thru.common for the recent fix to the zoom-in.timeseries drill. Update the
underlying-records drill to use the common lib code rather than duplicating it.
`replace-clause` is for editing an existing clause, which should
*preserve* its identity. But the new clause was getting its own
random ident which didn't match the original.
I've added tests for this, and fixed the `lib.remove-replace` logic to
make them pass.
Hat tip to Mike for catching this case!
Previously, cumulative aggregators were being done in clojure after the mongo query was finished. However, this
didn't work on multi-step queries where the cumulative aggregator wasn't the last step (for example, do a cumulative
sum and then filter the result). Implementing cumulative operators directly in mongo fixes this issue.
* Fix#51020
* Add a repro for #51020
* Improve test name
* Update comment and naming
* Improve diff
* Add repro for when primary key is not called 'id'
* Update repro
* Drop prefetchesInitialValues & getValuesForUpdate
It's unclear what the real intention behind these two was, but it's clear that it made two differences:
(1) It decreased the probability of overwriting someone else's changes (if someone else saved them after we opened the form)
(2) It decreases the payload sent over the wire
(1) We can ignore this, as this problem is prevalent everywhere in our app
(2) This is negligible
Dropping these is simpler than enhancing getValuesForUpdate with extra logic for primary keys (and who knows what else)
* Break beforeEach into 2
* Remove no longer valid test
* save only grid on fixed width dashboards
* split export pdf into pages
* trim dashboard export pdf into pages
* specs
* add padding to pdf pages
* dark mode
* skip tracking non react-sdk x-metabase-clients
- Do not do promethus tracking for sdk.
- tracking ignores x-metabase-client headers when they're not == "embedding-sdk-react"
* cleanup indentation
* add prometheus tracking counters for iframe
* remedy 24h query execution inaccuracy
We were getting the `dashboard_tab_id` from the existing cards on the
first tab... but if no such card exists (the first tab is empty) then we
end up with a dashboard with tabs, but a card with
`dashboard_tab_id=null`, which doesn't show up.
This also fixes a bug in the revision history: we were publishing an
event for the dashboard being updated. The object was the updated
dashboard, but it had the old dashcards that we'd hydrated in order to
autoplace. We can dissoc the dashcards to ensure that the handler
hydrates them with the fresh, updated dashcards.
* checks if there are dashboards in the personal collection to check if we should show items below
* adding e2e test
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* add bug report to command palette and profile menu
add bug report to command palette and profile menu
* refactor and update diagnostic window modal
* add analytics to track bug reporter modal activity
* add file id to to the upload-file method
* add the ability to collect browser information for diagnostic window
* implement feedback from PR comments
* Update frontend/src/metabase/components/ErrorPages/ErrorDiagnosticModal.tsx
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* implement feedback from PR comments #2
* rename "file a bug" to "report an issue"
* resolve conflict
* fix lint issues
* fix bad merge conflict resolution
* fix failing onboarding tests
---------
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* wip: better documentation for developers that need to run the sdk
locally
* morning fixes
* remove developers-guide/sdk in favor of dev.md in the sdk folder
* add note about using the sdk with link to the docs
* fix mising word
* addressed feedback
* added notes about the envs for sdk e2e test
* Prevent newly opened items from expanding outside viewport
* Expand the unti test coverage
* Use the existing helper
* Change scroll view to center
This works better because it ensures that the top bar doesn't cover the element.
* Update frontend/src/metabase/home/components/Onboarding/Onboarding.tsx
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Extract the `stopVideo` function
* Fix the type name
* Set the top scroll margin on the `main` element
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Closes#49682. Closes#49683.
Updates `lib.convert` and legacy MBQL normalization to allow
`:ident`s, and fixes some QP middleware and utils so they can
still match up refs correctly.
No user-visible impact.
Newly created queries will have `:ident`s generated for joins,
aggregations, etc. by the previous PR. This starts saving them
into appDBs. It is still acceptable to have missing idents at
this stage, including for new queries created without the lib.
Pre-existing clauses don't have refs added yet.
* Fix loading in segments
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Our revision system has a race condition in the case of two concurrent requests to modify the same object. You can reproduce the exact issue here by opening two `psql` shells. In each one, run
```
begin;
insert into revision (model, model_id, user_id, timestamp, object, most_recent, metabase_version) values ('Dashboard', 1, 1, '2024-01-01T00:00:00Z', '{}', true, 'v1.0.0') returning id;
update revision set most_recent = false where model='Dashboard' and model_id = 10 and id != [[the id from above]];
```
after running all of the above, `commit` the transaction and note that **two** revisions are now marked as `most_recent`.
This is the way our revision system works: `t2/before-insert` sets `most_recent=true`, while `t2/after-insert` updates all the *other* revisions with `most_recent=false`.
The issue here is that when each transaction is running, it doesn't see
the uncommitted result of the other transaction (the new revision), so
the UPDATE does not set `most_recent` to `false` for either new
revision.
The solution here is to just select all existing `revision`s for this
model `FOR UPDATE`. This effectively locks the revisions for a single
entity, so we can't concurrently update revisions for the same entity.
* Add EntityObjectLoaderRtkQuery boilerplate
* Incorporate EntityType logic into the new component
* Improve comment
* Add childrenProps
* Move component into its own directory, extract types to a separate file
* Add entityObjectLoaderRtkQuery and temporary mappings
* Hook up the useGetQuery hook (tables only)
* Pass all the props to children
* Add Playground
* Gather more info in the Playground
* Integrate RTK with redux store
* Adjust props passed down
* Improve typing
* Make the request hook dynamic
* Add missing id in the query
* Handle fetchMetadata
* Change requestType to fetchType
* Add rtk mappings for databases in entity framework
* Use requestType to get request state path
* Enable new component everywhere
* Use legacy loader for entities without rtk mappings
* Improve comment
* Handle fetchMetadataAndForeignTables
* Remove unused function
* Handle reload and remove
* Add TODO
* Add transformResponse option and use it for fetchIdFields
* Format code
* Remove Playground
* Remove unused getUseGetListQuery
* Add rtk mapping for alerts
* Add rtk mapping for dashboards
* Add rtk mapping for groups
* Add rtk mapping for pulses
* Remove options
* Fix permissions tests
* Fix notifications tests
* Add rtk mapping for segments
* Use params instead of body
- It's leftover because this code got reverted after body -> params refactoring for GET endpoint has been merged to master
* Add rtk mapping for snippet collections
* Add TODO
* ESLint
* ESLint
* Update TODO
* Deprecate entity HOCs
* Update TODO
* Revert other entities for now
* Revert other entities for now
* Bring tables back
* Improve todos
* Add comment
* Improve comment
* Remove TODO
* Use reload param
* Remove properties prop
This was used to short-circuit and avoid triggering api calls if we already had an object with given attributes in cache.
With RTK query now managing the cache, we don't want to reproduce this behavior.
* Remove reloadInterval
This is used in EntityListLoader, not EntityObjectLoader
* Don't use setRequestPromise action
* Fix deprecations
* Format code
* Fix flake :(
* Try to fix flake
* Revert "Try to fix flake"
This reverts commit 6408a22a37.
* Try to fix flake
* Fix UseQuery type
* Implement useGetMetadataAndForeignTables
* Fix useMemo and move things down
* Fix typo
* Bring back original hack
* Move file
* Enable RTK for alerts
* Undeprecate a test
* Enable RTK for dashboards
* Port Segments to use new EntityObjectLoaderRtkQuery (#50635)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* [WIP] Port Groups to use new EntityObjectLoaderRtkQuery (#50631)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Fix failing test
* Enable RTK for dashboards
* Port Snippet collections to use new EntityObjectLoaderRtkQuery (#50636)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* Revert "Enable RTK for segments"
This reverts commit ef65fe78f3.
* Enable RTK for snippet collections
* Fix skipToken passing
* Enable RTK for dashboards
* Port Pulses to use new EntityObjectLoaderRtkQuery (#50633)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Fix hook usage
* Enable RTK for dashboards
* Port Fields to use new EntityObjectLoaderRtkQuery (#50639)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* Revert "Enable RTK for segments"
This reverts commit ef65fe78f3.
* Enable RTK for snippet collections
* Revert "Enable RTK for snippet collections"
This reverts commit e1439cb28b.
* Enable RTK for fields
* Account for skipToken
* Fix fetchFieldValues
* Optimize hooks
* Format code
* Fix loading state not kicking in during first render
* Rename
* Add comment
* Use consistent reference name for revisions
* Port Questions to use new EntityObjectLoaderRtkQuery (#50634)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Rename import
* Fix weirdness
- Apparently what makes the difference here is the lack of children render pattern ({childProps => <ComposedComponent {...childProps} />})
* Refactor
* Use isFetching instead of isLoading in EntityObjectLoaderRtkQuery, replace useEffect+useState with useMemo in SavedQuestionLoader
* Change order of HOCs to prevent loading state from redundantly kicking in
* Rename import
* [WIP] Port Persisted Models to use new EntityObjectLoaderRtkQuery (#50853)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Rename import
* Fix weirdness
- Apparently what makes the difference here is the lack of children render pattern ({childProps => <ComposedComponent {...childProps} />})
* Refactor
* Use isFetching instead of isLoading in EntityObjectLoaderRtkQuery, replace useEffect+useState with useMemo in SavedQuestionLoader
* Change order of HOCs to prevent loading state from redundantly kicking in
* Port persisted models
* Port schemas (#50854)
* Port snippets (#50855)
* Port timelines (#50858)
* Do TODOs
* Port timeline events (#50857)
* Port users (#50866)
* Port Collections to use new EntityObjectLoaderRtkQuery (#50869)
* Port collections
* Fix imports
* Drop unused params
* Add missing property
* Fix collection not getting loaded
* Replace useCollectionQuery with useGetCollectionQuery
* Remove useCollectionQuery
* Format code
* Skip request on null id
* Revert "Remove useCollectionQuery"
This reverts commit 9b1068e611.
* Revert "Replace useCollectionQuery with useGetCollectionQuery"
This reverts commit 0c5ee660a3.
* Fix test
* Extract dedicated providePermissionsGroupListQueryTags
* Format code
* Fix test setup
* Get rid of Database.Link usage in TableBrowser
* Fix browseDatabase typing
* Get rid of Collection.Link usage in DashboardMoveModal
* Get rid of Collection.Link usage in QuestionMoveToast
* Style links consistently
* Remove EntityLink
* Use database from the store if it's already there
* Remove filter
* Fix typo
* Use ts-pattern
* Bring spacing back
* Extract components for links
---------
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* BE: Dashboard Questions
Setting this makes the card a "dashboard internal card" that should only
be used within a single dashboard, is moved along with the dashboard, etc.
prevent invalid inserts/updates
POST /api/card for dashboard internal cards
GET /api/collection/:id/items
Search
PUT /api/card/:id
WIP: last bits of card-in-dashboard API
Sticking point here is the copy, I think I need to do a small refactor
to make the code less hairy.
rm line
optionally show DQs for collection items
copying dashboards 🎉
wip
quick fix
Disallow moving dashboard questions & test archive
remove unused columns
fix argument order 🤦
serdes for dashboard questions
serialization baseline tests
Fix tests in `metabase.models.card`
Fix download exports test
Lint migrations file
cljfmt
cljfmt, again
the cljfmt job in CI doesn't run on this branch, it runs on the PR
branch merged with `origin/master` - which is nice, because it catches
cljfmt issues that would happen if you merged, but don't happen on your
branch, but also confusing, because it catches issues like
"metabase.api.common is never used", and then you stare at the file for
a while like "YES IT IS" before realizing what's going on.
migration to add idx for report_card.dashboard_id
don't rollback the `report_card.dashboard_id` bits
a couple missed API changes
- no DQs w/o DashboardCards in search
- no DQs in autocomplete
- fix check for "currently a dashboard question" in card validation
Fix some comments, clean up some existing code
Before, `cards-to-copy` returned just two lists of cards: those to
discard (no permissions) and those to copy. But `maybe-duplicate-cards!`
had some extra logic: if the card was a `model`, don't copy it, just
reference it.
I modified `cards-to-copy` to return three lists instead: cards to copy,
cards to discard, *and* cards to reference. This was initially just used
for making sure we copy Dashboard Questions instead of referencing them
- but I realized we could use the same for Models as well. I moved the
logic for models to `cards-to-copy` instead: if the card is a Model,
return it in the `reference` list instead of the `copy` list.
Fix commented-out test
cljfmt
Optionally include dashboard questions in search
Add a test for failing card name change
Opt. include dashboard questions for autocomplete
Update src/metabase/api/dashboard.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
PR review response
lint migrations
quick impl of dashboard items
copied from collection items. should be firmed up for real going
forward. notable divergences:
- i'm not doing any limit/offset in the query
- i'm not including limit/offset in the response
- no reuse. just grabbed it to unblock nick
```
❯ http get "localhost:3000/api/dashboard/22/items" x-api-key:$API_KEY -pb
{
"data": [
{
"archived": false,
"archived_directly": false,
"collection_id": 20,
"collection_position": null,
"collection_preview": true,
"database_id": 48,
"dataset_query": {
"database": 48,
"query": {
"source-table": 360
},
"type": "query"
},
"description": null,
"display": "table",
"entity_id": "_TmF-T7m_bBVxV_jfaNtM",
"id": 192,
"last-edit-info": {
"email": "dan@metabase.com",
"first_name": "dan",
"id": 1,
"last_name": "sutton",
"timestamp": "2024-09-13T22:13:30.220311Z"
},
"last_used_at": "2024-10-01T17:48:36.128915Z",
"model": "card",
"moderated_status": null,
"name": "Cynthia Example"
}
],
"models": [
"card"
],
"total": 1
}
```
contrast with collection items:
```
❯ http get "localhost:3000/api/collection/root/items?models=card&limit=1" x-api-key:$API_KEY -pb
{
"data": [
{
"archived": false,
"can_delete": false,
"can_restore": false,
"can_write": true,
"collection_id": null,
"collection_position": null,
"collection_preview": true,
"database_id": 4,
"description": null,
"display": "table",
"entity_id": "e_jABu05u9nT9ocgfQWfZ",
"fully_parameterized": true,
"id": 143,
"last-edit-info": {
"email": "dan@metabase.com",
"first_name": "dan",
"id": 1,
"last_name": "sutton",
"timestamp": "2024-08-27T14:11:15.214254Z"
},
"last_used_at": "2024-08-27T14:11:27.138735Z",
"location": null,
"model": "card",
"moderated_status": null,
"name": "Collection + query on sleep question"
}
],
"limit": 1,
"models": [
"card"
],
"offset": 0,
"total": 13
}
```
Clean up last-edit-info and exclude archived cards
before: over-selecting, copy-paste function to clean it up.
after: reuse (bulk-aware) query that annotates with last-edit-info, ALSO
EXCLUDE CARDS
also condensed query a bit. was each field on a separate line for no
real reason.
`location` on dashboards
dashcards are still active for archived DQs
WIP: autoplace dashcards for dashboard cards
more wip for autoplacement
WIP for moving DQs
more WIP on moving DQs/autoplacement
delete old dashcards when moving to new dashboard
populate `here` on dashboards in coll items
reject moving card if in another dashboard
fix silly typo
allow optionally deleting old dashcards on move
WIP
add dashboard details to recents
untested wip
fix auto delete of old dashcards for move
Add another test for DQs getting unarchived
Fix dashboard details on recents, with a test
Archive and unarchive DQs when used/unused
including in revisions
Handle reversions containing unowned DQs
lint
Tests and fixes for dashboard items endpoint
remove stray #p
fix a test
Add more tests for DQs/reversions
Fix/update tests for updating DQs
Don't revert to dashcards with archived cards
This is the existing behavior, we should keep it.
lint
small fixes for recents tests
add more tests
Autoplace Dashboard Questions on creation
Improve autoplace to be like frontend
I'd hacked together something - this actually makes autoplacement work
the same way on the frontend and backend.
fix a couple tests
`delete_old_dashboards` as a query param
Fix autoplace for tabbed dashboards
and add a test
do not autoplace DQs when duplicating a dashboard
This was resulting in us accidentally creating two dashcards for a
duplicated DQ.
add effective location to collections on dash
add dashboard_count to (card) collection items
autoplace on unarchive, better error messages
Exclude DQs from dashboard if archived directly
DOH
workaround a bug with "changing" a type
Fix bug causing dashboard cards to be removed
small refactor autoplacing / removing dashcards
Create Dashboard revisions on autoplace/remove
When we modify Dashboard Questions, we autoplace or autoremove relevant
cards from the Dashboards involved. For example, we remove DashCards
from the old dashboard if we're moving to a new dashboard, and we
autoplace DashCards into the new dashboard.
When we do this, publish an event that the dashboard itself has changed.
This updates the revision history for the dashboard so that you can
revert correctly.
Note that when using the revision history, DQs will automatically be
archived or unarchived as necessary. So e.g. if you manually archive a
DQ, then go to the Dashboard and revert, the DQ will be auto-unarchived.
If you go to the dashboard and remove all cards for a DQ, the DQ will be
archived - if you then manually unarchive it, it'll be autoplaced on its
dashboard again.
Fix bug on card modification
The conditions here were wrong. Say you had a card on dashboard 1 that
was archived. If you specified `{:dashboard_id 1 :archived false}`, it
would be correctly autoplaced on the dashboard. But if you just
specified `{:archived false}`, the "new dashboard ID" is nil.
The logic here is pretty painful, because to figure out the resulting
value for something like `dashboard_id` we can't just get it from the
old version or the changes. Instead, we need to:
- check to see whether it's changing, then
- if it changed, use the version from the update, otherwise
- use the old version
The same is true for `archived`.
After we figure out the old dashboard_id, new dashboard_id, and whether
the card will end up archived or not, we can actually do the checks we
need:
- if we're ending up an unarchived card on a dashboard, autoplace
- if we're moving from one dashboard to another, or moving from a
dashboard to a collection (and the user told us to), auto-remove
offset/limits for dashboard items
Reuse some code between dashboard/collection items
Fix some failing tests in api.dashboard
I was getting stack overflow errors when running these tests. I followed
along with the demonstartion
[here](https://github.com/metabase/metabase/pull/43496) to fix.
cljfmt
add additional tests
Always delete dashcards when archiving a DQ
If we're archiving a DQ, let's delete the dashcards so this appears in
the revision history.
publish revision when autoplacing
Publish dashboard events when cards are autoplaced
new endpoint to get dashboards a card is in
allow moving to a dashboard deleting old dashcards
fix the migrations 😭
fix migration IDs
* work on dedicated dashboard question page
* wip on dashboard page for handling dashboard questions
* hacking linking to dashboard for question and books marks - does not handle navigating to the correct tab yet
* make it easier to change the dashboard_id for hacking
* misc. changes
* action menu and dashbcard action updates
* enables trashing dashboard questions from the dashboard view, enables undoing of that case, updates archive state for dedicated dashboard question page
* wip - mostly done creation flow
* wip
* add question directly on dashboard flow
* significant clean up of fe code, revert some wierd differences between base branch, allows selecting dashboard when duplicating a question
* type fixes
* clean up isDashboardQuestion checks, fixes bug where save question form name and description state would get cleared after selecting a dashboard from the picker, fixed bug where editing a dashboard question and choosing save as new question would only let you save it to the dashboard the original question was on
* implement bulk of team feedback
This commit accomplishes the following things:
- Updates copy of the dashboard/collection picker
- Fixes the tooltip copy for the add question button in the edit dashboard view
- Removes the ability to trash a dashboard question + dashboard view removes rather than trashes the question
- Adds option in dashboard edit view to add a native or notebook question
* prompt to save rather than discard changes when leaving a dashboard while editing to create a new question
* include dashboard questions in search results
* fix lint & kick CI
* type fixes
* fix unit tests
* two small fixes
* wip on failing e2e tests
* bulk e2e test fix
* fixed most tests worth fixing - leaving the ones that need to be replaced
* Dashboard Question as a source
* replace dashcard flow
* starting to update collection picker to treat dashboards as folders
* fix spelling mistake
* fix bad resolution and stop double adding newly created dashboard questions to their dashboards
* allow dashboards in collection picker
* remove checkbox to only clone dashboard w/o cards, fix collection id calculation for saving a question when defaults to a dashboard
* rough pass at allowing questions to be moved to, from, and between dashboards
* building state path with initial value as a dashboard
* simplify moving code and fix encountering BE error
* add confirm modals when moving a dashboard question to a collection or another dashboard, allow users to keep or remove dashcards for a dashboard question getting moved to a collection
* improve the breadcrumbs data consistency, fix bug in moving logic and dump user in edit state of dashboard when moving from collection to dashboard
* dynamic select button text
* Search showing available models
* hopefully this resolves some type sadness
* handle inconsistent state issues between latest question and query builder internal state
* default question picker to showing dashboards as folders
* silly type things
* disallow saving metrics | models | events to dashboards, added back ability to keep or remove dashboard ca
rds when moving a DQ to a collection, fixed dedicated DQ page not appearing archived after archiving the parent dashboard
* use currentData instead of data for collection breadcrumbs
* dashboards are not folders when they are empty
* fix moving question to Our Analytics, Update sidesheet question into for DQs, always allowed to select a dashboard for moving question
* confirm moving questions into dashboards
* trying to sort out types
* types are still a mess
* only regular questions can be moved into a dashboard
* adjusting some unit tests
* add e2e tests for dashboard questions
* should have happy linter and unit tests
* fix some type errors
* loading state
* e2e test fixes
* more fixed tests
* loading state message
* filter series options, fix create new collection logic
* fix some more e2e tests
* e2e test adjustments
* even more e2e tests fixes
* fix named export
* more e2e test fixes
* fixing some e2e stuff
* hopefully final e2e adjustments
* last failing e2e (for now)
* More BE: Dashboard Questions
add dashboard question dimension to stats
Handle series correctly
and test for them
Don't restore archived-directly questions
* unit test adjustments, linter changes
* obey the linter
* unit tests for new breadcumb functionality
* allow creating a question directly to a dashboard
* More BE: Dashboard Questions
use dashboard constants for default size
add missing sql clause
add dummmy alias to subquery
* more cards in dashboards e2e tests
* rebase whoopsie
* more tests
* fix types
* More BE: Dashboard Questions
include dashboard info on coll items & cards
include dashboard info in search
also modify the moderation info a bit, now all three endpoints return
the same shape for `dashboard` (if present):
```
{:id dashboard-id
:name dashboard-name
:moderation_status moderation-status-if-present}
```
Bulk endpoints for checking dashboards and moving
For now the move endpoint just runs everything in a transaction and
spits out an error if anything failed. I'd like to take more time to
think of the exact shape of the result we want in the case of an error,
even though I think ideally we should allow partial success.
* reproduce card-pocalypse
* adding dashboard link to search results
* more trash tests
* lint fixes
* fix crash when archiving dashboard with virtual cards
* add loading state to archive modal
* obey the clojure linter
* More BE: Dashboard Questions
cljfmt and fix tests
small test fix
* more e2e tests
* embedding tests
* type fix
* More BE: Dashboard Questions
two more small test fixes
more test fixes
* add sdk tests
* another test
* add series removal e2e test
* More BE: Dashboard Questions
yet another test 😭
work around old MariaDB limitations
* permission check and confirmation modal when moving cards into dashboards
* e2e adjustments
* add some missing tests
* prettier + rebase fixes
* fix search result bug
* Entity syncing when moving a question from a collection to a dashboard
* hello rebase hades
* gree dashboard-questions suite
* fix more e2e tests
* happy little test fixes
* Sloan Feedback
* fix move modal error swallowing
* fix embedding test
* fixes breadcrumbs for embedding, fixes associated failing tests, and updates some tests to account for reduction in # of requests
* fix embedding e2e tests
* Bryan's review response
* fix usage of `offset-paging`
* fix request count errors
* fix unit test
* finish feedback from FE code review
* CR response for Noah
* bad sloan
* no need for dashboards in ad hoc
* get the dash name from the question object
* fix migrations again
* fix test + kondo
* lint fix
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Add new user-level setting
* Add selector
* Make link dismissible from the sidebar
* Transfer the onboarding link to the main "gear" menu
* Add E2E test to cover the link dismissal behavior
* Add tests for the settings menu link
* Fix tests
* Provide useful comment in the e2e test
* Provide contextual explanations to translators
* We need to look at the context field to categorize public views
* pull poll-unil into util ns
* fix comment typo
* add test
* update docstring
* remove true? call, it will return `true` on pass
* improve poll-until
* Fix the zoom-in.timeseries drill for multi-stage queries
If no dimensions are provided to the zoom-in.timeseries drill, then attempt to find the matching dimension from the FE-provided row data instead, if present.
Related to https://github.com/metabase/metabase/issues/46932
* Only return zoom-in.timeseries drills for clicks on aggregated columns
This PR removes the deprecated `BodyComponent` HOC in favor of Mantine's `Portal` component. This helps pave the way for https://github.com/metabase/metabase/pull/49442
### How to verify
1. Perform an action that triggers an undo toast (e.g., move a question into the trash)
2. Verify the undo toast appears correctly at the bottom of the screen
3. In a collection, drag and drop items to verify the drag preview renders correctly
### Testing
This change is covered by the following tests:
* `collections.cy.spec.js > "should be able to drag an item to the root collection (metabase#16498)"`
* `collection-pinned-overview.cy.spec.js:269: "should be able to pin a visualization by dragging it up"`
Note the following comment, however, which mentions the limitations of our drag-n-drop e2e tests:
(from https://github.com/metabase/metabase/blob/92d6150/e2e/test/scenarios/collections/collection-pinned-overview.cy.spec.js#L282-L289)
```
// this test can give us some degree of confidence, but its effectiveness is limited
// because we are manually firing events on the correct elements. It doesn't seem that there's
// a way to actually simulate the raw user interaction of dragging a certain distance in cypress.
// this will not guarantee that the drag and drop functionality will work in the real world, e.g
// when our various drag + drop libraries start interfering with events on one another.
// for example, this test would not have caught https://github.com/metabase/metabase/issues/30614
// even libraries like https://github.com/dmtrKovalenko/cypress-real-events rely on firing events
// on specific elements rather than truly simulating mouse movements across the screen
```
Closes#49682. Closes#49683.
Updates `lib.convert` and legacy MBQL normalization to allow
`:ident`s, and fixes some QP middleware and utils so they can
still match up refs correctly.
No user-visible impact.
Newly created queries will have `:ident`s generated for joins,
aggregations, etc. by the previous query, and this will start
saving them into appDBs.
Pre-existing clauses don't have refs added yet.
* Fix loading in segments
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
For now, we've decided not to trigger index updates when elements are deleted, due to the overhead that toucan2 would add.
We have the code to calculate what *should* be done, so we might as well keep it around in a test that documents these relationships.
It's good to be mindful of the implications, and we could even use this information to do manual calls from the relevant deletion call sites.
* Add list/create/update/delete to actions api
* Add api mappings in entity framework
* Port actions to RTK
* Remove old api mappings
* Fix a test
* Fix test
* Fix test
* Fix test
* Fix method
* Fix types
* Define request types
* Remove ActionsApi.create
* Fix deleting actions
* Fix invalidation
* Fix invalidation
* Remove redundant spreads
* Remove type attribute from UpdateActionRequest
The underlying issue is that nest-query/nest-expression included temporal units when copying over joined fields into the nested query. This caused both the nested query and the parent query to include the "extract the unit from the date" code. In cases where the extraction didn't change the type, this duplication wasn't a big deal. However, for temporal units like hour-of-day, the first extraction changed the type of the field from date to string, and then the second extraction broke. This change ensures that the inner query doesn't try to extract temporal units, which should avoid the duplicate extractions.
* wip next sdk compat
* better code generation, dts works
* Last line of file should end in a newline character
* feat(sdk): make MetabaseProvider not break ssr
* aUpdate enterprise/frontend/src/embedding-sdk/package.template.json
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* address pr comments
* refactor(sdk): use relative paths when fixing d.ts files
* make types work on moduleResolution: node
* add loading text
* fix(sdk): #50736 make sure we don't break the host app if a sdk
component is rendered outside of the provider for a render
* fix: this should make types work for real?
* self review fixes
* update test for PublicComponentWrapper
* Update enterprise/frontend/src/embedding-sdk/components/private/PublicComponentWrapper/PublicComponentWrapper.tsx
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* remove unused component
* eslint-disable-next-line for MetabaseProvider
* list tsx as dependency
* /next -> /nextjs
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* introduce public metabase question type
* revise internal question type
* update stories
* use public question type in dashboard card plugins
* add tests for transforming question properties
* add tests for question description
* re-export the MetabaseQuestion type
* fix lint imports
* change publicQuestion to sdkQuestion
* name cannot be null if the card is present
* Fix embed loader not translated with #locale
* Fix unit tests
* Add E2E tests
* Fix a case where the lcoale is failed to load
* Simplify conditions in `LocaleProvider`
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Make E2E tests faster
Co-authored-by: Nicolò Pretto <info@npretto.com>
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
* WIP
* metabase.request.cookies namespace
* Server API namespace
* `metabase.server` => `metabase.server.instance`
* Reformat code
* Revert indentation changes
* Unrevert comment
* Oops delete moved file
* Don't be lazy CAM!!!
* Add TODO
* Please cljfmt
* Bump Kondo version to fix metadata ignores in namespaces being ignored
* Update Kondo and enable a few more linters
* Update Clojure linux install script location
* Ok so apparently the old URL does work
* Make sure things live in the right places
* Why do I always have to clean up other people's messes
* Fix formatting
* People need to knocc it off with writing slow tests
* Fix another test that took 45 SECONDS
* Fix Kondo warning
* fix scatter chart tooltip shows irrelevant info
* remove outdated spec as we always show the tooltip_column setting
* fix missing tooltip on the total bar on waterfall charts
* Always show embed sharing option for admins (#50513)
* Always render the public link option
This is possible because metabase#50504
* Always render the embed option
* Fix unit tests
* Fix E2E tests
* Correct test semantic
* Hide Embed option for non-admins
* New Embed modal (#50580)
* Open "Compare options" in a new tab
* Add an external link icon to interactive embed option when it should
* Add disabled state to SDK
* Add a disable state to static and interactive embedding
* Add interactive embedding disabled state
* Implement disabled link
* Prepare folder for testing
* Fix tests and add SDK tests
* Add interactive embedding tests
* Correct test desribe block levels
* Add static embedding tests
* Add premium interactive embedding tests
* Add enterprise tests (EE without tokens)
* Fix the rest of the failed unit tests
* Update E2E tests
* Fix a case where OSS see disabled interactive embedding
This should be impossible as interactive embedding isn't available on
OSS, so we should not display the disabled state.
* Address review
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Address review
Co-Authored-by: Poom poom@poom.dev
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Update Interactive Embedding Admin Settings page (#50693)
* Add a quick start link to interactive embedding settings page
* Update UTM tags for interactive embedding card
* Don't show interactive embedding option as enabled on OSS
* Fix unit tests
* Add a new test
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* fix the script, and add the new edn
* make the migrations run in the proper order
* it works (needs cleanup)
* remove broken binning
- These bins rely on fingerprinting for the custom Age column
- The fingerprinting for that custom column are not properly exported in
serialization, hence do not get loaded into metabase, causing binning by
Age to fail.
* cleanup / linting
* cleanup
* update migration test ids to test
* realign test name to check for to see if sample content was created
* remove Product collection + move Examples into root
* update collection_ids for the content too
* remove click behavior for 'Revenue goal for this quarter'
* Apply suggestions from code review
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
* for the sample collection is_sample should be true
* Drop CreateInternalUserNoOp
* typo
* update migration comments
* 2 small fixes
* fix data perms for example coll
* add a comment so we don't have 2 blank lines in a row
---------
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
* Add delay helper
* Remove delays in storybook
* Use skippable delay in ListField and SingleSelectListField
* Use skippable delay in ExplicitSize
* Remove waitTimeContext in storybook
* Remove waitTimeContext altogether
* Add documentation to delay
* Remove all context references in ExplicitSize
* Update flaky snapshots
Fixes: #50771
The new sync process failed to qualify enums outside public and
therefore failed to properly type them in database_type. This could
either result in matching the wrong type if there was a conflict or not
being typed as an enum.
* test: see which drivers pass these tests
* Fix build
* Change test to use standard dataset, add initial implementation for bigquery
* Use fully qualified names for views
* Don't use transactions for databricks
* Make drop table only if exists, fix describe-index in mongo for views
* don't sync system.views
* Fix oracle drop-views, disable non-syncing views
* Use metabase-instance to find the table, this is important because redshift and oracle munge table name
* Fix oracle tests
* Fix oracle views
* Fix formatting
* Fix snowflake qualified components
* Change tests to opt out
* Fix sqlsserver
* Fix postgres test
* Disable h2 describe views test
* Address review
* Add trial banner
* Use the semantic "warning" color for background
* Simplify the "reappearance" logic
* Refine the banner logic
- Pass the bare minimum information (expiry timestamp) to the `TrialBanner`
- Take timezones into account
- Introduce utilities
- Add unit tests
* Update the remaining tests
* Clean up
* Unbreak my `defsetting` 🎶
* Add comments
* Defer var assignment
* Fix failing tests 🤦
* Adjust the types in order to match `master`
* Move the logic up to the parent component
* Convert the `index` component to TS
* Fix typo in the JSDoc
* Fix another typo
* Make the return statement more readable
* Remove redundant test
* Move modals out of resizable area
* Split off ace editor from NativeQueryEditor
* Query for selection target in AceEditor
* Rename editor ref
* Add custom type for selection range
* Move more items into shared types file
* Add shared interface for editors
* Add shared Editor wrapper
* Add SQL colors in core/colors.module.css
* Allow importing global css with .global.css
* Move ace styles to AceEditor.global.css
* Fix Range type
* Fix Ace editor tests
* Scope the ace editor styles
* Remove EditorRoot styled component
* Use parent wrapper instead of classname on editor
* Remove ace editor styles from global styles
* Remove custom ace styles from embedding sdk
* Move NativeQueryEditor styles to css modules
* Fix styles for notebook sql preview
* Remove unused props
* Remove unused variable warnings
* Remove width prop
* Rename openDataReferenceAtQuestion and make it optional
* Rename setNativeEditorSelectedRange and make it optional
* Move selection popover logic out of AceEditor
* Make autocompleteResultsFn optional
* Make cardAutocompleteResultsFn optional
* Remove fetchQuestion prop
* Remove nativeEditorSelectedText prop
* Make props explicit
* Make onChange optional
* Set editor height
* Use Editor in NotebookNativePreview
* Remove cardAutocompleteResultsFn and autocompleteResultsFn props from Editor
* Remove unused snippetCollections prop
* Add missing native-query-autocomplete-match-style setting
* Remove snippets from Editor props
* Resize ace editor on load
* Add custom snippets wrapper
* Flatter completer styles
* Add container class to autocomplete popover
* Fix flex size for editor
* Fix AceEditor types
* Flatten ace styles
* Avoid hard-coded color
* Set height and width on editor
* Flatten ace wrapper
* Do not call api with empty prefix
* Add helpers to target native editor
* Use nativeQueryEditor in action helper
* Use nativeQueryEditor in ad hoc helper
* Remove ace_* references from tests
* Align nativeEditor helpers
* Revert "Align nativeEditor helpers"
This reverts commit e629aecb18.
* Move focusNativeEditor to native editor helpers
* Use focusNativeEditor where applicable
* Remove ace_editor references in test
* Fix test
* Remove references to native-query-editor testid in test
* Remove duplicate test assertion
* Fix nesting in css
* Use font-family from variable
* Use default module.css with :global instead of custom .global.css file
* Move AceEditor into subdirectory
* Fix import path
* More strongly type native-query-autocomplete-match-style
* Remove duplicate default
* Rename EditorHandle to EditorRef
* Add more specificity to ace syntax overwrites
* Simplify EditorRef
* Remove HOC
* Do not perform visibility check inside nativeEditor helper
* Explicitly assert elements are visible
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix formatting issue
* Add highlight for identifier
* Improve gutter cell
* Use relative imports for helpers
* Focus editor before typing
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* rename onClose to onCancel
* rename questionHeight to drillThroughQuestionHeight
* rename showVisualizationSelector to withVisualizationSelector
* rename parameter value props
* doc updates for parameter values
* rename the chart type selector prop
* document the withChartTypeSelector prop
* do not show column selection error on empty datasets
* add more specs to chart-definition logic
* fix entry nodes tooltip and click behavior values
* fix nodes formatting
* do not show sankey data settings on dashboards
* do not initiallt open sankey data columns selects
* hide sankey tooltip on scroll
* drop incorrect types
* set sankey node limit
* prevent clicks on edges close tooltips on native sankey questions
* tweak copy
* update sankey spec according to fixes
* revert changed settings behavior
* fix unit specs
* Pivot Exports Can Handle `nil` in Breakout Col, and Multiple Cols of the Same Aggregation Type
Fixes#50551
If a pivot table has several measures configured, they might both use the same kind of
aggregation (eg. 'sum'). Previously, this would lead to columns with the same name preventing the pivot-measures from
properly making it through the export post processing. Now, the correct aliased/deduped name is used and all columns
can be properly included in the pivot export.
Also fixes the case where a breakout column (pivot-row) contains `nil` values, which caused the pivot export to
fail. Now, all nil values are grouped and handled appropriately.
* when pivot is disabled, header should still be included
* min and max don't default to 0, add a test for the aggregations
* Add EntityObjectLoaderRtkQuery boilerplate
* Incorporate EntityType logic into the new component
* Improve comment
* Add childrenProps
* Move component into its own directory, extract types to a separate file
* Add entityObjectLoaderRtkQuery and temporary mappings
* Hook up the useGetQuery hook (tables only)
* Pass all the props to children
* Add Playground
* Gather more info in the Playground
* Integrate RTK with redux store
* Adjust props passed down
* Improve typing
* Make the request hook dynamic
* Add missing id in the query
* Handle fetchMetadata
* Change requestType to fetchType
* Add rtk mappings for databases in entity framework
* Use requestType to get request state path
* Enable new component everywhere
* Use legacy loader for entities without rtk mappings
* Improve comment
* Handle fetchMetadataAndForeignTables
* Remove unused function
* Handle reload and remove
* Add TODO
* Add transformResponse option and use it for fetchIdFields
* Format code
* Remove Playground
* Remove unused getUseGetListQuery
* Add rtk mapping for alerts
* Add rtk mapping for dashboards
* Add rtk mapping for groups
* Add rtk mapping for pulses
* Remove options
* Fix permissions tests
* Fix notifications tests
* Add rtk mapping for segments
* Use params instead of body
- It's leftover because this code got reverted after body -> params refactoring for GET endpoint has been merged to master
* Add rtk mapping for snippet collections
* Add TODO
* ESLint
* ESLint
* Update TODO
* Deprecate entity HOCs
* Update TODO
* Revert other entities for now
* Revert other entities for now
* Bring tables back
* Improve todos
* Add comment
* Improve comment
* Remove TODO
* Use reload param
* Remove properties prop
This was used to short-circuit and avoid triggering api calls if we already had an object with given attributes in cache.
With RTK query now managing the cache, we don't want to reproduce this behavior.
* Remove reloadInterval
This is used in EntityListLoader, not EntityObjectLoader
* Don't use setRequestPromise action
* Fix deprecations
* Format code
* Fix flake :(
* Try to fix flake
* Revert "Try to fix flake"
This reverts commit 6408a22a37.
* Try to fix flake
* Fix UseQuery type
* Implement useGetMetadataAndForeignTables
* Fix useMemo and move things down
* Fix typo
* Bring back original hack
* Move file
* Add comment
* match against any hexadecimal bundle name
* add e2e tests skeleton
* Support display-name for ISO get-day-of-week exclusion expressions (#50606)
* Support display-name for ISO get-day-of-week exclusion expressions
Fixes#50574
* Fix light theme embeddings has different text colors (#50678)
* add cache-control tests
* support local runs
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* add withVisualizationSelector prop
* add tests
* add to docs
* use default layout in tests
* only use custom layout when needed
* make withChartTypeSelector false by default
Adds `dev.portal/diff->`, a drop-in replacement for `->` that `tap>`s
a diff at each stage. (With the metadata for Portal to default to diff viewer)
I found myself wanting this several times while debugging the refs overhaul.
Specifically, for expressions, aggregations, breakouts, and joins.
Adds MBQL lib unit tests for this behaviour.
Currently these idents are discarded during `lib.convert` and don't
survive a round trip to legacy. That will come with a later change.
This protects against us trying to read or write from an incompatible search index. In particular it prevents existing instances from being switched over to an incompatible table during an HA upgrade or downgrade.
~~It doesn't yet get rid of the needless re-initialization, but that will be a fast follow.~~ It breaks up the jobs so that we won't reindex on startup unless necessary.
See https://www.notion.so/metabase/Search-Index-Maintenance-14b69354c9018047b670cbba1c3be50d
This ensures that when one instance starts or finishes reindexing, the other instances pick up and switch to the same tables.
Note: this does not stop blowing away the currently active table on deployment yet, we need to have a notion of version for that first.
* Enable underlying-records drill in table viz for multi-stage queries
* Fix e2e test
* Add tests for underyling-records drill for multi-stage queries
* Review feedback: filterv not-empty etc
* Review feedback: add a breakout-only query to drill-thru canned-queries
* do not show column selection error on empty datasets
* persist default sankey column settings when saving
* add more specs to chart-definition logic
* more specs
* Table Column Ordering Should Not Prevent Pivot Exports
Prior, when viz-settings existed for table column ordering, or if some columns were hidden in a regular table viz, the
pivot export could fail.
Now, if a pivot table is being exported, the table column sorting is properly ignored and the pivot export should work.
* address feedback
* use css variables for skeleton background color
* apply theme to folder color
* replace theme.white with themeColor
* use name of folder chevron color
* add background-inverse color mapping
* Revert "add background-inverse color mapping"
This reverts commit d0b546c583.
* avoid overriding folder chevron color in emotion
* use Mantine style overrides for NavLink
* temporarily hiding group into other
* hide group into other
* remove assertions related to group into other
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Adding .png version of no_results.svg to fix#49012
* Adjusting text color to to match existing color scheme
* Removing old unused no results assets; adding new png version of no_results.svg at matching resolution to pulse_no_results@2x.png
* Fixing namespace import cycle
Fixes: #50422
Although we passed additional-options through subname in url parameters
to DriverManager, we were also passing `:schema nil` as connection property.
* Pivot Export Aggregations Improved and Won't Error
Fixes#50207
WIP
- Totals now have appropriate aggregation implementation
- Totals for different row and column partial paths could collide before (eg. if a pivot-col value was equal to the
- measure idx), now the paths are unique and will not collide, causing errors
* add a test
* multiple measures need to have the correct order in each row
* clean up, add some comments explaining totals paths
* fix tests
* add test to show add-rows working correctly after :rows-part :cols-part added
* remove pivot-grouping row and columns in tandem
* add pivot card scalar test for csvs and xlsx.
* improve test feedback
* add a 'todo' for pivoted json exports
* indent
* Add stage-number parameter to NativeParameterDimensionTarget
* Update unit test
* Recognize native query dimension parameters with stage-number
* Update test setup
---------
Co-authored-by: Tamás Benkő <tamas@metabase.com>
* Don't dissoc `:id` in `before-update`
There seems to be a nasty footgun in toucan2 here.
Say you're executing an `update!` command on a set of IDs, e.g.:
```
(t2/update! :model/Card :id [:in 1 2] {:view_count 1})
```
This works if:
- both cards 1 and 2 have `view_count=1` already
- both cards 1 and 2 have `view_count!=1`
However.
If one of the two cards has `view_count=1` and another has a different
view_count, then (if the `before-update` method doesn't have the primary
key attached) Toucan emits a call to update *every card in the
database*, without a where clause at all.
* Trigger CI tests
* Add dummy job generator
* [DO NOT MERGE] Remove driver test config from the list of backend changes
* Add missing runs-on
* Avoid too much nesting
* Revert changes to file paths and README
* FE - Allow to use in dashboard filters both pre- and post- last aggregation columns (#46670)
* Extract getFilterStageIndexes
* Update comment
* Include 2 last stages in filterable columns in getParameterColumns
* Move getJoinQueryHelpers to test-helpers
* Add basic query creation
* Add breakout columns
* Make tableName optional
* Implement 1-stage test, refactor assertions
* Rename
* Move filter utils up
* Use getGroupItems and appendStageIfAggregated in getFilterableColums
* Introduce Lib.filterGroups
* Revert "Introduce Lib.filterGroups"
This reverts commit 5f8b2d098e.
* Don't reuse getGroupItems or appendStageIfAggregated
* Move getFilterStageIndexes out of Lib
* Format code
* Revert types move
* Revert functions move
* Improve diff
* Group dashcard mapping options
* Adjust existing getParameterColumns usages (except unit tests)
* Remove redundant call
* Fix filtering
* Update unit tests
* Format code, remove TODO
* Add stage index assertions
* Add model test for 1-stage query
* Add partial boilerplate for 2-stage query
* Add a 2-stage case
* Add a 2-stage case for model
* Update unit test
* Fix click behavior mapping
* Remove duplicated functions
* Remove complex return type
* Move appendStageIfAggregated and getFilterStageIndexes to metabase-lib/filter.ts
* Re-group columns based on stage index instead of using ML-returned references for comparison
* Add new options to dimension types
* Use the new stage-number option
* Rename appendStageIfSummarized to ensureFilterStage and add explanatory comment
* Remove duplicated ensureFilterStage after merge
* FE - e2e - Cover dashboard filters with tests (#46958)
* Extract getFilterStageIndexes
* Update comment
* Include 2 last stages in filterable columns in getParameterColumns
* Move getJoinQueryHelpers to test-helpers
* Add basic query creation
* Add breakout columns
* Make tableName optional
* Implement 1-stage test, refactor assertions
* Rename
* Move filter utils up
* Use getGroupItems and appendStageIfAggregated in getFilterableColums
* Introduce Lib.filterGroups
* Revert "Introduce Lib.filterGroups"
This reverts commit 5f8b2d098e.
* Don't reuse getGroupItems or appendStageIfAggregated
* Move getFilterStageIndexes out of Lib
* Format code
* Revert types move
* Revert functions move
* Improve diff
* Group dashcard mapping options
* Adjust existing getParameterColumns usages (except unit tests)
* Remove redundant call
* Fix filtering
* Update unit tests
* Format code, remove TODO
* Add stage index assertions
* Add model test for 1-stage query
* Add partial boilerplate for 2-stage query
* Add a 2-stage case
* Add a 2-stage case for model
* Update unit test
* Fix click behavior mapping
* Remove duplicated functions
* Remove complex return type
* Remove unused @ts-expect-error directive
- use createQuestionAndDashboard helper instead of command
- sort imports
* Add boilerplate for query stages test suite
* Add createDashboard
* Try out the new function
* Organize code
* Add descriptions
* Move visitDashboard call out of createDashboard
* Move appendStageIfAggregated and getFilterStageIndexes to metabase-lib/filter.ts
* Re-group columns based on stage index instead of using ML-returned references for comparison
* Add boilerplate for base questions
* Add assertions for mapping options
* Simplify verifyDashcardMappingOptions interface
* Add assertions for all types of parameters
* Update test name
* Format code
* Add new options to dimension types
* Use the new stage-number option
* Add Q2 creation
* Introduce helper constants
* Rename appendStageIfSummarized to ensureFilterStage and add explanatory comment
* Add cases for models
* Extract subroutines
* Add Q3
* Extract createQ2Query and createQ3Query
* Extract createQ1uery, remove createM1
* Optimize verifyPopoverMappingOptions
* Add more assertions
* Add model assertions
* Add tooltip assertion
* Break large functions down
* Restructure tests to prevent memory-related crashes
* Refactor
* Update comment
* Update comment
* Remove createQ1
* Remove createQ2, createQ3
* Refactor
* Refactor
* Comment out failing assertions
* Add Q3 assertions
* Add Q3 assertions
* Add comment
* Introduce createAndVisitDashboardWithQueryMatrix
* Add Q4
* Fix assertions
* Add repro tags
* Add comment
* De-hardcode indexes
* Add Q5-Q8
* Rename
* Add Q5
* Add Q6
* Add Q7 & Q8
* Add Q9
* Fix structure
* Reuse Lib.ensureFilterStage
* Fix unit of time parameters
* Support explicit and implicit stage numbers for temporal-unit params
* Extend test with temporal unit parameter with stage number
* Update unit tests
* Let stage-path handle nil (default) stage-number
---------
Co-authored-by: Tamás Benkő <tamas@metabase.com>
* Allow filtering on columns from all stages in dashboards (#48254)
* Allow filtering on columns from all stages in dashboards
* Update tests
* Clean up
* Update unit tests
* Distinguish multiple "Summaries" column groups in dashboard filter mapping (#48261)
* Allow filtering on columns from all stages in dashboards
* Update tests
* Clean up
* Update unit tests
* Distinguish multiple "Summaries" column groups
* Fix imports
* Remove redundant attribute
* Update import
* Allow filter params targeting native stages (#48379)
* Move filters on nested native stages to parent stage in expand-mbql-params
Similar to the existing handling in move-join-condition-to-source-query.
This fixes filter params that directly target a native stage with a non-negative :stage-number, e.g. when applying
dashboard filters to a card where the card query is native.
Fixes#48258
* Add test for filter params explicitly targeting nested native queries
* FE - e2e - Cover dashboard drills with tests (#48289)
* Allow filtering on columns from all stages in dashboards
* Update tests
* Clean up
* Update unit tests
* Distinguish multiple "Summaries" column groups
* Fix imports
* Remove redundant attribute
* Fix type
* Add a test
* Add a test
* Reduce flakiness
* Fix race conditions with last_used_param_values
- see https://metaboat.slack.com/archives/C010L1Z4F9S/p1727947712609579
* Add a temporary workaround
* Fix name
* Fix applying parameters in Question
* Revert "Fix type"
This reverts commit 67f6778d50.
* Rename and reorder tests
* Add more tests
* Update tests to avoid hitting https://github.com/metabase/metabase/issues/46774
* Add Q8 tests and fix tests for Q9
* Update assertions
* Update assertions because of https://github.com/metabase/metabase/issues/48339
* Improve test names
* Add test
* Skip failing test
* Add more tests
* Add test
* Remove redundant code
* Remove invalid cases
* Add test
* Add tests
* Add more assertions
* Add more assertions
* Add assertions
* Add assertions
* Use helper functions
* Use helper functions
* Nest helpers consistently throughout the file
* Decrease flakiness
* Revert "Add a temporary workaround"
This reverts commit 48ed8770d6.
* Add a temporary workaround
* Add tests
* Add assertions
* Add tests
* Update unit tests
* Make temporary workaround safer
* Add TODOs to github links
* Extract a function to improve readability
* Simplify getParametersMappedToDashcard
* Fix failing tests by calling Lib.ensureFilterStage in the correct place and properly identifying the originalQuestion so that Question.prototype.isDirtyComparedTo returns expected result
* Remove redundant code
* Quick dirty fix to run in CI
* Format code
* Format code
* Update unit tests
* Fix test setup
* Revert "Remove redundant code"
This reverts commit 35b47a10f2.
* Use Lib.areLegacyQueriesEqual
* Create a new e2e test group: dashboard-filters-2 (#48610)
* Allow filtering on columns from all stages in dashboards
* Update tests
* Clean up
* Update unit tests
* Distinguish multiple "Summaries" column groups
* Fix imports
* Remove redundant attribute
* Fix type
* Add a test
* Add a test
* Reduce flakiness
* Fix race conditions with last_used_param_values
- see https://metaboat.slack.com/archives/C010L1Z4F9S/p1727947712609579
* Add a temporary workaround
* Fix name
* Fix applying parameters in Question
* Revert "Fix type"
This reverts commit 67f6778d50.
* Rename and reorder tests
* Add more tests
* Update tests to avoid hitting https://github.com/metabase/metabase/issues/46774
* Add Q8 tests and fix tests for Q9
* Update assertions
* Update assertions because of https://github.com/metabase/metabase/issues/48339
* Improve test names
* Add test
* Skip failing test
* Add more tests
* Add test
* Remove redundant code
* Remove invalid cases
* Add test
* Add tests
* Add more assertions
* Add more assertions
* Add assertions
* Add assertions
* Use helper functions
* Use helper functions
* Nest helpers consistently throughout the file
* Decrease flakiness
* Revert "Add a temporary workaround"
This reverts commit 48ed8770d6.
* Add a temporary workaround
* Add tests
* Add assertions
* Add tests
* Update unit tests
* Make temporary workaround safer
* Add TODOs to github links
* Extract a function to improve readability
* Simplify getParametersMappedToDashcard
* Fix failing tests by calling Lib.ensureFilterStage in the correct place and properly identifying the originalQuestion so that Question.prototype.isDirtyComparedTo returns expected result
* Remove redundant code
* Quick dirty fix to run in CI
* Format code
* Format code
* Update unit tests
* Fix test setup
* Create a new e2e test group because the old one was timing out in CI
* Add the group to GH action
* Revert "Remove redundant code"
This reverts commit 35b47a10f2.
* Use Lib.areLegacyQueriesEqual
* FE - e2e - Cover combination of unit of time parameter + non-last-stage filters with tests (#48726)
* Add JSDoc for entityPickerModalItem
* Add addToDashboard parameter to saveQuestion
* Add JSDoc for saveQuestion
* Fix import
* Add a test reproducing the bug
* Try to fix unit of time parameters
* Fix wrong stage-index for temporal unit parameters on composed ad hoc questions
* Fix ad-hoc question naming when there is an empty extra filter stage
* Fix unit tests
* FE - Do not add extra filtering stage for pivot tables (#48975)
* Avoid using Lib.ensureFilterStage for pivoted questions
* Fix unit test setup
* Add tests for pivot tables
* Optimize tests
* Test filter modal
* Revert redundant changes
* [WIP] Dashboard filter columns be fixes (#48906)
* Revert "Add a temporary workaround"
This reverts commit 4ea374870d.
* Fix some of the issues and unskip some of the E2E tests
* Acknowledge that metrics are transparent
* Fix breakouts by the same column with different temporal-units
* Use non-negative stageIndex for temporal unit parameters
* Add stage if there is an explicit reference to stage after the last
Instead of duplicating the ensure-filter-stage logic, just check if there is a parameter referencing the filter
stage (which is always one after the last in the query). If there is such a reference, add that stage.
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Require only breakouts when adding new stage in ensure-filter-stage (#49067)
* Only require breakouts when adding new stage in ensure-filter-stage
Previously, this function required both breakouts and aggregations.
Closes#48339
* Update e2e tests after changes to Lib.ensureFilterStage
This resolves all-but-one of the TODOs related to #49339
Related to #49022
* Implementing parameter_mappings migration (#49256)
* FE - e2e - Update tests after `Lib.ensureFilterStage` changes (#49222)
* Unskip #19744 repro
* Remove sanity checks
* Update comment
* Fix typo
* Remove invalid test
* Ignore pivoted "table" viz and and inline Question.prototype.isPivoted
* Remove test for pivoted table
* FE - e2e - Click behavior (#49239)
* Unskip #19744 repro
* Remove sanity checks
* Update comment
* Fix typo
* Remove invalid test
* Ignore pivoted "table" viz and and inline Question.prototype.isPivoted
* Remove test for pivoted table
* Add a test
* Disable columns from non-last stage in click behavior
* Finish the test
* Refactor
* FE - e2e - Cover public & embedded dashboards with tests (#49280)
* Add test for public dashboard
* Add test for embedded dashboard
* Add comment
* Add test for 2nd stage column
* Add test for 2nd stage aggregation column
* Add test for 2nd stage breakout column
* Add assertions for models
* Handle no auto-pivoting of ad-hoc questions from drill thrus
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Add missing test setup
* Address some review comments (#49429)
* FE - Show columns from all stages in click behavior (target question) (#48828)
* Add JSDoc for entityPickerModalItem
* Add addToDashboard parameter to saveQuestion
* Add JSDoc for saveQuestion
* Fix import
* Add a test reproducing the bug
* Try to fix unit of time parameters
* Fix wrong stage-index for temporal unit parameters on composed ad hoc questions
* Fix ad-hoc question naming when there is an empty extra filter stage
* Fix unit tests
* Fix non-unique keys
* Add stage-number to click behavior dimension target
* Call Lib.ensureFilteringStage in click behavior
* Use describeEE because click behavior tests use setTokenFeatures("all")
* Stop testing implementation details
* Stop testing implementation details
* Stop testing implementation details
* Modernize test and make it less flaky
* Address flake source
* Fix SAMPLE_DB_ID
* Unskip repro for #19744
* Add a test
* Update test name
* Introduce verifyNotebookQuery
* Use verifyNotebookQuery
* Use verifyNotebookQuery
* Move verifyNotebookQuery to helpers
* Assert items count in verifyNotebookQuery
* Extract helper functions
* Rename helpers
* Refactor
* Consistent comments
* Extract verifyVizTypeIsLine
* Add scrollIntoView calls for extra safety
* Add TODO
* Use verifyNotebookQuery
* Make stages optional
* Add createMultiStageQuery
* Remove redundant scrollIntoView calls
- We don't need them because the assertions are not using "be.visible"
* Improve test setup
* Add more assertions
* Add more assertions
* Add basic join support to verifyNotebookJoins
* Unnest code
* Implement join assertions
* Implement expressions and sort assertions
* Simplify code
* Add commented-out assertions
* Yes, it is correct
- it's stageIndex which sourceColumn comes from in sourceQuery
* Conditionally apply filter stage
* Allow mapping to all stages in click behavior
* Remove redundant argument
* Simplify diff
* Simplify diff
* Update test setup
* Remove TODO related to metabase#49282
- Closes#49282
* Support fetching param values on filter stage (#50120)
* Support fetching param values on filter stage
Fixes#48613
* Fix temporal-unit-parameters
* Enrich dimension targets in click_behavior with stage-numbers (#49911)
* Enrich dimension targets in click_behavior with stage-numbers
Fixes#49110
* Add missing stage-number
* Remove changeSynchronousBatchUpdateSetting references
* Update assertions to include time bucket
* Use mbql.u/legacy-last-stage-number in metabase.query-processor.card
---------
Co-authored-by: Tamás Benkő <tamas@metabase.com>
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* prevent the UI from breaking if there is an invalid enum value set by MB_LOADING_MESSAGE
* add test case
* add enum validation on backend
* oops - fix set-value-of-type! call
---------
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* use -1 for stage index for breakout and aggregation list
* add e2e tests
* remove sum of total in e2e
* use test id matcher
* add issue to e2e name
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* move stageIndex to a separate variable
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* hide pivoted exports under env var
* only export pivoted when the env var is true
Also added tests to confirm.
* must specify :export for public settings
* spec
* change setting to :internal since it's only via env var
* use correct visibility add setting to settings.yaml for serialization
---------
Co-authored-by: Adam James <adam.vermeer2@gmail.com>
On MySQL 8.0.17, the following is prohibited:
```
WITH RECURSIVE MyRecursiveCTE (...) AS
(
SELECT ... FROM bar UNION ALL
SELECT ... FROM bar JOIN MyRecursiveCTE ON ...
)
```
Instead, you need to do:
```
WITH RECURSIVE MyRecursiveCTE (...) AS
(
SELECT ... FROM bar UNION ALL
SELECT ... FROM MyRecursiveCTE JOIN bar ON ...
)
```
Basically, just putting the CTE on the left vs. the right side of the
join (which is exactly equivalent under the hood) fixes the issue.
I can't actually find any documentation of this change, but it seems
that sometime between MySQL 8.0.17 and MySQL 8.0.33, MySQL started
allowing this, perhaps just an optimization where it now notices the
equivalence and does the right thing.
I originally tested this on what I believed was the oldest version of
MySQL we supported, but didn't read carefully enough. We recommend
8.0.33 or higher here:
https://www.metabase.com/docs/latest/databases/connections/mysql
But that's not for the AppDB - for that, we recommend 8.0.17 or higher.
* Fix serdes of missing cards in click behavior
When exporting a dashboard, we export its dashcards, including cards
referenced inside its viz_settings.
The problem is that viz_settings might have references to
now-nonexistent cards.
This is an initial stab at cleaning up the problem, specifically for
click behavior. If Dashboard D has Dashcard DC, which has a
click-behavior that links to Card C, then when we collect the
descendants of D, we'll include a reference to C - even if C has been
deleted.
So this makes two changes: first, when calculating the descendants of a
dashcard, don't include links to cards that don't exist. And, when
exporting the click behavior, remove the whole thing entirely if the
card it points to doesn't exist.
* Put `#locale` behind the feature flag
* Prepare folder structure for unit testing
* Refactor: extract a setup function as a shared util
* Add #locale tests
* Add all tests
* Prepare folder structure for unit testing
* Refactor: extract a setup function as a shared util
* Add locale tests
* Add all questions tests
* Fix failed E2E
* Tidy up questions tests
* Tidy up dashboards tests
* Fix test name typo
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Allow searching a single select list picker
* Fix unit tests
* Add repro for 49642
* Remove selection when searching
* Rename parameter to value because it does not need to be a prefix
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Simplify loading state for ListField
* Use same DEBOUNCE_FILTER_TIME in ListField
* Don't rely on waitTimeContext in test
* Fix lint
* Add initialization state for filter widget
* Update Loki Snapshots
* Update Loki Snapshots
* Allow picking other values when selecting a single value dropdown value
* Add comment about initialization
* Use setup in test
* Update Loki Snapshots
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* revamp the create question component
* docs updates
* add stories and update styles
* prevent switching to visualization when it is not ready
* ensure CreateQuestion works in flex parents
* add basic e2e tests for CreateQuestion
* ensure that switching between editor and visualization works
* use the save button for the disabled state
* update the question title when the question is saved
* implement updating questions in place
* rename replaceQuestion
* make CreateQuestion props all optional
* add background color to modal header
* hide the save button if question is dirty
* hide save button instead of disabling per design feedback
* update e2e tests
* revert e2e helpers
* add title update assertions
* whitespace changes
* Add new strings for v51.4
* Update translations
* kick ci
---------
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Don't try to fetch dbs for zero cards in api.table/batch-fetch-card-query-metadatas
Closes#48461
Ensure we don't throw an exception by attempting to fetch the Databases associated with potentially-deleted card ids in
api.table/batch-fetch-card-query-metadatas.
In particular, this ensures we don't generate an exception if the user does the following
1. Create a card "Source Card"
2. Create another card "Derived Card" with a :source-table of "Source Card"
3. Add Derived Card to a dashboard
4. Permanently delete (not just archive) the Source Card
* Add test for virtual-table query-metadata with deleted source card
* Add test for dataset query_metadata with deleted source card
* Add test for dashboard query-metadata with deleted source card
* Add test for card query-metadata with deleted source card
* Add metabase.api.test-util and use it in query-metadata-with-deleted-card tests
* Replace copy/pasted debug helpers with select-query-metadata-keys-for-debugging
* trigger save handlers on question create
* add skeleton example stories for save forms
* update storybook stories
* add tests for saving questions
* docs update
* add createCard assertions
* rollback docs
* feat(sdk): detect mismatch between sdk version and mb version
* refactor: log utils to print the big headers
* Update enterprise/frontend/src/embedding-sdk/lib/version-utils.unit.spec.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* refactor: call -> callArguments
* add link to the docs
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* remove LegendVertical logic that preserves legend width
* memoize pie legend items
* prevent pie from jumping when toggle legend
* fix responsive legend logic, cleanup
* drop hyphens from the pie legend (#29597)
* eliminate flickering on render due to not having dimensions
* shortcut
* specs
* test vertical layout has no hyphens
* setup func
* set explicit showLegend=true default
* [xlsx-export] Cache the custom datetime formatter
* [xlsx-export] Optimize functions in m.q_p.pivot.postprocess
* cljfmt
---------
Co-authored-by: Adam James <adam.vermeer2@gmail.com>
* Revert "fix(sdk): put a bandage on the flashing error on static question in strict mode (#49659)"
This reverts commit c57b3a38e0.
* refactor(sdk): use ignore flag instead of mounted ref
* Add temporal unit to display name
* Remove temporal unit adjustments from FE
* formatColumn to displayNameForColumn
* Factor out getFriendlyName
* Adjust BE tests
* Add require
* Adjust FE unit tests
* Adjust temporal-unit-parameters.cy.spec.js
* Adjust dashboard-reproductions.cy.spec
* Convert formatColumn occurrence
* Adjust legacy display name logic generation
After `getFriendlyName` was removed from FE, capitalization
of display name should happen here. Logic is similar, should
be adjusted prior merging the PR.
* Adjust embedding-linked-filters.cy.spec.js
* Adjust joins-reproductions.cy.spec.js
* Adjust metrics-editing.cy.spec.js
* Adjust timelines-question.cy.spec.js
* Adjust offset.cy.spec.ts
* TMP Avoid duplication of temporal unit in display name in MLv2 code
* Probably FE fix: add default temporal bucket only if not present
* WIP Add ends-with-temporal-unit? function
Current strategy is to add temporal units to display names if same is not already present
* Adjust display-name-method :metadata/column
Append TU to display name only if different to what's already present
* Adjust FE: show temporal unit that's present on field instead of default on hover on QueryColumnPicker element
* Return also temporal-unit from visible-columns
* Enable tempora-unit to be returned from previous-stage-or-source-visible-columns
* WIP Adjust multi breakout multi stage e2e multiple-column-breakouts.cy.spec
* Adjust multiple-column-breakouts.cy.spec.ts
* It was not a bug.
* Revert changes in _BaseBucketPickerPopover
* Restore temporal-unit for next stages removal in stage.cljc
* Add inherent-temporal-unit
* Adjust display name temporal bucket logic
* Rename to inherited-temporal-unit
* Shutdown too broad display name capitalization
* Eastwood
* Adjust pivot-rows-order-test
* Adjust pivot-non-numeric-values-in-aggregations
* Adjust inherent to inherited
* Rename to inherited
* Add inherent-temporal-unit only to returned-columns
* Update e2e/test/scenarios/dashboard-filters/temporal-unit-parameters.cy.spec.js
* Update e2e/test/scenarios/question/multiple-column-breakouts.cy.spec.ts
* Update consistent-date-formatting-test
* Update dashboard/x-rays.cy.spec.js
* Update waterfall.cy.spec.js
* Return month default for default inherited-temporal-unit
* Update reproductions-3.cy.spec.js
* Update downloads/downloads.cy.spec.js
* Update legend.cy.spec.js
* Update pie_chart.cy.spec.js
* Update click-behavior.cy.spec.js
* Update embedding-linked-filters.cy.spec.js
* Return QueryColumnPicker's condition to original state
* Remove reundant export
* Update e2e/test/scenarios/question/multiple-column-breakouts.cy.spec.ts
* Comment
* Update nested.cy.spec.js
* Update e2e/test/scenarios/sharing/sharing-reproductions.cy.spec.js
* Adjust to TS types
* Adjust for cljfmt
* Capitalize
* Add propagation tests
* Add inherited-temporal-unit propagation for expressions
* Add default inherited-temporal-unit test cases
* Add qp temporal-bucket-test ns
* Simplify default unit logic
* Comment
* Use NULL_DISPLAY_VALUE
* Comment
* Comment
* Comment
* Comment
* Comment
* Comment
* Remove redundant code (for now)
* Update comments
* Comment
* Restore way too daring simplification
* cljfmt
* Adjust capitalize
* Update comment
* Update test
* Address remarks in tests
* Update test
* Address remarks
* typo
* Use Flex instead of Group for items in button
* Allow column picker to stretch to match the height of the element
* Fix height of data picker buttons
* Make sure the whole button is clickable
* Add e2e test for metabase/metabase#50038
* Use correct id for issue in e2e test
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* attempt to fix 49537 by removing position relative from the tooltip
container
* Make echart tooltip container cover all iframe area
* Update the E2E test
* Update loki snapshots from CI
* Re-enable echarts tooltip assertion.
I couldn't find a better way to fail these assertions on the previous
version of the code, but these assertions are better than nothing.
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* XLSX Pivot Export - Initialize Pivot Table with Small Area Ref
This PR changes the native pivot export so that we first initialize the pivot table with an area ref that's only the
first 2 rows. Prior, I used `AreaReference/GetWholecolumn` which had the side effect of using lots of memory.
I think this is related to how the Pivot Table's Cache is built up as you add rows/cols to the pivot definition. So,
if you keep the area ref only as wide as the number of columns in the raw data, and just 2 rows, then the cache can
stay small.
After adding row/column data to the pivot table, you can then modify the area ref to be all rows in the relevant
columns (`A:E` for example).
This keeps the pivot-table object much smaller in size.
* need the extra xml schemas to get this to work
I'd like to try find a way around including this extra dep, it's a 13mb jar (ish), and I think that's a little bigger
than I'd like
* oops, didnt mean to have this in
* Add test for pivot table initialization being fast and lean on memory
* remove time based assertion, add comments in impl
* cljfmt
* fix test
* Pesky little whitespace
---------
Co-authored-by: Oleksandr Yakushev <alex@bytopia.org>
* XLSX Pivot Exports Now Use The Sort Settings from Viz Settings
Prior to this PR, the xlsx pivot exports worked correctly except for the fact that no sorting (ascending or
descending) was applied to the pivot rows or columns. Now, those settings are used when they exist.
* only try to set sort when a correct setting exists
* add ooxml-full dep for correct classes
* Update src/metabase/query_processor/streaming/xlsx.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
---------
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Fixes: #50072
When we `describe-fields` we are passing db details, but sometimes we
have the database name in `dbname` rather than in `db` depending on the
connection specs and possibly environment. So we check for both now.
== Goal ==
Hide attached DWH database details from anyone incl. admins:
* Do not show them in the UI
* Do not permit to change them
* Do not serialize them
- This is what we're adding for H2 snapshots (dumps) with this PR.
The aim is that customers cannot gain access to (parts of) credentials,
and they cannot break a feature they are paying for by changing
connection details.
== Implementation ==
In 592360c9fa I wrongly understood that
database details would be omitted when dumping to H2, but this is only
true for dumping H2 databases.
Fix this by omitting database details also when dumping databases with
`is_attached_dwh` set.
== How to test ==
To prepare, download a H2 JAR matching the H2 version used by Metabase:
```
wget https://repo1.maven.org/maven2/com/h2database/h2/2.1.214/h2-2.1.214.jar
```
If the H2 version does not match, you will get an error when trying to
open the H2 shell:
```
Exception in thread "main" org.h2.jdbc.JdbcSQLNonTransientConnectionException: Unsupported database file version or invalid file header in file "[REDACTED]" [90048-232]
```
=== New behaviour ===
Setting the `is_attached_dwh` field hides the database details from H2 dumps:
1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
- In addition to the fields you would normally set, also set
`is_attached_dwh: true`.
- This also works when adding this flag to a database that previously
did not have this flag set.
2. Start your Metabase instance.
3. Run `${metabase} dump-to-h2 ./dump-file-h2` to create the H2 snapshot file.
4. Run `java -cp h2-2.1.214.jar org.h2.tools.Shell -url "jdbc:h2:file:./dump-file-h2;ifexists=true"` to open a H2 shell.
5. Verify that `SELECT name,details FROM metabase_database;` shows `{}` for the database you added in step 1
=== Original behaviour ===
Behaviour without setting the `is_attached_dwh` field is unchanged:
1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
- Only set the fields you would normally set. Do not set
`is_attached_dwh` (or set it to `false`).
2. Start your Metabase instance.
3. Run `${metabase} dump-to-h2 ./dump-file-h2` to create the H2 snapshot file.
4. Run `java -cp h2-2.1.214.jar org.h2.tools.Shell -url "jdbc:h2:file:./dump-file-h2;ifexists=true"` to open a H2 shell.
5. Verify that `SELECT name,details FROM metabase_database;` shows a non-empty object (i.e. not `{}`) for the database you added in step 1
Fixes: 592360c9fa
Closes: https://github.com/metabase/harbormaster/issues/5526
* fix(sdk): remove `Cannot update a component XX while rendering a
different component` error
* test(sdk): introduce jest test to make sure api.basename is set before
loading the locale
* sync the attached datawarehouse
requires either "sync_db" true or a table name
it will error
```shell
❯ http post 'localhost:3000/api/notify/db/attached_datawarehouse' x-metabase-apikey:apikey -pb
Must provide `sync_db` or a `table_name` to sync
```
It will sync the db when sync_db is true [NOTE: this is outdated. now just omitting the table name will sync the db]
```shell
❯ echo '{"synchronous?": true, "sync_db": true}' | http post 'localhost:3000/api/notify/db/attached_datawarehouse' x-metabase-apikey:apikey -pb
{
"success": true
}
```
It will sync existing tables
```shell
❯ http post 'localhost:3000/api/notify/db/attached_datawarehouse' table_name=existing schema_name=public x-metabase-apikey:apikey -pb
{
"success": true
}
```
it will error if it cannot find a table
```shell
❯ http post 'localhost:3000/api/notify/db/attached_datawarehouse' table_name=new schema_name=public x-metabase-apikey:apikey -pb
{
"cause": "Unable to identify table 'public.new'",
"data": {
"schema_name": "public",
"status-code": 404,
"table_name": "new"
},
"message": "Unable to identify table 'public.new'",
"schema_name": "public",
"table_name": "new",
"trace": [],
"via": [
{
"data": {
"schema_name": "public",
"status-code": 404,
"table_name": "new"
},
"message": "Unable to identify table 'public.new'",
"type": "clojure.lang.ExceptionInfo"
}
]
}
```
if i create that table
```sql
attached=# create table new (id int);
CREATE TABLE
```
it will then find and sync it
```shell
❯ http post 'localhost:3000/api/notify/db/attached_datawarehouse' table_name=new schema_name=public x-metabase-apikey:apikey -pb
{
"success": true
}
```
* formatting
* Tests for attached datawarehouse
* PR nits
typo in `thorw` -> `throw`
comment explaining why `find-and-sync-new-table` is always sync
gensym the db name in test
remove a binding for a fn `sync!` that's only called once
test `{:sync_db true}` syncs the whole db
* stupid formatting 🦾
* Use absence of table_name as indicator to sync database
* Fix smartscalar texts to use user locale
* Fix Pie chart texts to use user locale
* Fix line chart tooltip texts to use user locale
* Add trend chart assertions
* Add pie chart assertions
* Add line chart other series tooltip assertions
* Update frontend/src/metabase/visualizations/echarts/pie/constants.ts
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Fix failed E2E tests
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Apply Column Sort To Pivot Sections
Fixes#49437
This PR fixes the dataset API so that column sorts (ascending/descending settings on pivot-rows and pivot-cols) are
properly parsed and passed through the QP.
After that, I've also modified the post processor to use the sort orders properly in pivoted exports.
* println debugging :)
* fixing a few tests
* fix a few more tests
* Update workflow on master
* [DEBUG] Disable steps for testing
* Add alberto as a reviewer
* [DEBUG] modify the branch name for testing
* Revert "[DEBUG] modify the branch name for testing"
This reverts commit 7d5d8bf802.
* Revert "[DEBUG] Disable steps for testing"
This reverts commit a8662c9e8b.
* Try snowflake 3.19.0
* Deafault to java 21 for drivers
* Trigger CI
* Try running things with --add-opens to see if it solves our problems
* Fix `update-view-dashboard-timestamp-test`
* Try snowflake 3.19.0
* Deafault to java 21 for drivers
* Try running things with --add-opens to see if it solves our problems
* Fix `update-view-dashboard-timestamp-test`
* Update deps.edn
* Switch to Java 21
* Docker image needs to use --add-opens option
* Add note about change to Java 21 to driver changelog
---------
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Luis Paolini <paoliniluis@gmail.com>
* add filter picker skeleton
* add filter picker skeleton
* add picker in popover use case
* reuse FilterColumnPicker
* add question filter logic
* remove fixed width from popover
* add onClose handler
* add e2e test skeleton
* contain FilterBar in Box in story
* add a test for adding filters
* update e2e tests
* add filter picker to the docs
* inline props and functions
* update the props
* Allow using the arrow buttons to change the limit value
* Add a min prop to the number input
* Use base-10
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Ensure number is integer
* Add issue name to unit test
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Use ParameterValueWidget for default value picker in native parameter settings
* Fix e2e test
* Fix native filter tests
* Fix e2e test
* Fix e2e test
* Add repro for #44665
* Assert that filter values are shown in the popover.
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
This also cleans up the testing to check everything more carefully,
and to expect the right things across multiple stages.
(This is a bit tricky since the order-bys get removed when the stage
gets aggregated, because the columns they were using are now gone.)
* Remove dead code from browse models page
* Use the grid to control the gap
* Implement "Add database" button
* Add basic unit tests
* Make sure the feature works when there are no databases
* Add analytics to the db list
* Add "Add database" button to the sidebar
* Add E2E Snowplow tests
* Add unit tests
* Deal with authentication in each beforeEeach separately
* Add Snowplow E2E test for a database card
* Slightly tweak the title line height
Search currently returns a detailed list of the scores for each item, which is pretty hefty, and the type is also coupled to the current rankers, which will change in future.
Removing from the type to start to just validate that nothing uses it.
Best to just not model this. Going to see about just removing it from the BE as well, and put in some other way to debug. Worst case an opt-in query parameter.
* Add blurb from C3P0 docs to comment for unreturnedConnectionTimeout
* Enable debugUnreturnedConnectionStackTraces in data-warehouse-connection-pool-properties
Closes#47981
* Add defsetting for jdbc-data-warehouse-debug-unreturned-connection-stack-traces
And only enable debugUnreturnedConnectionStackTraces in the C3P0 config if the setting is enabled. There is some
memory cost to enabling the setting, and since maxPoolSize is user-configurable, it seems safer to disable
debugUnreturnedConnectionStackTraces by default.
* Add metabase.logger/level-enabled?
* Log a warning if C3P0 logs are not configured correctly to log stack traces
Log a warning if the user enables the jdbc-data-warehouse-debug-unreturned-connection-stack-traces but does not raise
the log level for the com.mchange package to INFO or higher.
* Add docs for MB_JDBC_DATA_WAREHOUSE_DEBUG_UNRETURNED_CONNECTION_STACK_TRACES
* Improve markdown docs for MB_JDBC_DATA_WAREHOUSE_DEBUG_UNRETURNED_CONNECTION_STACK_TRACES
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Keep Pivoted Option works in Subscriptions
Fixes: #49525
Even though the user could toggle 'Keep data pivoted` in the Sharing Sidebar UI, the setting was not being propagated
to the backend, so attachments would not be pivoted.
* add a cypress test
* add viz settings to get the pivot working
* fix for linter
* remove eslint ignore
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
I had to add `only-safe-columns` to filter out columns with known issues
in current pMBQL. In particular, implicitly joining on a column which
appears twice (eg. because of a double- or self-join) is ambiguous with
classic `:field` refs, and will cause errors.
There's a `:dynamic` var to re-enable all the columns for tests that are
only exercising new `:column` refs.
* Revert "[streaming-response] Use HttpEndpoint.isOpen() instead of reading a byte from TCP socket (#49460)"
This reverts commit 060baf0ce4.
* Disable connection reuse for requests that serve streaming responses
This introduces a reusable `weighted-choice` function and a "weight"
system for the various steps. `:order-by` should be more rare than
filters, etc., `:limit` and `:append-stage` should be more rare still.
Adds basic expressions to the query generators.
There's a lot more to do to robustly generate realistic expressions! But
this is enough to make expressions exist on queries, which then get
consumed as inputs to filters etc.
* build on java 21
* update docker image
* update test
* more using 21
* back to 11 for build
* update docker command
* update containerization
* update docker actions
* revert dockerfile change
* update other docker action versions
* use release commit for containerization
* Remove @flaky tag
* Unskip test and remove @flaky tag
* Remove @flaky tag
* Remove @flaky tag
* Revert "Remove @flaky tag"
This reverts commit 08c57add88.
* Try to fix flake
* See if timeout is necessary
* Revert "See if timeout is necessary"
This reverts commit 181f3c5e63.
* Revert change for coordinates filter
* Remove @flaky tag
* Make workaround test pass
* Disable workaround test
* Uncomment and unquarantine the workaround test
* Disable normal test for now
* Revert "Disable normal test for now"
This reverts commit 68fa2175a7.
* Bring back quarantine
* Update comment
* Move the upload CSV button to the main nav bar
* Remove the old `SidebarOnboardingSection`
* Remove obsolete unit tests
* Rework E2E tests
* Add some spacing around the button
* Fix E2E test
* Break `MainNavbar` unit test down
* Expand unit tests
* Update E2E tests
* Fix E2E test failure
* Move JSDoc where it belongs
* Provide context for translators
* Simplify the condition
* Pivot Measures Order Used in Pivot Exports
Fixes#48442
A pivot table can have any number of measures, and the user can order these by dragging in the UI. Before this PR,
that order was ignored and measures would alway be in index order, which is confusing for any user who needs the
measures to be displayed in a particular order, especially if they've re-ordered them in the pivot viz settings UI.
A test has been added to check that measure order is used.
A few minor changes to the pivot qp and post-processor
- measure indices are looked up in the pivot qp and added in viz-settings order
- the pivot measures are only added if the qp has not already added them.
- pivot-opts Malli spec has been made in the namespace, adjusted to allow `nil` as a valid pivot-opts output, and
used in relevant functions
* address review points.
* add a rows order test
* Filter out Empty Rows from Pivot Exports
Fixes#49353
The linked issue is not actually related to pivot export size but is instead related to the 'Min of Created At: Month'
aggregation; the default aggregation function was `+`, so it broke when the date string was encountered.
That was fixed.
As I was trynig to keep the export small if possible, I noticed that in some cases empty rows are appended, so I added
the filter so that if a pivot row's values are completely empty, it doesn't add it.
Finally, this PR also adds the 'sub section totals' which I noticed were missing from the exports. This comes up when
you have 3+ pivot-rows configured, so you can see the subtotals for the first pivot row and the subtotals nested
within those sections for the second pivot row, and so on.
* add test for non-numeric values
* Make sure the new name refs match on aggregations not just breakouts
* cljfmt
* move the database prompt to the very beginning
* ability to use sample database for cli
* print username and password in console
* no tenancy isolation as a default option
* highlight sample credentials file
* update credential log messages
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* continue without tenancy isolation
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Fix :pivot-measures with column names
* Fix the test
* Update src/metabase/query_processor/pivot.clj
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Add a test
* Add a test
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
Adds filters to generative testing for queries.
There's a lot of new code here, but don't panic, it's quite straightforward:
generating filter values, in the right shapes for various filter expressions.
* refactor + more specific auth errors
* fix linting issues
* make tests pass, again
* make test pass for real?
* continues the refactor and hopefully makes tests pass
* try to simplify the logic a little bit more
* also logs to the console for the edge case of invalid session
* remove setLoginStatus
* fix typo
* fix debug line that should have been commented
* remove dead code
* remove test for code that doesnt exist
* address feedback from PR
* update test copy
* Update enterprise/frontend/src/embedding-sdk/store/auth.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Update enterprise/frontend/src/embedding-sdk/store/auth.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* update copy for site settings error
* ex -> exception
* Update enterprise/frontend/src/embedding-sdk/store/auth.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Update enterprise/frontend/src/embedding-sdk/store/auth.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* update tests to use the updated copy + small typos
* refactor: use jest spy on to check for console.error
Co-authored-by: "Mahatthana (Kelvin) Nomsawadi" <me@bboykelvin.dev>
* use full error message in test
* renamed response to session
* mock implementation of console.log
* fix(49482): don't create a new store on each MetabaseProvider render (#49492)
* fix(49482): don't create a new store on each MetabaseProvider render
* remove store prop as it seems to not be used
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
The query can contain all kinds of things, in particular the whole original query in the case of pivot questions, so
we should check only the part that actually gets resolved by the middleware.
* perf: Implement faster sync methods for postgres
Fixes#48575
Pulls work from redshift into the common postgres driver.
* Fix tests and formatting
* Move nested-field-column sync to sync functions so describe-fields will also get them
* Fix test
* Fix test
* Remove fixed safety test
* Add test specific database-supports feature for pk metadata
* Fix test
* perf: faster mysql sync with describe-fields
Fixes: #49010
* Adrress PR feedback
* Fix tests
* Fix test
* Add nil table-schema
* Don't use subselect for field-comment
* Fix quoting weird identifiers
* Make format string inline
* Update src/metabase/driver/mysql.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update src/metabase/driver/mysql.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Fix tests
* Fix database-type
* Fix tests
* Fix test
* Fix tests
* Exclude mysql table_schema
* Handle tinyInt1IsBit
* Fix test
* Only get one db at a time
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
These are helpful for creating literal dates and times in a
cross-platform way. Useful for generating queries, in particular for
expression and filter values.
* add base color palette
* add some more comments
* fix formatting and color copy-pasta errors
* ignore existing bad base color usage
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
The ref (usually) contains enough information to unambiguously find the
matching implictly-joined column, even if there are several FKs that
could join the target table.
The remaining ambiguous case is if the *same* FK column exists
multiple times - current MBQL uses raw field IDs for the FK field, so it
can't distinguish between eg. an FK in the main table and the one from a
self-join.
* Update saveDashboard helper to wait for the GET /api/dashboard/:id request
- update aliases to be unique so that they don't interfere with aliases used in tests
* Remove no longer needed intercept + wait
* only tag nightly on the current version
* add release channel setting to publish job
* clean up release workflow
* remove code from something else
* reduce depth of the release process stack
* update build job too
* remove dead arg
* Pivot Measures Order Used in Pivot Exports
Fixes#48442
A pivot table can have any number of measures, and the user can order these by dragging in the UI. Before this PR,
that order was ignored and measures would alway be in index order, which is confusing for any user who needs the
measures to be displayed in a particular order, especially if they've re-ordered them in the pivot viz settings UI.
A test has been added to check that measure order is used.
A few minor changes to the pivot qp and post-processor
- measure indices are looked up in the pivot qp and added in viz-settings order
- the pivot measures are only added if the qp has not already added them.
- pivot-opts Malli spec has been made in the namespace, adjusted to allow `nil` as a valid pivot-opts output, and
used in relevant functions
* address review points.
* add a rows order test
* only tag nightly on the current version
* add release channel setting to publish job
* clean up release workflow
* remove code from something else
* fix step title
Also fix a bug in `lib.breakout/existing-breakouts` that treated
breakouts on duplicate columns from different joins as existing, even
though they're not.
* add ability to hide the collection picker
* add save options
* make sure collectionId is fresh
* update sdk question form component
* make collectionId accept null
* apply saveOptions to sdk components
* revert to use initialCollectionId as context name
* target the correct collection id
* add e2e tests
* update e2e tests
* move into within
* update test
* remove collectionId from default args
* revert e2e of hide columns
* make e2e helper more generic
* use saveToCollectionId prop to save to a specific collection
* update e2e tests
* update comment formatting
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* clean up leftover changes from an earlier version
* readme update
* indicate that saveToCollectionId is only applicable to static questions
* revert QuestionEditor.isSaveEnabled defaults
* update comment format
* remove unused initialCollectionId prop
* experiment: try to fix failing custom-columns e2e
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Adding moderated status to entity picker
* adjusting verified icon size
* size adjustments
* e2e test adjustments
* removing verified model icon
* fixing e2e test
* unit test
* unit test adjustment
* Adding moderation status to Dashboards
* react sins
* verification coll: subselect on card + dashboard
* fix the 3 failing BE tests
* add an api test for dashboard verification
* verified dashboards are never stale
* e2e tests
* code cleanup
* unit test and e2e test cleanup
* PR Feedback
* backend test fixes
* Nick shouldn't touch clojure
* type fix
* indent
* fix last 2 BE tests
---------
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* WIP: use list picker when searching a single value
* Add test case for multi vs non-multi search fields
* Update loki snapshots
* Add special case to test for multiautocomplete filter
* Use single select value in test
* Add test for using list picker
* Explicitly make sure elements are visible
* Do not use list selector when search is disabled
* Add margin to multi autocomplete dropdown
* Update loki snapshots
* Update loki snapshots
* Fix scroll for EditableDashboard in fullscreen mode
* Fix scroll for InteractiveDashboard in fullscreen mode
* Display fullscreen control to dashboards by default
* Fix issue with public shared questions empty footer
Previously these were omitted, and it caused invalid SQL to be
generated, since the ORDER BY (time unit not updated) and GROUP BY
(time unit updated) did not line up.
Fixes#49263.
Lays the foundation for a weighted probability approach to generating infinite sequences of valid queries.
Only has aggregations so far, but we can expand it to filters, breakouts, order-by, removals etc. etc. in later PRs.
* Update fullscreen dashboard filter on dark theme
* Use the latest base color name
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
* perf: Implement faster sync methods for postgres
Fixes#48575
Pulls work from redshift into the common postgres driver.
* Fix tests and formatting
* Move nested-field-column sync to sync functions so describe-fields will also get them
* Fix test
* Fix test
* Remove fixed safety test
* Add test specific database-supports feature for pk metadata
* Fix test
* Adrress PR feedback
* Fix test
* Don't use subselect for field-comment
* Fix quoting weird identifiers
* Make format string inline
* Remove unused component
* Remove extra space
* Remove cross-component dependency on the `Banner`
* Use named export
* Convert the Banner to TS
* Make `ReadOnlyBanner` use `Banner` under the hood
* Add test id to the banner
* Simplify render method and tests
* Add the missing test for ReadOnly banner
* Avoid using special ASCII characters for the common apostrophe
* Remove E2E spec
* Use `Banner` for `DatabaseInfoField`
* Use colors defined in the theme
* Remove the dependency on the `Banner` component again!
* Add a comment
* Use pattern matching
* Mongo objects should download as JSON, not EDN
Fixes#48198
Prior to this change, object columns (base or effective type of :type/Dictionary) were just formatted with `(str
value)` which results in a csv or json download containing EDN formatted objects.
This is a bug because we present object column values as json in the app, so the expected formatting of the download
should match this.
The formatter function now takes this type into account. As well, since this is a type of formatting that should be
always applied (even when format_rows is false), the function is modified to unconditionally apply the json/encode
formatting to dictionary types when encountered.
* add a test
* add proper condition to test
* card-download should be public
* uncomment json encoding formatter
* set-cell! should keep encoded json string for Objects
I think this is the correct change; I don't really understand the reason for wrapping, encoding, decoding, and then
string-ing that value. Maybe I'm missing something.
* Adjusted xlsx Object set-cell! implementation
* forgot the not... inverted
* Update cursor position when selection changes
* Add reproduction for #33640
* Revert "Add reproduction for #33640"
This reverts commit 573f6320f5.
We cannot reproduce the bug with tests, so having a test is useless.
* Add reproduction for #12586
* Do not show the dirty overlay when an error occurred
* Fix tests that rely on dirty overlay
* Add issue tag to test case
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add max-height to the click actions popover to prevent overflow and infinite scroll
* Add reproduction for #48851
* Update reproduction to speed it up
* Use existing helper for table header click
* Add max height to string filter picker
* Revert "Add max-height to the click actions popover to prevent overflow and infinite scroll"
This reverts commit 166478d892.
The audit collection is a bit weird. Users have permissions on the
collection but even superusers don't actually have write permissions on
the elements within it.
For now, I'm just excluding anything in the audit collection from the
"stale" endpoint, so we won't suggest cleaning these up, but I wonder if
maybe there's a cleaner way here. It feels weird to fake permissions
like this.
Fixes https://github.com/metabase/metabase/issues/49164
* Add new route
* Add the navigation link
* Guard the route
* Add first pass of the page structure
* Add the bottom section
* Add placeholder dummy YT videos
* Add images
* Track checklist being opened
* Track checklist item expanded
* Track checklist item CTA clicked
* PoC: Stop the video on accordion change
* Use actual onboarding videos
* Fix missing unique `key` prop
* Final content tweaks
* Override the accordion styles defined in the `theme`
* Use predefined t-shirt sizes where possible
* Update copy
* Conditionally display the example dashboard CTA
* Add very basic unit test coverage
* Show onboarding link to new instances only
* Remember the last position in the onboarding
* Fix the mock settings types
* Scroll last opened item into view
* Properly display docs links
* Another accordion style override
* Add tests for the Onboarding page contents
* Add E2E Snowplow tests
* Conditionally display xray item contents based on settings
* Split tests
* Make sure items scroll into view when expanded
* Do not show example question CTA if example dashboard doesn't exist
* Allow YT videos to play in fullscreen
* Add comments about the example question
* Update rules for when we show the onboarding nav link
* Test the conditional documentation links display
* Reduce E2E test flakiness
* document loader and error states
* document loader and error states in the docs
* revert changes to README.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* pie chart crashes when light or dark accents of instance colors overriden
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Add verifyNotebookQuery helper
* Use describeEE
* Fix DB id
* Refactor test
* Refactor tests
* Introduce customizeLinkText helper
* Use verifyNotebookQuery
* Fix test setup
* Fix static embed card download when param has accentuation
* Fix E2E util type
* Add a dashboard E2E test
* test question download with params
* Fix some names regarding tests
* Fix failed E2E tests
* Fix csp directives for embed previews
We set content security directives to allow for iframes on
dashboards. This list did not include 'self' so we can't actually host
an iframe pointing at our, well, self.
Embed previews work by just embedding an iframe with the dashboard and
this breaks if we don't allow iframes from our self.
* e2e test
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
### Description
This PR updates Macaw to the latest version, which exposes a new analyzer which handles compound queries.
It also finishes the migration to using tagged-union result maps consistently across all the interfaces.
This should also remove the atrociously verbose analysis error messages in CI.
* investigate hide columns
* add update viz settings in the sdk
* cleanup implementation
* add storybook for interactive question
* use the question callback to update viz settings
* remove onUpdateVisualizationSettings
* revert change to onUpdateVisualizationSettings
* add click action behaviour
* fix misspelling
* add onUpdateQuestion mock function
* add unit test for click action behaviour
* add e2e test for hiding question column header
This standards on a "just maps" approach to returning disabled, disallowed, or failed query analysis. This is as opposed to the hodgepodge approach of significant `nil`, untagged unions with keywords, and exceptions.
Essentially we have a classic "either" style tagged union, where `:error` is the tag.
* test(sdk): add tests for auth flow
* make the fetch token fn return something that will pass the validations
we'll add in the next project
* waitForRequest helper + add expects for card endpoints on api key test
* upgrades is-airgapped? defn to defsetting airgap-enabled
- uses the airgap-enabled settings value to remove Cloud section in
SettingsEditor
- "Metabase Enterprise Airgap" -> "Enterprise Airgap"
* hide still need help section in airgap mode
* move flag checking into selectors
* fe lint
* ts type check
* use optional chaining and nullish coalescing
* hide StillNeedHelp in ag mode
* adding airgap-enabled to createMockSettings
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* add allowed iframe host setting wip
* use allowed-iframe-hosts setting in the CSP header
* add a test for the frame-src csp directive
* Update allowed-iframe-hosts setting definition
* Add error state for forbidden iframe url domains
* Move out iframe e2e test suite
* Add e2e test
* Update error message in view mode
* Fix unit tests
* Update setting on the admin page
* Update error state
* Add links to error states
* Update docs links
* Update link anchors
* Add default allowed hosts to public setting
* Update allowed domain check logic
* Fix default value display in admin page
* Don't update setting without changes
* Update error message spacing
* correct the parsing of allowed-hosts string for CSP header entries
* fix test
* fix not handling wildcard ports
* Fix failing e2e test
* Fix subdomain test
* address review
- the parse-allowed-iframe-hosts fn is now memoized
- a * entry is handled and doesn't produce a weird *:* entry
- no more try/catch, errors in parsing will be logged but the list returns all valid entries
- when www. is encountered, an entry including www. is added
- trailing / is 'cleaned' and the entry is used as if there was no trailing /
* Fixup test for expecting a few more frame sources
* indentation fix for linter 🥲
* Fix type error
---------
Co-authored-by: Adam James <adam.vermeer2@gmail.com>
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
Co-authored-by: dan sutton <dan@dpsutton.com>
* ui implementation of new emb homepage, tests will come later
* change padding of the last card to be consistent with the top one
* update beta badge spacing
* update tests
* fix stories type error
* use quick-start for url
* update HomeContent tests
* update events + related e2e
* Update frontend/src/metabase/home/components/EmbedHomepage/StaticEmbedContent.tsx
* Add `graph.max_categories` viz setting
* Allow grouping series into "other" category
* Add tooltip
* Toggle "other" series visibility
* WIP: new implementation for `graph.series_order`
* Remove not used function
* Make "other" series a regular `SeriesModel`
* Fix stacked bar charts
* Remove colors from "other" tooltip
* Sort "other" tooltip values
* Update `graph.max_categories` setting
- negative number validation
- 0 value turns off grouping
* Fix Loki test
* Add basic loki test
* Add loki test for stacked chart
* Add loki test for stacked normalized chart
* slight refactor of series_order setting
* Add screenshots
* Handle different kinds of aggregations for "other"
* Handle different aggregation kinds in tooltips
* Add `graph.other_series_aggregation_fn` viz setting
* Fix type errors
* Fix unit tests
* settings ui
* remove sorting by value to match series order setting, other color
* hide grouped series controls
* Add e2e test
* Update screenshots
* Fix incorrect series grouping for stacked charts
* group series into other settings
* Update e2e test to work with new viz settings
* Disable drills for the "Other" series
* fix max_categories setting popover positioning
* fix total row has misaligned columns on other series values
* Remove redundant `click({ force: true })`
* Rename aggregation fn viz setting
* Move aggregation fn setting to the popover
* WIP
* Update popover
* Fix `it.only`
* Fix legend sync issue
* Disable "Other" category by default (temporary)
* Enable aggregation fn picker for MBQL queries (temporary)
* Move "Other" category summary to a tooltip row
* Add `graph.max_categories_enabled` to viz settings type
* Reuse already computed "Other" value in tooltips
* Add null check
* Fix series length check
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
This is to allow for the possibiliy of adding more metricsv2 usage and error metrics in the future while still
distinguishing the source of the errors.
### Description
This is a low level step towards applying table-level permissions to native queries.
It replaces the more generic and powerful `:ast-walker-1` analyzer with the more conservative `:basic-select` version we recently landed. The advantage of this version is that it protects against dynamic table references and false negatives due to masking.
This was throwing errors trying to `nippy/freeze!` an Atom.
That was because in certain circumstances a pMBQL query can end up in
`(:json_query metadata)` for a query, and those can contain an atom
due to `metabase.lib.cache` and both `CachedMetadataProvider` and
`InvocationTracker`.
I added `lib.query/serializable` as a single location to strip out
anything that isn't serializable, such as atoms. (And the entire
`:lib/metadata`, which is a wasted effort to serialize.)
* Fix reference in deprecation notice
* Add lock_filled icon
* Add view-only tag to the question header for questions with hidden source tables
* Simplify the useHasHiddenSourceTables helper
* Add typings to useHasHiddenSourceTables
* Show 'view-only' badge when the question is not editable
* Set up card endpoints
* Add test for hidden tables
* Add e2e test for View-only tag
* Add visibilityType to TableDisplayInfo
* Use unfiltered table display info to determine which tables are hidden
* Rename util to hooks
* Use rem for border-radius
* Sign in in beforeEach
* Add tests for more visibility types
* Fix unit tests
* Remove empty file
* Add test for unavailable cards
* Remove unnecessary card endpoint
* Add unit test for inaccessible card
* Move ViewOnlyTag to it's own file
* Add more unit tests for ViewOnly tag
* Use MBQL to create queryies where possible
* Increase the size of the overlay spacer
* Increase the delay of the popover opening
* Hard-code the popover delays
* Remove delay props from info components
* Fix unit test
* Shorten popover timeout to 500ms
* Make popover delay configurable again
* Increase openDelay for header info
* Fix a Couple bugs related to pivot exports
- pivot-grouping value, when not an int (in the case of Oracle dbs), would cause exports to contain zero rows. This
fixes that
- json formatted/unformatted for the dataset API endpoints (unsaved questions) was not being used, so all exports
were formatted
* json and csv use (int group) to check row inclusion
* grouping change fix tests
* json formatted/unformatted test
* xlsx formatters are correct when pivot-grouping col is removed
* cljfmt
* fix up failing tests
* fix test
* Update src/metabase/api/dataset.clj
* Reset the question's viz settings when converting to a model
* Update e2e/test/scenarios/models/reproductions.cy.spec.ts
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Test model metadata
* Test model metadata
* Test model metadata
* Test model metadata
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix#48878 by avoiding to use entity framework which can serve incomplete card objects
* Make sure canWrite is a boolean
* Remove redundant optional chaining
* Add repro for #48878
* Optimize
* Improve comment
* Fix test setup
* Automate npm version
* Make troubleshooting SDK release process easier
* Fix quote situation
* Allow deploying SDK to different branches at the same time
Because they won't be in conflict with each other
* Make checkout a bit faster
* Remove unnecessary console.log
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Tidy up `jq` command
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Tidy up multiple GitHub actions summary log
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Add helper to add utm params to url
* Add utm param to getDocsUrl
* Allow passing utm to docs url
* Add utm parameters to expression widget docs url
* Add useUrlWithUtm hook
* Make more utm props optional
* Use the utm hook and docs hook instead of adding utm tags manually to urls
* Simplify getDocsUrl
* Update unit tests for expression editor to contain utm params
* Only add parameters if they have a value
* Fix unit test
* Fix unit test
* Simplify docs url
* Use useDocsUrl on embedding page
* Use named arguments in useDocsUrl
* Modularize metabase.pulse; add metabase.pulse.core API namespace
* Rename metabase.pulse.send-test
* Split email result attachment stuff into its own namespace
* Update dashboard-subscription-test
* Allow metabase.email.result-attachment externally... for now
* More code cleanup
* Decouple models.card and email.messages
* Decouple models.collection and metabase.email.messages
* Hopefully the last fix 🔧
* Fix#48647🙁
* Fix event handler
* :render/text, not :pulse/text
* Test fixes 🔧
* Test fix 🔧
* RESPECT the modules
* Remove unused namespace
* defineEmbeddingSdkConfig to make typing easier when the config is created outside of the jsx
* also add defineEmbeddingSdkTheme, move them to index.ts as i don't have a good name for what file to put them into :shrugs:
* updated the readme to use the defineXXX functions
== Behaviour without this fix ==
See #48881
== Behaviour with this fix ==
1. Metabase starts with an empty application database, and a config file
containing configuration for a connected database
3. By the time it reaches "INFO metabase.task :: Task scheduler started",
`qrtz_triggers` contains `metabase.task.sync-and-analyze.trigger.1`
(sample DB) and `metabase.task.sync-and-analyze.trigger.2` (DB from
config file)
Closes: https://github.com/metabase/metabase/issues/48881
Co-authored-by: Noah Moss <noah@metabase.com>
* Use type date for case expression when there are Date and DateTime args
* Use logic for getting case expr type from annotate middleware
* Add e2e test
* Add bigquery test
* Update test
* cljfmt and comment
* Adjust e2e test
* Let non-admins see tables in recents
```shell
❯ http get "localhost:3000/api/activity/recents?context=views" Cookie:"metabase.SESSION=ba..3d" -pb
{
"recents": [
{
"can_write": false,
"database": {
"id": 6,
"initial_sync_status": "complete",
"name": "pg restaurants"
},
"description": null,
"display_name": "Restaurants",
"id": 112,
"model": "table",
"name": "restaurants",
"table_schema": "public",
"timestamp": "2024-10-15T22:11:39.412100Z"
},
{
"can_write": false,
"database": {
"id": 6,
"initial_sync_status": "complete",
"name": "pg restaurants"
},
"description": null,
"display_name": "Reviews",
"id": 110,
"model": "table",
"name": "reviews",
"table_schema": "public",
"timestamp": "2024-10-15T19:53:19.999445Z"
}
]
}
```
We were getting permissions of a "fake" table. Instead, let the db
select it and get the proper stuff. Worked for admins because
mi/can-read? is presumably always true for tables, and worked in tests
because we mocked mi/can-read?
* bump ci
* Add new promethues counter for :metabase-metrics/adjust-errors
To record errors in metabase.query-processor.middleware.metrics/adjust
* Add prometheus metrics test to metrics middleware tests
* Also inc :metabase-metrics/adjust-errors when throwing an exception
* Make splice-compatible-metrics private
It doesn't seem to be used outside of metabase.query-processor.middleware.metrics.
* Add test for :metabase-metrics/adjust-errors when exception is thrown
* Move inc-and-throw! helper into metabase.analytics.prometheus
* Add inc-and-throw! test to prometheus tests
* Rename inc-test to inc!-test and use approx= and metric-value helpers
* Add try/catch to metrics adjust and record metric errors in a single place
Record promethues metrics in a single place in metabase.query-processor.middleware.metrics and ensure we record
failures for any exception throw.
* Improve testing description string
* Rename :metabase-metrics/adjust-errors to :metabase-query-processor/metrics-errors
* Move metrics adjustment try/catch into a new helper
This ensures we count all exceptions generated by adjust-metric-stages.
* Add new prometheus metric to count number of MB metrics seen by the QP
This new metric is intended to count the number of metrics that the query processor metrics middleware sees and
attempts to adjust.
* Add more metrics metrics tests
- Ensure that queries with no metrics do not incremement either counter
- Ensure that successful calls to adjust do not increment the error counter
- Ensure that exceptions throw from other libs also increment the error counter
* Rename test to adjust-prometheus-metrics-test and add helper to reduce code dup
* If no metrics are found immediately return query in metrics middleware adjust
Don't attempt to adjust query if we don't find any :metric clauses.
* Appease cljfmt
* Improve metric description
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Improve epsilon test in approx=
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Remove adjust-metric-stages-counting-errors
This function was an attempt to avoid double-counting errors, but probably not worth the effort.
See
https://github.com/metabase/metabase/pull/48598#discussion_r1800197964https://github.com/metabase/metabase/pull/48598#discussion_r1800200421
* Count metrics directly when mocking prometheus/inc!
Based on PR feedback here:
https://github.com/metabase/metabase/pull/48598#discussion_r1800207674
* Remove prometheus/inc-and-throw!
Value prop was always questionable, but now that it's only called from a single place, just remove it and inline the
call to prometheus/inc!. Easy to add back later if it proves generally useful.
PR discussion thread
https://github.com/metabase/metabase/pull/48598#discussion_r1799728596
* Rename match-one-metric to find-first-metric
https://github.com/metabase/metabase/pull/48598#discussion_r1803321025
* Rename :metabase-query-processor/metrics-errors to metric-errors
https://github.com/metabase/metabase/pull/48598#discussion_r1803313818
* Throw an exception rather than logging a warning if we fail to adjust a metric
https://github.com/metabase/metabase/pull/48598#discussion_r1803328482
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* faster --watch command for the sdk
* apparently we want an empty line at the end of jsons, but prettier removes it?
* format tasks.json with prettier 😕
* vscods tasks, move watch task from webpack config to the fixup script
* cleanup webpack config
* adds clean + waits for dist folder when not existing
* minimal doc for the sdk commands
* adds data + schema for metrics stats ping
* remove comment
* annotate todos
* fill in the rest of the metrics values
- and add defaults
* fix some definitions + use a single timestamp
* shuffle stuff around to appease the namespace linter
- we aren't reaching into the api API, so the linter is more of a
formality here.
* update docstring
* fix jsonschema, maybe
* review responses
- revert changes to 1-0-0
- add metrics section to new file: 1-0-1
- bump ::instance_stats to "1-0-1"
- add tags into 1-0-1
- make the code return tags (empty for now until we know what to tag things.)
- also fix test that broke from shuffling settings around
* remove a footgun
* add tags to metrics,
add grouped_metrics to jsonschema 1-0-1
format 1-0-1
* indent
* cljfmt
* version should match filename
* update instance stats 1-0-1 schema
* require `tags` in grouped_metric + snowcat comment
* fix formatting noise
* update schema to make it valid
* remove grouped_metrics from instance_stats schema
* cljfmt appeasement
* unbin cache_num_queries_cached value
- alphabetize metrics generation
* we can now guarantee metric values will be ints
* jsonschema for instance_uuid, settings, and grouped_metrics
* add analytics_uuid and make it required
* lint + alphabetize instance stats json schema
* update setting key type
- add maxLength to some strings
* lint jsonschema
- Add description to setting.items.tags
- add maxLength, and {min,max}imum to setting.items.value
* Bump instance stats to 2-0-0
- remove analytics_uuid string length == 36 check
- adds assertion to ensure required fields are set (and it passes)
- adds info for embedding settings
* adds a grouped-metric to stats ping
- adds length info to the schema to pass jsonschema linting
* cljfmt
* Incremental Pivot Processing for Exports
WIP
Fixes pivot exports for CSV and xlsx.
The CSV export should use less memory by incrementally building up the data structure and aggregating necessary row
data right away, so the memory overhead becomes only as large as the total pivot result.
In cases where the pivot rows/cols do combine into many many columns and rows, this can still be a large set of data,
but it should behave much better now in most cases.
The Excel export is a little more straightforward: create the export rows in the same fashion, streaming one row at a
time, and just post-process the sheet to add the pivot table in one shot at the end.
* WIP adding row totals.
* aggregate totals as rows are added
Row, column, section, and grand totals are all aggregated as each row is added.
This means the final step of building pivot output becomes just an exercise of lookups/arrangement, no further
aggregation is needed.
* CSV pivot works per-row, export respects formatting
This is a big step forward; we don't need to hold the entire dataset in memory, we instead aggregate a row's data into
the pivot datastructure, which only holds onto:
- unique values for each pivot-row in a sorted set
- unique values for each pivot-col in a sorted set
- grand total for each measure N values, where N is number of measures, ususally 1 or 2
- row totals for each combination of each pivot-row * N measures
- col totals for each combination of each pivot-col * N measures
- totals for each 'section', determined by unique values of first pivot-row * N measures
- values for each measure in every 'cell'; Row Combos * Col Combos * N Measures
So, there can still be a decent amount of data to store; but it will never hold onto all of the 'raw rows' from the
dataset.
We can never completely guarantee that Row Combos * Col Combos * N Measures remains small, but two things let us move
forward anyway:
- there's now visible feedback in the app that the download is running (or if it's failed)
- Pivot table utility diminishes rapidly with huge output anyway; users still need to curate/set up their data
- effectively to improve the table's utility, so we can assume that a slow-to-download pivot table is also slow to
- use/less effective, and will likely be something the user doesn't want (as often).
* some test fixes
* now, if we export 'raw pivot rows', they don't show pivot-grouping
and they also don't include the 'extra' rows for totals/subtotals/grand totals (any row with pivot-grouping > 0).
This means that now the non-pivot version of a pivot table export will match what a user sees if they change the viz
to a regular table.
* remove old test
* re-incorporate some changes from master
* fix csv for non-pivots due to oversight in my changes
This is just a temporary change, I think I should clean up this bit of the code a little, I can probably make it a
little more readable and use some cleaner logic regarding if the rows are 'raw pivot rows' or not.
* start moving format_rows to POST bod, add pivot_results too
There's still wiring work to do, but this starts to add format_rows and pivot_results to POST body for the various API
endpoints. Also modify tests to improve coverage/consistency across downloads and alerts/subscriptions.
The tests will not pass on this commit, but fixes will be incoming
* native pivot tables in xlsx
* add precondition to pass migration linter
* try to get migrations fixed
* pasing pivot-results through api and attachments
* fix tests for format_rows in BODY vs query param
* tests!
* might have the tests all fixed now
* the pivoted export now respects col/row totals settings
* add test coverage for public questions and dashboards
* col and row totals work as expected
* build-pivot refactor for clarity
* docstring change + tiny refactor in helper fn
* see if dashcard download works with format_rows
* csv pivot handles nil values
* pass format_rows and pivot_results in :params not :body
* fix some other tests
* pivot-grouping col filtered out of xlsx
* pivot-grouping-col removed for all rows
* configurable pivot exports and attachments (#47880)
* exports fe
* specs
* ui
* specs
* format/unformatted now works for xlsx
* format test changes for xlsx formatting
* embedding endpoints accept pivot_results
* cljfmt and eslint fix
* empty
* embedding test should have formatting defaulted to true
* embed test fixes
* Use `Chip` for export settings widget
* downloads e2e test fix
* fix public download limit test
* public card download defaults
* fix public download defaults in some tests
* Fix visual test
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
* Remove 'Save as new question' toggle when saving questions from editing a metrics' ad-hoc question
* Allow visitQuestion to work for metrics
* Use visitQuestion in metric test
* Add test for #48555 (showing the 'Save as new?' ui for metrics)
* Add issue info to repro
* Use visitMetric, visitModel and visitQuestion helpers in createQuestion
* Move check to make it more clear what the save ui does
* Use cy.button for Done button
* Remove unnecessary .within
* Sign in as normal user
* Add missing dataset alias
* Refactor createDashboardWithQuestions, createQuestionAndAddToDashboard and editDashboardCard to TS function helpers
* Fix createNativeQuestionAndDashboard that got out of sync
* Rename Common Metrics to Metrics
* Rename Basic Metrics to Summaries
* Update chill mode button copy to 'Add a summary or metric'
* Rename 'Pick the metric you want to see' button to include summary
* Fix type in test
* Rename Summaries to Basic functions
* Fix test
* Undo cross-version test changes
* Add eslint-plugin-jest-formatting
* Turn on jest formatting for unit.spec and cy.spec files
* Fix e2e test formatting
* Fix unit test formatting
* Fix formatting in frontend/test
* Remove commented out code
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix more files
* Fix more lint
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add repro for #44171
* Do not save viz settings for metrics
* Lowercase description
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Rename variable to be more descriptive
* Use visitDashboard
* Fix typo in excludeVisualisationSettings
* Check the request does not contain visualization_settings too
* Allow visitQuestion to work for metrics
---------
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Remove `MB_API_KEY` env var
A bit awkwardly, we never set `:deprecated` on the setting before. We
can retroactively deprecate this as of v50.
I'm keeping the setting purely to emit the warning message on startup.
* Experimental: try splitting MySQL test jobs into 4 partitions intstead of 2
* user-http-request should make sure users are initialized
* Fix MySQL deadlocks in tests
* Bump init timeout to 90 seconds
* Fix metabase.api.session-test/logout-test
* Add 'Create metric' button to empty metrics page
* Add e2e test for 'Create metric' button
* Remove duplicate main on Browse page
* Add docs link
* Split off common test for BrowseMetrics
* Add whitespace
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Use cy.button
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Hide Create metric button when the user does not have data access
* Use findByText for link
* Add test for hiding the 'Create metric' button when the user does not have data access
* Fix unit tests
* Improve copy
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add whitespace
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add explanatory comment for enterprise
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add repro for #47058
* Delay the response and test for the actual metric button
* Move setup to before and find summarize block using helper
* Use beforeEach
* Write Vitess limitation for row limits in MySQL databases
* Update docs/databases/connections/mysql.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Relax the arg types to ExpressionArg for concat expressions in the legacy schema
Relax the arg types to ExpressionArg for concat since many DBs allow to concatenate non-string types. This also aligns
with the corresponding MLv2 schema and with the reference docs we publish.
Fixes#39439
* Add nested concat schema tests
* Add nested-concat query-processor tests
* Always startup prometheus metrics
previously only started up when a port was provided
```
MB_PROMETHEUS_SERVER_PORT=9191 java -jar metabase.jar
```
But these counters are useful to be included in anonymous stats. So
let's start up the collectors and then we can get their values like:
```clojure
prometheus=> (dotimes [_ 500] (inc :metabase-email/messages))
nil
;; prometheus/value is iapetos.core/value here
prometheus=> (-> system :registry :metabase-email/messages prometheus/value)
501.0
```
* rename `metabase.analytics.prometheus/inc` to `inc!`
it side effects a value and now no longer shadows `clojure.core/inc` so
we're all happy
* don't set enable-embedding and setup-embedding-autoenabled in setup actions + cleanup
* update jest test
* shouldn't expect the secret key to be there either 🙃
This PR reimplements driver/describe-table for MongoDB. Before we would query a sample of documents from a collection and analyse them in Clojure. Instead, we now now execute a query that does a similar aggregation, but most of the calculation is done in the Mongo database.
Based on a few tests the performance is slightly slower when the collection contains small or deeply nested documents but much faster for large ones. But the main difference is in memory usage. This uses very little memory in the Metabase instance because all of the aggregation is done in the database.
Nested fields are a naturally recursive problem but here we unroll potential recursions to a `max-depth` number of queries that look for nesting at each depth level.
* ~ use DB to describe the table
* ~ optimize root query
* ~ nested-level-query works and gets objects too
* + root query gets objects too
* + driver/describe-table :mongo works
* ~ remove old implementation
* Various fixes for faster sync
Upgraded driver to 5.2.0
Updated data load to insert many rather than 1 row at a time.
Dropped max-depth to 7, see comment.
---------
Co-authored-by: Case Nelson <case@metabase.com>
* Move mbql.u/{mbql-clause?,is-clause?,check-clause} to schema.helpers
And remove duplicate definitions.
Closes#39333
* Fix docstring of schema.helpers.mbql-clause?
This func is apparently no longer used by code in `normalize`, which has it's own local def.
* Use more granular permissions
* Simplify the "upload spreadsheet" button logic
* Simplify the "add database" button logic
* Go back to using the existing `databases` prop
* Conditionally display `UploadInput`
* Refine `canUpload` logic
* Reduce prop drilling
* Make visual logic more "reader-friendly"
* Fix and expand tests
* Pass down the existing prop `hasDataAccess`
* Remove the Metabase "learn" link
* Move sub-components out of the render
https://github.com/metabase/metabase/issues/41919#issuecomment-2400542908
This issue goes away in 5.3.1 of apache poi. Seems like they have a
yearly release cadence so we can ensure this exists for the time being
and then remove this when we can bump to 5.3.1
```diff
index b763a1ffdf..c87992c935 100644
--- a/deps.edn
+++ b/deps.edn
@@ -122,7 +122,7 @@
{:mvn/version "2.23.1"} ; allows the slf4j2 API to work with log4j 2
org.apache.logging.log4j/log4j-layout-template-json
{:mvn/version "2.23.1"} ; allows the custom json logging format
- org.apache.poi/poi {:mvn/version "5.2.5"} ; Work with Office documents (e.g. Excel spreadsheets) -- newer version than one specified by Docjure
+ org.apache.poi/poi {:mvn/version "5.3.1"} ; Work with Office documents (e.g. Excel spreadsheets) -- newer version than one specified by Docjure
org.apache.poi/poi-ooxml {:mvn/version "5.2.5"
:exclusions [org.bouncycastle/bcpkix-jdk15on
org.bouncycastle/bcprov-jdk15on]}
```
* [MS 1.1] Better Onboarding via First-Time UX (#48202)
* Add new `learn` icon
* Add initial visual elements WIP
* Add a menu WIP
* Use named export instead of a named one
* Encapsulate the menu item logic
* Redo the styling
* Add todo notes
* Get rid of the `style` prop where possible
* Fix CSS logic for the container
* Collapse section when the sidebar is closed
* Use Mantine prop instead of the `style`
* Fix the translation issue
* Collapse section completely when the sidebar is closed
* Remove hard coded color reference from SVG
* Simplify element positioning
* Collapse sidebar when closed
* Fix E2E tests unscoped selectors
* Rename props
* Extract `SidebarOnboardingSection` logic into a separate component
* Allow only admins to add data
* Remove "Add your own data" button
* Simplify menuitem component
* Add initial unit tests
* Use predefined vars for the CSV button subtitle
* Show Upload CSV modal
* Enable file uploads
* Improve file organization
* Tidy up types
* Add analytics
* Remove "Upload CSV" button from the sidebar
* Fix "unmocked GET request"
* Make menu usable on small screens
* Make sure sidebar is scrollable
This PR
* mocks audit requests in tests of `DashboardInfoSidebar` and `QuestionInfoSidebar` (these tests were flaking)
* introduces a helper, `setupAuditEndpoints`, to facilitate this in the future
* uses the helper in one other place where audit requests were getting mocked
* Maintain selected folder state in Entity Picker (#47309)
* Keep tabs mounted
* Rename TablePicker onChange to onItemSelect to match QuestionPicker and also call it for folders
* Inline small functions
* Remove unused type
* Rename
* Remove TODOs
* Remove duplication
* Reduce diff
* Rename
* Lift state from TabsView
* Remove redundant callback
* Store folder state in DataPickerModal
* Refactor computing tabs
* Consolidate search tab definition with recents tab definition
* Simplify computeInitialTab interface
* Extract RECENTS_CONTEXT
* Store tab state in DataPickerModal
* Propagate initial tab state
* Fix a flicker
* Keep tab state in EntityPickerModal
* Fix typing computeInitialTab
* Remove Notebook prefix
* Pass onItemSelect to individual tabs from EntityPickerModal
* Auto select folder
* Fix types
* Adjust remaining tabs definitions to use render
* Adjust tabs definitions to use render in tests
* Fix inverted condition
* Fix TS error
* Fix types
* Rename EntityTab to EntityPickerTab, TabId to EntityPickerTabId
* Rename EntityTabRenderProps to EntityPickerTabRenderProps
* Separate tab models from tab ids
* ESLint
* ESLint
* Recognize folders inside EntityPickerModal
* Add missing prop
* Remove console.log
* Simplify handleFolderSelect
* Revert changes that will go to a different PR
* Fix unit tests
* Refactor
* Remove keepMounted prop
* Fix type
* Remove useMemo from tabs computation (#47505)
* Extract getTabs for DataPickerModal
* Extract getTabs for EntityPickerModal
* Add explicit return type
* Revert "Add explicit return type"
This reverts commit feb3f58c3c.
* Revert "Extract getTabs for EntityPickerModal"
This reverts commit 4e875a1fc1.
* Revert "Extract getTabs for DataPickerModal"
This reverts commit cdda8ea185.
* Remove useMemo
* Maintain tab state in Entity Picker (#47794)
* Store picker path state in data picker modal
* Adjust remaining QuestionPicker usages
* Store collection picker state
* Use dedicated onPathChange prop
* Store dashboard picker state
* Improve naming
* Fix unit tests
* Remove accidental code
* Return to previously opened tab when search input is cleared in Entity Picker (#47798)
* Restore previous tab when clearing search input
* Add a unit test for new behavior
* Update comment
* Don't use effect
* Add more tests
* Reduce diff
* Fix flakes
* Change search input placeholder depending on selected folder & tab in Entity Picker (#47801)
* Make TabFolderState generic, pass folder to EntityPickerSearchInput
* Simplify interface
* Extract getSearchInputPlaceholder
* Reduce diff
* Remove TODO
* Implement it
* Shorten code
* Reduce diff
* Use database to scope search if there's only 1 schema
* Add scoped search in the Search tab in Entity Picker (#47839)
* Cleanup entity picker search types (#48098)
* Add e2e tests for local search in the entity picker (#48122)
* Sync the value back into selectedItem when it changes
* Await element
* Handle empty tabs
* Pick everywhere tab
* Click Everywhere in unit tests
* Use deep equality to sync value
* Sync back the path value to the dashboard too
* Run test as admin to avoid (unrelated) permissions issues
* Filter scoped results too
* Fix post-merge issues
* Fix type issues
* Fix null scoped search results
* Sign in as admin to appease the tests
* Use useDeepCompareEffect when syncing back state
* Add default value for searchResultFilter to avoid double condition
* Avoid using useEffect to sync selectedItem
* Avoid overwriting path in useEnsureCollectionSelected
* Fix e2e test
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Parallel driver tests PoC
* Set fail-fast to false for now
* Try splitting up non-driver tests to see how broken tests are
* Whoops fix plain BE tests
* Ok nvm I'll test this in another branch
* Fix fail-fast
* Experiment with the improved Hawk split logic
* Fix some broken/flaky tests
* Experiment: try splitting MySQL 8 tests into FOUR jobs
* Divide other Postgres and MySQL tests up and use num-partitions = 2
* Another test fix 🔧
* Flaky test fix 🔧
* Try making more stuff fast
* Make athena fast??
* Fix a few more things
* Test fixes? 🔧
* Fix configs
* Fix Mongo job syntax
* Fix busted test from #46942
* Fix Mongo config again
* wait-for-port needs to specify shell I guess
* More cleanup
* await-port can't have a timeout-minutes I guess
* Let's only parallelize MySQL for now.
* Cleanup action
* Cleanup wait-for-port action
* Fix another flaky test
* NOW driver tests will be FAST
* Need to mark driver tests too
* Fix wrong tag
* Use Hawk 1.0.5
* Fix busted metabase.public-settings-test/landing-page-setting-test
* Fix busted `metabase.api.database-test/get-database-test` etc. (hopefully)
* Fix busted `metabase.sync.sync-metadata.fields-test/sync-fks-and-fields-test` for Oracle
* Maybe this fixed `metabase.query-processor.middleware.permissions-test/e2e-ignore-user-supplied-perms-test` maybe not
* Fix busted metabase.api.dashboard-test/dependent-metadata-test because endpoint had differemt sort order than test
* Ok my test fix did not work
* Fix metabase.sync.sync-metadata.fields-test/sync-fks-and-fields-test for Redshift
* Better test name
* More test fixes 🔧
* Schema fix
* PR feedback
* Split off test partitioning into separate PR
* Fix failing Oracle tests
* Another round of test fixes, hopefully
* Fix failing Redshift tests
* Maybe the last round of test fixes
* Fix Oracle
* Fix stray line
* add update channels in product
* support for changing release notes to show beta and nightly info
* dont export setting
* obey the linter and add tests
* export setting
* update e2e tests
* clojure magic
* clojure-foo
* better localization
* sorry mr linter
* add more tests 💪
* embed cards
* safety
* rename params
* review, renaming
* min and default size of iframe cards
* move ml auto to parent
* use truncate
* spec failures
* empty state
* consistent iFrame copy
* renaming, no fe tracking
* iframe copy
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
* wip locale header middleware
* wip fe code for the locale provider for both public/static and sdk
* clean up code
* move api.baseurl assignment outside of useEffect to make it work on first render
* e2e test for locale
* Revert "wip locale header middleware"
This reverts commit c215eac3f8.
* remove public/static code as we want to focus on the sdk
* adds docs for `X-Metabase-Locale` header and for why we don't use I18NApi.locale
* Update frontend/src/metabase/lib/api.js
* doc: explain why we have the unused state
* Update the labels for the Edit question action for models and metrics
* Use the metric and model editor by default
* Add e2e tests for metric and model cards
* Remove console.log
* Add extra mode for editing metrics or models
* Use slug to do exact match of path
* Rename editor mode to query
* Rename as param to mode
* Add BE test for exporting self-joined renamed columns
* Add e2e test for exporting self-joined renamed columns
See also
- Not renamed fields in a same table join inherit the renamed name in exports #48046
- backport: Support both name & field ref-based column keys in viz settings on read and upgrade on write #48243
* add sunburst to pie chart
* fix type errors
* consider all slice percent values for formatting percent on chart
* fix lint error
* refactor nested functions outside of parent
* create and use getArrayFromMapValues helper
* fix bug with percent significant digits
* improve color picker
* fix total visibility in small dashcards
* fix missing field remappings
* use aggregated value for click behavior
* update minimum slice percentage logic
* fix total text truncation
* remove colors from tooltip for middle and outer slices
* update e2e spec
* update snapshot
* update ring setting titles
* address feedback
* update comment
* fix: bigquery more resilient querying
Inline some function calls to make it easier to track what's happening.
Make sure that cancellation during the initial query and subsequent page
fetches are handled properly. Explicitly throw when cancelled.
Only retry queries if bigquery says they are retryable.
Try to cancel the BigQuery job if an exception or cancellation occurs.
* Add comment explaining execution flow
* Bump bigquery deps
* Bump biquery dependencies
* Fix tests
* Fix formatting
* Improvements to SDK readme
* Add live demo site to SDK readme
* Remove suggested workarounds for SSR
* Use - for bullets for consistency
* Remove Loom
* Update enterprise/frontend/src/embedding-sdk/README.md
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* fix(sdk): fix zindex issues after switch to portals
* Target portal to a different root than popovers and co
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* trash
* Trash update
* review comments
* link
* why are you complaining
* text to tables
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* review comments
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Add metrics to table pane
* Add test for metric in data reference sidebar
* Clean up other reference tests
* Add test for when the table has no metrics
* Use helper for sidebar header title
* Rename sidebar to create distinction from existing helper
* fix(sdk): modal pushing content below when opened
* render the portal below the content so we don't need the zIndex
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Simplify browse components imports
* Move metrics-related browse components to separate folder
* Move models related browse components to separate subdir
* Split up browse/test-utils
* Move schemas related browse code to separate folder
* Move databases related browse code to separate folder
* Move tables related browse code to separate folder
* Fix import paths
* Avoid drilling into metric module
* Avoid drilling into models module
* Move enterprise tests for browse models
* Format comments
* Remove unused sortModelsByVerification plugin helper
* Support skipToken in useFetchModels
* Simplify models content verification
* Fix filter button query
* Do not filter recent models if there are no verified models
* Use filterButton helper everywhere
* Fixup comment
* Use new var-style color
* Await dashcard query when editing
* Reload to fix flake
* Reset filters to fix flake
* Await dashcard query when editing
* Wait for query
* WIP
* Hard-visit dashboard
* Split up test to avoid navigation issues
* Do not cache all token check failures
We want to cache token checks to avoid an issue where we repeatedly ask
the store "hey, is this token valid?? is this token valid?? is this
token valid??" for the same token.
However, transient errors can also occur. For example, maybe a network
issue causes the HTTP request to fail entirely. In this case, if we
cache the result, the user needs to restart metabase (or wait 5 minutes
until the cache is cleared) before they can attempt to validate their
token again.
This PR moves the cache logic deeper into the stack. We want to cache
"successful" responses from the store API - cases where the store has
told us categorically that the token is or is not valid. We don't want
or need to cache other things that might happen. Maybe your token isn't
the right length - we can recalculate that, it's ok. Maybe you get a 503
error from the Store - we should let you retry. Maybe your network is
having issues and you can't contact the Store at all - again, we should
let you retry.
The one potential issue I see here is that if the store goes down, we'll
massively increase the number of requests we send to the store,
potentially making it harder to recover. If this is a concern, I can add
a circuit breaker: if we repeatedly get errors back from the store, back
off and stop making requests for a while.
* Add a circuit breaker to store API requests
In the pathological case where the store goes down for >5 minutes, the
cache will expire and all instances everywhere will start repeatedly
making requests for token validation at once. This might make recovering
from an outage more difficult.
This adds a circuit breaker around the API request. If the call
repeatedly throws (5XX errors, socket timeouts, etc.) then we'll pause
for 1 minute, during which time all calls to token validation will
immediately fail without making any request to the API.
After one minute, we'll allow one request through to the API. If it
succeeds, we'll go back to normal operation. Otherwise, we'll wait
another minute.
This was old logic to support certain drivers (eg. pre-JDBC Druid) and
isn't required for most. It's perfectly sound to filter or even break
out on a datetime column without bucketing.
Adds a new `:temporal/requires-default-unit` driver feature, and enables it only for the legacy Druid driver.
Fixes#47341
* Reset the dashboard parameter when setting the default value
* Fix test for filter default
* Add test for setting parameters when editing the defaults
* Use plain dispatch
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Use cy.log instead of comments
* Use icon selector
---------
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
Fixes 46575
Creating a Pivot Table Question that is based off of a model that has at least one column derived from a join failed
to display row totals.
This is because the pivot-options map was being mis-calculated; not all column indices were correctly found/passed in
to the :pivot-rows or :pivot-cols keys, causing the pivot query not to compute all necessary data.
Here, I just modify the :lib/source key of the columns whose source is a card (as determined by the existence of
:lib/card-id). The columns being checked will all have :source/breakout, which caused, in the issue's repro example,
the "NAME" column to be missed. If it instead has :lib/source :source/card, the logic inside
`lib/find-matching-column` works.
* IS_EMBEDDING_SDK_BUILD -> IS_EMBEDDING_SDK to keep things simple, as we already have that name in other places
* don't load metabase css in sdk stories
* first two stories
* make the store not a global "singleton" anymore
* basic test for font-family styles in the SDK
* allow any custom font in theme.fontFamily as we actually support it
* fix tests using hardcoded default font family
* Databricks JDBC driver base
* Add databricks CI job
* WIP data loading -- it works, further cleanup needed
* Cleanup
* Implement ->honeysql to enable data loading
* Hardcode catalog job var
* Implement driver methods and update tests
* Derive hive instead of sql-jdbc
* Cleanup leftovers after deriving hive
* Run databricks tests on push
* Cleanp and enable set-timezone
* Disable database creation by tests
* Add Databricks to broken drivers for timezone tests
* Exclude Databricks from test
* Enable have-select-privilege?-test
* Restore sql-jdbc-drivers-using-default-describe-table-or-fields-impl post rebase
* Restore joined-date-filter-test
* Adjust to work with dataset definition tests
* Adjust alternative date tests
* Remove leftover reflecttion warning set
* Update test exts
* cljfmt vscode
* Add databricks to kondo drivers
* Update metabase-plugin.yaml
* Update databricks_jdbc.clj
* Rework test extensions
* Update general data loading code to work with Databricks
* Reset tests to orig
* Use DateTimeWithLocalTZ for TIMESTAMP database type
* Convert to LocalDateTime for set-parameter
* Update test extensions field-base-type->sql-tyoe
* Update database-type->base-type
* Enable creation of time columns in test data even though not supported
* Fix typo
* Update tests
* Udpate tests
* Update drivers.yml
* Disable dynamic dataset loading tests
* Adjust the iso-8601-text-fields-should-be-queryable-date-test
* Update load-data/row-xform
* Add time type exception to test
* Update test data loading and enable test
* Whitespace
* Enable all driver jobs
* Update comment
* Make catalog mandatory
* Remove comment
* Remove log level from spec generation
* Update sql.qp/datetime-diff
* Update read-column-thunk
* Remove comment
* Simplify date-time->results-local-date-time
* Update comment
* Move definitions
* Update test extension types mapping
* Remove now obsolete ddl/insert-rows-honeysql-form implementation
* Update sql-jdbc.conn/connection-details->spec for perturb-db-details
* Update load-data/do-insert!
* Remove ssh tunnel from driver as tests do not work with it
* Update test
* Promote ::dynamic-dataset-loading to :test/dynamic-dataset-loading and modify corresponding tests
* Adjust to broken TIMESTAMP_NTZ sync
* Update read-column-thunk to return timestamps always in Z
* Comment
* Disable tests for dynamic datasets
* Return spark jobs into drivers.yml
* Update Databricks CI catalog
* Remove vscode cljfmt tweak
* Update iso-8601-text-fields-expected-rows
* Update datetime-diff
* Formatting
* cljfmt
* Add placeholder test
* Remove comment
* cljfmt
* Use EnableArrow=0 connection param
* Remove comment
* Comment
* Update tests
* cljfmt
* Update driver's deps.edn
* Update tests
* Implement alternative `describe-table`
* WIP Workaround for timestamp_ntz sync, will be thrown away probably
* Update metabase-plugin.yaml with schema filters
* Update driver to use schema filters and remove now redundant sync implemnetations
* Update tests
* Update tests extensions
* Update test
* Add feature flags for fast sync
* Implement describe-fields
* Implement describe-fks-sql
* Enable fast sync features
* Use full_data_type
* Comment
* Add exception for timestamp_ntz columns to new sync code
* Implement db-default-timezone
* Add timestamp_ntz ignored test
* Add db-default-timezone-test
* Fix typo
* Update setReadOnly
* Add comment on setAutoCommit
* Update chunk-size
* Add timezone-in-set-and-read-functions-test
* Drop Athena from driver exceptions
* Use set/intersection instead of a filter
* Add explicit fast-sync tests
* Update describe-fields-sql and add comment
* Add preprocess-additional-options
* Add leading semicolon test
* Disable dataset creation and update comment
* Rename driver to `databricks`
* Use old secret names
* Fix wrongly copied hsql list
* Temporarily allow database creation
* Add *allow-database-deletion*
* Temporarily allow database creation
* Disable database creation
* cljfmt
* cljfmt
* Require `:encryption` for string settings
For settings that are not typed as JSON, CSV, or strings, encryption now
defaults to `:no` (*except* if you have
explicitly marked your setting as `:sensitive?` - these will default to
`:when-encryption-key-set`).
I went through all our settings and provided what I think are reasonable
values here. I tried to be conservative - when I wasn't sure whether a
stored setting was sensitive, I kept it as encrypted. For example, the
`ldap-port` setting is probably non-sensitive but theoretically someone
could be using a weird port for security-by-obscurity, so I kept that
encrypted.
* Change possible values for `:encryption`
`:maybe` was confusing: let's be more explicit that the value will be
encrypted `:when-encryption-key-set` to make it obvious what actually
turns encryption on and off.
* Pre-fill cloud user info from search params
* Autofocus password field for hosted instances
* Add tests
* Parse setup user info and store it immediatelly
* Autofocus password input field for hosted instances
* Add e2e reproduction
* Fix the user info
* Fix unit test
* Support skipToken in useFetchMetrics
* WIP
* Add user setting for verified metrics
* Add verified metrics to the VERIFIED_CONTENT plugin
* Use metric plugins for verified metrics
* Move helpers down
* Do not filter for verified metrics if there are none, regardless of user setting
* Add label to filter button
* Revalidate search when card gets (un-)verified
* Add e2e test for verified metrics
* Avoid checking for verified metrics on when content-verification is not enabled
* Fix broken test
* Simplify content verification plugin structure for metrics
* Add content moderation types and API helpers
* Update content moderation plugin to make use of the moderation api in metabase/api
* Fix lint
* Fix missing list tag
* Store verified metric filter preference
* Move all helpers to the bottom
* Ensure the user setting gets updated when chaning the filter
* Add comment about component similarity
* Undo changes to embedding-sdk
* Use default values for parameters in dashboard filters when used from click action
* Add test for setting non-specified parameters to default in dashboard accessed via click action
* size and padding fixes
* linkify and align stuff in question details
* reorganize and divide question overflow menu
* hide history for analytics content
* fix verification icon alignment
* whoopsie lets commit all the files
* fix css file name
* removes the default-ee snapshot in favor for a blockUserGroupPermissions util
* commit default-ee removal i forgot, replace default strings with undefined
* fix oopsie
* fixes failures for not removing enough permissions for tests in sandboxes.cy.spec.js
* rename var
* fixes some tests where data permissions were overwriting sandboxing permissions, makes assertions in tests where sandboxing user is used but the test also passes if you were to sign in as the normal user, fixes one test that sandboxes the wrong table for the test
* do not push - wip changes
* add assertions to that network requests are provided sandboxed data, fix faulty test in dashboard-filters-source.cy.spec.js
* use helpers to clean up parts of pagination assertions
* rename assertDatasetIsSandboxed to assertDatasetReqIsSandboxed
* adds assertions on api response data
* clean up
* fix typo
* refactor helper to be easier to follow
* golfing
* pr feedback
* fix
* fix
* Return the table id from csv upload (as a header)
* Add date format that google sheets uses
* Smuggle id
* done?
* comment
* with empty h2 db
* dashing
* fix flaky test
* Parse the correct date class
---------
Co-authored-by: Alexander Solovyov <alexander@solovyov.net>
* Pivot Exports/Downloads Should Not include pivot-grouping or 'Extra' Rows
Fixes 46561
When downloading a pivot table's data, a column 'pivot-grouping' is included as well as 'extra' rows that correspond
to various totals in the table.
This PR removes the pivot-grouping column and any of these 'extra' rows from the downloads/exports.
* fix xlsx
* Fix csv
* fix json
* add test and fix some failing tests
* fmt
* Update src/metabase/query_processor/streaming/csv.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* static viz table renders filter out pivot-grouping
* address review feedback
* add some comments to try explain the changes
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Fixes#47426
Date operations return values based on the timestamp offset rather than
the session/db timezone. Converting to `timestamp_ltz` first ensures
that we get predictable results.
In the test, it is important that database timezone matches
report_timezone so that post-processing gives proper results.
* Ensure :effective-type for columns from an aggregation in a card
Fixes#47184
* Synchronize effective-type with the base-type on override
When :base-type in the column metadata is overridden with the :base-type in the field ref, set it as :effective-type
too. If :effective-type is also set in the field ref, it wins.
* add license problem hook
* add warning banner
* style cue component
* update cue component
* report issue to console
* report issue to console
* use sdk loader for pre-initialization
* extract print problem method
* use non-standard shadows
* add unit tests
* prevent console log spam during tests
* mock the embedding token feature
* mock the embedding token feature by default
* add more test cases
* add test case
* add test cases
* increase popover z-index
* simplify test setup
* simplify test setup
* do not load individual component when a license error happens
* move SdkLicenseProblemDisplay component out
* replace colors with color-mix
* replace colors with color-mix
* make the text unthemed to ensure legibility
* hard-code colors to prevent them from being themed
* hard-code colors to prevent them from being themed
* update border styles
* make the chevron unthemed
* unset console text color
* only set stroke on errors
* optimistic query settings
* document the sdk license condition
* remove redundant condition
* add notes on api keys always being allowed on localhost
* move the no auth method message to warning component
* rename sdk license problem to usage problem
* add eslint rule for stories
* rename sdk license problem to usage problem
* move sdk usage problem display out of the init controller
* specify fallback font if instance is down or CORS error
* add comments on conditions pending on the CORS PR
* add notes on non-localhost checks
* make test ids constant
* add test case for neither provided
* fix types
* address code reviews
* pass `embedding-sdk` flag in `token-features`
* address code review
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* remove appName from console logs
* rename embedding-sdk to embedding_sdk
* add the embedding_sdk premium token feature
* do not setup enterprise plugin in sdk test
* do not localize visual cue messages
* fix unit test in setup-sdk-auth
* rename tests
* remove unused eslint-disable
* enable embedding_sdk premium feature for sdk unit tests
* revert security_test.clj to use embedding instead of embedding-sdk
* remove unneeded test
* changes to support the style leak fixes
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Rollback to legacy unique-name-generator in annotate-native-cols
* Add native query long column alias test to nested queries tests
* Add native query long column alias test to dataset api tests
* Add native query long column alias test to models e2e tests
* Move limit into inner native query
* PR suggestion: simplify lambda argslist
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* PR suggestion: remove unnecessary mt/with-temp-copy-of-db
* Tweak test name
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* add offset e2e tests
* mark test not flaky
* add basic tests
* add more tests
* WIP
* working version
* cleanup
* oops
* improve naming
* fix test
* improve formatting
* update comment and ts
* add a test for metric
* space
* update tests and create a new issue
* add one more test
* re-enable test
* make imports consistent
* rewrite tests to put data through UI
* first iteration to scope sdk styles to our components
* basic css reset for buttons
* clean up duplicated css
* make sdk components use instance font if no font is passed
* cleanin up and remove defaultProps not needed
* fix tests
* adds EMBEDDING_SDK_PORTAL_CONTAINER_ELEMENT_ID
* Update enterprise/frontend/src/embedding-sdk/components/public/MetabaseProvider.tsx
* Update enterprise/frontend/src/embedding-sdk/components/public/MetabaseProvider.tsx
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* fix import
* prettier --write
* we don't need theme.fontFamily, added a comment
* rename EMBEDDING_SDK_PORTAL_CONTAINER_ELEMENT_ID to EMBEDDING_SDK_PORTAL_ROOT_ELEMENT_ID
* fix unit test
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* progress bar
* rephrase
* better word
* streamline the article
* rename image
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Initial commit of controlled upgrades
- new setting upgrade-threshold (`MB_UPGRADE_THRESHOLD`)
number 0-100
- conditionally removing latest from upgrade checks
AS OF RIGHT NOW IT ALWAYS REMOVES the latest.
Should help FE make this optional
* dumb mistake
* handle no latest version info
* implement and basic test
* More tests, refactor name
* more tests
* cljfmt does not like commas in source code.
* Just move site-uuid to the declaration and don't declare it
nothing is sacred in the settings order. no reason. i think i was just
trying to minimize the diff, which a dumb and ignoble goal.
* add context strings
* update unit tests
* use display version
* Have upgrade threshold include current major version
hopefully helps rotate people from early to late, and late to early
across major versions. Idea from sanya and quite nice!
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* more detailed question info
small styling updates
refine styling to match designs
update more e2e test
remove some changes
small styling updates
more question details
more question details
DateTime to typescript
test updates
type updates
export fix
fixes
* Icon Updates
* consolidate styles
* test updates
* address review comments
* Render values for temporal metrics
* Render data as rich
* Add tests for temporal metric
* Add TODO about isDate
* Check for errors in dataset
* Add unit test for timeseries dataset
* Add check for rows earlier
* Group tests better
* Add e2e test for metrics with no value
* Add test for metric with error
* Fix some minor collections issues
- explicitly provide access to the trash to all users, in the same way
we provide access to personal collections. Due to migration order, we
don't necessarily have a permissions row with the correct `perm_type`,
`perm_value`, and `collection_id` for the Trash collection. That's ok -
we don't actually move things to the Trash, so there isn't any
item where `collection_id=$trash.id` - but this may affect things like
effective ancestors for children of the trash. Let's be explicit about
the permissions that users have.
- replace a case where we manually calculated effective location and
then got the parent_id with just hydrating `:effective_parent`. This is
more efficient.
- replace the simple hydration method for `effective-location-path` with
a batched hydration method that fetches `visible-collection-ids` *once*
and then uses it to figure out the effective location path for each
collection passed.
This PR tests that the user can assign any of three cache invalidation strategies ("duration", "adaptive", and "don't cache") to dashboards and questions. The tests check that the following things are true:
* These strategies can be applied directly (via the sidebar)
* Dashboards and questions can inherit their caching strategy from the database that underlies them
* Dashboards and questions can inherit their caching strategy from the instance-wide default policy
* When a question inside a dashboard has been assigned a strategy, this is overridden by the strategy assigned to the dashboard.
What these tests do not examine:
* the schedule strategy
* the minimum query duration
* edge-case strategies, such as very long durations or very large values for the adaptive strategy multiplier
These may be addressed in follow-up work.
Closes#45355
* Alway render models and metrics links
* Remove useHasMetrics and useHasModels helpers'
* Update metrics test for metrics in sidebar
* Fix unit test
* Fix e2e tests
* Move stale endpoint to /ee/stale/${collection-id}
It's an enterprise-only endpoint, so it makes sense to put the whole
thing under enterprise.
* refactors ui to use ee stale endpoint
lint fixes, change endpoint location
fix failure e2e test
* Fix endpoint location
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* Don't encrypt `site-locale`
* Fix test for `site-locale`
The previous test required encryption. Theoretically the referenced
error just depends on the fact that something is logged during the
process of getting the value, so I reworked it to depend on that
directly rather than triggering it through an encryption key rotation.
* Add metric actions
* Fix dot menu width
* Refetch card when it's being (un-)bookmarked
* Add tests for metric dot menu
* Make sure bookmarked metric appears in sidebar
* Make sure restoring a metric works
* Add more tests for readonly users
* Use non-root collection for metric test
* Render Scalar metric values
* Add tooltip
* Render Scalar metric value using formatValue
* Move isMetricScalar and getMetricValue to utils
* Add e2e tests for scalar metric values
* Fix unit tests
* Add card query endpoint to metabase/api
* Use card query from api instead of question loader
* Add test for non-numeric scalar metric
* Move providesCardQueryTag to utils
* Use skipToken for query
* Clean up getDatasetScalarValueForMetric
* Add metric actions
* Fix dot menu width
* Refetch card when it's being (un-)bookmarked
* Add tests for metric dot menu
* Make sure bookmarked metric appears in sidebar
* Make sure restoring a metric works
* Add more tests for readonly users
* Use non-root collection for metric test
Fixes#45938.
Remapping is handled at the top level of the query, and should be
ignored for inner queries. This PR hides it from `results_metadata` on
source queries.
It also fixes the interaction with `large-int-id` logic to make big
integers JS-safe.
* Add series visibility props to `ChartWithLegend`
* Mark pie slices as visible or hidden
* Don't let to hide the last visible slice
* Add space between legend dot and title
* Fix legend dot outer circle is clipped by overflow
* Use `ChartSettingSeriesOrder` for `pie.rows` setting
* Actually hide slices and exclude from total calc
* Don't show "0%" next to hidden slice legend items
* Hide the "Other" slice
* Make overflow slices popover interactive
* Fix hover
* Fix incorrect visibility state in legend popover
* Add e2e test
* Remove debug stuff
* Fix hover and drill issues with slice index calc
* Fix `pie.rows` ordering bug
* Revert "Fix hover and drill issues with slice index calc"
This reverts commit f104aaa584.
* Fix hover and drill issues with slice index calc (2)
`dataIndex` based approach
* Use a special data key for the "Other" slice
* Fixed viewport size in legend e2e test
* Fix jumping legend width
* Fix dot and label vertical alignment
* Rework pie e2e test
* Fix legend dot and label alignment
* fix: bigquery add null checks to result processing
Fixes#47339
On the related issue there are different stacktraces indicating
likely sources of null pointer exceptions.
1. `.getNextPage` is likely returning a nil value. I was unable to reproduce this but one thing I did notice is that `hasNextPage` is recommended over checking `.getNextPageToken`. Added nil handling around `page` possibly being nil.
2. `cancel-chan` may be triggered before processing begins and such `execute-bigquery` would pass nil as a TableResult to the initial reducer. Testing if cancel-chan happens at just the right moment would be too flaky for CI testing but I was able to reproduce this locally and is fixed by the nil handling added.
3. `cancel-chan` may be triggered during query processing. This is covered by a test now.
* Check hasNextPage in test:
* Add test for null getNextPage
* Fix cljfmt
* Add custom image for empty metrics
* Add useFetchMetrics helper
* Copy browse models page to metrics page
* Add route for browse metrics page
* Add useHasMetrics hook
* Add link to browse metrics page to the sidebar
* Simplify MetricsTable component
* Render description markdown in table too
* Add unit tests for browse metrics page
* Remove custom padding
* Fix line height of markdown cell
* Add e2e tests for browse page
* Fix metrics sorting
* Fix types
* Fix column widths and cell padding
* Typo
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Remove name check
* Simplify alphabet
* Remove resetSnowplow
* Use helper to get metric in table
* Use MouseEvent from React directly
* Use --mb-color-icon-primary
* Test ellipsification of description
* Simplify useHasMetrics
* Move useHasMetrics MainNavBar
* Merge sortMetrics and sortModels into sortCards
* Fix metric default description
* Extract SidebarSkeleton to a separate component
* Test sorting via Collection too
* Use ModelTableRow to fix cursor
* Share more components between ModelsTable and MetricsTable
* Use the same simplified structure in ModelsTable as for MetricsTable
* Render block-level elements as inline so ellipsis detection works
* Pick better testid for metric name
* Remove delay
* Avoid type specificty error in sortCards
* Add test for not opening in new tab
* Remove useFetchMetrics defaults
* Fix unit test import
* Test meta-click with cypress
* Add comment on inline markdown elements
* Use model-name test id
* Use correct colors for icon
* Remove delay from browse models page too
* Make meta key in test platform-dependent
* Fix hover color on metrics table
* Add focus styling for celss in the models and metrics table
* Use MarkdownPreview for table cells
* Define --mb-color-icon-primary
* Make the collection link take up the whole height of the cell
* Remove tabIndex from table rows
* Rename ModelsTable.module.css to BrowseTable.module.css
* Rename CardsTable to BrowseTable
* Remove single module.css class in favor of styled components
* Allow customising allowed elements
* Allow forcing all markdown on one line
* Render bold and italics in markdown descriptions and move content onto one line
* Fix legacy tooltip in test
* Make sure getValueForSorting returns a string
* Remove empty file
* Remove icon for metrics in the table
* Remove else statement.
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Remove async findBy where possible
* Add github issue link for TODO item
* Disambiguate between getMetricDescription type
* Be explicit with sortModelOrMetric
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
We had a security issue where it was possible to query timelines without
having the requisite permissions on the collection. This test checks to
make sure this doesn't happen again.
* walk the error to remove any visualization settings
Thanks Dan for the patch, this approach is nice because it doesn't require changing all 3 streaming-response-writer
implementations (json csv and xlsx). It keeps behaviour basically the same, but just cleans up the output to not
include potentially sensitive column/dataset details.
* add a test that confirms json/csv errors are cleaned
* walk the error to remove any visualization settings
Thanks Dan for the patch, this approach is nice because it doesn't require changing all 3 streaming-response-writer
implementations (json csv and xlsx). It keeps behaviour basically the same, but just cleans up the output to not
include potentially sensitive column/dataset details.
* add a test that confirms json/csv errors are cleaned
* fix dataset error test
* cljfmt
* try to be a little more surgical with what we take out of the error
* formatting again, oops
* pass export format, only modify the obj for file exports
In the case of errors being saved to files, the obj is a map with a :status :failed key assoc'd by the qp somewhere
along the way. The format-exception cond branch is not used in this case, so we can't rely on modifying
`format-exception` to just dissoc ex-data.
Instead, I've kept the obj modification, but it excludes the query and the preprocessed keys, which are the only 2
keys where we see :viz-settings. This doesn't eliminate the problem of needing to exclude other keys in the future,
but it does improve the surface area - the query and viz settings are the most likely candidates for holding
potentially sensitive info.
I'd prefer to stick with this solution, as we can get a fix for the security concern in quicker, and perhaps a
non-security bug can be opened regarding the most correct way to indicate errors to users whose downloads have failed.
* keep format-exception the same
* cleaner passing of export-format to write-error!
* newsletter-toggle-clicked event
* use ValidateEvent,
* rename event -> simple_event and trackEvent -> trackSimpleEvent
* use the renamed trackSimpleEvent function
* Fix typo in MetadataProvider docstring
* Fix typo in caching-test: s/Second/Third/
* Store ::nil markers in CachedMetadataProvider for failed lookups
Store ::nil markers in CachedMetadataProvider for any ids for which the wrapped/uncached upstream provider fails to return
metadatas. This prevents repeatedly querying the uncached-provider for ids that don't exist.
The downside is that if the uncached-provider suddenly starts returning metadata for an id that previously did not
exist, we won't pick up on it, but the assumption here is that this is no different / worse than cache invalidation
for existing ids that happen to change after we cache them.
* PR suggestion: remove unnecessary test assertion
* fix: sqlserver handle uniqueidentifier uuids
Fixes#46148
Include sqlserver in `uuid-type` handling as its `uniqueidentifier` type
stores uuids.
* Don't be so precise with varchar size
* Add seam for drivers to cast to text type
* Fix arg order
* rename event -> simple_event and trackEvent -> trackSimpleEvent
* add empty line at the end of the file
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
---------
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
Fixes#47341.
### Description
The QP now supports temporal bucketing of custom expressions, but that has
not been surfaced in the UI until now.
### How to verify
Describe the steps to verify that the changes are working as expected.
1. Use a database other than H2, since it doesn't support time zones.
1. New question -> Anything with a DateTime column
1. Create a custom expression like `convertTimezone([Date Column], "America/New_York", "UTC")`
1. Break out by that expression
The temporal bucketing should work in the UI and in the query results, save properly, etc.
* Move current click handler to legend title
* Add `onToggleSeriesVisibility` prop to `Legend`
* Update `LegendItemDot` component
* Update `LegendItemTitle`
* Toggle series visibility
* Fix hover effect
* Don't let to hide the last visible series
* Add `visible` property to `BaseSeriesModel`
* Fix dangling split axis label when hiding a series
* Update `graph.series_order` viz setting widget
* Fix can't toggle series visibility from legend popover
* Fix legend dot clipped by `overflow: hidden`
* Fix eslint error
* Fix legend popover scroll
* Add unit tests
* Add e2e tests
* Fix show/hide series for scatter charts
* Default `LegendItemDot` to visible state
* Only apply legend dot hover effect if there's a click handler
* Don't allow to remove the last series in `ChartSettingSeriesOrder`
* Fix tooltip performance drop
* Fix e2e tests around `series_order` viz setting widget
* Use `text-medium` for outer circle background
* Update dot style
* tweak inner circle color
* Update copy
* Use div instead of button if can't toggle visibility
* Fix legend layout issue
* Don't show trendlines for hidden series
* Fix missing prop value
* Disable visibility toggle outside of dashboard/qb view
* Add public dashboard test
* Add tooltip footer e2e helpers
* Fix chart tooltip shows hidden series data
---------
Co-authored-by: Kyle Doherty <5248953+kdoh@users.noreply.github.com>
This migration is supposed to be backported to v50 and thus needs a v50
migration ID. Since v51.2024-08-27T00:00:00 was already applied to dev
instances, we set `preConditions[].onFail = "MARK_RAN"`, so that if the
table is already present, the backdated migration does not fail. See
the discussion on PR #47419 for more details.
This will not need an (automatic) backport, since I will include it in
PR #47419.
Fixes: 592360c9fa
References: https://github.com/metabase/metabase/pull/47419#discussion_r1738929516
I still don't want users' eyes to bleed looking at (unnecessary?) stacktraces, but we need to be able to see exception
causes to determine what is going wrong
references #47520
* Move query-processor.util/remove-lib-uuids to lib.schema.util
This function will soon be needed in lib.schema.util in order to implement distinct order-by clause schema checking.
This function feels like it should live in lib.util instead of lib.schema.util, but that would create the following
circular import dependency since lib.util indirectly requires stuff from lib.schema.util:
lib.schema.util -> lib.util -> lib.schema.util
* Don't create queries with duplicate order-bys in remove_replace_test.cljc
Otherwise, theses tests would fail soon when we add update the order-by schema to reject duplicates.
* Declare lib.order-by/order-bys to return ::lib.schema.order-by/order-bys
Previously it inlined the schema instead.
* Disallow duplicate order-by clauses in ::lib.schema.order-by/order-bys
Fixes: 39384
* ask for tenancy isolation columns
* deny all permissions for all users group
* create new collections
* add jwt group mappings
* add the permissions step
* add multi-tenancy message in helper text format
* add permission graph
* wire together permissions
* use schema permissions
* use fields from table metadata from query_metadata
* add tenancy field reference
* remove log messages
* deny access to unsandboxed tables
* make permission graph more explicit
* deny access to sample database for customer groups
* add unit test for permission graph
* split permission groups and sandboxes
* jwt settings and hard-coded user attributes
* handle errors when updating sso mappings
* add express api and user switcher
* only fallback to api keys when license is invalid
* add util to sample tenancy column values
* conditional BASE_SSO_API imports
* improve embedding error message
* setup jwt configuration after license step
* setup permissions at the last step
* add missing import
* update steps that requires license
* fix incorrect imports
* add missing useContext
* handle permission update error
* remove tenancyIsolationEnabled field
* add tenancy column sampling
* differentiate tenancy column query error
* rename tenancyColumnValues to tenantIds
* assign sampled tenant ids to user attributes
* add tenant ids
* define collection permissions
* reference sandboxing group by name
* update snippet to be same as the README
* extract ask for tenancy columns to a separate step
* use the customer_id attribute
* query the table query metadata at origin
* append tables correctly
* improve error handling in table scanning
* add retry logic to metadata fetching
* only query metadata for selected fields
* fix race condition with retry
* update loading state and retries
* update comments on jwt license
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* filter the target table by id
* highlight last selected tenant column
* use breakout to get list of ids
* temporary workaround to reload the whole page
* update row value types
* update row value types
* ask if they want to setup a pro license
* post-installation improvements for cli
* block non-selected tables
* remove the source-field from sandboxing
* use the fk_target_field_id as instead of target.id
* update unit test
* add learn more message
* install mock server dependencies
* fix post-setup step formatting
* fix text wrapping
* skip tenancy columns and permissions if token is not provided
* only show the server dir step when license is enabled
* skip permission setup if there is no tenancy column
* remove source-field as we only reference our own column
* add runIf flag to skip steps
* remove the gitignore mention as this is automatic
* configure permission if there are some tenancy columns
* make native permission types more strict
* add notice about setting up local mb instance
* support multiple tenancy columns
* map the tenancy column name as the user attribute
* add component import guide
* fix deprecated punycode in node-fetch
* warn on unsampled tables
* show helper text on tenancy columns
* allow sampling repeated tenant
* add instruction to change directory into mock server
* align the learn more message
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Snowflake Variant cols should also work in attachments
A previous PR fixed the snowflake variant bug by using a try/catch aroung the global-type-settings function.
I realized there was a second codepath using the same function and it gets used in xlsx attachment code, so I've made
the change there as well to prevent the same error in attachments.
* add a test to confirm the viz-settings-for-col function doesn't throw
* Snowflake Variant Type should not prevent downloads or attachments
Fixes#46981
The :type/SnowflakeVariant key matches 2 methods in `metabase.query-processor.streaming.common/global-type-settings`.
In this case, it seems that the variant type can be any type, so we shouldn't try to guess anything here, except if
the user has provided a :semantic_type, which we can use.
Otherwise, we'll keep going without formatting details about the variant column, which is likely fine already, as it
should result in strings in the export/attachments.
* add a test
Fixes#30602
We were overriding localdate in bigquery driver to produce
offsetdatetimes for no apparent reason. This causes a number of problems
when dealing with date type columns.
* Bigquery verify bignumeric fingerprint and bin
Fixes#28573
Field values must be scanned.
In 45.3, these fields were not being fingerprinted properly. (tag `v1.45.3`)
In a 45 point release that appears to have been fixed. (branch `release-x.45.x`)
Confirmed fixed in master, adding test
* cljfmt
The Portal diff view wants `[before after]` pairs of the original data,
not the three-part output of `clojure.data/diff`. This fixes the
`qp.debug/debug>` output from the preprocessing middleware to emit
correct diffs.
Also, since the diffs are not useful when the query is converted between
legacy and pMBQL, this also attaches the *converted* input query to the
output query's Clojure metadata, and this converted value is used as the
basis for the diff.
* entity id translation + tests
* add api level test
* simplify definition of eid-table->model + add test
* update tests to take keywords
* improve comment
* generate the eid-table->model map
* delete now-obsolete test
* make it work in oss
* put the resulting response into a key, so we can add more information later
* formatting
* use model names without the model/ prefix as keys
* Creates list of `api/model->db-model`
- update keys for util/entity_id request
- update shape of util/entity_id response
- add test for not-found eids
* formatting
* Respond to code review feedback
* entity-ids work on /embed/{card,dashboard}* routes
* make entity id translation work for all tokens
* Merge branch 'static-embed-dashboard-and-question-endpoints' of github.com:metabase/metabase into static-embed-dashboard-and-question-endpoints
* massage schema. add tons of tests
* decode more eids
* test alignment
* don't require ns inside of itself :think:
* make the test functions to create tokens better
* bring back `dashboard-url`
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Query Validator FE
* collection path, unit tests
* wrapping feature with token flag
* updating util function, adding row type
* updating session_test.clj
* type adjustment
* fixing other table sorting
* Empty state, clean up utils
* unit test adjustment
* e2e adjustment
* another attempt at perms fix
* try again
* try manual graphql auth
* try with global env
* Try setting defaults in graphql
* try some more things
* use my own graphql instance
* one more test
* remove test code
* remove rogue curly brace
== Goal ==
Hide attached DWH database details from anyone incl. admins:
* Do not show them in the UI
* Do not permit to change them
* Do not serialize them
The aim is that customers cannot gain access to (parts of) credentials,
and they cannot break a feature they are paying for by changing
connection details.
== Implementation ==
The Metabase backend already contains provisions in the implementation
of `metabase.models.interface/to-json` for `:model/Database` to hide the
`details` of the database in HTTP responses, if the user lacks write
permission on the database. We utilize this by adding an
`is_attached_dwh` column to the `database` table and rejecting
`metabase.models.interface/can-write?` when this flag is enabled. In
the "admin" UI, we show a replacement text instead of the edit form when
the flag is set. (It might be correct to show this whenever `details`
is absent. See below for possible follow-up work.)
However, several sections of the frontend code expected the `details`
field to always be present. In order to make `details` optional, as the
backend seems to handle it, we fix the respective code to treat this
case in the way that appears appropriate in the context.
Database details are already generally excluded from H2 dump snapshots
(see `metabase.cmd.copy/*copy-h2-database-details*`), thus nothing
changes there.
== How to test ==
=== New behaviour ===
Setting the `is_attached_dwh` field hides the database details:
1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
- In addition to the fields you would normally set, also set
`is_attached_dwh: true`.
- This also works when adding this flag to a database that previously
did not have this flag set.
2. Start your Metabase instance.
3. Verify the database shows up in the "admin" section
(`/admin/databases`).
4. Verify that clicking the database to see its details only reveals
"This database cannot be modified."
5. Verify that responses from the backend do not include a `details`
field for this database.
=== Original behaviour ===
Behaviour without setting the `is_attached_dwh` field is unchanged:
1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
- Only set the fields you would normally set. Do not set
`is_attached_dwh` (or set it to `false`).
2. Start your Metabase instance.
3. Verify the database shows up in the "admin" section
(`/admin/databases`).
4. Verify that clicking the database to see its details only reveal the
regular edit form, showing connection fields like `host`, `user`,
etc. with the values you configured.
== How this will be rolled out ==
1. Upgrade existing Metabase Cloud instances with data warehouse to a
Metabase version that supports `is_attached_dwh`.
2. Set `is_attached_dwh` in the database section of the config file for
Metabase Cloud instances with a data warehouse.
== Possible follow-up work ==
In https://github.com/metabase/metabase/issues/25715, absent
`database.details` was identified as a bug. Since then, `details` was
made `NOT NULL` in the application database, so this bug can no longer
occur. However, today backend responses can be missing the `details`
field, if the current user lacks write permission to the database
setting (see above). Fully re-evaluating the fix to #25715 in this
context is outside the scope of this PR.
Closes: https://github.com/metabase/harbormaster/issues/5051
* Add `clojure -M:kondo` and `clojure -M:kondo:kondo/all` and bump version
* Fix Kondo errors
* Fix Kondo+LSP issues with `defendpoint`, `defenterprise`, etc.
* Use replace-deps instead of deps for speed
* Ok apparently maybe we do need to copy configs when we run Kondo on CI
* Oops `./bin/kondo.sh` should not try to use `clj-kondo`
* Remove references to GA driver folders
These migrations will be backported to v49.
Three changes:
- update IDs/locations of the migrations to v49 vs v51
- add one preCondition (when adding the foreign key constraint)
- add `onFail: MARK_RAN` to the index preconditions. I forgot this
before, so it blows up when the precondition doesn't hold.
* allow re-ordering and re-naming pie chart slices
* aggregate rows with same dimension value
* show warning when dimension has unaggregated values
* change colors from sidebar
* implement renaming
* styles for slice name widget
* handle numeric slice keys for binned values relative dates etc
* implement slice disabling
* remove old colors setting
* implement merging new and removing existing rows from query results
* fix formatting bugs
* port settings to static viz
* fix bug with unaggregated data
* address todos
* add loki test
* fix measure column settings widget erroring
* fix settings sidebar crashing
* add e2e tests
* update snapshots
* fix pie_chart.cy.spec.js CI-only flakes
* fix#21504 repro failure
* fix click action value
* use nestedSettings instead of seriesSettings
* fix settings sidebar crashing
* fix row with 0 key not being draggable
* fix flaky test
* filter other slices from sidebar
* combine filter header buttons
* refined styling
* unit tests
* another import conflict
* omg another one
* remove double border for filter button
* use vars and update imports
* Increase log level to INFO from error
Closes#47396
logs before:
```
2024-08-29 08:37:03,836 INFO db.liquibase :: Checking if Database has unrun migrations...
2024-08-29 08:37:04,045 INFO db.liquibase :: Database has unrun migrations. Checking if migration lock is taken...
2024-08-29 08:37:04,047 INFO db.liquibase :: No migration lock found.
2024-08-29 08:37:04,047 INFO db.liquibase :: Migration lock acquired.
2024-08-29 08:37:04,167 INFO db.liquibase :: Running 83 migrations ...
```
logs after:
```
2024-08-29 08:39:45,019 INFO db.liquibase :: Running 83 migrations ...
...
2024-08-29 08:39:45,166 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:00::calherries ran successfully in 0ms
2024-08-29 08:39:45,182 INFO liquibase.changelog :: Columns is_defective_duplicate(boolean) added to metabase_field
2024-08-29 08:39:45,182 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:01::calherries ran successfully in 14ms
2024-08-29 08:39:45,199 INFO liquibase.changelog :: Custom SQL executed
2024-08-29 08:39:45,200 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:02::calherries ran successfully in 16ms
2024-08-29 08:39:45,202 INFO liquibase.changelog :: Foreign key fk_field_parent_ref_field_id dropped
2024-08-29 08:39:45,202 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:03::calherries ran successfully in 1ms
2024-08-29 08:39:45,204 INFO liquibase.changelog :: Foreign key constraint added to metabase_field (parent_id)
2024-08-29 08:39:45,205 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:04::calherries ran successfully in 2ms
...
```
* empty commit to bump CI
* handle cache config overflowed
* fix root too
* do not comment out test
* make cljfmt happy
---------
Co-authored-by: Alexander Solovyov <alexander@solovyov.net>
* Make metric metadata keep the metric's name
This is a fix for the problem reported in #40355, which is fixed separately for v50 and backported to v49.
* Patch broken test
* Normalize dataset-query
* Normalize legacy queries only
* ask for tenancy isolation columns
* deny all permissions for all users group
* create new collections
* add jwt group mappings
* add the permissions step
* add multi-tenancy message in helper text format
* add permission graph
* wire together permissions
* use schema permissions
* use fields from table metadata from query_metadata
* add tenancy field reference
* remove log messages
* deny access to unsandboxed tables
* make permission graph more explicit
* deny access to sample database for customer groups
* add unit test for permission graph
* split permission groups and sandboxes
* jwt settings and hard-coded user attributes
* handle errors when updating sso mappings
* add express api and user switcher
* only fallback to api keys when license is invalid
* add util to sample tenancy column values
* conditional BASE_SSO_API imports
* improve embedding error message
* setup jwt configuration after license step
* setup permissions at the last step
* add missing import
* update steps that requires license
* fix incorrect imports
* add missing useContext
* handle permission update error
* remove tenancyIsolationEnabled field
* add tenancy column sampling
* differentiate tenancy column query error
* rename tenancyColumnValues to tenantIds
* assign sampled tenant ids to user attributes
* add tenant ids
* define collection permissions
* reference sandboxing group by name
* update snippet to be same as the README
* extract ask for tenancy columns to a separate step
* use the customer_id attribute
* query the table query metadata at origin
* append tables correctly
* improve error handling in table scanning
* add retry logic to metadata fetching
* only query metadata for selected fields
* fix race condition with retry
* update loading state and retries
* update comments on jwt license
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* filter the target table by id
* highlight last selected tenant column
* use breakout to get list of ids
* temporary workaround to reload the whole page
* update row value types
* update row value types
* block non-selected tables
* remove the source-field from sandboxing
* use the fk_target_field_id as instead of target.id
* update unit test
* remove source-field as we only reference our own column
* make native permission types more strict
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* ask for tenancy isolation columns
* deny all permissions for all users group
* create new collections
* add jwt group mappings
* add the permissions step
* add multi-tenancy message in helper text format
* add permission graph
* wire together permissions
* use schema permissions
* use fields from table metadata from query_metadata
* add tenancy field reference
* remove log messages
* deny access to unsandboxed tables
* make permission graph more explicit
* deny access to sample database for customer groups
* add unit test for permission graph
* split permission groups and sandboxes
* jwt settings and hard-coded user attributes
* handle errors when updating sso mappings
* add util to sample tenancy column values
* improve embedding error message
* setup jwt configuration after license step
* setup permissions at the last step
* handle permission update error
* add tenancy column sampling
* differentiate tenancy column query error
* rename tenancyColumnValues to tenantIds
* define collection permissions
* reference sandboxing group by name
* extract ask for tenancy columns to a separate step
* query the table query metadata at origin
* append tables correctly
* improve error handling in table scanning
* add retry logic to metadata fetching
* only query metadata for selected fields
* fix race condition with retry
* update loading state and retries
* filter the target table by id
* highlight last selected tenant column
* use breakout to get list of ids
* update row value types
* block non-selected tables
* remove the source-field from sandboxing
* use the fk_target_field_id as instead of target.id
* update unit test
* remove source-field as we only reference our own column
* make native permission types more strict
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
Previously, truncating a `:type/DateTime` column by `:month` or `:day`
would return a `:type/Date`, which subtly broke the query.
In particular, if you try to order-by the breakout column `Created At (month)`
then it would not get de-duplicated, causing a SQL error about conflicting
ORDER BY clauses.
Fixes#46992.
Previously, `visible-collection-ids` was effectively "free" in that we'd
cached `collection-id->collection` for *all* collections, within a
single request, and then locally filtered it for permissions without
needing to hit the database again.
To be honest, there is probably a better fix for this - we're repeatedly
calling `visible-collection-ids` when we probably could just save a
single copy of it and use it when calculating the effective location of
every collection.
However, this is a very quick and low-risk fix, and I want to prioritize
getting this done, and then we can improve it later.
Locally, I copied down the database from stats and timed the
`/api/search?model_ancestors=true` endpoint.
Before my "speedup" PR (https://github.com/metabase/metabase/pull/46942)
it took ~7 seconds to return results.
After my "speedup" PR, it took ~15s to return results. :grimace:
With this change, it takes 818ms to return results. 🎉
* Fix typo in expression-clause-normalization-test
s/time-interval/relative-time-interval/
* Add missing table-id to card def to fix failing legacy-ref-test
* Ensure that if yarn test-cljs fails it fails the CI checks
When configured to autorun, the command
shadow-cljs compile test
will exit success even if test failures occur in the node test. See
https://github.com/thheller/shadow-cljs/issues/425
In order to ensure our CI checks fail when cljs tests fail, we need to invoke
node directly. To prevent running the tests twice, remove `:autorun true` from
the `:test` target in shadow-cljs.edn.
* zoom-in-timeseries-drill: do not include :hour and :minute for DATE columns
When the user attempts to drill through a DATE column, we should not return :hour or :minute drill throughs from
available-drill-thrus. Previously we did, which resulted in an error being displayed when the user attempted to select
them.
Modify next-breakout-unit in zoom_in_timeseries.cljc to not return :hour or :minute units for :type/Date columns,
which prevents these from being returned by available-drill-thrus, which prevents them appearing in UI context menus.
* Use lib.temporal-bucket/available-temporal-buckets
Per review feedback, use lib.temporal-bucket/available-temporal-buckets to get the valid-current-units in
metabase.lib.drill-thru.zoom-in-timeseries.
* Do not construct zipmap just to lookup the next unit
Rather than constructing a zipmap from `valid-current-units', which requires iterating over the seq twice, just
traverse the seq once and return the next item in the seq (if any).
Constructing a zipmap made sense in the old code because it was constructed once from static data and used many times
for lookup. In the new version, the vec of `valid-current-units` is dynamic and dependent on the type of the breakout.
* PR suggestion: use threading macro rather than nested sexps
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* PR suggestion: use threading macro rather than transducer / eduction
* PR suggestion: reduce boilerplate in zoom-in-timeseries tests
Reduce code dup by looping over aggregation and bucketing options rather than having a separate deftest for each.
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
Fixes#39366
* use loki to test png exports on questions
* try to put the img in the body to see if it fixes the scrollbar issues in ci
* fix linting issues
* let's try again in ci
* Attempt to make CI wait before taking a snapshot for downloaded PNGs
* Use the snapshots from CI for PNG downloads
* pdf export test thanks to kelvin suggestion about just testing the png
* refactored the code to extract a util function
* remove pdf tests as they broke in ci 🤷
* Update frontend/src/metabase/env.ts
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Update frontend/src/metabase/lib/loki-utils.ts
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* entity id translation + tests
* add api level test
* simplify definition of eid-table->model + add test
* update tests to take keywords
* improve comment
* generate the eid-table->model map
* delete now-obsolete test
* make it work in oss
* put the resulting response into a key, so we can add more information later
* formatting
* use model names without the model/ prefix as keys
* Creates list of `api/model->db-model`
- update keys for util/entity_id request
- update shape of util/entity_id response
- add test for not-found eids
* formatting
* Respond to code review feedback
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Don't encrypt boolean settings (by default)
We have tooling to disable encryption on settings even when the
`MB_ENCRYPTION_SECRET_KEY` is set. Turn it on by default for all boolean
settings, which don't need to be encrypted.
I also optimized the code that decrypts settings on startup because I
didn't want to delay startup if someone had set a bunch of boolean
settings. With 20 set, the old version added about 200+ ms to startup,
about 10ms per boolean setting, whether or not it was encrypted or in
the DB already. The optimized version selects all the never-encrypt
values from the database at once (a bit silly, but we also just exclude
raw `true` and `false` values so we don't bother checking them) and
updates them if they're encrypted - this adds ~40ms to startup with 20
encrypted boolean settings (about 2ms per boolean setting) and ~5ms to
startup on subsequent runs, when no encrypted values are in the DB.
This testing is too strict to support some legitimate use cases that
have no workaround at present. We can bring back this type-checking
eventually, once it's possible to correctly express things like
automatic coercion of strings to numbers in `SUM` aggregations.
Fixes#44431.
* Update appearance.md
Adding information about how colors are selected by Metabase because there is the confusion that the order in which you select colors is the order you get on the chart
* Update appearance.md
Removed code reference and tried to simplify the context
* Update docs/configuring-metabase/appearance.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
There are a few separate changes here:
- Migrations: add and populate indexed columns perm_value, perm_type, and collection_id to permissions
These fields allows us to efficiently run queries based on collection permissions in the DB without string manipulation. Keeping the table as permissions allows us to do this migration in-place. Note that in some cases collections may have been deleted from the database without deleting the associated permissions row (since there was no foreign key before). We need to be defensive here: if we have a permissions row without a corresponding collection, delete the row before running the rest of the migration.
- Write perm_value, perm_type, and collection_id for new collection permissions
A very simple before-insert method sets these fields before a collection permission is written to the DB.
- Replace collection/permissions-set->visible-collection-ids and collection/visible-collection-ids->honeysql-filter-clause with collection/honeysql-filter-clause
Previously, just about everywhere we used permissions-set->visible-collection-ids, what we were essentially doing was an in-app join: select all the collection IDs you have permission on, then convert it to a SQL clause like WHERE collection_id IN ( all of those collection IDs).
Replace both of these with honeysql-filter-clause, which uses the new fields we added to permissions above to construct a honeysql filter representing "all the collections I have permissions on", without needing to round-trip them to the application and back to the DB.
Of course, we can then write a function visible-collection-ids, which uses honeysql-filter-clause, for those cases where we do actually need the whole bunch in the application (we use this, for example, when constructing the effective-location for a collection).
I also added one more toggle to the VisibilityConfig that's passed into the honeysql-filter-clause (and used to be passed to permissions-set->visible-collection-ids), allowing you to select only the effective children of some collection.
* [MBQL lib] Map dashboard filters to last stage w/aggregations
Originally we used the last stage's columns for dashboard filters, but
then if you eg. use a custom expression to clean up the results of the
aggregation, then you can't filter at the most useful spot - before the
aggregations.
Fixes#19744. Related to #26244, #33958.
* Add stage-number option to the dimension not to the field
* Remove FE changes
* Support negative stage numbers
* Add tests for multi-stage parameter expansion
* Specify schema for dimension options
* Fix legacy MBQL schema
* Append stage when running cards with top level summary
* Fix schema
* Only ensure filter stage when the parameters say it's used
* Implement ensure-filter-stage
* Format clj
* Add ensure-filter-stage test
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Fix DateTimeRange substitution for open ranges
* Add E2E repro for #47172
* cljfmt
* Add test for datetime column
* Update range boundary for open range
* Update test names
* Apply suggestions from code review
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix e2e
---------
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add database type adjustments to server side generated temporal values
* Speculative base type change
* Infer timezone only from _actual_ user, not global user level
* Set timezone for actual user in test extensions
* Add test case
* Update type info generation
* Adjust server side relative datetime
* Update src/metabase/driver/sql/query_processor.clj
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Fix base type logic
* Fix typo
* Fix formatting
* Update user name handling
* cljfmt
* Address review comments
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* reorganizes data-permissions.ts to be a folder split by a file per concern, moves some logic out into other utils files where it makes sense
* pr feedback and fix lints
* only use viz height for visualization view
* move the default viz height fallback
* default to full-height if height is zero or unavailable
* remove test wrapper
* recents updates to better support metrics
allows updating metric views through the API
improves parent_collection calculation
* add 2 tests for metrics
* Use global search for aggregation picker
* Add names to aggregation picker items so they can be searched
* Render info icon metric descriptions
* Add unit tests for global aggregation picker search
* Add test for searching metrics
* Add a test for metrics descriptions
* Sort metrics
* Fix e2e test
* Use Flex from metabase/ui
* Handle markdown in metric description
* Add test for sorted metrics
* Update Markdown styling in metric description
* Fix clojure formatting
* Test for markdown in the Metric description
* Fix lint
* Fix test for markdown
* Use simpler equality test
* Fix overflow in test
* QuestionAlertWidget to FC + TS
* update e2e tests
* use regular alert for unsubscribes
* Create Question and Dashboard Sharing Menu
* Handle some edge cases
* top notch unit tests
* more testing and tweaking
* fix imports
* more tests
* add alerts to question sharing menu
* add tests for alerts
* update e2e tests
* use Oisin's toolbarbutton
* fix rebase
* prompt to save before sharing questions
* show prompt to set up notification channels
* fix import
* update tests and hide on notebook screen
* fixes
* clean up pulse types
* update e2e tests
* fix a bunch more tests
* more e2e test fixes
* maybe green now? 🤞
* last one 🥺
* design updates
* make channel setup modal nicer
* simpler component props
* lint fix
* lint fixes
* poc of removing data usage step for the tech plan
* pretend it's not cloud if hasAddedPaidPlanInPreviousStep is true
* split the jest tests files to simulate pro cloud, pro selfhosted and starter cloud scenarios
* open metabase store to get trial token
* remove license env from instance setup as we provide the license key later
* activate metabase license
* fix password field missing
* fix formatting for generated component files message
* activate license key
* add missing auth options for postgres
* add function to print with padding
* prevent infinite loop and update section representation
* show the activate license error in red
* rename variable
* Add offset presets
* Render OffsetPresets initially
* Add BucketInput
* Render BucketInput and do not rely on referential equality for Lib.Bucket
* Make sure the currently selected bucket shows in the presets
* Lift label into CompareAggregations
* Fix spacing of label
* Fix labels based on comparison type
* Use Switch for include current period
* Include current bucket in CurrentPeriodInput label
* Fix unit tests
* Add more unit tests for CompareAggregations
* Update e2e test for column compare to handle presets
* Group offset tests
* Add tests for moving average
* Add tests to verify changing the bucket works
* Fix type
* Make sure temporal columns are buckatable before picking them
* Make getPreferredPresets more clear
* Add test for temporal expression column
* Fix lint
* refactor: sort import members inside destructuring
* Show an error message when the temporal unit is not applicable to a dashboard card (#46983)
* Fix tests
* Add tests
* Add tests
* Fix types
* Fix types
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* make the newsletter signup a checkbox instead of a button
* update copy on unit test
* fix describe nesting
* update the ui to be like the design
* make tests use userEvent.click to avoid warning about missing act
* use the subscribeToNewsletter function
* add basic e2e test for newsletter subscription
* move wait to the end of the test to see if it passes in ci
* remove async to see if it fixes the issue
* trying to mock the response
* make it clear why we're mocking the response
* use getUserEmail as suggested by Sloan
* inline var a suggested on code review
* Kondo Config cleanup
* Restore warnings for with-log-messages-for-level until #28827 is merged
* Updated clojure.test hooks
* Fix kondo warning
* Test defn/defmacro exclamation point linter should also use :parallel/disallowed (part 1)
* WIP
* Time for me to learn to spell
* Finish the cleanup
* Remove code I was typing as an example
* Fix renamed var
* Fix the SAML tests
* Fix Kondo warning
* allow `:blocked` to be saved for table level perms
* Adds 2 tests for table level blocked permission settings
- N.B. these are NOT ENFORCED YET
* update test that asserted we cannot set block on tables (we can)
* WIP: Pairing on making perms checking less wild
* cleanup, update docs, and add a test for view-data perm only
- Added a test where we have data permissions, but not create query, and
I think it is failing when we have create query and blocked data
permissions.
- renamed some functions from check-x -> has-x? since they return a
value instead of throwing now
* Revert "WIP: Pairing on making perms checking less wild"
Keep the same behavior, but stick with the saner flow control
This reverts commit 63bcb5b401138555a081378574b15d6506469e60.
* update docs
* update test to be passing
- TODO: make sure it's correct w.r.t. perm settings
* Allow schema level blocked setting in permgraph
* remove invalid test cases
- continue to have a forcing function to test newly added perms
* conform function output
* ensure a single blocked table blocks native queries to its DB
* update error message
- we now catch this error in `metabase.models.query.permissions/has-perm-for-query?`
* we now check for data permissions to process query for card
* add more explanation to what we are testing
- to help see why it fails on CI and passes locally
* remove excess `def`
* Add test for table-level data X collection perms
- update test found to be in-error
* update param values qp permission check style
* set view-data and create-query explicitly
* set viewdata and createquery explicitly in qp test
* Respond to review comments (which fixes a case)
* setting a table to blocked: leave other tables the same
* [Permissions] Add "No access" schema/table permission (#46509)
* first pass
* refactors downgrading native permission logic and updates calculation so that "No access" downgrades native permissions to "No"
* stub for permissions help info on table block
* modal changes wip, updates downgrading create queries permissions to all happen at a single call site
* clean up, sandboxing modal copy changes, removes rekoke/limit access modal changes to make the diff smaller and move code to a seperate PR
* updates permissions help section to contain the final copy
* sandboxing copy fix and remove modal that was dropped from requirements
* adds blocked at the schema level, updates no access copy to blocked, updates permissions help section to contain new blocked and schema level changes
* fixes failed unit and e2e tests after sandboxing copy changes
* improve the block e2e test to include table blocking
* fixes failing blocked test, fixes other schemas create queries permissions getting correct with one schema was droped to blocked view data access, fixes a bug that prevents the save bar from going away when all permissions for group are set to the default values
* clean up
* remove color changes
* prevents parent being set to blocked preventing edits for children entities
* add new hasPermissionValueInSubgraph fn, adds modal to warn users we have to upgrade the view data permissions when they upgrade create queries permissions when a child entity is set to blocked
* adds test coverage for new modal
* removes unused function, adds new updateEntityPermission fn to help consolidate some logic elsewhere
* unit test fix and type fix
* most pr feedback
* updates the confirmation modal copy when changing a parent entity that contains a child with blocked permissions and/or sandboxed children, adds test coverage for that, adds test coverage for permission view data column not appearing in oss
* type fix
* [Permissions] Add e2e test coverage for blocked permissions enforcements (#46663)
* adds test coverage for enforcement of blocked permissions
* moves tests around based on pr feedback
* copy changes
* adds fix to make sure that blocked permissions are not removed from sibling tables that have the create queries permissions upgraded (#46854)
* Fix table name lookup for dbs w/ 1 schema per db
* add test for blank schema identifiers
* Refine sandboxed user perms for query builder access (#46939)
* Refine sandboxed user perms for query builder access
- Limit create-queries permissions to unblocked tables only
- Check user permissions for each table before granting query builder access
- Prevent querying of blocked joined tables from query builder for sandboxed users
* Adjust permissions for sandboxed users
- Grant view-data permissions only for unblocked tables
- Revert create-queries permissions to all tables in sandbox
- Remove unnecessary intermediate variable
* when sandboxing we no longer grant unrestricted view perms for blocked tables
* Update enterprise/backend/src/metabase_enterprise/sandbox/query_processor/middleware/row_level_restrictions.clj
remove blank line
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* - make coalesce-test exhaustive (except for sandbox)
* Update enterprise/backend/src/metabase_enterprise/sandbox/query_processor/middleware/row_level_restrictions.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* t2/select ... -> database/table-id->database-id
* update comment
* [Permissions] Prevent "Granular" option in DB View Data options from changing permissions to unrestricted (#46976)
* fix
* adds back most of the code and limits it to only happen with impersonations, updates test to handle differing logic between the two flows
* removes test that is not longer needed
* more sandbox join table perms tests
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update outdated loki snapshots
* Update the line snapshot using the one from CI
* try bumping the timeout to reduce flake
* Revert the linechart snapshot since it's still flaking
* skip the flaking story for themed line chart
* skip the flaking story for line chart
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
If the content-type header is missing entirely, let it through. We're
not trying to be overly restrictive here, just prohibit things that are
clearly not GeoJSON.
Fixes https://github.com/metabase/metabase/issues/46918
* Add new strings for v50.20
* update translations for 50.21
---------
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Throw reasonable error when incompatible temporal unit is used in time unit param
* Remove redundant function
* Add :curated flag to catch exceptions
* Update test/metabase/query_processor_test/date_bucketing_test.clj
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Address review remarks
* Update test
* Make vars private
---------
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* use ee api to find custom reports collection
* remove deprecated hook
* fix race condition
* add unit tests
* fix circular dependency
* fix unit tests
* another plugin strategy
* Remove unused file
* Refactor getCurrentUser command to a TS function util
* Refactor createAlert command to a TS function util
* Refactor createModerationReview command to a TS function util
* Reuse createModerationReview
* Refactor createPulse command to a TS function util
* Remove references to deleted files
* Improve typing
* Kondo Config cleanup
* Restore warnings for with-log-messages-for-level until #28827 is merged
* Updated clojure.test hooks
* Fix kondo warning
* Time for me to learn to spell
* use ECharts tooltip on cartesian charts
* fix tooltip lag when changing hovered series
* facelift tooltip
* fix static viz crashes due to tooltip option
* facelift
* pie chart echarts tooltip
* facelift
* facelift
* facelift
* tweaks
* e2e specs
* return back the pie other slices tooltip
* more specs
* specs
* fix pie total
* fix tooltip overflow
* fix specs
* fix infinite rerender in embedding mode for charts that have warnings
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Beautiful log message capturing ⭐😻
* Don't have log.cljs try to load log.capture, don't know how to make that work
* Remove restriction against use in parallel tests
* Fix tests using invalid syntax
* Port legacy tests
* Make this stuff work with Cljs
* Fix bad syntax
* Convert usages of old version of with-log-messages-for-level to new version
* Update other stuff to use the updated macro
* Fix stuff
* Fix Cljs tests
* Fix world's largest test
* Appease Kondo
* Fix comment
* ClojureScript: only emit capture code in dev builds
* perf: Use more efficient iteration and formatting in CSV export
* perf: Replace clojure.data.csv functions with more efficient implementations
* fix: Fix formatting for integer numbers in scientific notation
* concat
* we fixed this!
* format
* links
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* refactor: memoize getSteps selector
* screen.getXXX.click() => userEvent.click(...) to avoid missing act() warning
* use mockSettings so that MetabaseSettings.setAll is called and we avoid the unknown features warning
* Add type picker to comparison modal
* Adjust offset input labels based of comarison type
* Pass comparisonType to column picker
* Add current period picker
* Use deepCompareEffect to avoid rerender loops
* Add moving average helpers
* Add moving average clause to query
* Prevent less than 1 period moving average from being created
* Fix unit test for CompareAggregations
* Use the ComparisonType type for the comparisonType arg
* Hardcode the mapping for offset <-> moving-average column types
* Convert column types in the comparisonType callback instead of a hook
* Remove unused parameter
* Automatically change to 2 when setting an invalid value for moving average periods
* Use expression parts in test
* Use named arguments for movingAverage
* Fixup test for moving average offset input
* Fix type issue
* Pass on includePreviousPeriod to getAggregations
* Add defaultTemporalBucket helper
* Add helper to pick default breakout
* Move breakout to the front or add one if it does not exist
* Add updateQuery to AggregationPicker props
* Add tests for creation and reordering of temporal breakouts
* Reuse defaultTemporalBucket in withDefaultTemporalBucket
* Reorder query operations to make sure the correct names get generated
* Remove dependency on referential equality of breakout
* Rename updateQuery to onQueryChange
* Remove onAddAggregations and onUpdateAggregation for onQueryChange
* Extract query update in to separate function
* Remove on{Add,Update,Remove}Aggregation in favor of onQueryChange
* Remove aggregations from summarize details
* Fix e2e tests
* Remove conflicting test
* Fix recent clause in unit tests
* Fix superflouous check for columnAndBucket
* Future proof breakout check
* Remove reliance on details in the useSummarize hook
* Remove details in unit tests
* Removal of default summarization should not trigger a query change
* new version of data_access migration + run ANALYZE beforehand
* relax constraints on migration timestamps
* more migration rewrites
* fix bug in migration
* fix another bug and remove extraneous logging
* reorder migrations to fix ID validation
* start adapting the changes to mysql as well
* wip
* add more ANALYZE calls
* revert logging changes
* finish duplicating work for mysql + a bit of clean up
* create H2 versions of migrations without the ANALYZE calls
* remove ANALYZE call from view_data.sql (probably not necessary)
* fix mysql syntax
* fix another mysql bug
* another mysql fix
* Don't exclude inactive tables for data_access
* Don't write rows for Administrators group
* Don't write rows for Admins (H2/MySQL)
* disable triggers before perms migration
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* add setup commands
* fix settings definition
* update environment variables for cli
* handle instances not being ready
* update error messages
* add more specific loading messages
* loading spinner state
* improve error message
* use a fixed demo setup token
* remove extraneous spinner
* update status checks
* update container messages
* update wait timing
* create api keys
* extract constants
* remove manual steps
* Add anonymous tracking + other things. will need to clean up
* Modify SDK for better structure
* remove line from print.ts
* Update webpack.embedding-sdk-cli.config.js back to production
* Add types and add quick note
* Fix a typo
* Add index file, simplify types, use an array
* Add safer json parsing
* use delay of 100ms between each setup call
* Suggestions from review
* ensure that cli works
* Attempt to fix jest errors
* Remove node-fetch from sdk code to hopefully get unit tests working again
* add database connection
* add connection details handling
* refactor asking for database connection info
* apply actual database id for syncing schema
* fix failing database sync step
* allow table selection
* create model for each table
* handle errors in model creation
* fix incorrect model display name
* create x-rays based on user data
* consolidate instance setup message
* workaround for inquirer eventemitter issue
* add sample components
* fix yarn.lock file
* add sample components
* set spinner to fail state when instance setup fails
* use a transparent background by default
* update cli welcome messages
* add import notice
* refactor imports
* update snippet to add dashboard dropdown
* cli add credential file to users .gitignore
* add docs on quickstart
* clear screen with console.clear
* address code review feedback
Co-authored-by: Nicolò Pretto <info@npretto.com>
* allow node 18 lts
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Fix tooltip styles leaking to popover
* Fix snapshot now showing the tooltips and dropdowns
* Update the loki snapshot using built storybook for consistency
* Show new model loading state while fetching
* Show new model loading state while fetching
* Add unit test
* Use `DelayedLoadingSpinner` instead
* Update the test
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Prettier
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add Timeseries insights even when Custom Columns exist
WIP
Fixes 46244
I don't know why it was (maybe still is) required that the :other columns be empty before providing insights.
* might be a solution to changing trendlines when data doesn't change
The insights code that powers the frontend trendline requires taking a sample of the dataset.
This sampling has some randomness associated by design, and should maintain its randomness. But it makes some sense to
use the exact same sample when the input has not changed at all, hence trying this memoize approach out.
Might not be the final solution, but it's a start
* add test to show that insights are computed
* use java.util.random with a seed to keep insights stable
* Add testcase from https://github.com/metabase/metabase/issues/43915
* Try a custom formatter if the default formatter for OffsetDateTime doesn't work
* Add testcase
* Make auxillary formatter private
* Catch a more specific exception
* Throw IllegalArgumentException on invalid dates
* Use strict resolver and fix test
* Empty commit to assign credit where it's due
---------
Co-authored-by: Chris Truter <chris@metabase.com>
* Dowload Row Limit Env Var
Adds `MB_DOWNLOAD_ROW_LIMIT` to enable changing the row limit on downloads and alert/subscription attachments (not the
rendered tables, but the .csv, etc.).
Based on: #44982 (Thanks, @r-kot)
Partially implements: #28144
The difference in this PR compared to #44982 is that the download limit applies to all downloads, the only exception
being when the limit is above `qp.i/absolute-max-results` (1048575, based on Excel's limitation); in such a case, the
user supplied limit is only used if the download is csv or json, and `qp.i/absolute-max-results` is used for xlsx.
This PR also fixes alert/subscription attachment limits; prior to this, they were set to the in-app limit of 2000
rows, but now they will follow the user supplied download-row-limit.
This PR also adds a test to the downloads-and-exports test namespace, confirming that they follow the supplied limit,
or the max limit if none is supplied.
* add test confirming the default limit works
* fix test to use the download-row-limit
* address review feedback
* Update src/metabase/public_settings.clj
Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>
* add test covering case where download-row-limit is unset
---------
Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>
* Revert "Remove unused humanizeCoercionStrategy and its test suite (#36136)"
This reverts commit 5375347e33.
* humanize coercion strategies, add e2e test
* unit test adjustment
* pr feedback
* conditional dismissal step
* check for download status in the container
* try fixing flakes again
* try again
* chartPathWithFillColor already verifies chart has been rendered
* Introduce subpath-safe URL utils
* Open data source links from a notebook safely on a subpath
* Open browse model links safely on a subpath
* Uniform naming
* Use the existing URL helper
* Trim url
* Add unit tests
* Add one more test case
* Fix test name 😅
* Make URLs non-optional strings
* Export Formatting Correctness
WIP
Several bugs exist around the correctness of export formatting, this aims to fix them.
Fixes#27374
When a model has viz settings/metadata changed after save, the download won't incorporate said changes. At least it's
clear that scale doesn't work.
* number formatter merges settings from columns too
* column formatting on aggregated cols works in xlsx now too
* fixes 43040 by grabbing global-type-settings in csv formatter
* Refactor
* Fix wrong colors when passing empty SDK `colors`
* Clean up CSS custom properties
* Add missing color mapping to the SDK
* Add missing colors to the main app
* Only override additional dynamic colors when values are passed
* Fix unit tests
* Address review
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Move the compare to the past aggregation to the bottom of the dropdown
* Fix missing icon property
* Hide 'Compare to the past' when there are no temporal columns
* Update copy in tests
* Add test for questions with no temporal columns
* Fix unit test for AggregationPicker
* Allow adding a compare aggregation when the first breakout is not temporal too
* Rename to 'Compare to the past' in custom column action
* Rename to 'Compare to the past' in column drill
* Update copy in tests for actions and drills
* Move canAddTemporalAggregation to CompareAggregation utils
* Use aggregations from the query always
* Remove unnecessary displayName check
* Last Used Param Value doesn't fill in with default value if cleared
Fixes#46368
The Last used parameter value is cleared when the frontend sends `nil` as the parameter's value. This works except in
cases where a user clears a parameter with a default value. In this case, we want to indicate that the parameter is
cleared, so we store `nil` in the appdb in such a case.
* Update src/metabase/query_processor/dashboard.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* add assertion showing that 'nil' with default is still stored
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Fixes: #46558
In #45575 we introduced casting uuid fields for drilldowns.
However we failed to include a check that the rhs was itself another
field of uuid type and so no casting should occur. This broke joins on
models when the database_type of the field may not be known.
* Add tooltip to the data step cell
* Conditionally display tooltip
* Add unit tests for the data step
* Add unit tests for the join step
* Move the abstraction to the NotebookDataPicker
* Cover the tooltip behavior with E2E test
* Update test
* Add test boilerplate
* Rename test and add assertions
* Add broken test for models
* Add unit of time tests
* Deduplicate code
* Add date parameter tests
* Add test for multiple stages
* Fix model test setup and refactor getColumnsInfos
* Remove redundant assertions
* Format codee
* Extract createQuestion and createModel
* Add unit of time model tests
* Add test for model
* Extract constants
* Add a warning
* Build dataset queries with mbql
* Introduce queryOrders
* Revert redundant change
* Improve Question creation
* Fix spammy GitHub Actions comments
* [DEBUG] trigger conventional commit reminder
* fix syntax errors
* Mock update
* mock another update
* 3rd update
* test if the comment would be updated only if the actual commet changes
* Remove all code relating to testing
* Fix type not compatible with node 18
* Update new Metabase base colors
* Make hover and selected state consistency
* Fix wrong embed frame border color
* Refactor: extract props
* Fix border color
* Fix pivot table cell hover color
* Fix smart scalar text contrast
* Add smart scalar tests + update text color design
* Theme dashboard card menu
* Remove dashcard action menu tests since they don't work in loki
* Fix storybook background color for dark theme
* Update loki snapshots
* Disable untestable tests
* Make sure all storybook inherit all necessary base styles
* Skip untestable tests
* Update loki snapshots
* add setup commands
* fix settings definition
* update environment variables for cli
* handle instances not being ready
* update error messages
* add more specific loading messages
* loading spinner state
* improve error message
* use a fixed demo setup token
* remove extraneous spinner
* update status checks
* update container messages
* update wait timing
* create api keys
* extract constants
* remove manual steps
* Add anonymous tracking + other things. will need to clean up
* Modify SDK for better structure
* remove line from print.ts
* Update webpack.embedding-sdk-cli.config.js back to production
* Add types and add quick note
* Fix a typo
* Add index file, simplify types, use an array
* Add safer json parsing
* use delay of 100ms between each setup call
* Suggestions from review
* ensure that cli works
* Attempt to fix jest errors
* Remove node-fetch from sdk code to hopefully get unit tests working again
* add database connection
* add connection details handling
* refactor asking for database connection info
* apply actual database id for syncing schema
* fix failing database sync step
* allow table selection
* create model for each table
* handle errors in model creation
* fix incorrect model display name
* create x-rays based on user data
* consolidate instance setup message
* workaround for inquirer eventemitter issue
* fix yarn.lock file
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
== Goal ==
Prevent admins of Metabase Cloud instances with Metabase Cloud Storage
from accidentally breaking their upload configuration, since in this
scenario that is automatically managed by Metabase Cloud to use the
attached data warehouse.
== How to test ==
On a subscription with Metabase Cloud Storage:
* The "Uploads" section in the left sidebar of `/admin/settings` does
not show
* A "not found" page is shown at `/admin/settings/uploads` where one
would otherwise expect the "Uploads Settings" page
* `PUT /api/setting` with an `uploads-settings` field in the body fails
with `403 Forbidden` and the setting is not changed
* `PUT /api/setting/uploads-settings` fails with `403 Forbidden` and
the setting is not changed
* `GET /api/setting` contains an element with `key: "uploads-settings"`
and a value
* `GET /api/setting/uploads-settings` responds with the value
* `GET /api/session/properties` contains the `uploads-settings` field
Closes: https://github.com/metabase/harbormaster/issues/5121
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* truncate total text in pie chart
* add story
* Update Loki Snapshots
* add e2e test
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Add unit test for #46574
* Handle non-numeric values in NumberFingerprint
* Be more realistic about NumberFingerprintDisplayInfo type
* Add comment on why the types are unknown
* add setup commands
* fix settings definition
* update environment variables for cli
* handle instances not being ready
* update error messages
* add more specific loading messages
* loading spinner state
* improve error message
* use a fixed demo setup token
* remove extraneous spinner
* update status checks
* update container messages
* update wait timing
* create api keys
* extract constants
* remove manual steps
* Add anonymous tracking + other things. will need to clean up
* Modify SDK for better structure
* remove line from print.ts
* Update webpack.embedding-sdk-cli.config.js back to production
* Add types and add quick note
* Fix a typo
* Add index file, simplify types, use an array
* Add safer json parsing
* use delay of 100ms between each setup call
* Suggestions from review
* ensure that cli works
* Attempt to fix jest errors
* Remove node-fetch from sdk code to hopefully get unit tests working again
* fix yarn.lock file
* revert capture groups change
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
== Goal ==
Currently the config file (`MB_CONFIG_FILE_PATH`) only allows adding or
changing databases, but not removing them. Since connected databases
can be managed as a service by us, it becomes necessary to delete any
information Metabase has about them, including any stored credentials.
Since this is a destructive operation with no way to revert (without
restoring from a backup), we raise the bar for accidental deletion and
require not just a (boolean) flag, but a magic string to be set.
== How to test ==
1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
2. Start your Metabase instance.
3. Verify the database shows up in the "admin" section and it works.
Create a dashboard and questions using the database.
4. Add a `delete: "DELETE_WITH_DEPENDENTS:${name}"` line to your
database configuration section.
5. Restart your Metabase instance.
6. Verify the database no longer shows up in the "admin" section.
Notice that questions using it are gone, too.
References: https://github.com/metabase/harbormaster/issues/5173
* WIP Open notebook data source in new tab
* WIP
* Implement the new functionality with `pickerInfo`
* Use destructured value
* Add the simplest possible E2E test as PoC
* Smoke test
* Add basic coverage for questions
* Add basic coverage for models
* Source question has been archived scenario
* Add basic permissions test
* Expand to the user without wite query permissions
* Fix title
* Add another permission scenario
* Add E2E coverage for a source being native
* Add coverage for the native model
* Address review comment re: `pickerInfo`
* Extract `getUrl` to utils
* Add support for the middle/scroll click
* Add unit tests
* Add a negative test case
* Fix title
Fixes#46440
couple things come together here.
Bad escaping:
The `{` characters are used for format strings and placeholders like
```clojure
impl=> (translate "fr" "After {0}" ["Bob"])
"Après Bob"
```
And if you want _literal_ {'s in your translation string, you must
escape them like "'{" or "'}" using an apostrophe. But this isn't the
end of our worries.
in `resources/zh-CN.edn`, we actually have _two_ translations for the
original string:
```clojure
"User lookup filter. The placeholder '{login'} will be replaced by the user supplied login."
"用户找过滤器，占位符 '{login'} 将替换为用户提供的登录名。"
,,,
"User lookup filter. The placeholder '{login}' will be replaced by the user supplied login."
"用户查找筛选器。占位符“{login}”将替换为用户提供的登录名。"
```
And this later has unescaped curlies.
The fix for that is at
https://poeditor.com/projects/po_edit?per_page=20&id=200535&id_language=31&filter=&order=&search=User+lookup+filter
but will need to wait for the new source translation string in this
instance with the corrected escaping.
```clojure
;; simulate what happens at runtime
impl=> (translated-format-string "zh_CN"
"User lookup filter. The placeholder '{login}' will be replaced by the user supplied login."
{})
"用户查找筛选器。占位符“{login}”将替换为用户提供的登录名。"
impl=> (MessageFormat. *1 (locale "zh_CN"))
Execution error (NumberFormatException) at java.lang.NumberFormatException/forInputString (NumberFormatException.java:67).
For input string: "login"
;; the problematic zh-CN string
impl=> (MessageFormat. "用户查找筛选器。占位符“{login}”将替换为用户提供的登录名。")
Execution error (NumberFormatException) at java.lang.NumberFormatException/forInputString (NumberFormatException.java:67).
For input string: "login"
;; minimal reproduction
impl=> (MessageFormat. "“{login}”")
Execution error (NumberFormatException) at java.lang.NumberFormatException/forInputString (NumberFormatException.java:67).
For input string: "login"
```
* fix: mongo ObjectId should be orderable
Fixes#46259
Mongo ObjectId derive :type/TextLike but that can't be an orderable
type, so we mark the specific Mongo type as orderable for malli schemas.
* Fix test
* prevents users from moving items to the trash if they are in the trash already and archives instead of moves in cases where item is moved to or from the trash
* reverts detecting if dragging in or out of the trash not setting the collection id, this will be prevented on the BE instead
* adds test coverage for dragging and dropping items to/from/within the trash (one of which is failing until we can make a BE change)
* Throw away `collection_id` when marking archived
This is a hack around a frontend issue. Apparently, the undo
functionality depends on calculating a diff between the current state
and the previous state. Sometimes this results in the frontend telling
us to *both* mark an item as archived *and* "move" it to the Trash.
Let's just say that if you're marking something as archived, we throw
away any `collection_id` you passed in along with it.
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
### Description
This change should fix some explosions in stats due to phantom fields, while also making things a bit more rigorous.
- Record the table schema in query_field - especially useful for unknown tables
- Allow null table names in query_field - they might be phantoms, or they might not
- We still have the option to filter them out in tools we build on top
- Tighten up matching logic to consider schema
* Update popover dark theme on part of ParameterFieldWidget (#45813)
* Add filter visual tests
* Simplify embed CSS colors
* [WIP] theme filter popover
* Move theme to global node, so it affects portal too
* Fix embed colors
* Theme part of ParameterFieldWidget
* Add story for parameter with dropdown
* Update loki snapshots
* Update more popover style
* Update loki snapshots
* Remove outdated snapshots
* Update loki snapshots with production build
* Fix loki not working with `userEvent`
* Theme mantine popover
* Improve dark theme text-brand visibility
* Update loki snapshots from master change
* Fix new colors not working in SDK
* Fix flaky Loki tests
* Filter popover dark theme part 2 (#46284)
* Theme Date all options
* wip add month and year filter
* Fix wrong CSS custom property declaration
* Test hover state
* Update Loki snapshots
* Theme date month and year
* Add quarter and year tests
* Theme date single and date range
* Style relative date
* Update frontend/src/metabase/admin/datamodel/components/filters/pickers/DatePicker/DatePickerHeader.styled.tsx
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Complete theming filter popover night mode (#46365)
* Style unit of time and number widget
* Attempt to fix flaky Loki snapshots
* Address white background checkbox on dark theme
* Style search filter single value
* Fix mis color
* Update snapshots
* Remove duplication on play functions since they're long
* Milestone 2 polishing (#46382)
* Matches the footer padding with header padding
* Update snapshots
* Add E2E to prevent further problem
* Try to reduce flaky Loki tests
* Remove unused color shades (#46421)
* Update no results illustration to work better on dark theme
* Update loki snapshots
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Refactor dashboard helpers to TypeScript
* Reuse existing helper
* Refactor cy.createQuestionAndDashboard to a function helper
- there was no good place to put it, so I also created new helpers/api directory
- and I moved all helpers using cy.request (with 1 exception, where a CSV download helper also does a bunch of assertions) in there
* Merge TS command definitions into a single file
* Export types
* Make dashboardDetails optional
* Export StructuredQuestionDetails
* Remove duplicated function
* Extract createNativeQuestion to separate file
- 1 exported helper per file, no exceptions!
* Reuse const
* Add test for unit of time parameters
* Improve assertions
* Extract createDashboardWithParameters
* Add tests for time parameters
* Fix hardcoded parameter_mappings ids
* Format code
* Extract checkButtonVisible
* Improve test names
* Add tests for location parameters
* Reuse const
* Update test name
* Add WidgetStatusButton
* Add unit tests for WidgetStatusButton
* Rename handler
* Remove redundant assertions
* Add iconSize prop
* Fix styling
* Replace WidgetStatusIcon with WidgetStatusButton
- Remove WidgetStatusIcon
* Fix tooltip issue
* Fix tooltip for good
* Use default compact button size
* Prevent overflow
* Fix alignment issues in parameter sidebar
- Remove iconSize prop
- Simplify StatusConfig
* Update padding in Unit of Time input to make it consistent with other inputs in dashboard parameter sidebar
* Update unit tests
* Rename WidgetStatusButton to WidgetStatus
* Explicitly specify what is a button
* Update helpers in tests
* Update when buttons are displayed according to the new spec, update tests
* Simplify label
* Remove utils
* Adjust padding
* Update reset icon
* Make the button gray when color is undesired
* Remove old definitions
* Add missing export
* Remove unused type
* Add test for location (single value)
* Add test for id (single value)
* Add more assertions
* Add test for id (multiple values)
* Improve name
* Add tests for number (single and multi value)
* Add tests for text (single and multi value)
* Rename file
* Update comment
* Update test name
* Use unique ids
* Use new color name
* Add a test for chevron icons alignment
* Lift className up
* Make types more specific
* Sort attributes
* Revert "Make types more specific"
This reverts commit 2e6994b5ed.
* Revert "Use new color name"
This reverts commit 2cc68df6ba.
* Remove TODO
* Introduce editFilter util, refactor chevron assertions to work with any number of chevrons
* Connect label to the default value input in ParameterSettings
* Add a whitespace in the default value label when required "(required)" is shown
* Add parameter sidebar tests for unit of time
* Add parameter sidebar tests for date filters
* Extract checkParameterSidebarDefaultValue
* Add parameter sidebar tests for location - single values
* Add parameter sidebar tests for location - multiple values
* Use constants for test case names
* Add parameter sidebar tests for id parameters - single values
* Add parameter sidebar tests for id parameters - multi values
* Add parameter sidebar tests for number parameters - single values
* Add parameter sidebar tests for number parameters - multi values
* Add parameter sidebar tests for text parameters - single values
* Add parameter sidebar tests for text parameters - multi values
* Extract checkDashboardFilters
* Use checkDashboardFilters in time parameters
* Use checkParameterSidebarDefaultValue in checkDashboardFilters
* Use checkDashboardFilters everywhere
* Sort helpers
* Rename checkDashboardFilters to checkDashboardParameters
* Rename checkOnlyOneButtonVisible to checkStatusIcon
* Add test suite for sql filters, coverage for text filter - single value
* Add a whitespace in the default value label when required "(required)" is shown - SQL filters
- Similar to 930dd3a50a
* Add test suite for sql filters, coverage for number filters
* Add test for sql filters - date
* Remove unused things, rename placeholder to labelOrPlaceholder
* Apply label only to 1 input
* Add test for sql filters - field parameters
* Fix date tests
* Make all sql filters tests work
* Add assertions for required without default in text filters
* Add assertions for required without default in date filters
* Add assertions for required without default in field filters
* Allow clearing required SQL filters without default value
* Add a test to ensure comma-separated numeric values work
* Update unit test
* Update helper
* Update e2e tests
* Use != null and == null checks with defaultValue instead of just checking if its truthy/falsy
* Run `yarn test-visual:loki` and then `yarn loki approve`
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Add button boilerplate
* Add getCanResetFilters selector
* Move canResetFilters to utils
* Add explicit return types
* Implement the button action
* Rename
* Simplify action
* Optimize: use single action
* Use parameterHasNoDisplayValue to account for empty strings
* Run `yarn test-visual:loki` and then `yarn loki approve` after merging master
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Rename getHiddenParameterSlugs to getTabHiddenParameterSlugs
* Take all tabs into account
* Add unit tests for canResetFilter
* Format test name
* Fix non-human-friendly aria-label
* Add assertions for "Reset all filters" button's visibility
* Unnest helpers from describe
* Add a test for comma-separated values
* Improve typing
* Fix typing
* Add tests for multiple tabs and auto-apply-filters on/off
* Improve test name
* Remove redundant util
* Remove redundant condition
* Verify card results get filtered
* Test that all tabs are affected
* Fix test case with autoApplyFilters: false
* Add tests for resetting to default value
* Fix wrong initial state in reduce
* Fix updated parameters not being applied
* Add reset all filters assertions
* Remove unused attribute
* Run `yarn test-visual:loki` and then `yarn loki approve`
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Run `yarn test-visual:loki` and then `yarn loki approve`
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Refactor dashboard helpers to TypeScript
* Reuse existing helper
* Refactor cy.createQuestionAndDashboard to a function helper
- there was no good place to put it, so I also created new helpers/api directory
- and I moved all helpers using cy.request (with 1 exception, where a CSV download helper also does a bunch of assertions) in there
* Merge TS command definitions into a single file
* Export types
* Make dashboardDetails optional
* Export StructuredQuestionDetails
* Remove duplicated function
* Extract createNativeQuestion to separate file
- 1 exported helper per file, no exceptions!
* Reuse const
* Add test for unit of time parameters
* Improve assertions
* Extract createDashboardWithParameters
* Add tests for time parameters
* Fix hardcoded parameter_mappings ids
* Format code
* Extract checkButtonVisible
* Improve test names
* Add tests for location parameters
* Reuse const
* Update test name
* Add WidgetStatusButton
* Add unit tests for WidgetStatusButton
* Rename handler
* Remove redundant assertions
* Add iconSize prop
* Fix styling
* Replace WidgetStatusIcon with WidgetStatusButton
- Remove WidgetStatusIcon
* Fix tooltip issue
* Fix tooltip for good
* Use default compact button size
* Prevent overflow
* Fix alignment issues in parameter sidebar
- Remove iconSize prop
- Simplify StatusConfig
* Update padding in Unit of Time input to make it consistent with other inputs in dashboard parameter sidebar
* Update unit tests
* Rename WidgetStatusButton to WidgetStatus
* Explicitly specify what is a button
* Update helpers in tests
* Update when buttons are displayed according to the new spec, update tests
* Simplify label
* Remove utils
* Adjust padding
* Update reset icon
* Make the button gray when color is undesired
* Remove old definitions
* Add missing export
* Remove unused type
* Add test for location (single value)
* Add test for id (single value)
* Add more assertions
* Add test for id (multiple values)
* Improve name
* Add tests for number (single and multi value)
* Add tests for text (single and multi value)
* Rename file
* Update comment
* Update test name
* Use unique ids
* Use new color name
* Add a test for chevron icons alignment
* Lift className up
* Make types more specific
* Sort attributes
* Revert "Make types more specific"
This reverts commit 2e6994b5ed.
* Revert "Use new color name"
This reverts commit 2cc68df6ba.
* Remove TODO
* Introduce editFilter util, refactor chevron assertions to work with any number of chevrons
* Connect label to the default value input in ParameterSettings
* Add a whitespace in the default value label when required "(required)" is shown
* Add parameter sidebar tests for unit of time
* Add parameter sidebar tests for date filters
* Extract checkParameterSidebarDefaultValue
* Add parameter sidebar tests for location - single values
* Add parameter sidebar tests for location - multiple values
* Use constants for test case names
* Add parameter sidebar tests for id parameters - single values
* Add parameter sidebar tests for id parameters - multi values
* Add parameter sidebar tests for number parameters - single values
* Add parameter sidebar tests for number parameters - multi values
* Add parameter sidebar tests for text parameters - single values
* Add parameter sidebar tests for text parameters - multi values
* Extract checkDashboardFilters
* Use checkDashboardFilters in time parameters
* Use checkParameterSidebarDefaultValue in checkDashboardFilters
* Use checkDashboardFilters everywhere
* Sort helpers
* Rename checkDashboardFilters to checkDashboardParameters
* Rename checkOnlyOneButtonVisible to checkStatusIcon
* Add test suite for sql filters, coverage for text filter - single value
* Add a whitespace in the default value label when required "(required)" is shown - SQL filters
- Similar to 930dd3a50a
* Add test suite for sql filters, coverage for number filters
* Add test for sql filters - date
* Remove unused things, rename placeholder to labelOrPlaceholder
* Apply label only to 1 input
* Add test for sql filters - field parameters
* Fix date tests
* Make all sql filters tests work
* Add assertions for required without default in text filters
* Add assertions for required without default in date filters
* Add assertions for required without default in field filters
* Allow clearing required SQL filters without default value
* Add a test to ensure comma-separated numeric values work
* Update unit test
* Update helper
* Update e2e tests
* Use != null and == null checks with defaultValue instead of just checking if its truthy/falsy
* Run `yarn test-visual:loki` and then `yarn loki approve`
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Use parameterHasNoDisplayValue to account for empty strings
* Run `yarn test-visual:loki` and then `yarn loki approve` after merging master
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Run `yarn test-visual:loki` and then `yarn loki approve`
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Run `yarn test-visual:loki` and then `yarn loki approve`
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Make number also have custom parameter values
* Render the correct label in the dropdown for number inputs
* Mock paramter values endpoint in number input test
* Add test for custom labels on number input
* Fix number input test
* Allow entering comma separated values using labels
* Only show custom labels in popover
* Use getLabel and getValue helpers to avoid indexing confusion
* Remove calls to parameter endpoints for number input
* Do not parse the value from the label
* Add support for number parameters in the source picker modal
* Filter number values
* Add e2e test for number parameters with custom source
* Remove card option for number parameters
* Set the correct initial value for number parameters without fields
* Do not render number parameter as search box when it is a list
* Fix e2e tests for number lists
* Remove e2e tests for card source dashboard filters
* Add tests for native filter custom source
* Show the dropdown when picking the default value for number parameters
* Clear values_source_config when changing parameter type
* Restore a parameters' values_source_config when changing the parameter type back
* Clear and restore values_source_config for template tags when chaning type
* Add e2e test for restoring of template tag sources_value_config
* Add e2e test for clear and restore of static-list for dashboard parameters
* Fix input type for numbers
* Fix broken test
* Display labels for ParameterValuePicker too
* Clear and restore values_source_type when changing parameter type
* Clear and restore values_source_type for tag parameters
* Clear and restore values_query_type when changing parameter type
* Clear and restore value_query_type for parameters tags
* Update tests to test for reset to also apply to values_source_config and values_query_type
* wip
* moves code to enterprise folder, adds enterprise guards, updates conditionals as to when to show the feature
* impls filter date logic, improves styling, mocks out real endpoint with collection items endpoint, updates table with isLink boolean rath
er than using custom item renderer, removes dead code
* use the actual url for stale
* so many things... updated design, custom table component, polish, misc. refactors, more correct api typing
* updates to pagination component and reverts some now unused changes to the base items table
* impls pr feedback
* fixes
* fixes failing unit test
* completes pr feedback from raffi
* implements second round of feedback from raffi
* adds tests and couple of small fixes / feedback
* migrate clean up action to enterprise plugin and fix related e2e test, also skip the collection items count query when we do not need the value
* finish main base case e2e test
* clean up helper code for manual testing and add OSS test to make sure feature is not available in OSS
* better handling of min-height on smaller screens
* adds internal undo notification for bulk archiving in the clean up modal
* reset pagination when items are archived/unarchived to keep users from getting stuck on a page beyond the current view
* pr feedback
* further cleanup
* unit test fix
* fixes issue where archiving all items in the modal and then undoing does not result in the stale items list getting refetched
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* hide notebook button for brand new questions
* test notebook button visibility for new questions
* move brand new check inside shouldRender
* test embedding SDK notebook button
* update unit tests
* add spy to shouldRender
* Fix coll permissions for audit collection
We've had a function in `models.collection` for a while that has taken a
permissions set and returned a set of collection IDs that the user has
permissions on. I refactored this recently, but didn't notice that it
was actually doing the permissions checks slightly incorrectly.
Specifically, because it only looks at the user's permissions and the
collection IDs and doesn't use `mi/can-read?` or `mi/can-write?`, it's
completely indifferent to whether a collection is an audit collection or
not.
This is arguably not a *permissions* issue: the two errors that could
come about here are:
- someone sees the audit collection even though the audit feature is
disabled, or
- someone is presented with the audit collection in a context where only
writable collections should be present - when they try to actually write
to it, it fails (since then we're doing the real permissions check).
The primary motivation for this fix was to prevent audit dashboards and
cards from appearing in the list of stale items.
* Fix hardcoded collection ID
We were creating a timeline in a fixed collection ID, that in tests
happened to be the ID of the Metabase Analytics collection.
* properly handle SMTP environment variables
* disable clear button when set by env var
* allow the SMTP form to validate with any combination of environment variables
* add unit tests for SMTP connection form
* BONUS: fix console error for jt keys
* fix types
* updates the ordering of create queries permissions options so that granular permissions is the last option
* fix unit test
* move granular option above no
* update static_embed_code_copied to use downloads (#45944)
* copy schema file for better diff
* adds downloads field to appearance in embed_flow schema
* update event to send `downloads` parameter, update e2e tests
* fix test for dwonloads: false
* fix: use correct schema version for embed flow analytics (#46053)
* update `dashboard_pdf_exported` and use it in embedding context (#46076)
* copy schema for better diff
* adds `dashboard_accessed_via` to the `dashboard_pdf_exported` event
* download_results_clicked event (#46132)
* adds download_results_clicked event + e2e tests
* make strings nullable
* adds maxLength
* switch back to if/then as it seems more readable
* refactor: use the resourceType to determine the download params instead of re-doing the logic
* fix ''question' is possibly 'undefined'.'
* split resource_type into resource_type and accessed_via to also consider interactive-embedding and sdk
* edit comment
* adds test for ad-hoc-question
* Update frontend/src/metabase/query_builder/actions/downloading.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* makes describe message more consistent
* "else if" => if
* fix describe block name
* convert downloads event to use enum for download_type and export_type
* resource_type should be an enum too
* fix line added in the wrong place during merges
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Update frontend/src/metabase/public/components/EmbedFrame/EmbedFrame.tsx
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* adds enum schema to dashboard_accessed_via
* fix accessed_via for sdk embeds
* update comments to explain why we send 0 when dashboardId is null
* only send dashboard_id as a number when it is a number, as expected by snowplow
* remove un-needed async/await from saveAsPDF function
* rename frontend/src/metabase/redux/downloading-analytics.ts -> frontend/src/metabase/redux/downloads-analytics.ts
* fix cy.log message
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* update embed frame pdf button colors to use --mb-color- vars
* change download pdf button color to brand color
* testing adding the images that actually differ
Previously we unpacked everything and picked up first entry as the directory
containing the data. This works well when archives are passed as is. However,
if a user unpacks and then packs an archive, there is a good chance that some
hidden directories will appear - and they are first in order.
So we skip those right now, and additionally check known markers of the
serialization data directory.
* Squash
* Clean up query modifier for validating card references (#46283)
* Fixup bad git merge on test expectation
* Tweak docstrings and de-densify logic
* Allow filtering on collection
* Add `effective_ancestors` to collection data
Note: I removed the `qv=` comparator. I think it'.s better to be more explicit
about what exactly we're asserting. In particular, I hadn't updated the
`:count` when we were filtering on collection type, so the count was
still the total for *every* collection even though the results were being
restricted by the collection. However the `qv=-data` function was still
reporting that everything was OK, because it was accounting for other
data from the App DB by just asserting that the count we'd specified was
lower than the actual count. I'd argue it's better for tests to fail
locally because I have garbage in my app DB (which I can clean up) than
succeed when they really shouldn't.
* Consolidate effective ancestors
For the Search API, we're currently returning slightly different
ancestors:
- it never has the Root Collection, and
- the ancestors don't have a `personal_owner_id`
We could change the API (and maybe will soon - it seems pretty
unexpected to have `effective_ancestors` sometimes act one way and
sometimes act another) but for now we can just reshape things to look
the same as they did before, while using the same code under the hood.
* echarts pie
* setup rendering pipeline for static pie chart (#43547)
* setup rendering pipeline for static pie chart
* remove more donut code from BE
* compute viz settings for static pie chart (#43548)
* compute viz settings for static pie chart
* create chartModel for pie chart (#43549)
* create chartModel for pie chart
* render static pie chart (#43550)
* render static pie chart
* render static pie chart legend (#43551)
* disable animation
* alter static-viz pie chart be tests to account for echarts-impl
The old implementation had a different implementation for legends, separate from the 'categorical/donut' js
render. Echarts nicely combines the legend into the pie render, so we don't need to worry about that stuff anymore.
Instead, I've written a simple set of assertions on the echarts pie implementation now.
* Get rid of this test that was attempting to use old code path
* echarts pie
* One last spot where we don't need old pie stuff anymore.
* add loki specs
* save snapshots
* fix small slices disappearing
* update snapshots
* add repro story for exteremely small slice
* add snapshot
* fix rounded total
* fix percent formatting in legend
* fix date dimension formatting
* fix binned and relative date dimension labels on legend
* add specs and snapshots for null and unaggregated dimensions
* hide overlaping labels on chart
* fix null dimension color
* show error when metric column has a negative value
* move legend to botom
* make dot margin smaller
* add columns to legend
* sort legend vertically
* dont use grid if only one row for legend
* never hide legend in static viz
* fix empty column culling
* show single legend item
* add long dimension name story
* fix long dimension name in legend
* right justify percentages in legend
* remove negative error and re-add stories
* handle negative values
* create chart definition for dynamic pie chart
* render chart with responsiveness
* add comment for borderWidth calculation
* center legend
* fix legend centering
* add story for missing currency formatting
* fix missing currency formatting
* fix graalvm crashing
* fix column settings computation
* add another currency formatting repro
* fix wrong order of col settings computations
* convert colors to hex
* fix colors missing from viz settings
* temp todo comment
* fix colors in dynamic viz
* update snapshots
* fix crashing due to invalid dimension/metric setting
* fix chart crashing for boolean dimension with labels on chart
* add repro
* fix regression in label formatting
* fix labels on small slices
* update specs
* emphasis state for slice hover
* recreate old label hiding logic
* include height measurement in getIsLabelVisible
* move d3 slices to chart model
* fix labels on static viz
* wip basic hover state and tooltip
* fix emphasis style getting stuck by memoizing everything
* update total display value
* fix stuck emphasis by setting notMerge to false for pie only
* handle other slice in tooltip
* fix bugs in total graphic
* use OTHER_SLICE_KEY const in static viz legend
* implement legend and connect to hover state
* implement click actions
* fix slice not clickable
* handle all zero slices
* add repro
* update specs
* update pie_chart.cy.spec.js
* update chart_drill.cy.spec.js
* add repro for single column legend
* use minimum 2 legend columns
* fix legend text overlap
* use more margin for single row legend
* update specs
* add repro for 28568
* add repro for 38424
* fix percent label not showing on large slices
* hide total text when chart is small
* replace comment about animations
* show warning for mixed positive and negative values
* delete old chart files
* fix type errors
* update d3 import
* update specs
* remove branch from uberjar.yml
* dont put single slice into other
* fix lint errors and pie render test
* update more d3 imports
* remove unused settings
* revert preview-card-width change
* remove unused import
* remove majorWidth option from formatting
* use debounceLeading refresh mode in ChartWithLegend
* remove TOTAL_GRAPHIC_OPTION constant
* delete unit test for pie chart
* fix failing test by doing set comparison instead
* fix typo in comment
* remove SUNBURST_SERIES_OPTION constant
* fix lint by removing unused eslint-disable
* move default color computation to getValue
* fix static pie test to show that we will always render the legend
* update pie border color for night mode and embedding
* fix styles for embedding case and add stories
* skip embedding stories
---------
Co-authored-by: Adam James <adam.vermeer2@gmail.com>
* Make an internal API for retrieving stale content
This introduces the `metabase.stale` module, which has one public
function (so far): `metabase.stale/find-candidates`. This takes a set of
`collection-ids`, a cutoff date, and pagination/sort info.
Next, we introduce an HTTP API endpoint that will figure out the appropriate
set of `collection-ids` for a given user/request, call this function,
and return the results in the shape we want.
Also: for e2e tests, we need a way to mark a card or dashboard as stale. I
think the easiest way to do this is to just create a testing endpoint
that we can call to update the `last_used_at` or `last_viewed_at` of the
card/dashboard to set it to ~7 months ago.
---------
Co-authored-by: bryan <bryan.maass@gmail.com>
* use mantine buttons for filter and summarize
* Restore fancy color hover states
* Add css modules
* oh the cleverness of me
* better var name
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
Expands and refactors `metabase.util.memoize` to support better cross-platform memoization.
**Better basic memoization**
Use `clojure.core.memoize/memo` (and its clone in CLJS) rather than `clojure.core/memoize` for
the "everything forever" strategy.
**Bounded memoization**
There are a few places (eg. `u/kebab-case-en`) where we want zero-overhead *hits* and expect the input space
to be fixed. To guard against runaway memory usage, this **bounded** strategy dumps the entire cache when it
overflows. It logs an INFO level note when that happens - the intent here is that the threshold should never
actually get hit!
**Fast JVM versions**
In the special case where the function to memoize takes exactly 1 argument which is a valid map key, we can
use `ConcurrentHashMap.computeIfAbsent` as the basis of the cache and have much less overhead.
`fast-memo` and `fast-bounded` implement this in CLJ; in CLJS they just call `memo` and `bounded`.
* Use function helpers
* Only assert the resize down
* Refactor repro 31701
* Remove flaky tag
* Better organize describe blocks
* Move custom `requestTimeout` config to the common block for resizing tests
* Refactor dashboard helpers to TypeScript
* Reuse existing helper
* Refactor cy.createQuestionAndDashboard to a function helper
- there was no good place to put it, so I also created new helpers/api directory
- and I moved all helpers using cy.request (with 1 exception, where a CSV download helper also does a bunch of assertions) in there
* Merge TS command definitions into a single file
* Export types
* Make dashboardDetails optional
* Export StructuredQuestionDetails
* Remove duplicated function
* Extract createNativeQuestion to separate file
- 1 exported helper per file, no exceptions!
* Reuse const
* Remove old definitions
* Add missing export
The `dashboard_load_id` is attached to the body for queries, not as a
query parameter. This correctly receives the `dashboard_load_id` and
uses it for `/query` calls.
* pull sdk info into data on-thread
* include api tests for query_execution and view_log
* Try mt/test-helpers-set-global-values!
- disambiguate names used in tests
* pass test + cleanup test helper
* make sure view logging is enabled in test
* fix conditional formatting color lookup with correct col names
The column names for color lookup are expected to be the same as the :name key of each column, and applying column
formatting broke this lookup. This fixes it.
* move overflow-x into pulse body div to exlude card title from scroll
* add a test for conditional formatting w/ a hidden row.
This still fails at the moment, working on a fix yet.
* apply conditional formatting in static viz even when col is hidden
Conditional formatting that depends on a hidden column was not being rendered in the static viz, but now we pass all
columns into the color selector fn, thus when rendering the table with the ordered/hidden columns, all other columns
will still use their appropriate conditional formatting
* I missed a linter warning
* fix tab rendering
* fix test
* fix up tests with some better structural searching
* Adds a middleware that sets 2 dynamic vars
- These vars will be used to insert context for embedding analytics
* migration adding embedding_client and embedding_version to qe and vl tables
* clarify function api
* headers come in -> dynamic vars -> written to db
When the vars are set we insert their values into query execution and
query_log under embedding_client and embedding_version.
* update update migrations
* add predconditions to migrations
* fix tablenames + dataset_test
* Code Review responses
- remove commented out requires
- clarify test for merge-count-maps
* Lock down the trash collection more
Oops. I was very thorough in checking that you couldn't modify the
trash collection in the sense of moving things to or from it. I missed
the most obvious thing: actually modifying the trash collection itself.
So if someone wanted, they could change the name of the Trash to
"Definitely Not the Trash".
* Remove the Trash Collection from serialization
Serialization tests were failing due to the previous change, but this
got me thinking, and I realized that serializing the Trash collection is
completely unnecessary. We want to create it programatically on
startup (which we are) and then nothing should change it afterwards.
I tightened this up by removing the condition here, thinking that CI
would catch it if it was necessary - but I'm getting failures in
development. Oops - didn't realize that this defmethod wasn't actually
applied during CI runs.
* Rewrite repro for 27643
* Add TODO for the remaining tasks
* Make a test name consistent with others
* Finish the E2E test for #27643
* Fix the test to use PostgresDB
Since H2 originally doesn't fail the test
* Attempt to fix a flake on CI by waiting for table to finish syncing
* Create QA snapshot with fully synced field metadata
* Fix failed tests from field metadata change
* Simplify question scenarios
* Simplify dashboard scenarios
* Fix permissions
* Rework titles
* Add comment about PG
* Add limit to speed things up
* Fix and improve dashboard assertions
* Fix and improve question assertions
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Bigquery Pagination Use next page to get proper results
Fixes 45953
WIP
* use new-page in values-iterator
* disable just the part of the test that is still flaky
Want to keep the other part of the test that does prove we're not getting duplicates due to incorrect pagination.
* Number formatter Reports correct Decimal places for negative Numbers
Fixes#41640
Prior to this, the number formatter would get the incorrect number of decimal places for negative numbers.
The way the decimal places count is derived works as follows:
- convert number to a string
- split the string on the separator (decimal, comma, whatever the instance has set)
- destructure that split into [int decimal]
- count decimal to get the number of decimals.
That all works for positive numbers, but for negative numbers, the regex was just splitting on 'not digits', so we got
an extra group due to the leading negative sign. This would incorrectly destructure the 'int' part of the number into
the count, thus we would always get 1 instead of the correct value.
This fixes it by just taking last instead of destructuring.
* correct comparison to always use abs value in conditonal check
* tiny incorrectness in the test
* Modify the E2E stress test workflow to allow running PostgresDB
* Fix syntax
* Add qa_db to workflow summary
* Only run necessary containers
* Fix QA DB snapshots running when QA_DB_ENABLED=false
* Use more generic option that's not only specific to postgres
* perf: Add explicit .entryAt method for SnakeHatingMap
Otherwise, SHM invokes .keySet on itself which triggers the contstruction of the list of keys on each map access.
* perf: Cache (un)successful requiring of EE in dynamic-ee-oss-fn
* perf: Add a quicker path for getting metadatas from cache
* perf: Minor improvements
It's based on a (global, synchronized) `WeakHashMap` using the query map
itself as the key. That gives the cache the same lifespan as the query
map, which is generally brief but often includes many calls to
expensive, cached functions like `visible-columns`.
* Make filter corrupted again™
* Speed up the test by limiting the number of rows
* Make the repro more correct, and more resilient
* Remove intercepts
* Improve assertions
* add webpack configuration for cli
* add requirements checking
* docker check
* docker start command
* port checks
* add getting started steps
* add final install message
* cleanup cli text
* handle force-closing prompt
* build the cli when running build-embedding-sdk
* wording updates
* add redundant prompt in case of small terminals
* remove redundant instruction
* make sure the user confirms the step
* change container name to metabase-enterprise-embedding
* migrate cli to es modules instead of webpack
* add shebang line for npx
* add cli dependencies to devDependencies
* Revert "add cli dependencies to devDependencies"
This reverts commit 5d5e952c2b.
* Revert "add shebang line for npx"
This reverts commit 766fdda09e.
* Revert "migrate cli to es modules instead of webpack"
This reverts commit 7d1b065163.
* Embedding Parameter Values set in Tokens work properly when `nil` or empty list
Fixes: #40292
If a locked parameter's value is set to `nil` or `[]` in the embedding JWT, we want to treat this as 'no value has
been set' on that filter, but we should still run the query as normal. Likewise, any linked filters should still
properly derive their values from the unfiltered values of the filter to which they are linked.
* when a parameter's value is nil or empty seq, don't include it
This is a partial solution as I still need to find the place that's still erroring on the empty list case in terms of
the linked parameter's values.
* Treat empty list like `nil` for the token param
When you're using a filter in the App, it makes intuitive sense that when you've not selected anything from a filter's
drop down list (or entered any text, etc.), then the filter is effectively not applied.
For token params, we should behave similarly; `nil` and `[]` can both be accepted, and signal that the user intends
for that filter to basically NOT apply.
This also means we should not pass the constraint to the linked filters, since it errors, but even if it worked, it
would return 0 results, making it impossible to properly apply the linked filter at all.
So, this change causes empty lists to be treated as nil, AND will remove any parameters with `nil` as a value before
being passed to the query processor, which results in the proper behaviour of not having constraints on the linked
filter, and having the 'parent' filter not filtering anything.
* Embed preview should always use params from the token
In making a screenshot, I discovered a slight bug in previews, where if the embed had been previously published, it
could pull embedding params from the db that don't match the settings on screen. So, always pull the settings from the
token, which updates as the user is making changes in the Preview Embed UI.
* for previews, merge the embedding params sources instead
* alter 'nil' behaviour to work just as if the key doesn't exist
* fix one spot where I was passing maps into set functions
* Adjust tests to check for consistent behaviour in dash/qstn embeds
* fix an e2e test
* fix up last 2 failing e2e tests
* add the test case of a locked param having no value set in token
* some further e2e changes
We have N+1 detection for simple hydration methods that checks to see
whether the hydration method made any DB queries at all. This makes
sense, because a simple hydration method is called on each instance
individually - if we make database calls *each time*, we're in trouble.
Where this runs into issues is with cached data, where the simple
hydration method is populating the cache - e.g. `:can_read` or
`:can_write` dereferencing `api/*current-user-permissions-set*`.
Currently we specifically exclude those hydration methods from the
check.
But after running into this again for a different simple hydration
method, I think there's a more robust approach:
- first, run the hydration method without counting DB queries
- then, run it again and make sure we don't make any additional DB
queries.
There is a potential downside here: if someone wrote a simple hydration
method with side effects, they'd potentially be very confused (in
development - this doesn't run in prod!) when those side effects
happened twice for a single hydration. But I *think* that's fairly
unlikely.
* handle errors when useMetabaseAuthStatus is used outside of sdk context
* address review comments feedback
* extract useSdkContext
* fix import type error
* polyfill useSyncExternalStore for React 17
* adds the shim
* use ReactDOM.render instead of createRoot for backwards compat
* use createRoot for React 18 environment
* refactor renderRoot compatibility
* remove semver dependency
* add small test for version
* use the same check for react 17
- babel is kept to transpile fonts file as swc emotion plugin doesn't handle our case correctly. I didn't find a root cause of wrong behaviour, but the safe past is to keep using babel + emotion plugin for one file and all others can be transpiled with swc
- react fast-refresh now handled by swc
- swc-loader can't read browserslist config from package.json, so we had to manually specify supported browsers in webpack
* Uninstall ReplayIO
* Remove Replay config
* Install and use custom Chrome v111
* Simplify comment
* Set a specific Chrome version commit
* Skip repro for #22517
* fix: replacing joins and conditions updates alias
Fixes#37074
* Only change alias if conditions or table changed
* Fix test
* Fix fe test
* Address PR feedback
* Remove can_run_adhoc_query from the FE
* Remove can_run_adhoc_query from the FE
* Disable can_run_adhoc_query on dashboard
Was impacting performance on dashboard load. Timings locally for me:
get dash: 3178 -> 199
get metadata: 3604 -> 1002
So dashboard went from 6 seconds to load to 1 second.
* Fixup test from disabling can_run_adhoc_query
---------
Co-authored-by: dan sutton <dan@dpsutton.com>
* adds database_id to recents, and question collection items
* don't show unjoinable things in recents either
* unskip repro
* fix types
* fix repro
* fix tests in model/recent_views
* add unit test for invalid join in recents
* fix tests in test/metabase/api/collection_test.clj
* update type
* improve unit tests
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Add `report_dashboard.last_viewed_at`
- defaults to `now()` (it's intended to be used as a timestamp after which
we're *sure* the item hasn't been used, not a definitive timestamp we're
sure the dashboard was actually used at)
- updated whenever the QP processes a query for a dashcard
---------
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* remove dynamic import to make html2canvas work on the sdk
* Revert "remove dynamic import to make html2canvas work on the sdk"
This reverts commit a8b511fa28.
* manually import html2canvas-pro and jspdf in the sdk
Previously, due to over-eager `.flush()` calls in Cheshire, we were
flushing the output stream for every `data.rows` entry. Typically this
resulted in 3 + N TCP packets sent, where N is the number of rows being
returned.
This PR rebuilds how we stream the JSON responses to avoid extra
`.flush()` calls. For the unit test in this PR, that reduced the number
of packets sent over the wire from 37524 to 4.
Fixes#34795.
* Post sync hook stub for implicit joins testing stub
* Add join alias to field lvalues to enable sorting on joined fields
* Disable :foreign-keys on Mongo
* Require :left-join support for implicit joins instead of :foreign-keys
* Update implicit joins tests
* Adjust sync-fields-test
* Update implicit joins feature check test
* Transform post-sync-hook to normal function
* Add foreign key relationships only for dbmses without :foreign-keys feature
* Update test to handle Oracle correctly
* Split convoluted fn
* Avoid unnecessary computations for datasets with no fks
* Update docstring
* Fix driver usage
* Transform :foreign-keys to :metadata/key-constraints in test data loading code
* Update sync_test.clj
* Update driver_test.clj
* Update moviedb.clj
* Update dataset_definition_test.clj
* Update fetch_metadata.clj
* Update fields_test.clj
* Update driver.clj
* Update driver/sql.clj
* Set sql driver join support to true for all joins
Deriving drivers are expected to set to false where applicable.
* Update sqllite.clj
* Remove foreign-keys from spark
* Remove :foreign-keys from presto
* Remove :foreign-keys from Athena
* Remove foreign-keys from big query
Reading docs it seems fk inference should be ok. Let's see the test results. Act based on that.
* Update test_metadata.cljc
* Set key-constraints to false for bigquery
* Add foreign keys to sqlite manually until sync is fixed
* Return driver require to Athena
* Correct typo
* Add naive primary key heuristic
* Update pk fk logic to handle name components correctly
* Add alias escaping to presto
* Add ordering to test
* Add order by to test
* Update test
* Remove use of rewrite-fields-to-force-using-column-aliases in order by fields
* Add exception to alias forcing
* Different approach to exception from alias forcing
* Alternative approach for prefixing idents in bigq
* All seelcted fields by desired alias
* Rewrite only fields not from this source table
* Update test
* Enable breakout-on-fk-field-test for :left-join drivers
* Add feature comment
* Explicit joins tests foreign-keys removal
* Update nested_queries_test.clj
* Update remapping tests
* Update tests
* Update tests to handle sqlite results format
* Disable metadata/key-constraints on sqlite during tests until
* Address remarks
* Remove mt/with-mock-fks-for-drivers-without-fk-constraints
* Update bigquery test
* Update tests
* Adjust row level restrictions
* Add parameterized-sql feature
* Update comment
* Update leftovers
* Order keys
* Remove foreign keys from frontend
* Fix FE unit
* Update driver changelog
* Address review remark
* Update test/metabase/query_processor/test_util.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update docs/developers-guide/driver-changelog.md
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update docs/developers-guide/driver-changelog.md
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update modules/drivers/bigquery-cloud-sdk/src/metabase/driver/bigquery_cloud_sdk/query_processor.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update docs/developers-guide/driver-changelog.md
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update test
* Update comment and reduce expression
* Update comment
* Address remarks
* Fix merge
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Derive MySQLEnum from Text instead of TextLike
Fixes#44807.
This change is in accordance with https://dev.mysql.com/doc/refman/8.0/en/enum.html describing the ENUM types as a
kind of string. All our string operations seem to work with MySQL ENUM fields.
* docs: extract requires a db with regex
* Update docs/questions/sharing/visualizations/table.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* tests for drivers with spaces in their identifiers
* Add driver feature
* Fix tests
* Remove ambiguous alias for sqlite
* Try to fix bigquery and sqlite tests not seeing left-join support
* Add explicit support for left-join because of foreign-keys during test hacks
* Handle left-join support weirdness
* Fix test
* Fix falsy tabSelected/tabRenamed checks
* Update URL management (WIP)
* Fix tests failing because of whitespace changes
* Fix tests failing because of parameter order
* Use one hook for parameters and tab URL management
* Fix tests failing because of parameter order
* Fix test failing because of whitespace changes
* Revert
* Fix DashboardTabs tests
* Fix tests failing because of parameter order
* Fix test
* Fix test failing because of whitespace changes
* Remove unit tests that should be e2e tests now
* Fix tab navigation
* Fix string encoding/decoding
* Revert test changes
* Consistent question search params
* Update DashboardTabs tests
* Fix `SyncedParametersList`
* Fix e2e tests
* Fix nav usage
* Fix types
* Remove `useSyncedQueryString`
* Fix test
* Move slug utils to dashboard utils
* Fix nav
* Revert nav change
* Fix e2e tests
* Fix public/embedded dashboards, handle site url
* Add repro test
* Bring back and fix `PublicOrEmbeddedDashboardPage` tests
* Move `SyncedParametersList` to qb
* Refactor EditoTabs
- get rid of options prop
- use named export
* Remove unused data-testid
* Remove redundant id and aria-labelledby
* Remove redundant props spread
* Bring back data-testid in use
- Thanks VSCode for randomly showing me search results or not...
* Fix navigation issue
* Add a repro for first issue
* Adjust EntityPickerModal root element so that it works with "be.visible" assertion in Cypress
* Add a repro for second issue
* Refactor tests
- Fix not enough history entries issue
* Change order of tests to start with simpler one
* Fix invalid request issue
* Add more assertions
* Format code
* customize popover z-index
* make popover z-index customizable
* make the popover z-index customizable
* add tests for popover z-index
* also specify z-index for mantine popover
* Reuse connection for test data loading & JDBC data loading overhaul
* Revert silly change
* Remove unneeded logic for recording which datasets are loaded or not-yet-loaded
* Ok so I guess we do need the code to detect duplicate dataset loading after all
* B I G improvements
* Appease Kondo
* Test fixes 🔧
* More test fixes 🔧
* Fix dataset-already-loaded?-test
* Fix test
* Fix Presto JDBC and greatly improve test data load speed for it
* Postgres test fix 🔧
* Fix compilation errors
* MySQL fixes 🔧
* Fix syntax error
* MySQL fixes 🔧
* Implement `dataset-already-loaded?` for Athena and cache the set of existing databases
* Fix Redshift
* Fix Presto JDBC
* Vertica test fix 🔧
* Fix Oracle dataset-already-loaded?
* Test fixes and parallelize some API tests.
* Test fixes 🔧
* Oracle test fix? 🔧
* Another test fix 🔧
* Stop putting random spaces in code bodies this is not JavaScript code
* with-temp-vals-in-db should merge in original value of Database settings
* The most test fixes 🔧
* 3 or 4 test fixes 🔧
* Add schemas around stuff to catch errors
* Appease Kondo
* Implement dataset-already-loaded? for vertica
* Implement dataset-already-loaded? for SQLite and Snowflake
* [WIP] Testing driver filters
* Only test returned values in CI
* Add :uuid-type for dbs that support that type (test oracle's support)
* Try again for oracle test, redshift does not support uuid-type
* Add tests, support where available
* Add empty/null tests
* Adjust test query for athena
* Reset logging
* Fix comments
* Add Athena uuid conversion
* Address PR review
* Add comments
* Address PR comments
* Use varchar for portability. Add inline uuid casting for athena
---------
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Field Filter Value Normalization in Embedding Endpoints
Fixes 27643
Since we use query-params to pass paramter values, we have to normalize the parameter map values.
This PR is a WIP, as I'm trying to find a way to do this hollistically/properly.
First commit here is mostly as a marker for where to start working on this properly, as I think the json parsing won't
catch all cases, nor should the normalization happen without some attempt to use a schema to properly validate, maybe
even coerce things.
* read strings if boolean or numbers, replace blank str with nil
Also update the common dashboard param values fn to use embedding params from the URL blob if the app db doesn't have
embedding params. This happens upon first preview where nothing will have been written to the Appdb yet.
* Add query-param normalization test, add related embedding test
* schema didn't work in this case after all.
Worth a shot, but don't want this to block the PR, so reverting to the previous schema
* Add comment pointing out the downsides/reasons for this solution
It's nto a perfect solution, but it's probably the best we can get at the moment, without adding a new filter
type (boolean) and/or improving the design of hte embedding endpoints and how filters/parameters work
When we run queries, we preprocess them, during which the sandboxing
middleware adds information that we use when checking permissions
later on in query processing.
When we checked data permissions before saving a card, we were checking
permissions on the raw query, without preprocessing. Without the data
provided by the sandboxing middleware, the permission check sometimes
failed (depending on the permissions for other tables in the DB).
Note the big hack here: preprocessing can fail, for lots of reasons.
This broke a *lot* of tests that were passing in invalid queries. If an
exception occurs during preprocessing, we catch and just use the
original query like we were before.
* refactor: Simplify code (#44275)
* Remove dead code
* Remove values same as default options
* Simplify hook API: Removing unused argument
* review: Remove redundant code
* Update look and feel copy (#44277)
* Update static embed tab name
* Update copies + rearrange settings
* Use Mantine Select
* Fix E2E tests by applying new copies
* Fix unit tests
* Fix more E2E tests
* review: Add a comment on Mantine `value` quirk
* Adjust spacing and relevant tests
* Move `transparent` theme to its own toggle option (#44362)
* Reorder the function to match the visual order
* Make function name consistent
* Introduce the new background option (do nothing yet)
* Remove redundant exports
* Tighten up values to reduce confusion
* Make the parameters order consistent
* Only show background option on dashboads (spec)
* Allow dashboard embed background to be transparent
* Make filter background and header/footer transparent
* Fix type error
* Fix unit tests
* Fix E2E tests
* Fix checkerboard pattern not showing when no background
* Add unit tests
* Add missing copy update
* Remove remnants from outdated solutions within the PR
* Improve code comment references
* Fix E2E tests since the last copy change
* Fix type errors from merging master
* Update light theme style (#44488)
* Add background to dashboard filter on light theme
* Make position sticky works with transparent background
* Fix sticky filters not working on theme=transparent
* Fix unit tests
* Simplify IntersectionObserver components + fix E2E tests
* Add storybook for embedded dashboard
* Clarify the use of ExplicitSize context
* Remove unused code
* Formatting
* Review: Better mock `isWithinIframe`
* Update dark theme to match the new design (#45134)
* Update light theme checkerboard colors
* Update checkerboard colors
* Consolidate dashboard light/dark border style
* Use `--mb-color-text-primary` for light theme dashboard
* Update embed dashboard dark theme primary color
* Migrate dashboard text-secondary color
* Migrate text-tertiary color
* Migrate gold, axis, and split line to CSS custom properties
* Fix failed e2e tests
* Fix static viz not working
* Update loki snapshots
* Remove unnecessary CSS custom property
* Make diff more consistent
* Address review: reusing type
* Support backward compatibility when passing `theme=transparent` (#45214)
* Match preview query with the actual code query
* Handle backward compatibility when `theme=transparent` on dashboard
* Add unit tests
* Address review: Make tests less repetitive
* Update Metabase banner upsell UI (#45424)
* Prepare folder for nested components
* Update upsell UI
- add `fullWidth` variant
- align title to the left
* Fix Upsell banner having horizontal scroll on full width
* Add new Metabase banner upsell component
* Fix unit tests
* Address review: Fix typo
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
* Address review: Move upsel UI to `admin/upsells`
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
* Remove unnecessary ESLint disable comment
We can remove it since the file is in admin folder, and is already excluded from the rule.
* Address review: remove default value property
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Address review: Extract magic number
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* More UI updates (#45450)
* Update dark theme text colors to match the new design
* Update powered by metabase logo
* Update loki snapshots
* Fix unit tests
* Fix formatting
* Fix wrong dark theme metabase logo badge color
* Address review: Compress SVG with svgo
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Fix E2E tests
* Update loki snapshots
* Clarify Metabase base colors with comments
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Add more tests (#45628)
* Add static embed question stories
* Add loki snapshots
* Add transparent theme for static dashboard tests
* Add transparent theme tests for static questions
* Assert `background` option in analytics tests
* Remove tests for logic that has been moved
* Update loki snapshots
* Fix failed E2E tests from merging master
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Refactor EditoTabs
- get rid of options prop
- use named export
* Remove unused data-testid
* Remove redundant id and aria-labelledby
* Remove redundant props spread
* Bring back data-testid in use
- Thanks VSCode for randomly showing me search results or not...
* add pdf export button to static embed and public sharing (#43743)
* add export button to embed frame, it works on static embed and public sharing
* move css around to make dashcard border show up on static/public embed pdf exports
the library we use to render an html node to an image doesn't seem to support box shadow
the original fix was done in ParametersAndCardsContainer, but that component is not used in the static/public embeds
* fix types
* fix night theme pdf export
- use padding for the title instead of starting to draw the image from 60px, to keep the bg color
- change the css to also apply the bg-color to the first child of the EmbedFrame, to make it work when exporting to pdfs, where the EmbedFrame is not part of the exported dom
* refactor: exportTabAsPdfButtonText function
* give a default to the filename, even though it should always be there
* exportTabAsPdfButtonText -> getExportTabAsPdfButtonText
* introduce .WithThemeBackground to not have a css rule too much generic
* feat: adds endpoint to download results of a dashcard part of a public dashboard (#44156)
* Flag to hide download buttons (#44206)
* pass down downloadsEnabled to hide downloads buttons in public and embedded questions or dashcards
* Update frontend/src/metabase/dashboard/components/DashCard/DashCardMenu/DashCardMenu.tsx
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* PLUGIN_DOWNLOADS_ENABLED -> PLUGIN_RESOURCE_DOWNLOADS
* adds comment to areDownloadsEnabled function to explain what it does
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* logic to handle downloads url parameter (#44363)
* minimal support for downloads url parameter
* updates StaticEmbedSetupPane test to epxect downloads=false instead of hide_download_button=true
* fix prettier mistakes
* tmp fix to the tests until we have the new analytics events
* fix eslint issues
* update embedding snippets e2e
* update embedding questions e2e
* try to clean up the mess of downloads/downloadsEnabled
* adds `hasPremiumFeature` check to the plugin, adds unit test for the logic
* adds TODOS for analytics
* unit tests for downloads flag (#45178)
* fix download pdf button not showing up when titled=false and there's only one tab
* unit tests for downloads flag on dashboards and questions
* adds test about the container id, not sure if we should keep it
* adds a comment to explain why we enable premium features in the tests
* move downloadsEnabled outside of the Boolean cast as it's already a boolean
* E2E tests for public/static downloads (#45240)
* wip e2e tests
* make downloadUtils ts friendly
* assertNotEmpty function
* adds tests for public questions
* e2e tests for static embed downloads
* publicUid -> publicUuid
* fix assertNotEmpty
* fix parameter not renamed
* Apply suggestions from code review
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* remove duplicated test
* fix another typo i forgot to rename
* fix another rename issue
* convert e2e-downloads-helpers to ts
* refactor: put method calculation in getEndpoint function
* adds note to downloadUtils
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Update e2e/test/scenarios/embedding/embedding-snippets.cy.spec.js
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* change requires to import to reflect that we changed to export the source code in downloadUtils
* makes downloadsEnabled prop more consistent across components
* colocate downloadsEnabled with hideDownloadButton in embed-display-options
* defaults displayOptions.downloads to true to simplify logic
* remove unused queryString parameter in tests setup function
* reword TODO
* Revert "defaults displayOptions.downloads to true to simplify logic"
This reverts commit aebf4cf3cf.
* fix png export when on dark theme (public/embed) not having the dark … (#45497)
* fix png export when on dark theme (public/embed) not having the dark background
* move the border-radius fix for the dashcard export to saveChartImage
* remove comments
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Do not display alert icon for models
* Add e2e repro for #37893
* Put the guard higher
* Fix unrelated test
* Go back to keeping the logic in `shouldRender`
* Remove whitespace
* Get rid of tolerance
* Use range rect and element rect instead of clientWidth/clientHeight/scrollWidth/scrollHeight
* Fix types
* Mock Range.prototype.getBoundingClientRect
* Mock getBoundingClientRect in unit tests
* Add repro for #32252
* Have a test for archiving a question and a collection
* Refactor cy.createCollection and cy.archiveCollection to TS
* Refactor collection and user helpers to TS
* Keep collection helpers together
* Sort code
* Format code
* Use new createCollection helper
* Add return type to createCollection
* Fix typing
* Fix#32252
* Fix typing
* Deprecate openNewCollectionItemFlowFor
* Revert "Fix typing"
This reverts commit ecf3179868.
* Fix TS error
* Refactor cy.createCollection and cy.archiveCollection to TS
* Refactor collection and user helpers to TS
* Keep collection helpers together
* Sort code
* Format code
* Add return type to createCollection
* Fix typing
* Deprecate openNewCollectionItemFlowFor
* Add a repro for #43219
* Fix#43219
* Create questions in parallel
* Make 2 elements scrollable so that the 'Make sure to make a selection ...' element is not weirdly cut off when scrolling horizontally to see inside of parameter-mapper-container
* Fix assertion
* Be more specific which element is to be scrolled
* Avoid realMouseWheel which does not work reliably in CI
* updates the fe to skip refetching the graph when updating the collection graph, adds tests
* Moves the skip_graph flag to be part of the body.
- the FE tooling makes it awkward for PUTs to include query params
* fixes failing unit tests
* pr feedback
---------
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* Add Temporal Units List to Dashboard Parameters Schema
Closes: #44361
This adds an entry to the Dashboard Parameter schema so that we can validate
the list of temporal_units that might be passed from the frontend when adding
or updating the `Unit of Time` type parameters.
* This is the way
the real reason the schema wasn't working... well, it was, but the un-decoded versoin of parameters was being passed
to `update-dashboard` because inside defendpoint, you can use map destructuring and bind the whole map to whatever
with :as keyword.
If you decode/validate on all of the symbols in the destructuring, that's great, but if you then pass the map bound
with `:as whatever`, you don't get that decoded goodness, just the data that our http middleware produces (that is, it
might not be the shape you expect).
So, here I've created a private Schema for the :body of the request and use the same schemas to validate each key in
that map. The result should be the same but we can pass a fully decoded/validated map along.
* change test to look for the error in the right place
* automate updating translations for major releases
* add reviewer to PR
* auto translate
* auto-translate works!
* update
* automate translation updates in CI
* fixes
* whitespace
* sort scripts
* just checkout the branch to begin with
* fix tests!
* fix import
* parameterized static questions
* sdk parameter values
* use parameters to construct parameter input
* add a unit test for passing the correct parameters
* fix parameter parsing
* move TEST_PARAM to top of file
* update readme docs
* address code review
* add skip_graph to coll perm PUT
- also no longer ignores revision and groups
so we don't print this warning:
2024-07-11 16:17:32,293 WARN api.common :: Unexpected parameters at [:put "/api/collection/graph"]: [:revision :groups]
* namespace should be a keyword
* decode the keyword
* add a typehint to avoid reflection
* Adds test for skip_graph
* handle revision 0 🤦
* Do not reset `table.columns` when some column is deleted
* Do not reset `table.columns` when some column is deleted
* Fix
* Fix
* Make API calls more obvious
* do not allow to render empty SaveQuestionModal content
when question is already saved and QB rerenders still open SaveQuestionModal there is a moment when content is empty before modal is closed
* fixup! do not allow to render empty SaveQuestionModal content
* parallelize coll perm graph group lookup w/ core.async
* respond to code review
* add + uses ExecutorCompletionService map function
* rename test
* more tests
* no concurrency for cljs
* fix timeout + add more tests
* use claypoole
* cleanup
* add kondo hooks for claypoole
* cleanup
* Can't use libraries that don't have a license
Revert "add kondo hooks for claypoole"
This reverts commit 9d93b55b28.
* ignore unused value
-- TODO: check back in on claypoole to see if the kondo config gets merged
* fix linter
This was a huge footgun.
We were accidentally limiting *ALL* queries run in our transducer to
1000 results. This included the query to fetch permissions, so when we
checked permissions we were checking a small percentage of your *actual*
permissions (e.g. 1000 / ~6300 for me on stats).
This comes in two parts.
**Rebuilt query logic**
I rewrote the query exec logic to use one `future` and one `a/go` block,
reporting results to a `promise`. This is easier to follow and doesn't leak
threads like the original implementation with two `future`s.
The single future is guaranteed to exit. It blocks on the BigQuery call.
That will either complete or throw an exception. Either the exception or
the BigQuery result is sent to the promise, and then the future is
complete. The only reference to that future is overwritten after the `promise`
resolves, so the `future`, its closure, and its thread don't persist longer than needed.
**Return a reducible result**
Previously the results were returned as a lazy sequence, but that brings
a lot of `seq` overhead, and is a risk of retaining the entire result
set in memory until the query is all done. Above all, it doesn't support
`reduced?` to explicitly drop any further results.
The `thunk` function style of `qp.reducible/reducible-rows` doesn't fit
neatly here, so I implemented the reducible logic directly on top of the
`Iterable` BigQuery results.
**Verifying**
This is a bit tricky. I've been testing it like this:
- Add a BigQuery database to my instance
- Create a saved question and note its ID (say, 123)
- Create an API key in (EE) Metabase and save it in an env var
- Install the `siege` load testing tool
- Install VisualVM and monitor my Clojure REPL process's thread count
- `siege --concurrent=15 --time=30s --header="x-api-key: $API_KEY" "http://localhost:3000/api/card/123/query POST"`
That will create a bunch of new threads while it's getting called, but
within a minute of the last queries they'll gradually drop out of the
thread pools and it will return to the baseline number of threads.
* create sdk load question
* store question result in sdk context
* remove dataset data type
* update question
* handle navigate to new card
* add loading states
* fix on query change
* update question change logic
* refactor question loading and query params
* fix ad-hoc question in dashboards
* extract useLoadQuestion into a separate hook
* cleanup run question on load
* revert changes in interactive question component
* store original question in the question result state
* turn the question into ad-hoc questions
* refactor run question methods
* add a separate isQueryRunning state for better ux
* isQueryRunning should default to false
* simulate re-running the query in unit tests
* add tests for rendering multiple valid questions
* refactor sync question method
* extract raw series and computed pivot logic
* add getCardAfterVisualizationClick
* separate single and multiple question tests
* fix import paths
* is question native
* pass queryResult instead of queryResults
* temporarily remove checkNotNull for aggregation picker to avoid sdk crash
* Dashboard Paramter Value Now Works in Preview Embed
Fixes: #44962
Prior, a common util fn used both in the embed and preview_embed endpoints was causing the preview_embed to error out
when fetching dashboard parameter values because it was checking if embedding was enabled. This check is needed for
the normal embedding endpoints, but we want the preview embed endpoints to work without needing to publish the
dashboard first.
* pass proper args
* put args back in proper order.
oops
* change form back to previous, but keep preview check
* autocorrect mistake? Or perhaps a silly typo?
* change one test to show that the preview flag is working
Didn't create a whole new test for this flag as it is provable just by adjusting this test to not bother enabling
embedding on the dashboard.
* improve the docstring
* Log changes when they make sense + async coll perm-graph revision
* - Make revision-number consistent
Decouples the creation of a revision number from the (potentially
expensive) writing of before and after for the coll perms-graph
revision.
* get id from the perms revision entity
* Move the revision id creation into the transaction
The revision id creation should only happen iff the perm updates went
through successfully.
* write a test that checks the incomplete revision
* remove sleep from tests
* Migrate Model Indexes to RTK
* Sorting out types... I think?
* remove unneeded comment
* replacing the selector with an initiate to ensure that there is data
* catching and throwing error
* Updating unit tests
This introduces a new metadata on commands, `:requires-init`. If set,
we'll run `metabase.core/init!` before running the command. `init!` will
initialize the database and run our normal startup, though it won't
actually start the Metabase server.
This allows you to use
```
env MB_CONFIG_FILE_PATH=... java -jar metabase.jar import /my/metabase/export
```
We'll run the normal init process (creating the internal user, loading
from the config file, etc), then run the command.
On my machine this adds about 2-3 seconds to the time it takes to run an
`import`.
* start adding some util fns to download-exports-test ns
* wip improving test util fns
* add subscription attachment util fn
* add tests to catch download/export differences
* viz-settings from card and dashcard merged properly
* revert these changes which probably aren't needed
* split test: downloads can be parallel, email attachments cannot
* normalize viz-settings no matter their origin
* Update test/metabase/api/downloads_exports_test.clj
Co-authored-by: Alexander Solovyov <alexander@solovyov.net>
* Incorporate some review feedback
* rendered table cards also respect column names
* Clean up table render tests and remove unhelpful render test utils
The big removal of code in the render test utils namespace is a clean up task that could've happened sooner, but this
table test adjustment is the perfect opportunity to clean up.
I had made the test utils before I truly knew how best to use `mt/with-temp`, so it had a lot of poorly implemented
'generators' that were doing a poor job of creating mock data for the rendering functions anyway.
So, reworking a few table tests here to just use a temp card is way more inline with the rest of the codebase, and
actually improves confidence that the tests are testing on well-formed data to begin with.
---------
Co-authored-by: Alexander Solovyov <alexander@solovyov.net>
* add admin-only user-local setting that controls perms tutorial
* Adding modal and banner, user setting for dismissable banner
* copy changes
* adjust so modal only appears on data permission page
* e2e adjustments, adding banner test
* never show the modal or banner on new instances
* add test
* e2e adjustments, copy update
* more PR feedback
* move modal/banner settings & tests + clarify tests
* query for v50 migration execution works on mysql
- move changelog lookup into metabase.db.liquibase
* no race conditions in changelog-table-name
The previous implementation of this function was hitting this
liquibase issue: https://github.com/liquibase/liquibase/issues/2966
* revert changelog-table-name
- simply lookup the changlog row
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* Add a test verifying that fonts are loaded
* Verify font is mentioned in HTML
* Bring back working assertions
* Add wait
* Disable speedy
* Disable CSP and emotion cache
* Revert "Disable CSP and emotion cache"
This reverts commit 573e99bfa4.
* Revert "Disable speedy"
This reverts commit 6f9962b869.
* Don't use local fonts
* Revert "Don't use local fonts"
This reverts commit 17c58aa73d.
* Force Lato
* Fix storybook file name
* ESLint
* Add more !important
* Revert !important
* Add font-family assertion
* Move test
* try fixing cross-version tests
* restore legacy metrics
* its not legacy in v43
* Dont got echarts either
* newfangled echarts
* run cross-version tests on code changes in the tests
* enable slack message
* don't always run cross-version tests
* whitespace
* add refresh auth token function
* provide a default implementation for refreshing auth tokens
* rename getRequestToken to fetchRequestToken
* add docs on customizing jwt authentication
* clarify docs, add error handling and failing tests
* update tests to pass
* re-export FetchRequestTokenFn type
* re-export EmbeddingSessionToken type
* perf: pass uuid to BE to enable metadata caching during fetching dashboard
* fixup! perf: pass uuid to BE to enable metadata caching during fetching dashboard
* Add wildcard on interceptions
* Add tests
Before
====
```
2024-07-04 15:54:39,084 INFO metabase.server :: Shutting Down Embedded Jetty Webserver
Exception in thread "Thread-9" java.util.IllegalFormatConversionException: f != java.lang.Long
2024-07-04 15:54:39,122 WARN db.liquibase :: (#object[liquibase.Liquibase 0x2b7e6614 liquibase.Liquibase@2b7e6614])
at java.base/java.util.Formatter$FormatSpecifier.failConversion(Unknown Source)
at java.base/java.util.Formatter$FormatSpecifier.printFloat(Unknown Source)
at java.base/java.util.Formatter$FormatSpecifier.print(Unknown Source)
at java.base/java.util.Formatter.format(Unknown Source)
at java.base/java.util.Formatter.format(Unknown Source)
at java.base/java.lang.String.format(Unknown Source)
at clojure.core$format.invokeStatic(core.clj:5770)
at clojure.core$format.doInvoke(core.clj:5764)
at clojure.lang.RestFn.invoke(RestFn.java:423)
at metabase.db.liquibase$wait_for_all_locks.invokeStatic(liquibase.clj:258)
at metabase.db.liquibase$wait_for_all_locks.invoke(liquibase.clj:251)
at metabase.db.setup$release_migration_locks_BANG_.invokeStatic(setup.clj:175)
at metabase.db.setup$release_migration_locks_BANG_.invoke(setup.clj:170)
at metabase.db$release_migration_locks_BANG_.invokeStatic(db.clj:91)
at metabase.db$release_migration_locks_BANG_.invoke(db.clj:86)
at metabase.core$destroy_BANG_.invokeStatic(core.clj:90)
at metabase.core$destroy_BANG_.invoke(core.clj:81)
at clojure.lang.AFn.run(AFn.java:22)
at java.base/java.lang.Thread.run(Unknown Source)
```
And now it will correctly log
```
2024-07-09 16:04:28,345 INFO db.liquibase :: Waiting for migration lock(s) to be released (max 0.4 secs)
```
Why no tests?
====
There _is_ a test that would have caught this. And in fact running at
the repl will catch it:
```clojure
liquibase=> (do (require 'clojure.test 'metabase.db.liquibase-test)
(clojure.test/run-test metabase.db.liquibase-test/wait-for-all-locks-test))
Testing metabase.db.liquibase-test
ERROR in (wait-for-all-locks-test) (Formatter.java:4515)
:h2
Will timeout if a lock is not released
expected: (= :timed-out (liquibase/wait-for-all-locks sleep-ms timeout-ms))
actual: java.util.IllegalFormatConversionException: f != clojure.lang.Ratio
```
But this _passes_ in CI:
```
❯ clj -X:dev:test :only metabase.db.liquibase-test/wait-for-all-locks-test
Finding tests took 23.5 s.
Running 1 tests
Running before-run hooks...
Ran 1 tests in 1.461 seconds
3 assertions, 0 failures, 0 errors.
{:test 1, :pass 3, :fail 0, :error 0, :type :summary, :duration 1461.175042, :single-threaded 1}
Ran 0 tests in parallel, 1 single-threaded.
Finding and running tests took 25.0 s.
All tests passed.
Running after-run hooks...
```
And that's because the log level in testing is FATAL and this is an INFO
level log.
Discussed ways to improve this like adding a null logger during CI but
we aren't ready to increase CI times for that just yet.
* Move focus to separate helper
* Use focus method to do initial focus
* Use focus method for other _editor.focus calls
* Move focus to componentDidMount
* Cancel focus timeout on unmount
* Add delay to broken test
* Switch to requestAnimationFrame
* Remove wait in test
* fix combined cards crash when some cards have invalid viz settings
* fix mock column was not metric
* ensure combined scalar column has proper type to handle the case when the card returns empty results
This PR also specifies the 'filled' variant for the question duplication
modal, since the previous default 'filled' variant for submit button was
added to make that specific button filled.
* update e2e test for command pallete to use a valid setting
* remove the setting from the ui, make it always true, unless set false from env
* remove enable-nested-queries from admin section on fe
- fix unused oldValue -> _oldValue
* tests aren't allowed to change enable-nested-queries
* fix test + disallow setting nested queries from code
* we cannot set enable-nested-queries
- set the env var instead
* add checks for the enable-nested-queries setting
* make resolve-source-cards work with enable-nested-queries setting
* fix recents uri interception
* return enable_nested_queries setting to client
* adjust fetchmock url matcher
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* Nicer error messages for linting migrations
In general, just throw exceptions instead of using clojure spec here.
You'll only get one error per run, but that seems fine.
I did keep some specs around, but run them slightly differently. Rather
than validating the whole collection of changeSets at once with `s/+`,
just `doseq` through the changeSets and validate each one separately.
That way, the value that's presented as erroneous is much smaller (a
single changeSet) and it's easier to see what went wrong.
* Update bin/lint-migrations-file/test/lint_migrations_file_test.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update bin/lint-migrations-file/src/lint_migrations_file.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Require preConditions for certain liquibase change types to encourage idempotence (#44578)
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* remove `!`s from non-side-effecty fns
* add preConditions to a few v51 migrations
---------
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Split unrelated changes into separate PR
* Fix test since user perms set reset fix split into other PR
* Fix syntax error
* Test fixes 🔧
* Test fix??? 🔧
* Test fix 🔧
* lint: add stylelint support for emotion components
* drop comment
* fix pattern order
* Use one version of typescript
* use :focus not :outline
* Use the logical CSS property "inset-inline-start:" (not "inline-start:" which doesn't exist) as replacement for the physical css property "left:"
* Use mb-color-focus for focus color
* Drop dead code
---------
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* fix: limit xrays on agg models to source fields
Fixes#43793
1. `table-like?` was checking aggregations but not breakouts. This meant
that legacy metric definitions would try to apply to the second stage
which is generally illegal and would cause things like implicit joins
to come in where they make no sense.
2. Trying to bin a `count` metric-like model would give a bin span of
0.0 which would blow up those questions.
3. Trying to self reference breakout fields, in the precense of a
question's aggregation meant that most fields would not be exposed to
the next stage and should be ignored. (similar to 1.)
* Fix tests
* Add csv-stringify package
* Split values at comma
WIP: Parse values using json
* Add label to ParameterValue and add ParameterValue to value config
* Use custom label in FieldValuesWidget
* Allow passing displayValue from outside of RemappedValue
* Allow passing displayValue in ParameterFieldWidgetValue
* Get custom value label in FormatterParameterValue
* Allow rendering custom displayValue in FieldValuesWidget
* Add custom ParameterValue types
* Fix ParameterValuePicker type
* Fix jest import for csv-stringify
* Add missing RTKQ exports for paramters
* Use direct implementation of useGetParameterValues
* Fix ValueSourceModal tests
* Add tests for value parsing and stringifying
* Use custom label in list too
* Render the custom label in a parameter value
* Setup parameter mocks in embed test
* Add info about splitting values with comma
* Add model hint when user is using labels
* Add test for cusom labels on dashboards
* Handle label-value pairs in test
* Test custom labels in parameter search input
* Use the label to parse the value for text inputs
* Fix comma's to commas
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Use strong instead of styling link
* Use getInput instead of ignoring eslint rule
* Use arrow function
* Add newlines between tests
* Use uppercase for test value
* Name value
* Use getLabel and getValue helpers to avoid indexing confusion
* Pull out confusing condition
* Move out getLabel helper
* Be more explicit about non-existent config values
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Move setupParameterValuesEndpoints to setup
* Add loading state
* Do not use alert to implement info box
* Fix formatting
* Remove undefined from getValue signature
* Use getLearnUrl
* Remove dependency on column
* Make custom field labels work in the absence of fields
* Support custom values for non-field parameters
* Add test for native question parameters
* Add newline in test
* Add test case for when show-metabase-links is false
* Just fall back to rendering value when fetching label, or in case it failed
* Add test for loading state of FormattedParameterValue
* Do not fetch custom label for FormattedParameterValue
* Add table with ip addresses
* Add test for IPAddress filter with custom labels
* Add test for custom labels on type/Quantity fields
* Remove getInput helper in test
* Remove unused hasCustomLabel value
* Split up into common and premium tests
* Make fieldValues a map for performance
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* fix: param-fields when native model is aggregated
Fixes#42829
The param-field lookup required result_metadata to match
filterable-columns in order to find parameter targets. This adds a
fallback where if the param field is missing from result_metadata,
then we look at filterable-columns.
* Unskip test
Per discussion in Slack, I made a mistake with the migration IDs as I wanted to backport to 50 but used 51 IDs.
This PR uses 50 IDs and places the migrations in the correct place.
Not sure if it's mergable in this state because Stats has run with the PR containing the incorrect migration IDs.
* Refine the reproduction for 44794
* Move skip to top-level again
* Remove superflouous click
* Verify wether or not the recents tab exists and is open
* Make the test description more descriptive
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
* Move question details to a variable
---------
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
* User Parameter Values Are Unique Per Dashboard
Fixes: #43001
Might be related: #44858
User Parameter Values previously stored just the `parameter-id` and `user-id` as data to look up the last used
parameter value. This isn't sufficient as the parameter id is not guaranteed to be unique, in particular this is true
when a dashboard is duplicated: the parameters are copied to the new dashboard without changing the parameter id at
all.
This means that we need to also store the dashboard-id in the user_parameter_value table and use that to update/get
the last used value.
The migration removes existing entries to the user_parameter_value table because I want a non-nullable constraint on
the dashboard_id column, and existing entries will not have a value. The only way to backfill those values would be to
look through every dashboard containing parameters, and for every parameter check for a matching ID. Even if you can
do this, there's no way to disambiguate if the same paramter exists on 2 dashboards anyway, so one of them will be
wrong. I think it's not worth the trouble considering that removing the entries in this table doesn't break anything;
it just becomes a mild inconvenience that a user has to select a filter value again (since the dashboard will use the
default value).
* alter test to check for uniqueness across dashboard
This test makes 2 dashboards with parameters of the same ID and asserts that changing the value on dashboard-a does
not change the value on dashboard-b.
* adjust migration to pass linter rules
* remove the extra rollback on migration
* Update src/metabase/models/user_parameter_value.clj
Co-authored-by: bryan <bryan.maass@gmail.com>
* Update src/metabase/models/user_parameter_value.clj
Co-authored-by: bryan <bryan.maass@gmail.com>
* adjust parameters in test so we don't get logged warnings
* put update!/insert! in a transaction to avoid any race conditions
---------
Co-authored-by: bryan <bryan.maass@gmail.com>
* Support compiling SQL with inlined parameters directly
* Test fix
* Make `format-honeysql` a multimethod
* Add test for #32543
* Test also fixes#44915
* Add explicit test for #44915
* Better test for #44915
* More test fixes 🔧
* Fix Kondo error
* Optimize: compute chart title href only on mouseenter and focus
* Improve comment
* Improve comment
* Format code
* Update existing tests
* Extract helpers
* Update unit tests
* Make getHref argument-less
`Lib.diagnoseExpression` is stricter about this than the QP or FE
expression parser, making it reject expressions that tried to compare
datetimes to strings like `2024-07-02 14:34:17Z`.
Those used to work in 49.x, but a new `Lib.diagnoseExpression` call
while editing expressions caused it to regress.
Fixes#44916.
**MetadataProvider cache**
The reusable MetadataProviders is a generally useful thing - within some
dynamic scope like an API request, a `clojure.core.cache` instance can
be provided as `lib.metadata.jvm/*metadata-provider-cache*`, and that
will be used to cache the provider for each `database-id`.
Reusing those providers can significantly reduce appdb traffic,
re-fetching the same metadata in different `MetadataProvider`
instances.
**Dashboard loads**
The FE can supply `?dashboard_load_id=uuid_string` on requests for a
dashboard load:
- `/api/dashboard/:id`
- `/api/dashboard/:id/query_metadata`
- `/api/dashboard/:id/dashcard/:dashcard_id/card/:card_id/query`
That provides a good cache key across that burst of N+2 API requests
that are part of a single load. Currently it is used for two things:
- Caching `get-dashboard` (used by the first two requests)
- Selecting the `clojure.core.cache` for `*metadata-provider-cache*`
Both use a TTL of 10 seconds, which is generally enough for the first
screenful of a dashboard load, if not the whole thing. (Browsers will
only issue a handful of requests at once, so a few long-running queries
might delay later parts of the dashboard from loading until after the
TTL expires.)
* Make sure the query is native
* Convert the comment to JSDoc
* Add E2E reproduction
* Fix mapping
* Add test scenarios for the public sharing and embedding
* Simplify API payload
* Extract and significantly expand the reproduction scope
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* only send permissions updates for groups that have been modified in the graph
* Support partial permission graph updates on the BE (#44797)
* handles keeping track of advanced permissions changes to send relevant group data when updating the graph, merges response
* tests and optimizations
* respect group-ids set when fetching connection impersonations, sandboxes, and admin group perms
* fix typo
* add e2e test
* fix BE test
* impls pr feedback, reduces code needed since advanced permissions state is always reset on save
* last fixes
---------
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* recent views sometimes don't show special colls
colls with a non-null namespace are special: they can't interact with
normal collections, dashboards, cards, etc at all. (Currently the only
use for these we have is snippets.) At some point we could add a
`namespace` argument to this endpoint to support looking at recent
collections in a different namespace, but that'd be a bigger refactor
without a whole lot to gain. For now we can just exclude these
completely, which is what we want for current usages of the recents
endpoint anyway.
We won't exclude non-null types across the board, because sometimes
we'll want to see these. But we will be particular about when to display
them.
We'll never display the `trash` collection, but will display the
instance analytics collection when we're not trying to make selections.
* Hack an e2e test around an existing bug
So, the entity picker that pops up when you're selecting a collection to
put a Snippet into shows you "Recent" collections - which previously
included snippet collections.
We should either make the Recents API support a namespace parameter for
collections so it could actually support this case, or the entity picker
shouldn't show a recents tab here (realistically, how many collections
are people going to have for snippets that they would need this tab
anyway).
* Calculate metadata without running queries [JDBC]
* Appease Kondo
* Test fixes 🔧
* Overhauled Card metadata calculation code
* Test fixes 🔧
* Kondo fix 🔧
* Ok maybe I fixed stuff
* Work around https://github.com/aws/amazon-redshift-jdbc-driver/issues/118
* Maybe fixx the Cypress tests
* Experimental possible e2e test fix 🔧
* Test fixes 🔧
* Do things a little differently.
* Try re-enabling the field refs
* Add reproduction for #38989
* Use saveQuestion instead of clicking trough the UI
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Use openOrdersTable instead of manually clicking through the ui
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Create question programmatically in test
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Wrap non-latin characters in a span specifying working font
Fixes: #38753
CSSBox seems to have a bug where font fallback doesn't work properly. This is noticeable when a font does not contain
glyphs that are present in the string being rendered. For example, Lato does not have many international characters,
so the rendered version of tables (that show up in Slack messages) will not render properly, making the card
unreadable.
Since this appears to be a downstream bug, I've opted to work around this limitation by wrapping any non-latin
characters in a <span> that specifies the font family to be sans-serif, which should contain the glyphs to properly
render.
This leaves Lato in place for other characters.
For now, I figured it's worth trying this solution over using Noto for 2 reasons:
- we can keep Lato, which has been the decided font style for the app for some time (this keeps things consistent
where possible)
- the Noto font containing all glyphs is quite a large font (>20mb, I think) and it would be nice to avoid bundling
that if we can.
* stub installed fonts test
* typo
* Do wrapping, but now per-string, and in Clojure data not html string
I've decided that a reasonable solution is to still wrap strings containing non-lato characters. But it's not done
with str/replace to the html string but rather in a postwalk over the Hiccup data prior to rendering.
Seems like a decent compromise of issues without patching CSSBox or fixing upstream (may be good to do, but will take
longer to get a fix in).
* add test checking that glyphs render correctly
* Add a test that directly checks the wrapping fn
* Change the string to keep the linter quiet
* Change how we check if string can be rendered to faster method, new Lato Font
With Sanya's help, the way I check if a given string is renderable with Lato is now faster.
Also use the full Lato font, not just the 'latin' lato so we can cover more chars
* change lato so that it loads the fn even in a fresh test run
* prevent pin maps from filtering out rows with null values
- since pin maps are just displaying where the data is by latitude and longitude, we do not need to filter the pin just because one of the rows is null.
- the metric column selection does not make sense for pins as they just represent that data through coordinates (no aggregations or values displayed), so we should not be filtering out any data just because it is null
* add unit test
I was thinking about the potential options here.
On one hand, we could initialize the app db ourselves. This is a little
problematic, because:
- our `init!` function is in `metabase.core` and calling it yields a
painful cyclical dependency. It's also pretty tied to outputting things
to the user that they might not expect during an import, e.g. "Looks
like this is a new installation ... preparing setup wizard", and starts
up jobs, etc.
- if we choose to *only* initialize the database ourselves (only the
bits that we actually *need* initialized), I'm a little nervous about
maybe missing something. Right now we need to create the internal user
and create users from the config file before running the import. If we
added a third thing, would we remember to update this?
- we could refactor to put all of the above in one spot, but this is a
P1 and it feels important to close this out as quickly as possible. We
also don't claim to support initializing the app DB from an import in
the first place.
So: just throw an exception if the app db hasn't been set up yet. Our
heuristic here is whether a single non-internal user has been created.
So if you want to use your config, you can run:
```
MB_CONFIG_FILE_PATH=dev/config.yml java -jar metabase.jar
java -jar metabase.jar import /my/exported/dump
```
If you try to run `java -jar metabase.jar import ...` on a fresh
DB, it'll now fail with the error message:
```
You cannot `import` into an empty database. Please set up Metabase
normally, then retry.
```
* highlights the trash link in the sidebar when viewing root trash collection or a trashed item
* impls pr feedback
* impls more pr feedback
* fixes unit tests and logical mistake that allowed urls with invalid collection ids to match on undefined trash id due to it not being loaded yet
* stop using strings for enum values (prefer kw)
* only do SLO when saml-slo-enabled is true
* make defsetting docstring formatting consistent
* log-direction is a string
* add test for enabling / disabling saml slo
- also return a 403 when the client tries, but it's disabled
* let's see how many things break
* add missing useSyncURLSlug
* remove cardTitled prop that's not needed
* remove DashboardControls
* remove @ts-expect-error for dashboardId
* move props destructuring at the top
* remove use-dashboard-nav.ts as it's not needed anymore
* remove import of use-dashboard-nav from dashboard/hooks/index.ts
* remove unused props from dashboard
* Add dashboard question header
* Add dashboard data preserve after question drill
* Review fixes
* Review fixes
* Potential fix for e2e
* Potential fix for e2e
* Add basic unit tests
* Fix Dashboard reloading on unmount, fix ad hoc question title in SDK
* Move dashboard reset to SDK component
* Fix issue with dashboard rerender
* Fix one more reported issue with rerender
* fix map default display type with un-binned coordinates
- default to `"map.type"` = "pin" when coordinates are not binned
- default to `"map.type"` = "grid" when coordinates are binned
* add tests
* refactor breakouts usage
* Fix broken field normalization in metadata
* Handle Fields.actionTypes.UPDATE action in tables reducer to keep table's original_fields in sync
* Add an e2e test for fields
* Format code
* Use a different table, since accounts table is not available in e2e tests
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* remove invalid else conditional
* fix RowChart firing invalid API request on legend item click
* fix clicking on non-breakout legend on dashboards
* add E2E test to validate and enforce the desired behavior
* Bump dashboard description max len limit to 1.5K
* Extract `DASHBOARD_DESCRIPTION_MAX_LENGTH` const
* Handle description max length in dashboard sidebar
* Add red border to dashboard description box
* Add test
* fixes spacing and color issue of trash icon in archive banner
* css var for color
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* WIP PublicOrCustomEmbedDashboardPage without using dashboard controls
* add usePublicDashboardEndpoints instead of WithPublicDashboardEndpoints
* fix type error by setting null as default value for hide_parameters and hide_download_button
* use usePublicDashboardEndpoints inside WithPublicDashboardEndpoints
* also migrate embed
* i don't think we need useDashboardNav public/static embed
* SetTitle component to use instead of the `title` hoc
* wrap setEmbedDashboardEndpoints/setPublicDashboardEndpoints in useEffect as per PR comment
* after the fix for metabase#38640 we don't need the public routes in the main app
* renamed test file
* change tests to use PublicOrEmbeddedDashboardPage
* remove PublicOrEmbeddedDashboardControlled and change imports for PublicOrEmbeddedDashboardPage
* remove debug prop left by mistake
* Move SimpleSpecificDatePicker to the form
* Extract `isSpecificValue` helper
* Integrate the toggle into `SimpleDateIntervalPicker`
* Make DateOperatorPicker do only one thing
Separation of concerns - lift SimpleRelativeDatePicker up to the form.
* Fix unit tests
* Add back the test for the `SimpleSpecificDatePicker`
* Keep the rendering logic in the `SimpleDatePicker`
* apply css variables injection right under the public component
* refactor css variables
* fix name changes
* apply css variables to global styles
* update path
* render component wrapper as full-width
* move style resets to public component styles wrapper
* update unit tests for global styles
* Add Temporal Units List to Dashboard Parameters Schema
Closes: #44361
This adds an entry to the Dashboard Parameter schema so that we can validate
the list of temporal_units that might be passed from the frontend when adding
or updating the `Unit of Time` type parameters.
* Sanitize the SVG before parsing it with Batik
WIP
Fixes: #43677
There are characters that are invalid according to the Batik XML Parser. This sanitizes the svg string to strip out
some of those characters so that the render can continue.
* Use the xml 1.0 spec allowed unicode chars list for the regex
* Remove accidentally merged code
* still miseed a line
* fix: dont always optimize between expressions
Fixes#42291
The frontend produces expressions like
```
[:between
[+ [:field ... {:temporal-bucket :day] [:interval 2 :day]]
[:relative-datetime -1 :week]
[:relative-datetime 0 :week]]
```
This should not be optimized because of the mixed `:day` and `:week`
units. However, it was being optimized since the compatible units
weren't being properly picked up by the match.
* Disable suspcicios args eastwood lint, kondo does it better
* Stop autobucket to day when adding a time interval
* Fix the operator list for `type/TextLike` PKs
* Fix the bug with using wrong operators
* Fix the bug with using wrong operators
* Update test/metabase/lib/filter_test.cljc
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* Test FKs too
* Update test/metabase/lib/filter_test.cljc
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
---------
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* Get rid of the styled component
* Stack date operator on top of actual filter pickers
* Use the existing `CurrentDatePicker`
* Fix unit test
* Conditionally apply padding to the `CurrentDatePicker`
* Remove the custom prop
This PR removes `:role` removal from spec, even if it was added in `:connection-uri`.
It seems that `:connection-uri` is ignored on `do-with-connection-options` code path.
Hence use of non-default roles is again available on Snowflake.
This PR disables the 'make exports look pivoted' post processing step for csv and xlsx exports.
This is in response to a reported issue that Large Pivot Table downloads are failing entirely: #44556
A more hollistic fix and improvement to the post processing feature will be worked on, so the post processing code is
left untouched in this PR.
The dynamic vars in the csv and xlsx namespaces each default `false` and disable the export from running any post
processing. The var should remain `false` until we've got a plan for these types of exports, and there is no way for
users (either through ENV or UI) to change this, so the feature is effectively disabled.
* convert embed.ts to use createSlice from rtk
* only store in the state what we care about for interactive embedding
* use DEFAULT_EMBED_OPTIONS keys to filter searchOptions
* removed unused hash parameter
* adds basic unit test
There were two problems here:
- wrong args to `mdb/migrate!` (passing the db-type as well as the data
source), and
- in MySQL, the changelog table name is not `:databasechangelog`. Use
`metabase.db.liquibase/changelog-table-name` to get the actual table name.
Add a new query parameter to `/api/collection/:id/items` and
`/api/collection/root/items`. If `official_collections_first` is passed,
we'll sort official collections first, or not, as requested. If it isn't
passed, we'll default to `false` for the trash collection, or `true` for
normal collections.
* simplify bordered logic
* 🧹 setTitled
* 🧹 setHideDownloadButton
* 🧹 setHideParameters
* inline useEmbedDisplayOptions
* move DEFAULT_EMBED_DISPLAY_OPTIONS to costants file so we can remove use-embed-display-options.ts
* move consts up in the file
* update deps of useEffect
* fix ts error in the sdk
* use useEmbedFrameOptions in useDashboardUrlParams
* embed frame should not touch the `embed` slice, which is used for interactive embedding
* funnel start color should be text-dark
* change pivot table font size unit to em for scaling
* change funnel and legend item unit to em
* change funnel font size to em
* change object detail font size to em
* change gauge font size to em
* add cartesian chart padding config
* update docs
* remove isEmbeddingSdk prop
* simulate the 0.875em base font size
* chart padding override should apply in query builder
* add cartesian padding to stories
* update readme docs
* fix collection browser option in readme docs
* fix pivot table font size scaling
* update loki
* update loki
* use em for smart scalar
* update chart default padding
* revert smart scalar font size scaling due to measurement
* dynamic font family and font size as defaults
* pivot table
* add more properties to sdk docs
* make FONT_SIZES constant local
* link to discussion on query builder extra spacing
* enforce that font is passed to getLeftHeaderWidths
* Update e2e test
* Update e2e test
* fix: populate param-fields for named fields on public dashboards
* Add name to public hydration
* Add tests
* Add tests
* Fix tests
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Ensures that minimum query durations are correctly displayed in the form post-save (#44249) and that all times can be used in the Schedule component (#44257)
* UserParameterValue transforms wrapped to properly escape string vals
The `json-in` and `json-out` functions used for toucan model transforms do not perform any string escaping logic. This
makes sense as we don't want to make assumptions about the shape of data flowing in/out of the db. But, this did mean
that for the User paramter values table, string values were causing an error to be logged. This doesn't break
anything, as the function will still correctly return the string, but it can clutter up logs.
So, this PR wraps all incoming values in a map and unwraps it for outgoing values.
* Add migration to wrap existing user param values with ::wrapper key
Since the in/out transform for UserParameterValues is updated, we need to migrate any existing values to have this wrapping.
* Simplify the solution to the problem.
Since we're already getting what we need from the json-in/json-out *except* that it's logging a parse error, we can
create a json-out that works the same way (tries to parse and returns the string as-is if it fails) without logging an
error at all.
* take away the arg and keywordize inside the json-out fn
* remove irrelevant comment
Signed-off-by: Adam James <adam.vermeer2@gmail.com>
* Add a little more test coverage for different value types
* faster test thanks to Dan!
* Add a comment about the data being tested
---------
Signed-off-by: Adam James <adam.vermeer2@gmail.com>
* XLSX Pivot Table Downloads have 'Native' Pivot Table
Addresses: #2473
The xlsx downloads work fine for regular tables. However, on Pivot Table downloads, the resulting file isn't pivoted
and is formatted somewhat strangely (it contains additional rows related to totals, has an extra 'pivot-grouping'
column).
Now, the downloaded xlsx file contains 2 sheets:
- the data sheet containing the question's rows unpivoted. If you were constructing a pivot table yourself, you would
probably start with this shape of data.
- the pivot sheet containing a 'native' Pivot table whose reference points to the data on the data sheet, and whose
columns, rows, and values align with the cols, rows, and measures set up in Metabase
* Silly typo!
* Exported Pivot uses correct aggregation (sum, count, stddev, avg, min, max)
* data for pivot exports now also uses the formatting
* Add a test asserting that a pivot table exists in the xlsx file
* add-row! is a method dispatching on sheet class
* remove unnecessary hint
* address review feedback:
- consolidate the aggregation function key fns into one function in the .xlsx namespace
this was moved out of postprocessing so that the xlsx specific fns don't litter the post processing namespace at
this time
- clean up an unnecessary destructiring inside the native-pivot function
- don't shadow 'name' inside the body of a fn
* Add tests that confirm zero-col and zero-row situations work
* Consolidate the atoms used to store pivot rows/options
* Expose "Include this {period}" in timeseries chrome
* Fix type errors
* Simplify type guards
* Add unit tests
* Remove Flex wrapper around Switch
* Extract `IncludeCurrentSwitch` component
* Add E2E smoke tests
* Cover the toggle behavior with E2E tests
* Simplify test with a simple helper
* Temp
* Add support for saved questions
* Try to fix question not found issue
* Try to fix question not found issue
* Fix unit tests
* Add basic unit test
* Fix unit tests
* Fix unit tests, fix question not found issue
* Review fixes
* Review fixes
* Refactor components structure, review fixes
* Adjust readme
Fixes#43993 for real.
The earlier fix#44182 was needlessly restricting the
"nominal refs" check to fields which were using numeric IDs,
when really any match on nominal refs is valid.
* automatically add milestones to prs and issues
* test code
* test code
* test commit (#3138)
* test commit (#43320)
* test commit (#44357)
* remove test code
* add newlines
* prefer older version milestones
* Upgrade @reduxjs/toolkit from ^1.9.3 to ^2.2.5
* Fix typing
* Upgrade redux-actions and @types/redux-actions
* Fix type errors
* Fix type errors
* Upgrade react-redux from ^8.0.5 to ^9.1.2
- remove @types/react-redux as it's no-longer-needed
* Upgrade redux-promise from ^0.5.0 to ^0.6.0
- upgrade @types/redux-promise
* Remove unused @types/redux-logger
* Revert "Upgrade redux-actions and @types/redux-actions"
This reverts commit 65005a0053.
* Remove createWeakSelector as weakMapMemoize is the default memoization strategy now
* Do not call unsubscribe if it does not exist
* Do not subscribe
* Remove console.log
* Make DelayedLoadingSpinner always render something with data-testid="spinner" so that tests can be aware of when things are loading
* Fix JoinStep unit tests
* Fix AddToDashSelectDashModal unit tests
* Fix SegmentApp unit tests
* Bring back original subscription logic
* Fix DataUsageStep unit tests
* Fix ManageApiKeys unit tests
* Simplify comment
* Fix MainNavbar unit tests
* Fix HomeRecentSection unit tests
* Use afterEach instead of afterAll to reset mocks
* Remove explanatory variable
* Pivot Tables with no Pivot Columns should still download
Fixes#44159
Pivot Tables can still be valid without columns, and therefore the download should respect such a query and be
successfully exported.
In this PR, the postprocessing of pivot results into 'visual pivot' exports for csv takes into account the zero-column
configuration and the download no longer fails.
* add a test to confirm that zero cols and >1 measures works as well
* Handle the case where there are zero pivot-rows.
This is a similar problem to the zero pivot-cols case, just with rows.
The app actually doesn't render these tables correctly, but that's a separate frontend bug
* Convert drill-through (CLJ) values to JS
* Convert the drill-through `:null` to an actual `null`
Resolves#44232
* Use new function in `filter-drill-details`
* display-info->js should check for nil before seqable?
* fix js object comparison
* Add test for the `drill-value->js` function
* Simplify the test
* Fix the JS object equality in the test
---------
Co-authored-by: Alexander Solovyov <alexander@solovyov.net>
* Change run to make-run in process-query-for-card
Fixes#44160
There were two levers where a card could change how it was run.
`qp` this is basically `qp/process-query` with some optional userland
flags and constraints marked on the passed in query.
In the case of pivot tables in the
`/pivot/:card-id/query` endpoints it was `qp.pivot/process-query`
'run' this defaults to a streaming response that takes the `qp` param
and `export-format` and passes a query to qp. Most often this is
overriden to not stream the response.
Unfortunately, if `run` is overwritten, then `qp` is thrown away, and
the caller who overrides `run` must determine the correct `qp` to use.
However, when running `process-query-for-dashcard` it is difficult to properly set
`qp` for all of the dashcards without doing card lookups. So [this commit](https://github.com/metabase/metabase/blob/release-x.50.x/src/metabase/query_processor/card.clj#L230-L232) did a check to swap the runner in the presence of a pivot query that calls the **default** runner with a `qp.pivot/run-pivot-query`. The problem is that pulse needs to use a non-default runner, but there's no way for `process-query-for-card` to do both.
This commit changes `run` to `make-run` which raises the abstraction to
a function that produces a runer given a `qp`. This allows
`process-query-for-card` to change the `qp` as needed while keeping the
passed in runner.
There's really three things going on that could be teased apart
further/better.
1. `qp` - this conflates the processor (`qp/process-query` `qp.pivot/run-pivot-query`)
and modifying query (`qp/userland-query`,
`qp/userland-query-with-default-constraints` `(update query :info merge info)`)
2. `make-run` - How the above are processed: (streaming/realized)
* Fix tests
* Elevate include this period in the relative date picker (#44179)
* Mock new row with "include {period}" string
* Implement "include this" using a toggle
* Elevate "starting from" option in the relative date picker (#44242)
* Elevate "Starting from" option in the filter picker
* Tweak the date offset close icon
* Fix tests
* fix charts crash in safari when dataset timezone is not in iana format
* Update Loki Snapshots
* better fix: apply offset instead of finding a timezone with such offset
* remove unused specs
* Update Loki Snapshots
* review
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Fix a potential bug in `hydrate-can-restore`
I had a bug here. I wanted to hydrate `:collection` on the items and
then return the original items, but instead I was `dissoc`ing
`:collection` from the items. If the original items passed in already
had `:collection` keys this would break them.
AFAIK this didn't cause any issues, but it could have - so fixing it here.
* Add a test
* Present collection items in the collection
When fetching collection items, we need to fetch the true
`collection_id` in order to figure out things like `can_restore` and
`can_delete`, but set it to the displayed/effective `collection_id` when
we actually send it out.
This fixes a bug where collections had their `collection_id` match their
`id` when returned by this endpoint.
Previously we were just adding this when the item was archived. Instead,
always send a value - if the item is not archived, it can't be restored,
so it will be `false` in this case.
Making this change will make fixing a frontend bug easier, because the
frontend will be able to just trust the `can_restore` value coming in
from the backend.
* Use MultiAutocomplete in dashboard filters
* Handle prefix on MultiAutocomplete
* Remove unnecessary tests for empty values
* Remove input className
* Handle numbers in MultiAutocomplete
* Accept number values in MultiAutocomplete
* Use MultiAutocomplete in NumberInputWidget
* Filter out empty items
* Do not expect formatted number in values
* Do not allow adding multiple values when multi is false
* Handle undefined values
* Make value optional
* Fix MultiAutocomplete types
* Make MultiAutoComplete generic in its value type
* Refine MultiAutocomplete type in NumberInputWidget
* Refine MultiAutocomplete type in FieldValuesWidget
* Fix typo
* Fix MultiAutocomplete tests
* Add helpers to work with MultiSelect and MultiAutocomplete inputs
* Handle MultiAutocomplete input in tests
* Fix filter helper
* Use multiAutocompleteInput helper
* Find the value remove button in test
* Fix tests for NumberInputWidget
* Fix tests for DefaultPicker
* Fix type in NumberInput
* Fix dasboard test
* Fix embedding test
* Add helpers to remove values
* Fix chained filter test
* Fix nested test
* Fix embedding tests
* Fix filter tests
* Use removeMultiAutocompleteValue
* Use removeMultiAutocompleteValue
* Remove unused variable
* Use multiAutocompleteInput helper
* Handle comma's in values
* Handle multiselect
* Fix filter test
* Add filter
* Format field value
* Fix embedding tests
* Allow rendering custom label for item
* Use renderValue in FieldValuesWidget
* Fix data prop type
* Be more explicit about broken types in FieldValuesWidget
* Fix default behaviour for label
* Fix embedding tests
* Force click
* Fix sharing test
* Use removeMultiAutocompleteValue
* Handle multiautocomplete inputs
* Force click filter
* Close the suggestions menu
* Fix email test
* Fix sharing filter
* Set viewport in test
* Fix sharing test
* Remove debugging logs
* Use removeMultiAutocompleteValue
* Close the suggestions menu using blur
* Remove commented out code
* Remove the unused color prop
* Move shared types to types file
* Rename ValueType to TValue
* Add missing forwardRef
* Add tests for filter and renderValue
* Use TokenField for single value parameters
* Fix single value test
* Be more specific about when to select a single input
* Fix single value e2e test
* Handle undefined parameter
* Accept all RowValue types in MultiSelect
* Accept all RowValue types in ListField
* Accept all RowValue types in SingleSelectListField
* Be more precise about the FieldValuesWidget value types
* Be more specific about when to render normal input
* Remove duplicate import
* Remove maxSelectedValues from MultiAutocomplete
* Parse values in FieldValuesWidget
* Remove parsing complexity from MultiAutocomplete
* Be more strict when parsing numbers
* Do not use multiselect when arity is 1
* Remove spurious changes
* Pass prefix from FieldValuesWidget
* Remove number MultiAutocomplete story
* Remove renderValue from MultiAutocomplete
* Add custom component to MultiAutocomplete
* Add missing data prop
* Remove spurious changes in MultiAutocomplete
* Remove SelectItem reference
* Use renderValue in FieldValuesWidget
This PR removes:
- a warning about missing React keys in the `Schedule` component
- a warning about invalid props in the `StrategyFormLauncher` component
* adds a shimmer effect to the skeletons
* adds a `natural` prop that makes the skeletons have a random width between 50% and 80%
* adds a skeleton to Browse models
Closes#44111
* expose card title settings
* Add dashboard test skeleton
* add a sample dashcard to the test
* dashboard grid should have card title by default
* add unit test for card title
* remove unused delay in unit test
* fix width and height being smaller than min values
* flip grid layout condition
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Handle errors when inferring separator of csv
This file
```
"c1","c2"
"a,b,c","d"
```
was failing to upload for a silly reason: when we were checking for
which separator it used, we had a parse error when using a semi-colon
and that blew up the whole pipeline.
* Use cal's helpful test suggestions
Although a user may have the necessary permissions to delete an item, if we're not going to allow deleting unarchived items, can_delete should be false for them too.
* [Metrics V2] Metrics with filters on implicitly joined columns produce query error when added in a Summarize block
* Handle multiple metrics with same implicit join
* Address PR feedback
* adds the endpoint and a test
* add recents endpoint, todo: tests
* add post recents endpoint
* return recent views for both lists, but make the endpoint work
* Make recent-views context aware
- pruning is context aware, only checks for the recently-inserted
context
- Adds endpoints:
- POST to create selection recents,
- GET activity/recents
- requres context query param to be one of:
- all, views, selections
- Adds context arg to update-users-recent-views!
- Cleans up arg schema for update-users-recent-views
* impl GET api/activity/recents
- return recent-selections and recent-views from
- send context == collection from pinned card reads
* update callsites of recent-views/update-users-recent-views!
- to use :view param where necessary
* fixes models/recent-view tests
* adds more activity/recent-view tests
* wip
- fix whitespace linter
* Fix command palette e2e test
- reuse util snake-keys
* updates fe to use new recents endpoints
* fixes fe type issue
* snake-keys -> deep-snake-keys
- I've been betrayed by lsp rename
* snake-keys -> deep-snake-keys
- I've been betrayed by lsp rename
* log selection events for created collections
* mysql doesn't allow default values for TEXT types
* log a recent view on data-picker selection
* decouple view-log context from card-event view context
* fix a doc typo
* stop double logging recent-views on POST
* maybe fixes some tests
* some e2e fixes
* fix mysterious divide by zero during score search
* fix divide by zero possibilities everywhere in score-items
metabase.api.activity/score-items used to throw when there weren't any
items being scored (even though there's a `(when (seq items) ...)` check)
* more test fixes
* fix more e2e tests, + rename endpoint in tests
* fix oopsie
* fixes a few more tests
* address review comments/questions
* allow for a comma delimited list in qps like ?context=views,selections
returns all recent view items with those contexts, most recent first
under the `:recents` key.
* refactors FE around new endpoint
* fixes for unit tests
* use ms/QueryVectorOf for context
* fix models/recent_views tests
* use multiple query params instead of comma delimited value on FE
* ignore timestamp when deduping recents
* review comments + test fixing
* update docstring
* fix api/activity_test s
* actually dedupe
* add test for deduping by context
* e2e fix: shows up-to-date list of recently viewed items after another page is visited
* e2e fix: should undo the question replace action
* e2e fix: should replace a dashboard card question (metabase#36984)
* e2e fix: should preselect the most recently visited dashboard
* fix 6 more e2e tests
* fixes fe type check and unit failure
* renames unit test mocking function
* fix a flaky e2e test
* widen Item to accept str or kw where sensible
- allow strings or keywords for moderated_status, and authority_level
* simplify impl + add test
* add view-log to events schema
* add collection context to view log
* fix the final 2 failing e2e tests
* click dashboard tab when the user has can-read? on recent entities
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* Improve the Trash
Ok, so I had a realization at the PERFECT time, immediately after the RC
cutoff. Great job, brain!
Here's the realization. For the Trash, we need to keep track of two
things:
- where the item actually is located in the hierarchy, and
- what collection we should look at to see what permissions apply to the
item.
For example, a Card might be in the Trash, but we need to look at
Collection 1234 to see that a user has permission to Write that card.
My implementation of this was to add a column,
`trashed_from_collection_id`, so that we could move a Card or a
Dashboard to a new `collection_id`, but keep track of the permissions we
actually needed to check.
So:
- `collection_id` was where the item was located in the collection
hierarchy, and
- `trashed_from_collection_id` was where we needed to look to check
permissions.
Today I had the realization that it's much, much more important to get
PERMISSIONS right than to get collection hierarchy right. Like if we
mess up and show something as in the Trash when it's not in the Trash,
or show something in the wrong Collection - that's not great, sure. But
if we mess up and show a Card when we shouldn't, or show a Dashboard
when we shouldn't, that's Super Duper Bad.
So the problem with my initial implementation was that we needed to
change everywhere that checked permissions, to make sure they checked
BOTH `trashed_from_collection_id` and `collection_id` as appropriate.
So... there's a much better solution. Instead of adding a column to
represent the *permissions* that we should apply to the dashboard or
card, add a column to represent the *location in the hierarchy* that
should apply to the dashboard or the card.
We can simplify further: the *only time* we want to display something in
a different place in the hierarchy than usual is when it was put
directly into the trash. If you trash a dashboard as a part of a
collection, then we should display it in that collection just like
normal.
So, we can do the following:
- add a `trashed_directly` column to Cards and Dashboards, representing
whether they should be displayed in the Trash instead of their actual
parent collection
- use the `collection_id` column of Cards and Dashboards without
modification to represent permissions.
There's one main downside of this approach. If you trash a dashboard,
and then delete the collection that the dashboard was originally in,
what do we do with `dashboard.collection_id`?
- we have to change it, because it's a foreign key
- we can't set it to null, because that represents the root collection
In this initial implementation, I've just cascaded the delete: if you
delete a dashboard and then delete a collection, the dashboard will be
deleted. This is not ideal. I'm not totally sure what we should do in
this situation.
* Rip out all the `trashed_from_collection_id`
* Migration to delete trashed_from_collection_id
* fixes
* don't move collections
And don't allow deleting collections
* only show cards/dashboards with write perms
* Show the correct archived/unarchived branch
* some cleanup
* add a todo for tomorrow
* Fix for yesterday's TODO
* more wip
* refactor
* memoize collection info
* move around memoization a bit
* fix schema migration test
* oops, delete server.middleware.memo
* Use a migration script (postgres only for now)
* Fix some tests
* remove n+1 queries in `collection-can-restore`
* fix test
* fix more tests, and x-db migration script
* fix h2 rollback
* fix mysql/mariadb migration
* lint
* fix some mariadb/mysql tests
* fix h2 rollback
* Fix mysql rollback
* Fix Postgres migration
* "Real" `trash_operation_id` UUIDs from migration
* fix mariadb migration
* Separate MySQL/MariaDB migrations
* trashed directly bit->boolean
* Remove `trashed_from_*` from migrations
* Rename `api/updates-with-trashed-directly`
Previously named `move-on-archive-or-unarchive`, which was no longer
accurate since we're not moving anything!
* Add `can_delete`
* Delete test of deleted code
* Can't move anything to the Trash
The Trash exists as a real collection so that we can list items in and
such without needing special cases. But we don't want anything to
_actually_ get moved to it.
* integrates can_delete flag on FE
* Update src/metabase/models/collection.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update src/metabase/search/impl.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Better name for `fix-collection-id`
Also consolidated it into a single function. It takes the
trash-collection-id as an argument to prevent a circular dependency.
* s/trashed/archived/ on the backend
In the product, we want to move to "trashed" as the descriptor for
things that are currently-known-as-archived.
It'd be nice to switch to that on the backend as well, but that'll
require quite a lot of churn on the frontend as we'd need to change the
API (or have an awkward translation layer, where we call it `trashed` on
the backend, turn it into `archived` when sending data out via the API,
and then the FE turns around and called it "trashed" again).
For now, let's just be consistent on the backend and call it `archived`
everywhere. So:
- `archived_directly` instead of `trashed_directly`, and
- `archive_operation_id` instead of `trash_operation_id`
* Fix up a couple docstrings
* Rename `visible-collection-ids` args
for `collection/permissions-set->visible-collection-ids`:
- `:include-archived` => `:include-archived-items`
- `:include-trash?` => `:include-trash-collection`
* select affected-collection-ids in the tx
* Stop dealing with `:all` visible collection ids
This used to be a possible return value for
`permissions-set->visible-collection-ids` indicating that the user could
view all collections. Now that function always returns a set of
collection IDs (possibly including "root" for the root collection) so we
don't need to deal with that anymore.
* Don't use separate hydration keys for collections
Dispatch on the model of each item (but still keep them in batches for
efficiency.)
* vars for collectable/archived-directly models
* Round up loose `trashed_directly`s
* Fix a test
* Use `clojure.core.memoize` instead of handrolled
It's slightly different (TTL instead of exactly per-request) but that
should be fine.
* FE lint fixes
* 😭 e2e test fix
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* add option to customize dashcard border
* use the shadow CSS variable
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* make the shadow and border decoration call more explicit
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* pass in theme as argument
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Allow picking arity of number parameters
* Limit to input to have arity 1 when multi is false
* Use MultiAutocomplete in NumberInputWidget when arity is 1
* Add test cases for single value numbers
* Only change arity when it is n
* Fix unit tests for NumberInputWidget
* Move arity check for numbers to getNumberParameterArity
* Fix the typo
* Replace `realHover` with the `mouseMove` trigger
* Be explicit when targetting elements by their index
* fix combo chart tooltip e2e specs
* cy.wait as a last resort
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
* Improve typing in Question
* Remove dead CSS
* Improve typing in getUrlWithParameters
* Add LegendLabel
* Drop navigateToNewCardFromDashboard in PublicOrEmbeddedDashboard
* Extract getNewCardUrl and move getParametersMappedToDashcard closer to it
* Add link to dashboard chart title
* Update title-drill tests
* Format code
* Update test
* Add HrefMemo
* Use state to memoize href
* Recompute state on getHref prop change
* Format code
* Refactor href computation
* Update test
* Use selector instead of useMemo
* Fix unit tests
* Ensure type safety
* Revert getDashCardById type change to not make this PR too large
* Use complete dashboard
* Add a workaround for metabase#43990
* Remind embedding SDK PR author to use conventional commits
* Only remind PR author when PR title isn't conventional commits
* Add branch for testing
* Fix gh cli command
* Fix getting wrong user login
* Ensure we don't create more than 1 comment
* Improve comment format
* Add more info for non-changelog related type
* Check non-changelog related types
* Update the comment for consistency
* Revert "Add branch for testing"
This reverts commit 4a1be68828.
* Migrate text-brand
* Migrate text-dark
* Migrate text-light
* Migrate text-medium
* Migrate admin-navbar
* make compact formatting consistent across all series (#43472)
* make compact formatting consistent across all series
* fix rebase
* [Browse] Show Browse models link in nav only when there are models (#43286)
* Upgrade to TypeScript v5.4 (#43736)
* use correct github api for CI (#43750)
* update list of releases (#43754)
* Multi Series Dashcards Render with Filters Applied To All Series (#43428)
* Multi Series Dashcards Render with Filters Applied To All Series
Fixes: #39083
Multi-series dashcards now properly render when a subscription is set up with filters that impact the values of each
card in the multi-series card.
* Small refactor for executing dashcards in a pulse context.
- move execute-dashboard-subscription to pulse.util
- this fn now will also execute all card series cards
and put those results into a list on the key :series-results in the dashcard
which is then used in the render multimethod to send the correct data to the javascript context
A small todo was taken care of as well, we can now use :dashboard-subscription in the :context key
* Fix up render fn
The shape of hte data going into the javascript was a bit unclear before. This keeps things a little more
understandable.
Next step is probably to create some Malli schemas inside the js_svg namespace to document and enforce the needed data shapes
* pull :data up a level from :result in the series cards
* Update src/metabase/pulse/render/body.clj
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Remove unneeded let
* set/rename-keys won't work here because we raise the :data key
I've realised a suggested change to just use set/rename-keys won't do what we need. We need instead to raise :data
from within the result key up to the same level (and dissoc result).
I've renamed the helper function to make its purpose a bit clearer
---------
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Hydrate can_run_adhoc_query and address review comments (#43747)
This is a followup on #43625.
* Consolidate dashboard-card specs (#43716)
* Consolidate joins specs (#43717)
* Update translations for v50 and add Danish Language Support (#43314)
* update translations for v50
* add Danish
* update translations for v50 release
* alphabetical order is nice
* Move database role to url parameters from connection properties on Snowflake (#43602)
* Move role to url parameters from connection properties
* Fix test
* Fix width of save button in cache sidebar (#43755)
* Remove dead code (#43603)
* Combine search e2e specs (#43718)
* Highlight the table cell data in dashboards when the cell is an ID column (#43728)
* highlight id cell column
* add simpletable stories
* add simple table test data
* add loki reference image
* Update frontend/src/metabase/visualizations/components/TableSimple/TableCell.styled.tsx
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Remove old repro for 39150 (#43749)
* Fix remaining `tinyint` booleans in MySQL (#43296)
* Correct type for `report_card.dataset`
This was missed in the original migration of boolean types to
${boolean.type} in MySQL/MariaDB, and then missed again by me when I
migrated `collection_preview` over a week ago.
* Change all boolean types to `bit(1)` in MySQL
Liquibase changed their boolean type in MySQL from `bit(1)` to
`tinyint(4)` in version 4.25.1. Our JDBC driver does not recognize these
as booleans, so we needed to migrate them to `bit(1)`s.
As discussed [here](#36964), we
changed all existing `boolean` types that were in the
`001_update_migrations.yml` but not the SQL initialization file.
For new installations, this works: things in the SQL initialization file
get created with the `bit(1)` type.
However, for existing installations, there's a potential issue. Say I'm
on v42 and am upgrading to v49. In v43, a new `boolean` was added.
In this case, I'll get the `boolean` from the liquibase migration rather
than from the SQL initialization file, and it need to be changed to a
`bit(1)`.
I installed Metabase v41 with MySQL, migrated the database, and then
installed Metabase v49 and migrated again. I made a list of all the
columns that had the type `tinyint`:
```
mysql> SELECT TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME, COLUMN_TYPE, COLUMN_DEFAULT, IS_NULLABLE FROM INFORMATION_SCHEMA.COLUMNS WHERE COLUMN_TYPE = 'tinyint' AND TABLE_SCHEMA='metabase_test';
+---------------+------------------------------+-------------------+-------------+----------------+-------------+
| TABLE_SCHEMA | TABLE_NAME | COLUMN_NAME | COLUMN_TYPE | COLUMN_DEFAULT | IS_NULLABLE |
+---------------+------------------------------+-------------------+-------------+----------------+-------------+
| metabase_test | core_user | is_datasetnewb | tinyint | 1 | NO |
| metabase_test | metabase_field | database_required | tinyint | 0 | NO |
| metabase_test | metabase_fieldvalues | has_more_values | tinyint | 0 | YES |
| metabase_test | permissions_group_membership | is_group_manager | tinyint | 0 | NO |
| metabase_test | persisted_info | active | tinyint | 0 | NO |
| metabase_test | report_card | dataset | tinyint | 0 | NO |
| metabase_test | timeline | archived | tinyint | 0 | NO |
| metabase_test | timeline | default | tinyint | 0 | NO |
| metabase_test | timeline_event | archived | tinyint | 0 | NO |
| metabase_test | timeline_event | time_matters | tinyint | NULL | NO |
+---------------+------------------------------+-------------------+-------------+----------------+-------------+
10 rows in set (0.01 sec)
```
Then wrote migrations. For each column, we:
- turn it into a `bit(1)`,
- re-set the previously existing default value, and
- re-add the NOT NULL constraint, if applicable.
* Change author and add missing `dbms`
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* Add ts-pattern library (#43752)
* Add events to measure search performance (#43135)
* new_search_query event
* Filters in object, search click event
* updating event schema
* type and event schema adjustments
* delete comments BE code
* adjusting tests
* PR feedback
* typos and spacing
* Fixup missing colors
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
Co-authored-by: adam-james <21064735+adam-james-v@users.noreply.github.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: SakuragiYoshimasa <ysrhsp@outlook.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Nick Fitzpatrick <nick@metabase.com>
* Add repro test
* Move home selectors to `metabase/selectors/app`
* Prevent unnecessary redirects to home dashboard
* Rename
* Handle navbar home link
* Fix unit tests
* Fix navigating away from an archived dashboard
* Handle sub-root Metabase URLs
* Sort functions
* Add boilerplate for aggregateOffset and tests
* Simplify assertion
* Make basic case work
* Handle name generation
* Do not require "lib/uuid" attribute to be present since MLv2 will normalize it under the hood anyway
* Fix typing
* Group tests
* Refactor offsetClause to return a new clause instead of a new query
- Move offsetClause to expressions.ts
- Add assertion
* Revert "Sort functions"
This reverts commit ab9ce2b24e.
* Move tests
* Handle names dynamically
* Shorten a test
* Update comment
* Add TODO
* Update expression types
* Add diffOffsetClause & percentDiffOffsetClause
* Add a test for diffOffsetClause
* Add TODOs
* Add tests for diffOffsetClause and percentDiffOffsetClause
* Unwrap tests
* Add skeleton for new tests
* Refactor first offsetClause tests
* Add tests for names
* Fix case of non-datetime columns
* Handle a case with offset < 1 and no breakouts
* Handle a case with offset < 1 and breakouts on non-datetime column
* Handle a case with offset < 1 and breakouts binned datetime column
* Handle a case with offset < 1 and breakout on non-binned datetime column
* Refactor
* Refactor
* Remove TODO
* Add tests for diffOffsetClause
* Add tests for percentDiffOffsetClause
* Move offset stuff to offset.ts and offset.unit.spec.ts
* Use template string for prefixes
* Use breakoutColumn + isDate to check column type
* Refactor
* Fix error message
* Add boilerplate for CompareAggregations
* Fix title
* Render aggregations list
* Style AccordionList
* Sort props
* Fix bucket name
* Use displayName instead shortName
* Support parseValue prop in NumberInput
* Add period input accepting integers only
* Accept non-negative values only
* Do not accept zeros
* Add state
* Add submit button
* Export offset functions via Lib
* Make it possible to skip rightSection
* Add column picker
* Map offset user input to api input (negative value)
* Add label
* Fix crash
* Extract shouldCreate
* Make onSelect in AggregationPicker support multiple values
* Extract ReferenceAggregationPicker
* Extract ColumnPicker
* Extract getAggregations
* Rename
* Add custom items
* Refactor item component
* Extract OffsetInput
* Remove unused data-testid
* Style OffsetInput
* Generate titles according to specs
* Generate label
* Generate help
* Extract utils
* Use different width for the 1st step
* Format code
* Use MultiSelect directly
* Avoid custom parseValue
* Revert MultiaAutocomplete changes
* Improve typing in describeTemporalInterval and
describeRelativeDatetime
* Use describeTemporalUnit to pluralize
* Use interface
* Avoid setting value as DOM attribute
* Fix test
* Add onAdd prop to AggregationPicker and revert the change to have onSelect pass multiple aggregations
* Reduce number of props
* Render checkboxes in custom items
* Introduce and use --mb-color-brand-lighter
* Avoid !important
* Remove redundant prop
* Rename ExtractColumn to ExtractColumnAction
* Add lines.svg
* Fix positioning in case dir attribute is not present in any parent component
* Add CompareAggregationsAction
* Fix warning about isSelected being used as DOM attribute
* Fix positioning in case dir attribute is not present in any parent component
* Remove redundant Box
* Compute the title
* Add compareAggregationsDrill boilerplate
* Add compare-aggregations drill stub
* Update comment
* Compute title and aggregation
* Add TODO
* Use proper title when there's only 1 aggregation
* Show visualization on submit
* Limit aggregation choice to 1 in compare aggregations drill
* Redirect to chart upon submission
* Do not navigate to chart view
* Do not navigate to chart view
* Make drill-thru tests expect the new compare-aggregations drill
* Add type attribute to all ListItems
* Add aggregationIndex to drillInfo
* Add aggregationDrillDetails and wire it up
* Add tracking helpers
* Add new schema
* Add tracking in column header
* Add tracking in plus modal
* Add tracking in aggregation picker
* Fix destructuring
* Add functionsUsedByExpression
* Fix schema number
* Add tests boilerplate
* Implement test for notebook editor
* Add test for column header
* Add a test for summarize sidebar
* Add a test for the plus button
* Add expectNoBadSnowplowEvents call in afterEach
* Format code
* Add missing redux store state in tests
* Add data for tests
* Move createQuestion call out of beforeEach
* Add tests for labels when there are no aggregations
* Add describe
* Add group helper
* Revert "Add group helper"
This reverts commit e047a223e9.
* Optimize: use cy.log() instead of it()
* Format code
* Add tests for 1 breakout, add helpers
* Add test for non-binned datetime column text
* Add test for non-datetime column text
* Nest tests
* Add test for multiple aggregations and no breakout, fix getTitle in this case
* Add all text tests
* Reduce nesting
* Enable test
* Format code
* Add tests
* Rename function
* Format code
* Simplify code
* Add assertions
* Format code
* Add column assertions
* Run tests with all 3 aggregations
* Remove logs
* Introduce CUSTOM_EXPRESSIONS_USED
* Avoid force click
* Remove explicit assertion
* Use be.visible instead of exist
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
I'm not sure it's any less hacky in the CLJS but it keeps the TS
wrappers thin.
Context:
- #43799 is the original issue
- #43848 added the hack being removed here
This PR simplifies a bit of text used in the cache cron schedule component: we can say "spaces" (which is normal English) not "white spaces" (which is developerese)
* Automate SDK doc PR
* Prevent auto merge
* Dry run npm publish for testing
* Speed up testing by not building jars
* Fix auto approve step
* disable concurrency so I could test multiple runs in parallel
* Test auto-merge
* Fix auto merge
* Revert "disable concurrency so I could test multiple runs in parallel"
This reverts commit 60e90076eb.
* Revert "Speed up testing by not building jars"
This reverts commit 4e53a54414.
* Revert "Dry run npm publish for testing"
This reverts commit 9b5d761c8d.
* Enable all commented out steps
* Remove unnecessary steps
* Mock that the changelog is different from the one in master
* Apply changelog patch incase the changelog is created from another branch
* Dry run npm publish so I could test
* Fix failed patch
* Speed up testing by not building jar
* Fix adding new changelog to master
* feat(sdk): A placeholder for an actual feature in the changelog
* Mock that the changelog diverges from master
* Revert "Speed up testing by not building jar"
This reverts commit acb80d7465.
* Revert "Dry run npm publish so I could test"
This reverts commit f4e679efd9.
* Revert "Mock that the changelog diverges from master"
This reverts commit 0e77a68e1b.
* dashcard background color
* add bg-dashboard and bg-dashboard-card
* rename background-visualization to background-question
* apply bg-question to embedded questions
* use CSS variables along with theme object
* add css variables to more places
* add ability to reference design system colors for defaults
* use CSS variables for defaults
* fix chart skeleton test
* fix legend item title
* Move expressionClause to handleSubmit
* Allow passing the first column as a prop and allow auto picking of columns
* Use CombineColumns component in header drills
* Remove duplicate CombineColumnsDrill component and helpers
* Allow separators to be null
* Calculate example in Example component to avoid useMemo call
* Fix test reference in test
* Append suffix to duplicate columns
* Rename parameter to something more expressive
* Enable autoPickColumn for all instances of the combine column popover
* Fix tests
* Fix notebook tests
* Make test typescript
* Correct types of openTable function
* Add a helper to extract linked issues from PR body
* Add tests
* Tweak the workflow
* Fix the workflow
* Delete obsolete code
* Improve the message
* Fix whitespace error
* Correct type for `report_card.dataset`
This was missed in the original migration of boolean types to
${boolean.type} in MySQL/MariaDB, and then missed again by me when I
migrated `collection_preview` over a week ago.
* Change all boolean types to `bit(1)` in MySQL
Liquibase changed their boolean type in MySQL from `bit(1)` to
`tinyint(4)` in version 4.25.1. Our JDBC driver does not recognize these
as booleans, so we needed to migrate them to `bit(1)`s.
As discussed [here](#36964), we
changed all existing `boolean` types that were in the
`001_update_migrations.yml` but not the SQL initialization file.
For new installations, this works: things in the SQL initialization file
get created with the `bit(1)` type.
However, for existing installations, there's a potential issue. Say I'm
on v42 and am upgrading to v49. In v43, a new `boolean` was added.
In this case, I'll get the `boolean` from the liquibase migration rather
than from the SQL initialization file, and it need to be changed to a
`bit(1)`.
I installed Metabase v41 with MySQL, migrated the database, and then
installed Metabase v49 and migrated again. I made a list of all the
columns that had the type `tinyint`:
```
mysql> SELECT TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME, COLUMN_TYPE, COLUMN_DEFAULT, IS_NULLABLE FROM INFORMATION_SCHEMA.COLUMNS WHERE COLUMN_TYPE = 'tinyint' AND TABLE_SCHEMA='metabase_test';
+---------------+------------------------------+-------------------+-------------+----------------+-------------+
| TABLE_SCHEMA | TABLE_NAME | COLUMN_NAME | COLUMN_TYPE | COLUMN_DEFAULT | IS_NULLABLE |
+---------------+------------------------------+-------------------+-------------+----------------+-------------+
| metabase_test | core_user | is_datasetnewb | tinyint | 1 | NO |
| metabase_test | metabase_field | database_required | tinyint | 0 | NO |
| metabase_test | metabase_fieldvalues | has_more_values | tinyint | 0 | YES |
| metabase_test | permissions_group_membership | is_group_manager | tinyint | 0 | NO |
| metabase_test | persisted_info | active | tinyint | 0 | NO |
| metabase_test | report_card | dataset | tinyint | 0 | NO |
| metabase_test | timeline | archived | tinyint | 0 | NO |
| metabase_test | timeline | default | tinyint | 0 | NO |
| metabase_test | timeline_event | archived | tinyint | 0 | NO |
| metabase_test | timeline_event | time_matters | tinyint | NULL | NO |
+---------------+------------------------------+-------------------+-------------+----------------+-------------+
10 rows in set (0.01 sec)
```
Then wrote migrations. For each column, we:
- turn it into a `bit(1)`,
- re-set the previously existing default value, and
- re-add the NOT NULL constraint, if applicable.
* Change author and add missing `dbms`
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* Multi Series Dashcards Render with Filters Applied To All Series
Fixes: #39083
Multi-series dashcards now properly render when a subscription is set up with filters that impact the values of each
card in the multi-series card.
* Small refactor for executing dashcards in a pulse context.
- move execute-dashboard-subscription to pulse.util
- this fn now will also execute all card series cards
and put those results into a list on the key :series-results in the dashcard
which is then used in the render multimethod to send the correct data to the javascript context
A small todo was taken care of as well, we can now use :dashboard-subscription in the :context key
* Fix up render fn
The shape of hte data going into the javascript was a bit unclear before. This keeps things a little more
understandable.
Next step is probably to create some Malli schemas inside the js_svg namespace to document and enforce the needed data shapes
* pull :data up a level from :result in the series cards
* Update src/metabase/pulse/render/body.clj
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Remove unneeded let
* set/rename-keys won't work here because we raise the :data key
I've realised a suggested change to just use set/rename-keys won't do what we need. We need instead to raise :data
from within the result key up to the same level (and dissoc result).
I've renamed the helper function to make its purpose a bit clearer
---------
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Sort functions
* Add boilerplate for aggregateOffset and tests
* Simplify assertion
* Make basic case work
* Handle name generation
* Do not require "lib/uuid" attribute to be present since MLv2 will normalize it under the hood anyway
* Fix typing
* Group tests
* Refactor offsetClause to return a new clause instead of a new query
- Move offsetClause to expressions.ts
- Add assertion
* Revert "Sort functions"
This reverts commit ab9ce2b24e.
* Move tests
* Handle names dynamically
* Shorten a test
* Update comment
* Add TODO
* Update expression types
* Add diffOffsetClause & percentDiffOffsetClause
* Add a test for diffOffsetClause
* Add TODOs
* Add tests for diffOffsetClause and percentDiffOffsetClause
* Unwrap tests
* Add skeleton for new tests
* Refactor first offsetClause tests
* Add tests for names
* Fix case of non-datetime columns
* Handle a case with offset < 1 and no breakouts
* Handle a case with offset < 1 and breakouts on non-datetime column
* Handle a case with offset < 1 and breakouts binned datetime column
* Handle a case with offset < 1 and breakout on non-binned datetime column
* Refactor
* Refactor
* Remove TODO
* Add tests for diffOffsetClause
* Add tests for percentDiffOffsetClause
* Move offset stuff to offset.ts and offset.unit.spec.ts
* Use template string for prefixes
* Use breakoutColumn + isDate to check column type
* Refactor
* Fix error message
* Add boilerplate for CompareAggregations
* Fix title
* Render aggregations list
* Style AccordionList
* Sort props
* Fix bucket name
* Use displayName instead shortName
* Support parseValue prop in NumberInput
* Add period input accepting integers only
* Accept non-negative values only
* Do not accept zeros
* Add state
* Add submit button
* Export offset functions via Lib
* Make it possible to skip rightSection
* Add column picker
* Map offset user input to api input (negative value)
* Add label
* Fix crash
* Extract shouldCreate
* Make onSelect in AggregationPicker support multiple values
* Extract ReferenceAggregationPicker
* Extract ColumnPicker
* Extract getAggregations
* Rename
* Add custom items
* Refactor item component
* Extract OffsetInput
* Remove unused data-testid
* Style OffsetInput
* Generate titles according to specs
* Generate label
* Generate help
* Extract utils
* Use different width for the 1st step
* Format code
* Use MultiSelect directly
* Avoid custom parseValue
* Revert MultiaAutocomplete changes
* Improve typing in describeTemporalInterval and
describeRelativeDatetime
* Use describeTemporalUnit to pluralize
* Use interface
* Avoid setting value as DOM attribute
* Fix test
* Add onAdd prop to AggregationPicker and revert the change to have onSelect pass multiple aggregations
* Reduce number of props
* Render checkboxes in custom items
* Introduce and use --mb-color-brand-lighter
* Avoid !important
* Remove redundant prop
* Rename ExtractColumn to ExtractColumnAction
* Add lines.svg
* Fix positioning in case dir attribute is not present in any parent component
* Add CompareAggregationsAction
* Fix warning about isSelected being used as DOM attribute
* Fix positioning in case dir attribute is not present in any parent component
* Remove redundant Box
* Compute the title
* Add compareAggregationsDrill boilerplate
* Add compare-aggregations drill stub
* Update comment
* Compute title and aggregation
* Add TODO
* Use proper title when there's only 1 aggregation
* Show visualization on submit
* Limit aggregation choice to 1 in compare aggregations drill
* Redirect to chart upon submission
* Do not navigate to chart view
* Do not navigate to chart view
* Make drill-thru tests expect the new compare-aggregations drill
* Add type attribute to all ListItems
* Add aggregationIndex to drillInfo
* Add aggregationDrillDetails and wire it up
* Add tracking helpers
* Add new schema
* Add tracking in column header
* Add tracking in plus modal
* Add tracking in aggregation picker
* Fix destructuring
* Add functionsUsedByExpression
* Fix schema number
* Add tests boilerplate
* Implement test for notebook editor
* Add test for column header
* Add a test for summarize sidebar
* Add a test for the plus button
* Add expectNoBadSnowplowEvents call in afterEach
* Format code
* Add missing redux store state in tests
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Use createQueryWithClauses in createQueryWithCountAggregation
* Support column-specific aggregations in createQueryWithClauses
* Add tests for compare option presence
* Fix case
* Add a test for onAdd
* Split tests
* Rename tests
* Add boilerplate for CompareAggregations unit tests
* Fix assertion
* Add more tests
* Add offset input tests
* Add tests
* Update frontend/src/metabase/query_builder/components/CompareAggregations/CompareAggregations.unit.spec.tsx
* Update tests after merge
* Use constants instead of factory functions
* Check git tag for an exact match
* Set npm publish to dry run, so I can test
* Uses new github automation account for creating a PR
* Remove unnecessary step
* Automatically set `no-backport` on the SDK doc version bump PR
* Fix step to edit PR with correct permission
* Remove npm dry run after finished testing
* return empty string so non-empty labels do not get hidden by empty labels
* fix function name
* apply scaling in transformed dataset
* add missing option to OptionsType
* fix double scaling
* add column scaling to waterfall transformed dataset
* add column value scaling to dataset transform
* fix stacked bar segment labels not responding to column scaling
* fix tooltip values displayed
* since scaling as added to transformedDataset, use transformedDataset for formatters
* revert changes to display percentage instead of absolute value
* fix stacked data labels
* add loki variant for custom y-axis range with column scaling
* fix stack segmented label formatters compact calculation
* scale original dataset and use that instead of modifying transformedDataset
* update loki snapshots
* fix scatter plot dataset scaling
* fix tooltip values being double scaled
* add E2E test to validate behavior
* add helper for not duplicating the column scaling
* port multiple y-axis metrics with column scaling fixes to RowChart
* add waterfall and scatterplot loki variants
* fix type errors and failing unit tests
* fix failing unit tests
* update loki snapshots
* Add E2E reproduction for #39150
* Fix typo in the test title
* Move the viewport config to the `describe` block
* Add the second repro scenario as per the review comment
* Scope the variable to the relevant test
* Sort functions
* Add boilerplate for aggregateOffset and tests
* Simplify assertion
* Make basic case work
* Handle name generation
* Do not require "lib/uuid" attribute to be present since MLv2 will normalize it under the hood anyway
* Fix typing
* Group tests
* Refactor offsetClause to return a new clause instead of a new query
- Move offsetClause to expressions.ts
- Add assertion
* Revert "Sort functions"
This reverts commit ab9ce2b24e.
* Move tests
* Handle names dynamically
* Shorten a test
* Update comment
* Add TODO
* Update expression types
* Add diffOffsetClause & percentDiffOffsetClause
* Add a test for diffOffsetClause
* Add TODOs
* Add tests for diffOffsetClause and percentDiffOffsetClause
* Unwrap tests
* Add skeleton for new tests
* Refactor first offsetClause tests
* Add tests for names
* Fix case of non-datetime columns
* Handle a case with offset < 1 and no breakouts
* Handle a case with offset < 1 and breakouts on non-datetime column
* Handle a case with offset < 1 and breakouts binned datetime column
* Handle a case with offset < 1 and breakout on non-binned datetime column
* Refactor
* Refactor
* Remove TODO
* Add tests for diffOffsetClause
* Add tests for percentDiffOffsetClause
* Move offset stuff to offset.ts and offset.unit.spec.ts
* Use template string for prefixes
* Use breakoutColumn + isDate to check column type
* Refactor
* Fix error message
* Add boilerplate for CompareAggregations
* Fix title
* Render aggregations list
* Style AccordionList
* Sort props
* Fix bucket name
* Use displayName instead shortName
* Support parseValue prop in NumberInput
* Add period input accepting integers only
* Accept non-negative values only
* Do not accept zeros
* Add state
* Add submit button
* Export offset functions via Lib
* Make it possible to skip rightSection
* Add column picker
* Map offset user input to api input (negative value)
* Add label
* Fix crash
* Extract shouldCreate
* Make onSelect in AggregationPicker support multiple values
* Extract ReferenceAggregationPicker
* Extract ColumnPicker
* Extract getAggregations
* Rename
* Add custom items
* Refactor item component
* Extract OffsetInput
* Remove unused data-testid
* Style OffsetInput
* Generate titles according to specs
* Generate label
* Generate help
* Extract utils
* Use different width for the 1st step
* Format code
* Use MultiSelect directly
* Avoid custom parseValue
* Revert MultiaAutocomplete changes
* Improve typing in describeTemporalInterval and
describeRelativeDatetime
* Use describeTemporalUnit to pluralize
* Use interface
* Avoid setting value as DOM attribute
* Fix test
* Add onAdd prop to AggregationPicker and revert the change to have onSelect pass multiple aggregations
* Reduce number of props
* Render checkboxes in custom items
* Introduce and use --mb-color-brand-lighter
* Avoid !important
* Remove redundant prop
* Rename ExtractColumn to ExtractColumnAction
* Add lines.svg
* Fix positioning in case dir attribute is not present in any parent component
* Add CompareAggregationsAction
* Fix warning about isSelected being used as DOM attribute
* Fix positioning in case dir attribute is not present in any parent component
* Remove redundant Box
* Compute the title
* Add compareAggregationsDrill boilerplate
* Add compare-aggregations drill stub
* Update comment
* Compute title and aggregation
* Add TODO
* Use proper title when there's only 1 aggregation
* Show visualization on submit
* Limit aggregation choice to 1 in compare aggregations drill
* Redirect to chart upon submission
* Do not navigate to chart view
* Do not navigate to chart view
* Make drill-thru tests expect the new compare-aggregations drill
* Add type attribute to all ListItems
* Add aggregationIndex to drillInfo
* Add aggregationDrillDetails and wire it up
* Fix destructuring
* Fix tests
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* tag flakiest tests as such
* let well behaved tests out of flake jail
* more bad actors off the streets
* add another flake
* sandboxes re-offended already
* support test utilities in sdk import
* add relative unit default chart font size in embedding defaults
* make font size follow sdk root
* feat(sdk): add font size settings to axis ticks
* feat(sdk): add font size settings to series labels
* re-add hasTimelineEvents from merge commits
* apply default font size at the theme level
* add table header cell font size
* calculate cell font size based on 12.5px
* rename theme.chart to theme.cartesian
* refactor visualization theme
* refactor default viz theme
* remove chart style config
* add goal line
* pass font size to chart measurements
* use px as reference unit
* support row chart viz
* add missing echarts font size
* add table cell font sizes
* use css variables for funnel color
* fix theme type in smart scalar story
* add bar chart to storybook for visual test
* add line chart to storybook for visual test
* add loki reference images
* add loki reference images for row chart
* add loki shared reference images to stories filter
* update loki reference image
* synchronize base font size
* add theme provider to funnel test setup
* add missing default visualization theme
* add unit test for get size in px
* update loki reference image
* fix static viz build error
* Sort functions
* Add boilerplate for aggregateOffset and tests
* Simplify assertion
* Make basic case work
* Handle name generation
* Do not require "lib/uuid" attribute to be present since MLv2 will normalize it under the hood anyway
* Fix typing
* Group tests
* Refactor offsetClause to return a new clause instead of a new query
- Move offsetClause to expressions.ts
- Add assertion
* Revert "Sort functions"
This reverts commit ab9ce2b24e.
* Move tests
* Handle names dynamically
* Shorten a test
* Update comment
* Add TODO
* Update expression types
* Add diffOffsetClause & percentDiffOffsetClause
* Add a test for diffOffsetClause
* Add TODOs
* Add tests for diffOffsetClause and percentDiffOffsetClause
* Unwrap tests
* Add skeleton for new tests
* Refactor first offsetClause tests
* Add tests for names
* Fix case of non-datetime columns
* Handle a case with offset < 1 and no breakouts
* Handle a case with offset < 1 and breakouts on non-datetime column
* Handle a case with offset < 1 and breakouts binned datetime column
* Handle a case with offset < 1 and breakout on non-binned datetime column
* Refactor
* Refactor
* Remove TODO
* Add tests for diffOffsetClause
* Add tests for percentDiffOffsetClause
* Move offset stuff to offset.ts and offset.unit.spec.ts
* Use template string for prefixes
* Use breakoutColumn + isDate to check column type
* Refactor
* Fix error message
* Add boilerplate for CompareAggregations
* Fix title
* Render aggregations list
* Style AccordionList
* Sort props
* Fix bucket name
* Use displayName instead shortName
* Support parseValue prop in NumberInput
* Add period input accepting integers only
* Accept non-negative values only
* Do not accept zeros
* Add state
* Add submit button
* Export offset functions via Lib
* Make it possible to skip rightSection
* Add column picker
* Map offset user input to api input (negative value)
* Add label
* Fix crash
* Extract shouldCreate
* Make onSelect in AggregationPicker support multiple values
* Extract ReferenceAggregationPicker
* Extract ColumnPicker
* Extract getAggregations
* Rename
* Add custom items
* Refactor item component
* Extract OffsetInput
* Remove unused data-testid
* Style OffsetInput
* Generate titles according to specs
* Generate label
* Generate help
* Extract utils
* Use different width for the 1st step
* Format code
* Use MultiSelect directly
* Avoid custom parseValue
* Revert MultiaAutocomplete changes
* Improve typing in describeTemporalInterval and
describeRelativeDatetime
* Use describeTemporalUnit to pluralize
* Use interface
* Avoid setting value as DOM attribute
* Fix test
* Add onAdd prop to AggregationPicker and revert the change to have onSelect pass multiple aggregations
* Reduce number of props
* Render checkboxes in custom items
* Introduce and use --mb-color-brand-lighter
* Avoid !important
* Remove redundant prop
* Rename ExtractColumn to ExtractColumnAction
* Add lines.svg
* Fix positioning in case dir attribute is not present in any parent component
* Add CompareAggregationsAction
* Fix warning about isSelected being used as DOM attribute
* Fix positioning in case dir attribute is not present in any parent component
* Remove redundant Box
* Compute the title
* Use proper title when there's only 1 aggregation
* Show visualization on submit
* Do not navigate to chart view
* Add type attribute to all ListItems
* move parameters and dashboard reducers to the common ones as they're used everywhere
* convert reducers-{common,main,public}.js to typescript, done with --no-verify
* convert reducers to named exports
* Sort functions
* Add boilerplate for aggregateOffset and tests
* Simplify assertion
* Make basic case work
* Handle name generation
* Do not require "lib/uuid" attribute to be present since MLv2 will normalize it under the hood anyway
* Fix typing
* Group tests
* Refactor offsetClause to return a new clause instead of a new query
- Move offsetClause to expressions.ts
- Add assertion
* Revert "Sort functions"
This reverts commit ab9ce2b24e.
* Move tests
* Handle names dynamically
* Shorten a test
* Update comment
* Add TODO
* Update expression types
* Add diffOffsetClause & percentDiffOffsetClause
* Add a test for diffOffsetClause
* Add TODOs
* Add tests for diffOffsetClause and percentDiffOffsetClause
* Unwrap tests
* Add skeleton for new tests
* Refactor first offsetClause tests
* Add tests for names
* Fix case of non-datetime columns
* Handle a case with offset < 1 and no breakouts
* Handle a case with offset < 1 and breakouts on non-datetime column
* Handle a case with offset < 1 and breakouts binned datetime column
* Handle a case with offset < 1 and breakout on non-binned datetime column
* Refactor
* Refactor
* Remove TODO
* Add tests for diffOffsetClause
* Add tests for percentDiffOffsetClause
* Move offset stuff to offset.ts and offset.unit.spec.ts
* Use template string for prefixes
* Use breakoutColumn + isDate to check column type
* Refactor
* Fix error message
* Add boilerplate for CompareAggregations
* Fix title
* Render aggregations list
* Style AccordionList
* Sort props
* Fix bucket name
* Use displayName instead shortName
* Support parseValue prop in NumberInput
* Add period input accepting integers only
* Accept non-negative values only
* Do not accept zeros
* Add state
* Add submit button
* Export offset functions via Lib
* Make it possible to skip rightSection
* Add column picker
* Map offset user input to api input (negative value)
* Add label
* Fix crash
* Extract shouldCreate
* Make onSelect in AggregationPicker support multiple values
* Extract ReferenceAggregationPicker
* Extract ColumnPicker
* Extract getAggregations
* Rename
* Add custom items
* Refactor item component
* Extract OffsetInput
* Remove unused data-testid
* Style OffsetInput
* Generate titles according to specs
* Generate label
* Generate help
* Extract utils
* Use different width for the 1st step
* Format code
* Use MultiSelect directly
* Avoid custom parseValue
* Revert MultiaAutocomplete changes
* Improve typing in describeTemporalInterval and
describeRelativeDatetime
* Use describeTemporalUnit to pluralize
* Use interface
* Avoid setting value as DOM attribute
* Fix test
* Add onAdd prop to AggregationPicker and revert the change to have onSelect pass multiple aggregations
* Reduce number of props
* Render checkboxes in custom items
* Introduce and use --mb-color-brand-lighter
* Avoid !important
* Remove redundant prop
* Fix warning about isSelected being used as DOM attribute
* Fix positioning in case dir attribute is not present in any parent component
* Add type attribute to all ListItems
On the Browse models page we have a filter icon which opens a popover with a switch that toggles "Show only verified models". With this change, the icon is not rendered if there are no verified models at all.
:message(format"The name of this %s should end with `!` because it contains calls to non thread safe form `%s`. [:metabase/test-helpers-use-non-thread-safe-functions]"
(error!form(format"destructive functions like %s are not allowed inside a ^:parallel test or test fixture. If this should be allowed, add it to the whitelist in .clj-kondo/hooks/clojure/test.clj"
(error!form(format"destructive functions like %s are not allowed inside a ^:parallel test or test fixture. If this should be allowed, add it to the whitelist in the Kondo config file [:metabase/validate-deftest]"
qualified-symbol)))))
(walk[form]
(fform)
@@ -135,7 +28,7 @@
(defn- deftest-check-parallel
"1. Check if test is marked ^:parallel / ^:synchronized correctly
2. Make sure disallowed forms are not used in ^:parallel tests"
[{[_test-name&body]:children, :as_node}]
[{[_test-name&body]:children, :as_node}config]
(let [test-metadata(:metatest-name)
metadata-sexprs(map hooks/sexprtest-metadata)
combined-metadata(transduce
@@ -152,9 +45,9 @@
(hooks/reg-finding!(assoc (meta test-name)
:message"Test should not be marked both ^:parallel and ^:synchronized"
:type:metabase/validate-deftest)))
;; only when the custom `:metabase/deftest-not-marked-parallel` is enabled: complain if tests are not explicitly
;; marked `^:parallel` or `^:synchronized`. This is mostly to encourage people to mark everything `^:parallel` in
;; places like `metabase.lib` tests unless there is a really good reason not to.
;; only when the custom `:metabase/deftest-not-marked-parallel-or-synchronized` is enabled: complain if tests are
;; not explicitly marked `^:parallel` or `^:synchronized`. This is mostly to encourage people to mark everything
;; `^:parallel` in places like `metabase.lib` tests unless there is a really good reason not to.
> in the Metabase Branching Strategy document for more details. If you're not employed by Metabase, this section does not
> apply to you, and the label will be taken care of by your reviewer.
> For those employed by Metabase: if you are merging into master, please add either a `backport` or a `no-backport` label to this PR. You will not be able to merge until you do this step. Refer to the section [Do I need to backport this PR?](https://www.notion.so/metabase/Metabase-Branching-Strategy-6eb577d5f61142aa960a626d6bbdfeb3?pvs=4#89f80d6f17714a0198aeb66c0efd1b71) in the Metabase Branching Strategy document for more details. If you're not employed by Metabase, this section does not apply to you, and the label will be taken care of by your reviewer.
> **Warning**
>
> If that is your first contribution to Metabase, please sign the [Contributor License Agreement](https://docs.google.com/a/metabase.com/forms/d/1oV38o7b9ONFSwuzwmERRMi9SYrhYeOrkbmNaq9pOJ_E/viewform) (unless it's a tiny documentation change). Also, if you're attempting to fix a translation issue, please submit your changes to our [POEditor project](https://poeditor.com/join/project/ynjQmwSsGh) instead of opening a PR.
> If that is your first contribution to Metabase, please sign the [Contributor License Agreement](https://docs.google.com/a/metabase.com/forms/d/1oV38o7b9ONFSwuzwmERRMi9SYrhYeOrkbmNaq9pOJ_E/viewform) (unless it's a tiny documentation change). Also, if you're attempting to fix a translation issue, please submit your changes to our [Crowdin project](https://crowdin.com/project/metabase-i18n) instead of opening a PR.
while ! curl -s 'http://localhost:3000/api/health' | grep '{"status":"ok"}' > /dev/null; do
if ! kill -0 $(cat metabase_pid) 2>/dev/null; then
cat metabase.log
echo "Metabase process has exited. Check that your migration changes work on an instance that has been previously initialized on your PR's target branch."
@${githubUsername} You have modified embedding SDK code. Please make sure the PR title follows [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) style.
Here are all the supported types that will be shown in the changelog:
- \`feat\`
- \`fix\`
- \`perf\`
These types \`docs\`, \`style\`, \`refactor\`, \`test\`, \`build\`, and \`ci\` are also encouraged for non-changelog related tasks.
Please also make sure to include \`sdk\` scope, otherwise, the changelog will not include this task.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
