The Portal diff view wants `[before after]` pairs of the original data,
not the three-part output of `clojure.data/diff`. This fixes the
`qp.debug/debug>` output from the preprocessing middleware to emit
correct diffs.
Also, since the diffs are not useful when the query is converted between
legacy and pMBQL, this also attaches the *converted* input query to the
output query's Clojure metadata, and this converted value is used as the
basis for the diff.
* entity id translation + tests
* add api level test
* simplify definition of eid-table->model + add test
* update tests to take keywords
* improve comment
* generate the eid-table->model map
* delete now-obsolete test
* make it work in oss
* put the resulting response into a key, so we can add more information later
* formatting
* use model names without the model/ prefix as keys
* Creates list of `api/model->db-model`
- update keys for util/entity_id request
- update shape of util/entity_id response
- add test for not-found eids
* formatting
* Respond to code review feedback
* entity-ids work on /embed/{card,dashboard}* routes
* make entity id translation work for all tokens
* Merge branch 'static-embed-dashboard-and-question-endpoints' of github.com:metabase/metabase into static-embed-dashboard-and-question-endpoints
* massage schema. add tons of tests
* decode more eids
* test alignment
* don't require ns inside of itself :think:
* make the test functions to create tokens better
* bring back `dashboard-url`
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Query Validator FE
* collection path, unit tests
* wrapping feature with token flag
* updating util function, adding row type
* updating session_test.clj
* type adjustment
* fixing other table sorting
* Empty state, clean up utils
* unit test adjustment
* e2e adjustment
* another attempt at perms fix
* try again
* try manual graphql auth
* try with global env
* Try setting defaults in graphql
* try some more things
* use my own graphql instance
* one more test
* remove test code
* remove rogue curly brace
== Goal ==
Hide attached DWH database details from anyone incl. admins:
* Do not show them in the UI
* Do not permit to change them
* Do not serialize them
The aim is that customers cannot gain access to (parts of) credentials,
and they cannot break a feature they are paying for by changing
connection details.
== Implementation ==
The Metabase backend already contains provisions in the implementation
of `metabase.models.interface/to-json` for `:model/Database` to hide the
`details` of the database in HTTP responses, if the user lacks write
permission on the database. We utilize this by adding an
`is_attached_dwh` column to the `database` table and rejecting
`metabase.models.interface/can-write?` when this flag is enabled. In
the "admin" UI, we show a replacement text instead of the edit form when
the flag is set. (It might be correct to show this whenever `details`
is absent. See below for possible follow-up work.)
However, several sections of the frontend code expected the `details`
field to always be present. In order to make `details` optional, as the
backend seems to handle it, we fix the respective code to treat this
case in the way that appears appropriate in the context.
Database details are already generally excluded from H2 dump snapshots
(see `metabase.cmd.copy/*copy-h2-database-details*`), thus nothing
changes there.
== How to test ==
=== New behaviour ===
Setting the `is_attached_dwh` field hides the database details:
1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
- In addition to the fields you would normally set, also set
`is_attached_dwh: true`.
- This also works when adding this flag to a database that previously
did not have this flag set.
2. Start your Metabase instance.
3. Verify the database shows up in the "admin" section
(`/admin/databases`).
4. Verify that clicking the database to see its details only reveals
"This database cannot be modified."
5. Verify that responses from the backend do not include a `details`
field for this database.
=== Original behaviour ===
Behaviour without setting the `is_attached_dwh` field is unchanged:
1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
- Only set the fields you would normally set. Do not set
`is_attached_dwh` (or set it to `false`).
2. Start your Metabase instance.
3. Verify the database shows up in the "admin" section
(`/admin/databases`).
4. Verify that clicking the database to see its details only reveal the
regular edit form, showing connection fields like `host`, `user`,
etc. with the values you configured.
== How this will be rolled out ==
1. Upgrade existing Metabase Cloud instances with data warehouse to a
Metabase version that supports `is_attached_dwh`.
2. Set `is_attached_dwh` in the database section of the config file for
Metabase Cloud instances with a data warehouse.
== Possible follow-up work ==
In https://github.com/metabase/metabase/issues/25715, absent
`database.details` was identified as a bug. Since then, `details` was
made `NOT NULL` in the application database, so this bug can no longer
occur. However, today backend responses can be missing the `details`
field, if the current user lacks write permission to the database
setting (see above). Fully re-evaluating the fix to #25715 in this
context is outside the scope of this PR.
Closes: https://github.com/metabase/harbormaster/issues/5051
* Add `clojure -M:kondo` and `clojure -M:kondo:kondo/all` and bump version
* Fix Kondo errors
* Fix Kondo+LSP issues with `defendpoint`, `defenterprise`, etc.
* Use replace-deps instead of deps for speed
* Ok apparently maybe we do need to copy configs when we run Kondo on CI
* Oops `./bin/kondo.sh` should not try to use `clj-kondo`
* Remove references to GA driver folders
These migrations will be backported to v49.
Three changes:
- update IDs/locations of the migrations to v49 vs v51
- add one preCondition (when adding the foreign key constraint)
- add `onFail: MARK_RAN` to the index preconditions. I forgot this
before, so it blows up when the precondition doesn't hold.
* allow re-ordering and re-naming pie chart slices
* aggregate rows with same dimension value
* show warning when dimension has unaggregated values
* change colors from sidebar
* implement renaming
* styles for slice name widget
* handle numeric slice keys for binned values relative dates etc
* implement slice disabling
* remove old colors setting
* implement merging new and removing existing rows from query results
* fix formatting bugs
* port settings to static viz
* fix bug with unaggregated data
* address todos
* add loki test
* fix measure column settings widget erroring
* fix settings sidebar crashing
* add e2e tests
* update snapshots
* fix pie_chart.cy.spec.js CI-only flakes
* fix#21504 repro failure
* fix click action value
* use nestedSettings instead of seriesSettings
* fix settings sidebar crashing
* fix row with 0 key not being draggable
* fix flaky test
* filter other slices from sidebar
* combine filter header buttons
* refined styling
* unit tests
* another import conflict
* omg another one
* remove double border for filter button
* use vars and update imports
* Increase log level to INFO from error
Closes#47396
logs before:
```
2024-08-29 08:37:03,836 INFO db.liquibase :: Checking if Database has unrun migrations...
2024-08-29 08:37:04,045 INFO db.liquibase :: Database has unrun migrations. Checking if migration lock is taken...
2024-08-29 08:37:04,047 INFO db.liquibase :: No migration lock found.
2024-08-29 08:37:04,047 INFO db.liquibase :: Migration lock acquired.
2024-08-29 08:37:04,167 INFO db.liquibase :: Running 83 migrations ...
```
logs after:
```
2024-08-29 08:39:45,019 INFO db.liquibase :: Running 83 migrations ...
...
2024-08-29 08:39:45,166 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:00::calherries ran successfully in 0ms
2024-08-29 08:39:45,182 INFO liquibase.changelog :: Columns is_defective_duplicate(boolean) added to metabase_field
2024-08-29 08:39:45,182 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:01::calherries ran successfully in 14ms
2024-08-29 08:39:45,199 INFO liquibase.changelog :: Custom SQL executed
2024-08-29 08:39:45,200 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:02::calherries ran successfully in 16ms
2024-08-29 08:39:45,202 INFO liquibase.changelog :: Foreign key fk_field_parent_ref_field_id dropped
2024-08-29 08:39:45,202 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:03::calherries ran successfully in 1ms
2024-08-29 08:39:45,204 INFO liquibase.changelog :: Foreign key constraint added to metabase_field (parent_id)
2024-08-29 08:39:45,205 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:04::calherries ran successfully in 2ms
...
```
* empty commit to bump CI
* handle cache config overflowed
* fix root too
* do not comment out test
* make cljfmt happy
---------
Co-authored-by: Alexander Solovyov <alexander@solovyov.net>
* Make metric metadata keep the metric's name
This is a fix for the problem reported in #40355, which is fixed separately for v50 and backported to v49.
* Patch broken test
* Normalize dataset-query
* Normalize legacy queries only
* ask for tenancy isolation columns
* deny all permissions for all users group
* create new collections
* add jwt group mappings
* add the permissions step
* add multi-tenancy message in helper text format
* add permission graph
* wire together permissions
* use schema permissions
* use fields from table metadata from query_metadata
* add tenancy field reference
* remove log messages
* deny access to unsandboxed tables
* make permission graph more explicit
* deny access to sample database for customer groups
* add unit test for permission graph
* split permission groups and sandboxes
* jwt settings and hard-coded user attributes
* handle errors when updating sso mappings
* add express api and user switcher
* only fallback to api keys when license is invalid
* add util to sample tenancy column values
* conditional BASE_SSO_API imports
* improve embedding error message
* setup jwt configuration after license step
* setup permissions at the last step
* add missing import
* update steps that requires license
* fix incorrect imports
* add missing useContext
* handle permission update error
* remove tenancyIsolationEnabled field
* add tenancy column sampling
* differentiate tenancy column query error
* rename tenancyColumnValues to tenantIds
* assign sampled tenant ids to user attributes
* add tenant ids
* define collection permissions
* reference sandboxing group by name
* update snippet to be same as the README
* extract ask for tenancy columns to a separate step
* use the customer_id attribute
* query the table query metadata at origin
* append tables correctly
* improve error handling in table scanning
* add retry logic to metadata fetching
* only query metadata for selected fields
* fix race condition with retry
* update loading state and retries
* update comments on jwt license
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* filter the target table by id
* highlight last selected tenant column
* use breakout to get list of ids
* temporary workaround to reload the whole page
* update row value types
* update row value types
* block non-selected tables
* remove the source-field from sandboxing
* use the fk_target_field_id as instead of target.id
* update unit test
* remove source-field as we only reference our own column
* make native permission types more strict
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* ask for tenancy isolation columns
* deny all permissions for all users group
* create new collections
* add jwt group mappings
* add the permissions step
* add multi-tenancy message in helper text format
* add permission graph
* wire together permissions
* use schema permissions
* use fields from table metadata from query_metadata
* add tenancy field reference
* remove log messages
* deny access to unsandboxed tables
* make permission graph more explicit
* deny access to sample database for customer groups
* add unit test for permission graph
* split permission groups and sandboxes
* jwt settings and hard-coded user attributes
* handle errors when updating sso mappings
* add util to sample tenancy column values
* improve embedding error message
* setup jwt configuration after license step
* setup permissions at the last step
* handle permission update error
* add tenancy column sampling
* differentiate tenancy column query error
* rename tenancyColumnValues to tenantIds
* define collection permissions
* reference sandboxing group by name
* extract ask for tenancy columns to a separate step
* query the table query metadata at origin
* append tables correctly
* improve error handling in table scanning
* add retry logic to metadata fetching
* only query metadata for selected fields
* fix race condition with retry
* update loading state and retries
* filter the target table by id
* highlight last selected tenant column
* use breakout to get list of ids
* update row value types
* block non-selected tables
* remove the source-field from sandboxing
* use the fk_target_field_id as instead of target.id
* update unit test
* remove source-field as we only reference our own column
* make native permission types more strict
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
Previously, truncating a `:type/DateTime` column by `:month` or `:day`
would return a `:type/Date`, which subtly broke the query.
In particular, if you try to order-by the breakout column `Created At (month)`
then it would not get de-duplicated, causing a SQL error about conflicting
ORDER BY clauses.
Fixes#46992.
Previously, `visible-collection-ids` was effectively "free" in that we'd
cached `collection-id->collection` for *all* collections, within a
single request, and then locally filtered it for permissions without
needing to hit the database again.
To be honest, there is probably a better fix for this - we're repeatedly
calling `visible-collection-ids` when we probably could just save a
single copy of it and use it when calculating the effective location of
every collection.
However, this is a very quick and low-risk fix, and I want to prioritize
getting this done, and then we can improve it later.
Locally, I copied down the database from stats and timed the
`/api/search?model_ancestors=true` endpoint.
Before my "speedup" PR (https://github.com/metabase/metabase/pull/46942)
it took ~7 seconds to return results.
After my "speedup" PR, it took ~15s to return results. :grimace:
With this change, it takes 818ms to return results. 🎉
* Fix typo in expression-clause-normalization-test
s/time-interval/relative-time-interval/
* Add missing table-id to card def to fix failing legacy-ref-test
* Ensure that if yarn test-cljs fails it fails the CI checks
When configured to autorun, the command
shadow-cljs compile test
will exit success even if test failures occur in the node test. See
https://github.com/thheller/shadow-cljs/issues/425
In order to ensure our CI checks fail when cljs tests fail, we need to invoke
node directly. To prevent running the tests twice, remove `:autorun true` from
the `:test` target in shadow-cljs.edn.
* zoom-in-timeseries-drill: do not include :hour and :minute for DATE columns
When the user attempts to drill through a DATE column, we should not return :hour or :minute drill throughs from
available-drill-thrus. Previously we did, which resulted in an error being displayed when the user attempted to select
them.
Modify next-breakout-unit in zoom_in_timeseries.cljc to not return :hour or :minute units for :type/Date columns,
which prevents these from being returned by available-drill-thrus, which prevents them appearing in UI context menus.
* Use lib.temporal-bucket/available-temporal-buckets
Per review feedback, use lib.temporal-bucket/available-temporal-buckets to get the valid-current-units in
metabase.lib.drill-thru.zoom-in-timeseries.
* Do not construct zipmap just to lookup the next unit
Rather than constructing a zipmap from `valid-current-units', which requires iterating over the seq twice, just
traverse the seq once and return the next item in the seq (if any).
Constructing a zipmap made sense in the old code because it was constructed once from static data and used many times
for lookup. In the new version, the vec of `valid-current-units` is dynamic and dependent on the type of the breakout.
* PR suggestion: use threading macro rather than nested sexps
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* PR suggestion: use threading macro rather than transducer / eduction
* PR suggestion: reduce boilerplate in zoom-in-timeseries tests
Reduce code dup by looping over aggregation and bucketing options rather than having a separate deftest for each.
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
Fixes#39366
* use loki to test png exports on questions
* try to put the img in the body to see if it fixes the scrollbar issues in ci
* fix linting issues
* let's try again in ci
* Attempt to make CI wait before taking a snapshot for downloaded PNGs
* Use the snapshots from CI for PNG downloads
* pdf export test thanks to kelvin suggestion about just testing the png
* refactored the code to extract a util function
* remove pdf tests as they broke in ci 🤷
* Update frontend/src/metabase/env.ts
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Update frontend/src/metabase/lib/loki-utils.ts
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* entity id translation + tests
* add api level test
* simplify definition of eid-table->model + add test
* update tests to take keywords
* improve comment
* generate the eid-table->model map
* delete now-obsolete test
* make it work in oss
* put the resulting response into a key, so we can add more information later
* formatting
* use model names without the model/ prefix as keys
* Creates list of `api/model->db-model`
- update keys for util/entity_id request
- update shape of util/entity_id response
- add test for not-found eids
* formatting
* Respond to code review feedback
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Don't encrypt boolean settings (by default)
We have tooling to disable encryption on settings even when the
`MB_ENCRYPTION_SECRET_KEY` is set. Turn it on by default for all boolean
settings, which don't need to be encrypted.
I also optimized the code that decrypts settings on startup because I
didn't want to delay startup if someone had set a bunch of boolean
settings. With 20 set, the old version added about 200+ ms to startup,
about 10ms per boolean setting, whether or not it was encrypted or in
the DB already. The optimized version selects all the never-encrypt
values from the database at once (a bit silly, but we also just exclude
raw `true` and `false` values so we don't bother checking them) and
updates them if they're encrypted - this adds ~40ms to startup with 20
encrypted boolean settings (about 2ms per boolean setting) and ~5ms to
startup on subsequent runs, when no encrypted values are in the DB.
This testing is too strict to support some legitimate use cases that
have no workaround at present. We can bring back this type-checking
eventually, once it's possible to correctly express things like
automatic coercion of strings to numbers in `SUM` aggregations.
Fixes#44431.
* Update appearance.md
Adding information about how colors are selected by Metabase because there is the confusion that the order in which you select colors is the order you get on the chart
* Update appearance.md
Removed code reference and tried to simplify the context
* Update docs/configuring-metabase/appearance.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
There are a few separate changes here:
- Migrations: add and populate indexed columns perm_value, perm_type, and collection_id to permissions
These fields allows us to efficiently run queries based on collection permissions in the DB without string manipulation. Keeping the table as permissions allows us to do this migration in-place. Note that in some cases collections may have been deleted from the database without deleting the associated permissions row (since there was no foreign key before). We need to be defensive here: if we have a permissions row without a corresponding collection, delete the row before running the rest of the migration.
- Write perm_value, perm_type, and collection_id for new collection permissions
A very simple before-insert method sets these fields before a collection permission is written to the DB.
- Replace collection/permissions-set->visible-collection-ids and collection/visible-collection-ids->honeysql-filter-clause with collection/honeysql-filter-clause
Previously, just about everywhere we used permissions-set->visible-collection-ids, what we were essentially doing was an in-app join: select all the collection IDs you have permission on, then convert it to a SQL clause like WHERE collection_id IN ( all of those collection IDs).
Replace both of these with honeysql-filter-clause, which uses the new fields we added to permissions above to construct a honeysql filter representing "all the collections I have permissions on", without needing to round-trip them to the application and back to the DB.
Of course, we can then write a function visible-collection-ids, which uses honeysql-filter-clause, for those cases where we do actually need the whole bunch in the application (we use this, for example, when constructing the effective-location for a collection).
I also added one more toggle to the VisibilityConfig that's passed into the honeysql-filter-clause (and used to be passed to permissions-set->visible-collection-ids), allowing you to select only the effective children of some collection.
* [MBQL lib] Map dashboard filters to last stage w/aggregations
Originally we used the last stage's columns for dashboard filters, but
then if you eg. use a custom expression to clean up the results of the
aggregation, then you can't filter at the most useful spot - before the
aggregations.
Fixes#19744. Related to #26244, #33958.
* Add stage-number option to the dimension not to the field
* Remove FE changes
* Support negative stage numbers
* Add tests for multi-stage parameter expansion
* Specify schema for dimension options
* Fix legacy MBQL schema
* Append stage when running cards with top level summary
* Fix schema
* Only ensure filter stage when the parameters say it's used
* Implement ensure-filter-stage
* Format clj
* Add ensure-filter-stage test
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Fix DateTimeRange substitution for open ranges
* Add E2E repro for #47172
* cljfmt
* Add test for datetime column
* Update range boundary for open range
* Update test names
* Apply suggestions from code review
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix e2e
---------
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add database type adjustments to server side generated temporal values
* Speculative base type change
* Infer timezone only from _actual_ user, not global user level
* Set timezone for actual user in test extensions
* Add test case
* Update type info generation
* Adjust server side relative datetime
* Update src/metabase/driver/sql/query_processor.clj
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Fix base type logic
* Fix typo
* Fix formatting
* Update user name handling
* cljfmt
* Address review comments
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* reorganizes data-permissions.ts to be a folder split by a file per concern, moves some logic out into other utils files where it makes sense
* pr feedback and fix lints
* only use viz height for visualization view
* move the default viz height fallback
* default to full-height if height is zero or unavailable
* remove test wrapper
* recents updates to better support metrics
allows updating metric views through the API
improves parent_collection calculation
* add 2 tests for metrics
* Use global search for aggregation picker
* Add names to aggregation picker items so they can be searched
* Render info icon metric descriptions
* Add unit tests for global aggregation picker search
* Add test for searching metrics
* Add a test for metrics descriptions
* Sort metrics
* Fix e2e test
* Use Flex from metabase/ui
* Handle markdown in metric description
* Add test for sorted metrics
* Update Markdown styling in metric description
* Fix clojure formatting
* Test for markdown in the Metric description
* Fix lint
* Fix test for markdown
* Use simpler equality test
* Fix overflow in test
* QuestionAlertWidget to FC + TS
* update e2e tests
* use regular alert for unsubscribes
* Create Question and Dashboard Sharing Menu
* Handle some edge cases
* top notch unit tests
* more testing and tweaking
* fix imports
* more tests
* add alerts to question sharing menu
* add tests for alerts
* update e2e tests
* use Oisin's toolbarbutton
* fix rebase
* prompt to save before sharing questions
* show prompt to set up notification channels
* fix import
* update tests and hide on notebook screen
* fixes
* clean up pulse types
* update e2e tests
* fix a bunch more tests
* more e2e test fixes
* maybe green now? 🤞
* last one 🥺
* design updates
* make channel setup modal nicer
* simpler component props
* lint fix
* lint fixes
* poc of removing data usage step for the tech plan
* pretend it's not cloud if hasAddedPaidPlanInPreviousStep is true
* split the jest tests files to simulate pro cloud, pro selfhosted and starter cloud scenarios
* open metabase store to get trial token
* remove license env from instance setup as we provide the license key later
* activate metabase license
* fix password field missing
* fix formatting for generated component files message
* activate license key
* add missing auth options for postgres
* add function to print with padding
* prevent infinite loop and update section representation
* show the activate license error in red
* rename variable
* Add offset presets
* Render OffsetPresets initially
* Add BucketInput
* Render BucketInput and do not rely on referential equality for Lib.Bucket
* Make sure the currently selected bucket shows in the presets
* Lift label into CompareAggregations
* Fix spacing of label
* Fix labels based on comparison type
* Use Switch for include current period
* Include current bucket in CurrentPeriodInput label
* Fix unit tests
* Add more unit tests for CompareAggregations
* Update e2e test for column compare to handle presets
* Group offset tests
* Add tests for moving average
* Add tests to verify changing the bucket works
* Fix type
* Make sure temporal columns are buckatable before picking them
* Make getPreferredPresets more clear
* Add test for temporal expression column
* Fix lint
* refactor: sort import members inside destructuring
* Show an error message when the temporal unit is not applicable to a dashboard card (#46983)
* Fix tests
* Add tests
* Add tests
* Fix types
* Fix types
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* make the newsletter signup a checkbox instead of a button
* update copy on unit test
* fix describe nesting
* update the ui to be like the design
* make tests use userEvent.click to avoid warning about missing act
* use the subscribeToNewsletter function
* add basic e2e test for newsletter subscription
* move wait to the end of the test to see if it passes in ci
* remove async to see if it fixes the issue
* trying to mock the response
* make it clear why we're mocking the response
* use getUserEmail as suggested by Sloan
* inline var a suggested on code review
* Kondo Config cleanup
* Restore warnings for with-log-messages-for-level until #28827 is merged
* Updated clojure.test hooks
* Fix kondo warning
* Test defn/defmacro exclamation point linter should also use :parallel/disallowed (part 1)
* WIP
* Time for me to learn to spell
* Finish the cleanup
* Remove code I was typing as an example
* Fix renamed var
* Fix the SAML tests
* Fix Kondo warning
* allow `:blocked` to be saved for table level perms
* Adds 2 tests for table level blocked permission settings
- N.B. these are NOT ENFORCED YET
* update test that asserted we cannot set block on tables (we can)
* WIP: Pairing on making perms checking less wild
* cleanup, update docs, and add a test for view-data perm only
- Added a test where we have data permissions, but not create query, and
I think it is failing when we have create query and blocked data
permissions.
- renamed some functions from check-x -> has-x? since they return a
value instead of throwing now
* Revert "WIP: Pairing on making perms checking less wild"
Keep the same behavior, but stick with the saner flow control
This reverts commit 63bcb5b401138555a081378574b15d6506469e60.
* update docs
* update test to be passing
- TODO: make sure it's correct w.r.t. perm settings
* Allow schema level blocked setting in permgraph
* remove invalid test cases
- continue to have a forcing function to test newly added perms
* conform function output
* ensure a single blocked table blocks native queries to its DB
* update error message
- we now catch this error in `metabase.models.query.permissions/has-perm-for-query?`
* we now check for data permissions to process query for card
* add more explanation to what we are testing
- to help see why it fails on CI and passes locally
* remove excess `def`
* Add test for table-level data X collection perms
- update test found to be in-error
* update param values qp permission check style
* set view-data and create-query explicitly
* set viewdata and createquery explicitly in qp test
* Respond to review comments (which fixes a case)
* setting a table to blocked: leave other tables the same
* [Permissions] Add "No access" schema/table permission (#46509)
* first pass
* refactors downgrading native permission logic and updates calculation so that "No access" downgrades native permissions to "No"
* stub for permissions help info on table block
* modal changes wip, updates downgrading create queries permissions to all happen at a single call site
* clean up, sandboxing modal copy changes, removes rekoke/limit access modal changes to make the diff smaller and move code to a seperate PR
* updates permissions help section to contain the final copy
* sandboxing copy fix and remove modal that was dropped from requirements
* adds blocked at the schema level, updates no access copy to blocked, updates permissions help section to contain new blocked and schema level changes
* fixes failed unit and e2e tests after sandboxing copy changes
* improve the block e2e test to include table blocking
* fixes failing blocked test, fixes other schemas create queries permissions getting correct with one schema was droped to blocked view data access, fixes a bug that prevents the save bar from going away when all permissions for group are set to the default values
* clean up
* remove color changes
* prevents parent being set to blocked preventing edits for children entities
* add new hasPermissionValueInSubgraph fn, adds modal to warn users we have to upgrade the view data permissions when they upgrade create queries permissions when a child entity is set to blocked
* adds test coverage for new modal
* removes unused function, adds new updateEntityPermission fn to help consolidate some logic elsewhere
* unit test fix and type fix
* most pr feedback
* updates the confirmation modal copy when changing a parent entity that contains a child with blocked permissions and/or sandboxed children, adds test coverage for that, adds test coverage for permission view data column not appearing in oss
* type fix
* [Permissions] Add e2e test coverage for blocked permissions enforcements (#46663)
* adds test coverage for enforcement of blocked permissions
* moves tests around based on pr feedback
* copy changes
* adds fix to make sure that blocked permissions are not removed from sibling tables that have the create queries permissions upgraded (#46854)
* Fix table name lookup for dbs w/ 1 schema per db
* add test for blank schema identifiers
* Refine sandboxed user perms for query builder access (#46939)
* Refine sandboxed user perms for query builder access
- Limit create-queries permissions to unblocked tables only
- Check user permissions for each table before granting query builder access
- Prevent querying of blocked joined tables from query builder for sandboxed users
* Adjust permissions for sandboxed users
- Grant view-data permissions only for unblocked tables
- Revert create-queries permissions to all tables in sandbox
- Remove unnecessary intermediate variable
* when sandboxing we no longer grant unrestricted view perms for blocked tables
* Update enterprise/backend/src/metabase_enterprise/sandbox/query_processor/middleware/row_level_restrictions.clj
remove blank line
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* - make coalesce-test exhaustive (except for sandbox)
* Update enterprise/backend/src/metabase_enterprise/sandbox/query_processor/middleware/row_level_restrictions.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* t2/select ... -> database/table-id->database-id
* update comment
* [Permissions] Prevent "Granular" option in DB View Data options from changing permissions to unrestricted (#46976)
* fix
* adds back most of the code and limits it to only happen with impersonations, updates test to handle differing logic between the two flows
* removes test that is not longer needed
* more sandbox join table perms tests
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update outdated loki snapshots
* Update the line snapshot using the one from CI
* try bumping the timeout to reduce flake
* Revert the linechart snapshot since it's still flaking
* skip the flaking story for themed line chart
* skip the flaking story for line chart
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
If the content-type header is missing entirely, let it through. We're
not trying to be overly restrictive here, just prohibit things that are
clearly not GeoJSON.
Fixes https://github.com/metabase/metabase/issues/46918
* Add new strings for v50.20
* update translations for 50.21
---------
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Throw reasonable error when incompatible temporal unit is used in time unit param
* Remove redundant function
* Add :curated flag to catch exceptions
* Update test/metabase/query_processor_test/date_bucketing_test.clj
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Address review remarks
* Update test
* Make vars private
---------
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* use ee api to find custom reports collection
* remove deprecated hook
* fix race condition
* add unit tests
* fix circular dependency
* fix unit tests
* another plugin strategy
* Remove unused file
* Refactor getCurrentUser command to a TS function util
* Refactor createAlert command to a TS function util
* Refactor createModerationReview command to a TS function util
* Reuse createModerationReview
* Refactor createPulse command to a TS function util
* Remove references to deleted files
* Improve typing
* Kondo Config cleanup
* Restore warnings for with-log-messages-for-level until #28827 is merged
* Updated clojure.test hooks
* Fix kondo warning
* Time for me to learn to spell
* use ECharts tooltip on cartesian charts
* fix tooltip lag when changing hovered series
* facelift tooltip
* fix static viz crashes due to tooltip option
* facelift
* pie chart echarts tooltip
* facelift
* facelift
* facelift
* tweaks
* e2e specs
* return back the pie other slices tooltip
* more specs
* specs
* fix pie total
* fix tooltip overflow
* fix specs
* fix infinite rerender in embedding mode for charts that have warnings
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Beautiful log message capturing ⭐😻
* Don't have log.cljs try to load log.capture, don't know how to make that work
* Remove restriction against use in parallel tests
* Fix tests using invalid syntax
* Port legacy tests
* Make this stuff work with Cljs
* Fix bad syntax
* Convert usages of old version of with-log-messages-for-level to new version
* Update other stuff to use the updated macro
* Fix stuff
* Fix Cljs tests
* Fix world's largest test
* Appease Kondo
* Fix comment
* ClojureScript: only emit capture code in dev builds
* perf: Use more efficient iteration and formatting in CSV export
* perf: Replace clojure.data.csv functions with more efficient implementations
* fix: Fix formatting for integer numbers in scientific notation
* concat
* we fixed this!
* format
* links
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* refactor: memoize getSteps selector
* screen.getXXX.click() => userEvent.click(...) to avoid missing act() warning
* use mockSettings so that MetabaseSettings.setAll is called and we avoid the unknown features warning
* Add type picker to comparison modal
* Adjust offset input labels based of comarison type
* Pass comparisonType to column picker
* Add current period picker
* Use deepCompareEffect to avoid rerender loops
* Add moving average helpers
* Add moving average clause to query
* Prevent less than 1 period moving average from being created
* Fix unit test for CompareAggregations
* Use the ComparisonType type for the comparisonType arg
* Hardcode the mapping for offset <-> moving-average column types
* Convert column types in the comparisonType callback instead of a hook
* Remove unused parameter
* Automatically change to 2 when setting an invalid value for moving average periods
* Use expression parts in test
* Use named arguments for movingAverage
* Fixup test for moving average offset input
* Fix type issue
* Pass on includePreviousPeriod to getAggregations
* Add defaultTemporalBucket helper
* Add helper to pick default breakout
* Move breakout to the front or add one if it does not exist
* Add updateQuery to AggregationPicker props
* Add tests for creation and reordering of temporal breakouts
* Reuse defaultTemporalBucket in withDefaultTemporalBucket
* Reorder query operations to make sure the correct names get generated
* Remove dependency on referential equality of breakout
* Rename updateQuery to onQueryChange
* Remove onAddAggregations and onUpdateAggregation for onQueryChange
* Extract query update in to separate function
* Remove on{Add,Update,Remove}Aggregation in favor of onQueryChange
* Remove aggregations from summarize details
* Fix e2e tests
* Remove conflicting test
* Fix recent clause in unit tests
* Fix superflouous check for columnAndBucket
* Future proof breakout check
* Remove reliance on details in the useSummarize hook
* Remove details in unit tests
* Removal of default summarization should not trigger a query change
* new version of data_access migration + run ANALYZE beforehand
* relax constraints on migration timestamps
* more migration rewrites
* fix bug in migration
* fix another bug and remove extraneous logging
* reorder migrations to fix ID validation
* start adapting the changes to mysql as well
* wip
* add more ANALYZE calls
* revert logging changes
* finish duplicating work for mysql + a bit of clean up
* create H2 versions of migrations without the ANALYZE calls
* remove ANALYZE call from view_data.sql (probably not necessary)
* fix mysql syntax
* fix another mysql bug
* another mysql fix
* Don't exclude inactive tables for data_access
* Don't write rows for Administrators group
* Don't write rows for Admins (H2/MySQL)
* disable triggers before perms migration
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* add setup commands
* fix settings definition
* update environment variables for cli
* handle instances not being ready
* update error messages
* add more specific loading messages
* loading spinner state
* improve error message
* use a fixed demo setup token
* remove extraneous spinner
* update status checks
* update container messages
* update wait timing
* create api keys
* extract constants
* remove manual steps
* Add anonymous tracking + other things. will need to clean up
* Modify SDK for better structure
* remove line from print.ts
* Update webpack.embedding-sdk-cli.config.js back to production
* Add types and add quick note
* Fix a typo
* Add index file, simplify types, use an array
* Add safer json parsing
* use delay of 100ms between each setup call
* Suggestions from review
* ensure that cli works
* Attempt to fix jest errors
* Remove node-fetch from sdk code to hopefully get unit tests working again
* add database connection
* add connection details handling
* refactor asking for database connection info
* apply actual database id for syncing schema
* fix failing database sync step
* allow table selection
* create model for each table
* handle errors in model creation
* fix incorrect model display name
* create x-rays based on user data
* consolidate instance setup message
* workaround for inquirer eventemitter issue
* add sample components
* fix yarn.lock file
* add sample components
* set spinner to fail state when instance setup fails
* use a transparent background by default
* update cli welcome messages
* add import notice
* refactor imports
* update snippet to add dashboard dropdown
* cli add credential file to users .gitignore
* add docs on quickstart
* clear screen with console.clear
* address code review feedback
Co-authored-by: Nicolò Pretto <info@npretto.com>
* allow node 18 lts
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Fix tooltip styles leaking to popover
* Fix snapshot now showing the tooltips and dropdowns
* Update the loki snapshot using built storybook for consistency
* Show new model loading state while fetching
* Show new model loading state while fetching
* Add unit test
* Use `DelayedLoadingSpinner` instead
* Update the test
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Prettier
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add Timeseries insights even when Custom Columns exist
WIP
Fixes 46244
I don't know why it was (maybe still is) required that the :other columns be empty before providing insights.
* might be a solution to changing trendlines when data doesn't change
The insights code that powers the frontend trendline requires taking a sample of the dataset.
This sampling has some randomness associated by design, and should maintain its randomness. But it makes some sense to
use the exact same sample when the input has not changed at all, hence trying this memoize approach out.
Might not be the final solution, but it's a start
* add test to show that insights are computed
* use java.util.random with a seed to keep insights stable
* Add testcase from https://github.com/metabase/metabase/issues/43915
* Try a custom formatter if the default formatter for OffsetDateTime doesn't work
* Add testcase
* Make auxillary formatter private
* Catch a more specific exception
* Throw IllegalArgumentException on invalid dates
* Use strict resolver and fix test
* Empty commit to assign credit where it's due
---------
Co-authored-by: Chris Truter <chris@metabase.com>
* Dowload Row Limit Env Var
Adds `MB_DOWNLOAD_ROW_LIMIT` to enable changing the row limit on downloads and alert/subscription attachments (not the
rendered tables, but the .csv, etc.).
Based on: #44982 (Thanks, @r-kot)
Partially implements: #28144
The difference in this PR compared to #44982 is that the download limit applies to all downloads, the only exception
being when the limit is above `qp.i/absolute-max-results` (1048575, based on Excel's limitation); in such a case, the
user supplied limit is only used if the download is csv or json, and `qp.i/absolute-max-results` is used for xlsx.
This PR also fixes alert/subscription attachment limits; prior to this, they were set to the in-app limit of 2000
rows, but now they will follow the user supplied download-row-limit.
This PR also adds a test to the downloads-and-exports test namespace, confirming that they follow the supplied limit,
or the max limit if none is supplied.
* add test confirming the default limit works
* fix test to use the download-row-limit
* address review feedback
* Update src/metabase/public_settings.clj
Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>
* add test covering case where download-row-limit is unset
---------
Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>
* Revert "Remove unused humanizeCoercionStrategy and its test suite (#36136)"
This reverts commit 5375347e33.
* humanize coercion strategies, add e2e test
* unit test adjustment
* pr feedback
* conditional dismissal step
* check for download status in the container
* try fixing flakes again
* try again
* chartPathWithFillColor already verifies chart has been rendered
* Introduce subpath-safe URL utils
* Open data source links from a notebook safely on a subpath
* Open browse model links safely on a subpath
* Uniform naming
* Use the existing URL helper
* Trim url
* Add unit tests
* Add one more test case
* Fix test name 😅
* Make URLs non-optional strings
* Export Formatting Correctness
WIP
Several bugs exist around the correctness of export formatting, this aims to fix them.
Fixes#27374
When a model has viz settings/metadata changed after save, the download won't incorporate said changes. At least it's
clear that scale doesn't work.
* number formatter merges settings from columns too
* column formatting on aggregated cols works in xlsx now too
* fixes 43040 by grabbing global-type-settings in csv formatter
* Refactor
* Fix wrong colors when passing empty SDK `colors`
* Clean up CSS custom properties
* Add missing color mapping to the SDK
* Add missing colors to the main app
* Only override additional dynamic colors when values are passed
* Fix unit tests
* Address review
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Move the compare to the past aggregation to the bottom of the dropdown
* Fix missing icon property
* Hide 'Compare to the past' when there are no temporal columns
* Update copy in tests
* Add test for questions with no temporal columns
* Fix unit test for AggregationPicker
* Allow adding a compare aggregation when the first breakout is not temporal too
* Rename to 'Compare to the past' in custom column action
* Rename to 'Compare to the past' in column drill
* Update copy in tests for actions and drills
* Move canAddTemporalAggregation to CompareAggregation utils
* Use aggregations from the query always
* Remove unnecessary displayName check
* Last Used Param Value doesn't fill in with default value if cleared
Fixes#46368
The Last used parameter value is cleared when the frontend sends `nil` as the parameter's value. This works except in
cases where a user clears a parameter with a default value. In this case, we want to indicate that the parameter is
cleared, so we store `nil` in the appdb in such a case.
* Update src/metabase/query_processor/dashboard.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* add assertion showing that 'nil' with default is still stored
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Fixes: #46558
In #45575 we introduced casting uuid fields for drilldowns.
However we failed to include a check that the rhs was itself another
field of uuid type and so no casting should occur. This broke joins on
models when the database_type of the field may not be known.
* Add tooltip to the data step cell
* Conditionally display tooltip
* Add unit tests for the data step
* Add unit tests for the join step
* Move the abstraction to the NotebookDataPicker
* Cover the tooltip behavior with E2E test
* Update test
* Add test boilerplate
* Rename test and add assertions
* Add broken test for models
* Add unit of time tests
* Deduplicate code
* Add date parameter tests
* Add test for multiple stages
* Fix model test setup and refactor getColumnsInfos
* Remove redundant assertions
* Format codee
* Extract createQuestion and createModel
* Add unit of time model tests
* Add test for model
* Extract constants
* Add a warning
* Build dataset queries with mbql
* Introduce queryOrders
* Revert redundant change
* Improve Question creation
* Fix spammy GitHub Actions comments
* [DEBUG] trigger conventional commit reminder
* fix syntax errors
* Mock update
* mock another update
* 3rd update
* test if the comment would be updated only if the actual commet changes
* Remove all code relating to testing
* Fix type not compatible with node 18
* Update new Metabase base colors
* Make hover and selected state consistency
* Fix wrong embed frame border color
* Refactor: extract props
* Fix border color
* Fix pivot table cell hover color
* Fix smart scalar text contrast
* Add smart scalar tests + update text color design
* Theme dashboard card menu
* Remove dashcard action menu tests since they don't work in loki
* Fix storybook background color for dark theme
* Update loki snapshots
* Disable untestable tests
* Make sure all storybook inherit all necessary base styles
* Skip untestable tests
* Update loki snapshots
* add setup commands
* fix settings definition
* update environment variables for cli
* handle instances not being ready
* update error messages
* add more specific loading messages
* loading spinner state
* improve error message
* use a fixed demo setup token
* remove extraneous spinner
* update status checks
* update container messages
* update wait timing
* create api keys
* extract constants
* remove manual steps
* Add anonymous tracking + other things. will need to clean up
* Modify SDK for better structure
* remove line from print.ts
* Update webpack.embedding-sdk-cli.config.js back to production
* Add types and add quick note
* Fix a typo
* Add index file, simplify types, use an array
* Add safer json parsing
* use delay of 100ms between each setup call
* Suggestions from review
* ensure that cli works
* Attempt to fix jest errors
* Remove node-fetch from sdk code to hopefully get unit tests working again
* add database connection
* add connection details handling
* refactor asking for database connection info
* apply actual database id for syncing schema
* fix failing database sync step
* allow table selection
* create model for each table
* handle errors in model creation
* fix incorrect model display name
* create x-rays based on user data
* consolidate instance setup message
* workaround for inquirer eventemitter issue
* fix yarn.lock file
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
== Goal ==
Prevent admins of Metabase Cloud instances with Metabase Cloud Storage
from accidentally breaking their upload configuration, since in this
scenario that is automatically managed by Metabase Cloud to use the
attached data warehouse.
== How to test ==
On a subscription with Metabase Cloud Storage:
* The "Uploads" section in the left sidebar of `/admin/settings` does
not show
* A "not found" page is shown at `/admin/settings/uploads` where one
would otherwise expect the "Uploads Settings" page
* `PUT /api/setting` with an `uploads-settings` field in the body fails
with `403 Forbidden` and the setting is not changed
* `PUT /api/setting/uploads-settings` fails with `403 Forbidden` and
the setting is not changed
* `GET /api/setting` contains an element with `key: "uploads-settings"`
and a value
* `GET /api/setting/uploads-settings` responds with the value
* `GET /api/session/properties` contains the `uploads-settings` field
Closes: https://github.com/metabase/harbormaster/issues/5121
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* truncate total text in pie chart
* add story
* Update Loki Snapshots
* add e2e test
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Add unit test for #46574
* Handle non-numeric values in NumberFingerprint
* Be more realistic about NumberFingerprintDisplayInfo type
* Add comment on why the types are unknown
* add setup commands
* fix settings definition
* update environment variables for cli
* handle instances not being ready
* update error messages
* add more specific loading messages
* loading spinner state
* improve error message
* use a fixed demo setup token
* remove extraneous spinner
* update status checks
* update container messages
* update wait timing
* create api keys
* extract constants
* remove manual steps
* Add anonymous tracking + other things. will need to clean up
* Modify SDK for better structure
* remove line from print.ts
* Update webpack.embedding-sdk-cli.config.js back to production
* Add types and add quick note
* Fix a typo
* Add index file, simplify types, use an array
* Add safer json parsing
* use delay of 100ms between each setup call
* Suggestions from review
* ensure that cli works
* Attempt to fix jest errors
* Remove node-fetch from sdk code to hopefully get unit tests working again
* fix yarn.lock file
* revert capture groups change
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
== Goal ==
Currently the config file (`MB_CONFIG_FILE_PATH`) only allows adding or
changing databases, but not removing them. Since connected databases
can be managed as a service by us, it becomes necessary to delete any
information Metabase has about them, including any stored credentials.
Since this is a destructive operation with no way to revert (without
restoring from a backup), we raise the bar for accidental deletion and
require not just a (boolean) flag, but a magic string to be set.
== How to test ==
1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
2. Start your Metabase instance.
3. Verify the database shows up in the "admin" section and it works.
Create a dashboard and questions using the database.
4. Add a `delete: "DELETE_WITH_DEPENDENTS:${name}"` line to your
database configuration section.
5. Restart your Metabase instance.
6. Verify the database no longer shows up in the "admin" section.
Notice that questions using it are gone, too.
References: https://github.com/metabase/harbormaster/issues/5173
* WIP Open notebook data source in new tab
* WIP
* Implement the new functionality with `pickerInfo`
* Use destructured value
* Add the simplest possible E2E test as PoC
* Smoke test
* Add basic coverage for questions
* Add basic coverage for models
* Source question has been archived scenario
* Add basic permissions test
* Expand to the user without wite query permissions
* Fix title
* Add another permission scenario
* Add E2E coverage for a source being native
* Add coverage for the native model
* Address review comment re: `pickerInfo`
* Extract `getUrl` to utils
* Add support for the middle/scroll click
* Add unit tests
* Add a negative test case
* Fix title
Fixes#46440
couple things come together here.
Bad escaping:
The `{` characters are used for format strings and placeholders like
```clojure
impl=> (translate "fr" "After {0}" ["Bob"])
"Après Bob"
```
And if you want _literal_ {'s in your translation string, you must
escape them like "'{" or "'}" using an apostrophe. But this isn't the
end of our worries.
in `resources/zh-CN.edn`, we actually have _two_ translations for the
original string:
```clojure
"User lookup filter. The placeholder '{login'} will be replaced by the user supplied login."
"用户找过滤器，占位符 '{login'} 将替换为用户提供的登录名。"
,,,
"User lookup filter. The placeholder '{login}' will be replaced by the user supplied login."
"用户查找筛选器。占位符“{login}”将替换为用户提供的登录名。"
```
And this later has unescaped curlies.
The fix for that is at
https://poeditor.com/projects/po_edit?per_page=20&id=200535&id_language=31&filter=&order=&search=User+lookup+filter
but will need to wait for the new source translation string in this
instance with the corrected escaping.
```clojure
;; simulate what happens at runtime
impl=> (translated-format-string "zh_CN"
"User lookup filter. The placeholder '{login}' will be replaced by the user supplied login."
{})
"用户查找筛选器。占位符“{login}”将替换为用户提供的登录名。"
impl=> (MessageFormat. *1 (locale "zh_CN"))
Execution error (NumberFormatException) at java.lang.NumberFormatException/forInputString (NumberFormatException.java:67).
For input string: "login"
;; the problematic zh-CN string
impl=> (MessageFormat. "用户查找筛选器。占位符“{login}”将替换为用户提供的登录名。")
Execution error (NumberFormatException) at java.lang.NumberFormatException/forInputString (NumberFormatException.java:67).
For input string: "login"
;; minimal reproduction
impl=> (MessageFormat. "“{login}”")
Execution error (NumberFormatException) at java.lang.NumberFormatException/forInputString (NumberFormatException.java:67).
For input string: "login"
```
* fix: mongo ObjectId should be orderable
Fixes#46259
Mongo ObjectId derive :type/TextLike but that can't be an orderable
type, so we mark the specific Mongo type as orderable for malli schemas.
* Fix test
* prevents users from moving items to the trash if they are in the trash already and archives instead of moves in cases where item is moved to or from the trash
* reverts detecting if dragging in or out of the trash not setting the collection id, this will be prevented on the BE instead
* adds test coverage for dragging and dropping items to/from/within the trash (one of which is failing until we can make a BE change)
* Throw away `collection_id` when marking archived
This is a hack around a frontend issue. Apparently, the undo
functionality depends on calculating a diff between the current state
and the previous state. Sometimes this results in the frontend telling
us to *both* mark an item as archived *and* "move" it to the Trash.
Let's just say that if you're marking something as archived, we throw
away any `collection_id` you passed in along with it.
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
### Description
This change should fix some explosions in stats due to phantom fields, while also making things a bit more rigorous.
- Record the table schema in query_field - especially useful for unknown tables
- Allow null table names in query_field - they might be phantoms, or they might not
- We still have the option to filter them out in tools we build on top
- Tighten up matching logic to consider schema
* Update popover dark theme on part of ParameterFieldWidget (#45813)
* Add filter visual tests
* Simplify embed CSS colors
* [WIP] theme filter popover
* Move theme to global node, so it affects portal too
* Fix embed colors
* Theme part of ParameterFieldWidget
* Add story for parameter with dropdown
* Update loki snapshots
* Update more popover style
* Update loki snapshots
* Remove outdated snapshots
* Update loki snapshots with production build
* Fix loki not working with `userEvent`
* Theme mantine popover
* Improve dark theme text-brand visibility
* Update loki snapshots from master change
* Fix new colors not working in SDK
* Fix flaky Loki tests
* Filter popover dark theme part 2 (#46284)
* Theme Date all options
* wip add month and year filter
* Fix wrong CSS custom property declaration
* Test hover state
* Update Loki snapshots
* Theme date month and year
* Add quarter and year tests
* Theme date single and date range
* Style relative date
* Update frontend/src/metabase/admin/datamodel/components/filters/pickers/DatePicker/DatePickerHeader.styled.tsx
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Complete theming filter popover night mode (#46365)
* Style unit of time and number widget
* Attempt to fix flaky Loki snapshots
* Address white background checkbox on dark theme
* Style search filter single value
* Fix mis color
* Update snapshots
* Remove duplication on play functions since they're long
* Milestone 2 polishing (#46382)
* Matches the footer padding with header padding
* Update snapshots
* Add E2E to prevent further problem
* Try to reduce flaky Loki tests
* Remove unused color shades (#46421)
* Update no results illustration to work better on dark theme
* Update loki snapshots
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Refactor dashboard helpers to TypeScript
* Reuse existing helper
* Refactor cy.createQuestionAndDashboard to a function helper
- there was no good place to put it, so I also created new helpers/api directory
- and I moved all helpers using cy.request (with 1 exception, where a CSV download helper also does a bunch of assertions) in there
* Merge TS command definitions into a single file
* Export types
* Make dashboardDetails optional
* Export StructuredQuestionDetails
* Remove duplicated function
* Extract createNativeQuestion to separate file
- 1 exported helper per file, no exceptions!
* Reuse const
* Add test for unit of time parameters
* Improve assertions
* Extract createDashboardWithParameters
* Add tests for time parameters
* Fix hardcoded parameter_mappings ids
* Format code
* Extract checkButtonVisible
* Improve test names
* Add tests for location parameters
* Reuse const
* Update test name
* Add WidgetStatusButton
* Add unit tests for WidgetStatusButton
* Rename handler
* Remove redundant assertions
* Add iconSize prop
* Fix styling
* Replace WidgetStatusIcon with WidgetStatusButton
- Remove WidgetStatusIcon
* Fix tooltip issue
* Fix tooltip for good
* Use default compact button size
* Prevent overflow
* Fix alignment issues in parameter sidebar
- Remove iconSize prop
- Simplify StatusConfig
* Update padding in Unit of Time input to make it consistent with other inputs in dashboard parameter sidebar
* Update unit tests
* Rename WidgetStatusButton to WidgetStatus
* Explicitly specify what is a button
* Update helpers in tests
* Update when buttons are displayed according to the new spec, update tests
* Simplify label
* Remove utils
* Adjust padding
* Update reset icon
* Make the button gray when color is undesired
* Remove old definitions
* Add missing export
* Remove unused type
* Add test for location (single value)
* Add test for id (single value)
* Add more assertions
* Add test for id (multiple values)
* Improve name
* Add tests for number (single and multi value)
* Add tests for text (single and multi value)
* Rename file
* Update comment
* Update test name
* Use unique ids
* Use new color name
* Add a test for chevron icons alignment
* Lift className up
* Make types more specific
* Sort attributes
* Revert "Make types more specific"
This reverts commit 2e6994b5ed.
* Revert "Use new color name"
This reverts commit 2cc68df6ba.
* Remove TODO
* Introduce editFilter util, refactor chevron assertions to work with any number of chevrons
* Connect label to the default value input in ParameterSettings
* Add a whitespace in the default value label when required "(required)" is shown
* Add parameter sidebar tests for unit of time
* Add parameter sidebar tests for date filters
* Extract checkParameterSidebarDefaultValue
* Add parameter sidebar tests for location - single values
* Add parameter sidebar tests for location - multiple values
* Use constants for test case names
* Add parameter sidebar tests for id parameters - single values
* Add parameter sidebar tests for id parameters - multi values
* Add parameter sidebar tests for number parameters - single values
* Add parameter sidebar tests for number parameters - multi values
* Add parameter sidebar tests for text parameters - single values
* Add parameter sidebar tests for text parameters - multi values
* Extract checkDashboardFilters
* Use checkDashboardFilters in time parameters
* Use checkParameterSidebarDefaultValue in checkDashboardFilters
* Use checkDashboardFilters everywhere
* Sort helpers
* Rename checkDashboardFilters to checkDashboardParameters
* Rename checkOnlyOneButtonVisible to checkStatusIcon
* Add test suite for sql filters, coverage for text filter - single value
* Add a whitespace in the default value label when required "(required)" is shown - SQL filters
- Similar to 930dd3a50a
* Add test suite for sql filters, coverage for number filters
* Add test for sql filters - date
* Remove unused things, rename placeholder to labelOrPlaceholder
* Apply label only to 1 input
* Add test for sql filters - field parameters
* Fix date tests
* Make all sql filters tests work
* Add assertions for required without default in text filters
* Add assertions for required without default in date filters
* Add assertions for required without default in field filters
* Allow clearing required SQL filters without default value
* Add a test to ensure comma-separated numeric values work
* Update unit test
* Update helper
* Update e2e tests
* Use != null and == null checks with defaultValue instead of just checking if its truthy/falsy
* Run `yarn test-visual:loki` and then `yarn loki approve`
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Add button boilerplate
* Add getCanResetFilters selector
* Move canResetFilters to utils
* Add explicit return types
* Implement the button action
* Rename
* Simplify action
* Optimize: use single action
* Use parameterHasNoDisplayValue to account for empty strings
* Run `yarn test-visual:loki` and then `yarn loki approve` after merging master
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Rename getHiddenParameterSlugs to getTabHiddenParameterSlugs
* Take all tabs into account
* Add unit tests for canResetFilter
* Format test name
* Fix non-human-friendly aria-label
* Add assertions for "Reset all filters" button's visibility
* Unnest helpers from describe
* Add a test for comma-separated values
* Improve typing
* Fix typing
* Add tests for multiple tabs and auto-apply-filters on/off
* Improve test name
* Remove redundant util
* Remove redundant condition
* Verify card results get filtered
* Test that all tabs are affected
* Fix test case with autoApplyFilters: false
* Add tests for resetting to default value
* Fix wrong initial state in reduce
* Fix updated parameters not being applied
* Add reset all filters assertions
* Remove unused attribute
* Run `yarn test-visual:loki` and then `yarn loki approve`
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Run `yarn test-visual:loki` and then `yarn loki approve`
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Refactor dashboard helpers to TypeScript
* Reuse existing helper
* Refactor cy.createQuestionAndDashboard to a function helper
- there was no good place to put it, so I also created new helpers/api directory
- and I moved all helpers using cy.request (with 1 exception, where a CSV download helper also does a bunch of assertions) in there
* Merge TS command definitions into a single file
* Export types
* Make dashboardDetails optional
* Export StructuredQuestionDetails
* Remove duplicated function
* Extract createNativeQuestion to separate file
- 1 exported helper per file, no exceptions!
* Reuse const
* Add test for unit of time parameters
* Improve assertions
* Extract createDashboardWithParameters
* Add tests for time parameters
* Fix hardcoded parameter_mappings ids
* Format code
* Extract checkButtonVisible
* Improve test names
* Add tests for location parameters
* Reuse const
* Update test name
* Add WidgetStatusButton
* Add unit tests for WidgetStatusButton
* Rename handler
* Remove redundant assertions
* Add iconSize prop
* Fix styling
* Replace WidgetStatusIcon with WidgetStatusButton
- Remove WidgetStatusIcon
* Fix tooltip issue
* Fix tooltip for good
* Use default compact button size
* Prevent overflow
* Fix alignment issues in parameter sidebar
- Remove iconSize prop
- Simplify StatusConfig
* Update padding in Unit of Time input to make it consistent with other inputs in dashboard parameter sidebar
* Update unit tests
* Rename WidgetStatusButton to WidgetStatus
* Explicitly specify what is a button
* Update helpers in tests
* Update when buttons are displayed according to the new spec, update tests
* Simplify label
* Remove utils
* Adjust padding
* Update reset icon
* Make the button gray when color is undesired
* Remove old definitions
* Add missing export
* Remove unused type
* Add test for location (single value)
* Add test for id (single value)
* Add more assertions
* Add test for id (multiple values)
* Improve name
* Add tests for number (single and multi value)
* Add tests for text (single and multi value)
* Rename file
* Update comment
* Update test name
* Use unique ids
* Use new color name
* Add a test for chevron icons alignment
* Lift className up
* Make types more specific
* Sort attributes
* Revert "Make types more specific"
This reverts commit 2e6994b5ed.
* Revert "Use new color name"
This reverts commit 2cc68df6ba.
* Remove TODO
* Introduce editFilter util, refactor chevron assertions to work with any number of chevrons
* Connect label to the default value input in ParameterSettings
* Add a whitespace in the default value label when required "(required)" is shown
* Add parameter sidebar tests for unit of time
* Add parameter sidebar tests for date filters
* Extract checkParameterSidebarDefaultValue
* Add parameter sidebar tests for location - single values
* Add parameter sidebar tests for location - multiple values
* Use constants for test case names
* Add parameter sidebar tests for id parameters - single values
* Add parameter sidebar tests for id parameters - multi values
* Add parameter sidebar tests for number parameters - single values
* Add parameter sidebar tests for number parameters - multi values
* Add parameter sidebar tests for text parameters - single values
* Add parameter sidebar tests for text parameters - multi values
* Extract checkDashboardFilters
* Use checkDashboardFilters in time parameters
* Use checkParameterSidebarDefaultValue in checkDashboardFilters
* Use checkDashboardFilters everywhere
* Sort helpers
* Rename checkDashboardFilters to checkDashboardParameters
* Rename checkOnlyOneButtonVisible to checkStatusIcon
* Add test suite for sql filters, coverage for text filter - single value
* Add a whitespace in the default value label when required "(required)" is shown - SQL filters
- Similar to 930dd3a50a
* Add test suite for sql filters, coverage for number filters
* Add test for sql filters - date
* Remove unused things, rename placeholder to labelOrPlaceholder
* Apply label only to 1 input
* Add test for sql filters - field parameters
* Fix date tests
* Make all sql filters tests work
* Add assertions for required without default in text filters
* Add assertions for required without default in date filters
* Add assertions for required without default in field filters
* Allow clearing required SQL filters without default value
* Add a test to ensure comma-separated numeric values work
* Update unit test
* Update helper
* Update e2e tests
* Use != null and == null checks with defaultValue instead of just checking if its truthy/falsy
* Run `yarn test-visual:loki` and then `yarn loki approve`
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Use parameterHasNoDisplayValue to account for empty strings
* Run `yarn test-visual:loki` and then `yarn loki approve` after merging master
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Run `yarn test-visual:loki` and then `yarn loki approve`
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Run `yarn test-visual:loki` and then `yarn loki approve`
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Make number also have custom parameter values
* Render the correct label in the dropdown for number inputs
* Mock paramter values endpoint in number input test
* Add test for custom labels on number input
* Fix number input test
* Allow entering comma separated values using labels
* Only show custom labels in popover
* Use getLabel and getValue helpers to avoid indexing confusion
* Remove calls to parameter endpoints for number input
* Do not parse the value from the label
* Add support for number parameters in the source picker modal
* Filter number values
* Add e2e test for number parameters with custom source
* Remove card option for number parameters
* Set the correct initial value for number parameters without fields
* Do not render number parameter as search box when it is a list
* Fix e2e tests for number lists
* Remove e2e tests for card source dashboard filters
* Add tests for native filter custom source
* Show the dropdown when picking the default value for number parameters
* Clear values_source_config when changing parameter type
* Restore a parameters' values_source_config when changing the parameter type back
* Clear and restore values_source_config for template tags when chaning type
* Add e2e test for restoring of template tag sources_value_config
* Add e2e test for clear and restore of static-list for dashboard parameters
* Fix input type for numbers
* Fix broken test
* Display labels for ParameterValuePicker too
* Clear and restore values_source_type when changing parameter type
* Clear and restore values_source_type for tag parameters
* Clear and restore values_query_type when changing parameter type
* Clear and restore value_query_type for parameters tags
* Update tests to test for reset to also apply to values_source_config and values_query_type
* wip
* moves code to enterprise folder, adds enterprise guards, updates conditionals as to when to show the feature
* impls filter date logic, improves styling, mocks out real endpoint with collection items endpoint, updates table with isLink boolean rath
er than using custom item renderer, removes dead code
* use the actual url for stale
* so many things... updated design, custom table component, polish, misc. refactors, more correct api typing
* updates to pagination component and reverts some now unused changes to the base items table
* impls pr feedback
* fixes
* fixes failing unit test
* completes pr feedback from raffi
* implements second round of feedback from raffi
* adds tests and couple of small fixes / feedback
* migrate clean up action to enterprise plugin and fix related e2e test, also skip the collection items count query when we do not need the value
* finish main base case e2e test
* clean up helper code for manual testing and add OSS test to make sure feature is not available in OSS
* better handling of min-height on smaller screens
* adds internal undo notification for bulk archiving in the clean up modal
* reset pagination when items are archived/unarchived to keep users from getting stuck on a page beyond the current view
* pr feedback
* further cleanup
* unit test fix
* fixes issue where archiving all items in the modal and then undoing does not result in the stale items list getting refetched
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* hide notebook button for brand new questions
* test notebook button visibility for new questions
* move brand new check inside shouldRender
* test embedding SDK notebook button
* update unit tests
* add spy to shouldRender
* Fix coll permissions for audit collection
We've had a function in `models.collection` for a while that has taken a
permissions set and returned a set of collection IDs that the user has
permissions on. I refactored this recently, but didn't notice that it
was actually doing the permissions checks slightly incorrectly.
Specifically, because it only looks at the user's permissions and the
collection IDs and doesn't use `mi/can-read?` or `mi/can-write?`, it's
completely indifferent to whether a collection is an audit collection or
not.
This is arguably not a *permissions* issue: the two errors that could
come about here are:
- someone sees the audit collection even though the audit feature is
disabled, or
- someone is presented with the audit collection in a context where only
writable collections should be present - when they try to actually write
to it, it fails (since then we're doing the real permissions check).
The primary motivation for this fix was to prevent audit dashboards and
cards from appearing in the list of stale items.
* Fix hardcoded collection ID
We were creating a timeline in a fixed collection ID, that in tests
happened to be the ID of the Metabase Analytics collection.
* properly handle SMTP environment variables
* disable clear button when set by env var
* allow the SMTP form to validate with any combination of environment variables
* add unit tests for SMTP connection form
* BONUS: fix console error for jt keys
* fix types
* updates the ordering of create queries permissions options so that granular permissions is the last option
* fix unit test
* move granular option above no
* update static_embed_code_copied to use downloads (#45944)
* copy schema file for better diff
* adds downloads field to appearance in embed_flow schema
* update event to send `downloads` parameter, update e2e tests
* fix test for dwonloads: false
* fix: use correct schema version for embed flow analytics (#46053)
* update `dashboard_pdf_exported` and use it in embedding context (#46076)
* copy schema for better diff
* adds `dashboard_accessed_via` to the `dashboard_pdf_exported` event
* download_results_clicked event (#46132)
* adds download_results_clicked event + e2e tests
* make strings nullable
* adds maxLength
* switch back to if/then as it seems more readable
* refactor: use the resourceType to determine the download params instead of re-doing the logic
* fix ''question' is possibly 'undefined'.'
* split resource_type into resource_type and accessed_via to also consider interactive-embedding and sdk
* edit comment
* adds test for ad-hoc-question
* Update frontend/src/metabase/query_builder/actions/downloading.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* makes describe message more consistent
* "else if" => if
* fix describe block name
* convert downloads event to use enum for download_type and export_type
* resource_type should be an enum too
* fix line added in the wrong place during merges
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Update frontend/src/metabase/public/components/EmbedFrame/EmbedFrame.tsx
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* adds enum schema to dashboard_accessed_via
* fix accessed_via for sdk embeds
* update comments to explain why we send 0 when dashboardId is null
* only send dashboard_id as a number when it is a number, as expected by snowplow
* remove un-needed async/await from saveAsPDF function
* rename frontend/src/metabase/redux/downloading-analytics.ts -> frontend/src/metabase/redux/downloads-analytics.ts
* fix cy.log message
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* update embed frame pdf button colors to use --mb-color- vars
* change download pdf button color to brand color
* testing adding the images that actually differ
Previously we unpacked everything and picked up first entry as the directory
containing the data. This works well when archives are passed as is. However,
if a user unpacks and then packs an archive, there is a good chance that some
hidden directories will appear - and they are first in order.
So we skip those right now, and additionally check known markers of the
serialization data directory.
* Squash
* Clean up query modifier for validating card references (#46283)
* Fixup bad git merge on test expectation
* Tweak docstrings and de-densify logic
* Allow filtering on collection
* Add `effective_ancestors` to collection data
Note: I removed the `qv=` comparator. I think it'.s better to be more explicit
about what exactly we're asserting. In particular, I hadn't updated the
`:count` when we were filtering on collection type, so the count was
still the total for *every* collection even though the results were being
restricted by the collection. However the `qv=-data` function was still
reporting that everything was OK, because it was accounting for other
data from the App DB by just asserting that the count we'd specified was
lower than the actual count. I'd argue it's better for tests to fail
locally because I have garbage in my app DB (which I can clean up) than
succeed when they really shouldn't.
* Consolidate effective ancestors
For the Search API, we're currently returning slightly different
ancestors:
- it never has the Root Collection, and
- the ancestors don't have a `personal_owner_id`
We could change the API (and maybe will soon - it seems pretty
unexpected to have `effective_ancestors` sometimes act one way and
sometimes act another) but for now we can just reshape things to look
the same as they did before, while using the same code under the hood.
* echarts pie
* setup rendering pipeline for static pie chart (#43547)
* setup rendering pipeline for static pie chart
* remove more donut code from BE
* compute viz settings for static pie chart (#43548)
* compute viz settings for static pie chart
* create chartModel for pie chart (#43549)
* create chartModel for pie chart
* render static pie chart (#43550)
* render static pie chart
* render static pie chart legend (#43551)
* disable animation
* alter static-viz pie chart be tests to account for echarts-impl
The old implementation had a different implementation for legends, separate from the 'categorical/donut' js
render. Echarts nicely combines the legend into the pie render, so we don't need to worry about that stuff anymore.
Instead, I've written a simple set of assertions on the echarts pie implementation now.
* Get rid of this test that was attempting to use old code path
* echarts pie
* One last spot where we don't need old pie stuff anymore.
* add loki specs
* save snapshots
* fix small slices disappearing
* update snapshots
* add repro story for exteremely small slice
* add snapshot
* fix rounded total
* fix percent formatting in legend
* fix date dimension formatting
* fix binned and relative date dimension labels on legend
* add specs and snapshots for null and unaggregated dimensions
* hide overlaping labels on chart
* fix null dimension color
* show error when metric column has a negative value
* move legend to botom
* make dot margin smaller
* add columns to legend
* sort legend vertically
* dont use grid if only one row for legend
* never hide legend in static viz
* fix empty column culling
* show single legend item
* add long dimension name story
* fix long dimension name in legend
* right justify percentages in legend
* remove negative error and re-add stories
* handle negative values
* create chart definition for dynamic pie chart
* render chart with responsiveness
* add comment for borderWidth calculation
* center legend
* fix legend centering
* add story for missing currency formatting
* fix missing currency formatting
* fix graalvm crashing
* fix column settings computation
* add another currency formatting repro
* fix wrong order of col settings computations
* convert colors to hex
* fix colors missing from viz settings
* temp todo comment
* fix colors in dynamic viz
* update snapshots
* fix crashing due to invalid dimension/metric setting
* fix chart crashing for boolean dimension with labels on chart
* add repro
* fix regression in label formatting
* fix labels on small slices
* update specs
* emphasis state for slice hover
* recreate old label hiding logic
* include height measurement in getIsLabelVisible
* move d3 slices to chart model
* fix labels on static viz
* wip basic hover state and tooltip
* fix emphasis style getting stuck by memoizing everything
* update total display value
* fix stuck emphasis by setting notMerge to false for pie only
* handle other slice in tooltip
* fix bugs in total graphic
* use OTHER_SLICE_KEY const in static viz legend
* implement legend and connect to hover state
* implement click actions
* fix slice not clickable
* handle all zero slices
* add repro
* update specs
* update pie_chart.cy.spec.js
* update chart_drill.cy.spec.js
* add repro for single column legend
* use minimum 2 legend columns
* fix legend text overlap
* use more margin for single row legend
* update specs
* add repro for 28568
* add repro for 38424
* fix percent label not showing on large slices
* hide total text when chart is small
* replace comment about animations
* show warning for mixed positive and negative values
* delete old chart files
* fix type errors
* update d3 import
* update specs
* remove branch from uberjar.yml
* dont put single slice into other
* fix lint errors and pie render test
* update more d3 imports
* remove unused settings
* revert preview-card-width change
* remove unused import
* remove majorWidth option from formatting
* use debounceLeading refresh mode in ChartWithLegend
* remove TOTAL_GRAPHIC_OPTION constant
* delete unit test for pie chart
* fix failing test by doing set comparison instead
* fix typo in comment
* remove SUNBURST_SERIES_OPTION constant
* fix lint by removing unused eslint-disable
* move default color computation to getValue
* fix static pie test to show that we will always render the legend
* update pie border color for night mode and embedding
* fix styles for embedding case and add stories
* skip embedding stories
---------
Co-authored-by: Adam James <adam.vermeer2@gmail.com>
* Make an internal API for retrieving stale content
This introduces the `metabase.stale` module, which has one public
function (so far): `metabase.stale/find-candidates`. This takes a set of
`collection-ids`, a cutoff date, and pagination/sort info.
Next, we introduce an HTTP API endpoint that will figure out the appropriate
set of `collection-ids` for a given user/request, call this function,
and return the results in the shape we want.
Also: for e2e tests, we need a way to mark a card or dashboard as stale. I
think the easiest way to do this is to just create a testing endpoint
that we can call to update the `last_used_at` or `last_viewed_at` of the
card/dashboard to set it to ~7 months ago.
---------
Co-authored-by: bryan <bryan.maass@gmail.com>
* use mantine buttons for filter and summarize
* Restore fancy color hover states
* Add css modules
* oh the cleverness of me
* better var name
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
Expands and refactors `metabase.util.memoize` to support better cross-platform memoization.
**Better basic memoization**
Use `clojure.core.memoize/memo` (and its clone in CLJS) rather than `clojure.core/memoize` for
the "everything forever" strategy.
**Bounded memoization**
There are a few places (eg. `u/kebab-case-en`) where we want zero-overhead *hits* and expect the input space
to be fixed. To guard against runaway memory usage, this **bounded** strategy dumps the entire cache when it
overflows. It logs an INFO level note when that happens - the intent here is that the threshold should never
actually get hit!
**Fast JVM versions**
In the special case where the function to memoize takes exactly 1 argument which is a valid map key, we can
use `ConcurrentHashMap.computeIfAbsent` as the basis of the cache and have much less overhead.
`fast-memo` and `fast-bounded` implement this in CLJ; in CLJS they just call `memo` and `bounded`.
* Use function helpers
* Only assert the resize down
* Refactor repro 31701
* Remove flaky tag
* Better organize describe blocks
* Move custom `requestTimeout` config to the common block for resizing tests
* Refactor dashboard helpers to TypeScript
* Reuse existing helper
* Refactor cy.createQuestionAndDashboard to a function helper
- there was no good place to put it, so I also created new helpers/api directory
- and I moved all helpers using cy.request (with 1 exception, where a CSV download helper also does a bunch of assertions) in there
* Merge TS command definitions into a single file
* Export types
* Make dashboardDetails optional
* Export StructuredQuestionDetails
* Remove duplicated function
* Extract createNativeQuestion to separate file
- 1 exported helper per file, no exceptions!
* Reuse const
* Remove old definitions
* Add missing export
The `dashboard_load_id` is attached to the body for queries, not as a
query parameter. This correctly receives the `dashboard_load_id` and
uses it for `/query` calls.
* pull sdk info into data on-thread
* include api tests for query_execution and view_log
* Try mt/test-helpers-set-global-values!
- disambiguate names used in tests
* pass test + cleanup test helper
* make sure view logging is enabled in test
* fix conditional formatting color lookup with correct col names
The column names for color lookup are expected to be the same as the :name key of each column, and applying column
formatting broke this lookup. This fixes it.
* move overflow-x into pulse body div to exlude card title from scroll
* add a test for conditional formatting w/ a hidden row.
This still fails at the moment, working on a fix yet.
* apply conditional formatting in static viz even when col is hidden
Conditional formatting that depends on a hidden column was not being rendered in the static viz, but now we pass all
columns into the color selector fn, thus when rendering the table with the ordered/hidden columns, all other columns
will still use their appropriate conditional formatting
* I missed a linter warning
* fix tab rendering
* fix test
* fix up tests with some better structural searching
* Adds a middleware that sets 2 dynamic vars
- These vars will be used to insert context for embedding analytics
* migration adding embedding_client and embedding_version to qe and vl tables
* clarify function api
* headers come in -> dynamic vars -> written to db
When the vars are set we insert their values into query execution and
query_log under embedding_client and embedding_version.
* update update migrations
* add predconditions to migrations
* fix tablenames + dataset_test
* Code Review responses
- remove commented out requires
- clarify test for merge-count-maps
* Lock down the trash collection more
Oops. I was very thorough in checking that you couldn't modify the
trash collection in the sense of moving things to or from it. I missed
the most obvious thing: actually modifying the trash collection itself.
So if someone wanted, they could change the name of the Trash to
"Definitely Not the Trash".
* Remove the Trash Collection from serialization
Serialization tests were failing due to the previous change, but this
got me thinking, and I realized that serializing the Trash collection is
completely unnecessary. We want to create it programatically on
startup (which we are) and then nothing should change it afterwards.
I tightened this up by removing the condition here, thinking that CI
would catch it if it was necessary - but I'm getting failures in
development. Oops - didn't realize that this defmethod wasn't actually
applied during CI runs.
* Rewrite repro for 27643
* Add TODO for the remaining tasks
* Make a test name consistent with others
* Finish the E2E test for #27643
* Fix the test to use PostgresDB
Since H2 originally doesn't fail the test
* Attempt to fix a flake on CI by waiting for table to finish syncing
* Create QA snapshot with fully synced field metadata
* Fix failed tests from field metadata change
* Simplify question scenarios
* Simplify dashboard scenarios
* Fix permissions
* Rework titles
* Add comment about PG
* Add limit to speed things up
* Fix and improve dashboard assertions
* Fix and improve question assertions
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Bigquery Pagination Use next page to get proper results
Fixes 45953
WIP
* use new-page in values-iterator
* disable just the part of the test that is still flaky
Want to keep the other part of the test that does prove we're not getting duplicates due to incorrect pagination.
* Number formatter Reports correct Decimal places for negative Numbers
Fixes#41640
Prior to this, the number formatter would get the incorrect number of decimal places for negative numbers.
The way the decimal places count is derived works as follows:
- convert number to a string
- split the string on the separator (decimal, comma, whatever the instance has set)
- destructure that split into [int decimal]
- count decimal to get the number of decimals.
That all works for positive numbers, but for negative numbers, the regex was just splitting on 'not digits', so we got
an extra group due to the leading negative sign. This would incorrectly destructure the 'int' part of the number into
the count, thus we would always get 1 instead of the correct value.
This fixes it by just taking last instead of destructuring.
* correct comparison to always use abs value in conditonal check
* tiny incorrectness in the test
* Modify the E2E stress test workflow to allow running PostgresDB
* Fix syntax
* Add qa_db to workflow summary
* Only run necessary containers
* Fix QA DB snapshots running when QA_DB_ENABLED=false
* Use more generic option that's not only specific to postgres
* perf: Add explicit .entryAt method for SnakeHatingMap
Otherwise, SHM invokes .keySet on itself which triggers the contstruction of the list of keys on each map access.
* perf: Cache (un)successful requiring of EE in dynamic-ee-oss-fn
* perf: Add a quicker path for getting metadatas from cache
* perf: Minor improvements
It's based on a (global, synchronized) `WeakHashMap` using the query map
itself as the key. That gives the cache the same lifespan as the query
map, which is generally brief but often includes many calls to
expensive, cached functions like `visible-columns`.
* Make filter corrupted again™
* Speed up the test by limiting the number of rows
* Make the repro more correct, and more resilient
* Remove intercepts
* Improve assertions
* add webpack configuration for cli
* add requirements checking
* docker check
* docker start command
* port checks
* add getting started steps
* add final install message
* cleanup cli text
* handle force-closing prompt
* build the cli when running build-embedding-sdk
* wording updates
* add redundant prompt in case of small terminals
* remove redundant instruction
* make sure the user confirms the step
* change container name to metabase-enterprise-embedding
* migrate cli to es modules instead of webpack
* add shebang line for npx
* add cli dependencies to devDependencies
* Revert "add cli dependencies to devDependencies"
This reverts commit 5d5e952c2b.
* Revert "add shebang line for npx"
This reverts commit 766fdda09e.
* Revert "migrate cli to es modules instead of webpack"
This reverts commit 7d1b065163.
* Embedding Parameter Values set in Tokens work properly when `nil` or empty list
Fixes: #40292
If a locked parameter's value is set to `nil` or `[]` in the embedding JWT, we want to treat this as 'no value has
been set' on that filter, but we should still run the query as normal. Likewise, any linked filters should still
properly derive their values from the unfiltered values of the filter to which they are linked.
* when a parameter's value is nil or empty seq, don't include it
This is a partial solution as I still need to find the place that's still erroring on the empty list case in terms of
the linked parameter's values.
* Treat empty list like `nil` for the token param
When you're using a filter in the App, it makes intuitive sense that when you've not selected anything from a filter's
drop down list (or entered any text, etc.), then the filter is effectively not applied.
For token params, we should behave similarly; `nil` and `[]` can both be accepted, and signal that the user intends
for that filter to basically NOT apply.
This also means we should not pass the constraint to the linked filters, since it errors, but even if it worked, it
would return 0 results, making it impossible to properly apply the linked filter at all.
So, this change causes empty lists to be treated as nil, AND will remove any parameters with `nil` as a value before
being passed to the query processor, which results in the proper behaviour of not having constraints on the linked
filter, and having the 'parent' filter not filtering anything.
* Embed preview should always use params from the token
In making a screenshot, I discovered a slight bug in previews, where if the embed had been previously published, it
could pull embedding params from the db that don't match the settings on screen. So, always pull the settings from the
token, which updates as the user is making changes in the Preview Embed UI.
* for previews, merge the embedding params sources instead
* alter 'nil' behaviour to work just as if the key doesn't exist
* fix one spot where I was passing maps into set functions
* Adjust tests to check for consistent behaviour in dash/qstn embeds
* fix an e2e test
* fix up last 2 failing e2e tests
* add the test case of a locked param having no value set in token
* some further e2e changes
We have N+1 detection for simple hydration methods that checks to see
whether the hydration method made any DB queries at all. This makes
sense, because a simple hydration method is called on each instance
individually - if we make database calls *each time*, we're in trouble.
Where this runs into issues is with cached data, where the simple
hydration method is populating the cache - e.g. `:can_read` or
`:can_write` dereferencing `api/*current-user-permissions-set*`.
Currently we specifically exclude those hydration methods from the
check.
But after running into this again for a different simple hydration
method, I think there's a more robust approach:
- first, run the hydration method without counting DB queries
- then, run it again and make sure we don't make any additional DB
queries.
There is a potential downside here: if someone wrote a simple hydration
method with side effects, they'd potentially be very confused (in
development - this doesn't run in prod!) when those side effects
happened twice for a single hydration. But I *think* that's fairly
unlikely.
* handle errors when useMetabaseAuthStatus is used outside of sdk context
* address review comments feedback
* extract useSdkContext
* fix import type error
* polyfill useSyncExternalStore for React 17
* adds the shim
* use ReactDOM.render instead of createRoot for backwards compat
* use createRoot for React 18 environment
* refactor renderRoot compatibility
* remove semver dependency
* add small test for version
* use the same check for react 17
- babel is kept to transpile fonts file as swc emotion plugin doesn't handle our case correctly. I didn't find a root cause of wrong behaviour, but the safe past is to keep using babel + emotion plugin for one file and all others can be transpiled with swc
- react fast-refresh now handled by swc
- swc-loader can't read browserslist config from package.json, so we had to manually specify supported browsers in webpack
* Uninstall ReplayIO
* Remove Replay config
* Install and use custom Chrome v111
* Simplify comment
* Set a specific Chrome version commit
* Skip repro for #22517
* fix: replacing joins and conditions updates alias
Fixes#37074
* Only change alias if conditions or table changed
* Fix test
* Fix fe test
* Address PR feedback
* Remove can_run_adhoc_query from the FE
* Remove can_run_adhoc_query from the FE
* Disable can_run_adhoc_query on dashboard
Was impacting performance on dashboard load. Timings locally for me:
get dash: 3178 -> 199
get metadata: 3604 -> 1002
So dashboard went from 6 seconds to load to 1 second.
* Fixup test from disabling can_run_adhoc_query
---------
Co-authored-by: dan sutton <dan@dpsutton.com>
* adds database_id to recents, and question collection items
* don't show unjoinable things in recents either
* unskip repro
* fix types
* fix repro
* fix tests in model/recent_views
* add unit test for invalid join in recents
* fix tests in test/metabase/api/collection_test.clj
* update type
* improve unit tests
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Add `report_dashboard.last_viewed_at`
- defaults to `now()` (it's intended to be used as a timestamp after which
we're *sure* the item hasn't been used, not a definitive timestamp we're
sure the dashboard was actually used at)
- updated whenever the QP processes a query for a dashcard
---------
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* remove dynamic import to make html2canvas work on the sdk
* Revert "remove dynamic import to make html2canvas work on the sdk"
This reverts commit a8b511fa28.
* manually import html2canvas-pro and jspdf in the sdk
Previously, due to over-eager `.flush()` calls in Cheshire, we were
flushing the output stream for every `data.rows` entry. Typically this
resulted in 3 + N TCP packets sent, where N is the number of rows being
returned.
This PR rebuilds how we stream the JSON responses to avoid extra
`.flush()` calls. For the unit test in this PR, that reduced the number
of packets sent over the wire from 37524 to 4.
Fixes#34795.
* Post sync hook stub for implicit joins testing stub
* Add join alias to field lvalues to enable sorting on joined fields
* Disable :foreign-keys on Mongo
* Require :left-join support for implicit joins instead of :foreign-keys
* Update implicit joins tests
* Adjust sync-fields-test
* Update implicit joins feature check test
* Transform post-sync-hook to normal function
* Add foreign key relationships only for dbmses without :foreign-keys feature
* Update test to handle Oracle correctly
* Split convoluted fn
* Avoid unnecessary computations for datasets with no fks
* Update docstring
* Fix driver usage
* Transform :foreign-keys to :metadata/key-constraints in test data loading code
* Update sync_test.clj
* Update driver_test.clj
* Update moviedb.clj
* Update dataset_definition_test.clj
* Update fetch_metadata.clj
* Update fields_test.clj
* Update driver.clj
* Update driver/sql.clj
* Set sql driver join support to true for all joins
Deriving drivers are expected to set to false where applicable.
* Update sqllite.clj
* Remove foreign-keys from spark
* Remove :foreign-keys from presto
* Remove :foreign-keys from Athena
* Remove foreign-keys from big query
Reading docs it seems fk inference should be ok. Let's see the test results. Act based on that.
* Update test_metadata.cljc
* Set key-constraints to false for bigquery
* Add foreign keys to sqlite manually until sync is fixed
* Return driver require to Athena
* Correct typo
* Add naive primary key heuristic
* Update pk fk logic to handle name components correctly
* Add alias escaping to presto
* Add ordering to test
* Add order by to test
* Update test
* Remove use of rewrite-fields-to-force-using-column-aliases in order by fields
* Add exception to alias forcing
* Different approach to exception from alias forcing
* Alternative approach for prefixing idents in bigq
* All seelcted fields by desired alias
* Rewrite only fields not from this source table
* Update test
* Enable breakout-on-fk-field-test for :left-join drivers
* Add feature comment
* Explicit joins tests foreign-keys removal
* Update nested_queries_test.clj
* Update remapping tests
* Update tests
* Update tests to handle sqlite results format
* Disable metadata/key-constraints on sqlite during tests until
* Address remarks
* Remove mt/with-mock-fks-for-drivers-without-fk-constraints
* Update bigquery test
* Update tests
* Adjust row level restrictions
* Add parameterized-sql feature
* Update comment
* Update leftovers
* Order keys
* Remove foreign keys from frontend
* Fix FE unit
* Update driver changelog
* Address review remark
* Update test/metabase/query_processor/test_util.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update docs/developers-guide/driver-changelog.md
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update docs/developers-guide/driver-changelog.md
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update modules/drivers/bigquery-cloud-sdk/src/metabase/driver/bigquery_cloud_sdk/query_processor.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update docs/developers-guide/driver-changelog.md
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update test
* Update comment and reduce expression
* Update comment
* Address remarks
* Fix merge
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Derive MySQLEnum from Text instead of TextLike
Fixes#44807.
This change is in accordance with https://dev.mysql.com/doc/refman/8.0/en/enum.html describing the ENUM types as a
kind of string. All our string operations seem to work with MySQL ENUM fields.
* docs: extract requires a db with regex
* Update docs/questions/sharing/visualizations/table.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* tests for drivers with spaces in their identifiers
* Add driver feature
* Fix tests
* Remove ambiguous alias for sqlite
* Try to fix bigquery and sqlite tests not seeing left-join support
* Add explicit support for left-join because of foreign-keys during test hacks
* Handle left-join support weirdness
* Fix test
* Fix falsy tabSelected/tabRenamed checks
* Update URL management (WIP)
* Fix tests failing because of whitespace changes
* Fix tests failing because of parameter order
* Use one hook for parameters and tab URL management
* Fix tests failing because of parameter order
* Fix test failing because of whitespace changes
* Revert
* Fix DashboardTabs tests
* Fix tests failing because of parameter order
* Fix test
* Fix test failing because of whitespace changes
* Remove unit tests that should be e2e tests now
* Fix tab navigation
* Fix string encoding/decoding
* Revert test changes
* Consistent question search params
* Update DashboardTabs tests
* Fix `SyncedParametersList`
* Fix e2e tests
* Fix nav usage
* Fix types
* Remove `useSyncedQueryString`
* Fix test
* Move slug utils to dashboard utils
* Fix nav
* Revert nav change
* Fix e2e tests
* Fix public/embedded dashboards, handle site url
* Add repro test
* Bring back and fix `PublicOrEmbeddedDashboardPage` tests
* Move `SyncedParametersList` to qb
* Refactor EditoTabs
- get rid of options prop
- use named export
* Remove unused data-testid
* Remove redundant id and aria-labelledby
* Remove redundant props spread
* Bring back data-testid in use
- Thanks VSCode for randomly showing me search results or not...
* Fix navigation issue
* Add a repro for first issue
* Adjust EntityPickerModal root element so that it works with "be.visible" assertion in Cypress
* Add a repro for second issue
* Refactor tests
- Fix not enough history entries issue
* Change order of tests to start with simpler one
* Fix invalid request issue
* Add more assertions
* Format code
* customize popover z-index
* make popover z-index customizable
* make the popover z-index customizable
* add tests for popover z-index
* also specify z-index for mantine popover
* Reuse connection for test data loading & JDBC data loading overhaul
* Revert silly change
* Remove unneeded logic for recording which datasets are loaded or not-yet-loaded
* Ok so I guess we do need the code to detect duplicate dataset loading after all
* B I G improvements
* Appease Kondo
* Test fixes 🔧
* More test fixes 🔧
* Fix dataset-already-loaded?-test
* Fix test
* Fix Presto JDBC and greatly improve test data load speed for it
* Postgres test fix 🔧
* Fix compilation errors
* MySQL fixes 🔧
* Fix syntax error
* MySQL fixes 🔧
* Implement `dataset-already-loaded?` for Athena and cache the set of existing databases
* Fix Redshift
* Fix Presto JDBC
* Vertica test fix 🔧
* Fix Oracle dataset-already-loaded?
* Test fixes and parallelize some API tests.
* Test fixes 🔧
* Oracle test fix? 🔧
* Another test fix 🔧
* Stop putting random spaces in code bodies this is not JavaScript code
* with-temp-vals-in-db should merge in original value of Database settings
* The most test fixes 🔧
* 3 or 4 test fixes 🔧
* Add schemas around stuff to catch errors
* Appease Kondo
* Implement dataset-already-loaded? for vertica
* Implement dataset-already-loaded? for SQLite and Snowflake
* [WIP] Testing driver filters
* Only test returned values in CI
* Add :uuid-type for dbs that support that type (test oracle's support)
* Try again for oracle test, redshift does not support uuid-type
* Add tests, support where available
* Add empty/null tests
* Adjust test query for athena
* Reset logging
* Fix comments
* Add Athena uuid conversion
* Address PR review
* Add comments
* Address PR comments
* Use varchar for portability. Add inline uuid casting for athena
---------
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Field Filter Value Normalization in Embedding Endpoints
Fixes 27643
Since we use query-params to pass paramter values, we have to normalize the parameter map values.
This PR is a WIP, as I'm trying to find a way to do this hollistically/properly.
First commit here is mostly as a marker for where to start working on this properly, as I think the json parsing won't
catch all cases, nor should the normalization happen without some attempt to use a schema to properly validate, maybe
even coerce things.
* read strings if boolean or numbers, replace blank str with nil
Also update the common dashboard param values fn to use embedding params from the URL blob if the app db doesn't have
embedding params. This happens upon first preview where nothing will have been written to the Appdb yet.
* Add query-param normalization test, add related embedding test
* schema didn't work in this case after all.
Worth a shot, but don't want this to block the PR, so reverting to the previous schema
* Add comment pointing out the downsides/reasons for this solution
It's nto a perfect solution, but it's probably the best we can get at the moment, without adding a new filter
type (boolean) and/or improving the design of hte embedding endpoints and how filters/parameters work
When we run queries, we preprocess them, during which the sandboxing
middleware adds information that we use when checking permissions
later on in query processing.
When we checked data permissions before saving a card, we were checking
permissions on the raw query, without preprocessing. Without the data
provided by the sandboxing middleware, the permission check sometimes
failed (depending on the permissions for other tables in the DB).
Note the big hack here: preprocessing can fail, for lots of reasons.
This broke a *lot* of tests that were passing in invalid queries. If an
exception occurs during preprocessing, we catch and just use the
original query like we were before.
* refactor: Simplify code (#44275)
* Remove dead code
* Remove values same as default options
* Simplify hook API: Removing unused argument
* review: Remove redundant code
* Update look and feel copy (#44277)
* Update static embed tab name
* Update copies + rearrange settings
* Use Mantine Select
* Fix E2E tests by applying new copies
* Fix unit tests
* Fix more E2E tests
* review: Add a comment on Mantine `value` quirk
* Adjust spacing and relevant tests
* Move `transparent` theme to its own toggle option (#44362)
* Reorder the function to match the visual order
* Make function name consistent
* Introduce the new background option (do nothing yet)
* Remove redundant exports
* Tighten up values to reduce confusion
* Make the parameters order consistent
* Only show background option on dashboads (spec)
* Allow dashboard embed background to be transparent
* Make filter background and header/footer transparent
* Fix type error
* Fix unit tests
* Fix E2E tests
* Fix checkerboard pattern not showing when no background
* Add unit tests
* Add missing copy update
* Remove remnants from outdated solutions within the PR
* Improve code comment references
* Fix E2E tests since the last copy change
* Fix type errors from merging master
* Update light theme style (#44488)
* Add background to dashboard filter on light theme
* Make position sticky works with transparent background
* Fix sticky filters not working on theme=transparent
* Fix unit tests
* Simplify IntersectionObserver components + fix E2E tests
* Add storybook for embedded dashboard
* Clarify the use of ExplicitSize context
* Remove unused code
* Formatting
* Review: Better mock `isWithinIframe`
* Update dark theme to match the new design (#45134)
* Update light theme checkerboard colors
* Update checkerboard colors
* Consolidate dashboard light/dark border style
* Use `--mb-color-text-primary` for light theme dashboard
* Update embed dashboard dark theme primary color
* Migrate dashboard text-secondary color
* Migrate text-tertiary color
* Migrate gold, axis, and split line to CSS custom properties
* Fix failed e2e tests
* Fix static viz not working
* Update loki snapshots
* Remove unnecessary CSS custom property
* Make diff more consistent
* Address review: reusing type
* Support backward compatibility when passing `theme=transparent` (#45214)
* Match preview query with the actual code query
* Handle backward compatibility when `theme=transparent` on dashboard
* Add unit tests
* Address review: Make tests less repetitive
* Update Metabase banner upsell UI (#45424)
* Prepare folder for nested components
* Update upsell UI
- add `fullWidth` variant
- align title to the left
* Fix Upsell banner having horizontal scroll on full width
* Add new Metabase banner upsell component
* Fix unit tests
* Address review: Fix typo
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
* Address review: Move upsel UI to `admin/upsells`
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
* Remove unnecessary ESLint disable comment
We can remove it since the file is in admin folder, and is already excluded from the rule.
* Address review: remove default value property
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Address review: Extract magic number
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* More UI updates (#45450)
* Update dark theme text colors to match the new design
* Update powered by metabase logo
* Update loki snapshots
* Fix unit tests
* Fix formatting
* Fix wrong dark theme metabase logo badge color
* Address review: Compress SVG with svgo
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Fix E2E tests
* Update loki snapshots
* Clarify Metabase base colors with comments
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Add more tests (#45628)
* Add static embed question stories
* Add loki snapshots
* Add transparent theme for static dashboard tests
* Add transparent theme tests for static questions
* Assert `background` option in analytics tests
* Remove tests for logic that has been moved
* Update loki snapshots
* Fix failed E2E tests from merging master
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Refactor EditoTabs
- get rid of options prop
- use named export
* Remove unused data-testid
* Remove redundant id and aria-labelledby
* Remove redundant props spread
* Bring back data-testid in use
- Thanks VSCode for randomly showing me search results or not...
* add pdf export button to static embed and public sharing (#43743)
* add export button to embed frame, it works on static embed and public sharing
* move css around to make dashcard border show up on static/public embed pdf exports
the library we use to render an html node to an image doesn't seem to support box shadow
the original fix was done in ParametersAndCardsContainer, but that component is not used in the static/public embeds
* fix types
* fix night theme pdf export
- use padding for the title instead of starting to draw the image from 60px, to keep the bg color
- change the css to also apply the bg-color to the first child of the EmbedFrame, to make it work when exporting to pdfs, where the EmbedFrame is not part of the exported dom
* refactor: exportTabAsPdfButtonText function
* give a default to the filename, even though it should always be there
* exportTabAsPdfButtonText -> getExportTabAsPdfButtonText
* introduce .WithThemeBackground to not have a css rule too much generic
* feat: adds endpoint to download results of a dashcard part of a public dashboard (#44156)
* Flag to hide download buttons (#44206)
* pass down downloadsEnabled to hide downloads buttons in public and embedded questions or dashcards
* Update frontend/src/metabase/dashboard/components/DashCard/DashCardMenu/DashCardMenu.tsx
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* PLUGIN_DOWNLOADS_ENABLED -> PLUGIN_RESOURCE_DOWNLOADS
* adds comment to areDownloadsEnabled function to explain what it does
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* logic to handle downloads url parameter (#44363)
* minimal support for downloads url parameter
* updates StaticEmbedSetupPane test to epxect downloads=false instead of hide_download_button=true
* fix prettier mistakes
* tmp fix to the tests until we have the new analytics events
* fix eslint issues
* update embedding snippets e2e
* update embedding questions e2e
* try to clean up the mess of downloads/downloadsEnabled
* adds `hasPremiumFeature` check to the plugin, adds unit test for the logic
* adds TODOS for analytics
* unit tests for downloads flag (#45178)
* fix download pdf button not showing up when titled=false and there's only one tab
* unit tests for downloads flag on dashboards and questions
* adds test about the container id, not sure if we should keep it
* adds a comment to explain why we enable premium features in the tests
* move downloadsEnabled outside of the Boolean cast as it's already a boolean
* E2E tests for public/static downloads (#45240)
* wip e2e tests
* make downloadUtils ts friendly
* assertNotEmpty function
* adds tests for public questions
* e2e tests for static embed downloads
* publicUid -> publicUuid
* fix assertNotEmpty
* fix parameter not renamed
* Apply suggestions from code review
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* remove duplicated test
* fix another typo i forgot to rename
* fix another rename issue
* convert e2e-downloads-helpers to ts
* refactor: put method calculation in getEndpoint function
* adds note to downloadUtils
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Update e2e/test/scenarios/embedding/embedding-snippets.cy.spec.js
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* change requires to import to reflect that we changed to export the source code in downloadUtils
* makes downloadsEnabled prop more consistent across components
* colocate downloadsEnabled with hideDownloadButton in embed-display-options
* defaults displayOptions.downloads to true to simplify logic
* remove unused queryString parameter in tests setup function
* reword TODO
* Revert "defaults displayOptions.downloads to true to simplify logic"
This reverts commit aebf4cf3cf.
* fix png export when on dark theme (public/embed) not having the dark … (#45497)
* fix png export when on dark theme (public/embed) not having the dark background
* move the border-radius fix for the dashcard export to saveChartImage
* remove comments
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Do not display alert icon for models
* Add e2e repro for #37893
* Put the guard higher
* Fix unrelated test
* Go back to keeping the logic in `shouldRender`
* Remove whitespace
* Get rid of tolerance
* Use range rect and element rect instead of clientWidth/clientHeight/scrollWidth/scrollHeight
* Fix types
* Mock Range.prototype.getBoundingClientRect
* Mock getBoundingClientRect in unit tests
* Add repro for #32252
* Have a test for archiving a question and a collection
* Refactor cy.createCollection and cy.archiveCollection to TS
* Refactor collection and user helpers to TS
* Keep collection helpers together
* Sort code
* Format code
* Use new createCollection helper
* Add return type to createCollection
* Fix typing
* Fix#32252
* Fix typing
* Deprecate openNewCollectionItemFlowFor
* Revert "Fix typing"
This reverts commit ecf3179868.
* Fix TS error
* Refactor cy.createCollection and cy.archiveCollection to TS
* Refactor collection and user helpers to TS
* Keep collection helpers together
* Sort code
* Format code
* Add return type to createCollection
* Fix typing
* Deprecate openNewCollectionItemFlowFor
* Add a repro for #43219
* Fix#43219
* Create questions in parallel
* Make 2 elements scrollable so that the 'Make sure to make a selection ...' element is not weirdly cut off when scrolling horizontally to see inside of parameter-mapper-container
* Fix assertion
* Be more specific which element is to be scrolled
* Avoid realMouseWheel which does not work reliably in CI
* updates the fe to skip refetching the graph when updating the collection graph, adds tests
* Moves the skip_graph flag to be part of the body.
- the FE tooling makes it awkward for PUTs to include query params
* fixes failing unit tests
* pr feedback
---------
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* Add Temporal Units List to Dashboard Parameters Schema
Closes: #44361
This adds an entry to the Dashboard Parameter schema so that we can validate
the list of temporal_units that might be passed from the frontend when adding
or updating the `Unit of Time` type parameters.
* This is the way
the real reason the schema wasn't working... well, it was, but the un-decoded versoin of parameters was being passed
to `update-dashboard` because inside defendpoint, you can use map destructuring and bind the whole map to whatever
with :as keyword.
If you decode/validate on all of the symbols in the destructuring, that's great, but if you then pass the map bound
with `:as whatever`, you don't get that decoded goodness, just the data that our http middleware produces (that is, it
might not be the shape you expect).
So, here I've created a private Schema for the :body of the request and use the same schemas to validate each key in
that map. The result should be the same but we can pass a fully decoded/validated map along.
* change test to look for the error in the right place
* automate updating translations for major releases
* add reviewer to PR
* auto translate
* auto-translate works!
* update
* automate translation updates in CI
* fixes
* whitespace
* sort scripts
* just checkout the branch to begin with
* fix tests!
* fix import
* parameterized static questions
* sdk parameter values
* use parameters to construct parameter input
* add a unit test for passing the correct parameters
* fix parameter parsing
* move TEST_PARAM to top of file
* update readme docs
* address code review
* add skip_graph to coll perm PUT
- also no longer ignores revision and groups
so we don't print this warning:
2024-07-11 16:17:32,293 WARN api.common :: Unexpected parameters at [:put "/api/collection/graph"]: [:revision :groups]
* namespace should be a keyword
* decode the keyword
* add a typehint to avoid reflection
* Adds test for skip_graph
* handle revision 0 🤦
* Do not reset `table.columns` when some column is deleted
* Do not reset `table.columns` when some column is deleted
* Fix
* Fix
* Make API calls more obvious
* do not allow to render empty SaveQuestionModal content
when question is already saved and QB rerenders still open SaveQuestionModal there is a moment when content is empty before modal is closed
* fixup! do not allow to render empty SaveQuestionModal content
* parallelize coll perm graph group lookup w/ core.async
* respond to code review
* add + uses ExecutorCompletionService map function
* rename test
* more tests
* no concurrency for cljs
* fix timeout + add more tests
* use claypoole
* cleanup
* add kondo hooks for claypoole
* cleanup
* Can't use libraries that don't have a license
Revert "add kondo hooks for claypoole"
This reverts commit 9d93b55b28.
* ignore unused value
-- TODO: check back in on claypoole to see if the kondo config gets merged
* fix linter
This was a huge footgun.
We were accidentally limiting *ALL* queries run in our transducer to
1000 results. This included the query to fetch permissions, so when we
checked permissions we were checking a small percentage of your *actual*
permissions (e.g. 1000 / ~6300 for me on stats).
This comes in two parts.
**Rebuilt query logic**
I rewrote the query exec logic to use one `future` and one `a/go` block,
reporting results to a `promise`. This is easier to follow and doesn't leak
threads like the original implementation with two `future`s.
The single future is guaranteed to exit. It blocks on the BigQuery call.
That will either complete or throw an exception. Either the exception or
the BigQuery result is sent to the promise, and then the future is
complete. The only reference to that future is overwritten after the `promise`
resolves, so the `future`, its closure, and its thread don't persist longer than needed.
**Return a reducible result**
Previously the results were returned as a lazy sequence, but that brings
a lot of `seq` overhead, and is a risk of retaining the entire result
set in memory until the query is all done. Above all, it doesn't support
`reduced?` to explicitly drop any further results.
The `thunk` function style of `qp.reducible/reducible-rows` doesn't fit
neatly here, so I implemented the reducible logic directly on top of the
`Iterable` BigQuery results.
**Verifying**
This is a bit tricky. I've been testing it like this:
- Add a BigQuery database to my instance
- Create a saved question and note its ID (say, 123)
- Create an API key in (EE) Metabase and save it in an env var
- Install the `siege` load testing tool
- Install VisualVM and monitor my Clojure REPL process's thread count
- `siege --concurrent=15 --time=30s --header="x-api-key: $API_KEY" "http://localhost:3000/api/card/123/query POST"`
That will create a bunch of new threads while it's getting called, but
within a minute of the last queries they'll gradually drop out of the
thread pools and it will return to the baseline number of threads.
* create sdk load question
* store question result in sdk context
* remove dataset data type
* update question
* handle navigate to new card
* add loading states
* fix on query change
* update question change logic
* refactor question loading and query params
* fix ad-hoc question in dashboards
* extract useLoadQuestion into a separate hook
* cleanup run question on load
* revert changes in interactive question component
* store original question in the question result state
* turn the question into ad-hoc questions
* refactor run question methods
* add a separate isQueryRunning state for better ux
* isQueryRunning should default to false
* simulate re-running the query in unit tests
* add tests for rendering multiple valid questions
* refactor sync question method
* extract raw series and computed pivot logic
* add getCardAfterVisualizationClick
* separate single and multiple question tests
* fix import paths
* is question native
* pass queryResult instead of queryResults
* temporarily remove checkNotNull for aggregation picker to avoid sdk crash
* Dashboard Paramter Value Now Works in Preview Embed
Fixes: #44962
Prior, a common util fn used both in the embed and preview_embed endpoints was causing the preview_embed to error out
when fetching dashboard parameter values because it was checking if embedding was enabled. This check is needed for
the normal embedding endpoints, but we want the preview embed endpoints to work without needing to publish the
dashboard first.
* pass proper args
* put args back in proper order.
oops
* change form back to previous, but keep preview check
* autocorrect mistake? Or perhaps a silly typo?
* change one test to show that the preview flag is working
Didn't create a whole new test for this flag as it is provable just by adjusting this test to not bother enabling
embedding on the dashboard.
* improve the docstring
* Log changes when they make sense + async coll perm-graph revision
* - Make revision-number consistent
Decouples the creation of a revision number from the (potentially
expensive) writing of before and after for the coll perms-graph
revision.
* get id from the perms revision entity
* Move the revision id creation into the transaction
The revision id creation should only happen iff the perm updates went
through successfully.
* write a test that checks the incomplete revision
* remove sleep from tests
* Migrate Model Indexes to RTK
* Sorting out types... I think?
* remove unneeded comment
* replacing the selector with an initiate to ensure that there is data
* catching and throwing error
* Updating unit tests
This introduces a new metadata on commands, `:requires-init`. If set,
we'll run `metabase.core/init!` before running the command. `init!` will
initialize the database and run our normal startup, though it won't
actually start the Metabase server.
This allows you to use
```
env MB_CONFIG_FILE_PATH=... java -jar metabase.jar import /my/metabase/export
```
We'll run the normal init process (creating the internal user, loading
from the config file, etc), then run the command.
On my machine this adds about 2-3 seconds to the time it takes to run an
`import`.
* start adding some util fns to download-exports-test ns
* wip improving test util fns
* add subscription attachment util fn
* add tests to catch download/export differences
* viz-settings from card and dashcard merged properly
* revert these changes which probably aren't needed
* split test: downloads can be parallel, email attachments cannot
* normalize viz-settings no matter their origin
* Update test/metabase/api/downloads_exports_test.clj
Co-authored-by: Alexander Solovyov <alexander@solovyov.net>
* Incorporate some review feedback
* rendered table cards also respect column names
* Clean up table render tests and remove unhelpful render test utils
The big removal of code in the render test utils namespace is a clean up task that could've happened sooner, but this
table test adjustment is the perfect opportunity to clean up.
I had made the test utils before I truly knew how best to use `mt/with-temp`, so it had a lot of poorly implemented
'generators' that were doing a poor job of creating mock data for the rendering functions anyway.
So, reworking a few table tests here to just use a temp card is way more inline with the rest of the codebase, and
actually improves confidence that the tests are testing on well-formed data to begin with.
---------
Co-authored-by: Alexander Solovyov <alexander@solovyov.net>
* add admin-only user-local setting that controls perms tutorial
* Adding modal and banner, user setting for dismissable banner
* copy changes
* adjust so modal only appears on data permission page
* e2e adjustments, adding banner test
* never show the modal or banner on new instances
* add test
* e2e adjustments, copy update
* more PR feedback
* move modal/banner settings & tests + clarify tests
* query for v50 migration execution works on mysql
- move changelog lookup into metabase.db.liquibase
* no race conditions in changelog-table-name
The previous implementation of this function was hitting this
liquibase issue: https://github.com/liquibase/liquibase/issues/2966
* revert changelog-table-name
- simply lookup the changlog row
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* Add a test verifying that fonts are loaded
* Verify font is mentioned in HTML
* Bring back working assertions
* Add wait
* Disable speedy
* Disable CSP and emotion cache
* Revert "Disable CSP and emotion cache"
This reverts commit 573e99bfa4.
* Revert "Disable speedy"
This reverts commit 6f9962b869.
* Don't use local fonts
* Revert "Don't use local fonts"
This reverts commit 17c58aa73d.
* Force Lato
* Fix storybook file name
* ESLint
* Add more !important
* Revert !important
* Add font-family assertion
* Move test
* try fixing cross-version tests
* restore legacy metrics
* its not legacy in v43
* Dont got echarts either
* newfangled echarts
* run cross-version tests on code changes in the tests
* enable slack message
* don't always run cross-version tests
* whitespace
* add refresh auth token function
* provide a default implementation for refreshing auth tokens
* rename getRequestToken to fetchRequestToken
* add docs on customizing jwt authentication
* clarify docs, add error handling and failing tests
* update tests to pass
* re-export FetchRequestTokenFn type
* re-export EmbeddingSessionToken type
* perf: pass uuid to BE to enable metadata caching during fetching dashboard
* fixup! perf: pass uuid to BE to enable metadata caching during fetching dashboard
* Add wildcard on interceptions
* Add tests
Before
====
```
2024-07-04 15:54:39,084 INFO metabase.server :: Shutting Down Embedded Jetty Webserver
Exception in thread "Thread-9" java.util.IllegalFormatConversionException: f != java.lang.Long
2024-07-04 15:54:39,122 WARN db.liquibase :: (#object[liquibase.Liquibase 0x2b7e6614 liquibase.Liquibase@2b7e6614])
at java.base/java.util.Formatter$FormatSpecifier.failConversion(Unknown Source)
at java.base/java.util.Formatter$FormatSpecifier.printFloat(Unknown Source)
at java.base/java.util.Formatter$FormatSpecifier.print(Unknown Source)
at java.base/java.util.Formatter.format(Unknown Source)
at java.base/java.util.Formatter.format(Unknown Source)
at java.base/java.lang.String.format(Unknown Source)
at clojure.core$format.invokeStatic(core.clj:5770)
at clojure.core$format.doInvoke(core.clj:5764)
at clojure.lang.RestFn.invoke(RestFn.java:423)
at metabase.db.liquibase$wait_for_all_locks.invokeStatic(liquibase.clj:258)
at metabase.db.liquibase$wait_for_all_locks.invoke(liquibase.clj:251)
at metabase.db.setup$release_migration_locks_BANG_.invokeStatic(setup.clj:175)
at metabase.db.setup$release_migration_locks_BANG_.invoke(setup.clj:170)
at metabase.db$release_migration_locks_BANG_.invokeStatic(db.clj:91)
at metabase.db$release_migration_locks_BANG_.invoke(db.clj:86)
at metabase.core$destroy_BANG_.invokeStatic(core.clj:90)
at metabase.core$destroy_BANG_.invoke(core.clj:81)
at clojure.lang.AFn.run(AFn.java:22)
at java.base/java.lang.Thread.run(Unknown Source)
```
And now it will correctly log
```
2024-07-09 16:04:28,345 INFO db.liquibase :: Waiting for migration lock(s) to be released (max 0.4 secs)
```
Why no tests?
====
There _is_ a test that would have caught this. And in fact running at
the repl will catch it:
```clojure
liquibase=> (do (require 'clojure.test 'metabase.db.liquibase-test)
(clojure.test/run-test metabase.db.liquibase-test/wait-for-all-locks-test))
Testing metabase.db.liquibase-test
ERROR in (wait-for-all-locks-test) (Formatter.java:4515)
:h2
Will timeout if a lock is not released
expected: (= :timed-out (liquibase/wait-for-all-locks sleep-ms timeout-ms))
actual: java.util.IllegalFormatConversionException: f != clojure.lang.Ratio
```
But this _passes_ in CI:
```
❯ clj -X:dev:test :only metabase.db.liquibase-test/wait-for-all-locks-test
Finding tests took 23.5 s.
Running 1 tests
Running before-run hooks...
Ran 1 tests in 1.461 seconds
3 assertions, 0 failures, 0 errors.
{:test 1, :pass 3, :fail 0, :error 0, :type :summary, :duration 1461.175042, :single-threaded 1}
Ran 0 tests in parallel, 1 single-threaded.
Finding and running tests took 25.0 s.
All tests passed.
Running after-run hooks...
```
And that's because the log level in testing is FATAL and this is an INFO
level log.
Discussed ways to improve this like adding a null logger during CI but
we aren't ready to increase CI times for that just yet.
* Move focus to separate helper
* Use focus method to do initial focus
* Use focus method for other _editor.focus calls
* Move focus to componentDidMount
* Cancel focus timeout on unmount
* Add delay to broken test
* Switch to requestAnimationFrame
* Remove wait in test
* fix combined cards crash when some cards have invalid viz settings
* fix mock column was not metric
* ensure combined scalar column has proper type to handle the case when the card returns empty results
This PR also specifies the 'filled' variant for the question duplication
modal, since the previous default 'filled' variant for submit button was
added to make that specific button filled.
* update e2e test for command pallete to use a valid setting
* remove the setting from the ui, make it always true, unless set false from env
* remove enable-nested-queries from admin section on fe
- fix unused oldValue -> _oldValue
* tests aren't allowed to change enable-nested-queries
* fix test + disallow setting nested queries from code
* we cannot set enable-nested-queries
- set the env var instead
* add checks for the enable-nested-queries setting
* make resolve-source-cards work with enable-nested-queries setting
* fix recents uri interception
* return enable_nested_queries setting to client
* adjust fetchmock url matcher
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* Nicer error messages for linting migrations
In general, just throw exceptions instead of using clojure spec here.
You'll only get one error per run, but that seems fine.
I did keep some specs around, but run them slightly differently. Rather
than validating the whole collection of changeSets at once with `s/+`,
just `doseq` through the changeSets and validate each one separately.
That way, the value that's presented as erroneous is much smaller (a
single changeSet) and it's easier to see what went wrong.
* Update bin/lint-migrations-file/test/lint_migrations_file_test.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update bin/lint-migrations-file/src/lint_migrations_file.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Require preConditions for certain liquibase change types to encourage idempotence (#44578)
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* remove `!`s from non-side-effecty fns
* add preConditions to a few v51 migrations
---------
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Split unrelated changes into separate PR
* Fix test since user perms set reset fix split into other PR
* Fix syntax error
* Test fixes 🔧
* Test fix??? 🔧
* Test fix 🔧
* lint: add stylelint support for emotion components
* drop comment
* fix pattern order
* Use one version of typescript
* use :focus not :outline
* Use the logical CSS property "inset-inline-start:" (not "inline-start:" which doesn't exist) as replacement for the physical css property "left:"
* Use mb-color-focus for focus color
* Drop dead code
---------
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* fix: limit xrays on agg models to source fields
Fixes#43793
1. `table-like?` was checking aggregations but not breakouts. This meant
that legacy metric definitions would try to apply to the second stage
which is generally illegal and would cause things like implicit joins
to come in where they make no sense.
2. Trying to bin a `count` metric-like model would give a bin span of
0.0 which would blow up those questions.
3. Trying to self reference breakout fields, in the precense of a
question's aggregation meant that most fields would not be exposed to
the next stage and should be ignored. (similar to 1.)
* Fix tests
* Add csv-stringify package
* Split values at comma
WIP: Parse values using json
* Add label to ParameterValue and add ParameterValue to value config
* Use custom label in FieldValuesWidget
* Allow passing displayValue from outside of RemappedValue
* Allow passing displayValue in ParameterFieldWidgetValue
* Get custom value label in FormatterParameterValue
* Allow rendering custom displayValue in FieldValuesWidget
* Add custom ParameterValue types
* Fix ParameterValuePicker type
* Fix jest import for csv-stringify
* Add missing RTKQ exports for paramters
* Use direct implementation of useGetParameterValues
* Fix ValueSourceModal tests
* Add tests for value parsing and stringifying
* Use custom label in list too
* Render the custom label in a parameter value
* Setup parameter mocks in embed test
* Add info about splitting values with comma
* Add model hint when user is using labels
* Add test for cusom labels on dashboards
* Handle label-value pairs in test
* Test custom labels in parameter search input
* Use the label to parse the value for text inputs
* Fix comma's to commas
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Use strong instead of styling link
* Use getInput instead of ignoring eslint rule
* Use arrow function
* Add newlines between tests
* Use uppercase for test value
* Name value
* Use getLabel and getValue helpers to avoid indexing confusion
* Pull out confusing condition
* Move out getLabel helper
* Be more explicit about non-existent config values
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Move setupParameterValuesEndpoints to setup
* Add loading state
* Do not use alert to implement info box
* Fix formatting
* Remove undefined from getValue signature
* Use getLearnUrl
* Remove dependency on column
* Make custom field labels work in the absence of fields
* Support custom values for non-field parameters
* Add test for native question parameters
* Add newline in test
* Add test case for when show-metabase-links is false
* Just fall back to rendering value when fetching label, or in case it failed
* Add test for loading state of FormattedParameterValue
* Do not fetch custom label for FormattedParameterValue
* Add table with ip addresses
* Add test for IPAddress filter with custom labels
* Add test for custom labels on type/Quantity fields
* Remove getInput helper in test
* Remove unused hasCustomLabel value
* Split up into common and premium tests
* Make fieldValues a map for performance
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* fix: param-fields when native model is aggregated
Fixes#42829
The param-field lookup required result_metadata to match
filterable-columns in order to find parameter targets. This adds a
fallback where if the param field is missing from result_metadata,
then we look at filterable-columns.
* Unskip test
Per discussion in Slack, I made a mistake with the migration IDs as I wanted to backport to 50 but used 51 IDs.
This PR uses 50 IDs and places the migrations in the correct place.
Not sure if it's mergable in this state because Stats has run with the PR containing the incorrect migration IDs.
* Refine the reproduction for 44794
* Move skip to top-level again
* Remove superflouous click
* Verify wether or not the recents tab exists and is open
* Make the test description more descriptive
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
* Move question details to a variable
---------
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
* User Parameter Values Are Unique Per Dashboard
Fixes: #43001
Might be related: #44858
User Parameter Values previously stored just the `parameter-id` and `user-id` as data to look up the last used
parameter value. This isn't sufficient as the parameter id is not guaranteed to be unique, in particular this is true
when a dashboard is duplicated: the parameters are copied to the new dashboard without changing the parameter id at
all.
This means that we need to also store the dashboard-id in the user_parameter_value table and use that to update/get
the last used value.
The migration removes existing entries to the user_parameter_value table because I want a non-nullable constraint on
the dashboard_id column, and existing entries will not have a value. The only way to backfill those values would be to
look through every dashboard containing parameters, and for every parameter check for a matching ID. Even if you can
do this, there's no way to disambiguate if the same paramter exists on 2 dashboards anyway, so one of them will be
wrong. I think it's not worth the trouble considering that removing the entries in this table doesn't break anything;
it just becomes a mild inconvenience that a user has to select a filter value again (since the dashboard will use the
default value).
* alter test to check for uniqueness across dashboard
This test makes 2 dashboards with parameters of the same ID and asserts that changing the value on dashboard-a does
not change the value on dashboard-b.
* adjust migration to pass linter rules
* remove the extra rollback on migration
* Update src/metabase/models/user_parameter_value.clj
Co-authored-by: bryan <bryan.maass@gmail.com>
* Update src/metabase/models/user_parameter_value.clj
Co-authored-by: bryan <bryan.maass@gmail.com>
* adjust parameters in test so we don't get logged warnings
* put update!/insert! in a transaction to avoid any race conditions
---------
Co-authored-by: bryan <bryan.maass@gmail.com>
* Support compiling SQL with inlined parameters directly
* Test fix
* Make `format-honeysql` a multimethod
* Add test for #32543
* Test also fixes#44915
* Add explicit test for #44915
* Better test for #44915
* More test fixes 🔧
* Fix Kondo error
* Optimize: compute chart title href only on mouseenter and focus
* Improve comment
* Improve comment
* Format code
* Update existing tests
* Extract helpers
* Update unit tests
* Make getHref argument-less
`Lib.diagnoseExpression` is stricter about this than the QP or FE
expression parser, making it reject expressions that tried to compare
datetimes to strings like `2024-07-02 14:34:17Z`.
Those used to work in 49.x, but a new `Lib.diagnoseExpression` call
while editing expressions caused it to regress.
Fixes#44916.
**MetadataProvider cache**
The reusable MetadataProviders is a generally useful thing - within some
dynamic scope like an API request, a `clojure.core.cache` instance can
be provided as `lib.metadata.jvm/*metadata-provider-cache*`, and that
will be used to cache the provider for each `database-id`.
Reusing those providers can significantly reduce appdb traffic,
re-fetching the same metadata in different `MetadataProvider`
instances.
**Dashboard loads**
The FE can supply `?dashboard_load_id=uuid_string` on requests for a
dashboard load:
- `/api/dashboard/:id`
- `/api/dashboard/:id/query_metadata`
- `/api/dashboard/:id/dashcard/:dashcard_id/card/:card_id/query`
That provides a good cache key across that burst of N+2 API requests
that are part of a single load. Currently it is used for two things:
- Caching `get-dashboard` (used by the first two requests)
- Selecting the `clojure.core.cache` for `*metadata-provider-cache*`
Both use a TTL of 10 seconds, which is generally enough for the first
screenful of a dashboard load, if not the whole thing. (Browsers will
only issue a handful of requests at once, so a few long-running queries
might delay later parts of the dashboard from loading until after the
TTL expires.)
* Make sure the query is native
* Convert the comment to JSDoc
* Add E2E reproduction
* Fix mapping
* Add test scenarios for the public sharing and embedding
* Simplify API payload
* Extract and significantly expand the reproduction scope
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* only send permissions updates for groups that have been modified in the graph
* Support partial permission graph updates on the BE (#44797)
* handles keeping track of advanced permissions changes to send relevant group data when updating the graph, merges response
* tests and optimizations
* respect group-ids set when fetching connection impersonations, sandboxes, and admin group perms
* fix typo
* add e2e test
* fix BE test
* impls pr feedback, reduces code needed since advanced permissions state is always reset on save
* last fixes
---------
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* recent views sometimes don't show special colls
colls with a non-null namespace are special: they can't interact with
normal collections, dashboards, cards, etc at all. (Currently the only
use for these we have is snippets.) At some point we could add a
`namespace` argument to this endpoint to support looking at recent
collections in a different namespace, but that'd be a bigger refactor
without a whole lot to gain. For now we can just exclude these
completely, which is what we want for current usages of the recents
endpoint anyway.
We won't exclude non-null types across the board, because sometimes
we'll want to see these. But we will be particular about when to display
them.
We'll never display the `trash` collection, but will display the
instance analytics collection when we're not trying to make selections.
* Hack an e2e test around an existing bug
So, the entity picker that pops up when you're selecting a collection to
put a Snippet into shows you "Recent" collections - which previously
included snippet collections.
We should either make the Recents API support a namespace parameter for
collections so it could actually support this case, or the entity picker
shouldn't show a recents tab here (realistically, how many collections
are people going to have for snippets that they would need this tab
anyway).
* Calculate metadata without running queries [JDBC]
* Appease Kondo
* Test fixes 🔧
* Overhauled Card metadata calculation code
* Test fixes 🔧
* Kondo fix 🔧
* Ok maybe I fixed stuff
* Work around https://github.com/aws/amazon-redshift-jdbc-driver/issues/118
* Maybe fixx the Cypress tests
* Experimental possible e2e test fix 🔧
* Test fixes 🔧
* Do things a little differently.
* Try re-enabling the field refs
* Add reproduction for #38989
* Use saveQuestion instead of clicking trough the UI
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Use openOrdersTable instead of manually clicking through the ui
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Create question programmatically in test
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Wrap non-latin characters in a span specifying working font
Fixes: #38753
CSSBox seems to have a bug where font fallback doesn't work properly. This is noticeable when a font does not contain
glyphs that are present in the string being rendered. For example, Lato does not have many international characters,
so the rendered version of tables (that show up in Slack messages) will not render properly, making the card
unreadable.
Since this appears to be a downstream bug, I've opted to work around this limitation by wrapping any non-latin
characters in a <span> that specifies the font family to be sans-serif, which should contain the glyphs to properly
render.
This leaves Lato in place for other characters.
For now, I figured it's worth trying this solution over using Noto for 2 reasons:
- we can keep Lato, which has been the decided font style for the app for some time (this keeps things consistent
where possible)
- the Noto font containing all glyphs is quite a large font (>20mb, I think) and it would be nice to avoid bundling
that if we can.
* stub installed fonts test
* typo
* Do wrapping, but now per-string, and in Clojure data not html string
I've decided that a reasonable solution is to still wrap strings containing non-lato characters. But it's not done
with str/replace to the html string but rather in a postwalk over the Hiccup data prior to rendering.
Seems like a decent compromise of issues without patching CSSBox or fixing upstream (may be good to do, but will take
longer to get a fix in).
* add test checking that glyphs render correctly
* Add a test that directly checks the wrapping fn
* Change the string to keep the linter quiet
* Change how we check if string can be rendered to faster method, new Lato Font
With Sanya's help, the way I check if a given string is renderable with Lato is now faster.
Also use the full Lato font, not just the 'latin' lato so we can cover more chars
* change lato so that it loads the fn even in a fresh test run
* prevent pin maps from filtering out rows with null values
- since pin maps are just displaying where the data is by latitude and longitude, we do not need to filter the pin just because one of the rows is null.
- the metric column selection does not make sense for pins as they just represent that data through coordinates (no aggregations or values displayed), so we should not be filtering out any data just because it is null
* add unit test
I was thinking about the potential options here.
On one hand, we could initialize the app db ourselves. This is a little
problematic, because:
- our `init!` function is in `metabase.core` and calling it yields a
painful cyclical dependency. It's also pretty tied to outputting things
to the user that they might not expect during an import, e.g. "Looks
like this is a new installation ... preparing setup wizard", and starts
up jobs, etc.
- if we choose to *only* initialize the database ourselves (only the
bits that we actually *need* initialized), I'm a little nervous about
maybe missing something. Right now we need to create the internal user
and create users from the config file before running the import. If we
added a third thing, would we remember to update this?
- we could refactor to put all of the above in one spot, but this is a
P1 and it feels important to close this out as quickly as possible. We
also don't claim to support initializing the app DB from an import in
the first place.
So: just throw an exception if the app db hasn't been set up yet. Our
heuristic here is whether a single non-internal user has been created.
So if you want to use your config, you can run:
```
MB_CONFIG_FILE_PATH=dev/config.yml java -jar metabase.jar
java -jar metabase.jar import /my/exported/dump
```
If you try to run `java -jar metabase.jar import ...` on a fresh
DB, it'll now fail with the error message:
```
You cannot `import` into an empty database. Please set up Metabase
normally, then retry.
```
* highlights the trash link in the sidebar when viewing root trash collection or a trashed item
* impls pr feedback
* impls more pr feedback
* fixes unit tests and logical mistake that allowed urls with invalid collection ids to match on undefined trash id due to it not being loaded yet
* stop using strings for enum values (prefer kw)
* only do SLO when saml-slo-enabled is true
* make defsetting docstring formatting consistent
* log-direction is a string
* add test for enabling / disabling saml slo
- also return a 403 when the client tries, but it's disabled
* let's see how many things break
* add missing useSyncURLSlug
* remove cardTitled prop that's not needed
* remove DashboardControls
* remove @ts-expect-error for dashboardId
* move props destructuring at the top
* remove use-dashboard-nav.ts as it's not needed anymore
* remove import of use-dashboard-nav from dashboard/hooks/index.ts
* remove unused props from dashboard
* Add dashboard question header
* Add dashboard data preserve after question drill
* Review fixes
* Review fixes
* Potential fix for e2e
* Potential fix for e2e
* Add basic unit tests
* Fix Dashboard reloading on unmount, fix ad hoc question title in SDK
* Move dashboard reset to SDK component
* Fix issue with dashboard rerender
* Fix one more reported issue with rerender
* fix map default display type with un-binned coordinates
- default to `"map.type"` = "pin" when coordinates are not binned
- default to `"map.type"` = "grid" when coordinates are binned
* add tests
* refactor breakouts usage
* Fix broken field normalization in metadata
* Handle Fields.actionTypes.UPDATE action in tables reducer to keep table's original_fields in sync
* Add an e2e test for fields
* Format code
* Use a different table, since accounts table is not available in e2e tests
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* remove invalid else conditional
* fix RowChart firing invalid API request on legend item click
* fix clicking on non-breakout legend on dashboards
* add E2E test to validate and enforce the desired behavior
* Bump dashboard description max len limit to 1.5K
* Extract `DASHBOARD_DESCRIPTION_MAX_LENGTH` const
* Handle description max length in dashboard sidebar
* Add red border to dashboard description box
* Add test
* fixes spacing and color issue of trash icon in archive banner
* css var for color
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* WIP PublicOrCustomEmbedDashboardPage without using dashboard controls
* add usePublicDashboardEndpoints instead of WithPublicDashboardEndpoints
* fix type error by setting null as default value for hide_parameters and hide_download_button
* use usePublicDashboardEndpoints inside WithPublicDashboardEndpoints
* also migrate embed
* i don't think we need useDashboardNav public/static embed
* SetTitle component to use instead of the `title` hoc
* wrap setEmbedDashboardEndpoints/setPublicDashboardEndpoints in useEffect as per PR comment
* after the fix for metabase#38640 we don't need the public routes in the main app
* renamed test file
* change tests to use PublicOrEmbeddedDashboardPage
* remove PublicOrEmbeddedDashboardControlled and change imports for PublicOrEmbeddedDashboardPage
* remove debug prop left by mistake
* Move SimpleSpecificDatePicker to the form
* Extract `isSpecificValue` helper
* Integrate the toggle into `SimpleDateIntervalPicker`
* Make DateOperatorPicker do only one thing
Separation of concerns - lift SimpleRelativeDatePicker up to the form.
* Fix unit tests
* Add back the test for the `SimpleSpecificDatePicker`
* Keep the rendering logic in the `SimpleDatePicker`
* apply css variables injection right under the public component
* refactor css variables
* fix name changes
* apply css variables to global styles
* update path
* render component wrapper as full-width
* move style resets to public component styles wrapper
* update unit tests for global styles
* Add Temporal Units List to Dashboard Parameters Schema
Closes: #44361
This adds an entry to the Dashboard Parameter schema so that we can validate
the list of temporal_units that might be passed from the frontend when adding
or updating the `Unit of Time` type parameters.
* Sanitize the SVG before parsing it with Batik
WIP
Fixes: #43677
There are characters that are invalid according to the Batik XML Parser. This sanitizes the svg string to strip out
some of those characters so that the render can continue.
* Use the xml 1.0 spec allowed unicode chars list for the regex
* Remove accidentally merged code
* still miseed a line
* fix: dont always optimize between expressions
Fixes#42291
The frontend produces expressions like
```
[:between
[+ [:field ... {:temporal-bucket :day] [:interval 2 :day]]
[:relative-datetime -1 :week]
[:relative-datetime 0 :week]]
```
This should not be optimized because of the mixed `:day` and `:week`
units. However, it was being optimized since the compatible units
weren't being properly picked up by the match.
* Disable suspcicios args eastwood lint, kondo does it better
* Stop autobucket to day when adding a time interval
* Fix the operator list for `type/TextLike` PKs
* Fix the bug with using wrong operators
* Fix the bug with using wrong operators
* Update test/metabase/lib/filter_test.cljc
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* Test FKs too
* Update test/metabase/lib/filter_test.cljc
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
---------
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* Get rid of the styled component
* Stack date operator on top of actual filter pickers
* Use the existing `CurrentDatePicker`
* Fix unit test
* Conditionally apply padding to the `CurrentDatePicker`
* Remove the custom prop
This PR removes `:role` removal from spec, even if it was added in `:connection-uri`.
It seems that `:connection-uri` is ignored on `do-with-connection-options` code path.
Hence use of non-default roles is again available on Snowflake.
This PR disables the 'make exports look pivoted' post processing step for csv and xlsx exports.
This is in response to a reported issue that Large Pivot Table downloads are failing entirely: #44556
A more hollistic fix and improvement to the post processing feature will be worked on, so the post processing code is
left untouched in this PR.
The dynamic vars in the csv and xlsx namespaces each default `false` and disable the export from running any post
processing. The var should remain `false` until we've got a plan for these types of exports, and there is no way for
users (either through ENV or UI) to change this, so the feature is effectively disabled.
* convert embed.ts to use createSlice from rtk
* only store in the state what we care about for interactive embedding
* use DEFAULT_EMBED_OPTIONS keys to filter searchOptions
* removed unused hash parameter
* adds basic unit test
There were two problems here:
- wrong args to `mdb/migrate!` (passing the db-type as well as the data
source), and
- in MySQL, the changelog table name is not `:databasechangelog`. Use
`metabase.db.liquibase/changelog-table-name` to get the actual table name.
Add a new query parameter to `/api/collection/:id/items` and
`/api/collection/root/items`. If `official_collections_first` is passed,
we'll sort official collections first, or not, as requested. If it isn't
passed, we'll default to `false` for the trash collection, or `true` for
normal collections.
* simplify bordered logic
* 🧹 setTitled
* 🧹 setHideDownloadButton
* 🧹 setHideParameters
* inline useEmbedDisplayOptions
* move DEFAULT_EMBED_DISPLAY_OPTIONS to costants file so we can remove use-embed-display-options.ts
* move consts up in the file
* update deps of useEffect
* fix ts error in the sdk
* use useEmbedFrameOptions in useDashboardUrlParams
* embed frame should not touch the `embed` slice, which is used for interactive embedding
* funnel start color should be text-dark
* change pivot table font size unit to em for scaling
* change funnel and legend item unit to em
* change funnel font size to em
* change object detail font size to em
* change gauge font size to em
* add cartesian chart padding config
* update docs
* remove isEmbeddingSdk prop
* simulate the 0.875em base font size
* chart padding override should apply in query builder
* add cartesian padding to stories
* update readme docs
* fix collection browser option in readme docs
* fix pivot table font size scaling
* update loki
* update loki
* use em for smart scalar
* update chart default padding
* revert smart scalar font size scaling due to measurement
* dynamic font family and font size as defaults
* pivot table
* add more properties to sdk docs
* make FONT_SIZES constant local
* link to discussion on query builder extra spacing
* enforce that font is passed to getLeftHeaderWidths
* Update e2e test
* Update e2e test
* fix: populate param-fields for named fields on public dashboards
* Add name to public hydration
* Add tests
* Add tests
* Fix tests
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Ensures that minimum query durations are correctly displayed in the form post-save (#44249) and that all times can be used in the Schedule component (#44257)
* UserParameterValue transforms wrapped to properly escape string vals
The `json-in` and `json-out` functions used for toucan model transforms do not perform any string escaping logic. This
makes sense as we don't want to make assumptions about the shape of data flowing in/out of the db. But, this did mean
that for the User paramter values table, string values were causing an error to be logged. This doesn't break
anything, as the function will still correctly return the string, but it can clutter up logs.
So, this PR wraps all incoming values in a map and unwraps it for outgoing values.
* Add migration to wrap existing user param values with ::wrapper key
Since the in/out transform for UserParameterValues is updated, we need to migrate any existing values to have this wrapping.
* Simplify the solution to the problem.
Since we're already getting what we need from the json-in/json-out *except* that it's logging a parse error, we can
create a json-out that works the same way (tries to parse and returns the string as-is if it fails) without logging an
error at all.
* take away the arg and keywordize inside the json-out fn
* remove irrelevant comment
Signed-off-by: Adam James <adam.vermeer2@gmail.com>
* Add a little more test coverage for different value types
* faster test thanks to Dan!
* Add a comment about the data being tested
---------
Signed-off-by: Adam James <adam.vermeer2@gmail.com>
* XLSX Pivot Table Downloads have 'Native' Pivot Table
Addresses: #2473
The xlsx downloads work fine for regular tables. However, on Pivot Table downloads, the resulting file isn't pivoted
and is formatted somewhat strangely (it contains additional rows related to totals, has an extra 'pivot-grouping'
column).
Now, the downloaded xlsx file contains 2 sheets:
- the data sheet containing the question's rows unpivoted. If you were constructing a pivot table yourself, you would
probably start with this shape of data.
- the pivot sheet containing a 'native' Pivot table whose reference points to the data on the data sheet, and whose
columns, rows, and values align with the cols, rows, and measures set up in Metabase
* Silly typo!
* Exported Pivot uses correct aggregation (sum, count, stddev, avg, min, max)
* data for pivot exports now also uses the formatting
* Add a test asserting that a pivot table exists in the xlsx file
* add-row! is a method dispatching on sheet class
* remove unnecessary hint
* address review feedback:
- consolidate the aggregation function key fns into one function in the .xlsx namespace
this was moved out of postprocessing so that the xlsx specific fns don't litter the post processing namespace at
this time
- clean up an unnecessary destructiring inside the native-pivot function
- don't shadow 'name' inside the body of a fn
* Add tests that confirm zero-col and zero-row situations work
* Consolidate the atoms used to store pivot rows/options
* Expose "Include this {period}" in timeseries chrome
* Fix type errors
* Simplify type guards
* Add unit tests
* Remove Flex wrapper around Switch
* Extract `IncludeCurrentSwitch` component
* Add E2E smoke tests
* Cover the toggle behavior with E2E tests
* Simplify test with a simple helper
* Temp
* Add support for saved questions
* Try to fix question not found issue
* Try to fix question not found issue
* Fix unit tests
* Add basic unit test
* Fix unit tests
* Fix unit tests, fix question not found issue
* Review fixes
* Review fixes
* Refactor components structure, review fixes
* Adjust readme
Fixes#43993 for real.
The earlier fix#44182 was needlessly restricting the
"nominal refs" check to fields which were using numeric IDs,
when really any match on nominal refs is valid.
* automatically add milestones to prs and issues
* test code
* test code
* test commit (#3138)
* test commit (#43320)
* test commit (#44357)
* remove test code
* add newlines
* prefer older version milestones
* Upgrade @reduxjs/toolkit from ^1.9.3 to ^2.2.5
* Fix typing
* Upgrade redux-actions and @types/redux-actions
* Fix type errors
* Fix type errors
* Upgrade react-redux from ^8.0.5 to ^9.1.2
- remove @types/react-redux as it's no-longer-needed
* Upgrade redux-promise from ^0.5.0 to ^0.6.0
- upgrade @types/redux-promise
* Remove unused @types/redux-logger
* Revert "Upgrade redux-actions and @types/redux-actions"
This reverts commit 65005a0053.
* Remove createWeakSelector as weakMapMemoize is the default memoization strategy now
* Do not call unsubscribe if it does not exist
* Do not subscribe
* Remove console.log
* Make DelayedLoadingSpinner always render something with data-testid="spinner" so that tests can be aware of when things are loading
* Fix JoinStep unit tests
* Fix AddToDashSelectDashModal unit tests
* Fix SegmentApp unit tests
* Bring back original subscription logic
* Fix DataUsageStep unit tests
* Fix ManageApiKeys unit tests
* Simplify comment
* Fix MainNavbar unit tests
* Fix HomeRecentSection unit tests
* Use afterEach instead of afterAll to reset mocks
* Remove explanatory variable
* Pivot Tables with no Pivot Columns should still download
Fixes#44159
Pivot Tables can still be valid without columns, and therefore the download should respect such a query and be
successfully exported.
In this PR, the postprocessing of pivot results into 'visual pivot' exports for csv takes into account the zero-column
configuration and the download no longer fails.
* add a test to confirm that zero cols and >1 measures works as well
* Handle the case where there are zero pivot-rows.
This is a similar problem to the zero pivot-cols case, just with rows.
The app actually doesn't render these tables correctly, but that's a separate frontend bug
* Convert drill-through (CLJ) values to JS
* Convert the drill-through `:null` to an actual `null`
Resolves#44232
* Use new function in `filter-drill-details`
* display-info->js should check for nil before seqable?
* fix js object comparison
* Add test for the `drill-value->js` function
* Simplify the test
* Fix the JS object equality in the test
---------
Co-authored-by: Alexander Solovyov <alexander@solovyov.net>
* Change run to make-run in process-query-for-card
Fixes#44160
There were two levers where a card could change how it was run.
`qp` this is basically `qp/process-query` with some optional userland
flags and constraints marked on the passed in query.
In the case of pivot tables in the
`/pivot/:card-id/query` endpoints it was `qp.pivot/process-query`
'run' this defaults to a streaming response that takes the `qp` param
and `export-format` and passes a query to qp. Most often this is
overriden to not stream the response.
Unfortunately, if `run` is overwritten, then `qp` is thrown away, and
the caller who overrides `run` must determine the correct `qp` to use.
However, when running `process-query-for-dashcard` it is difficult to properly set
`qp` for all of the dashcards without doing card lookups. So [this commit](https://github.com/metabase/metabase/blob/release-x.50.x/src/metabase/query_processor/card.clj#L230-L232) did a check to swap the runner in the presence of a pivot query that calls the **default** runner with a `qp.pivot/run-pivot-query`. The problem is that pulse needs to use a non-default runner, but there's no way for `process-query-for-card` to do both.
This commit changes `run` to `make-run` which raises the abstraction to
a function that produces a runer given a `qp`. This allows
`process-query-for-card` to change the `qp` as needed while keeping the
passed in runner.
There's really three things going on that could be teased apart
further/better.
1. `qp` - this conflates the processor (`qp/process-query` `qp.pivot/run-pivot-query`)
and modifying query (`qp/userland-query`,
`qp/userland-query-with-default-constraints` `(update query :info merge info)`)
2. `make-run` - How the above are processed: (streaming/realized)
* Fix tests
* Elevate include this period in the relative date picker (#44179)
* Mock new row with "include {period}" string
* Implement "include this" using a toggle
* Elevate "starting from" option in the relative date picker (#44242)
* Elevate "Starting from" option in the filter picker
* Tweak the date offset close icon
* Fix tests
* fix charts crash in safari when dataset timezone is not in iana format
* Update Loki Snapshots
* better fix: apply offset instead of finding a timezone with such offset
* remove unused specs
* Update Loki Snapshots
* review
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Fix a potential bug in `hydrate-can-restore`
I had a bug here. I wanted to hydrate `:collection` on the items and
then return the original items, but instead I was `dissoc`ing
`:collection` from the items. If the original items passed in already
had `:collection` keys this would break them.
AFAIK this didn't cause any issues, but it could have - so fixing it here.
* Add a test
* Present collection items in the collection
When fetching collection items, we need to fetch the true
`collection_id` in order to figure out things like `can_restore` and
`can_delete`, but set it to the displayed/effective `collection_id` when
we actually send it out.
This fixes a bug where collections had their `collection_id` match their
`id` when returned by this endpoint.
Previously we were just adding this when the item was archived. Instead,
always send a value - if the item is not archived, it can't be restored,
so it will be `false` in this case.
Making this change will make fixing a frontend bug easier, because the
frontend will be able to just trust the `can_restore` value coming in
from the backend.
* Use MultiAutocomplete in dashboard filters
* Handle prefix on MultiAutocomplete
* Remove unnecessary tests for empty values
* Remove input className
* Handle numbers in MultiAutocomplete
* Accept number values in MultiAutocomplete
* Use MultiAutocomplete in NumberInputWidget
* Filter out empty items
* Do not expect formatted number in values
* Do not allow adding multiple values when multi is false
* Handle undefined values
* Make value optional
* Fix MultiAutocomplete types
* Make MultiAutoComplete generic in its value type
* Refine MultiAutocomplete type in NumberInputWidget
* Refine MultiAutocomplete type in FieldValuesWidget
* Fix typo
* Fix MultiAutocomplete tests
* Add helpers to work with MultiSelect and MultiAutocomplete inputs
* Handle MultiAutocomplete input in tests
* Fix filter helper
* Use multiAutocompleteInput helper
* Find the value remove button in test
* Fix tests for NumberInputWidget
* Fix tests for DefaultPicker
* Fix type in NumberInput
* Fix dasboard test
* Fix embedding test
* Add helpers to remove values
* Fix chained filter test
* Fix nested test
* Fix embedding tests
* Fix filter tests
* Use removeMultiAutocompleteValue
* Use removeMultiAutocompleteValue
* Remove unused variable
* Use multiAutocompleteInput helper
* Handle comma's in values
* Handle multiselect
* Fix filter test
* Add filter
* Format field value
* Fix embedding tests
* Allow rendering custom label for item
* Use renderValue in FieldValuesWidget
* Fix data prop type
* Be more explicit about broken types in FieldValuesWidget
* Fix default behaviour for label
* Fix embedding tests
* Force click
* Fix sharing test
* Use removeMultiAutocompleteValue
* Handle multiautocomplete inputs
* Force click filter
* Close the suggestions menu
* Fix email test
* Fix sharing filter
* Set viewport in test
* Fix sharing test
* Remove debugging logs
* Use removeMultiAutocompleteValue
* Close the suggestions menu using blur
* Remove commented out code
* Remove the unused color prop
* Move shared types to types file
* Rename ValueType to TValue
* Add missing forwardRef
* Add tests for filter and renderValue
* Use TokenField for single value parameters
* Fix single value test
* Be more specific about when to select a single input
* Fix single value e2e test
* Handle undefined parameter
* Accept all RowValue types in MultiSelect
* Accept all RowValue types in ListField
* Accept all RowValue types in SingleSelectListField
* Be more precise about the FieldValuesWidget value types
* Be more specific about when to render normal input
* Remove duplicate import
* Remove maxSelectedValues from MultiAutocomplete
* Parse values in FieldValuesWidget
* Remove parsing complexity from MultiAutocomplete
* Be more strict when parsing numbers
* Do not use multiselect when arity is 1
* Remove spurious changes
* Pass prefix from FieldValuesWidget
* Remove number MultiAutocomplete story
* Remove renderValue from MultiAutocomplete
* Add custom component to MultiAutocomplete
* Add missing data prop
* Remove spurious changes in MultiAutocomplete
* Remove SelectItem reference
* Use renderValue in FieldValuesWidget
This PR removes:
- a warning about missing React keys in the `Schedule` component
- a warning about invalid props in the `StrategyFormLauncher` component
* adds a shimmer effect to the skeletons
* adds a `natural` prop that makes the skeletons have a random width between 50% and 80%
* adds a skeleton to Browse models
Closes#44111
* expose card title settings
* Add dashboard test skeleton
* add a sample dashcard to the test
* dashboard grid should have card title by default
* add unit test for card title
* remove unused delay in unit test
* fix width and height being smaller than min values
* flip grid layout condition
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Handle errors when inferring separator of csv
This file
```
"c1","c2"
"a,b,c","d"
```
was failing to upload for a silly reason: when we were checking for
which separator it used, we had a parse error when using a semi-colon
and that blew up the whole pipeline.
* Use cal's helpful test suggestions
Although a user may have the necessary permissions to delete an item, if we're not going to allow deleting unarchived items, can_delete should be false for them too.
* [Metrics V2] Metrics with filters on implicitly joined columns produce query error when added in a Summarize block
* Handle multiple metrics with same implicit join
* Address PR feedback
* adds the endpoint and a test
* add recents endpoint, todo: tests
* add post recents endpoint
* return recent views for both lists, but make the endpoint work
* Make recent-views context aware
- pruning is context aware, only checks for the recently-inserted
context
- Adds endpoints:
- POST to create selection recents,
- GET activity/recents
- requres context query param to be one of:
- all, views, selections
- Adds context arg to update-users-recent-views!
- Cleans up arg schema for update-users-recent-views
* impl GET api/activity/recents
- return recent-selections and recent-views from
- send context == collection from pinned card reads
* update callsites of recent-views/update-users-recent-views!
- to use :view param where necessary
* fixes models/recent-view tests
* adds more activity/recent-view tests
* wip
- fix whitespace linter
* Fix command palette e2e test
- reuse util snake-keys
* updates fe to use new recents endpoints
* fixes fe type issue
* snake-keys -> deep-snake-keys
- I've been betrayed by lsp rename
* snake-keys -> deep-snake-keys
- I've been betrayed by lsp rename
* log selection events for created collections
* mysql doesn't allow default values for TEXT types
* log a recent view on data-picker selection
* decouple view-log context from card-event view context
* fix a doc typo
* stop double logging recent-views on POST
* maybe fixes some tests
* some e2e fixes
* fix mysterious divide by zero during score search
* fix divide by zero possibilities everywhere in score-items
metabase.api.activity/score-items used to throw when there weren't any
items being scored (even though there's a `(when (seq items) ...)` check)
* more test fixes
* fix more e2e tests, + rename endpoint in tests
* fix oopsie
* fixes a few more tests
* address review comments/questions
* allow for a comma delimited list in qps like ?context=views,selections
returns all recent view items with those contexts, most recent first
under the `:recents` key.
* refactors FE around new endpoint
* fixes for unit tests
* use ms/QueryVectorOf for context
* fix models/recent_views tests
* use multiple query params instead of comma delimited value on FE
* ignore timestamp when deduping recents
* review comments + test fixing
* update docstring
* fix api/activity_test s
* actually dedupe
* add test for deduping by context
* e2e fix: shows up-to-date list of recently viewed items after another page is visited
* e2e fix: should undo the question replace action
* e2e fix: should replace a dashboard card question (metabase#36984)
* e2e fix: should preselect the most recently visited dashboard
* fix 6 more e2e tests
* fixes fe type check and unit failure
* renames unit test mocking function
* fix a flaky e2e test
* widen Item to accept str or kw where sensible
- allow strings or keywords for moderated_status, and authority_level
* simplify impl + add test
* add view-log to events schema
* add collection context to view log
* fix the final 2 failing e2e tests
* click dashboard tab when the user has can-read? on recent entities
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* Improve the Trash
Ok, so I had a realization at the PERFECT time, immediately after the RC
cutoff. Great job, brain!
Here's the realization. For the Trash, we need to keep track of two
things:
- where the item actually is located in the hierarchy, and
- what collection we should look at to see what permissions apply to the
item.
For example, a Card might be in the Trash, but we need to look at
Collection 1234 to see that a user has permission to Write that card.
My implementation of this was to add a column,
`trashed_from_collection_id`, so that we could move a Card or a
Dashboard to a new `collection_id`, but keep track of the permissions we
actually needed to check.
So:
- `collection_id` was where the item was located in the collection
hierarchy, and
- `trashed_from_collection_id` was where we needed to look to check
permissions.
Today I had the realization that it's much, much more important to get
PERMISSIONS right than to get collection hierarchy right. Like if we
mess up and show something as in the Trash when it's not in the Trash,
or show something in the wrong Collection - that's not great, sure. But
if we mess up and show a Card when we shouldn't, or show a Dashboard
when we shouldn't, that's Super Duper Bad.
So the problem with my initial implementation was that we needed to
change everywhere that checked permissions, to make sure they checked
BOTH `trashed_from_collection_id` and `collection_id` as appropriate.
So... there's a much better solution. Instead of adding a column to
represent the *permissions* that we should apply to the dashboard or
card, add a column to represent the *location in the hierarchy* that
should apply to the dashboard or the card.
We can simplify further: the *only time* we want to display something in
a different place in the hierarchy than usual is when it was put
directly into the trash. If you trash a dashboard as a part of a
collection, then we should display it in that collection just like
normal.
So, we can do the following:
- add a `trashed_directly` column to Cards and Dashboards, representing
whether they should be displayed in the Trash instead of their actual
parent collection
- use the `collection_id` column of Cards and Dashboards without
modification to represent permissions.
There's one main downside of this approach. If you trash a dashboard,
and then delete the collection that the dashboard was originally in,
what do we do with `dashboard.collection_id`?
- we have to change it, because it's a foreign key
- we can't set it to null, because that represents the root collection
In this initial implementation, I've just cascaded the delete: if you
delete a dashboard and then delete a collection, the dashboard will be
deleted. This is not ideal. I'm not totally sure what we should do in
this situation.
* Rip out all the `trashed_from_collection_id`
* Migration to delete trashed_from_collection_id
* fixes
* don't move collections
And don't allow deleting collections
* only show cards/dashboards with write perms
* Show the correct archived/unarchived branch
* some cleanup
* add a todo for tomorrow
* Fix for yesterday's TODO
* more wip
* refactor
* memoize collection info
* move around memoization a bit
* fix schema migration test
* oops, delete server.middleware.memo
* Use a migration script (postgres only for now)
* Fix some tests
* remove n+1 queries in `collection-can-restore`
* fix test
* fix more tests, and x-db migration script
* fix h2 rollback
* fix mysql/mariadb migration
* lint
* fix some mariadb/mysql tests
* fix h2 rollback
* Fix mysql rollback
* Fix Postgres migration
* "Real" `trash_operation_id` UUIDs from migration
* fix mariadb migration
* Separate MySQL/MariaDB migrations
* trashed directly bit->boolean
* Remove `trashed_from_*` from migrations
* Rename `api/updates-with-trashed-directly`
Previously named `move-on-archive-or-unarchive`, which was no longer
accurate since we're not moving anything!
* Add `can_delete`
* Delete test of deleted code
* Can't move anything to the Trash
The Trash exists as a real collection so that we can list items in and
such without needing special cases. But we don't want anything to
_actually_ get moved to it.
* integrates can_delete flag on FE
* Update src/metabase/models/collection.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update src/metabase/search/impl.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Better name for `fix-collection-id`
Also consolidated it into a single function. It takes the
trash-collection-id as an argument to prevent a circular dependency.
* s/trashed/archived/ on the backend
In the product, we want to move to "trashed" as the descriptor for
things that are currently-known-as-archived.
It'd be nice to switch to that on the backend as well, but that'll
require quite a lot of churn on the frontend as we'd need to change the
API (or have an awkward translation layer, where we call it `trashed` on
the backend, turn it into `archived` when sending data out via the API,
and then the FE turns around and called it "trashed" again).
For now, let's just be consistent on the backend and call it `archived`
everywhere. So:
- `archived_directly` instead of `trashed_directly`, and
- `archive_operation_id` instead of `trash_operation_id`
* Fix up a couple docstrings
* Rename `visible-collection-ids` args
for `collection/permissions-set->visible-collection-ids`:
- `:include-archived` => `:include-archived-items`
- `:include-trash?` => `:include-trash-collection`
* select affected-collection-ids in the tx
* Stop dealing with `:all` visible collection ids
This used to be a possible return value for
`permissions-set->visible-collection-ids` indicating that the user could
view all collections. Now that function always returns a set of
collection IDs (possibly including "root" for the root collection) so we
don't need to deal with that anymore.
* Don't use separate hydration keys for collections
Dispatch on the model of each item (but still keep them in batches for
efficiency.)
* vars for collectable/archived-directly models
* Round up loose `trashed_directly`s
* Fix a test
* Use `clojure.core.memoize` instead of handrolled
It's slightly different (TTL instead of exactly per-request) but that
should be fine.
* FE lint fixes
* 😭 e2e test fix
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* add option to customize dashcard border
* use the shadow CSS variable
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* make the shadow and border decoration call more explicit
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* pass in theme as argument
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Allow picking arity of number parameters
* Limit to input to have arity 1 when multi is false
* Use MultiAutocomplete in NumberInputWidget when arity is 1
* Add test cases for single value numbers
* Only change arity when it is n
* Fix unit tests for NumberInputWidget
* Move arity check for numbers to getNumberParameterArity
* Fix the typo
* Replace `realHover` with the `mouseMove` trigger
* Be explicit when targetting elements by their index
* fix combo chart tooltip e2e specs
* cy.wait as a last resort
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
* Improve typing in Question
* Remove dead CSS
* Improve typing in getUrlWithParameters
* Add LegendLabel
* Drop navigateToNewCardFromDashboard in PublicOrEmbeddedDashboard
* Extract getNewCardUrl and move getParametersMappedToDashcard closer to it
* Add link to dashboard chart title
* Update title-drill tests
* Format code
* Update test
* Add HrefMemo
* Use state to memoize href
* Recompute state on getHref prop change
* Format code
* Refactor href computation
* Update test
* Use selector instead of useMemo
* Fix unit tests
* Ensure type safety
* Revert getDashCardById type change to not make this PR too large
* Use complete dashboard
* Add a workaround for metabase#43990
* Remind embedding SDK PR author to use conventional commits
* Only remind PR author when PR title isn't conventional commits
* Add branch for testing
* Fix gh cli command
* Fix getting wrong user login
* Ensure we don't create more than 1 comment
* Improve comment format
* Add more info for non-changelog related type
* Check non-changelog related types
* Update the comment for consistency
* Revert "Add branch for testing"
This reverts commit 4a1be68828.
* Migrate text-brand
* Migrate text-dark
* Migrate text-light
* Migrate text-medium
* Migrate admin-navbar
* make compact formatting consistent across all series (#43472)
* make compact formatting consistent across all series
* fix rebase
* [Browse] Show Browse models link in nav only when there are models (#43286)
* Upgrade to TypeScript v5.4 (#43736)
* use correct github api for CI (#43750)
* update list of releases (#43754)
* Multi Series Dashcards Render with Filters Applied To All Series (#43428)
* Multi Series Dashcards Render with Filters Applied To All Series
Fixes: #39083
Multi-series dashcards now properly render when a subscription is set up with filters that impact the values of each
card in the multi-series card.
* Small refactor for executing dashcards in a pulse context.
- move execute-dashboard-subscription to pulse.util
- this fn now will also execute all card series cards
and put those results into a list on the key :series-results in the dashcard
which is then used in the render multimethod to send the correct data to the javascript context
A small todo was taken care of as well, we can now use :dashboard-subscription in the :context key
* Fix up render fn
The shape of hte data going into the javascript was a bit unclear before. This keeps things a little more
understandable.
Next step is probably to create some Malli schemas inside the js_svg namespace to document and enforce the needed data shapes
* pull :data up a level from :result in the series cards
* Update src/metabase/pulse/render/body.clj
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Remove unneeded let
* set/rename-keys won't work here because we raise the :data key
I've realised a suggested change to just use set/rename-keys won't do what we need. We need instead to raise :data
from within the result key up to the same level (and dissoc result).
I've renamed the helper function to make its purpose a bit clearer
---------
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Hydrate can_run_adhoc_query and address review comments (#43747)
This is a followup on #43625.
* Consolidate dashboard-card specs (#43716)
* Consolidate joins specs (#43717)
* Update translations for v50 and add Danish Language Support (#43314)
* update translations for v50
* add Danish
* update translations for v50 release
* alphabetical order is nice
* Move database role to url parameters from connection properties on Snowflake (#43602)
* Move role to url parameters from connection properties
* Fix test
* Fix width of save button in cache sidebar (#43755)
* Remove dead code (#43603)
* Combine search e2e specs (#43718)
* Highlight the table cell data in dashboards when the cell is an ID column (#43728)
* highlight id cell column
* add simpletable stories
* add simple table test data
* add loki reference image
* Update frontend/src/metabase/visualizations/components/TableSimple/TableCell.styled.tsx
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Remove old repro for 39150 (#43749)
* Fix remaining `tinyint` booleans in MySQL (#43296)
* Correct type for `report_card.dataset`
This was missed in the original migration of boolean types to
${boolean.type} in MySQL/MariaDB, and then missed again by me when I
migrated `collection_preview` over a week ago.
* Change all boolean types to `bit(1)` in MySQL
Liquibase changed their boolean type in MySQL from `bit(1)` to
`tinyint(4)` in version 4.25.1. Our JDBC driver does not recognize these
as booleans, so we needed to migrate them to `bit(1)`s.
As discussed [here](#36964), we
changed all existing `boolean` types that were in the
`001_update_migrations.yml` but not the SQL initialization file.
For new installations, this works: things in the SQL initialization file
get created with the `bit(1)` type.
However, for existing installations, there's a potential issue. Say I'm
on v42 and am upgrading to v49. In v43, a new `boolean` was added.
In this case, I'll get the `boolean` from the liquibase migration rather
than from the SQL initialization file, and it need to be changed to a
`bit(1)`.
I installed Metabase v41 with MySQL, migrated the database, and then
installed Metabase v49 and migrated again. I made a list of all the
columns that had the type `tinyint`:
```
mysql> SELECT TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME, COLUMN_TYPE, COLUMN_DEFAULT, IS_NULLABLE FROM INFORMATION_SCHEMA.COLUMNS WHERE COLUMN_TYPE = 'tinyint' AND TABLE_SCHEMA='metabase_test';
+---------------+------------------------------+-------------------+-------------+----------------+-------------+
| TABLE_SCHEMA | TABLE_NAME | COLUMN_NAME | COLUMN_TYPE | COLUMN_DEFAULT | IS_NULLABLE |
+---------------+------------------------------+-------------------+-------------+----------------+-------------+
| metabase_test | core_user | is_datasetnewb | tinyint | 1 | NO |
| metabase_test | metabase_field | database_required | tinyint | 0 | NO |
| metabase_test | metabase_fieldvalues | has_more_values | tinyint | 0 | YES |
| metabase_test | permissions_group_membership | is_group_manager | tinyint | 0 | NO |
| metabase_test | persisted_info | active | tinyint | 0 | NO |
| metabase_test | report_card | dataset | tinyint | 0 | NO |
| metabase_test | timeline | archived | tinyint | 0 | NO |
| metabase_test | timeline | default | tinyint | 0 | NO |
| metabase_test | timeline_event | archived | tinyint | 0 | NO |
| metabase_test | timeline_event | time_matters | tinyint | NULL | NO |
+---------------+------------------------------+-------------------+-------------+----------------+-------------+
10 rows in set (0.01 sec)
```
Then wrote migrations. For each column, we:
- turn it into a `bit(1)`,
- re-set the previously existing default value, and
- re-add the NOT NULL constraint, if applicable.
* Change author and add missing `dbms`
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* Add ts-pattern library (#43752)
* Add events to measure search performance (#43135)
* new_search_query event
* Filters in object, search click event
* updating event schema
* type and event schema adjustments
* delete comments BE code
* adjusting tests
* PR feedback
* typos and spacing
* Fixup missing colors
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
Co-authored-by: adam-james <21064735+adam-james-v@users.noreply.github.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: SakuragiYoshimasa <ysrhsp@outlook.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Nick Fitzpatrick <nick@metabase.com>
* Add repro test
* Move home selectors to `metabase/selectors/app`
* Prevent unnecessary redirects to home dashboard
* Rename
* Handle navbar home link
* Fix unit tests
* Fix navigating away from an archived dashboard
* Handle sub-root Metabase URLs
* Sort functions
* Add boilerplate for aggregateOffset and tests
* Simplify assertion
* Make basic case work
* Handle name generation
* Do not require "lib/uuid" attribute to be present since MLv2 will normalize it under the hood anyway
* Fix typing
* Group tests
* Refactor offsetClause to return a new clause instead of a new query
- Move offsetClause to expressions.ts
- Add assertion
* Revert "Sort functions"
This reverts commit ab9ce2b24e.
* Move tests
* Handle names dynamically
* Shorten a test
* Update comment
* Add TODO
* Update expression types
* Add diffOffsetClause & percentDiffOffsetClause
* Add a test for diffOffsetClause
* Add TODOs
* Add tests for diffOffsetClause and percentDiffOffsetClause
* Unwrap tests
* Add skeleton for new tests
* Refactor first offsetClause tests
* Add tests for names
* Fix case of non-datetime columns
* Handle a case with offset < 1 and no breakouts
* Handle a case with offset < 1 and breakouts on non-datetime column
* Handle a case with offset < 1 and breakouts binned datetime column
* Handle a case with offset < 1 and breakout on non-binned datetime column
* Refactor
* Refactor
* Remove TODO
* Add tests for diffOffsetClause
* Add tests for percentDiffOffsetClause
* Move offset stuff to offset.ts and offset.unit.spec.ts
* Use template string for prefixes
* Use breakoutColumn + isDate to check column type
* Refactor
* Fix error message
* Add boilerplate for CompareAggregations
* Fix title
* Render aggregations list
* Style AccordionList
* Sort props
* Fix bucket name
* Use displayName instead shortName
* Support parseValue prop in NumberInput
* Add period input accepting integers only
* Accept non-negative values only
* Do not accept zeros
* Add state
* Add submit button
* Export offset functions via Lib
* Make it possible to skip rightSection
* Add column picker
* Map offset user input to api input (negative value)
* Add label
* Fix crash
* Extract shouldCreate
* Make onSelect in AggregationPicker support multiple values
* Extract ReferenceAggregationPicker
* Extract ColumnPicker
* Extract getAggregations
* Rename
* Add custom items
* Refactor item component
* Extract OffsetInput
* Remove unused data-testid
* Style OffsetInput
* Generate titles according to specs
* Generate label
* Generate help
* Extract utils
* Use different width for the 1st step
* Format code
* Use MultiSelect directly
* Avoid custom parseValue
* Revert MultiaAutocomplete changes
* Improve typing in describeTemporalInterval and
describeRelativeDatetime
* Use describeTemporalUnit to pluralize
* Use interface
* Avoid setting value as DOM attribute
* Fix test
* Add onAdd prop to AggregationPicker and revert the change to have onSelect pass multiple aggregations
* Reduce number of props
* Render checkboxes in custom items
* Introduce and use --mb-color-brand-lighter
* Avoid !important
* Remove redundant prop
* Rename ExtractColumn to ExtractColumnAction
* Add lines.svg
* Fix positioning in case dir attribute is not present in any parent component
* Add CompareAggregationsAction
* Fix warning about isSelected being used as DOM attribute
* Fix positioning in case dir attribute is not present in any parent component
* Remove redundant Box
* Compute the title
* Add compareAggregationsDrill boilerplate
* Add compare-aggregations drill stub
* Update comment
* Compute title and aggregation
* Add TODO
* Use proper title when there's only 1 aggregation
* Show visualization on submit
* Limit aggregation choice to 1 in compare aggregations drill
* Redirect to chart upon submission
* Do not navigate to chart view
* Do not navigate to chart view
* Make drill-thru tests expect the new compare-aggregations drill
* Add type attribute to all ListItems
* Add aggregationIndex to drillInfo
* Add aggregationDrillDetails and wire it up
* Add tracking helpers
* Add new schema
* Add tracking in column header
* Add tracking in plus modal
* Add tracking in aggregation picker
* Fix destructuring
* Add functionsUsedByExpression
* Fix schema number
* Add tests boilerplate
* Implement test for notebook editor
* Add test for column header
* Add a test for summarize sidebar
* Add a test for the plus button
* Add expectNoBadSnowplowEvents call in afterEach
* Format code
* Add missing redux store state in tests
* Add data for tests
* Move createQuestion call out of beforeEach
* Add tests for labels when there are no aggregations
* Add describe
* Add group helper
* Revert "Add group helper"
This reverts commit e047a223e9.
* Optimize: use cy.log() instead of it()
* Format code
* Add tests for 1 breakout, add helpers
* Add test for non-binned datetime column text
* Add test for non-datetime column text
* Nest tests
* Add test for multiple aggregations and no breakout, fix getTitle in this case
* Add all text tests
* Reduce nesting
* Enable test
* Format code
* Add tests
* Rename function
* Format code
* Simplify code
* Add assertions
* Format code
* Add column assertions
* Run tests with all 3 aggregations
* Remove logs
* Introduce CUSTOM_EXPRESSIONS_USED
* Avoid force click
* Remove explicit assertion
* Use be.visible instead of exist
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
I'm not sure it's any less hacky in the CLJS but it keeps the TS
wrappers thin.
Context:
- #43799 is the original issue
- #43848 added the hack being removed here
This PR simplifies a bit of text used in the cache cron schedule component: we can say "spaces" (which is normal English) not "white spaces" (which is developerese)
* Automate SDK doc PR
* Prevent auto merge
* Dry run npm publish for testing
* Speed up testing by not building jars
* Fix auto approve step
* disable concurrency so I could test multiple runs in parallel
* Test auto-merge
* Fix auto merge
* Revert "disable concurrency so I could test multiple runs in parallel"
This reverts commit 60e90076eb.
* Revert "Speed up testing by not building jars"
This reverts commit 4e53a54414.
* Revert "Dry run npm publish for testing"
This reverts commit 9b5d761c8d.
* Enable all commented out steps
* Remove unnecessary steps
* Mock that the changelog is different from the one in master
* Apply changelog patch incase the changelog is created from another branch
* Dry run npm publish so I could test
* Fix failed patch
* Speed up testing by not building jar
* Fix adding new changelog to master
* feat(sdk): A placeholder for an actual feature in the changelog
* Mock that the changelog diverges from master
* Revert "Speed up testing by not building jar"
This reverts commit acb80d7465.
* Revert "Dry run npm publish so I could test"
This reverts commit f4e679efd9.
* Revert "Mock that the changelog diverges from master"
This reverts commit 0e77a68e1b.
* dashcard background color
* add bg-dashboard and bg-dashboard-card
* rename background-visualization to background-question
* apply bg-question to embedded questions
* use CSS variables along with theme object
* add css variables to more places
* add ability to reference design system colors for defaults
* use CSS variables for defaults
* fix chart skeleton test
* fix legend item title
* Move expressionClause to handleSubmit
* Allow passing the first column as a prop and allow auto picking of columns
* Use CombineColumns component in header drills
* Remove duplicate CombineColumnsDrill component and helpers
* Allow separators to be null
* Calculate example in Example component to avoid useMemo call
* Fix test reference in test
* Append suffix to duplicate columns
* Rename parameter to something more expressive
* Enable autoPickColumn for all instances of the combine column popover
* Fix tests
* Fix notebook tests
* Make test typescript
* Correct types of openTable function
* Add a helper to extract linked issues from PR body
* Add tests
* Tweak the workflow
* Fix the workflow
* Delete obsolete code
* Improve the message
* Fix whitespace error
* Correct type for `report_card.dataset`
This was missed in the original migration of boolean types to
${boolean.type} in MySQL/MariaDB, and then missed again by me when I
migrated `collection_preview` over a week ago.
* Change all boolean types to `bit(1)` in MySQL
Liquibase changed their boolean type in MySQL from `bit(1)` to
`tinyint(4)` in version 4.25.1. Our JDBC driver does not recognize these
as booleans, so we needed to migrate them to `bit(1)`s.
As discussed [here](#36964), we
changed all existing `boolean` types that were in the
`001_update_migrations.yml` but not the SQL initialization file.
For new installations, this works: things in the SQL initialization file
get created with the `bit(1)` type.
However, for existing installations, there's a potential issue. Say I'm
on v42 and am upgrading to v49. In v43, a new `boolean` was added.
In this case, I'll get the `boolean` from the liquibase migration rather
than from the SQL initialization file, and it need to be changed to a
`bit(1)`.
I installed Metabase v41 with MySQL, migrated the database, and then
installed Metabase v49 and migrated again. I made a list of all the
columns that had the type `tinyint`:
```
mysql> SELECT TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME, COLUMN_TYPE, COLUMN_DEFAULT, IS_NULLABLE FROM INFORMATION_SCHEMA.COLUMNS WHERE COLUMN_TYPE = 'tinyint' AND TABLE_SCHEMA='metabase_test';
+---------------+------------------------------+-------------------+-------------+----------------+-------------+
| TABLE_SCHEMA | TABLE_NAME | COLUMN_NAME | COLUMN_TYPE | COLUMN_DEFAULT | IS_NULLABLE |
+---------------+------------------------------+-------------------+-------------+----------------+-------------+
| metabase_test | core_user | is_datasetnewb | tinyint | 1 | NO |
| metabase_test | metabase_field | database_required | tinyint | 0 | NO |
| metabase_test | metabase_fieldvalues | has_more_values | tinyint | 0 | YES |
| metabase_test | permissions_group_membership | is_group_manager | tinyint | 0 | NO |
| metabase_test | persisted_info | active | tinyint | 0 | NO |
| metabase_test | report_card | dataset | tinyint | 0 | NO |
| metabase_test | timeline | archived | tinyint | 0 | NO |
| metabase_test | timeline | default | tinyint | 0 | NO |
| metabase_test | timeline_event | archived | tinyint | 0 | NO |
| metabase_test | timeline_event | time_matters | tinyint | NULL | NO |
+---------------+------------------------------+-------------------+-------------+----------------+-------------+
10 rows in set (0.01 sec)
```
Then wrote migrations. For each column, we:
- turn it into a `bit(1)`,
- re-set the previously existing default value, and
- re-add the NOT NULL constraint, if applicable.
* Change author and add missing `dbms`
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* Multi Series Dashcards Render with Filters Applied To All Series
Fixes: #39083
Multi-series dashcards now properly render when a subscription is set up with filters that impact the values of each
card in the multi-series card.
* Small refactor for executing dashcards in a pulse context.
- move execute-dashboard-subscription to pulse.util
- this fn now will also execute all card series cards
and put those results into a list on the key :series-results in the dashcard
which is then used in the render multimethod to send the correct data to the javascript context
A small todo was taken care of as well, we can now use :dashboard-subscription in the :context key
* Fix up render fn
The shape of hte data going into the javascript was a bit unclear before. This keeps things a little more
understandable.
Next step is probably to create some Malli schemas inside the js_svg namespace to document and enforce the needed data shapes
* pull :data up a level from :result in the series cards
* Update src/metabase/pulse/render/body.clj
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Remove unneeded let
* set/rename-keys won't work here because we raise the :data key
I've realised a suggested change to just use set/rename-keys won't do what we need. We need instead to raise :data
from within the result key up to the same level (and dissoc result).
I've renamed the helper function to make its purpose a bit clearer
---------
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Sort functions
* Add boilerplate for aggregateOffset and tests
* Simplify assertion
* Make basic case work
* Handle name generation
* Do not require "lib/uuid" attribute to be present since MLv2 will normalize it under the hood anyway
* Fix typing
* Group tests
* Refactor offsetClause to return a new clause instead of a new query
- Move offsetClause to expressions.ts
- Add assertion
* Revert "Sort functions"
This reverts commit ab9ce2b24e.
* Move tests
* Handle names dynamically
* Shorten a test
* Update comment
* Add TODO
* Update expression types
* Add diffOffsetClause & percentDiffOffsetClause
* Add a test for diffOffsetClause
* Add TODOs
* Add tests for diffOffsetClause and percentDiffOffsetClause
* Unwrap tests
* Add skeleton for new tests
* Refactor first offsetClause tests
* Add tests for names
* Fix case of non-datetime columns
* Handle a case with offset < 1 and no breakouts
* Handle a case with offset < 1 and breakouts on non-datetime column
* Handle a case with offset < 1 and breakouts binned datetime column
* Handle a case with offset < 1 and breakout on non-binned datetime column
* Refactor
* Refactor
* Remove TODO
* Add tests for diffOffsetClause
* Add tests for percentDiffOffsetClause
* Move offset stuff to offset.ts and offset.unit.spec.ts
* Use template string for prefixes
* Use breakoutColumn + isDate to check column type
* Refactor
* Fix error message
* Add boilerplate for CompareAggregations
* Fix title
* Render aggregations list
* Style AccordionList
* Sort props
* Fix bucket name
* Use displayName instead shortName
* Support parseValue prop in NumberInput
* Add period input accepting integers only
* Accept non-negative values only
* Do not accept zeros
* Add state
* Add submit button
* Export offset functions via Lib
* Make it possible to skip rightSection
* Add column picker
* Map offset user input to api input (negative value)
* Add label
* Fix crash
* Extract shouldCreate
* Make onSelect in AggregationPicker support multiple values
* Extract ReferenceAggregationPicker
* Extract ColumnPicker
* Extract getAggregations
* Rename
* Add custom items
* Refactor item component
* Extract OffsetInput
* Remove unused data-testid
* Style OffsetInput
* Generate titles according to specs
* Generate label
* Generate help
* Extract utils
* Use different width for the 1st step
* Format code
* Use MultiSelect directly
* Avoid custom parseValue
* Revert MultiaAutocomplete changes
* Improve typing in describeTemporalInterval and
describeRelativeDatetime
* Use describeTemporalUnit to pluralize
* Use interface
* Avoid setting value as DOM attribute
* Fix test
* Add onAdd prop to AggregationPicker and revert the change to have onSelect pass multiple aggregations
* Reduce number of props
* Render checkboxes in custom items
* Introduce and use --mb-color-brand-lighter
* Avoid !important
* Remove redundant prop
* Rename ExtractColumn to ExtractColumnAction
* Add lines.svg
* Fix positioning in case dir attribute is not present in any parent component
* Add CompareAggregationsAction
* Fix warning about isSelected being used as DOM attribute
* Fix positioning in case dir attribute is not present in any parent component
* Remove redundant Box
* Compute the title
* Add compareAggregationsDrill boilerplate
* Add compare-aggregations drill stub
* Update comment
* Compute title and aggregation
* Add TODO
* Use proper title when there's only 1 aggregation
* Show visualization on submit
* Limit aggregation choice to 1 in compare aggregations drill
* Redirect to chart upon submission
* Do not navigate to chart view
* Do not navigate to chart view
* Make drill-thru tests expect the new compare-aggregations drill
* Add type attribute to all ListItems
* Add aggregationIndex to drillInfo
* Add aggregationDrillDetails and wire it up
* Add tracking helpers
* Add new schema
* Add tracking in column header
* Add tracking in plus modal
* Add tracking in aggregation picker
* Fix destructuring
* Add functionsUsedByExpression
* Fix schema number
* Add tests boilerplate
* Implement test for notebook editor
* Add test for column header
* Add a test for summarize sidebar
* Add a test for the plus button
* Add expectNoBadSnowplowEvents call in afterEach
* Format code
* Add missing redux store state in tests
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Use createQueryWithClauses in createQueryWithCountAggregation
* Support column-specific aggregations in createQueryWithClauses
* Add tests for compare option presence
* Fix case
* Add a test for onAdd
* Split tests
* Rename tests
* Add boilerplate for CompareAggregations unit tests
* Fix assertion
* Add more tests
* Add offset input tests
* Add tests
* Update frontend/src/metabase/query_builder/components/CompareAggregations/CompareAggregations.unit.spec.tsx
* Update tests after merge
* Use constants instead of factory functions
* Check git tag for an exact match
* Set npm publish to dry run, so I can test
* Uses new github automation account for creating a PR
* Remove unnecessary step
* Automatically set `no-backport` on the SDK doc version bump PR
* Fix step to edit PR with correct permission
* Remove npm dry run after finished testing
* return empty string so non-empty labels do not get hidden by empty labels
* fix function name
* apply scaling in transformed dataset
* add missing option to OptionsType
* fix double scaling
* add column scaling to waterfall transformed dataset
* add column value scaling to dataset transform
* fix stacked bar segment labels not responding to column scaling
* fix tooltip values displayed
* since scaling as added to transformedDataset, use transformedDataset for formatters
* revert changes to display percentage instead of absolute value
* fix stacked data labels
* add loki variant for custom y-axis range with column scaling
* fix stack segmented label formatters compact calculation
* scale original dataset and use that instead of modifying transformedDataset
* update loki snapshots
* fix scatter plot dataset scaling
* fix tooltip values being double scaled
* add E2E test to validate behavior
* add helper for not duplicating the column scaling
* port multiple y-axis metrics with column scaling fixes to RowChart
* add waterfall and scatterplot loki variants
* fix type errors and failing unit tests
* fix failing unit tests
* update loki snapshots
* Add E2E reproduction for #39150
* Fix typo in the test title
* Move the viewport config to the `describe` block
* Add the second repro scenario as per the review comment
* Scope the variable to the relevant test
* Sort functions
* Add boilerplate for aggregateOffset and tests
* Simplify assertion
* Make basic case work
* Handle name generation
* Do not require "lib/uuid" attribute to be present since MLv2 will normalize it under the hood anyway
* Fix typing
* Group tests
* Refactor offsetClause to return a new clause instead of a new query
- Move offsetClause to expressions.ts
- Add assertion
* Revert "Sort functions"
This reverts commit ab9ce2b24e.
* Move tests
* Handle names dynamically
* Shorten a test
* Update comment
* Add TODO
* Update expression types
* Add diffOffsetClause & percentDiffOffsetClause
* Add a test for diffOffsetClause
* Add TODOs
* Add tests for diffOffsetClause and percentDiffOffsetClause
* Unwrap tests
* Add skeleton for new tests
* Refactor first offsetClause tests
* Add tests for names
* Fix case of non-datetime columns
* Handle a case with offset < 1 and no breakouts
* Handle a case with offset < 1 and breakouts on non-datetime column
* Handle a case with offset < 1 and breakouts binned datetime column
* Handle a case with offset < 1 and breakout on non-binned datetime column
* Refactor
* Refactor
* Remove TODO
* Add tests for diffOffsetClause
* Add tests for percentDiffOffsetClause
* Move offset stuff to offset.ts and offset.unit.spec.ts
* Use template string for prefixes
* Use breakoutColumn + isDate to check column type
* Refactor
* Fix error message
* Add boilerplate for CompareAggregations
* Fix title
* Render aggregations list
* Style AccordionList
* Sort props
* Fix bucket name
* Use displayName instead shortName
* Support parseValue prop in NumberInput
* Add period input accepting integers only
* Accept non-negative values only
* Do not accept zeros
* Add state
* Add submit button
* Export offset functions via Lib
* Make it possible to skip rightSection
* Add column picker
* Map offset user input to api input (negative value)
* Add label
* Fix crash
* Extract shouldCreate
* Make onSelect in AggregationPicker support multiple values
* Extract ReferenceAggregationPicker
* Extract ColumnPicker
* Extract getAggregations
* Rename
* Add custom items
* Refactor item component
* Extract OffsetInput
* Remove unused data-testid
* Style OffsetInput
* Generate titles according to specs
* Generate label
* Generate help
* Extract utils
* Use different width for the 1st step
* Format code
* Use MultiSelect directly
* Avoid custom parseValue
* Revert MultiaAutocomplete changes
* Improve typing in describeTemporalInterval and
describeRelativeDatetime
* Use describeTemporalUnit to pluralize
* Use interface
* Avoid setting value as DOM attribute
* Fix test
* Add onAdd prop to AggregationPicker and revert the change to have onSelect pass multiple aggregations
* Reduce number of props
* Render checkboxes in custom items
* Introduce and use --mb-color-brand-lighter
* Avoid !important
* Remove redundant prop
* Rename ExtractColumn to ExtractColumnAction
* Add lines.svg
* Fix positioning in case dir attribute is not present in any parent component
* Add CompareAggregationsAction
* Fix warning about isSelected being used as DOM attribute
* Fix positioning in case dir attribute is not present in any parent component
* Remove redundant Box
* Compute the title
* Add compareAggregationsDrill boilerplate
* Add compare-aggregations drill stub
* Update comment
* Compute title and aggregation
* Add TODO
* Use proper title when there's only 1 aggregation
* Show visualization on submit
* Limit aggregation choice to 1 in compare aggregations drill
* Redirect to chart upon submission
* Do not navigate to chart view
* Do not navigate to chart view
* Make drill-thru tests expect the new compare-aggregations drill
* Add type attribute to all ListItems
* Add aggregationIndex to drillInfo
* Add aggregationDrillDetails and wire it up
* Fix destructuring
* Fix tests
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* tag flakiest tests as such
* let well behaved tests out of flake jail
* more bad actors off the streets
* add another flake
* sandboxes re-offended already
* support test utilities in sdk import
* add relative unit default chart font size in embedding defaults
* make font size follow sdk root
* feat(sdk): add font size settings to axis ticks
* feat(sdk): add font size settings to series labels
* re-add hasTimelineEvents from merge commits
* apply default font size at the theme level
* add table header cell font size
* calculate cell font size based on 12.5px
* rename theme.chart to theme.cartesian
* refactor visualization theme
* refactor default viz theme
* remove chart style config
* add goal line
* pass font size to chart measurements
* use px as reference unit
* support row chart viz
* add missing echarts font size
* add table cell font sizes
* use css variables for funnel color
* fix theme type in smart scalar story
* add bar chart to storybook for visual test
* add line chart to storybook for visual test
* add loki reference images
* add loki reference images for row chart
* add loki shared reference images to stories filter
* update loki reference image
* synchronize base font size
* add theme provider to funnel test setup
* add missing default visualization theme
* add unit test for get size in px
* update loki reference image
* fix static viz build error
* Sort functions
* Add boilerplate for aggregateOffset and tests
* Simplify assertion
* Make basic case work
* Handle name generation
* Do not require "lib/uuid" attribute to be present since MLv2 will normalize it under the hood anyway
* Fix typing
* Group tests
* Refactor offsetClause to return a new clause instead of a new query
- Move offsetClause to expressions.ts
- Add assertion
* Revert "Sort functions"
This reverts commit ab9ce2b24e.
* Move tests
* Handle names dynamically
* Shorten a test
* Update comment
* Add TODO
* Update expression types
* Add diffOffsetClause & percentDiffOffsetClause
* Add a test for diffOffsetClause
* Add TODOs
* Add tests for diffOffsetClause and percentDiffOffsetClause
* Unwrap tests
* Add skeleton for new tests
* Refactor first offsetClause tests
* Add tests for names
* Fix case of non-datetime columns
* Handle a case with offset < 1 and no breakouts
* Handle a case with offset < 1 and breakouts on non-datetime column
* Handle a case with offset < 1 and breakouts binned datetime column
* Handle a case with offset < 1 and breakout on non-binned datetime column
* Refactor
* Refactor
* Remove TODO
* Add tests for diffOffsetClause
* Add tests for percentDiffOffsetClause
* Move offset stuff to offset.ts and offset.unit.spec.ts
* Use template string for prefixes
* Use breakoutColumn + isDate to check column type
* Refactor
* Fix error message
* Add boilerplate for CompareAggregations
* Fix title
* Render aggregations list
* Style AccordionList
* Sort props
* Fix bucket name
* Use displayName instead shortName
* Support parseValue prop in NumberInput
* Add period input accepting integers only
* Accept non-negative values only
* Do not accept zeros
* Add state
* Add submit button
* Export offset functions via Lib
* Make it possible to skip rightSection
* Add column picker
* Map offset user input to api input (negative value)
* Add label
* Fix crash
* Extract shouldCreate
* Make onSelect in AggregationPicker support multiple values
* Extract ReferenceAggregationPicker
* Extract ColumnPicker
* Extract getAggregations
* Rename
* Add custom items
* Refactor item component
* Extract OffsetInput
* Remove unused data-testid
* Style OffsetInput
* Generate titles according to specs
* Generate label
* Generate help
* Extract utils
* Use different width for the 1st step
* Format code
* Use MultiSelect directly
* Avoid custom parseValue
* Revert MultiaAutocomplete changes
* Improve typing in describeTemporalInterval and
describeRelativeDatetime
* Use describeTemporalUnit to pluralize
* Use interface
* Avoid setting value as DOM attribute
* Fix test
* Add onAdd prop to AggregationPicker and revert the change to have onSelect pass multiple aggregations
* Reduce number of props
* Render checkboxes in custom items
* Introduce and use --mb-color-brand-lighter
* Avoid !important
* Remove redundant prop
* Rename ExtractColumn to ExtractColumnAction
* Add lines.svg
* Fix positioning in case dir attribute is not present in any parent component
* Add CompareAggregationsAction
* Fix warning about isSelected being used as DOM attribute
* Fix positioning in case dir attribute is not present in any parent component
* Remove redundant Box
* Compute the title
* Use proper title when there's only 1 aggregation
* Show visualization on submit
* Do not navigate to chart view
* Add type attribute to all ListItems
* move parameters and dashboard reducers to the common ones as they're used everywhere
* convert reducers-{common,main,public}.js to typescript, done with --no-verify
* convert reducers to named exports
* Sort functions
* Add boilerplate for aggregateOffset and tests
* Simplify assertion
* Make basic case work
* Handle name generation
* Do not require "lib/uuid" attribute to be present since MLv2 will normalize it under the hood anyway
* Fix typing
* Group tests
* Refactor offsetClause to return a new clause instead of a new query
- Move offsetClause to expressions.ts
- Add assertion
* Revert "Sort functions"
This reverts commit ab9ce2b24e.
* Move tests
* Handle names dynamically
* Shorten a test
* Update comment
* Add TODO
* Update expression types
* Add diffOffsetClause & percentDiffOffsetClause
* Add a test for diffOffsetClause
* Add TODOs
* Add tests for diffOffsetClause and percentDiffOffsetClause
* Unwrap tests
* Add skeleton for new tests
* Refactor first offsetClause tests
* Add tests for names
* Fix case of non-datetime columns
* Handle a case with offset < 1 and no breakouts
* Handle a case with offset < 1 and breakouts on non-datetime column
* Handle a case with offset < 1 and breakouts binned datetime column
* Handle a case with offset < 1 and breakout on non-binned datetime column
* Refactor
* Refactor
* Remove TODO
* Add tests for diffOffsetClause
* Add tests for percentDiffOffsetClause
* Move offset stuff to offset.ts and offset.unit.spec.ts
* Use template string for prefixes
* Use breakoutColumn + isDate to check column type
* Refactor
* Fix error message
* Add boilerplate for CompareAggregations
* Fix title
* Render aggregations list
* Style AccordionList
* Sort props
* Fix bucket name
* Use displayName instead shortName
* Support parseValue prop in NumberInput
* Add period input accepting integers only
* Accept non-negative values only
* Do not accept zeros
* Add state
* Add submit button
* Export offset functions via Lib
* Make it possible to skip rightSection
* Add column picker
* Map offset user input to api input (negative value)
* Add label
* Fix crash
* Extract shouldCreate
* Make onSelect in AggregationPicker support multiple values
* Extract ReferenceAggregationPicker
* Extract ColumnPicker
* Extract getAggregations
* Rename
* Add custom items
* Refactor item component
* Extract OffsetInput
* Remove unused data-testid
* Style OffsetInput
* Generate titles according to specs
* Generate label
* Generate help
* Extract utils
* Use different width for the 1st step
* Format code
* Use MultiSelect directly
* Avoid custom parseValue
* Revert MultiaAutocomplete changes
* Improve typing in describeTemporalInterval and
describeRelativeDatetime
* Use describeTemporalUnit to pluralize
* Use interface
* Avoid setting value as DOM attribute
* Fix test
* Add onAdd prop to AggregationPicker and revert the change to have onSelect pass multiple aggregations
* Reduce number of props
* Render checkboxes in custom items
* Introduce and use --mb-color-brand-lighter
* Avoid !important
* Remove redundant prop
* Fix warning about isSelected being used as DOM attribute
* Fix positioning in case dir attribute is not present in any parent component
* Add type attribute to all ListItems
On the Browse models page we have a filter icon which opens a popover with a switch that toggles "Show only verified models". With this change, the icon is not rendered if there are no verified models at all.
* remove unused variable
* conditionally render labels sparsely if large amount of data points
* update loki snapshots
* refactor data label formatter conditions
* setup intelligent label plotting
* handle the case when there are no labels
* add sparse data label rendering to bar charts
* do not compute if no labels need to be shown or hidden
* add sparse data labeling to waterfall charts
* adjustment for large series waterfall labels
* revert change
* refactor name
* add sparse labels to stacked bar and area
* improve scaleFactor
* fix type errors
* remove unused labelFormatter
* update snapshots
* patch ECharts
- labels with no text, i.e. "", were still being considered as plot-able labels which was causing labels with real text "asdfec.." to be hidden because of overlap.
- This patch makes it so that labels with no text, i.e. "", are not considered to be candidates for overlap checking.
* update loki snapshots
* only loop over dataset once for getWaterfallChartDataDensity
* improve performance of chart data density calculations
* update loki snapshots
* update E2E spec
* update E2E spec
* increase cartesian label density allotment
* update loki snapshots
* update loki snapshots
* comment out node_modules cache since it cannot be skipped by commit and is breaking CI
* fix type errors
* Reset node modules if patches are changed
* refactor type naming
---------
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* fix(sdk): add Static Dashboards to README
* Add more clarification comments for API
* Clarify dashboard ID type
* Prevent crashing SDK static dashboard if not provided optional values
* Document fallback behavior for `font`
---------
Co-authored-by: Mahatthana Nomsawadi <mahatthana.n@gmail.com>
* Auto tag embedding team for PR review + dry run npm publish
* Install changelog generation package
* Add the initial changelog
* feat(sdk): This is an empty commit. For testing purposes.
* Update SDk build script to include changelog
* Check if a tag exists
* Ensure changelog is in the released package
* Fix workflow
* Remove unnecessary step
* Ensure we can have access to release utils script when checking out master
* Fix the step to create a PR updating readme and changelog
* Fix changelog generation
* Fix couldn't download changelog
* Clean up steps
* initial commit with most types sorted
* admin, binning, collections, custom-column and dashboard-cards suites
* Filters, Joins, Metrics, Models, Native and Native filters suites
* rest of e2e specs
* filters, actions, onboarding and custom column will hopfully be green?
* fixing static viz, brush filtering, login redirecting, and a few other tests
* types green
* linter green
* hopefully fixes filter tests
* I don't often cry, but when I do it's because of filter-types
* hopfully sorted the last test that's failing CI
* PR Feedback
* yarn prettier
* React 18 Upgrade - Unit Tests (#41577)
* fixes for most unit tests to work with react 18 + upgraded rtl to v15
* fixes all unit tests locally
* fixes from rebasing
* linter shames me
* fix flake in maps.cy.spec.js
---------
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* Do not use asReturned to determine columns for extractions and combine columns
* Add reproduction for #43226 for combine columns
* Add reproduction for #43226 for extract columns
* Remove commented out code
* Only render extraction shortcut when there are extractions
* Only render combination shortcut when there are combinations
* Use hasCombinations from CombineColumns
* Add hasExtractions from ExtractColumn
* Use appendStageIfAggregated over Lib.asReturned
* Only show combine column shortcut when there are two or more columns to be combined
* Lift appendStageIfAggregated to top-level drill
* Add test for extraction on table with just breakout
* Add test for combinations on table with just breakouts
* Disable the 2-column requirement for combinations
* Remove Lib.asReturned in appendStageIfAggregated
* Remove + 1 to stageIndex in appendStageIfAggregated
* Check for empty breakouts too in as-returned
* Switch back to Lib.asReturned
* Fix test for combine column on breakouts
* Reference the correct query and stageIndex
* add CLJS unit tests for as-returned with only aggs, breakouts
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
Models are treated as opaque here, but for a saved question we expose
filters on the query before its final aggregation step.
This drops the aggregation clauses and gets the possible values for the
column of interest.
Fixes#43154.
* Multiple Measures Requires Additional Header Row in Pivot Exports
Mulitple measures can be set in our pivots. We indicate these properly in the fronted with a header row that shows
which measure is reprsented in that particular column.
The header builder fn in the pivot export post processing didn't handle this case, but after this PR, it correctly
adds the final header when necessary.
* Fix up header tests
* Remove comment form; the tests are derived from that work already
These databases previously had impoverished support for regular
expressions. The `:regex` driver feature requires advanced,
"Perl-compatible" regular expressions that can do things like lookahead
assertions.
- MySQL supports this from 8.0.4; we recommend 8.0.33+.
- MariaDB supports this from 10.0.5; we recommend 10.4+.
Fixes#43180.
* Avoid overflowing dropdown menus
* Hide most bucket picker items behind a More... button
* Add test for More… button on TimeseriesBucketPicker
* Add color to More... button
* Add test for showing all columns when a lower bucket is picked
* Handles card events for CRU (not D) => recent view
* add tests
* avoid some errors in dev
We expect recent-views/post-process to return nil occasionally (for
permissions reasons, or more), so let's filter those out instead of
printing a noisy unactionable warning
* uncomment and fix test
* fix some cypress tests + docs improvements
* update more cypress tests
* more cypress test updates
* adding a card puts it into your recents now
* fix notebook-data-source
* fix notebook-data-source
* fixing more e2e tests
* fix command palette e2e tests
* run prettier
* fix style issue
* update notebook-data-source tests
* update homepage e2e tests
* Next question is in recents, now
* update command palette: skip the first pagedown, since it is a noop
* check the buttons for their respective names
* ensure that the model is there, and no questions are
* Keep a single null dimension value
* Add Loki test
* Remove Loki test
* Throw an error if dataset is empty after filtering
* Extract error message into a constant
* Add color to icon directly
* Remove unused css file
* Only show info icon when the field is focussed
* Use metabase blue for background on value tags
* Use larger font size for value labels
* Make the value labels bold
* Tweak padding on search input
* Move custom MultiAutocomplete styles to MultiSelect overrides and adapt to accept xs size too
* feat(sdk): colors for pivot tables
* fix theme type
* feat(sdk): basic pivot table theming
* fix emphasis hover color
* customizable row toggle color
* add storybook story and refactor hover color logic
* add sdk visualization wrapper
* make theme values with defaults internally non-optional but externally optional
* update test to render with providers
* update test wrapper to have height container
* update pivot table loki reference image
* update theme names
* make all theme options optional in the public config
* update readme
* readme update for consistency
- Update how base type and effective type is added in query-method :mbql/query.
- Base type and effective type is added also during the conversion of query-method :query.
- Adds warning suppression for expressions to places where normalize is
called something that does not conform schme (eg. ->pMBQL :dispatch-type/map called
in query-method :query).
* Allow disabling encryption for settings
For two settings, we want to disable encryption entirely.
On the way out of the data source, we don't need to worry - the
decryption is already backwards-compatible with non-encrypted data.
On the way in, things are a bit less straightforward. The core of the
change is replacing the `t2/deftransform` that *always* runs
`encryption/maybe-encrypt` on the value with:
- a define-after-select that always tries to decrypt the value (so we
can handle existing encrypted values in the db)
- a define-before-insert that encrypts the value if the `defsetting`
told us to, and
- a define-before-update that does the same thing.
For now, set two settings to be never-encrypted: `analytics-uuid` and
`anon-tracking-enabled`.
* Sort functions
* Add boilerplate for aggregateOffset and tests
* Simplify assertion
* Make basic case work
* Handle name generation
* Do not require "lib/uuid" attribute to be present since MLv2 will normalize it under the hood anyway
* Fix typing
* Group tests
* Refactor offsetClause to return a new clause instead of a new query
- Move offsetClause to expressions.ts
- Add assertion
* Revert "Sort functions"
This reverts commit ab9ce2b24e.
* Move tests
* Handle names dynamically
* Shorten a test
* Update comment
* Add TODO
* Update expression types
* Add diffOffsetClause & percentDiffOffsetClause
* Add a test for diffOffsetClause
* Add TODOs
* Add tests for diffOffsetClause and percentDiffOffsetClause
* Unwrap tests
* Add skeleton for new tests
* Refactor first offsetClause tests
* Add tests for names
* Fix case of non-datetime columns
* Handle a case with offset < 1 and no breakouts
* Handle a case with offset < 1 and breakouts on non-datetime column
* Handle a case with offset < 1 and breakouts binned datetime column
* Handle a case with offset < 1 and breakout on non-binned datetime column
* Refactor
* Refactor
* Remove TODO
* Add tests for diffOffsetClause
* Add tests for percentDiffOffsetClause
* Move offset stuff to offset.ts and offset.unit.spec.ts
* Use template string for prefixes
* Use breakoutColumn + isDate to check column type
* Refactor
* Fix error message
* Use displayName instead shortName
* Use sentence case for suggestion headers
* Remove suggestion group dividers
* Highlight suggestions when mousing over them
* Remove unused hoverparent
* Remove custom highlighting for icons
* Allow passing color to metadata info icon
* Make suggestions icons have a consisten color
* Update documentation suggestion style
* Add margin to suggestion sections
* Reorder suggestion sections
* Add test for mouse over highlight on suggestions
* Fix test for suggestion footers
* Remove debugging true
* Rename 'Most used functions/aggregations' to 'Common functions/aggregations'
* Highlight suggestion on mouse move
* Only call onHighlightSuggestion if it is not already highlighted
* Use arrow functions
* Fix test for suggestion highlight
* Do not render 'Add column' shortcut when viewing a raw table
* Do not show 'Add column' shortcut when a query has aggregations
* List shortcut guards out of component
* Make 'Add column' icon sticky in the header
* Add test for hiding the column shortcut
* Remove Number wrapper
* Move padding to a constant
* Use actions directly to determine if there are any shortcuts
* Make sure there are expressionable columns for the combine columns shortcut
* Make sure there are extractable columns for the extract column shortcut
* Remove padding when not overflowing
* Do not shadow query in popover components
* Allow combine and extract for raw tables
* Add test for combining summarization columns
* Add test for combining summarization columns
* Use a more precise way of calculating the offset for the 'Add column' button
* Handle null question
* Change to the outline button
* Use actual column widths to calculate total width
* Handle empty mode
* Remove extra measure call
* Move shortcutColumn to separate method and avoid looping over all available actions when possible
* Revert "Revert "minimal support for multiple values for Access-Control-Allow-Origin (#42888)" (#43125)"
This reverts commit a4312230a8.
* refactor: extracted parse-approved-origins + supported "*" without special case
* fix: parse-approved-origins won't break on multiple spaces in a row
* handles invalid urls in the origins + some formatting
* log instead of throwing in parse-url + simplify the logic in parse-approved-origins
* simplify parse-approved-origins logic by splitting on "at least one space"
* remove extra boolean call
* approved-origins -> approved-origins-raw to better reflect that it's a raw string
* swap the order of the arguments to `is` in the tests
* indentation 🙃
* fix trend charts erroring on latest value being null
* refactor previousRowIndex calculation
* handle errors gracefully
* refactor error messages
* update computeTrend unit tests
* add error handling integration test
* add E2E test to check that errors are gracefully handled
* add integration test to enforce the fix
* fix static viz
* display more meaningful error on static-viz trend charts
* move error handling out of trend chart and fix error handling in Visualization.tsx
* update compute.unit.spec.js
* update SmartScalar.unit.spec.js
* use separate error handler for ErrorBoundary errors in Visualization.jsx
* update E2E test
* update error handling
* update compute.unit.spec.js
* fix react state change error in render function
* show generic error message for ErrorBoundary caught errors
* revert ErrorBoundary changes
* remove generic error border for visualizations
* Pass current values to shouldCreate
* Allow comma to create new item
* Simplify getAvailableSelectItems
* Add unit test for MultiAutocomplete input
* Use text instead of Text in paste
* Handle spaces in MultiAutocomplete
* Remove cast
* Reuse last var
* Use haveLastBeenCalledWith over haveBeenCalledWith
* Handle pasting when there is text in the input already
* Accept quote-delimited values
* Add info icon
* Handle tab as a separator
* Fix test cases for unsanitized parseValues
* Add csv-parse
* Use csv-parse to parse values
* Apply the correct color to the info icon
* Rename util to utils
* Break up tooltip text
* Only return unique values
* Be more accepting around quotes in the middle of unescaped values
* provide Buffer for csv-parse package in storybook
* Use browser-friendly version of csv-parse
* Remove unused eslint rule
* Add specital case for csv-parse to jest config
* Put uniqueness check in each state update
* Only add partial value if it is valid
* Only add partial value if it is valid
* Fix test case for FilterValuePicker
* Just use Set
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Allow parsing more than one value
* Use isValid everywhere it makes sense
* Add test for RTL languages
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Remove dead CSS
* Use proper types
* Properly type parameterValues
* Add href prop support to LegendCaption
* Extract getNewCardUrl
- move getParametersMappedToDashcard
* Use href in CartesianChart
* Handle RowChart
* Handle Funnel
* Format code
* Fix types
* Update first test in title-drill
* Update test
* Update test
* Update tests
- pass questionId in createQuestionAndDashboard
* Add test for links
* Fix createDashboardWithQuestions
* Add missing click handler
* Fix crash
* Fix typo
* Add a workaround for #42999
* Fix tests
* Fix hover color
* Update comment
* Get rid of mapStateToProps in DashboardGrid
* Invert condition
* Use useCallback instead useMemo
* Do not pass getNewCardUrl down if navigateToNewCardFromDashboard is not present
- Remove default noop for navigateToNewCardFromDashboard in PublicDashboard
* Remove explicit undefined prop
* add color types for chart sdk colors
* simplify chart color types
* chart color mappings
* apply mapping to master branch
* add chart mapping to mantine
* feat(sdk): apply chart accent colors to mantine and color getter
* rename tint and shade
* remove chart color logic from mantine
* add a color mapping test
* docs(sdk): add chart customizations to readme
* add type comments on MappableSdkColor
* cleanup sdk colors
* Convert DashboardControls to TS hooks
* fix types
* Modify dashboard url params to ensure that we keep hash options that haven't changed
* Fix PublicDashboard and DashboardTabs tests
* Use synced dashboard tabs for automatic dashboard, and use normal tabs for those with DashboardControls
* reduce hash option location changes to only the ones we can change through the UI
* Add comment on state management
* Remove console
* Attempt to solve more tests
* Remove loadDashboardParams as a passed prop
* Fix some tests, but will cause unit test to break
* Change names to reflect actual usage
* Fix lint and type issues
* Add simpler url param control, simplify types, and simplify fullscreen hook
* Add two way binding with url params to attempt to fix tests
* Fix unit test + types
* Move folders and fix imports
* Use Record<string, unknown> for refresh hook
* Remove console.log
* Clear query parameters when ParametersList unmounts
* Add comment
* Revert DisplayTheme changes
* Fix weird reloading problem
* Fix reducers test
* add code comments on selecting the tab once tab data is loaded
* add code comments on unsyncing parameters
* Fix e2e tests, move url reset to tabs context
* Fix e2e tests
* Hotfix
---------
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Avoid unnecessary re-renderings in dashboard - getEmbeddedParameterVisibility
* Avoid unnecessary re-renderings in dashboard - favicon, page title
* Exclude isLoadingComplete from passing down
* Avoid unnecessary re-renderings in dashboard - metadata
* Drop metadata from Public Dashboard
* Avoid unnecessary re-renderings in dashboard - dashcardData
* minimal support for multiple origins in security middleware
* if -> when to be more idiomatic
* adds tests for parse-url
* if -> when
* add missing new line in tests
* split the logic of validation in functions
* adds tests for the small functions
* use namespace in tests instead of :refer
* Make accordion list item content 100% wide
* Remove unused Dot component
* Move bucket popover trigger to the end of the item
* Revert QueryColumnPicker styles to what they were before adding the info icons
* Fix the height of the bucket picker
* refactor scatter plot model to separate function
* (2/3) refactor scatter plot option to separate function (#43033)
* use getScatterPlotOption
* use buildEChartsScatterSeries directly
* remove hoveredSeriesDataKey from scatter
* (3/3) improve chart model types (#43034)
* create WaterfallChartModel type
* create ScatterPlotModel type
* rename BaseCartesianChartModel to CartesianChartModel
* simplify chartModel creation in hook
* Search API namespace
* Search API namespace
* Search API namespace
* Fix Kondo warnings
* Fix typo
* Finally solve the MySQL ^:parallel test failures <3
* Make some search tests REPL-friendly
* Did I finally fix `^:parallel` MySQL tests?
* Ok I give up just make the search tests single-threaded for MySQL/MariaDB.
* Revert the stuff to make paralell tests work for MySQL... for now.
* Keep new name
* Revert deps-graph changes
* Remove unused namespaces
* Fix Kondo linter for log/info + format
* Misc fixes 🔧
* Test fixes 🔧
This is a bit painful. I merged this change, but realized we need to
backport the fix to v49. However:
- we don't want to have two versions of the migration (one with a v49
id, one with a v50 id) because then if someone upgrades to 50, then
downgrades to 49, the `rollback` will run and change the type back,
leading to a bug.
- we don't want to push a v51 changeSet ID to v49 or v50, because we
give the user a helpful notice when their database needs a downgrade.
We do this by checking for the latest *executed* migration in the
database and comparing it to the latest migration that Liquibase knows
about, and making sure the executed isn't bigger than the known (e.g.
you can't have executed a v51 migration if it isn't in the local
migration yaml). That would all work fine, except that then we want to
tell you how to downgrade your database, and we use the latest-executed
version for that. So if, for example, someone upgraded from 48 to 49 and
got a v51 changeset, then downgraded back to 48, they would get an error
telling them to run the *v51* jar to roll back their DB.
In this case though, I think it's fine to just move the migration around
to v49, then we can backport it to 49 and 50.
* handle visibilty_type on tables
- remove def wtf
- uncomment + fix tests
* make sure parent colls exist on report_cards
- official is a string
- ensure the database for a recent table exists
* remove def + improve arg name
* docstring
* improve + extract valid parent collection check
* grab the entity's colllection id and use that for the validity test
- This seems like something that should never happen, but I've seen it
happen. I think it might be possible from a down/up migration.
* fix tests
* fix more tests
* cleanup
* fixing another test
* add error-avoider, to filter bad items in prod instead of throwing
- improve recent view testing for table
* remove wrap (it double prints the value)
* [Metrics V2] Backfill viz settings for compatible-series
Fixes#42575
The BE should rely on the query when there is no corresponding viz setting.
If there is no graph.metrics setting, assume that all last-stage aggregations are used.
No graph.dimensions -> use breakouts.
* Fix tests
* Unskip e2e test
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* hide symbols for charts with too many of them
* update loki
* prevent series dots showing up after manually disabling another series
* update loki snapshots
* refactor conditional
* update loki snapshots
* Migrate druid metrics-inside-aggregation-clauses-test to druid-jdbc
* Fix test-runner: druid should not exclude druid-jdbc
* Implement set-parameter and unprepare-value for LocalDateTime for druid-jdbc
* feat(sdk): apply ui color overrides to the sdk
selector state
set global embedding colors
rename the method
* test(sdk): add unit test on color overrides
* [MBQL lib] Reject broken uses of `offset` in expressions, filters (#42662)
- Using `offset` in custom expressions is supported only when there is
an order-by defined on that stage.
- Using `offset` in custom filters is not supported at all, currently.
* Support `Offset()` in custom columns [frontend] (#42326)
* Only nest expressions referenced in breakouts or aggregations
* Support Offset() as expression with no breakouts
* Test fixes 🔧
* Oracle test update
* Improved Oracle test
* Test update 🔧
* Fix busted test
* Add naive support for Offset() in expressions
* Introduce MBQLClauseFunctionReturnType
* Add "window" to MBQLClauseFunctionReturnType and use it for the offset function
* Handle offset type inference
* Remove unused export
* Use "expression" instead of "window" return type
- Rename identifiers in isCompatible
- Make isCompatible accept a clause object instead of just the type
- Handle offset as a special case in isCompatible
* Use any type
* Rename expectedArgumentType to expectedType
* Format code
* Sort types
* Do not suggest offset function in filters
* Fix offset not working in case
* Revert "Do not suggest offset function in filters"
This reverts commit e63790b780.
* Fix order of adjustments
---------
Co-authored-by: Cam Saul <github@camsaul.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Disable offsets in filters expressions
* Group existing aggregations-specific tests
* Remove repro for a closed issue
* Use findByText instead of contains
* Add a test for filter expressions
* Add a test for aggregation expressions suggestions
* Disable offsets in filters expressions (#42755)
* Add a test for custom column suggestions
* Add "typing" to enterCustomColumnDetails
* Use enterCustomColumnDetails, improve assertions
* Add more assertions
* Optimize queries
* Add typing for offset expressions
* Add a repro for metabase#42764
* Add a TODO
* Add a TODO
* Add a TODO
* Use NumericLiteral
* Post-merge fixes
* Update test
* Add tests for other custom expressions
* Test drills
* Format code
* Update test name
* Add an assertion
* Add assertions for the preview
* Unskip fixed issue
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
Co-authored-by: Cam Saul <github@camsaul.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
In v49, we converted all boolean types to `bit(1)` on mysql/mariadb. It
looks like we missed this one, so when you `GET` a card it has
`"collection_preview": 1`. Then we get an error when trying to save,
because `collection_preview` should be a boolean.
I verified that after this migration, `GET`ting a card results in a
boolean `"collection_preview"`.
* Set scrollToLastColumn ui control from column extract action
* Scroll to last column when the uiControl is set
* Scroll to last column when combining a column
* Scroll to last column when combining a column via the + shortcut
* Scroll to last column when extracting a column via the + shortcut
* Remove all references to settingsSyncOptions
* Check that the column is added at the end
* Disable scrollToLastColumn once used
* Add test that verifies scrollToLastColumn ui control has been reset
* Fix nil schema in /api/table/:id/query_metadata
* Fix nil schema in /api/table/:id/query_metadata
* Fix nil schema in /api/table/:id/query_metadata
* Fix nil schema in /api/table/:id/query_metadata
* Fix nil schema in /api/table/:id/query_metadata
* Revert "Fix nil schema in /api/table/:id/query_metadata"
This reverts commit 38ac092335.
* Revert "Fix nil schema in /api/table/:id/query_metadata"
This reverts commit 2b8fe78ed4.
* Revert "Fix nil schema in /api/table/:id/query_metadata"
This reverts commit 219506f0e0.
* Revert "Fix nil schema in /api/table/:id/query_metadata"
This reverts commit 9430478bdf.
* Revert "Fix nil schema in /api/table/:id/query_metadata"
This reverts commit 8c6cdd9068.
* Table Picker (#40509)
* Generic types
* Generic types
* Generic types + make generateKey a prop
* Generic types
* Extract AutoScrollBox.styled.tsx
* Move AutoScrollBox to a separate directory
* Generic types
* Generic types
* Generic types
* Unhookify searchFilter
* Generic types
* Generic types
* Generic types
* Generic types
* Generic types
* Generic types
* Generic types
* Generic types
* Finish dealing with a cast
* Destructure import
* Inline type
* Generic types
* Remove cast
* Remove commented code
* Remove redundant fallback
* Move CollectionPicker out of EntityPicker
* Avoid as unknown
* Remove TODOs
* Rename TisFolder to IsFolder
* Fix any
* Fix anys
* Remove NestedItemPicker's storybook
* Remove a cast
* Remove a cast
* Remove a cast
* Revert "Remove a cast"
This reverts commit b762d0f076.
* Use extends SearchModelType instead of extends string everywhere
* Revert "Use extends SearchModelType instead of extends string everywhere"
This reverts commit 708190d63e.
* Fix SearchResult["available_models"] type
* Handle options.allowCreateNew
* Add missing description attribute to Database type
* Move allowCreateNew to CollectionPickerOptions
* Add missing description attribute in Database mock
* Add types
* Add TableList component
* Add SchemaList component
* Fix naming
* Add DatabaseList component
* Add NotebookDataItemPickerResolver
* Fix typing
* Add TablePicker and NotebookDataPickerModal
* Update types
* Fix types
* Add folder type
* Fix initial state
* Update title
* Make query model-dependent
* Render tables
* Fix schema icon
* Adjust key generation
* Rename utilts to utils
* Use ItemList error prop
* Use value prop
* Rename NotebookDataPickerModal to DataPickerModal
* Automatically open datapicker if there is no value
* Highlight current item
* Rename value to initial value
* Fix item highlighting
* Rework TablePicker state
* Leave TODOs for names
* Fix selecting items
* Fix highlighting selected item
* Hide confirmation button
* Add fetchMetadata to tableApi
* Fix Table['schema'] type
* Fetch table metadata upon selection
* Remove options from DataPickerModal
* Fix collectionId typing in Question
* Add collectionId prop
* Remove title prop
* Add todo
* Fix confirmation button
* Remove old DataSourceSelector
* Bring back useSchemaListQuery
* Fix crash
* Fix picker not opening
* Hide dbs list and schemas list if there's only 1
* Remove unused ref
* Remove unused DataPickerListResolver
* Remove unused types
* Change conditional rendering to allow for loading state
* Revert collection-related changes
* Use RTK
* Avoid using stale data
* Fix highlighted db
* Remove todo
* Remove unused type
* Remove useSchemaListQuery
* Rename Value to TablePickerValue
* Move isValueEqual to utils and rename it to isTablePickerValueEqual
* Improve TODO
* Remove unused options prop
* Introduce tablePickerValueFromTable
* Avoid having 2 table metadata requests
* Update comment
* Update comment
* Remove unused function
* Allow null in tablePickerValueFromTable
* Extract helpers
* Rename schemaId to schemaName
* Remove description
* Improve naming
* Simplify types
* Add explanatory comments
* Fall back to empty state when there are no items
* Account for null schemas
* Account for null schema in tests
* Revert "Account for null schema in tests"
This reverts commit 0c96d1cb88.
* Revert "Account for null schemas"
This reverts commit 4f5c4e99ee.
* Improve typing around schema name
* Refactor auto-select logic
* Refactor
* Avoid comment
* Make useAutoSelectOnlyItem generic
* Do not enforce presence of at least 1 tab with TypeScript
* Fix types
* Remove obsolete comment
* Post-merge fix
* Integrate QuestionPicker in DataPicker (#42268)
* Add question and model pickers to Data Picker
* Add model attribute to data picker value
* Introduce DataPickerValue
* Start with the right tab when opening data picker modal
* Do not use stale sourceCard value
* Fix switching tabs when a table is selected
* Use undefined instead of null for consistency
* Do not show models/questions tabs when there are no models/questions
* Fix selecting models/questions in search results
* Remove dead code
* Remove dead code
* Ignore selecting collections
* Refactor useHasModels and useHasQuestions into useAvailableData
* Use models param to speed up the query
* Do not require `onConfirm` prop in `EntityPickerModal` (#42317)
* Make onConfirm prop optional
* Add a test for onConfirm prop assertion
* Table does not appear selected in single-schema dbs (#42336)
* Convert useAutoSelectOnlyItem arguments to param object
* Disable auto-selection when there already is a selection
* Use `DataPicker` in `JoinTablePicker` (#42330)
* Refactor DataStep:
- rename handleTableSelect to handleTableChange
- extract value and memoize it
* Remove redundant fragment
* Add new DataPicker props
* Use DataPickerModal in JoinTablePicker
* Remove initialTab prop since the default tab is always the models tab anyway
* Limit table selection to particular database
* Limit search results to a given database
* Support databaseId in useAvailableData
* Filter collection items by databaseId
* Use databaseId prop only for CollectionItemList component
* Fix types
* Fix filtering
* Use useLatest for onChange
* Fix typing
* Introduce shouldShowItem and use it instead of databaseId prop
* Improve PickerInfo type
* Use MLv2 to generate DataPickerModal value in JoinTablePicker
* Fix title
* Add isModel to TableDisplayInfo
* Add a hack to populate query metadata with info that table is a model
* Remove NewQuestionView & QuestionDataSelector (#42562)
* Temporarily disable recents tab in the data picker
* Update Data Picker tests (#42565)
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Fix metadata loading
* Update test
* Update test
* Update tests
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update tests
* Update test
* Update test
* Update test and make it pass
* Update test
* Update tests
* Update test
* Use entityPickerModal() instead of modal()
* Use entityPickerModal() instead of modal()
* Update test
* Update test
* Update test
* Revert change that broke tests
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Remove test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update tests
* Update test
* Update test
* Rename createDatabaseIdItemFilter to createShouldShowItem
* Update test
* Update and skip a test
* Remove test
* Update tests
* Update tests
* Update tests
* Remove test
* Update tests
* Update test
* Update test
* Update test
* Extract utils
* Use helpers
* Extract helpers
* Update test
* Update test
* Update test
* Update test
* Update test
* Filter out collections that do not have card or dataset
* Improve filtering
* Always show root and personal collections, see https://github.com/metabase/metabase/issues/42687
* Remove failing assertion
* Update tests
* Use entityPickerModalTab
* Accept strings in pickEntity
* Use entityPickerModalLevel
* Use entityPickerModalTab
* Use entityPickerModalTab
* Update tests
* Update test
* Update tests
* Update test
* Fix assertion
* Update test
* Update tests
* Update test
* Update tests
* Update tests
* Update test
* Update test
* Update tests
- ensure database list gets loaded like in a real app
* Update test
* Update test
* Integrate Recents tab in Data Picker (#42821)
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Fix metadata loading
* Update test
* Update test
* Update tests
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update tests
* Update test
* Update test
* Update test and make it pass
* Update test
* Update tests
* Update test
* Use entityPickerModal() instead of modal()
* Use entityPickerModal() instead of modal()
* Update test
* Update test
* Update test
* Revert change that broke tests
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Remove test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update tests
* Update test
* Update test
* Rename createDatabaseIdItemFilter to createShouldShowItem
* Update test
* Update and skip a test
* Remove test
* Update tests
* Update tests
* Update tests
* Remove test
* Update tests
* Update test
* Update test
* Update test
* Extract utils
* Use helpers
* Extract helpers
* Update test
* Update test
* Update test
* Update test
* Update test
* Filter out collections that do not have card or dataset
* Improve filtering
* Always show root and personal collections, see https://github.com/metabase/metabase/issues/42687
* Remove failing assertion
* Update tests
* Use entityPickerModalTab
* Accept strings in pickEntity
* Use entityPickerModalLevel
* Use entityPickerModalTab
* Use entityPickerModalTab
* Update tests
* Update test
* Update tests
* Update test
* Fix assertion
* Update test
* Update tests
* Update test
* Update tests
* Update tests
* Update test
* Update test
* Update tests
- ensure database list gets loaded like in a real app
* Update test
* Enable recents tab and update tests
* Update test
* Update tests
* Add Recents-related assertions
* Update test
* Clean up mocks
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Migration to add `trashed_from_*` (#41529)
We want to record where things were trashed *from* for two purposes:
- first, we want to be able to put things back if they're "untrashed".
- second, we want to be able to enforce permissions *as if* something is
still in its previous location. That is, if we trash a card or a
dashboard from Collection A, the permissions of Collection A should
continue to apply to the card or dashboard (e.g. in terms of who can
view it).
To achieve this, collections get a `trashed_from_location` (paralleling
their `location`) and dashboards/cards get a
`trashed_from_collection_id` (paralleling their `collection_id`).
* Create the trash collection on startup (#41535)
* Create the trash collection on startup
The trash collection (and its descendants) can't have its permissions
modified.
Note that right now, it's possible to move the Trash collection. I'll
fix this when I implement the API for moving things to the Trash.
* s/TRASH_COLLECTION_ID/trash-collection-id/g
* Add a comment to explain null comparison
* Move archived items to the trash (#41543)
This PR is probably too big. Hopefully documenting all the changes made here will make it easier to review and digest. So:
Tables with a `collection_id` had `ON DELETE SET NULL` on the foreign key. Since we only deleted collections in testing and development, this didn't really affect anything. But now that we are actually deleting collections in production, it's important that nothing accidentally get moved to the root collection, which is what `SET NULL` actually does.
When we get an API request to update the `archived` flag on a card, collection, or dashboard, we either move the item *to* the trash (if `archived` is `true`) or *from* the trash (if `archived` is `false`). We set the `trashed_from_collection_id` flag as appropriate, and use it when restoring an item if possible.
Because moving something to the trash by itself would have permissions implications (since permissions are based on the parent collection) we need to change the way permissions are enforced for trashed items.
First, we change the definition of `perms-objects-set-for-parent-collection` so that it returns the permission set for the collection the object was *trashed from*, if it is archived.
Second, when listing objects inside the Trash itself, we need to check permissions to make sure the user can actually read the object - usually, of course, if you can read a collection you can read the contents, but this is not true for the trash so we need to check each one. In this case we're actually a little extra restrictive and only return objects the user can *write*. The reasoning here is that you only really want to browse the Trash to see things you could "act on" - unarchive or permanently delete. So there's no reason to show you things you only have read permissions on.
Because previously the Collections API expected archived collections to live anywhere, not just in a single tree based in the Trash, I needed to make some changes to some API endpoints.
This endpoint still takes an `exclude-archived` parameter, which defaults to `false`. When it's `false`, we return the entire tree including the Trash collection and archived children. When it's `true`, we exclude the Trash collection (and its subtree) from the results.
Previously, this endpoint took an `archived` parameter, which defaulted to `false`. Thus it would only return non-archived results. This is a problem, because we want the frontend to be able to ask for the contents of the Trash or an archived subcollection without explicitly asking for archived results. We just want to treat these like normal collections.
The change made to support this was to just default `archived` to the `archived` status of the collection itself. If you're asking for the items in an archived collection, you'll only get archived results. If you're asking for the items in a non-archived collection, you'll only get unarchived results.
This is, for normal collections, the same thing as just returning all results. But see the section on namespaced collections for details on why we needed this slightly awkward default.
No change - this endpoint still takes an `archived` parameter. When `archived=true`, we return the Trash collection, as it is in the root collection. Otherwise, we don't.
* Make Trash Usable - UI (#41666)
* Remove Archive Page + Add `/trash` routing (#42226)
* removes archive page and related resources, adds new /trash route for trash collection, adds redirects to ensure consistent /trash routing instead of collection path
* fixes unit + e2e tests, corrects links generated for trash collection to use /trash over /collect/:trashId route
* updates comment
* Serialize trash correctly (#42345)
Also, create the Trash in a migration rather than on startup. Don't set a specific trash collection.id, instead just select based on the `type` when necessary.
* Fix collection data for trashed items (#42284)
* Fix collection IDs for trashed items
When something is in the trash, we need to check permissions on the
`trashed_from_collection_id` rather than the `collection_id`. We were
doing this. However, we want the actual collection data on the search
result to represent the actual collection it's in (the trash). I added
the code to do this, a test to make sure it works as intended, and a
comment to explain what we're doing here and why.
* Refactor permission for trashed_from_collection_id
Noah pointed out that the logic of "what collection do I check for
permissions" was getting sprinkled into numerous places, and it felt a
little scary. In fact, there was a bug in the previous implementation.
If you selected a collection with `(t2/select [:model/Collection ...])`,
selecting a subset of columns, and *didn't* include the
`trashed_from_collection_id` in that set of columns, then called
`mi/can-write?` on the result, you'd get erroneous results.
Specifically, we'd return `nil` (representing the root collection).
I think this is a reasonable fix, though I'm pretty new to using fancy
multimethods with `derive` and such. But basically, I wanted a way to
annotate a model to say "check these permissions using
`:trashed_from_collection_id` if the item is archived." And in this
case, we'll throw an exception if `:trashed_from_collection_id` is not
present, just like we currently throw an exception if `:collection_id`
is not present when checking permissions the normal way.
* Move existing archived items to the trash (#42241)
Move everything that's archived directly to the trash. It's not ideal
because we're wiping out the existing collection structure, but the
existing Archive page also has no collection structure.
* lint fixes from rebase
* Fix backend tests (#42506)
`can_write` on collection items was wrong because we didn't have a
`trashed_from_collection_id` - the refactor to ensure we didn't make
that mistake worked! 🎉
* Add an /api/collections/trash endpoint (#42476)
This fetches the Trash in exactly the same way as if we'd fetched it
with `/api/collection/:id` with the Trash ID. I hadn't realized that the
frontend was doing this with the previously hardcoded Trash ID.
* Make Trash Usable - Dynamic Trash Collection Id (#42532)
* wip
* fixes hardcoded reference to trash id in sidebar
* remove TRAHS_COLLECTION
* fixes line and e2e tests
* fix invert logic mistake and fixes lint
* Make Trash Usable - Search Filter UI (#42402)
* adds filtering for archived items on the search page
* fix typing mistake
* Make Trash Usable - Bug Bash 1 (#42541)
* disables reordering columns in archived questions, disables modifying archived question name in question header, collection picker no longer defaults to archived item, keeps trashed collection from appearing collection picker search results, shops showing empty area in trashed dashboard info sidebar, disables uploading csvs to trashed collections
* impls pr feedback
* fix e2e failure
* Localize the name of the Trash (#42514)
There are at least two spots where we can't just rely on the
after-select hook, and select the collection name directly from the
database: the Search and Collection API.
In these cases we need to call `collection/maybe-localize-trash-name`,
which will localize the name if the passed Collection is the Trash
collection.
* Update migration IDs
Migration IDs need to be in order.
* Fix failing mariadb:latest test (#42608)
Hooooly cow. Glad to finally track this down. The issue was that
booleans come back from MariaDB as NON BOOLEANS, and clojure says 0 is
truthy, and together that makes for FUN TIMES.
We need to turn MariaDB's bits to booleans before we can work with them.
* Make Trash Usable: Add `can_restore` to API endpoints (#42654)
Rather than having two separate implementations of the `can_restore`
hydration for cards and dashboards, I set up automagic hydration for the
`trashed_from_collection_id`. Then the hydration method for
`can_restore` can just first hydrate `trashed_from_collection` and then
operate based on that.
* fix can_restore for collections trashed from root
* Fix `trashed_from_location` for trashed subcols
We were setting `trashed_from_location` on subcollections to the
`trashed_from_location` of the ancestor that was trashed - which is
wrong.
This would have caused bugs where collections would be restored to the
root collection, regardless of where they were originally trashed from.
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* Fix Trash rollbacks (#42710)
* Fix Trash rollbacks
There were a few errors in my initial implementation.
First, we can't simply assume that `trashed_from_location` and
`trashed_from_collection_id` are set for anything in the archive. They
should be set for things *directly* trashed, but not for things trashed
as part of a collection.
Therefore, we need to set `location` and `collection_id` to something
like "if the item is in the trash, then the `trashed_from` - otherwise,
don't modify it".
Second, because `trashed_from_collection_id` is not a foreign key but
`collection_id` is, we have a potential problem. If someone upgrades,
Trashes a dashboard, then Trashes and permanently deletes the collection
that dashboard _was_ in, then downgrades, how do we move the dashboard
"back"? What do we set the dashboard's `collection_id` to?
The solution here is that when we downgrade, we don't actually move
dashboards, collections, or cards out of the Trash collection. Instead
we just make Trash a normal collection and leave everything in it.
* Make Trash Usable - Bug Bash 2 (#42787)
* wip
* fixes access to property on null value
* pr clean up
* more clean up
* Fix up tests
- permissions will check the archived from location. So recents need to
select :report_card.trashed_from_collection_id and
:dash.trashed_from_collection_id to satisfy mi/can-read?
- some commented out code with `(def wtf (mt/user-http-request
...))`. Restore tests.
- probably commented out because the recent views come back in an order
but we don't care about the order. And it was asserting against an
ordered collection. Just go from [{:id <id> :model <model>} ...] to a
map of {<id> <model>} and then our comparison works fine and is not
sensitive to order.
* put try/finally around read-only-mode
ensure the setting read-only-mode! to false happens in a finally
block. Also, set read-only-mode to false in
```clojure
(deftest read-only-login-test
(try
(cloud-migration/read-only-mode! true)
(mt/client :post 200 "session" (mt/user->credentials :rasta))
(finally
(cloud-migration/read-only-mode! false))))
```
But worryingly, I think we have a lurking problem that I'm not sure why
we haven't hit yet. We run tests in parallel and then put all of the
non-parallel tests on the same main thread. And when one of these puts
the app in read-only-mode, the parallel tests will fail. HOPEFULLY since
they are parallel they won't be hitting the app-db necessarily, but
there could be lots of silly things that break.
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: dan sutton <dan@dpsutton.com>
* move series data label formatters to the model
- TODO: move stacked data label formatters to the model
* combo stacked chart
* spec
* remove irrelevant spec
* combo stacking
* fix selecting y-axis on stacked charts
* snapshot
* linter, specs, types
* propagate combo stacking to area and bar charts
* enable combo stacking for bar/area charts, fix data transform
* fix types, specs
* fix y-axis extents calculations
* remove spec that tested the removed control, update specs, update test data
* propagate combo stacking to area and bar charts (#42600)
* propagate combo stacking to area and bar charts
* enable combo stacking for bar/area charts, fix data transform
* fix types, specs
* fix y-axis extents calculations
* remove spec that tested the removed control, update specs, update test data
* review
* remove unused formatting options parameter
* move stacked labels formatter to model
* fix stacked combo chart label display for non-stacked lines
* combo stacked chart
* spec
* remove irrelevant spec
* combo stacking
* fix selecting y-axis on stacked charts
* snapshot
* linter, specs, types
* propagate combo stacking to area and bar charts (#42600)
* propagate combo stacking to area and bar charts
* enable combo stacking for bar/area charts, fix data transform
* fix types, specs
* fix y-axis extents calculations
* remove spec that tested the removed control, update specs, update test data
* review
* fix waterfall chart labels
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
* handle recent view for non-existant entity
* test that recent views works with missing entities
- just doesn't return them.
* dont count on there being entities in the app-db
* Query for the entity data upfront for recent-views
- Still need to check for the model_object due to popular-items calls
* cleanup, fix ellide logic
memoize get-parent-coll
* more cleanup
* rewire half of the queries
* rewrite the other queries, and improve docs
* align some queries
* lookup all entities ahead of time
- always return effective parent collections
* seperate n+1 tests
* use = instead of a set
* no need for a vector
* fixup simple table test
* go from effective-location-path to id instead of hydrating
* feat: cloud migration endpoints
* fix: don't leave open conn behind when checking migrations
`unrun-migrations` would open a new connection but not close it.
Since `with-liquibase` is happy enough using a data-source the fix is straightforward.
You can verify this by running the following code:
```
(comment
(require '[metabase.cmd.dump-to-h2 :as dump-to-h2]
'[metabase.analytics.prometheus :as prometheus])
(defn dump []
(-> (str "cloud_migration_dump_" (random-uuid) ".mv.db")
io/file
.getAbsolutePath
dump-to-h2/dump-to-h2!))
(defn busy-conns []
(-> (prometheus/connection-pool-info)
first
second
:numBusyConnections))
;; each dump leaves behind 1 busy conn
(busy-conns)
;; => 0
(dump)
(busy-conns)
;; => 1
(dump)
(busy-conns)
;; => 2
(dump)
(busy-conns)
;; => 3
)
```
* fix: flush h2 before returning from dump
* rfct: move code to models.cloud-migration
* test: add login while on read-only test
* fix: assorted cloud_migration fixes from review
* test: allow overriding uploaded dump
* fix: add UserParameterValue to read-only exceptions
Also make the list a little bit nicer.
* fix: only block dumped tables on read-only
* fix: recover on startup if read-only was left on
* feat: block migration when hosted already
* test: test settings for migration
* feat: cloud migration supports retries and multipart
* test: sane dev defaults for migration
* fix: upload 100% shouldn't be migration 100%
* chore: make up a new migration id after merge
* Cloud Migration FE (#42542)
* it's a start
* ui wip
* wip
* dynamic polling intervals, and custom modal for migrate confirmation modal
* cleans out most of the remainig UI TODOs
* adding progress component
* impls team feedback
* makes component more testable, starts some a unit test for the CloudPanel
* finish unit testing
* reverts api changes
* update progress styling
* fix type issues
* fix e2e failure, fix feature unit tests by holding last migration state in fetchMock if more requests than expected happen at the end of a test, remove white spacing change in clj file
* second pass at fixing tests
* fix copy from ready-only to read-only
* copy fix
* Update frontend/src/metabase/admin/settings/components/CloudPanel/MigrationError.tsx
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* Update frontend/src/metabase/admin/settings/components/CloudPanel/MigrationInProgress.tsx
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* adding e2e test
* pr feedback
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
---------
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* combo stacked chart
* spec
* remove irrelevant spec
* combo stacking
* fix selecting y-axis on stacked charts
* snapshot
* linter, specs, types
* propagate combo stacking to area and bar charts (#42600)
* propagate combo stacking to area and bar charts
* enable combo stacking for bar/area charts, fix data transform
* fix types, specs
* fix y-axis extents calculations
* remove spec that tested the removed control, update specs, update test data
* review
* show show data values setting on normalized charts that have a line series
* fixes
* [MBQL lib] Add `canPreview` and use it from the notebook
* Fix previewQuery
* Apply can-preview across stages; add unit tests
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Correct Data Shape for Multi-Series Static Viz
Fixes: #41787
During the echarts migration, some backend tests needed fixing. In this process, the shape of data (eg. list of cards
and their data results from execution) were altered.
This was done to fix failing tests and renders for single series cards (all Questions and many dashcards), but didn't
correctly handle a multi-series card.
This conditionally adds back the expected shape.
Perhaps in a follow on PR it's worth inspecting the implementation of the :javascript_visualization render method to
clean it up. Probably worth putting some Malli schemas around it too to make it clearer what shape we need.
* Make sure card/data maps going into static-viz are distinct
* Add a test util to render dashcards, add multiseries card test
We haven't made a 'unit' (perhaps more like a small integration test at this point) test for multi series dashcards
yet.
In most other cases, making sure a card (Question) renders properly is sufficient, but the multi series dashcards are
a special case. It's probably useufl to have a test or two, and the dashcards test util should hopefully help future
test writing a bit now too.
* WIP some workflows
* Add reminder todo comment
* Complete the workflow
* Fix YAML syntax error
* Trigger the workflow
* Fix resuable workflow
* Fix uberjar workflow
* Add missing steps before testing
* Pass recrets to resuable workflow, so it could run
* Fix sdk artifact location
* Remove unneeded output
* fetch tags, so it doesn't fail on push
* Only running EE uberjar in SDK workflow
Because OSS is never needed
* Fix incorrect syntax
* Shuffle workflow so it failed fast
* Fix syntax error
* Attempt to bulid only EE jar
* Fix the error handling step
* Fix failing to upload to s3
* Fix create a PR step
* Fix the last temporary command, so the build looks green
* Remove hack to trigger the workflow
* Remove completed todo comment
* Clean up S3 step
* Add todo comment
* Upload correct docker image name
* Bump the package version before building SDK
* Fix uploading the correct Metabase dev container name
* Fix one too many double quote
* Push the correct docker image
* Fix PR creation step
* Dry run publishing SDK
* Use a separate uberjar workflow for SDK
* Remove a bunch of unused steps
* Update variable names, step names
* Improve tag error message
* Correct m2-cache-key to be unique
* Fix the push tag failure step to only run when failed to push tag
* Fix changed artifact name
* Improve env name for clarity
* Fix extra trailing empty line
* Remove conditions from running certian steps
* Removing npm publish dry-run
* Add a simple replace-names test
* Support find-and-replace in cards with variables and field filters
[Fixes#42580]
[Fixes#42581]
* Add models.card/replace-fields-and-tables!
* Docstring fix
* Little cleanup
3 TODOs were removed here:
- "a way to SET Database-local values." More like TODONE :haaa:
- "we should probably also bind this [that is, *database-local-values*]
in sync contexts e.g. functions in [[metabase.sync]]" I think we don't
need to do this. The full set of maybe-database-local settings right now
is `aggregated-query-row-limit`, `unaggregated-query-row-limit`,
`database-enable-actions`, and `persist-models-enabled`. None of these
need to be set when `sync`ing. If at some point we do add a db-local
setting that should affect the way syncing works, we will need to do
this - but I don't think this TODO will help figure that out.
- "settings sent via an env var aren't returned for security purposes.
It is an open question whether we should obfuscate them and still show
the last two characters like we do for sensitive values that are set via
the UI." I think if it's been this way for 5 years, it no longer
qualifies as an "open question." We can reopen if down the road if this
is something we want.
* PoC Pivot table exporting.
Works so far!
This just hacks a couple things in. I added the :query via format_rows, which should instead have its own proper
middleware
Pivot export should probably not just be in the middleware namespace but in qp.pivot.
Should not break the streaming results writer for non-pivots... obviously this breaks ordinary results, so that needs
to be handled properly.
Better documentation about how the logic actually works. Pivot tables are hard to remember/easy to forget, so some
very clear diagrams/examples would probably be good.
* adjust csv export to export a pivot table when pivot options exist
* post processing of pivot table rows for card/dashcard downloads
* WIP on getting pivot exports to work with unsaved questions
* WIP getting pivot exports working on unsaved questions
The dataset POST /pivot endpoint is used to get the pivot data to the frontend, but also to get the pivot query.
The query that is returned by the POST comes from the json_query, which is populated by the userland-query stuff with
the last run query. The pivot qp runs mulitple queries and each query runs through the regular userland query qp, so
we'll 1. see lots of executions recorded and 2. end up with the wrong query in the POST return. It'll look mostly
correct, but due to how the pivot qp works won't necessarily have all of the fields.
That incorrect pivot query is passed to the /:export-format endpoint and we don't end up seeing correct results.
Since that json_query gets fed back to the backend when doing an export, we do need it to be correct, like the query
that was sent to /pivot in the first place.
Since this is a WIP, I've just passed the query as a string to :original-query in :info, and if that exists, use that
to fill the :json_query key.
Then, also add a :was-pivot flag so that the dataset download endpoint can look at it and use the pivot qp instead of
the regular qp. Even if we have the correct 'top level' query, we need to run it through the pivot qp to get the full,
correct pivoted results.
* Pivot Export Test
Here I've added a pivot export test that has various assertions trying to capture the correct output from the pivot
csv export.
I've done my best to explain the tests with comments, but have also left a (comment ) form with a function
`explore-pivot` to help *show* rather than tell what the expected pivot output should be.
I've created this new namespace to first capture this pivot export test with the intent to add tests that can improve
our confidence that ALL exports/downloads in ALL places behave as we expect in the future. It just seemed nice to
start fresh here with the pivot export test first.
* Don't need to make this public.
* Add lots of comments to hopefully make this a bit easier to follow
* Added an explore-raw-pivot fn in comment form
* Update test/metabase/api/downloads_exports_test.clj
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* begin addressing feedback
- finish an unfinished docstring
- merge info instead of assoc
- use a native query to build the pivot data instead of using a custom dataset, to keep test speed ok
* Create a new ns for the pivot post processing utils
Separate the post processing fns into their own namespace. This means I can easily add a test namespace for the post
processing specifically as well.
* Add pivot postprocessing test ns, not complete, but decent start
* fix mistake in endpoint. Only add original query to first pivot q
* Figured out how to keep the original query without storing as a string
* Perhaps this'll fix things properly?
---------
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* support theming in table
* theme injection with hoc
* reasonable default theme options
* customizable text color
* theming table header
* table theming
* change theme configuration order
* apply sorted cell color directly
* gutter background
* remove header theming for now
* fix: id column styling
* id column borders
* border color
* add cell hover color
* conditional hover color
* use brand color to accent
* add default theme values to unit test
* cell background should default to white
* update unit test
* remove unused properties
* remove duplicated color declaration as inner cell data styles it
* remove duplicate fallback
* remove unused use-embedding-theme hook
* use theme color from Mantine for css variables
* update defaults
* inject default component theme
* add notes on cell data theme
* update unit tests
* update unit tests
* remove no-op text-brand hover color
* remove no-op is sorted cell color
* Explicitly decide whether to upload or not
* Upload in the main e2e workflow, but skip uploading in stress-test workflow
* Do not record runs in the stress-test workflow
* Add Replay API key
Even though we're not using it in this workflow. 🤷
* Omit env var from the stress-test workflow
this lowers memory usage 100x for 50kb strings (our limit before this change) and speeds up parsing by an order of magnitude.
For example, results for parsing 42kb string (already in memory):
Before: Time per call: 6.32 ms Alloc per call: 8,732,850b
After: Time per call: 55.16 us Alloc per call: 83,254b
* Render item icon inline with column icon
* Use inline info icon in field picker
* Use new QueryColumnInfoIcon on filter columns picker
* Use new QueryColumnInfoIcon on suggestions dropdown
* Use new QueryColumnInfoIcon aggregations dropdown
* Use new QueryColumnInfoIcon filter modal
* Use new QueryColumnInfoIcon in BreakoutColumnListItem
* Use new QueryColumnInfoIcon in FieldList
* Use new QueryColumnInfoIcon in DataSelectorFieldPicker
* Update TableInfoIcon to render fallback too
* Use new TableInfoIcon in DataSelector
* Use new TableInfoIcon in ViewHeader
* Fix types in breakout column
* Move aria-label to wrapper
* Move binning back to the right
* Always give info icon full opacity
* Remove asDot props everywhere
Follow up to #42424.
This enforces that all drills do not appear on native stages.
Also fixes one more drill on top of the two in #42424 that were wrongly
showing up for native queries.
- `:metadata/key-constraints` for those databases which formally track
PK and FK key constraints. Some databases (eg. Presto/Athena/Starburst
family) don't support constraints like that, but do support querying
based on foreign keys (which is the `:foreign-keys` feature).
- `:connection/multiple-databases` for databases where a single
connection can connect to many databases. This is the case for Athena,
since it connects to an S3 bucket and exposes all files in the bucket
as databases through that single connection.
- Also removed a troublesome "all DBs except a huge list" case in the
SSH tunnel tests with a shorter list of supported drivers.
* Fix broken metric revision API call
Fixes#42633
* Add E2E repro for #42633
* Fix metrics revision history in the data reference
* Add repro for #42633 for the reference page
* Consolidate tests
* Do not use hard coded id
* Fix lint errors
* handle recent view for non-existant entity
* test that recent views works with missing entities
- just doesn't return them.
* dont count on there being entities in the app-db
* add recents tab
type updates
fix unit test
change where we filter recents
WIP Recents Tab
wip recents
design updates
update recents design to match search
use new recents format
initial recents implementation
fix types
* fix e2e tests
* update more e2e tests
* allow recents filters
* update test mocks
* add unit tests for recents tab
* update e2e tests
* add e2e tests for recents picker tab
* fix lost access test
* Expand the single or multi-selectable string filter choices
* Mark operators as `multi`
* Wrap an existing test in a separate `describe` block
* Test multiple values settings
* Expand repro to include multiple dashboard filter values
* Ease the load by limiting the card results
* Switch to the more flexible data structure for the dashboard filters
* Expand the assertions
* Fix typo
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* wip
* adds more collection-touch events
* remove sneaky dep that slipped in
* cleaning up, and adding tests
* respond to review comments + mini bug reports
* make it official
* double official
* cleanup + return recent-views/Item for `GET activity/popular_items`
* get popular_items returning the proper shape
* compare doesn't what I thought
* add display_name to tables
* table.database.name should be the db name, not the table name
* add table.database.initial_sync_status
* improve call to toucan2
* light renaming + many text fixes
* include moderated_status in cards and datasets
* look up Dashboard permissions for Dashboards
* stop double-querying for `can-write?` checks
- also include some debug logging
* fix card <-> model dispatching
also gets tests passing locally
* let the test runner run the tests
* fix gc test
* adds description
* description can be null
* better tests
no more nils passed through the app
* debugging test
* fix tests and skip archived
* mysql finesse
* ellide inactive tables
* wip
* adds more collection-touch events
* remove sneaky dep that slipped in
* cleaning up, and adding tests
* respond to review comments + mini bug reports
* make it official
* double official
* cleanup + return recent-views/Item for `GET activity/popular_items`
* get popular_items returning the proper shape
* compare doesn't what I thought
* add display_name to tables
* table.database.name should be the db name, not the table name
* add table.database.initial_sync_status
* improve call to toucan2
* light renaming + many text fixes
* include moderated_status in cards and datasets
* look up Dashboard permissions for Dashboards
* stop double-querying for `can-write?` checks
- also include some debug logging
* fix card <-> model dispatching
also gets tests passing locally
* let the test runner run the tests
* fix gc test
* adds description
* description can be null
* better tests
no more nils passed through the app
* debugging test
* fix tests and skip archived
* mysql finesse
* New Recents + Popular API Format - Frontend (#42305)
* fix tests
* new API payload format for recents and populars
* update url generation
* update recents components
* update tests
* only show 5 recents in command palette
* remove unused types
* update tests
* obey the linter
* type updates
* fix unit test
* change where we filter recents
---------
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* fix modelToUrl types
* respond to review comments
* post merge test fix
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Copy question event to new version
* Add new events for adding column via plus modal
* Add plus modal variants
* Track combine column via plus header
* Track extract column via plus header
* Add test for combine question event
* Add e2e test for extract column shortcut snowplow events
* Add index file to CombineColumns
* Add CombinColumnAction
* Add CombinColumnsAction to default and embedding modes
* Fix outdated --color-brand references
* Set up sequenced close handler in combine columns
* Allow outside clicks to happen when popover is not open
* Add e2e test for combine column shortcut
* Remove unused prop
* Rename file to match component name: CombineColumnsAction
* Simplify test assertion
* Set width on combine columns action
* Remove superflouous ? check
* Add repro for 32323
* Fix offset not working in case
* Make offset function return any
* Add a repro for #42377
* Fix order of adjustments
* Revert unrelated changes
* Remove commented code
* Revert unrelated changes
* Refactor test
* Type offset aggregation
* Add a test for no order by or breakout clause
* Add a basic test for breakout clause
* Fix assertion
* Remove problematic dependency
* Fix uuids generation
* Remove redundant limit
* Add a test for multiple breakouts
* Update test names
* Extract OFFSET_SUM_TOTAL_AGGREGATION
* Add a repro for metabase#42509
* Add a test for multiple aggregations and breakouts
* Remove unused intercept
* Move uuid utils to separate file
- it wasn't possible to import the utils file in e2e tests without it
* Make isUuid a type guard
* Add tests for sorting
* Tag the test
* Add extra assertion to verify expression parsing
* Add a complex scenario
* Reverse isFirst logic
* migrate smart scalar to css modules
* lighten theme color
* adjust margin
* map space 1 to xs
* use c prop instead of color
* use rem unit
* use rem unit
* add y margin
* return null when no children
* comment on text variation
* cleanup unused components and add dark mode todo
* remove todo comment
* support night mode in embedding
* lift dashboard theme to redux
* make theme non-nullable with default
* use component prop instead of title tag
* optional display theme
* restore tooltip state
* add default mock state
* remove theme from embed
* replace handleActions with createReducer
* Render plus button in interactive table header
* Add columnShortcuts on ClickObject
* Dispatch click action for columnShortcuts on plus button
* Make onCancel an optional prop
* Add ExtractColumn action
* Add ExtractColumn action to Default and Embedding mode
* Use arrow_split icon for extract column action
* Add title to extract column popover
* Add e2e test for Extract column shortcut
* Use asReturned in extract column
* Rename to Extract part of column
* Add a custom title to the new column shortcut
* Add onClose to ExtractColumn popover
* Hide shortcuts icon when question is not editable
* Fix null error
* Do not use submodules in import
* Scroll more in test
* Format Rows toggle also applies to Column Names
Fixes: #42500
When format-rows is false, we want to also pass that true/false value to the title column formatting.
That is, if a column is renamed and formatting is applied, we want to use that column's name. But, if a column is
renamed and the user exports unformatted csv or json, the custom column name should NOT be used, as was the behaviour
in prior versions.
* write a better test explanation
Ensures that models are sorted as follows in Browse models:
* Models are sorted by collection, by default
* When models are sorted by collection, they are secondarily sorted by name (so that models with the same collection path appear in alphabetic order)
* use echarts tree shaking api
* fix specs
* update types
* move echarts module registration back to individual components
* Transpile echarts in jest
---------
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
These params were failing to match. This matches them up and puts the
options in the correct place. (`[:contains x y opts]` but `[:contains opts x y z]`).
* how optional clauses work
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Fix offset not working in case
* Make offset function return any
* Add a repro for #42377
* Fix order of adjustments
* Revert unrelated changes
* Remove commented code
* Revert unrelated changes
* add upload management API
* add upload management interface
* remove trailing slash
* add e2e upload management test
* add upload management to token_features
* jsdoc cleanup
* fixes tests
- include :upload_management in session_test
- fix list-url in upload management
- ensure local tests have upload management disabled
(causes tests locally to 422 instead of 402: it has sufficient
permissions from a local token to try to delete the table but the
table is not an upload table, so you get a 422)
* add card archive param
* update tests
* use common component
* add success/error toasts
* test success/error notifications
* update e2e tests
* address review comments
* make sure we have fresh data
---------
Co-authored-by: dan sutton <dan@dpsutton.com>
* Fix flaky question management test
* Update assertions order to reduce probability of a flake
* Use explicit assertions
* Change assertion order
* Decrease flakiness
* Remove unused functions
* Remove `export` from a local function
* Remove googleanalytics related engine code
* Use `getEngines` selector instead of the deprecated Metabase `Settings`
* Use `getDefaultEngineKey`
* Use `formatSQL` function locally
* Remove unused `getEngineLogo` function
* Convert to TS
* Remove unused export
* Fix types
* Expand test coverage
* Update type
* Remove `googleanalitics` from an assertion
* Address review comment
* Expand types
* Add tests for `isDeprecatedEngine`
* Expand tests
* wip dashboard picker
* use in homepage selector
* remove dead code
* remove generateKey prop
* add unit tests
* cleanup
* WIP unit tests
* update personal/public interactions and tests
* update e2e tests
* allow disabling items in dashboardPicker
* add new dashboard tag invalidation
* use rtk in dashboardpicker
* use collection_id from api
* horrendous way of disabling read only items
* remove badness
* disallow selecting read-only dashboards in dashboard picker search
* pass search filter as prop
* update types and tests
* update types and tests
* address review comments
* convert `SocketChannelEndpoint` to `SocketChannel`
* Remove fatal logging (used for debugging)
* check for nil channel + log on unknown type
* only log unexpected transport types once
* prop to reserve height and use full height by default
* require height to be passed
* add default height based on viz type
* default to 250px
* reserve table height
* remove memoization
* rely on the card size defaults constant
* multiply by a fixed constant
* correct vis to viz
* Dashboard Param Values Common Impl for Embedding/Preview Embedding
Refactor the embedding api namespace to allow a bit of reuse between the embedding api and the embedding preview
api. In some cases, the implementations are not identical (in terms of the shape of data expected/endpoints
available), so this is a first step to making them the same.
Related:
https://www.notion.so/metabase/Make-embedding-preview-behave-consistently-with-actual-static-embeds-855353d8e5e8411d8164c7ac563c7d2f
* Add a test to verify that preview-embed param-values endpoint works
* Use preview embed endpoint and fix relevant tests
* Add tests
* Change name to reflect what actually happens in the function
---------
Co-authored-by: Mahatthana Nomsawadi <mahatthana.n@gmail.com>
now:
```clojure
❯ clj -M:"$ALIASES"
Warning: environ value /Users/dan/.sdkman/candidates/java/current for key :java-home has been overwritten with /Users/dan/.sdkman/candidates/java/21.0.2-tem
2024-05-07 19:55:25,167 INFO metabase.util :: Maximum memory available to JVM: 8.0 GB
```
no logs about recur targets unboxing
* Only nest expressions referenced in breakouts or aggregations
* Support Offset() as expression with no breakouts
* Test fixes 🔧
* Oracle test update
* Improved Oracle test
* Test update 🔧
* Fix busted test
* Fix running and saving Offset over the REST API
* Remove stray :wow
* More test fixes 🔧
* Add suffix to name if column already exists
* Add test for duplicate combine columns
* Use direct question in test
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix formatting
* Use createQuestion for other tests too
* Fix table references
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* initial attempt
* search button
* small adjustments
* adding back the search bar
* adding an extra check in e2e
* i18n adjustment
* impls pr feedback
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
before:
```clojure
❯ clj -M:"$ALIASES"
Warning: environ value /Users/dan/.sdkman/candidates/java/current for key :java-home has been overwritten with /Users/dan/.sdkman/candidates/java/21.0.2-tem
WARNING: abs already refers to: #'clojure.core/abs in namespace: clojure.algo.generic.math-functions, being replaced by: #'clojure.algo.generic.math-functions/abs
2024-05-07 04:39:37,890 INFO metabase.util :: Maximum memory available to JVM: 8.0 GB
```
after:
```clojure
❯ clj -M:"$ALIASES"
Warning: environ value /Users/dan/.sdkman/candidates/java/current for key :java-home has been overwritten with /Users/dan/.sdkman/candidates/java/21.0.2-tem
2024-05-07 04:40:42,561 INFO metabase.util :: Maximum memory available to JVM: 8.0 GB
```
Now no warning about `abs`
* add percent change to timeseries tooltip
* move computeChange to lib/numeric
* cleaning up code in numeric.js
* color text
* add unit to key
* move change to tooltip footer
* fix extra bottom spacing
* remove color
* handle gaps in data
* clean up code
* add change to funnel tooltip
* move retained to footer in funnel
* update copy
* remove min decimal count
* rename events.tsx -> events.ts
* add test cases to line-bar-tooltips.cy.spec.js
* add specs for edge cases
* readd exports to fix numeric.unit.spec.js
* don't show for change for scatterplot
* translate retained
* use parseTimestamp instead of String
* handle quarter interval
* remove redundant dayjs calls
* handle quarter with Feburary correctly
* api details
* Update docs/installation-and-operation/serialization.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* public metabase theme type
* add tests for transforming embedding theme
* export public types
* load transformed theme override into theme provider
* migrate font family inside theme configuration
* add assertion for font family
* update README to add font family in theme options
* add font enum, fix tuple type and cleanup tests
* color tuple argument name
* use fill for colortuple
* use css modules instead of inline styles for cursor: default (#41644)
* add toast notification after the feedback (#41575)
* fix modal submit sending data while button says skip (#41784)
* fix: it should not send the feedback if button says skip
* add comment explaining why we check that getLastFeedbackCall() is undefined
* embedding homepage analytics (#41725)
* rename defaultTab -> initialTab
* add events schema for embedding-homepage
* fix stories
* add analytics, e2e tests and fix typo in dismiss reason
* copy embed_flow schema for better diff in the next commit
* adds isExampleDashboard to trackStaticEmbedPublished
* utm tags
* fix bug of is_example_dashboard
* better test names
* refactor embed homepage status type
* "() => {}" => "_.noop"
* add "Setup embedding" to the admin setup checklist (#41638)
* add setup embedding to the admin setup checklist
* restore whitespace as it was before
* i miss prettier (whitespace again)
* Clean up check list for embedding
Largely making a new entry for the embedding info just to fight long
lines. `:embedding-homepage-dismissed-as-done` just made the maps really
long. If we have a `:done?` under an `:embedding` keyword it reads much
more naturally.
Also, The previous diff was calling `boolean` on a var rather than
invoking the var:
```clojure
:embedding-app-origin (boolean embed.settings/embedding-app-origin)
;; ^^^^^^^^^^^^^^ needs to be (invoked)
```
---------
Co-authored-by: dan sutton <dan@dpsutton.com>
* Change utm tags for embedding homepage links (#42042)
* update utm tags
* utm_media -> utm_content
* add description to the checklist setup step (#42200)
---------
Co-authored-by: dan sutton <dan@dpsutton.com>
* Update readme adding more ways to run Metabase
* Remove running from `master` because it could break
* Update enterprise/frontend/src/embedding-sdk/README.md
Co-authored-by: Alberto Perdomo <albertoperdomo@users.noreply.github.com>
* Update disclaimer, makes it more obvious
---------
Co-authored-by: Alberto Perdomo <albertoperdomo@users.noreply.github.com>
* Rename "Line style" to "Line shape"
* Add "line.style" viz setting
* Add "line.size" viz setting
* Apply new viz settings in line/area charts
* Clean up `ChartSettings` test suite
* Fix default values not applied
* Add visual test
* Add E2E test
* Rework default values
* Use constant S/M/L values for "line.size" setting
Lets the `ItemsTable` component have customizable columns.
The logic for this is contained in `BaseItemsTable`. This is preliminary work for #41614
This doesn't introduce any new UI, but there is a unit test for the new logic.
The main thing to check is whether the table on the collections page stills work correctly. Note how the columns adjust as you resize the viewport horizontally. (In fact, the columns will sometimes adjust as you close and open the sidebar, because the width they are paying attention to is the width of the area to the right of the sidebar.)
Closes#41908
The API/model schemas for dashboard parameters are general enough
that they just work for this new type, which doesn't look very different
from a value parameter right now.
Part of #42118.
* Add test for email extractions
* Fix swapped domain and host examples
* Add tests for URL columns
* Fix examples for shortcut too
* Use arrow function instead of function
* Simplified MetadataProvider protocol
* Test fix
* Ok make CachedMetadataProvider a separate schema again after all.
* Consolidate the Metadata schemas
* Clean Kondo warnings
* Malli-ify the stuff in metabase.lib.metadata.protocols
* Fix Kondo warnings
* Test fix 🔧
* PR feedback
* Window functions should use all breakouts except first for partitioning
* Update the offset() test
* Update tests
* Rework how we use driver escape-alias in MLv2; fix Oracle
* PR feedback
* Test fixes 🔧
This is a new parameter `:type` (and `:widget-type`) to choose a time
granularity (a `:temporal-unit`, eg. `month`, `day`, `day-of-week`)
which should replace the unit on the target field for the parameter.
The target field should be a breakout with a temporal type (date,
datetime) that's compatible with the input unit. The value of the
parameter should be a string or keyword naming one of the units:
`"month"`, `:month`.
Part of #42118.
* Add trackColumnExtractViaHeader helper
* Add Lib.extract wrapper
* Track column extractions via header
* Fix analytics event name
* Add tests for extract action analytics
* Set up extractions metadata helper
* Use functionsUseByExtraction in analytics
* Add custom type for ExtractionTag
* Use ExtractionTag in analytics
* [MBQL lib] Add `Lib.columnExtractDrillExtractions`
This gets the extractions from a column-extract drill, which is needed
in the UI.
* Add extractionExpression wrapper
* Add extractionsForDrill helper
* Fix type signature of extractionExpression
* Use expressionParts for getting expression function
* Walk expression to get functions used
* Simplify ColumnExtractionTag
* Fix expected expression
* Be more explicit with what the guard does
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* use palette for rendering
* add unit tests for chart label colors
* cleanup test code
* update unit test name
* cleanup color handler
* use theme.fn.themeColor to get color
* Adding Descriptions to search results in palette
* Palette Results List component
* adjusting e2e and unit tests
* type fix
* removing duplicate theme font
* fix typo, palette types adjustment
* ensures metabase theme override is merged with user-supplied theme
* ensure both metabase and user themes are merged
* ensure existing theme override is applied
* add tests for default text colors
* [MLv2] Add `preview-expression` to eval an expression on sample data
This is needed to show the previews in the UX for the combining columns
epic #39977.
Fixes#39979.
* [MBQL lib] Add "Combine columns" drill-thru
This is following the original plan of using a drill for this. It's hard
to combine that with the "preview" functionality.
See [thread](https://metaboat.slack.com/archives/C06P22KS4JH/p1712264174056569)
for discussion of how we might approach that issue.
Fixes#39978.
* [MBQL lib] Add "Combine columns" drill-thru
This is following the original plan of using a drill for this. It's hard
to combine that with the "preview" functionality.
See [thread](https://metaboat.slack.com/archives/C06P22KS4JH/p1712264174056569)
for discussion of how we might approach that issue.
Fixes#39978.
* [MLv2] Add `preview-expression` to eval an expression on sample data
This is needed to show the previews in the UX for the combining columns
epic #39977.
Fixes#39979.
* [MLv2] Add `preview-expression` to eval an expression on sample data
This is needed to show the previews in the UX for the combining columns
epic #39977.
Fixes#39979.
* Revert "Merge branch 'mblib-preview-expressions' into combine-columns"
This reverts commit dac17f84aa, reversing
changes made to 2934fab19d.
* Revert "Merge branch 'mblib-preview-expressions' into combine-columns"
This reverts commit 2934fab19d, reversing
changes made to 6a43c22e78.
* Revert "Merge branch 'mblib-preview-expressions' into combine-columns"
This reverts commit 6a43c22e78, reversing
changes made to 2257333a8a.
* [FE] "Combine columns" drill thru (#40082)
* Type Section['icon'] properly, including the hack for summarize section
* Add combine columns section
* Add types for combine columns drill
* Add combine columns drill component stub
* Mock combine columns drill
* Add component stub
* Encapsulate typing hacks
* Extract CombineColumnsDrill
* Extract ColumnAndSeparatorRow
* Improve naming, add button to edit separators
* Translate string
* Improve styling
* Separator input
* Handle removing columns
* Rename lib.ts to utils.ts
* Implement onSubmit
* Add getNextColumnAndSeparator
* Use form for a11y
* Handle vertical overflow
* Handle horizontal overflow
* Extract formatSeparator
* Use proper translations
* Adjust min/max sizes
* Fix scrollbar being unnecessarily shown
* Display source column name
* Avoid serializing the entire column and putting it into DOM
* Use module.css extension
* Format code
* Improve a11y
* Mock combineColumnsDrillExpression and previewExpression
* Add Preview component
* Move preview logic to Preview component
* Style Preview
* Format code
* Move styles to CSS modules
* Update UI to latest designs
- move "Add another column" button
- remove "Separated by X" button
- always show separator inputs
* Remove displayInfo mock
* Add asReturned
* Update CombineColumnsDrillThruInfo
* Integrate new way of working with combine columns drill
* Fix generating expression names
* Integrate drill with click actions
* Integrate new preview workflow
* Revert "Update UI to latest designs"
This reverts commit cda039dffe.
* Use previewExpression interface correctly
* Use new preview_expression API
* Make sure columns are created with original query and stage index
* Extract usePreview and handle preview errors
* Update popover title
* Move add column button
* Fix outline being cut off
* Style preview label as per design
* Format empty separator
* Make preview scrollable
* Use project convention
* Simplify ScrollArea usage
* Use ScrollArea in CombineColumnsDrill
* Update comment
* Remove Lib.previewExpression
* Clean up the preview after removing Lib.previewExpression
* Implement getPreview
* Change default column preview to 'text'
* Remove usePreview hook which is no longer in use
* Rename Preview to Example
* Use different placeholder for separator input
* Add whitespace placeholder
* Add ColumnPicker boilerplate
* Export Input
* Use QueryColumnPicker for ColumnPicker
* Pass width to AccordionList
* Add sequences popover events with tippy
* Remove unused ColumnOption helpers
* Add testid to Example
* Add e2e test for combine column in header
* Select content of separator input on focus
* Reduce padding on example
* Add monospace variant to Text
* Use monospace variant in Example
* Use monospace font in textinput
* Match faux-select label styles to our TextInput label styles
* Add missing color
* Remove !important
* Use pre whitespace so consecutive spaces show up
* Remove empty default
* Add example for boolean
* Remove !important
* Use should.have text over contain
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Remove duplicated code
* Export useMantineTheme from metabase/ui
* Use useMantineTheme instead of reading theme directly
* Remove setTimeout in popover
* Use plain t instead of jt
* Move Popover dropdown sequence hack to metabase/ui
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add string like filtering capabilities to filter widget
* Address the review suggestions
* Add e2e test
* Address review suggestions
* Update mongo e2e test id so it hopefully corresponds to CI data
* Update e2e test to use id gathered from actual data
* Expand and improve the E2E reproduction
* Explicitly reference the underlying issues we're reproducing
* Update frontend/src/metabase/querying/components/FilterPicker/FilterPickerBody/FilterPickerBody.tsx
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
* Fix parens
---------
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
These string matching clauses only allowed two arguments previously.
Typically `[:contains field x]` to match a field against a literal.
This adds similar desugaring for `:contains`, `:does-not-contain`,
`:starts-with` and `:ends-with` that is currently done for
multi-argument `:=` and `:!=`:
```clojure
[:contains field x y z] ;; ->
[:or [:contains field x] [:contains field y] [:contains field z]]
[:does-not-contain field x y z] ;; ->
[:and [:does-not-contain field x]
[:does-not-contain field y]
[:does-not-contain field z]]
```
* Add shortcuts to suggestions
* Add shortcuts group
* Add split icon
* Start implementing extract column modal
* Pass alwaysExpanded from prop to accordion items
* Allow disabling search on QueryColumnPicker
* Only render columns that support extraction
* Add columnExtractions helper to Lib
* WIP
* Add examples for column extraction
* Use Lib.columnExtractions to create extraction picker
* WIP: Create clause from extraction
* Use mb- prefix in color
* Add extract wrapper
* Use updated column extraction types
* WIP: extract clause
* Use JSDoc for todo
* Handle extract column submit in expression editor
* Set name based on extract expression
* Suffix column displayNames to avoid conflicts
* Rename combine test to disambiguate it from extract test
* Add e2e test for column extractions
* Correctly render button as button
* Add tests for email extractions
* Use correct font in expression button
* Remove error when selecting an extraction
* Use last expression
* Use handleExpressionChange
* Add tests for url extractions
* Rename to lastExpression
* Add issue link to the todo
* Move Button into ExtractColumn
* Use unstyled button for ExtractColumnButton
* Fix Snowflake filtering on date for `timestamp with time zone` columns
* Snowflake should just generate raw SQL for literals
* Go ahead and fix other Snowflake stuff and reenable a bunch of disabled tests
* Snowflake test fixes 🔧
* Minor test improvements
* Fix Redshift and Vertica as well.
* Address PR feedback
* update the option name
* Update docs/configuring-metabase/slack.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Use RTK Query endpoint to fetch native dataset in preview query modal
* Remove unused selector
* Remove endpoint from `services`
* Fix type error
* Use `getLearnUrl` selector instead of the deprecated MetabaseSettings
* Fix the error potentially being undefined when passed to a helper
* Fix weird destructuring syntax
* Store Parameter Values Set by User on a per-user basis
This is a WIP for #28181 and the notion doc:
https://www.notion.so/metabase/Tech-Maintain-user-level-state-in-dashboards-for-filters-fc16909a3216482f896934dd94b54f9a
Still to do:
- [ ] validate the table/model design
- [ ] hook up the endpoints mentioned in the doc (2 in api/dashboard)
- [ ] return the user specific parameter values on /api/dashboard/:dashboardID endpoints
- [ ] write a few tests to capture the intent of this feature
* Accidentally deleted a digit in the change ID timestamp
* first pass at writing user param values to the db.
It's in a doseq here which is probably not the correct way to go yet, but it's a step in the right direction.
* Hydrate dashboard response with `:last_used_param_values` key
If the user has previously set a parameter's value, it will show up in the map under that parameter id.
If the user has no parameter value set, that entry will be 'null'.
* Use proper fn name
* Only save or retreive user params if there's a current user id
* Add model to necessary lists
* Only run query when there are parameter ids to run it with
* Add a test to confirm that last_used_param_values works
* Add models test namespace for CRUD test
* The hydration is checked in the dashboard api test already
* Fix `sdk` key not expected on a reducer
* Use existing type instead
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Add default SDK state when using `renderWithProvider` with `sdk` mode
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Add analytics event for combine columns shortcut
* Add new version for question analytics schema
* Add column_combine_via_shortcut event
* Add custom_expressions_used field
* Use trackSchemaEvent to track combine column event
* Move trackColumnCombineViaShortcut to separate file
* Fix typo in event key
* Add column_combine_via_column_header
* Add column_extract_via_shortcut
* Add column_extract_via_column_header
* Add database id for question
* Move analytics file to frontend/src/metabase/query_builder
* Allow null for custom_expressions_used
* Add test for snowplow event
* Fix typo in schema
* Add max length to custom expressions
* Add description to list item
* Fix add-alias-info matching for nominal field literal refs
* Minor cleanup
* Minimize diff
* A bit more cleanup
* Cumulative aggregations w/ multiple breakouts should reset counts (#2862)
* Update dox
* Do the nesting FIRST before adding alias info.
* Remove unused
* FE for Offset()
* Cleanup stray code
* Prettier
* Kondo
* Test fixes 🔧
* Better error message when no breakout
* Support offset as an expression with no breakouts. Still require an ORDER BY tho.
* Fix most stuff
* Hack city??????
* Revert nest-query changes and disable the offset-no-breakout test for now.
* Appease TypeScript
* Test fix 🔧
* Oracle & MySQL test fixes
* Test fixes 🔧
* Remove maxWorkers for FE unit tests
* Add Embedding SDK readme
* Update enterprise/frontend/src/embedding-sdk/docs/README.md
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Update enterprise/frontend/src/embedding-sdk/docs/README.md
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Update enterprise/frontend/src/embedding-sdk/docs/README.md
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Clarify the result from `useQuestionSearch`
* Remove the mentioning of API key
Since we want to drop API key support for embedding SDK
* Add more detail
* Update enterprise/frontend/src/embedding-sdk/docs/README.md
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Update enterprise/frontend/src/embedding-sdk/docs/README.md
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Update enterprise/frontend/src/embedding-sdk/docs/README.md
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Update enterprise/frontend/src/embedding-sdk/docs/README.md
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Update enterprise/frontend/src/embedding-sdk/docs/README.md
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Update enterprise/frontend/src/embedding-sdk/docs/README.md
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Remove private GH repo
* Remove the compatibility table until we've made the first release
* Fix typo
* Add details how to run with locally built SDK
* Correct Metabase Embedding SDK case
* Make the instruction more concise
* Add a disclaimer
* Update config removing deleted property
* Add Oisin suggestion that was missed from rebasing
* Clarify plugin section
* Tweak SDK Readme
* Polish the limitation section
* Update the code sample into 4 sections making it easier to read
* Tweak overview and features section of README
* Add metabase configuration section
* Clean up section
* Replace the doc in the root embedding SDK directory
* Some tweaks and improvements to SDK README
* Tweaks to SDK README: wording, change flow, add section on running MB from master
* Update enterprise/frontend/src/embedding-sdk/README.md
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Feedback section
* Add simple InteractiveQuestion example
* SDK README: Tweak Slack wording
* Standardize fullstops + fix typo
* Clean up formatting
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Oisin Coveney <oisin@metabase.com>
Co-authored-by: Alberto Perdomo <alberto.perdomo@gmail.com>
* Add `collection.effective_ancestors` to search
Fixes#41909
The ultimate goal here is to be able to display a UI that showed the entire (effective) path to each collection.
This was a bit more painful than I'd hoped. The primary issue is that we
tied two things together:
- scoring results
- "serializing" results (really, partially serializing results, because
we do more work to shape the data later)
Why is this a problem? The `top-scorers` function took a transducer and
returned the *serialized* top scorers. Later, when we need to operate
on *all* the top results (in this case, to collect all the collection
IDs and names that we know of), we're looking at the serialized results.
In our case, we needed some data (collection location) that wasn't
included after serialization. We could add it, of course, but it's
frustrating to need to change the API of what data is *returned* from
the endpoint for purely internal purposes (changing the data available
during *processing*).
I ended up pulling serialization out of the scoring function. At some
point down the road, we should go ahead and move the `serialize`
function out of the `search.scoring` namespace entirely - there's really
no reason for it to belong there as far as I can tell.
* temp
* rework DatasetFieldMetadataSidebar to use new form components
* temp
* removing unneeded import
* fixing display name form field
* temp
* more temp
* onChange for form elements
* e2e adjustment
The new expressions `:month-name`, `:quarter-name` and `:day-name`
return the (user localized) names for these units, given the
corresponding number.
For example, 4 yields `"Apr"`, `"Q4"` or `"Wednesday"` respectively.
The `column-extract` drill uses these new expressions, rather than
generating its own `:case` clauses for them.
Note that `:day-of-week` and `:month-of-year` are tricky cases for i18n, but there are tests that exercise them thoroughly.
Part of the follow-up for Extract Column epic #38964.
* Add subTitle to ClickActions
* Add examples to column extractions
* Add test for examples
* Add test for examples
* Use rem helper
* Use JSDoc for todo
* Disambiguate between ColumnExtraction and ColumnExtractionInfo
* Update type of getExample
* Use rem helper
* Notebook combine columns (#41379)
* Add CombineColumns dialog component
* Add combine columns to the expression editor widget
* Add placeholder for empty and space to the input
* Drop the first separator
* Remove empty string from eventual concat
* Add example to Combine Colunmns builder
* Show example in monospace
* Do not collapse spaces in example
* Use monospace in separator input
* Reorganise column flatten helper
* Add back button on combine buttons modal
* Select text when focussing separator input
* Add minimum height to example
* Use that same separator as the last row
* Use ExpressionWidgetHeader instead of hand-rolled header
* Fix type when generating example
* Correctly update default separator when a url column gets set
* Swap url and email checks
* Move column input into separate component
* Use QuerySelector for column select
* WIP: customise dropdown
* Export Input
* Use a button to render the column selector
* Add combine icon
* Use shortcut suggestion shortcut for combine columns
* Allow interacting with buttons inside the popover
* Use longDisplayName for button label
* Add asReturned
* Use asReturned in CombineColumns modal
* Make faux-select button styles match normal input
* Return focus to button when select closes
* Fix type of focus event
* Move ColumnInput component to a separate file
* Only show combine column shortcut on custom column
* Add missing test ids
* Add e2e test for combine column shortcut
* Remove white fill from paths
* Add color to combine icon
* Use function expression, not declaration
* Remove comment
* Move initial default separator to constant
* Remove unused select option helpers
* Add isInteger helper to Lib
* Add case for integer columns
* Add unit tests for util
* Add SuggestionShortcut type to props for Suggestions
* Remove {enter} which broke test
* Do not use !important
* Simplify classname into object-style
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Use direct override on Button instead of class
* Fix formatting issue
* Use .select() to select input
* Avoid selecting placeholder text
* Avoid losing focus when window loses focus
* Rename evt to event
* Add monospace variant to Text
* Use monospace variant in Example
* Use monospace font from theme
* Match faux-select label styles to our TextInput label styles
* Remove empty default variant
* Add limit to the test case
* Remove unnecessary within wrapper
* Use rem
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
---------
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
* Remove intercept we're not waiting for
* Use have.text instead of contain
* Use cy.button
* Expand cancel test
* Remove duplicate title
---------
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Column extractions are "canned" expressions based on a column's
type. For example, we might extract the weekday from a temporal column,
or the domain from an email or URL column.
This logic already existed inside the `column-extract` drill; this pulls
it out as a top-level concept, since extractions are also being
integrated into the notebook editor apart from drills.
Part of the follow-up for Extract Column epic #38964.
It has a slightly different API shape. We're not using it right this
minute, but we're about to and it seemed nice to separate this out into
its own PR
* Export Input
* Move TextInput overrides to Input
* Remove the :not(:only-child) hack the Schedule component
* Remove the :not(:only-child) hack from DateInput
* Remove the :not(:only-child) hack from FileInput
* Remove the :not(:only-child) hack from Select
* Remove the :not(:only-child) hack from TextArea
* Remove the :not(:only-child) hack from TimeInput
* Reproduce pre-hack margin in Schedule component
* Undo overzealous refactor
* Add persisted info types
* Keep exports sorted alphabetically
* Add boilerplate `perist` API
* Use RTK Query for the `PersistedModels.list`
* Throw for unused `PersistedModels` endpoints
* Update types
* Re-wire PersistedModels `get` requests to use RTK Query
* Fix the endpoint
* Refresh model cache using RTK Query
* Add cache invalidation
* Fix `PersistedModels.api.get` endpoint options
* Remove stray TODO comment
* Explain the difference between the `persisted-info` and the `persisted-model` tags
* Invalidate `persisted-info` list upon refreshing a single model
* Be more consistent with borders and avoid double borders
* Mark the aggregation 'Custom Expression' button as an action
* Remove color from action headers
* Remove bottom border from last child
* Remove top border from first child
* Make section header classes consistent
* Add border to search header
* Use correct color for borders
* Add props to enable borders in AccordionList
* Add borders to column picker elements
* Dates Binned by Week Export Formatting Matches App
Fixes: #41492
Before, the dates binned by week would be formatted as follows in CSV and JSON exports:
`Week 1 - 2024`
But that doesn't match the App's format of a Date Range:
`January 1, 2024 - January 7, 2024`
So, now the exports will apply formatting in this same way.
Note that a second bug (#41616) exists on the Frontend preventing the column formatting being applied in the app. This is
frontend only, and the csv and json exports will match the column formatting settings (eg. abbreviated dates and
alternative separators will be used in the export).
* add test and change default week style
* Week format test change
* also fix incorrect :hour formatting
* [tech debt] Stop using ms/BooleanString and only use ms/BooleanValue
ms/BooleanString will coerce to a string, and we should probably get rid of it and only ever use ms/BooleanValue which will always coerce to a boolean
We want the coercion layer to give us values that are helpful, and true is more helpful than "true".
So we should remove ms/BooleanString and calls to parseBoolean
* Remove ms/BooleanString usage in api/native_query_snippet.clj
* Remove test case related to ms/BooleanString
* Remove the BooleanString schema def as no longer used
* Add RTK Query `revision` API
* Re-wire `Revisions` entity `list` to work with RTK Query
* Re-wire the `Revisions` entity `revert` to work with RTK Query
These functions are implemented with hairy regular expressions, and
it's more user-friendly and future-proof to name those functions in MBQL
rather than baking the `regexextract` and regex into the user's query.
It lets us evolve the regexes in the future if we detect a bug, and it
improves the UX since the user sees a meaningful function instead of
regexextract([My URL Column], "(?<=[@\.])(?!www\.)[^@\.]+(?=\.[^@\.]{1,3}\.[^@\.]+$|\.[^@\.]+$)")
Also refactors the regexes somewhat so that they work for emails as well as URLs, and there's always just one layer of `:regex-match-first`.
Previously this was separated into two steps: URL or email to host, and host to (sub)domain.
Part of the follow-up for Extract Column epic #38964.
* Delete unused endpoint from `PermissionsApi`
* Add group tag helpers
* Define RTK Query 5 main endpoints for `permissionApi` (group)
* Re-wire 5 main api methods in groups entity to work with RTK Query
* Re-wire `clearMember` action to work with RTK Query
* Add `Groups.api.update`
* Update types
* Fix membership failing to update after `CREATE_MEMBERSHIP`
* Make sure `members` is not undefined
* Fix cache invalidation
* Throw for unused Tasks endpoints
* RTK Query `taskApi`
* Wire tasks entity list and get endpoints to use RTK Query
* Fix typo
* Expose `taskApi`
* Rework `TaskModal` to use RTK Query api
* Throw on `Tasks.api.get`
* Rework `TaskApp` to use RTK Query api
* Delete `Tasks` entity
... which never should've been an entity in the first place.
* Delete unused `TaskApi` entry
* Fix next pagination page not being properly disabled
* Do not export `isLastPage` function
* Convert TasksApp to TS
* Use Mantine Tooltip
* Remove stray comment
* Add empty lines for better legibility
* Update the response type
* Define `task_details` type more strictly
* Use the existing `DatabaseId` type
* Provide tags for `list` and `get` endpoints
* Address code review
* Handle error and loading in the `TasksApp`
* Handle error and loading in the `TaskModal`
* Extract the page size into a variable
* Index databases without mutation
* Refactor TaskModal
When we actually run the job to cache models, we select "refreshable
models", which selects `PersistedInfo` where:
- the `state` is something refreshable, and
- the question marked for caching is actually a model, and
- the question hasn't been archived.
However, when we selected `PersistedInfo`s to display to the user, we
didn't apply any conditions. Instead, we included PersistedInfo where
the question was not a model, or where the question had been archived.
To prevent confusion, let's hide these.
Fixes https://github.com/metabase/metabase/issues/41318
The filters for _numeric_ PK and FK filters are slightly different than a general numeric column. That's fine for numeric keys, but with string keys that same set of numeric filters was being returned, and then the ones for strings were removed. That left only `Is` and `Is Not` for a string PK or FK.
This PR allows more flexibility in filtering by string PKs and FKs, by checking the base type of the column and using the right set of filters for the type of key.
Closes#40665.
* Add the pagination type
* Add the pagination helper
* Update types
* Update `user` API
* Update `collection` API
* Update `search` API
* Update `UserListQuery` type
* Explicitly define default pagination inputs
* Remove the pagination helper
* Update and fix types
`limit` and `offset` are actually nullable.
* Separate types (again)
* Fix `PaginationResponse` type
* FeedbackModal for the Embedding Homepage
* add source and change to correct endpoint
* fix tests, adding await and fixing endpoint
* use toEqual to also check for the source
* addresses PR feedback
* button becomes Send when email is written
* make 'comments' optional, trim inputs and refactor tests
* product-feedback endpoint (#40974)
* mvp of the proxy endpoint for the product feedback
* cleanup and fixes
* email :string -> ms/NonBlankString
* revert my solution
* Dan diff file
* makes fields other than source optional
* don't use default value in dev, to not spam the prod endpoint
* change from setting to function for product-feedback-url, as it's not meant to be changed by end users
* changed endpoint as per https://metaboat.slack.com/archives/C010ZSXQY87/p1712918951362979?thread_ts=1704910781.153589&cid=C010ZSXQY87
* fix wrong dismiss reason sent for 'I'm not interested right now'
* Update frontend/src/metabase/home/components/EmbedHomepage/EmbedHomepage.tsx
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* renamed field: feedback -> comment
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Add provideDashboardListTags and provideDashboardTags
* Keep the utils sorted alphabetically
* Include collection tags in dashboard tags
* Add getDashboard
* Migrate basic CRUD dashboard endpoints to RTK
* Migrate remaining dashboard endpoints to RTK
* Fix save dashboard action
* Remove unused favorite/unfavorite dashboard endpoints
* Fix copy dashboard action
* Move types to types file
* Type dashboard API requests
* Fix get
* Fix typo
* Pass body
* Add a comment
* Support noEvent option to fix a failing test
Add a Github workflow to nag on a PR that changes `metabase.lib.js` without updating `docs/developers-guide/mbql-library-changelog.md` as well.
You can set the `.metabase-lib/no-change` label to override for a change that edits the namespace without materially changing the API.
Not all drivers support regular expressions. Don't return the
Column Extractions for URL -> (sub)domain and email -> domain unless the
database can support the regular expression matches.
Part of the follow-up for Extract Column epic #38964.
This improves our Malli performance substantially! Many of our code paths were calling vanilla `malli.core/validate` or `malli.core/explain`; these redo the (possibly expensive) compilation of the schema into a validator or explainer for every call.
We should use the caching versions in `metabase.util.malli.registry` everywhere, and our runtime performance will be much improved.
However when I started using these cached versions, I found that the memory use was growing out of control. Eventually I tracked this down to `:fn` schemas. Functions are only comparable by pointer equality, so they make poor cache keys. `(fn ...)` or `comp` calls in a schema on a `mu/defn` function **get re-created for every call of the `mu/defn`'d function**! That's a big time sink recompiling the schemas if we're *not* caching, and a huge memory sink if we *are* caching!
This PR pulls every such schema I could find out into a `def` so it uses the same closure and is cacheable. I'd like to automate that in some part of the Malli pipeline, or maybe a linter rule, but I haven't found a good way to do it yet.
Part of #39946.
* Define the types for the `create` user request
* First draft for the create user API using RTK Query
* Export RTK user API
* Define explicit `create` api for the user entity
* Define `updatePassword` mutation
* Wire users entity `resetPasswordManual` to use RTK under the hood
* Use `updatePassword` mutation
* Wire users list calls
* Remove unused `update_password` from services
* Fix the dispatch used to list users
* Dispatch the action type and the payload
* Wire `deactivateUser` to work with RTK
* Wire `reactivateUser` to work with RTK
* Delete unused entries from `UserApi`
* Convert the reducer to use `switch` statement
* Remove unused user `send_invite` endpoint from FE
* Wire `resetPasswordEmail` in users entity to work with RTK
* Provide cache invalidation for the `userApi`
* Add `provideUserListTags` helper
* Pass `email` as a sole argument
* Throw on `get` a user
This endpoint hasn't been used in a long itme.
Removed in https://github.com/metabase/metabase/pull/41334/files#diff-241e0afa11f03a0942eb322aa47695e58496379b0b6d42e6e59ecf6a20fe623cL448
* Use RTK Query to `updateUser`
* Fix wrong syntax
* Use RTK Query to `getUser`
It was used by `.load` and `fetch` after all.
* Export `useUpdateUserMutation`
* Refactor `getRecipientsList`
* Fix response type for the `getUser`
* Pass user id as a sole argument instead of in an object
* Fix users entity `get` method
* Fix the user update password form
* Fix unit tests
* Fix deactivate and reactivate query invocations
See e949812.
* Adjust types according to the feedback
* Order utils alphabetically
* Throw on `delete`
* Revert "Fix unit tests"
This reverts commit 2c329e8364.
* Fix FE unit test mocks
* Rename imports to use plural for consistency
* Fix one last remaining FE unit test
* Add list of most popular functions and aggregations
* Render 'All functions' link on expression dropdown
* Suggest popular functions when no input is given
* Do not return focus to search bar
* Default to not matching any substring
* Open suggestions when the expression editor opens
* Group expressions by group
* Render group titles in dropdown
* Match the expression suggestions height to the design
* Add tests for popular functions
* Convert tests to it-style
* Only return supported popular functions
* Limit most used functions group to 5 items
* Make the suggestion footer keyboard-accessible
* Make a different group for popular aggregations
* Remove padding on suggestion list
* Resize list
* Add test for popular functions in suggest
* Move showMetabaseLinks into args
* Remove unneeded optional chain
* Clarify popup issue in ACE hack
* Use correct target for links
* Reference correct method in comment
* Split up popular expressions
* Remove Set around popular functions/aggregations
* Do not allow interacting with suggestion header
* Do not use _ prefix for variable
* Add verb in test
* Remove unused async in test
* Test if functions are in document
* Move tests for suggestWithFooter to separate file
* Use correct target in link
* Fix typo in key
* Allow picking suggestion when textbox is empty
* Remove unnecessary cast
* Avoid double scroll bars
* Only show link to all functions when editor is empty
* Split up popular functions between expressions and filters
* Fix type of popular variable
* Use the same heuristic for footer as for popular expressions
* Close suggestions when the editor is not focused
* Add missing open prop in test
* Simplify test
* Add selector for expression editor widget
* Fix e2e tests
Fix e2e tests
* Avoid non-interactive clicks from closing the dropdown
* Use custom expression helpers in broken test
* Fix test
* Remove should exist check for selector
The drill-thru is mostly a stub to put an entry in the context menu. Most of the logic for this is built on the FE using the existing functions for consuming and creating expressions.
See [thread](https://metaboat.slack.com/archives/C06P22KS4JH/p1712264174056569) for a lot more context.
This also adds `docs/developer-guide/mbql-library-changelog.md` and documents adding this function to the library as the first change in the API. I'll follow up in another PR with Github automation to expect the changelog to be updated whenever `src/metabase/lib/js.cljs` changes.
Fixes#39978.
* Keep Filter Values for enabled Parameters in Embedding
Fixes: #37914
Previously, we removed locked parameters entirely. This is to prevent leaking potentially sensitive values.
However, for the situation where the same field backs 1 locked or disabled paramter and 1 enabled parameter, we do
still want to send the paramater values, because the enabled parameter implies that the values are permissible to see
in the embed.
So, this change will still remove parameters based on their 'disabled/locked/enabled' status, but will NOT remove the
linked field ids if they're also being used by 'enabled' parameters.
This results in the backend correctly sending necessary parameter values to the embed, where the frontend can then
render the appropriate UI instead of falling back to just text filters.
* test the case where a locked and enabled param share same field
* Address feedback. Added comment to explain classify fn
* Fix add-alias-info matching for nominal field literal refs
* Minor cleanup
* Minimize diff
* A bit more cleanup
* Cumulative aggregations w/ multiple breakouts should reset counts (#2862)
* Update dox
* Do the nesting FIRST before adding alias info.
* Remove unused
* Add help text icon in expression editor suggestions (#41092)
* Add helptext to expression suggestions
* Deduplicate getDatabase calls
* Allow customising popover width
* Split out help text content component
* Render an info icon with the help text
* Remove duplicated defaults for popover
* Explicitly wait for suggestion items in test
* Remove commented out code
* Use React imports directly
* Rename e to evt
* Inline getHelpText when creating suggestions
* Avoid overlapping content
* Remove unnecessary Fragment
* Avoid selecting item when clicking link or button inside element
* Allow shifting popover
* Stop Popover mouse events for bubbling
* Use onMouseDown instead of onMouseDownCapture to avoid clicks inside popover from triggering select
* Add help text to all suggestions that are know mbql functions
* Expression suggestions clean up (#41236)
* Move ExpressionEditorSuggestions to tsx
* Convert ExpressionEditorSuggestions to functional component
* Simplify suggestion match component
* Move ExpressionEditorSuggestions a separate folder
* Remove default export
* Use Popover instead of Tippy for suggestions list
* Fix suggestions tests
* Remove data-ignore-outside-clicks
* Add default for suggestions
* Fix e2e test
* Move helptext wrapper to ExpressionEditorTextfield
* Switch to metabase/ui for Button
* Use arrow function
* connect the example dashboard id setting to the embedding homepage (#41138)
* connect the example dashboard id setting to the embedding homepage
* fix types
* Update frontend/src/metabase/home/components/EmbedHomepage/StaticTabContent.tsx
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* use isNotNull
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* collection and db breadcrumbs
* compact breadcrumbs
* adding navigation keybindings
* recent_views and search changes to always include collection or db name
* removing breadcrumbs, display parent name
* removing unneeded components
* e2e and FE unit tests
* waitFor
* e2e tests for keybindings
* help
* be tests passing
* PR feedback and type adjustments
* unit test adjustment
* Keep the original question name in the save modal where possible
Resolves#41196
* Update tests
* Append "- Modified" to a suggested question name
* Preserve the original question description
* Address reviews - adjust the translation string
* Add E2E reproduction
* Remove extra space
* Improve assertions
* .eslintrc -> .eslintrc.js
* This might work?
* add env for linting css modules class names
* avoid extending postcss-modules/recommended, specify just the used rule instead
* add missing . on comment
* Remove unused .detailName references
* Fix ManageApiKeys and GuiQueryEditor
* Fix SettingText and DashboardControls
* fix missing textUnspaced class
* remove usave of S.headerButton, that's actually never used because we never pass headerLink to ReferenceHeader
* Remove flexNoShrink
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Fix embed download endpoint
We have added format_rows as a query parameter. In most endpoints this just works fine.
Here, however, the problem is that we use query parameters to pass parameter values. We run some validation over these
provided parameter values (eg. don't allow a user to over ride a 'locked' parameter). Since we have these 2 different
use cases for query parameters, the validation was failing on :format_rows, which isn't expected to exist at all.
Here, I dissoc the :format_rows key prior to this validation step, so normal validation can continue.
* Add test that uses this card embedding endpoint
* Nested queries have limits which can defeat their purpose
Consider a query
```sql
select count(*) from {{#199}}
```
This query should return the number of distinct rows in the query
defined by 199. But it's actually limited by the excel limit of
1048575. And that's because when the value of `{{#199}}` is expanded it
has that limit applied as normal.
```clojure
qp=> (let [card-id 199] ;; use a valid card id for you
(-> {:database 1,
:type :native,
:native {:query "select count(*) from {{ref}}"
:template-tags {:ref {:card-id card-id
:type :card
:name "ref"
:display-name "ref"}}}
:middleware {:disable-max-results? true}}
qp.compile/compile
:query
(metabase.db.query/format-sql )
println))
select
count(*)
from
(
SELECT
"PUBLIC"."ORDERS"."ID" AS "ID",
"PUBLIC"."ORDERS"."TOTAL" AS "TOTAL"
FROM
"PUBLIC"."ORDERS"
LIMIT
1048575
)
```
But we can suppress this limit when substituting a query inside yielding
```sql
select
count(*)
from
(
SELECT
"PUBLIC"."ORDERS"."ID" AS "ID",
"PUBLIC"."ORDERS"."TOTAL" AS "TOTAL"
FROM
"PUBLIC"."ORDERS"
)
```
And this is proper because we want to limit the _outer_ query, not
internal queries.
* Remove limit from test expectation
* stupid trailing space
* another subquery test
* Use helper function to disable limit middleware
* Fix#39138 again
Fixes#40934
Got unfixed somehow in #40578. seems like some file renames caught it off guard
But the long and short is that its possible for a persisted model to end
up in the state "refreshing" (possible an instance restart during
refreshing). The refreshing job doesn't look for these so they become
effectively invisible.
Since the job to refresh them will only run one at a time cluster wide,
any jobs that are in the "refreshing" state when the refresher begins to
refresh are necessarilly stuck (no one else could currently be
refreshing them). So we can just add them to the queue of models to
refresh.
```clojure
(jobs/defjob ^{org.quartz.DisallowConcurrentExecution true ;; <----
:doc "Refresh persisted tables job"}
PersistenceRefresh [job-context]
(refresh-job-fn! job-context))
```
* Ensure ee/oss pathways are taken in tests
Annoying little footgun here. CI does not run with an ee token, so in
order to ensure that test pathway goes through ee version of a
defenterprise we _must_ use the `mt/with-premium-features
{:cache-granular-controls}`, but we also want to ensure that it goes
through the oss version.
So two options, a `doseq` on both features (empty set and the feature
that triggers this). But want a test in the enterprise folder as well to
ensure.
The real trickiness comes from running tests at a repl and CLI. My REPL
always has an ee token in it. My command line always lacks that as
well. So want to be explicit about the token features in effect at test
time.
That's why I'm essentially duplicating tests (ee in ee folder, oss in
regular pathway)
* Restore tabs display for embedded dashboard without title
* Add unit test
* Remove empty file
---------
Co-authored-by: Mahatthana Nomsawadi <mahatthana.n@gmail.com>
* H2 compilation improvements: avoid unneeded casting
* Legacy MBQL schema improvements
* Add additional test for wrap-value-literals
* Simpler version of the fixes from #36068
* Test and lint fixes 🔧
* Pivot QP use MLv2
* Wow
* Test fix 🔧
* Fixes
* Actions should use strings for column names (fix :update-row and :create-row normalization)
* MLv2 schema should check against keys for the other query type
* Ok, have I fixed things?
* More fixes 🔧
* Fix indentation
* Another round of test fixes. 🔧
* Hopefully the last few test fixes 🔧
* We need to test normalization for queries that have keyword keys as well.
* Fix Cljs i18n namespaces
* Sort namespaces
* Only test against H2
* Test fixes 🔧
* Register MBQL clause schemas and test fixes 🔧
* Test fixes and PR feedback
* Test fix
* Remove the normalization tests
* Test fixes 🔧
* Fix kondo
* PR feedback
* Resurrect remove-all-breakouts
* Test fixes 🔧
These type checks are useful when constructing clauses with full MLv2
metadata available, but when converting from legacy the `:field` refs
have only `:base-type`.
If a column has its metadata overridden to eg. coerce a string or
integer into a date, its `:base-type` will remain `:type/Integer` or
`:type/Text` and the `:effective-type` will be `:type/Date` etc.
This change skips the type check while converting from legacy, and
while checking `can-run`. Eventually this should be replaced with
tracking `:effective-type` on refs or using real metdata, instead of
relying on the `:base-type` only.
Fixes#41122.
We need to serialize it for any card, since when a normal question depends on a native question - their result_metadata won't be updated when opening a dashboard, which leads to front-end not understanding how to connect the parameters from a dashboard to a card, and errors during queries.
* Group axes with the correct map shape.
Prior to this, any time axes were all explicitly set :left or :right by the user, the `group-axes-at-once` function
would incorrectly return the map in the following form:
`{:left ["a" "b" "c"]}` which is incorrect. It must instead look like this:
`{"a" :left "b" :left "c" :left}`
This fixes the map shape and permits the user to explicitly set all axes to the left or to the right.
* Add a test to confirm the axis settings are respected and do render
* Wow
* Test fix 🔧
* Fixes
* Actions should use strings for column names (fix :update-row and :create-row normalization)
* MLv2 schema should check against keys for the other query type
* Ok, have I fixed things?
* More fixes 🔧
* Fix indentation
* Another round of test fixes. 🔧
* Hopefully the last few test fixes 🔧
* We need to test normalization for queries that have keyword keys as well.
* Fix Cljs i18n namespaces
* Sort namespaces
* Only test against H2
* Rename `metabase.mbql` to `metabase.legacy-mbql`
* Fix Kondo warnings
* Test fixes 🔧
* Register MBQL clause schemas and test fixes 🔧
* Test fixes and PR feedback
* Test fix
* Remove the normalization tests
* Test fixes 🔧
* Fix kondo
* Fix import
* Another fix 🔧
* Merge
* FIXES
* Add another missing REQUIRE
* More Metric => LegacyMetric renaming
* metabase.models.metric-important-field => metabase.models.legacy-metric-important-field
* Even more renaming
* More renaming
* Even more fixes.
* More renaming
* Wow, even more renaming!!
* Use window functions for SQL drivers for cumulative count and sum
* H2/Snowflake/Redshift implementation
* Test fixes 🔧
* Use SELECT expression positions as the default window function implementation
* Some test fixes.
* Fix bugs in add-alias-info
* MongoDB fixes
* Another pass
* BigQuery should use positions?
* BigQuery almost works 100% now.
* MLv2 fix conversion behavior for expressions with type info
* Add transformations for nesting cumulative aggregations in expressions
* Minor tweaks
* MLv2 should escape aliases using driver/escape-alias (dynamically bind this)
* Kondo fix
* Lint fixes 🔧
* More test fixes and revert test changes
* Update some tests to use correct case for nominal Field literal refs
* Wow! BigQuery is fixed! Remove upgrade-field-literals middleware
* Disable tests for the failing drivers, we can fix in follow-on PRs.
* Enable BigQuery :window-functions
* Extra info
* Add GH issue numbers
* Update more GH issue numbers
* Fix SQL Server!!!! wooo
* I THOUGHT WE WERE DISABLING SPARK SQL!!!!!!!!!!1
* Cleanup to get things ready for review.
* Snowflake + Vertica shouldn't use output column numbers for ORDER BY in OVER
* Code cleanup
* Snowflake + Vertica shouldn't use output column numbers for ORDER BY in OVER
* Code cleanup
* Test fixes 🔧
* Much simpler BigQuery implementation
* Add GH issue context
* Remove commented-out code
* Remove unused add-stage-at-index
* Documentation
* Test fix? 🔧
* Test fix 🔧
* Add new user-local setting `notebook-native-preview-preferences`
* Update user settings on FE
* Use two distinct setting entries
* Update FE types
* More explicit `notebook-native-preview-sidebar-width` definition
* Add selectors
* Initialize QB uiControls based on the BE settings
* Rename QB store entry
`isNativePreviewSidebarOpen` -> `isShowingNotebookNativePreview`
* Rename QB store entry
`nativePreviewSidebarWidth` -> `notebookNativePreviewSidebarWidth`
* Add Redux actions
* Dispatch user settings on top of Redux store
* Fix test title typo
* Add E2E test to make sure sidebar is resizable
* Expand the E2E test to cover persisting settings across sessions
* Be more explicit that we're resizing along x axis
* Store window breakpoint as a variable
* Calculate handle's left position more explicitly
* Make canSelectSection a function
* Use a custom type for actions in the accordionlist
* Add the globalSearch prop to AccordionList
* Improve AccordionList tests
* Enable globalSearch in the AccordionPicker on the FilterColumn and QueryColumn picker
* Assume alwaysVisible for action sections
* Add state for empty search results
* Fix test case
* Allow passing globalSearch to Select
* Set globalSearch in semantic type picker
* Handle search when a section does not have items
* Rename variable to be more descriptive
* Add borders to headings in select
* Add border to back button too
This optional fourth argument will disregard all joins at `position` and
later when suggesting conditions. This is useful when editing a join's
RHS, which wipes out the condition. We don't want to generate a
suggested condition that's based on keys from the RHS of the join we
just edited, or from later joins.
Fixes#40916. Part of #40890.
New endpoint accepts entity and entity ids in form of `database=1&dashboard=2&question=3`. If you don't supply
`&include=overrides`, then it tries to find configs directly referencing supplied entities and updates their
`invalidated_at`. If you supply `&include=overrides`, all the referenced cards are
updated (`report_card.cache_invalidated_at`).
Cache strategies then select the maximum `invalidated_at` in their logic. EE-only.
resolves#40548
* Display sql preview on sizes less than 1280px
* Handle min notebook width using Mantine responsitve style prop
* Display the resizable sidebar on sizes >= 1280px
* Set sidebar width on resize
* Add custom handle
* Simplify window resizing with `useWindowSize` hook
* Use "white" from the theme colors
* Improve sidebar semantics with `aside`
* Define the `NativeQueryPreviewSidebar` as `aside`
* Use Mantine style props instead of CSS module
* Run the E2E test again using regular browser
We removed the nested media query that ReplayIO doesn't support.
Now this test can run using Replay Chromium as all other tests do.
* Add some breathing room between the scrollbar and the resizable sidebar
* MLv2 normalization: add :lib/uuid automatically to options maps if it's missing
* Remove unneeded normalization code in QP
* Fix 🔧
* Ignore invalid metadata providers when passed in from the REST API
* create ui for the full embedding homepage, without any logic (#40251)
* wip ui for the full embedding homepage
* adds missing link + fix keys
* remove done TODO
* Update frontend/src/metabase/home/components/EmbedHomepage/InteractiveTabContent.tsx
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* destructure props in the function arguments
* rename customerFacingAnalyticsDocsUrl to analyticsDocsUrl
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* add new settings for embedding homepage (#40455)
* remove old minimal version
* adds new embedding homepage settings and sets them at the end of the setup
* comment e2e test
* add tests for embedding homepage flags
* more tests
* remove unit tests related to old minimal version
* fixes flags descriptions
* :keyword for embedding-homepage
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* replace reloadSettings and refreshSettingsList with initializeSettings
* `value => value === true` -> `Boolean`
* toMatchObject -> toEqual for more strict tests
* remove useless undefined check for settings, the component is conditionally rendered based on a setting so it means they're loaded anyway
* fix: use :hidden as we changed the type to :keyword
* specifically check for the `embedding` feature in the tokenFeatures object instead of checking if any feature is active
* remove double \n
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* connect settings to the embed homepage and show it (#40528)
* fix nested a warnings in browser console
* connect settings to the embed homepage and show it
* fix stories
* remove localStorage.clear() as we're not using it in the logic anymore
* dismiss without feedback for the embedding homepage (#40587)
* feat: add dismiss functionality to the embed homepage
* update setup e2e to check for the dismissal of embedding homepage
* use List from metabase/ui (#40638)
* set embedding-secret-key when embedding is enabled (#40813)
* set embedding-secret-key when embedding is enabled
* use "named" setters and getters as suggested by Noah
* Fix tests for ms1 after updating on master (#41069)
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Use window functions for SQL drivers for cumulative count and sum
* H2/Snowflake/Redshift implementation
* Test fixes 🔧
* Use SELECT expression positions as the default window function implementation
* Some test fixes.
* Fix bugs in add-alias-info
* MongoDB fixes
* Another pass
* BigQuery should use positions?
* Disable tests for the failing drivers, we can fix in follow-on PRs.
* Extra info
* Add GH issue numbers
* Update more GH issue numbers
* Fix SQL Server!!!! wooo
* I THOUGHT WE WERE DISABLING SPARK SQL!!!!!!!!!!1
* Cleanup to get things ready for review.
* Snowflake + Vertica shouldn't use output column numbers for ORDER BY in OVER
* Code cleanup
* Test fix? 🔧
Goal is to properly document the interface between MLv2 and the FE. In
particular I want to call out the health and fate of each function. Some
of them only exist to support legacy compatibility, for example, and
that should be documented to discourage new usage and make clear the
conditions for removing the functions.
* Hide info icons when no description exists
* Move defaults for InfoIcon to separate file
* Add open delay to popover
* Remove inverted background from type label
* Fix margin on semantic type label
* Make semantic type not be bold
* Fix unit tests for hidden info icons
* Rename file to constants
* Remove unused showIfEmpty prop altogether
* Fix the height of the expression suggestions that don't have an info icon
* Fix padding on filter modal
* Fix the height on dataref
* Enable not emptyable arguments for is-empty and not-empty clauses
* Add review remarks
* Shorten dataset name so it is usable with some drivers
* Disable test for Athena
* Handle possibly missing base type
* Update comment
* Missing base type field is treated as not emptyable
* Update desugar middleware
Now middleware adds base type to fields with integer id that are missing that first. This ensures correct function of is-empty expansion.
* Update desugar middleware
Cleanup base types necessary for desugaring is-empty
* Update desugar middleware cleanup
* Update test
* Adjust tests
* Remove now redundant e2e test
* Update comment
The first time you zoom in on a bin in a binned query, it works fine.
For example, Orders COUNT by SUBTOTAL auto-binned has a bin for 40-60.
If you zoom in on that, it correctly shows the range 40 to 60 broken
down into smaller bins which are 2.5 units wide. If you then try to
zoom in on one of these smaller bins, say 50-52.5, there are two
problems:
1. The filters from the first zoom (`>= 40` and `< 60`) are still
there, and so are the new ones (`>= 50` and `< 52.5`).
2. The new filter is actually wrong; it uses the big width (20)
from the original bins, not the width of the nested bins (2.5),
so the new filters are `>= 50` and `< 70`.
This PR fixes both issues, so nested bins (1) remove the old filters,
and (2) use the correct width for the nested bins.
This logic was trying to match `[name join-alias]` pairs previously, but
that won't work since the later stage field doesn't have a `:join-alias`
on it. This falls back to matching on only the ID or name, but only if
it's unambiguous.
Fixes#40252.
* Make Noah the codeowner for permissions et al
et al = `integrations` (OSS google auth, LDAP and Slack integrations) + `enterprise` (audit, advanced SSO, advanced config stuff). `enterprise` also has LLM which doesn't really have a codeowner now, but it also doesn't need one.
* Correct the paths and the scopes
* Bump the question schema and update it with the new events
* Enable snowplow tracking on the frontend
* Add E2E test for tracking snowplow events related to native preview sidebar
* Fix question id type in the schema
* Fix overflow static dashboard
* Fix `frame` event's height for interactive dashboards
* Fix E2E tests
* Fix test names to match the correct GitHub issue numbers
* Use `data-element-id` for consistency
* Make the test selector more consistent
* Clarify that backport/no-backport is an internal thing
A few times per month, external contributors ask about the notion doc they can't access and about the labels they are not expected to set. This is not the experience we should give to people who not only take their time to contribute to metabase, but also actually read the PR template.
(slightly duplicated condition in the beginning and the end of the paragraph — "For those employed [...] If you're not employed [...]" — is on purpose)
* Add linebreaks in PR template
Co-authored-by: Tim Macdonald <tim@metabase.com>
---------
Co-authored-by: Tim Macdonald <tim@metabase.com>
* Command Palette UI adjustments
* move search to end of results list
* ellipsis long command palette items, min icon size and showing debounced text
* setting fixed width. adjusting admin result icons and text
* fixing e2e test
* Use MLv2 to determine whether a query can be previewed
* Add E2E repro for #40608
* Expand E2E repro
* Fix test
* Make sure the step is active and visible before offering to preview its query
* Expand E2E test
* Address review comment - use `getNotebookStep` helper
* Do not put js code into page title
* Typecheck
* Add e2e test
* Update e2e/test/scenarios/question/document-title.cy.spec.js
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add login page illustration (#39299)
* Move Metabot setting to a new place
* Add setting for login page illustration
* fix UI styling
* Fix not able to upload the same file twice
* Update the admin > appearance > conceal metabase copy
* Prepare the component for tests
* Add some unit tests
* Prevent settings not saved sometimes
After setting the value to/from `custom` one of the setting could be
failed to save, I suspect this happens because the race condition from
trying to save the setting values relatively at the same time.
* Make unit tests easier to read
* Add all unit test test cases
* Add basic E2E test
* Show fallback icon for broken images
* Make use of the proper component
* Update the comment to affect the reality
* Add an error message when file size is over 2MB
* Show an error message when the file size > 2MB
* Add the error message tests
* Preserve the preview image aspect-ratio
* Detect corrupted images
* Fix failing unit tests
* Address feedback: Type
* Feedback: Create meaningful variables
* Feedback: Make selector API more sensible
* Prevent race condition
Makes sure we save the settings completely before making other actions
that also change settings otherwise, there could be race conditions.
* Feedback: tighten up a type
* Feedback: Fix inconsistency callback calls
* 2. Add the rest 3 illustration settings (#40487)
* Add the rest 3 illustration settings
* Don't make the X icon feel lonely
* Fix illustration widget saving
* Fix type error
* 3. Update the select UX to match new design (#40491)
* Update the select UX to match new design
* Move info dot next to file upload button
* Simplify IllustrationWidget API
* Update setting title copy
* Clean up test comments
* Feedback: Clarify `any` type
* Feedback: Use shorthand object creation
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Feedback: Use global CSS glass instead of inlining values
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Feedback: Remove unnecessary curly brackets in JSX
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* 4. Connect illustrations to new setting values (#40139)
* Wire landing page illustration to the setting value
* Wire the unsubscribe page to the setting value
* Prepare no rows error to be wired to the setting value
* Wire no question results illustration to the setting value
* Wire no search results illustration to setting value
* Fix error from fixing conflict
* Fix required unnecsseary prop on Image
* Use Image component
* Link dashboard card no results image to the setting value
* Move the no data results to more a generic directory
* Link collection picker no results image to the setting value
* Make illustrations more accessible
* Fix failed tests
* Test unscribe page illustration + test no illustration
* Add landing page tests
* Simplify Cypress selectors
* Refactor: Rename settings to match the new requirements
No question data -> No Data
No search results -> No object
These changes are made so, these 2 settings are generic enough to
include other illustrations later on.
* Add no data illustration test
* Add no object illustration tests
* Fix unsubscribe page background style
* Fix not able to build static viz
* 5. Follow-up on milestone 1 (#40710)
* Update custom background illustration scale behavior
* Remove the need to put an SVG file in static folder
* Fix test from rebasing off master with @testing-library/user-event upgrade
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Send creator emails every week
mod the email by 52 instead of by 12 and use the current week as an
anchor
```clojure
user=> (require '[java-time.api :as t])
nil
user=> (let [every-day (take 1000 (t/iterate t/plus (t/local-date 2024 1 1) (t/days 1)))
f (fn [d]
(let [wf (java.time.temporal.WeekFields/of (java.util.Locale/getDefault))]
(.get d (.weekOfWeekBasedYear wf))))]
(sort (keys (frequencies (map f every-day)))))
(1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
40 41 42 43 44 45 46 47 48 49 50 51 52)
```
```clojure
user=> (metabase.util.cron/cron-string->schedule-map "0 0 2 ? * 7")
{:schedule_minute 0,
:schedule_day "sat",
:schedule_frame nil,
:schedule_hour 2,
:schedule_type "weekly"}
```
* fix typo in creator sentiment email, clean up a bit
typo: had `:verison` instead of `version`.
And while I'm in there, make the payload all in one file and have the
email namespace json->bytes->b64-encode so the data is all in one place
and the setting that allows it to be present or not is co-located.
* Move all notebook containers styles to one place
* Make NotebookStep wide again
* Tweak sidebar width and breakpoints
* Remove top margin from NotebookSteps
* Stop line wrapping in an ace editor
* Uniform width between step header and step body
* Remove stray styles
* Scroll content independently
* Use named export
* Use Mantine button for "Visualize"
* Convert notebook step width to rem
* Adjust the position of the step close button
* Adjust notebook step max-width to 1200px
* Remove the `cx` where not needed
* Use HTML 5 `aside` element for the sidebar
* Use Mantine responsive styles
* Fix E2E test
* Add repro test for a model that uses another model and both have custom column with the same name
* Skip test
* Update e2e/test/scenarios/models/reproductions/39150-model-with-same-custom-column-name.cy.spec.js
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Update e2e/test/scenarios/models/reproductions/39150-model-with-same-custom-column-name.cy.spec.js
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Convert to ts and add question
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Migrated .PopoverContainer, .PopoverBody
* Make global CSS more prominent
* Migrate 2 classes
* Remove unused class
* Migrate .PopoverBodyWithArrow
* Remove unused class
* Migrate .PopoverBodyMarginBottom
* Remove unused class
* Migrate .PopoverFooter
* Remove unused class
* Preserve classes order for easier review
* Fix unit tests
* Add E2E test
* Move `getDatasetQueryParams` to `data-fetching.js`
* Use `null` as a parameter value when missing
* Fix missing default `datasetQuery`
... among other things.
The order of arguments was wrong here, causing the default
implementation of `post-process-collection-childre` to be called. The
implementation for `:card` ends up removing the `:dataset_query`
entirely, which is what was intended.
* Cards with Results Should Show up in Subscriptions
Fixes: #40726
This PR fixes a bug where setting 'Hide thie card if there are no results' in a Dashcard's viz-settings would hide the
card even when there were results.
* adjust test to show that the setting is respected
Before, the setting would always hide the card. Now, the card is only hidden when there are no results. This test sets
up 4 cards: 2 are normal, 2 have no results, but 1 has the setting to hide empty. THerefore, we should expect to see 3 results.
* upgrades @testing-library/user-event to v14, updates test to accomidate events always returning promises and other breaking API changes
* Fixing up broken suites. Half way
* rest of tests
* green plz
* Fix new entries of userEvent usage
* unit test adjustments
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
* Make Formatting on Exports Optional
Formatting on exports can be turned off via a middleware key:
`:format-export?` which will control if the 'app style' formatting will be applied to the exported results.
`:format-export? false` will return the file as in previous Metabase versions.
* format_export's default value provided in the QP middleware
Should still default to formatted for v. 49, as the formatting is intended as a feature. This default value should be
provided in a central place, so is added to the middleware and is therefore not necessary for the API to provide a default.
* weird indentation
* Add format_export param to dashboard dashcard query endpoint
* Add format_export option to embedded dashboard downloads endpoint
* Move default true value to the streaming-results-writer impls
* Default true but for real this time
* Add format_export to the public Question endpoint
* Add test to card api
* Dashboard exports endpoint test
* Add test to dashboard embeds
Test still fails and I don't know why yet
* the dangers of println debugging....
Well, I was passing a string false the whole time. Turns out the problem was me
* Public Card downloads endpoint now has format_export and test
* export_format also works for adhoc queries using api/dataset
* Let defendpoint do the boolean parsing
* Embed endpoint should now work too.
* Unify format-rows and format-export middlewares
* Sneaky endpoint!
* In xlsx, always parse temporal strings so that viz-settings formatting is applied
* Snuck in an incomplete change by accident. removed it.
* Can now specify format/no format on subscriptions
* Fix and move migration
* default format rows to true in attachments
* Ok, I think that should actually work for subscriptions.
* Change endpoints to use format_rows instead of confusing format_export
* format_rows everywhere
* Fix more test failures
* redef'd function had wrong signature
* Make format export optional (FE part) (#40643)
* Refactor download URL builder code
Need to handle query `params` and request `body` separately
* Export `useHover` from `metabase/ui`
* Add unformatted export for dashboard cards
* Add unformatted export for saved questions
* Fix POST body encoding
* Add unformatted export for ad-hoc questions
* Ensure `format_export` is always false for Excel
* Add unformatted export to public questions
* Add unformatted export to embedded questions and dashboards
* Fix behavior for PNG format
* Add tests for `QueryDownloadPopover`
* Fix e2e downloads helper (allow URL query params)
* Rework alt key handling
* Add e2e test
* Switch to `format_rows` instead of `format_export`
* Add unformatted export option to subscriptions
* Fix `isHoldingAltKey` initial state
* Rework e2e test to use Total column
* Fix initial checkbox state for subscriptions
* Use "Alt" in tooltip label for Windows/Linux
* Show the subscriptions toggle only for CSV attachments
---------
Co-authored-by: adam-james <21064735+adam-james-v@users.noreply.github.com>
* Address review feedback
---------
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
* Fix warning about passing a ref to a function component
- this didn't work and fortunately was not used (anymore) at all
* Move comment where it belongs
* Fix warning that functions cannot be react children
* Parse wildcards in native queries
Add query_field.direct_reference
* Do away with old Model refs in Card model test
* Do not show stale cards with a select *
* Upgrade node to 18.20
* Trigger CI
* Try 20 again
* Revert "Try 20 again"
This reverts commit 8f4b39e7a6.
* Revert change
* Add nvmrc to the frontend_sources list
Fixes#40328
[Our documentation](https://www.metabase.com/docs/latest/people-and-groups/managing#group-managers) states that:
> Group managers can:
>
> - View all people in the Admin settings > People tab.
This fixes enforcement to be aligned with the documentation. This behavior makes sense, because as the docs also state, Group Managers should be allowed to *add* people to the group they manage, which they can only do if they can see those people!
Initially, I also removed a faulty test, which was described as:
```
;; Non-segmented users are allowed to ask for a list of all of the users in the Metabase instance. Pulse email lists
;; are an example usage of this. Segmented users should not have that ability. Instead they should only see
;; themselves. This test checks that GET /api/user for a segmented user only returns themselves
```
but actually failed to test this in a relevant way (because it was testing the `GET /api/user` endpoint rather than the `GET /api/user/recipients` endpoint). The test had continued to pass only because the user was the only member of the group they managed.
I initially thought this behavior wasn't desired, but as it turns out, it *is* in fact documented behavior to disallow sandboxed users from seeing any email suggestions. Investigating, I found that a bug was allowing sandboxed users to see all users on the `/api/user/recipients` endpoint if the user-visibility setting was set to `:all`.
Thus, the second commit fixes the bug and re-adds the (fixed) test. A sandboxed user now only sees their own user when hitting `/api/user/recipients`.
* Trigger the query limit change on blur
* Use named export
* Add a unit test
* Fix E2E tests
* Fix unit tests
* Additional E2E test fix
* Address review comments
* Revert "Remove Models from Browse data (#40108)"
This reverts commit 45a2e8689f.
* Make rtl friendly
* Add 'no description' for blank-description models
* Fix vertical position of collection header toggle
* Move FixedSizeIcon into metabase/ui and increase height of empty state
* Move FixedSizeIcon back into BrowseModels.styled.tsx
* Localize 'No description'
* Add comment
* Fix type issue
* Use expression fix on both pk-ref and value-ref
The gist: inside the query expressions are referred to as [:expression
"foo"]. But as a nested query, it's just another field, and the fact
that it was an expression is irrelevant and should not leak from that
stage. So "downstream" should just be [:field "foo" base-type].
Oddly, if you just fix one field or the other, the query would be
"valid", would log an error, but no valid values would be returned.
It seems like the invalid expression reference was just dropped so we
only selected a single column and those were removed (we run `(filter
valid-tuples?)` as we only want valid key/value pairs).
So let's add a `(> (count values) 0)` which was failing previously.
* Fix indexing error
When updating the model index, it diffs the existing values vs the new
values and deletes appropriate and adds the new values. But it was using
the wrong column when updating the model index values. It used `pk_ref`
but that table uses `model_pk`. This led to the following errors on
stats:
```sql
select pk_ref, value_ref, state, error from model_index where state = 'error'
pk_ref,value_ref,state,error
"[""field"",563028,null]","[""field"",563018,null]",error,"ERROR: column ""pk_ref"" does not exist Position: 68"
"[""field"",534030,null]","[""field"",534037,null]",error,"ERROR: column ""pk_ref"" does not exist Position: 68"
"[""field"",545945,null]","[""field"",545948,null]",error,"ERROR: column ""pk_ref"" does not exist Position: 68"
```
Now those errors will be fixed and the model indexes will correctly
track the values
* Bump indexing threshold to 25,000
Had been limited to 5,000 rows of model indexing. This was a made up
number as we went into this endeavor. But it's proving to be far too
small. Bumping to 25,000
* Parititon deletions and inserts
With a higher limit need to ensure that we don't send too large of a
query.
```clojure
model-index=> (t2/insert! ModelIndexValue
(map (fn [id]
{:name (str (gensym "value"))
:model_pk id
:model_index_id 1})
(range 25000)))
Execution error (PSQLException) at org.postgresql.jdbc.PgPreparedStatement/<init> (PgPreparedStatement.java:105).
PreparedStatement can have at most 65,535 parameters. Please consider using arrays, or splitting the query in several ones, or using COPY. Given query has 75,000 parameters
```
* move comment and fix test failure explanation
comment should be next to the number? check. and also need to use
`mc/explain` which returns info rather than `mc/validate` which returns
a bool:
```clojure
;; bad
model-index-test=> (let [values [[1 "foo"] [2 "foo"] ["foo" "foo"]]]
(-> (mc/validate [:sequential [:tuple number? string?]] values)
(me/humanize)))
nil
;; good
model-index-test=> (let [values [[1 "foo"] [2 "foo"] ["foo" "foo"]]]
(-> (mc/explain [:sequential [:tuple number? string?]] values)
(me/humanize)))
[nil nil [["should be a number"]]]
```
* Split from #40146: only the filter changes
* Don't include format-rows changes here.
* Test fixes 🔧
* Test fix 🔧
* Misc test fixes 🔧
* Test fix 🔧
* Tweak SQL Server fix
They were showing "break out by" and quick filters, when they should be
showing "break out by" and "See these Orders".
These are the `pivot` and `underlying-records` drills, internally.
Fixes#40174.
* Show SQL in a notebook sidebar [PoC/task-1] (#40279)
* Display SQL in the notebook sidebar
* Remove unused modal
* Rename the component to `SQLPreviewSidebar`
* Hide the close button in a SQL sidebar
* Rename the component to `NativeQueryPreviewSidebar`
* Toggle a native query sidebar preview [PoC/task-2] (#40327)
* Use Mantine components where possible
* Rename the button to `ToggleNativeQueryPreview`
* Toggle the native query preview sidebar on and off
* Update the button tooltip based on the sidebar state
* Shorten var names
* Read the state directly from the store
* Re-enable the button that converts a query into its native counterpart [PoC/task-3] (#40362)
* Re-enable the button that converts a query into its native counterpart
* Use Mantine button
* Remove references to the modal
* Fix tests
* Use RTKQuery for API calls in the native preview sidebar [PoC/extras] (#40430)
* Use RTKQuery for API calls
* Fetch question inside the component
* Export named component
* Add type guard
* Use RTK query for the native dataset endpoint
* Improve semantics around error message
* Improve native sidebar basic styling [PoC/task-4] (#40463)
* Refactor sidebar to use `AceEditor`
* Fix the query display condition
* Style the code as in the native query editor
* Style the toggle icon
* Make native preview usable on small screens
* Cover "Show SQL/native query in a sidebar" with E2E tests [PoC/task-5] (#40540)
* Fix query formatting for Mongo
* Add initial smoke tests
* Cover "converting a question to SQL" scenarios
* Cover "converting a question to a native query" scenarios
* Add test that covers behavior on small screens
* Adjust the viewport width to counter CI's behavior
* Revert "Adjust the viewport width to counter CI's behavior"
This reverts commit 370a76b621.
* Add E2E repro for #38181
* Add repro for #40557
* Try a workaround for the failing test in CI
* Show empty sidebar if query has no data source
* Add E2E coverage for an empty sidebar state
* Sort official collections first in API endpoints
For both `/api/collection/tree` and `/api/collection/items`, sort
collections with official collections coming first.
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Fixes#40383
If a native query parameter is set up with a string widget type that
supports a `case-sensitive` option, and that parameter is tied to a
dashboard filter that did not support that option, the option was still
leaking into the filter as an option where options were not expected.
For instance the native query parameter used `string/contains` and the
dashboard used `string/=`, the QP would try to build a filter like `[:=
field value {:case-sensitive false}]` which is an illegal filter.
* In #37160 we added a migration to convert all datetime columns to
timestamp with time zone by using a fixed list of what to convert. but
it turns out some customer instances has a different list. So this PR
removed the fixed list and use a query to get a list of all datetime
columns.
* Remove unused marginalia aliases
* Upgrade deps.edn
From `neil deps upgrade`
* Un-bump jetty-server
* Downgrade H2 due to many problems
* Dep bumping for Athena
* Dep bumps for BigQuery
* Dep bumps for Presto
* Dep bumps for Redshift
* Dep bumps for Snowflake
* Dep bumps for SQLite
* Upgrade to Mongo 5
* Bump next.jdbc
* Funnel Chart Static-Viz should handle row data more robustly
The row data coming into the funnel chart render method can evidently have a few slightly different shapes, and may
need to be 'coordinated' with the :funnel.rows key from viz-settings (names and order are specified there).
This is a WIP to make the method more robust to different row data shapes. For example:
[[1 100] [2 200]] should work as well as [["A" 100] ["B" 200]] and correctly handle the viz-settings
WIP because I need to add a test or two, try to get a handle on the exact schema that is allowed for 'row shape' and
the funnel.row viz key, and see if there are failing cases that I haven't considered yet.
* Funnel rows should now work more effectively with possible raw-rows
* Add a test that checks the success of the render and correct order
* Update src/metabase/pulse/render/body.clj
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
* Update src/metabase/pulse/render/body.clj
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
* Addressing feedback from the review
- renamed build-funnel-rows to funnel-rows
- used 'funnel-viz' as the binding for the {:key "asdf" :name "asdf" :enabled true} maps from the viz settings
- added docstring to the function to try clarify its 2 branches (keys vs indices on the rows)
---------
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
* Trend Chart Viz Should Use Proper Card execute in Rendering
WIP
Fixes: #39854
The trend chart follows a code path shared with multi-series static-viz. This means there was an implicit expectation
that a dashcard is associated with the card at all times. This isn't true for alerts, so the code is fixed to remove
the failure.
* Add a test
* add missed :breakout that actually made the test work
oops
* Adding comments to try clarify what's going on
* sort expression functions by use case
* custom expression edits
* formatting cleanup
* fix link
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* address comments
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Often slo won't be setup, but we should still delete the session
* test that we delete the session even when "/handle_slo" is not called
* cleanup test
* take the cookie from the session, not as a parameter
* /api/collection/:id/items gets `here` and `below`
During postprocessing of `collection` children retrieved by
`/api/collection/:id/items`, annotate each of the children with `below`
and `here` keys representing the presence of questions, models, or
collections at this level of the hierarchy or below it.
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Fixes: #40306
Our datetime formatter relies on the `java-time.api`, for which there are many different, sometimes confusing, formatter patterns: https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatterBuilder.html#appendPattern-java.lang.String-
In this case, 'YYYY' is a week-of-year style year, which calculates which week a date falls into before returning the
year. Sometimes days near the start/end of a year will fall into a week in the wrong year.
For example, apparently 2023-12-31 falls into the 1st week of 2024, which probably not the year you'd expect to
see. What we probably do want is 'yyyy' which calculates what day of the year the date is and then returns the year
based off of that instead of the week number.
For an explanation, you can check out this SO answer: https://stackoverflow.com/a/46395342 provides an explanation.
* Wow
* Test fix 🔧
* Fixes
* Actions should use strings for column names (fix :update-row and :create-row normalization)
* MLv2 schema should check against keys for the other query type
* Ok, have I fixed things?
* More fixes 🔧
* Fix indentation
* Another round of test fixes. 🔧
* Hopefully the last few test fixes 🔧
* We need to test normalization for queries that have keyword keys as well.
* Fix Cljs i18n namespaces
* Sort namespaces
* Only test against H2
* Rename `metabase.mbql` to `metabase.legacy-mbql`
* Fix Kondo warnings
* Test fixes 🔧
* Register MBQL clause schemas and test fixes 🔧
* Test fixes and PR feedback
* Test fix
* Remove the normalization tests
* Test fixes 🔧
* Fix kondo
* Fix import
* Another fix 🔧
* Merge
* FIXES
* Add another missing REQUIRE
* E2E support for CRUD + running pMBQL queries; new MLv2 normalization
* More work on improved normalization
* Update schema references
* Test fixes 🔧
* Move the MBQL parameters schema
* Performance improvements
* Improved pivot code etc.
* Clean namespaces
* Fixes 🔧
* Wow
* Test fix 🔧
* Fixes
* Actions should use strings for column names (fix :update-row and :create-row normalization)
* MLv2 schema should check against keys for the other query type
* Ok, have I fixed things?
* More fixes 🔧
* Fix indentation
* Another round of test fixes. 🔧
* Split pivot stuff off
* Hopefully the last few test fixes 🔧
* We need to test normalization for queries that have keyword keys as well.
* Fix Cljs i18n namespaces
* Sort namespaces
* Only test against H2
* Test fixes 🔧
* Register MBQL clause schemas and test fixes 🔧
* Test fixes and PR feedback
* Test fix
* Remove the normalization tests
* Test fixes 🔧
* Fix kondo
* Test fix
* Another fix
* Test fix 🔧
* decrypt and read values from airgap token
- Adds notion of "AirgapToken"
- Adds max-users and company optional fields to `TokenStatus`
- Adds branch to `fetch-token-status*` for handling airgapped tokens
* Add support for handling airgap tokens
- validate airgap token valid-thru date
* enforce user creation limits using airgap token
* fix airgap token reads + fill in the token data
* in oss mode don't try to read the airgap_ token
* use airgap-token? helper
* tighten the user-creation logic
- make the check correct
- account for archived users
Co-authored-by: John Swanson <john.swanson@metabase.com>
* check airgap user count
setting premium-embedding-token
app startup
* move decryption code into ee namespace
* add some tests for token decryption
* fix tests and add tests
* move ee features into ee tests
* add typehint + warn on reflection
* add a check for missing public key resource
* respond to review
- add docstring to AirgapToken
- remove outdated comment
- fix off-by-1 error
* valid-now? takes a TokenStatus instead of a :map
* revert usage of mt/with-temporary-setting-values
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* wip
* refreshes list through tag invalidation rather than prop drilling refresh list fn
* invalidate count, added invalidation utils, general clean up
* custom fetcher that makes use of our api service
* reorganizes files and impls util to createEntityApis more easily
* documents different approaches
* moves main file
* wip
* stash
* clean up
* further clean up
* type annotations clean up
* fixes api key unit test
* pull out api keys feature from branch
* moves files core rtkquery files into metabase/api
* replace trivial loginHistory entity
* export hook too
* fix rebase
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
In a working meeting, some prompt adjustments were tried. This change felt like the best incremental improvement,
considering feedback from users on stats.
We attempted to guide the LLM to return descriptions that:
- don't state the chart type as if its *very* important (we can see what it is already, so its not that important)
- do call out caveats and important specifics as implied by the question's filters, aggregations, and breakouts
- are descriptive but not verbose
- try not to state caveats too strongly
* Use Mantine Flex for the NotebookContainer
* Rename the component to `NotebookContainer`
* Convert to TS
* Explain the delay
* Use regular div
* Destructure the import type
* Simplify styling
* Shorten the prop name to `isOpen`
* do not pass null values to link url
* update null check to include query builder nulls replaced with "(empty")
* add test to verify behavior
* replace magic string with constant
* fix breakout in cypress spec
This issue was previously punted, because the schema for aggregation
clauses allowed the tag to be `:any`. Now it's a short list of permitted
arithmetic expressions, but that list is too short and excludes things
like `:round`.
This change searches for an aggregation clause transitively in the
expression, and so allows any arithmetic expression.
* Move search-related snowplow tests to a separate file
* Update test names
* Move filters-related search tests to a separate file
* Extract search helpers into a separate file
* Revert test rename
* Export named `NativeCodePanel`
* Export named `NativeQueryModal`
* Rename `NativeQueryModal` to `NativeQueryPreview`
* Do not use styled components in the `NativeQueryPreview`
There were two problems here:
- one endpoint was sorting in the database, but not putting 'analytics'
last.
- one endpoint was sorting in the database, but then sorting again in
clojure, which had subtly different behavior for special characters
Fixes#39965
* Use proper :display when using 'zoom' in Dashboard
Fixes: #38307
WIP, I want to change at a different call site so that defaults are still used in case there is no :display set.
* Lock Display so that drill through doesn't overwrite it with defaults
* Lock display only when drill action coming from a dashboard
* Use isDashboard properly
* Add test to confirm Display is locked from Dashboards
* minor changes to the test
* Lock Display specifically for zoom-in-binning and timeseries
* Fix wrong string quotes
* Try a simple change to see if the e2e test will pass in CI
* Fix test
* Minor refactor
* Wait for dashcard queries in e2e test
* Wait for loading spinner to be gone
* Fix type errors
* Fix incorrect dashboard ID used in test
---------
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
* Add the initial test for the empty app db
* Add reproduction for #25142
* Add test that covers only saved questions
* Cover scenarios with a table as a source
* Add tests that cover nested questions and models
* Add test for simple model's source table
* Remove intercepted schema routes
* Improve assertion
* Fix the test failing on enterprise instance
* Add E2E reproduction for #40223
* Remove superfluous code block
* Avoid casting
* upgrade to react 17
* fix popovers
* upgrade formik to fix types
* fix sso nested form failures
* fix WindowModal failing unit test
* update RowChart test to accommodate changes to event pooling in react 17
* updates input blur change components to use layout effect to ensure input ref is available at the time of unmount
* fixes failing map visualization e2e test
(error!form(format"destructive functions like %s are not allowed inside a ^:parallel test or test fixture. If this should be allowed, add it to the whitelist in .clj-kondo/hooks/clojure/test.clj"
qualified-symbol)))))
(walk[form]
(fform)
(doseq [child(:childrenform)]
(walkchild)))]
(walkform)))
(defn- deftest-check-parallel
"1. Check if test is marked ^:parallel / ^:synchronized correctly
2. Make sure disallowed forms are not used in ^:parallel tests"
[{[_test-name&body]:children, :as_node}]
(let [test-metadata(:metatest-name)
metadata-sexprs(map hooks/sexprtest-metadata)
combined-metadata(transduce
(map (fn [x]
(if (map? x)
x
{xtrue})))
(completingmerge)
{}
metadata-sexprs)
parallel?(:parallelcombined-metadata)
synchronized?(:synchronizedcombined-metadata)]
(when (and parallel?synchronized?)
(hooks/reg-finding!(assoc (meta test-name)
:message"Test should not be marked both ^:parallel and ^:synchronized"
:type:metabase/validate-deftest)))
;; only when the custom `:metabase/deftest-not-marked-parallel` is enabled: complain if tests are not explicitly
;; marked `^:parallel` or `^:synchronized`. This is mostly to encourage people to mark everything `^:parallel` in
;; places like `metabase.lib` tests unless there is a really good reason not to.
(when-not (or parallel?synchronized?)
(hooks/reg-finding!
(assoc (meta test-name)
:message"Test should be marked either ^:parallel or ^:synchronized"
:message(format"The name of this %s should end with `!` because it contains calls to non thread safe form `%s`. [:metabase/test-helpers-use-non-thread-safe-functions]"
(error!form(format"destructive functions like %s are not allowed inside a ^:parallel test or test fixture. If this should be allowed, add it to the whitelist in the Kondo config file [:metabase/validate-deftest]"
qualified-symbol)))))
(walk[form]
(fform)
(doseq [child(:childrenform)]
(walkchild)))]
(walkform)))
(defn- deftest-check-parallel
"1. Check if test is marked ^:parallel / ^:synchronized correctly
2. Make sure disallowed forms are not used in ^:parallel tests"
[{[_test-name&body]:children, :as_node}config]
(let [test-metadata(:metatest-name)
metadata-sexprs(map hooks/sexprtest-metadata)
combined-metadata(transduce
(map (fn [x]
(if (map? x)
x
{xtrue})))
(completingmerge)
{}
metadata-sexprs)
parallel?(:parallelcombined-metadata)
synchronized?(:synchronizedcombined-metadata)]
(when (and parallel?synchronized?)
(hooks/reg-finding!(assoc (meta test-name)
:message"Test should not be marked both ^:parallel and ^:synchronized"
:type:metabase/validate-deftest)))
;; only when the custom `:metabase/deftest-not-marked-parallel-or-synchronized` is enabled: complain if tests are
;; not explicitly marked `^:parallel` or `^:synchronized`. This is mostly to encourage people to mark everything
;; `^:parallel` in places like `metabase.lib` tests unless there is a really good reason not to.
(when-not (or parallel?synchronized?)
(hooks/reg-finding!
(assoc (meta test-name)
:message"Test should be marked either ^:parallel or ^:synchronized"
> If you are merging into master, please add either a `backport` or a `no-backport` label to this PR. You will not be able to merge until you do this step. Refer to the section [Do I need to backport this PR?](https://www.notion.so/metabase/Metabase-Branching-Strategy-6eb577d5f61142aa960a626d6bbdfeb3?pvs=4#89f80d6f17714a0198aeb66c0efd1b71) in the Metabase Branching Strategy document for more details.
> For those employed by Metabase: if you are merging into master, please add either a `backport` or a `no-backport` label to this PR. You will not be able to merge until you do this step. Refer to the section [Do I need to backport this PR?](https://www.notion.so/metabase/Metabase-Branching-Strategy-6eb577d5f61142aa960a626d6bbdfeb3?pvs=4#89f80d6f17714a0198aeb66c0efd1b71) in the Metabase Branching Strategy document for more details. If you're not employed by Metabase, this section does not apply to you, and the label will be taken care of by your reviewer.
while ! curl -s 'http://localhost:3000/api/health' | grep '{"status":"ok"}' > /dev/null; do
if ! kill -0 $(cat metabase_pid) 2>/dev/null; then
cat metabase.log
echo "Metabase process has exited. Check that your migration changes work on an instance that has been previously initialized on your PR's target branch."
@${githubUsername} You have modified embedding SDK code. Please make sure the PR title follows [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) style.
Here are all the supported types that will be shown in the changelog:
- \`feat\`
- \`fix\`
- \`perf\`
These types \`docs\`, \`style\`, \`refactor\`, \`test\`, \`build\`, and \`ci\` are also encouraged for non-changelog related tasks.
Please also make sure to include \`sdk\` scope, otherwise, the changelog will not include this task.
"text": `:broken_heart:Cross-version tests are failing on ${BRANCH}`,
"text": `:broken_heart:Cross-version tests are failing on ${REF_NAME}`,
"emoji": true,
}
},
@@ -149,7 +157,7 @@ jobs:
"type": "section",
"text": {
"type": "mrkdwn",
"text": `Commit <https://github.com/metabase/metabase/commit/${BREAKING_COMMIT}|${BREAKING_COMMIT.slice(0,7)}> has failing <${FAILED_RUN_URL}|${FAILED_RUN_NAME}> tests on the <https://github.com/metabase/metabase/commits/${BRANCH}|\`${BRANCH}\`> branch.`
"text": `See <${FAILED_RUN_URL}|failing run details>.`
# TODO: Add a version validation, so the workflow won't fail when publishing to npm
sdk_version:
description:'SDK version (e.g. 0.1.3)'
type:string
required:true
git_ref:
description:'A git commit hash that will be used to build the SDK'
type:string
required:true
concurrency:
# We want to ensure only one job is running at a time because
# there could be a conflict when updating the readme file.
group:${{ github.workflow }}
cancel-in-progress:true
jobs:
check-git-tag:
runs-on:ubuntu-22.04
timeout-minutes:20
env:
tag:embedding-sdk-${{ inputs.sdk_version }}
steps:
- name:Check out the code using the provided ref
uses:actions/checkout@v4
with:
ref:${{ inputs.git_ref }}
fetch-depth:0
fetch-tags:true
- name:check tags
run:|
if [ $(git tag -l | grep $tag$) ]
then
echo "::error::Tag '${{ env.tag }}' already exists. If you expect to run this workflow with the same tag, please remove the tag first and rerun this workflow again."
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
