* add bug report to command palette and profile menu
add bug report to command palette and profile menu
* refactor and update diagnostic window modal
* add analytics to track bug reporter modal activity
* add file id to to the upload-file method
* add the ability to collect browser information for diagnostic window
* implement feedback from PR comments
* Update frontend/src/metabase/components/ErrorPages/ErrorDiagnosticModal.tsx
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* implement feedback from PR comments #2
* rename "file a bug" to "report an issue"
* resolve conflict
* fix lint issues
* fix bad merge conflict resolution
* fix failing onboarding tests
---------
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* wip: better documentation for developers that need to run the sdk
locally
* morning fixes
* remove developers-guide/sdk in favor of dev.md in the sdk folder
* add note about using the sdk with link to the docs
* fix mising word
* addressed feedback
* added notes about the envs for sdk e2e test
* Prevent newly opened items from expanding outside viewport
* Expand the unti test coverage
* Use the existing helper
* Change scroll view to center
This works better because it ensures that the top bar doesn't cover the element.
* Update frontend/src/metabase/home/components/Onboarding/Onboarding.tsx
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Extract the `stopVideo` function
* Fix the type name
* Set the top scroll margin on the `main` element
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Closes#49682. Closes#49683.
Updates `lib.convert` and legacy MBQL normalization to allow
`:ident`s, and fixes some QP middleware and utils so they can
still match up refs correctly.
No user-visible impact.
Newly created queries will have `:ident`s generated for joins,
aggregations, etc. by the previous PR. This starts saving them
into appDBs. It is still acceptable to have missing idents at
this stage, including for new queries created without the lib.
Pre-existing clauses don't have refs added yet.
* Fix loading in segments
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Our revision system has a race condition in the case of two concurrent requests to modify the same object. You can reproduce the exact issue here by opening two `psql` shells. In each one, run
```
begin;
insert into revision (model, model_id, user_id, timestamp, object, most_recent, metabase_version) values ('Dashboard', 1, 1, '2024-01-01T00:00:00Z', '{}', true, 'v1.0.0') returning id;
update revision set most_recent = false where model='Dashboard' and model_id = 10 and id != [[the id from above]];
```
after running all of the above, `commit` the transaction and note that **two** revisions are now marked as `most_recent`.
This is the way our revision system works: `t2/before-insert` sets `most_recent=true`, while `t2/after-insert` updates all the *other* revisions with `most_recent=false`.
The issue here is that when each transaction is running, it doesn't see
the uncommitted result of the other transaction (the new revision), so
the UPDATE does not set `most_recent` to `false` for either new
revision.
The solution here is to just select all existing `revision`s for this
model `FOR UPDATE`. This effectively locks the revisions for a single
entity, so we can't concurrently update revisions for the same entity.
* Add EntityObjectLoaderRtkQuery boilerplate
* Incorporate EntityType logic into the new component
* Improve comment
* Add childrenProps
* Move component into its own directory, extract types to a separate file
* Add entityObjectLoaderRtkQuery and temporary mappings
* Hook up the useGetQuery hook (tables only)
* Pass all the props to children
* Add Playground
* Gather more info in the Playground
* Integrate RTK with redux store
* Adjust props passed down
* Improve typing
* Make the request hook dynamic
* Add missing id in the query
* Handle fetchMetadata
* Change requestType to fetchType
* Add rtk mappings for databases in entity framework
* Use requestType to get request state path
* Enable new component everywhere
* Use legacy loader for entities without rtk mappings
* Improve comment
* Handle fetchMetadataAndForeignTables
* Remove unused function
* Handle reload and remove
* Add TODO
* Add transformResponse option and use it for fetchIdFields
* Format code
* Remove Playground
* Remove unused getUseGetListQuery
* Add rtk mapping for alerts
* Add rtk mapping for dashboards
* Add rtk mapping for groups
* Add rtk mapping for pulses
* Remove options
* Fix permissions tests
* Fix notifications tests
* Add rtk mapping for segments
* Use params instead of body
- It's leftover because this code got reverted after body -> params refactoring for GET endpoint has been merged to master
* Add rtk mapping for snippet collections
* Add TODO
* ESLint
* ESLint
* Update TODO
* Deprecate entity HOCs
* Update TODO
* Revert other entities for now
* Revert other entities for now
* Bring tables back
* Improve todos
* Add comment
* Improve comment
* Remove TODO
* Use reload param
* Remove properties prop
This was used to short-circuit and avoid triggering api calls if we already had an object with given attributes in cache.
With RTK query now managing the cache, we don't want to reproduce this behavior.
* Remove reloadInterval
This is used in EntityListLoader, not EntityObjectLoader
* Don't use setRequestPromise action
* Fix deprecations
* Format code
* Fix flake :(
* Try to fix flake
* Revert "Try to fix flake"
This reverts commit 6408a22a37.
* Try to fix flake
* Fix UseQuery type
* Implement useGetMetadataAndForeignTables
* Fix useMemo and move things down
* Fix typo
* Bring back original hack
* Move file
* Enable RTK for alerts
* Undeprecate a test
* Enable RTK for dashboards
* Port Segments to use new EntityObjectLoaderRtkQuery (#50635)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* [WIP] Port Groups to use new EntityObjectLoaderRtkQuery (#50631)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Fix failing test
* Enable RTK for dashboards
* Port Snippet collections to use new EntityObjectLoaderRtkQuery (#50636)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* Revert "Enable RTK for segments"
This reverts commit ef65fe78f3.
* Enable RTK for snippet collections
* Fix skipToken passing
* Enable RTK for dashboards
* Port Pulses to use new EntityObjectLoaderRtkQuery (#50633)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Fix hook usage
* Enable RTK for dashboards
* Port Fields to use new EntityObjectLoaderRtkQuery (#50639)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Revert "Enable RTK for questions"
This reverts commit b9ad637ae0.
* Enable RTK for segments
* Revert "Enable RTK for segments"
This reverts commit ef65fe78f3.
* Enable RTK for snippet collections
* Revert "Enable RTK for snippet collections"
This reverts commit e1439cb28b.
* Enable RTK for fields
* Account for skipToken
* Fix fetchFieldValues
* Optimize hooks
* Format code
* Fix loading state not kicking in during first render
* Rename
* Add comment
* Use consistent reference name for revisions
* Port Questions to use new EntityObjectLoaderRtkQuery (#50634)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Rename import
* Fix weirdness
- Apparently what makes the difference here is the lack of children render pattern ({childProps => <ComposedComponent {...childProps} />})
* Refactor
* Use isFetching instead of isLoading in EntityObjectLoaderRtkQuery, replace useEffect+useState with useMemo in SavedQuestionLoader
* Change order of HOCs to prevent loading state from redundantly kicking in
* Rename import
* [WIP] Port Persisted Models to use new EntityObjectLoaderRtkQuery (#50853)
* Revert "Enable RTK for dashboards"
This reverts commit dd74f51b63.
* Enable RTK for groups
* Revert "Enable RTK for groups"
This reverts commit bbf6954cfa.
* Enable RTK for pulses
* Revert "Enable RTK for pulses"
This reverts commit cabcc7e297.
* Enable RTK for questions
* Rename import
* Fix weirdness
- Apparently what makes the difference here is the lack of children render pattern ({childProps => <ComposedComponent {...childProps} />})
* Refactor
* Use isFetching instead of isLoading in EntityObjectLoaderRtkQuery, replace useEffect+useState with useMemo in SavedQuestionLoader
* Change order of HOCs to prevent loading state from redundantly kicking in
* Port persisted models
* Port schemas (#50854)
* Port snippets (#50855)
* Port timelines (#50858)
* Do TODOs
* Port timeline events (#50857)
* Port users (#50866)
* Port Collections to use new EntityObjectLoaderRtkQuery (#50869)
* Port collections
* Fix imports
* Drop unused params
* Add missing property
* Fix collection not getting loaded
* Replace useCollectionQuery with useGetCollectionQuery
* Remove useCollectionQuery
* Format code
* Skip request on null id
* Revert "Remove useCollectionQuery"
This reverts commit 9b1068e611.
* Revert "Replace useCollectionQuery with useGetCollectionQuery"
This reverts commit 0c5ee660a3.
* Fix test
* Extract dedicated providePermissionsGroupListQueryTags
* Format code
* Fix test setup
* Get rid of Database.Link usage in TableBrowser
* Fix browseDatabase typing
* Get rid of Collection.Link usage in DashboardMoveModal
* Get rid of Collection.Link usage in QuestionMoveToast
* Style links consistently
* Remove EntityLink
* Use database from the store if it's already there
* Remove filter
* Fix typo
* Use ts-pattern
* Bring spacing back
* Extract components for links
---------
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* BE: Dashboard Questions
Setting this makes the card a "dashboard internal card" that should only
be used within a single dashboard, is moved along with the dashboard, etc.
prevent invalid inserts/updates
POST /api/card for dashboard internal cards
GET /api/collection/:id/items
Search
PUT /api/card/:id
WIP: last bits of card-in-dashboard API
Sticking point here is the copy, I think I need to do a small refactor
to make the code less hairy.
rm line
optionally show DQs for collection items
copying dashboards 🎉
wip
quick fix
Disallow moving dashboard questions & test archive
remove unused columns
fix argument order 🤦
serdes for dashboard questions
serialization baseline tests
Fix tests in `metabase.models.card`
Fix download exports test
Lint migrations file
cljfmt
cljfmt, again
the cljfmt job in CI doesn't run on this branch, it runs on the PR
branch merged with `origin/master` - which is nice, because it catches
cljfmt issues that would happen if you merged, but don't happen on your
branch, but also confusing, because it catches issues like
"metabase.api.common is never used", and then you stare at the file for
a while like "YES IT IS" before realizing what's going on.
migration to add idx for report_card.dashboard_id
don't rollback the `report_card.dashboard_id` bits
a couple missed API changes
- no DQs w/o DashboardCards in search
- no DQs in autocomplete
- fix check for "currently a dashboard question" in card validation
Fix some comments, clean up some existing code
Before, `cards-to-copy` returned just two lists of cards: those to
discard (no permissions) and those to copy. But `maybe-duplicate-cards!`
had some extra logic: if the card was a `model`, don't copy it, just
reference it.
I modified `cards-to-copy` to return three lists instead: cards to copy,
cards to discard, *and* cards to reference. This was initially just used
for making sure we copy Dashboard Questions instead of referencing them
- but I realized we could use the same for Models as well. I moved the
logic for models to `cards-to-copy` instead: if the card is a Model,
return it in the `reference` list instead of the `copy` list.
Fix commented-out test
cljfmt
Optionally include dashboard questions in search
Add a test for failing card name change
Opt. include dashboard questions for autocomplete
Update src/metabase/api/dashboard.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
PR review response
lint migrations
quick impl of dashboard items
copied from collection items. should be firmed up for real going
forward. notable divergences:
- i'm not doing any limit/offset in the query
- i'm not including limit/offset in the response
- no reuse. just grabbed it to unblock nick
```
❯ http get "localhost:3000/api/dashboard/22/items" x-api-key:$API_KEY -pb
{
"data": [
{
"archived": false,
"archived_directly": false,
"collection_id": 20,
"collection_position": null,
"collection_preview": true,
"database_id": 48,
"dataset_query": {
"database": 48,
"query": {
"source-table": 360
},
"type": "query"
},
"description": null,
"display": "table",
"entity_id": "_TmF-T7m_bBVxV_jfaNtM",
"id": 192,
"last-edit-info": {
"email": "dan@metabase.com",
"first_name": "dan",
"id": 1,
"last_name": "sutton",
"timestamp": "2024-09-13T22:13:30.220311Z"
},
"last_used_at": "2024-10-01T17:48:36.128915Z",
"model": "card",
"moderated_status": null,
"name": "Cynthia Example"
}
],
"models": [
"card"
],
"total": 1
}
```
contrast with collection items:
```
❯ http get "localhost:3000/api/collection/root/items?models=card&limit=1" x-api-key:$API_KEY -pb
{
"data": [
{
"archived": false,
"can_delete": false,
"can_restore": false,
"can_write": true,
"collection_id": null,
"collection_position": null,
"collection_preview": true,
"database_id": 4,
"description": null,
"display": "table",
"entity_id": "e_jABu05u9nT9ocgfQWfZ",
"fully_parameterized": true,
"id": 143,
"last-edit-info": {
"email": "dan@metabase.com",
"first_name": "dan",
"id": 1,
"last_name": "sutton",
"timestamp": "2024-08-27T14:11:15.214254Z"
},
"last_used_at": "2024-08-27T14:11:27.138735Z",
"location": null,
"model": "card",
"moderated_status": null,
"name": "Collection + query on sleep question"
}
],
"limit": 1,
"models": [
"card"
],
"offset": 0,
"total": 13
}
```
Clean up last-edit-info and exclude archived cards
before: over-selecting, copy-paste function to clean it up.
after: reuse (bulk-aware) query that annotates with last-edit-info, ALSO
EXCLUDE CARDS
also condensed query a bit. was each field on a separate line for no
real reason.
`location` on dashboards
dashcards are still active for archived DQs
WIP: autoplace dashcards for dashboard cards
more wip for autoplacement
WIP for moving DQs
more WIP on moving DQs/autoplacement
delete old dashcards when moving to new dashboard
populate `here` on dashboards in coll items
reject moving card if in another dashboard
fix silly typo
allow optionally deleting old dashcards on move
WIP
add dashboard details to recents
untested wip
fix auto delete of old dashcards for move
Add another test for DQs getting unarchived
Fix dashboard details on recents, with a test
Archive and unarchive DQs when used/unused
including in revisions
Handle reversions containing unowned DQs
lint
Tests and fixes for dashboard items endpoint
remove stray #p
fix a test
Add more tests for DQs/reversions
Fix/update tests for updating DQs
Don't revert to dashcards with archived cards
This is the existing behavior, we should keep it.
lint
small fixes for recents tests
add more tests
Autoplace Dashboard Questions on creation
Improve autoplace to be like frontend
I'd hacked together something - this actually makes autoplacement work
the same way on the frontend and backend.
fix a couple tests
`delete_old_dashboards` as a query param
Fix autoplace for tabbed dashboards
and add a test
do not autoplace DQs when duplicating a dashboard
This was resulting in us accidentally creating two dashcards for a
duplicated DQ.
add effective location to collections on dash
add dashboard_count to (card) collection items
autoplace on unarchive, better error messages
Exclude DQs from dashboard if archived directly
DOH
workaround a bug with "changing" a type
Fix bug causing dashboard cards to be removed
small refactor autoplacing / removing dashcards
Create Dashboard revisions on autoplace/remove
When we modify Dashboard Questions, we autoplace or autoremove relevant
cards from the Dashboards involved. For example, we remove DashCards
from the old dashboard if we're moving to a new dashboard, and we
autoplace DashCards into the new dashboard.
When we do this, publish an event that the dashboard itself has changed.
This updates the revision history for the dashboard so that you can
revert correctly.
Note that when using the revision history, DQs will automatically be
archived or unarchived as necessary. So e.g. if you manually archive a
DQ, then go to the Dashboard and revert, the DQ will be auto-unarchived.
If you go to the dashboard and remove all cards for a DQ, the DQ will be
archived - if you then manually unarchive it, it'll be autoplaced on its
dashboard again.
Fix bug on card modification
The conditions here were wrong. Say you had a card on dashboard 1 that
was archived. If you specified `{:dashboard_id 1 :archived false}`, it
would be correctly autoplaced on the dashboard. But if you just
specified `{:archived false}`, the "new dashboard ID" is nil.
The logic here is pretty painful, because to figure out the resulting
value for something like `dashboard_id` we can't just get it from the
old version or the changes. Instead, we need to:
- check to see whether it's changing, then
- if it changed, use the version from the update, otherwise
- use the old version
The same is true for `archived`.
After we figure out the old dashboard_id, new dashboard_id, and whether
the card will end up archived or not, we can actually do the checks we
need:
- if we're ending up an unarchived card on a dashboard, autoplace
- if we're moving from one dashboard to another, or moving from a
dashboard to a collection (and the user told us to), auto-remove
offset/limits for dashboard items
Reuse some code between dashboard/collection items
Fix some failing tests in api.dashboard
I was getting stack overflow errors when running these tests. I followed
along with the demonstartion
[here](https://github.com/metabase/metabase/pull/43496) to fix.
cljfmt
add additional tests
Always delete dashcards when archiving a DQ
If we're archiving a DQ, let's delete the dashcards so this appears in
the revision history.
publish revision when autoplacing
Publish dashboard events when cards are autoplaced
new endpoint to get dashboards a card is in
allow moving to a dashboard deleting old dashcards
fix the migrations 😭
fix migration IDs
* work on dedicated dashboard question page
* wip on dashboard page for handling dashboard questions
* hacking linking to dashboard for question and books marks - does not handle navigating to the correct tab yet
* make it easier to change the dashboard_id for hacking
* misc. changes
* action menu and dashbcard action updates
* enables trashing dashboard questions from the dashboard view, enables undoing of that case, updates archive state for dedicated dashboard question page
* wip - mostly done creation flow
* wip
* add question directly on dashboard flow
* significant clean up of fe code, revert some wierd differences between base branch, allows selecting dashboard when duplicating a question
* type fixes
* clean up isDashboardQuestion checks, fixes bug where save question form name and description state would get cleared after selecting a dashboard from the picker, fixed bug where editing a dashboard question and choosing save as new question would only let you save it to the dashboard the original question was on
* implement bulk of team feedback
This commit accomplishes the following things:
- Updates copy of the dashboard/collection picker
- Fixes the tooltip copy for the add question button in the edit dashboard view
- Removes the ability to trash a dashboard question + dashboard view removes rather than trashes the question
- Adds option in dashboard edit view to add a native or notebook question
* prompt to save rather than discard changes when leaving a dashboard while editing to create a new question
* include dashboard questions in search results
* fix lint & kick CI
* type fixes
* fix unit tests
* two small fixes
* wip on failing e2e tests
* bulk e2e test fix
* fixed most tests worth fixing - leaving the ones that need to be replaced
* Dashboard Question as a source
* replace dashcard flow
* starting to update collection picker to treat dashboards as folders
* fix spelling mistake
* fix bad resolution and stop double adding newly created dashboard questions to their dashboards
* allow dashboards in collection picker
* remove checkbox to only clone dashboard w/o cards, fix collection id calculation for saving a question when defaults to a dashboard
* rough pass at allowing questions to be moved to, from, and between dashboards
* building state path with initial value as a dashboard
* simplify moving code and fix encountering BE error
* add confirm modals when moving a dashboard question to a collection or another dashboard, allow users to keep or remove dashcards for a dashboard question getting moved to a collection
* improve the breadcrumbs data consistency, fix bug in moving logic and dump user in edit state of dashboard when moving from collection to dashboard
* dynamic select button text
* Search showing available models
* hopefully this resolves some type sadness
* handle inconsistent state issues between latest question and query builder internal state
* default question picker to showing dashboards as folders
* silly type things
* disallow saving metrics | models | events to dashboards, added back ability to keep or remove dashboard ca
rds when moving a DQ to a collection, fixed dedicated DQ page not appearing archived after archiving the parent dashboard
* use currentData instead of data for collection breadcrumbs
* dashboards are not folders when they are empty
* fix moving question to Our Analytics, Update sidesheet question into for DQs, always allowed to select a dashboard for moving question
* confirm moving questions into dashboards
* trying to sort out types
* types are still a mess
* only regular questions can be moved into a dashboard
* adjusting some unit tests
* add e2e tests for dashboard questions
* should have happy linter and unit tests
* fix some type errors
* loading state
* e2e test fixes
* more fixed tests
* loading state message
* filter series options, fix create new collection logic
* fix some more e2e tests
* e2e test adjustments
* even more e2e tests fixes
* fix named export
* more e2e test fixes
* fixing some e2e stuff
* hopefully final e2e adjustments
* last failing e2e (for now)
* More BE: Dashboard Questions
add dashboard question dimension to stats
Handle series correctly
and test for them
Don't restore archived-directly questions
* unit test adjustments, linter changes
* obey the linter
* unit tests for new breadcumb functionality
* allow creating a question directly to a dashboard
* More BE: Dashboard Questions
use dashboard constants for default size
add missing sql clause
add dummmy alias to subquery
* more cards in dashboards e2e tests
* rebase whoopsie
* more tests
* fix types
* More BE: Dashboard Questions
include dashboard info on coll items & cards
include dashboard info in search
also modify the moderation info a bit, now all three endpoints return
the same shape for `dashboard` (if present):
```
{:id dashboard-id
:name dashboard-name
:moderation_status moderation-status-if-present}
```
Bulk endpoints for checking dashboards and moving
For now the move endpoint just runs everything in a transaction and
spits out an error if anything failed. I'd like to take more time to
think of the exact shape of the result we want in the case of an error,
even though I think ideally we should allow partial success.
* reproduce card-pocalypse
* adding dashboard link to search results
* more trash tests
* lint fixes
* fix crash when archiving dashboard with virtual cards
* add loading state to archive modal
* obey the clojure linter
* More BE: Dashboard Questions
cljfmt and fix tests
small test fix
* more e2e tests
* embedding tests
* type fix
* More BE: Dashboard Questions
two more small test fixes
more test fixes
* add sdk tests
* another test
* add series removal e2e test
* More BE: Dashboard Questions
yet another test 😭
work around old MariaDB limitations
* permission check and confirmation modal when moving cards into dashboards
* e2e adjustments
* add some missing tests
* prettier + rebase fixes
* fix search result bug
* Entity syncing when moving a question from a collection to a dashboard
* hello rebase hades
* gree dashboard-questions suite
* fix more e2e tests
* happy little test fixes
* Sloan Feedback
* fix move modal error swallowing
* fix embedding test
* fixes breadcrumbs for embedding, fixes associated failing tests, and updates some tests to account for reduction in # of requests
* fix embedding e2e tests
* Bryan's review response
* fix usage of `offset-paging`
* fix request count errors
* fix unit test
* finish feedback from FE code review
* CR response for Noah
* bad sloan
* no need for dashboards in ad hoc
* get the dash name from the question object
* fix migrations again
* fix test + kondo
* lint fix
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Add new user-level setting
* Add selector
* Make link dismissible from the sidebar
* Transfer the onboarding link to the main "gear" menu
* Add E2E test to cover the link dismissal behavior
* Add tests for the settings menu link
* Fix tests
* Provide useful comment in the e2e test
* Provide contextual explanations to translators
* We need to look at the context field to categorize public views
* pull poll-unil into util ns
* fix comment typo
* add test
* update docstring
* remove true? call, it will return `true` on pass
* improve poll-until
* Fix the zoom-in.timeseries drill for multi-stage queries
If no dimensions are provided to the zoom-in.timeseries drill, then attempt to find the matching dimension from the FE-provided row data instead, if present.
Related to https://github.com/metabase/metabase/issues/46932
* Only return zoom-in.timeseries drills for clicks on aggregated columns
This PR removes the deprecated `BodyComponent` HOC in favor of Mantine's `Portal` component. This helps pave the way for https://github.com/metabase/metabase/pull/49442
### How to verify
1. Perform an action that triggers an undo toast (e.g., move a question into the trash)
2. Verify the undo toast appears correctly at the bottom of the screen
3. In a collection, drag and drop items to verify the drag preview renders correctly
### Testing
This change is covered by the following tests:
* `collections.cy.spec.js > "should be able to drag an item to the root collection (metabase#16498)"`
* `collection-pinned-overview.cy.spec.js:269: "should be able to pin a visualization by dragging it up"`
Note the following comment, however, which mentions the limitations of our drag-n-drop e2e tests:
(from https://github.com/metabase/metabase/blob/92d6150/e2e/test/scenarios/collections/collection-pinned-overview.cy.spec.js#L282-L289)
```
// this test can give us some degree of confidence, but its effectiveness is limited
// because we are manually firing events on the correct elements. It doesn't seem that there's
// a way to actually simulate the raw user interaction of dragging a certain distance in cypress.
// this will not guarantee that the drag and drop functionality will work in the real world, e.g
// when our various drag + drop libraries start interfering with events on one another.
// for example, this test would not have caught https://github.com/metabase/metabase/issues/30614
// even libraries like https://github.com/dmtrKovalenko/cypress-real-events rely on firing events
// on specific elements rather than truly simulating mouse movements across the screen
```
Closes#49682. Closes#49683.
Updates `lib.convert` and legacy MBQL normalization to allow
`:ident`s, and fixes some QP middleware and utils so they can
still match up refs correctly.
No user-visible impact.
Newly created queries will have `:ident`s generated for joins,
aggregations, etc. by the previous query, and this will start
saving them into appDBs.
Pre-existing clauses don't have refs added yet.
* Fix loading in segments
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
For now, we've decided not to trigger index updates when elements are deleted, due to the overhead that toucan2 would add.
We have the code to calculate what *should* be done, so we might as well keep it around in a test that documents these relationships.
It's good to be mindful of the implications, and we could even use this information to do manual calls from the relevant deletion call sites.
* Add list/create/update/delete to actions api
* Add api mappings in entity framework
* Port actions to RTK
* Remove old api mappings
* Fix a test
* Fix test
* Fix test
* Fix test
* Fix method
* Fix types
* Define request types
* Remove ActionsApi.create
* Fix deleting actions
* Fix invalidation
* Fix invalidation
* Remove redundant spreads
* Remove type attribute from UpdateActionRequest
The underlying issue is that nest-query/nest-expression included temporal units when copying over joined fields into the nested query. This caused both the nested query and the parent query to include the "extract the unit from the date" code. In cases where the extraction didn't change the type, this duplication wasn't a big deal. However, for temporal units like hour-of-day, the first extraction changed the type of the field from date to string, and then the second extraction broke. This change ensures that the inner query doesn't try to extract temporal units, which should avoid the duplicate extractions.
* wip next sdk compat
* better code generation, dts works
* Last line of file should end in a newline character
* feat(sdk): make MetabaseProvider not break ssr
* aUpdate enterprise/frontend/src/embedding-sdk/package.template.json
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* address pr comments
* refactor(sdk): use relative paths when fixing d.ts files
* make types work on moduleResolution: node
* add loading text
* fix(sdk): #50736 make sure we don't break the host app if a sdk
component is rendered outside of the provider for a render
* fix: this should make types work for real?
* self review fixes
* update test for PublicComponentWrapper
* Update enterprise/frontend/src/embedding-sdk/components/private/PublicComponentWrapper/PublicComponentWrapper.tsx
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* remove unused component
* eslint-disable-next-line for MetabaseProvider
* list tsx as dependency
* /next -> /nextjs
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* introduce public metabase question type
* revise internal question type
* update stories
* use public question type in dashboard card plugins
* add tests for transforming question properties
* add tests for question description
* re-export the MetabaseQuestion type
* fix lint imports
* change publicQuestion to sdkQuestion
* name cannot be null if the card is present
* Fix embed loader not translated with #locale
* Fix unit tests
* Add E2E tests
* Fix a case where the lcoale is failed to load
* Simplify conditions in `LocaleProvider`
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Make E2E tests faster
Co-authored-by: Nicolò Pretto <info@npretto.com>
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
* WIP
* metabase.request.cookies namespace
* Server API namespace
* `metabase.server` => `metabase.server.instance`
* Reformat code
* Revert indentation changes
* Unrevert comment
* Oops delete moved file
* Don't be lazy CAM!!!
* Add TODO
* Please cljfmt
* Bump Kondo version to fix metadata ignores in namespaces being ignored
* Update Kondo and enable a few more linters
* Update Clojure linux install script location
* Ok so apparently the old URL does work
* Make sure things live in the right places
* Why do I always have to clean up other people's messes
* Fix formatting
* People need to knocc it off with writing slow tests
* Fix another test that took 45 SECONDS
* Fix Kondo warning
* fix scatter chart tooltip shows irrelevant info
* remove outdated spec as we always show the tooltip_column setting
* fix missing tooltip on the total bar on waterfall charts
* Always show embed sharing option for admins (#50513)
* Always render the public link option
This is possible because metabase#50504
* Always render the embed option
* Fix unit tests
* Fix E2E tests
* Correct test semantic
* Hide Embed option for non-admins
* New Embed modal (#50580)
* Open "Compare options" in a new tab
* Add an external link icon to interactive embed option when it should
* Add disabled state to SDK
* Add a disable state to static and interactive embedding
* Add interactive embedding disabled state
* Implement disabled link
* Prepare folder for testing
* Fix tests and add SDK tests
* Add interactive embedding tests
* Correct test desribe block levels
* Add static embedding tests
* Add premium interactive embedding tests
* Add enterprise tests (EE without tokens)
* Fix the rest of the failed unit tests
* Update E2E tests
* Fix a case where OSS see disabled interactive embedding
This should be impossible as interactive embedding isn't available on
OSS, so we should not display the disabled state.
* Address review
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Address review
Co-Authored-by: Poom poom@poom.dev
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Update Interactive Embedding Admin Settings page (#50693)
* Add a quick start link to interactive embedding settings page
* Update UTM tags for interactive embedding card
* Don't show interactive embedding option as enabled on OSS
* Fix unit tests
* Add a new test
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* fix the script, and add the new edn
* make the migrations run in the proper order
* it works (needs cleanup)
* remove broken binning
- These bins rely on fingerprinting for the custom Age column
- The fingerprinting for that custom column are not properly exported in
serialization, hence do not get loaded into metabase, causing binning by
Age to fail.
* cleanup / linting
* cleanup
* update migration test ids to test
* realign test name to check for to see if sample content was created
* remove Product collection + move Examples into root
* update collection_ids for the content too
* remove click behavior for 'Revenue goal for this quarter'
* Apply suggestions from code review
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
* for the sample collection is_sample should be true
* Drop CreateInternalUserNoOp
* typo
* update migration comments
* 2 small fixes
* fix data perms for example coll
* add a comment so we don't have 2 blank lines in a row
---------
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
* Add delay helper
* Remove delays in storybook
* Use skippable delay in ListField and SingleSelectListField
* Use skippable delay in ExplicitSize
* Remove waitTimeContext in storybook
* Remove waitTimeContext altogether
* Add documentation to delay
* Remove all context references in ExplicitSize
* Update flaky snapshots
Fixes: #50771
The new sync process failed to qualify enums outside public and
therefore failed to properly type them in database_type. This could
either result in matching the wrong type if there was a conflict or not
being typed as an enum.
* test: see which drivers pass these tests
* Fix build
* Change test to use standard dataset, add initial implementation for bigquery
* Use fully qualified names for views
* Don't use transactions for databricks
* Make drop table only if exists, fix describe-index in mongo for views
* don't sync system.views
* Fix oracle drop-views, disable non-syncing views
* Use metabase-instance to find the table, this is important because redshift and oracle munge table name
* Fix oracle tests
* Fix oracle views
* Fix formatting
* Fix snowflake qualified components
* Change tests to opt out
* Fix sqlsserver
* Fix postgres test
* Disable h2 describe views test
* Address review
* Add trial banner
* Use the semantic "warning" color for background
* Simplify the "reappearance" logic
* Refine the banner logic
- Pass the bare minimum information (expiry timestamp) to the `TrialBanner`
- Take timezones into account
- Introduce utilities
- Add unit tests
* Update the remaining tests
* Clean up
* Unbreak my `defsetting` 🎶
* Add comments
* Defer var assignment
* Fix failing tests 🤦
* Adjust the types in order to match `master`
* Move the logic up to the parent component
* Convert the `index` component to TS
* Fix typo in the JSDoc
* Fix another typo
* Make the return statement more readable
* Remove redundant test
* Move modals out of resizable area
* Split off ace editor from NativeQueryEditor
* Query for selection target in AceEditor
* Rename editor ref
* Add custom type for selection range
* Move more items into shared types file
* Add shared interface for editors
* Add shared Editor wrapper
* Add SQL colors in core/colors.module.css
* Allow importing global css with .global.css
* Move ace styles to AceEditor.global.css
* Fix Range type
* Fix Ace editor tests
* Scope the ace editor styles
* Remove EditorRoot styled component
* Use parent wrapper instead of classname on editor
* Remove ace editor styles from global styles
* Remove custom ace styles from embedding sdk
* Move NativeQueryEditor styles to css modules
* Fix styles for notebook sql preview
* Remove unused props
* Remove unused variable warnings
* Remove width prop
* Rename openDataReferenceAtQuestion and make it optional
* Rename setNativeEditorSelectedRange and make it optional
* Move selection popover logic out of AceEditor
* Make autocompleteResultsFn optional
* Make cardAutocompleteResultsFn optional
* Remove fetchQuestion prop
* Remove nativeEditorSelectedText prop
* Make props explicit
* Make onChange optional
* Set editor height
* Use Editor in NotebookNativePreview
* Remove cardAutocompleteResultsFn and autocompleteResultsFn props from Editor
* Remove unused snippetCollections prop
* Add missing native-query-autocomplete-match-style setting
* Remove snippets from Editor props
* Resize ace editor on load
* Add custom snippets wrapper
* Flatter completer styles
* Add container class to autocomplete popover
* Fix flex size for editor
* Fix AceEditor types
* Flatten ace styles
* Avoid hard-coded color
* Set height and width on editor
* Flatten ace wrapper
* Do not call api with empty prefix
* Add helpers to target native editor
* Use nativeQueryEditor in action helper
* Use nativeQueryEditor in ad hoc helper
* Remove ace_* references from tests
* Align nativeEditor helpers
* Revert "Align nativeEditor helpers"
This reverts commit e629aecb18.
* Move focusNativeEditor to native editor helpers
* Use focusNativeEditor where applicable
* Remove ace_editor references in test
* Fix test
* Remove references to native-query-editor testid in test
* Remove duplicate test assertion
* Fix nesting in css
* Use font-family from variable
* Use default module.css with :global instead of custom .global.css file
* Move AceEditor into subdirectory
* Fix import path
* More strongly type native-query-autocomplete-match-style
* Remove duplicate default
* Rename EditorHandle to EditorRef
* Add more specificity to ace syntax overwrites
* Simplify EditorRef
* Remove HOC
* Do not perform visibility check inside nativeEditor helper
* Explicitly assert elements are visible
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix formatting issue
* Add highlight for identifier
* Improve gutter cell
* Use relative imports for helpers
* Focus editor before typing
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* rename onClose to onCancel
* rename questionHeight to drillThroughQuestionHeight
* rename showVisualizationSelector to withVisualizationSelector
* rename parameter value props
* doc updates for parameter values
* rename the chart type selector prop
* document the withChartTypeSelector prop
* do not show column selection error on empty datasets
* add more specs to chart-definition logic
* fix entry nodes tooltip and click behavior values
* fix nodes formatting
* do not show sankey data settings on dashboards
* do not initiallt open sankey data columns selects
* hide sankey tooltip on scroll
* drop incorrect types
* set sankey node limit
* prevent clicks on edges close tooltips on native sankey questions
* tweak copy
* update sankey spec according to fixes
* revert changed settings behavior
* fix unit specs
* Pivot Exports Can Handle `nil` in Breakout Col, and Multiple Cols of the Same Aggregation Type
Fixes#50551
If a pivot table has several measures configured, they might both use the same kind of
aggregation (eg. 'sum'). Previously, this would lead to columns with the same name preventing the pivot-measures from
properly making it through the export post processing. Now, the correct aliased/deduped name is used and all columns
can be properly included in the pivot export.
Also fixes the case where a breakout column (pivot-row) contains `nil` values, which caused the pivot export to
fail. Now, all nil values are grouped and handled appropriately.
* when pivot is disabled, header should still be included
* min and max don't default to 0, add a test for the aggregations
* Add EntityObjectLoaderRtkQuery boilerplate
* Incorporate EntityType logic into the new component
* Improve comment
* Add childrenProps
* Move component into its own directory, extract types to a separate file
* Add entityObjectLoaderRtkQuery and temporary mappings
* Hook up the useGetQuery hook (tables only)
* Pass all the props to children
* Add Playground
* Gather more info in the Playground
* Integrate RTK with redux store
* Adjust props passed down
* Improve typing
* Make the request hook dynamic
* Add missing id in the query
* Handle fetchMetadata
* Change requestType to fetchType
* Add rtk mappings for databases in entity framework
* Use requestType to get request state path
* Enable new component everywhere
* Use legacy loader for entities without rtk mappings
* Improve comment
* Handle fetchMetadataAndForeignTables
* Remove unused function
* Handle reload and remove
* Add TODO
* Add transformResponse option and use it for fetchIdFields
* Format code
* Remove Playground
* Remove unused getUseGetListQuery
* Add rtk mapping for alerts
* Add rtk mapping for dashboards
* Add rtk mapping for groups
* Add rtk mapping for pulses
* Remove options
* Fix permissions tests
* Fix notifications tests
* Add rtk mapping for segments
* Use params instead of body
- It's leftover because this code got reverted after body -> params refactoring for GET endpoint has been merged to master
* Add rtk mapping for snippet collections
* Add TODO
* ESLint
* ESLint
* Update TODO
* Deprecate entity HOCs
* Update TODO
* Revert other entities for now
* Revert other entities for now
* Bring tables back
* Improve todos
* Add comment
* Improve comment
* Remove TODO
* Use reload param
* Remove properties prop
This was used to short-circuit and avoid triggering api calls if we already had an object with given attributes in cache.
With RTK query now managing the cache, we don't want to reproduce this behavior.
* Remove reloadInterval
This is used in EntityListLoader, not EntityObjectLoader
* Don't use setRequestPromise action
* Fix deprecations
* Format code
* Fix flake :(
* Try to fix flake
* Revert "Try to fix flake"
This reverts commit 6408a22a37.
* Try to fix flake
* Fix UseQuery type
* Implement useGetMetadataAndForeignTables
* Fix useMemo and move things down
* Fix typo
* Bring back original hack
* Move file
* Add comment
* match against any hexadecimal bundle name
* add e2e tests skeleton
* Support display-name for ISO get-day-of-week exclusion expressions (#50606)
* Support display-name for ISO get-day-of-week exclusion expressions
Fixes#50574
* Fix light theme embeddings has different text colors (#50678)
* add cache-control tests
* support local runs
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* add withVisualizationSelector prop
* add tests
* add to docs
* use default layout in tests
* only use custom layout when needed
* make withChartTypeSelector false by default
Adds `dev.portal/diff->`, a drop-in replacement for `->` that `tap>`s
a diff at each stage. (With the metadata for Portal to default to diff viewer)
I found myself wanting this several times while debugging the refs overhaul.
Specifically, for expressions, aggregations, breakouts, and joins.
Adds MBQL lib unit tests for this behaviour.
Currently these idents are discarded during `lib.convert` and don't
survive a round trip to legacy. That will come with a later change.
This protects against us trying to read or write from an incompatible search index. In particular it prevents existing instances from being switched over to an incompatible table during an HA upgrade or downgrade.
~~It doesn't yet get rid of the needless re-initialization, but that will be a fast follow.~~ It breaks up the jobs so that we won't reindex on startup unless necessary.
See https://www.notion.so/metabase/Search-Index-Maintenance-14b69354c9018047b670cbba1c3be50d
This ensures that when one instance starts or finishes reindexing, the other instances pick up and switch to the same tables.
Note: this does not stop blowing away the currently active table on deployment yet, we need to have a notion of version for that first.
* Enable underlying-records drill in table viz for multi-stage queries
* Fix e2e test
* Add tests for underyling-records drill for multi-stage queries
* Review feedback: filterv not-empty etc
* Review feedback: add a breakout-only query to drill-thru canned-queries
* do not show column selection error on empty datasets
* persist default sankey column settings when saving
* add more specs to chart-definition logic
* more specs
* Table Column Ordering Should Not Prevent Pivot Exports
Prior, when viz-settings existed for table column ordering, or if some columns were hidden in a regular table viz, the
pivot export could fail.
Now, if a pivot table is being exported, the table column sorting is properly ignored and the pivot export should work.
* address feedback
* use css variables for skeleton background color
* apply theme to folder color
* replace theme.white with themeColor
* use name of folder chevron color
* add background-inverse color mapping
* Revert "add background-inverse color mapping"
This reverts commit d0b546c583.
* avoid overriding folder chevron color in emotion
* use Mantine style overrides for NavLink
* temporarily hiding group into other
* hide group into other
* remove assertions related to group into other
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Adding .png version of no_results.svg to fix#49012
* Adjusting text color to to match existing color scheme
* Removing old unused no results assets; adding new png version of no_results.svg at matching resolution to pulse_no_results@2x.png
* Fixing namespace import cycle
Fixes: #50422
Although we passed additional-options through subname in url parameters
to DriverManager, we were also passing `:schema nil` as connection property.
* Pivot Export Aggregations Improved and Won't Error
Fixes#50207
WIP
- Totals now have appropriate aggregation implementation
- Totals for different row and column partial paths could collide before (eg. if a pivot-col value was equal to the
- measure idx), now the paths are unique and will not collide, causing errors
* add a test
* multiple measures need to have the correct order in each row
* clean up, add some comments explaining totals paths
* fix tests
* add test to show add-rows working correctly after :rows-part :cols-part added
* remove pivot-grouping row and columns in tandem
* add pivot card scalar test for csvs and xlsx.
* improve test feedback
* add a 'todo' for pivoted json exports
* indent
* Add stage-number parameter to NativeParameterDimensionTarget
* Update unit test
* Recognize native query dimension parameters with stage-number
* Update test setup
---------
Co-authored-by: Tamás Benkő <tamas@metabase.com>
* Don't dissoc `:id` in `before-update`
There seems to be a nasty footgun in toucan2 here.
Say you're executing an `update!` command on a set of IDs, e.g.:
```
(t2/update! :model/Card :id [:in 1 2] {:view_count 1})
```
This works if:
- both cards 1 and 2 have `view_count=1` already
- both cards 1 and 2 have `view_count!=1`
However.
If one of the two cards has `view_count=1` and another has a different
view_count, then (if the `before-update` method doesn't have the primary
key attached) Toucan emits a call to update *every card in the
database*, without a where clause at all.
* Trigger CI tests
* Add dummy job generator
* [DO NOT MERGE] Remove driver test config from the list of backend changes
* Add missing runs-on
* Avoid too much nesting
* Revert changes to file paths and README
* FE - Allow to use in dashboard filters both pre- and post- last aggregation columns (#46670)
* Extract getFilterStageIndexes
* Update comment
* Include 2 last stages in filterable columns in getParameterColumns
* Move getJoinQueryHelpers to test-helpers
* Add basic query creation
* Add breakout columns
* Make tableName optional
* Implement 1-stage test, refactor assertions
* Rename
* Move filter utils up
* Use getGroupItems and appendStageIfAggregated in getFilterableColums
* Introduce Lib.filterGroups
* Revert "Introduce Lib.filterGroups"
This reverts commit 5f8b2d098e.
* Don't reuse getGroupItems or appendStageIfAggregated
* Move getFilterStageIndexes out of Lib
* Format code
* Revert types move
* Revert functions move
* Improve diff
* Group dashcard mapping options
* Adjust existing getParameterColumns usages (except unit tests)
* Remove redundant call
* Fix filtering
* Update unit tests
* Format code, remove TODO
* Add stage index assertions
* Add model test for 1-stage query
* Add partial boilerplate for 2-stage query
* Add a 2-stage case
* Add a 2-stage case for model
* Update unit test
* Fix click behavior mapping
* Remove duplicated functions
* Remove complex return type
* Move appendStageIfAggregated and getFilterStageIndexes to metabase-lib/filter.ts
* Re-group columns based on stage index instead of using ML-returned references for comparison
* Add new options to dimension types
* Use the new stage-number option
* Rename appendStageIfSummarized to ensureFilterStage and add explanatory comment
* Remove duplicated ensureFilterStage after merge
* FE - e2e - Cover dashboard filters with tests (#46958)
* Extract getFilterStageIndexes
* Update comment
* Include 2 last stages in filterable columns in getParameterColumns
* Move getJoinQueryHelpers to test-helpers
* Add basic query creation
* Add breakout columns
* Make tableName optional
* Implement 1-stage test, refactor assertions
* Rename
* Move filter utils up
* Use getGroupItems and appendStageIfAggregated in getFilterableColums
* Introduce Lib.filterGroups
* Revert "Introduce Lib.filterGroups"
This reverts commit 5f8b2d098e.
* Don't reuse getGroupItems or appendStageIfAggregated
* Move getFilterStageIndexes out of Lib
* Format code
* Revert types move
* Revert functions move
* Improve diff
* Group dashcard mapping options
* Adjust existing getParameterColumns usages (except unit tests)
* Remove redundant call
* Fix filtering
* Update unit tests
* Format code, remove TODO
* Add stage index assertions
* Add model test for 1-stage query
* Add partial boilerplate for 2-stage query
* Add a 2-stage case
* Add a 2-stage case for model
* Update unit test
* Fix click behavior mapping
* Remove duplicated functions
* Remove complex return type
* Remove unused @ts-expect-error directive
- use createQuestionAndDashboard helper instead of command
- sort imports
* Add boilerplate for query stages test suite
* Add createDashboard
* Try out the new function
* Organize code
* Add descriptions
* Move visitDashboard call out of createDashboard
* Move appendStageIfAggregated and getFilterStageIndexes to metabase-lib/filter.ts
* Re-group columns based on stage index instead of using ML-returned references for comparison
* Add boilerplate for base questions
* Add assertions for mapping options
* Simplify verifyDashcardMappingOptions interface
* Add assertions for all types of parameters
* Update test name
* Format code
* Add new options to dimension types
* Use the new stage-number option
* Add Q2 creation
* Introduce helper constants
* Rename appendStageIfSummarized to ensureFilterStage and add explanatory comment
* Add cases for models
* Extract subroutines
* Add Q3
* Extract createQ2Query and createQ3Query
* Extract createQ1uery, remove createM1
* Optimize verifyPopoverMappingOptions
* Add more assertions
* Add model assertions
* Add tooltip assertion
* Break large functions down
* Restructure tests to prevent memory-related crashes
* Refactor
* Update comment
* Update comment
* Remove createQ1
* Remove createQ2, createQ3
* Refactor
* Refactor
* Comment out failing assertions
* Add Q3 assertions
* Add Q3 assertions
* Add comment
* Introduce createAndVisitDashboardWithQueryMatrix
* Add Q4
* Fix assertions
* Add repro tags
* Add comment
* De-hardcode indexes
* Add Q5-Q8
* Rename
* Add Q5
* Add Q6
* Add Q7 & Q8
* Add Q9
* Fix structure
* Reuse Lib.ensureFilterStage
* Fix unit of time parameters
* Support explicit and implicit stage numbers for temporal-unit params
* Extend test with temporal unit parameter with stage number
* Update unit tests
* Let stage-path handle nil (default) stage-number
---------
Co-authored-by: Tamás Benkő <tamas@metabase.com>
* Allow filtering on columns from all stages in dashboards (#48254)
* Allow filtering on columns from all stages in dashboards
* Update tests
* Clean up
* Update unit tests
* Distinguish multiple "Summaries" column groups in dashboard filter mapping (#48261)
* Allow filtering on columns from all stages in dashboards
* Update tests
* Clean up
* Update unit tests
* Distinguish multiple "Summaries" column groups
* Fix imports
* Remove redundant attribute
* Update import
* Allow filter params targeting native stages (#48379)
* Move filters on nested native stages to parent stage in expand-mbql-params
Similar to the existing handling in move-join-condition-to-source-query.
This fixes filter params that directly target a native stage with a non-negative :stage-number, e.g. when applying
dashboard filters to a card where the card query is native.
Fixes#48258
* Add test for filter params explicitly targeting nested native queries
* FE - e2e - Cover dashboard drills with tests (#48289)
* Allow filtering on columns from all stages in dashboards
* Update tests
* Clean up
* Update unit tests
* Distinguish multiple "Summaries" column groups
* Fix imports
* Remove redundant attribute
* Fix type
* Add a test
* Add a test
* Reduce flakiness
* Fix race conditions with last_used_param_values
- see https://metaboat.slack.com/archives/C010L1Z4F9S/p1727947712609579
* Add a temporary workaround
* Fix name
* Fix applying parameters in Question
* Revert "Fix type"
This reverts commit 67f6778d50.
* Rename and reorder tests
* Add more tests
* Update tests to avoid hitting https://github.com/metabase/metabase/issues/46774
* Add Q8 tests and fix tests for Q9
* Update assertions
* Update assertions because of https://github.com/metabase/metabase/issues/48339
* Improve test names
* Add test
* Skip failing test
* Add more tests
* Add test
* Remove redundant code
* Remove invalid cases
* Add test
* Add tests
* Add more assertions
* Add more assertions
* Add assertions
* Add assertions
* Use helper functions
* Use helper functions
* Nest helpers consistently throughout the file
* Decrease flakiness
* Revert "Add a temporary workaround"
This reverts commit 48ed8770d6.
* Add a temporary workaround
* Add tests
* Add assertions
* Add tests
* Update unit tests
* Make temporary workaround safer
* Add TODOs to github links
* Extract a function to improve readability
* Simplify getParametersMappedToDashcard
* Fix failing tests by calling Lib.ensureFilterStage in the correct place and properly identifying the originalQuestion so that Question.prototype.isDirtyComparedTo returns expected result
* Remove redundant code
* Quick dirty fix to run in CI
* Format code
* Format code
* Update unit tests
* Fix test setup
* Revert "Remove redundant code"
This reverts commit 35b47a10f2.
* Use Lib.areLegacyQueriesEqual
* Create a new e2e test group: dashboard-filters-2 (#48610)
* Allow filtering on columns from all stages in dashboards
* Update tests
* Clean up
* Update unit tests
* Distinguish multiple "Summaries" column groups
* Fix imports
* Remove redundant attribute
* Fix type
* Add a test
* Add a test
* Reduce flakiness
* Fix race conditions with last_used_param_values
- see https://metaboat.slack.com/archives/C010L1Z4F9S/p1727947712609579
* Add a temporary workaround
* Fix name
* Fix applying parameters in Question
* Revert "Fix type"
This reverts commit 67f6778d50.
* Rename and reorder tests
* Add more tests
* Update tests to avoid hitting https://github.com/metabase/metabase/issues/46774
* Add Q8 tests and fix tests for Q9
* Update assertions
* Update assertions because of https://github.com/metabase/metabase/issues/48339
* Improve test names
* Add test
* Skip failing test
* Add more tests
* Add test
* Remove redundant code
* Remove invalid cases
* Add test
* Add tests
* Add more assertions
* Add more assertions
* Add assertions
* Add assertions
* Use helper functions
* Use helper functions
* Nest helpers consistently throughout the file
* Decrease flakiness
* Revert "Add a temporary workaround"
This reverts commit 48ed8770d6.
* Add a temporary workaround
* Add tests
* Add assertions
* Add tests
* Update unit tests
* Make temporary workaround safer
* Add TODOs to github links
* Extract a function to improve readability
* Simplify getParametersMappedToDashcard
* Fix failing tests by calling Lib.ensureFilterStage in the correct place and properly identifying the originalQuestion so that Question.prototype.isDirtyComparedTo returns expected result
* Remove redundant code
* Quick dirty fix to run in CI
* Format code
* Format code
* Update unit tests
* Fix test setup
* Create a new e2e test group because the old one was timing out in CI
* Add the group to GH action
* Revert "Remove redundant code"
This reverts commit 35b47a10f2.
* Use Lib.areLegacyQueriesEqual
* FE - e2e - Cover combination of unit of time parameter + non-last-stage filters with tests (#48726)
* Add JSDoc for entityPickerModalItem
* Add addToDashboard parameter to saveQuestion
* Add JSDoc for saveQuestion
* Fix import
* Add a test reproducing the bug
* Try to fix unit of time parameters
* Fix wrong stage-index for temporal unit parameters on composed ad hoc questions
* Fix ad-hoc question naming when there is an empty extra filter stage
* Fix unit tests
* FE - Do not add extra filtering stage for pivot tables (#48975)
* Avoid using Lib.ensureFilterStage for pivoted questions
* Fix unit test setup
* Add tests for pivot tables
* Optimize tests
* Test filter modal
* Revert redundant changes
* [WIP] Dashboard filter columns be fixes (#48906)
* Revert "Add a temporary workaround"
This reverts commit 4ea374870d.
* Fix some of the issues and unskip some of the E2E tests
* Acknowledge that metrics are transparent
* Fix breakouts by the same column with different temporal-units
* Use non-negative stageIndex for temporal unit parameters
* Add stage if there is an explicit reference to stage after the last
Instead of duplicating the ensure-filter-stage logic, just check if there is a parameter referencing the filter
stage (which is always one after the last in the query). If there is such a reference, add that stage.
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Require only breakouts when adding new stage in ensure-filter-stage (#49067)
* Only require breakouts when adding new stage in ensure-filter-stage
Previously, this function required both breakouts and aggregations.
Closes#48339
* Update e2e tests after changes to Lib.ensureFilterStage
This resolves all-but-one of the TODOs related to #49339
Related to #49022
* Implementing parameter_mappings migration (#49256)
* FE - e2e - Update tests after `Lib.ensureFilterStage` changes (#49222)
* Unskip #19744 repro
* Remove sanity checks
* Update comment
* Fix typo
* Remove invalid test
* Ignore pivoted "table" viz and and inline Question.prototype.isPivoted
* Remove test for pivoted table
* FE - e2e - Click behavior (#49239)
* Unskip #19744 repro
* Remove sanity checks
* Update comment
* Fix typo
* Remove invalid test
* Ignore pivoted "table" viz and and inline Question.prototype.isPivoted
* Remove test for pivoted table
* Add a test
* Disable columns from non-last stage in click behavior
* Finish the test
* Refactor
* FE - e2e - Cover public & embedded dashboards with tests (#49280)
* Add test for public dashboard
* Add test for embedded dashboard
* Add comment
* Add test for 2nd stage column
* Add test for 2nd stage aggregation column
* Add test for 2nd stage breakout column
* Add assertions for models
* Handle no auto-pivoting of ad-hoc questions from drill thrus
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Add missing test setup
* Address some review comments (#49429)
* FE - Show columns from all stages in click behavior (target question) (#48828)
* Add JSDoc for entityPickerModalItem
* Add addToDashboard parameter to saveQuestion
* Add JSDoc for saveQuestion
* Fix import
* Add a test reproducing the bug
* Try to fix unit of time parameters
* Fix wrong stage-index for temporal unit parameters on composed ad hoc questions
* Fix ad-hoc question naming when there is an empty extra filter stage
* Fix unit tests
* Fix non-unique keys
* Add stage-number to click behavior dimension target
* Call Lib.ensureFilteringStage in click behavior
* Use describeEE because click behavior tests use setTokenFeatures("all")
* Stop testing implementation details
* Stop testing implementation details
* Stop testing implementation details
* Modernize test and make it less flaky
* Address flake source
* Fix SAMPLE_DB_ID
* Unskip repro for #19744
* Add a test
* Update test name
* Introduce verifyNotebookQuery
* Use verifyNotebookQuery
* Use verifyNotebookQuery
* Move verifyNotebookQuery to helpers
* Assert items count in verifyNotebookQuery
* Extract helper functions
* Rename helpers
* Refactor
* Consistent comments
* Extract verifyVizTypeIsLine
* Add scrollIntoView calls for extra safety
* Add TODO
* Use verifyNotebookQuery
* Make stages optional
* Add createMultiStageQuery
* Remove redundant scrollIntoView calls
- We don't need them because the assertions are not using "be.visible"
* Improve test setup
* Add more assertions
* Add more assertions
* Add basic join support to verifyNotebookJoins
* Unnest code
* Implement join assertions
* Implement expressions and sort assertions
* Simplify code
* Add commented-out assertions
* Yes, it is correct
- it's stageIndex which sourceColumn comes from in sourceQuery
* Conditionally apply filter stage
* Allow mapping to all stages in click behavior
* Remove redundant argument
* Simplify diff
* Simplify diff
* Update test setup
* Remove TODO related to metabase#49282
- Closes#49282
* Support fetching param values on filter stage (#50120)
* Support fetching param values on filter stage
Fixes#48613
* Fix temporal-unit-parameters
* Enrich dimension targets in click_behavior with stage-numbers (#49911)
* Enrich dimension targets in click_behavior with stage-numbers
Fixes#49110
* Add missing stage-number
* Remove changeSynchronousBatchUpdateSetting references
* Update assertions to include time bucket
* Use mbql.u/legacy-last-stage-number in metabase.query-processor.card
---------
Co-authored-by: Tamás Benkő <tamas@metabase.com>
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* prevent the UI from breaking if there is an invalid enum value set by MB_LOADING_MESSAGE
* add test case
* add enum validation on backend
* oops - fix set-value-of-type! call
---------
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* use -1 for stage index for breakout and aggregation list
* add e2e tests
* remove sum of total in e2e
* use test id matcher
* add issue to e2e name
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* move stageIndex to a separate variable
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* hide pivoted exports under env var
* only export pivoted when the env var is true
Also added tests to confirm.
* must specify :export for public settings
* spec
* change setting to :internal since it's only via env var
* use correct visibility add setting to settings.yaml for serialization
---------
Co-authored-by: Adam James <adam.vermeer2@gmail.com>
On MySQL 8.0.17, the following is prohibited:
```
WITH RECURSIVE MyRecursiveCTE (...) AS
(
SELECT ... FROM bar UNION ALL
SELECT ... FROM bar JOIN MyRecursiveCTE ON ...
)
```
Instead, you need to do:
```
WITH RECURSIVE MyRecursiveCTE (...) AS
(
SELECT ... FROM bar UNION ALL
SELECT ... FROM MyRecursiveCTE JOIN bar ON ...
)
```
Basically, just putting the CTE on the left vs. the right side of the
join (which is exactly equivalent under the hood) fixes the issue.
I can't actually find any documentation of this change, but it seems
that sometime between MySQL 8.0.17 and MySQL 8.0.33, MySQL started
allowing this, perhaps just an optimization where it now notices the
equivalence and does the right thing.
I originally tested this on what I believed was the oldest version of
MySQL we supported, but didn't read carefully enough. We recommend
8.0.33 or higher here:
https://www.metabase.com/docs/latest/databases/connections/mysql
But that's not for the AppDB - for that, we recommend 8.0.17 or higher.
* Fix serdes of missing cards in click behavior
When exporting a dashboard, we export its dashcards, including cards
referenced inside its viz_settings.
The problem is that viz_settings might have references to
now-nonexistent cards.
This is an initial stab at cleaning up the problem, specifically for
click behavior. If Dashboard D has Dashcard DC, which has a
click-behavior that links to Card C, then when we collect the
descendants of D, we'll include a reference to C - even if C has been
deleted.
So this makes two changes: first, when calculating the descendants of a
dashcard, don't include links to cards that don't exist. And, when
exporting the click behavior, remove the whole thing entirely if the
card it points to doesn't exist.
* Put `#locale` behind the feature flag
* Prepare folder structure for unit testing
* Refactor: extract a setup function as a shared util
* Add #locale tests
* Add all tests
* Prepare folder structure for unit testing
* Refactor: extract a setup function as a shared util
* Add locale tests
* Add all questions tests
* Fix failed E2E
* Tidy up questions tests
* Tidy up dashboards tests
* Fix test name typo
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Allow searching a single select list picker
* Fix unit tests
* Add repro for 49642
* Remove selection when searching
* Rename parameter to value because it does not need to be a prefix
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Simplify loading state for ListField
* Use same DEBOUNCE_FILTER_TIME in ListField
* Don't rely on waitTimeContext in test
* Fix lint
* Add initialization state for filter widget
* Update Loki Snapshots
* Update Loki Snapshots
* Allow picking other values when selecting a single value dropdown value
* Add comment about initialization
* Use setup in test
* Update Loki Snapshots
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* revamp the create question component
* docs updates
* add stories and update styles
* prevent switching to visualization when it is not ready
* ensure CreateQuestion works in flex parents
* add basic e2e tests for CreateQuestion
* ensure that switching between editor and visualization works
* use the save button for the disabled state
* update the question title when the question is saved
* implement updating questions in place
* rename replaceQuestion
* make CreateQuestion props all optional
* add background color to modal header
* hide the save button if question is dirty
* hide save button instead of disabling per design feedback
* update e2e tests
* revert e2e helpers
* add title update assertions
* whitespace changes
* Add new strings for v51.4
* Update translations
* kick ci
---------
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Don't try to fetch dbs for zero cards in api.table/batch-fetch-card-query-metadatas
Closes#48461
Ensure we don't throw an exception by attempting to fetch the Databases associated with potentially-deleted card ids in
api.table/batch-fetch-card-query-metadatas.
In particular, this ensures we don't generate an exception if the user does the following
1. Create a card "Source Card"
2. Create another card "Derived Card" with a :source-table of "Source Card"
3. Add Derived Card to a dashboard
4. Permanently delete (not just archive) the Source Card
* Add test for virtual-table query-metadata with deleted source card
* Add test for dataset query_metadata with deleted source card
* Add test for dashboard query-metadata with deleted source card
* Add test for card query-metadata with deleted source card
* Add metabase.api.test-util and use it in query-metadata-with-deleted-card tests
* Replace copy/pasted debug helpers with select-query-metadata-keys-for-debugging
* trigger save handlers on question create
* add skeleton example stories for save forms
* update storybook stories
* add tests for saving questions
* docs update
* add createCard assertions
* rollback docs
* feat(sdk): detect mismatch between sdk version and mb version
* refactor: log utils to print the big headers
* Update enterprise/frontend/src/embedding-sdk/lib/version-utils.unit.spec.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* refactor: call -> callArguments
* add link to the docs
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* remove LegendVertical logic that preserves legend width
* memoize pie legend items
* prevent pie from jumping when toggle legend
* fix responsive legend logic, cleanup
* drop hyphens from the pie legend (#29597)
* eliminate flickering on render due to not having dimensions
* shortcut
* specs
* test vertical layout has no hyphens
* setup func
* set explicit showLegend=true default
* [xlsx-export] Cache the custom datetime formatter
* [xlsx-export] Optimize functions in m.q_p.pivot.postprocess
* cljfmt
---------
Co-authored-by: Adam James <adam.vermeer2@gmail.com>
* Revert "fix(sdk): put a bandage on the flashing error on static question in strict mode (#49659)"
This reverts commit c57b3a38e0.
* refactor(sdk): use ignore flag instead of mounted ref
* Add temporal unit to display name
* Remove temporal unit adjustments from FE
* formatColumn to displayNameForColumn
* Factor out getFriendlyName
* Adjust BE tests
* Add require
* Adjust FE unit tests
* Adjust temporal-unit-parameters.cy.spec.js
* Adjust dashboard-reproductions.cy.spec
* Convert formatColumn occurrence
* Adjust legacy display name logic generation
After `getFriendlyName` was removed from FE, capitalization
of display name should happen here. Logic is similar, should
be adjusted prior merging the PR.
* Adjust embedding-linked-filters.cy.spec.js
* Adjust joins-reproductions.cy.spec.js
* Adjust metrics-editing.cy.spec.js
* Adjust timelines-question.cy.spec.js
* Adjust offset.cy.spec.ts
* TMP Avoid duplication of temporal unit in display name in MLv2 code
* Probably FE fix: add default temporal bucket only if not present
* WIP Add ends-with-temporal-unit? function
Current strategy is to add temporal units to display names if same is not already present
* Adjust display-name-method :metadata/column
Append TU to display name only if different to what's already present
* Adjust FE: show temporal unit that's present on field instead of default on hover on QueryColumnPicker element
* Return also temporal-unit from visible-columns
* Enable tempora-unit to be returned from previous-stage-or-source-visible-columns
* WIP Adjust multi breakout multi stage e2e multiple-column-breakouts.cy.spec
* Adjust multiple-column-breakouts.cy.spec.ts
* It was not a bug.
* Revert changes in _BaseBucketPickerPopover
* Restore temporal-unit for next stages removal in stage.cljc
* Add inherent-temporal-unit
* Adjust display name temporal bucket logic
* Rename to inherited-temporal-unit
* Shutdown too broad display name capitalization
* Eastwood
* Adjust pivot-rows-order-test
* Adjust pivot-non-numeric-values-in-aggregations
* Adjust inherent to inherited
* Rename to inherited
* Add inherent-temporal-unit only to returned-columns
* Update e2e/test/scenarios/dashboard-filters/temporal-unit-parameters.cy.spec.js
* Update e2e/test/scenarios/question/multiple-column-breakouts.cy.spec.ts
* Update consistent-date-formatting-test
* Update dashboard/x-rays.cy.spec.js
* Update waterfall.cy.spec.js
* Return month default for default inherited-temporal-unit
* Update reproductions-3.cy.spec.js
* Update downloads/downloads.cy.spec.js
* Update legend.cy.spec.js
* Update pie_chart.cy.spec.js
* Update click-behavior.cy.spec.js
* Update embedding-linked-filters.cy.spec.js
* Return QueryColumnPicker's condition to original state
* Remove reundant export
* Update e2e/test/scenarios/question/multiple-column-breakouts.cy.spec.ts
* Comment
* Update nested.cy.spec.js
* Update e2e/test/scenarios/sharing/sharing-reproductions.cy.spec.js
* Adjust to TS types
* Adjust for cljfmt
* Capitalize
* Add propagation tests
* Add inherited-temporal-unit propagation for expressions
* Add default inherited-temporal-unit test cases
* Add qp temporal-bucket-test ns
* Simplify default unit logic
* Comment
* Use NULL_DISPLAY_VALUE
* Comment
* Comment
* Comment
* Comment
* Comment
* Comment
* Remove redundant code (for now)
* Update comments
* Comment
* Restore way too daring simplification
* cljfmt
* Adjust capitalize
* Update comment
* Update test
* Address remarks in tests
* Update test
* Address remarks
* typo
* Use Flex instead of Group for items in button
* Allow column picker to stretch to match the height of the element
* Fix height of data picker buttons
* Make sure the whole button is clickable
* Add e2e test for metabase/metabase#50038
* Use correct id for issue in e2e test
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* attempt to fix 49537 by removing position relative from the tooltip
container
* Make echart tooltip container cover all iframe area
* Update the E2E test
* Update loki snapshots from CI
* Re-enable echarts tooltip assertion.
I couldn't find a better way to fail these assertions on the previous
version of the code, but these assertions are better than nothing.
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* XLSX Pivot Export - Initialize Pivot Table with Small Area Ref
This PR changes the native pivot export so that we first initialize the pivot table with an area ref that's only the
first 2 rows. Prior, I used `AreaReference/GetWholecolumn` which had the side effect of using lots of memory.
I think this is related to how the Pivot Table's Cache is built up as you add rows/cols to the pivot definition. So,
if you keep the area ref only as wide as the number of columns in the raw data, and just 2 rows, then the cache can
stay small.
After adding row/column data to the pivot table, you can then modify the area ref to be all rows in the relevant
columns (`A:E` for example).
This keeps the pivot-table object much smaller in size.
* need the extra xml schemas to get this to work
I'd like to try find a way around including this extra dep, it's a 13mb jar (ish), and I think that's a little bigger
than I'd like
* oops, didnt mean to have this in
* Add test for pivot table initialization being fast and lean on memory
* remove time based assertion, add comments in impl
* cljfmt
* fix test
* Pesky little whitespace
---------
Co-authored-by: Oleksandr Yakushev <alex@bytopia.org>
* XLSX Pivot Exports Now Use The Sort Settings from Viz Settings
Prior to this PR, the xlsx pivot exports worked correctly except for the fact that no sorting (ascending or
descending) was applied to the pivot rows or columns. Now, those settings are used when they exist.
* only try to set sort when a correct setting exists
* add ooxml-full dep for correct classes
* Update src/metabase/query_processor/streaming/xlsx.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
---------
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Fixes: #50072
When we `describe-fields` we are passing db details, but sometimes we
have the database name in `dbname` rather than in `db` depending on the
connection specs and possibly environment. So we check for both now.
== Goal ==
Hide attached DWH database details from anyone incl. admins:
* Do not show them in the UI
* Do not permit to change them
* Do not serialize them
- This is what we're adding for H2 snapshots (dumps) with this PR.
The aim is that customers cannot gain access to (parts of) credentials,
and they cannot break a feature they are paying for by changing
connection details.
== Implementation ==
In 592360c9fa I wrongly understood that
database details would be omitted when dumping to H2, but this is only
true for dumping H2 databases.
Fix this by omitting database details also when dumping databases with
`is_attached_dwh` set.
== How to test ==
To prepare, download a H2 JAR matching the H2 version used by Metabase:
```
wget https://repo1.maven.org/maven2/com/h2database/h2/2.1.214/h2-2.1.214.jar
```
If the H2 version does not match, you will get an error when trying to
open the H2 shell:
```
Exception in thread "main" org.h2.jdbc.JdbcSQLNonTransientConnectionException: Unsupported database file version or invalid file header in file "[REDACTED]" [90048-232]
```
=== New behaviour ===
Setting the `is_attached_dwh` field hides the database details from H2 dumps:
1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
- In addition to the fields you would normally set, also set
`is_attached_dwh: true`.
- This also works when adding this flag to a database that previously
did not have this flag set.
2. Start your Metabase instance.
3. Run `${metabase} dump-to-h2 ./dump-file-h2` to create the H2 snapshot file.
4. Run `java -cp h2-2.1.214.jar org.h2.tools.Shell -url "jdbc:h2:file:./dump-file-h2;ifexists=true"` to open a H2 shell.
5. Verify that `SELECT name,details FROM metabase_database;` shows `{}` for the database you added in step 1
=== Original behaviour ===
Behaviour without setting the `is_attached_dwh` field is unchanged:
1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
- Only set the fields you would normally set. Do not set
`is_attached_dwh` (or set it to `false`).
2. Start your Metabase instance.
3. Run `${metabase} dump-to-h2 ./dump-file-h2` to create the H2 snapshot file.
4. Run `java -cp h2-2.1.214.jar org.h2.tools.Shell -url "jdbc:h2:file:./dump-file-h2;ifexists=true"` to open a H2 shell.
5. Verify that `SELECT name,details FROM metabase_database;` shows a non-empty object (i.e. not `{}`) for the database you added in step 1
Fixes: 592360c9fa
Closes: https://github.com/metabase/harbormaster/issues/5526
* fix(sdk): remove `Cannot update a component XX while rendering a
different component` error
* test(sdk): introduce jest test to make sure api.basename is set before
loading the locale
* sync the attached datawarehouse
requires either "sync_db" true or a table name
it will error
```shell
❯ http post 'localhost:3000/api/notify/db/attached_datawarehouse' x-metabase-apikey:apikey -pb
Must provide `sync_db` or a `table_name` to sync
```
It will sync the db when sync_db is true [NOTE: this is outdated. now just omitting the table name will sync the db]
```shell
❯ echo '{"synchronous?": true, "sync_db": true}' | http post 'localhost:3000/api/notify/db/attached_datawarehouse' x-metabase-apikey:apikey -pb
{
"success": true
}
```
It will sync existing tables
```shell
❯ http post 'localhost:3000/api/notify/db/attached_datawarehouse' table_name=existing schema_name=public x-metabase-apikey:apikey -pb
{
"success": true
}
```
it will error if it cannot find a table
```shell
❯ http post 'localhost:3000/api/notify/db/attached_datawarehouse' table_name=new schema_name=public x-metabase-apikey:apikey -pb
{
"cause": "Unable to identify table 'public.new'",
"data": {
"schema_name": "public",
"status-code": 404,
"table_name": "new"
},
"message": "Unable to identify table 'public.new'",
"schema_name": "public",
"table_name": "new",
"trace": [],
"via": [
{
"data": {
"schema_name": "public",
"status-code": 404,
"table_name": "new"
},
"message": "Unable to identify table 'public.new'",
"type": "clojure.lang.ExceptionInfo"
}
]
}
```
if i create that table
```sql
attached=# create table new (id int);
CREATE TABLE
```
it will then find and sync it
```shell
❯ http post 'localhost:3000/api/notify/db/attached_datawarehouse' table_name=new schema_name=public x-metabase-apikey:apikey -pb
{
"success": true
}
```
* formatting
* Tests for attached datawarehouse
* PR nits
typo in `thorw` -> `throw`
comment explaining why `find-and-sync-new-table` is always sync
gensym the db name in test
remove a binding for a fn `sync!` that's only called once
test `{:sync_db true}` syncs the whole db
* stupid formatting 🦾
* Use absence of table_name as indicator to sync database
* Fix smartscalar texts to use user locale
* Fix Pie chart texts to use user locale
* Fix line chart tooltip texts to use user locale
* Add trend chart assertions
* Add pie chart assertions
* Add line chart other series tooltip assertions
* Update frontend/src/metabase/visualizations/echarts/pie/constants.ts
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Fix failed E2E tests
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Apply Column Sort To Pivot Sections
Fixes#49437
This PR fixes the dataset API so that column sorts (ascending/descending settings on pivot-rows and pivot-cols) are
properly parsed and passed through the QP.
After that, I've also modified the post processor to use the sort orders properly in pivoted exports.
* println debugging :)
* fixing a few tests
* fix a few more tests
* Update workflow on master
* [DEBUG] Disable steps for testing
* Add alberto as a reviewer
* [DEBUG] modify the branch name for testing
* Revert "[DEBUG] modify the branch name for testing"
This reverts commit 7d5d8bf802.
* Revert "[DEBUG] Disable steps for testing"
This reverts commit a8662c9e8b.
* Try snowflake 3.19.0
* Deafault to java 21 for drivers
* Trigger CI
* Try running things with --add-opens to see if it solves our problems
* Fix `update-view-dashboard-timestamp-test`
* Try snowflake 3.19.0
* Deafault to java 21 for drivers
* Try running things with --add-opens to see if it solves our problems
* Fix `update-view-dashboard-timestamp-test`
* Update deps.edn
* Switch to Java 21
* Docker image needs to use --add-opens option
* Add note about change to Java 21 to driver changelog
---------
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Luis Paolini <paoliniluis@gmail.com>
* add filter picker skeleton
* add filter picker skeleton
* add picker in popover use case
* reuse FilterColumnPicker
* add question filter logic
* remove fixed width from popover
* add onClose handler
* add e2e test skeleton
* contain FilterBar in Box in story
* add a test for adding filters
* update e2e tests
* add filter picker to the docs
* inline props and functions
* update the props
* Allow using the arrow buttons to change the limit value
* Add a min prop to the number input
* Use base-10
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Ensure number is integer
* Add issue name to unit test
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Use ParameterValueWidget for default value picker in native parameter settings
* Fix e2e test
* Fix native filter tests
* Fix e2e test
* Fix e2e test
* Add repro for #44665
* Assert that filter values are shown in the popover.
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
This also cleans up the testing to check everything more carefully,
and to expect the right things across multiple stages.
(This is a bit tricky since the order-bys get removed when the stage
gets aggregated, because the columns they were using are now gone.)
* Remove dead code from browse models page
* Use the grid to control the gap
* Implement "Add database" button
* Add basic unit tests
* Make sure the feature works when there are no databases
* Add analytics to the db list
* Add "Add database" button to the sidebar
* Add E2E Snowplow tests
* Add unit tests
* Deal with authentication in each beforeEeach separately
* Add Snowplow E2E test for a database card
* Slightly tweak the title line height
Search currently returns a detailed list of the scores for each item, which is pretty hefty, and the type is also coupled to the current rankers, which will change in future.
Removing from the type to start to just validate that nothing uses it.
Best to just not model this. Going to see about just removing it from the BE as well, and put in some other way to debug. Worst case an opt-in query parameter.
* Add blurb from C3P0 docs to comment for unreturnedConnectionTimeout
* Enable debugUnreturnedConnectionStackTraces in data-warehouse-connection-pool-properties
Closes#47981
* Add defsetting for jdbc-data-warehouse-debug-unreturned-connection-stack-traces
And only enable debugUnreturnedConnectionStackTraces in the C3P0 config if the setting is enabled. There is some
memory cost to enabling the setting, and since maxPoolSize is user-configurable, it seems safer to disable
debugUnreturnedConnectionStackTraces by default.
* Add metabase.logger/level-enabled?
* Log a warning if C3P0 logs are not configured correctly to log stack traces
Log a warning if the user enables the jdbc-data-warehouse-debug-unreturned-connection-stack-traces but does not raise
the log level for the com.mchange package to INFO or higher.
* Add docs for MB_JDBC_DATA_WAREHOUSE_DEBUG_UNRETURNED_CONNECTION_STACK_TRACES
* Improve markdown docs for MB_JDBC_DATA_WAREHOUSE_DEBUG_UNRETURNED_CONNECTION_STACK_TRACES
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Keep Pivoted Option works in Subscriptions
Fixes: #49525
Even though the user could toggle 'Keep data pivoted` in the Sharing Sidebar UI, the setting was not being propagated
to the backend, so attachments would not be pivoted.
* add a cypress test
* add viz settings to get the pivot working
* fix for linter
* remove eslint ignore
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
I had to add `only-safe-columns` to filter out columns with known issues
in current pMBQL. In particular, implicitly joining on a column which
appears twice (eg. because of a double- or self-join) is ambiguous with
classic `:field` refs, and will cause errors.
There's a `:dynamic` var to re-enable all the columns for tests that are
only exercising new `:column` refs.
* Revert "[streaming-response] Use HttpEndpoint.isOpen() instead of reading a byte from TCP socket (#49460)"
This reverts commit 060baf0ce4.
* Disable connection reuse for requests that serve streaming responses
This introduces a reusable `weighted-choice` function and a "weight"
system for the various steps. `:order-by` should be more rare than
filters, etc., `:limit` and `:append-stage` should be more rare still.
Adds basic expressions to the query generators.
There's a lot more to do to robustly generate realistic expressions! But
this is enough to make expressions exist on queries, which then get
consumed as inputs to filters etc.
* build on java 21
* update docker image
* update test
* more using 21
* back to 11 for build
* update docker command
* update containerization
* update docker actions
* revert dockerfile change
* update other docker action versions
* use release commit for containerization
* Remove @flaky tag
* Unskip test and remove @flaky tag
* Remove @flaky tag
* Remove @flaky tag
* Revert "Remove @flaky tag"
This reverts commit 08c57add88.
* Try to fix flake
* See if timeout is necessary
* Revert "See if timeout is necessary"
This reverts commit 181f3c5e63.
* Revert change for coordinates filter
* Remove @flaky tag
* Make workaround test pass
* Disable workaround test
* Uncomment and unquarantine the workaround test
* Disable normal test for now
* Revert "Disable normal test for now"
This reverts commit 68fa2175a7.
* Bring back quarantine
* Update comment
* Move the upload CSV button to the main nav bar
* Remove the old `SidebarOnboardingSection`
* Remove obsolete unit tests
* Rework E2E tests
* Add some spacing around the button
* Fix E2E test
* Break `MainNavbar` unit test down
* Expand unit tests
* Update E2E tests
* Fix E2E test failure
* Move JSDoc where it belongs
* Provide context for translators
* Simplify the condition
* Pivot Measures Order Used in Pivot Exports
Fixes#48442
A pivot table can have any number of measures, and the user can order these by dragging in the UI. Before this PR,
that order was ignored and measures would alway be in index order, which is confusing for any user who needs the
measures to be displayed in a particular order, especially if they've re-ordered them in the pivot viz settings UI.
A test has been added to check that measure order is used.
A few minor changes to the pivot qp and post-processor
- measure indices are looked up in the pivot qp and added in viz-settings order
- the pivot measures are only added if the qp has not already added them.
- pivot-opts Malli spec has been made in the namespace, adjusted to allow `nil` as a valid pivot-opts output, and
used in relevant functions
* address review points.
* add a rows order test
* Filter out Empty Rows from Pivot Exports
Fixes#49353
The linked issue is not actually related to pivot export size but is instead related to the 'Min of Created At: Month'
aggregation; the default aggregation function was `+`, so it broke when the date string was encountered.
That was fixed.
As I was trynig to keep the export small if possible, I noticed that in some cases empty rows are appended, so I added
the filter so that if a pivot row's values are completely empty, it doesn't add it.
Finally, this PR also adds the 'sub section totals' which I noticed were missing from the exports. This comes up when
you have 3+ pivot-rows configured, so you can see the subtotals for the first pivot row and the subtotals nested
within those sections for the second pivot row, and so on.
* add test for non-numeric values
* Make sure the new name refs match on aggregations not just breakouts
* cljfmt
* move the database prompt to the very beginning
* ability to use sample database for cli
* print username and password in console
* no tenancy isolation as a default option
* highlight sample credentials file
* update credential log messages
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* continue without tenancy isolation
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Fix :pivot-measures with column names
* Fix the test
* Update src/metabase/query_processor/pivot.clj
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Add a test
* Add a test
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
Adds filters to generative testing for queries.
There's a lot of new code here, but don't panic, it's quite straightforward:
generating filter values, in the right shapes for various filter expressions.
* refactor + more specific auth errors
* fix linting issues
* make tests pass, again
* make test pass for real?
* continues the refactor and hopefully makes tests pass
* try to simplify the logic a little bit more
* also logs to the console for the edge case of invalid session
* remove setLoginStatus
* fix typo
* fix debug line that should have been commented
* remove dead code
* remove test for code that doesnt exist
* address feedback from PR
* update test copy
* Update enterprise/frontend/src/embedding-sdk/store/auth.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Update enterprise/frontend/src/embedding-sdk/store/auth.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* update copy for site settings error
* ex -> exception
* Update enterprise/frontend/src/embedding-sdk/store/auth.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Update enterprise/frontend/src/embedding-sdk/store/auth.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* update tests to use the updated copy + small typos
* refactor: use jest spy on to check for console.error
Co-authored-by: "Mahatthana (Kelvin) Nomsawadi" <me@bboykelvin.dev>
* use full error message in test
* renamed response to session
* mock implementation of console.log
* fix(49482): don't create a new store on each MetabaseProvider render (#49492)
* fix(49482): don't create a new store on each MetabaseProvider render
* remove store prop as it seems to not be used
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
The query can contain all kinds of things, in particular the whole original query in the case of pivot questions, so
we should check only the part that actually gets resolved by the middleware.
* perf: Implement faster sync methods for postgres
Fixes#48575
Pulls work from redshift into the common postgres driver.
* Fix tests and formatting
* Move nested-field-column sync to sync functions so describe-fields will also get them
* Fix test
* Fix test
* Remove fixed safety test
* Add test specific database-supports feature for pk metadata
* Fix test
* perf: faster mysql sync with describe-fields
Fixes: #49010
* Adrress PR feedback
* Fix tests
* Fix test
* Add nil table-schema
* Don't use subselect for field-comment
* Fix quoting weird identifiers
* Make format string inline
* Update src/metabase/driver/mysql.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update src/metabase/driver/mysql.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Fix tests
* Fix database-type
* Fix tests
* Fix test
* Fix tests
* Exclude mysql table_schema
* Handle tinyInt1IsBit
* Fix test
* Only get one db at a time
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
These are helpful for creating literal dates and times in a
cross-platform way. Useful for generating queries, in particular for
expression and filter values.
* add base color palette
* add some more comments
* fix formatting and color copy-pasta errors
* ignore existing bad base color usage
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
The ref (usually) contains enough information to unambiguously find the
matching implictly-joined column, even if there are several FKs that
could join the target table.
The remaining ambiguous case is if the *same* FK column exists
multiple times - current MBQL uses raw field IDs for the FK field, so it
can't distinguish between eg. an FK in the main table and the one from a
self-join.
* Update saveDashboard helper to wait for the GET /api/dashboard/:id request
- update aliases to be unique so that they don't interfere with aliases used in tests
* Remove no longer needed intercept + wait
* only tag nightly on the current version
* add release channel setting to publish job
* clean up release workflow
* remove code from something else
* reduce depth of the release process stack
* update build job too
* remove dead arg
* Pivot Measures Order Used in Pivot Exports
Fixes#48442
A pivot table can have any number of measures, and the user can order these by dragging in the UI. Before this PR,
that order was ignored and measures would alway be in index order, which is confusing for any user who needs the
measures to be displayed in a particular order, especially if they've re-ordered them in the pivot viz settings UI.
A test has been added to check that measure order is used.
A few minor changes to the pivot qp and post-processor
- measure indices are looked up in the pivot qp and added in viz-settings order
- the pivot measures are only added if the qp has not already added them.
- pivot-opts Malli spec has been made in the namespace, adjusted to allow `nil` as a valid pivot-opts output, and
used in relevant functions
* address review points.
* add a rows order test
* only tag nightly on the current version
* add release channel setting to publish job
* clean up release workflow
* remove code from something else
* fix step title
Also fix a bug in `lib.breakout/existing-breakouts` that treated
breakouts on duplicate columns from different joins as existing, even
though they're not.
* add ability to hide the collection picker
* add save options
* make sure collectionId is fresh
* update sdk question form component
* make collectionId accept null
* apply saveOptions to sdk components
* revert to use initialCollectionId as context name
* target the correct collection id
* add e2e tests
* update e2e tests
* move into within
* update test
* remove collectionId from default args
* revert e2e of hide columns
* make e2e helper more generic
* use saveToCollectionId prop to save to a specific collection
* update e2e tests
* update comment formatting
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* clean up leftover changes from an earlier version
* readme update
* indicate that saveToCollectionId is only applicable to static questions
* revert QuestionEditor.isSaveEnabled defaults
* update comment format
* remove unused initialCollectionId prop
* experiment: try to fix failing custom-columns e2e
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Adding moderated status to entity picker
* adjusting verified icon size
* size adjustments
* e2e test adjustments
* removing verified model icon
* fixing e2e test
* unit test
* unit test adjustment
* Adding moderation status to Dashboards
* react sins
* verification coll: subselect on card + dashboard
* fix the 3 failing BE tests
* add an api test for dashboard verification
* verified dashboards are never stale
* e2e tests
* code cleanup
* unit test and e2e test cleanup
* PR Feedback
* backend test fixes
* Nick shouldn't touch clojure
* type fix
* indent
* fix last 2 BE tests
---------
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* WIP: use list picker when searching a single value
* Add test case for multi vs non-multi search fields
* Update loki snapshots
* Add special case to test for multiautocomplete filter
* Use single select value in test
* Add test for using list picker
* Explicitly make sure elements are visible
* Do not use list selector when search is disabled
* Add margin to multi autocomplete dropdown
* Update loki snapshots
* Update loki snapshots
* Fix scroll for EditableDashboard in fullscreen mode
* Fix scroll for InteractiveDashboard in fullscreen mode
* Display fullscreen control to dashboards by default
* Fix issue with public shared questions empty footer
Previously these were omitted, and it caused invalid SQL to be
generated, since the ORDER BY (time unit not updated) and GROUP BY
(time unit updated) did not line up.
Fixes#49263.
Lays the foundation for a weighted probability approach to generating infinite sequences of valid queries.
Only has aggregations so far, but we can expand it to filters, breakouts, order-by, removals etc. etc. in later PRs.
* Update fullscreen dashboard filter on dark theme
* Use the latest base color name
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
* perf: Implement faster sync methods for postgres
Fixes#48575
Pulls work from redshift into the common postgres driver.
* Fix tests and formatting
* Move nested-field-column sync to sync functions so describe-fields will also get them
* Fix test
* Fix test
* Remove fixed safety test
* Add test specific database-supports feature for pk metadata
* Fix test
* Adrress PR feedback
* Fix test
* Don't use subselect for field-comment
* Fix quoting weird identifiers
* Make format string inline
* Remove unused component
* Remove extra space
* Remove cross-component dependency on the `Banner`
* Use named export
* Convert the Banner to TS
* Make `ReadOnlyBanner` use `Banner` under the hood
* Add test id to the banner
* Simplify render method and tests
* Add the missing test for ReadOnly banner
* Avoid using special ASCII characters for the common apostrophe
* Remove E2E spec
* Use `Banner` for `DatabaseInfoField`
* Use colors defined in the theme
* Remove the dependency on the `Banner` component again!
* Add a comment
* Use pattern matching
* Mongo objects should download as JSON, not EDN
Fixes#48198
Prior to this change, object columns (base or effective type of :type/Dictionary) were just formatted with `(str
value)` which results in a csv or json download containing EDN formatted objects.
This is a bug because we present object column values as json in the app, so the expected formatting of the download
should match this.
The formatter function now takes this type into account. As well, since this is a type of formatting that should be
always applied (even when format_rows is false), the function is modified to unconditionally apply the json/encode
formatting to dictionary types when encountered.
* add a test
* add proper condition to test
* card-download should be public
* uncomment json encoding formatter
* set-cell! should keep encoded json string for Objects
I think this is the correct change; I don't really understand the reason for wrapping, encoding, decoding, and then
string-ing that value. Maybe I'm missing something.
* Adjusted xlsx Object set-cell! implementation
* forgot the not... inverted
* Update cursor position when selection changes
* Add reproduction for #33640
* Revert "Add reproduction for #33640"
This reverts commit 573f6320f5.
We cannot reproduce the bug with tests, so having a test is useless.
* Add reproduction for #12586
* Do not show the dirty overlay when an error occurred
* Fix tests that rely on dirty overlay
* Add issue tag to test case
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add max-height to the click actions popover to prevent overflow and infinite scroll
* Add reproduction for #48851
* Update reproduction to speed it up
* Use existing helper for table header click
* Add max height to string filter picker
* Revert "Add max-height to the click actions popover to prevent overflow and infinite scroll"
This reverts commit 166478d892.
The audit collection is a bit weird. Users have permissions on the
collection but even superusers don't actually have write permissions on
the elements within it.
For now, I'm just excluding anything in the audit collection from the
"stale" endpoint, so we won't suggest cleaning these up, but I wonder if
maybe there's a cleaner way here. It feels weird to fake permissions
like this.
Fixes https://github.com/metabase/metabase/issues/49164
* Add new route
* Add the navigation link
* Guard the route
* Add first pass of the page structure
* Add the bottom section
* Add placeholder dummy YT videos
* Add images
* Track checklist being opened
* Track checklist item expanded
* Track checklist item CTA clicked
* PoC: Stop the video on accordion change
* Use actual onboarding videos
* Fix missing unique `key` prop
* Final content tweaks
* Override the accordion styles defined in the `theme`
* Use predefined t-shirt sizes where possible
* Update copy
* Conditionally display the example dashboard CTA
* Add very basic unit test coverage
* Show onboarding link to new instances only
* Remember the last position in the onboarding
* Fix the mock settings types
* Scroll last opened item into view
* Properly display docs links
* Another accordion style override
* Add tests for the Onboarding page contents
* Add E2E Snowplow tests
* Conditionally display xray item contents based on settings
* Split tests
* Make sure items scroll into view when expanded
* Do not show example question CTA if example dashboard doesn't exist
* Allow YT videos to play in fullscreen
* Add comments about the example question
* Update rules for when we show the onboarding nav link
* Test the conditional documentation links display
* Reduce E2E test flakiness
* document loader and error states
* document loader and error states in the docs
* revert changes to README.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* pie chart crashes when light or dark accents of instance colors overriden
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Add verifyNotebookQuery helper
* Use describeEE
* Fix DB id
* Refactor test
* Refactor tests
* Introduce customizeLinkText helper
* Use verifyNotebookQuery
* Fix test setup
* Fix static embed card download when param has accentuation
* Fix E2E util type
* Add a dashboard E2E test
* test question download with params
* Fix some names regarding tests
* Fix failed E2E tests
* Fix csp directives for embed previews
We set content security directives to allow for iframes on
dashboards. This list did not include 'self' so we can't actually host
an iframe pointing at our, well, self.
Embed previews work by just embedding an iframe with the dashboard and
this breaks if we don't allow iframes from our self.
* e2e test
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
### Description
This PR updates Macaw to the latest version, which exposes a new analyzer which handles compound queries.
It also finishes the migration to using tagged-union result maps consistently across all the interfaces.
This should also remove the atrociously verbose analysis error messages in CI.
* investigate hide columns
* add update viz settings in the sdk
* cleanup implementation
* add storybook for interactive question
* use the question callback to update viz settings
* remove onUpdateVisualizationSettings
* revert change to onUpdateVisualizationSettings
* add click action behaviour
* fix misspelling
* add onUpdateQuestion mock function
* add unit test for click action behaviour
* add e2e test for hiding question column header
This standards on a "just maps" approach to returning disabled, disallowed, or failed query analysis. This is as opposed to the hodgepodge approach of significant `nil`, untagged unions with keywords, and exceptions.
Essentially we have a classic "either" style tagged union, where `:error` is the tag.
* test(sdk): add tests for auth flow
* make the fetch token fn return something that will pass the validations
we'll add in the next project
* waitForRequest helper + add expects for card endpoints on api key test
* upgrades is-airgapped? defn to defsetting airgap-enabled
- uses the airgap-enabled settings value to remove Cloud section in
SettingsEditor
- "Metabase Enterprise Airgap" -> "Enterprise Airgap"
* hide still need help section in airgap mode
* move flag checking into selectors
* fe lint
* ts type check
* use optional chaining and nullish coalescing
* hide StillNeedHelp in ag mode
* adding airgap-enabled to createMockSettings
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* add allowed iframe host setting wip
* use allowed-iframe-hosts setting in the CSP header
* add a test for the frame-src csp directive
* Update allowed-iframe-hosts setting definition
* Add error state for forbidden iframe url domains
* Move out iframe e2e test suite
* Add e2e test
* Update error message in view mode
* Fix unit tests
* Update setting on the admin page
* Update error state
* Add links to error states
* Update docs links
* Update link anchors
* Add default allowed hosts to public setting
* Update allowed domain check logic
* Fix default value display in admin page
* Don't update setting without changes
* Update error message spacing
* correct the parsing of allowed-hosts string for CSP header entries
* fix test
* fix not handling wildcard ports
* Fix failing e2e test
* Fix subdomain test
* address review
- the parse-allowed-iframe-hosts fn is now memoized
- a * entry is handled and doesn't produce a weird *:* entry
- no more try/catch, errors in parsing will be logged but the list returns all valid entries
- when www. is encountered, an entry including www. is added
- trailing / is 'cleaned' and the entry is used as if there was no trailing /
* Fixup test for expecting a few more frame sources
* indentation fix for linter 🥲
* Fix type error
---------
Co-authored-by: Adam James <adam.vermeer2@gmail.com>
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
Co-authored-by: dan sutton <dan@dpsutton.com>
* ui implementation of new emb homepage, tests will come later
* change padding of the last card to be consistent with the top one
* update beta badge spacing
* update tests
* fix stories type error
* use quick-start for url
* update HomeContent tests
* update events + related e2e
* Update frontend/src/metabase/home/components/EmbedHomepage/StaticEmbedContent.tsx
* Add `graph.max_categories` viz setting
* Allow grouping series into "other" category
* Add tooltip
* Toggle "other" series visibility
* WIP: new implementation for `graph.series_order`
* Remove not used function
* Make "other" series a regular `SeriesModel`
* Fix stacked bar charts
* Remove colors from "other" tooltip
* Sort "other" tooltip values
* Update `graph.max_categories` setting
- negative number validation
- 0 value turns off grouping
* Fix Loki test
* Add basic loki test
* Add loki test for stacked chart
* Add loki test for stacked normalized chart
* slight refactor of series_order setting
* Add screenshots
* Handle different kinds of aggregations for "other"
* Handle different aggregation kinds in tooltips
* Add `graph.other_series_aggregation_fn` viz setting
* Fix type errors
* Fix unit tests
* settings ui
* remove sorting by value to match series order setting, other color
* hide grouped series controls
* Add e2e test
* Update screenshots
* Fix incorrect series grouping for stacked charts
* group series into other settings
* Update e2e test to work with new viz settings
* Disable drills for the "Other" series
* fix max_categories setting popover positioning
* fix total row has misaligned columns on other series values
* Remove redundant `click({ force: true })`
* Rename aggregation fn viz setting
* Move aggregation fn setting to the popover
* WIP
* Update popover
* Fix `it.only`
* Fix legend sync issue
* Disable "Other" category by default (temporary)
* Enable aggregation fn picker for MBQL queries (temporary)
* Move "Other" category summary to a tooltip row
* Add `graph.max_categories_enabled` to viz settings type
* Reuse already computed "Other" value in tooltips
* Add null check
* Fix series length check
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
This is to allow for the possibiliy of adding more metricsv2 usage and error metrics in the future while still
distinguishing the source of the errors.
### Description
This is a low level step towards applying table-level permissions to native queries.
It replaces the more generic and powerful `:ast-walker-1` analyzer with the more conservative `:basic-select` version we recently landed. The advantage of this version is that it protects against dynamic table references and false negatives due to masking.
This was throwing errors trying to `nippy/freeze!` an Atom.
That was because in certain circumstances a pMBQL query can end up in
`(:json_query metadata)` for a query, and those can contain an atom
due to `metabase.lib.cache` and both `CachedMetadataProvider` and
`InvocationTracker`.
I added `lib.query/serializable` as a single location to strip out
anything that isn't serializable, such as atoms. (And the entire
`:lib/metadata`, which is a wasted effort to serialize.)
* Fix reference in deprecation notice
* Add lock_filled icon
* Add view-only tag to the question header for questions with hidden source tables
* Simplify the useHasHiddenSourceTables helper
* Add typings to useHasHiddenSourceTables
* Show 'view-only' badge when the question is not editable
* Set up card endpoints
* Add test for hidden tables
* Add e2e test for View-only tag
* Add visibilityType to TableDisplayInfo
* Use unfiltered table display info to determine which tables are hidden
* Rename util to hooks
* Use rem for border-radius
* Sign in in beforeEach
* Add tests for more visibility types
* Fix unit tests
* Remove empty file
* Add test for unavailable cards
* Remove unnecessary card endpoint
* Add unit test for inaccessible card
* Move ViewOnlyTag to it's own file
* Add more unit tests for ViewOnly tag
* Use MBQL to create queryies where possible
* Increase the size of the overlay spacer
* Increase the delay of the popover opening
* Hard-code the popover delays
* Remove delay props from info components
* Fix unit test
* Shorten popover timeout to 500ms
* Make popover delay configurable again
* Increase openDelay for header info
* Fix a Couple bugs related to pivot exports
- pivot-grouping value, when not an int (in the case of Oracle dbs), would cause exports to contain zero rows. This
fixes that
- json formatted/unformatted for the dataset API endpoints (unsaved questions) was not being used, so all exports
were formatted
* json and csv use (int group) to check row inclusion
* grouping change fix tests
* json formatted/unformatted test
* xlsx formatters are correct when pivot-grouping col is removed
* cljfmt
* fix up failing tests
* fix test
* Update src/metabase/api/dataset.clj
* Reset the question's viz settings when converting to a model
* Update e2e/test/scenarios/models/reproductions.cy.spec.ts
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Test model metadata
* Test model metadata
* Test model metadata
* Test model metadata
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix#48878 by avoiding to use entity framework which can serve incomplete card objects
* Make sure canWrite is a boolean
* Remove redundant optional chaining
* Add repro for #48878
* Optimize
* Improve comment
* Fix test setup
* Automate npm version
* Make troubleshooting SDK release process easier
* Fix quote situation
* Allow deploying SDK to different branches at the same time
Because they won't be in conflict with each other
* Make checkout a bit faster
* Remove unnecessary console.log
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Tidy up `jq` command
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Tidy up multiple GitHub actions summary log
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Add helper to add utm params to url
* Add utm param to getDocsUrl
* Allow passing utm to docs url
* Add utm parameters to expression widget docs url
* Add useUrlWithUtm hook
* Make more utm props optional
* Use the utm hook and docs hook instead of adding utm tags manually to urls
* Simplify getDocsUrl
* Update unit tests for expression editor to contain utm params
* Only add parameters if they have a value
* Fix unit test
* Fix unit test
* Simplify docs url
* Use useDocsUrl on embedding page
* Use named arguments in useDocsUrl
* Modularize metabase.pulse; add metabase.pulse.core API namespace
* Rename metabase.pulse.send-test
* Split email result attachment stuff into its own namespace
* Update dashboard-subscription-test
* Allow metabase.email.result-attachment externally... for now
* More code cleanup
* Decouple models.card and email.messages
* Decouple models.collection and metabase.email.messages
* Hopefully the last fix 🔧
* Fix#48647🙁
* Fix event handler
* :render/text, not :pulse/text
* Test fixes 🔧
* Test fix 🔧
* RESPECT the modules
* Remove unused namespace
* defineEmbeddingSdkConfig to make typing easier when the config is created outside of the jsx
* also add defineEmbeddingSdkTheme, move them to index.ts as i don't have a good name for what file to put them into :shrugs:
* updated the readme to use the defineXXX functions
== Behaviour without this fix ==
See #48881
== Behaviour with this fix ==
1. Metabase starts with an empty application database, and a config file
containing configuration for a connected database
3. By the time it reaches "INFO metabase.task :: Task scheduler started",
`qrtz_triggers` contains `metabase.task.sync-and-analyze.trigger.1`
(sample DB) and `metabase.task.sync-and-analyze.trigger.2` (DB from
config file)
Closes: https://github.com/metabase/metabase/issues/48881
Co-authored-by: Noah Moss <noah@metabase.com>
* Use type date for case expression when there are Date and DateTime args
* Use logic for getting case expr type from annotate middleware
* Add e2e test
* Add bigquery test
* Update test
* cljfmt and comment
* Adjust e2e test
* Let non-admins see tables in recents
```shell
❯ http get "localhost:3000/api/activity/recents?context=views" Cookie:"metabase.SESSION=ba..3d" -pb
{
"recents": [
{
"can_write": false,
"database": {
"id": 6,
"initial_sync_status": "complete",
"name": "pg restaurants"
},
"description": null,
"display_name": "Restaurants",
"id": 112,
"model": "table",
"name": "restaurants",
"table_schema": "public",
"timestamp": "2024-10-15T22:11:39.412100Z"
},
{
"can_write": false,
"database": {
"id": 6,
"initial_sync_status": "complete",
"name": "pg restaurants"
},
"description": null,
"display_name": "Reviews",
"id": 110,
"model": "table",
"name": "reviews",
"table_schema": "public",
"timestamp": "2024-10-15T19:53:19.999445Z"
}
]
}
```
We were getting permissions of a "fake" table. Instead, let the db
select it and get the proper stuff. Worked for admins because
mi/can-read? is presumably always true for tables, and worked in tests
because we mocked mi/can-read?
* bump ci
* Add new promethues counter for :metabase-metrics/adjust-errors
To record errors in metabase.query-processor.middleware.metrics/adjust
* Add prometheus metrics test to metrics middleware tests
* Also inc :metabase-metrics/adjust-errors when throwing an exception
* Make splice-compatible-metrics private
It doesn't seem to be used outside of metabase.query-processor.middleware.metrics.
* Add test for :metabase-metrics/adjust-errors when exception is thrown
* Move inc-and-throw! helper into metabase.analytics.prometheus
* Add inc-and-throw! test to prometheus tests
* Rename inc-test to inc!-test and use approx= and metric-value helpers
* Add try/catch to metrics adjust and record metric errors in a single place
Record promethues metrics in a single place in metabase.query-processor.middleware.metrics and ensure we record
failures for any exception throw.
* Improve testing description string
* Rename :metabase-metrics/adjust-errors to :metabase-query-processor/metrics-errors
* Move metrics adjustment try/catch into a new helper
This ensures we count all exceptions generated by adjust-metric-stages.
* Add new prometheus metric to count number of MB metrics seen by the QP
This new metric is intended to count the number of metrics that the query processor metrics middleware sees and
attempts to adjust.
* Add more metrics metrics tests
- Ensure that queries with no metrics do not incremement either counter
- Ensure that successful calls to adjust do not increment the error counter
- Ensure that exceptions throw from other libs also increment the error counter
* Rename test to adjust-prometheus-metrics-test and add helper to reduce code dup
* If no metrics are found immediately return query in metrics middleware adjust
Don't attempt to adjust query if we don't find any :metric clauses.
* Appease cljfmt
* Improve metric description
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Improve epsilon test in approx=
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Remove adjust-metric-stages-counting-errors
This function was an attempt to avoid double-counting errors, but probably not worth the effort.
See
https://github.com/metabase/metabase/pull/48598#discussion_r1800197964https://github.com/metabase/metabase/pull/48598#discussion_r1800200421
* Count metrics directly when mocking prometheus/inc!
Based on PR feedback here:
https://github.com/metabase/metabase/pull/48598#discussion_r1800207674
* Remove prometheus/inc-and-throw!
Value prop was always questionable, but now that it's only called from a single place, just remove it and inline the
call to prometheus/inc!. Easy to add back later if it proves generally useful.
PR discussion thread
https://github.com/metabase/metabase/pull/48598#discussion_r1799728596
* Rename match-one-metric to find-first-metric
https://github.com/metabase/metabase/pull/48598#discussion_r1803321025
* Rename :metabase-query-processor/metrics-errors to metric-errors
https://github.com/metabase/metabase/pull/48598#discussion_r1803313818
* Throw an exception rather than logging a warning if we fail to adjust a metric
https://github.com/metabase/metabase/pull/48598#discussion_r1803328482
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* faster --watch command for the sdk
* apparently we want an empty line at the end of jsons, but prettier removes it?
* format tasks.json with prettier 😕
* vscods tasks, move watch task from webpack config to the fixup script
* cleanup webpack config
* adds clean + waits for dist folder when not existing
* minimal doc for the sdk commands
* adds data + schema for metrics stats ping
* remove comment
* annotate todos
* fill in the rest of the metrics values
- and add defaults
* fix some definitions + use a single timestamp
* shuffle stuff around to appease the namespace linter
- we aren't reaching into the api API, so the linter is more of a
formality here.
* update docstring
* fix jsonschema, maybe
* review responses
- revert changes to 1-0-0
- add metrics section to new file: 1-0-1
- bump ::instance_stats to "1-0-1"
- add tags into 1-0-1
- make the code return tags (empty for now until we know what to tag things.)
- also fix test that broke from shuffling settings around
* remove a footgun
* add tags to metrics,
add grouped_metrics to jsonschema 1-0-1
format 1-0-1
* indent
* cljfmt
* version should match filename
* update instance stats 1-0-1 schema
* require `tags` in grouped_metric + snowcat comment
* fix formatting noise
* update schema to make it valid
* remove grouped_metrics from instance_stats schema
* cljfmt appeasement
* unbin cache_num_queries_cached value
- alphabetize metrics generation
* we can now guarantee metric values will be ints
* jsonschema for instance_uuid, settings, and grouped_metrics
* add analytics_uuid and make it required
* lint + alphabetize instance stats json schema
* update setting key type
- add maxLength to some strings
* lint jsonschema
- Add description to setting.items.tags
- add maxLength, and {min,max}imum to setting.items.value
* Bump instance stats to 2-0-0
- remove analytics_uuid string length == 36 check
- adds assertion to ensure required fields are set (and it passes)
- adds info for embedding settings
* adds a grouped-metric to stats ping
- adds length info to the schema to pass jsonschema linting
* cljfmt
* Incremental Pivot Processing for Exports
WIP
Fixes pivot exports for CSV and xlsx.
The CSV export should use less memory by incrementally building up the data structure and aggregating necessary row
data right away, so the memory overhead becomes only as large as the total pivot result.
In cases where the pivot rows/cols do combine into many many columns and rows, this can still be a large set of data,
but it should behave much better now in most cases.
The Excel export is a little more straightforward: create the export rows in the same fashion, streaming one row at a
time, and just post-process the sheet to add the pivot table in one shot at the end.
* WIP adding row totals.
* aggregate totals as rows are added
Row, column, section, and grand totals are all aggregated as each row is added.
This means the final step of building pivot output becomes just an exercise of lookups/arrangement, no further
aggregation is needed.
* CSV pivot works per-row, export respects formatting
This is a big step forward; we don't need to hold the entire dataset in memory, we instead aggregate a row's data into
the pivot datastructure, which only holds onto:
- unique values for each pivot-row in a sorted set
- unique values for each pivot-col in a sorted set
- grand total for each measure N values, where N is number of measures, ususally 1 or 2
- row totals for each combination of each pivot-row * N measures
- col totals for each combination of each pivot-col * N measures
- totals for each 'section', determined by unique values of first pivot-row * N measures
- values for each measure in every 'cell'; Row Combos * Col Combos * N Measures
So, there can still be a decent amount of data to store; but it will never hold onto all of the 'raw rows' from the
dataset.
We can never completely guarantee that Row Combos * Col Combos * N Measures remains small, but two things let us move
forward anyway:
- there's now visible feedback in the app that the download is running (or if it's failed)
- Pivot table utility diminishes rapidly with huge output anyway; users still need to curate/set up their data
- effectively to improve the table's utility, so we can assume that a slow-to-download pivot table is also slow to
- use/less effective, and will likely be something the user doesn't want (as often).
* some test fixes
* now, if we export 'raw pivot rows', they don't show pivot-grouping
and they also don't include the 'extra' rows for totals/subtotals/grand totals (any row with pivot-grouping > 0).
This means that now the non-pivot version of a pivot table export will match what a user sees if they change the viz
to a regular table.
* remove old test
* re-incorporate some changes from master
* fix csv for non-pivots due to oversight in my changes
This is just a temporary change, I think I should clean up this bit of the code a little, I can probably make it a
little more readable and use some cleaner logic regarding if the rows are 'raw pivot rows' or not.
* start moving format_rows to POST bod, add pivot_results too
There's still wiring work to do, but this starts to add format_rows and pivot_results to POST body for the various API
endpoints. Also modify tests to improve coverage/consistency across downloads and alerts/subscriptions.
The tests will not pass on this commit, but fixes will be incoming
* native pivot tables in xlsx
* add precondition to pass migration linter
* try to get migrations fixed
* pasing pivot-results through api and attachments
* fix tests for format_rows in BODY vs query param
* tests!
* might have the tests all fixed now
* the pivoted export now respects col/row totals settings
* add test coverage for public questions and dashboards
* col and row totals work as expected
* build-pivot refactor for clarity
* docstring change + tiny refactor in helper fn
* see if dashcard download works with format_rows
* csv pivot handles nil values
* pass format_rows and pivot_results in :params not :body
* fix some other tests
* pivot-grouping col filtered out of xlsx
* pivot-grouping-col removed for all rows
* configurable pivot exports and attachments (#47880)
* exports fe
* specs
* ui
* specs
* format/unformatted now works for xlsx
* format test changes for xlsx formatting
* embedding endpoints accept pivot_results
* cljfmt and eslint fix
* empty
* embedding test should have formatting defaulted to true
* embed test fixes
* Use `Chip` for export settings widget
* downloads e2e test fix
* fix public download limit test
* public card download defaults
* fix public download defaults in some tests
* Fix visual test
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
* Remove 'Save as new question' toggle when saving questions from editing a metrics' ad-hoc question
* Allow visitQuestion to work for metrics
* Use visitQuestion in metric test
* Add test for #48555 (showing the 'Save as new?' ui for metrics)
* Add issue info to repro
* Use visitMetric, visitModel and visitQuestion helpers in createQuestion
* Move check to make it more clear what the save ui does
* Use cy.button for Done button
* Remove unnecessary .within
* Sign in as normal user
* Add missing dataset alias
* Refactor createDashboardWithQuestions, createQuestionAndAddToDashboard and editDashboardCard to TS function helpers
* Fix createNativeQuestionAndDashboard that got out of sync
* Rename Common Metrics to Metrics
* Rename Basic Metrics to Summaries
* Update chill mode button copy to 'Add a summary or metric'
* Rename 'Pick the metric you want to see' button to include summary
* Fix type in test
* Rename Summaries to Basic functions
* Fix test
* Undo cross-version test changes
* Add eslint-plugin-jest-formatting
* Turn on jest formatting for unit.spec and cy.spec files
* Fix e2e test formatting
* Fix unit test formatting
* Fix formatting in frontend/test
* Remove commented out code
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix more files
* Fix more lint
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add repro for #44171
* Do not save viz settings for metrics
* Lowercase description
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Rename variable to be more descriptive
* Use visitDashboard
* Fix typo in excludeVisualisationSettings
* Check the request does not contain visualization_settings too
* Allow visitQuestion to work for metrics
---------
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Remove `MB_API_KEY` env var
A bit awkwardly, we never set `:deprecated` on the setting before. We
can retroactively deprecate this as of v50.
I'm keeping the setting purely to emit the warning message on startup.
* Experimental: try splitting MySQL test jobs into 4 partitions intstead of 2
* user-http-request should make sure users are initialized
* Fix MySQL deadlocks in tests
* Bump init timeout to 90 seconds
* Fix metabase.api.session-test/logout-test
* Add 'Create metric' button to empty metrics page
* Add e2e test for 'Create metric' button
* Remove duplicate main on Browse page
* Add docs link
* Split off common test for BrowseMetrics
* Add whitespace
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Use cy.button
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Hide Create metric button when the user does not have data access
* Use findByText for link
* Add test for hiding the 'Create metric' button when the user does not have data access
* Fix unit tests
* Improve copy
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add whitespace
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add explanatory comment for enterprise
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add repro for #47058
* Delay the response and test for the actual metric button
* Move setup to before and find summarize block using helper
* Use beforeEach
* Write Vitess limitation for row limits in MySQL databases
* Update docs/databases/connections/mysql.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Relax the arg types to ExpressionArg for concat expressions in the legacy schema
Relax the arg types to ExpressionArg for concat since many DBs allow to concatenate non-string types. This also aligns
with the corresponding MLv2 schema and with the reference docs we publish.
Fixes#39439
* Add nested concat schema tests
* Add nested-concat query-processor tests
* Always startup prometheus metrics
previously only started up when a port was provided
```
MB_PROMETHEUS_SERVER_PORT=9191 java -jar metabase.jar
```
But these counters are useful to be included in anonymous stats. So
let's start up the collectors and then we can get their values like:
```clojure
prometheus=> (dotimes [_ 500] (inc :metabase-email/messages))
nil
;; prometheus/value is iapetos.core/value here
prometheus=> (-> system :registry :metabase-email/messages prometheus/value)
501.0
```
* rename `metabase.analytics.prometheus/inc` to `inc!`
it side effects a value and now no longer shadows `clojure.core/inc` so
we're all happy
* don't set enable-embedding and setup-embedding-autoenabled in setup actions + cleanup
* update jest test
* shouldn't expect the secret key to be there either 🙃
This PR reimplements driver/describe-table for MongoDB. Before we would query a sample of documents from a collection and analyse them in Clojure. Instead, we now now execute a query that does a similar aggregation, but most of the calculation is done in the Mongo database.
Based on a few tests the performance is slightly slower when the collection contains small or deeply nested documents but much faster for large ones. But the main difference is in memory usage. This uses very little memory in the Metabase instance because all of the aggregation is done in the database.
Nested fields are a naturally recursive problem but here we unroll potential recursions to a `max-depth` number of queries that look for nesting at each depth level.
* ~ use DB to describe the table
* ~ optimize root query
* ~ nested-level-query works and gets objects too
* + root query gets objects too
* + driver/describe-table :mongo works
* ~ remove old implementation
* Various fixes for faster sync
Upgraded driver to 5.2.0
Updated data load to insert many rather than 1 row at a time.
Dropped max-depth to 7, see comment.
---------
Co-authored-by: Case Nelson <case@metabase.com>
* Move mbql.u/{mbql-clause?,is-clause?,check-clause} to schema.helpers
And remove duplicate definitions.
Closes#39333
* Fix docstring of schema.helpers.mbql-clause?
This func is apparently no longer used by code in `normalize`, which has it's own local def.
* Use more granular permissions
* Simplify the "upload spreadsheet" button logic
* Simplify the "add database" button logic
* Go back to using the existing `databases` prop
* Conditionally display `UploadInput`
* Refine `canUpload` logic
* Reduce prop drilling
* Make visual logic more "reader-friendly"
* Fix and expand tests
* Pass down the existing prop `hasDataAccess`
* Remove the Metabase "learn" link
* Move sub-components out of the render
https://github.com/metabase/metabase/issues/41919#issuecomment-2400542908
This issue goes away in 5.3.1 of apache poi. Seems like they have a
yearly release cadence so we can ensure this exists for the time being
and then remove this when we can bump to 5.3.1
```diff
index b763a1ffdf..c87992c935 100644
--- a/deps.edn
+++ b/deps.edn
@@ -122,7 +122,7 @@
{:mvn/version "2.23.1"} ; allows the slf4j2 API to work with log4j 2
org.apache.logging.log4j/log4j-layout-template-json
{:mvn/version "2.23.1"} ; allows the custom json logging format
- org.apache.poi/poi {:mvn/version "5.2.5"} ; Work with Office documents (e.g. Excel spreadsheets) -- newer version than one specified by Docjure
+ org.apache.poi/poi {:mvn/version "5.3.1"} ; Work with Office documents (e.g. Excel spreadsheets) -- newer version than one specified by Docjure
org.apache.poi/poi-ooxml {:mvn/version "5.2.5"
:exclusions [org.bouncycastle/bcpkix-jdk15on
org.bouncycastle/bcprov-jdk15on]}
```
* [MS 1.1] Better Onboarding via First-Time UX (#48202)
* Add new `learn` icon
* Add initial visual elements WIP
* Add a menu WIP
* Use named export instead of a named one
* Encapsulate the menu item logic
* Redo the styling
* Add todo notes
* Get rid of the `style` prop where possible
* Fix CSS logic for the container
* Collapse section when the sidebar is closed
* Use Mantine prop instead of the `style`
* Fix the translation issue
* Collapse section completely when the sidebar is closed
* Remove hard coded color reference from SVG
* Simplify element positioning
* Collapse sidebar when closed
* Fix E2E tests unscoped selectors
* Rename props
* Extract `SidebarOnboardingSection` logic into a separate component
* Allow only admins to add data
* Remove "Add your own data" button
* Simplify menuitem component
* Add initial unit tests
* Use predefined vars for the CSV button subtitle
* Show Upload CSV modal
* Enable file uploads
* Improve file organization
* Tidy up types
* Add analytics
* Remove "Upload CSV" button from the sidebar
* Fix "unmocked GET request"
* Make menu usable on small screens
* Make sure sidebar is scrollable
This PR
* mocks audit requests in tests of `DashboardInfoSidebar` and `QuestionInfoSidebar` (these tests were flaking)
* introduces a helper, `setupAuditEndpoints`, to facilitate this in the future
* uses the helper in one other place where audit requests were getting mocked
* Maintain selected folder state in Entity Picker (#47309)
* Keep tabs mounted
* Rename TablePicker onChange to onItemSelect to match QuestionPicker and also call it for folders
* Inline small functions
* Remove unused type
* Rename
* Remove TODOs
* Remove duplication
* Reduce diff
* Rename
* Lift state from TabsView
* Remove redundant callback
* Store folder state in DataPickerModal
* Refactor computing tabs
* Consolidate search tab definition with recents tab definition
* Simplify computeInitialTab interface
* Extract RECENTS_CONTEXT
* Store tab state in DataPickerModal
* Propagate initial tab state
* Fix a flicker
* Keep tab state in EntityPickerModal
* Fix typing computeInitialTab
* Remove Notebook prefix
* Pass onItemSelect to individual tabs from EntityPickerModal
* Auto select folder
* Fix types
* Adjust remaining tabs definitions to use render
* Adjust tabs definitions to use render in tests
* Fix inverted condition
* Fix TS error
* Fix types
* Rename EntityTab to EntityPickerTab, TabId to EntityPickerTabId
* Rename EntityTabRenderProps to EntityPickerTabRenderProps
* Separate tab models from tab ids
* ESLint
* ESLint
* Recognize folders inside EntityPickerModal
* Add missing prop
* Remove console.log
* Simplify handleFolderSelect
* Revert changes that will go to a different PR
* Fix unit tests
* Refactor
* Remove keepMounted prop
* Fix type
* Remove useMemo from tabs computation (#47505)
* Extract getTabs for DataPickerModal
* Extract getTabs for EntityPickerModal
* Add explicit return type
* Revert "Add explicit return type"
This reverts commit feb3f58c3c.
* Revert "Extract getTabs for EntityPickerModal"
This reverts commit 4e875a1fc1.
* Revert "Extract getTabs for DataPickerModal"
This reverts commit cdda8ea185.
* Remove useMemo
* Maintain tab state in Entity Picker (#47794)
* Store picker path state in data picker modal
* Adjust remaining QuestionPicker usages
* Store collection picker state
* Use dedicated onPathChange prop
* Store dashboard picker state
* Improve naming
* Fix unit tests
* Remove accidental code
* Return to previously opened tab when search input is cleared in Entity Picker (#47798)
* Restore previous tab when clearing search input
* Add a unit test for new behavior
* Update comment
* Don't use effect
* Add more tests
* Reduce diff
* Fix flakes
* Change search input placeholder depending on selected folder & tab in Entity Picker (#47801)
* Make TabFolderState generic, pass folder to EntityPickerSearchInput
* Simplify interface
* Extract getSearchInputPlaceholder
* Reduce diff
* Remove TODO
* Implement it
* Shorten code
* Reduce diff
* Use database to scope search if there's only 1 schema
* Add scoped search in the Search tab in Entity Picker (#47839)
* Cleanup entity picker search types (#48098)
* Add e2e tests for local search in the entity picker (#48122)
* Sync the value back into selectedItem when it changes
* Await element
* Handle empty tabs
* Pick everywhere tab
* Click Everywhere in unit tests
* Use deep equality to sync value
* Sync back the path value to the dashboard too
* Run test as admin to avoid (unrelated) permissions issues
* Filter scoped results too
* Fix post-merge issues
* Fix type issues
* Fix null scoped search results
* Sign in as admin to appease the tests
* Use useDeepCompareEffect when syncing back state
* Add default value for searchResultFilter to avoid double condition
* Avoid using useEffect to sync selectedItem
* Avoid overwriting path in useEnsureCollectionSelected
* Fix e2e test
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Parallel driver tests PoC
* Set fail-fast to false for now
* Try splitting up non-driver tests to see how broken tests are
* Whoops fix plain BE tests
* Ok nvm I'll test this in another branch
* Fix fail-fast
* Experiment with the improved Hawk split logic
* Fix some broken/flaky tests
* Experiment: try splitting MySQL 8 tests into FOUR jobs
* Divide other Postgres and MySQL tests up and use num-partitions = 2
* Another test fix 🔧
* Flaky test fix 🔧
* Try making more stuff fast
* Make athena fast??
* Fix a few more things
* Test fixes? 🔧
* Fix configs
* Fix Mongo job syntax
* Fix busted test from #46942
* Fix Mongo config again
* wait-for-port needs to specify shell I guess
* More cleanup
* await-port can't have a timeout-minutes I guess
* Let's only parallelize MySQL for now.
* Cleanup action
* Cleanup wait-for-port action
* Fix another flaky test
* NOW driver tests will be FAST
* Need to mark driver tests too
* Fix wrong tag
* Use Hawk 1.0.5
* Fix busted metabase.public-settings-test/landing-page-setting-test
* Fix busted `metabase.api.database-test/get-database-test` etc. (hopefully)
* Fix busted `metabase.sync.sync-metadata.fields-test/sync-fks-and-fields-test` for Oracle
* Maybe this fixed `metabase.query-processor.middleware.permissions-test/e2e-ignore-user-supplied-perms-test` maybe not
* Fix busted metabase.api.dashboard-test/dependent-metadata-test because endpoint had differemt sort order than test
* Ok my test fix did not work
* Fix metabase.sync.sync-metadata.fields-test/sync-fks-and-fields-test for Redshift
* Better test name
* More test fixes 🔧
* Schema fix
* PR feedback
* Split off test partitioning into separate PR
* Fix failing Oracle tests
* Another round of test fixes, hopefully
* Fix failing Redshift tests
* Maybe the last round of test fixes
* Fix Oracle
* Fix stray line
* add update channels in product
* support for changing release notes to show beta and nightly info
* dont export setting
* obey the linter and add tests
* export setting
* update e2e tests
* clojure magic
* clojure-foo
* better localization
* sorry mr linter
* add more tests 💪
* embed cards
* safety
* rename params
* review, renaming
* min and default size of iframe cards
* move ml auto to parent
* use truncate
* spec failures
* empty state
* consistent iFrame copy
* renaming, no fe tracking
* iframe copy
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
* wip locale header middleware
* wip fe code for the locale provider for both public/static and sdk
* clean up code
* move api.baseurl assignment outside of useEffect to make it work on first render
* e2e test for locale
* Revert "wip locale header middleware"
This reverts commit c215eac3f8.
* remove public/static code as we want to focus on the sdk
* adds docs for `X-Metabase-Locale` header and for why we don't use I18NApi.locale
* Update frontend/src/metabase/lib/api.js
* doc: explain why we have the unused state
* Update the labels for the Edit question action for models and metrics
* Use the metric and model editor by default
* Add e2e tests for metric and model cards
* Remove console.log
* Add extra mode for editing metrics or models
* Use slug to do exact match of path
* Rename editor mode to query
* Rename as param to mode
* Add BE test for exporting self-joined renamed columns
* Add e2e test for exporting self-joined renamed columns
See also
- Not renamed fields in a same table join inherit the renamed name in exports #48046
- backport: Support both name & field ref-based column keys in viz settings on read and upgrade on write #48243
* add sunburst to pie chart
* fix type errors
* consider all slice percent values for formatting percent on chart
* fix lint error
* refactor nested functions outside of parent
* create and use getArrayFromMapValues helper
* fix bug with percent significant digits
* improve color picker
* fix total visibility in small dashcards
* fix missing field remappings
* use aggregated value for click behavior
* update minimum slice percentage logic
* fix total text truncation
* remove colors from tooltip for middle and outer slices
* update e2e spec
* update snapshot
* update ring setting titles
* address feedback
* update comment
* fix: bigquery more resilient querying
Inline some function calls to make it easier to track what's happening.
Make sure that cancellation during the initial query and subsequent page
fetches are handled properly. Explicitly throw when cancelled.
Only retry queries if bigquery says they are retryable.
Try to cancel the BigQuery job if an exception or cancellation occurs.
* Add comment explaining execution flow
* Bump bigquery deps
* Bump biquery dependencies
* Fix tests
* Fix formatting
* Improvements to SDK readme
* Add live demo site to SDK readme
* Remove suggested workarounds for SSR
* Use - for bullets for consistency
* Remove Loom
* Update enterprise/frontend/src/embedding-sdk/README.md
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* fix(sdk): fix zindex issues after switch to portals
* Target portal to a different root than popovers and co
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* trash
* Trash update
* review comments
* link
* why are you complaining
* text to tables
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* review comments
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Add metrics to table pane
* Add test for metric in data reference sidebar
* Clean up other reference tests
* Add test for when the table has no metrics
* Use helper for sidebar header title
* Rename sidebar to create distinction from existing helper
* fix(sdk): modal pushing content below when opened
* render the portal below the content so we don't need the zIndex
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Simplify browse components imports
* Move metrics-related browse components to separate folder
* Move models related browse components to separate subdir
* Split up browse/test-utils
* Move schemas related browse code to separate folder
* Move databases related browse code to separate folder
* Move tables related browse code to separate folder
* Fix import paths
* Avoid drilling into metric module
* Avoid drilling into models module
* Move enterprise tests for browse models
* Format comments
* Remove unused sortModelsByVerification plugin helper
* Support skipToken in useFetchModels
* Simplify models content verification
* Fix filter button query
* Do not filter recent models if there are no verified models
* Use filterButton helper everywhere
* Fixup comment
* Use new var-style color
* Await dashcard query when editing
* Reload to fix flake
* Reset filters to fix flake
* Await dashcard query when editing
* Wait for query
* WIP
* Hard-visit dashboard
* Split up test to avoid navigation issues
* Do not cache all token check failures
We want to cache token checks to avoid an issue where we repeatedly ask
the store "hey, is this token valid?? is this token valid?? is this
token valid??" for the same token.
However, transient errors can also occur. For example, maybe a network
issue causes the HTTP request to fail entirely. In this case, if we
cache the result, the user needs to restart metabase (or wait 5 minutes
until the cache is cleared) before they can attempt to validate their
token again.
This PR moves the cache logic deeper into the stack. We want to cache
"successful" responses from the store API - cases where the store has
told us categorically that the token is or is not valid. We don't want
or need to cache other things that might happen. Maybe your token isn't
the right length - we can recalculate that, it's ok. Maybe you get a 503
error from the Store - we should let you retry. Maybe your network is
having issues and you can't contact the Store at all - again, we should
let you retry.
The one potential issue I see here is that if the store goes down, we'll
massively increase the number of requests we send to the store,
potentially making it harder to recover. If this is a concern, I can add
a circuit breaker: if we repeatedly get errors back from the store, back
off and stop making requests for a while.
* Add a circuit breaker to store API requests
In the pathological case where the store goes down for >5 minutes, the
cache will expire and all instances everywhere will start repeatedly
making requests for token validation at once. This might make recovering
from an outage more difficult.
This adds a circuit breaker around the API request. If the call
repeatedly throws (5XX errors, socket timeouts, etc.) then we'll pause
for 1 minute, during which time all calls to token validation will
immediately fail without making any request to the API.
After one minute, we'll allow one request through to the API. If it
succeeds, we'll go back to normal operation. Otherwise, we'll wait
another minute.
This was old logic to support certain drivers (eg. pre-JDBC Druid) and
isn't required for most. It's perfectly sound to filter or even break
out on a datetime column without bucketing.
Adds a new `:temporal/requires-default-unit` driver feature, and enables it only for the legacy Druid driver.
Fixes#47341
* Reset the dashboard parameter when setting the default value
* Fix test for filter default
* Add test for setting parameters when editing the defaults
* Use plain dispatch
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Use cy.log instead of comments
* Use icon selector
---------
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
Fixes 46575
Creating a Pivot Table Question that is based off of a model that has at least one column derived from a join failed
to display row totals.
This is because the pivot-options map was being mis-calculated; not all column indices were correctly found/passed in
to the :pivot-rows or :pivot-cols keys, causing the pivot query not to compute all necessary data.
Here, I just modify the :lib/source key of the columns whose source is a card (as determined by the existence of
:lib/card-id). The columns being checked will all have :source/breakout, which caused, in the issue's repro example,
the "NAME" column to be missed. If it instead has :lib/source :source/card, the logic inside
`lib/find-matching-column` works.
* IS_EMBEDDING_SDK_BUILD -> IS_EMBEDDING_SDK to keep things simple, as we already have that name in other places
* don't load metabase css in sdk stories
* first two stories
* make the store not a global "singleton" anymore
* basic test for font-family styles in the SDK
* allow any custom font in theme.fontFamily as we actually support it
* fix tests using hardcoded default font family
* Databricks JDBC driver base
* Add databricks CI job
* WIP data loading -- it works, further cleanup needed
* Cleanup
* Implement ->honeysql to enable data loading
* Hardcode catalog job var
* Implement driver methods and update tests
* Derive hive instead of sql-jdbc
* Cleanup leftovers after deriving hive
* Run databricks tests on push
* Cleanp and enable set-timezone
* Disable database creation by tests
* Add Databricks to broken drivers for timezone tests
* Exclude Databricks from test
* Enable have-select-privilege?-test
* Restore sql-jdbc-drivers-using-default-describe-table-or-fields-impl post rebase
* Restore joined-date-filter-test
* Adjust to work with dataset definition tests
* Adjust alternative date tests
* Remove leftover reflecttion warning set
* Update test exts
* cljfmt vscode
* Add databricks to kondo drivers
* Update metabase-plugin.yaml
* Update databricks_jdbc.clj
* Rework test extensions
* Update general data loading code to work with Databricks
* Reset tests to orig
* Use DateTimeWithLocalTZ for TIMESTAMP database type
* Convert to LocalDateTime for set-parameter
* Update test extensions field-base-type->sql-tyoe
* Update database-type->base-type
* Enable creation of time columns in test data even though not supported
* Fix typo
* Update tests
* Udpate tests
* Update drivers.yml
* Disable dynamic dataset loading tests
* Adjust the iso-8601-text-fields-should-be-queryable-date-test
* Update load-data/row-xform
* Add time type exception to test
* Update test data loading and enable test
* Whitespace
* Enable all driver jobs
* Update comment
* Make catalog mandatory
* Remove comment
* Remove log level from spec generation
* Update sql.qp/datetime-diff
* Update read-column-thunk
* Remove comment
* Simplify date-time->results-local-date-time
* Update comment
* Move definitions
* Update test extension types mapping
* Remove now obsolete ddl/insert-rows-honeysql-form implementation
* Update sql-jdbc.conn/connection-details->spec for perturb-db-details
* Update load-data/do-insert!
* Remove ssh tunnel from driver as tests do not work with it
* Update test
* Promote ::dynamic-dataset-loading to :test/dynamic-dataset-loading and modify corresponding tests
* Adjust to broken TIMESTAMP_NTZ sync
* Update read-column-thunk to return timestamps always in Z
* Comment
* Disable tests for dynamic datasets
* Return spark jobs into drivers.yml
* Update Databricks CI catalog
* Remove vscode cljfmt tweak
* Update iso-8601-text-fields-expected-rows
* Update datetime-diff
* Formatting
* cljfmt
* Add placeholder test
* Remove comment
* cljfmt
* Use EnableArrow=0 connection param
* Remove comment
* Comment
* Update tests
* cljfmt
* Update driver's deps.edn
* Update tests
* Implement alternative `describe-table`
* WIP Workaround for timestamp_ntz sync, will be thrown away probably
* Update metabase-plugin.yaml with schema filters
* Update driver to use schema filters and remove now redundant sync implemnetations
* Update tests
* Update tests extensions
* Update test
* Add feature flags for fast sync
* Implement describe-fields
* Implement describe-fks-sql
* Enable fast sync features
* Use full_data_type
* Comment
* Add exception for timestamp_ntz columns to new sync code
* Implement db-default-timezone
* Add timestamp_ntz ignored test
* Add db-default-timezone-test
* Fix typo
* Update setReadOnly
* Add comment on setAutoCommit
* Update chunk-size
* Add timezone-in-set-and-read-functions-test
* Drop Athena from driver exceptions
* Use set/intersection instead of a filter
* Add explicit fast-sync tests
* Update describe-fields-sql and add comment
* Add preprocess-additional-options
* Add leading semicolon test
* Disable dataset creation and update comment
* Rename driver to `databricks`
* Use old secret names
* Fix wrongly copied hsql list
* Temporarily allow database creation
* Add *allow-database-deletion*
* Temporarily allow database creation
* Disable database creation
* cljfmt
* cljfmt
* Require `:encryption` for string settings
For settings that are not typed as JSON, CSV, or strings, encryption now
defaults to `:no` (*except* if you have
explicitly marked your setting as `:sensitive?` - these will default to
`:when-encryption-key-set`).
I went through all our settings and provided what I think are reasonable
values here. I tried to be conservative - when I wasn't sure whether a
stored setting was sensitive, I kept it as encrypted. For example, the
`ldap-port` setting is probably non-sensitive but theoretically someone
could be using a weird port for security-by-obscurity, so I kept that
encrypted.
* Change possible values for `:encryption`
`:maybe` was confusing: let's be more explicit that the value will be
encrypted `:when-encryption-key-set` to make it obvious what actually
turns encryption on and off.
* Pre-fill cloud user info from search params
* Autofocus password field for hosted instances
* Add tests
* Parse setup user info and store it immediatelly
* Autofocus password input field for hosted instances
* Add e2e reproduction
* Fix the user info
* Fix unit test
* Support skipToken in useFetchMetrics
* WIP
* Add user setting for verified metrics
* Add verified metrics to the VERIFIED_CONTENT plugin
* Use metric plugins for verified metrics
* Move helpers down
* Do not filter for verified metrics if there are none, regardless of user setting
* Add label to filter button
* Revalidate search when card gets (un-)verified
* Add e2e test for verified metrics
* Avoid checking for verified metrics on when content-verification is not enabled
* Fix broken test
* Simplify content verification plugin structure for metrics
* Add content moderation types and API helpers
* Update content moderation plugin to make use of the moderation api in metabase/api
* Fix lint
* Fix missing list tag
* Store verified metric filter preference
* Move all helpers to the bottom
* Ensure the user setting gets updated when chaning the filter
* Add comment about component similarity
* Undo changes to embedding-sdk
* Use default values for parameters in dashboard filters when used from click action
* Add test for setting non-specified parameters to default in dashboard accessed via click action
* size and padding fixes
* linkify and align stuff in question details
* reorganize and divide question overflow menu
* hide history for analytics content
* fix verification icon alignment
* whoopsie lets commit all the files
* fix css file name
* removes the default-ee snapshot in favor for a blockUserGroupPermissions util
* commit default-ee removal i forgot, replace default strings with undefined
* fix oopsie
* fixes failures for not removing enough permissions for tests in sandboxes.cy.spec.js
* rename var
* fixes some tests where data permissions were overwriting sandboxing permissions, makes assertions in tests where sandboxing user is used but the test also passes if you were to sign in as the normal user, fixes one test that sandboxes the wrong table for the test
* do not push - wip changes
* add assertions to that network requests are provided sandboxed data, fix faulty test in dashboard-filters-source.cy.spec.js
* use helpers to clean up parts of pagination assertions
* rename assertDatasetIsSandboxed to assertDatasetReqIsSandboxed
* adds assertions on api response data
* clean up
* fix typo
* refactor helper to be easier to follow
* golfing
* pr feedback
* fix
* fix
* Return the table id from csv upload (as a header)
* Add date format that google sheets uses
* Smuggle id
* done?
* comment
* with empty h2 db
* dashing
* fix flaky test
* Parse the correct date class
---------
Co-authored-by: Alexander Solovyov <alexander@solovyov.net>
* Pivot Exports/Downloads Should Not include pivot-grouping or 'Extra' Rows
Fixes 46561
When downloading a pivot table's data, a column 'pivot-grouping' is included as well as 'extra' rows that correspond
to various totals in the table.
This PR removes the pivot-grouping column and any of these 'extra' rows from the downloads/exports.
* fix xlsx
* Fix csv
* fix json
* add test and fix some failing tests
* fmt
* Update src/metabase/query_processor/streaming/csv.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* static viz table renders filter out pivot-grouping
* address review feedback
* add some comments to try explain the changes
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Fixes#47426
Date operations return values based on the timestamp offset rather than
the session/db timezone. Converting to `timestamp_ltz` first ensures
that we get predictable results.
In the test, it is important that database timezone matches
report_timezone so that post-processing gives proper results.
* Ensure :effective-type for columns from an aggregation in a card
Fixes#47184
* Synchronize effective-type with the base-type on override
When :base-type in the column metadata is overridden with the :base-type in the field ref, set it as :effective-type
too. If :effective-type is also set in the field ref, it wins.
* add license problem hook
* add warning banner
* style cue component
* update cue component
* report issue to console
* report issue to console
* use sdk loader for pre-initialization
* extract print problem method
* use non-standard shadows
* add unit tests
* prevent console log spam during tests
* mock the embedding token feature
* mock the embedding token feature by default
* add more test cases
* add test case
* add test cases
* increase popover z-index
* simplify test setup
* simplify test setup
* do not load individual component when a license error happens
* move SdkLicenseProblemDisplay component out
* replace colors with color-mix
* replace colors with color-mix
* make the text unthemed to ensure legibility
* hard-code colors to prevent them from being themed
* hard-code colors to prevent them from being themed
* update border styles
* make the chevron unthemed
* unset console text color
* only set stroke on errors
* optimistic query settings
* document the sdk license condition
* remove redundant condition
* add notes on api keys always being allowed on localhost
* move the no auth method message to warning component
* rename sdk license problem to usage problem
* add eslint rule for stories
* rename sdk license problem to usage problem
* move sdk usage problem display out of the init controller
* specify fallback font if instance is down or CORS error
* add comments on conditions pending on the CORS PR
* add notes on non-localhost checks
* make test ids constant
* add test case for neither provided
* fix types
* address code reviews
* pass `embedding-sdk` flag in `token-features`
* address code review
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* remove appName from console logs
* rename embedding-sdk to embedding_sdk
* add the embedding_sdk premium token feature
* do not setup enterprise plugin in sdk test
* do not localize visual cue messages
* fix unit test in setup-sdk-auth
* rename tests
* remove unused eslint-disable
* enable embedding_sdk premium feature for sdk unit tests
* revert security_test.clj to use embedding instead of embedding-sdk
* remove unneeded test
* changes to support the style leak fixes
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Rollback to legacy unique-name-generator in annotate-native-cols
* Add native query long column alias test to nested queries tests
* Add native query long column alias test to dataset api tests
* Add native query long column alias test to models e2e tests
* Move limit into inner native query
* PR suggestion: simplify lambda argslist
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* PR suggestion: remove unnecessary mt/with-temp-copy-of-db
* Tweak test name
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* add offset e2e tests
* mark test not flaky
* add basic tests
* add more tests
* WIP
* working version
* cleanup
* oops
* improve naming
* fix test
* improve formatting
* update comment and ts
* add a test for metric
* space
* update tests and create a new issue
* add one more test
* re-enable test
* make imports consistent
* rewrite tests to put data through UI
* first iteration to scope sdk styles to our components
* basic css reset for buttons
* clean up duplicated css
* make sdk components use instance font if no font is passed
* cleanin up and remove defaultProps not needed
* fix tests
* adds EMBEDDING_SDK_PORTAL_CONTAINER_ELEMENT_ID
* Update enterprise/frontend/src/embedding-sdk/components/public/MetabaseProvider.tsx
* Update enterprise/frontend/src/embedding-sdk/components/public/MetabaseProvider.tsx
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* fix import
* prettier --write
* we don't need theme.fontFamily, added a comment
* rename EMBEDDING_SDK_PORTAL_CONTAINER_ELEMENT_ID to EMBEDDING_SDK_PORTAL_ROOT_ELEMENT_ID
* fix unit test
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* progress bar
* rephrase
* better word
* streamline the article
* rename image
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Initial commit of controlled upgrades
- new setting upgrade-threshold (`MB_UPGRADE_THRESHOLD`)
number 0-100
- conditionally removing latest from upgrade checks
AS OF RIGHT NOW IT ALWAYS REMOVES the latest.
Should help FE make this optional
* dumb mistake
* handle no latest version info
* implement and basic test
* More tests, refactor name
* more tests
* cljfmt does not like commas in source code.
* Just move site-uuid to the declaration and don't declare it
nothing is sacred in the settings order. no reason. i think i was just
trying to minimize the diff, which a dumb and ignoble goal.
* add context strings
* update unit tests
* use display version
* Have upgrade threshold include current major version
hopefully helps rotate people from early to late, and late to early
across major versions. Idea from sanya and quite nice!
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* more detailed question info
small styling updates
refine styling to match designs
update more e2e test
remove some changes
small styling updates
more question details
more question details
DateTime to typescript
test updates
type updates
export fix
fixes
* Icon Updates
* consolidate styles
* test updates
* address review comments
* Render values for temporal metrics
* Render data as rich
* Add tests for temporal metric
* Add TODO about isDate
* Check for errors in dataset
* Add unit test for timeseries dataset
* Add check for rows earlier
* Group tests better
* Add e2e test for metrics with no value
* Add test for metric with error
* Fix some minor collections issues
- explicitly provide access to the trash to all users, in the same way
we provide access to personal collections. Due to migration order, we
don't necessarily have a permissions row with the correct `perm_type`,
`perm_value`, and `collection_id` for the Trash collection. That's ok -
we don't actually move things to the Trash, so there isn't any
item where `collection_id=$trash.id` - but this may affect things like
effective ancestors for children of the trash. Let's be explicit about
the permissions that users have.
- replace a case where we manually calculated effective location and
then got the parent_id with just hydrating `:effective_parent`. This is
more efficient.
- replace the simple hydration method for `effective-location-path` with
a batched hydration method that fetches `visible-collection-ids` *once*
and then uses it to figure out the effective location path for each
collection passed.
This PR tests that the user can assign any of three cache invalidation strategies ("duration", "adaptive", and "don't cache") to dashboards and questions. The tests check that the following things are true:
* These strategies can be applied directly (via the sidebar)
* Dashboards and questions can inherit their caching strategy from the database that underlies them
* Dashboards and questions can inherit their caching strategy from the instance-wide default policy
* When a question inside a dashboard has been assigned a strategy, this is overridden by the strategy assigned to the dashboard.
What these tests do not examine:
* the schedule strategy
* the minimum query duration
* edge-case strategies, such as very long durations or very large values for the adaptive strategy multiplier
These may be addressed in follow-up work.
Closes#45355
* Alway render models and metrics links
* Remove useHasMetrics and useHasModels helpers'
* Update metrics test for metrics in sidebar
* Fix unit test
* Fix e2e tests
* Move stale endpoint to /ee/stale/${collection-id}
It's an enterprise-only endpoint, so it makes sense to put the whole
thing under enterprise.
* refactors ui to use ee stale endpoint
lint fixes, change endpoint location
fix failure e2e test
* Fix endpoint location
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* Don't encrypt `site-locale`
* Fix test for `site-locale`
The previous test required encryption. Theoretically the referenced
error just depends on the fact that something is logged during the
process of getting the value, so I reworked it to depend on that
directly rather than triggering it through an encryption key rotation.
* Add metric actions
* Fix dot menu width
* Refetch card when it's being (un-)bookmarked
* Add tests for metric dot menu
* Make sure bookmarked metric appears in sidebar
* Make sure restoring a metric works
* Add more tests for readonly users
* Use non-root collection for metric test
* Render Scalar metric values
* Add tooltip
* Render Scalar metric value using formatValue
* Move isMetricScalar and getMetricValue to utils
* Add e2e tests for scalar metric values
* Fix unit tests
* Add card query endpoint to metabase/api
* Use card query from api instead of question loader
* Add test for non-numeric scalar metric
* Move providesCardQueryTag to utils
* Use skipToken for query
* Clean up getDatasetScalarValueForMetric
* Add metric actions
* Fix dot menu width
* Refetch card when it's being (un-)bookmarked
* Add tests for metric dot menu
* Make sure bookmarked metric appears in sidebar
* Make sure restoring a metric works
* Add more tests for readonly users
* Use non-root collection for metric test
Fixes#45938.
Remapping is handled at the top level of the query, and should be
ignored for inner queries. This PR hides it from `results_metadata` on
source queries.
It also fixes the interaction with `large-int-id` logic to make big
integers JS-safe.
* Add series visibility props to `ChartWithLegend`
* Mark pie slices as visible or hidden
* Don't let to hide the last visible slice
* Add space between legend dot and title
* Fix legend dot outer circle is clipped by overflow
* Use `ChartSettingSeriesOrder` for `pie.rows` setting
* Actually hide slices and exclude from total calc
* Don't show "0%" next to hidden slice legend items
* Hide the "Other" slice
* Make overflow slices popover interactive
* Fix hover
* Fix incorrect visibility state in legend popover
* Add e2e test
* Remove debug stuff
* Fix hover and drill issues with slice index calc
* Fix `pie.rows` ordering bug
* Revert "Fix hover and drill issues with slice index calc"
This reverts commit f104aaa584.
* Fix hover and drill issues with slice index calc (2)
`dataIndex` based approach
* Use a special data key for the "Other" slice
* Fixed viewport size in legend e2e test
* Fix jumping legend width
* Fix dot and label vertical alignment
* Rework pie e2e test
* Fix legend dot and label alignment
* fix: bigquery add null checks to result processing
Fixes#47339
On the related issue there are different stacktraces indicating
likely sources of null pointer exceptions.
1. `.getNextPage` is likely returning a nil value. I was unable to reproduce this but one thing I did notice is that `hasNextPage` is recommended over checking `.getNextPageToken`. Added nil handling around `page` possibly being nil.
2. `cancel-chan` may be triggered before processing begins and such `execute-bigquery` would pass nil as a TableResult to the initial reducer. Testing if cancel-chan happens at just the right moment would be too flaky for CI testing but I was able to reproduce this locally and is fixed by the nil handling added.
3. `cancel-chan` may be triggered during query processing. This is covered by a test now.
* Check hasNextPage in test:
* Add test for null getNextPage
* Fix cljfmt
* Add custom image for empty metrics
* Add useFetchMetrics helper
* Copy browse models page to metrics page
* Add route for browse metrics page
* Add useHasMetrics hook
* Add link to browse metrics page to the sidebar
* Simplify MetricsTable component
* Render description markdown in table too
* Add unit tests for browse metrics page
* Remove custom padding
* Fix line height of markdown cell
* Add e2e tests for browse page
* Fix metrics sorting
* Fix types
* Fix column widths and cell padding
* Typo
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Remove name check
* Simplify alphabet
* Remove resetSnowplow
* Use helper to get metric in table
* Use MouseEvent from React directly
* Use --mb-color-icon-primary
* Test ellipsification of description
* Simplify useHasMetrics
* Move useHasMetrics MainNavBar
* Merge sortMetrics and sortModels into sortCards
* Fix metric default description
* Extract SidebarSkeleton to a separate component
* Test sorting via Collection too
* Use ModelTableRow to fix cursor
* Share more components between ModelsTable and MetricsTable
* Use the same simplified structure in ModelsTable as for MetricsTable
* Render block-level elements as inline so ellipsis detection works
* Pick better testid for metric name
* Remove delay
* Avoid type specificty error in sortCards
* Add test for not opening in new tab
* Remove useFetchMetrics defaults
* Fix unit test import
* Test meta-click with cypress
* Add comment on inline markdown elements
* Use model-name test id
* Use correct colors for icon
* Remove delay from browse models page too
* Make meta key in test platform-dependent
* Fix hover color on metrics table
* Add focus styling for celss in the models and metrics table
* Use MarkdownPreview for table cells
* Define --mb-color-icon-primary
* Make the collection link take up the whole height of the cell
* Remove tabIndex from table rows
* Rename ModelsTable.module.css to BrowseTable.module.css
* Rename CardsTable to BrowseTable
* Remove single module.css class in favor of styled components
* Allow customising allowed elements
* Allow forcing all markdown on one line
* Render bold and italics in markdown descriptions and move content onto one line
* Fix legacy tooltip in test
* Make sure getValueForSorting returns a string
* Remove empty file
* Remove icon for metrics in the table
* Remove else statement.
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Remove async findBy where possible
* Add github issue link for TODO item
* Disambiguate between getMetricDescription type
* Be explicit with sortModelOrMetric
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
We had a security issue where it was possible to query timelines without
having the requisite permissions on the collection. This test checks to
make sure this doesn't happen again.
* walk the error to remove any visualization settings
Thanks Dan for the patch, this approach is nice because it doesn't require changing all 3 streaming-response-writer
implementations (json csv and xlsx). It keeps behaviour basically the same, but just cleans up the output to not
include potentially sensitive column/dataset details.
* add a test that confirms json/csv errors are cleaned
* walk the error to remove any visualization settings
Thanks Dan for the patch, this approach is nice because it doesn't require changing all 3 streaming-response-writer
implementations (json csv and xlsx). It keeps behaviour basically the same, but just cleans up the output to not
include potentially sensitive column/dataset details.
* add a test that confirms json/csv errors are cleaned
* fix dataset error test
* cljfmt
* try to be a little more surgical with what we take out of the error
* formatting again, oops
* pass export format, only modify the obj for file exports
In the case of errors being saved to files, the obj is a map with a :status :failed key assoc'd by the qp somewhere
along the way. The format-exception cond branch is not used in this case, so we can't rely on modifying
`format-exception` to just dissoc ex-data.
Instead, I've kept the obj modification, but it excludes the query and the preprocessed keys, which are the only 2
keys where we see :viz-settings. This doesn't eliminate the problem of needing to exclude other keys in the future,
but it does improve the surface area - the query and viz settings are the most likely candidates for holding
potentially sensitive info.
I'd prefer to stick with this solution, as we can get a fix for the security concern in quicker, and perhaps a
non-security bug can be opened regarding the most correct way to indicate errors to users whose downloads have failed.
* keep format-exception the same
* cleaner passing of export-format to write-error!
* newsletter-toggle-clicked event
* use ValidateEvent,
* rename event -> simple_event and trackEvent -> trackSimpleEvent
* use the renamed trackSimpleEvent function
* Fix typo in MetadataProvider docstring
* Fix typo in caching-test: s/Second/Third/
* Store ::nil markers in CachedMetadataProvider for failed lookups
Store ::nil markers in CachedMetadataProvider for any ids for which the wrapped/uncached upstream provider fails to return
metadatas. This prevents repeatedly querying the uncached-provider for ids that don't exist.
The downside is that if the uncached-provider suddenly starts returning metadata for an id that previously did not
exist, we won't pick up on it, but the assumption here is that this is no different / worse than cache invalidation
for existing ids that happen to change after we cache them.
* PR suggestion: remove unnecessary test assertion
* fix: sqlserver handle uniqueidentifier uuids
Fixes#46148
Include sqlserver in `uuid-type` handling as its `uniqueidentifier` type
stores uuids.
* Don't be so precise with varchar size
* Add seam for drivers to cast to text type
* Fix arg order
* rename event -> simple_event and trackEvent -> trackSimpleEvent
* add empty line at the end of the file
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
---------
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
Fixes#47341.
### Description
The QP now supports temporal bucketing of custom expressions, but that has
not been surfaced in the UI until now.
### How to verify
Describe the steps to verify that the changes are working as expected.
1. Use a database other than H2, since it doesn't support time zones.
1. New question -> Anything with a DateTime column
1. Create a custom expression like `convertTimezone([Date Column], "America/New_York", "UTC")`
1. Break out by that expression
The temporal bucketing should work in the UI and in the query results, save properly, etc.
* Move current click handler to legend title
* Add `onToggleSeriesVisibility` prop to `Legend`
* Update `LegendItemDot` component
* Update `LegendItemTitle`
* Toggle series visibility
* Fix hover effect
* Don't let to hide the last visible series
* Add `visible` property to `BaseSeriesModel`
* Fix dangling split axis label when hiding a series
* Update `graph.series_order` viz setting widget
* Fix can't toggle series visibility from legend popover
* Fix legend dot clipped by `overflow: hidden`
* Fix eslint error
* Fix legend popover scroll
* Add unit tests
* Add e2e tests
* Fix show/hide series for scatter charts
* Default `LegendItemDot` to visible state
* Only apply legend dot hover effect if there's a click handler
* Don't allow to remove the last series in `ChartSettingSeriesOrder`
* Fix tooltip performance drop
* Fix e2e tests around `series_order` viz setting widget
* Use `text-medium` for outer circle background
* Update dot style
* tweak inner circle color
* Update copy
* Use div instead of button if can't toggle visibility
* Fix legend layout issue
* Don't show trendlines for hidden series
* Fix missing prop value
* Disable visibility toggle outside of dashboard/qb view
* Add public dashboard test
* Add tooltip footer e2e helpers
* Fix chart tooltip shows hidden series data
---------
Co-authored-by: Kyle Doherty <5248953+kdoh@users.noreply.github.com>
This migration is supposed to be backported to v50 and thus needs a v50
migration ID. Since v51.2024-08-27T00:00:00 was already applied to dev
instances, we set `preConditions[].onFail = "MARK_RAN"`, so that if the
table is already present, the backdated migration does not fail. See
the discussion on PR #47419 for more details.
This will not need an (automatic) backport, since I will include it in
PR #47419.
Fixes: 592360c9fa
References: https://github.com/metabase/metabase/pull/47419#discussion_r1738929516
I still don't want users' eyes to bleed looking at (unnecessary?) stacktraces, but we need to be able to see exception
causes to determine what is going wrong
references #47520
* Move query-processor.util/remove-lib-uuids to lib.schema.util
This function will soon be needed in lib.schema.util in order to implement distinct order-by clause schema checking.
This function feels like it should live in lib.util instead of lib.schema.util, but that would create the following
circular import dependency since lib.util indirectly requires stuff from lib.schema.util:
lib.schema.util -> lib.util -> lib.schema.util
* Don't create queries with duplicate order-bys in remove_replace_test.cljc
Otherwise, theses tests would fail soon when we add update the order-by schema to reject duplicates.
* Declare lib.order-by/order-bys to return ::lib.schema.order-by/order-bys
Previously it inlined the schema instead.
* Disallow duplicate order-by clauses in ::lib.schema.order-by/order-bys
Fixes: 39384
* ask for tenancy isolation columns
* deny all permissions for all users group
* create new collections
* add jwt group mappings
* add the permissions step
* add multi-tenancy message in helper text format
* add permission graph
* wire together permissions
* use schema permissions
* use fields from table metadata from query_metadata
* add tenancy field reference
* remove log messages
* deny access to unsandboxed tables
* make permission graph more explicit
* deny access to sample database for customer groups
* add unit test for permission graph
* split permission groups and sandboxes
* jwt settings and hard-coded user attributes
* handle errors when updating sso mappings
* add express api and user switcher
* only fallback to api keys when license is invalid
* add util to sample tenancy column values
* conditional BASE_SSO_API imports
* improve embedding error message
* setup jwt configuration after license step
* setup permissions at the last step
* add missing import
* update steps that requires license
* fix incorrect imports
* add missing useContext
* handle permission update error
* remove tenancyIsolationEnabled field
* add tenancy column sampling
* differentiate tenancy column query error
* rename tenancyColumnValues to tenantIds
* assign sampled tenant ids to user attributes
* add tenant ids
* define collection permissions
* reference sandboxing group by name
* update snippet to be same as the README
* extract ask for tenancy columns to a separate step
* use the customer_id attribute
* query the table query metadata at origin
* append tables correctly
* improve error handling in table scanning
* add retry logic to metadata fetching
* only query metadata for selected fields
* fix race condition with retry
* update loading state and retries
* update comments on jwt license
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* filter the target table by id
* highlight last selected tenant column
* use breakout to get list of ids
* temporary workaround to reload the whole page
* update row value types
* update row value types
* ask if they want to setup a pro license
* post-installation improvements for cli
* block non-selected tables
* remove the source-field from sandboxing
* use the fk_target_field_id as instead of target.id
* update unit test
* add learn more message
* install mock server dependencies
* fix post-setup step formatting
* fix text wrapping
* skip tenancy columns and permissions if token is not provided
* only show the server dir step when license is enabled
* skip permission setup if there is no tenancy column
* remove source-field as we only reference our own column
* add runIf flag to skip steps
* remove the gitignore mention as this is automatic
* configure permission if there are some tenancy columns
* make native permission types more strict
* add notice about setting up local mb instance
* support multiple tenancy columns
* map the tenancy column name as the user attribute
* add component import guide
* fix deprecated punycode in node-fetch
* warn on unsampled tables
* show helper text on tenancy columns
* allow sampling repeated tenant
* add instruction to change directory into mock server
* align the learn more message
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Snowflake Variant cols should also work in attachments
A previous PR fixed the snowflake variant bug by using a try/catch aroung the global-type-settings function.
I realized there was a second codepath using the same function and it gets used in xlsx attachment code, so I've made
the change there as well to prevent the same error in attachments.
* add a test to confirm the viz-settings-for-col function doesn't throw
* Snowflake Variant Type should not prevent downloads or attachments
Fixes#46981
The :type/SnowflakeVariant key matches 2 methods in `metabase.query-processor.streaming.common/global-type-settings`.
In this case, it seems that the variant type can be any type, so we shouldn't try to guess anything here, except if
the user has provided a :semantic_type, which we can use.
Otherwise, we'll keep going without formatting details about the variant column, which is likely fine already, as it
should result in strings in the export/attachments.
* add a test
Fixes#30602
We were overriding localdate in bigquery driver to produce
offsetdatetimes for no apparent reason. This causes a number of problems
when dealing with date type columns.
* Bigquery verify bignumeric fingerprint and bin
Fixes#28573
Field values must be scanned.
In 45.3, these fields were not being fingerprinted properly. (tag `v1.45.3`)
In a 45 point release that appears to have been fixed. (branch `release-x.45.x`)
Confirmed fixed in master, adding test
* cljfmt
The Portal diff view wants `[before after]` pairs of the original data,
not the three-part output of `clojure.data/diff`. This fixes the
`qp.debug/debug>` output from the preprocessing middleware to emit
correct diffs.
Also, since the diffs are not useful when the query is converted between
legacy and pMBQL, this also attaches the *converted* input query to the
output query's Clojure metadata, and this converted value is used as the
basis for the diff.
* entity id translation + tests
* add api level test
* simplify definition of eid-table->model + add test
* update tests to take keywords
* improve comment
* generate the eid-table->model map
* delete now-obsolete test
* make it work in oss
* put the resulting response into a key, so we can add more information later
* formatting
* use model names without the model/ prefix as keys
* Creates list of `api/model->db-model`
- update keys for util/entity_id request
- update shape of util/entity_id response
- add test for not-found eids
* formatting
* Respond to code review feedback
* entity-ids work on /embed/{card,dashboard}* routes
* make entity id translation work for all tokens
* Merge branch 'static-embed-dashboard-and-question-endpoints' of github.com:metabase/metabase into static-embed-dashboard-and-question-endpoints
* massage schema. add tons of tests
* decode more eids
* test alignment
* don't require ns inside of itself :think:
* make the test functions to create tokens better
* bring back `dashboard-url`
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Query Validator FE
* collection path, unit tests
* wrapping feature with token flag
* updating util function, adding row type
* updating session_test.clj
* type adjustment
* fixing other table sorting
* Empty state, clean up utils
* unit test adjustment
* e2e adjustment
* another attempt at perms fix
* try again
* try manual graphql auth
* try with global env
* Try setting defaults in graphql
* try some more things
* use my own graphql instance
* one more test
* remove test code
* remove rogue curly brace
== Goal ==
Hide attached DWH database details from anyone incl. admins:
* Do not show them in the UI
* Do not permit to change them
* Do not serialize them
The aim is that customers cannot gain access to (parts of) credentials,
and they cannot break a feature they are paying for by changing
connection details.
== Implementation ==
The Metabase backend already contains provisions in the implementation
of `metabase.models.interface/to-json` for `:model/Database` to hide the
`details` of the database in HTTP responses, if the user lacks write
permission on the database. We utilize this by adding an
`is_attached_dwh` column to the `database` table and rejecting
`metabase.models.interface/can-write?` when this flag is enabled. In
the "admin" UI, we show a replacement text instead of the edit form when
the flag is set. (It might be correct to show this whenever `details`
is absent. See below for possible follow-up work.)
However, several sections of the frontend code expected the `details`
field to always be present. In order to make `details` optional, as the
backend seems to handle it, we fix the respective code to treat this
case in the way that appears appropriate in the context.
Database details are already generally excluded from H2 dump snapshots
(see `metabase.cmd.copy/*copy-h2-database-details*`), thus nothing
changes there.
== How to test ==
=== New behaviour ===
Setting the `is_attached_dwh` field hides the database details:
1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
- In addition to the fields you would normally set, also set
`is_attached_dwh: true`.
- This also works when adding this flag to a database that previously
did not have this flag set.
2. Start your Metabase instance.
3. Verify the database shows up in the "admin" section
(`/admin/databases`).
4. Verify that clicking the database to see its details only reveals
"This database cannot be modified."
5. Verify that responses from the backend do not include a `details`
field for this database.
=== Original behaviour ===
Behaviour without setting the `is_attached_dwh` field is unchanged:
1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
- Only set the fields you would normally set. Do not set
`is_attached_dwh` (or set it to `false`).
2. Start your Metabase instance.
3. Verify the database shows up in the "admin" section
(`/admin/databases`).
4. Verify that clicking the database to see its details only reveal the
regular edit form, showing connection fields like `host`, `user`,
etc. with the values you configured.
== How this will be rolled out ==
1. Upgrade existing Metabase Cloud instances with data warehouse to a
Metabase version that supports `is_attached_dwh`.
2. Set `is_attached_dwh` in the database section of the config file for
Metabase Cloud instances with a data warehouse.
== Possible follow-up work ==
In https://github.com/metabase/metabase/issues/25715, absent
`database.details` was identified as a bug. Since then, `details` was
made `NOT NULL` in the application database, so this bug can no longer
occur. However, today backend responses can be missing the `details`
field, if the current user lacks write permission to the database
setting (see above). Fully re-evaluating the fix to #25715 in this
context is outside the scope of this PR.
Closes: https://github.com/metabase/harbormaster/issues/5051
* Add `clojure -M:kondo` and `clojure -M:kondo:kondo/all` and bump version
* Fix Kondo errors
* Fix Kondo+LSP issues with `defendpoint`, `defenterprise`, etc.
* Use replace-deps instead of deps for speed
* Ok apparently maybe we do need to copy configs when we run Kondo on CI
* Oops `./bin/kondo.sh` should not try to use `clj-kondo`
* Remove references to GA driver folders
These migrations will be backported to v49.
Three changes:
- update IDs/locations of the migrations to v49 vs v51
- add one preCondition (when adding the foreign key constraint)
- add `onFail: MARK_RAN` to the index preconditions. I forgot this
before, so it blows up when the precondition doesn't hold.
* allow re-ordering and re-naming pie chart slices
* aggregate rows with same dimension value
* show warning when dimension has unaggregated values
* change colors from sidebar
* implement renaming
* styles for slice name widget
* handle numeric slice keys for binned values relative dates etc
* implement slice disabling
* remove old colors setting
* implement merging new and removing existing rows from query results
* fix formatting bugs
* port settings to static viz
* fix bug with unaggregated data
* address todos
* add loki test
* fix measure column settings widget erroring
* fix settings sidebar crashing
* add e2e tests
* update snapshots
* fix pie_chart.cy.spec.js CI-only flakes
* fix#21504 repro failure
* fix click action value
* use nestedSettings instead of seriesSettings
* fix settings sidebar crashing
* fix row with 0 key not being draggable
* fix flaky test
* filter other slices from sidebar
* combine filter header buttons
* refined styling
* unit tests
* another import conflict
* omg another one
* remove double border for filter button
* use vars and update imports
* Increase log level to INFO from error
Closes#47396
logs before:
```
2024-08-29 08:37:03,836 INFO db.liquibase :: Checking if Database has unrun migrations...
2024-08-29 08:37:04,045 INFO db.liquibase :: Database has unrun migrations. Checking if migration lock is taken...
2024-08-29 08:37:04,047 INFO db.liquibase :: No migration lock found.
2024-08-29 08:37:04,047 INFO db.liquibase :: Migration lock acquired.
2024-08-29 08:37:04,167 INFO db.liquibase :: Running 83 migrations ...
```
logs after:
```
2024-08-29 08:39:45,019 INFO db.liquibase :: Running 83 migrations ...
...
2024-08-29 08:39:45,166 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:00::calherries ran successfully in 0ms
2024-08-29 08:39:45,182 INFO liquibase.changelog :: Columns is_defective_duplicate(boolean) added to metabase_field
2024-08-29 08:39:45,182 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:01::calherries ran successfully in 14ms
2024-08-29 08:39:45,199 INFO liquibase.changelog :: Custom SQL executed
2024-08-29 08:39:45,200 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:02::calherries ran successfully in 16ms
2024-08-29 08:39:45,202 INFO liquibase.changelog :: Foreign key fk_field_parent_ref_field_id dropped
2024-08-29 08:39:45,202 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:03::calherries ran successfully in 1ms
2024-08-29 08:39:45,204 INFO liquibase.changelog :: Foreign key constraint added to metabase_field (parent_id)
2024-08-29 08:39:45,205 INFO liquibase.changelog :: ChangeSet migrations/001_update_migrations.yaml::v49.2024-06-27T00:00:04::calherries ran successfully in 2ms
...
```
* empty commit to bump CI
* handle cache config overflowed
* fix root too
* do not comment out test
* make cljfmt happy
---------
Co-authored-by: Alexander Solovyov <alexander@solovyov.net>
* Make metric metadata keep the metric's name
This is a fix for the problem reported in #40355, which is fixed separately for v50 and backported to v49.
* Patch broken test
* Normalize dataset-query
* Normalize legacy queries only
* ask for tenancy isolation columns
* deny all permissions for all users group
* create new collections
* add jwt group mappings
* add the permissions step
* add multi-tenancy message in helper text format
* add permission graph
* wire together permissions
* use schema permissions
* use fields from table metadata from query_metadata
* add tenancy field reference
* remove log messages
* deny access to unsandboxed tables
* make permission graph more explicit
* deny access to sample database for customer groups
* add unit test for permission graph
* split permission groups and sandboxes
* jwt settings and hard-coded user attributes
* handle errors when updating sso mappings
* add express api and user switcher
* only fallback to api keys when license is invalid
* add util to sample tenancy column values
* conditional BASE_SSO_API imports
* improve embedding error message
* setup jwt configuration after license step
* setup permissions at the last step
* add missing import
* update steps that requires license
* fix incorrect imports
* add missing useContext
* handle permission update error
* remove tenancyIsolationEnabled field
* add tenancy column sampling
* differentiate tenancy column query error
* rename tenancyColumnValues to tenantIds
* assign sampled tenant ids to user attributes
* add tenant ids
* define collection permissions
* reference sandboxing group by name
* update snippet to be same as the README
* extract ask for tenancy columns to a separate step
* use the customer_id attribute
* query the table query metadata at origin
* append tables correctly
* improve error handling in table scanning
* add retry logic to metadata fetching
* only query metadata for selected fields
* fix race condition with retry
* update loading state and retries
* update comments on jwt license
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* filter the target table by id
* highlight last selected tenant column
* use breakout to get list of ids
* temporary workaround to reload the whole page
* update row value types
* update row value types
* block non-selected tables
* remove the source-field from sandboxing
* use the fk_target_field_id as instead of target.id
* update unit test
* remove source-field as we only reference our own column
* make native permission types more strict
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* ask for tenancy isolation columns
* deny all permissions for all users group
* create new collections
* add jwt group mappings
* add the permissions step
* add multi-tenancy message in helper text format
* add permission graph
* wire together permissions
* use schema permissions
* use fields from table metadata from query_metadata
* add tenancy field reference
* remove log messages
* deny access to unsandboxed tables
* make permission graph more explicit
* deny access to sample database for customer groups
* add unit test for permission graph
* split permission groups and sandboxes
* jwt settings and hard-coded user attributes
* handle errors when updating sso mappings
* add util to sample tenancy column values
* improve embedding error message
* setup jwt configuration after license step
* setup permissions at the last step
* handle permission update error
* add tenancy column sampling
* differentiate tenancy column query error
* rename tenancyColumnValues to tenantIds
* define collection permissions
* reference sandboxing group by name
* extract ask for tenancy columns to a separate step
* query the table query metadata at origin
* append tables correctly
* improve error handling in table scanning
* add retry logic to metadata fetching
* only query metadata for selected fields
* fix race condition with retry
* update loading state and retries
* filter the target table by id
* highlight last selected tenant column
* use breakout to get list of ids
* update row value types
* block non-selected tables
* remove the source-field from sandboxing
* use the fk_target_field_id as instead of target.id
* update unit test
* remove source-field as we only reference our own column
* make native permission types more strict
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
Previously, truncating a `:type/DateTime` column by `:month` or `:day`
would return a `:type/Date`, which subtly broke the query.
In particular, if you try to order-by the breakout column `Created At (month)`
then it would not get de-duplicated, causing a SQL error about conflicting
ORDER BY clauses.
Fixes#46992.
Previously, `visible-collection-ids` was effectively "free" in that we'd
cached `collection-id->collection` for *all* collections, within a
single request, and then locally filtered it for permissions without
needing to hit the database again.
To be honest, there is probably a better fix for this - we're repeatedly
calling `visible-collection-ids` when we probably could just save a
single copy of it and use it when calculating the effective location of
every collection.
However, this is a very quick and low-risk fix, and I want to prioritize
getting this done, and then we can improve it later.
Locally, I copied down the database from stats and timed the
`/api/search?model_ancestors=true` endpoint.
Before my "speedup" PR (https://github.com/metabase/metabase/pull/46942)
it took ~7 seconds to return results.
After my "speedup" PR, it took ~15s to return results. :grimace:
With this change, it takes 818ms to return results. 🎉
* Fix typo in expression-clause-normalization-test
s/time-interval/relative-time-interval/
* Add missing table-id to card def to fix failing legacy-ref-test
* Ensure that if yarn test-cljs fails it fails the CI checks
When configured to autorun, the command
shadow-cljs compile test
will exit success even if test failures occur in the node test. See
https://github.com/thheller/shadow-cljs/issues/425
In order to ensure our CI checks fail when cljs tests fail, we need to invoke
node directly. To prevent running the tests twice, remove `:autorun true` from
the `:test` target in shadow-cljs.edn.
* zoom-in-timeseries-drill: do not include :hour and :minute for DATE columns
When the user attempts to drill through a DATE column, we should not return :hour or :minute drill throughs from
available-drill-thrus. Previously we did, which resulted in an error being displayed when the user attempted to select
them.
Modify next-breakout-unit in zoom_in_timeseries.cljc to not return :hour or :minute units for :type/Date columns,
which prevents these from being returned by available-drill-thrus, which prevents them appearing in UI context menus.
* Use lib.temporal-bucket/available-temporal-buckets
Per review feedback, use lib.temporal-bucket/available-temporal-buckets to get the valid-current-units in
metabase.lib.drill-thru.zoom-in-timeseries.
* Do not construct zipmap just to lookup the next unit
Rather than constructing a zipmap from `valid-current-units', which requires iterating over the seq twice, just
traverse the seq once and return the next item in the seq (if any).
Constructing a zipmap made sense in the old code because it was constructed once from static data and used many times
for lookup. In the new version, the vec of `valid-current-units` is dynamic and dependent on the type of the breakout.
* PR suggestion: use threading macro rather than nested sexps
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* PR suggestion: use threading macro rather than transducer / eduction
* PR suggestion: reduce boilerplate in zoom-in-timeseries tests
Reduce code dup by looping over aggregation and bucketing options rather than having a separate deftest for each.
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
Fixes#39366
* use loki to test png exports on questions
* try to put the img in the body to see if it fixes the scrollbar issues in ci
* fix linting issues
* let's try again in ci
* Attempt to make CI wait before taking a snapshot for downloaded PNGs
* Use the snapshots from CI for PNG downloads
* pdf export test thanks to kelvin suggestion about just testing the png
* refactored the code to extract a util function
* remove pdf tests as they broke in ci 🤷
* Update frontend/src/metabase/env.ts
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Update frontend/src/metabase/lib/loki-utils.ts
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* entity id translation + tests
* add api level test
* simplify definition of eid-table->model + add test
* update tests to take keywords
* improve comment
* generate the eid-table->model map
* delete now-obsolete test
* make it work in oss
* put the resulting response into a key, so we can add more information later
* formatting
* use model names without the model/ prefix as keys
* Creates list of `api/model->db-model`
- update keys for util/entity_id request
- update shape of util/entity_id response
- add test for not-found eids
* formatting
* Respond to code review feedback
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Don't encrypt boolean settings (by default)
We have tooling to disable encryption on settings even when the
`MB_ENCRYPTION_SECRET_KEY` is set. Turn it on by default for all boolean
settings, which don't need to be encrypted.
I also optimized the code that decrypts settings on startup because I
didn't want to delay startup if someone had set a bunch of boolean
settings. With 20 set, the old version added about 200+ ms to startup,
about 10ms per boolean setting, whether or not it was encrypted or in
the DB already. The optimized version selects all the never-encrypt
values from the database at once (a bit silly, but we also just exclude
raw `true` and `false` values so we don't bother checking them) and
updates them if they're encrypted - this adds ~40ms to startup with 20
encrypted boolean settings (about 2ms per boolean setting) and ~5ms to
startup on subsequent runs, when no encrypted values are in the DB.
This testing is too strict to support some legitimate use cases that
have no workaround at present. We can bring back this type-checking
eventually, once it's possible to correctly express things like
automatic coercion of strings to numbers in `SUM` aggregations.
Fixes#44431.
* Update appearance.md
Adding information about how colors are selected by Metabase because there is the confusion that the order in which you select colors is the order you get on the chart
* Update appearance.md
Removed code reference and tried to simplify the context
* Update docs/configuring-metabase/appearance.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
There are a few separate changes here:
- Migrations: add and populate indexed columns perm_value, perm_type, and collection_id to permissions
These fields allows us to efficiently run queries based on collection permissions in the DB without string manipulation. Keeping the table as permissions allows us to do this migration in-place. Note that in some cases collections may have been deleted from the database without deleting the associated permissions row (since there was no foreign key before). We need to be defensive here: if we have a permissions row without a corresponding collection, delete the row before running the rest of the migration.
- Write perm_value, perm_type, and collection_id for new collection permissions
A very simple before-insert method sets these fields before a collection permission is written to the DB.
- Replace collection/permissions-set->visible-collection-ids and collection/visible-collection-ids->honeysql-filter-clause with collection/honeysql-filter-clause
Previously, just about everywhere we used permissions-set->visible-collection-ids, what we were essentially doing was an in-app join: select all the collection IDs you have permission on, then convert it to a SQL clause like WHERE collection_id IN ( all of those collection IDs).
Replace both of these with honeysql-filter-clause, which uses the new fields we added to permissions above to construct a honeysql filter representing "all the collections I have permissions on", without needing to round-trip them to the application and back to the DB.
Of course, we can then write a function visible-collection-ids, which uses honeysql-filter-clause, for those cases where we do actually need the whole bunch in the application (we use this, for example, when constructing the effective-location for a collection).
I also added one more toggle to the VisibilityConfig that's passed into the honeysql-filter-clause (and used to be passed to permissions-set->visible-collection-ids), allowing you to select only the effective children of some collection.
* [MBQL lib] Map dashboard filters to last stage w/aggregations
Originally we used the last stage's columns for dashboard filters, but
then if you eg. use a custom expression to clean up the results of the
aggregation, then you can't filter at the most useful spot - before the
aggregations.
Fixes#19744. Related to #26244, #33958.
* Add stage-number option to the dimension not to the field
* Remove FE changes
* Support negative stage numbers
* Add tests for multi-stage parameter expansion
* Specify schema for dimension options
* Fix legacy MBQL schema
* Append stage when running cards with top level summary
* Fix schema
* Only ensure filter stage when the parameters say it's used
* Implement ensure-filter-stage
* Format clj
* Add ensure-filter-stage test
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Fix DateTimeRange substitution for open ranges
* Add E2E repro for #47172
* cljfmt
* Add test for datetime column
* Update range boundary for open range
* Update test names
* Apply suggestions from code review
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix e2e
---------
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add database type adjustments to server side generated temporal values
* Speculative base type change
* Infer timezone only from _actual_ user, not global user level
* Set timezone for actual user in test extensions
* Add test case
* Update type info generation
* Adjust server side relative datetime
* Update src/metabase/driver/sql/query_processor.clj
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Fix base type logic
* Fix typo
* Fix formatting
* Update user name handling
* cljfmt
* Address review comments
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* reorganizes data-permissions.ts to be a folder split by a file per concern, moves some logic out into other utils files where it makes sense
* pr feedback and fix lints
* only use viz height for visualization view
* move the default viz height fallback
* default to full-height if height is zero or unavailable
* remove test wrapper
* recents updates to better support metrics
allows updating metric views through the API
improves parent_collection calculation
* add 2 tests for metrics
* Use global search for aggregation picker
* Add names to aggregation picker items so they can be searched
* Render info icon metric descriptions
* Add unit tests for global aggregation picker search
* Add test for searching metrics
* Add a test for metrics descriptions
* Sort metrics
* Fix e2e test
* Use Flex from metabase/ui
* Handle markdown in metric description
* Add test for sorted metrics
* Update Markdown styling in metric description
* Fix clojure formatting
* Test for markdown in the Metric description
* Fix lint
* Fix test for markdown
* Use simpler equality test
* Fix overflow in test
* QuestionAlertWidget to FC + TS
* update e2e tests
* use regular alert for unsubscribes
* Create Question and Dashboard Sharing Menu
* Handle some edge cases
* top notch unit tests
* more testing and tweaking
* fix imports
* more tests
* add alerts to question sharing menu
* add tests for alerts
* update e2e tests
* use Oisin's toolbarbutton
* fix rebase
* prompt to save before sharing questions
* show prompt to set up notification channels
* fix import
* update tests and hide on notebook screen
* fixes
* clean up pulse types
* update e2e tests
* fix a bunch more tests
* more e2e test fixes
* maybe green now? 🤞
* last one 🥺
* design updates
* make channel setup modal nicer
* simpler component props
* lint fix
* lint fixes
* poc of removing data usage step for the tech plan
* pretend it's not cloud if hasAddedPaidPlanInPreviousStep is true
* split the jest tests files to simulate pro cloud, pro selfhosted and starter cloud scenarios
* open metabase store to get trial token
* remove license env from instance setup as we provide the license key later
* activate metabase license
* fix password field missing
* fix formatting for generated component files message
* activate license key
* add missing auth options for postgres
* add function to print with padding
* prevent infinite loop and update section representation
* show the activate license error in red
* rename variable
* Add offset presets
* Render OffsetPresets initially
* Add BucketInput
* Render BucketInput and do not rely on referential equality for Lib.Bucket
* Make sure the currently selected bucket shows in the presets
* Lift label into CompareAggregations
* Fix spacing of label
* Fix labels based on comparison type
* Use Switch for include current period
* Include current bucket in CurrentPeriodInput label
* Fix unit tests
* Add more unit tests for CompareAggregations
* Update e2e test for column compare to handle presets
* Group offset tests
* Add tests for moving average
* Add tests to verify changing the bucket works
* Fix type
* Make sure temporal columns are buckatable before picking them
* Make getPreferredPresets more clear
* Add test for temporal expression column
* Fix lint
* refactor: sort import members inside destructuring
* Show an error message when the temporal unit is not applicable to a dashboard card (#46983)
* Fix tests
* Add tests
* Add tests
* Fix types
* Fix types
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* make the newsletter signup a checkbox instead of a button
* update copy on unit test
* fix describe nesting
* update the ui to be like the design
* make tests use userEvent.click to avoid warning about missing act
* use the subscribeToNewsletter function
* add basic e2e test for newsletter subscription
* move wait to the end of the test to see if it passes in ci
* remove async to see if it fixes the issue
* trying to mock the response
* make it clear why we're mocking the response
* use getUserEmail as suggested by Sloan
* inline var a suggested on code review
* Kondo Config cleanup
* Restore warnings for with-log-messages-for-level until #28827 is merged
* Updated clojure.test hooks
* Fix kondo warning
* Test defn/defmacro exclamation point linter should also use :parallel/disallowed (part 1)
* WIP
* Time for me to learn to spell
* Finish the cleanup
* Remove code I was typing as an example
* Fix renamed var
* Fix the SAML tests
* Fix Kondo warning
* allow `:blocked` to be saved for table level perms
* Adds 2 tests for table level blocked permission settings
- N.B. these are NOT ENFORCED YET
* update test that asserted we cannot set block on tables (we can)
* WIP: Pairing on making perms checking less wild
* cleanup, update docs, and add a test for view-data perm only
- Added a test where we have data permissions, but not create query, and
I think it is failing when we have create query and blocked data
permissions.
- renamed some functions from check-x -> has-x? since they return a
value instead of throwing now
* Revert "WIP: Pairing on making perms checking less wild"
Keep the same behavior, but stick with the saner flow control
This reverts commit 63bcb5b401138555a081378574b15d6506469e60.
* update docs
* update test to be passing
- TODO: make sure it's correct w.r.t. perm settings
* Allow schema level blocked setting in permgraph
* remove invalid test cases
- continue to have a forcing function to test newly added perms
* conform function output
* ensure a single blocked table blocks native queries to its DB
* update error message
- we now catch this error in `metabase.models.query.permissions/has-perm-for-query?`
* we now check for data permissions to process query for card
* add more explanation to what we are testing
- to help see why it fails on CI and passes locally
* remove excess `def`
* Add test for table-level data X collection perms
- update test found to be in-error
* update param values qp permission check style
* set view-data and create-query explicitly
* set viewdata and createquery explicitly in qp test
* Respond to review comments (which fixes a case)
* setting a table to blocked: leave other tables the same
* [Permissions] Add "No access" schema/table permission (#46509)
* first pass
* refactors downgrading native permission logic and updates calculation so that "No access" downgrades native permissions to "No"
* stub for permissions help info on table block
* modal changes wip, updates downgrading create queries permissions to all happen at a single call site
* clean up, sandboxing modal copy changes, removes rekoke/limit access modal changes to make the diff smaller and move code to a seperate PR
* updates permissions help section to contain the final copy
* sandboxing copy fix and remove modal that was dropped from requirements
* adds blocked at the schema level, updates no access copy to blocked, updates permissions help section to contain new blocked and schema level changes
* fixes failed unit and e2e tests after sandboxing copy changes
* improve the block e2e test to include table blocking
* fixes failing blocked test, fixes other schemas create queries permissions getting correct with one schema was droped to blocked view data access, fixes a bug that prevents the save bar from going away when all permissions for group are set to the default values
* clean up
* remove color changes
* prevents parent being set to blocked preventing edits for children entities
* add new hasPermissionValueInSubgraph fn, adds modal to warn users we have to upgrade the view data permissions when they upgrade create queries permissions when a child entity is set to blocked
* adds test coverage for new modal
* removes unused function, adds new updateEntityPermission fn to help consolidate some logic elsewhere
* unit test fix and type fix
* most pr feedback
* updates the confirmation modal copy when changing a parent entity that contains a child with blocked permissions and/or sandboxed children, adds test coverage for that, adds test coverage for permission view data column not appearing in oss
* type fix
* [Permissions] Add e2e test coverage for blocked permissions enforcements (#46663)
* adds test coverage for enforcement of blocked permissions
* moves tests around based on pr feedback
* copy changes
* adds fix to make sure that blocked permissions are not removed from sibling tables that have the create queries permissions upgraded (#46854)
* Fix table name lookup for dbs w/ 1 schema per db
* add test for blank schema identifiers
* Refine sandboxed user perms for query builder access (#46939)
* Refine sandboxed user perms for query builder access
- Limit create-queries permissions to unblocked tables only
- Check user permissions for each table before granting query builder access
- Prevent querying of blocked joined tables from query builder for sandboxed users
* Adjust permissions for sandboxed users
- Grant view-data permissions only for unblocked tables
- Revert create-queries permissions to all tables in sandbox
- Remove unnecessary intermediate variable
* when sandboxing we no longer grant unrestricted view perms for blocked tables
* Update enterprise/backend/src/metabase_enterprise/sandbox/query_processor/middleware/row_level_restrictions.clj
remove blank line
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* - make coalesce-test exhaustive (except for sandbox)
* Update enterprise/backend/src/metabase_enterprise/sandbox/query_processor/middleware/row_level_restrictions.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* t2/select ... -> database/table-id->database-id
* update comment
* [Permissions] Prevent "Granular" option in DB View Data options from changing permissions to unrestricted (#46976)
* fix
* adds back most of the code and limits it to only happen with impersonations, updates test to handle differing logic between the two flows
* removes test that is not longer needed
* more sandbox join table perms tests
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update outdated loki snapshots
* Update the line snapshot using the one from CI
* try bumping the timeout to reduce flake
* Revert the linechart snapshot since it's still flaking
* skip the flaking story for themed line chart
* skip the flaking story for line chart
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
If the content-type header is missing entirely, let it through. We're
not trying to be overly restrictive here, just prohibit things that are
clearly not GeoJSON.
Fixes https://github.com/metabase/metabase/issues/46918
* Add new strings for v50.20
* update translations for 50.21
---------
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Throw reasonable error when incompatible temporal unit is used in time unit param
* Remove redundant function
* Add :curated flag to catch exceptions
* Update test/metabase/query_processor_test/date_bucketing_test.clj
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Address review remarks
* Update test
* Make vars private
---------
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* use ee api to find custom reports collection
* remove deprecated hook
* fix race condition
* add unit tests
* fix circular dependency
* fix unit tests
* another plugin strategy
* Remove unused file
* Refactor getCurrentUser command to a TS function util
* Refactor createAlert command to a TS function util
* Refactor createModerationReview command to a TS function util
* Reuse createModerationReview
* Refactor createPulse command to a TS function util
* Remove references to deleted files
* Improve typing
* Kondo Config cleanup
* Restore warnings for with-log-messages-for-level until #28827 is merged
* Updated clojure.test hooks
* Fix kondo warning
* Time for me to learn to spell
* use ECharts tooltip on cartesian charts
* fix tooltip lag when changing hovered series
* facelift tooltip
* fix static viz crashes due to tooltip option
* facelift
* pie chart echarts tooltip
* facelift
* facelift
* facelift
* tweaks
* e2e specs
* return back the pie other slices tooltip
* more specs
* specs
* fix pie total
* fix tooltip overflow
* fix specs
* fix infinite rerender in embedding mode for charts that have warnings
* Update Loki Snapshots
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Beautiful log message capturing ⭐😻
* Don't have log.cljs try to load log.capture, don't know how to make that work
* Remove restriction against use in parallel tests
* Fix tests using invalid syntax
* Port legacy tests
* Make this stuff work with Cljs
* Fix bad syntax
* Convert usages of old version of with-log-messages-for-level to new version
* Update other stuff to use the updated macro
* Fix stuff
* Fix Cljs tests
* Fix world's largest test
* Appease Kondo
* Fix comment
* ClojureScript: only emit capture code in dev builds
* perf: Use more efficient iteration and formatting in CSV export
* perf: Replace clojure.data.csv functions with more efficient implementations
* fix: Fix formatting for integer numbers in scientific notation
* concat
* we fixed this!
* format
* links
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* refactor: memoize getSteps selector
* screen.getXXX.click() => userEvent.click(...) to avoid missing act() warning
* use mockSettings so that MetabaseSettings.setAll is called and we avoid the unknown features warning
* Add type picker to comparison modal
* Adjust offset input labels based of comarison type
* Pass comparisonType to column picker
* Add current period picker
* Use deepCompareEffect to avoid rerender loops
* Add moving average helpers
* Add moving average clause to query
* Prevent less than 1 period moving average from being created
* Fix unit test for CompareAggregations
* Use the ComparisonType type for the comparisonType arg
* Hardcode the mapping for offset <-> moving-average column types
* Convert column types in the comparisonType callback instead of a hook
* Remove unused parameter
* Automatically change to 2 when setting an invalid value for moving average periods
* Use expression parts in test
* Use named arguments for movingAverage
* Fixup test for moving average offset input
* Fix type issue
* Pass on includePreviousPeriod to getAggregations
* Add defaultTemporalBucket helper
* Add helper to pick default breakout
* Move breakout to the front or add one if it does not exist
* Add updateQuery to AggregationPicker props
* Add tests for creation and reordering of temporal breakouts
* Reuse defaultTemporalBucket in withDefaultTemporalBucket
* Reorder query operations to make sure the correct names get generated
* Remove dependency on referential equality of breakout
* Rename updateQuery to onQueryChange
* Remove onAddAggregations and onUpdateAggregation for onQueryChange
* Extract query update in to separate function
* Remove on{Add,Update,Remove}Aggregation in favor of onQueryChange
* Remove aggregations from summarize details
* Fix e2e tests
* Remove conflicting test
* Fix recent clause in unit tests
* Fix superflouous check for columnAndBucket
* Future proof breakout check
* Remove reliance on details in the useSummarize hook
* Remove details in unit tests
* Removal of default summarization should not trigger a query change
* new version of data_access migration + run ANALYZE beforehand
* relax constraints on migration timestamps
* more migration rewrites
* fix bug in migration
* fix another bug and remove extraneous logging
* reorder migrations to fix ID validation
* start adapting the changes to mysql as well
* wip
* add more ANALYZE calls
* revert logging changes
* finish duplicating work for mysql + a bit of clean up
* create H2 versions of migrations without the ANALYZE calls
* remove ANALYZE call from view_data.sql (probably not necessary)
* fix mysql syntax
* fix another mysql bug
* another mysql fix
* Don't exclude inactive tables for data_access
* Don't write rows for Administrators group
* Don't write rows for Admins (H2/MySQL)
* disable triggers before perms migration
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* add setup commands
* fix settings definition
* update environment variables for cli
* handle instances not being ready
* update error messages
* add more specific loading messages
* loading spinner state
* improve error message
* use a fixed demo setup token
* remove extraneous spinner
* update status checks
* update container messages
* update wait timing
* create api keys
* extract constants
* remove manual steps
* Add anonymous tracking + other things. will need to clean up
* Modify SDK for better structure
* remove line from print.ts
* Update webpack.embedding-sdk-cli.config.js back to production
* Add types and add quick note
* Fix a typo
* Add index file, simplify types, use an array
* Add safer json parsing
* use delay of 100ms between each setup call
* Suggestions from review
* ensure that cli works
* Attempt to fix jest errors
* Remove node-fetch from sdk code to hopefully get unit tests working again
* add database connection
* add connection details handling
* refactor asking for database connection info
* apply actual database id for syncing schema
* fix failing database sync step
* allow table selection
* create model for each table
* handle errors in model creation
* fix incorrect model display name
* create x-rays based on user data
* consolidate instance setup message
* workaround for inquirer eventemitter issue
* add sample components
* fix yarn.lock file
* add sample components
* set spinner to fail state when instance setup fails
* use a transparent background by default
* update cli welcome messages
* add import notice
* refactor imports
* update snippet to add dashboard dropdown
* cli add credential file to users .gitignore
* add docs on quickstart
* clear screen with console.clear
* address code review feedback
Co-authored-by: Nicolò Pretto <info@npretto.com>
* allow node 18 lts
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Fix tooltip styles leaking to popover
* Fix snapshot now showing the tooltips and dropdowns
* Update the loki snapshot using built storybook for consistency
* Show new model loading state while fetching
* Show new model loading state while fetching
* Add unit test
* Use `DelayedLoadingSpinner` instead
* Update the test
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Prettier
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add Timeseries insights even when Custom Columns exist
WIP
Fixes 46244
I don't know why it was (maybe still is) required that the :other columns be empty before providing insights.
* might be a solution to changing trendlines when data doesn't change
The insights code that powers the frontend trendline requires taking a sample of the dataset.
This sampling has some randomness associated by design, and should maintain its randomness. But it makes some sense to
use the exact same sample when the input has not changed at all, hence trying this memoize approach out.
Might not be the final solution, but it's a start
* add test to show that insights are computed
* use java.util.random with a seed to keep insights stable
* Add testcase from https://github.com/metabase/metabase/issues/43915
* Try a custom formatter if the default formatter for OffsetDateTime doesn't work
* Add testcase
* Make auxillary formatter private
* Catch a more specific exception
* Throw IllegalArgumentException on invalid dates
* Use strict resolver and fix test
* Empty commit to assign credit where it's due
---------
Co-authored-by: Chris Truter <chris@metabase.com>
* Dowload Row Limit Env Var
Adds `MB_DOWNLOAD_ROW_LIMIT` to enable changing the row limit on downloads and alert/subscription attachments (not the
rendered tables, but the .csv, etc.).
Based on: #44982 (Thanks, @r-kot)
Partially implements: #28144
The difference in this PR compared to #44982 is that the download limit applies to all downloads, the only exception
being when the limit is above `qp.i/absolute-max-results` (1048575, based on Excel's limitation); in such a case, the
user supplied limit is only used if the download is csv or json, and `qp.i/absolute-max-results` is used for xlsx.
This PR also fixes alert/subscription attachment limits; prior to this, they were set to the in-app limit of 2000
rows, but now they will follow the user supplied download-row-limit.
This PR also adds a test to the downloads-and-exports test namespace, confirming that they follow the supplied limit,
or the max limit if none is supplied.
* add test confirming the default limit works
* fix test to use the download-row-limit
* address review feedback
* Update src/metabase/public_settings.clj
Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>
* add test covering case where download-row-limit is unset
---------
Co-authored-by: Cal Herries <39073188+calherries@users.noreply.github.com>
* Revert "Remove unused humanizeCoercionStrategy and its test suite (#36136)"
This reverts commit 5375347e33.
* humanize coercion strategies, add e2e test
* unit test adjustment
* pr feedback
* conditional dismissal step
* check for download status in the container
* try fixing flakes again
* try again
* chartPathWithFillColor already verifies chart has been rendered
* Introduce subpath-safe URL utils
* Open data source links from a notebook safely on a subpath
* Open browse model links safely on a subpath
* Uniform naming
* Use the existing URL helper
* Trim url
* Add unit tests
* Add one more test case
* Fix test name 😅
* Make URLs non-optional strings
* Export Formatting Correctness
WIP
Several bugs exist around the correctness of export formatting, this aims to fix them.
Fixes#27374
When a model has viz settings/metadata changed after save, the download won't incorporate said changes. At least it's
clear that scale doesn't work.
* number formatter merges settings from columns too
* column formatting on aggregated cols works in xlsx now too
* fixes 43040 by grabbing global-type-settings in csv formatter
* Refactor
* Fix wrong colors when passing empty SDK `colors`
* Clean up CSS custom properties
* Add missing color mapping to the SDK
* Add missing colors to the main app
* Only override additional dynamic colors when values are passed
* Fix unit tests
* Address review
Co-authored-by: Phoomparin Mano <poom@metabase.com>
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Move the compare to the past aggregation to the bottom of the dropdown
* Fix missing icon property
* Hide 'Compare to the past' when there are no temporal columns
* Update copy in tests
* Add test for questions with no temporal columns
* Fix unit test for AggregationPicker
* Allow adding a compare aggregation when the first breakout is not temporal too
* Rename to 'Compare to the past' in custom column action
* Rename to 'Compare to the past' in column drill
* Update copy in tests for actions and drills
* Move canAddTemporalAggregation to CompareAggregation utils
* Use aggregations from the query always
* Remove unnecessary displayName check
* Last Used Param Value doesn't fill in with default value if cleared
Fixes#46368
The Last used parameter value is cleared when the frontend sends `nil` as the parameter's value. This works except in
cases where a user clears a parameter with a default value. In this case, we want to indicate that the parameter is
cleared, so we store `nil` in the appdb in such a case.
* Update src/metabase/query_processor/dashboard.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* add assertion showing that 'nil' with default is still stored
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Fixes: #46558
In #45575 we introduced casting uuid fields for drilldowns.
However we failed to include a check that the rhs was itself another
field of uuid type and so no casting should occur. This broke joins on
models when the database_type of the field may not be known.
* Add tooltip to the data step cell
* Conditionally display tooltip
* Add unit tests for the data step
* Add unit tests for the join step
* Move the abstraction to the NotebookDataPicker
* Cover the tooltip behavior with E2E test
* Update test
* Add test boilerplate
* Rename test and add assertions
* Add broken test for models
* Add unit of time tests
* Deduplicate code
* Add date parameter tests
* Add test for multiple stages
* Fix model test setup and refactor getColumnsInfos
* Remove redundant assertions
* Format codee
* Extract createQuestion and createModel
* Add unit of time model tests
* Add test for model
* Extract constants
* Add a warning
* Build dataset queries with mbql
* Introduce queryOrders
* Revert redundant change
* Improve Question creation
* Fix spammy GitHub Actions comments
* [DEBUG] trigger conventional commit reminder
* fix syntax errors
* Mock update
* mock another update
* 3rd update
* test if the comment would be updated only if the actual commet changes
* Remove all code relating to testing
* Fix type not compatible with node 18
* Update new Metabase base colors
* Make hover and selected state consistency
* Fix wrong embed frame border color
* Refactor: extract props
* Fix border color
* Fix pivot table cell hover color
* Fix smart scalar text contrast
* Add smart scalar tests + update text color design
* Theme dashboard card menu
* Remove dashcard action menu tests since they don't work in loki
* Fix storybook background color for dark theme
* Update loki snapshots
* Disable untestable tests
* Make sure all storybook inherit all necessary base styles
* Skip untestable tests
* Update loki snapshots
* add setup commands
* fix settings definition
* update environment variables for cli
* handle instances not being ready
* update error messages
* add more specific loading messages
* loading spinner state
* improve error message
* use a fixed demo setup token
* remove extraneous spinner
* update status checks
* update container messages
* update wait timing
* create api keys
* extract constants
* remove manual steps
* Add anonymous tracking + other things. will need to clean up
* Modify SDK for better structure
* remove line from print.ts
* Update webpack.embedding-sdk-cli.config.js back to production
* Add types and add quick note
* Fix a typo
* Add index file, simplify types, use an array
* Add safer json parsing
* use delay of 100ms between each setup call
* Suggestions from review
* ensure that cli works
* Attempt to fix jest errors
* Remove node-fetch from sdk code to hopefully get unit tests working again
* add database connection
* add connection details handling
* refactor asking for database connection info
* apply actual database id for syncing schema
* fix failing database sync step
* allow table selection
* create model for each table
* handle errors in model creation
* fix incorrect model display name
* create x-rays based on user data
* consolidate instance setup message
* workaround for inquirer eventemitter issue
* fix yarn.lock file
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
== Goal ==
Prevent admins of Metabase Cloud instances with Metabase Cloud Storage
from accidentally breaking their upload configuration, since in this
scenario that is automatically managed by Metabase Cloud to use the
attached data warehouse.
== How to test ==
On a subscription with Metabase Cloud Storage:
* The "Uploads" section in the left sidebar of `/admin/settings` does
not show
* A "not found" page is shown at `/admin/settings/uploads` where one
would otherwise expect the "Uploads Settings" page
* `PUT /api/setting` with an `uploads-settings` field in the body fails
with `403 Forbidden` and the setting is not changed
* `PUT /api/setting/uploads-settings` fails with `403 Forbidden` and
the setting is not changed
* `GET /api/setting` contains an element with `key: "uploads-settings"`
and a value
* `GET /api/setting/uploads-settings` responds with the value
* `GET /api/session/properties` contains the `uploads-settings` field
Closes: https://github.com/metabase/harbormaster/issues/5121
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* truncate total text in pie chart
* add story
* Update Loki Snapshots
* add e2e test
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Add unit test for #46574
* Handle non-numeric values in NumberFingerprint
* Be more realistic about NumberFingerprintDisplayInfo type
* Add comment on why the types are unknown
* add setup commands
* fix settings definition
* update environment variables for cli
* handle instances not being ready
* update error messages
* add more specific loading messages
* loading spinner state
* improve error message
* use a fixed demo setup token
* remove extraneous spinner
* update status checks
* update container messages
* update wait timing
* create api keys
* extract constants
* remove manual steps
* Add anonymous tracking + other things. will need to clean up
* Modify SDK for better structure
* remove line from print.ts
* Update webpack.embedding-sdk-cli.config.js back to production
* Add types and add quick note
* Fix a typo
* Add index file, simplify types, use an array
* Add safer json parsing
* use delay of 100ms between each setup call
* Suggestions from review
* ensure that cli works
* Attempt to fix jest errors
* Remove node-fetch from sdk code to hopefully get unit tests working again
* fix yarn.lock file
* revert capture groups change
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
== Goal ==
Currently the config file (`MB_CONFIG_FILE_PATH`) only allows adding or
changing databases, but not removing them. Since connected databases
can be managed as a service by us, it becomes necessary to delete any
information Metabase has about them, including any stored credentials.
Since this is a destructive operation with no way to revert (without
restoring from a backup), we raise the bar for accidental deletion and
require not just a (boolean) flag, but a magic string to be set.
== How to test ==
1. Configure a database as described in https://www.metabase.com/docs/latest/configuring-metabase/config-file#databases.
2. Start your Metabase instance.
3. Verify the database shows up in the "admin" section and it works.
Create a dashboard and questions using the database.
4. Add a `delete: "DELETE_WITH_DEPENDENTS:${name}"` line to your
database configuration section.
5. Restart your Metabase instance.
6. Verify the database no longer shows up in the "admin" section.
Notice that questions using it are gone, too.
References: https://github.com/metabase/harbormaster/issues/5173
* WIP Open notebook data source in new tab
* WIP
* Implement the new functionality with `pickerInfo`
* Use destructured value
* Add the simplest possible E2E test as PoC
* Smoke test
* Add basic coverage for questions
* Add basic coverage for models
* Source question has been archived scenario
* Add basic permissions test
* Expand to the user without wite query permissions
* Fix title
* Add another permission scenario
* Add E2E coverage for a source being native
* Add coverage for the native model
* Address review comment re: `pickerInfo`
* Extract `getUrl` to utils
* Add support for the middle/scroll click
* Add unit tests
* Add a negative test case
* Fix title
Fixes#46440
couple things come together here.
Bad escaping:
The `{` characters are used for format strings and placeholders like
```clojure
impl=> (translate "fr" "After {0}" ["Bob"])
"Après Bob"
```
And if you want _literal_ {'s in your translation string, you must
escape them like "'{" or "'}" using an apostrophe. But this isn't the
end of our worries.
in `resources/zh-CN.edn`, we actually have _two_ translations for the
original string:
```clojure
"User lookup filter. The placeholder '{login'} will be replaced by the user supplied login."
"用户找过滤器，占位符 '{login'} 将替换为用户提供的登录名。"
,,,
"User lookup filter. The placeholder '{login}' will be replaced by the user supplied login."
"用户查找筛选器。占位符“{login}”将替换为用户提供的登录名。"
```
And this later has unescaped curlies.
The fix for that is at
https://poeditor.com/projects/po_edit?per_page=20&id=200535&id_language=31&filter=&order=&search=User+lookup+filter
but will need to wait for the new source translation string in this
instance with the corrected escaping.
```clojure
;; simulate what happens at runtime
impl=> (translated-format-string "zh_CN"
"User lookup filter. The placeholder '{login}' will be replaced by the user supplied login."
{})
"用户查找筛选器。占位符“{login}”将替换为用户提供的登录名。"
impl=> (MessageFormat. *1 (locale "zh_CN"))
Execution error (NumberFormatException) at java.lang.NumberFormatException/forInputString (NumberFormatException.java:67).
For input string: "login"
;; the problematic zh-CN string
impl=> (MessageFormat. "用户查找筛选器。占位符“{login}”将替换为用户提供的登录名。")
Execution error (NumberFormatException) at java.lang.NumberFormatException/forInputString (NumberFormatException.java:67).
For input string: "login"
;; minimal reproduction
impl=> (MessageFormat. "“{login}”")
Execution error (NumberFormatException) at java.lang.NumberFormatException/forInputString (NumberFormatException.java:67).
For input string: "login"
```
* fix: mongo ObjectId should be orderable
Fixes#46259
Mongo ObjectId derive :type/TextLike but that can't be an orderable
type, so we mark the specific Mongo type as orderable for malli schemas.
* Fix test
* prevents users from moving items to the trash if they are in the trash already and archives instead of moves in cases where item is moved to or from the trash
* reverts detecting if dragging in or out of the trash not setting the collection id, this will be prevented on the BE instead
* adds test coverage for dragging and dropping items to/from/within the trash (one of which is failing until we can make a BE change)
* Throw away `collection_id` when marking archived
This is a hack around a frontend issue. Apparently, the undo
functionality depends on calculating a diff between the current state
and the previous state. Sometimes this results in the frontend telling
us to *both* mark an item as archived *and* "move" it to the Trash.
Let's just say that if you're marking something as archived, we throw
away any `collection_id` you passed in along with it.
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
### Description
This change should fix some explosions in stats due to phantom fields, while also making things a bit more rigorous.
- Record the table schema in query_field - especially useful for unknown tables
- Allow null table names in query_field - they might be phantoms, or they might not
- We still have the option to filter them out in tools we build on top
- Tighten up matching logic to consider schema
* Update popover dark theme on part of ParameterFieldWidget (#45813)
* Add filter visual tests
* Simplify embed CSS colors
* [WIP] theme filter popover
* Move theme to global node, so it affects portal too
* Fix embed colors
* Theme part of ParameterFieldWidget
* Add story for parameter with dropdown
* Update loki snapshots
* Update more popover style
* Update loki snapshots
* Remove outdated snapshots
* Update loki snapshots with production build
* Fix loki not working with `userEvent`
* Theme mantine popover
* Improve dark theme text-brand visibility
* Update loki snapshots from master change
* Fix new colors not working in SDK
* Fix flaky Loki tests
* Filter popover dark theme part 2 (#46284)
* Theme Date all options
* wip add month and year filter
* Fix wrong CSS custom property declaration
* Test hover state
* Update Loki snapshots
* Theme date month and year
* Add quarter and year tests
* Theme date single and date range
* Style relative date
* Update frontend/src/metabase/admin/datamodel/components/filters/pickers/DatePicker/DatePickerHeader.styled.tsx
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Complete theming filter popover night mode (#46365)
* Style unit of time and number widget
* Attempt to fix flaky Loki snapshots
* Address white background checkbox on dark theme
* Style search filter single value
* Fix mis color
* Update snapshots
* Remove duplication on play functions since they're long
* Milestone 2 polishing (#46382)
* Matches the footer padding with header padding
* Update snapshots
* Add E2E to prevent further problem
* Try to reduce flaky Loki tests
* Remove unused color shades (#46421)
* Update no results illustration to work better on dark theme
* Update loki snapshots
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Refactor dashboard helpers to TypeScript
* Reuse existing helper
* Refactor cy.createQuestionAndDashboard to a function helper
- there was no good place to put it, so I also created new helpers/api directory
- and I moved all helpers using cy.request (with 1 exception, where a CSV download helper also does a bunch of assertions) in there
* Merge TS command definitions into a single file
* Export types
* Make dashboardDetails optional
* Export StructuredQuestionDetails
* Remove duplicated function
* Extract createNativeQuestion to separate file
- 1 exported helper per file, no exceptions!
* Reuse const
* Add test for unit of time parameters
* Improve assertions
* Extract createDashboardWithParameters
* Add tests for time parameters
* Fix hardcoded parameter_mappings ids
* Format code
* Extract checkButtonVisible
* Improve test names
* Add tests for location parameters
* Reuse const
* Update test name
* Add WidgetStatusButton
* Add unit tests for WidgetStatusButton
* Rename handler
* Remove redundant assertions
* Add iconSize prop
* Fix styling
* Replace WidgetStatusIcon with WidgetStatusButton
- Remove WidgetStatusIcon
* Fix tooltip issue
* Fix tooltip for good
* Use default compact button size
* Prevent overflow
* Fix alignment issues in parameter sidebar
- Remove iconSize prop
- Simplify StatusConfig
* Update padding in Unit of Time input to make it consistent with other inputs in dashboard parameter sidebar
* Update unit tests
* Rename WidgetStatusButton to WidgetStatus
* Explicitly specify what is a button
* Update helpers in tests
* Update when buttons are displayed according to the new spec, update tests
* Simplify label
* Remove utils
* Adjust padding
* Update reset icon
* Make the button gray when color is undesired
* Remove old definitions
* Add missing export
* Remove unused type
* Add test for location (single value)
* Add test for id (single value)
* Add more assertions
* Add test for id (multiple values)
* Improve name
* Add tests for number (single and multi value)
* Add tests for text (single and multi value)
* Rename file
* Update comment
* Update test name
* Use unique ids
* Use new color name
* Add a test for chevron icons alignment
* Lift className up
* Make types more specific
* Sort attributes
* Revert "Make types more specific"
This reverts commit 2e6994b5ed.
* Revert "Use new color name"
This reverts commit 2cc68df6ba.
* Remove TODO
* Introduce editFilter util, refactor chevron assertions to work with any number of chevrons
* Connect label to the default value input in ParameterSettings
* Add a whitespace in the default value label when required "(required)" is shown
* Add parameter sidebar tests for unit of time
* Add parameter sidebar tests for date filters
* Extract checkParameterSidebarDefaultValue
* Add parameter sidebar tests for location - single values
* Add parameter sidebar tests for location - multiple values
* Use constants for test case names
* Add parameter sidebar tests for id parameters - single values
* Add parameter sidebar tests for id parameters - multi values
* Add parameter sidebar tests for number parameters - single values
* Add parameter sidebar tests for number parameters - multi values
* Add parameter sidebar tests for text parameters - single values
* Add parameter sidebar tests for text parameters - multi values
* Extract checkDashboardFilters
* Use checkDashboardFilters in time parameters
* Use checkParameterSidebarDefaultValue in checkDashboardFilters
* Use checkDashboardFilters everywhere
* Sort helpers
* Rename checkDashboardFilters to checkDashboardParameters
* Rename checkOnlyOneButtonVisible to checkStatusIcon
* Add test suite for sql filters, coverage for text filter - single value
* Add a whitespace in the default value label when required "(required)" is shown - SQL filters
- Similar to 930dd3a50a
* Add test suite for sql filters, coverage for number filters
* Add test for sql filters - date
* Remove unused things, rename placeholder to labelOrPlaceholder
* Apply label only to 1 input
* Add test for sql filters - field parameters
* Fix date tests
* Make all sql filters tests work
* Add assertions for required without default in text filters
* Add assertions for required without default in date filters
* Add assertions for required without default in field filters
* Allow clearing required SQL filters without default value
* Add a test to ensure comma-separated numeric values work
* Update unit test
* Update helper
* Update e2e tests
* Use != null and == null checks with defaultValue instead of just checking if its truthy/falsy
* Run `yarn test-visual:loki` and then `yarn loki approve`
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Add button boilerplate
* Add getCanResetFilters selector
* Move canResetFilters to utils
* Add explicit return types
* Implement the button action
* Rename
* Simplify action
* Optimize: use single action
* Use parameterHasNoDisplayValue to account for empty strings
* Run `yarn test-visual:loki` and then `yarn loki approve` after merging master
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Rename getHiddenParameterSlugs to getTabHiddenParameterSlugs
* Take all tabs into account
* Add unit tests for canResetFilter
* Format test name
* Fix non-human-friendly aria-label
* Add assertions for "Reset all filters" button's visibility
* Unnest helpers from describe
* Add a test for comma-separated values
* Improve typing
* Fix typing
* Add tests for multiple tabs and auto-apply-filters on/off
* Improve test name
* Remove redundant util
* Remove redundant condition
* Verify card results get filtered
* Test that all tabs are affected
* Fix test case with autoApplyFilters: false
* Add tests for resetting to default value
* Fix wrong initial state in reduce
* Fix updated parameters not being applied
* Add reset all filters assertions
* Remove unused attribute
* Run `yarn test-visual:loki` and then `yarn loki approve`
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Run `yarn test-visual:loki` and then `yarn loki approve`
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Refactor dashboard helpers to TypeScript
* Reuse existing helper
* Refactor cy.createQuestionAndDashboard to a function helper
- there was no good place to put it, so I also created new helpers/api directory
- and I moved all helpers using cy.request (with 1 exception, where a CSV download helper also does a bunch of assertions) in there
* Merge TS command definitions into a single file
* Export types
* Make dashboardDetails optional
* Export StructuredQuestionDetails
* Remove duplicated function
* Extract createNativeQuestion to separate file
- 1 exported helper per file, no exceptions!
* Reuse const
* Add test for unit of time parameters
* Improve assertions
* Extract createDashboardWithParameters
* Add tests for time parameters
* Fix hardcoded parameter_mappings ids
* Format code
* Extract checkButtonVisible
* Improve test names
* Add tests for location parameters
* Reuse const
* Update test name
* Add WidgetStatusButton
* Add unit tests for WidgetStatusButton
* Rename handler
* Remove redundant assertions
* Add iconSize prop
* Fix styling
* Replace WidgetStatusIcon with WidgetStatusButton
- Remove WidgetStatusIcon
* Fix tooltip issue
* Fix tooltip for good
* Use default compact button size
* Prevent overflow
* Fix alignment issues in parameter sidebar
- Remove iconSize prop
- Simplify StatusConfig
* Update padding in Unit of Time input to make it consistent with other inputs in dashboard parameter sidebar
* Update unit tests
* Rename WidgetStatusButton to WidgetStatus
* Explicitly specify what is a button
* Update helpers in tests
* Update when buttons are displayed according to the new spec, update tests
* Simplify label
* Remove utils
* Adjust padding
* Update reset icon
* Make the button gray when color is undesired
* Remove old definitions
* Add missing export
* Remove unused type
* Add test for location (single value)
* Add test for id (single value)
* Add more assertions
* Add test for id (multiple values)
* Improve name
* Add tests for number (single and multi value)
* Add tests for text (single and multi value)
* Rename file
* Update comment
* Update test name
* Use unique ids
* Use new color name
* Add a test for chevron icons alignment
* Lift className up
* Make types more specific
* Sort attributes
* Revert "Make types more specific"
This reverts commit 2e6994b5ed.
* Revert "Use new color name"
This reverts commit 2cc68df6ba.
* Remove TODO
* Introduce editFilter util, refactor chevron assertions to work with any number of chevrons
* Connect label to the default value input in ParameterSettings
* Add a whitespace in the default value label when required "(required)" is shown
* Add parameter sidebar tests for unit of time
* Add parameter sidebar tests for date filters
* Extract checkParameterSidebarDefaultValue
* Add parameter sidebar tests for location - single values
* Add parameter sidebar tests for location - multiple values
* Use constants for test case names
* Add parameter sidebar tests for id parameters - single values
* Add parameter sidebar tests for id parameters - multi values
* Add parameter sidebar tests for number parameters - single values
* Add parameter sidebar tests for number parameters - multi values
* Add parameter sidebar tests for text parameters - single values
* Add parameter sidebar tests for text parameters - multi values
* Extract checkDashboardFilters
* Use checkDashboardFilters in time parameters
* Use checkParameterSidebarDefaultValue in checkDashboardFilters
* Use checkDashboardFilters everywhere
* Sort helpers
* Rename checkDashboardFilters to checkDashboardParameters
* Rename checkOnlyOneButtonVisible to checkStatusIcon
* Add test suite for sql filters, coverage for text filter - single value
* Add a whitespace in the default value label when required "(required)" is shown - SQL filters
- Similar to 930dd3a50a
* Add test suite for sql filters, coverage for number filters
* Add test for sql filters - date
* Remove unused things, rename placeholder to labelOrPlaceholder
* Apply label only to 1 input
* Add test for sql filters - field parameters
* Fix date tests
* Make all sql filters tests work
* Add assertions for required without default in text filters
* Add assertions for required without default in date filters
* Add assertions for required without default in field filters
* Allow clearing required SQL filters without default value
* Add a test to ensure comma-separated numeric values work
* Update unit test
* Update helper
* Update e2e tests
* Use != null and == null checks with defaultValue instead of just checking if its truthy/falsy
* Run `yarn test-visual:loki` and then `yarn loki approve`
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Use parameterHasNoDisplayValue to account for empty strings
* Run `yarn test-visual:loki` and then `yarn loki approve` after merging master
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Run `yarn test-visual:loki` and then `yarn loki approve`
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Run `yarn test-visual:loki` and then `yarn loki approve`
- see https://metaboat.slack.com/archives/C505ZNNH4/p1721289287037069
* Make number also have custom parameter values
* Render the correct label in the dropdown for number inputs
* Mock paramter values endpoint in number input test
* Add test for custom labels on number input
* Fix number input test
* Allow entering comma separated values using labels
* Only show custom labels in popover
* Use getLabel and getValue helpers to avoid indexing confusion
* Remove calls to parameter endpoints for number input
* Do not parse the value from the label
* Add support for number parameters in the source picker modal
* Filter number values
* Add e2e test for number parameters with custom source
* Remove card option for number parameters
* Set the correct initial value for number parameters without fields
* Do not render number parameter as search box when it is a list
* Fix e2e tests for number lists
* Remove e2e tests for card source dashboard filters
* Add tests for native filter custom source
* Show the dropdown when picking the default value for number parameters
* Clear values_source_config when changing parameter type
* Restore a parameters' values_source_config when changing the parameter type back
* Clear and restore values_source_config for template tags when chaning type
* Add e2e test for restoring of template tag sources_value_config
* Add e2e test for clear and restore of static-list for dashboard parameters
* Fix input type for numbers
* Fix broken test
* Display labels for ParameterValuePicker too
* Clear and restore values_source_type when changing parameter type
* Clear and restore values_source_type for tag parameters
* Clear and restore values_query_type when changing parameter type
* Clear and restore value_query_type for parameters tags
* Update tests to test for reset to also apply to values_source_config and values_query_type
* wip
* moves code to enterprise folder, adds enterprise guards, updates conditionals as to when to show the feature
* impls filter date logic, improves styling, mocks out real endpoint with collection items endpoint, updates table with isLink boolean rath
er than using custom item renderer, removes dead code
* use the actual url for stale
* so many things... updated design, custom table component, polish, misc. refactors, more correct api typing
* updates to pagination component and reverts some now unused changes to the base items table
* impls pr feedback
* fixes
* fixes failing unit test
* completes pr feedback from raffi
* implements second round of feedback from raffi
* adds tests and couple of small fixes / feedback
* migrate clean up action to enterprise plugin and fix related e2e test, also skip the collection items count query when we do not need the value
* finish main base case e2e test
* clean up helper code for manual testing and add OSS test to make sure feature is not available in OSS
* better handling of min-height on smaller screens
* adds internal undo notification for bulk archiving in the clean up modal
* reset pagination when items are archived/unarchived to keep users from getting stuck on a page beyond the current view
* pr feedback
* further cleanup
* unit test fix
* fixes issue where archiving all items in the modal and then undoing does not result in the stale items list getting refetched
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* hide notebook button for brand new questions
* test notebook button visibility for new questions
* move brand new check inside shouldRender
* test embedding SDK notebook button
* update unit tests
* add spy to shouldRender
* Fix coll permissions for audit collection
We've had a function in `models.collection` for a while that has taken a
permissions set and returned a set of collection IDs that the user has
permissions on. I refactored this recently, but didn't notice that it
was actually doing the permissions checks slightly incorrectly.
Specifically, because it only looks at the user's permissions and the
collection IDs and doesn't use `mi/can-read?` or `mi/can-write?`, it's
completely indifferent to whether a collection is an audit collection or
not.
This is arguably not a *permissions* issue: the two errors that could
come about here are:
- someone sees the audit collection even though the audit feature is
disabled, or
- someone is presented with the audit collection in a context where only
writable collections should be present - when they try to actually write
to it, it fails (since then we're doing the real permissions check).
The primary motivation for this fix was to prevent audit dashboards and
cards from appearing in the list of stale items.
* Fix hardcoded collection ID
We were creating a timeline in a fixed collection ID, that in tests
happened to be the ID of the Metabase Analytics collection.
* properly handle SMTP environment variables
* disable clear button when set by env var
* allow the SMTP form to validate with any combination of environment variables
* add unit tests for SMTP connection form
* BONUS: fix console error for jt keys
* fix types
* updates the ordering of create queries permissions options so that granular permissions is the last option
* fix unit test
* move granular option above no
* update static_embed_code_copied to use downloads (#45944)
* copy schema file for better diff
* adds downloads field to appearance in embed_flow schema
* update event to send `downloads` parameter, update e2e tests
* fix test for dwonloads: false
* fix: use correct schema version for embed flow analytics (#46053)
* update `dashboard_pdf_exported` and use it in embedding context (#46076)
* copy schema for better diff
* adds `dashboard_accessed_via` to the `dashboard_pdf_exported` event
* download_results_clicked event (#46132)
* adds download_results_clicked event + e2e tests
* make strings nullable
* adds maxLength
* switch back to if/then as it seems more readable
* refactor: use the resourceType to determine the download params instead of re-doing the logic
* fix ''question' is possibly 'undefined'.'
* split resource_type into resource_type and accessed_via to also consider interactive-embedding and sdk
* edit comment
* adds test for ad-hoc-question
* Update frontend/src/metabase/query_builder/actions/downloading.ts
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* makes describe message more consistent
* "else if" => if
* fix describe block name
* convert downloads event to use enum for download_type and export_type
* resource_type should be an enum too
* fix line added in the wrong place during merges
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Update frontend/src/metabase/public/components/EmbedFrame/EmbedFrame.tsx
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* adds enum schema to dashboard_accessed_via
* fix accessed_via for sdk embeds
* update comments to explain why we send 0 when dashboardId is null
* only send dashboard_id as a number when it is a number, as expected by snowplow
* remove un-needed async/await from saveAsPDF function
* rename frontend/src/metabase/redux/downloading-analytics.ts -> frontend/src/metabase/redux/downloads-analytics.ts
* fix cy.log message
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* update embed frame pdf button colors to use --mb-color- vars
* change download pdf button color to brand color
* testing adding the images that actually differ
Previously we unpacked everything and picked up first entry as the directory
containing the data. This works well when archives are passed as is. However,
if a user unpacks and then packs an archive, there is a good chance that some
hidden directories will appear - and they are first in order.
So we skip those right now, and additionally check known markers of the
serialization data directory.
* Squash
* Clean up query modifier for validating card references (#46283)
* Fixup bad git merge on test expectation
* Tweak docstrings and de-densify logic
* Allow filtering on collection
* Add `effective_ancestors` to collection data
Note: I removed the `qv=` comparator. I think it'.s better to be more explicit
about what exactly we're asserting. In particular, I hadn't updated the
`:count` when we were filtering on collection type, so the count was
still the total for *every* collection even though the results were being
restricted by the collection. However the `qv=-data` function was still
reporting that everything was OK, because it was accounting for other
data from the App DB by just asserting that the count we'd specified was
lower than the actual count. I'd argue it's better for tests to fail
locally because I have garbage in my app DB (which I can clean up) than
succeed when they really shouldn't.
* Consolidate effective ancestors
For the Search API, we're currently returning slightly different
ancestors:
- it never has the Root Collection, and
- the ancestors don't have a `personal_owner_id`
We could change the API (and maybe will soon - it seems pretty
unexpected to have `effective_ancestors` sometimes act one way and
sometimes act another) but for now we can just reshape things to look
the same as they did before, while using the same code under the hood.
* echarts pie
* setup rendering pipeline for static pie chart (#43547)
* setup rendering pipeline for static pie chart
* remove more donut code from BE
* compute viz settings for static pie chart (#43548)
* compute viz settings for static pie chart
* create chartModel for pie chart (#43549)
* create chartModel for pie chart
* render static pie chart (#43550)
* render static pie chart
* render static pie chart legend (#43551)
* disable animation
* alter static-viz pie chart be tests to account for echarts-impl
The old implementation had a different implementation for legends, separate from the 'categorical/donut' js
render. Echarts nicely combines the legend into the pie render, so we don't need to worry about that stuff anymore.
Instead, I've written a simple set of assertions on the echarts pie implementation now.
* Get rid of this test that was attempting to use old code path
* echarts pie
* One last spot where we don't need old pie stuff anymore.
* add loki specs
* save snapshots
* fix small slices disappearing
* update snapshots
* add repro story for exteremely small slice
* add snapshot
* fix rounded total
* fix percent formatting in legend
* fix date dimension formatting
* fix binned and relative date dimension labels on legend
* add specs and snapshots for null and unaggregated dimensions
* hide overlaping labels on chart
* fix null dimension color
* show error when metric column has a negative value
* move legend to botom
* make dot margin smaller
* add columns to legend
* sort legend vertically
* dont use grid if only one row for legend
* never hide legend in static viz
* fix empty column culling
* show single legend item
* add long dimension name story
* fix long dimension name in legend
* right justify percentages in legend
* remove negative error and re-add stories
* handle negative values
* create chart definition for dynamic pie chart
* render chart with responsiveness
* add comment for borderWidth calculation
* center legend
* fix legend centering
* add story for missing currency formatting
* fix missing currency formatting
* fix graalvm crashing
* fix column settings computation
* add another currency formatting repro
* fix wrong order of col settings computations
* convert colors to hex
* fix colors missing from viz settings
* temp todo comment
* fix colors in dynamic viz
* update snapshots
* fix crashing due to invalid dimension/metric setting
* fix chart crashing for boolean dimension with labels on chart
* add repro
* fix regression in label formatting
* fix labels on small slices
* update specs
* emphasis state for slice hover
* recreate old label hiding logic
* include height measurement in getIsLabelVisible
* move d3 slices to chart model
* fix labels on static viz
* wip basic hover state and tooltip
* fix emphasis style getting stuck by memoizing everything
* update total display value
* fix stuck emphasis by setting notMerge to false for pie only
* handle other slice in tooltip
* fix bugs in total graphic
* use OTHER_SLICE_KEY const in static viz legend
* implement legend and connect to hover state
* implement click actions
* fix slice not clickable
* handle all zero slices
* add repro
* update specs
* update pie_chart.cy.spec.js
* update chart_drill.cy.spec.js
* add repro for single column legend
* use minimum 2 legend columns
* fix legend text overlap
* use more margin for single row legend
* update specs
* add repro for 28568
* add repro for 38424
* fix percent label not showing on large slices
* hide total text when chart is small
* replace comment about animations
* show warning for mixed positive and negative values
* delete old chart files
* fix type errors
* update d3 import
* update specs
* remove branch from uberjar.yml
* dont put single slice into other
* fix lint errors and pie render test
* update more d3 imports
* remove unused settings
* revert preview-card-width change
* remove unused import
* remove majorWidth option from formatting
* use debounceLeading refresh mode in ChartWithLegend
* remove TOTAL_GRAPHIC_OPTION constant
* delete unit test for pie chart
* fix failing test by doing set comparison instead
* fix typo in comment
* remove SUNBURST_SERIES_OPTION constant
* fix lint by removing unused eslint-disable
* move default color computation to getValue
* fix static pie test to show that we will always render the legend
* update pie border color for night mode and embedding
* fix styles for embedding case and add stories
* skip embedding stories
---------
Co-authored-by: Adam James <adam.vermeer2@gmail.com>
* Make an internal API for retrieving stale content
This introduces the `metabase.stale` module, which has one public
function (so far): `metabase.stale/find-candidates`. This takes a set of
`collection-ids`, a cutoff date, and pagination/sort info.
Next, we introduce an HTTP API endpoint that will figure out the appropriate
set of `collection-ids` for a given user/request, call this function,
and return the results in the shape we want.
Also: for e2e tests, we need a way to mark a card or dashboard as stale. I
think the easiest way to do this is to just create a testing endpoint
that we can call to update the `last_used_at` or `last_viewed_at` of the
card/dashboard to set it to ~7 months ago.
---------
Co-authored-by: bryan <bryan.maass@gmail.com>
* use mantine buttons for filter and summarize
* Restore fancy color hover states
* Add css modules
* oh the cleverness of me
* better var name
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
Expands and refactors `metabase.util.memoize` to support better cross-platform memoization.
**Better basic memoization**
Use `clojure.core.memoize/memo` (and its clone in CLJS) rather than `clojure.core/memoize` for
the "everything forever" strategy.
**Bounded memoization**
There are a few places (eg. `u/kebab-case-en`) where we want zero-overhead *hits* and expect the input space
to be fixed. To guard against runaway memory usage, this **bounded** strategy dumps the entire cache when it
overflows. It logs an INFO level note when that happens - the intent here is that the threshold should never
actually get hit!
**Fast JVM versions**
In the special case where the function to memoize takes exactly 1 argument which is a valid map key, we can
use `ConcurrentHashMap.computeIfAbsent` as the basis of the cache and have much less overhead.
`fast-memo` and `fast-bounded` implement this in CLJ; in CLJS they just call `memo` and `bounded`.
* Use function helpers
* Only assert the resize down
* Refactor repro 31701
* Remove flaky tag
* Better organize describe blocks
* Move custom `requestTimeout` config to the common block for resizing tests
* Refactor dashboard helpers to TypeScript
* Reuse existing helper
* Refactor cy.createQuestionAndDashboard to a function helper
- there was no good place to put it, so I also created new helpers/api directory
- and I moved all helpers using cy.request (with 1 exception, where a CSV download helper also does a bunch of assertions) in there
* Merge TS command definitions into a single file
* Export types
* Make dashboardDetails optional
* Export StructuredQuestionDetails
* Remove duplicated function
* Extract createNativeQuestion to separate file
- 1 exported helper per file, no exceptions!
* Reuse const
* Remove old definitions
* Add missing export
The `dashboard_load_id` is attached to the body for queries, not as a
query parameter. This correctly receives the `dashboard_load_id` and
uses it for `/query` calls.
* pull sdk info into data on-thread
* include api tests for query_execution and view_log
* Try mt/test-helpers-set-global-values!
- disambiguate names used in tests
* pass test + cleanup test helper
* make sure view logging is enabled in test
* fix conditional formatting color lookup with correct col names
The column names for color lookup are expected to be the same as the :name key of each column, and applying column
formatting broke this lookup. This fixes it.
* move overflow-x into pulse body div to exlude card title from scroll
* add a test for conditional formatting w/ a hidden row.
This still fails at the moment, working on a fix yet.
* apply conditional formatting in static viz even when col is hidden
Conditional formatting that depends on a hidden column was not being rendered in the static viz, but now we pass all
columns into the color selector fn, thus when rendering the table with the ordered/hidden columns, all other columns
will still use their appropriate conditional formatting
* I missed a linter warning
* fix tab rendering
* fix test
* fix up tests with some better structural searching
* Adds a middleware that sets 2 dynamic vars
- These vars will be used to insert context for embedding analytics
* migration adding embedding_client and embedding_version to qe and vl tables
* clarify function api
* headers come in -> dynamic vars -> written to db
When the vars are set we insert their values into query execution and
query_log under embedding_client and embedding_version.
* update update migrations
* add predconditions to migrations
* fix tablenames + dataset_test
* Code Review responses
- remove commented out requires
- clarify test for merge-count-maps
* Lock down the trash collection more
Oops. I was very thorough in checking that you couldn't modify the
trash collection in the sense of moving things to or from it. I missed
the most obvious thing: actually modifying the trash collection itself.
So if someone wanted, they could change the name of the Trash to
"Definitely Not the Trash".
* Remove the Trash Collection from serialization
Serialization tests were failing due to the previous change, but this
got me thinking, and I realized that serializing the Trash collection is
completely unnecessary. We want to create it programatically on
startup (which we are) and then nothing should change it afterwards.
I tightened this up by removing the condition here, thinking that CI
would catch it if it was necessary - but I'm getting failures in
development. Oops - didn't realize that this defmethod wasn't actually
applied during CI runs.
* Rewrite repro for 27643
* Add TODO for the remaining tasks
* Make a test name consistent with others
* Finish the E2E test for #27643
* Fix the test to use PostgresDB
Since H2 originally doesn't fail the test
* Attempt to fix a flake on CI by waiting for table to finish syncing
* Create QA snapshot with fully synced field metadata
* Fix failed tests from field metadata change
* Simplify question scenarios
* Simplify dashboard scenarios
* Fix permissions
* Rework titles
* Add comment about PG
* Add limit to speed things up
* Fix and improve dashboard assertions
* Fix and improve question assertions
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Bigquery Pagination Use next page to get proper results
Fixes 45953
WIP
* use new-page in values-iterator
* disable just the part of the test that is still flaky
Want to keep the other part of the test that does prove we're not getting duplicates due to incorrect pagination.
* Number formatter Reports correct Decimal places for negative Numbers
Fixes#41640
Prior to this, the number formatter would get the incorrect number of decimal places for negative numbers.
The way the decimal places count is derived works as follows:
- convert number to a string
- split the string on the separator (decimal, comma, whatever the instance has set)
- destructure that split into [int decimal]
- count decimal to get the number of decimals.
That all works for positive numbers, but for negative numbers, the regex was just splitting on 'not digits', so we got
an extra group due to the leading negative sign. This would incorrectly destructure the 'int' part of the number into
the count, thus we would always get 1 instead of the correct value.
This fixes it by just taking last instead of destructuring.
* correct comparison to always use abs value in conditonal check
* tiny incorrectness in the test
* Modify the E2E stress test workflow to allow running PostgresDB
* Fix syntax
* Add qa_db to workflow summary
* Only run necessary containers
* Fix QA DB snapshots running when QA_DB_ENABLED=false
* Use more generic option that's not only specific to postgres
* perf: Add explicit .entryAt method for SnakeHatingMap
Otherwise, SHM invokes .keySet on itself which triggers the contstruction of the list of keys on each map access.
* perf: Cache (un)successful requiring of EE in dynamic-ee-oss-fn
* perf: Add a quicker path for getting metadatas from cache
* perf: Minor improvements
It's based on a (global, synchronized) `WeakHashMap` using the query map
itself as the key. That gives the cache the same lifespan as the query
map, which is generally brief but often includes many calls to
expensive, cached functions like `visible-columns`.
* Make filter corrupted again™
* Speed up the test by limiting the number of rows
* Make the repro more correct, and more resilient
* Remove intercepts
* Improve assertions
* add webpack configuration for cli
* add requirements checking
* docker check
* docker start command
* port checks
* add getting started steps
* add final install message
* cleanup cli text
* handle force-closing prompt
* build the cli when running build-embedding-sdk
* wording updates
* add redundant prompt in case of small terminals
* remove redundant instruction
* make sure the user confirms the step
* change container name to metabase-enterprise-embedding
* migrate cli to es modules instead of webpack
* add shebang line for npx
* add cli dependencies to devDependencies
* Revert "add cli dependencies to devDependencies"
This reverts commit 5d5e952c2b.
* Revert "add shebang line for npx"
This reverts commit 766fdda09e.
* Revert "migrate cli to es modules instead of webpack"
This reverts commit 7d1b065163.
* Embedding Parameter Values set in Tokens work properly when `nil` or empty list
Fixes: #40292
If a locked parameter's value is set to `nil` or `[]` in the embedding JWT, we want to treat this as 'no value has
been set' on that filter, but we should still run the query as normal. Likewise, any linked filters should still
properly derive their values from the unfiltered values of the filter to which they are linked.
* when a parameter's value is nil or empty seq, don't include it
This is a partial solution as I still need to find the place that's still erroring on the empty list case in terms of
the linked parameter's values.
* Treat empty list like `nil` for the token param
When you're using a filter in the App, it makes intuitive sense that when you've not selected anything from a filter's
drop down list (or entered any text, etc.), then the filter is effectively not applied.
For token params, we should behave similarly; `nil` and `[]` can both be accepted, and signal that the user intends
for that filter to basically NOT apply.
This also means we should not pass the constraint to the linked filters, since it errors, but even if it worked, it
would return 0 results, making it impossible to properly apply the linked filter at all.
So, this change causes empty lists to be treated as nil, AND will remove any parameters with `nil` as a value before
being passed to the query processor, which results in the proper behaviour of not having constraints on the linked
filter, and having the 'parent' filter not filtering anything.
* Embed preview should always use params from the token
In making a screenshot, I discovered a slight bug in previews, where if the embed had been previously published, it
could pull embedding params from the db that don't match the settings on screen. So, always pull the settings from the
token, which updates as the user is making changes in the Preview Embed UI.
* for previews, merge the embedding params sources instead
* alter 'nil' behaviour to work just as if the key doesn't exist
* fix one spot where I was passing maps into set functions
* Adjust tests to check for consistent behaviour in dash/qstn embeds
* fix an e2e test
* fix up last 2 failing e2e tests
* add the test case of a locked param having no value set in token
* some further e2e changes
We have N+1 detection for simple hydration methods that checks to see
whether the hydration method made any DB queries at all. This makes
sense, because a simple hydration method is called on each instance
individually - if we make database calls *each time*, we're in trouble.
Where this runs into issues is with cached data, where the simple
hydration method is populating the cache - e.g. `:can_read` or
`:can_write` dereferencing `api/*current-user-permissions-set*`.
Currently we specifically exclude those hydration methods from the
check.
But after running into this again for a different simple hydration
method, I think there's a more robust approach:
- first, run the hydration method without counting DB queries
- then, run it again and make sure we don't make any additional DB
queries.
There is a potential downside here: if someone wrote a simple hydration
method with side effects, they'd potentially be very confused (in
development - this doesn't run in prod!) when those side effects
happened twice for a single hydration. But I *think* that's fairly
unlikely.
* handle errors when useMetabaseAuthStatus is used outside of sdk context
* address review comments feedback
* extract useSdkContext
* fix import type error
* polyfill useSyncExternalStore for React 17
* adds the shim
* use ReactDOM.render instead of createRoot for backwards compat
* use createRoot for React 18 environment
* refactor renderRoot compatibility
* remove semver dependency
* add small test for version
* use the same check for react 17
- babel is kept to transpile fonts file as swc emotion plugin doesn't handle our case correctly. I didn't find a root cause of wrong behaviour, but the safe past is to keep using babel + emotion plugin for one file and all others can be transpiled with swc
- react fast-refresh now handled by swc
- swc-loader can't read browserslist config from package.json, so we had to manually specify supported browsers in webpack
* Uninstall ReplayIO
* Remove Replay config
* Install and use custom Chrome v111
* Simplify comment
* Set a specific Chrome version commit
* Skip repro for #22517
* fix: replacing joins and conditions updates alias
Fixes#37074
* Only change alias if conditions or table changed
* Fix test
* Fix fe test
* Address PR feedback
* Remove can_run_adhoc_query from the FE
* Remove can_run_adhoc_query from the FE
* Disable can_run_adhoc_query on dashboard
Was impacting performance on dashboard load. Timings locally for me:
get dash: 3178 -> 199
get metadata: 3604 -> 1002
So dashboard went from 6 seconds to load to 1 second.
* Fixup test from disabling can_run_adhoc_query
---------
Co-authored-by: dan sutton <dan@dpsutton.com>
* adds database_id to recents, and question collection items
* don't show unjoinable things in recents either
* unskip repro
* fix types
* fix repro
* fix tests in model/recent_views
* add unit test for invalid join in recents
* fix tests in test/metabase/api/collection_test.clj
* update type
* improve unit tests
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Add `report_dashboard.last_viewed_at`
- defaults to `now()` (it's intended to be used as a timestamp after which
we're *sure* the item hasn't been used, not a definitive timestamp we're
sure the dashboard was actually used at)
- updated whenever the QP processes a query for a dashcard
---------
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* remove dynamic import to make html2canvas work on the sdk
* Revert "remove dynamic import to make html2canvas work on the sdk"
This reverts commit a8b511fa28.
* manually import html2canvas-pro and jspdf in the sdk
Previously, due to over-eager `.flush()` calls in Cheshire, we were
flushing the output stream for every `data.rows` entry. Typically this
resulted in 3 + N TCP packets sent, where N is the number of rows being
returned.
This PR rebuilds how we stream the JSON responses to avoid extra
`.flush()` calls. For the unit test in this PR, that reduced the number
of packets sent over the wire from 37524 to 4.
Fixes#34795.
* Post sync hook stub for implicit joins testing stub
* Add join alias to field lvalues to enable sorting on joined fields
* Disable :foreign-keys on Mongo
* Require :left-join support for implicit joins instead of :foreign-keys
* Update implicit joins tests
* Adjust sync-fields-test
* Update implicit joins feature check test
* Transform post-sync-hook to normal function
* Add foreign key relationships only for dbmses without :foreign-keys feature
* Update test to handle Oracle correctly
* Split convoluted fn
* Avoid unnecessary computations for datasets with no fks
* Update docstring
* Fix driver usage
* Transform :foreign-keys to :metadata/key-constraints in test data loading code
* Update sync_test.clj
* Update driver_test.clj
* Update moviedb.clj
* Update dataset_definition_test.clj
* Update fetch_metadata.clj
* Update fields_test.clj
* Update driver.clj
* Update driver/sql.clj
* Set sql driver join support to true for all joins
Deriving drivers are expected to set to false where applicable.
* Update sqllite.clj
* Remove foreign-keys from spark
* Remove :foreign-keys from presto
* Remove :foreign-keys from Athena
* Remove foreign-keys from big query
Reading docs it seems fk inference should be ok. Let's see the test results. Act based on that.
* Update test_metadata.cljc
* Set key-constraints to false for bigquery
* Add foreign keys to sqlite manually until sync is fixed
* Return driver require to Athena
* Correct typo
* Add naive primary key heuristic
* Update pk fk logic to handle name components correctly
* Add alias escaping to presto
* Add ordering to test
* Add order by to test
* Update test
* Remove use of rewrite-fields-to-force-using-column-aliases in order by fields
* Add exception to alias forcing
* Different approach to exception from alias forcing
* Alternative approach for prefixing idents in bigq
* All seelcted fields by desired alias
* Rewrite only fields not from this source table
* Update test
* Enable breakout-on-fk-field-test for :left-join drivers
* Add feature comment
* Explicit joins tests foreign-keys removal
* Update nested_queries_test.clj
* Update remapping tests
* Update tests
* Update tests to handle sqlite results format
* Disable metadata/key-constraints on sqlite during tests until
* Address remarks
* Remove mt/with-mock-fks-for-drivers-without-fk-constraints
* Update bigquery test
* Update tests
* Adjust row level restrictions
* Add parameterized-sql feature
* Update comment
* Update leftovers
* Order keys
* Remove foreign keys from frontend
* Fix FE unit
* Update driver changelog
* Address review remark
* Update test/metabase/query_processor/test_util.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update docs/developers-guide/driver-changelog.md
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update docs/developers-guide/driver-changelog.md
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update modules/drivers/bigquery-cloud-sdk/src/metabase/driver/bigquery_cloud_sdk/query_processor.clj
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update docs/developers-guide/driver-changelog.md
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Update test
* Update comment and reduce expression
* Update comment
* Address remarks
* Fix merge
---------
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
* Derive MySQLEnum from Text instead of TextLike
Fixes#44807.
This change is in accordance with https://dev.mysql.com/doc/refman/8.0/en/enum.html describing the ENUM types as a
kind of string. All our string operations seem to work with MySQL ENUM fields.
* docs: extract requires a db with regex
* Update docs/questions/sharing/visualizations/table.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* tests for drivers with spaces in their identifiers
* Add driver feature
* Fix tests
* Remove ambiguous alias for sqlite
* Try to fix bigquery and sqlite tests not seeing left-join support
* Add explicit support for left-join because of foreign-keys during test hacks
* Handle left-join support weirdness
* Fix test
* Fix falsy tabSelected/tabRenamed checks
* Update URL management (WIP)
* Fix tests failing because of whitespace changes
* Fix tests failing because of parameter order
* Use one hook for parameters and tab URL management
* Fix tests failing because of parameter order
* Fix test failing because of whitespace changes
* Revert
* Fix DashboardTabs tests
* Fix tests failing because of parameter order
* Fix test
* Fix test failing because of whitespace changes
* Remove unit tests that should be e2e tests now
* Fix tab navigation
* Fix string encoding/decoding
* Revert test changes
* Consistent question search params
* Update DashboardTabs tests
* Fix `SyncedParametersList`
* Fix e2e tests
* Fix nav usage
* Fix types
* Remove `useSyncedQueryString`
* Fix test
* Move slug utils to dashboard utils
* Fix nav
* Revert nav change
* Fix e2e tests
* Fix public/embedded dashboards, handle site url
* Add repro test
* Bring back and fix `PublicOrEmbeddedDashboardPage` tests
* Move `SyncedParametersList` to qb
* Refactor EditoTabs
- get rid of options prop
- use named export
* Remove unused data-testid
* Remove redundant id and aria-labelledby
* Remove redundant props spread
* Bring back data-testid in use
- Thanks VSCode for randomly showing me search results or not...
* Fix navigation issue
* Add a repro for first issue
* Adjust EntityPickerModal root element so that it works with "be.visible" assertion in Cypress
* Add a repro for second issue
* Refactor tests
- Fix not enough history entries issue
* Change order of tests to start with simpler one
* Fix invalid request issue
* Add more assertions
* Format code
* customize popover z-index
* make popover z-index customizable
* make the popover z-index customizable
* add tests for popover z-index
* also specify z-index for mantine popover
* Reuse connection for test data loading & JDBC data loading overhaul
* Revert silly change
* Remove unneeded logic for recording which datasets are loaded or not-yet-loaded
* Ok so I guess we do need the code to detect duplicate dataset loading after all
* B I G improvements
* Appease Kondo
* Test fixes 🔧
* More test fixes 🔧
* Fix dataset-already-loaded?-test
* Fix test
* Fix Presto JDBC and greatly improve test data load speed for it
* Postgres test fix 🔧
* Fix compilation errors
* MySQL fixes 🔧
* Fix syntax error
* MySQL fixes 🔧
* Implement `dataset-already-loaded?` for Athena and cache the set of existing databases
* Fix Redshift
* Fix Presto JDBC
* Vertica test fix 🔧
* Fix Oracle dataset-already-loaded?
* Test fixes and parallelize some API tests.
* Test fixes 🔧
* Oracle test fix? 🔧
* Another test fix 🔧
* Stop putting random spaces in code bodies this is not JavaScript code
* with-temp-vals-in-db should merge in original value of Database settings
* The most test fixes 🔧
* 3 or 4 test fixes 🔧
* Add schemas around stuff to catch errors
* Appease Kondo
* Implement dataset-already-loaded? for vertica
* Implement dataset-already-loaded? for SQLite and Snowflake
* [WIP] Testing driver filters
* Only test returned values in CI
* Add :uuid-type for dbs that support that type (test oracle's support)
* Try again for oracle test, redshift does not support uuid-type
* Add tests, support where available
* Add empty/null tests
* Adjust test query for athena
* Reset logging
* Fix comments
* Add Athena uuid conversion
* Address PR review
* Add comments
* Address PR comments
* Use varchar for portability. Add inline uuid casting for athena
---------
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Field Filter Value Normalization in Embedding Endpoints
Fixes 27643
Since we use query-params to pass paramter values, we have to normalize the parameter map values.
This PR is a WIP, as I'm trying to find a way to do this hollistically/properly.
First commit here is mostly as a marker for where to start working on this properly, as I think the json parsing won't
catch all cases, nor should the normalization happen without some attempt to use a schema to properly validate, maybe
even coerce things.
* read strings if boolean or numbers, replace blank str with nil
Also update the common dashboard param values fn to use embedding params from the URL blob if the app db doesn't have
embedding params. This happens upon first preview where nothing will have been written to the Appdb yet.
* Add query-param normalization test, add related embedding test
* schema didn't work in this case after all.
Worth a shot, but don't want this to block the PR, so reverting to the previous schema
* Add comment pointing out the downsides/reasons for this solution
It's nto a perfect solution, but it's probably the best we can get at the moment, without adding a new filter
type (boolean) and/or improving the design of hte embedding endpoints and how filters/parameters work
When we run queries, we preprocess them, during which the sandboxing
middleware adds information that we use when checking permissions
later on in query processing.
When we checked data permissions before saving a card, we were checking
permissions on the raw query, without preprocessing. Without the data
provided by the sandboxing middleware, the permission check sometimes
failed (depending on the permissions for other tables in the DB).
Note the big hack here: preprocessing can fail, for lots of reasons.
This broke a *lot* of tests that were passing in invalid queries. If an
exception occurs during preprocessing, we catch and just use the
original query like we were before.
* refactor: Simplify code (#44275)
* Remove dead code
* Remove values same as default options
* Simplify hook API: Removing unused argument
* review: Remove redundant code
* Update look and feel copy (#44277)
* Update static embed tab name
* Update copies + rearrange settings
* Use Mantine Select
* Fix E2E tests by applying new copies
* Fix unit tests
* Fix more E2E tests
* review: Add a comment on Mantine `value` quirk
* Adjust spacing and relevant tests
* Move `transparent` theme to its own toggle option (#44362)
* Reorder the function to match the visual order
* Make function name consistent
* Introduce the new background option (do nothing yet)
* Remove redundant exports
* Tighten up values to reduce confusion
* Make the parameters order consistent
* Only show background option on dashboads (spec)
* Allow dashboard embed background to be transparent
* Make filter background and header/footer transparent
* Fix type error
* Fix unit tests
* Fix E2E tests
* Fix checkerboard pattern not showing when no background
* Add unit tests
* Add missing copy update
* Remove remnants from outdated solutions within the PR
* Improve code comment references
* Fix E2E tests since the last copy change
* Fix type errors from merging master
* Update light theme style (#44488)
* Add background to dashboard filter on light theme
* Make position sticky works with transparent background
* Fix sticky filters not working on theme=transparent
* Fix unit tests
* Simplify IntersectionObserver components + fix E2E tests
* Add storybook for embedded dashboard
* Clarify the use of ExplicitSize context
* Remove unused code
* Formatting
* Review: Better mock `isWithinIframe`
* Update dark theme to match the new design (#45134)
* Update light theme checkerboard colors
* Update checkerboard colors
* Consolidate dashboard light/dark border style
* Use `--mb-color-text-primary` for light theme dashboard
* Update embed dashboard dark theme primary color
* Migrate dashboard text-secondary color
* Migrate text-tertiary color
* Migrate gold, axis, and split line to CSS custom properties
* Fix failed e2e tests
* Fix static viz not working
* Update loki snapshots
* Remove unnecessary CSS custom property
* Make diff more consistent
* Address review: reusing type
* Support backward compatibility when passing `theme=transparent` (#45214)
* Match preview query with the actual code query
* Handle backward compatibility when `theme=transparent` on dashboard
* Add unit tests
* Address review: Make tests less repetitive
* Update Metabase banner upsell UI (#45424)
* Prepare folder for nested components
* Update upsell UI
- add `fullWidth` variant
- align title to the left
* Fix Upsell banner having horizontal scroll on full width
* Add new Metabase banner upsell component
* Fix unit tests
* Address review: Fix typo
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
* Address review: Move upsel UI to `admin/upsells`
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
* Remove unnecessary ESLint disable comment
We can remove it since the file is in admin folder, and is already excluded from the rule.
* Address review: remove default value property
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Address review: Extract magic number
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* More UI updates (#45450)
* Update dark theme text colors to match the new design
* Update powered by metabase logo
* Update loki snapshots
* Fix unit tests
* Fix formatting
* Fix wrong dark theme metabase logo badge color
* Address review: Compress SVG with svgo
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Fix E2E tests
* Update loki snapshots
* Clarify Metabase base colors with comments
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Add more tests (#45628)
* Add static embed question stories
* Add loki snapshots
* Add transparent theme for static dashboard tests
* Add transparent theme tests for static questions
* Assert `background` option in analytics tests
* Remove tests for logic that has been moved
* Update loki snapshots
* Fix failed E2E tests from merging master
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Refactor EditoTabs
- get rid of options prop
- use named export
* Remove unused data-testid
* Remove redundant id and aria-labelledby
* Remove redundant props spread
* Bring back data-testid in use
- Thanks VSCode for randomly showing me search results or not...
* add pdf export button to static embed and public sharing (#43743)
* add export button to embed frame, it works on static embed and public sharing
* move css around to make dashcard border show up on static/public embed pdf exports
the library we use to render an html node to an image doesn't seem to support box shadow
the original fix was done in ParametersAndCardsContainer, but that component is not used in the static/public embeds
* fix types
* fix night theme pdf export
- use padding for the title instead of starting to draw the image from 60px, to keep the bg color
- change the css to also apply the bg-color to the first child of the EmbedFrame, to make it work when exporting to pdfs, where the EmbedFrame is not part of the exported dom
* refactor: exportTabAsPdfButtonText function
* give a default to the filename, even though it should always be there
* exportTabAsPdfButtonText -> getExportTabAsPdfButtonText
* introduce .WithThemeBackground to not have a css rule too much generic
* feat: adds endpoint to download results of a dashcard part of a public dashboard (#44156)
* Flag to hide download buttons (#44206)
* pass down downloadsEnabled to hide downloads buttons in public and embedded questions or dashcards
* Update frontend/src/metabase/dashboard/components/DashCard/DashCardMenu/DashCardMenu.tsx
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* PLUGIN_DOWNLOADS_ENABLED -> PLUGIN_RESOURCE_DOWNLOADS
* adds comment to areDownloadsEnabled function to explain what it does
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* logic to handle downloads url parameter (#44363)
* minimal support for downloads url parameter
* updates StaticEmbedSetupPane test to epxect downloads=false instead of hide_download_button=true
* fix prettier mistakes
* tmp fix to the tests until we have the new analytics events
* fix eslint issues
* update embedding snippets e2e
* update embedding questions e2e
* try to clean up the mess of downloads/downloadsEnabled
* adds `hasPremiumFeature` check to the plugin, adds unit test for the logic
* adds TODOS for analytics
* unit tests for downloads flag (#45178)
* fix download pdf button not showing up when titled=false and there's only one tab
* unit tests for downloads flag on dashboards and questions
* adds test about the container id, not sure if we should keep it
* adds a comment to explain why we enable premium features in the tests
* move downloadsEnabled outside of the Boolean cast as it's already a boolean
* E2E tests for public/static downloads (#45240)
* wip e2e tests
* make downloadUtils ts friendly
* assertNotEmpty function
* adds tests for public questions
* e2e tests for static embed downloads
* publicUid -> publicUuid
* fix assertNotEmpty
* fix parameter not renamed
* Apply suggestions from code review
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* remove duplicated test
* fix another typo i forgot to rename
* fix another rename issue
* convert e2e-downloads-helpers to ts
* refactor: put method calculation in getEndpoint function
* adds note to downloadUtils
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Update e2e/test/scenarios/embedding/embedding-snippets.cy.spec.js
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* change requires to import to reflect that we changed to export the source code in downloadUtils
* makes downloadsEnabled prop more consistent across components
* colocate downloadsEnabled with hideDownloadButton in embed-display-options
* defaults displayOptions.downloads to true to simplify logic
* remove unused queryString parameter in tests setup function
* reword TODO
* Revert "defaults displayOptions.downloads to true to simplify logic"
This reverts commit aebf4cf3cf.
* fix png export when on dark theme (public/embed) not having the dark … (#45497)
* fix png export when on dark theme (public/embed) not having the dark background
* move the border-radius fix for the dashcard export to saveChartImage
* remove comments
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Do not display alert icon for models
* Add e2e repro for #37893
* Put the guard higher
* Fix unrelated test
* Go back to keeping the logic in `shouldRender`
* Remove whitespace
* Get rid of tolerance
* Use range rect and element rect instead of clientWidth/clientHeight/scrollWidth/scrollHeight
* Fix types
* Mock Range.prototype.getBoundingClientRect
* Mock getBoundingClientRect in unit tests
* Add repro for #32252
* Have a test for archiving a question and a collection
* Refactor cy.createCollection and cy.archiveCollection to TS
* Refactor collection and user helpers to TS
* Keep collection helpers together
* Sort code
* Format code
* Use new createCollection helper
* Add return type to createCollection
* Fix typing
* Fix#32252
* Fix typing
* Deprecate openNewCollectionItemFlowFor
* Revert "Fix typing"
This reverts commit ecf3179868.
* Fix TS error
* Refactor cy.createCollection and cy.archiveCollection to TS
* Refactor collection and user helpers to TS
* Keep collection helpers together
* Sort code
* Format code
* Add return type to createCollection
* Fix typing
* Deprecate openNewCollectionItemFlowFor
* Add a repro for #43219
* Fix#43219
* Create questions in parallel
* Make 2 elements scrollable so that the 'Make sure to make a selection ...' element is not weirdly cut off when scrolling horizontally to see inside of parameter-mapper-container
* Fix assertion
* Be more specific which element is to be scrolled
* Avoid realMouseWheel which does not work reliably in CI
* updates the fe to skip refetching the graph when updating the collection graph, adds tests
* Moves the skip_graph flag to be part of the body.
- the FE tooling makes it awkward for PUTs to include query params
* fixes failing unit tests
* pr feedback
---------
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* Add Temporal Units List to Dashboard Parameters Schema
Closes: #44361
This adds an entry to the Dashboard Parameter schema so that we can validate
the list of temporal_units that might be passed from the frontend when adding
or updating the `Unit of Time` type parameters.
* This is the way
the real reason the schema wasn't working... well, it was, but the un-decoded versoin of parameters was being passed
to `update-dashboard` because inside defendpoint, you can use map destructuring and bind the whole map to whatever
with :as keyword.
If you decode/validate on all of the symbols in the destructuring, that's great, but if you then pass the map bound
with `:as whatever`, you don't get that decoded goodness, just the data that our http middleware produces (that is, it
might not be the shape you expect).
So, here I've created a private Schema for the :body of the request and use the same schemas to validate each key in
that map. The result should be the same but we can pass a fully decoded/validated map along.
* change test to look for the error in the right place
* automate updating translations for major releases
* add reviewer to PR
* auto translate
* auto-translate works!
* update
* automate translation updates in CI
* fixes
* whitespace
* sort scripts
* just checkout the branch to begin with
* fix tests!
* fix import
* parameterized static questions
* sdk parameter values
* use parameters to construct parameter input
* add a unit test for passing the correct parameters
* fix parameter parsing
* move TEST_PARAM to top of file
* update readme docs
* address code review
* add skip_graph to coll perm PUT
- also no longer ignores revision and groups
so we don't print this warning:
2024-07-11 16:17:32,293 WARN api.common :: Unexpected parameters at [:put "/api/collection/graph"]: [:revision :groups]
* namespace should be a keyword
* decode the keyword
* add a typehint to avoid reflection
* Adds test for skip_graph
* handle revision 0 🤦
* Do not reset `table.columns` when some column is deleted
* Do not reset `table.columns` when some column is deleted
* Fix
* Fix
* Make API calls more obvious
* do not allow to render empty SaveQuestionModal content
when question is already saved and QB rerenders still open SaveQuestionModal there is a moment when content is empty before modal is closed
* fixup! do not allow to render empty SaveQuestionModal content
* parallelize coll perm graph group lookup w/ core.async
* respond to code review
* add + uses ExecutorCompletionService map function
* rename test
* more tests
* no concurrency for cljs
* fix timeout + add more tests
* use claypoole
* cleanup
* add kondo hooks for claypoole
* cleanup
* Can't use libraries that don't have a license
Revert "add kondo hooks for claypoole"
This reverts commit 9d93b55b28.
* ignore unused value
-- TODO: check back in on claypoole to see if the kondo config gets merged
* fix linter
This was a huge footgun.
We were accidentally limiting *ALL* queries run in our transducer to
1000 results. This included the query to fetch permissions, so when we
checked permissions we were checking a small percentage of your *actual*
permissions (e.g. 1000 / ~6300 for me on stats).
This comes in two parts.
**Rebuilt query logic**
I rewrote the query exec logic to use one `future` and one `a/go` block,
reporting results to a `promise`. This is easier to follow and doesn't leak
threads like the original implementation with two `future`s.
The single future is guaranteed to exit. It blocks on the BigQuery call.
That will either complete or throw an exception. Either the exception or
the BigQuery result is sent to the promise, and then the future is
complete. The only reference to that future is overwritten after the `promise`
resolves, so the `future`, its closure, and its thread don't persist longer than needed.
**Return a reducible result**
Previously the results were returned as a lazy sequence, but that brings
a lot of `seq` overhead, and is a risk of retaining the entire result
set in memory until the query is all done. Above all, it doesn't support
`reduced?` to explicitly drop any further results.
The `thunk` function style of `qp.reducible/reducible-rows` doesn't fit
neatly here, so I implemented the reducible logic directly on top of the
`Iterable` BigQuery results.
**Verifying**
This is a bit tricky. I've been testing it like this:
- Add a BigQuery database to my instance
- Create a saved question and note its ID (say, 123)
- Create an API key in (EE) Metabase and save it in an env var
- Install the `siege` load testing tool
- Install VisualVM and monitor my Clojure REPL process's thread count
- `siege --concurrent=15 --time=30s --header="x-api-key: $API_KEY" "http://localhost:3000/api/card/123/query POST"`
That will create a bunch of new threads while it's getting called, but
within a minute of the last queries they'll gradually drop out of the
thread pools and it will return to the baseline number of threads.
* create sdk load question
* store question result in sdk context
* remove dataset data type
* update question
* handle navigate to new card
* add loading states
* fix on query change
* update question change logic
* refactor question loading and query params
* fix ad-hoc question in dashboards
* extract useLoadQuestion into a separate hook
* cleanup run question on load
* revert changes in interactive question component
* store original question in the question result state
* turn the question into ad-hoc questions
* refactor run question methods
* add a separate isQueryRunning state for better ux
* isQueryRunning should default to false
* simulate re-running the query in unit tests
* add tests for rendering multiple valid questions
* refactor sync question method
* extract raw series and computed pivot logic
* add getCardAfterVisualizationClick
* separate single and multiple question tests
* fix import paths
* is question native
* pass queryResult instead of queryResults
* temporarily remove checkNotNull for aggregation picker to avoid sdk crash
* Dashboard Paramter Value Now Works in Preview Embed
Fixes: #44962
Prior, a common util fn used both in the embed and preview_embed endpoints was causing the preview_embed to error out
when fetching dashboard parameter values because it was checking if embedding was enabled. This check is needed for
the normal embedding endpoints, but we want the preview embed endpoints to work without needing to publish the
dashboard first.
* pass proper args
* put args back in proper order.
oops
* change form back to previous, but keep preview check
* autocorrect mistake? Or perhaps a silly typo?
* change one test to show that the preview flag is working
Didn't create a whole new test for this flag as it is provable just by adjusting this test to not bother enabling
embedding on the dashboard.
* improve the docstring
* Log changes when they make sense + async coll perm-graph revision
* - Make revision-number consistent
Decouples the creation of a revision number from the (potentially
expensive) writing of before and after for the coll perms-graph
revision.
* get id from the perms revision entity
* Move the revision id creation into the transaction
The revision id creation should only happen iff the perm updates went
through successfully.
* write a test that checks the incomplete revision
* remove sleep from tests
* Migrate Model Indexes to RTK
* Sorting out types... I think?
* remove unneeded comment
* replacing the selector with an initiate to ensure that there is data
* catching and throwing error
* Updating unit tests
This introduces a new metadata on commands, `:requires-init`. If set,
we'll run `metabase.core/init!` before running the command. `init!` will
initialize the database and run our normal startup, though it won't
actually start the Metabase server.
This allows you to use
```
env MB_CONFIG_FILE_PATH=... java -jar metabase.jar import /my/metabase/export
```
We'll run the normal init process (creating the internal user, loading
from the config file, etc), then run the command.
On my machine this adds about 2-3 seconds to the time it takes to run an
`import`.
* start adding some util fns to download-exports-test ns
* wip improving test util fns
* add subscription attachment util fn
* add tests to catch download/export differences
* viz-settings from card and dashcard merged properly
* revert these changes which probably aren't needed
* split test: downloads can be parallel, email attachments cannot
* normalize viz-settings no matter their origin
* Update test/metabase/api/downloads_exports_test.clj
Co-authored-by: Alexander Solovyov <alexander@solovyov.net>
* Incorporate some review feedback
* rendered table cards also respect column names
* Clean up table render tests and remove unhelpful render test utils
The big removal of code in the render test utils namespace is a clean up task that could've happened sooner, but this
table test adjustment is the perfect opportunity to clean up.
I had made the test utils before I truly knew how best to use `mt/with-temp`, so it had a lot of poorly implemented
'generators' that were doing a poor job of creating mock data for the rendering functions anyway.
So, reworking a few table tests here to just use a temp card is way more inline with the rest of the codebase, and
actually improves confidence that the tests are testing on well-formed data to begin with.
---------
Co-authored-by: Alexander Solovyov <alexander@solovyov.net>
* add admin-only user-local setting that controls perms tutorial
* Adding modal and banner, user setting for dismissable banner
* copy changes
* adjust so modal only appears on data permission page
* e2e adjustments, adding banner test
* never show the modal or banner on new instances
* add test
* e2e adjustments, copy update
* more PR feedback
* move modal/banner settings & tests + clarify tests
* query for v50 migration execution works on mysql
- move changelog lookup into metabase.db.liquibase
* no race conditions in changelog-table-name
The previous implementation of this function was hitting this
liquibase issue: https://github.com/liquibase/liquibase/issues/2966
* revert changelog-table-name
- simply lookup the changlog row
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* Add a test verifying that fonts are loaded
* Verify font is mentioned in HTML
* Bring back working assertions
* Add wait
* Disable speedy
* Disable CSP and emotion cache
* Revert "Disable CSP and emotion cache"
This reverts commit 573e99bfa4.
* Revert "Disable speedy"
This reverts commit 6f9962b869.
* Don't use local fonts
* Revert "Don't use local fonts"
This reverts commit 17c58aa73d.
* Force Lato
* Fix storybook file name
* ESLint
* Add more !important
* Revert !important
* Add font-family assertion
* Move test
* try fixing cross-version tests
* restore legacy metrics
* its not legacy in v43
* Dont got echarts either
* newfangled echarts
* run cross-version tests on code changes in the tests
* enable slack message
* don't always run cross-version tests
* whitespace
* add refresh auth token function
* provide a default implementation for refreshing auth tokens
* rename getRequestToken to fetchRequestToken
* add docs on customizing jwt authentication
* clarify docs, add error handling and failing tests
* update tests to pass
* re-export FetchRequestTokenFn type
* re-export EmbeddingSessionToken type
* perf: pass uuid to BE to enable metadata caching during fetching dashboard
* fixup! perf: pass uuid to BE to enable metadata caching during fetching dashboard
* Add wildcard on interceptions
* Add tests
Before
====
```
2024-07-04 15:54:39,084 INFO metabase.server :: Shutting Down Embedded Jetty Webserver
Exception in thread "Thread-9" java.util.IllegalFormatConversionException: f != java.lang.Long
2024-07-04 15:54:39,122 WARN db.liquibase :: (#object[liquibase.Liquibase 0x2b7e6614 liquibase.Liquibase@2b7e6614])
at java.base/java.util.Formatter$FormatSpecifier.failConversion(Unknown Source)
at java.base/java.util.Formatter$FormatSpecifier.printFloat(Unknown Source)
at java.base/java.util.Formatter$FormatSpecifier.print(Unknown Source)
at java.base/java.util.Formatter.format(Unknown Source)
at java.base/java.util.Formatter.format(Unknown Source)
at java.base/java.lang.String.format(Unknown Source)
at clojure.core$format.invokeStatic(core.clj:5770)
at clojure.core$format.doInvoke(core.clj:5764)
at clojure.lang.RestFn.invoke(RestFn.java:423)
at metabase.db.liquibase$wait_for_all_locks.invokeStatic(liquibase.clj:258)
at metabase.db.liquibase$wait_for_all_locks.invoke(liquibase.clj:251)
at metabase.db.setup$release_migration_locks_BANG_.invokeStatic(setup.clj:175)
at metabase.db.setup$release_migration_locks_BANG_.invoke(setup.clj:170)
at metabase.db$release_migration_locks_BANG_.invokeStatic(db.clj:91)
at metabase.db$release_migration_locks_BANG_.invoke(db.clj:86)
at metabase.core$destroy_BANG_.invokeStatic(core.clj:90)
at metabase.core$destroy_BANG_.invoke(core.clj:81)
at clojure.lang.AFn.run(AFn.java:22)
at java.base/java.lang.Thread.run(Unknown Source)
```
And now it will correctly log
```
2024-07-09 16:04:28,345 INFO db.liquibase :: Waiting for migration lock(s) to be released (max 0.4 secs)
```
Why no tests?
====
There _is_ a test that would have caught this. And in fact running at
the repl will catch it:
```clojure
liquibase=> (do (require 'clojure.test 'metabase.db.liquibase-test)
(clojure.test/run-test metabase.db.liquibase-test/wait-for-all-locks-test))
Testing metabase.db.liquibase-test
ERROR in (wait-for-all-locks-test) (Formatter.java:4515)
:h2
Will timeout if a lock is not released
expected: (= :timed-out (liquibase/wait-for-all-locks sleep-ms timeout-ms))
actual: java.util.IllegalFormatConversionException: f != clojure.lang.Ratio
```
But this _passes_ in CI:
```
❯ clj -X:dev:test :only metabase.db.liquibase-test/wait-for-all-locks-test
Finding tests took 23.5 s.
Running 1 tests
Running before-run hooks...
Ran 1 tests in 1.461 seconds
3 assertions, 0 failures, 0 errors.
{:test 1, :pass 3, :fail 0, :error 0, :type :summary, :duration 1461.175042, :single-threaded 1}
Ran 0 tests in parallel, 1 single-threaded.
Finding and running tests took 25.0 s.
All tests passed.
Running after-run hooks...
```
And that's because the log level in testing is FATAL and this is an INFO
level log.
Discussed ways to improve this like adding a null logger during CI but
we aren't ready to increase CI times for that just yet.
* Move focus to separate helper
* Use focus method to do initial focus
* Use focus method for other _editor.focus calls
* Move focus to componentDidMount
* Cancel focus timeout on unmount
* Add delay to broken test
* Switch to requestAnimationFrame
* Remove wait in test
* fix combined cards crash when some cards have invalid viz settings
* fix mock column was not metric
* ensure combined scalar column has proper type to handle the case when the card returns empty results
This PR also specifies the 'filled' variant for the question duplication
modal, since the previous default 'filled' variant for submit button was
added to make that specific button filled.
* update e2e test for command pallete to use a valid setting
* remove the setting from the ui, make it always true, unless set false from env
* remove enable-nested-queries from admin section on fe
- fix unused oldValue -> _oldValue
* tests aren't allowed to change enable-nested-queries
* fix test + disallow setting nested queries from code
* we cannot set enable-nested-queries
- set the env var instead
* add checks for the enable-nested-queries setting
* make resolve-source-cards work with enable-nested-queries setting
* fix recents uri interception
* return enable_nested_queries setting to client
* adjust fetchmock url matcher
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* Nicer error messages for linting migrations
In general, just throw exceptions instead of using clojure spec here.
You'll only get one error per run, but that seems fine.
I did keep some specs around, but run them slightly differently. Rather
than validating the whole collection of changeSets at once with `s/+`,
just `doseq` through the changeSets and validate each one separately.
That way, the value that's presented as erroneous is much smaller (a
single changeSet) and it's easier to see what went wrong.
* Update bin/lint-migrations-file/test/lint_migrations_file_test.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update bin/lint-migrations-file/src/lint_migrations_file.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Require preConditions for certain liquibase change types to encourage idempotence (#44578)
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* remove `!`s from non-side-effecty fns
* add preConditions to a few v51 migrations
---------
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Split unrelated changes into separate PR
* Fix test since user perms set reset fix split into other PR
* Fix syntax error
* Test fixes 🔧
* Test fix??? 🔧
* Test fix 🔧
* lint: add stylelint support for emotion components
* drop comment
* fix pattern order
* Use one version of typescript
* use :focus not :outline
* Use the logical CSS property "inset-inline-start:" (not "inline-start:" which doesn't exist) as replacement for the physical css property "left:"
* Use mb-color-focus for focus color
* Drop dead code
---------
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* fix: limit xrays on agg models to source fields
Fixes#43793
1. `table-like?` was checking aggregations but not breakouts. This meant
that legacy metric definitions would try to apply to the second stage
which is generally illegal and would cause things like implicit joins
to come in where they make no sense.
2. Trying to bin a `count` metric-like model would give a bin span of
0.0 which would blow up those questions.
3. Trying to self reference breakout fields, in the precense of a
question's aggregation meant that most fields would not be exposed to
the next stage and should be ignored. (similar to 1.)
* Fix tests
* Add csv-stringify package
* Split values at comma
WIP: Parse values using json
* Add label to ParameterValue and add ParameterValue to value config
* Use custom label in FieldValuesWidget
* Allow passing displayValue from outside of RemappedValue
* Allow passing displayValue in ParameterFieldWidgetValue
* Get custom value label in FormatterParameterValue
* Allow rendering custom displayValue in FieldValuesWidget
* Add custom ParameterValue types
* Fix ParameterValuePicker type
* Fix jest import for csv-stringify
* Add missing RTKQ exports for paramters
* Use direct implementation of useGetParameterValues
* Fix ValueSourceModal tests
* Add tests for value parsing and stringifying
* Use custom label in list too
* Render the custom label in a parameter value
* Setup parameter mocks in embed test
* Add info about splitting values with comma
* Add model hint when user is using labels
* Add test for cusom labels on dashboards
* Handle label-value pairs in test
* Test custom labels in parameter search input
* Use the label to parse the value for text inputs
* Fix comma's to commas
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Use strong instead of styling link
* Use getInput instead of ignoring eslint rule
* Use arrow function
* Add newlines between tests
* Use uppercase for test value
* Name value
* Use getLabel and getValue helpers to avoid indexing confusion
* Pull out confusing condition
* Move out getLabel helper
* Be more explicit about non-existent config values
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Move setupParameterValuesEndpoints to setup
* Add loading state
* Do not use alert to implement info box
* Fix formatting
* Remove undefined from getValue signature
* Use getLearnUrl
* Remove dependency on column
* Make custom field labels work in the absence of fields
* Support custom values for non-field parameters
* Add test for native question parameters
* Add newline in test
* Add test case for when show-metabase-links is false
* Just fall back to rendering value when fetching label, or in case it failed
* Add test for loading state of FormattedParameterValue
* Do not fetch custom label for FormattedParameterValue
* Add table with ip addresses
* Add test for IPAddress filter with custom labels
* Add test for custom labels on type/Quantity fields
* Remove getInput helper in test
* Remove unused hasCustomLabel value
* Split up into common and premium tests
* Make fieldValues a map for performance
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* fix: param-fields when native model is aggregated
Fixes#42829
The param-field lookup required result_metadata to match
filterable-columns in order to find parameter targets. This adds a
fallback where if the param field is missing from result_metadata,
then we look at filterable-columns.
* Unskip test
Per discussion in Slack, I made a mistake with the migration IDs as I wanted to backport to 50 but used 51 IDs.
This PR uses 50 IDs and places the migrations in the correct place.
Not sure if it's mergable in this state because Stats has run with the PR containing the incorrect migration IDs.
* Refine the reproduction for 44794
* Move skip to top-level again
* Remove superflouous click
* Verify wether or not the recents tab exists and is open
* Make the test description more descriptive
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
* Move question details to a variable
---------
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
* User Parameter Values Are Unique Per Dashboard
Fixes: #43001
Might be related: #44858
User Parameter Values previously stored just the `parameter-id` and `user-id` as data to look up the last used
parameter value. This isn't sufficient as the parameter id is not guaranteed to be unique, in particular this is true
when a dashboard is duplicated: the parameters are copied to the new dashboard without changing the parameter id at
all.
This means that we need to also store the dashboard-id in the user_parameter_value table and use that to update/get
the last used value.
The migration removes existing entries to the user_parameter_value table because I want a non-nullable constraint on
the dashboard_id column, and existing entries will not have a value. The only way to backfill those values would be to
look through every dashboard containing parameters, and for every parameter check for a matching ID. Even if you can
do this, there's no way to disambiguate if the same paramter exists on 2 dashboards anyway, so one of them will be
wrong. I think it's not worth the trouble considering that removing the entries in this table doesn't break anything;
it just becomes a mild inconvenience that a user has to select a filter value again (since the dashboard will use the
default value).
* alter test to check for uniqueness across dashboard
This test makes 2 dashboards with parameters of the same ID and asserts that changing the value on dashboard-a does
not change the value on dashboard-b.
* adjust migration to pass linter rules
* remove the extra rollback on migration
* Update src/metabase/models/user_parameter_value.clj
Co-authored-by: bryan <bryan.maass@gmail.com>
* Update src/metabase/models/user_parameter_value.clj
Co-authored-by: bryan <bryan.maass@gmail.com>
* adjust parameters in test so we don't get logged warnings
* put update!/insert! in a transaction to avoid any race conditions
---------
Co-authored-by: bryan <bryan.maass@gmail.com>
* Support compiling SQL with inlined parameters directly
* Test fix
* Make `format-honeysql` a multimethod
* Add test for #32543
* Test also fixes#44915
* Add explicit test for #44915
* Better test for #44915
* More test fixes 🔧
* Fix Kondo error
* Optimize: compute chart title href only on mouseenter and focus
* Improve comment
* Improve comment
* Format code
* Update existing tests
* Extract helpers
* Update unit tests
* Make getHref argument-less
`Lib.diagnoseExpression` is stricter about this than the QP or FE
expression parser, making it reject expressions that tried to compare
datetimes to strings like `2024-07-02 14:34:17Z`.
Those used to work in 49.x, but a new `Lib.diagnoseExpression` call
while editing expressions caused it to regress.
Fixes#44916.
**MetadataProvider cache**
The reusable MetadataProviders is a generally useful thing - within some
dynamic scope like an API request, a `clojure.core.cache` instance can
be provided as `lib.metadata.jvm/*metadata-provider-cache*`, and that
will be used to cache the provider for each `database-id`.
Reusing those providers can significantly reduce appdb traffic,
re-fetching the same metadata in different `MetadataProvider`
instances.
**Dashboard loads**
The FE can supply `?dashboard_load_id=uuid_string` on requests for a
dashboard load:
- `/api/dashboard/:id`
- `/api/dashboard/:id/query_metadata`
- `/api/dashboard/:id/dashcard/:dashcard_id/card/:card_id/query`
That provides a good cache key across that burst of N+2 API requests
that are part of a single load. Currently it is used for two things:
- Caching `get-dashboard` (used by the first two requests)
- Selecting the `clojure.core.cache` for `*metadata-provider-cache*`
Both use a TTL of 10 seconds, which is generally enough for the first
screenful of a dashboard load, if not the whole thing. (Browsers will
only issue a handful of requests at once, so a few long-running queries
might delay later parts of the dashboard from loading until after the
TTL expires.)
* Make sure the query is native
* Convert the comment to JSDoc
* Add E2E reproduction
* Fix mapping
* Add test scenarios for the public sharing and embedding
* Simplify API payload
* Extract and significantly expand the reproduction scope
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* only send permissions updates for groups that have been modified in the graph
* Support partial permission graph updates on the BE (#44797)
* handles keeping track of advanced permissions changes to send relevant group data when updating the graph, merges response
* tests and optimizations
* respect group-ids set when fetching connection impersonations, sandboxes, and admin group perms
* fix typo
* add e2e test
* fix BE test
* impls pr feedback, reduces code needed since advanced permissions state is always reset on save
* last fixes
---------
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* recent views sometimes don't show special colls
colls with a non-null namespace are special: they can't interact with
normal collections, dashboards, cards, etc at all. (Currently the only
use for these we have is snippets.) At some point we could add a
`namespace` argument to this endpoint to support looking at recent
collections in a different namespace, but that'd be a bigger refactor
without a whole lot to gain. For now we can just exclude these
completely, which is what we want for current usages of the recents
endpoint anyway.
We won't exclude non-null types across the board, because sometimes
we'll want to see these. But we will be particular about when to display
them.
We'll never display the `trash` collection, but will display the
instance analytics collection when we're not trying to make selections.
* Hack an e2e test around an existing bug
So, the entity picker that pops up when you're selecting a collection to
put a Snippet into shows you "Recent" collections - which previously
included snippet collections.
We should either make the Recents API support a namespace parameter for
collections so it could actually support this case, or the entity picker
shouldn't show a recents tab here (realistically, how many collections
are people going to have for snippets that they would need this tab
anyway).
* Calculate metadata without running queries [JDBC]
* Appease Kondo
* Test fixes 🔧
* Overhauled Card metadata calculation code
* Test fixes 🔧
* Kondo fix 🔧
* Ok maybe I fixed stuff
* Work around https://github.com/aws/amazon-redshift-jdbc-driver/issues/118
* Maybe fixx the Cypress tests
* Experimental possible e2e test fix 🔧
* Test fixes 🔧
* Do things a little differently.
* Try re-enabling the field refs
* Add reproduction for #38989
* Use saveQuestion instead of clicking trough the UI
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Use openOrdersTable instead of manually clicking through the ui
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Create question programmatically in test
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Wrap non-latin characters in a span specifying working font
Fixes: #38753
CSSBox seems to have a bug where font fallback doesn't work properly. This is noticeable when a font does not contain
glyphs that are present in the string being rendered. For example, Lato does not have many international characters,
so the rendered version of tables (that show up in Slack messages) will not render properly, making the card
unreadable.
Since this appears to be a downstream bug, I've opted to work around this limitation by wrapping any non-latin
characters in a <span> that specifies the font family to be sans-serif, which should contain the glyphs to properly
render.
This leaves Lato in place for other characters.
For now, I figured it's worth trying this solution over using Noto for 2 reasons:
- we can keep Lato, which has been the decided font style for the app for some time (this keeps things consistent
where possible)
- the Noto font containing all glyphs is quite a large font (>20mb, I think) and it would be nice to avoid bundling
that if we can.
* stub installed fonts test
* typo
* Do wrapping, but now per-string, and in Clojure data not html string
I've decided that a reasonable solution is to still wrap strings containing non-lato characters. But it's not done
with str/replace to the html string but rather in a postwalk over the Hiccup data prior to rendering.
Seems like a decent compromise of issues without patching CSSBox or fixing upstream (may be good to do, but will take
longer to get a fix in).
* add test checking that glyphs render correctly
* Add a test that directly checks the wrapping fn
* Change the string to keep the linter quiet
* Change how we check if string can be rendered to faster method, new Lato Font
With Sanya's help, the way I check if a given string is renderable with Lato is now faster.
Also use the full Lato font, not just the 'latin' lato so we can cover more chars
* change lato so that it loads the fn even in a fresh test run
* prevent pin maps from filtering out rows with null values
- since pin maps are just displaying where the data is by latitude and longitude, we do not need to filter the pin just because one of the rows is null.
- the metric column selection does not make sense for pins as they just represent that data through coordinates (no aggregations or values displayed), so we should not be filtering out any data just because it is null
* add unit test
I was thinking about the potential options here.
On one hand, we could initialize the app db ourselves. This is a little
problematic, because:
- our `init!` function is in `metabase.core` and calling it yields a
painful cyclical dependency. It's also pretty tied to outputting things
to the user that they might not expect during an import, e.g. "Looks
like this is a new installation ... preparing setup wizard", and starts
up jobs, etc.
- if we choose to *only* initialize the database ourselves (only the
bits that we actually *need* initialized), I'm a little nervous about
maybe missing something. Right now we need to create the internal user
and create users from the config file before running the import. If we
added a third thing, would we remember to update this?
- we could refactor to put all of the above in one spot, but this is a
P1 and it feels important to close this out as quickly as possible. We
also don't claim to support initializing the app DB from an import in
the first place.
So: just throw an exception if the app db hasn't been set up yet. Our
heuristic here is whether a single non-internal user has been created.
So if you want to use your config, you can run:
```
MB_CONFIG_FILE_PATH=dev/config.yml java -jar metabase.jar
java -jar metabase.jar import /my/exported/dump
```
If you try to run `java -jar metabase.jar import ...` on a fresh
DB, it'll now fail with the error message:
```
You cannot `import` into an empty database. Please set up Metabase
normally, then retry.
```
* highlights the trash link in the sidebar when viewing root trash collection or a trashed item
* impls pr feedback
* impls more pr feedback
* fixes unit tests and logical mistake that allowed urls with invalid collection ids to match on undefined trash id due to it not being loaded yet
* stop using strings for enum values (prefer kw)
* only do SLO when saml-slo-enabled is true
* make defsetting docstring formatting consistent
* log-direction is a string
* add test for enabling / disabling saml slo
- also return a 403 when the client tries, but it's disabled
* let's see how many things break
* add missing useSyncURLSlug
* remove cardTitled prop that's not needed
* remove DashboardControls
* remove @ts-expect-error for dashboardId
* move props destructuring at the top
* remove use-dashboard-nav.ts as it's not needed anymore
* remove import of use-dashboard-nav from dashboard/hooks/index.ts
* remove unused props from dashboard
* Add dashboard question header
* Add dashboard data preserve after question drill
* Review fixes
* Review fixes
* Potential fix for e2e
* Potential fix for e2e
* Add basic unit tests
* Fix Dashboard reloading on unmount, fix ad hoc question title in SDK
* Move dashboard reset to SDK component
* Fix issue with dashboard rerender
* Fix one more reported issue with rerender
* fix map default display type with un-binned coordinates
- default to `"map.type"` = "pin" when coordinates are not binned
- default to `"map.type"` = "grid" when coordinates are binned
* add tests
* refactor breakouts usage
* Fix broken field normalization in metadata
* Handle Fields.actionTypes.UPDATE action in tables reducer to keep table's original_fields in sync
* Add an e2e test for fields
* Format code
* Use a different table, since accounts table is not available in e2e tests
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* remove invalid else conditional
* fix RowChart firing invalid API request on legend item click
* fix clicking on non-breakout legend on dashboards
* add E2E test to validate and enforce the desired behavior
* Bump dashboard description max len limit to 1.5K
* Extract `DASHBOARD_DESCRIPTION_MAX_LENGTH` const
* Handle description max length in dashboard sidebar
* Add red border to dashboard description box
* Add test
* fixes spacing and color issue of trash icon in archive banner
* css var for color
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* WIP PublicOrCustomEmbedDashboardPage without using dashboard controls
* add usePublicDashboardEndpoints instead of WithPublicDashboardEndpoints
* fix type error by setting null as default value for hide_parameters and hide_download_button
* use usePublicDashboardEndpoints inside WithPublicDashboardEndpoints
* also migrate embed
* i don't think we need useDashboardNav public/static embed
* SetTitle component to use instead of the `title` hoc
* wrap setEmbedDashboardEndpoints/setPublicDashboardEndpoints in useEffect as per PR comment
* after the fix for metabase#38640 we don't need the public routes in the main app
* renamed test file
* change tests to use PublicOrEmbeddedDashboardPage
* remove PublicOrEmbeddedDashboardControlled and change imports for PublicOrEmbeddedDashboardPage
* remove debug prop left by mistake
* Move SimpleSpecificDatePicker to the form
* Extract `isSpecificValue` helper
* Integrate the toggle into `SimpleDateIntervalPicker`
* Make DateOperatorPicker do only one thing
Separation of concerns - lift SimpleRelativeDatePicker up to the form.
* Fix unit tests
* Add back the test for the `SimpleSpecificDatePicker`
* Keep the rendering logic in the `SimpleDatePicker`
* apply css variables injection right under the public component
* refactor css variables
* fix name changes
* apply css variables to global styles
* update path
* render component wrapper as full-width
* move style resets to public component styles wrapper
* update unit tests for global styles
* Add Temporal Units List to Dashboard Parameters Schema
Closes: #44361
This adds an entry to the Dashboard Parameter schema so that we can validate
the list of temporal_units that might be passed from the frontend when adding
or updating the `Unit of Time` type parameters.
* Sanitize the SVG before parsing it with Batik
WIP
Fixes: #43677
There are characters that are invalid according to the Batik XML Parser. This sanitizes the svg string to strip out
some of those characters so that the render can continue.
* Use the xml 1.0 spec allowed unicode chars list for the regex
* Remove accidentally merged code
* still miseed a line
* fix: dont always optimize between expressions
Fixes#42291
The frontend produces expressions like
```
[:between
[+ [:field ... {:temporal-bucket :day] [:interval 2 :day]]
[:relative-datetime -1 :week]
[:relative-datetime 0 :week]]
```
This should not be optimized because of the mixed `:day` and `:week`
units. However, it was being optimized since the compatible units
weren't being properly picked up by the match.
* Disable suspcicios args eastwood lint, kondo does it better
* Stop autobucket to day when adding a time interval
* Fix the operator list for `type/TextLike` PKs
* Fix the bug with using wrong operators
* Fix the bug with using wrong operators
* Update test/metabase/lib/filter_test.cljc
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* Test FKs too
* Update test/metabase/lib/filter_test.cljc
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
---------
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
* Get rid of the styled component
* Stack date operator on top of actual filter pickers
* Use the existing `CurrentDatePicker`
* Fix unit test
* Conditionally apply padding to the `CurrentDatePicker`
* Remove the custom prop
This PR removes `:role` removal from spec, even if it was added in `:connection-uri`.
It seems that `:connection-uri` is ignored on `do-with-connection-options` code path.
Hence use of non-default roles is again available on Snowflake.
This PR disables the 'make exports look pivoted' post processing step for csv and xlsx exports.
This is in response to a reported issue that Large Pivot Table downloads are failing entirely: #44556
A more hollistic fix and improvement to the post processing feature will be worked on, so the post processing code is
left untouched in this PR.
The dynamic vars in the csv and xlsx namespaces each default `false` and disable the export from running any post
processing. The var should remain `false` until we've got a plan for these types of exports, and there is no way for
users (either through ENV or UI) to change this, so the feature is effectively disabled.
* convert embed.ts to use createSlice from rtk
* only store in the state what we care about for interactive embedding
* use DEFAULT_EMBED_OPTIONS keys to filter searchOptions
* removed unused hash parameter
* adds basic unit test
There were two problems here:
- wrong args to `mdb/migrate!` (passing the db-type as well as the data
source), and
- in MySQL, the changelog table name is not `:databasechangelog`. Use
`metabase.db.liquibase/changelog-table-name` to get the actual table name.
Add a new query parameter to `/api/collection/:id/items` and
`/api/collection/root/items`. If `official_collections_first` is passed,
we'll sort official collections first, or not, as requested. If it isn't
passed, we'll default to `false` for the trash collection, or `true` for
normal collections.
* simplify bordered logic
* 🧹 setTitled
* 🧹 setHideDownloadButton
* 🧹 setHideParameters
* inline useEmbedDisplayOptions
* move DEFAULT_EMBED_DISPLAY_OPTIONS to costants file so we can remove use-embed-display-options.ts
* move consts up in the file
* update deps of useEffect
* fix ts error in the sdk
* use useEmbedFrameOptions in useDashboardUrlParams
* embed frame should not touch the `embed` slice, which is used for interactive embedding
* funnel start color should be text-dark
* change pivot table font size unit to em for scaling
* change funnel and legend item unit to em
* change funnel font size to em
* change object detail font size to em
* change gauge font size to em
* add cartesian chart padding config
* update docs
* remove isEmbeddingSdk prop
* simulate the 0.875em base font size
* chart padding override should apply in query builder
* add cartesian padding to stories
* update readme docs
* fix collection browser option in readme docs
* fix pivot table font size scaling
* update loki
* update loki
* use em for smart scalar
* update chart default padding
* revert smart scalar font size scaling due to measurement
* dynamic font family and font size as defaults
* pivot table
* add more properties to sdk docs
* make FONT_SIZES constant local
* link to discussion on query builder extra spacing
* enforce that font is passed to getLeftHeaderWidths
* Update e2e test
* Update e2e test
* fix: populate param-fields for named fields on public dashboards
* Add name to public hydration
* Add tests
* Add tests
* Fix tests
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Ensures that minimum query durations are correctly displayed in the form post-save (#44249) and that all times can be used in the Schedule component (#44257)
* UserParameterValue transforms wrapped to properly escape string vals
The `json-in` and `json-out` functions used for toucan model transforms do not perform any string escaping logic. This
makes sense as we don't want to make assumptions about the shape of data flowing in/out of the db. But, this did mean
that for the User paramter values table, string values were causing an error to be logged. This doesn't break
anything, as the function will still correctly return the string, but it can clutter up logs.
So, this PR wraps all incoming values in a map and unwraps it for outgoing values.
* Add migration to wrap existing user param values with ::wrapper key
Since the in/out transform for UserParameterValues is updated, we need to migrate any existing values to have this wrapping.
* Simplify the solution to the problem.
Since we're already getting what we need from the json-in/json-out *except* that it's logging a parse error, we can
create a json-out that works the same way (tries to parse and returns the string as-is if it fails) without logging an
error at all.
* take away the arg and keywordize inside the json-out fn
* remove irrelevant comment
Signed-off-by: Adam James <adam.vermeer2@gmail.com>
* Add a little more test coverage for different value types
* faster test thanks to Dan!
* Add a comment about the data being tested
---------
Signed-off-by: Adam James <adam.vermeer2@gmail.com>
* XLSX Pivot Table Downloads have 'Native' Pivot Table
Addresses: #2473
The xlsx downloads work fine for regular tables. However, on Pivot Table downloads, the resulting file isn't pivoted
and is formatted somewhat strangely (it contains additional rows related to totals, has an extra 'pivot-grouping'
column).
Now, the downloaded xlsx file contains 2 sheets:
- the data sheet containing the question's rows unpivoted. If you were constructing a pivot table yourself, you would
probably start with this shape of data.
- the pivot sheet containing a 'native' Pivot table whose reference points to the data on the data sheet, and whose
columns, rows, and values align with the cols, rows, and measures set up in Metabase
* Silly typo!
* Exported Pivot uses correct aggregation (sum, count, stddev, avg, min, max)
* data for pivot exports now also uses the formatting
* Add a test asserting that a pivot table exists in the xlsx file
* add-row! is a method dispatching on sheet class
* remove unnecessary hint
* address review feedback:
- consolidate the aggregation function key fns into one function in the .xlsx namespace
this was moved out of postprocessing so that the xlsx specific fns don't litter the post processing namespace at
this time
- clean up an unnecessary destructiring inside the native-pivot function
- don't shadow 'name' inside the body of a fn
* Add tests that confirm zero-col and zero-row situations work
* Consolidate the atoms used to store pivot rows/options
* Expose "Include this {period}" in timeseries chrome
* Fix type errors
* Simplify type guards
* Add unit tests
* Remove Flex wrapper around Switch
* Extract `IncludeCurrentSwitch` component
* Add E2E smoke tests
* Cover the toggle behavior with E2E tests
* Simplify test with a simple helper
* Temp
* Add support for saved questions
* Try to fix question not found issue
* Try to fix question not found issue
* Fix unit tests
* Add basic unit test
* Fix unit tests
* Fix unit tests, fix question not found issue
* Review fixes
* Review fixes
* Refactor components structure, review fixes
* Adjust readme
Fixes#43993 for real.
The earlier fix#44182 was needlessly restricting the
"nominal refs" check to fields which were using numeric IDs,
when really any match on nominal refs is valid.
* automatically add milestones to prs and issues
* test code
* test code
* test commit (#3138)
* test commit (#43320)
* test commit (#44357)
* remove test code
* add newlines
* prefer older version milestones
* Upgrade @reduxjs/toolkit from ^1.9.3 to ^2.2.5
* Fix typing
* Upgrade redux-actions and @types/redux-actions
* Fix type errors
* Fix type errors
* Upgrade react-redux from ^8.0.5 to ^9.1.2
- remove @types/react-redux as it's no-longer-needed
* Upgrade redux-promise from ^0.5.0 to ^0.6.0
- upgrade @types/redux-promise
* Remove unused @types/redux-logger
* Revert "Upgrade redux-actions and @types/redux-actions"
This reverts commit 65005a0053.
* Remove createWeakSelector as weakMapMemoize is the default memoization strategy now
* Do not call unsubscribe if it does not exist
* Do not subscribe
* Remove console.log
* Make DelayedLoadingSpinner always render something with data-testid="spinner" so that tests can be aware of when things are loading
* Fix JoinStep unit tests
* Fix AddToDashSelectDashModal unit tests
* Fix SegmentApp unit tests
* Bring back original subscription logic
* Fix DataUsageStep unit tests
* Fix ManageApiKeys unit tests
* Simplify comment
* Fix MainNavbar unit tests
* Fix HomeRecentSection unit tests
* Use afterEach instead of afterAll to reset mocks
* Remove explanatory variable
* Pivot Tables with no Pivot Columns should still download
Fixes#44159
Pivot Tables can still be valid without columns, and therefore the download should respect such a query and be
successfully exported.
In this PR, the postprocessing of pivot results into 'visual pivot' exports for csv takes into account the zero-column
configuration and the download no longer fails.
* add a test to confirm that zero cols and >1 measures works as well
* Handle the case where there are zero pivot-rows.
This is a similar problem to the zero pivot-cols case, just with rows.
The app actually doesn't render these tables correctly, but that's a separate frontend bug
* Convert drill-through (CLJ) values to JS
* Convert the drill-through `:null` to an actual `null`
Resolves#44232
* Use new function in `filter-drill-details`
* display-info->js should check for nil before seqable?
* fix js object comparison
* Add test for the `drill-value->js` function
* Simplify the test
* Fix the JS object equality in the test
---------
Co-authored-by: Alexander Solovyov <alexander@solovyov.net>
* Change run to make-run in process-query-for-card
Fixes#44160
There were two levers where a card could change how it was run.
`qp` this is basically `qp/process-query` with some optional userland
flags and constraints marked on the passed in query.
In the case of pivot tables in the
`/pivot/:card-id/query` endpoints it was `qp.pivot/process-query`
'run' this defaults to a streaming response that takes the `qp` param
and `export-format` and passes a query to qp. Most often this is
overriden to not stream the response.
Unfortunately, if `run` is overwritten, then `qp` is thrown away, and
the caller who overrides `run` must determine the correct `qp` to use.
However, when running `process-query-for-dashcard` it is difficult to properly set
`qp` for all of the dashcards without doing card lookups. So [this commit](https://github.com/metabase/metabase/blob/release-x.50.x/src/metabase/query_processor/card.clj#L230-L232) did a check to swap the runner in the presence of a pivot query that calls the **default** runner with a `qp.pivot/run-pivot-query`. The problem is that pulse needs to use a non-default runner, but there's no way for `process-query-for-card` to do both.
This commit changes `run` to `make-run` which raises the abstraction to
a function that produces a runer given a `qp`. This allows
`process-query-for-card` to change the `qp` as needed while keeping the
passed in runner.
There's really three things going on that could be teased apart
further/better.
1. `qp` - this conflates the processor (`qp/process-query` `qp.pivot/run-pivot-query`)
and modifying query (`qp/userland-query`,
`qp/userland-query-with-default-constraints` `(update query :info merge info)`)
2. `make-run` - How the above are processed: (streaming/realized)
* Fix tests
* Elevate include this period in the relative date picker (#44179)
* Mock new row with "include {period}" string
* Implement "include this" using a toggle
* Elevate "starting from" option in the relative date picker (#44242)
* Elevate "Starting from" option in the filter picker
* Tweak the date offset close icon
* Fix tests
* fix charts crash in safari when dataset timezone is not in iana format
* Update Loki Snapshots
* better fix: apply offset instead of finding a timezone with such offset
* remove unused specs
* Update Loki Snapshots
* review
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
* Fix a potential bug in `hydrate-can-restore`
I had a bug here. I wanted to hydrate `:collection` on the items and
then return the original items, but instead I was `dissoc`ing
`:collection` from the items. If the original items passed in already
had `:collection` keys this would break them.
AFAIK this didn't cause any issues, but it could have - so fixing it here.
* Add a test
* Present collection items in the collection
When fetching collection items, we need to fetch the true
`collection_id` in order to figure out things like `can_restore` and
`can_delete`, but set it to the displayed/effective `collection_id` when
we actually send it out.
This fixes a bug where collections had their `collection_id` match their
`id` when returned by this endpoint.
Previously we were just adding this when the item was archived. Instead,
always send a value - if the item is not archived, it can't be restored,
so it will be `false` in this case.
Making this change will make fixing a frontend bug easier, because the
frontend will be able to just trust the `can_restore` value coming in
from the backend.
* Use MultiAutocomplete in dashboard filters
* Handle prefix on MultiAutocomplete
* Remove unnecessary tests for empty values
* Remove input className
* Handle numbers in MultiAutocomplete
* Accept number values in MultiAutocomplete
* Use MultiAutocomplete in NumberInputWidget
* Filter out empty items
* Do not expect formatted number in values
* Do not allow adding multiple values when multi is false
* Handle undefined values
* Make value optional
* Fix MultiAutocomplete types
* Make MultiAutoComplete generic in its value type
* Refine MultiAutocomplete type in NumberInputWidget
* Refine MultiAutocomplete type in FieldValuesWidget
* Fix typo
* Fix MultiAutocomplete tests
* Add helpers to work with MultiSelect and MultiAutocomplete inputs
* Handle MultiAutocomplete input in tests
* Fix filter helper
* Use multiAutocompleteInput helper
* Find the value remove button in test
* Fix tests for NumberInputWidget
* Fix tests for DefaultPicker
* Fix type in NumberInput
* Fix dasboard test
* Fix embedding test
* Add helpers to remove values
* Fix chained filter test
* Fix nested test
* Fix embedding tests
* Fix filter tests
* Use removeMultiAutocompleteValue
* Use removeMultiAutocompleteValue
* Remove unused variable
* Use multiAutocompleteInput helper
* Handle comma's in values
* Handle multiselect
* Fix filter test
* Add filter
* Format field value
* Fix embedding tests
* Allow rendering custom label for item
* Use renderValue in FieldValuesWidget
* Fix data prop type
* Be more explicit about broken types in FieldValuesWidget
* Fix default behaviour for label
* Fix embedding tests
* Force click
* Fix sharing test
* Use removeMultiAutocompleteValue
* Handle multiautocomplete inputs
* Force click filter
* Close the suggestions menu
* Fix email test
* Fix sharing filter
* Set viewport in test
* Fix sharing test
* Remove debugging logs
* Use removeMultiAutocompleteValue
* Close the suggestions menu using blur
* Remove commented out code
* Remove the unused color prop
* Move shared types to types file
* Rename ValueType to TValue
* Add missing forwardRef
* Add tests for filter and renderValue
* Use TokenField for single value parameters
* Fix single value test
* Be more specific about when to select a single input
* Fix single value e2e test
* Handle undefined parameter
* Accept all RowValue types in MultiSelect
* Accept all RowValue types in ListField
* Accept all RowValue types in SingleSelectListField
* Be more precise about the FieldValuesWidget value types
* Be more specific about when to render normal input
* Remove duplicate import
* Remove maxSelectedValues from MultiAutocomplete
* Parse values in FieldValuesWidget
* Remove parsing complexity from MultiAutocomplete
* Be more strict when parsing numbers
* Do not use multiselect when arity is 1
* Remove spurious changes
* Pass prefix from FieldValuesWidget
* Remove number MultiAutocomplete story
* Remove renderValue from MultiAutocomplete
* Add custom component to MultiAutocomplete
* Add missing data prop
* Remove spurious changes in MultiAutocomplete
* Remove SelectItem reference
* Use renderValue in FieldValuesWidget
This PR removes:
- a warning about missing React keys in the `Schedule` component
- a warning about invalid props in the `StrategyFormLauncher` component
* adds a shimmer effect to the skeletons
* adds a `natural` prop that makes the skeletons have a random width between 50% and 80%
* adds a skeleton to Browse models
Closes#44111
* expose card title settings
* Add dashboard test skeleton
* add a sample dashcard to the test
* dashboard grid should have card title by default
* add unit test for card title
* remove unused delay in unit test
* fix width and height being smaller than min values
* flip grid layout condition
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Handle errors when inferring separator of csv
This file
```
"c1","c2"
"a,b,c","d"
```
was failing to upload for a silly reason: when we were checking for
which separator it used, we had a parse error when using a semi-colon
and that blew up the whole pipeline.
* Use cal's helpful test suggestions
Although a user may have the necessary permissions to delete an item, if we're not going to allow deleting unarchived items, can_delete should be false for them too.
* [Metrics V2] Metrics with filters on implicitly joined columns produce query error when added in a Summarize block
* Handle multiple metrics with same implicit join
* Address PR feedback
* adds the endpoint and a test
* add recents endpoint, todo: tests
* add post recents endpoint
* return recent views for both lists, but make the endpoint work
* Make recent-views context aware
- pruning is context aware, only checks for the recently-inserted
context
- Adds endpoints:
- POST to create selection recents,
- GET activity/recents
- requres context query param to be one of:
- all, views, selections
- Adds context arg to update-users-recent-views!
- Cleans up arg schema for update-users-recent-views
* impl GET api/activity/recents
- return recent-selections and recent-views from
- send context == collection from pinned card reads
* update callsites of recent-views/update-users-recent-views!
- to use :view param where necessary
* fixes models/recent-view tests
* adds more activity/recent-view tests
* wip
- fix whitespace linter
* Fix command palette e2e test
- reuse util snake-keys
* updates fe to use new recents endpoints
* fixes fe type issue
* snake-keys -> deep-snake-keys
- I've been betrayed by lsp rename
* snake-keys -> deep-snake-keys
- I've been betrayed by lsp rename
* log selection events for created collections
* mysql doesn't allow default values for TEXT types
* log a recent view on data-picker selection
* decouple view-log context from card-event view context
* fix a doc typo
* stop double logging recent-views on POST
* maybe fixes some tests
* some e2e fixes
* fix mysterious divide by zero during score search
* fix divide by zero possibilities everywhere in score-items
metabase.api.activity/score-items used to throw when there weren't any
items being scored (even though there's a `(when (seq items) ...)` check)
* more test fixes
* fix more e2e tests, + rename endpoint in tests
* fix oopsie
* fixes a few more tests
* address review comments/questions
* allow for a comma delimited list in qps like ?context=views,selections
returns all recent view items with those contexts, most recent first
under the `:recents` key.
* refactors FE around new endpoint
* fixes for unit tests
* use ms/QueryVectorOf for context
* fix models/recent_views tests
* use multiple query params instead of comma delimited value on FE
* ignore timestamp when deduping recents
* review comments + test fixing
* update docstring
* fix api/activity_test s
* actually dedupe
* add test for deduping by context
* e2e fix: shows up-to-date list of recently viewed items after another page is visited
* e2e fix: should undo the question replace action
* e2e fix: should replace a dashboard card question (metabase#36984)
* e2e fix: should preselect the most recently visited dashboard
* fix 6 more e2e tests
* fixes fe type check and unit failure
* renames unit test mocking function
* fix a flaky e2e test
* widen Item to accept str or kw where sensible
- allow strings or keywords for moderated_status, and authority_level
* simplify impl + add test
* add view-log to events schema
* add collection context to view log
* fix the final 2 failing e2e tests
* click dashboard tab when the user has can-read? on recent entities
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* Improve the Trash
Ok, so I had a realization at the PERFECT time, immediately after the RC
cutoff. Great job, brain!
Here's the realization. For the Trash, we need to keep track of two
things:
- where the item actually is located in the hierarchy, and
- what collection we should look at to see what permissions apply to the
item.
For example, a Card might be in the Trash, but we need to look at
Collection 1234 to see that a user has permission to Write that card.
My implementation of this was to add a column,
`trashed_from_collection_id`, so that we could move a Card or a
Dashboard to a new `collection_id`, but keep track of the permissions we
actually needed to check.
So:
- `collection_id` was where the item was located in the collection
hierarchy, and
- `trashed_from_collection_id` was where we needed to look to check
permissions.
Today I had the realization that it's much, much more important to get
PERMISSIONS right than to get collection hierarchy right. Like if we
mess up and show something as in the Trash when it's not in the Trash,
or show something in the wrong Collection - that's not great, sure. But
if we mess up and show a Card when we shouldn't, or show a Dashboard
when we shouldn't, that's Super Duper Bad.
So the problem with my initial implementation was that we needed to
change everywhere that checked permissions, to make sure they checked
BOTH `trashed_from_collection_id` and `collection_id` as appropriate.
So... there's a much better solution. Instead of adding a column to
represent the *permissions* that we should apply to the dashboard or
card, add a column to represent the *location in the hierarchy* that
should apply to the dashboard or the card.
We can simplify further: the *only time* we want to display something in
a different place in the hierarchy than usual is when it was put
directly into the trash. If you trash a dashboard as a part of a
collection, then we should display it in that collection just like
normal.
So, we can do the following:
- add a `trashed_directly` column to Cards and Dashboards, representing
whether they should be displayed in the Trash instead of their actual
parent collection
- use the `collection_id` column of Cards and Dashboards without
modification to represent permissions.
There's one main downside of this approach. If you trash a dashboard,
and then delete the collection that the dashboard was originally in,
what do we do with `dashboard.collection_id`?
- we have to change it, because it's a foreign key
- we can't set it to null, because that represents the root collection
In this initial implementation, I've just cascaded the delete: if you
delete a dashboard and then delete a collection, the dashboard will be
deleted. This is not ideal. I'm not totally sure what we should do in
this situation.
* Rip out all the `trashed_from_collection_id`
* Migration to delete trashed_from_collection_id
* fixes
* don't move collections
And don't allow deleting collections
* only show cards/dashboards with write perms
* Show the correct archived/unarchived branch
* some cleanup
* add a todo for tomorrow
* Fix for yesterday's TODO
* more wip
* refactor
* memoize collection info
* move around memoization a bit
* fix schema migration test
* oops, delete server.middleware.memo
* Use a migration script (postgres only for now)
* Fix some tests
* remove n+1 queries in `collection-can-restore`
* fix test
* fix more tests, and x-db migration script
* fix h2 rollback
* fix mysql/mariadb migration
* lint
* fix some mariadb/mysql tests
* fix h2 rollback
* Fix mysql rollback
* Fix Postgres migration
* "Real" `trash_operation_id` UUIDs from migration
* fix mariadb migration
* Separate MySQL/MariaDB migrations
* trashed directly bit->boolean
* Remove `trashed_from_*` from migrations
* Rename `api/updates-with-trashed-directly`
Previously named `move-on-archive-or-unarchive`, which was no longer
accurate since we're not moving anything!
* Add `can_delete`
* Delete test of deleted code
* Can't move anything to the Trash
The Trash exists as a real collection so that we can list items in and
such without needing special cases. But we don't want anything to
_actually_ get moved to it.
* integrates can_delete flag on FE
* Update src/metabase/models/collection.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Update src/metabase/search/impl.clj
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Better name for `fix-collection-id`
Also consolidated it into a single function. It takes the
trash-collection-id as an argument to prevent a circular dependency.
* s/trashed/archived/ on the backend
In the product, we want to move to "trashed" as the descriptor for
things that are currently-known-as-archived.
It'd be nice to switch to that on the backend as well, but that'll
require quite a lot of churn on the frontend as we'd need to change the
API (or have an awkward translation layer, where we call it `trashed` on
the backend, turn it into `archived` when sending data out via the API,
and then the FE turns around and called it "trashed" again).
For now, let's just be consistent on the backend and call it `archived`
everywhere. So:
- `archived_directly` instead of `trashed_directly`, and
- `archive_operation_id` instead of `trash_operation_id`
* Fix up a couple docstrings
* Rename `visible-collection-ids` args
for `collection/permissions-set->visible-collection-ids`:
- `:include-archived` => `:include-archived-items`
- `:include-trash?` => `:include-trash-collection`
* select affected-collection-ids in the tx
* Stop dealing with `:all` visible collection ids
This used to be a possible return value for
`permissions-set->visible-collection-ids` indicating that the user could
view all collections. Now that function always returns a set of
collection IDs (possibly including "root" for the root collection) so we
don't need to deal with that anymore.
* Don't use separate hydration keys for collections
Dispatch on the model of each item (but still keep them in batches for
efficiency.)
* vars for collectable/archived-directly models
* Round up loose `trashed_directly`s
* Fix a test
* Use `clojure.core.memoize` instead of handrolled
It's slightly different (TTL instead of exactly per-request) but that
should be fine.
* FE lint fixes
* 😭 e2e test fix
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* add option to customize dashcard border
* use the shadow CSS variable
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* make the shadow and border decoration call more explicit
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* pass in theme as argument
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Allow picking arity of number parameters
* Limit to input to have arity 1 when multi is false
* Use MultiAutocomplete in NumberInputWidget when arity is 1
* Add test cases for single value numbers
* Only change arity when it is n
* Fix unit tests for NumberInputWidget
* Move arity check for numbers to getNumberParameterArity
* Fix the typo
* Replace `realHover` with the `mouseMove` trigger
* Be explicit when targetting elements by their index
* fix combo chart tooltip e2e specs
* cy.wait as a last resort
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
* Improve typing in Question
* Remove dead CSS
* Improve typing in getUrlWithParameters
* Add LegendLabel
* Drop navigateToNewCardFromDashboard in PublicOrEmbeddedDashboard
* Extract getNewCardUrl and move getParametersMappedToDashcard closer to it
* Add link to dashboard chart title
* Update title-drill tests
* Format code
* Update test
* Add HrefMemo
* Use state to memoize href
* Recompute state on getHref prop change
* Format code
* Refactor href computation
* Update test
* Use selector instead of useMemo
* Fix unit tests
* Ensure type safety
* Revert getDashCardById type change to not make this PR too large
* Use complete dashboard
* Add a workaround for metabase#43990
* Remind embedding SDK PR author to use conventional commits
* Only remind PR author when PR title isn't conventional commits
* Add branch for testing
* Fix gh cli command
* Fix getting wrong user login
* Ensure we don't create more than 1 comment
* Improve comment format
* Add more info for non-changelog related type
* Check non-changelog related types
* Update the comment for consistency
* Revert "Add branch for testing"
This reverts commit 4a1be68828.
* Migrate text-brand
* Migrate text-dark
* Migrate text-light
* Migrate text-medium
* Migrate admin-navbar
* make compact formatting consistent across all series (#43472)
* make compact formatting consistent across all series
* fix rebase
* [Browse] Show Browse models link in nav only when there are models (#43286)
* Upgrade to TypeScript v5.4 (#43736)
* use correct github api for CI (#43750)
* update list of releases (#43754)
* Multi Series Dashcards Render with Filters Applied To All Series (#43428)
* Multi Series Dashcards Render with Filters Applied To All Series
Fixes: #39083
Multi-series dashcards now properly render when a subscription is set up with filters that impact the values of each
card in the multi-series card.
* Small refactor for executing dashcards in a pulse context.
- move execute-dashboard-subscription to pulse.util
- this fn now will also execute all card series cards
and put those results into a list on the key :series-results in the dashcard
which is then used in the render multimethod to send the correct data to the javascript context
A small todo was taken care of as well, we can now use :dashboard-subscription in the :context key
* Fix up render fn
The shape of hte data going into the javascript was a bit unclear before. This keeps things a little more
understandable.
Next step is probably to create some Malli schemas inside the js_svg namespace to document and enforce the needed data shapes
* pull :data up a level from :result in the series cards
* Update src/metabase/pulse/render/body.clj
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Remove unneeded let
* set/rename-keys won't work here because we raise the :data key
I've realised a suggested change to just use set/rename-keys won't do what we need. We need instead to raise :data
from within the result key up to the same level (and dissoc result).
I've renamed the helper function to make its purpose a bit clearer
---------
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Hydrate can_run_adhoc_query and address review comments (#43747)
This is a followup on #43625.
* Consolidate dashboard-card specs (#43716)
* Consolidate joins specs (#43717)
* Update translations for v50 and add Danish Language Support (#43314)
* update translations for v50
* add Danish
* update translations for v50 release
* alphabetical order is nice
* Move database role to url parameters from connection properties on Snowflake (#43602)
* Move role to url parameters from connection properties
* Fix test
* Fix width of save button in cache sidebar (#43755)
* Remove dead code (#43603)
* Combine search e2e specs (#43718)
* Highlight the table cell data in dashboards when the cell is an ID column (#43728)
* highlight id cell column
* add simpletable stories
* add simple table test data
* add loki reference image
* Update frontend/src/metabase/visualizations/components/TableSimple/TableCell.styled.tsx
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Remove old repro for 39150 (#43749)
* Fix remaining `tinyint` booleans in MySQL (#43296)
* Correct type for `report_card.dataset`
This was missed in the original migration of boolean types to
${boolean.type} in MySQL/MariaDB, and then missed again by me when I
migrated `collection_preview` over a week ago.
* Change all boolean types to `bit(1)` in MySQL
Liquibase changed their boolean type in MySQL from `bit(1)` to
`tinyint(4)` in version 4.25.1. Our JDBC driver does not recognize these
as booleans, so we needed to migrate them to `bit(1)`s.
As discussed [here](#36964), we
changed all existing `boolean` types that were in the
`001_update_migrations.yml` but not the SQL initialization file.
For new installations, this works: things in the SQL initialization file
get created with the `bit(1)` type.
However, for existing installations, there's a potential issue. Say I'm
on v42 and am upgrading to v49. In v43, a new `boolean` was added.
In this case, I'll get the `boolean` from the liquibase migration rather
than from the SQL initialization file, and it need to be changed to a
`bit(1)`.
I installed Metabase v41 with MySQL, migrated the database, and then
installed Metabase v49 and migrated again. I made a list of all the
columns that had the type `tinyint`:
```
mysql> SELECT TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME, COLUMN_TYPE, COLUMN_DEFAULT, IS_NULLABLE FROM INFORMATION_SCHEMA.COLUMNS WHERE COLUMN_TYPE = 'tinyint' AND TABLE_SCHEMA='metabase_test';
+---------------+------------------------------+-------------------+-------------+----------------+-------------+
| TABLE_SCHEMA | TABLE_NAME | COLUMN_NAME | COLUMN_TYPE | COLUMN_DEFAULT | IS_NULLABLE |
+---------------+------------------------------+-------------------+-------------+----------------+-------------+
| metabase_test | core_user | is_datasetnewb | tinyint | 1 | NO |
| metabase_test | metabase_field | database_required | tinyint | 0 | NO |
| metabase_test | metabase_fieldvalues | has_more_values | tinyint | 0 | YES |
| metabase_test | permissions_group_membership | is_group_manager | tinyint | 0 | NO |
| metabase_test | persisted_info | active | tinyint | 0 | NO |
| metabase_test | report_card | dataset | tinyint | 0 | NO |
| metabase_test | timeline | archived | tinyint | 0 | NO |
| metabase_test | timeline | default | tinyint | 0 | NO |
| metabase_test | timeline_event | archived | tinyint | 0 | NO |
| metabase_test | timeline_event | time_matters | tinyint | NULL | NO |
+---------------+------------------------------+-------------------+-------------+----------------+-------------+
10 rows in set (0.01 sec)
```
Then wrote migrations. For each column, we:
- turn it into a `bit(1)`,
- re-set the previously existing default value, and
- re-add the NOT NULL constraint, if applicable.
* Change author and add missing `dbms`
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* Add ts-pattern library (#43752)
* Add events to measure search performance (#43135)
* new_search_query event
* Filters in object, search click event
* updating event schema
* type and event schema adjustments
* delete comments BE code
* adjusting tests
* PR feedback
* typos and spacing
* Fixup missing colors
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
Co-authored-by: adam-james <21064735+adam-james-v@users.noreply.github.com>
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: lbrdnk <lbrdnk@users.noreply.github.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: SakuragiYoshimasa <ysrhsp@outlook.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Nick Fitzpatrick <nick@metabase.com>
* Add repro test
* Move home selectors to `metabase/selectors/app`
* Prevent unnecessary redirects to home dashboard
* Rename
* Handle navbar home link
* Fix unit tests
* Fix navigating away from an archived dashboard
* Handle sub-root Metabase URLs
* Sort functions
* Add boilerplate for aggregateOffset and tests
* Simplify assertion
* Make basic case work
* Handle name generation
* Do not require "lib/uuid" attribute to be present since MLv2 will normalize it under the hood anyway
* Fix typing
* Group tests
* Refactor offsetClause to return a new clause instead of a new query
- Move offsetClause to expressions.ts
- Add assertion
* Revert "Sort functions"
This reverts commit ab9ce2b24e.
* Move tests
* Handle names dynamically
* Shorten a test
* Update comment
* Add TODO
* Update expression types
* Add diffOffsetClause & percentDiffOffsetClause
* Add a test for diffOffsetClause
* Add TODOs
* Add tests for diffOffsetClause and percentDiffOffsetClause
* Unwrap tests
* Add skeleton for new tests
* Refactor first offsetClause tests
* Add tests for names
* Fix case of non-datetime columns
* Handle a case with offset < 1 and no breakouts
* Handle a case with offset < 1 and breakouts on non-datetime column
* Handle a case with offset < 1 and breakouts binned datetime column
* Handle a case with offset < 1 and breakout on non-binned datetime column
* Refactor
* Refactor
* Remove TODO
* Add tests for diffOffsetClause
* Add tests for percentDiffOffsetClause
* Move offset stuff to offset.ts and offset.unit.spec.ts
* Use template string for prefixes
* Use breakoutColumn + isDate to check column type
* Refactor
* Fix error message
* Add boilerplate for CompareAggregations
* Fix title
* Render aggregations list
* Style AccordionList
* Sort props
* Fix bucket name
* Use displayName instead shortName
* Support parseValue prop in NumberInput
* Add period input accepting integers only
* Accept non-negative values only
* Do not accept zeros
* Add state
* Add submit button
* Export offset functions via Lib
* Make it possible to skip rightSection
* Add column picker
* Map offset user input to api input (negative value)
* Add label
* Fix crash
* Extract shouldCreate
* Make onSelect in AggregationPicker support multiple values
* Extract ReferenceAggregationPicker
* Extract ColumnPicker
* Extract getAggregations
* Rename
* Add custom items
* Refactor item component
* Extract OffsetInput
* Remove unused data-testid
* Style OffsetInput
* Generate titles according to specs
* Generate label
* Generate help
* Extract utils
* Use different width for the 1st step
* Format code
* Use MultiSelect directly
* Avoid custom parseValue
* Revert MultiaAutocomplete changes
* Improve typing in describeTemporalInterval and
describeRelativeDatetime
* Use describeTemporalUnit to pluralize
* Use interface
* Avoid setting value as DOM attribute
* Fix test
* Add onAdd prop to AggregationPicker and revert the change to have onSelect pass multiple aggregations
* Reduce number of props
* Render checkboxes in custom items
* Introduce and use --mb-color-brand-lighter
* Avoid !important
* Remove redundant prop
* Rename ExtractColumn to ExtractColumnAction
* Add lines.svg
* Fix positioning in case dir attribute is not present in any parent component
* Add CompareAggregationsAction
* Fix warning about isSelected being used as DOM attribute
* Fix positioning in case dir attribute is not present in any parent component
* Remove redundant Box
* Compute the title
* Add compareAggregationsDrill boilerplate
* Add compare-aggregations drill stub
* Update comment
* Compute title and aggregation
* Add TODO
* Use proper title when there's only 1 aggregation
* Show visualization on submit
* Limit aggregation choice to 1 in compare aggregations drill
* Redirect to chart upon submission
* Do not navigate to chart view
* Do not navigate to chart view
* Make drill-thru tests expect the new compare-aggregations drill
* Add type attribute to all ListItems
* Add aggregationIndex to drillInfo
* Add aggregationDrillDetails and wire it up
* Add tracking helpers
* Add new schema
* Add tracking in column header
* Add tracking in plus modal
* Add tracking in aggregation picker
* Fix destructuring
* Add functionsUsedByExpression
* Fix schema number
* Add tests boilerplate
* Implement test for notebook editor
* Add test for column header
* Add a test for summarize sidebar
* Add a test for the plus button
* Add expectNoBadSnowplowEvents call in afterEach
* Format code
* Add missing redux store state in tests
* Add data for tests
* Move createQuestion call out of beforeEach
* Add tests for labels when there are no aggregations
* Add describe
* Add group helper
* Revert "Add group helper"
This reverts commit e047a223e9.
* Optimize: use cy.log() instead of it()
* Format code
* Add tests for 1 breakout, add helpers
* Add test for non-binned datetime column text
* Add test for non-datetime column text
* Nest tests
* Add test for multiple aggregations and no breakout, fix getTitle in this case
* Add all text tests
* Reduce nesting
* Enable test
* Format code
* Add tests
* Rename function
* Format code
* Simplify code
* Add assertions
* Format code
* Add column assertions
* Run tests with all 3 aggregations
* Remove logs
* Introduce CUSTOM_EXPRESSIONS_USED
* Avoid force click
* Remove explicit assertion
* Use be.visible instead of exist
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
I'm not sure it's any less hacky in the CLJS but it keeps the TS
wrappers thin.
Context:
- #43799 is the original issue
- #43848 added the hack being removed here
This PR simplifies a bit of text used in the cache cron schedule component: we can say "spaces" (which is normal English) not "white spaces" (which is developerese)
* Automate SDK doc PR
* Prevent auto merge
* Dry run npm publish for testing
* Speed up testing by not building jars
* Fix auto approve step
* disable concurrency so I could test multiple runs in parallel
* Test auto-merge
* Fix auto merge
* Revert "disable concurrency so I could test multiple runs in parallel"
This reverts commit 60e90076eb.
* Revert "Speed up testing by not building jars"
This reverts commit 4e53a54414.
* Revert "Dry run npm publish for testing"
This reverts commit 9b5d761c8d.
* Enable all commented out steps
* Remove unnecessary steps
* Mock that the changelog is different from the one in master
* Apply changelog patch incase the changelog is created from another branch
* Dry run npm publish so I could test
* Fix failed patch
* Speed up testing by not building jar
* Fix adding new changelog to master
* feat(sdk): A placeholder for an actual feature in the changelog
* Mock that the changelog diverges from master
* Revert "Speed up testing by not building jar"
This reverts commit acb80d7465.
* Revert "Dry run npm publish so I could test"
This reverts commit f4e679efd9.
* Revert "Mock that the changelog diverges from master"
This reverts commit 0e77a68e1b.
* dashcard background color
* add bg-dashboard and bg-dashboard-card
* rename background-visualization to background-question
* apply bg-question to embedded questions
* use CSS variables along with theme object
* add css variables to more places
* add ability to reference design system colors for defaults
* use CSS variables for defaults
* fix chart skeleton test
* fix legend item title
* Move expressionClause to handleSubmit
* Allow passing the first column as a prop and allow auto picking of columns
* Use CombineColumns component in header drills
* Remove duplicate CombineColumnsDrill component and helpers
* Allow separators to be null
* Calculate example in Example component to avoid useMemo call
* Fix test reference in test
* Append suffix to duplicate columns
* Rename parameter to something more expressive
* Enable autoPickColumn for all instances of the combine column popover
* Fix tests
* Fix notebook tests
* Make test typescript
* Correct types of openTable function
* Add a helper to extract linked issues from PR body
* Add tests
* Tweak the workflow
* Fix the workflow
* Delete obsolete code
* Improve the message
* Fix whitespace error
* Correct type for `report_card.dataset`
This was missed in the original migration of boolean types to
${boolean.type} in MySQL/MariaDB, and then missed again by me when I
migrated `collection_preview` over a week ago.
* Change all boolean types to `bit(1)` in MySQL
Liquibase changed their boolean type in MySQL from `bit(1)` to
`tinyint(4)` in version 4.25.1. Our JDBC driver does not recognize these
as booleans, so we needed to migrate them to `bit(1)`s.
As discussed [here](#36964), we
changed all existing `boolean` types that were in the
`001_update_migrations.yml` but not the SQL initialization file.
For new installations, this works: things in the SQL initialization file
get created with the `bit(1)` type.
However, for existing installations, there's a potential issue. Say I'm
on v42 and am upgrading to v49. In v43, a new `boolean` was added.
In this case, I'll get the `boolean` from the liquibase migration rather
than from the SQL initialization file, and it need to be changed to a
`bit(1)`.
I installed Metabase v41 with MySQL, migrated the database, and then
installed Metabase v49 and migrated again. I made a list of all the
columns that had the type `tinyint`:
```
mysql> SELECT TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME, COLUMN_TYPE, COLUMN_DEFAULT, IS_NULLABLE FROM INFORMATION_SCHEMA.COLUMNS WHERE COLUMN_TYPE = 'tinyint' AND TABLE_SCHEMA='metabase_test';
+---------------+------------------------------+-------------------+-------------+----------------+-------------+
| TABLE_SCHEMA | TABLE_NAME | COLUMN_NAME | COLUMN_TYPE | COLUMN_DEFAULT | IS_NULLABLE |
+---------------+------------------------------+-------------------+-------------+----------------+-------------+
| metabase_test | core_user | is_datasetnewb | tinyint | 1 | NO |
| metabase_test | metabase_field | database_required | tinyint | 0 | NO |
| metabase_test | metabase_fieldvalues | has_more_values | tinyint | 0 | YES |
| metabase_test | permissions_group_membership | is_group_manager | tinyint | 0 | NO |
| metabase_test | persisted_info | active | tinyint | 0 | NO |
| metabase_test | report_card | dataset | tinyint | 0 | NO |
| metabase_test | timeline | archived | tinyint | 0 | NO |
| metabase_test | timeline | default | tinyint | 0 | NO |
| metabase_test | timeline_event | archived | tinyint | 0 | NO |
| metabase_test | timeline_event | time_matters | tinyint | NULL | NO |
+---------------+------------------------------+-------------------+-------------+----------------+-------------+
10 rows in set (0.01 sec)
```
Then wrote migrations. For each column, we:
- turn it into a `bit(1)`,
- re-set the previously existing default value, and
- re-add the NOT NULL constraint, if applicable.
* Change author and add missing `dbms`
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* Multi Series Dashcards Render with Filters Applied To All Series
Fixes: #39083
Multi-series dashcards now properly render when a subscription is set up with filters that impact the values of each
card in the multi-series card.
* Small refactor for executing dashcards in a pulse context.
- move execute-dashboard-subscription to pulse.util
- this fn now will also execute all card series cards
and put those results into a list on the key :series-results in the dashcard
which is then used in the render multimethod to send the correct data to the javascript context
A small todo was taken care of as well, we can now use :dashboard-subscription in the :context key
* Fix up render fn
The shape of hte data going into the javascript was a bit unclear before. This keeps things a little more
understandable.
Next step is probably to create some Malli schemas inside the js_svg namespace to document and enforce the needed data shapes
* pull :data up a level from :result in the series cards
* Update src/metabase/pulse/render/body.clj
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Remove unneeded let
* set/rename-keys won't work here because we raise the :data key
I've realised a suggested change to just use set/rename-keys won't do what we need. We need instead to raise :data
from within the result key up to the same level (and dissoc result).
I've renamed the helper function to make its purpose a bit clearer
---------
Co-authored-by: Ngoc Khuat <qn.khuat@gmail.com>
* Sort functions
* Add boilerplate for aggregateOffset and tests
* Simplify assertion
* Make basic case work
* Handle name generation
* Do not require "lib/uuid" attribute to be present since MLv2 will normalize it under the hood anyway
* Fix typing
* Group tests
* Refactor offsetClause to return a new clause instead of a new query
- Move offsetClause to expressions.ts
- Add assertion
* Revert "Sort functions"
This reverts commit ab9ce2b24e.
* Move tests
* Handle names dynamically
* Shorten a test
* Update comment
* Add TODO
* Update expression types
* Add diffOffsetClause & percentDiffOffsetClause
* Add a test for diffOffsetClause
* Add TODOs
* Add tests for diffOffsetClause and percentDiffOffsetClause
* Unwrap tests
* Add skeleton for new tests
* Refactor first offsetClause tests
* Add tests for names
* Fix case of non-datetime columns
* Handle a case with offset < 1 and no breakouts
* Handle a case with offset < 1 and breakouts on non-datetime column
* Handle a case with offset < 1 and breakouts binned datetime column
* Handle a case with offset < 1 and breakout on non-binned datetime column
* Refactor
* Refactor
* Remove TODO
* Add tests for diffOffsetClause
* Add tests for percentDiffOffsetClause
* Move offset stuff to offset.ts and offset.unit.spec.ts
* Use template string for prefixes
* Use breakoutColumn + isDate to check column type
* Refactor
* Fix error message
* Add boilerplate for CompareAggregations
* Fix title
* Render aggregations list
* Style AccordionList
* Sort props
* Fix bucket name
* Use displayName instead shortName
* Support parseValue prop in NumberInput
* Add period input accepting integers only
* Accept non-negative values only
* Do not accept zeros
* Add state
* Add submit button
* Export offset functions via Lib
* Make it possible to skip rightSection
* Add column picker
* Map offset user input to api input (negative value)
* Add label
* Fix crash
* Extract shouldCreate
* Make onSelect in AggregationPicker support multiple values
* Extract ReferenceAggregationPicker
* Extract ColumnPicker
* Extract getAggregations
* Rename
* Add custom items
* Refactor item component
* Extract OffsetInput
* Remove unused data-testid
* Style OffsetInput
* Generate titles according to specs
* Generate label
* Generate help
* Extract utils
* Use different width for the 1st step
* Format code
* Use MultiSelect directly
* Avoid custom parseValue
* Revert MultiaAutocomplete changes
* Improve typing in describeTemporalInterval and
describeRelativeDatetime
* Use describeTemporalUnit to pluralize
* Use interface
* Avoid setting value as DOM attribute
* Fix test
* Add onAdd prop to AggregationPicker and revert the change to have onSelect pass multiple aggregations
* Reduce number of props
* Render checkboxes in custom items
* Introduce and use --mb-color-brand-lighter
* Avoid !important
* Remove redundant prop
* Rename ExtractColumn to ExtractColumnAction
* Add lines.svg
* Fix positioning in case dir attribute is not present in any parent component
* Add CompareAggregationsAction
* Fix warning about isSelected being used as DOM attribute
* Fix positioning in case dir attribute is not present in any parent component
* Remove redundant Box
* Compute the title
* Add compareAggregationsDrill boilerplate
* Add compare-aggregations drill stub
* Update comment
* Compute title and aggregation
* Add TODO
* Use proper title when there's only 1 aggregation
* Show visualization on submit
* Limit aggregation choice to 1 in compare aggregations drill
* Redirect to chart upon submission
* Do not navigate to chart view
* Do not navigate to chart view
* Make drill-thru tests expect the new compare-aggregations drill
* Add type attribute to all ListItems
* Add aggregationIndex to drillInfo
* Add aggregationDrillDetails and wire it up
* Add tracking helpers
* Add new schema
* Add tracking in column header
* Add tracking in plus modal
* Add tracking in aggregation picker
* Fix destructuring
* Add functionsUsedByExpression
* Fix schema number
* Add tests boilerplate
* Implement test for notebook editor
* Add test for column header
* Add a test for summarize sidebar
* Add a test for the plus button
* Add expectNoBadSnowplowEvents call in afterEach
* Format code
* Add missing redux store state in tests
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Use createQueryWithClauses in createQueryWithCountAggregation
* Support column-specific aggregations in createQueryWithClauses
* Add tests for compare option presence
* Fix case
* Add a test for onAdd
* Split tests
* Rename tests
* Add boilerplate for CompareAggregations unit tests
* Fix assertion
* Add more tests
* Add offset input tests
* Add tests
* Update frontend/src/metabase/query_builder/components/CompareAggregations/CompareAggregations.unit.spec.tsx
* Update tests after merge
* Use constants instead of factory functions
* Check git tag for an exact match
* Set npm publish to dry run, so I can test
* Uses new github automation account for creating a PR
* Remove unnecessary step
* Automatically set `no-backport` on the SDK doc version bump PR
* Fix step to edit PR with correct permission
* Remove npm dry run after finished testing
* return empty string so non-empty labels do not get hidden by empty labels
* fix function name
* apply scaling in transformed dataset
* add missing option to OptionsType
* fix double scaling
* add column scaling to waterfall transformed dataset
* add column value scaling to dataset transform
* fix stacked bar segment labels not responding to column scaling
* fix tooltip values displayed
* since scaling as added to transformedDataset, use transformedDataset for formatters
* revert changes to display percentage instead of absolute value
* fix stacked data labels
* add loki variant for custom y-axis range with column scaling
* fix stack segmented label formatters compact calculation
* scale original dataset and use that instead of modifying transformedDataset
* update loki snapshots
* fix scatter plot dataset scaling
* fix tooltip values being double scaled
* add E2E test to validate behavior
* add helper for not duplicating the column scaling
* port multiple y-axis metrics with column scaling fixes to RowChart
* add waterfall and scatterplot loki variants
* fix type errors and failing unit tests
* fix failing unit tests
* update loki snapshots
* Add E2E reproduction for #39150
* Fix typo in the test title
* Move the viewport config to the `describe` block
* Add the second repro scenario as per the review comment
* Scope the variable to the relevant test
* Sort functions
* Add boilerplate for aggregateOffset and tests
* Simplify assertion
* Make basic case work
* Handle name generation
* Do not require "lib/uuid" attribute to be present since MLv2 will normalize it under the hood anyway
* Fix typing
* Group tests
* Refactor offsetClause to return a new clause instead of a new query
- Move offsetClause to expressions.ts
- Add assertion
* Revert "Sort functions"
This reverts commit ab9ce2b24e.
* Move tests
* Handle names dynamically
* Shorten a test
* Update comment
* Add TODO
* Update expression types
* Add diffOffsetClause & percentDiffOffsetClause
* Add a test for diffOffsetClause
* Add TODOs
* Add tests for diffOffsetClause and percentDiffOffsetClause
* Unwrap tests
* Add skeleton for new tests
* Refactor first offsetClause tests
* Add tests for names
* Fix case of non-datetime columns
* Handle a case with offset < 1 and no breakouts
* Handle a case with offset < 1 and breakouts on non-datetime column
* Handle a case with offset < 1 and breakouts binned datetime column
* Handle a case with offset < 1 and breakout on non-binned datetime column
* Refactor
* Refactor
* Remove TODO
* Add tests for diffOffsetClause
* Add tests for percentDiffOffsetClause
* Move offset stuff to offset.ts and offset.unit.spec.ts
* Use template string for prefixes
* Use breakoutColumn + isDate to check column type
* Refactor
* Fix error message
* Add boilerplate for CompareAggregations
* Fix title
* Render aggregations list
* Style AccordionList
* Sort props
* Fix bucket name
* Use displayName instead shortName
* Support parseValue prop in NumberInput
* Add period input accepting integers only
* Accept non-negative values only
* Do not accept zeros
* Add state
* Add submit button
* Export offset functions via Lib
* Make it possible to skip rightSection
* Add column picker
* Map offset user input to api input (negative value)
* Add label
* Fix crash
* Extract shouldCreate
* Make onSelect in AggregationPicker support multiple values
* Extract ReferenceAggregationPicker
* Extract ColumnPicker
* Extract getAggregations
* Rename
* Add custom items
* Refactor item component
* Extract OffsetInput
* Remove unused data-testid
* Style OffsetInput
* Generate titles according to specs
* Generate label
* Generate help
* Extract utils
* Use different width for the 1st step
* Format code
* Use MultiSelect directly
* Avoid custom parseValue
* Revert MultiaAutocomplete changes
* Improve typing in describeTemporalInterval and
describeRelativeDatetime
* Use describeTemporalUnit to pluralize
* Use interface
* Avoid setting value as DOM attribute
* Fix test
* Add onAdd prop to AggregationPicker and revert the change to have onSelect pass multiple aggregations
* Reduce number of props
* Render checkboxes in custom items
* Introduce and use --mb-color-brand-lighter
* Avoid !important
* Remove redundant prop
* Rename ExtractColumn to ExtractColumnAction
* Add lines.svg
* Fix positioning in case dir attribute is not present in any parent component
* Add CompareAggregationsAction
* Fix warning about isSelected being used as DOM attribute
* Fix positioning in case dir attribute is not present in any parent component
* Remove redundant Box
* Compute the title
* Add compareAggregationsDrill boilerplate
* Add compare-aggregations drill stub
* Update comment
* Compute title and aggregation
* Add TODO
* Use proper title when there's only 1 aggregation
* Show visualization on submit
* Limit aggregation choice to 1 in compare aggregations drill
* Redirect to chart upon submission
* Do not navigate to chart view
* Do not navigate to chart view
* Make drill-thru tests expect the new compare-aggregations drill
* Add type attribute to all ListItems
* Add aggregationIndex to drillInfo
* Add aggregationDrillDetails and wire it up
* Fix destructuring
* Fix tests
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* tag flakiest tests as such
* let well behaved tests out of flake jail
* more bad actors off the streets
* add another flake
* sandboxes re-offended already
* support test utilities in sdk import
* add relative unit default chart font size in embedding defaults
* make font size follow sdk root
* feat(sdk): add font size settings to axis ticks
* feat(sdk): add font size settings to series labels
* re-add hasTimelineEvents from merge commits
* apply default font size at the theme level
* add table header cell font size
* calculate cell font size based on 12.5px
* rename theme.chart to theme.cartesian
* refactor visualization theme
* refactor default viz theme
* remove chart style config
* add goal line
* pass font size to chart measurements
* use px as reference unit
* support row chart viz
* add missing echarts font size
* add table cell font sizes
* use css variables for funnel color
* fix theme type in smart scalar story
* add bar chart to storybook for visual test
* add line chart to storybook for visual test
* add loki reference images
* add loki reference images for row chart
* add loki shared reference images to stories filter
* update loki reference image
* synchronize base font size
* add theme provider to funnel test setup
* add missing default visualization theme
* add unit test for get size in px
* update loki reference image
* fix static viz build error
* Sort functions
* Add boilerplate for aggregateOffset and tests
* Simplify assertion
* Make basic case work
* Handle name generation
* Do not require "lib/uuid" attribute to be present since MLv2 will normalize it under the hood anyway
* Fix typing
* Group tests
* Refactor offsetClause to return a new clause instead of a new query
- Move offsetClause to expressions.ts
- Add assertion
* Revert "Sort functions"
This reverts commit ab9ce2b24e.
* Move tests
* Handle names dynamically
* Shorten a test
* Update comment
* Add TODO
* Update expression types
* Add diffOffsetClause & percentDiffOffsetClause
* Add a test for diffOffsetClause
* Add TODOs
* Add tests for diffOffsetClause and percentDiffOffsetClause
* Unwrap tests
* Add skeleton for new tests
* Refactor first offsetClause tests
* Add tests for names
* Fix case of non-datetime columns
* Handle a case with offset < 1 and no breakouts
* Handle a case with offset < 1 and breakouts on non-datetime column
* Handle a case with offset < 1 and breakouts binned datetime column
* Handle a case with offset < 1 and breakout on non-binned datetime column
* Refactor
* Refactor
* Remove TODO
* Add tests for diffOffsetClause
* Add tests for percentDiffOffsetClause
* Move offset stuff to offset.ts and offset.unit.spec.ts
* Use template string for prefixes
* Use breakoutColumn + isDate to check column type
* Refactor
* Fix error message
* Add boilerplate for CompareAggregations
* Fix title
* Render aggregations list
* Style AccordionList
* Sort props
* Fix bucket name
* Use displayName instead shortName
* Support parseValue prop in NumberInput
* Add period input accepting integers only
* Accept non-negative values only
* Do not accept zeros
* Add state
* Add submit button
* Export offset functions via Lib
* Make it possible to skip rightSection
* Add column picker
* Map offset user input to api input (negative value)
* Add label
* Fix crash
* Extract shouldCreate
* Make onSelect in AggregationPicker support multiple values
* Extract ReferenceAggregationPicker
* Extract ColumnPicker
* Extract getAggregations
* Rename
* Add custom items
* Refactor item component
* Extract OffsetInput
* Remove unused data-testid
* Style OffsetInput
* Generate titles according to specs
* Generate label
* Generate help
* Extract utils
* Use different width for the 1st step
* Format code
* Use MultiSelect directly
* Avoid custom parseValue
* Revert MultiaAutocomplete changes
* Improve typing in describeTemporalInterval and
describeRelativeDatetime
* Use describeTemporalUnit to pluralize
* Use interface
* Avoid setting value as DOM attribute
* Fix test
* Add onAdd prop to AggregationPicker and revert the change to have onSelect pass multiple aggregations
* Reduce number of props
* Render checkboxes in custom items
* Introduce and use --mb-color-brand-lighter
* Avoid !important
* Remove redundant prop
* Rename ExtractColumn to ExtractColumnAction
* Add lines.svg
* Fix positioning in case dir attribute is not present in any parent component
* Add CompareAggregationsAction
* Fix warning about isSelected being used as DOM attribute
* Fix positioning in case dir attribute is not present in any parent component
* Remove redundant Box
* Compute the title
* Use proper title when there's only 1 aggregation
* Show visualization on submit
* Do not navigate to chart view
* Add type attribute to all ListItems
* move parameters and dashboard reducers to the common ones as they're used everywhere
* convert reducers-{common,main,public}.js to typescript, done with --no-verify
* convert reducers to named exports
* Sort functions
* Add boilerplate for aggregateOffset and tests
* Simplify assertion
* Make basic case work
* Handle name generation
* Do not require "lib/uuid" attribute to be present since MLv2 will normalize it under the hood anyway
* Fix typing
* Group tests
* Refactor offsetClause to return a new clause instead of a new query
- Move offsetClause to expressions.ts
- Add assertion
* Revert "Sort functions"
This reverts commit ab9ce2b24e.
* Move tests
* Handle names dynamically
* Shorten a test
* Update comment
* Add TODO
* Update expression types
* Add diffOffsetClause & percentDiffOffsetClause
* Add a test for diffOffsetClause
* Add TODOs
* Add tests for diffOffsetClause and percentDiffOffsetClause
* Unwrap tests
* Add skeleton for new tests
* Refactor first offsetClause tests
* Add tests for names
* Fix case of non-datetime columns
* Handle a case with offset < 1 and no breakouts
* Handle a case with offset < 1 and breakouts on non-datetime column
* Handle a case with offset < 1 and breakouts binned datetime column
* Handle a case with offset < 1 and breakout on non-binned datetime column
* Refactor
* Refactor
* Remove TODO
* Add tests for diffOffsetClause
* Add tests for percentDiffOffsetClause
* Move offset stuff to offset.ts and offset.unit.spec.ts
* Use template string for prefixes
* Use breakoutColumn + isDate to check column type
* Refactor
* Fix error message
* Add boilerplate for CompareAggregations
* Fix title
* Render aggregations list
* Style AccordionList
* Sort props
* Fix bucket name
* Use displayName instead shortName
* Support parseValue prop in NumberInput
* Add period input accepting integers only
* Accept non-negative values only
* Do not accept zeros
* Add state
* Add submit button
* Export offset functions via Lib
* Make it possible to skip rightSection
* Add column picker
* Map offset user input to api input (negative value)
* Add label
* Fix crash
* Extract shouldCreate
* Make onSelect in AggregationPicker support multiple values
* Extract ReferenceAggregationPicker
* Extract ColumnPicker
* Extract getAggregations
* Rename
* Add custom items
* Refactor item component
* Extract OffsetInput
* Remove unused data-testid
* Style OffsetInput
* Generate titles according to specs
* Generate label
* Generate help
* Extract utils
* Use different width for the 1st step
* Format code
* Use MultiSelect directly
* Avoid custom parseValue
* Revert MultiaAutocomplete changes
* Improve typing in describeTemporalInterval and
describeRelativeDatetime
* Use describeTemporalUnit to pluralize
* Use interface
* Avoid setting value as DOM attribute
* Fix test
* Add onAdd prop to AggregationPicker and revert the change to have onSelect pass multiple aggregations
* Reduce number of props
* Render checkboxes in custom items
* Introduce and use --mb-color-brand-lighter
* Avoid !important
* Remove redundant prop
* Fix warning about isSelected being used as DOM attribute
* Fix positioning in case dir attribute is not present in any parent component
* Add type attribute to all ListItems
On the Browse models page we have a filter icon which opens a popover with a switch that toggles "Show only verified models". With this change, the icon is not rendered if there are no verified models at all.
malli.core/explainer{:message"Use metabase.util.malli.registry/explainer instead of malli.core/explainer"}
me.flowthing.pp/pprint{:message"Use metabase.util.log instead of me.flowthing.pp/pprint"}
medley.core/random-uuid{:message"Use clojure.core/random-uuid instead of medley.core/random-uuid"}
metabase.driver/database-supports?{:message"Use metabase.driver.util/supports? instead of metabase.driver/database-supports?"}
metabase.lib.equality/find-column-indexes-for-refs{:message"Use lib.equality/closest-matches-in-metadata or lib.equality/find-closest-matches-for-refs instead of lib.equality/find-column-indexes-for-refs"}
metabase.test/with-temp*{:message"Use mt/with-temp instead of mt/with-temp*"}
toucan2.tools.with-temp{:message"Use mt/with-temp instead of t2.with-temp/with-temp"}}
:discouraged-namespace
{camel-snake-kebab.core{:message"CSK is not Turkish-safe, use the versions in metabase.util instead."}
cheshire.core{:message"Use metabase.util.json instead of cheshire.core directly."}
clojure.tools.logging{:message"Use metabase.util.log instead of clojure.tools.logging directly"}
grouper.core{:message"Use metabase.util.grouper instead of grouper.core"}
honeysql.core{:message"Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}
honeysql.format{:message"Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}
honeysql.helpers{:message"Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}
@@ -135,7 +303,7 @@
;; namespace(s). This is mostly a temporary placeholder until we go in and create module namespaces, which means
;; you should go create one.
;;
;; 2. A set of namespace symbols. All namespaces in other modules will only be allowed ;; to use namespaces from
;; 2. A set of namespace symbols. All namespaces in other modules will only be allowed to use namespaces from
;; this set. Ideally this set should only have one namespace, but restricting it to a set of several is still
;; better than `:any`.
;;
@@ -149,7 +317,8 @@
{metabase.actions#{metabase.actions.core}
metabase.analytics#{metabase.analytics.prometheus
metabase.analytics.snowplow
metabase.analytics.stats}; TODO -- consolidate these into a real API namespace.
metabase.analytics.stats
metabase.analytics.sdk}; TODO -- consolidate these into a real API namespace.
metabase.analyze#{metabase.analyze}
metabase.api#{metabase.api.common
metabase.api.dataset
@@ -158,8 +327,12 @@
metabase.async#{metabase.async.streaming-response
metabase.async.streaming-response.thread-pool
metabase.async.util}; TODO -- consolidate these into a real API namespace.
metabase.audit#{metabase.audit}
metabase.bootstrap#{metabase.bootstrap}
metabase.cmd#{}; there are no namespaces here since you shouldn't be using this in any other module.
metabase.channel#{metabase.channel.core
metabase.channel.render.core
metabase.channel.template.core}
metabase.config#{metabase.config}
metabase.core#{metabase.core.initialization-status}; TODO -- only namespace used outside of EE, this probably belongs in `metabase.server` anyway since that's the only place it's used.
metabase.db#{metabase.db
@@ -168,7 +341,8 @@
metabase.db.setup}; TODO these are only calling `metabase.db.setup/setup-db!` and there's a slightly different version in `metabase.db`
metabase.driver:any; TODO -- 19 namespaces!!!! CRY
metabase.email#{metabase.email
metabase.email.messages}; TODO -- consolidate these into a real API namespace.
metabase.email.messages
metabase.email.result-attachment}; TODO -- consolidate these into a real API namespace.
metabase.embed#{metabase.embed.settings}
metabase.events#{metabase.events}
metabase.formatter#{metabase.formatter}
@@ -186,46 +360,38 @@
metabase.metabot#{metabase.metabot}
metabase.models:any; TODO -- scream, 62 namespaces used elsewhere, but to be fair a lot of these don't *need* to be required.
:message(format"The name of this %s should end with `!` because it contains calls to non thread safe form `%s`. [:metabase/test-helpers-use-non-thread-safe-functions]"
(error!form(format"destructive functions like %s are not allowed inside a ^:parallel test or test fixture. If this should be allowed, add it to the whitelist in .clj-kondo/hooks/clojure/test.clj"
(error!form(format"destructive functions like %s are not allowed inside a ^:parallel test or test fixture. If this should be allowed, add it to the whitelist in the Kondo config file [:metabase/validate-deftest]"
qualified-symbol)))))
(walk[form]
(fform)
@@ -135,7 +28,7 @@
(defn- deftest-check-parallel
"1. Check if test is marked ^:parallel / ^:synchronized correctly
2. Make sure disallowed forms are not used in ^:parallel tests"
[{[_test-name&body]:children, :as_node}]
[{[_test-name&body]:children, :as_node}config]
(let [test-metadata(:metatest-name)
metadata-sexprs(map hooks/sexprtest-metadata)
combined-metadata(transduce
@@ -152,9 +45,9 @@
(hooks/reg-finding!(assoc (meta test-name)
:message"Test should not be marked both ^:parallel and ^:synchronized"
:type:metabase/validate-deftest)))
;; only when the custom `:metabase/deftest-not-marked-parallel` is enabled: complain if tests are not explicitly
;; marked `^:parallel` or `^:synchronized`. This is mostly to encourage people to mark everything `^:parallel` in
;; places like `metabase.lib` tests unless there is a really good reason not to.
;; only when the custom `:metabase/deftest-not-marked-parallel-or-synchronized` is enabled: complain if tests are
;; not explicitly marked `^:parallel` or `^:synchronized`. This is mostly to encourage people to mark everything
;; `^:parallel` in places like `metabase.lib` tests unless there is a really good reason not to.
> in the Metabase Branching Strategy document for more details. If you're not employed by Metabase, this section does not
> apply to you, and the label will be taken care of by your reviewer.
> For those employed by Metabase: if you are merging into master, please add either a `backport` or a `no-backport` label to this PR. You will not be able to merge until you do this step. Refer to the section [Do I need to backport this PR?](https://www.notion.so/metabase/Metabase-Branching-Strategy-6eb577d5f61142aa960a626d6bbdfeb3?pvs=4#89f80d6f17714a0198aeb66c0efd1b71) in the Metabase Branching Strategy document for more details. If you're not employed by Metabase, this section does not apply to you, and the label will be taken care of by your reviewer.
while ! curl -s 'http://localhost:3000/api/health' | grep '{"status":"ok"}' > /dev/null; do
if ! kill -0 $(cat metabase_pid) 2>/dev/null; then
cat metabase.log
echo "Metabase process has exited. Check that your migration changes work on an instance that has been previously initialized on your PR's target branch."
@${githubUsername} You have modified embedding SDK code. Please make sure the PR title follows [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) style.
Here are all the supported types that will be shown in the changelog:
- \`feat\`
- \`fix\`
- \`perf\`
These types \`docs\`, \`style\`, \`refactor\`, \`test\`, \`build\`, and \`ci\` are also encouraged for non-changelog related tasks.
Please also make sure to include \`sdk\` scope, otherwise, the changelog will not include this task.
"text": `:broken_heart:Cross-version tests are failing on ${BRANCH}`,
"text": `:broken_heart:Cross-version tests are failing on ${REF_NAME}`,
"emoji": true,
}
},
@@ -149,7 +159,7 @@ jobs:
"type": "section",
"text": {
"type": "mrkdwn",
"text": `Commit <https://github.com/metabase/metabase/commit/${BREAKING_COMMIT}|${BREAKING_COMMIT.slice(0,7)}> has failing <${FAILED_RUN_URL}|${FAILED_RUN_NAME}> tests on the <https://github.com/metabase/metabase/commits/${BRANCH}|\`${BRANCH}\`> branch.`
"text": `See <${FAILED_RUN_URL}|failing run details>.`
- name:Check out the code using the provided branch
uses:actions/checkout@v4
with:
ref:${{ inputs.git_ref }}
ref:${{ inputs.branch }}
fetch-depth:0
fetch-tags:true
- name:check tags
run:|
if [ $(git tag -l | grep $tag) ]
if [ $(git tag -l | grep $tag$) ]
then
echo "::error::Tag '${{ env.tag }}' already exists. If you expect to run this workflow with the same tag, please remove the tag first and rerun this workflow again."
exit 1
@@ -48,10 +110,10 @@ jobs:
runs-on:ubuntu-22.04
timeout-minutes:20
steps:
- name:Check out the code using the provided ref
- name:Check out the code using the provided branch
uses:actions/checkout@v4
with:
ref:${{ inputs.git_ref }}
ref:${{ inputs.branch }}
- name:Prepare front-end environment
uses:./.github/actions/prepare-frontend
@@ -65,14 +127,14 @@ jobs:
run:yarn embedding-sdk:test-unit
build-sdk:
needs:test
needs:[test, determine-version]
runs-on:ubuntu-22.04
timeout-minutes:20
steps:
- name:Check out the code using the provided ref
- name:Check out the code using the provided branch
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
