* remove unused variable
* conditionally render labels sparsely if large amount of data points
* update loki snapshots
* refactor data label formatter conditions
* setup intelligent label plotting
* handle the case when there are no labels
* add sparse data label rendering to bar charts
* do not compute if no labels need to be shown or hidden
* add sparse data labeling to waterfall charts
* adjustment for large series waterfall labels
* revert change
* refactor name
* add sparse labels to stacked bar and area
* improve scaleFactor
* fix type errors
* remove unused labelFormatter
* update snapshots
* patch ECharts
- labels with no text, i.e. "", were still being considered as plot-able labels which was causing labels with real text "asdfec.." to be hidden because of overlap.
- This patch makes it so that labels with no text, i.e. "", are not considered to be candidates for overlap checking.
* update loki snapshots
* only loop over dataset once for getWaterfallChartDataDensity
* improve performance of chart data density calculations
* update loki snapshots
* update E2E spec
* update E2E spec
* increase cartesian label density allotment
* update loki snapshots
* update loki snapshots
* comment out node_modules cache since it cannot be skipped by commit and is breaking CI
* fix type errors
* Reset node modules if patches are changed
* refactor type naming
---------
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* fix(sdk): add Static Dashboards to README
* Add more clarification comments for API
* Clarify dashboard ID type
* Prevent crashing SDK static dashboard if not provided optional values
* Document fallback behavior for `font`
---------
Co-authored-by: Mahatthana Nomsawadi <mahatthana.n@gmail.com>
* Auto tag embedding team for PR review + dry run npm publish
* Install changelog generation package
* Add the initial changelog
* feat(sdk): This is an empty commit. For testing purposes.
* Update SDk build script to include changelog
* Check if a tag exists
* Ensure changelog is in the released package
* Fix workflow
* Remove unnecessary step
* Ensure we can have access to release utils script when checking out master
* Fix the step to create a PR updating readme and changelog
* Fix changelog generation
* Fix couldn't download changelog
* Clean up steps
* initial commit with most types sorted
* admin, binning, collections, custom-column and dashboard-cards suites
* Filters, Joins, Metrics, Models, Native and Native filters suites
* rest of e2e specs
* filters, actions, onboarding and custom column will hopfully be green?
* fixing static viz, brush filtering, login redirecting, and a few other tests
* types green
* linter green
* hopefully fixes filter tests
* I don't often cry, but when I do it's because of filter-types
* hopfully sorted the last test that's failing CI
* PR Feedback
* yarn prettier
* React 18 Upgrade - Unit Tests (#41577)
* fixes for most unit tests to work with react 18 + upgraded rtl to v15
* fixes all unit tests locally
* fixes from rebasing
* linter shames me
* fix flake in maps.cy.spec.js
---------
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* Do not use asReturned to determine columns for extractions and combine columns
* Add reproduction for #43226 for combine columns
* Add reproduction for #43226 for extract columns
* Remove commented out code
* Only render extraction shortcut when there are extractions
* Only render combination shortcut when there are combinations
* Use hasCombinations from CombineColumns
* Add hasExtractions from ExtractColumn
* Use appendStageIfAggregated over Lib.asReturned
* Only show combine column shortcut when there are two or more columns to be combined
* Lift appendStageIfAggregated to top-level drill
* Add test for extraction on table with just breakout
* Add test for combinations on table with just breakouts
* Disable the 2-column requirement for combinations
* Remove Lib.asReturned in appendStageIfAggregated
* Remove + 1 to stageIndex in appendStageIfAggregated
* Check for empty breakouts too in as-returned
* Switch back to Lib.asReturned
* Fix test for combine column on breakouts
* Reference the correct query and stageIndex
* add CLJS unit tests for as-returned with only aggs, breakouts
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
Models are treated as opaque here, but for a saved question we expose
filters on the query before its final aggregation step.
This drops the aggregation clauses and gets the possible values for the
column of interest.
Fixes#43154.
* Multiple Measures Requires Additional Header Row in Pivot Exports
Mulitple measures can be set in our pivots. We indicate these properly in the fronted with a header row that shows
which measure is reprsented in that particular column.
The header builder fn in the pivot export post processing didn't handle this case, but after this PR, it correctly
adds the final header when necessary.
* Fix up header tests
* Remove comment form; the tests are derived from that work already
These databases previously had impoverished support for regular
expressions. The `:regex` driver feature requires advanced,
"Perl-compatible" regular expressions that can do things like lookahead
assertions.
- MySQL supports this from 8.0.4; we recommend 8.0.33+.
- MariaDB supports this from 10.0.5; we recommend 10.4+.
Fixes#43180.
* Avoid overflowing dropdown menus
* Hide most bucket picker items behind a More... button
* Add test for More… button on TimeseriesBucketPicker
* Add color to More... button
* Add test for showing all columns when a lower bucket is picked
* Handles card events for CRU (not D) => recent view
* add tests
* avoid some errors in dev
We expect recent-views/post-process to return nil occasionally (for
permissions reasons, or more), so let's filter those out instead of
printing a noisy unactionable warning
* uncomment and fix test
* fix some cypress tests + docs improvements
* update more cypress tests
* more cypress test updates
* adding a card puts it into your recents now
* fix notebook-data-source
* fix notebook-data-source
* fixing more e2e tests
* fix command palette e2e tests
* run prettier
* fix style issue
* update notebook-data-source tests
* update homepage e2e tests
* Next question is in recents, now
* update command palette: skip the first pagedown, since it is a noop
* check the buttons for their respective names
* ensure that the model is there, and no questions are
* Keep a single null dimension value
* Add Loki test
* Remove Loki test
* Throw an error if dataset is empty after filtering
* Extract error message into a constant
* Add color to icon directly
* Remove unused css file
* Only show info icon when the field is focussed
* Use metabase blue for background on value tags
* Use larger font size for value labels
* Make the value labels bold
* Tweak padding on search input
* Move custom MultiAutocomplete styles to MultiSelect overrides and adapt to accept xs size too
* feat(sdk): colors for pivot tables
* fix theme type
* feat(sdk): basic pivot table theming
* fix emphasis hover color
* customizable row toggle color
* add storybook story and refactor hover color logic
* add sdk visualization wrapper
* make theme values with defaults internally non-optional but externally optional
* update test to render with providers
* update test wrapper to have height container
* update pivot table loki reference image
* update theme names
* make all theme options optional in the public config
* update readme
* readme update for consistency
- Update how base type and effective type is added in query-method :mbql/query.
- Base type and effective type is added also during the conversion of query-method :query.
- Adds warning suppression for expressions to places where normalize is
called something that does not conform schme (eg. ->pMBQL :dispatch-type/map called
in query-method :query).
* Allow disabling encryption for settings
For two settings, we want to disable encryption entirely.
On the way out of the data source, we don't need to worry - the
decryption is already backwards-compatible with non-encrypted data.
On the way in, things are a bit less straightforward. The core of the
change is replacing the `t2/deftransform` that *always* runs
`encryption/maybe-encrypt` on the value with:
- a define-after-select that always tries to decrypt the value (so we
can handle existing encrypted values in the db)
- a define-before-insert that encrypts the value if the `defsetting`
told us to, and
- a define-before-update that does the same thing.
For now, set two settings to be never-encrypted: `analytics-uuid` and
`anon-tracking-enabled`.
* Sort functions
* Add boilerplate for aggregateOffset and tests
* Simplify assertion
* Make basic case work
* Handle name generation
* Do not require "lib/uuid" attribute to be present since MLv2 will normalize it under the hood anyway
* Fix typing
* Group tests
* Refactor offsetClause to return a new clause instead of a new query
- Move offsetClause to expressions.ts
- Add assertion
* Revert "Sort functions"
This reverts commit ab9ce2b24e.
* Move tests
* Handle names dynamically
* Shorten a test
* Update comment
* Add TODO
* Update expression types
* Add diffOffsetClause & percentDiffOffsetClause
* Add a test for diffOffsetClause
* Add TODOs
* Add tests for diffOffsetClause and percentDiffOffsetClause
* Unwrap tests
* Add skeleton for new tests
* Refactor first offsetClause tests
* Add tests for names
* Fix case of non-datetime columns
* Handle a case with offset < 1 and no breakouts
* Handle a case with offset < 1 and breakouts on non-datetime column
* Handle a case with offset < 1 and breakouts binned datetime column
* Handle a case with offset < 1 and breakout on non-binned datetime column
* Refactor
* Refactor
* Remove TODO
* Add tests for diffOffsetClause
* Add tests for percentDiffOffsetClause
* Move offset stuff to offset.ts and offset.unit.spec.ts
* Use template string for prefixes
* Use breakoutColumn + isDate to check column type
* Refactor
* Fix error message
* Use displayName instead shortName
* Use sentence case for suggestion headers
* Remove suggestion group dividers
* Highlight suggestions when mousing over them
* Remove unused hoverparent
* Remove custom highlighting for icons
* Allow passing color to metadata info icon
* Make suggestions icons have a consisten color
* Update documentation suggestion style
* Add margin to suggestion sections
* Reorder suggestion sections
* Add test for mouse over highlight on suggestions
* Fix test for suggestion footers
* Remove debugging true
* Rename 'Most used functions/aggregations' to 'Common functions/aggregations'
* Highlight suggestion on mouse move
* Only call onHighlightSuggestion if it is not already highlighted
* Use arrow functions
* Fix test for suggestion highlight
* Do not render 'Add column' shortcut when viewing a raw table
* Do not show 'Add column' shortcut when a query has aggregations
* List shortcut guards out of component
* Make 'Add column' icon sticky in the header
* Add test for hiding the column shortcut
* Remove Number wrapper
* Move padding to a constant
* Use actions directly to determine if there are any shortcuts
* Make sure there are expressionable columns for the combine columns shortcut
* Make sure there are extractable columns for the extract column shortcut
* Remove padding when not overflowing
* Do not shadow query in popover components
* Allow combine and extract for raw tables
* Add test for combining summarization columns
* Add test for combining summarization columns
* Use a more precise way of calculating the offset for the 'Add column' button
* Handle null question
* Change to the outline button
* Use actual column widths to calculate total width
* Handle empty mode
* Remove extra measure call
* Move shortcutColumn to separate method and avoid looping over all available actions when possible
* Revert "Revert "minimal support for multiple values for Access-Control-Allow-Origin (#42888)" (#43125)"
This reverts commit a4312230a8.
* refactor: extracted parse-approved-origins + supported "*" without special case
* fix: parse-approved-origins won't break on multiple spaces in a row
* handles invalid urls in the origins + some formatting
* log instead of throwing in parse-url + simplify the logic in parse-approved-origins
* simplify parse-approved-origins logic by splitting on "at least one space"
* remove extra boolean call
* approved-origins -> approved-origins-raw to better reflect that it's a raw string
* swap the order of the arguments to `is` in the tests
* indentation 🙃
* fix trend charts erroring on latest value being null
* refactor previousRowIndex calculation
* handle errors gracefully
* refactor error messages
* update computeTrend unit tests
* add error handling integration test
* add E2E test to check that errors are gracefully handled
* add integration test to enforce the fix
* fix static viz
* display more meaningful error on static-viz trend charts
* move error handling out of trend chart and fix error handling in Visualization.tsx
* update compute.unit.spec.js
* update SmartScalar.unit.spec.js
* use separate error handler for ErrorBoundary errors in Visualization.jsx
* update E2E test
* update error handling
* update compute.unit.spec.js
* fix react state change error in render function
* show generic error message for ErrorBoundary caught errors
* revert ErrorBoundary changes
* remove generic error border for visualizations
* Pass current values to shouldCreate
* Allow comma to create new item
* Simplify getAvailableSelectItems
* Add unit test for MultiAutocomplete input
* Use text instead of Text in paste
* Handle spaces in MultiAutocomplete
* Remove cast
* Reuse last var
* Use haveLastBeenCalledWith over haveBeenCalledWith
* Handle pasting when there is text in the input already
* Accept quote-delimited values
* Add info icon
* Handle tab as a separator
* Fix test cases for unsanitized parseValues
* Add csv-parse
* Use csv-parse to parse values
* Apply the correct color to the info icon
* Rename util to utils
* Break up tooltip text
* Only return unique values
* Be more accepting around quotes in the middle of unescaped values
* provide Buffer for csv-parse package in storybook
* Use browser-friendly version of csv-parse
* Remove unused eslint rule
* Add specital case for csv-parse to jest config
* Put uniqueness check in each state update
* Only add partial value if it is valid
* Only add partial value if it is valid
* Fix test case for FilterValuePicker
* Just use Set
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Allow parsing more than one value
* Use isValid everywhere it makes sense
* Add test for RTL languages
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Remove dead CSS
* Use proper types
* Properly type parameterValues
* Add href prop support to LegendCaption
* Extract getNewCardUrl
- move getParametersMappedToDashcard
* Use href in CartesianChart
* Handle RowChart
* Handle Funnel
* Format code
* Fix types
* Update first test in title-drill
* Update test
* Update test
* Update tests
- pass questionId in createQuestionAndDashboard
* Add test for links
* Fix createDashboardWithQuestions
* Add missing click handler
* Fix crash
* Fix typo
* Add a workaround for #42999
* Fix tests
* Fix hover color
* Update comment
* Get rid of mapStateToProps in DashboardGrid
* Invert condition
* Use useCallback instead useMemo
* Do not pass getNewCardUrl down if navigateToNewCardFromDashboard is not present
- Remove default noop for navigateToNewCardFromDashboard in PublicDashboard
* Remove explicit undefined prop
* add color types for chart sdk colors
* simplify chart color types
* chart color mappings
* apply mapping to master branch
* add chart mapping to mantine
* feat(sdk): apply chart accent colors to mantine and color getter
* rename tint and shade
* remove chart color logic from mantine
* add a color mapping test
* docs(sdk): add chart customizations to readme
* add type comments on MappableSdkColor
* cleanup sdk colors
* Convert DashboardControls to TS hooks
* fix types
* Modify dashboard url params to ensure that we keep hash options that haven't changed
* Fix PublicDashboard and DashboardTabs tests
* Use synced dashboard tabs for automatic dashboard, and use normal tabs for those with DashboardControls
* reduce hash option location changes to only the ones we can change through the UI
* Add comment on state management
* Remove console
* Attempt to solve more tests
* Remove loadDashboardParams as a passed prop
* Fix some tests, but will cause unit test to break
* Change names to reflect actual usage
* Fix lint and type issues
* Add simpler url param control, simplify types, and simplify fullscreen hook
* Add two way binding with url params to attempt to fix tests
* Fix unit test + types
* Move folders and fix imports
* Use Record<string, unknown> for refresh hook
* Remove console.log
* Clear query parameters when ParametersList unmounts
* Add comment
* Revert DisplayTheme changes
* Fix weird reloading problem
* Fix reducers test
* add code comments on selecting the tab once tab data is loaded
* add code comments on unsyncing parameters
* Fix e2e tests, move url reset to tabs context
* Fix e2e tests
* Hotfix
---------
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Avoid unnecessary re-renderings in dashboard - getEmbeddedParameterVisibility
* Avoid unnecessary re-renderings in dashboard - favicon, page title
* Exclude isLoadingComplete from passing down
* Avoid unnecessary re-renderings in dashboard - metadata
* Drop metadata from Public Dashboard
* Avoid unnecessary re-renderings in dashboard - dashcardData
* minimal support for multiple origins in security middleware
* if -> when to be more idiomatic
* adds tests for parse-url
* if -> when
* add missing new line in tests
* split the logic of validation in functions
* adds tests for the small functions
* use namespace in tests instead of :refer
* Make accordion list item content 100% wide
* Remove unused Dot component
* Move bucket popover trigger to the end of the item
* Revert QueryColumnPicker styles to what they were before adding the info icons
* Fix the height of the bucket picker
* refactor scatter plot model to separate function
* (2/3) refactor scatter plot option to separate function (#43033)
* use getScatterPlotOption
* use buildEChartsScatterSeries directly
* remove hoveredSeriesDataKey from scatter
* (3/3) improve chart model types (#43034)
* create WaterfallChartModel type
* create ScatterPlotModel type
* rename BaseCartesianChartModel to CartesianChartModel
* simplify chartModel creation in hook
* Search API namespace
* Search API namespace
* Search API namespace
* Fix Kondo warnings
* Fix typo
* Finally solve the MySQL ^:parallel test failures <3
* Make some search tests REPL-friendly
* Did I finally fix `^:parallel` MySQL tests?
* Ok I give up just make the search tests single-threaded for MySQL/MariaDB.
* Revert the stuff to make paralell tests work for MySQL... for now.
* Keep new name
* Revert deps-graph changes
* Remove unused namespaces
* Fix Kondo linter for log/info + format
* Misc fixes 🔧
* Test fixes 🔧
This is a bit painful. I merged this change, but realized we need to
backport the fix to v49. However:
- we don't want to have two versions of the migration (one with a v49
id, one with a v50 id) because then if someone upgrades to 50, then
downgrades to 49, the `rollback` will run and change the type back,
leading to a bug.
- we don't want to push a v51 changeSet ID to v49 or v50, because we
give the user a helpful notice when their database needs a downgrade.
We do this by checking for the latest *executed* migration in the
database and comparing it to the latest migration that Liquibase knows
about, and making sure the executed isn't bigger than the known (e.g.
you can't have executed a v51 migration if it isn't in the local
migration yaml). That would all work fine, except that then we want to
tell you how to downgrade your database, and we use the latest-executed
version for that. So if, for example, someone upgraded from 48 to 49 and
got a v51 changeset, then downgraded back to 48, they would get an error
telling them to run the *v51* jar to roll back their DB.
In this case though, I think it's fine to just move the migration around
to v49, then we can backport it to 49 and 50.
* handle visibilty_type on tables
- remove def wtf
- uncomment + fix tests
* make sure parent colls exist on report_cards
- official is a string
- ensure the database for a recent table exists
* remove def + improve arg name
* docstring
* improve + extract valid parent collection check
* grab the entity's colllection id and use that for the validity test
- This seems like something that should never happen, but I've seen it
happen. I think it might be possible from a down/up migration.
* fix tests
* fix more tests
* cleanup
* fixing another test
* add error-avoider, to filter bad items in prod instead of throwing
- improve recent view testing for table
* remove wrap (it double prints the value)
* [Metrics V2] Backfill viz settings for compatible-series
Fixes#42575
The BE should rely on the query when there is no corresponding viz setting.
If there is no graph.metrics setting, assume that all last-stage aggregations are used.
No graph.dimensions -> use breakouts.
* Fix tests
* Unskip e2e test
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* hide symbols for charts with too many of them
* update loki
* prevent series dots showing up after manually disabling another series
* update loki snapshots
* refactor conditional
* update loki snapshots
* Migrate druid metrics-inside-aggregation-clauses-test to druid-jdbc
* Fix test-runner: druid should not exclude druid-jdbc
* Implement set-parameter and unprepare-value for LocalDateTime for druid-jdbc
* feat(sdk): apply ui color overrides to the sdk
selector state
set global embedding colors
rename the method
* test(sdk): add unit test on color overrides
* [MBQL lib] Reject broken uses of `offset` in expressions, filters (#42662)
- Using `offset` in custom expressions is supported only when there is
an order-by defined on that stage.
- Using `offset` in custom filters is not supported at all, currently.
* Support `Offset()` in custom columns [frontend] (#42326)
* Only nest expressions referenced in breakouts or aggregations
* Support Offset() as expression with no breakouts
* Test fixes 🔧
* Oracle test update
* Improved Oracle test
* Test update 🔧
* Fix busted test
* Add naive support for Offset() in expressions
* Introduce MBQLClauseFunctionReturnType
* Add "window" to MBQLClauseFunctionReturnType and use it for the offset function
* Handle offset type inference
* Remove unused export
* Use "expression" instead of "window" return type
- Rename identifiers in isCompatible
- Make isCompatible accept a clause object instead of just the type
- Handle offset as a special case in isCompatible
* Use any type
* Rename expectedArgumentType to expectedType
* Format code
* Sort types
* Do not suggest offset function in filters
* Fix offset not working in case
* Revert "Do not suggest offset function in filters"
This reverts commit e63790b780.
* Fix order of adjustments
---------
Co-authored-by: Cam Saul <github@camsaul.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* Disable offsets in filters expressions
* Group existing aggregations-specific tests
* Remove repro for a closed issue
* Use findByText instead of contains
* Add a test for filter expressions
* Add a test for aggregation expressions suggestions
* Disable offsets in filters expressions (#42755)
* Add a test for custom column suggestions
* Add "typing" to enterCustomColumnDetails
* Use enterCustomColumnDetails, improve assertions
* Add more assertions
* Optimize queries
* Add typing for offset expressions
* Add a repro for metabase#42764
* Add a TODO
* Add a TODO
* Add a TODO
* Use NumericLiteral
* Post-merge fixes
* Update test
* Add tests for other custom expressions
* Test drills
* Format code
* Update test name
* Add an assertion
* Add assertions for the preview
* Unskip fixed issue
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
Co-authored-by: Cam Saul <github@camsaul.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
In v49, we converted all boolean types to `bit(1)` on mysql/mariadb. It
looks like we missed this one, so when you `GET` a card it has
`"collection_preview": 1`. Then we get an error when trying to save,
because `collection_preview` should be a boolean.
I verified that after this migration, `GET`ting a card results in a
boolean `"collection_preview"`.
* Set scrollToLastColumn ui control from column extract action
* Scroll to last column when the uiControl is set
* Scroll to last column when combining a column
* Scroll to last column when combining a column via the + shortcut
* Scroll to last column when extracting a column via the + shortcut
* Remove all references to settingsSyncOptions
* Check that the column is added at the end
* Disable scrollToLastColumn once used
* Add test that verifies scrollToLastColumn ui control has been reset
* Fix nil schema in /api/table/:id/query_metadata
* Fix nil schema in /api/table/:id/query_metadata
* Fix nil schema in /api/table/:id/query_metadata
* Fix nil schema in /api/table/:id/query_metadata
* Fix nil schema in /api/table/:id/query_metadata
* Revert "Fix nil schema in /api/table/:id/query_metadata"
This reverts commit 38ac092335.
* Revert "Fix nil schema in /api/table/:id/query_metadata"
This reverts commit 2b8fe78ed4.
* Revert "Fix nil schema in /api/table/:id/query_metadata"
This reverts commit 219506f0e0.
* Revert "Fix nil schema in /api/table/:id/query_metadata"
This reverts commit 9430478bdf.
* Revert "Fix nil schema in /api/table/:id/query_metadata"
This reverts commit 8c6cdd9068.
* Table Picker (#40509)
* Generic types
* Generic types
* Generic types + make generateKey a prop
* Generic types
* Extract AutoScrollBox.styled.tsx
* Move AutoScrollBox to a separate directory
* Generic types
* Generic types
* Generic types
* Unhookify searchFilter
* Generic types
* Generic types
* Generic types
* Generic types
* Generic types
* Generic types
* Generic types
* Generic types
* Finish dealing with a cast
* Destructure import
* Inline type
* Generic types
* Remove cast
* Remove commented code
* Remove redundant fallback
* Move CollectionPicker out of EntityPicker
* Avoid as unknown
* Remove TODOs
* Rename TisFolder to IsFolder
* Fix any
* Fix anys
* Remove NestedItemPicker's storybook
* Remove a cast
* Remove a cast
* Remove a cast
* Revert "Remove a cast"
This reverts commit b762d0f076.
* Use extends SearchModelType instead of extends string everywhere
* Revert "Use extends SearchModelType instead of extends string everywhere"
This reverts commit 708190d63e.
* Fix SearchResult["available_models"] type
* Handle options.allowCreateNew
* Add missing description attribute to Database type
* Move allowCreateNew to CollectionPickerOptions
* Add missing description attribute in Database mock
* Add types
* Add TableList component
* Add SchemaList component
* Fix naming
* Add DatabaseList component
* Add NotebookDataItemPickerResolver
* Fix typing
* Add TablePicker and NotebookDataPickerModal
* Update types
* Fix types
* Add folder type
* Fix initial state
* Update title
* Make query model-dependent
* Render tables
* Fix schema icon
* Adjust key generation
* Rename utilts to utils
* Use ItemList error prop
* Use value prop
* Rename NotebookDataPickerModal to DataPickerModal
* Automatically open datapicker if there is no value
* Highlight current item
* Rename value to initial value
* Fix item highlighting
* Rework TablePicker state
* Leave TODOs for names
* Fix selecting items
* Fix highlighting selected item
* Hide confirmation button
* Add fetchMetadata to tableApi
* Fix Table['schema'] type
* Fetch table metadata upon selection
* Remove options from DataPickerModal
* Fix collectionId typing in Question
* Add collectionId prop
* Remove title prop
* Add todo
* Fix confirmation button
* Remove old DataSourceSelector
* Bring back useSchemaListQuery
* Fix crash
* Fix picker not opening
* Hide dbs list and schemas list if there's only 1
* Remove unused ref
* Remove unused DataPickerListResolver
* Remove unused types
* Change conditional rendering to allow for loading state
* Revert collection-related changes
* Use RTK
* Avoid using stale data
* Fix highlighted db
* Remove todo
* Remove unused type
* Remove useSchemaListQuery
* Rename Value to TablePickerValue
* Move isValueEqual to utils and rename it to isTablePickerValueEqual
* Improve TODO
* Remove unused options prop
* Introduce tablePickerValueFromTable
* Avoid having 2 table metadata requests
* Update comment
* Update comment
* Remove unused function
* Allow null in tablePickerValueFromTable
* Extract helpers
* Rename schemaId to schemaName
* Remove description
* Improve naming
* Simplify types
* Add explanatory comments
* Fall back to empty state when there are no items
* Account for null schemas
* Account for null schema in tests
* Revert "Account for null schema in tests"
This reverts commit 0c96d1cb88.
* Revert "Account for null schemas"
This reverts commit 4f5c4e99ee.
* Improve typing around schema name
* Refactor auto-select logic
* Refactor
* Avoid comment
* Make useAutoSelectOnlyItem generic
* Do not enforce presence of at least 1 tab with TypeScript
* Fix types
* Remove obsolete comment
* Post-merge fix
* Integrate QuestionPicker in DataPicker (#42268)
* Add question and model pickers to Data Picker
* Add model attribute to data picker value
* Introduce DataPickerValue
* Start with the right tab when opening data picker modal
* Do not use stale sourceCard value
* Fix switching tabs when a table is selected
* Use undefined instead of null for consistency
* Do not show models/questions tabs when there are no models/questions
* Fix selecting models/questions in search results
* Remove dead code
* Remove dead code
* Ignore selecting collections
* Refactor useHasModels and useHasQuestions into useAvailableData
* Use models param to speed up the query
* Do not require `onConfirm` prop in `EntityPickerModal` (#42317)
* Make onConfirm prop optional
* Add a test for onConfirm prop assertion
* Table does not appear selected in single-schema dbs (#42336)
* Convert useAutoSelectOnlyItem arguments to param object
* Disable auto-selection when there already is a selection
* Use `DataPicker` in `JoinTablePicker` (#42330)
* Refactor DataStep:
- rename handleTableSelect to handleTableChange
- extract value and memoize it
* Remove redundant fragment
* Add new DataPicker props
* Use DataPickerModal in JoinTablePicker
* Remove initialTab prop since the default tab is always the models tab anyway
* Limit table selection to particular database
* Limit search results to a given database
* Support databaseId in useAvailableData
* Filter collection items by databaseId
* Use databaseId prop only for CollectionItemList component
* Fix types
* Fix filtering
* Use useLatest for onChange
* Fix typing
* Introduce shouldShowItem and use it instead of databaseId prop
* Improve PickerInfo type
* Use MLv2 to generate DataPickerModal value in JoinTablePicker
* Fix title
* Add isModel to TableDisplayInfo
* Add a hack to populate query metadata with info that table is a model
* Remove NewQuestionView & QuestionDataSelector (#42562)
* Temporarily disable recents tab in the data picker
* Update Data Picker tests (#42565)
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Fix metadata loading
* Update test
* Update test
* Update tests
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update tests
* Update test
* Update test
* Update test and make it pass
* Update test
* Update tests
* Update test
* Use entityPickerModal() instead of modal()
* Use entityPickerModal() instead of modal()
* Update test
* Update test
* Update test
* Revert change that broke tests
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Remove test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update tests
* Update test
* Update test
* Rename createDatabaseIdItemFilter to createShouldShowItem
* Update test
* Update and skip a test
* Remove test
* Update tests
* Update tests
* Update tests
* Remove test
* Update tests
* Update test
* Update test
* Update test
* Extract utils
* Use helpers
* Extract helpers
* Update test
* Update test
* Update test
* Update test
* Update test
* Filter out collections that do not have card or dataset
* Improve filtering
* Always show root and personal collections, see https://github.com/metabase/metabase/issues/42687
* Remove failing assertion
* Update tests
* Use entityPickerModalTab
* Accept strings in pickEntity
* Use entityPickerModalLevel
* Use entityPickerModalTab
* Use entityPickerModalTab
* Update tests
* Update test
* Update tests
* Update test
* Fix assertion
* Update test
* Update tests
* Update test
* Update tests
* Update tests
* Update test
* Update test
* Update tests
- ensure database list gets loaded like in a real app
* Update test
* Update test
* Integrate Recents tab in Data Picker (#42821)
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Fix metadata loading
* Update test
* Update test
* Update tests
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update tests
* Update test
* Update test
* Update test and make it pass
* Update test
* Update tests
* Update test
* Use entityPickerModal() instead of modal()
* Use entityPickerModal() instead of modal()
* Update test
* Update test
* Update test
* Revert change that broke tests
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Remove test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update test
* Update tests
* Update test
* Update test
* Rename createDatabaseIdItemFilter to createShouldShowItem
* Update test
* Update and skip a test
* Remove test
* Update tests
* Update tests
* Update tests
* Remove test
* Update tests
* Update test
* Update test
* Update test
* Extract utils
* Use helpers
* Extract helpers
* Update test
* Update test
* Update test
* Update test
* Update test
* Filter out collections that do not have card or dataset
* Improve filtering
* Always show root and personal collections, see https://github.com/metabase/metabase/issues/42687
* Remove failing assertion
* Update tests
* Use entityPickerModalTab
* Accept strings in pickEntity
* Use entityPickerModalLevel
* Use entityPickerModalTab
* Use entityPickerModalTab
* Update tests
* Update test
* Update tests
* Update test
* Fix assertion
* Update test
* Update tests
* Update test
* Update tests
* Update tests
* Update test
* Update test
* Update tests
- ensure database list gets loaded like in a real app
* Update test
* Enable recents tab and update tests
* Update test
* Update tests
* Add Recents-related assertions
* Update test
* Clean up mocks
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Migration to add `trashed_from_*` (#41529)
We want to record where things were trashed *from* for two purposes:
- first, we want to be able to put things back if they're "untrashed".
- second, we want to be able to enforce permissions *as if* something is
still in its previous location. That is, if we trash a card or a
dashboard from Collection A, the permissions of Collection A should
continue to apply to the card or dashboard (e.g. in terms of who can
view it).
To achieve this, collections get a `trashed_from_location` (paralleling
their `location`) and dashboards/cards get a
`trashed_from_collection_id` (paralleling their `collection_id`).
* Create the trash collection on startup (#41535)
* Create the trash collection on startup
The trash collection (and its descendants) can't have its permissions
modified.
Note that right now, it's possible to move the Trash collection. I'll
fix this when I implement the API for moving things to the Trash.
* s/TRASH_COLLECTION_ID/trash-collection-id/g
* Add a comment to explain null comparison
* Move archived items to the trash (#41543)
This PR is probably too big. Hopefully documenting all the changes made here will make it easier to review and digest. So:
Tables with a `collection_id` had `ON DELETE SET NULL` on the foreign key. Since we only deleted collections in testing and development, this didn't really affect anything. But now that we are actually deleting collections in production, it's important that nothing accidentally get moved to the root collection, which is what `SET NULL` actually does.
When we get an API request to update the `archived` flag on a card, collection, or dashboard, we either move the item *to* the trash (if `archived` is `true`) or *from* the trash (if `archived` is `false`). We set the `trashed_from_collection_id` flag as appropriate, and use it when restoring an item if possible.
Because moving something to the trash by itself would have permissions implications (since permissions are based on the parent collection) we need to change the way permissions are enforced for trashed items.
First, we change the definition of `perms-objects-set-for-parent-collection` so that it returns the permission set for the collection the object was *trashed from*, if it is archived.
Second, when listing objects inside the Trash itself, we need to check permissions to make sure the user can actually read the object - usually, of course, if you can read a collection you can read the contents, but this is not true for the trash so we need to check each one. In this case we're actually a little extra restrictive and only return objects the user can *write*. The reasoning here is that you only really want to browse the Trash to see things you could "act on" - unarchive or permanently delete. So there's no reason to show you things you only have read permissions on.
Because previously the Collections API expected archived collections to live anywhere, not just in a single tree based in the Trash, I needed to make some changes to some API endpoints.
This endpoint still takes an `exclude-archived` parameter, which defaults to `false`. When it's `false`, we return the entire tree including the Trash collection and archived children. When it's `true`, we exclude the Trash collection (and its subtree) from the results.
Previously, this endpoint took an `archived` parameter, which defaulted to `false`. Thus it would only return non-archived results. This is a problem, because we want the frontend to be able to ask for the contents of the Trash or an archived subcollection without explicitly asking for archived results. We just want to treat these like normal collections.
The change made to support this was to just default `archived` to the `archived` status of the collection itself. If you're asking for the items in an archived collection, you'll only get archived results. If you're asking for the items in a non-archived collection, you'll only get unarchived results.
This is, for normal collections, the same thing as just returning all results. But see the section on namespaced collections for details on why we needed this slightly awkward default.
No change - this endpoint still takes an `archived` parameter. When `archived=true`, we return the Trash collection, as it is in the root collection. Otherwise, we don't.
* Make Trash Usable - UI (#41666)
* Remove Archive Page + Add `/trash` routing (#42226)
* removes archive page and related resources, adds new /trash route for trash collection, adds redirects to ensure consistent /trash routing instead of collection path
* fixes unit + e2e tests, corrects links generated for trash collection to use /trash over /collect/:trashId route
* updates comment
* Serialize trash correctly (#42345)
Also, create the Trash in a migration rather than on startup. Don't set a specific trash collection.id, instead just select based on the `type` when necessary.
* Fix collection data for trashed items (#42284)
* Fix collection IDs for trashed items
When something is in the trash, we need to check permissions on the
`trashed_from_collection_id` rather than the `collection_id`. We were
doing this. However, we want the actual collection data on the search
result to represent the actual collection it's in (the trash). I added
the code to do this, a test to make sure it works as intended, and a
comment to explain what we're doing here and why.
* Refactor permission for trashed_from_collection_id
Noah pointed out that the logic of "what collection do I check for
permissions" was getting sprinkled into numerous places, and it felt a
little scary. In fact, there was a bug in the previous implementation.
If you selected a collection with `(t2/select [:model/Collection ...])`,
selecting a subset of columns, and *didn't* include the
`trashed_from_collection_id` in that set of columns, then called
`mi/can-write?` on the result, you'd get erroneous results.
Specifically, we'd return `nil` (representing the root collection).
I think this is a reasonable fix, though I'm pretty new to using fancy
multimethods with `derive` and such. But basically, I wanted a way to
annotate a model to say "check these permissions using
`:trashed_from_collection_id` if the item is archived." And in this
case, we'll throw an exception if `:trashed_from_collection_id` is not
present, just like we currently throw an exception if `:collection_id`
is not present when checking permissions the normal way.
* Move existing archived items to the trash (#42241)
Move everything that's archived directly to the trash. It's not ideal
because we're wiping out the existing collection structure, but the
existing Archive page also has no collection structure.
* lint fixes from rebase
* Fix backend tests (#42506)
`can_write` on collection items was wrong because we didn't have a
`trashed_from_collection_id` - the refactor to ensure we didn't make
that mistake worked! 🎉
* Add an /api/collections/trash endpoint (#42476)
This fetches the Trash in exactly the same way as if we'd fetched it
with `/api/collection/:id` with the Trash ID. I hadn't realized that the
frontend was doing this with the previously hardcoded Trash ID.
* Make Trash Usable - Dynamic Trash Collection Id (#42532)
* wip
* fixes hardcoded reference to trash id in sidebar
* remove TRAHS_COLLECTION
* fixes line and e2e tests
* fix invert logic mistake and fixes lint
* Make Trash Usable - Search Filter UI (#42402)
* adds filtering for archived items on the search page
* fix typing mistake
* Make Trash Usable - Bug Bash 1 (#42541)
* disables reordering columns in archived questions, disables modifying archived question name in question header, collection picker no longer defaults to archived item, keeps trashed collection from appearing collection picker search results, shops showing empty area in trashed dashboard info sidebar, disables uploading csvs to trashed collections
* impls pr feedback
* fix e2e failure
* Localize the name of the Trash (#42514)
There are at least two spots where we can't just rely on the
after-select hook, and select the collection name directly from the
database: the Search and Collection API.
In these cases we need to call `collection/maybe-localize-trash-name`,
which will localize the name if the passed Collection is the Trash
collection.
* Update migration IDs
Migration IDs need to be in order.
* Fix failing mariadb:latest test (#42608)
Hooooly cow. Glad to finally track this down. The issue was that
booleans come back from MariaDB as NON BOOLEANS, and clojure says 0 is
truthy, and together that makes for FUN TIMES.
We need to turn MariaDB's bits to booleans before we can work with them.
* Make Trash Usable: Add `can_restore` to API endpoints (#42654)
Rather than having two separate implementations of the `can_restore`
hydration for cards and dashboards, I set up automagic hydration for the
`trashed_from_collection_id`. Then the hydration method for
`can_restore` can just first hydrate `trashed_from_collection` and then
operate based on that.
* fix can_restore for collections trashed from root
* Fix `trashed_from_location` for trashed subcols
We were setting `trashed_from_location` on subcollections to the
`trashed_from_location` of the ancestor that was trashed - which is
wrong.
This would have caused bugs where collections would be restored to the
root collection, regardless of where they were originally trashed from.
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* Fix Trash rollbacks (#42710)
* Fix Trash rollbacks
There were a few errors in my initial implementation.
First, we can't simply assume that `trashed_from_location` and
`trashed_from_collection_id` are set for anything in the archive. They
should be set for things *directly* trashed, but not for things trashed
as part of a collection.
Therefore, we need to set `location` and `collection_id` to something
like "if the item is in the trash, then the `trashed_from` - otherwise,
don't modify it".
Second, because `trashed_from_collection_id` is not a foreign key but
`collection_id` is, we have a potential problem. If someone upgrades,
Trashes a dashboard, then Trashes and permanently deletes the collection
that dashboard _was_ in, then downgrades, how do we move the dashboard
"back"? What do we set the dashboard's `collection_id` to?
The solution here is that when we downgrade, we don't actually move
dashboards, collections, or cards out of the Trash collection. Instead
we just make Trash a normal collection and leave everything in it.
* Make Trash Usable - Bug Bash 2 (#42787)
* wip
* fixes access to property on null value
* pr clean up
* more clean up
* Fix up tests
- permissions will check the archived from location. So recents need to
select :report_card.trashed_from_collection_id and
:dash.trashed_from_collection_id to satisfy mi/can-read?
- some commented out code with `(def wtf (mt/user-http-request
...))`. Restore tests.
- probably commented out because the recent views come back in an order
but we don't care about the order. And it was asserting against an
ordered collection. Just go from [{:id <id> :model <model>} ...] to a
map of {<id> <model>} and then our comparison works fine and is not
sensitive to order.
* put try/finally around read-only-mode
ensure the setting read-only-mode! to false happens in a finally
block. Also, set read-only-mode to false in
```clojure
(deftest read-only-login-test
(try
(cloud-migration/read-only-mode! true)
(mt/client :post 200 "session" (mt/user->credentials :rasta))
(finally
(cloud-migration/read-only-mode! false))))
```
But worryingly, I think we have a lurking problem that I'm not sure why
we haven't hit yet. We run tests in parallel and then put all of the
non-parallel tests on the same main thread. And when one of these puts
the app in read-only-mode, the parallel tests will fail. HOPEFULLY since
they are parallel they won't be hitting the app-db necessarily, but
there could be lots of silly things that break.
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: dan sutton <dan@dpsutton.com>
* move series data label formatters to the model
- TODO: move stacked data label formatters to the model
* combo stacked chart
* spec
* remove irrelevant spec
* combo stacking
* fix selecting y-axis on stacked charts
* snapshot
* linter, specs, types
* propagate combo stacking to area and bar charts
* enable combo stacking for bar/area charts, fix data transform
* fix types, specs
* fix y-axis extents calculations
* remove spec that tested the removed control, update specs, update test data
* propagate combo stacking to area and bar charts (#42600)
* propagate combo stacking to area and bar charts
* enable combo stacking for bar/area charts, fix data transform
* fix types, specs
* fix y-axis extents calculations
* remove spec that tested the removed control, update specs, update test data
* review
* remove unused formatting options parameter
* move stacked labels formatter to model
* fix stacked combo chart label display for non-stacked lines
* combo stacked chart
* spec
* remove irrelevant spec
* combo stacking
* fix selecting y-axis on stacked charts
* snapshot
* linter, specs, types
* propagate combo stacking to area and bar charts (#42600)
* propagate combo stacking to area and bar charts
* enable combo stacking for bar/area charts, fix data transform
* fix types, specs
* fix y-axis extents calculations
* remove spec that tested the removed control, update specs, update test data
* review
* fix waterfall chart labels
---------
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
* handle recent view for non-existant entity
* test that recent views works with missing entities
- just doesn't return them.
* dont count on there being entities in the app-db
* Query for the entity data upfront for recent-views
- Still need to check for the model_object due to popular-items calls
* cleanup, fix ellide logic
memoize get-parent-coll
* more cleanup
* rewire half of the queries
* rewrite the other queries, and improve docs
* align some queries
* lookup all entities ahead of time
- always return effective parent collections
* seperate n+1 tests
* use = instead of a set
* no need for a vector
* fixup simple table test
* go from effective-location-path to id instead of hydrating
* feat: cloud migration endpoints
* fix: don't leave open conn behind when checking migrations
`unrun-migrations` would open a new connection but not close it.
Since `with-liquibase` is happy enough using a data-source the fix is straightforward.
You can verify this by running the following code:
```
(comment
(require '[metabase.cmd.dump-to-h2 :as dump-to-h2]
'[metabase.analytics.prometheus :as prometheus])
(defn dump []
(-> (str "cloud_migration_dump_" (random-uuid) ".mv.db")
io/file
.getAbsolutePath
dump-to-h2/dump-to-h2!))
(defn busy-conns []
(-> (prometheus/connection-pool-info)
first
second
:numBusyConnections))
;; each dump leaves behind 1 busy conn
(busy-conns)
;; => 0
(dump)
(busy-conns)
;; => 1
(dump)
(busy-conns)
;; => 2
(dump)
(busy-conns)
;; => 3
)
```
* fix: flush h2 before returning from dump
* rfct: move code to models.cloud-migration
* test: add login while on read-only test
* fix: assorted cloud_migration fixes from review
* test: allow overriding uploaded dump
* fix: add UserParameterValue to read-only exceptions
Also make the list a little bit nicer.
* fix: only block dumped tables on read-only
* fix: recover on startup if read-only was left on
* feat: block migration when hosted already
* test: test settings for migration
* feat: cloud migration supports retries and multipart
* test: sane dev defaults for migration
* fix: upload 100% shouldn't be migration 100%
* chore: make up a new migration id after merge
* Cloud Migration FE (#42542)
* it's a start
* ui wip
* wip
* dynamic polling intervals, and custom modal for migrate confirmation modal
* cleans out most of the remainig UI TODOs
* adding progress component
* impls team feedback
* makes component more testable, starts some a unit test for the CloudPanel
* finish unit testing
* reverts api changes
* update progress styling
* fix type issues
* fix e2e failure, fix feature unit tests by holding last migration state in fetchMock if more requests than expected happen at the end of a test, remove white spacing change in clj file
* second pass at fixing tests
* fix copy from ready-only to read-only
* copy fix
* Update frontend/src/metabase/admin/settings/components/CloudPanel/MigrationError.tsx
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* Update frontend/src/metabase/admin/settings/components/CloudPanel/MigrationInProgress.tsx
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* adding e2e test
* pr feedback
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
---------
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Raphael Krut-Landau <raphael.kl@gmail.com>
* combo stacked chart
* spec
* remove irrelevant spec
* combo stacking
* fix selecting y-axis on stacked charts
* snapshot
* linter, specs, types
* propagate combo stacking to area and bar charts (#42600)
* propagate combo stacking to area and bar charts
* enable combo stacking for bar/area charts, fix data transform
* fix types, specs
* fix y-axis extents calculations
* remove spec that tested the removed control, update specs, update test data
* review
* show show data values setting on normalized charts that have a line series
* fixes
* [MBQL lib] Add `canPreview` and use it from the notebook
* Fix previewQuery
* Apply can-preview across stages; add unit tests
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Correct Data Shape for Multi-Series Static Viz
Fixes: #41787
During the echarts migration, some backend tests needed fixing. In this process, the shape of data (eg. list of cards
and their data results from execution) were altered.
This was done to fix failing tests and renders for single series cards (all Questions and many dashcards), but didn't
correctly handle a multi-series card.
This conditionally adds back the expected shape.
Perhaps in a follow on PR it's worth inspecting the implementation of the :javascript_visualization render method to
clean it up. Probably worth putting some Malli schemas around it too to make it clearer what shape we need.
* Make sure card/data maps going into static-viz are distinct
* Add a test util to render dashcards, add multiseries card test
We haven't made a 'unit' (perhaps more like a small integration test at this point) test for multi series dashcards
yet.
In most other cases, making sure a card (Question) renders properly is sufficient, but the multi series dashcards are
a special case. It's probably useufl to have a test or two, and the dashcards test util should hopefully help future
test writing a bit now too.
* WIP some workflows
* Add reminder todo comment
* Complete the workflow
* Fix YAML syntax error
* Trigger the workflow
* Fix resuable workflow
* Fix uberjar workflow
* Add missing steps before testing
* Pass recrets to resuable workflow, so it could run
* Fix sdk artifact location
* Remove unneeded output
* fetch tags, so it doesn't fail on push
* Only running EE uberjar in SDK workflow
Because OSS is never needed
* Fix incorrect syntax
* Shuffle workflow so it failed fast
* Fix syntax error
* Attempt to bulid only EE jar
* Fix the error handling step
* Fix failing to upload to s3
* Fix create a PR step
* Fix the last temporary command, so the build looks green
* Remove hack to trigger the workflow
* Remove completed todo comment
* Clean up S3 step
* Add todo comment
* Upload correct docker image name
* Bump the package version before building SDK
* Fix uploading the correct Metabase dev container name
* Fix one too many double quote
* Push the correct docker image
* Fix PR creation step
* Dry run publishing SDK
* Use a separate uberjar workflow for SDK
* Remove a bunch of unused steps
* Update variable names, step names
* Improve tag error message
* Correct m2-cache-key to be unique
* Fix the push tag failure step to only run when failed to push tag
* Fix changed artifact name
* Improve env name for clarity
* Fix extra trailing empty line
* Remove conditions from running certian steps
* Removing npm publish dry-run
* Add a simple replace-names test
* Support find-and-replace in cards with variables and field filters
[Fixes#42580]
[Fixes#42581]
* Add models.card/replace-fields-and-tables!
* Docstring fix
* Little cleanup
3 TODOs were removed here:
- "a way to SET Database-local values." More like TODONE :haaa:
- "we should probably also bind this [that is, *database-local-values*]
in sync contexts e.g. functions in [[metabase.sync]]" I think we don't
need to do this. The full set of maybe-database-local settings right now
is `aggregated-query-row-limit`, `unaggregated-query-row-limit`,
`database-enable-actions`, and `persist-models-enabled`. None of these
need to be set when `sync`ing. If at some point we do add a db-local
setting that should affect the way syncing works, we will need to do
this - but I don't think this TODO will help figure that out.
- "settings sent via an env var aren't returned for security purposes.
It is an open question whether we should obfuscate them and still show
the last two characters like we do for sensitive values that are set via
the UI." I think if it's been this way for 5 years, it no longer
qualifies as an "open question." We can reopen if down the road if this
is something we want.
* PoC Pivot table exporting.
Works so far!
This just hacks a couple things in. I added the :query via format_rows, which should instead have its own proper
middleware
Pivot export should probably not just be in the middleware namespace but in qp.pivot.
Should not break the streaming results writer for non-pivots... obviously this breaks ordinary results, so that needs
to be handled properly.
Better documentation about how the logic actually works. Pivot tables are hard to remember/easy to forget, so some
very clear diagrams/examples would probably be good.
* adjust csv export to export a pivot table when pivot options exist
* post processing of pivot table rows for card/dashcard downloads
* WIP on getting pivot exports to work with unsaved questions
* WIP getting pivot exports working on unsaved questions
The dataset POST /pivot endpoint is used to get the pivot data to the frontend, but also to get the pivot query.
The query that is returned by the POST comes from the json_query, which is populated by the userland-query stuff with
the last run query. The pivot qp runs mulitple queries and each query runs through the regular userland query qp, so
we'll 1. see lots of executions recorded and 2. end up with the wrong query in the POST return. It'll look mostly
correct, but due to how the pivot qp works won't necessarily have all of the fields.
That incorrect pivot query is passed to the /:export-format endpoint and we don't end up seeing correct results.
Since that json_query gets fed back to the backend when doing an export, we do need it to be correct, like the query
that was sent to /pivot in the first place.
Since this is a WIP, I've just passed the query as a string to :original-query in :info, and if that exists, use that
to fill the :json_query key.
Then, also add a :was-pivot flag so that the dataset download endpoint can look at it and use the pivot qp instead of
the regular qp. Even if we have the correct 'top level' query, we need to run it through the pivot qp to get the full,
correct pivoted results.
* Pivot Export Test
Here I've added a pivot export test that has various assertions trying to capture the correct output from the pivot
csv export.
I've done my best to explain the tests with comments, but have also left a (comment ) form with a function
`explore-pivot` to help *show* rather than tell what the expected pivot output should be.
I've created this new namespace to first capture this pivot export test with the intent to add tests that can improve
our confidence that ALL exports/downloads in ALL places behave as we expect in the future. It just seemed nice to
start fresh here with the pivot export test first.
* Don't need to make this public.
* Add lots of comments to hopefully make this a bit easier to follow
* Added an explore-raw-pivot fn in comment form
* Update test/metabase/api/downloads_exports_test.clj
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* begin addressing feedback
- finish an unfinished docstring
- merge info instead of assoc
- use a native query to build the pivot data instead of using a custom dataset, to keep test speed ok
* Create a new ns for the pivot post processing utils
Separate the post processing fns into their own namespace. This means I can easily add a test namespace for the post
processing specifically as well.
* Add pivot postprocessing test ns, not complete, but decent start
* fix mistake in endpoint. Only add original query to first pivot q
* Figured out how to keep the original query without storing as a string
* Perhaps this'll fix things properly?
---------
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
* support theming in table
* theme injection with hoc
* reasonable default theme options
* customizable text color
* theming table header
* table theming
* change theme configuration order
* apply sorted cell color directly
* gutter background
* remove header theming for now
* fix: id column styling
* id column borders
* border color
* add cell hover color
* conditional hover color
* use brand color to accent
* add default theme values to unit test
* cell background should default to white
* update unit test
* remove unused properties
* remove duplicated color declaration as inner cell data styles it
* remove duplicate fallback
* remove unused use-embedding-theme hook
* use theme color from Mantine for css variables
* update defaults
* inject default component theme
* add notes on cell data theme
* update unit tests
* update unit tests
* remove no-op text-brand hover color
* remove no-op is sorted cell color
* Explicitly decide whether to upload or not
* Upload in the main e2e workflow, but skip uploading in stress-test workflow
* Do not record runs in the stress-test workflow
* Add Replay API key
Even though we're not using it in this workflow. 🤷
* Omit env var from the stress-test workflow
this lowers memory usage 100x for 50kb strings (our limit before this change) and speeds up parsing by an order of magnitude.
For example, results for parsing 42kb string (already in memory):
Before: Time per call: 6.32 ms Alloc per call: 8,732,850b
After: Time per call: 55.16 us Alloc per call: 83,254b
* Render item icon inline with column icon
* Use inline info icon in field picker
* Use new QueryColumnInfoIcon on filter columns picker
* Use new QueryColumnInfoIcon on suggestions dropdown
* Use new QueryColumnInfoIcon aggregations dropdown
* Use new QueryColumnInfoIcon filter modal
* Use new QueryColumnInfoIcon in BreakoutColumnListItem
* Use new QueryColumnInfoIcon in FieldList
* Use new QueryColumnInfoIcon in DataSelectorFieldPicker
* Update TableInfoIcon to render fallback too
* Use new TableInfoIcon in DataSelector
* Use new TableInfoIcon in ViewHeader
* Fix types in breakout column
* Move aria-label to wrapper
* Move binning back to the right
* Always give info icon full opacity
* Remove asDot props everywhere
Follow up to #42424.
This enforces that all drills do not appear on native stages.
Also fixes one more drill on top of the two in #42424 that were wrongly
showing up for native queries.
- `:metadata/key-constraints` for those databases which formally track
PK and FK key constraints. Some databases (eg. Presto/Athena/Starburst
family) don't support constraints like that, but do support querying
based on foreign keys (which is the `:foreign-keys` feature).
- `:connection/multiple-databases` for databases where a single
connection can connect to many databases. This is the case for Athena,
since it connects to an S3 bucket and exposes all files in the bucket
as databases through that single connection.
- Also removed a troublesome "all DBs except a huge list" case in the
SSH tunnel tests with a shorter list of supported drivers.
* Fix broken metric revision API call
Fixes#42633
* Add E2E repro for #42633
* Fix metrics revision history in the data reference
* Add repro for #42633 for the reference page
* Consolidate tests
* Do not use hard coded id
* Fix lint errors
* handle recent view for non-existant entity
* test that recent views works with missing entities
- just doesn't return them.
* dont count on there being entities in the app-db
* add recents tab
type updates
fix unit test
change where we filter recents
WIP Recents Tab
wip recents
design updates
update recents design to match search
use new recents format
initial recents implementation
fix types
* fix e2e tests
* update more e2e tests
* allow recents filters
* update test mocks
* add unit tests for recents tab
* update e2e tests
* add e2e tests for recents picker tab
* fix lost access test
* Expand the single or multi-selectable string filter choices
* Mark operators as `multi`
* Wrap an existing test in a separate `describe` block
* Test multiple values settings
* Expand repro to include multiple dashboard filter values
* Ease the load by limiting the card results
* Switch to the more flexible data structure for the dashboard filters
* Expand the assertions
* Fix typo
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* wip
* adds more collection-touch events
* remove sneaky dep that slipped in
* cleaning up, and adding tests
* respond to review comments + mini bug reports
* make it official
* double official
* cleanup + return recent-views/Item for `GET activity/popular_items`
* get popular_items returning the proper shape
* compare doesn't what I thought
* add display_name to tables
* table.database.name should be the db name, not the table name
* add table.database.initial_sync_status
* improve call to toucan2
* light renaming + many text fixes
* include moderated_status in cards and datasets
* look up Dashboard permissions for Dashboards
* stop double-querying for `can-write?` checks
- also include some debug logging
* fix card <-> model dispatching
also gets tests passing locally
* let the test runner run the tests
* fix gc test
* adds description
* description can be null
* better tests
no more nils passed through the app
* debugging test
* fix tests and skip archived
* mysql finesse
* ellide inactive tables
* wip
* adds more collection-touch events
* remove sneaky dep that slipped in
* cleaning up, and adding tests
* respond to review comments + mini bug reports
* make it official
* double official
* cleanup + return recent-views/Item for `GET activity/popular_items`
* get popular_items returning the proper shape
* compare doesn't what I thought
* add display_name to tables
* table.database.name should be the db name, not the table name
* add table.database.initial_sync_status
* improve call to toucan2
* light renaming + many text fixes
* include moderated_status in cards and datasets
* look up Dashboard permissions for Dashboards
* stop double-querying for `can-write?` checks
- also include some debug logging
* fix card <-> model dispatching
also gets tests passing locally
* let the test runner run the tests
* fix gc test
* adds description
* description can be null
* better tests
no more nils passed through the app
* debugging test
* fix tests and skip archived
* mysql finesse
* New Recents + Popular API Format - Frontend (#42305)
* fix tests
* new API payload format for recents and populars
* update url generation
* update recents components
* update tests
* only show 5 recents in command palette
* remove unused types
* update tests
* obey the linter
* type updates
* fix unit test
* change where we filter recents
---------
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* fix modelToUrl types
* respond to review comments
* post merge test fix
---------
Co-authored-by: Ryan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Copy question event to new version
* Add new events for adding column via plus modal
* Add plus modal variants
* Track combine column via plus header
* Track extract column via plus header
* Add test for combine question event
* Add e2e test for extract column shortcut snowplow events
* Add index file to CombineColumns
* Add CombinColumnAction
* Add CombinColumnsAction to default and embedding modes
* Fix outdated --color-brand references
* Set up sequenced close handler in combine columns
* Allow outside clicks to happen when popover is not open
* Add e2e test for combine column shortcut
* Remove unused prop
* Rename file to match component name: CombineColumnsAction
* Simplify test assertion
* Set width on combine columns action
* Remove superflouous ? check
* Add repro for 32323
* Fix offset not working in case
* Make offset function return any
* Add a repro for #42377
* Fix order of adjustments
* Revert unrelated changes
* Remove commented code
* Revert unrelated changes
* Refactor test
* Type offset aggregation
* Add a test for no order by or breakout clause
* Add a basic test for breakout clause
* Fix assertion
* Remove problematic dependency
* Fix uuids generation
* Remove redundant limit
* Add a test for multiple breakouts
* Update test names
* Extract OFFSET_SUM_TOTAL_AGGREGATION
* Add a repro for metabase#42509
* Add a test for multiple aggregations and breakouts
* Remove unused intercept
* Move uuid utils to separate file
- it wasn't possible to import the utils file in e2e tests without it
* Make isUuid a type guard
* Add tests for sorting
* Tag the test
* Add extra assertion to verify expression parsing
* Add a complex scenario
* Reverse isFirst logic
* migrate smart scalar to css modules
* lighten theme color
* adjust margin
* map space 1 to xs
* use c prop instead of color
* use rem unit
* use rem unit
* add y margin
* return null when no children
* comment on text variation
* cleanup unused components and add dark mode todo
* remove todo comment
* support night mode in embedding
* lift dashboard theme to redux
* make theme non-nullable with default
* use component prop instead of title tag
* optional display theme
* restore tooltip state
* add default mock state
* remove theme from embed
* replace handleActions with createReducer
* Render plus button in interactive table header
* Add columnShortcuts on ClickObject
* Dispatch click action for columnShortcuts on plus button
* Make onCancel an optional prop
* Add ExtractColumn action
* Add ExtractColumn action to Default and Embedding mode
* Use arrow_split icon for extract column action
* Add title to extract column popover
* Add e2e test for Extract column shortcut
* Use asReturned in extract column
* Rename to Extract part of column
* Add a custom title to the new column shortcut
* Add onClose to ExtractColumn popover
* Hide shortcuts icon when question is not editable
* Fix null error
* Do not use submodules in import
* Scroll more in test
* Format Rows toggle also applies to Column Names
Fixes: #42500
When format-rows is false, we want to also pass that true/false value to the title column formatting.
That is, if a column is renamed and formatting is applied, we want to use that column's name. But, if a column is
renamed and the user exports unformatted csv or json, the custom column name should NOT be used, as was the behaviour
in prior versions.
* write a better test explanation
Ensures that models are sorted as follows in Browse models:
* Models are sorted by collection, by default
* When models are sorted by collection, they are secondarily sorted by name (so that models with the same collection path appear in alphabetic order)
* use echarts tree shaking api
* fix specs
* update types
* move echarts module registration back to individual components
* Transpile echarts in jest
---------
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
These params were failing to match. This matches them up and puts the
options in the correct place. (`[:contains x y opts]` but `[:contains opts x y z]`).
* how optional clauses work
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Fix offset not working in case
* Make offset function return any
* Add a repro for #42377
* Fix order of adjustments
* Revert unrelated changes
* Remove commented code
* Revert unrelated changes
* add upload management API
* add upload management interface
* remove trailing slash
* add e2e upload management test
* add upload management to token_features
* jsdoc cleanup
* fixes tests
- include :upload_management in session_test
- fix list-url in upload management
- ensure local tests have upload management disabled
(causes tests locally to 422 instead of 402: it has sufficient
permissions from a local token to try to delete the table but the
table is not an upload table, so you get a 422)
* add card archive param
* update tests
* use common component
* add success/error toasts
* test success/error notifications
* update e2e tests
* address review comments
* make sure we have fresh data
---------
Co-authored-by: dan sutton <dan@dpsutton.com>
* Fix flaky question management test
* Update assertions order to reduce probability of a flake
* Use explicit assertions
* Change assertion order
* Decrease flakiness
* Remove unused functions
* Remove `export` from a local function
* Remove googleanalytics related engine code
* Use `getEngines` selector instead of the deprecated Metabase `Settings`
* Use `getDefaultEngineKey`
* Use `formatSQL` function locally
* Remove unused `getEngineLogo` function
* Convert to TS
* Remove unused export
* Fix types
* Expand test coverage
* Update type
* Remove `googleanalitics` from an assertion
* Address review comment
* Expand types
* Add tests for `isDeprecatedEngine`
* Expand tests
* wip dashboard picker
* use in homepage selector
* remove dead code
* remove generateKey prop
* add unit tests
* cleanup
* WIP unit tests
* update personal/public interactions and tests
* update e2e tests
* allow disabling items in dashboardPicker
* add new dashboard tag invalidation
* use rtk in dashboardpicker
* use collection_id from api
* horrendous way of disabling read only items
* remove badness
* disallow selecting read-only dashboards in dashboard picker search
* pass search filter as prop
* update types and tests
* update types and tests
* address review comments
* convert `SocketChannelEndpoint` to `SocketChannel`
* Remove fatal logging (used for debugging)
* check for nil channel + log on unknown type
* only log unexpected transport types once
* prop to reserve height and use full height by default
* require height to be passed
* add default height based on viz type
* default to 250px
* reserve table height
* remove memoization
* rely on the card size defaults constant
* multiply by a fixed constant
* correct vis to viz
* Dashboard Param Values Common Impl for Embedding/Preview Embedding
Refactor the embedding api namespace to allow a bit of reuse between the embedding api and the embedding preview
api. In some cases, the implementations are not identical (in terms of the shape of data expected/endpoints
available), so this is a first step to making them the same.
Related:
https://www.notion.so/metabase/Make-embedding-preview-behave-consistently-with-actual-static-embeds-855353d8e5e8411d8164c7ac563c7d2f
* Add a test to verify that preview-embed param-values endpoint works
* Use preview embed endpoint and fix relevant tests
* Add tests
* Change name to reflect what actually happens in the function
---------
Co-authored-by: Mahatthana Nomsawadi <mahatthana.n@gmail.com>
now:
```clojure
❯ clj -M:"$ALIASES"
Warning: environ value /Users/dan/.sdkman/candidates/java/current for key :java-home has been overwritten with /Users/dan/.sdkman/candidates/java/21.0.2-tem
2024-05-07 19:55:25,167 INFO metabase.util :: Maximum memory available to JVM: 8.0 GB
```
no logs about recur targets unboxing
* Only nest expressions referenced in breakouts or aggregations
* Support Offset() as expression with no breakouts
* Test fixes 🔧
* Oracle test update
* Improved Oracle test
* Test update 🔧
* Fix busted test
* Fix running and saving Offset over the REST API
* Remove stray :wow
* More test fixes 🔧
* Add suffix to name if column already exists
* Add test for duplicate combine columns
* Use direct question in test
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Fix formatting
* Use createQuestion for other tests too
* Fix table references
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* initial attempt
* search button
* small adjustments
* adding back the search bar
* adding an extra check in e2e
* i18n adjustment
* impls pr feedback
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
before:
```clojure
❯ clj -M:"$ALIASES"
Warning: environ value /Users/dan/.sdkman/candidates/java/current for key :java-home has been overwritten with /Users/dan/.sdkman/candidates/java/21.0.2-tem
WARNING: abs already refers to: #'clojure.core/abs in namespace: clojure.algo.generic.math-functions, being replaced by: #'clojure.algo.generic.math-functions/abs
2024-05-07 04:39:37,890 INFO metabase.util :: Maximum memory available to JVM: 8.0 GB
```
after:
```clojure
❯ clj -M:"$ALIASES"
Warning: environ value /Users/dan/.sdkman/candidates/java/current for key :java-home has been overwritten with /Users/dan/.sdkman/candidates/java/21.0.2-tem
2024-05-07 04:40:42,561 INFO metabase.util :: Maximum memory available to JVM: 8.0 GB
```
Now no warning about `abs`
* add percent change to timeseries tooltip
* move computeChange to lib/numeric
* cleaning up code in numeric.js
* color text
* add unit to key
* move change to tooltip footer
* fix extra bottom spacing
* remove color
* handle gaps in data
* clean up code
* add change to funnel tooltip
* move retained to footer in funnel
* update copy
* remove min decimal count
* rename events.tsx -> events.ts
* add test cases to line-bar-tooltips.cy.spec.js
* add specs for edge cases
* readd exports to fix numeric.unit.spec.js
* don't show for change for scatterplot
* translate retained
* use parseTimestamp instead of String
* handle quarter interval
* remove redundant dayjs calls
* handle quarter with Feburary correctly
* api details
* Update docs/installation-and-operation/serialization.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* public metabase theme type
* add tests for transforming embedding theme
* export public types
* load transformed theme override into theme provider
* migrate font family inside theme configuration
* add assertion for font family
* update README to add font family in theme options
* add font enum, fix tuple type and cleanup tests
* color tuple argument name
* use fill for colortuple
* use css modules instead of inline styles for cursor: default (#41644)
* add toast notification after the feedback (#41575)
* fix modal submit sending data while button says skip (#41784)
* fix: it should not send the feedback if button says skip
* add comment explaining why we check that getLastFeedbackCall() is undefined
* embedding homepage analytics (#41725)
* rename defaultTab -> initialTab
* add events schema for embedding-homepage
* fix stories
* add analytics, e2e tests and fix typo in dismiss reason
* copy embed_flow schema for better diff in the next commit
* adds isExampleDashboard to trackStaticEmbedPublished
* utm tags
* fix bug of is_example_dashboard
* better test names
* refactor embed homepage status type
* "() => {}" => "_.noop"
* add "Setup embedding" to the admin setup checklist (#41638)
* add setup embedding to the admin setup checklist
* restore whitespace as it was before
* i miss prettier (whitespace again)
* Clean up check list for embedding
Largely making a new entry for the embedding info just to fight long
lines. `:embedding-homepage-dismissed-as-done` just made the maps really
long. If we have a `:done?` under an `:embedding` keyword it reads much
more naturally.
Also, The previous diff was calling `boolean` on a var rather than
invoking the var:
```clojure
:embedding-app-origin (boolean embed.settings/embedding-app-origin)
;; ^^^^^^^^^^^^^^ needs to be (invoked)
```
---------
Co-authored-by: dan sutton <dan@dpsutton.com>
* Change utm tags for embedding homepage links (#42042)
* update utm tags
* utm_media -> utm_content
* add description to the checklist setup step (#42200)
---------
Co-authored-by: dan sutton <dan@dpsutton.com>
* Update readme adding more ways to run Metabase
* Remove running from `master` because it could break
* Update enterprise/frontend/src/embedding-sdk/README.md
Co-authored-by: Alberto Perdomo <albertoperdomo@users.noreply.github.com>
* Update disclaimer, makes it more obvious
---------
Co-authored-by: Alberto Perdomo <albertoperdomo@users.noreply.github.com>
* Rename "Line style" to "Line shape"
* Add "line.style" viz setting
* Add "line.size" viz setting
* Apply new viz settings in line/area charts
* Clean up `ChartSettings` test suite
* Fix default values not applied
* Add visual test
* Add E2E test
* Rework default values
* Use constant S/M/L values for "line.size" setting
Lets the `ItemsTable` component have customizable columns.
The logic for this is contained in `BaseItemsTable`. This is preliminary work for #41614
This doesn't introduce any new UI, but there is a unit test for the new logic.
The main thing to check is whether the table on the collections page stills work correctly. Note how the columns adjust as you resize the viewport horizontally. (In fact, the columns will sometimes adjust as you close and open the sidebar, because the width they are paying attention to is the width of the area to the right of the sidebar.)
Closes#41908
The API/model schemas for dashboard parameters are general enough
that they just work for this new type, which doesn't look very different
from a value parameter right now.
Part of #42118.
* Add test for email extractions
* Fix swapped domain and host examples
* Add tests for URL columns
* Fix examples for shortcut too
* Use arrow function instead of function
* Simplified MetadataProvider protocol
* Test fix
* Ok make CachedMetadataProvider a separate schema again after all.
* Consolidate the Metadata schemas
* Clean Kondo warnings
* Malli-ify the stuff in metabase.lib.metadata.protocols
* Fix Kondo warnings
* Test fix 🔧
* PR feedback
* Window functions should use all breakouts except first for partitioning
* Update the offset() test
* Update tests
* Rework how we use driver escape-alias in MLv2; fix Oracle
* PR feedback
* Test fixes 🔧
This is a new parameter `:type` (and `:widget-type`) to choose a time
granularity (a `:temporal-unit`, eg. `month`, `day`, `day-of-week`)
which should replace the unit on the target field for the parameter.
The target field should be a breakout with a temporal type (date,
datetime) that's compatible with the input unit. The value of the
parameter should be a string or keyword naming one of the units:
`"month"`, `:month`.
Part of #42118.
* Add trackColumnExtractViaHeader helper
* Add Lib.extract wrapper
* Track column extractions via header
* Fix analytics event name
* Add tests for extract action analytics
* Set up extractions metadata helper
* Use functionsUseByExtraction in analytics
* Add custom type for ExtractionTag
* Use ExtractionTag in analytics
* [MBQL lib] Add `Lib.columnExtractDrillExtractions`
This gets the extractions from a column-extract drill, which is needed
in the UI.
* Add extractionExpression wrapper
* Add extractionsForDrill helper
* Fix type signature of extractionExpression
* Use expressionParts for getting expression function
* Walk expression to get functions used
* Simplify ColumnExtractionTag
* Fix expected expression
* Be more explicit with what the guard does
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* use palette for rendering
* add unit tests for chart label colors
* cleanup test code
* update unit test name
* cleanup color handler
* use theme.fn.themeColor to get color
* Adding Descriptions to search results in palette
* Palette Results List component
* adjusting e2e and unit tests
* type fix
* removing duplicate theme font
* fix typo, palette types adjustment
* ensures metabase theme override is merged with user-supplied theme
* ensure both metabase and user themes are merged
* ensure existing theme override is applied
* add tests for default text colors
* [MLv2] Add `preview-expression` to eval an expression on sample data
This is needed to show the previews in the UX for the combining columns
epic #39977.
Fixes#39979.
* [MBQL lib] Add "Combine columns" drill-thru
This is following the original plan of using a drill for this. It's hard
to combine that with the "preview" functionality.
See [thread](https://metaboat.slack.com/archives/C06P22KS4JH/p1712264174056569)
for discussion of how we might approach that issue.
Fixes#39978.
* [MBQL lib] Add "Combine columns" drill-thru
This is following the original plan of using a drill for this. It's hard
to combine that with the "preview" functionality.
See [thread](https://metaboat.slack.com/archives/C06P22KS4JH/p1712264174056569)
for discussion of how we might approach that issue.
Fixes#39978.
* [MLv2] Add `preview-expression` to eval an expression on sample data
This is needed to show the previews in the UX for the combining columns
epic #39977.
Fixes#39979.
* [MLv2] Add `preview-expression` to eval an expression on sample data
This is needed to show the previews in the UX for the combining columns
epic #39977.
Fixes#39979.
* Revert "Merge branch 'mblib-preview-expressions' into combine-columns"
This reverts commit dac17f84aa, reversing
changes made to 2934fab19d.
* Revert "Merge branch 'mblib-preview-expressions' into combine-columns"
This reverts commit 2934fab19d, reversing
changes made to 6a43c22e78.
* Revert "Merge branch 'mblib-preview-expressions' into combine-columns"
This reverts commit 6a43c22e78, reversing
changes made to 2257333a8a.
* [FE] "Combine columns" drill thru (#40082)
* Type Section['icon'] properly, including the hack for summarize section
* Add combine columns section
* Add types for combine columns drill
* Add combine columns drill component stub
* Mock combine columns drill
* Add component stub
* Encapsulate typing hacks
* Extract CombineColumnsDrill
* Extract ColumnAndSeparatorRow
* Improve naming, add button to edit separators
* Translate string
* Improve styling
* Separator input
* Handle removing columns
* Rename lib.ts to utils.ts
* Implement onSubmit
* Add getNextColumnAndSeparator
* Use form for a11y
* Handle vertical overflow
* Handle horizontal overflow
* Extract formatSeparator
* Use proper translations
* Adjust min/max sizes
* Fix scrollbar being unnecessarily shown
* Display source column name
* Avoid serializing the entire column and putting it into DOM
* Use module.css extension
* Format code
* Improve a11y
* Mock combineColumnsDrillExpression and previewExpression
* Add Preview component
* Move preview logic to Preview component
* Style Preview
* Format code
* Move styles to CSS modules
* Update UI to latest designs
- move "Add another column" button
- remove "Separated by X" button
- always show separator inputs
* Remove displayInfo mock
* Add asReturned
* Update CombineColumnsDrillThruInfo
* Integrate new way of working with combine columns drill
* Fix generating expression names
* Integrate drill with click actions
* Integrate new preview workflow
* Revert "Update UI to latest designs"
This reverts commit cda039dffe.
* Use previewExpression interface correctly
* Use new preview_expression API
* Make sure columns are created with original query and stage index
* Extract usePreview and handle preview errors
* Update popover title
* Move add column button
* Fix outline being cut off
* Style preview label as per design
* Format empty separator
* Make preview scrollable
* Use project convention
* Simplify ScrollArea usage
* Use ScrollArea in CombineColumnsDrill
* Update comment
* Remove Lib.previewExpression
* Clean up the preview after removing Lib.previewExpression
* Implement getPreview
* Change default column preview to 'text'
* Remove usePreview hook which is no longer in use
* Rename Preview to Example
* Use different placeholder for separator input
* Add whitespace placeholder
* Add ColumnPicker boilerplate
* Export Input
* Use QueryColumnPicker for ColumnPicker
* Pass width to AccordionList
* Add sequences popover events with tippy
* Remove unused ColumnOption helpers
* Add testid to Example
* Add e2e test for combine column in header
* Select content of separator input on focus
* Reduce padding on example
* Add monospace variant to Text
* Use monospace variant in Example
* Use monospace font in textinput
* Match faux-select label styles to our TextInput label styles
* Add missing color
* Remove !important
* Use pre whitespace so consecutive spaces show up
* Remove empty default
* Add example for boolean
* Remove !important
* Use should.have text over contain
---------
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
* Remove duplicated code
* Export useMantineTheme from metabase/ui
* Use useMantineTheme instead of reading theme directly
* Remove setTimeout in popover
* Use plain t instead of jt
* Move Popover dropdown sequence hack to metabase/ui
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add string like filtering capabilities to filter widget
* Address the review suggestions
* Add e2e test
* Address review suggestions
* Update mongo e2e test id so it hopefully corresponds to CI data
* Update e2e test to use id gathered from actual data
* Expand and improve the E2E reproduction
* Explicitly reference the underlying issues we're reproducing
* Update frontend/src/metabase/querying/components/FilterPicker/FilterPickerBody/FilterPickerBody.tsx
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
* Fix parens
---------
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
These string matching clauses only allowed two arguments previously.
Typically `[:contains field x]` to match a field against a literal.
This adds similar desugaring for `:contains`, `:does-not-contain`,
`:starts-with` and `:ends-with` that is currently done for
multi-argument `:=` and `:!=`:
```clojure
[:contains field x y z] ;; ->
[:or [:contains field x] [:contains field y] [:contains field z]]
[:does-not-contain field x y z] ;; ->
[:and [:does-not-contain field x]
[:does-not-contain field y]
[:does-not-contain field z]]
```
* Add shortcuts to suggestions
* Add shortcuts group
* Add split icon
* Start implementing extract column modal
* Pass alwaysExpanded from prop to accordion items
* Allow disabling search on QueryColumnPicker
* Only render columns that support extraction
* Add columnExtractions helper to Lib
* WIP
* Add examples for column extraction
* Use Lib.columnExtractions to create extraction picker
* WIP: Create clause from extraction
* Use mb- prefix in color
* Add extract wrapper
* Use updated column extraction types
* WIP: extract clause
* Use JSDoc for todo
* Handle extract column submit in expression editor
* Set name based on extract expression
* Suffix column displayNames to avoid conflicts
* Rename combine test to disambiguate it from extract test
* Add e2e test for column extractions
* Correctly render button as button
* Add tests for email extractions
* Use correct font in expression button
* Remove error when selecting an extraction
* Use last expression
* Use handleExpressionChange
* Add tests for url extractions
* Rename to lastExpression
* Add issue link to the todo
* Move Button into ExtractColumn
* Use unstyled button for ExtractColumnButton
* Fix Snowflake filtering on date for `timestamp with time zone` columns
* Snowflake should just generate raw SQL for literals
* Go ahead and fix other Snowflake stuff and reenable a bunch of disabled tests
* Snowflake test fixes 🔧
* Minor test improvements
* Fix Redshift and Vertica as well.
* Address PR feedback
* update the option name
* Update docs/configuring-metabase/slack.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Use RTK Query endpoint to fetch native dataset in preview query modal
* Remove unused selector
* Remove endpoint from `services`
* Fix type error
* Use `getLearnUrl` selector instead of the deprecated MetabaseSettings
* Fix the error potentially being undefined when passed to a helper
* Fix weird destructuring syntax
* Store Parameter Values Set by User on a per-user basis
This is a WIP for #28181 and the notion doc:
https://www.notion.so/metabase/Tech-Maintain-user-level-state-in-dashboards-for-filters-fc16909a3216482f896934dd94b54f9a
Still to do:
- [ ] validate the table/model design
- [ ] hook up the endpoints mentioned in the doc (2 in api/dashboard)
- [ ] return the user specific parameter values on /api/dashboard/:dashboardID endpoints
- [ ] write a few tests to capture the intent of this feature
* Accidentally deleted a digit in the change ID timestamp
* first pass at writing user param values to the db.
It's in a doseq here which is probably not the correct way to go yet, but it's a step in the right direction.
* Hydrate dashboard response with `:last_used_param_values` key
If the user has previously set a parameter's value, it will show up in the map under that parameter id.
If the user has no parameter value set, that entry will be 'null'.
* Use proper fn name
* Only save or retreive user params if there's a current user id
* Add model to necessary lists
* Only run query when there are parameter ids to run it with
* Add a test to confirm that last_used_param_values works
* Add models test namespace for CRUD test
* The hydration is checked in the dashboard api test already
* Fix `sdk` key not expected on a reducer
* Use existing type instead
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Add default SDK state when using `renderWithProvider` with `sdk` mode
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Add analytics event for combine columns shortcut
* Add new version for question analytics schema
* Add column_combine_via_shortcut event
* Add custom_expressions_used field
* Use trackSchemaEvent to track combine column event
* Move trackColumnCombineViaShortcut to separate file
* Fix typo in event key
* Add column_combine_via_column_header
* Add column_extract_via_shortcut
* Add column_extract_via_column_header
* Add database id for question
* Move analytics file to frontend/src/metabase/query_builder
* Allow null for custom_expressions_used
* Add test for snowplow event
* Fix typo in schema
* Add max length to custom expressions
* Add description to list item
* Fix add-alias-info matching for nominal field literal refs
* Minor cleanup
* Minimize diff
* A bit more cleanup
* Cumulative aggregations w/ multiple breakouts should reset counts (#2862)
* Update dox
* Do the nesting FIRST before adding alias info.
* Remove unused
* FE for Offset()
* Cleanup stray code
* Prettier
* Kondo
* Test fixes 🔧
* Better error message when no breakout
* Support offset as an expression with no breakouts. Still require an ORDER BY tho.
* Fix most stuff
* Hack city??????
* Revert nest-query changes and disable the offset-no-breakout test for now.
* Appease TypeScript
* Test fix 🔧
* Oracle & MySQL test fixes
* Test fixes 🔧
* Remove maxWorkers for FE unit tests
* Add Embedding SDK readme
* Update enterprise/frontend/src/embedding-sdk/docs/README.md
Co-authored-by: Nicolò Pretto <info@npretto.com>
* Update enterprise/frontend/src/embedding-sdk/docs/README.md
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Update enterprise/frontend/src/embedding-sdk/docs/README.md
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Clarify the result from `useQuestionSearch`
* Remove the mentioning of API key
Since we want to drop API key support for embedding SDK
* Add more detail
* Update enterprise/frontend/src/embedding-sdk/docs/README.md
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Update enterprise/frontend/src/embedding-sdk/docs/README.md
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Update enterprise/frontend/src/embedding-sdk/docs/README.md
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Update enterprise/frontend/src/embedding-sdk/docs/README.md
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Update enterprise/frontend/src/embedding-sdk/docs/README.md
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Update enterprise/frontend/src/embedding-sdk/docs/README.md
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Remove private GH repo
* Remove the compatibility table until we've made the first release
* Fix typo
* Add details how to run with locally built SDK
* Correct Metabase Embedding SDK case
* Make the instruction more concise
* Add a disclaimer
* Update config removing deleted property
* Add Oisin suggestion that was missed from rebasing
* Clarify plugin section
* Tweak SDK Readme
* Polish the limitation section
* Update the code sample into 4 sections making it easier to read
* Tweak overview and features section of README
* Add metabase configuration section
* Clean up section
* Replace the doc in the root embedding SDK directory
* Some tweaks and improvements to SDK README
* Tweaks to SDK README: wording, change flow, add section on running MB from master
* Update enterprise/frontend/src/embedding-sdk/README.md
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Feedback section
* Add simple InteractiveQuestion example
* SDK README: Tweak Slack wording
* Standardize fullstops + fix typo
* Clean up formatting
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Oisin Coveney <oisin@metabase.com>
Co-authored-by: Alberto Perdomo <alberto.perdomo@gmail.com>
* Add `collection.effective_ancestors` to search
Fixes#41909
The ultimate goal here is to be able to display a UI that showed the entire (effective) path to each collection.
This was a bit more painful than I'd hoped. The primary issue is that we
tied two things together:
- scoring results
- "serializing" results (really, partially serializing results, because
we do more work to shape the data later)
Why is this a problem? The `top-scorers` function took a transducer and
returned the *serialized* top scorers. Later, when we need to operate
on *all* the top results (in this case, to collect all the collection
IDs and names that we know of), we're looking at the serialized results.
In our case, we needed some data (collection location) that wasn't
included after serialization. We could add it, of course, but it's
frustrating to need to change the API of what data is *returned* from
the endpoint for purely internal purposes (changing the data available
during *processing*).
I ended up pulling serialization out of the scoring function. At some
point down the road, we should go ahead and move the `serialize`
function out of the `search.scoring` namespace entirely - there's really
no reason for it to belong there as far as I can tell.
* temp
* rework DatasetFieldMetadataSidebar to use new form components
* temp
* removing unneeded import
* fixing display name form field
* temp
* more temp
* onChange for form elements
* e2e adjustment
The new expressions `:month-name`, `:quarter-name` and `:day-name`
return the (user localized) names for these units, given the
corresponding number.
For example, 4 yields `"Apr"`, `"Q4"` or `"Wednesday"` respectively.
The `column-extract` drill uses these new expressions, rather than
generating its own `:case` clauses for them.
Note that `:day-of-week` and `:month-of-year` are tricky cases for i18n, but there are tests that exercise them thoroughly.
Part of the follow-up for Extract Column epic #38964.
* Add subTitle to ClickActions
* Add examples to column extractions
* Add test for examples
* Add test for examples
* Use rem helper
* Use JSDoc for todo
* Disambiguate between ColumnExtraction and ColumnExtractionInfo
* Update type of getExample
* Use rem helper
* Notebook combine columns (#41379)
* Add CombineColumns dialog component
* Add combine columns to the expression editor widget
* Add placeholder for empty and space to the input
* Drop the first separator
* Remove empty string from eventual concat
* Add example to Combine Colunmns builder
* Show example in monospace
* Do not collapse spaces in example
* Use monospace in separator input
* Reorganise column flatten helper
* Add back button on combine buttons modal
* Select text when focussing separator input
* Add minimum height to example
* Use that same separator as the last row
* Use ExpressionWidgetHeader instead of hand-rolled header
* Fix type when generating example
* Correctly update default separator when a url column gets set
* Swap url and email checks
* Move column input into separate component
* Use QuerySelector for column select
* WIP: customise dropdown
* Export Input
* Use a button to render the column selector
* Add combine icon
* Use shortcut suggestion shortcut for combine columns
* Allow interacting with buttons inside the popover
* Use longDisplayName for button label
* Add asReturned
* Use asReturned in CombineColumns modal
* Make faux-select button styles match normal input
* Return focus to button when select closes
* Fix type of focus event
* Move ColumnInput component to a separate file
* Only show combine column shortcut on custom column
* Add missing test ids
* Add e2e test for combine column shortcut
* Remove white fill from paths
* Add color to combine icon
* Use function expression, not declaration
* Remove comment
* Move initial default separator to constant
* Remove unused select option helpers
* Add isInteger helper to Lib
* Add case for integer columns
* Add unit tests for util
* Add SuggestionShortcut type to props for Suggestions
* Remove {enter} which broke test
* Do not use !important
* Simplify classname into object-style
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
* Use direct override on Button instead of class
* Fix formatting issue
* Use .select() to select input
* Avoid selecting placeholder text
* Avoid losing focus when window loses focus
* Rename evt to event
* Add monospace variant to Text
* Use monospace variant in Example
* Use monospace font from theme
* Match faux-select label styles to our TextInput label styles
* Remove empty default variant
* Add limit to the test case
* Remove unnecessary within wrapper
* Use rem
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
---------
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
* Remove intercept we're not waiting for
* Use have.text instead of contain
* Use cy.button
* Expand cancel test
* Remove duplicate title
---------
Co-authored-by: Uladzimir Havenchyk <125459446+uladzimirdev@users.noreply.github.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Column extractions are "canned" expressions based on a column's
type. For example, we might extract the weekday from a temporal column,
or the domain from an email or URL column.
This logic already existed inside the `column-extract` drill; this pulls
it out as a top-level concept, since extractions are also being
integrated into the notebook editor apart from drills.
Part of the follow-up for Extract Column epic #38964.
It has a slightly different API shape. We're not using it right this
minute, but we're about to and it seemed nice to separate this out into
its own PR
* Export Input
* Move TextInput overrides to Input
* Remove the :not(:only-child) hack the Schedule component
* Remove the :not(:only-child) hack from DateInput
* Remove the :not(:only-child) hack from FileInput
* Remove the :not(:only-child) hack from Select
* Remove the :not(:only-child) hack from TextArea
* Remove the :not(:only-child) hack from TimeInput
* Reproduce pre-hack margin in Schedule component
* Undo overzealous refactor
* Add persisted info types
* Keep exports sorted alphabetically
* Add boilerplate `perist` API
* Use RTK Query for the `PersistedModels.list`
* Throw for unused `PersistedModels` endpoints
* Update types
* Re-wire PersistedModels `get` requests to use RTK Query
* Fix the endpoint
* Refresh model cache using RTK Query
* Add cache invalidation
* Fix `PersistedModels.api.get` endpoint options
* Remove stray TODO comment
* Explain the difference between the `persisted-info` and the `persisted-model` tags
* Invalidate `persisted-info` list upon refreshing a single model
* Be more consistent with borders and avoid double borders
* Mark the aggregation 'Custom Expression' button as an action
* Remove color from action headers
* Remove bottom border from last child
* Remove top border from first child
* Make section header classes consistent
* Add border to search header
* Use correct color for borders
* Add props to enable borders in AccordionList
* Add borders to column picker elements
* Dates Binned by Week Export Formatting Matches App
Fixes: #41492
Before, the dates binned by week would be formatted as follows in CSV and JSON exports:
`Week 1 - 2024`
But that doesn't match the App's format of a Date Range:
`January 1, 2024 - January 7, 2024`
So, now the exports will apply formatting in this same way.
Note that a second bug (#41616) exists on the Frontend preventing the column formatting being applied in the app. This is
frontend only, and the csv and json exports will match the column formatting settings (eg. abbreviated dates and
alternative separators will be used in the export).
* add test and change default week style
* Week format test change
* also fix incorrect :hour formatting
* [tech debt] Stop using ms/BooleanString and only use ms/BooleanValue
ms/BooleanString will coerce to a string, and we should probably get rid of it and only ever use ms/BooleanValue which will always coerce to a boolean
We want the coercion layer to give us values that are helpful, and true is more helpful than "true".
So we should remove ms/BooleanString and calls to parseBoolean
* Remove ms/BooleanString usage in api/native_query_snippet.clj
* Remove test case related to ms/BooleanString
* Remove the BooleanString schema def as no longer used
* Add RTK Query `revision` API
* Re-wire `Revisions` entity `list` to work with RTK Query
* Re-wire the `Revisions` entity `revert` to work with RTK Query
These functions are implemented with hairy regular expressions, and
it's more user-friendly and future-proof to name those functions in MBQL
rather than baking the `regexextract` and regex into the user's query.
It lets us evolve the regexes in the future if we detect a bug, and it
improves the UX since the user sees a meaningful function instead of
regexextract([My URL Column], "(?<=[@\.])(?!www\.)[^@\.]+(?=\.[^@\.]{1,3}\.[^@\.]+$|\.[^@\.]+$)")
Also refactors the regexes somewhat so that they work for emails as well as URLs, and there's always just one layer of `:regex-match-first`.
Previously this was separated into two steps: URL or email to host, and host to (sub)domain.
Part of the follow-up for Extract Column epic #38964.
* Delete unused endpoint from `PermissionsApi`
* Add group tag helpers
* Define RTK Query 5 main endpoints for `permissionApi` (group)
* Re-wire 5 main api methods in groups entity to work with RTK Query
* Re-wire `clearMember` action to work with RTK Query
* Add `Groups.api.update`
* Update types
* Fix membership failing to update after `CREATE_MEMBERSHIP`
* Make sure `members` is not undefined
* Fix cache invalidation
* Throw for unused Tasks endpoints
* RTK Query `taskApi`
* Wire tasks entity list and get endpoints to use RTK Query
* Fix typo
* Expose `taskApi`
* Rework `TaskModal` to use RTK Query api
* Throw on `Tasks.api.get`
* Rework `TaskApp` to use RTK Query api
* Delete `Tasks` entity
... which never should've been an entity in the first place.
* Delete unused `TaskApi` entry
* Fix next pagination page not being properly disabled
* Do not export `isLastPage` function
* Convert TasksApp to TS
* Use Mantine Tooltip
* Remove stray comment
* Add empty lines for better legibility
* Update the response type
* Define `task_details` type more strictly
* Use the existing `DatabaseId` type
* Provide tags for `list` and `get` endpoints
* Address code review
* Handle error and loading in the `TasksApp`
* Handle error and loading in the `TaskModal`
* Extract the page size into a variable
* Index databases without mutation
* Refactor TaskModal
When we actually run the job to cache models, we select "refreshable
models", which selects `PersistedInfo` where:
- the `state` is something refreshable, and
- the question marked for caching is actually a model, and
- the question hasn't been archived.
However, when we selected `PersistedInfo`s to display to the user, we
didn't apply any conditions. Instead, we included PersistedInfo where
the question was not a model, or where the question had been archived.
To prevent confusion, let's hide these.
Fixes https://github.com/metabase/metabase/issues/41318
The filters for _numeric_ PK and FK filters are slightly different than a general numeric column. That's fine for numeric keys, but with string keys that same set of numeric filters was being returned, and then the ones for strings were removed. That left only `Is` and `Is Not` for a string PK or FK.
This PR allows more flexibility in filtering by string PKs and FKs, by checking the base type of the column and using the right set of filters for the type of key.
Closes#40665.
* Add the pagination type
* Add the pagination helper
* Update types
* Update `user` API
* Update `collection` API
* Update `search` API
* Update `UserListQuery` type
* Explicitly define default pagination inputs
* Remove the pagination helper
* Update and fix types
`limit` and `offset` are actually nullable.
* Separate types (again)
* Fix `PaginationResponse` type
* FeedbackModal for the Embedding Homepage
* add source and change to correct endpoint
* fix tests, adding await and fixing endpoint
* use toEqual to also check for the source
* addresses PR feedback
* button becomes Send when email is written
* make 'comments' optional, trim inputs and refactor tests
* product-feedback endpoint (#40974)
* mvp of the proxy endpoint for the product feedback
* cleanup and fixes
* email :string -> ms/NonBlankString
* revert my solution
* Dan diff file
* makes fields other than source optional
* don't use default value in dev, to not spam the prod endpoint
* change from setting to function for product-feedback-url, as it's not meant to be changed by end users
* changed endpoint as per https://metaboat.slack.com/archives/C010ZSXQY87/p1712918951362979?thread_ts=1704910781.153589&cid=C010ZSXQY87
* fix wrong dismiss reason sent for 'I'm not interested right now'
* Update frontend/src/metabase/home/components/EmbedHomepage/EmbedHomepage.tsx
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* renamed field: feedback -> comment
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* Add provideDashboardListTags and provideDashboardTags
* Keep the utils sorted alphabetically
* Include collection tags in dashboard tags
* Add getDashboard
* Migrate basic CRUD dashboard endpoints to RTK
* Migrate remaining dashboard endpoints to RTK
* Fix save dashboard action
* Remove unused favorite/unfavorite dashboard endpoints
* Fix copy dashboard action
* Move types to types file
* Type dashboard API requests
* Fix get
* Fix typo
* Pass body
* Add a comment
* Support noEvent option to fix a failing test
Add a Github workflow to nag on a PR that changes `metabase.lib.js` without updating `docs/developers-guide/mbql-library-changelog.md` as well.
You can set the `.metabase-lib/no-change` label to override for a change that edits the namespace without materially changing the API.
Not all drivers support regular expressions. Don't return the
Column Extractions for URL -> (sub)domain and email -> domain unless the
database can support the regular expression matches.
Part of the follow-up for Extract Column epic #38964.
This improves our Malli performance substantially! Many of our code paths were calling vanilla `malli.core/validate` or `malli.core/explain`; these redo the (possibly expensive) compilation of the schema into a validator or explainer for every call.
We should use the caching versions in `metabase.util.malli.registry` everywhere, and our runtime performance will be much improved.
However when I started using these cached versions, I found that the memory use was growing out of control. Eventually I tracked this down to `:fn` schemas. Functions are only comparable by pointer equality, so they make poor cache keys. `(fn ...)` or `comp` calls in a schema on a `mu/defn` function **get re-created for every call of the `mu/defn`'d function**! That's a big time sink recompiling the schemas if we're *not* caching, and a huge memory sink if we *are* caching!
This PR pulls every such schema I could find out into a `def` so it uses the same closure and is cacheable. I'd like to automate that in some part of the Malli pipeline, or maybe a linter rule, but I haven't found a good way to do it yet.
Part of #39946.
* Define the types for the `create` user request
* First draft for the create user API using RTK Query
* Export RTK user API
* Define explicit `create` api for the user entity
* Define `updatePassword` mutation
* Wire users entity `resetPasswordManual` to use RTK under the hood
* Use `updatePassword` mutation
* Wire users list calls
* Remove unused `update_password` from services
* Fix the dispatch used to list users
* Dispatch the action type and the payload
* Wire `deactivateUser` to work with RTK
* Wire `reactivateUser` to work with RTK
* Delete unused entries from `UserApi`
* Convert the reducer to use `switch` statement
* Remove unused user `send_invite` endpoint from FE
* Wire `resetPasswordEmail` in users entity to work with RTK
* Provide cache invalidation for the `userApi`
* Add `provideUserListTags` helper
* Pass `email` as a sole argument
* Throw on `get` a user
This endpoint hasn't been used in a long itme.
Removed in https://github.com/metabase/metabase/pull/41334/files#diff-241e0afa11f03a0942eb322aa47695e58496379b0b6d42e6e59ecf6a20fe623cL448
* Use RTK Query to `updateUser`
* Fix wrong syntax
* Use RTK Query to `getUser`
It was used by `.load` and `fetch` after all.
* Export `useUpdateUserMutation`
* Refactor `getRecipientsList`
* Fix response type for the `getUser`
* Pass user id as a sole argument instead of in an object
* Fix users entity `get` method
* Fix the user update password form
* Fix unit tests
* Fix deactivate and reactivate query invocations
See e949812.
* Adjust types according to the feedback
* Order utils alphabetically
* Throw on `delete`
* Revert "Fix unit tests"
This reverts commit 2c329e8364.
* Fix FE unit test mocks
* Rename imports to use plural for consistency
* Fix one last remaining FE unit test
* Add list of most popular functions and aggregations
* Render 'All functions' link on expression dropdown
* Suggest popular functions when no input is given
* Do not return focus to search bar
* Default to not matching any substring
* Open suggestions when the expression editor opens
* Group expressions by group
* Render group titles in dropdown
* Match the expression suggestions height to the design
* Add tests for popular functions
* Convert tests to it-style
* Only return supported popular functions
* Limit most used functions group to 5 items
* Make the suggestion footer keyboard-accessible
* Make a different group for popular aggregations
* Remove padding on suggestion list
* Resize list
* Add test for popular functions in suggest
* Move showMetabaseLinks into args
* Remove unneeded optional chain
* Clarify popup issue in ACE hack
* Use correct target for links
* Reference correct method in comment
* Split up popular expressions
* Remove Set around popular functions/aggregations
* Do not allow interacting with suggestion header
* Do not use _ prefix for variable
* Add verb in test
* Remove unused async in test
* Test if functions are in document
* Move tests for suggestWithFooter to separate file
* Use correct target in link
* Fix typo in key
* Allow picking suggestion when textbox is empty
* Remove unnecessary cast
* Avoid double scroll bars
* Only show link to all functions when editor is empty
* Split up popular functions between expressions and filters
* Fix type of popular variable
* Use the same heuristic for footer as for popular expressions
* Close suggestions when the editor is not focused
* Add missing open prop in test
* Simplify test
* Add selector for expression editor widget
* Fix e2e tests
Fix e2e tests
* Avoid non-interactive clicks from closing the dropdown
* Use custom expression helpers in broken test
* Fix test
* Remove should exist check for selector
The drill-thru is mostly a stub to put an entry in the context menu. Most of the logic for this is built on the FE using the existing functions for consuming and creating expressions.
See [thread](https://metaboat.slack.com/archives/C06P22KS4JH/p1712264174056569) for a lot more context.
This also adds `docs/developer-guide/mbql-library-changelog.md` and documents adding this function to the library as the first change in the API. I'll follow up in another PR with Github automation to expect the changelog to be updated whenever `src/metabase/lib/js.cljs` changes.
Fixes#39978.
* Keep Filter Values for enabled Parameters in Embedding
Fixes: #37914
Previously, we removed locked parameters entirely. This is to prevent leaking potentially sensitive values.
However, for the situation where the same field backs 1 locked or disabled paramter and 1 enabled parameter, we do
still want to send the paramater values, because the enabled parameter implies that the values are permissible to see
in the embed.
So, this change will still remove parameters based on their 'disabled/locked/enabled' status, but will NOT remove the
linked field ids if they're also being used by 'enabled' parameters.
This results in the backend correctly sending necessary parameter values to the embed, where the frontend can then
render the appropriate UI instead of falling back to just text filters.
* test the case where a locked and enabled param share same field
* Address feedback. Added comment to explain classify fn
* Fix add-alias-info matching for nominal field literal refs
* Minor cleanup
* Minimize diff
* A bit more cleanup
* Cumulative aggregations w/ multiple breakouts should reset counts (#2862)
* Update dox
* Do the nesting FIRST before adding alias info.
* Remove unused
* Add help text icon in expression editor suggestions (#41092)
* Add helptext to expression suggestions
* Deduplicate getDatabase calls
* Allow customising popover width
* Split out help text content component
* Render an info icon with the help text
* Remove duplicated defaults for popover
* Explicitly wait for suggestion items in test
* Remove commented out code
* Use React imports directly
* Rename e to evt
* Inline getHelpText when creating suggestions
* Avoid overlapping content
* Remove unnecessary Fragment
* Avoid selecting item when clicking link or button inside element
* Allow shifting popover
* Stop Popover mouse events for bubbling
* Use onMouseDown instead of onMouseDownCapture to avoid clicks inside popover from triggering select
* Add help text to all suggestions that are know mbql functions
* Expression suggestions clean up (#41236)
* Move ExpressionEditorSuggestions to tsx
* Convert ExpressionEditorSuggestions to functional component
* Simplify suggestion match component
* Move ExpressionEditorSuggestions a separate folder
* Remove default export
* Use Popover instead of Tippy for suggestions list
* Fix suggestions tests
* Remove data-ignore-outside-clicks
* Add default for suggestions
* Fix e2e test
* Move helptext wrapper to ExpressionEditorTextfield
* Switch to metabase/ui for Button
* Use arrow function
* connect the example dashboard id setting to the embedding homepage (#41138)
* connect the example dashboard id setting to the embedding homepage
* fix types
* Update frontend/src/metabase/home/components/EmbedHomepage/StaticTabContent.tsx
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* use isNotNull
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* collection and db breadcrumbs
* compact breadcrumbs
* adding navigation keybindings
* recent_views and search changes to always include collection or db name
* removing breadcrumbs, display parent name
* removing unneeded components
* e2e and FE unit tests
* waitFor
* e2e tests for keybindings
* help
* be tests passing
* PR feedback and type adjustments
* unit test adjustment
* Keep the original question name in the save modal where possible
Resolves#41196
* Update tests
* Append "- Modified" to a suggested question name
* Preserve the original question description
* Address reviews - adjust the translation string
* Add E2E reproduction
* Remove extra space
* Improve assertions
* .eslintrc -> .eslintrc.js
* This might work?
* add env for linting css modules class names
* avoid extending postcss-modules/recommended, specify just the used rule instead
* add missing . on comment
* Remove unused .detailName references
* Fix ManageApiKeys and GuiQueryEditor
* Fix SettingText and DashboardControls
* fix missing textUnspaced class
* remove usave of S.headerButton, that's actually never used because we never pass headerLink to ReferenceHeader
* Remove flexNoShrink
---------
Co-authored-by: Oisin Coveney <oisin@metabase.com>
* Fix embed download endpoint
We have added format_rows as a query parameter. In most endpoints this just works fine.
Here, however, the problem is that we use query parameters to pass parameter values. We run some validation over these
provided parameter values (eg. don't allow a user to over ride a 'locked' parameter). Since we have these 2 different
use cases for query parameters, the validation was failing on :format_rows, which isn't expected to exist at all.
Here, I dissoc the :format_rows key prior to this validation step, so normal validation can continue.
* Add test that uses this card embedding endpoint
* Nested queries have limits which can defeat their purpose
Consider a query
```sql
select count(*) from {{#199}}
```
This query should return the number of distinct rows in the query
defined by 199. But it's actually limited by the excel limit of
1048575. And that's because when the value of `{{#199}}` is expanded it
has that limit applied as normal.
```clojure
qp=> (let [card-id 199] ;; use a valid card id for you
(-> {:database 1,
:type :native,
:native {:query "select count(*) from {{ref}}"
:template-tags {:ref {:card-id card-id
:type :card
:name "ref"
:display-name "ref"}}}
:middleware {:disable-max-results? true}}
qp.compile/compile
:query
(metabase.db.query/format-sql )
println))
select
count(*)
from
(
SELECT
"PUBLIC"."ORDERS"."ID" AS "ID",
"PUBLIC"."ORDERS"."TOTAL" AS "TOTAL"
FROM
"PUBLIC"."ORDERS"
LIMIT
1048575
)
```
But we can suppress this limit when substituting a query inside yielding
```sql
select
count(*)
from
(
SELECT
"PUBLIC"."ORDERS"."ID" AS "ID",
"PUBLIC"."ORDERS"."TOTAL" AS "TOTAL"
FROM
"PUBLIC"."ORDERS"
)
```
And this is proper because we want to limit the _outer_ query, not
internal queries.
* Remove limit from test expectation
* stupid trailing space
* another subquery test
* Use helper function to disable limit middleware
* Fix#39138 again
Fixes#40934
Got unfixed somehow in #40578. seems like some file renames caught it off guard
But the long and short is that its possible for a persisted model to end
up in the state "refreshing" (possible an instance restart during
refreshing). The refreshing job doesn't look for these so they become
effectively invisible.
Since the job to refresh them will only run one at a time cluster wide,
any jobs that are in the "refreshing" state when the refresher begins to
refresh are necessarilly stuck (no one else could currently be
refreshing them). So we can just add them to the queue of models to
refresh.
```clojure
(jobs/defjob ^{org.quartz.DisallowConcurrentExecution true ;; <----
:doc "Refresh persisted tables job"}
PersistenceRefresh [job-context]
(refresh-job-fn! job-context))
```
* Ensure ee/oss pathways are taken in tests
Annoying little footgun here. CI does not run with an ee token, so in
order to ensure that test pathway goes through ee version of a
defenterprise we _must_ use the `mt/with-premium-features
{:cache-granular-controls}`, but we also want to ensure that it goes
through the oss version.
So two options, a `doseq` on both features (empty set and the feature
that triggers this). But want a test in the enterprise folder as well to
ensure.
The real trickiness comes from running tests at a repl and CLI. My REPL
always has an ee token in it. My command line always lacks that as
well. So want to be explicit about the token features in effect at test
time.
That's why I'm essentially duplicating tests (ee in ee folder, oss in
regular pathway)
* Restore tabs display for embedded dashboard without title
* Add unit test
* Remove empty file
---------
Co-authored-by: Mahatthana Nomsawadi <mahatthana.n@gmail.com>
* H2 compilation improvements: avoid unneeded casting
* Legacy MBQL schema improvements
* Add additional test for wrap-value-literals
* Simpler version of the fixes from #36068
* Test and lint fixes 🔧
* Pivot QP use MLv2
* Wow
* Test fix 🔧
* Fixes
* Actions should use strings for column names (fix :update-row and :create-row normalization)
* MLv2 schema should check against keys for the other query type
* Ok, have I fixed things?
* More fixes 🔧
* Fix indentation
* Another round of test fixes. 🔧
* Hopefully the last few test fixes 🔧
* We need to test normalization for queries that have keyword keys as well.
* Fix Cljs i18n namespaces
* Sort namespaces
* Only test against H2
* Test fixes 🔧
* Register MBQL clause schemas and test fixes 🔧
* Test fixes and PR feedback
* Test fix
* Remove the normalization tests
* Test fixes 🔧
* Fix kondo
* PR feedback
* Resurrect remove-all-breakouts
* Test fixes 🔧
These type checks are useful when constructing clauses with full MLv2
metadata available, but when converting from legacy the `:field` refs
have only `:base-type`.
If a column has its metadata overridden to eg. coerce a string or
integer into a date, its `:base-type` will remain `:type/Integer` or
`:type/Text` and the `:effective-type` will be `:type/Date` etc.
This change skips the type check while converting from legacy, and
while checking `can-run`. Eventually this should be replaced with
tracking `:effective-type` on refs or using real metdata, instead of
relying on the `:base-type` only.
Fixes#41122.
We need to serialize it for any card, since when a normal question depends on a native question - their result_metadata won't be updated when opening a dashboard, which leads to front-end not understanding how to connect the parameters from a dashboard to a card, and errors during queries.
* Group axes with the correct map shape.
Prior to this, any time axes were all explicitly set :left or :right by the user, the `group-axes-at-once` function
would incorrectly return the map in the following form:
`{:left ["a" "b" "c"]}` which is incorrect. It must instead look like this:
`{"a" :left "b" :left "c" :left}`
This fixes the map shape and permits the user to explicitly set all axes to the left or to the right.
* Add a test to confirm the axis settings are respected and do render
* Wow
* Test fix 🔧
* Fixes
* Actions should use strings for column names (fix :update-row and :create-row normalization)
* MLv2 schema should check against keys for the other query type
* Ok, have I fixed things?
* More fixes 🔧
* Fix indentation
* Another round of test fixes. 🔧
* Hopefully the last few test fixes 🔧
* We need to test normalization for queries that have keyword keys as well.
* Fix Cljs i18n namespaces
* Sort namespaces
* Only test against H2
* Rename `metabase.mbql` to `metabase.legacy-mbql`
* Fix Kondo warnings
* Test fixes 🔧
* Register MBQL clause schemas and test fixes 🔧
* Test fixes and PR feedback
* Test fix
* Remove the normalization tests
* Test fixes 🔧
* Fix kondo
* Fix import
* Another fix 🔧
* Merge
* FIXES
* Add another missing REQUIRE
* More Metric => LegacyMetric renaming
* metabase.models.metric-important-field => metabase.models.legacy-metric-important-field
* Even more renaming
* More renaming
* Even more fixes.
* More renaming
* Wow, even more renaming!!
* Use window functions for SQL drivers for cumulative count and sum
* H2/Snowflake/Redshift implementation
* Test fixes 🔧
* Use SELECT expression positions as the default window function implementation
* Some test fixes.
* Fix bugs in add-alias-info
* MongoDB fixes
* Another pass
* BigQuery should use positions?
* BigQuery almost works 100% now.
* MLv2 fix conversion behavior for expressions with type info
* Add transformations for nesting cumulative aggregations in expressions
* Minor tweaks
* MLv2 should escape aliases using driver/escape-alias (dynamically bind this)
* Kondo fix
* Lint fixes 🔧
* More test fixes and revert test changes
* Update some tests to use correct case for nominal Field literal refs
* Wow! BigQuery is fixed! Remove upgrade-field-literals middleware
* Disable tests for the failing drivers, we can fix in follow-on PRs.
* Enable BigQuery :window-functions
* Extra info
* Add GH issue numbers
* Update more GH issue numbers
* Fix SQL Server!!!! wooo
* I THOUGHT WE WERE DISABLING SPARK SQL!!!!!!!!!!1
* Cleanup to get things ready for review.
* Snowflake + Vertica shouldn't use output column numbers for ORDER BY in OVER
* Code cleanup
* Snowflake + Vertica shouldn't use output column numbers for ORDER BY in OVER
* Code cleanup
* Test fixes 🔧
* Much simpler BigQuery implementation
* Add GH issue context
* Remove commented-out code
* Remove unused add-stage-at-index
* Documentation
* Test fix? 🔧
* Test fix 🔧
* Add new user-local setting `notebook-native-preview-preferences`
* Update user settings on FE
* Use two distinct setting entries
* Update FE types
* More explicit `notebook-native-preview-sidebar-width` definition
* Add selectors
* Initialize QB uiControls based on the BE settings
* Rename QB store entry
`isNativePreviewSidebarOpen` -> `isShowingNotebookNativePreview`
* Rename QB store entry
`nativePreviewSidebarWidth` -> `notebookNativePreviewSidebarWidth`
* Add Redux actions
* Dispatch user settings on top of Redux store
* Fix test title typo
* Add E2E test to make sure sidebar is resizable
* Expand the E2E test to cover persisting settings across sessions
* Be more explicit that we're resizing along x axis
* Store window breakpoint as a variable
* Calculate handle's left position more explicitly
* Make canSelectSection a function
* Use a custom type for actions in the accordionlist
* Add the globalSearch prop to AccordionList
* Improve AccordionList tests
* Enable globalSearch in the AccordionPicker on the FilterColumn and QueryColumn picker
* Assume alwaysVisible for action sections
* Add state for empty search results
* Fix test case
* Allow passing globalSearch to Select
* Set globalSearch in semantic type picker
* Handle search when a section does not have items
* Rename variable to be more descriptive
* Add borders to headings in select
* Add border to back button too
This optional fourth argument will disregard all joins at `position` and
later when suggesting conditions. This is useful when editing a join's
RHS, which wipes out the condition. We don't want to generate a
suggested condition that's based on keys from the RHS of the join we
just edited, or from later joins.
Fixes#40916. Part of #40890.
New endpoint accepts entity and entity ids in form of `database=1&dashboard=2&question=3`. If you don't supply
`&include=overrides`, then it tries to find configs directly referencing supplied entities and updates their
`invalidated_at`. If you supply `&include=overrides`, all the referenced cards are
updated (`report_card.cache_invalidated_at`).
Cache strategies then select the maximum `invalidated_at` in their logic. EE-only.
resolves#40548
* Display sql preview on sizes less than 1280px
* Handle min notebook width using Mantine responsitve style prop
* Display the resizable sidebar on sizes >= 1280px
* Set sidebar width on resize
* Add custom handle
* Simplify window resizing with `useWindowSize` hook
* Use "white" from the theme colors
* Improve sidebar semantics with `aside`
* Define the `NativeQueryPreviewSidebar` as `aside`
* Use Mantine style props instead of CSS module
* Run the E2E test again using regular browser
We removed the nested media query that ReplayIO doesn't support.
Now this test can run using Replay Chromium as all other tests do.
* Add some breathing room between the scrollbar and the resizable sidebar
* MLv2 normalization: add :lib/uuid automatically to options maps if it's missing
* Remove unneeded normalization code in QP
* Fix 🔧
* Ignore invalid metadata providers when passed in from the REST API
* create ui for the full embedding homepage, without any logic (#40251)
* wip ui for the full embedding homepage
* adds missing link + fix keys
* remove done TODO
* Update frontend/src/metabase/home/components/EmbedHomepage/InteractiveTabContent.tsx
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* destructure props in the function arguments
* rename customerFacingAnalyticsDocsUrl to analyticsDocsUrl
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* add new settings for embedding homepage (#40455)
* remove old minimal version
* adds new embedding homepage settings and sets them at the end of the setup
* comment e2e test
* add tests for embedding homepage flags
* more tests
* remove unit tests related to old minimal version
* fixes flags descriptions
* :keyword for embedding-homepage
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
* replace reloadSettings and refreshSettingsList with initializeSettings
* `value => value === true` -> `Boolean`
* toMatchObject -> toEqual for more strict tests
* remove useless undefined check for settings, the component is conditionally rendered based on a setting so it means they're loaded anyway
* fix: use :hidden as we changed the type to :keyword
* specifically check for the `embedding` feature in the tokenFeatures object instead of checking if any feature is active
* remove double \n
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* connect settings to the embed homepage and show it (#40528)
* fix nested a warnings in browser console
* connect settings to the embed homepage and show it
* fix stories
* remove localStorage.clear() as we're not using it in the logic anymore
* dismiss without feedback for the embedding homepage (#40587)
* feat: add dismiss functionality to the embed homepage
* update setup e2e to check for the dismissal of embedding homepage
* use List from metabase/ui (#40638)
* set embedding-secret-key when embedding is enabled (#40813)
* set embedding-secret-key when embedding is enabled
* use "named" setters and getters as suggested by Noah
* Fix tests for ms1 after updating on master (#41069)
---------
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Use window functions for SQL drivers for cumulative count and sum
* H2/Snowflake/Redshift implementation
* Test fixes 🔧
* Use SELECT expression positions as the default window function implementation
* Some test fixes.
* Fix bugs in add-alias-info
* MongoDB fixes
* Another pass
* BigQuery should use positions?
* Disable tests for the failing drivers, we can fix in follow-on PRs.
* Extra info
* Add GH issue numbers
* Update more GH issue numbers
* Fix SQL Server!!!! wooo
* I THOUGHT WE WERE DISABLING SPARK SQL!!!!!!!!!!1
* Cleanup to get things ready for review.
* Snowflake + Vertica shouldn't use output column numbers for ORDER BY in OVER
* Code cleanup
* Test fix? 🔧
Goal is to properly document the interface between MLv2 and the FE. In
particular I want to call out the health and fate of each function. Some
of them only exist to support legacy compatibility, for example, and
that should be documented to discourage new usage and make clear the
conditions for removing the functions.
* Hide info icons when no description exists
* Move defaults for InfoIcon to separate file
* Add open delay to popover
* Remove inverted background from type label
* Fix margin on semantic type label
* Make semantic type not be bold
* Fix unit tests for hidden info icons
* Rename file to constants
* Remove unused showIfEmpty prop altogether
* Fix the height of the expression suggestions that don't have an info icon
* Fix padding on filter modal
* Fix the height on dataref
* Enable not emptyable arguments for is-empty and not-empty clauses
* Add review remarks
* Shorten dataset name so it is usable with some drivers
* Disable test for Athena
* Handle possibly missing base type
* Update comment
* Missing base type field is treated as not emptyable
* Update desugar middleware
Now middleware adds base type to fields with integer id that are missing that first. This ensures correct function of is-empty expansion.
* Update desugar middleware
Cleanup base types necessary for desugaring is-empty
* Update desugar middleware cleanup
* Update test
* Adjust tests
* Remove now redundant e2e test
* Update comment
The first time you zoom in on a bin in a binned query, it works fine.
For example, Orders COUNT by SUBTOTAL auto-binned has a bin for 40-60.
If you zoom in on that, it correctly shows the range 40 to 60 broken
down into smaller bins which are 2.5 units wide. If you then try to
zoom in on one of these smaller bins, say 50-52.5, there are two
problems:
1. The filters from the first zoom (`>= 40` and `< 60`) are still
there, and so are the new ones (`>= 50` and `< 52.5`).
2. The new filter is actually wrong; it uses the big width (20)
from the original bins, not the width of the nested bins (2.5),
so the new filters are `>= 50` and `< 70`.
This PR fixes both issues, so nested bins (1) remove the old filters,
and (2) use the correct width for the nested bins.
This logic was trying to match `[name join-alias]` pairs previously, but
that won't work since the later stage field doesn't have a `:join-alias`
on it. This falls back to matching on only the ID or name, but only if
it's unambiguous.
Fixes#40252.
* Make Noah the codeowner for permissions et al
et al = `integrations` (OSS google auth, LDAP and Slack integrations) + `enterprise` (audit, advanced SSO, advanced config stuff). `enterprise` also has LLM which doesn't really have a codeowner now, but it also doesn't need one.
* Correct the paths and the scopes
* Bump the question schema and update it with the new events
* Enable snowplow tracking on the frontend
* Add E2E test for tracking snowplow events related to native preview sidebar
* Fix question id type in the schema
* Fix overflow static dashboard
* Fix `frame` event's height for interactive dashboards
* Fix E2E tests
* Fix test names to match the correct GitHub issue numbers
* Use `data-element-id` for consistency
* Make the test selector more consistent
* Clarify that backport/no-backport is an internal thing
A few times per month, external contributors ask about the notion doc they can't access and about the labels they are not expected to set. This is not the experience we should give to people who not only take their time to contribute to metabase, but also actually read the PR template.
(slightly duplicated condition in the beginning and the end of the paragraph — "For those employed [...] If you're not employed [...]" — is on purpose)
* Add linebreaks in PR template
Co-authored-by: Tim Macdonald <tim@metabase.com>
---------
Co-authored-by: Tim Macdonald <tim@metabase.com>
* Command Palette UI adjustments
* move search to end of results list
* ellipsis long command palette items, min icon size and showing debounced text
* setting fixed width. adjusting admin result icons and text
* fixing e2e test
* Use MLv2 to determine whether a query can be previewed
* Add E2E repro for #40608
* Expand E2E repro
* Fix test
* Make sure the step is active and visible before offering to preview its query
* Expand E2E test
* Address review comment - use `getNotebookStep` helper
* Do not put js code into page title
* Typecheck
* Add e2e test
* Update e2e/test/scenarios/question/document-title.cy.spec.js
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Add login page illustration (#39299)
* Move Metabot setting to a new place
* Add setting for login page illustration
* fix UI styling
* Fix not able to upload the same file twice
* Update the admin > appearance > conceal metabase copy
* Prepare the component for tests
* Add some unit tests
* Prevent settings not saved sometimes
After setting the value to/from `custom` one of the setting could be
failed to save, I suspect this happens because the race condition from
trying to save the setting values relatively at the same time.
* Make unit tests easier to read
* Add all unit test test cases
* Add basic E2E test
* Show fallback icon for broken images
* Make use of the proper component
* Update the comment to affect the reality
* Add an error message when file size is over 2MB
* Show an error message when the file size > 2MB
* Add the error message tests
* Preserve the preview image aspect-ratio
* Detect corrupted images
* Fix failing unit tests
* Address feedback: Type
* Feedback: Create meaningful variables
* Feedback: Make selector API more sensible
* Prevent race condition
Makes sure we save the settings completely before making other actions
that also change settings otherwise, there could be race conditions.
* Feedback: tighten up a type
* Feedback: Fix inconsistency callback calls
* 2. Add the rest 3 illustration settings (#40487)
* Add the rest 3 illustration settings
* Don't make the X icon feel lonely
* Fix illustration widget saving
* Fix type error
* 3. Update the select UX to match new design (#40491)
* Update the select UX to match new design
* Move info dot next to file upload button
* Simplify IllustrationWidget API
* Update setting title copy
* Clean up test comments
* Feedback: Clarify `any` type
* Feedback: Use shorthand object creation
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Feedback: Use global CSS glass instead of inlining values
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Feedback: Remove unnecessary curly brackets in JSX
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* 4. Connect illustrations to new setting values (#40139)
* Wire landing page illustration to the setting value
* Wire the unsubscribe page to the setting value
* Prepare no rows error to be wired to the setting value
* Wire no question results illustration to the setting value
* Wire no search results illustration to setting value
* Fix error from fixing conflict
* Fix required unnecsseary prop on Image
* Use Image component
* Link dashboard card no results image to the setting value
* Move the no data results to more a generic directory
* Link collection picker no results image to the setting value
* Make illustrations more accessible
* Fix failed tests
* Test unscribe page illustration + test no illustration
* Add landing page tests
* Simplify Cypress selectors
* Refactor: Rename settings to match the new requirements
No question data -> No Data
No search results -> No object
These changes are made so, these 2 settings are generic enough to
include other illustrations later on.
* Add no data illustration test
* Add no object illustration tests
* Fix unsubscribe page background style
* Fix not able to build static viz
* 5. Follow-up on milestone 1 (#40710)
* Update custom background illustration scale behavior
* Remove the need to put an SVG file in static folder
* Fix test from rebasing off master with @testing-library/user-event upgrade
---------
Co-authored-by: Denis Berezin <denis.berezin@metabase.com>
* Send creator emails every week
mod the email by 52 instead of by 12 and use the current week as an
anchor
```clojure
user=> (require '[java-time.api :as t])
nil
user=> (let [every-day (take 1000 (t/iterate t/plus (t/local-date 2024 1 1) (t/days 1)))
f (fn [d]
(let [wf (java.time.temporal.WeekFields/of (java.util.Locale/getDefault))]
(.get d (.weekOfWeekBasedYear wf))))]
(sort (keys (frequencies (map f every-day)))))
(1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
40 41 42 43 44 45 46 47 48 49 50 51 52)
```
```clojure
user=> (metabase.util.cron/cron-string->schedule-map "0 0 2 ? * 7")
{:schedule_minute 0,
:schedule_day "sat",
:schedule_frame nil,
:schedule_hour 2,
:schedule_type "weekly"}
```
* fix typo in creator sentiment email, clean up a bit
typo: had `:verison` instead of `version`.
And while I'm in there, make the payload all in one file and have the
email namespace json->bytes->b64-encode so the data is all in one place
and the setting that allows it to be present or not is co-located.
* Move all notebook containers styles to one place
* Make NotebookStep wide again
* Tweak sidebar width and breakpoints
* Remove top margin from NotebookSteps
* Stop line wrapping in an ace editor
* Uniform width between step header and step body
* Remove stray styles
* Scroll content independently
* Use named export
* Use Mantine button for "Visualize"
* Convert notebook step width to rem
* Adjust the position of the step close button
* Adjust notebook step max-width to 1200px
* Remove the `cx` where not needed
* Use HTML 5 `aside` element for the sidebar
* Use Mantine responsive styles
* Fix E2E test
* Add repro test for a model that uses another model and both have custom column with the same name
* Skip test
* Update e2e/test/scenarios/models/reproductions/39150-model-with-same-custom-column-name.cy.spec.js
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Update e2e/test/scenarios/models/reproductions/39150-model-with-same-custom-column-name.cy.spec.js
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Convert to ts and add question
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
* Migrated .PopoverContainer, .PopoverBody
* Make global CSS more prominent
* Migrate 2 classes
* Remove unused class
* Migrate .PopoverBodyWithArrow
* Remove unused class
* Migrate .PopoverBodyMarginBottom
* Remove unused class
* Migrate .PopoverFooter
* Remove unused class
* Preserve classes order for easier review
* Fix unit tests
* Add E2E test
* Move `getDatasetQueryParams` to `data-fetching.js`
* Use `null` as a parameter value when missing
* Fix missing default `datasetQuery`
... among other things.
The order of arguments was wrong here, causing the default
implementation of `post-process-collection-childre` to be called. The
implementation for `:card` ends up removing the `:dataset_query`
entirely, which is what was intended.
* Cards with Results Should Show up in Subscriptions
Fixes: #40726
This PR fixes a bug where setting 'Hide thie card if there are no results' in a Dashcard's viz-settings would hide the
card even when there were results.
* adjust test to show that the setting is respected
Before, the setting would always hide the card. Now, the card is only hidden when there are no results. This test sets
up 4 cards: 2 are normal, 2 have no results, but 1 has the setting to hide empty. THerefore, we should expect to see 3 results.
* upgrades @testing-library/user-event to v14, updates test to accomidate events always returning promises and other breaking API changes
* Fixing up broken suites. Half way
* rest of tests
* green plz
* Fix new entries of userEvent usage
* unit test adjustments
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Co-authored-by: Uladzimir Havenchyk <uladzimir.dev@gmail.com>
* Make Formatting on Exports Optional
Formatting on exports can be turned off via a middleware key:
`:format-export?` which will control if the 'app style' formatting will be applied to the exported results.
`:format-export? false` will return the file as in previous Metabase versions.
* format_export's default value provided in the QP middleware
Should still default to formatted for v. 49, as the formatting is intended as a feature. This default value should be
provided in a central place, so is added to the middleware and is therefore not necessary for the API to provide a default.
* weird indentation
* Add format_export param to dashboard dashcard query endpoint
* Add format_export option to embedded dashboard downloads endpoint
* Move default true value to the streaming-results-writer impls
* Default true but for real this time
* Add format_export to the public Question endpoint
* Add test to card api
* Dashboard exports endpoint test
* Add test to dashboard embeds
Test still fails and I don't know why yet
* the dangers of println debugging....
Well, I was passing a string false the whole time. Turns out the problem was me
* Public Card downloads endpoint now has format_export and test
* export_format also works for adhoc queries using api/dataset
* Let defendpoint do the boolean parsing
* Embed endpoint should now work too.
* Unify format-rows and format-export middlewares
* Sneaky endpoint!
* In xlsx, always parse temporal strings so that viz-settings formatting is applied
* Snuck in an incomplete change by accident. removed it.
* Can now specify format/no format on subscriptions
* Fix and move migration
* default format rows to true in attachments
* Ok, I think that should actually work for subscriptions.
* Change endpoints to use format_rows instead of confusing format_export
* format_rows everywhere
* Fix more test failures
* redef'd function had wrong signature
* Make format export optional (FE part) (#40643)
* Refactor download URL builder code
Need to handle query `params` and request `body` separately
* Export `useHover` from `metabase/ui`
* Add unformatted export for dashboard cards
* Add unformatted export for saved questions
* Fix POST body encoding
* Add unformatted export for ad-hoc questions
* Ensure `format_export` is always false for Excel
* Add unformatted export to public questions
* Add unformatted export to embedded questions and dashboards
* Fix behavior for PNG format
* Add tests for `QueryDownloadPopover`
* Fix e2e downloads helper (allow URL query params)
* Rework alt key handling
* Add e2e test
* Switch to `format_rows` instead of `format_export`
* Add unformatted export option to subscriptions
* Fix `isHoldingAltKey` initial state
* Rework e2e test to use Total column
* Fix initial checkbox state for subscriptions
* Use "Alt" in tooltip label for Windows/Linux
* Show the subscriptions toggle only for CSV attachments
---------
Co-authored-by: adam-james <21064735+adam-james-v@users.noreply.github.com>
* Address review feedback
---------
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
* Fix warning about passing a ref to a function component
- this didn't work and fortunately was not used (anymore) at all
* Move comment where it belongs
* Fix warning that functions cannot be react children
* Parse wildcards in native queries
Add query_field.direct_reference
* Do away with old Model refs in Card model test
* Do not show stale cards with a select *
* Upgrade node to 18.20
* Trigger CI
* Try 20 again
* Revert "Try 20 again"
This reverts commit 8f4b39e7a6.
* Revert change
* Add nvmrc to the frontend_sources list
Fixes#40328
[Our documentation](https://www.metabase.com/docs/latest/people-and-groups/managing#group-managers) states that:
> Group managers can:
>
> - View all people in the Admin settings > People tab.
This fixes enforcement to be aligned with the documentation. This behavior makes sense, because as the docs also state, Group Managers should be allowed to *add* people to the group they manage, which they can only do if they can see those people!
Initially, I also removed a faulty test, which was described as:
```
;; Non-segmented users are allowed to ask for a list of all of the users in the Metabase instance. Pulse email lists
;; are an example usage of this. Segmented users should not have that ability. Instead they should only see
;; themselves. This test checks that GET /api/user for a segmented user only returns themselves
```
but actually failed to test this in a relevant way (because it was testing the `GET /api/user` endpoint rather than the `GET /api/user/recipients` endpoint). The test had continued to pass only because the user was the only member of the group they managed.
I initially thought this behavior wasn't desired, but as it turns out, it *is* in fact documented behavior to disallow sandboxed users from seeing any email suggestions. Investigating, I found that a bug was allowing sandboxed users to see all users on the `/api/user/recipients` endpoint if the user-visibility setting was set to `:all`.
Thus, the second commit fixes the bug and re-adds the (fixed) test. A sandboxed user now only sees their own user when hitting `/api/user/recipients`.
* Trigger the query limit change on blur
* Use named export
* Add a unit test
* Fix E2E tests
* Fix unit tests
* Additional E2E test fix
* Address review comments
* Revert "Remove Models from Browse data (#40108)"
This reverts commit 45a2e8689f.
* Make rtl friendly
* Add 'no description' for blank-description models
* Fix vertical position of collection header toggle
* Move FixedSizeIcon into metabase/ui and increase height of empty state
* Move FixedSizeIcon back into BrowseModels.styled.tsx
* Localize 'No description'
* Add comment
* Fix type issue
* Use expression fix on both pk-ref and value-ref
The gist: inside the query expressions are referred to as [:expression
"foo"]. But as a nested query, it's just another field, and the fact
that it was an expression is irrelevant and should not leak from that
stage. So "downstream" should just be [:field "foo" base-type].
Oddly, if you just fix one field or the other, the query would be
"valid", would log an error, but no valid values would be returned.
It seems like the invalid expression reference was just dropped so we
only selected a single column and those were removed (we run `(filter
valid-tuples?)` as we only want valid key/value pairs).
So let's add a `(> (count values) 0)` which was failing previously.
* Fix indexing error
When updating the model index, it diffs the existing values vs the new
values and deletes appropriate and adds the new values. But it was using
the wrong column when updating the model index values. It used `pk_ref`
but that table uses `model_pk`. This led to the following errors on
stats:
```sql
select pk_ref, value_ref, state, error from model_index where state = 'error'
pk_ref,value_ref,state,error
"[""field"",563028,null]","[""field"",563018,null]",error,"ERROR: column ""pk_ref"" does not exist Position: 68"
"[""field"",534030,null]","[""field"",534037,null]",error,"ERROR: column ""pk_ref"" does not exist Position: 68"
"[""field"",545945,null]","[""field"",545948,null]",error,"ERROR: column ""pk_ref"" does not exist Position: 68"
```
Now those errors will be fixed and the model indexes will correctly
track the values
* Bump indexing threshold to 25,000
Had been limited to 5,000 rows of model indexing. This was a made up
number as we went into this endeavor. But it's proving to be far too
small. Bumping to 25,000
* Parititon deletions and inserts
With a higher limit need to ensure that we don't send too large of a
query.
```clojure
model-index=> (t2/insert! ModelIndexValue
(map (fn [id]
{:name (str (gensym "value"))
:model_pk id
:model_index_id 1})
(range 25000)))
Execution error (PSQLException) at org.postgresql.jdbc.PgPreparedStatement/<init> (PgPreparedStatement.java:105).
PreparedStatement can have at most 65,535 parameters. Please consider using arrays, or splitting the query in several ones, or using COPY. Given query has 75,000 parameters
```
* move comment and fix test failure explanation
comment should be next to the number? check. and also need to use
`mc/explain` which returns info rather than `mc/validate` which returns
a bool:
```clojure
;; bad
model-index-test=> (let [values [[1 "foo"] [2 "foo"] ["foo" "foo"]]]
(-> (mc/validate [:sequential [:tuple number? string?]] values)
(me/humanize)))
nil
;; good
model-index-test=> (let [values [[1 "foo"] [2 "foo"] ["foo" "foo"]]]
(-> (mc/explain [:sequential [:tuple number? string?]] values)
(me/humanize)))
[nil nil [["should be a number"]]]
```
* Split from #40146: only the filter changes
* Don't include format-rows changes here.
* Test fixes 🔧
* Test fix 🔧
* Misc test fixes 🔧
* Test fix 🔧
* Tweak SQL Server fix
They were showing "break out by" and quick filters, when they should be
showing "break out by" and "See these Orders".
These are the `pivot` and `underlying-records` drills, internally.
Fixes#40174.
* Show SQL in a notebook sidebar [PoC/task-1] (#40279)
* Display SQL in the notebook sidebar
* Remove unused modal
* Rename the component to `SQLPreviewSidebar`
* Hide the close button in a SQL sidebar
* Rename the component to `NativeQueryPreviewSidebar`
* Toggle a native query sidebar preview [PoC/task-2] (#40327)
* Use Mantine components where possible
* Rename the button to `ToggleNativeQueryPreview`
* Toggle the native query preview sidebar on and off
* Update the button tooltip based on the sidebar state
* Shorten var names
* Read the state directly from the store
* Re-enable the button that converts a query into its native counterpart [PoC/task-3] (#40362)
* Re-enable the button that converts a query into its native counterpart
* Use Mantine button
* Remove references to the modal
* Fix tests
* Use RTKQuery for API calls in the native preview sidebar [PoC/extras] (#40430)
* Use RTKQuery for API calls
* Fetch question inside the component
* Export named component
* Add type guard
* Use RTK query for the native dataset endpoint
* Improve semantics around error message
* Improve native sidebar basic styling [PoC/task-4] (#40463)
* Refactor sidebar to use `AceEditor`
* Fix the query display condition
* Style the code as in the native query editor
* Style the toggle icon
* Make native preview usable on small screens
* Cover "Show SQL/native query in a sidebar" with E2E tests [PoC/task-5] (#40540)
* Fix query formatting for Mongo
* Add initial smoke tests
* Cover "converting a question to SQL" scenarios
* Cover "converting a question to a native query" scenarios
* Add test that covers behavior on small screens
* Adjust the viewport width to counter CI's behavior
* Revert "Adjust the viewport width to counter CI's behavior"
This reverts commit 370a76b621.
* Add E2E repro for #38181
* Add repro for #40557
* Try a workaround for the failing test in CI
* Show empty sidebar if query has no data source
* Add E2E coverage for an empty sidebar state
* Sort official collections first in API endpoints
For both `/api/collection/tree` and `/api/collection/items`, sort
collections with official collections coming first.
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Fixes#40383
If a native query parameter is set up with a string widget type that
supports a `case-sensitive` option, and that parameter is tied to a
dashboard filter that did not support that option, the option was still
leaking into the filter as an option where options were not expected.
For instance the native query parameter used `string/contains` and the
dashboard used `string/=`, the QP would try to build a filter like `[:=
field value {:case-sensitive false}]` which is an illegal filter.
* In #37160 we added a migration to convert all datetime columns to
timestamp with time zone by using a fixed list of what to convert. but
it turns out some customer instances has a different list. So this PR
removed the fixed list and use a query to get a list of all datetime
columns.
* Remove unused marginalia aliases
* Upgrade deps.edn
From `neil deps upgrade`
* Un-bump jetty-server
* Downgrade H2 due to many problems
* Dep bumping for Athena
* Dep bumps for BigQuery
* Dep bumps for Presto
* Dep bumps for Redshift
* Dep bumps for Snowflake
* Dep bumps for SQLite
* Upgrade to Mongo 5
* Bump next.jdbc
* Funnel Chart Static-Viz should handle row data more robustly
The row data coming into the funnel chart render method can evidently have a few slightly different shapes, and may
need to be 'coordinated' with the :funnel.rows key from viz-settings (names and order are specified there).
This is a WIP to make the method more robust to different row data shapes. For example:
[[1 100] [2 200]] should work as well as [["A" 100] ["B" 200]] and correctly handle the viz-settings
WIP because I need to add a test or two, try to get a handle on the exact schema that is allowed for 'row shape' and
the funnel.row viz key, and see if there are failing cases that I haven't considered yet.
* Funnel rows should now work more effectively with possible raw-rows
* Add a test that checks the success of the render and correct order
* Update src/metabase/pulse/render/body.clj
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
* Update src/metabase/pulse/render/body.clj
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
* Addressing feedback from the review
- renamed build-funnel-rows to funnel-rows
- used 'funnel-viz' as the binding for the {:key "asdf" :name "asdf" :enabled true} maps from the viz settings
- added docstring to the function to try clarify its 2 branches (keys vs indices on the rows)
---------
Co-authored-by: Chris Truter <crisptrutski@users.noreply.github.com>
* Trend Chart Viz Should Use Proper Card execute in Rendering
WIP
Fixes: #39854
The trend chart follows a code path shared with multi-series static-viz. This means there was an implicit expectation
that a dashcard is associated with the card at all times. This isn't true for alerts, so the code is fixed to remove
the failure.
* Add a test
* add missed :breakout that actually made the test work
oops
* Adding comments to try clarify what's going on
* sort expression functions by use case
* custom expression edits
* formatting cleanup
* fix link
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* address comments
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Often slo won't be setup, but we should still delete the session
* test that we delete the session even when "/handle_slo" is not called
* cleanup test
* take the cookie from the session, not as a parameter
* /api/collection/:id/items gets `here` and `below`
During postprocessing of `collection` children retrieved by
`/api/collection/:id/items`, annotate each of the children with `below`
and `here` keys representing the presence of questions, models, or
collections at this level of the hierarchy or below it.
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
Fixes: #40306
Our datetime formatter relies on the `java-time.api`, for which there are many different, sometimes confusing, formatter patterns: https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatterBuilder.html#appendPattern-java.lang.String-
In this case, 'YYYY' is a week-of-year style year, which calculates which week a date falls into before returning the
year. Sometimes days near the start/end of a year will fall into a week in the wrong year.
For example, apparently 2023-12-31 falls into the 1st week of 2024, which probably not the year you'd expect to
see. What we probably do want is 'yyyy' which calculates what day of the year the date is and then returns the year
based off of that instead of the week number.
For an explanation, you can check out this SO answer: https://stackoverflow.com/a/46395342 provides an explanation.
* Wow
* Test fix 🔧
* Fixes
* Actions should use strings for column names (fix :update-row and :create-row normalization)
* MLv2 schema should check against keys for the other query type
* Ok, have I fixed things?
* More fixes 🔧
* Fix indentation
* Another round of test fixes. 🔧
* Hopefully the last few test fixes 🔧
* We need to test normalization for queries that have keyword keys as well.
* Fix Cljs i18n namespaces
* Sort namespaces
* Only test against H2
* Rename `metabase.mbql` to `metabase.legacy-mbql`
* Fix Kondo warnings
* Test fixes 🔧
* Register MBQL clause schemas and test fixes 🔧
* Test fixes and PR feedback
* Test fix
* Remove the normalization tests
* Test fixes 🔧
* Fix kondo
* Fix import
* Another fix 🔧
* Merge
* FIXES
* Add another missing REQUIRE
* E2E support for CRUD + running pMBQL queries; new MLv2 normalization
* More work on improved normalization
* Update schema references
* Test fixes 🔧
* Move the MBQL parameters schema
* Performance improvements
* Improved pivot code etc.
* Clean namespaces
* Fixes 🔧
* Wow
* Test fix 🔧
* Fixes
* Actions should use strings for column names (fix :update-row and :create-row normalization)
* MLv2 schema should check against keys for the other query type
* Ok, have I fixed things?
* More fixes 🔧
* Fix indentation
* Another round of test fixes. 🔧
* Split pivot stuff off
* Hopefully the last few test fixes 🔧
* We need to test normalization for queries that have keyword keys as well.
* Fix Cljs i18n namespaces
* Sort namespaces
* Only test against H2
* Test fixes 🔧
* Register MBQL clause schemas and test fixes 🔧
* Test fixes and PR feedback
* Test fix
* Remove the normalization tests
* Test fixes 🔧
* Fix kondo
* Test fix
* Another fix
* Test fix 🔧
* decrypt and read values from airgap token
- Adds notion of "AirgapToken"
- Adds max-users and company optional fields to `TokenStatus`
- Adds branch to `fetch-token-status*` for handling airgapped tokens
* Add support for handling airgap tokens
- validate airgap token valid-thru date
* enforce user creation limits using airgap token
* fix airgap token reads + fill in the token data
* in oss mode don't try to read the airgap_ token
* use airgap-token? helper
* tighten the user-creation logic
- make the check correct
- account for archived users
Co-authored-by: John Swanson <john.swanson@metabase.com>
* check airgap user count
setting premium-embedding-token
app startup
* move decryption code into ee namespace
* add some tests for token decryption
* fix tests and add tests
* move ee features into ee tests
* add typehint + warn on reflection
* add a check for missing public key resource
* respond to review
- add docstring to AirgapToken
- remove outdated comment
- fix off-by-1 error
* valid-now? takes a TokenStatus instead of a :map
* revert usage of mt/with-temporary-setting-values
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
* wip
* refreshes list through tag invalidation rather than prop drilling refresh list fn
* invalidate count, added invalidation utils, general clean up
* custom fetcher that makes use of our api service
* reorganizes files and impls util to createEntityApis more easily
* documents different approaches
* moves main file
* wip
* stash
* clean up
* further clean up
* type annotations clean up
* fixes api key unit test
* pull out api keys feature from branch
* moves files core rtkquery files into metabase/api
* replace trivial loginHistory entity
* export hook too
* fix rebase
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
In a working meeting, some prompt adjustments were tried. This change felt like the best incremental improvement,
considering feedback from users on stats.
We attempted to guide the LLM to return descriptions that:
- don't state the chart type as if its *very* important (we can see what it is already, so its not that important)
- do call out caveats and important specifics as implied by the question's filters, aggregations, and breakouts
- are descriptive but not verbose
- try not to state caveats too strongly
* Use Mantine Flex for the NotebookContainer
* Rename the component to `NotebookContainer`
* Convert to TS
* Explain the delay
* Use regular div
* Destructure the import type
* Simplify styling
* Shorten the prop name to `isOpen`
Fixes: #39814
Our static-viz renders an error message into subscription emails if the render failed, but it provides no further
details as to what caused the failure.
This PR adds a .txt attachment in the case of a card render or card query failure. The contents of the text file are a
short message about the problem with the card's name, the visualization_settings that the card was rendered with, and
the stacktrace.
This should be helpful if a user is reporting a render error as a bug since this can be attached or copy/pasted into
the bug report.
The attachment was used so that the actualy body of the email remains clean/legible.
* do not pass null values to link url
* update null check to include query builder nulls replaced with "(empty")
* add test to verify behavior
* replace magic string with constant
* fix breakout in cypress spec
This issue was previously punted, because the schema for aggregation
clauses allowed the tag to be `:any`. Now it's a short list of permitted
arithmetic expressions, but that list is too short and excludes things
like `:round`.
This change searches for an aggregation clause transitively in the
expression, and so allows any arithmetic expression.
* Move search-related snowplow tests to a separate file
* Update test names
* Move filters-related search tests to a separate file
* Extract search helpers into a separate file
* Revert test rename
* Export named `NativeCodePanel`
* Export named `NativeQueryModal`
* Rename `NativeQueryModal` to `NativeQueryPreview`
* Do not use styled components in the `NativeQueryPreview`
There were two problems here:
- one endpoint was sorting in the database, but not putting 'analytics'
last.
- one endpoint was sorting in the database, but then sorting again in
clojure, which had subtly different behavior for special characters
Fixes#39965
* Use proper :display when using 'zoom' in Dashboard
Fixes: #38307
WIP, I want to change at a different call site so that defaults are still used in case there is no :display set.
* Lock Display so that drill through doesn't overwrite it with defaults
* Lock display only when drill action coming from a dashboard
* Use isDashboard properly
* Add test to confirm Display is locked from Dashboards
* minor changes to the test
* Lock Display specifically for zoom-in-binning and timeseries
* Fix wrong string quotes
* Try a simple change to see if the e2e test will pass in CI
* Fix test
* Minor refactor
* Wait for dashcard queries in e2e test
* Wait for loading spinner to be gone
* Fix type errors
* Fix incorrect dashboard ID used in test
---------
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
* Add the initial test for the empty app db
* Add reproduction for #25142
* Add test that covers only saved questions
* Cover scenarios with a table as a source
* Add tests that cover nested questions and models
* Add test for simple model's source table
* Remove intercepted schema routes
* Improve assertion
* Fix the test failing on enterprise instance
* Add E2E reproduction for #40223
* Remove superfluous code block
* Avoid casting
* upgrade to react 17
* fix popovers
* upgrade formik to fix types
* fix sso nested form failures
* fix WindowModal failing unit test
* update RowChart test to accommodate changes to event pooling in react 17
* updates input blur change components to use layout effect to ensure input ref is available at the time of unmount
* fixes failing map visualization e2e test
* Wow
* Test fix 🔧
* Fixes
* Actions should use strings for column names (fix :update-row and :create-row normalization)
* MLv2 schema should check against keys for the other query type
* Ok, have I fixed things?
* More fixes 🔧
* Fix indentation
* Another round of test fixes. 🔧
* Hopefully the last few test fixes 🔧
* We need to test normalization for queries that have keyword keys as well.
* Fix Cljs i18n namespaces
* Sort namespaces
* Only test against H2
* Test fixes 🔧
* Register MBQL clause schemas and test fixes 🔧
* Test fixes and PR feedback
* Test fix
* Fix duplicate native column nesting
* Try wrapping source-query in WITH statement
* Fix tests
* Fix tests
* Only wrap in with when needed
* More fixes
* Update names and comments
* Address PR review
* Bridge out of sync collection information between the card and the state
* Account for models
* Use existing helpers
* Create virtual schema if it doesn't exist yet
* Add E2E reproduction for #39812
* Create virtual schema if it doesn't exist yet
* Create virtual schema if it doesn't exist yet
* Create virtual schema if it doesn't exist yet
* Expand the reproduction to include the nested question scenario
* Improve assertions
* Convert E2E repro to TS
* Fix tests
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Eg. for a date column without a time, don't offer "Hour of day" as a
choice. For a time column without the date, only offer "Hour of day".
Fixes#40125.
* Fix multi-schema data selector header styling
- Makes spacing equal on both sides of the divider
- Changes the divider from the hyphen to the forward slash
* Change the breadcrumb divider in the Table Metadata section
* SQL Lineage: Create the QueryField model
This links Cards (with native queries) to Fields, showing us which
queries use which Fields.
QueryFields are automatically maintained as part of the Card lifecycle
Cards with stale queries (powered by QueryField info) are exposed via the `card?f=stale` API
* some clarifications around subscriptions
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* Show selected table in the data selector
Fixes#39807.
* Build schema information when table is selected
* Remove now unused method
* Update only the selected schema id information
* Obtain schema id from the state
* Introduce multiple schemas to the writable database
* Add E2E reproduction for #39807
* Fix unit test
* Fetch schema if needed according to the new condition
* Repro #29943 - Wrong field highlighted when editing model metadata if custom column added to left of column
* Refactor assertions
* Reuse getHeaderCell
This matches how models are visualized, and how the parameters are
attached to them in the first place - an extra ad-hoc query is wrapped
around the model.
Because of that wrapping, a parameter will be targeting a model's field
by name rather than by ID, but that parameter cannot be applied directly
on the model's last stage.
Fixes#39940.
* move log-enabled? into premium-features
* only `record-views!` when `log-enabled?`
* test that view-log entries arent created in oss
* simplify test
* fix new test fails: we don't log views in OSS.
* fix 3 more view logging tests (they dont do anything in oss)
* fix dashboard-read-ee-test
- check for `log-enabled?` not `ee-available?`
* fix the rest of the tests
- Only check for view logs when (premium-features/log-enabled?) is falsey
* adjusting e2e test
---------
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
* Detect the use of unsupported features in the expression editor
* Use an object to pass parameters to resolve
* Handle clauses that do not require feature
* Use postgres-12 for testing datetime functions
* Convert QueryVisualization to a functional component
* Add a custom message after loading for a while
* Add test for slow database message
* Loop through different custom messages
* Use cy.clock to test timeouts
* Make getLoadingMessage a function
This includes some generative testing, since that felt like
a great way to try all kinds of permutations of swapping clauses.
This also patches `metabase.types/assignable?` to memoize it.
Deeply nested `assignable?` calls made 100 tests of swapping
filters take 1600ms rather than the 50-60ms of the other clauses.
I think this is safe and reasonable on memory to cache forever,
but speak up if you're concerned. There's perhaps O(60) type
keywords, so `type X type -> Boolean` is a small space impact.
Fixes#39215.
* Keep fetch_source_query.clj
* Copy fetch_source_query.clj into fetch_source_query_legacy.clj
* Set back fetch_source_query.clj file
* Convert QP to MLv2, part 1
* Keep legacy implementation around for time being
* Port thru convert-to-legacy
* expand-macros is converted to legacy
* Test fixes 🔧
* Test fixes and better keys
* Test fixes
* Test fixes 🔧
* More test fixes 🔧
* Test fix? 🔧
* Wow, maybe I finally fixed everything 🔧
* Last few test fixes 🔧
* PR feedback; lib.walk should support splicing in multiple replacement stages
* Update fetch-source-query to use improved lib.walk
* Have I fixed everything now? 🔧
* Remove stray println
* Another MongoDB fix 🔧
* Convert QP to MLv2 Part 2
* Remove most of the fetch-source-query-legacy stuff
* Remove the changes to expand-macros for now, we can do that in a follow-on PR.
* Test fix
* Test fix
* TEST FIXES?! 🔧
* Kondo & test fixes
* Oops, make sure we have the updated tests
* Fixes 🔧
* Test fixes 🔧
* Convert resolve-source-table
* Appease Kondo
* Unparallelize the tests
* Another test that shouldn't be parallel
* Make tests more robust
* Test fixes 🔧
* Update expression-parts to satisfy filter picker
* Address review remarks and implement to-range for quarter
* Add coercion check to extensibility condition
* Make coerce-to-timestamp return nil for uncoercible arg value
* first move to a db api
* mdb.spec into mdb
* mdb.u/isa -> mdb.query/isa
* mdb.u/qualify -> mdb.query/qualify
* last of mbql.u -> mbql.query and 🔫mbql.u🎇
* remove last of the mdb.connection/db-type and use from mdb/db-type
* remove the last non-cmd usages of mdb.connection
had some circularity in
models.interface requires mdb/db-type for sql flavored "now"
mdb requires db/setup
which requires db/custom_migrations to ensure they are on cp
custom migration requires json functions in models.interface
so had to do a dynamic require for the db-type
* get db-type from mdb not mdb.env
* invoke `(mdb/app-db)` and not pass a var. lots of things fail
* mdb.setup/migrate! into mdb with potemkin
* remove get-connection. call .getConnection on the app-db
perhaps invites clumsy usage outside of `with-open` if it's a top level
function
* namespace docstrings
* last few and grab db.env/db-file into api ns
* lint cleanup
* addresses PR comments
;; A "module" is any `metabase.x` namespace in `src`.
:metabase/ns-module-checker
{:level:warning
;; Map of module name => the "core" external public-facing API namespace(s). You have three options here:
;;
;; 1. Special sentinel value `:any` means means this module does not (yet) have external public-facing API
;; namespace(s). This is mostly a temporary placeholder until we go in and create module namespaces, which means
;; you should go create one.
;;
;; 2. A set of namespace symbols. All namespaces in other modules will only be allowed ;; to use namespaces from
;; this set. Ideally this set should only have one namespace, but restricting it to a set of several is still
;; better than `:any`.
;;
;; 3. `nil` or not listed here -- we default to assuming there is one API namespace that matches the name of the
;; module itself, e.g. the module namespace for `metabase.setup` would be `metabase.setup`.
;;
;; `nil` or an empty set Otherwise this should be a set of namespace symbols.
;;
;; PRO TIP: Check out the `dev.deps-graph` namespace for helpful tools to see where a module is used externally.
:api-namespaces
{metabase.actions#{metabase.actions.core}
metabase.analytics#{metabase.analytics.prometheus
metabase.analytics.snowplow
metabase.analytics.stats}; TODO -- consolidate these into a real API namespace.
metabase.analyze#{metabase.analyze}
metabase.api#{metabase.api.common
metabase.api.dataset
metabase.api.permission-graph
metabase.api.routes}; TODO -- consolidate these into a real API namespace. I think the `*current-user*` type stuff might need to be moved into a separate module.
metabase.async#{metabase.async.streaming-response
metabase.async.streaming-response.thread-pool
metabase.async.util}; TODO -- consolidate these into a real API namespace.
metabase.bootstrap#{metabase.bootstrap}
metabase.cmd#{}; there are no namespaces here since you shouldn't be using this in any other module.
metabase.config#{metabase.config}
metabase.core#{metabase.core.initialization-status}; TODO -- only namespace used outside of EE, this probably belongs in `metabase.server` anyway since that's the only place it's used.
metabase.db#{metabase.db
metabase.db.metadata-queries; TODO this should probably be separate from metabase.db
metabase.db.query; TODO this is mostly util stuff like `metabase.db.query/query` that we don't even need anymore.
metabase.db.setup}; TODO these are only calling `metabase.db.setup/setup-db!` and there's a slightly different version in `metabase.db`
metabase.driver:any; TODO -- 19 namespaces!!!! CRY
metabase.email#{metabase.email
metabase.email.messages}; TODO -- consolidate these into a real API namespace.
metabase.legacy-mbql.util}; TODO -- consolidate these into a real API namespace.
metabase.lib:any; TODO -- :cry: 34 externally referenced namespaces, but maybe half of them are schema namespaces which technically don't need to be required.
metabase.logger#{metabase.logger}
metabase.metabot#{metabase.metabot}
metabase.models:any; TODO -- scream, 62 namespaces used elsewhere, but to be fair a lot of these don't *need* to be required.
> If you are merging into master, please add either a `backport` or a `no-backport` label to this PR. You will not be able to merge until you do this step. Refer to the section [Do I need to backport this PR?](https://www.notion.so/metabase/Metabase-Branching-Strategy-6eb577d5f61142aa960a626d6bbdfeb3?pvs=4#89f80d6f17714a0198aeb66c0efd1b71) in the Metabase Branching Strategy document for more details.
> For those employed by Metabase: if you are merging into master, please add either a `backport` or a `no-backport` label
> to this PR. You will not be able to merge until you do this step. Refer to the section [Do I need to backport this
# TODO: Add a version validation, so the workflow won't fail when publishing to npm
sdk_version:
description:'SDK version (e.g. 0.1.3)'
type:string
required:true
git_ref:
description:'Valid git reference e.g. branches, tags, hashes'
type:string
required:true
concurrency:
# We want to ensure only one job is running at a time because
# there could be a conflict when updating the readme file.
group:${{ github.workflow }}
cancel-in-progress:true
jobs:
check-git-tag:
runs-on:ubuntu-22.04
timeout-minutes:20
env:
tag:embedding-sdk-${{ inputs.sdk_version }}
steps:
- name:Check out the code using the provided ref
uses:actions/checkout@v4
with:
ref:${{ inputs.git_ref }}
fetch-depth:0
fetch-tags:true
- name:check tags
run:|
if [ $(git tag -l | grep $tag) ]
then
echo "::error::Tag '${{ env.tag }}' already exists. If you expect to run this workflow with the same tag, please remove the tag first and rerun this workflow again."
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.