* Run E2E tests using Chrome
* Create snapshots using Chrome
* Test in GHA once again
* Set browser in the base Cypress config
* Change branch back to `master` for E2E workflow
* Extract repro for #16756
* Optimize looping through sql date field fitlers
* Optimize looping through sql number field fitlers
* Optimize looping through sql string field fitlers
* Extract common helper
* Run only native filters in the current branch
* Reduce flakiness in `type`
* Merge field filters together
* Run tests on `master` again
* add API to fetch general permisisons graph
* add API to update general permissionns
* change author of migration
* update documents
* misc fixes to applease the CIs
* Add tests for general permission APIs and models
* linting and fix a failed test case
* fix some failed tests
* update docs and change /subscription/ to /general/subscription/ for consistency
* Hook and migration to make sure subscription are created for new groups by default
* add schema migrations tests
* set for the win
* address noah's comments
* Parse number as is in http-client for tests
* address Cam's comments
* revert the last commit about parsing API response in tests
* change fk name
* delete a comment
* Changes:
- Rename `changes` column to `after` to keep things consistent
- If a group has no General Permisions, it'll not be included in the graph
- Update tests and some docs
* fix failing tests in ee
* add some tests and make docstring completes
* polishing comments
* namespaces
* fix namespaces
* Add general permisison flags to `GET /api/user/current` (#21250)
* return general permisison flags for /api/current
* namespaces
* move permission flags to under
* Enforce Subscription permissions (#21285)
* return general permisison flags for /api/current
* namespaces
* enforce general permissions for subscription and tests
* update check-has-general-permisison to add option to require superuser
* adding arity
* add tests for permissions helper function
* Move advanced permissions check funcs to ee namespace
* unpushed changes
* namespaces
* ignore exception when load namespaces
* change helper fn name
* Enforce Monitoring Permissions (#21321)
* return general permisison flags for /api/current
* namespaces
* enforce general permissions for subscription and tests
* update check-has-general-permisison to add option to require superuser
* adding arity
* enforce permissions to call /api/dataset for internal queries
* enforce monitoring permissions for api/task and api/util
* add tests for OSS
* add tests for db-connection-info endpoint
* change test schema
* update name func and fix ns
* whydon't CI run ?
* Enforce Setting Permissions (#21386)
* return general permisison flags for /api/current
* namespaces
* enforce general permissions for subscription and tests
* update check-has-general-permisison to add option to require superuser
* adding arity
* enforce permissions to call /api/dataset for internal queries
* enforce monitoring permissions for api/task and api/util
* add tests for OSS
* add tests for db-connection-info endpoint
* change test schema
* update name func and fix ns
* whydon't CI run ?
* Enforce Setting permissions
* fix failing test
* make sure we could run slack test twice
* make the mock consistent
* address Noah's comments
* shorter permissions check
* Create sidebar and add collections tree
* Fix dragged collection item's z-index
* Add links (data, archive, users' collections)
* Add ProfileLink
* Add bookmarks
* Use the new sidebar at Navbar
* Fix official collection icon opacity
* Add sidebar width to navigation constants
* Cut long item names
* Fix sidebar's horizontal scroll
* Fix remove bookmark icon position
* Fix bookmark URLs
* Fix how selected item is determined
* Jump to a collection opened from a bookmark
* Fix selected element highlighting edge cases
* Don't check for "users" collection
* Fix collapsing tree nodes
* Fix remove bookmark icon show on hover
* Fix removing a bookmark
* Fix archived collections are disabled in the tree
* Expand collection's parents after move
* Fix UI element lookups in tests
* Accept `icon` object as a SidebarLink prop
* Use "tree" a11y role for collections tree
* Always show the archive link
* Fix "other users' collections" link icon
* Fix e2e test
* Fix e2e tests
* Remove not used prop
* Make bookmarks collapsible
* Fix post-rebase issues
* Fix can't click expanded tree node to collapse
* Fix e2e test
* Remove not used code
* Use bookmark entities icon helper
* Extract renderBookmark function
* Allow custom `icon` element prop on SidebarLink
* Fix bookmarked official collection icon colors
* Fix `personal_owner_id` type
* Move Collections entity file to collections dir
* Reorganize collection entity file
* Give methods some space
* Extract `active` state setter methods
* Clean up styled components
* Extract more styled-components
* Extract methods
* Turn SearchBar into functional component
* Convert SearchBar to TypeScript
* Prototype new nav skeleton (#20875)
* yarn
* begin basic layout flip
* rough in collections list
* position query builder
* remove fitViewport from QB to fix layout quirks
* extract search
* clean up props
* position search / style
* fix dashboard layout
* fix search icon
* move ProfileLink
* tweak spacing
* fix burger
* add basic toggle
* tweak layout for sidebar toggle
* tweak style
* tweak sidebar sizing
* Quick fix to prototype starting points init (#20922)
* fix scroll overflow on sidebar
Co-authored-by: Gustavo Saiani <gustavo@poe.ma>
* add currentUser to profile link
* move footer items out of collection sidebar
* tweak footer style
* fix hover style
* clean up footer
* fixup
* remove un-needed overflow hidden
* hide app bar when in admin mode
* merge sidebar header into new color scheme / nav layout
* clean up headers
* fix root collection spacing
* smaller font size for nav items
* provide flex context, fixes metadata editing
* fix login page
* Remove console logs
* Put user's personal collection under Our analytics (#21061)
* Browsing | Handle long item names in the sidebar (#21063)
* Handle long names in the nav sidebar
* Show tooltip with a full item name if its cut
* Browsing | Fix data reference & admin permissions layout (#21101)
* Remove admin navbar CSS fix
* Handle admin navbar in root App component
* Extract styled components
* Fix data reference layout
* Fix main container height
* Browsing | Hide sidebar for questions, models and dashboards on page open (#21064)
* Manage default sidebar visibility
* Handle ad-hoc questions better
* Move nav elements based on feedback (#21190)
* scoot buttons around
* move logo to appbar, scoot search
* remove user footer, home link [ci skip]
* review feedback [ci skip]
* fix accidental removal of links (minimalism was a fad anyway) (#21192)
* add button classes to new button (#21213)
* Nav visual changes (#21223)
* switch nav bg + move styling to NavRoot
* readable text
* increase contrast
* selected icon color
* remove bottom padding
* search bar color
* restructure sidebar position
* fix sidebar header color
* move to fixed when small
* fix multiple scrollbars
* fix sidebar width
* home as own item, gear below
* fix admin nav
* extract nav constant, fix types
* remove unused constant [ci skip]
* New nav - get base colors in order (#21262)
* new sidebar button
* nav bg to bg-light
* fix link track, remove extra home link
* color tweaks
* swap colors
* Fix E2E tests for new navigation UI (#21196)
* Use `main` HTML tag for main page content
* Add extra UI helpers for Cypress
* Fix tests failing due to nav UI changes
* Remove not needed NavBar code
* Fix loading databases list
* Use `navigationSidebar` helper for collections
* Use `navigationSidebar` for collection tests
* Close sidebar in SQL filters tests
* Extract AppBar component to `metabase/nav`
* Fix test
* Fix test
* Skip sidebar visibility control test
* Fix JSON engine DB check
* Fix DatasetEditor edit bar colors
* Fix global containers heights
* Fix tests scrolling target
* Fix styles
* Fix navbar cypress helpers
* [ci nocache]
* Fix scroll in e2e test
* Browsing | Manage sidebar state with redux (#21226)
* Move navbar state to redux
* Hide sidbebar using CSS
* Fix native E2E test
* Fix native e2e test
* Fix model tests
* Use more accurate element getters
* Fix e2e tests
* Browsing | Fix dashboard UI (#21233)
* Fix parameter widgets event listeners
* Attach scroll events to "main" instead of window
* Resize filters panel when navigation sidebar open
* Reorder imports at App
* Hide AppBar and NavBar when editing dashboard
* Fix navbar & appbar visibility
* Fix filter positions
* Extract `getMainElement` helper
* Update unit tests
* Fix test
* Browse fix bulk actions (#21320)
* make bulk action bar aware of sidebar, remove extraneous layout components
* clean up styling
* convert BulkActionBar to tsx
* Fix types
Co-authored-by: Kyle Doherty <5248953+kdoh@users.noreply.github.com>
Co-authored-by: Gustavo Saiani <gustavo@poe.ma>
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
* Remove errant :bookmarked hydration key
This was left over from an incorrect implementation of :bookmarked status. No :bookmarked hydration function exists
anyway, the key does nothing.
* Remove old favorites models that are superceded by bookmarks
* Remove dashboard favorite require
* Added bookmarked-score to enable boosting of bookmarked items
* Collection Search Query doesn't error with bookmark left join
* adjust collection id AS in base query
* Set proper bookmark status in collection for search tests
* Actually score bookmarks when bookmarked
* Added a bookmark scoring test
* Remove old todo since it does work. (yay)
All under the grand path to #708.
Changes field API to be compatible with nfc-paths. Allows joining of nested field columns by shoving the full identifier into the postgres function call. Also fixes arcane macro test bug by not using the macro.
New releases of maildev were recently made which has UI changes
as well as changing the default port from 25 to 1025, causing tests
to fail.
While we may want to upgrade maildev in the future, let's stick to
the old version first to unbreak all the tests.
* Revert "Log database connection error before humanizing it (#10695)"
This reverts commit 426df1631d.
This is no longer needed. The original exception message is kept
in the inner exception and everywhere this is used, the whole
stack trace is logged anyway.
* Don't log exception when opportunistically trying SSL
To be usable in the postgres driver, the _uploaded_ root cert needs to:
- have its prefix stripped
- and be base64 decoded
(note: this was not an issue when pointing to a local file)
* Remove unneeded requests
* Remove tests not related to embedding
* Remove embedded question test
Covered in `embedding-native.cy.spec.js`
* Extract question details
* Extract dashboard details
* Fix question details
* Refactor an anti-pattern
Do not assign return values to previously defained variables.
* Do not use hard coded JWT signed token
* Simplify and merge dashboard tests into one
* Divide context into UI and API
* Extract common code into `beforeEeach` for UI context
* Merge UI tests into one
* Rename spec to `embedding-dashboard`
* Remove unnecessary `beforeEach` block
* Extract data and functions into a separate file
* Make sure we can disable previously set parameters
* add API to fetch general permisisons graph
* add API to update general permissionns
* change author of migration
* update documents
* misc fixes to applease the CIs
* Add tests for general permission APIs and models
* linting and fix a failed test case
* fix some failed tests
* update docs and change /subscription/ to /general/subscription/ for consistency
* address noah's comments
* Parse number as is in http-client for tests
* revert the last commit about parsing API response in tests
* change fk name
* Changes:
- Rename `changes` column to `after` to keep things consistent
- If a group has no General Permisions, it'll not be included in the graph
- Update tests and some docs
* fix failing tests in ee
* add some tests and make docstring completes
* fix namespaces
* Add :dataset key to card bookmarks
The :dataset key is boolean and indicates when a card (question) has been turned into a model.
The frontend will use this key to determine which icon is used to display the bookmark in the sidebar.
* Reduce some duplication in this test
* Removing from CCI
* Adding Currents to GHA
* Let there be run
* Returning to master
* Adding new key to new project on Currents
* Adding --ci-build-id to support GHA Re-Runs
* Returning to branch to test it more
* Removing not needed info (repo)
* Returning to master
* allow question paramater fields on mobile
* hide editor bar on mobile when there are no params
* show editor controls when already open
* add parameter mobile visibility test
* add visibilityToggler unit tests
* Create `visitDashboard` E2E helper
* Apply helper to the related tests
* Fix waiting for redundant xhrs
* Make helper work for users without permission to access dashboard
* Do not use helper when asserting on individual dashcard queries
* Refactor repro for 18454
* Improve dashboard permissions test
* Remove remapping as it does not have any effect on native questions
* Name filter in sync with the documentation reccommendation
* Enable sharing via UI
* Publish changes via UI
* Set the token expiration in the helper function itself
* Expand the helper to work for both questions and dashboards
* Expand helper to include filter values and hidden filters
* Asert on the `embedding_params` upon publishing embedding changes
* Expand tests with API context
* Properly fix FK filter
* Fix typo and expand helper example
* Add API test for locked parameters
* Separate UI and API tests
* Timeline has default icon, Timeline Event default icon from Timeline
* Move the icon schema to Timeline model.
The 'default icon' of a Timeline is concretely 'star', whereas the default icon for an event is taken from the
Timeline it's part of, which could be any of the icons.
Since the default should be one of the icons listed in the Schema, it makes sense for readability that the icon schema
is in the same namespace.
* When timeline's collection_id is nil, build the root collection
Timeline's always have :collection_id, and when the ID is a proper ID, :collection hydration works properrly. When the
ID is nil, we don't pass the necessary data to the frontend. So, match the behaviour for the `GET
/api/collection/root` endpoint and use that data on the :collection key
* Add root collection hydration to GET /api/timeline
* Add tests for collection hydration and timeline default icons
* Add a json schema
* Pass required data for snowplow events
* Pass required data for snowplow events
* Add snowplow event tracking for Timeline event creation
This changes the timeline_event API endpoint to send snowplow tracking when an event is created.
The FE now sends additional information for this purpose:
- source: collections or question. This is null if the event is added via API
- question_id: added if the source is from a question. null if the question is unsaved.
These are dissoc-ed from the body before the db/insert because the backend does not store this information, it's just
from the FE and only relevant for this tracking data.
* Fix json error
* Add ::new-event-created entry to the snowplow test
* Fix json error
* Fix json error
* Fix snowplow test
Co-authored-by: Adam James <adam.vermeer2@gmail.com>
Co-authored-by: adam-james <21064735+adam-james-v@users.noreply.github.com>
This is the first vertical slice of #708. There is a material ways to go, including mysql implementation, plinking away at the data model stuff, and frontend. There are also big putative sync speed gains I think I should chip away at.
* shouldNotForwardTransientProp method created for emotion
* BadgeIcon html attributes clean up
* Usage of transient property $hasMargin in Badge for BadgeIcon
* HTML attribute cleanup for LinkRoot
* React key added to ButtonBar child if not defined
* Properties for TableInfoPopover properly typed
* Misc
* analyticsContext property set as non required as non of the parents enforce it or pass a value
* Clean up attributes for SVG in Icon
* CSS fix for share icon special case in Icon
* Checkbox controlled component default value fix
* Misc clean up
* index file for collection side bar
* Collections collectionID type fix
* collection type in NewCollectionItemMenu fix
* Misc
* TokenField fix for value
* noPopover passed properly to WidgetStatusIcon
* Updates to controlled input/checkbox
* VS Code workspace file added to gitignore
* shouldNotForwardTransientProp renamed to shouldForwardNonTransientProp
* AddAggregationButton defaultProps updates
* propTypes for TableInfoPopover added back
* Tick range fix for timeseries scale
* Revert ticksForRange in timeseries
* Resetting the timestamp parsing based on the unit
* Move startOf reset to getDatas only
* Misc
* Misc
* Cater for null xValue
* Completely rewrite repro for #16170
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
* Add :display to the bookmark query (card specific).
Bookmark results can now optionally show a 'display' key, which is relevant only to cards.
This does not change the db in any way, it just pulls additional data from the left-join in the query already.
This particular key :display is used on the frontend to change the icon that the bookmark renders with in the
sidebar.
* Add a test that confirms `:display` key exists on card bookmarks
* Bookmarks on official collections now have `authority_level` key
* Add validation logic to TemplateTagDimension dimension tag handling
* Add basic error message to query builder
* Update error text
* Add unique error message for validation errors
* Update text of error state
Co-authored-by: Maz Ameli <maz@metabase.com>
* Fix types
* Fix NativeQuery unit test
* Add unit tests for new components
Co-authored-by: Maz Ameli <maz@metabase.com>
* Containerizing using the current release container process, as it's not only more representative, but also we use yarn and maven caching which should be faster
* rebasing
* Update uberjar.yml
* docs - update sql editor page
* change sql example to upper case
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Add tippy/popper replacement for PopoverWithTrigger
* Refactor and fix types
* Move PopoverWithTrigger components to own folder
* Add tests
* rename children prop to popoverContent
* Fix direct ParameterTargetWidget.jsx reference
* Increase timeout for QA DB sync
* Delete DB after the snapshot was taken
This is a hack that seems to prevent race conditions when one DB's sync interferes with the subsequent DB's sync.
* Get slack users and conversations simulatenously
- instead of waiting for the channels, and then getting the users
* Add cache+timeout capability to conversations-list
* extend async approach to users-list
- return timeout+cache warning
- adds tests
* updating docstrings + adding more tests
* Add cache clear and seed slack token settings on-change
- add dox for :on-change for defsettings
- fixing tests
* Fixing tests
- cannot pass a declared (not def'd) value
- some cleanup
* remove unused function
* remove need for slack-configured from timeout fns
- update some docstrings
* pre-populate slack caches
- on app init
* introduce blocking (and cache resetting) endpoint
- this one does not timeout, meant to be called when a user hits a
button like: "Refetch my slack channels"
* removes -timeout versions of users/channels fn
- the endpoint now returns the cached slack data
- removes tests
* fixes routing layer
* remove trailing whitespace
* add missing docstring
* move the cached slack data into a setting
* add missing docstring
* fix typo
* update form-input-test
* refresh cache in test
* remove un-needed anonymous fn wrapping
* refresh slack cache when user sets new valid token
* refresh the cache when reading from it
- the more its used the more up to date it will be
* code review responses + removes 2-arity refresh-cache!
- clear out cache when slack connection is broken or unset
- add to docstring for slack settings route
- wrap init refresh-cache! with a future
- re-spell parallel
* do not block when returning cached slack data
* rename cache defsetting
* Addressing code review issues
- Adds scheduled updates to the slack users and channels cache
every 4 hours
- Removes `form_input_refresh` route
- Automatically refreshes cache when `form_input` route is hit,
if 10 minutes have passed since the last cache fix
- Removes cache refresh on app startup
- Stores only user names, and channel names in the cache, not maps with
irrelevant data
- Clears out some noisy linter warnings via `#_:clj-kondo/ignore`
* Adds slack task RefreshCache jobs
- Sets up startup-triggered and cron-triggered (every 4 hours at a
random minute) slack channel/username cache refreshing
* Implements 2 jobs to update slack users/channels
- one on startup, one every 4 hours
- improve docstrings
* ns linter fixes
* more ns linter fixes
* refactor files-channel to look in the cache
- instead of making a tricky api call
- fixing tests
* update files-channel-test to test the cache
* fixup integrations.slack ns decl
* fixup test integrations.slack ns decl
* fix outdated mock of slack/files-channel
* fix a few more tests
* fix api slack-test
* adds locking to refresh-channels-and-usernames!
* re-order requires
* Use Service Accounts for Google Analytics auth going forward
* Update GA dox
* Dox tweak
* Fix e2e test
* Remove dead googleanalytics form code
Co-authored-by: Dalton Johnson <daltojohnso@users.noreply.github.com>
* Change u/profile from println to log/info
* Change most test printlns to log/info
* Make u/profile message a fn rather than a delay
* Apparently clj-http errors aren't wrapped in :object anymore
* Add running commentary
* Log the amount of time it took to find and run tests
* Sort namespaces
* Update test
Co-authored-by: Diogo Mendes <diogo@metabase.com>
* added status option for rendering widget icon
* mimicked default form behavior using the fetch API
* recreated default html form submit and download
* added GET request functionality to download form
* cleanup
* making requested changes
* onClose, fileName cleanup, blob(), GC link remove
Co-authored-by: Paul Wilson <54870992+wilsonpg@users.noreply.github.com>
* Try making the encryption clearer
Got the question a few times, so I thought we had to make it more explicit
* Update docs/operations-guide/encrypting-database-details-at-rest.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
Due to Cypress upgrade, we observed failing and/or flakey E2E tests.
Rather than disabling this completely, let's narrow down to a few good
tests for now while robustifying the rest of them.
* Upgrade Liquibase to latest version
* Try adjusting log
* Fix checksums for the TWO migrations with ID = 32
* FINALLY get Liquibase to use Log4j2
* Set Liquibase ConsoleUIService OutputStream to null OutputStream
* Manually define a package for our H2 proxy class so Java 8 works
* Fix package-name determination code
* Update migrations file spec
* `databasechangelog` shouldn't be upper-case
* Lower-case quartz table names
* More MySQL fixes 🔧
* Properties for all the Quartz tables 😢
* Formatting tweaks [ci skip]
* Revert a few more busted changes
* Fix more busted changes
* Bump Liquibase version to 4.8.0 to fix MySQL defaultValueBoolean bug
* OMG I think I finally fixed MySQL
* Remove Java source file and prep-deps code
* Remove two more references to bin/prep.sh
* Minor cleanup
* Revert unneeded changes
* Fix busted indentation
* Don't search inside java/ anymore since it's G-O-N-E
* Appease the namespace linter
* Update src/metabase/db/liquibase/h2.clj
This lets the Frontend get a list of timelines with the events included, useful in case of having an unsaved question
where timelines should still be showable.
> left comments but seems workable
I'm going to merge this to let FE continue events implementation.
I will discuss the above concerns in Slack, and we can make adjustments as needed.
* accessors constants created
* accessors dependencies removed from all charts
* Removed accessors dependencies from BE JS for static viz
* Removed accessors references in chart editor
* accessors added to defaultProps
* Remove accessors in tests
* defaultProps removed form static viz components
* TimeSeriesLineChart uses scaleBand instead of scaleTime for x-axis
Similar to TimeSeriesBarChart
* TimeSeriesAreaChart updated to match TimeSeriesLineChart
- sorting in query so don't have to sort in memory afterwards, and
leaves open spot on the api to sort with user preferences if
present (future work)
- adds schema so we know what these bookmarks look like, and return an
explicit structure instead of threading through dissoc and assoc
- combined the remove-nil-values into the normalize function since it
set up preconditions necessary for the normalize.
- use Models in query so we don't have to hardcode the underlying table
This way, if the field name contains [ or ], this character will be
handled properly (with escaping and unescaping at the right place).
Ensure escape vs unescape symmetry
* Improved error messages when query execution fails
* Pretty-print hx/identifier in a different way if it contains extra keys
* Fix#20806
* Rename key used to record that we should not qualify
* Fix line breaks
* Test fix 🔧
* use site-url when opening links via JS
* fix normalization of 0 value
* make dashboard click drill to same dashboard just update parameters
* fix race condition with location prop
* use Link instead of anchor tag
* Add the :archived status of card/dashboard/collection for filtering
If an item is bookmarked, we can follow the link to that item without issue. But, if we bookmark an item and then
archive that item, the bookmark shouldn't show up, as we don't want to link to an archived item.
The bookmark itself is not removed, it's just filtered away when :archived is true.
* Added test for bookmarked and archived items
The archived items that are still bookmarked should not be sent to the Frontend.
* Upgrade Cypress to the latest version (9.4.1)
* Fix collections tests failures after upgrade
* Fix binning tests failures after upgrade
* Fix dashboard test failures after upgrade
* Fix models test failures after upgrade
* Fix admin test failures after upgrade
* Fix visualizations test failures after upgrade
* Rewrite and fix repro for 17397
Added the missing issue reference.
* Fix question test failures after upgrade
* Fix interception handlers in metabase-visual tests
* Fix the rest of the binning tests after upgrade
* rewrite cypressWaitAll helper with a simple recursion
* Updating to 9.5.0
* Upgrade Cypress to v9.5.1
* Fix repro for 13332
* Fix pinned items `collections` test failures
* Speed `collection-pinned-overview` spec up using the API
* Fix failing localization test
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: Diogo Mendes <diogo@metabase.com>
* /api/user return only active users when include_deactivated=false
* revert changes in docs/api-documentation.md and fix trailing space for params section
* simplify tests and update api's doc
* Types and options constants added to all chart types
* Chart types and options added to StaticChart
* Static viz page refactored to include an editor
* Editing data added
* Error handling added
* Misc
* StaticChart converted to TypeScript
* Stories for StaticChart added to storybook
* Misc
* Updates from PR review
* FunnelChart missing key for React loop added
* BarSeries missing key for React loop added
* TickRendererProps formattedValue method removed from tag attributes
* React loop key added to internal routes
* Table head tagging fixed in internal
* BaseIcon uncheckedColor method removed from being added to SVG tag attributes
* Misc
* ColorPicker fix for maxHeight attribute
* Missing key for CollectionBadge in InfoText added
* Use the term "Native query"
* Fix/update e2e tests
* Add repro for #20499
* make text conditional on engine and write permissions
* Revert "Fix/update e2e tests"
This reverts commit 3462cd113e.
Co-authored-by: Dalton Johnson <daltojohnso@users.noreply.github.com>
* Add test id to the mini-bar
* Add initial embedding test for the regular GUI question
* Update test with slighlty more complicated formatting
* Extract question details into a separate file
* Add a test for the quesiton with aggregation
* Assert while being signed out
* Add test for the nested question
* Add test for the joined question
mostly unused bindings. Had a few redundant lets. One common pattern was
a let binding for a read check but the resulting value was never used,
just checked that we could read or write it. Removed that from a binding
since it throws otherwise
* Handle legacy Field filter parameters that did not specify `:widget-type`
* Fix indentation
* Test fix
* Revert to old template tag validation logic
* Update e2e test
Co-authored-by: Cam Saul <github@camsaul.com>
* Displace tests that are not related to embedding
* Rename and enable embedding snippets spec
* Add more assertions to the embedding code snippets spec
* Rewrite and enable premium ebmedding token tests
* Fix a timezone related issue in the validity date of the token
* Adding tests in collections and card. Fixing some minor test things
* Added Test for timeline event range selection via card API
* Cleaned up a few TODO comments
* Made the timestamp required when POST-ing a timeline-event
* Remove unnecessary nesting of with-temp* forms
* Moved Icons schema into timeline-event model
This schema seems to be validating more than just the API, so we've moved it.
* Added start/end to GET /api/timeline/:id
Also cleaned up some unnecessary nesting in a card API test
Added timeline range test in timeline API test ns too.
* Parse timestamp when it exists in a timeline-event PUT
We saw the db error presented in the FE when updating timeline events. This should fix that problem.
* Add a test to ensure we can change timestamps via timeline-event PUT
* Add `SAMPLE_DB_TABLES` reference
* Add notes to the cypress data references
* Ensure table ids are correct
Early warning mechanism.
* Use SAMPLE_DB_TABLES in e2e helpers and custom commands
* Use `SAMPLE_DB_ID` in the previously missed helper
* Replace hard coded ids in `question.js`
* Replace hard coded ids in ad hoc question helpers
* Replace hard coded ids in `sandboxTable` custom command
* Replace hard coded ids in `updatePermissions` custom command
* Avoid circular dependencies
* Run unadded additional dashcard series using card query endpoint
* add repro for #20637
* Update frontend/test/metabase/scenarios/dashboard/reproductions/20637-add-series-to-dashcard.cy.spec.js
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
* wait on all dashboard save requests in e2e test
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
* Export `SAMPLE_DB_ID`
* Replace hard coded sample database id values in the default snapshot creator
* Replace hard coded values for the SQLite database
* Update function name
* Stop using hard coded values for questions and dashboards
Step 1 towards the long path towards whacking #708. We need an interface for describing the weird thing every RDBMS vendor did where they have their own special way of doing JSON columns, every single one of them
* Copy over admin premium embedding settings to a new file
* Move embedding reproductions to a related folder
* Move over dashboard embedded parameters
* Move over embedding dashboard spec
* Move over repro for 20634
* Enable embedding in CI
* Migrations for the new 'Annotate Useful Events' project
Users will be able to create 'events' in timelines, which are accessible via collections.
Two migrations are introduced to facilitate the new feature: timeline table, and event table. The following columns
are presumed necessary based on the design document:
** TimeLine
- id
- name
- description
- collection_id
- archived
- creator_id
- created_at
- updated_by ??
- updated_at
*** Event
- id
- timeline_id
- name
- description markdown (max length 255 for some reason)
- date
- time is optional
- timezone (optional)
- icon (set of 6)
- archived status
- created by
- created at
- created through: api or UI
- modified by ??
- modified at
* Changes to events schema
- add icon onto timeline
- make icon on event nullable
- just move to a single timestamp on event, no boolean or anything
- rename event table to `timeline_events` since "event" is so generic
* dir-locals indentation
* Timeline api and model
- patched up the migration to make collection_id nullable since that is
the root collection
- followed the layout of api/metric and did the generic model stuff
* Select keys with keywords, not destructured nils :(
also, can't return the boolean from the update, but select it again
* Enable the automatic updated_at
* Boolean for "time matters" and string timezone column on events
* clean up migration, rename modified_at -> updated_at
for benefit of `:timestamped? true` bits
* basic timeline event model namespace
* Timeline Events initial API
Just beginning the basics for the API for timeline events.
- need to find a way to check permissions
- have to still check if the endpoint is returning stuff properly
* Singularize timeline_event tablename
* Timeline events api hooked up
* Singularize timeline event api namespace
* unused comment
* indent spec maps on routes
* Make name optional on timeline PUT
* Update collection api for timelines endpoints
- add /root/timelines endpoint for when collection id is null
- add /:id/timelines endpoint
Created a hydration function to hydrate the timeline events when fetching timelines.
* TimelineEvent permissions
- crate a permissions objects set via the timeline's permissions
* Move to using new permissions setup
previously was explicitly checking permissions of the underlying
timeline from the event, but now we just do the standard read/write
check on the event, and the permissions set knows it is based on the
permission set of the underlying timeline (which is based on the
permissions set of the underlying collection)
* Items api includes timelines
* Strip of icon from other rows
* Indices on timeline and timeline_event
timeline access by collection_id
timeline_event by both timeline_id and (timeline_id, timestamp) for when
looking for events within a certain range. We will always have been
narrowed by timeline ids (per the collection containing the question, or
by manually enabling certain timelines) so we don't need a huge index on
the timestamp itself.
* Skeleton of range-based query
* Initial timeline API tests
Began with some simple Auth tests and basic GET tests, using a Collection with an id as well as the special root collection.
* Fix docstring on api timeline namespace
* Put timeline's events at `:events` not `timeline-events`
* Add api/card/:id/timelines
At the moment api/card/:id/timelines and api/collection/:id/timelines do
the same thing: they return all of the timelines in the collection or in
the collection belonging to the card. In the future this may drift so
they share an implementation at the moment.
* Put creator on timeline
on api endpoints for timeline by id and timelines by card and collection
* Hydrate creator on events
Bit tricky stuff here. The FE wants the events at "events" not
"timeline-events" i guess due to the hyphen that js can never quite be
sure isn't subtraction.
To use the magic of nested hydration
`(hydrate timelines [:events :creator])`,
the events have to be put at the declared hydration key. I had wanted
the hydration `:timeline-events` for clarity in the code but want
`:events` to be the key. But when you do this, the hydration thinks it
didn't do any work and cannot add the creator. So the layout of the
datastructure is tied to the name of the hydration key. Makes sense but
I wish I had more control since `:events` is so generic.
* Add include param check to allow timeline GET to hydrate w/ events.
The basic check is in place for include=events in the following ways:
GET /api/timeline/:id?include=events
GET /api/collection/root/timelines?include=events
If include param is omitted, timelines are hydrated only with the creator. Events are always hydrated with creator.
* fix hyphen typo in api doc
* Change schema for include=events to s/enum to enforce proper param
Had used just a s/Str validation, which allows the include parameter value to be anything, which isn't great. So,
switched it to an enum. Can always change this later to be a set of valid enums, similar to the `model` parameter on
collection items.
* Fixed card/:id/timelines failing w/ wrong arity
The `include` parameter is passed to `timeline/timelines-for-collection` to control hydration of the events for that
timeline. Since the card API currently calls this function too, it requires that parameter be passed along.
As well, I abstracted the `include-events-schema` and have it in metabase.api.timeline. Not married to it, but since
the schema was being used across timeline, collection, and card API, it seems like a good idea to define it in one
place only. Not sure if it should be metabase.api.timeline or metabase.models.timeline
* used proper migration numbers (claimed in migrations slack channel)
* Add archived=true functionality
There's one subtle issue remaining, and that is that when archived=true, :creator is not hydrated on the events.
I'll look into fixing that.
In the meantime, the following changes are made:
- :events returs empty list `[]` instead of `null` when there are no events in a timeline
- archived events can be fetched via `/api/timeline/:id?include=events&archived=true`
- similarly, archived events can be fetched with:
- `/api/collection/root|:id/timelines?include=events&archived=true`
- `/api/card/:id/timelines?include=events&archived=true`
Just note the caveat for the time being (no creator hydrated on events when fetching archived) Fix pending.
* Altered the hydration so creator is always part of a hydrated event
Adjusted the hydration of timelines as follows:
- hydration always grabs all events
- at the endpoint's implementation function, the timeline(s) are altered by filtering on archived true or false
Not sure if this is the best approach, but for now it should work
* Create GET endpoint for /api/timeline and allow archived=true
Added a missed GET endpoint for fetching all the timelines (optionally fetch archived)
* reformat def with docstring
* Timeline api updated to properly filter archived/un-archived
Use archived=true pattern in the query params for the timeline endpoint to allow FE to get what they need.
Work in progress on the API tests too.
* Timeline-events API tests
Timeline Events API endpoint tests, at least the very basics.
* Timeline Hydration w/ Events tests
Added a test for Events hydration in the timeline API test namespace.
* TimelineEvent Model test NS
May be a bit unnecessary to test the hydrate function, but the namespace is there in case we need to add more tests
over time.
Also adjusted out a comment and added a library to timeline_test ns
* Added metabase.models.timeline-test NS
Once again, this may be a bit unnecessary as a test ns for now, but could be the right place to add tests if the
feature grows in the future.
* Clean up handling of archived
- we only want to show archived events when viewing a timeline by id and
specifically asking for archived events. Presumably this is some
administrative screen. We don't want to allow all these options when
viewing a viz as its just overload and they are archived for a
reason. If FE really want them they can go by id of the timeline. Note
it will return all events, not just the archived ones.
- use namespaced keywords in the options map. Getting timelines
necessarily requires getting their events sometimes. And to avoid
confusion about the archived state, we have options like
`:timeline/events?`, `:timeline/archived?`, `:events/start`,
`:events/end`, and `:events/all?` so they are all in one map and not
nested or ambiguous.
* Clean up some tests
noticable differences: timelines with archived=true return all events as
if it is "include archived" rather than only show archived.
* PUT /api/timeline/:id now archives all events when TL is archived
We can archive/un-archive timelines, and all timeline events associated with that timeline will follow suit.
This does lose state when, for example, there is a mix of archived/un-archived events, as there is no notion of
'archived-by' to check against. But, per discussion in the pod-discovery slack channel, this is ok for now. It is also
how we handle other archiving scenarios anyway, with items on collections being archived when a collection is archived.
* cleanup tests
* Include Timeline and TimelineEvent in models
* Add tt/WithTempDefaults impls for Timeline/TimelineEvent
lets us remove lots of the `(merge defaults {...})` that was in the code.
Defaults are
```clojure
Timeline
(fn [_]
{:name "Timeline of bird squawks"
:creator_id (rasta-id)})
TimelineEvent
(fn [_]
{:name "default timeline event"
:timestamp (t/zoned-date-time)
:timezone "US/Pacific"
:time_matters true
:creator_id (rasta-id)})
```
* Add timeline and timeline event to copy infra
* Timeline Test checking that archiving a Timeline archives its events
A Timeline can be archived and its events should also be archived.
A Timeline can also be unarchived, which should also unarchive the events.
* Docstring clarity on apis
* Remove commented out form
* Reorder migrations
* Clean ns
clj-kondo ignores those but our linter does not
* Correct casing on api schema for include
* Ensure cleanup of timeline from test
* DELETE for timeline and timeline-event
* Poison collection items timeline query when is_pinned
timelines have no notion of pinning and were coming back in both queries
for pinned and not pinned. This adds a poison clause 1=2 when searching
for pinned timelines since none are but they don't have a column saying
that.
* Clean up old comment and useless tests
comment still said to poison the query when getting pinned state and
that function had been added.
Tests were asserting count = 2 and also the set of names had two things
in them. Close enough since there are no duplicate names
* Use TemporalString schema on timeline event routes
Co-authored-by: dan sutton <dan@dpsutton.com>
The widget to filter time-series dimension should not assume that the
dimension is from a field, as it can be also an expression, i.e. when
the breakout is originated from a custom column. Hence, generalize the
case to avoid any crashing.
* Replace metabase.db.migrations/copy-site-url-setting-and-remove-trailing-slashes with a Liquibase migration
* Tweak comment
* `setting`, not `settings`
* Clean namespace
* `key` has to be quoted for MySQL
* Add tests for migrations v0.43.00-008 and v0.43.00-009
* fix new migration for MySQL 5.7
* Fix
* More logging
* Fix unbalanced parens
* Tweak logging
* Add another test
* Tweak test to use :string/=
* More logging
* Updated fix for #20493
* Enable test
* Try disabling the #19494 tests to see what happens.
* Rework #19494 test
* Replace metabase.db.migrations/migrate-humanization-setting with a Liquibase migration
* Apparently key has to be quoted these days in MySQL/MariaDB
* Need to quote key in the preCondition as well
* Fix merge conflicts
* Replace AOT Spark SQL deps with a `proxy` and a `DataSource`
* Support `connection-details->spec` returning a `DataSource`
* Remove target/classes
* Don't need to deps prep drivers anymore
* Fix duplicate `this` params in `proxy` methods; add `:test` alias for Eastwood to make it be a little quieter
* Make sure to call `pool/map->properties`
* Upgrade Hive JDBC driver version from 1.2.2 -> 3.1.2; Bump Spark SQL from 2.1.1 to 3.2.1
* Clean the namespaces
* Don't need to register the a proxy JDBC driver since we're not even using it anymore
* Fix Spark SQL :schema sync for newer versions
* Remove unneeded override
* Fix day-of-week extract for new :sparksql
* Hive/Spark SQL needs to escape question marks inside QUOTED identifiers now 😒
* Some minor SQL generation improvements to avoid duplicate casts
* Revert change to debug test
* for sample db do not allow certain updates
- namely to engine in the model layer
* only check for new schedules one time
* remove unnecessary *current-user-id* binding in test
- as per code review
* Use `thrown-with-message?` to check for exceptions
- Previously this would pass if db/update did _not_ throw an excetption,
which is not what we want here.
* Add failing test for #15978
* Improved test
* Add new metabase.driver.query-processor.escape-join-aliases QP middleware
* Test fix 🔧
* Add reference to #20307
* Add some extra dox
* Test fixes for BigQuery drivers
* revert unneeded change
* Fix :bigquery and :bigquery-cloud-sdk mixup
* Test fixes 🔧
* Test fix 🔧
* Remove comment I meant to remove
As documented, react-ace >=8 doesn't need brace anymore.
This reduces the bundle size,
resources/frontend_client/app/dist/vendor.bundle.js, by 218 KB, from
5286809 to 5063129 bytes.
* Optionally wrap question id to make it available outside of the original scope
* Use `visitQuestion` to simpliy tests
* Fix pivot_table tests
* Fix repro 12568 for nested questions
* Fix repro for 10803
* No need to rely on UI elements any more
* Fix source aliases for Fields from joins not being properly escaped
* Tweak TODO comment
* Disable the flaky tests when running driver tests
* Rename the new macro
* Dox
* Remove unused :require
* Fix BigQuery not qualifying source columns
* Don't fix on 42 for the deprecated BigQuery driver
* You know what I'll fix it after all. Why not.
* Oops I updated the wrong part of the test
In an expression `COUNT/2`, `COUNT` there should be treated as a
function (i.e. `COUNT()`) instead of a field.
Another function is the same category is `CUMULATIVECOUNT`.
Both don't require any arguments, hence the parentheses are optional.
* Group `Urls.question` args into object
* Use `Urls.serializedQuestion`
* Fix `tableRowsQuery` URL builder
* Add E2E tests
* Add routes with `/:objectId`
* Add `objectId` support to question URL builders
* Use `queryParams` from `location` in QB init
* Fix `useSyncedQueryString` drops other params
* Add `zoomInRow` action
* Add `resetRowZoom` action
* Add basic selectors
* Modify `getIsObjectDetail` selector
* Handle `objectId` during QB init
* Replace actions opening prev/next object detail
* Use zoomed row in object detail view
* Update `ObjectDetailDrill`
* Fix URLs update for detail views
* Handle browser "back" button in detail view
* Use question name as a fallback for object view title field
* Fix E2E tests
* Fix E2E test
* Fix routes order
* Fix opening a link to filtered out record
* Move tests for object details
* Fix regexp
* Fix link style
* Better routing handling
* Fix dashboard embedding locked parameter scripting err
We were not passing the embedding components the correct parameter
objects; instead of dashboard.parameters we need to pass the components
parameters that have had metadata added to them such as the fields they
are mapped to, as child parameters components now expect this metadata
to exist.
* Add e2e tests for embedded parameters
* Add org.quartz-scheduler/quartz as an explict dep and bump version from 2.17 -> 2.3.2
* Add no-op impl for `initialize` for our Quartz `ConnectionProvider`
* Remove :bigquery driver
Add "migration" to convert existing Database instances from :bigquery to :bigquery-cloud-sdk (with error log if using outdated OAuth mechanisms), by way of `normalize-db-details`
Removing references to :bigquery from various places in the code
* Remove from modules/drivers/deps.edn
* Remove stuff from CircleCI that is running the old driver
Remove `driver-switch-test` since there is no practical way to run this anymore (since we can't initialize the old driver)
Remove buggy redef of `isa?` from `semantic-type-migration-tests`
Co-authored-by: Cam Saul <github@camsaul.com>
* Fix locking in new Alpine release container
Seems like the new Alpine container (3.15) has some issue when doing apk upgrade first which ends up erroring out the process, so moving it to the last part of the process (after installing the packages) makes the trick.
You can test it out by using the current process (errors with a "Unable to lock database: temporary error (try again later)") and the one in this branch which will finish successfully
* Expand the `visitQuestionAdhoc` helper with the selection of the `mode`
* Rewrite `openTable` helpers using the `visitQuestionAdhoc`
* Remove superfluous wait for the `dataset`
* Fix `nulls` test
* Rewrite and fix `trendline` test
* Fix `saved` question test
* Fix User type
* Add mock user factory
* Add Event and EventTimeline types
* Add mock factories for Event and EventTimeline
* Add Events entity
* Add EventTimeline entity
* Fix exports
* Move new types to `types/api`
* Merge field `results_metadata` into field instance object
* Update unit tests
* Fix replacing result metadata on query completion
* Keep previous QB mode in redux
* Rerun model query after saving editor changes
* Ignore table column widths reset event after exiting model editor
* Reload query when entering model editor
* Remove commented out code
* Fix Dimension unit tests get stuck
* Make changes to header buttons as per designer feedback and remove secondary button style from core button
* removed last use of secondary
* fix active notebook button color
* fix View SQL button and modal
* Integrate waiting for the question dataset into the helper function itself
* Remove explicit waiting from visual tests
* Remove explicit waiting from most of the e2e tests
* Handle pivot tables dataset
* Handle assertion callback
* Simplify helper function
* Fix `bar_chart`
* Fix `line_chart`
* Fix `scatter` visualization tests
* Don't offer to save model changes without changes
* Rename variable
* Assert save button is initially disabled
* Disable save button if model isn't dirty
* Fix prop type
* Clean up metadataDiff after saving changes
* Prevent flaky tests based on contents of TaskHistory
Just check for subsets since we sometimes encounter fingerprint-fields
and analyze information
Output from CI:
```
task_history_cleanup_test.clj:35
Basic run of the cleanup task when it needs to remove rows. Should also add a TaskHistory row once complete
expected: #{"task-history-cleanup"
"metabase.task.task-history-cleanup-test/task-3"
"metabase.task.task-history-cleanup-test/task-2"}
actual: #{"classify-fields"
"analyze"
"fingerprint-fields"
"task-history-cleanup"
"metabase.task.task-history-cleanup-test/task-3"
"metabase.task.task-history-cleanup-test/task-2"}
diff: + #{"classify-fields" "analyze" "fingerprint-fields"}
```
* fix ns
* Retain property file path when revisiting database edit page
Ensuring that the file path (if that is the secret type) is returned via the API response to the admin, so the UX on the form edit page is better
This involved a lot of refactoring to clean up some stuff:
* removed the population of inferred secret properties (ex: last updated timestamp and creator ID) from `handle-db-details-secret-prop!`, since that stores those values into the details blob, defeating some of the purpose of normalizing secrets into a separate table
* moved the logic for injecting such inferred values to a new function, `expand-inferred-secret-values`, which lives within the secret model namespace
* delegating existing calls that require such expansion to the new function
* adding a new helper function, `reduce-over-details-secret-values`, to handle some common logic that is now showing up in a few places (reducing over db-details on secret values and doing some kind of manipulation)
Adding new database API test to ensure that the file path value is, in fact, returned from the API
*** Code:
- ignore hidden tables and archived cards/dashboards
- correct docstring to indicate we are only getting 5 not 10 as
previously claimed
- in query, set limit to 8 in the case we get archived or non-existing
things and then take 5 in memory
- select extra properties from models that we need for this
determination
*** Tests:
- for mysql we were inserting views with a thread/sleep of one second
between each. Just compute timestamps seconds apart and do them all at
once. Should save 7 seconds off mysql test runs. verified temporal
spacing works with
```
MB_DB_CONNECTION_URI="mysql://username:password@localhost:3306/cli" \
clj -X:dev:test :only metabase.api.activity-test
```
- cleanup these activities we were creating with `mt/with-model-cleanup`
which introduces a big looking diff. But checking diff with whitespace
shows its just an indentation and then add the extra keys needed for
identifying which are archived (:archived and :visibility_type for cards
and tables respectively)
* Rework QP.dashboard to expect dashcard-id
* Update /api/dashboard endpoints
* Rework public endpoints for running queries for Dashcards
* Tweak parameter order for Dashboard endpoints for consistency
* Pulse should pass Dashcard ID
* Embed should pass Dashcard ID
* Unskip cypress test
* Update API documentation
* Update endpoints and pass dashcardId to endpoint fns
* Update cypress tests that reference previous endpoint
* Update placement of pivot in endpoint
* Update another endpoint + more e2e fixes
* update QB to hold onto dashcardId like it does with dashboardId
* Address PR feedback
Co-authored-by: Dalton Johnson <daltojohnso@users.noreply.github.com>
* Bump backend dependencies (Jan 2022)
* Revert java-time version upgrade for now until https://github.com/dm3/clojure.java-time/issues/77 is fixed
* Add license overrides
* Bump a few more deps (again). Revert Google/BigQuery and Vertica version bumps
* Revert MariaDB and Redshift version changes
Right now, datasets are all cards. In the future this could change so
record the original model of the dataset as a card. In the future, maybe
we care about the underlying model and this allows us to at least query
and distinguish
* Remove superfluous `beforeEach`
* Simplify `locks display to table` test
Cuts the test running time in half. It goes from 10s to 4-5s.
* Simplify `allows to edit GUI model query` test
Cuts the test running time in half. It goes from ~16s to ~8s.
* Group GUI models related tests into a separate `describe` block
* Simplify `allows to edit native model query` test
Cuts the test running time almost in half. It goes from ~12s to ~7s.
* Group native models related tests into a separate `describe` block
* Handle NULL value for integer fingerprints
Fixes#19976.
* conditionally show some or none of component depending on if numbers are present in fingerprint
* Make jsdoc return type accurate
* add unit tests for this scenario
* rmv unused import
Co-authored-by: Dalton Johnson <daltojohnso@users.noreply.github.com>
* Fix regression breaking support for user:password@host Postgres connection strings
* Actually we need to handle either Postgres OR MySQL
* Fix the helper `question` function
* Wait for the card query to load before typing
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
* Disable remapping when syncing Field values. Handle human readable values in GET /api/field/:id/values
* Don't require `:has_field_values` to be list to actually return anything.
* Make sure GET /api/field/:id/values returns :field_id or it breaks the FE
* Fix add-dimension-projections middleware not respecting disable-remaps? key
More stuff in #18676. Supposed to be the BE for #20004 Just continuing static viz work. Stacked graphs only have a simple viz-settings flag turned on so making sure that gets to FE bit.
No implementation of normalized stacks.
To avoid double runs of various checks (linters, tests, etc) on feature
branch, we mainly run on pull_request trigger, i.e. the checks are
carried out once that feature branch appears in a PR, and not before.
We do however still run on push trigger for the important branches
(master, release-*).
* Unused bindings cleanup
`clj-kondo --lint src:shared/src > lint` and then just work my way
through it. Note that there are warnings about deprecations that we will
need to get rid of as well. Most likely by turning off that warning
until we are ready to tackle the project.
* More unused and a sort ns
* Removing unused svg helpers that eastwood is complaining about
* Restore accidentally deleted `begin!` impl for xlsx
* clean ns
* clean ns
* Last few unused bindings
* kondo checks for empty docstrings
* Little more cleanup
* update translation build script to catch ts and tsx files
* add blank file for indonesian
* makes password resetting less awesome
* get rid of 'this item' text
* add updated translations
* Adjust E2E tests expectation of the strings (after the modification)
Co-authored-by: Ariya Hidayat <ariya@metabase.com>
* Use javax.sql.DataSource everywhere for app DB connection sources
* Sort namespaces
* Un-remove Kondo warning
* Minor tweaks to support 'postgres:' ...
* Remove ConnectionDataSource
* Add warning about PG without sslmode back
* PR feedback
* Sort namespaces
* Remove top-level calls to trs (not allowed because app DB isn't initialized yet anyway)
* Test fix 🔧 (MySQL)
Still pursuant to #18676
1. Funnel was previously not doing lato font for the bold bit because visx's text doesn't take weights, it takes styles
2. Columns can be nontrivial with respect to funnels, which was an oversight
3. For some reason the first try for fixing 2 also impinged upon values being strings for funnels, which breaks things at SVG level
4. And let's just shove in a nit for the display of trends if the previous value and current value are the same
* Remove DataSelector info popovers
* Remove DimensionList info popover
* Remove expression editor suggestion list info popover
* Remove FieldPicker info popover
* Remove DimensionListItem info popover
* Remove info popover from data selector search results
* remove TableBrowser info popover
* Remove an e2e test
CI was broken inasmuch as it wasn't running, so master was broken without us knowing it. Thankfully, to the best of our knowledge, it was a couple of nits. Fixes master.
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
* onDrop unneeded
* basic impl of reordering pinned items
* split cards and dashboards again
* Improve pin drop target styling
* Fix card drop target weirdness
* brand-light border --> brand
* pull out separate styled drop target component
* replace style prop usage
* reduce the z-index
* Remove prop that triggers drill functionality
* Fix clobbering of pinned list on update
* Fix card queries on every rerender
* Add pin drop zone & split drop target components
* lint fix
* fix & add a few tests
* update pin drop zone styling to match sorting
* Improve lifecycle of metadata
Previously we have assumed that if valid metadata is passed into the
card create and udpate, that that is valid metadata for the query. There
are a few scenarios where this can fail, but they all boil down to
editing the query after running it. Examples:
- when creating a query, come up with a query, visualize, realize you
wanted to not select all columns, edit to not select those, possibly
select a few extra, and then save. The saved metadata might only have
the columns from when you ran the query and not after you saved it.
- edit an existing query and save without rerunning. Again, adding and
removing some columns.
How it affects:
- if you remove columns, the preserved metadata can cause us to attempt
to select columns that are no longer in that question, leading to errors
for nested queries
- if you add columns, the query builder won't show the new columns until
that information is populated in the metadata by running the query.
Lifecycle now:
- should be encapsulated in `api.card/result-metadata-async`. But we
only allow metadata to remain if is is valid metadata and the query has
not been edited. If the query has been edited (and new queries always
satisfy this since the previous query will be nil) we rerun the query to
get the metadata. No more drift. However, when dealing with datasets we
will attempt to blend the existing metadata into the newly computed
metadata so that metadata edits can persevere.
Other changes:
- removed an extra indirection in the card api that made it harder to
reason about when to recompute metadata
- moved the `combine-metadata` function into a util namespace and gave
some more generic parameter names
- deleted two tests that are no longer relevant after removing the
checksum on metadata. One had to do with how we hash floats vs ints
which is no longer relevant since we aren't hashing at all (RIP
checksum) and one checked that we accepted "valid" checksum-ed
metadata.
- new test that updating a card will update its metadata by rerunning
the query
- when validating the checksum, do not call s/validate since this WILL
THROW. Instead check `(nil? (s/check ...))`.
* Normalize query before comparing for changes
When comparing db vs api-sent query, need to normalize otherwise they
will always be different and our optimization will never kick in
* Fix ns
* Cleanup after ourselves
* Move `combine-metadata` util functions to a better namespace
* Allow for no query in api/card/:id patch
Cypress tests would do a
```javascript
describe("simple mode", () => {
beforeEach(() => {
cy.request("PUT", "/api/card/1", {
name: "Orders Model",
dataset: true,
});
});
```
The metadata testing stuff would always use the query from the PUT to
check whether to dump or retain the metadata. In these instances, it
would dump the metadata and then i guess ask the qp for the metadata for
a nil query. Similar scenario if anyone uses the API to update
descriptions, etc.
The method is PUT not PATCH, but in the saving function we have
```clojure
(u/select-keys-when
card-updates
:present #{:collection_id :collection_position :description :cache_ttl :dataset}
:non-nil #{:dataset_query :display :name :visualization_settings
:archived :enable_embedding :embedding_params
:result_metadata})
```
So we kinda have a PATCH anyways.
* Normalize metadata before blending
* Ensure api blends metadata correctly
When blending the metadata, need to update the field_ref to keyword the
original. Otherwise we are attempting to match up the field with ref
["field" "subtotal"] against [:field "subtotal"] and they don't match.
* Normalize metadata from api correctly
if you just call normalize/normalize it doesn't know not to change
`:field_ref` -> `:field-ref` so it gets out of whack. Need to pass it
the function appropriate for that level.
Also add a test that you can update _just_ the metadata. This actually
already worked for a subtle reason, and that is possibly buggy. The
reasoning is that the result_metadata is clobbered conditionally now
instead of unconditionally. If the async metadata stuff returns nil it
won't clobber and then whatever was sent over the wire goes in.
* Ensure previous metadata is there
When updating the query of a dataset, handle the case where just the
query is sent, and not the whole card. If the client didn't send the
previous metadata, we did not look it up in the database
* Card api always uses metadata from `result-metadata-async`
Originally this did
(assoc card-data :result_metadata (a/<! result-metadata-chan))
But I changed that so it only optionally did so when that channel had
information. The problem with this is that if there was :result_metadata
on the original request but the channel returned nil for some reason,
whatever was in the request was going in the db. This includes invalid
metadata, non-json strings, what ever. Obviously bad.
But we need to handle PATCHs where clients don't send metadata or they
_ONLY_ send metadata and no query, etc. So this always uses the results
of `result-metadata-async` so it can go through that logic and we always
clobber.
And remember that if we only update the result_metadata when it is
non-nil (`update-card-async!`). The end result is that switching from a
PATCH model where you expect the full object to a PUT where things can
be optional has lots of footguns. Some examples:
- only updating the query
- updating the query along with edits to the metadata on a card (throw
the metadata away)
- updating the query along with edits on a dataset (save the metadata
but blend it into the fresh computation of the query's metadata)
- updating the query but not sending along the metadata. Need to grab
from the db and blend in.
- Sending invalid metadata, changing the description. We want to toss
away the metadata but save the description.
Adds an endpoint to the cards which are basically materially like dashboard/:id/copy, only it's for card/:id/copy - POST to it, get a copy of that entity.
* Add has-user-setup setting
- iff no user is setup aka has-user-setup == false, then redirect to /setup
- when set from the env variable the setup-token is completely
immutable, so can remove clear-token!
* make /api/setup only setup accounts when not has-user-setup
- react router changes to respect 'has-user-setup' setting
- include docstring for print-setup-url
* allow tests to setup multiple users via POST /api/setup
* remove unused function hasSetupToken
* Adds a test to enforce a single creation only
- we now throw an ex-info with status-code 403 when has-user-setup and
/api/setup route is hit (with error message mentioning the route)
- more code review responses
- fixup test to not delete users, (or set locale to spanish...)
* addressing code review concerns
- *disallow... -> *allow... for the dynamic var name
- has-user-settings test: do not assume the test db has users in it
- update test
* Make `getRevisionEvents` accept params object
* Use "You" for current user's revision events
* Fix "moved to" revision display
* Add extra bottom padding to events timeline
* Extract conditions into variables
* Remove saving changes from `setColumnType` helper
* Use single POST request to create a model in cypress
* Add E2E test
* Display metadata editing revision events
* Change message
* Add `dataset` to Card type
* Fix "source-table" type
* Don't use original model query in ad-hoc mode
* Handle model using `/dataset` endpoint in Cypress
* Rerun query when entering model editor
* Make it possible to only close the details sidebar
* Close details sidebar when entering model ad-hoc mode
* Move `isAdHocModelQuestion` to data-modeling utils
* Handle TableInteracting column width reset event
* Fix old isAdHoc util usage
* Trim whitespace around commas in schema filtering code
Change impl of `schema-pattern->re-pattern` so that strings are trimmed before being joined on `|`
Add test to confirm
* Support overriding ROWCOUNT for SQL Server
Add new "ROWCOUNT Override" connection property for `:sqlserver`, which will provide a DB-level mechanism to override the `ROWCOUNT` session level setting as needed for specific DBs
Change `max-results-bare-rows` from a hardcoded constant to a setting definition instead, which permits a DB level override, and move the former constant default to a new def instead (`default-max-results-bare-rows`)
For `:sqlserver`, set the DB-level setting override (if the connection property is set), via the `driver/normalize-db-details` impl
Add test to confirm the original scenario from #9940 works using this new override (set to `0`)
Move common computation function of overall row limit to the `metabase.query-processor.middleware.limit` namespace, and invoke it from execute now, called `determine-query-max-rows`
Add new clause to the `determine-query-max-rows` function that preferentially takes the value from `row-limit-override` (if defined)
Waterfall can now have pretty colors and goals now. Y-axis positions reverted back to the way Dan liked less but is more congruent to the dynamic visualizations.
* Apply schema inclusion/exclusion filtering to sql-jdbc drivers
Update `sql-jdbc` namespaces to handle schema inclusion/exclusion patterns when filtering schemas
Add new generic schema inclusion/exclusion test for sql-jdbc drivers that define the property
Update Snowflake and Redshift driver manifests to include schema filtering property
Create `db-details->schema-filter-patterns` util fn to turn DB details into the inclusion/exclusion patterns
Move schema inclusion/exclusion filtering code to new namespace (since it's not strictly used by `:sql-jdbc` derived drivers)
Move existing tests accordingly
Add schema inclusion/exclusion check to the new `filtered-syncable-schemas` multimethod (and updating docstring)
Change `:redshift` impl of `filtered-syncable-schemas` to call the `:sql-jdbc` version instead
Use new multimethod instead for `filtered-syncable-schemas`, and have default impl of `syncable-schemas` call that
Mark `syncable-schemas` as deprecated and include notes on the new method (and update driver markdown file accordingly)
* Add PinnedItemCard component (#19765)
* convert Card component to typescript
* add open state class to EntityMenu
* Add PinnedItemCard component
* prevent the bubbling of EntityMenu events
* replace class with styled-component stuff
* define a const
* Replace pinned collection items table with pinned overview section (#19787)
* Add PinnedItemOverview component
* Replace the pinned table with overview
* add types to collection utils
* fix pinning + types
* remvoe the old PinnedItemsTable
* fix/remove e2e tests that no longer make sense
* replace 'Datasets' with 'Models'
* Update frontend/src/metabase/collections/components/PinnedItemOverview/PinnedItemOverview.tsx
Co-authored-by: Maz Ameli <maz@metabase.com>
* add unique text for root collection
Co-authored-by: Maz Ameli <maz@metabase.com>
* Collections page UI changes (#19792)
* make a bunch of ui tweaks to collections
* resize Titles for PinnedItemCards
* resize collection description headings
* Add tooltip to pin button
* Fix unit test
* fix e2e tests
* add a default description
* Tbody --> TBody
* Hide long titles & add tooltip to show entire title
Co-authored-by: Maz Ameli <maz@metabase.com>
* Add card visualizations to the pinned section of collections (#19813)
* First pass at card viz in collections
* minor style fixes
* move redux stuff up to CollectionContent
* padding tweaks
* Make the entire card clickable
* add generic error message
* Add propTypes
* a little cleanup
* create ActionMenu component
* Fix pin action fn
* Fix circular dep & storybook story
* actually import the function
* add comment // use EventSandbox
* Add preventDefault prop to avoid breaking things
* Make pinned cards drag-and-droppable into the collections sidebar (#19829)
* first pass at dnd for pinned cards
* use cards instead of viz for performance
* fix type
* Add tests for collection changes (#19851)
* first pass at dnd for pinned cards
* use cards instead of viz for performance
* fix type
* Add tests for collection changes
* fix Button import
* remove async
* Add empty pinned items section banner (#19857)
Fix e2e test
don't show empty states when loading
styling tweaks
Fix test
* datasets --> models
* fix dragging data models
* Make question icons in collections bg-dark
Co-authored-by: Maz Ameli <maz@metabase.com>
* Move repro for 12985 to "dashboard-filters" reproductions where it belongs
* Simplify filter tests by visiting question through a function
* Use `intercept` instead of `server/route`
* Extract `trend` visualization test
* Simplify steps for popover hover test
Reduces test time from ~12 to ~5s.
* Move tests over to binning reproductions where they belong
* Rename 6239 repro
* Update repro 4482
- Fix wrong issue references
- DRY code
* Change label for "enable-nested-queries"
* Fix query types compatibility
* Add utility to check if DB supports nested queries
* Accept store initial state for `renderWithProviders`
* Hide model button when nested queries disabled
* Handle nullish database in modeling utils
* Fix var
Static viz errors are of basically two kinds:
1. Someone futzed up their query, which we can't do anything about
2. We futzed up the rendering, which is our fault.
Display those two errors differently.
* Backport Debug QP improvements from #19754
* Enable test
* "Fuzzy" matching when looking for Fields in join source queries
* Add/update tests
* Test fix 🔧
* Remove trailing whitespace
* Fix a few docstrings
* Add whats new for driver authors in 0.42.0 [ci skip]
* Apply suggestions from code review [ci skip]
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Postgres: avoid unnecessary casts of DATE or TIMESTAMPTZ to TIMESTAMP
* Update src/metabase/util/honeysql_extensions.clj
* Update test/metabase/driver/postgres_test.clj
* Add drive scope to BigQuery client
Adding new drive auth scope in order to be able to query Google Drive files created in BigQuery as external tables
Adding new test to confirm such a table can be queried successfully
* Don't add `:join-alias` to `:field` clauses for fields that came from source query's source table
* Some test fixes 🔧
* Test fixes 🔧
* Test fix 🔧
* Test fix 🔧
* Add function returning column type icon
* Display column type icon in the results table
* Add `left` prop to SelectButton
* Display field type icon in SemanticTypePicker
* Remove "three_dots" icon
Replaced with "ellipsis" with background color
* Add `fallback` arg to `getSemanticTypeIcon`
* Switch to `getSemanticTypeIcon`
* Fix `getSemanticTypeIcon`
* RIP checksum
Now that users can edit the metadata the checksum prevents that and
would throw it away when edited. We'll treat the metadata as valid if it
conforms. The UI isn't really expected to modify type information anyways.
* Ns checker is a harsh disciplinarian
* Extract `fields` variable in dataset editor
* Allow extra onChange callback to form fields
* Only show metadata sidebar when field is selected
* Add onChange callback to semantic type picker
* Add action for overwriting results metadata
* Connect essential fields to form
* display_name
* description
* semantic_type
TBD: mapped database column for native datasets
* Fix missing import
* Fix sidebar inputs loosing focus while typing
* Ensure focused field is in sync
* Refactor FK and currency to be form fields
* Use `field.settings` to keep formatting changes
* Remove "display_as" from form definition
* Connect remaining fields
* Remove CurrencyPicker in favor of formatting input
* Fix incomplete field metadata
Results metadata doesn't contain everything we need for the editor.
'Global' metadata is the most complete, though it shouldn't be muted while in the editor, only once a user saves the changes
Now we only keep focused field ref in state and derive the metadata by merging global metadata and query's results metadata. Query results metadata is mutated by the editor, so merging them both will do the job
* Fix ViewSidebar prop type
* Fix prop type warnings
* Add default values for some fields
* Connect MappedFieldPicker
* Fix MappedFieldPicker closes after loading fields
Sidebar dependant on IDFields was rerendering once additional metadata was loaded, causing DataSelector to unmount
Fixed by moving IDFields subscription to FKTargetPicker directly (the only component that actually needs them)
* Hide metadata popovers in dataset editor
* Disable saving invalid dataset editor changes
Disables "Save changes" button if:
* there is at least one field with empty display name
* the underlying query is empty
* Add `onChangeSetting` prop to ColumnSettings
The existing `onChange` passes the whole settings object as an argument. On the other hand, `onChangeSetting` passes only the changed setting which is useful for storing metadata diff
* Keep field metadata diff in redux
* Ignore column width reset events in metadata editor
* Preserve metadata diff between query edits
* A few important changes:
- ensure that we don't throw away metadata and return an empty channel
in the card api if the query hasn't changed
- ensure api/dataset uses `:metadata/dataset-metadata`
- hydrate fields on api/table/card__id/query_metadata so that it has
target and fk information
- preserve id so this information can be recovered†
†: When a user enters an id we preserve it, but it also gets confusing
if that's the actual field's id or the user-identified id. This is
important when we drill down (and probably lots of other places too if
they start assuming an id means they know exactly what is going on).
A situation based on the orders table:
Suppose you make a native question
```sql
SELECT "PUBLIC"."ORDERS"."ID" AS "ID",
"PUBLIC"."ORDERS"."USER_ID" AS "user_id_bro", -- note this is renamed
"PUBLIC"."ORDERS"."PRODUCT_ID" AS "PRODUCT_ID",
"PUBLIC"."ORDERS"."SUBTOTAL" AS "SUBTOTAL",
"PUBLIC"."ORDERS"."TAX" AS "TAX",
"PUBLIC"."ORDERS"."TOTAL" AS "TOTAL",
"PUBLIC"."ORDERS"."DISCOUNT" AS "DISCOUNT",
"PUBLIC"."ORDERS"."CREATED_AT" AS "CREATED_AT",
"PUBLIC"."ORDERS"."QUANTITY" AS "QUANTITY"
FROM "PUBLIC"."ORDERS"
LIMIT 15
```
In my db, the "USER_ID" column is field 3 and points at field 22. Our db
knows that field 3 is called "USER_ID". But in the source query that
column is actually called "user_id_bro". And lots of places assume that
if there's a field's `:id` it knows what the name is. This is obviously
incorrect here. It is a very important distinction that this column "is
basically field 3 USER_ID" and this is literally the field "USER_ID".
So if you do a drilldown based on this information, the FE will issue a
query to "select field 3 from card__127" which expands to roughly
"select USER_ID from (select ... USER_ID as user_id_bro ...)" and this
query obviously will fail.
* Don't put nil on a channel, clean ns
i honestly don't know why the results-metadata and encrypt deps were
there. I didn't add them but they just got caught in the linter for some
reason. I don't understand it.
* Update tests to include target and has_field_values
* Correct final test
* Select enough information for FK to work in UI
* Pass mapped column's properties to BE
* Qualify which semantic_type FK/PK we remove
Previously always removed these semantic types. Now only remove when
they do not have an id there. One thing I'm confused about though is
that native fields should never have a semantic type anyways. There's no
way for one to end up there, certainly not FK or PK right? So i want to
delete this entire method because as I understand it it's not possible
to trigger its conditions except for the new user-entered information,
in which case we want to preserve it. But I have no idea why
Mr. Chesterton built this function so I don't want to delete it.
Bug reports and connecting them back to this change is so hard. Because
it can be things like "native query with an aggregation named "blah"
bucketed by month break when you drilldown into this column" and that's
because it includes a semantic_type that breaks something. It's all very
at-a-distance so hard to trace back to this.
* Fix failed queries handling
* Support custom column metadata editing
* Hide empty sections in semantic type picker
* Remove redundant import
* Fetch database ID fields in FKTargetPicker
* Allow aliasing EntityObjectLoader's entity prop
* Reset DataSelector's state when ID props change
* Fix MappedFieldPicker
* Fixed saved question picker regression
* Select description and settings when merging result_metadata
Annotate columns allows for settings to be preserved from
datasets. Example
```clojure
;; shows a little bar in the UI. quite nice
:settings {:show_mini_bar true, :decimals 0},
```
These settings were correctly preserved in annotate.clj but dropped in
the final merging in result_metadata.clj. Fun times in metadata land
* Let dataset revisions preserve metadata
* Only wrap form fields with custom onChange handler
This prevents form fields from extra re-renders
* Fix summarize sidebar behavior
* Fix e2e test
* Update cypress expectations
Previously expected really generic placeholder text in search boxes. But
in `api/table/card__id/query_metadata` we hydrate native columns with
`:has_field_values` so that when a user identifies an underlying field
on datasets, we have extra information.
This change comes about because the hydrate for `:has_field_values`
doesn't hit the db and look for field values but uses a heuristic. And
this heuristic works for native fields and finds that they are
searchable because when they are text or textlike and improves the
search bar.
Code from models/field.clj below
```clojure
(defn- is-searchable?
"Is this `field` a Field that you should be presented with a search widget for (to search its values)? If so, we can
give it a `has_field_values` value of `search`."
[{base-type :base_type}]
;; For the time being we will consider something to be "searchable" if it's a text Field since the `starts-with`
;; filter that powers the search queries (see `metabase.api.field/search-values`) doesn't work on anything else
(or (isa? base-type :type/Text)
(isa? base-type :type/TextLike)))
(defn- infer-has-field-values
"Determine the value of `has_field_values` we should return for a `Field` As of 0.29.1 this doesn't require any DB
calls! :D"
[{has-field-values :has_field_values, :as field}]
(or
;; if `has_field_values` is set in the DB, use that value; but if it's `auto-list`, return the value as `list` to
;; avoid confusing FE code, which can remain blissfully unaware that `auto-list` is a thing
(when has-field-values
(if (= (keyword has-field-values) :auto-list)
:list
has-field-values))
;; otherwise if it does not have value set in DB we will infer it
(if (is-searchable? field)
:search
:none)))
(defn with-has-field-values
"Infer what the value of the `has_field_values` should be for Fields where it's not set. See documentation for
`has-field-values-options` above for a more detailed explanation of what these values mean."
{:batched-hydrate :has_field_values}
[fields]
(for [field fields]
(when field
(assoc field :has_field_values (infer-has-field-values field)))))
```
* Ignore `has_field_values` for field literals
Field literals are field using a column title instead of a numeric ID. They're usually coming from native or nested queries.
Metadata propagation changes introduced for data models, made the BE send a guessed `has_field_values` property for field literals too. But searching through possible field values isn't properly supported yet and it's difficult to prevent `has_field_values` being sent to the FE, so this makes the FE ignore it for field literals
* Fix E2E test
* Fix field literal check
* Ensure `:has_field_values` only goes on fields with ids
when a user selects an id to back a native field, we want to hydrate the
field_value information and the target info. But _ONLY_ when they have
ids. The hydration for `:has_field_values` works on a heuristic, not by
hitting the db and seeing if we have field values and will give
information about native fields that do not have backing field information.
* Ensure no annotations on numeric field ids
we put the vector id on each field when there's no numeric id field. I
have no idea why and the comment `;; TODO -- what????` explains the
thinking there. So only annotate where we have numeric ids (fields that
we know how to identify fields in the db) and not when we have a made up
vector id
* Revert changes to FieldValuesWidget
* Fix character case
* Show `has_field_values` selector when ID is known
* Undo the testing for indiscriminant hydration
previously had put target and has_field_values on every native field
which doesn't work well for the UI. Had to clean up the remaining tests
to no longer expect it
* Select `:dataset` when creating a new question
when duplicating a question, we send over this info and need to select
it from the existing question.
* Ensure card is saved with boolean dataset info
largely just for testing which does not send a `:dataset false` property
in some cases. In regular usage there should always be an initial value
* Remove dead code
* Replace TODO with a meaningful comment
* Type Field type
* Select only what we need on the source card
Co-authored-by: dan sutton <dan@dpsutton.com>
* Forward form field ref
* Focus "display_name" when selecting a new field
* Hide query editor in metadata mode
This will remove it from the "tab-flow"
* Pass tabIndex prop to form text inputs
* Add basic tab flow for metadata editor
* Allow passing more props to SelectButton
* Allow passing props to AccordionList's search input
* Use numbers for tab indexes
* Fix SelectButton's props type
* Add special type picker to tab flow
* Fix initial field focusing
* Add "tab" icon
* Add basic TabHintToast component
* Show tab hint toast in metadata editor
* Export wrapped CustomFormField
* Forward SelectButton ref
* Add react-virtualized scrollToColumn prop to Table
* Don't open SemanticTypePicker on focus
* Open SemanticTypePicker on "Enter" key press
* Improve tabbing experience
* automatically scroll the InteractiveTable while tabbing through columns
* focus the first column on hitting "Enter" on the last column
* Remove failed experiment
It was my last hope to combine both props, but no lucj
* Animate sidebar when focused field changes
* Minor rename
* Fix mapped field picker component
* Pass visibility callbacks to trigger elements
* Handle focus for mapped field picker
* Pass searchPlaceholder to Select
* Pass triggerable's onClose prop to Select
* Add onClose prop to DataSelector
* Use Select from special type picker
* Handle focus for MappedFieldPicker
* Add :focus style for essential select inputs
* Fix jump from last to first column on tab key
* Convert SelectButton to TypeScript
* Add onClick prop to SelectButton
* Extract FormFieldDivider component
* Pass dataset into field metadata sidebar
* Add function to get field metadata form fields
* Simplify field visibility options generation
* Allow to provide custom form field widgets
* Add `renderSearchSection` prop to AccordionList
* Add SemanticTypePicker component
* Use SemanticTypePicker
* Add FieldSelector (DataSelector subset)
* Allow hiding DataSelector trigger expand control
* Add MappedFieldPicker component
* Use MappedFieldPicker for native datasets
* Simplify mapped field name formatting
* Fix dataset type usage
* Remove redundant props
* Disable fuzzy search for SemanticTypePicker
* Allow hiding empty AccordionList sections in search
* Don't show empty sections in SemanticTypePicker
* Add formatting section to field metadata form
* Fix FormFieldDivider margin
* Add FK target and currency pickers
* Hide column title from formatting section
* Wire up column formatting updates
* Convert ChartSettingsWidget to TypeScript
* Use styled-components for ChartSettingsWidget
* Add "variant" prop to ChartSettingsWidget
* Unify sidebar form UI
* Fix hidden column settings are forcefully shown
* Add "column type" field label
* Fix ChartSettingsWidget label size in defaul mode
* Use "radio" input for "view_as" field
* Use "Text" instead of "Off" for view_as field
* Use tab layout in field metadata sidebar
* Hide Formatting tab if missing formatting opts
* Fix hidden formatting fields behavior
* Fix E2E test using renamed label
* change view_as label to "Display as"
* Remove divider from expanded semantic type picker
* Move FormFieldDivider into sidebar's .styled file
* Choose "view_as" widget depending on options count
* use "radio" for exactly two options
* use "select" when there are > 2 options
* Change selected radio button look
* Use "text-dark" in sidebar form
* Set height for semantic type picker
* Use Field's displayName method for formatting
Co-authored-by: Maz Ameli <maz@metabase.com>
* add balloons icon
* add icon prop to semantic type metadata map
* add functions for getting semantic type icon and name
* update/rename DimensionLabel component
* Fix unit tests
* add tests for util fns
* fix e2e tests reliant on old label
* Remove hardcoded project-id from `:bigquery-cloud-sdk` tests
Pull out common helper fn to extract project-id from credentials
Update `bigquery_cloud_sdk_test.clj` to have a var that loads the service-account-json from the env var, then uses aforementioned function to determine project-id
Change sensitive-data-redacted-test to use an arbitrary/fake project-id to remove any sort of confusion
* Log a shortened version of the enterprise token
- when checking that an enterprise token is valid
- test captures the logs, and checks for no token
- in the test, regrettably we print the logs we need to capture
* fix namespace requires order
* remove "Using this URL to check ..." log line
- and update test
* respond to review comments
- no circular dependency with mt, so use that
- "fa3e"*16 -> random token
* print the shortened token one time, and remove let
* Update collection icon for collection permissions page
* Update collections icon for the serach filter page
* Update the collection icon in the "New" dropdown menu
We have a repo-wide CI run to check ESLint error anyway.
Meanwhile, for day-to-day development of static viz frontend, the main
webpack config is already equipped with ESLint check (along with hot
reload etc).
Our email tests can be a bit brittle. Sometimes we assert the first
email in the email inbox is some password reset email, but a race makes
it such that its a "logged in from new ip" email. Othertimes, we might
assert that there are only emails to a particular user but maybe another
test might have triggered an email to another user.
We often might just want two predicates: did the user receive an email
with this subject, or did the user receive an email with this body.
This simply adds those.
It should be safe from races as it can handle emails to multiple users
and multiple emails to the same user.
* Comparison for goals depends on alert_above_goal and if progressbar.
This commit sets up the timeseries predicate and writes tests for what I think the correct behaviours should be:
timeseries, alert above -> (<= goal-value value)
- (<= 5.9 6) true -> alert sent
- (<= 6 6) true -> alert sent
- (<= 6.1 6) false -> alert not sent
progressbar, alert above -> (<= goal-value value)
- (<= 5.9 6) true -> alert sent
- (<= 6 6) true -> alert sent
- (<= 6.1 6) false -> alert not sent
timeseries, alert below -> (>= goal-value value)
- (>= 6.1 6) true -> alert sent
- (>= 6 6) true -> alert sent. NOTE: this is to match prior behaviour, but I don't understand why it is this way?
- (>= 5.9 6) faluse -> alert not sent
progressbar, alert below -> (> goal-value value)
- (> 6.1 6) true -> alert sent
- (> 6 6) false -> alert not sent. NOTE: this is what should fix the bug (#10899)
- (> 5.9 6) faluse -> alert not sent
I think there may be a cleaner way to write and present the tests, so the next commit(s) will address that.
* Simplified Tests for goal-met? predicate.
The goal-met predicate is tested directly with mock data to make the tests more readable.
The change to `metabase.pulse/goal-met?` fixes#10899 while preserving the goal-met behaviour for any non-progress
goal. The behaviour should now be as follows:
```
| Timeseries? | alert_above? | goal | val | goal-met? |
+-------------+--------------+------+-----+-----------+
| f | t | 5 | 4 | f |
| f | t | 5 | 5 | t |
| f | t | 5 | 6 | t |
| f | f | 5 | 4 | t |
| f | f | 5 | 5 | f | <--- this is new behaviour
| f | f | 5 | 6 | f |
| t | t | 5 | 4 | f |
| t | t | 5 | 5 | t |
| t | t | 5 | 6 | t |
| t | f | 5 | 4 | t |
| t | f | 5 | 5 | t |
| t | f | 5 | 6 | f |
```
Otherwise, alerts should be triggered as before.
* added issue number to test
* fixed alignment issue
* swapped goal and value around in comparison for readability
* Fixing nits
* Timeseries and progress bar now follow same logic
I asked for clarification about when alerts fire in #ama-product in Metabase's Slack. There was consensus around the
idea that 'reaches the goal' does generally imply 'meets the goal', so we can use the same comparators between
progress bar and timeseries goals now.
This simplifies the goal-met? function again because it's no longer necessary to check if the given data is a
timeseries goal.
* Adjusted text in goal line alert modal
* ran prettier
* Fix duplicate model tooltip
* Fix archive model title
* Fix edit model modal title
* Fix model verification title
* Fix add model to dashboard title
* Fix failing unit test
* Update model's archive message
Move bulk of dataset-id normalization logic into new private helper fn
Perform app DB update for db-details after dataset-id has been turned into inclusion filter
Add test to confirm that normalization only happens once
* Turn this into a ~~dataset~~ model
* Model revision history copy updates
* Datasets -> Models copy update
* "Dataset management -> Model management" copy update
* This is a ~~dataset~~ model now
* Models in data picker copy changes
* Models in search app copy changes
* Update models copy in tests
* Update frontend/src/metabase/lib/revisions/revisions.unit.spec.js
Co-authored-by: Maz Ameli <maz@metabase.com>
* Update frontend/src/metabase/lib/revisions/revisions.js
Co-authored-by: Maz Ameli <maz@metabase.com>
* Fix and update e2e tests
Co-authored-by: Maz Ameli <maz@metabase.com>
* Extract models into a separate e2e group
* Split models tests into smaller logical units
* Keep `getCollectionItemRow` helper scoped to a single file
* Keep `testDataPickerSearch` helper scoped to a single file
* Update the name from `datasets` to `models`
* Don't update tables in a schema one by one
Code had wanted to log each table which is admirable. But on bigquery
there can be 15,000 tables and this will lock up calls to
/api/database. On stats they are taking about 18s for me. Need to do
once and let the db do the heavy lifting
* Also where schema is nil
* slightly better query
When the text representation gets reformatted, the old caret position
might not be sensible anymore. Hence, just push the caret to the end.
* Add e2e test for caret position after blur and refocus
Co-authored-by: Gustavo Saiani <gustavo@poe.ma>
* Add Pulse URL param to email settings
* Undo frontend changes -- make config only allowable by env var
* Make config enterprise only
* Address feedback
* Change namespace and token feature
* newline
* Rename var and add docs
Add functions to describe-database namespace for performing inclusive/exclusive schema filtering
Add tests for these functions in isolation
Update `:bigquery-cloud-sdk` driver and QP code to remove the single `dataset-id` conn-param
Changing :bigquery-cloud-sdk` driver to use dataset inclusion/exclusion filters instead
Updating `add.cy.spec.js` Cypress test to account for new structure
Add data upgrade (via `driver/normalize-db-details` implementation) to change `dataset-id` property to inclusion filter
Add test to confirm data upgrade
Adding server side expansion of `:schema-filters` type connection property
* pratt parser diagnostics
* ported over tests from custom parser branch
* updated error types to support pos/len for error markers
* updated compiler error to removed unnecessary plural
* updated resolver to use new error type and pass through node information
* updated error text in custom column e2e test
* updated diagnostics to use new resolver error class
* update recursive-parser passes to pass through node information
* roll back error text fix in e2e tests
* deleted fuzzing tests (they will be moved into the metabase/expression repo)
* this is weird, setting e2e error text back to "Expected expression"
* rolled back expression editor changes in handleInputBlur and moved diagnoseExpression first to fix exponent rewrite issue
* roll back changes to compileExpression
* update import paths in expressions case class identities get messed up
* Exclude archived collections from tree view
This is a bit more nuanced that it might first appear. The bug report is
the permissions editor for collections is showing child collections
which are archived. But the api sends back ALL collections, regardless
of archival state. So the FE knows to omit top-level archived
collections but not archived child collections.
So we thought about just having the FE remove all archived ones, but
best to remove it from the response. One worry is that we're not sure if
any callers want archived collections in the tree view or not. So right
now they are removed with a `?exclude-archived=true` query param, but if
we can demonstrate that no callers of the tree api ever want archived
items we can make this the behavior, rather than an optional behavior.
To verify that the tree api is sending back all collections, you can
check
```clj
collection=> (collection/collections->tree
{}
[{:archived false
:name "foo"
:id 1
:location "/"}
{:archived true
:name "foo"
:id 2
:location "/1/"}
{:archived true
:name "its archived"
:location "/"
:id 3}])
;; we have always sent back archived collections, both top level and children
({:archived false,
:name "foo",
:id 1,
:location "/",
:children ({:archived true,
:name "foo",
:id 2,
:location "/1/",
:children ()})}
{:archived true,
:name "its archived",
:location "/",
:id 3,
:children ()})
```
* hide archived collections from the collections permissions page (#19616)
* fix incorrect issue number
Co-authored-by: Alexander Lesnenko <alxnddr@users.noreply.github.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
This actually has nothing to do with waterfalls. Really, it's that static viz is brutally incompatible with bignum types, bigint and bigdec. However, nothing will fit in the screen anyways if you actually need a bignum or bigint type so the solution was to just coerce and be OK with exploding if it actually needed bignum range.
* Allow override of WEEK_START in Snowflake driver
Incorporate the following sources of WEEK_START, in decreasing order of precedence:
1) the connection string, a.k.a. additional-options (ex: WEEK_START=N)
2) the Metabase "start-of-week" setting (i.e. day of a week)
3) the default value (7: Sunday)
Adding new fn to the driver.common namespace to parse a connection string param out of additional-settings (plus a test for that)
Adding Snowflake driver test to ensure that all cases above are covered as expected, by invoking the DAYOFWEEK function on a set of fixed dates
Update documentation for start-of-week setting to clarify that it now influences this behavior
Swap out parse-additional-options-value for additional-options->map instead (more generic)
Update test accordingly
The previous Chevrotain-based parser had a few limitations, which need
to be taken account by the expression generator for the fuzzer. With the
new recursive parser (PR #19335), these limitations are not necessary
anymore:
* leading decimal digit, e.g. `.25`
* nested negatives, e.g. `-- 3`
* nested boolean NOT, e.g. `NOT NOT X`
If the expression is not valid, i.e. there is an error message
associated with processing it, do not blindly continue with it, since it
leads to a potential rewrite of the expression (the rest of
the parser/compiler can tolerate certain brokenness).
* JWT Login URL configured by user may have URL parameters
Since params may already exist in the login URL, a check is performed to make sure the generated URL
with 'return_to' param formats correctly. If a param already exists, all subsequent params should
have an '&
character.
* Sorted ns to pass linting
* added issue number to test string for added context.
One possible combination of the many ways to have multiplicity in charts is to have a single card with singular chart type with multiple y-axes. This makes the general line-area-bar chart compatible with this.
* Update suggestion from AdoptOpenJDK to Eclipse Temurin
Eclipse Temurin is the replacement for AdoptOpenJDK Hotspot, as the project now is part from Eclipse Foundation, and the OpenJ9 build is called now as Semeru.
* Update java-versions.md
* Update running-the-metabase-jar-file.md
* Rename setting/get and setting/all; GeoJSON via env var tests
* Fix typo (thanks @noahmoss)
* Support Database-local Settings in metabase.models.setting itself
* Rework Settings code so it can handle possibly-already-deserialized values
* Database-local Settings
* Remove empty part of docstring
* Appease linters
* Update dox again
* Use text.type for the new column
* Test fixes 🔧
* Test fix 🔧
* Test fix 🔧
* All negative integer values when setting :integer Setting with a string
* Handle nested queries which have agg at top level and nested
Previously when matching columns in the outer query with columns in the
inner query we had use id, and then recently, field_ref. This is
problematic for aggregations.
Consider https://github.com/metabase/metabase/issues/19403
The mbql for this query is
```clojure
{:type :query
:query {:aggregation [[:aggregation-options
[:count]
{:name "count"}]
[:aggregation-options
[:avg
[:field
"sum"
{:base-type :type/Float}]]
{:name "avg"}]]
:limit 10
:source-card-id 1960
:source-query {:source-table 1
:aggregation [[:aggregation-options
[:sum
[:field 23 nil]]
{:name "sum"}]
[:aggregation-options
[:max
[:field 28 nil]]
{:name "max"}]]
:breakout [[:field 26 nil]]
:order-by [[:asc
[:field 26 nil]]]}}
:database 1}
```
The aggregations in the top level select will be type checked as :name
"count" :field_ref [:aggregation 0]. The aggregations in the nested
query will be turned into :name "sum" :field_ref [:aggregation 0]! This
is because aggregations are numbered "on their level" and not
globally. So when the fields on the top level look at the metadata for
the nested query and merge it, it unifies the two [:aggregation 0]
fields but this is INCORRECT. These aggregations are not the same, they
just happen to be the first aggregations at each level.
Its illustrative to see what a (select * from (query with aggregations))
looks like:
```clojure
{:database 1
:query {:source-card-id 1960
:source-metadata [{:description "The type of product, valid values include: Doohicky, Gadget, Gizmo and Widget"
:semantic_type :type/Category
:coercion_strategy nil
:name "CATEGORY"
:field_ref [:field 26 nil]
:effective_type :type/Text
:id 26
:display_name "Category"
:fingerprint {:global {:distinct-count 4
:nil% 0}
:type {:type/Text {:percent-json 0
:percent-url 0
:percent-email 0
:percent-state 0
:average-length 6.375}}}
:base_type :type/Text}
{:name "sum"
:display_name "Sum of Price"
:base_type :type/Float
:effective_type :type/Float
:semantic_type nil
:field_ref [:aggregation 0]}
{:name "max"
:display_name "Max of Rating"
:base_type :type/Float
:effective_type :type/Float
:semantic_type :type/Score
:field_ref [:aggregation 1]}]
:fields ([:field 26 nil]
[:field
"sum"
{:base-type :type/Float}]
[:field
"max"
{:base-type :type/Float}])
:source-query {:source-table 1
:aggregation [[:aggregation-options
[:sum
[:field 23 nil]]
{:name "sum"}]
[:aggregation-options
[:max
[:field 28 nil]]
{:name "max"}]]
:breakout [[:field 26 nil]]
:order-by [[:asc
[:field 26 nil]]]}}
:type :query
:middleware {:js-int-to-string? true
:add-default-userland-constraints? true}
:info {:executed-by 1
:context :ad-hoc
:card-id 1960
:nested? true
:query-hash #object["[B" 0x10227bf4 "[B@10227bf4"]}
:constraints {:max-results 10000
:max-results-bare-rows 2000}}
```
The important bits are that it understands the nested query's metadata
to be
```clojure
{:name "sum"
:display_name "Sum of Price"
:field_ref [:aggregation 0]}
{:name "max"
:display_name "Max of Rating"
:field_ref [:aggregation 1]}
```
And the fields on the outer query to be:
```clojure
([:field
"sum"
{:base-type :type/Float}]
[:field
"max"
{:base-type :type/Float}])
```
So there's the mismatch: the field_ref on the outside is [:field "sum"]
but on the inside the field_ref is [:aggregation 0]. So the best way to
match up when "looking down" into sub queries is by id and then by name.
* Some drivers return 4.0 instead of 4 so make them all ints
* Handle dataset metadata in a special way
rather than trying to set confusing merge rules, just special case
metadata from datasets.
Previously, was trying to merge the "preserved keys" on top of the
normal merge order. This caused lots of issues. They were
trivial. Native display names are very close to the column name, whereas
mbql names go through some humanization. So when you select price
from (select PRICE ...) its an mbql with a nested native query. The
merge order meant that the display name went from "Price"
previously (the mbql nice name for the outer select) to "PRICE", the
underlying native column name. Now we don't special case the display
name (and other fields) of regular source-metadata.
Also, there were issues with mbql on top of an mbql dataset. Since it is
all mbql, everything is pulled from the database. So if there were
overrides in the nested mbql dataset, like description, display name,
etc, the outer field select already had the display name, etc. from the
database rather than allowing the edits to override from the nested
query.
Also, using a long `:source-query/dataset?` keyword so it is far easier
to find where this is set. With things called just `:dataset` it can be
quite hard to find where these keys are used. When using the namespaced
keyword, greping and finding usages is trivial. And the namespace gives
more context
* Add DimensionInfoPopover to DataSelector
* show popover when there isn't an attached query
* e2e test
* skip flaky e2e test
* use dimension display name as heuristic for showing popover
* Add TableInfoPopover to DataSelector
* handle undefined table prop and improve fetching logic
* Add popover to search
* add popover to qb header table name
* use TableInfo in TablePane
* Add a few e2e tests
* Make TablePane connected tables change sidebar
* add should("be.visible") before clicks on Sample Dataset
* add should("be.visible") before clicks when selecting a sample dataset
* added findByTextEnsureVisible command to Cypress
* docs clarify sql access in sandboxing
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* promote headings
* revisions from Luis feedback
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Add DimensionInfoPopover to TableInteractive column headers
* check for missing query before parsing dimension
* Don't render DimensionInfoPopover until all dependent data is available
* Add a min/max width to the DimensionInfoPopover content
* Don't show DimensionInfoPopover when user has no metadata
* Add e2e test for table column header popover
* some styling tweaks
* Move field value fetching back to FieldValuesList
* table styling tweaks
* lint fix
* add loading state for field values
* handle alternate field values format
* Format numbers
* make comment more specific
* Fix e2e test
* changes per review
* reconcile some styling differences btwn Dimension and Table popovers
Co-authored-by: Maz Ameli <maz@metabase.com>
* add TableInfoPopover
* Finish TableInfo implementation
* Add TableInfoPopover to TableBrowser
* TableInfo & TableInfoPopover styling tweaks
* add a cypress test for the /browse popover
* TableInfo unit tests update
* fix smoketest test
* shorter delay
* Add loading state for table metadata
* Fix table fk link
* make everything typescript
* add more unit tests
* fix types
* fix e2e test
* styles cleanup
* review changes
Co-authored-by: Maz Ameli <maz@metabase.com>
* Fix file upload of secret values
Fix logic in `db-details-client->server` to properly consider the `-options` suffixed connection property from the client version, to determine the treatment to be applied (i.e. base64 decode)
Change logic in `db-details-client->server` to `assoc` nil rather than `dissoc` unused keywords (so they override the saved db details on merge)
Update `driver.u/database->driver*` so that the engine loaded from the database instance is *always* a keyword
Update `metabase.test.data.oracle/connection-details` to pass the property vals through the client->server translation layer
Change `creator_id` in `secret` model to be nullable (since, when creating test DBs, the API current user is deliberately set to nil)
Switch the logic in Oracle SSL test to iterate on the file-path and uploaded variants to the test itself to get around weird CircleCI/environment issues
Use `rds_root_ca_truststore.jks` for the test instead since we don't need the full cacerts for this
Don't use yarn here since it will fetch dependencies specified in
package.json. Instead, use npm so that dependencies frozen in
package-lock.json, which metabase.github.io uses, are downloaded instead.
* Remove references to flag in FE code
* Remove flag from BE code
* Remove/update tests related to the feature flag
* Show widget-type selection of 'String' for category & location template tags
The fault-tolerant/recovery mode of the Chevrotain-based parser isn't
needed anymore since the suggestion was made to be lexical-based in
PR #15688. Therefore, there is no more reason to carry the Chevrotain
baggage. The grammar construct for Chevrotain was converted into a
simple recursive-descen parser. This represents a reduction from over
1,100 LoC to a leaner parser of only 270 LoC.
Also, we remove the dependency to Chevrotain, reducing the front-end
bundle by approx 18 KB.
As described in #15893, a number literal as the left-hand side in a
comparison isn't semantically valid. Therefore the expression generator
for the fuzzer has to avoid it.
* Preserve metadata and surface metadata of datasets
Need to handle two cases:
1. querying the dataset itself
2. a question that is a nested question of the dataset
1. Querying the dataset itself
This is a bit of a shift of how metadata works. Previously it was just
thrown away and saved on each query run. This kind of needs to be this
way because when you edit a question, we do not ensure that the metadata
stays in sync! There's some checksum operation that ensures that the
metadata hasn't been tampered with, but it doesn't ensure that it
actually matches the query any longer.
So imagine you add a new column to a query. The metadata is not changed,
but its checksum matches the original query's metadata and the backend
happily saves this. Then on a subsequent run of the query (or if you hit
visualize before saving) the metadata is tossed and updated.
So to handle this carelessness, we have to allow the metadata that can
be edited to persist across just running the dataset query. So when
hitting the card api, stick the original metadata in the middleware (and
update the normalize ns not to mangle field_ref -> field-ref among
others). Once we have this smuggled in, when computing the metadata in
annotate, we need a way to index the columns. The old and bad way was
the following:
```clojure
;; old
(let [field-id->metadata (u/key-by :id source-metadata)] ...)
;; new and better
(let [ref->metadata (u/key-by (comp u/field-ref->key :field_ref) source-metadata)] )
```
This change is important because ids are only for fields that map to
actual database columns. computed columns, case, manipulations, and all
native fields will lack this. But we can make field references.
Then for each field in the newly computed metadata, allow the non-type
information to persist. We do not want to override type information as
this can break a query, but things like description, display name,
semantic type can survive.
This metadata is then saved in the db as always so we can continue with
the bit of careless metadata saving that we do.
2. a question that is a nested question of the dataset
This was a simpler change to grab the source-metadata and ensure that it
is blended into the result metadata in the same way.
Things i haven't looked at yet: column renaming, if we need to allow
conversions to carry through or if those necessarily must be opaque (ie,
once it has been cast forget that it was originally a different type so
we don't try to cast the already cast value), and i'm sure some other
things. But it has been quite a pain to figure all of this stuff
out. Especially the divide between native and mbql since native requires
the first row of values back before it can detect some types.
* Add in base-type specially
Best to use field_refs to combine metadata from datasets. This means
that we add this ref before the base-type is known. So we have to update
this base-type later once they are known from sampling the results
* Allow column information through
I'm not sure how this base-type is set for
annotate-native-cols. Presumably we don't have and we get it from the
results but this is not true. I guess we do some analysis on count
types. I'm not sure why they failed though.
* Correctly infer this stuff
This was annoying. I like :field_ref over :name for indexing, as it has
a guaranteed unique name. But datasets will have unique names due to a
restriction*. The problem was that annotating the native results before
we had type information gave us refs like `[:field "foo" {:base-type
:type/*}]`, but then this ruined the merge strategy at the end and
prevented a proper ref being merged on top. Quite annoying. This stuff
is very whack-a-mole in that you fix one bit and another breaks
somewhere else**.
* cannot have identical names for a subselect:
select id from (select 1 as id, 2 as id)
** in fact, another test broke on this commit
* Revert "Correctly infer this stuff"
This reverts commit 1ffe44e900.
* Annotate but de-annotate in a way
To combine metadata from the db, really, really want to make sure they
actually match up. Cannot use name as this could collide when there are
two IDs in the same query. Combining metadata on that gets nasty real
quick.
For mbql and native, its best to use field_refs. Field_refs offer the
best of both worlds: if id, we are golden and its by id. If by name,
they have been uniquified already. So this will run into issues if you
reorder a query or add a new column in with the same name but i think
that's the theoretical best we can do.
BUT, we have to do a little cleanup for this stuff. When native adds the
field_ref, it needs to include some type information. But this isn't
known until after the query runs for native since its just an opaque
query until we run it. So annotating will add a `[:field name
{:base_type :type/*}]` and then our merging doesn't clobber that
later. So its best to add the field_refs, match up with any db metadata,
and then remove the field_refs.
* Test that metadata flows through
* Test mbql datasets and questions based on datasets
* Test mbql/native queries and nested queries
* Recognize that native query bubbles into nested
When using a nested query based on a native query, the metadata from the
underlying dataset is used. Previously we would clobber this with the
metadata from the expected cols of the wrapping mbql query. This would
process the display name with `humanization/name->human-readable-name`
whereas for native it goes through `u/qualified-name`.
I originally piped the native's name through the humanization but that
leads to lots of test failures, and perhaps correct failures. For
instance, a csv test asserts the column title is "COUNT(*)" but the
change would emit "Count(*)", a humanization of count(*) isn't
necessarily an improvement nor even correct.
It is possible that we could change this in the future but I'd want it
to be a deliberate change. It should be mechanical, just adjusting
`annotate-native-cols` in annotate.clj to return a humanized display
name and then fixing tests.
* Allow computed display name on top of source metadata name
If we have a join, we want the "pretty" name to land on top of the
underlying table's name. "alias → B Column" vs "B Column".
* Put dataset metadata in info, not middleware
* Move metadata back under dataset key in info
We want to ensure that dataset information is propagated, but card
information should be computed fresh each time. Including the card
information each time leads to errors as it erroneously thinks the
existing card info should shadow the dataset information. This is
actually a tricky case: figuring out when to care about information at
arbitrary points in the query processor.
* Update metadata to :info not :middleware in tests
* Make var private and comment about info metadata
Remove any parent fields from the field list when building projections, which should have the same effect as before (only child fields selected) while being compatible with MongoDB 4.4+
Add test to confirm the parent fields are removed
* Patch from a 1:1 with Dan.
This is a set of changes made while pairing with Dan, where we tried a few different approaches to getting the proper site url in the email.
* Changed predicate inside forgot password test
* Remove hostname as arg for password reset email
The email template was changed to use siteUrl, so hostname is unused.
* use DateAllOptionsWidget for single date and range date filters (#19360)
* use DateAllOptionsWidget for single date and range date filters
* fix specs
* fix specs
* add more specs
* Patch from a 1:1 with Dan.
This is a set of changes made while pairing with Dan, where we tried a few different approaches to getting the proper site url in the email.
* Changed predicate inside forgot password test
* Remove hostname as arg for password reset email
The email template was changed to use siteUrl, so hostname is unused.
Co-authored-by: Alexander Lesnenko <alxnddr@users.noreply.github.com>
* Handle unauthed/data mismatch dashcard query builder states better
* only run verify fn when deserializedCard has parameters
* fix unit test description
* add FieldFingerprintInfo component to DimensionInfo
* add getFieldValues selector to the fields entity
* add fingerprint data to example page
* prettier fix
* add a defined const for '35' value
* Refactor: move Card and Dashboard QP code into their own qp.* namespaces
* Disable extra validation for now so a million tests don't fail
* WIP
* Validate template tag :parameters in query in context of a Card
* Fixes
* Disable strict validation for now
* Test fixes [WIP]
* Make the parameter type schema a little more forgiving for now
* Tests & test fixes 🔧
* More test fixes 🔧
* 1. Need more tests
2. Need to actually validate stuff
* More test fixes. 🔧
* Test fixes (again)
* Test fix 🔧
* Some test fixes / PR feedback
* Disallow native queries with a tag widget-type of "none"
Template tags with a widget-type that is undefined, null, or "none" now
cause the query's isRunnable method to return false. Existing questions
that have this defect won't be runnable until they are resaved with a
set widget-type.
* Fix prettier error
* add snippet and card types to validation pass
* Make sure template tag map keys + `:names` agree + test fixes
* Have MBQL normalization reconcile template tags map key and :name
* Test fix 🔧
* Fix tests for Cljs
* Fix Mongo tests.
* Allow passing :category parameters for :text/:number/:date for now.
* Dashboard subscriptions should use qp.dashboard code for executing
* Make sure Dashboard QP parameter resolution code merges in default values
* Add a test for sending a test Dashboard subscription with default params
* Prettier
* If both Dashboard and Card have default param value, prefer Card's default
* Test fix 🔧
* More tests and more fixes 😢
Co-authored-by: Dalton Johnson <daltojohnso@users.noreply.github.com>
* Bump log4j from 2.14.1 to 2.15.0
* Disable failing logging tests when bumping log4j
0day in log4j requires bump in dependency. These tests look for logs in
testing but our test logger doesn't seem to have levels set
correctly. The disease is certainly worse than the remedy in this case
and each instance is annotated with the reason it is disabled, and we
can reenable them in calmer waters
* Fix unused ns
Co-authored-by: Youngho Kim <miku@korea.ac.kr>
Co-authored-by: dan sutton <dan@dpsutton.com>
* Added URL to test for repro of issue #14491
* Allow alphanumeric and _ characters in domains.
Hostnames and domains are not the same. Hostnames have the restriction of alphanum, hyphen, and must start with an
alpha char. This is what our regex seemed to match before.
Relevant RFC for domain names:
[https://www.rfc-editor.org/rfc/rfc2181#section-11](RFC2181 Section 11)
Relevant RFC for hostnames:
[https://www.ietf.org/rfc/rfc1123.html#section-2.1](RFC1123 Section 2.1)
It seems that our url? regex is for hostnames, and is based on the UrlValidator code from apache commons. Though the
issue #14491 author brings up the RFC1034 (which links to RFC1123 with a bit of a spec) to indicate that underscores
SHOULD be allowed, they aren't really in that case. But it seems that there's wiggle room in allowing it for 'domains'
generally, and perhaps we should permit the use of underscores in this URL validation to allow for more configuration
options for our users.
A stackoverflow page with helpful context and explanantion which helped me think about this:
https://stackoverflow.com/questions/2180465/can-domain-name-subdomains-have-an-underscore-in-it
* Fixed spacing
* docs - expanding description of LDAP features
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* docs - edits to LDAP docs based on feedback
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* first pass at adding parameters to gui qb
* handle refresh of qb when there are parameters
* second pass
* minor refactoring
* correctly map dashboard params to template tag params
* make click behavior question params work again
* lint fix
* run with dsahboard endpoint in query builder
* fix parameter click behavior
* fix title drill navigating to new question
* add some extra care when setting props on card
* refactor getUrlWithParameters & sprinkle comments
* fix and remove irrelevant unit tests in meta/Card
* fix metadata fetch race condition
* refactor logic
* fix click behavior
* rmv unit tests for function that no longer exists
* add Question unit tests
* add dashboardId to card serialization in qb
* add more title drill e2e tests
* rmv unneeded code in qb initialization
* fix click behavior query param problem
* add targets unit tests
* dashboards unit test + simplification of fn logic
* add cards unit test
* prevent passing of nil parameterValues to native qb
Combo type and multiple cards for static viz BE. These go into one FE endpoint but are two really separate things in BE. This one conforms to the FE type but the price is that the previous more-dynamic BE types needed to be changed to conform: this will require a refactoring to whack the js-viz types that already exist, when the FE is refactored also.
* Rewrite DashCardCardParameterMapper to use tippy + be functional
* add unauthed state
* make the mapper popover interactive
* make Popover and TippyPopover use same selector
* add targets unit tests
* add e2e test for this specific scenario
* kill tippyPopover util because selectors are now the same
* fix e2e tests that rely on anchor tag
* delete old component
* rmv console.log call
Caching node_modules (see PR #19202) doesn't seem to _always_ work for
Percy CI workflow, there must be some non-deterministic behavior
somewhere.
Until we figure out that, skip node_modules caching for now to unblock
Percy CI runs.
* select none keeps the first one selected
* make "None" and "All" lowercase
* fix select none when a query has joins
Co-authored-by: Maz Ameli <maz@metabase.com>
* Update presto-kerberos-docker branch used by run-presto-kerberos-test
Point the Presto Kerberos integration test to a `hacked-dec-2021` branch of `metabase/presto-kerberos-docker`, which contains
* the test_data catalog, which was added under `add-test_data-catalog`
* an updated download location for the `forego` binary dependency, which we now host
Add new deps for finding close misspellings of map keys (spell-spec) and printing more readable spec errors (expound)
Adding spec definitions for driver YAML format
Add call to driver YAML validation from verify.clj
Fixing a couple issues in existing drivers found by the validation
* Only select type information on type inference of coalesce
This previously just selected hte inferred information from the second
argument to the coalesce. For fields this would return the entire field
object which includes an id.
Later, in `source-metadata->fields` this assumes that since an id is
present, we should select `[:field id nil]`, but this confused that the
type of the coalesce column is the same as that field-id, not that we
want to select that field id.
```clojure
add-source-metadata=> (pprint
(mbql-source-query->metadata {:source-table 2
:joins [{:fields [[:field
22
{:join-alias "People - User"}]
[:field
15
{:join-alias "People - User"}]]
:source-table 3
:condition [:=
[:field 3 nil]
[:field
22
{:join-alias "People - User"}]]
:alias "People - User"}]
:expressions {:coalesce [:coalesce
[:field 3 nil]
[:field
22
{:join-alias "People - User"}]]}
:aggregation [[:count]]
:breakout [[:expression "coalesce"]]}))
({:semantic_type :type/FK,
:table_id 2,
:coercion_strategy nil,
:name "coalesce", ;; merging maintains the correct name
:settings nil,
:field_ref [:expression "coalesce"], ;; merging maintains the field ref
:effective_type :type/Integer,
:parent_id nil,
:id 3, ;; the type inference selected
;; the entire field, including its id
:display_name "coalesce",
:fingerprint {:global {:distinct-count 929, :nil% 0.0}},
:base_type :type/Integer}
{:name "count",
:display_name "Count",
:base_type :type/BigInteger,
:semantic_type :type/Quantity,
:field_ref [:aggregation 0]})
nil
```
updating to only select the type information:
```clojure
add-source-metadata=> (pprint
(mbql-source-query->metadata {:source-table 2
:joins [{:fields [[:field
22
{:join-alias "People - User"}]
[:field
15
{:join-alias "People - User"}]]
:source-table 3
:condition [:=
[:field 3 nil]
[:field
22
{:join-alias "People - User"}]]
:alias "People - User"}]
:expressions {:coalesce [:coalesce
[:field 3 nil]
[:field
22
{:join-alias "People - User"}]]}
:aggregation [[:count]]
:breakout [[:expression "coalesce"]]}))
({:name "coalesce",
:display_name "coalesce",
:base_type :type/Integer,
:effective_type :type/Integer, ;; no more field information beyond
;; the types
:coercion_strategy nil,
:semantic_type :type/FK,
:field_ref [:expression "coalesce"]}
{:name "count",
:display_name "Count",
:base_type :type/BigInteger,
:semantic_type :type/Quantity,
:field_ref [:aggregation 0]})
```
* Update question in repro now that it doesn't fail on the backend
previously this question would actually fail on the backend and
independently the frontend would go into a stack overflow killing the
page. Fixing the bug actually broke the test as it asserted that the
query would break and look for the error message. Now that the query
returns assert we can find the name of the question "test question"
* Correct same bug for case
When inferring type for a case, we were returning the type information
for the first field we could find in the case statement. And that
information included the field id. We always merge the current column's
name on top of that to remove the analyzed field's name and other
things, but the field-id would remain. `source-metadata->fields` in
add_implicit_clauses will add a field reference if it finds a field id.
Before, the following was returned
```clojure
({:description "The date and time an order was submitted."
:semantic_type :type/CreationTimestamp
:table_id 2
:coercion_strategy nil
:unit :month
:name "CREATED_AT"
:settings nil
:source :breakout
:field_ref [:field
4
{:temporal-unit :month}]
:effective_type :type/DateTime
:parent_id nil
:id 4
:visibility_type :normal
:display_name "Created At"
:fingerprint {:global {:distinct-count 9998
:nil% 0.0}
:type {:type/DateTime {:earliest "2016-04-30T18:56:13.352Z"
:latest "2020-04-19T14:07:15.657Z"}}}
:base_type :type/DateTime}
{:description "The raw, pre-tax cost of the order. Note that this might be different in the future from the product price due to promotions, credits, etc."
:semantic_type :type/Quantity
:table_id 2
:coercion_strategy nil
:name "distinct case"
:settings nil
:source :aggregation
:field_ref [:aggregation 0]
:effective_type :type/Float
:parent_id nil
:id 6
:visibility_type :normal
:display_name "distinct case"
:fingerprint {:global {:distinct-count 340
:nil% 0.0}
:type {:type/Number {:min 15.691943673970439
:q1 49.74894519060184
:q3 105.42965746993103
:max 148.22900526552291
:sd 32.53705013056317
:avg 77.01295465356547}}}
:base_type :type/BigInteger})
```
The important bit above is that the metadata for the "distinct case" has
the correct field_ref but also has a field id.
After, we now return the following metadata:
```clojure
({:description "The date and time an order was submitted."
:semantic_type :type/CreationTimestamp
:table_id 2
:coercion_strategy nil
:unit :month
:name "CREATED_AT"
:settings nil
:source :fields
:field_ref [:field
4
{:temporal-unit :default}]
:effective_type :type/DateTime
:parent_id nil
:id 4
:visibility_type :normal
:display_name "Created At"
:fingerprint {:global {:distinct-count 9998
:nil% 0.0}
:type {:type/DateTime {:earliest "2016-04-30T18:56:13.352Z"
:latest "2020-04-19T14:07:15.657Z"}}}
:base_type :type/DateTime}
{:base_type :type/BigInteger
:effective_type :type/Float
:coercion_strategy nil
:semantic_type :type/Quantity
:name "distinct case"
:display_name "distinct case"
:source :fields
:field_ref [:field
"distinct case"
{:base-type :type/BigInteger}]}
{:semantic_type :type/Quantity
:coercion_strategy nil
:name "cc"
:expression_name "cc"
:source :fields
:field_ref [:expression "cc"]
:effective_type :type/Float
:display_name "cc"
:base_type :type/Float})
```
* Add tests looking for full field analysis without id
Use an improved semantic validation layered on top of the existing
resolver (around ~40 LoC) instead of its own dedicated logic
(~240 LoC). Not only the improved type-checker is leaner and faster,
it's also more accurate in some corner cases.
The type-checking process is also deferred to the compilation stage,
instead of earlier in the parsing stage.
* Enable Postgres driver to use secrets for configuring SSL parameters
Adding secret related properties for SSL options to Postgres driver
Move `conn-props->secret-props-by-name` to secret.clj since it's needed directly there, too
Add `us-east-2-bundle.pem` to `test-resources/certificates` for testing Postgres with SSL connectivity to our RDS instance (and a README.md explaining how it differs from the existing `ssl` directory)
Updating `value->file!` to have better logic for building the error message when the existing file is not found (for better UX from the Database admin page)
Updating frontend to support the `visible-if` value being an array, in which case any value may match the current details value, in order for that field to be visible
Adding secret related properties for SSL options to Postgres driver
Updating CircleCI config.yml to refer to the RDS instance when running Postgres SSL test
Implement server side expansion of transitive visible-if dependency chain
Update shouldShowEngineProvidedField on client to consider multiple key/value pairs in the visible-if map, and only show the field if ALL are true
Adding new test to confirm transitive visible-if resolution, and cycle detection
Add Cypress test for SSL field visibility
* Add tests for buildCollectionTree
* Rename test objects for better readability
* Allow filtering collection tree items by models
* Hide collections without datasets in dataset picker
* Keep parent nodes if they don't have target models
* Require data permissions to set parameter mappings
* Impl for sandboxed permissions
* Allow updating dashboard if parameters haven't changed; more tests
* Fix premature i18n lookups
* Fix google-api-client version from google driver
Update google-api-client version `1.30.7` -> `1.32.1`, so that `google-http-client` ends up as `1.39.2` to match the other drivers
Add explicit `google-http-client-jackson2` dependency for google driver since it's no longer included transitively for some strange reason, but the `:google` driver itself directly uses it
Add new CircleCI test run for multiple Google related drivers (to ensure there aren't cross-driver classpath issues)
* Limit autocomplete results for tables and fields
Add limit of 50 total fields or tables returned from the autocomplete_suggestions API, which can be a mixture of tables and/or fields
Adding test that confirms results are limited properly
Field references in custom expression must be correctly resolved as
dimensions, segments, or metrics. For instance, `[Rating] > 4 AND
[Expensive]` means that [Rating] is a dimension and [Expensive] is a
segment.
Before this change, this process was carried out inside the parser. This
led to various corner cases which were not handled well, e.g.
`COALESCE(CASE([X], [Y]))` apparently resolved both [X] and [Y] as
dimensions. This was incorrect, as [X] has to be a segment, since it's
part of the conditional clause in CASE().
With this change, all field references are resolved as an additional
pass. Thus, the resolver has the full visibility of the operand types
for both unary and binary operators, as well as argument types of all
functions, including variadic ones such as CONCAT, COALESCE, and CASE.
* XYChart
* support axes formatting, support negative values for bars and areas
* calculate chart bounds based on axes presence, ticks sizes
* Add legend
* legend for single column, more examples
* polish, unit tests
* clean mess
* fix XYChart renders bars with the same names, add demo for an edge case
* goal line support
prettier
* legend folder rename
Update bigquery and googleanalytics drivers dependencies to newer versions, such that their transitive dependencies on google-api-client/google-http-client match the versions used in the new BigQuery driver
* Code cleanup
* Make the linters happy
* Add pivot version of the new endpoints
* implement usage of the new dashboard card query endpoint (#19012)
* add new endpoints to services.js
* replace CardApi.query with the new endpoint + pass dashboardId
* add parameter id to parameter object found on query body
* run dashchards using card query endpoint when they're new/unsaved on dashboard
* Update endpoint references in e2e tests
* Remove count check from e2e test
We can make sure a double-mapped parameter filter shows results from
both fields, but unfortunately with the new endpoint, the results don't
seem to work anymore. I think that's OK? Maybe?
* skip corrupted filters test
the query endpoint now results in a 500 error caused by a schema
mismatch
* fix a few e2e intercepted request mismatches
* unskip filter corruption test and remove the part that no longer works
Co-authored-by: Dalton <daltojohnso@users.noreply.github.com>
If the expression is invalid due to an unterminated string (perhaps the
user is still not finished typing the expression), do not try to suggest
possible matches (functions etc) for an identifier following that
dangling quote.
* add tippy and react-is
* add tippy styles to vendor.css
we'll probably override most things but it handles
apply the triangle to tooltips
* reimplement Tooltip using tippy
* reimplement ChartTooltip using Tooltip
* add tooltip theme styling
* fix StoreLink styling
* remove TooltipPopover
* add util for easy mapping to innerRef
* use innerRef util with StoreLink target
* react-is type defs
* convert Tooltip into tsx
* convert styled-components util into typescript
* fix Tooltip fallback return
* use Tooltip in Icon over Tooltipify
* remove Tooltipify
* pass ref all the way through Icon
* fix DimensionListItem styling caused by missing ref
* fix ChartTooltip event target bug
* fix unit test by making Icon.tsx grosser
* replace popover() util with tooltip() in cy tests
* fix random broken e2e test
* improve types in Tooltip
* move everything to a separate folder + add tests
* add more examples
* rmv accidental deps
* fix run button styling
* add Link tooltip prop to avoid tooltip on icon
* wrap MetabotLogo in forwardRef
* support reduced motion
* remove flow type from dom.js so we can use in ts files
* pass isEnabled prop to tippy's disabled prop
once you've activated "control mode" in tippy you can't turn it off so
using "visible" as a toggle doesn't work. instead, use disabled.
* fix uncentered tooltip the right way
tippy includes padding/margin in its centering calculations, unlike
tether, so we will need to be careful about adding one-sides
padding/margins to a tooltip target, otherwise the tooltip will appear
offcenter
* workaround for absolutely positioned element
the tooltip appeared on the far end of the target because of a child div
that is absolutely positioned. quick fix is to instead target the icon
* fix positioning of HintIcon tooltip
* lint fix
* add placement prop
* fix styled component tooltip target styling
* place QuestionNotebookButton tooltip below
* Make the NativeQueryButton an actual button
* set tooltip placement to bottom
* lint fix warning
* Display dataset model name in recent searches
* Fix dataset URLs in activity
* Fix icons in activity recent views
* Distinguish datesets from questions in activity
* Fix message
Had wanted to make a heuristic that would look at timestamp of the
bundle and timestamps of the sources and compare. But this could fail
when switching branches. Simpler and easier to just throw a warning on
all failures
```clojure
js-svg-test=> (binding [*test-out* *out*] (run-tests))
Testing metabase.pulse.render.js-svg-test
FAIL in (foo) (js_svg_test.clj:28)
[PRO TIP] If this test fails, you may need to rebuild the bundle with `yarn build-static-viz`
expected: 1
actual: 2
diff: - 1
+ 2
Ran 8 tests containing 29 assertions.
1 failures, 0 errors.
{:test 8, :pass 28, :fail 1, :error 0, :type :summary}
```
For posterity, attempt using timestamps was
```clojure
(deftest bundle-up-to-date-test
(let [js-bundle (io/file (io/resource js-svg/bundle-path))
to-date (fn [epoch] (java.time.Instant/ofEpochMilli epoch))]
(is (.exists js-bundle) bundle-recreate-warning)
(when (.exists js-bundle)
(let [source-files (file-seq (io/file "frontend/src/metabase/static-viz"))
most-recent-source (apply max-key #(.lastModified %) source-files)]
(is (.isAfter (to-date (.lastModified js-bundle))
(to-date (.lastModified most-recent-source)))
bundle-recreate-warning)))))
```
Docstring for `regex-email-bodies` states:
> "Return messages in the fake inbox whose body matches the
regex(es). The body will be replaced by a map with the stringified regex
as it's key and a boolean indicated that the regex returned results."
But it does ont return messages in the fake inbox whose body matches any
of the regex. It returns them all regardless, with information if they
match or not. Simply correct this behavior to match the docstring
```clojure
{"rasta@metabase.com"
({:from "notifications@metabase.com",
:to #{"rasta@metabase.com"},
:subject "We've Noticed a New Metabase Login, Rasta",
:body {"https://metabase.com/testmb" false, "has any results" false, "My question" false}}
{:from "notifications@metabase.com",
:to #{"rasta@metabase.com"},
:subject "We've Noticed a New Metabase Login, Rasta",
:body {"https://metabase.com/testmb" false, "has any results" false, "My question" false}}
{:from "notifications@metabase.com",
:to #{"rasta@metabase.com"},
:subject "You set up an alert",
:body {"https://metabase.com/testmb" true, "has any results" true, "My question" true}})}
```
* Add SSL keystore and truststore secret properties to Oracle driver YAML files
Update Oracle driver to set keystore and truststore connection options from secret values
Adding new `select-keys-sequentially` helper function for dealing with assertions on transformed conn props
Add new function to the secret namespace to return a lighter weight "secret map", and remove that functionality from the `handle-db-details-secret-prop!`, so that it can be reused from connection testing
Modifying CircleCI to set the corresponding truststore secret conn prop vars instead of JVM-level truststore settings, as the test had been doing previously
Expand Oracle test connection details to incorporate all SSL related properties, and also make it a function instead of delayed def (so changes can be picked up later)
Misc. fixes in Oracle driver YAML files
Update `exception-classes-not-to-retry` to include SSLHandshakeException
Fix misc issues in secret and database model code
Update CircleCI config to use correct secret based env var keys and values
Add *database-name-override* dynamic var in `metabase.test.data.interface` to allow for DB name overriding
Fix up ssl connectivity test so all parts pass successfully
Get rid of crazy with-redefs to swap in the existing test-data's schema, and instead, just dropping the DB if it was created under a different schema (similar to what H2 does)
Co-authored-by: Gustavo Saiani <gustavo@poe.ma>
* added links and edited headings
* documented getCallerClass error
* Added create directory section
* Wording and spacing edits per Jeff feedback
* Fixing error in ToC from last commit
* punctuation and spacing
Co-authored-by: ft-metabase <ft-metabase@ft-metabases-MacBook-Pro.local>
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* add fn that maps from our locale string to moment's
* use moment with locales
* add a require for the specific locale
* don't require en
* ensure locale is lowercased for moment more generally
* default locale to a string
* Get rid of the n-queries
old logic: select last view logs, and then for each one, select its
associated model (card, table, etc).
new logic: get the last view logs, and do three batch queries, one each
to card, dashboard, table, for the ids from the view logs
* Mark datasets in api/activity/recent_views
* Get rid of `(db/delete! Activity)` in tests
* Handle datasets in activity list
* Change query_cache.results column to correct blob type
Add Liquibase changeSet to modifyDataType of query_cache.results to ${blob.type} (an idempotent operation on most DBs)
Add schema migrations test to confirm, which simulates the broken app DB state, then runs the migration, then confirms the fixed type
* Speed up interger or string recognition
jeff helpfully pointed this out. I did some repl benchmarks
```clojure
(comment
(doseq [x ["3" "bob"]]
(dotimes [_ 3] (time (dotimes [_ 10000] (try (Integer/parseInt x)
(catch NumberFormatException _ x))))))
(doseq [x ["3" "bob"]]
(dotimes [_ 3] (time (dotimes [_ 10000] (if (re-matches #"\d+" x)
(Integer/parseInt x)
x))))))
tiles-test=> (doseq [x ["3" "bob"]]
(dotimes [_ 3] (time (dotimes [_ 10000] (try (Integer/parseInt x)
(catch NumberFormatException _ x))))))
"Elapsed time: 5.72825 msecs"
"Elapsed time: 1.531042 msecs"
"Elapsed time: 0.797041 msecs"
"Elapsed time: 32.120875 msecs"
"Elapsed time: 27.848375 msecs"
"Elapsed time: 23.820542 msecs"
nil
tiles-test=> (doseq [x ["3" "bob"]]
(dotimes [_ 3] (time (dotimes [_ 10000] (if (re-matches #"\d+" x)
(Integer/parseInt x)
x)))))
"Elapsed time: 5.356417 msecs"
"Elapsed time: 1.97225 msecs"
"Elapsed time: 1.868708 msecs"
"Elapsed time: 1.214666 msecs"
"Elapsed time: 1.185625 msecs"
"Elapsed time: 1.19025 msecs"
nil
tiles-test=>
```
* Only select lat/lon fields in api/tiles
We required the index of lat and long columns, ran the whole query, and
then just grabbed those columns in a `(for [row rows] [(nth row
lat-idx)..])`. But of course we can just update the fields we want to
select in the query.
MBQL and native queries flow through this codepath depending on the
source of the query of the card being mapped. For mbql queries, it is
straightforward, add a filter on the lat long and replace the fields
with just lat and long fields. For native, we nest the native query as a
nested query and then proceed on the resulting mbql query. The only
difference is requiring the type annotation for the field type.
mbql field: [:field 32 nil] vs [:field "latitude" {:base-type :type/Float}]
This addresses memory concerns. The query is not limited and we don't
need to select an entire row for these purposes. Dropping lots of text
fields, id fields, etc could save quite a bit of memory when resolving
the entire result set in memory.
* Add limit to number of coordinates per tile
these queries were unbounded in getting coordinates for each tile. But
in a visualization, can there really be more information provided? We
were leaving them unbounded and getting back millions of rows (reported
on #4844). So we were adding unnecessary detail at the price of OOM
errors.
Note tests have been changed to show that the limit of the original mbql
query is clobbered and the limit of the original native query is only
present in the nested query whereas the outer query selects its own
limit.
As always, should this number be exposed as a setting? Configurable in
the UI? Leaving as a hardcoded and documented number for the moment.
* docstring
* Remove col indices from api/tiles
since we just select the actual columns we want, we no longer need the
index in the results of those cols. they are always the first and second
columns
* Trim final slash on url
* Nest qbnewb under /modal to allow for other modals
move the api from api/user/:id/qbnewb -> api/user/:id/modal/qbnewb to
allow for the upcoming new dismissable modal
* Add additional datasetnewb route
staying with the same naming scheme as qbnewb. the db field is
is_datasetnewb, the route just takes datasetnewb.
I'm not wild about the default is true, set to false, and the naming
scheme. But will be easier to migrate and easier for FE to use if we
continue the defaults/logic etc.
* Remark on db change in migration file
* Slightly simplify QuestionDataSource
* Extract and cleanup HeaderBreadcrumbs
* Only keep database icon in breadcrumbs
* Fix object ID not shown in detail view
* Use `text-light` color for CollectionBadge
* Support icon name as a prop of Badge
* Add `inactiveColor` prop to Badge
* Unify head and subhead breadcrumb components
* Fix ad-hoc view items alignment
* Fix invalid CSS
* Use isStructured question method
* Fix filters display in adhoc question header
* Fix E2E selector
* Fix saved question filters alignment
* Translate secret connection-properties into existing types understood by frontend
For all :type :secret properties, before returning over the wire (i.e. from `available-drivers-info`), translate the connection-property into one or more new client side properties, using the existing types, to encapsulate the behavior of that individual secret "unit". Depending on the :secret-kind and hosting environment, this could mean there are multiple additional properties injected (one for the file upload versus local path option field, one for the value itself on upload, and one for the file path)
Adding utility function to do the reverse (i.e. translating those client side properties back to the server secret property definition)
Adding tests for both (including updating secret-test-driver to cover all of these)
Add support for the treat-before-posting connection property config
* Fix bug in `delete-orphaned-secrets!`
* Add `hasPlugin` check to moderation plugin
* Remove border from QuestionModerationSection
* Remove padding from question description
* Reorder imports and exports
* Remove not used prop
* Manage section border in the sidebar
* Add tests for question details sidebar
* Add basic dataset management section
* Add icon for dataset > question action
* Test dataset —> question change
* Add ability to turn a dataset into a question
* Use HTML buttons for dataset management
* Show question verification section
* Fix datasets e2e test helper
* Fix hover effect for dataset action buttons
* Fix question sidebar button alignment
* Use CSS to handle sidebar border
* Use SameSite=None for EMBEDDED_SESSION and DEVICE cookies
- EMBEDDED_SESSION previously did not specify SameSite attribute
and assumed that the browser would default to SameSite=None, but
recent browsers default to SameSite=Lax which does not work
with cross-domain full-app embedding.
- DEVICE was previously set to SameSite=Lax causing devices to not
be remembered during login with cross-domain full-app embedding
resulting in superfluous "We've Noticed a New Metabase Login"
emails. Setting it to SameSite=None is safe because the cookie is
not used to authenticate a user.
* Only print SameSite warning if not over https
`embedding-app-origin` may contain multiple origins but postMessage
supports only a single origin, so we need to use `*` instead.
The risk should be very low because
- the data we send is not sensitive data (frame size, current URL)
- we are already using frame ancestor policy to limit domains that
can embed Metabase
* Update new BigQuery driver to only use project-id in qualified names if it differs from credentials
Add new key to db-details for :bigquery-cloud-sdk when a database is updated, called :project-id-from-credentials, and extract the project-id from the service account creds when the database is saved
Updating logic in bigquery-cloud-sdk QP to only include the project-id when qualifying an Identifier, if the project-id (override) value is set, and its value differs from the :project-id-from-credentials value mentioned above
Adding a test to confirm that a card query continues to work after changing the associated DB's driver from :bigquery to :bigquery-cloud-sdk
Change all syncs in the test to {:scan :schema} to speed them up (deeper level not needed)
* Added documentation on how to connect to Google Analytics
* Added section explaining how to enable Google Analytics API
* Reduced png size by 35 percent using optipng
* Update docs/administration-guide/databases/google-analytics.md
Co-authored-by: Jeff Bruemmer <jeffbruemmer@gmail.com>
* Update docs/administration-guide/databases/google-analytics.md
Co-authored-by: Jeff Bruemmer <jeffbruemmer@gmail.com>
* Update docs/administration-guide/databases/google-analytics.md
Co-authored-by: Jeff Bruemmer <jeffbruemmer@gmail.com>
* Update docs/administration-guide/databases/google-analytics.md
Co-authored-by: Jeff Bruemmer <jeffbruemmer@gmail.com>
* Update docs/administration-guide/databases/google-analytics.md
Co-authored-by: Damon P. Cortesi <d.lifehacker@gmail.com>
* Update docs/administration-guide/databases/google-analytics.md
Co-authored-by: Damon P. Cortesi <d.lifehacker@gmail.com>
* Removed reference to Google Analytics versions for API scopes
* Update docs/administration-guide/databases/google-analytics.md
Co-authored-by: etoulas <etoulas@users.noreply.github.com>
* Editing the Google Analytics docs
Co-authored-by: Leandro Lamaison <leandro.lamaison@ank.app>
Co-authored-by: Leandro Lamaison <69637098+lama-ank@users.noreply.github.com>
Co-authored-by: Jeff Bruemmer <jeffbruemmer@gmail.com>
Co-authored-by: Damon P. Cortesi <d.lifehacker@gmail.com>
Co-authored-by: etoulas <etoulas@users.noreply.github.com>
* Add backend support for keystore type secrets
Adding code to handle keystore type secrets
Adding test code to create and verify a keystore through the secret DB roundtrip
Adding util fn to secret namespace for converting the value to file, and adding test for that
* Support datasets in search
* Extend SearchResult not to be a link on demand
* Use common SearchResult component for data picker
* Wrap data picker search results in `li`
* Fix E2E test selector
* Fix E2E test selector
* simplify logic for checking if a parameter's fields are populated with values
* lint fix
* always define a fields array
* make sure fields is defaulted to array in template tag sidebar
* add DimensionLabel ui component
add index.js
Update size
convert jsx to tsx
move logic from info file to unit test
go back to jsx
* replace IconWrapper with a styled(Icon)
Add a new dynamic var - *source-query* - to sqp.qp to bind the current source-query, and bind that in the same place that *table-alias* is currently bound
Update bigquery-cloud-sqp qp implementation to check the sqp.qp/*source-query*, and only qualify the source query alias if the type of source query is a table (which is apparently required; see #15074)
Add test for #18742
* Handle api/tiles requests for native queries
Ordinarily the api takes the source query and adds in a filter clause
for a region:
```clojure
{:database 19
:query {:source-table 88
:fields [[:field 562 nil]
[:field 574 nil]
[:field 576 nil]]
:limit 2000}
:type :query}
{:database 19
:query {:source-table 88
:fields [[:field 562 nil]
[:field 574 nil]
[:field 576 nil]]
:limit 2000
:filter [:inside
[:field 576 nil]
[:field 574 nil]
-40.97989944013222
-179.99999564141046
-66.51326189011354
-134.99999673105785]}
:type :query
:async? false}
```
But when native, this would break, for three different reasons:
- native queries don't necessarily (or possibly) know their field
ids. So the api request it would construct would have `undefined` in the
slot where the field-id would normally go. This would cause the route to
404 as it would fail to match the expected numeric part of the url
- just passing in the field name isn't suffiencient, because mbql cannot
natively use an mbql filter clause into a native snippet. We do this to
a limited extent with filters and substitution but this requires a
marker like `{{filter}}` placeholder for us to add the filter text.
- when using `[:field name ...]` type field references we need the
base-type of a field in order to construct a query.
So the solution:
- allow the route to take field ids or names
- if native, rewrite the query to be a nested query using the native
query as the source query. ie, "select name, lat, long from table" ->
"select source.* from (select name, lat, long from table) source" but in
mbql so we can add our fliter clause
- if a string name is passed in, annotate it with a base-type of
:type/Float since we are dealing with lats and longs.
A concern was that we need to include the source-metadata. I'm omitting
this because we are essentially "select * from nested" so we just take
what we want, and the route already expects the index of the lat and
long columns and then selects only those from the query results.
As it stands we end up with a working query but a log
```
2021-11-02 14:37:25,470 DEBUG middleware.log :: GET /api/tiles/2/1/1/latitude/longitude/1/2/ 200 47.3 ms (5 DB calls) App DB connections: 2/13 Jetty threads: 9/50 (2 idle, 0 queued) (128 total active threads) Queries in flight: 0 (0 queued); postgres DB 19 connections: 3/6 (0 threads blocked)
2021-11-02 14:37:25,474 WARN middleware.add-implicit-clauses :: Warning: cannot determine fields for an explicit `source-query` unless you also include `source-metadata`.
2021-11-02 14:37:25,490 WARN middleware.add-implicit-clauses :: Warning: cannot determine fields for an explicit `source-query` unless you also include `source-metadata`.
2021-11-02 14:37:25,501 DEBUG middleware.log :: GET /api/tiles/2/0/2/latitude/longitude/1/2/ 200 39.4 ms (5 DB calls) App DB connections: 1/13 Jetty threads: 8/50 (2 idle, 0 queued) (128 total active threads) Queries in flight: 0 (0 queued); postgres DB 19 connections: 2/6 (0 threads blocked)
```
It might be possible to suppress these logs when we are in a query
context of `:map-tiles`. Or we might be able to require the frontend to
send along the metadata to include in the query. I didn't do this yet
since it seems to work fine and it would just make the urls quite long
if there are lots of columns (and the metadata can get quite verbose).
```clojure
;; source query of the card
{:type :native
:native {:query "select name, latitude, longitude from zomato limit 2000;"
:template-tags {}}
:database 19}
;; nest it into a source query
{:database 19
:type :query
:query {:source-query {:template-tags {}
:native "select name, latitude, longitude from zomato limit 2000;"}}}
;; add the `:inside` filter for the tile
{:database 19
:type :query
:query {:source-query {:template-tags {}
:native "select name, latitude, longitude from zomato limit 2000;"}
:filter [:inside
[:field
"latitude"
{:base-type :type/Float}]
[:field
"longitude"
{:base-type :type/Float}]
0.0
89.99999782070523
-66.51326189011354
179.99999564141046]}
:async? false}
```
* Add docstring and make function private
* url encode column names to api/tiles
* Annotate collection tree with datasets and cards
In order to know which collections have items of interest, they are
annotated with two new keys: `:here` and `:below`. These keys will have
a collection (in clojure a set, in js an array) which could contain
dataset and card, depending if they have those items "here" or in
collections below this node.
We include information about which subtrees are interesting so we know
to show or highlight those. We include information about which nodes
have interesting things versus which contain nodes which have things so
we can possibly implement drilldown in an intelligent manner.
ie
```
.
├── root
```
could expand automatically to
```
.
├── root
│ ├── collection
│ │ ├── collection that actually has items
```
without requiring someone to click on the intermediate collection which
has no items itself, just a collection with items.
http://localhost:3000/api/collection/tree?tree=true
```js
{
"name": "super",
"id": 154,
"location": "/",
"below": ["dataset"] ;; indicates collections below have datasets
"children": [{
"name": "parent",
"id": 155,
"location": "/154/",
"below": ["dataset"] ;; indicates the same
"children": [{
"children": [],
"slug": "contains",
"name": "contains",
"here": ["dataset"], ;; this node has datasets
"id": 157,
"location": "/154/155/",
}],
}, {
"children": [],
"name": "sibling",
"id": 156,
"location": "/154/",
}],
}
```
* docstring
* Alignment for readability
* Change all active TEXT columns in MySQL app DB to LONGTEXT
Add new text.type Liquibase property to dynamically select the correct text type to use, per DB (which is LONGTEXT for MySQL), very similar to the existing blob.type one
Adding new migrations that update all existing app DB TEXT columns to LONGTEXT columns, only for mariadb/mysql
Update migrations linter to ensure no new text types get added, via a new predicate that searches for text types being added
Do the same logic for "blob" now (should be "${blob.type}"), update rule and test
Add test for migrations, that checks the new types, for all types in scope for this PR, to ensure that they have all been changed to their expected DB-specific type (either LONGTEXT for MySQL or TEXT/CLOB for others)
Updating a couple migrations to remove special casing that was originally done only for MySQL to simply make them universally "${text.type}", in order to unify behavior and reduce surprises going forward
Co-authored-by: Cam Saul <github@camsaul.com>
* Drop long-unused Table.entity_name column
* Add NOT NULL constraint to Card.database_id
* Test fix 🔧
* Test fix 🔧
* Fix migration
* Add note about H2 shell to deps.edn
* Add new SQL migration to attempt to set database_id when unset
* Remove data migration that is now done in Liquibase land
* Oops, '$.database', not '$.database_id'
* Parse ints as signed rather than unsigned just to be safe (they *might* be -1337 if they're really broken)
* Don't run for H2
* Update comment
* Fix migration indentation
* Clean namespace
* Use new migration number.
* Use the new migration numbers
* Adopt new migration numbering scheme
* Fix comments
* Fix MySQL + MariaDB insanity
* Fix ID range validation
* Actually 382 is the last legacy ID
* Improved validation and tests
* Adopt the new-new migration ID format.
* Test fixes 🔧
* Fix merge
* Simplify precondition
* Revert changes from Jeff's PR
* Add index to ModerationReview moderated_item_type + moderated_item_id
* Add to 0.41.2 instead
* Require explicit index name for createIndex
* Move 41.2 migrations to after the 41.0 migrations
* Adopt new migration numbering scheme
* Fix comments
* Fix MySQL + MariaDB insanity
* Fix ID range validation
* Actually 382 is the last legacy ID
* Improved validation and tests
* Adopt the new-new migration ID format.
* Test fixes 🔧
* Adopt new migration numbering scheme
* Fix comments
* Fix MySQL + MariaDB insanity
* Fix ID range validation
* Actually 382 is the last legacy ID
* Improved validation and tests
* Adopt the new-new migration ID format.
* Test fixes 🔧
* Add unit tests for schema utils
* Add data bucket step to DataSelector
* Go to next steps from the data bucket step
* Add different type of AccordionList icons layout
* Add back navigation from 'Raw Data' tab
* Fix selecting a database in 'Raw Data'
* Enable search on data bucket step
* Only render data bucket step if there are datasets
* Prevent opening the data bucket step without datasets
* Update picker search placeholder text
* Update searchable models
* Fix displaying datasets in search results
* Hide "Saved Questions" in "Raw Data" tab
* Encode data in schema str
* Add ability to request dataset virtual tables
* Support datasets in SavedQuestionPicker
* Use saved question picker for datasets too
* Add E2E tests
* Show dataset icon in search results
* Fix E2E selectors
* Fix AccordionList icon rendering
* Fix dataset can't be selected in search
* Fix data selector step hydration
* Fix JoinStep unit tests
* Fix dataset icon size in picker view
* Fix schema picker without datasets
* Fix DataSelector tests
* Fix error about null data selector step
* Move schema utils to `metabase/lib`
* Fix questions don't appear in pickers after created
* Fix JoinStep unit tests
* Test can create a question based on a dataset
* Simplify schema ID parsing
* Fix typo
* Fix search models
* Fix ellipsis character
* Fix "Raw Data" button click area
* Remove temp comment
* Fix URL in personal collection contents request
* move querystring-syncing logic to new hook
* rename <Parameters> to <SyncedParametersList>
* Remove syncQueryString prop
* remove unused 'query' prop
* remove 'isQB' prop
* remove ParametersWidget in favor of ParametersList handling empty parameters list
* Rely on value-populated parameters objects over parameterValues map
* remove unused pass-through prop 'vertical'
* make props passed from Dashboard to SyncedParametersList explicit
* Update name of component
* create a more generic hook for syncing the url query string
With the suggestions and highlighting being done through via lexical analysis
only, there is no need to run the computationally-expensive syntactical
analysis on every keystroke anymore. We only need to do that when the
user finishes the editing and wants to use the expression.
* Move the multimethods over to strings not classes
datasets are Cards so don't have a good way to intercept. And this kinda
took all of the keywords, turned them into classes, and then didn't
actually use those classes-- they were just opaque keys in the
hierarchy. No bueno.
* Include datasets in search results
* Docstring for the linter. Thank you clj-kondo
* Fix namespace decls
* Update tests
* Clean up ns and make test more resilient
these tests that test global collections can be sensitive to test order
and the CI dbs that continually exist rather than be
recreated. Sometimes "table" shows up in these available models. Kinda
don't care so just check for subset
* Rename for confusion
* Expose the e2e downloads helper function
* Apply the new e2e downloads helper to the #18382 repro
* Apply the new e2e downloads helper to the #18440 repro
* Update the helper so it applies both for saved and unsaved questions
* Add additional helper options
* Redirect to a non-existing page
* Reafactor `downloads` spec
* Extract repro for #10803
* Move UI mocks into separate file
* Mock ace and Google Analytics by default
* Add helper to run a test in EE environment
* Add test ID to CacheTTLField
* Migrate tests to new EE helper
Delete Secret instances from the app DB, when the corresponding Database instance is deleted
Updating existing test (and renamed it to `secret-db-details-integration-test`) to confirm this behavior
The possible matching/choices for function names, dimensions, segments,
and metrics are only shown when there is at least ONE matching character.
Thus, when the user hasn't inputted anything yet, there won't be any
suggestions.
* Update the keys we consider to unverify a card
Previously when almost anything changed we would unverify a card. But
this was way too aggresive. We really only want to unverify when a query
changes. Far simpler to specify the keys we do want to consider rather
than the keys to ignore.
Was prompted because when we marked a question as a dataset it would
unverify because the `:display` was changed and also `:dataset`. Neither
of these matter for verification purposes
* Update testing context string [ci skip]
* Empty commit to trigger CI
had skipped ci since it was a docstring change but that prevents merging
sad face :(
`org.apache.sshd` requires `net.i2p.crypto/eddsa` dependency
to work with ED25519 keys.
This worked in 0.40 because `eddsa` was included as a transitive
dependency of another unrelated dependency (`buddy`) but stopped
working in 0.41 because the new version of `buddy` no longer depends
on `eddsa`. Thus, we must explicitly include the dependency.
To prevent this from breaking again, switched one of the test
keys to an ED25519 key.
* Handle long running sync in connection pool hash check
Modify the logic in `db->pooled-connection-spec` to check for the latest `DatabaseInstance` from the app DB in case of a hash mismatch, before deciding that the pool needs to be invalidated (based on changing hash of db details)
This is to handle the case where a long running sync keeps passing in a stale version of the `DatabaseInstance` over a period of many minutes/hours, long after the app DB has received an updated version that was saved through the UI
In this case, the connection pool code will get the latest `DatabaseInstance` from the app DB, on a hash mismatch, then compare that against the in-memory has for the given DB ID. And if it still doesn't match, then the invalidation kicks in
Updating the test accordingly
* Add dataset icon
* Add getter and setter for Question's `dataset` flag
* Add action to turn a saved question into a dataset
* Add UI for turning a question into a dataset
* Use dataset icon for questions turned into datasets
* Show datasets on collections page
* Change colors for dataset icons in collections
* Test can turn a question into a dataset
* Fix unit test
* Fix color literal
* Hide "Turn into a dataset" button for datasets
* Rerun non table question when turned into a dataset
* This was a mistake
* Make the checkins_interval transient datasets more uniquely named
Append a timestamp suffix (initialized when the namespace was loaded) to transient dataset names
Add verbose comments explaining what we're doing
Add transient-dataset? functions for both BQ drivers to put logic in one place (which currently just see if the dataset is named `checkins_interval_*`
Ditch the "legacydriver_" prefix from the legacy BigQuery driver when creating new datasets; it's no longer needed now that transient datasets will have a much more unique suffix
Update the logic for the transient dataset suffix so that it can be more easily detected/parsed later (for cleanup)
Remove old (more than 2 hours) transient datasets when test extensions namespaces load
Add various docstrings
This reverts commit c6688f5caf.
Redshift only supports VACUUM command at a time on the whole
cluster, so running VACUUM can fail with parallel executions.
* add parameters/constants file for options-related constants
* add feature-flag utils file
* move dashboard parameter options logic to own utils file
* pull out mapping logic into own function
* move parameter type utils into own file
* move parameter operator logic into own utils file
* update reference in meta/Dashboard.js
* move parameter filter utils to own file (#18634)
Integrate secret handling with db-details
Add helper function to database model namespace to "swap out" secret values with inserted IDs
Calling this from the pre-update fn
Add pre-insert fn for database model to perform the same
Adding test to database model test ns
Add creator_id to secret model to capture the user who created this secret instance/version
Add creator-id and created-at sub properties for each secret in db details
Version 2.6.0 until 2.7.0 of org.apache.sshd has a bug that
will fail to connect to SSH servers configured to use
`-oKexAlgorithms=diffie-hellman-group-exchange-sha256`.
Reverting to the known working version. We can upgrade
to 2.7.1 once it is released.
Reference: https://issues.apache.org/jira/browse/SSHD-1163Fixes#18316
Add secret model
Add Liquibase migration for secret table
Implement bare bones model namespace for secret
Add simple test to ensure secret values are stored and retrieved under an encryption key (or not)
Add :creator_id to secret model to capture the user who created this secret instance/version
Support updating secret values as part of encryption key rotation
Add test assertions for secret values in the `rotate-encryption-key!-test`
* Fix wrong id applied when constructing a link to the question added to the dashboard
Closes#18547
* Unskip repro
* Add unit tests for question ids
* Guard against the dashboard questions with no names
Fix calculation in `ttl-hierarchy`
The `ttl-hierarchy` function falls back to `query-magic-ttl` if no stored (card, dashboard, database) cache settings are available. But that function returns its TTL value in milliseconds, while the model level, granular TTLs are stored in hours. Convert those to seconds before returning, so that `ttl-hierarchy` is always returning in seconds
Add backend test for this scenario (no stored model ttls, but an average query execution time is available, and hence the `query-magic-ttl` should be used)
Unskip Cypress repro
* Register new notebook e2e helper `visualize`
* Apply new `visualize` helper to a several different specs as a PoC
* Apply `visualize` helper to all related specs
Also, in case reduced motion is preferred, increase the spring stiffness
of the motion to finish the transition faster (timing-wide, the actual
animation won't appear due to the above snapping).
* qp: fetch_source_query: store card-id for each query
we want to be able to determine further in the pipeline
whether a query came from a card (i.e. saved question)
or not
* qp: annotate: remove join-alias if source is a card
If the source is a card, the front-end should be able to treat it
similar to a database view so we should not expose the join
aliases outside.
If the card is on the right side of the join though, the alias
should still exists and refers to the current-level join alias.
Update encryption code to be able to operate on byte arrays directly (without dealing in base64 String representation), delegating existing methods to point to these new ones
Adding test
Removing duplicate property declaration from presto-jdbc driver YAML
Add test that executes against all drivers to confirm that no duplicate names come out of connection-properties
Change the way the test runs to avoid needing to initialize test data namespace (to make googleanalytics happy)
Unskip repro Cypress test
Null query runs show up at the top previously. This is because they are a null set. This is a thing that there is an affordance to happen, because you can just create cards without running them in the notebook builder. Coalesces the null to 0 to make things sort right.
Underlying problem was that the cardinality of dates were getting limited by the default 1000 limit imposed by EE queries. Whack it for this specific instance. I think if we see it ever again we just remove the 1000 limit instead.
* add more parameters unit tests
* add a few meta/Dashboard tests
* remove some old, unnecessary comments
* fix assertions in tests
* fix getParameterTargetField tests
* delete nonsensical test
Previously sometimes error table blanks out in postgres because of limit semantics of postgres. Get it to not do that by whacking the limit and doing the display of latest error another way.
Previously mongo custexps were just columns: you couldn't have group bys with them and you couldn't have filters with them. This allows those, because those weren't really tested before. Also enabled nemanja's tests for them.
* fix virtual field access when connecting parameters to dimensions
* fix tests
* lint fix
* maybe keep things a little more backwards compatible
* fix cy test
* stop messing with field array ids
* look for virtual fields on nested questions + tests
* fix dashboard mapping test
* fix cy test
* add parameterToMBQLFilter tests
* remove direct FIELD_REF usage
* revert a few changes
* Update frontend/src/metabase-lib/lib/Dimension.js
Co-authored-by: Gustavo Saiani <gustavo@poe.ma>
Co-authored-by: Gustavo Saiani <gustavo@poe.ma>
* Ensure we are paginating resultsets
Made big tables in both pg and mysql
pg:
```sql
create table large_table
(
id serial primary key,
large_text text
);
insert into large_table (large_text)
select repeat('Z', 4000)
from generate_series(1, 500000)
```
In mysql use the repl:
```clojure
(jdbc/execute! (sql-jdbc.conn/db->pooled-connection-spec 5)
["CREATE TABLE large_table (id int NOT NULL PRIMARY KEY AUTO_INCREMENT, foo text);"])
(do
(jdbc/insert-multi! (sql-jdbc.conn/db->pooled-connection-spec 5)
:large_table
(repeat 50000 {:foo (apply str (repeat 5000 "Z"))}))
:done)
(jdbc/execute! (sql-jdbc.conn/db->pooled-connection-spec 5)
["ALTER TABLE large_table add column properties json default null"])
(jdbc/execute! (sql-jdbc.conn/db->pooled-connection-spec 5)
["update large_table set properties = '{\"data\":{\"cols\":null,\"native_form\":{\"query\":\"SELECT
`large_table`.`id` AS `id`, `large_table`.`foo` AS `foo` FROM `large_table` LIMIT
1\",\"params\":null},\"results_timezone\":\"UTC\",\"results_metadata\":{\"checksum\":\"0MnSKb8145UERWn18F5Uiw==\",\"columns\":[{\"semantic_type\":\"type/PK\",\"coercion_strategy\":null,\"name\":\"id\",\"field_ref\":[\"field\",200,null],\"effective_type\":\"type/Integer\",\"id\":200,\"display_name\":\"ID\",\"fingerprint\":null,\"base_type\":\"type/Integer\"},{\"semantic_type\":null,\"coercion_strategy\":null,\"name\":\"foo\",\"field_ref\":[\"field\",201,null],\"effective_type\":\"type/Text\",\"id\":201,\"display_name\":\"Foo\",\"fingerprint\":{\"global\":{\"distinct-count\":1,\"nil%\":0.0},\"type\":{\"type/Text\":{\"percent-json\":0.0,\"percent-url\":0.0,\"percent-email\":0.0,\"percent-state\":0.0,\"average-length\":500.0}}},\"base_type\":\"type/Text\"}]},\"insights\":null,\"count\":1}}'"])
```
and then from the terminal client repeat this until we have 800,000 rows:
```sql
insert into large_table (foo, properties) select foo, properties from large_table;
```
Then can exercise from code with the following:
```clojure
(-> (qp/process-query {:database 5 ; use appropriate db and tables here
:query {:source-table 42
;; :limit 1000000
},
:type :query}
;; don't retain any rows, purely just counting
;; so resultset is what retains too many rows
{:rff (fn [metadata]
(let [c (volatile! 0)]
(fn count-rff
([]
{:data metadata})
([result]
(assoc-in result [:data :count] @c))
([result _row]
(vswap! c inc)
result))))
})
:data :count)
```
PG was far easier to blow up. Mysql took quite a bit of data.
Then we just set a fetch size on the result set so that we (hopefully)
only have than many rows in memory in the resultset at once. The
streaming will write to the download stream as it goes.
PG has one other complication in that the fetch size can only be honored
if autoCommit is false. The reasoning seems to be that each statement is
in a transaction and commits and to commit it has to close resultsets
and therefore it has to realize the entire resultset otherwise you would
only get the initial page if any.
* Set default fetch size to 500
;; Long queries on gcloud pg
;; limit 10,000
;; fetch size | t1 | t2 | t3
;; -------------------------------
;; 100 | 6030 | 8804 | 5986
;; 500 | 1537 | 1535 | 1494
;; 1000 | 1714 | 1802 | 1611
;; 3000 | 1644 | 1595 | 2044
;; limit 30,000
;; fetch size | t1 | t2 | t3
;; -------------------------------
;; 100 | 17341 | 15991 | 16061
;; 500 | 4112 | 4182 | 4851
;; 1000 | 5075 | 4546 | 4284
;; 3000 | 5405 | 5055 | 4745
* Only set fetch size if not default (0)
Details of `:additional-options "defaultRowFetchSize=3000"` can set a
default fetch size and we can easily honor that. This allows overriding
per db without much work on our part.
* Remove redshift custom fetch size code
This removes the automatic insertion of a defaultRowFetchSize=5000 on
redshift dbs. Now we always set this to 500 in the sql-jdbc statement
and prepared statement fields. And we also allow custom ones to persist
over our default of 500.
One additional benefit of removing this is that it always included the
option even if a user added ?defaultRowFetchSize=300 themselves so this
should actually give more control to our users.
Profiling quickly on selecting 79,000 rows from redshift, there
essentially no difference between a fetch size of 500 (the default) and
5000 (the old redshift default); both were 12442 ms or so.
* unused require of settings in redshift tests
* Appease the linter
* Unnecessary redshift connection details tests
* Move revision helpers to own directory
* Add simple utility to format revision messages
* Add messages for basic dashboard cards changes
* Handle null values for dashboard card actions
* Add basic card series revision message support
* Batch multiple changes in a single revision
* Add `isValidRevision` helper
* Return title and description instead a single string
* Filter out unknown fields in revisions
* Fix viz settings revision descriptions
* Add helpers to revisions unit tests
* Use new revision messages util
* Filter out invalid or unknown revisions
* Capitalize revision descriptions
* Wrap new item name with double-quotes
* Move revisions unit tests to source code directory
* Add basic HistoryModal tests
* Add getChangedFields helper
* Revert getRevisionMessage return type back to str
* Extend isValidRevision check
* Fix getRevisionEventsForTimeline work with updated helper
* Expsoe revision utils
* Use new messages in HistoryModal
* Remove getRevisionDescription function
* Handle cases when revision's after / before state is null
* Simplify getRevisionMessage
* Use "description" instead of "message"
* Fix dataset_query revision not parsed correctly
* Filter out unknown field change types
* Support collection_id change event
* Return array of changes instead of batching in a single message
* Return JSX from getRevisionEventsForTimeline
* Fix UI
* Remove console.log
* Use "rearranged the cards" message
* Fix e2e test using old revision messages
* Prefer 'after' state to get changed fields
* Fix timeline revision event
* Fix translations
* Add `key` prop to `jt`
* Merge revision files
* Add an option not to lowercase the capitalize str
* Use updated capitalize function
* Fix test string
* Display question's "display" change messages
* [ci nocache]
* Fix tests
* [ci nocache]
Fix flaky `rotate-encryption-key!-test`
Bind a temporary site locale when running the test, and disable the settings cache
Now, a full explanation of what was making the test flaky, for posterity:
When this test runs, it first tries to create a blank app DB. This is either postgres, h2, or mysql, depending on the driver under test, which is slightly confusing, but anyway...
In order to populate all the app DB tables (since the "real" app DB has already been initialized at this point to even run the test), it first rebinds the appropriate dynamic vars for the db type, spec, connection, etc. and then loads a test fixture H2 file into the app DB. This h2 test fixture file, evidently, contains all the current app DB tables, etc.
When initializing the app DB from H2 (in `metabase.cmd.copy/copy!`), the code needs to internationalize certain loading message strings. And in order to internationalize messages, it has to look up the current site locale, which is itself an application setting, which of course comes from the setting entity table.
For the "regular" blank app DB scenario (i.e. when Metabase first starts with a blank app DB), there is some code in the `metabase.util.i18n.impl` namespace to handle this chicken and egg problem (basically, defaulting to "en" if it can't be loaded from the app DB because there is no app DB yet). But after this tricky process finishes, that temporary, hacked lookup is replaced by the real one (since now, the app DB exists and the locale can just be loaded normally).
For the purpose of our test, that is the problem. That state (which swaps out the `site-locale-from-setting` fn) has already run (remember, we have already initialized the app DB to even run this test in the first place, and now we are swapping in a temporary one). So the call to load the site locate from the settings table (which is needed to print the loading message to initialize this temp app DB) fails, and hence the test fails.
Now, the reason this test was flaky, instead of just always failing, is because of the settings cache. If the site locale had been loaded anywhere within some short time frame before this test needs to load it, to print init messages, then it succeeds! But that doesn't always end up happening, of course, since it's effectively a race condition (setting cache expiration versus test execution timing).
Currently it only displays JS errors. Now, it still displays JS errors first but if there's a query error and no JS error it'll display the query error
This is more of a quick workaround (read: hack) until we have a better
parsing infrastructure.
The idea is to have an extra argument to contains/startsWith/endsWith,
which will be "case-insensitive" if the comparison is to be carred out
by ignoring the case. If this extra argument wasn't specified, then
it'll be assumed that the case-sensitive comparison is intended.
IOW, this MBQL `["contains", X, Y`] or `["contains", X, Y,
{case-sensitive: true}]` will be equivalent to the user-visible custom
expression `CONTAINS(X)`.
Meanwhile, MBQL `["contains", X, Y, {case-sensitive: false}]`
will be mapped to the user-visible custom expresson
`CONTAINS(X, Y, "case-insensitive")`.
With this, we achieve a lossless conversion for 3 string functions
(contains, startsWith, endsWith) when involving case-sensitivity.
* Don't render bar charts are sparklines
https://github.com/metabase/metabase/issues/18352
A simple select `SELECT 'a', 1 UNION ALL SELECT 'b', 2` would trigger as
a sparkline and blow up when comparing. There are two issues:
- don't render tables as a sparkline even if it only has two columns
- don't compare things that aren't comparable.
this solves the first issue, ensuring that tables aren't matched as
sparklines
* Sparkline should only compare comparable values
Check that the columns are temporal or numeric before comparing them. It
is only to optionally reverse the order of the results which seems
questionable in itself, but we can at least check that they are
comparable before attempting to
* Ensure tests render as sparklines
default was a :table and that adds csv attachments which throw off our
tests
* unskip repro
* remove unnecessary extra line
Input: 12345.65432
Expected: 12346
Actual: 12346.
We were unconditionally adding "." (repeat decimals "0") to the base
formatting string. Obviously when the decimals are 0, we are appending
"." to the base format string
* Clean up query expressions unit tests
* Add tests for StructuredQuery's expression methods
* Append indexes to duplicate expression names
* Fix expression update test
* Test renaming an expression
* Fix handling gaps in expression names with indexes
* Handle duplicate names when updating expressions
* Add issue id to repro test
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
* Add E2E helper to select notebook steps
* Simplify Cypress test
* [ci nocache]
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
* add basic typescript configuration
* convert a tiny hook use-debounced-value to typescript
* allow any for now?
* update terser and static viz configs
* adjust config
* change extension of QuestionActivityTimeline from jsx to js to match jest config
* adjust config
* skip failing tests
* Fix missing question method in unit tests (#18314)
* review fix
* fix
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
* Enable repro
* Move saved questions virtual DB id to own file
* Add saved question helpers
* Add test-id to Metabase logo component
* Extend repro test to cover archived items handling
* Handle saved question actions in schema entity
* Handle saved question actions in table entity
* Fix moved file import
* Fix Cypress can't find logo in EE (whitelabeling)
* Fix saved questions order
* Fix saved questions included in database tables
Change implementation of docsUrl to handle RC suffixes for EE versions
Change the regex used to test the tag to detect any OSS or EE version, including (possibly) an arbitrary suffix after the major and minor parts, and normalize to the major OSS equivalent version instead. If the suffix, OTOH, is `-SNAPSHOT` (ignoring case), then the doc tag will be `latest`, on the theory that this is as local development build.
v0.41.0 => v0.41
v1.41.1 => v0.41
v1.41.2-RC1 => v0.41
v1.42.0-SNAPSHOT => latest
Adding unit tests for all cases
Update behavior so that -SNAPSHOT suffix points to "latest"
Previous PRs for error cards page were laser focused on getting everything to work, as opposed to actually conforming to spec. This one gets it closer to spec, although product wants reloading animations which are not there yet. To wit,
- Indicates to end user what the current running state is. Given the many orders of magnitude in DB query run times I think something robust and non-ephemeral is necessary here (looking at some sort of expanded QE model) but Maz suggested expedient method and I took it
- No more left nav on the table page
- Page heading for table page
- Independent scroll-x behavior on the table
- Styling for the error section on the table page
- Select-all component. Note that the interaction behavior with pagination is fiddly. (but less fiddly now that it's all based on card id's instead of row numbers)
- Coalescing default behavior for Our Analytics collection (default collection) and null number of dashboards
Rename error name to error content search
Remove the `driver/format-custom-field-name` implementation for :snowflake, which was causing the problem and appears to be no longer needed anyway
Add test for this scenario to `metabase.query-processor-test.expression-aggregations-test`
* Update Link component's code style
* Add disabled style to Link
* Disable "Save" button when can't run query
* Animate Link color and opacity changes
Cache TTL is defined in the nice new column things as per hour. The actual cache wanted it in seconds. There wasn't a conversion hours->secs before, which was the underlying illusion that gave rise to #18160 - with a properly set cache ttl the cache works fine. But the cache ttl's weren't getting set to proper values because of that lack of unit conversion. So here's the conversion, hours->secs.
* Make webpack output closer to webpack 4 and fix whitelabelling
* Fix recently-viewed.cy.spec.js
Not sure why it's started failing with this PR, but adding wait for text.
When all the necessary data hasn't been fetched you can find yourself in
stuck in a loop between the invalid state checks in DataSelector's
componentDidUpdate method and the skipSteps method.
1. there seems to always be a single schema (the "Everything Else"
schema) available on instantiation
2. the skipSteps method auto-selects this schema when no other schemas
exist
3. there's some pre-existing logic to avoid getting in this loop by
checking the is_saved_questions property of the database in the
selectedSchema. However, for whatever reason this property doesn't
exist until AFTER we've fetched some things.
The fix is checking a property that is guaranteed to exist, the ID for
the "Everything Else" database
NODE_ENV interferes with other tools (e.g. `yarn` won't pull
devDependencies). Since what we need is only a flag indicating Webpack
to bundle for production vs developement, invent our own env.
This also solves the mystery of unable to call `yarn build` in the build
script `bin/build-mb/src/build.clj`.
* move logic in dashboard selectors to meta/Dashboard
* rmv usage of syncQueryParamsWithURL
* add syncQueryParamsWithURL logic to meta/Parameters.js
* Fix public questions
* fix dashboards
* fix query builder
* don't normalize when value is empty string
* rmv unused setMultipleParameterValues fn
* replace selector logic with meta/Dashboard fn
* filter out nil fields
we need to do more work before we can properly use
fields from nested questions when parsing parameter values.
* add field to dash parameter so we can correctly type nested native query fields
* fix tests
* code style tweaks per review
- a56b640 broke Percy workflow on master and introduced a major inconvenience for developers who wish to run Cypress locally. In both cases, it's required to have all supported QA DB docker images running at all times. Otherwise, the Cypress snapshot creation phase breaks.
This PR:
- uses the existing ENV var to conditionally skip qa db snapshots
- this doesn't affect CircleCI workflow - it merely makes life easier for developers running Cypress locally and for a Percy workflow that runs on GitHub Actions (where we still don't have QA docker images available)
Add `escape-alias` multimethod to `sql.qp` to handle translating an alias (whether for a field or expression) into something supported by the driver in quesion
Add default impl that is essentially identity
Marking `sql.qp/field->alias` as deprecated, and changing its default implementation to invoke the new `sql.qp/escape-alias` on the field's `:name` (with the intention of removing it in a future release, since individual driver won't need to override this, so much as they'll need to override `escape-alias` itself).
Override `sql.qp/escape-alias` method for both BigQuery drivers to run through the same `->valid-field-identifier` function already defined for this purpose
Add test for a custom expression with weird characters
* Create `postgres-12` snapshot
* Use `postgres-12` snapshot in tests
* Relocate postgres `add` spec
* Relocate postgres `sandboxes` spec
* Relocate postgres `native` spec
This actually didn't have anything to do with the native questions.
We simply used native editor because it was easier to reproduce the issue.
* Relocate postgres `permissions` spec
* Relocate postgres `query` spec
* Relocate postgres `custom-column` spec
* Remove postgres from `metabase-db`
There is a problem when there are no card ids for a userland query (so not all userland queries are non-adhoc queries: they can be adhoc), where viewlog needs a card id so it chokes. Just adds a check so it doesn't even try to emit a viewlog, since the auditing is for non-adhoc queries only.
Check feature flag on backend /graph PUT API for block permission
Check for the `:advanced-permissions` flag in `update-db-permissions!`, when a `:schemas` `:block` entry is present in the graph
Set feature flag in tests when using API
Add new test to ensure failure in the case that feature flag is missing
Overall purpose of PR is to add row drill-ins to the tools to fix broken questions, so you can take a look-see into what the broken questions are.
A tiny nit thing wasn't caught by CI and broke master. This was because there's a CI cache. I can confirm that this didn't hit the CI cache and the other thing that broke isn't breaking anymore
- `TEXT` in Redshift is equivalent to `VARCHAR(256)`, use
`VARCHAR(1024)` instead
- Unskip all tests skipped on Redshift due to sample dataset issue
- Add explicit `:order-by` to `join-source-queries-with-joins-test`
to make it pass in Redshift as Redshift does not preserve order
after join (like a lot of other distributed query engines)
- Add `VACUUM` and `ANALYZE` call after INSERT to improve performance
from maz in slack:
Should not remove verification:
- Moving to another collection
- Pinning/unpinning
- Adding/removing an alert on the question
- Turning sharing on the question on/off
Should remove verification:
- Editing the query
- Editing the visualization or viz settings
- Editing the title or description
This really just identifies when not to remove verification, otherwise
does remove verification. Alerts are not related to questions through
the api so no worries there. Everything else is just excluding
attributes from the diff calculation.
In tests, went perhaps a bit haywire with the macrolet, allowing to
define macros inline rather than farther away from the call site. Used
twice:
- once to optionally add in moderation review into the with-temp. This
macro is handy to ensure we always have a fresh state of card and don't
have leftover changes or verifications. Has to be a macro since i want
to dynamically change whether there are verifications in the with-temp*
vector binding
- a way to more easily reduce repetition when asserting something is
verified, do an action, and assert it is still verified
Cache controls all landed but is lacking in the audit capacity wanted for in the notion doc. This PR adds that audit capability and by the by changes the ViewLog model in order to be able to deliver on the question of whether ViewLogs were cache hits or not.
Details page for tools to fix broken questions. Also the PR ran long enough that nemanja found some bugs in the details page. Does not include any loading overlay or anything to indicate that reloads are happening, which is for 41.1 I guess
Fix BigQuery Percentile and Median expressions not using correct backtick quoting
Ensure the identifier components are quoted by reifying `ToSql` for the `APPROX_QUANTILES` expression
Add test to ensure query is created properly
Mark `:percentile-aggregations` as being supported in the new driver
This is a combination of 2 issues:
1) pt got renamed to pt_BR in x.39.x but the old
"pt" value may still be stored in the database
2) when making a release, an unclean build
directory may be used containing old "pt"
resource which gets leaked into the release build
1) is fixed by adding a function to treat "pt" as "pt_BR"
to support users who were on "pt" since pre-x.39.
(This is done by finding the closest fallback locale)
2) is fixed by emptying the folder before generating locales
so any old locales are deleted.
Fixes#16690
`query->native` works by having `nativef` throws an exception
which causes `mbql->native` to set native query to `nil` which
then cause the rest of the execution to be skipped.
However, because the existing code skips overriding the native
query for queries already native, the execution is not skipped.
This causes bad performance for nested native question because
`query->native` is called for the sub-query which causes the
sub-query to be executed on its own, potentially without any
limit or filtering.
Because we can't fully implement all the driver deprecation UX in time for release 41, we will adjust the language to allow flexibility for when to actually start the deprecation clock
Add test for ordering when nils are present
Adding test to ensure that we can handle a column containing nils in an ORDER BY clause
Adding `format-name` override for `bigquery-cloud-sdk` to convert dash to underscore in test table names
Adding comment showing IDs of each row in the daily-bird-counts.edn file
Format last column as int to make Oracle happy
Add impls for `presto` and `presto-jdbc` `sorts-nil-first?` as false
Make the `sorts-nil-first?` multimethod accept a base type in order to return an answer
Vertica, at least, has a different answer for different types; see:
https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/AnalyzingData/Optimizations/NULLPlacementByAnalyticFunctions.htm#2
Updating Vertica implementation to consider the `base-type` as per its documentation
Updating all invocations of `sorts-nil-first?` in tests to pass the relevant base type
Support query cancellation in new BigQuery driver
Remove now unused 60 second timeout
Remove a bunch of unneeded code (`with-finished-response`) from the legacy driver
Add support for canceling query, which stops pagination of results
Add test to confirm pagination stops upon cancellation
Add test to confirm that max rows enforcement via middleware prevents unnecessary pages from being fetched
* Add initial e2e test for erroring questions
* Add repro for #18050
* Add repro for #18048
* Add test section with that contains broken questions
* Improve test and add another scenario
* added check for borderless buttons when setting compact padding
* fix my own silly regression
* added visual tests for run button regressions
* added minWidth to percy snapshot
* shoudl have read the docs
* Sync VIEWPORT_WIDTH and percy snapshot width
* add template tag fields as dependent metadata for native queries
* check for a field_id property added by getValueAndFieldIdPopulatedParametersFromCard
* Add NativeQuery dependentMetadata unit test
* add test for loadMetadataForQuery
* Formatting
return a format string so it can be used over multiple rows. The rules
for formatting are quite annoying because there are defaults that are
assumed rather than present. IE, if you set a column as currency it
doesn't set the number style sometimes and defaults to USD without that
being in the settings. In the future would be far preferable to always
create a fully fleshed map of defaults rather than inferring them in
random places.
* appease our doc string supervisor
* Include column information for formatting numbers
really important for requiring two decimal places by default on doubles
and none on integers. Its close to correct ignoring it and using
"#,###.##" and this leaves non decimal places on integers and allows
only up to two on doubles, but things like 31.10 will format as 31.1 and
that is improper
* Remove unneeded metabase.test
* Update tests with nil column
* Get global viz settings in middleware, pass entire to formatting
Passing the entire viz settings to the formatting stuff makes the
calling code much simpler and the places where we merge global settings,
possibly get currency settings, etc, just in one place. We already had
the col information there so just use it.
* Remove public settings now that we get it from the metadata
* Check for existing result set when displaying
"Showing [X] rows" instead of the max limit
* added cypress test
* broke up nested ternary operators in QuestionRowCount
We added cache control fields for dashboard and database and actually use it for cards: presumably EE folks want to audit that stuff. Here is the addition of those fields to the table views.
In `getTableClickedObjectRowData`, we try to get the
original row of the data from the rowIndex but because
the rowIndex of the pivoted table and the original table
does not necessarily match, we get the wrong data.
This commit fixes it by storing a mapping between
pivoted (row, col) and the original row number and using
that to determine which row to use for the data.
Fixes#17920
When loading dashboard filters, the default value is in `:default`
not in `:value`. In the field filter code path, the default is
copied to `:value`. Fix the issue by calling the same code for
non-field filters too.
Fixes#15705
* Move email->domain and email-in-domain? util functions to metabase.util
* Improve the error handling for with-temp a little more.
* Extra error context for with-temporary-setting-values
* Improve Setting docstring forms validation: handle aliased forms like `i18n/deferred-tru`
* Fix missing :require
* For sparklines, check that viz type is line
this actually goes to fix waterfalls. They would match these
requirements but the viz type of waterfall would give results that would
dip out of the viewport. Fallback to table rendering
* Prevent waterfall rendering as sparkline
* please run that cancelled test again, GitHub®
Add separate catch clause for BigQueryException, to `execute-bigquery`, which conveniently includes a property to indicate whether it's retryable, and add that to the ex-data map
Add test that confirms a retryable exception is, in fact, retried
* Test join's setDimension methods can remove dimensions
* Specify temporal-unit for Join unit tests data
* Ensure join dimensions' temporal units are the same
* Display dimension's temporal unit on Join UI
* Add E2E test for join fields temporal units
* Select temporal units for join dimensions
* Sync parent and join dimensions temporal units
* Refactor Join dimension setters
* Update subdimension check
* Update overwrite check
* fix reverting dashboard with card deletion
`delete-dashboard-card!` expects the :dashboard_id
parameter to be defined.
* fix frontend issues related to dashboard reverts
1) fix the dashboard not being reloaded to the
reverted version after revert
2) fix the history list being reloaded to include
the latest change
* Rename dashboard details edit action
* Use render props at dashboard edit form
* Add enterprise caching plugin
* Move dashboard entity forms to own file
* Make FormField's margin configurable
* Pass custom style to FormField
* Use null as a default cacheTTLFormField value
* Add tests for DashboardDetailsModal
* Add cache_ttl field to DashboardDetailsModal
* Add tests for CreateDashboardModal
* Test cache_ttl is invisible when creating a dashboard
* Remove not used mocks
* Add tests for EditQuestionInfoModal
* Move question entity forms to own file
* Add cache_ttl field to EditQuestionInfoModal
* Test cache_ttl is invisible when creating a question
* Extract CacheTTLField component
* Don't show cache field if caching is disabled
* Fix typo
* Update cache_ttl field look
* Revert "Pass custom style to FormField"
This reverts commit 3944ca8ff1dcad478f0f1c3a1973f454447c88c1.
* Revert "Make FormField's margin configurable"
This reverts commit ca7dc434532038da736012147f6889e6cc01ec11.
* Fix CacheTTLField width
* Add descriptionPosition prop to FormField
* Add DatabaseCacheTTLField
* Show cache_ttl field on database form
* Use 24h as a default database cache TTL field value
* Fallback to database cache_ttl in question form
* Remove console.log
* Fix request body assertion for CreateDashboardModal
* Fix request body assertions for EditQuestionModal
* Fix request body assertions for DashboardDetailsModal
* Fix cache_ttl not included in `PUT /api/card/:id`
* Use simple for for EditQuestionInfoModal
* Refactor how unit tests mock caching settings
* Add dashboard caching EE test
* Add question caching EE test
* Add caching e2e test for database add/edit
* Remove debug stuff
* Add error state to CacheTTLField
* Fix DB cache field reverts to default value on 0
* Validate cache_ttl field
* Add getQuestionsImplicitCacheTTL helper
* Remove redundant tests
* Remove not used import
* Nomalize `cache_ttl: 0` to null
* Fix duplicated card update request
* Add QuestionCacheTTLField component
* Add test-id to Radio's option names
* Use a new cache TTL field for questions
* Check setting cache_ttl to null in E2E tests
* Style question cache TTL field
* Add dashboard ID to query request body
* Extend `duration` formatting helper to support hours
* Fix default caching duration display
* Fix tests
* Fix old dashboard edit action title
* Fix missing translations
* Add prop types to EditQuestionInfoModal
* Remove `test.todo`
* Simplify `getQuestionsImplicitCacheTTL`
* Use `space` style helper
* Add `key` prop to `jt`
* Move time coverting utils to lib/time
* Hide caching field label if just showing the input
* Fix unit tests
* Fix flacky E2E test
* Add hover style for "More options" control
* Fix missing translation
* Use placeholder for cache inputs
* Fix E2E tests to use cache field's placeholder
* Feature flag for caching UI
* invalidate revision entities when questions/dashboards updated/deleted
* you can't update a revision
* add a cy test to cover revision cache clearing
* let's be a little looser with revision text
* Add subscription-allowed-domains allow list Setting
* Move email->domain and email-in-domain? util functions to metabase.util
* Fix missing require
* Test fix 🔧
* Remove useless namespace
* Make subscription-allowed-domains a :string Setting rather than :csv
The locale list sent to the UI is formatted "pt-BR" but
when it is saved to the database, we re-format the locale to
"pt_BR", thus on next load, the UI cannot match the saved locale
with the locale list.
Fix the issue by using normalized value (e.g. "pt_BR") in the
available locale list.
Fixes#15657, fixes#16654
The ttag library expect the plural array to be exactly the size as
the number of plural forms defined in the header.
If we remove empty plural, the array will have the wrong size and
the library will crash when the trying to use the non-existent
plural.
By leaving the empty string there, the ttag library will correctly
detect it and use the English version for the missing variant.
This does not affect the backend because the backend does not
support plurals.
Fixes#16323
* Add new Settings (placeholders) for the new 0.41.0+ premium features
* Code cleanup
* Add new features to the `:premium-features` Setting (for FE consumption)
* Fix busted ns declarations
* combine sidebar state for various dashboard sidebars
* replace hideClickBehaviorSidebar action with closeSidebar
* move sidebar name strings to a constants file
* add some tests
added the viz-settings parsing middleware to static viz. Column settings are
spec'd for regular columns and by name, but we forgot expressions apparently.
* Add test ID for notebook's action buttons
* Add repro for #17712
* Use `Join.clean` in join step's clean callback
* Always show a remove icon
* Fix referencing an empty dimension
* Improve Join's validity check
* Don't cleanup incomplete join sections
* Fix structured query tests
* Don't use MBQL field literals in notebook tests
https://github.com/metabase/metabase/issues/14824
* Delete user pulse/alert/dashboard subscription subscriptions when they get archived
* Automatically archive Pulses when last subscription is deleted (#17812)
* Automatically archive Pulse when its last subscription is deleted
* Test fix 🔧
* Fix occasionally-failing rotate-encryption-key!-test
* Update pulse_test.clj
Add parameter handling for BigDecimal that checks whether the value is in range for BIGNUMERIC or BIGDECIMAL and sets accordingly
Updating parameter test to hit both these cases
Modifying test code/macros to support a with-table semantic for testing the decimal types via a temp table
Writing new test to confirm that all numeric types are synced correctly and can be queried for
* Basic skeleton of front-end dependencies analyzer
* Parse enterprise/frontend
* Handle good ol' require
* For sanity, bail out immediately when encountering a parse errr
* Split into commands (currently returning JSON)
* Filter dependents from stdin
* Track down all dependents, including the indirect/transitive ones
* Fix filtering not including the enterprise
* Map a directory to its index.js, if it exists
* Refactor common functions
* Matrix and count dependents
This was a miserable grind in the old BQ syntax but a happy fun time in the new BQ syntax, basically just turning it on and squashing a bunch of test bugs.
One test bug to note especially is that the underscore-transformed set of bird scarcity data for the bird scarcity tests is actually necessary, because BQ will hit you with a complaint about underscores even if you successfully and correctly quote the table names.
* Pass along date formatting
* extract and rename some things
* Number formatting
* Update tests
* Improve test validation of svg labels
* Clean up tests
* Move static-viz interface file to custom file
* let users disable attachments with questions checked
* use more declarative assertions in unit tests
* use aria-label over testid
* translate aria label
OpenAM SAMLResponse contains "\r\n" every 76 characters
leading our SAML code to think it's not base64-encoded.
Fix by relaxing our valid base64 check to ignore all spaces.
Fixes#15567
* expression editor: allow clearing help text with Escape key
Move the code to clear help text to "clearSuggestions" method
allowing the Escape key to also be used to close the help popup.
* expression editor: don't show error on window unfocus
* expression editor: allow clicking on "Learn more"
Clicking on the "Lean more" link removes the focus
from the input box, causing the help dialog to get
hidden and become unclickable.
Fix by cancelling the mouse down event causing the
input box to maintain the focus.
Fixes#17548
* add cypress test
* Run percy (sanity check) visual tests in CircleCI
* Update path for visual tests
* Make `--folder` flag optional
* visual tests docs
Co-authored-by: Alexander Lesnenko <alxnddr@users.noreply.github.com>
* Fix the chain filter functions with nil FieldValues (fix#17659)
* Reorganize test a bit.
* Simplify test
* More simplification
* Test fix 🔧
* Test fix part 2
* [ci nocache]
* No Clojure core classes please
* Don't try to make paths absolute twice. Build scripts should handle keyword args like ':driver'
* parse-as-keyword should return keywords as is
* Test fix 🔧
* Test fix 🔧
* Test fix part 2
* Collapse small donut slices into a single "Other" category
* Locale aware decimal formatting
A followup PR will add all of the formatting options for scalars and
smart scalars so this function is largely temporary and will be
subsumed into a namespace that knows about all formatting options.
* empty with no cache [ci nocache]
Since the Webpack and Babel upgrades, this shouldn't be needed any more. In the past, we had to explicitly resolve to the specific version of Cypress. Now it resolves to this version by default.
The node_modules folder was being persisted in the first step (checkout) as it's doing the building of the static viz package. This had an impact in the fe-deps step, so wiping the folder and doing a clean install should make the trick
* update sqlite-jdbc to 3.36.0.1
The new version includes arm64 binaries and allows Cypress,
which uses SQLite, to run on Apple Silicon.
* Improve date/time handling in SQLite
* Fix setting Block permissions in the permissions/graph API endpoints
* Revert changes to .dir-locals
* Perms graph: don't validate DB :schemas + :native unless both are present.
* PR feedback
* Ensure that data_readers functions are included in tests
`clj -X:dev:test :only metabase.pulse.render.js-svg-test` uses the
`#t` data reader, which passes the value to
`metabase.util.date-2/parse` but this function hasn't been evaluated
yet and it errors
```shell
% clj -X:dev:test :only metabase.pulse.render.js-svg-test
Warning: environ value /Users/dan/.sdkman/candidates/java/current for key :java-home has been overwritten with /Users/dan/.sdkman/candidates/java/11.0.11.hs-adpt
Running tests in metabase.pulse.render.js-svg-test
Syntax error reading source at (metabase/pulse/render/js_svg_test.clj:71:27).
Attempting to call unbound fn: #'metabase.util.date-2/parse
```
* Throw an error if resource is not found when loading sources
```shell
ERROR in metabase.pulse.render.js-svg-test/categorical-donut-test (js_engine.clj:37)
Uncaught exception, not in assertion.
clojure.lang.ExceptionInfo: Javascript resource not found: frontend_client/app/dist/lib-static-viz.bundle.js
source: "frontend_client/app/dist/lib-static-viz.bundle.js"
metabase.pulse.render.js-engine/load-resource js_engine.clj: 37
metabase.pulse.render.js-svg/load-viz-bundle js_svg.clj: 94
metabase.pulse.render.js-svg/static-viz-context js_svg.clj: 100
metabase.pulse.render.js-svg/fn js_svg.clj: 106
...
clojure.core/deref core.clj: 2324
metabase.pulse.render.js-svg/timelineseries-line js_svg.clj: 167
metabase.pulse.render.js-svg-test/fn/fn js_svg_test.clj: 75
metabase.pulse.render.js-svg-test/fn js_svg_test.clj: 74
clojure.test/test-var/fn test.clj: 717
clojure.test/test-var test.clj: 717
metabase.test-runner/run-test test_runner.clj: 107
...
```
```shell
ERROR in metabase.pulse.render.js-svg-test/categorical-donut-test (Objects.java:221)
Uncaught exception, not in assertion.
java.lang.NullPointerException:
java.util.Objects.requireNonNull Objects.java: 221
org.graalvm.polyglot.Source$Builder.<init> Source.java: 725
org.graalvm.polyglot.Source.newBuilder Source.java: 546
metabase.pulse.render.js-engine/load-resource js_engine.clj: 34
metabase.pulse.render.js-svg/load-viz-bundle js_svg.clj: 94
metabase.pulse.render.js-svg/static-viz-context js_svg.clj: 100
metabase.pulse.render.js-svg/fn js_svg.clj: 106
```
* Sort effects namespace
* Turn dataset_query into structured data in search results
* sort scoring test ns
* Fixup tests
we are no longer returning string results. The cards in the test have
empty queries, which the model turns into {} but normalization removes
empty clauses and returns nil. The tests would be too annoying to hook
this up to work properly and this functionality is tested in other
places.
* Unskip repro for #15959
* Update repro test
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
* Invalidate the pulse if it contains no recipients
Switch statement was returning `true`, resulting in falsely "valid" pulse
in the case there were no recipients but the schedule was set to month.
This commit fixes that.
* Move all conditions before the switch statement
* Enable repro for #17657
* Guard against the error that causes the white screen
* Add another repro for #17657
* Fix typo in the test name
* Fix typo in the unrelated comment
With the code being instrumented for code coverage, and another
additional time to analyze + send the code coverage to codecov.io,
the typical execution runs awfully close to the 10-min limit.
Hence, add some extra room.
* Randomize BigQuery test view names
Use tu/random-name instead of a name based on the output of gensym
* Add extra prefix for legacy BigQuery driver created databases
* Update legacy BigQuery driver to have unique name for transient datasets
For those datasets created and destroyed by the legacy BigQuery driver QP tests, add an additional prefix so that it doesn't conflict with the new BigQuery driver tests
* Rename file to make it more specific
* Extract dashboard sql filters into separate folder and CI group
* Add short descriptions to dashboard filters repros
* Extract dashboard sql filters helpers (data objects)
* slightly split existing search components
* add recent views list
* fix recent items moving when reopened
* extract TablePath component
* Add e2e specs for recents
* add specs
* add in table view logs so we get recent table looks
* review fixes
* use display_name for tables in recents list
* display name
* add loader threshold, return only 5 recents
Co-authored-by: Howon Lee <hlee.howon@gmail.com>
* Add test for required and default values set on dashboard SQL field filters
* Add test for required and default values set on dashboard SQL simple filter
* Outline for login troubleshooting guide
* Update docs/troubleshooting-guide/cant-log-in.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Update docs/troubleshooting-guide/cant-log-in.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Importing material
* Prosifying
* Updates based on feedback
* Another update to login troubleshooting
* Fixing typo
* One more update
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Improve driver deprecation UX
Reworking the DriverWarning component to be able to change the engine itself by firing the event (so that the "go to older/newer version" links actually work)
Some styling fixes to match Figma mockup
Remove all grid-styled usage
Remove all styling (except class names) from DriverWarning jsx file, move to the styled-component
Implement db-ids-with-deprecated-drivers API endpoint for fetching DB IDs using deprecated drivers
Add some plugin test init code to properly handle the test legacy/new drivers
Adding test for new API
Adding new entry in `MetabaseApi` frontend component for new API method
* Determine MB project root directory relative to source directory of build scripts
* Support calling build-driver! as a -X fn, and support dirs outside of modules/drivers
* Support loading 3rd-party driver manifest files at (dev) launch
* Fix log4j not working on latest version of CIDER (unrelated)
* Fix dissoc
* project-root-directory is supposed to be a string
* ⛔ B L O C K ⚠ permissions ✅
* Fix test failure in OSS since EE namespace doesn't exist.
* Remove unused var
* Oops, uncomment a commented-out test
* Clean up the new perms documentation a bit
* Include the results base type as the cols effective type
previously was merged the effective type from the col which caused
issues with dates when selecting a particular temporal unit, ie month.
Queries like this return months as integers so the base and effective
types are :type/Integer, and something somewhere else is responsible
for the unit conversion of `1` to January (or feb, months get weird i
think)
```clojure
;; after
{:description "The date and time an order was submitted.",
:unit :month-of-year,
:name "CREATED_AT",
:field_ref [:field 4 {:temporal-unit :month-of-year}],
:effective_type :type/Integer, ;; we have the proper effective type
:display_name "Created At",
:base_type :type/Integer}
;; before:
{:description "The date and time an order was submitted.",
:unit :month-of-year,
:name "CREATED_AT",
:field_ref [:field 4 {:temporal-unit :month-of-year}],
:effective_type :type/DateTime, ;; included effective type from db
:display_name "Created At",
:base_type :type/Integer}
```
* tests
* Driver tests
* Ignore effective/base types in breakout tests
sparksql is weird and its totally unrelated to the file under test
* Use correct options for datetimes that are projected to month, etc
When aggregating by a datetime column, you can choose a unit: month,
hour-of-day, week, etc. You often (always?) end up with an
integer. Previously the base_type was (inexplicably) :type/Integer and
the effective_type was :type/DateTime which really just didn't make
sense. The ideal solution is :type/DateTime as the base_type and
:type/Integer as the effective_type. But this breaks the frontend as
it expects to treat these columns in a special way. But it expected
them to report as date columns.
I've changed it only here. I thought about making isDate understand
the `column.unit` attribute and recognize that these are in fact
dates, but that seems wrong. These are integers that are special
cases. It seems that the contexts that need to talk about dates should
understand integers in a special way than all date code needs to be
aware that integers might flow through.
This might mean extra work but ultimately feels better as the correct
solution.
* unit is not default
* Move JoinStep's NotebookCell down to JoinClause
* Break JoinClause into two NotebookCells
* Update labels
* Use `space` helper function
* Simplify conditional statement
* Fix not translated string
* Add Kerberos support to Presto JDBC Driver
Adding Kerberos properties as DB details properties
Refactoring utility methods for dealing with additional options
Adding test for Kerb properties -> connection string
Adding GitHub action to run integration test
Start to move logic for capturing necessary files to new script, and calling that from mba run
Adjust settings in `metabase.test.data.presto-jdbc` to capture test parameters
Fix counting on grouped aggregated field
Adding QP test
Reworking the BigQuery version of the :breakout to more closely match the structure of the standard sql.qp version
* add PermissionSelect component
* add PermissionSidebar component
* add PermissionTable component
* add PermissionEditor component
* add useLeaveConfirmation hook
* add selectors for the collections permissions page
* add selectors for the data permissions pages
* add confirmation creators to use in selectors
* sandboxing permissions modal: redirect to parent route when closed
* Add PermissionsPageLayout component
* Adjust CollectionPermissionsModal to use new permissions UI
* Add CollectionPermissionsPage
* Add data permissions pages
* Adjust collection permissions modal route
* Adjust permissions routes according new pages structure
* update permissions actions, reducers
* adjust sandboxing
* remove old permissions code
* support admin colorScheme in Radio
* support admin colorScheme in TextInput
* support admin colorScheme in Tree
* update permissions specs
* fix data permissions url typo
* fix sandboxing
* add search debounce to permission pages
* exclude schemas from search, make it flat
* fix permissions for databases without schemas
* fix copy for native queries permissions, show table display_name instead of raw name
* fix before leave warning shown when change collections
* permissions sidebar: make database name clickable
* fix string groupIds lead to bugs in permissions editor
* fix disabling table permissions shows all tables permission disabled
* fix permissions copy
* fix permissions disabled tooltips copy
* fix, add specs
* fix permissions edit bar button colors, fix breadcrumbs on schemaless dbs
* fix official collection color, make sidebar tree scrollable
* fix scroll in permissions modal
* fix breadcrumbs
* change table icon
* create helpers for entity ids and permissions url creation
* remove hardcoded colors
* make confirmations an array
* store confirmAction in state
* add ModalRoute specs
* fix propTypes, typo
* build permission pages urls with helpers
* revert switching from ref to state for confirm action function due to react invokes it on a render
* move mapStateToProps to the top of the files
* add visual specs
* add more specs
* fix edit sandbox access action
* add cypress spec for sandboxing
Mongo custom expressions now are turned on for mongo 5.0 and after only. One idiosyncrasy is that we don't really support the BSON ID format at all in our typesystem despite having it in there, but both of my attempts to get them to work with the native mongo pipeline commands bounced because they really aren't strings, they're BSON ID's. We do rely on the pipeline commands heavily which is the reason for the version requirement.
This requires unfortunately an additional layer of indirection, however
I believe it's super cheap and it shouldn't impact execution speed.
In the mean time, many other predicates (isFoo etc) which rely on isa()
will be correctly type-inferred, which gives a huge win for editing and
debugging purposes.
I'm not sure we can fix this. We need the graal compiler which doesn't
appear to be jdk 8 compatible but I can't actually find javadocs for
it. Its not clear from
https://github.com/oracle/graaljs/blob/master/docs/user/RunOnJDK.md
how you can actually run it.
I tried adding the graal compiler but that seems to assume you are in
a graal vm and fails not finding a class.
To get a list of all the options available for a context's engine,
```clojure
(doto 'metabase.pulse.render.js-engine require in-ns)
(map (comp (juxt :name :help) bean)
(.getOptions (.getEngine (context))))
```
I've done some profiling
```clojure
(dotimes [_ 5]
(time
(do (let [rows [["apples" 2]
["bananas" 3]]
colors {"apples" "red" "bananas" "yellow"}]
(js-svg/categorical-donut rows colors))
nil)))
"Elapsed time: 210.32659 msecs"
"Elapsed time: 193.211736 msecs"
"Elapsed time: 190.97775 msecs"
"Elapsed time: 195.843254 msecs"
"Elapsed time: 188.077405 msecs"
nil
```
For interpreted and in another language, this doesn't seem the end of
the world.
And adding more quantities
```clojure
(letfn [(rand-string [] (str/join "-" (repeatedly 4 #(rand-nth ["banana" "horse" "battery" "cloak"]))))]
(dotimes [_ 5]
(let [rows (repeatedly 20 (fn [] [(rand-string) (rand-int 100)]))
colors (zipmap (map first rows) (cycle ["red" "green" "blue" "yellow"]))]
(time
(js-svg/categorical-donut rows colors)))))
"Elapsed time: 288.659887 msecs"
"Elapsed time: 334.733071 msecs"
"Elapsed time: 317.369294 msecs"
"Elapsed time: 272.084918 msecs"
"Elapsed time: 272.7203 msecs"
nil
```
For a timeline line chart, rendering one datapoint per day for a whole
year:
```clojure
(dotimes [_ 5]
(let [dates (take 365 (iterate #(.plusDays % 1) #t "2020"))
rows (map (fn [d] [d (rand-int 200)]) dates)
labels {:left "count" :bottom "year"}]
(time
(js-svg/timelineseries-line rows labels))))
"Elapsed time: 569.691124 msecs"
"Elapsed time: 544.415676 msecs"
"Elapsed time: 539.131092 msecs"
"Elapsed time: 492.60486 msecs"
"Elapsed time: 523.765691 msecs"
nil
```
* Troubleshooting guide for filters.
After conversation with @flamber, split this into one guide for simple filters and one for linked filters. The explanation at the top of the second page may still need some work...
* Updating filter troubleshooting
* Troubleshooting when you can't see the tables you should.
* Why is my dashboard slow?
* Troubleshooting: can't send email
* Adding @flamber's batch script-style guide
* Update docs/troubleshooting-guide/cant-send-email.md
Co-authored-by: flamber <1447303+flamber@users.noreply.github.com>
* Update docs/troubleshooting-guide/cant-send-email.md
Co-authored-by: flamber <1447303+flamber@users.noreply.github.com>
* Update docs/troubleshooting-guide/cant-send-email.md
Co-authored-by: flamber <1447303+flamber@users.noreply.github.com>
* Fixing first item
* Addressing feedback
* Update docs/troubleshooting-guide/cant-send-email.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Update docs/troubleshooting-guide/cant-send-email.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Update docs/troubleshooting-guide/cant-send-email.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Adding flamber's comment
* Update docs/troubleshooting-guide/cant-see-tables.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Update docs/troubleshooting-guide/cant-see-tables.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Update docs/troubleshooting-guide/cant-see-tables.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Splitting entry
* Update docs/troubleshooting-guide/cant-see-tables.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Update docs/troubleshooting-guide/cant-see-tables.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Update docs/troubleshooting-guide/cant-see-tables.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Troubleshooting timezones
* Reformatting the filter troubleshooting guides
* Partial update to troubleshooting filters
* specify sql questions
* link
* update relative links
* more link fixes
* still more links
* specify dashboard filters
* fix typo
* Italicizing one more word
* Unifying all troubleshooting guides so far
* Update docs/troubleshooting-guide/cant-see-tables.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Update docs/troubleshooting-guide/cant-see-tables.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Incorporating suggestion
* Update docs/troubleshooting-guide/my-dashboard-is-slow.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Update docs/troubleshooting-guide/my-dashboard-is-slow.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Update docs/troubleshooting-guide/my-dashboard-is-slow.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Update docs/troubleshooting-guide/my-dashboard-is-slow.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Update docs/troubleshooting-guide/my-dashboard-is-slow.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Another small fix
Co-authored-by: flamber <1447303+flamber@users.noreply.github.com>
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
Co-authored-by: jeff bruemmer <jeff.bruemmer@gmail.com>
* Fix serialization dump error when there are no collections
Update `select-collections` to correctly handle the case where there are no collections
Adding new test that removes all collections, then ensures that dump works with no errors
* Fix serialization load error into empty/blank target DB
Rethrowing exception in cmd when overall load fails
Add test to ensure that a dump containing a user can be loaded into a blank target app DB successfully
Adding a few missing bindings to the `with-temp-empty-app-db` code to set the connection vars under metabase.db.connection
In upsert, add hooks for a :pre-insert-fn and :post-insert-fn to be invoked for new entity instances created by the upsert process, since whether an entity will be an insert or update isn't necessarily known by the load process (only by upsert once identity-condition is checked for each)
In load, setting the pre and post insert functions for a User instance to initialize :password with a random value, and to generate and send a password reset email to the newly inserted user's email, respectively
NOTE: this post insert fn (to send a password reset email for newly inserted users) is NOT hooked up for User as of x.41 release, since it is considered a bugfix, but this can be enabled in a future release)
Adding new defs for the magic permission group names to make those easier to override from tests that might need to (such as the one added for this commit)
* Fix serialization objects being incorrectly updated on skip
Remove `maybe-fixup-card-template-ids!`, which was forcing mode :update, since the existing retry logic should cover what it was trying to do
Update Card and Metric models to delete any Dependency instances for which they are the `:model_id` (to make serialization tests after other tests created temporary Card/Dependency pairs)
Adding missing assertions for Dependency serialization
* Serialization: Fix reload entity logic
Now that the `:mode` is always respected instead of being ignored sometimes, we need to update our "second pass" reload functions to always make the mode `:update` on the second pass, or else that entity would just be skipped, which is bad
Also updated the test to use `:mode` `:skip` from the beginning to be more stringent
* refactor/move parameter object transformations to one place
* use hasParameterValue in place of nil check
* remove enableDefaultParameters arg since it is unused
New BigQuery Driver
Create new :bigquery-cloud-sdk driver using the google-cloud-bigquery library instead, and whose source is adapted from the :bigquery driver
https://cloud.google.com/bigquery/docs/reference/libraries
Marking existing :bigquery driver as deprecated, and superseded-by the new one (bigquery-cloud-sdk)
Update new driver and query processor code to use newer Google SDK
Switch test data loading over to use new API
Add project-id connection property to override the value from the service account JSON, and use it as part of qualified names in the query processor if set
Updating google driver so its libraries are compatible with the newer ones used in BigQuery
Update date bucketing tests to skip :bigquery-cloud-sdk (new driver) where :bigquery is skipped
Update `with-bigquery-fks` to take in the driver, since there are now multiple ones
Adding test to confirm that overriding project-id for a public BQ project works (sync and query)
Fixing a bunch of whitespace alignment errors in tests
* Add tooltips to authority level configs
* Use tooltips for CollectionAuthorityLevelIcon
* Pass tooltips to getCollectionIcon result
* Display tooltip in CollectionHeader
* Display tooltip in collections sidebar
* Display tooltip on the homepage
* Display tooltip in ItemPicker
* Accept icon props as a TreeNode's icon prop value
* Display tooltip in saved question picker
* Accept icon props as a SelectListItem's icon prop value
* Display tooltip in dashboard's question picker
* Add "Belongs to an Official collection" tooltip
* Fix prop-types errors
* Add tests for CollectionAuthorityLevelIcon
* Add tests for isRegularCollection
* Fix search results collection icons
* Show "Official Collection" label in search results
* Don't show tooltips for official collections in search
* Fix ItemPicker's prop type
* Add basic test for ItemPicker
* Test read-only collections are not displayed
* Add test IDs to ItemPicker
* Refresh Redux store in each ItemPicker's test
* Test opening a nested collection
* Test default breadcrumbs state
* Fix ItemPicker's header test ID
* Test ItemPicker read-only items visibility
* Add test for onChange prop
* Test navigating back from an open collection
* Test personal collections are grouped in ItemPicker
* Assert number of ItemPicker items
* Test getCollectionIcon in EE
* Add a workaround for tooltip offsets
* Add describe block for CollectionsList tests
* Make CollectionsList test shorter
* Test collection types on CollectionList
* Fix SearchResult's info
* Add SearchResult tests
* Use isRegularCollection for AuthorityLevelIcon
* Refactor and dry up `question` custom Cypress commands
* Example of `loadMetadata` custom option in action
* Example of `visitQuestion` custom option in action
* Annotate functions using JSDoc
* Help avoid the .lein-env gotchas
* Learn to spell @camsaul
* Revert the .gitignore change -- too D A N G E R O U S
* Fix version numbers (thanks @jeff303)
* Change of plans. get-string (etc) only return default if it matches the type
* Better docstrings
* More tests & rework the validate-default-value logic a bit.
* Fix lint error
* Add an additional test for GET /api/setting
* Consolidate exception-chain util functions
* Email error message humanization improvements
* More tests for util ex-chain
* Clean up email error handling and add lots of extra tests
* Fix docstring indentation
* Even more email tests
* Test fixes 🔧
* Update Cypress test again
* Code cleanup
* Backend SVG rendering proof of concept [ci skip]
* Update cssbox to 5.0.0
* Render bar, line, and pie charts in js to svg
sparkline is now done in js, bar is now recognized and done in js, new
:categorical/donut as well
* Remove api route for render
* pass along render-type, not hardcoded to :inline
* Move bar chart above sparkline and remove line check
In order to introduce the bar chart type need it above the sparkline
check since it is otherwise the same except for display property of
the card. But lots of tests assume that this will get hit with a nil
display type set in testing so remove checking for `:line` allows all
the testing cases to hit the right type
* Fix tests now that bar graphs aren't html but images
* Include attachments for bar charts
* Move over to in-tree bundle
* Force everything [ci noskip]
trying to ensure that the built jar includes the newer
"resources/frontend_client/app/dist/lib-static-viz.bundle.js"
* Run `yarn build-static-viz` in backend-deps in CI
this js file is now a hard dependency of the backend so it fits in
this tsk. All such things that depend on the backend sources will need
it. Makes me think perhaps we want a checked in version but i'm not
sure yet.
* Look on classpath not filesystem for js bundle [ci noskip]
* Move yarn build-static-viz into the checkout step
* License information for antlr4-runtime
* create attachment for categorical donuts
* add ordinal legend to donuts (#17177)
* set widths of html image and svg image to 1200
* Revert "add ordinal legend to donuts (#17177)"
This reverts commit 1eb81d2ebf.
* Helper functions to render html easily
* readme in dev
* readme ensure that static viz bundle exists
* Cleanup ns after removing proxy
* Donut chart colors and legend (#17251)
* use external color map for fill per dimension
* Add support new color legend for donut
* Ensure text doesn't appear as link
entire thing is actually the body of a link tag for emails but we want
a decent text color rather than a default link color
* use chart colors from https://stats.metabase.com/_internal/colors
* Make checkers happy
- remove unused imports
- add a docstring
- don't shadow fn with a local
* cleanup ns import
* Remove reflective call
* Cleanup ns on correct branch
Co-authored-by: dan sutton <dan@dpsutton.com>
* X-axis: just use (approx) 5 ticks to avoid overlapping labels (#17287)
* increase gap between arcs (#17271)
* Set rendering hints on html->image
* ignore width for now and make them larger
* Ns deprecation and some cleanup
* make namespace checker happy
* Simple tests for detecting chart type
* Rename from poc
* Tests for scalar/smartscalar
* cleanup js svg namespace a bit
* Tests of svg engine
* ns sorting after renaming
* Unify our two different js engine usages
settled on the js context. Has typed returns `(.asString ^Value ...)`
instead of perhaps capturing std out?
https://www.graalvm.org/sdk/javadoc/org/graalvm/polyglot/Value.html
Context is a bit more friendly for getting source into it. One
downside is that the invocable bit isn't quite as nice. The old way
would return a java.util.functionFunction but the difference is
(.apply function (object-array args))
vs
(.execute fn-ref (object-array args))
* Don't io/resource the io/resource
* js engine tests
* Ns cleanup in js-svg
type hints in the js-engine ns mean we don't need as many classes from
polyglot here
* Include percentages in rendered donut charts
* Testing for percentages
Co-authored-by: Cam Saul <github@camsaul.com>
Co-authored-by: Kyle Doherty <5248953+kdoh@users.noreply.github.com>
Co-authored-by: Ariya Hidayat <ariya@metabase.com>
* Backend SVG rendering proof of concept [ci skip]
* Update cssbox to 5.0.0
* Render bar, line, and pie charts in js to svg
sparkline is now done in js, bar is now recognized and done in js, new
:categorical/donut as well
* Remove api route for render
* pass along render-type, not hardcoded to :inline
* Move bar chart above sparkline and remove line check
In order to introduce the bar chart type need it above the sparkline
check since it is otherwise the same except for display property of
the card. But lots of tests assume that this will get hit with a nil
display type set in testing so remove checking for `:line` allows all
the testing cases to hit the right type
* Fix tests now that bar graphs aren't html but images
* Include attachments for bar charts
* Move over to in-tree bundle
* Force everything [ci noskip]
trying to ensure that the built jar includes the newer
"resources/frontend_client/app/dist/lib-static-viz.bundle.js"
* Run `yarn build-static-viz` in backend-deps in CI
this js file is now a hard dependency of the backend so it fits in
this tsk. All such things that depend on the backend sources will need
it. Makes me think perhaps we want a checked in version but i'm not
sure yet.
* Look on classpath not filesystem for js bundle [ci noskip]
* Move yarn build-static-viz into the checkout step
* License information for antlr4-runtime
* create attachment for categorical donuts
* add ordinal legend to donuts (#17177)
* set widths of html image and svg image to 1200
* Revert "add ordinal legend to donuts (#17177)"
This reverts commit 1eb81d2ebf.
* Helper functions to render html easily
* readme in dev
* readme ensure that static viz bundle exists
* Cleanup ns after removing proxy
* Donut chart colors and legend (#17251)
* use external color map for fill per dimension
* Add support new color legend for donut
* Ensure text doesn't appear as link
entire thing is actually the body of a link tag for emails but we want
a decent text color rather than a default link color
* use chart colors from https://stats.metabase.com/_internal/colors
* Make checkers happy
- remove unused imports
- add a docstring
- don't shadow fn with a local
* cleanup ns import
* Remove reflective call
* Cleanup ns on correct branch
Co-authored-by: dan sutton <dan@dpsutton.com>
* X-axis: just use (approx) 5 ticks to avoid overlapping labels (#17287)
* increase gap between arcs (#17271)
* Set rendering hints on html->image
* ignore width for now and make them larger
* Ns deprecation and some cleanup
* make namespace checker happy
* Simple tests for detecting chart type
* Rename from poc
* Tests for scalar/smartscalar
* cleanup js svg namespace a bit
* Tests of svg engine
* ns sorting after renaming
* Unify our two different js engine usages
settled on the js context. Has typed returns `(.asString ^Value ...)`
instead of perhaps capturing std out?
https://www.graalvm.org/sdk/javadoc/org/graalvm/polyglot/Value.html
Context is a bit more friendly for getting source into it. One
downside is that the invocable bit isn't quite as nice. The old way
would return a java.util.functionFunction but the difference is
(.apply function (object-array args))
vs
(.execute fn-ref (object-array args))
* Don't io/resource the io/resource
* js engine tests
* Ns cleanup in js-svg
type hints in the js-engine ns mean we don't need as many classes from
polyglot here
* add custom labels
* clean up margin
* Include labels in backend
* alignment
Co-authored-by: Cam Saul <github@camsaul.com>
Co-authored-by: dan sutton <dan@dpsutton.com>
Co-authored-by: Ariya Hidayat <ariya@metabase.com>
* Backend SVG rendering proof of concept [ci skip]
* Update cssbox to 5.0.0
* Render bar, line, and pie charts in js to svg
sparkline is now done in js, bar is now recognized and done in js, new
:categorical/donut as well
* Remove api route for render
* pass along render-type, not hardcoded to :inline
* Move bar chart above sparkline and remove line check
In order to introduce the bar chart type need it above the sparkline
check since it is otherwise the same except for display property of
the card. But lots of tests assume that this will get hit with a nil
display type set in testing so remove checking for `:line` allows all
the testing cases to hit the right type
* Fix tests now that bar graphs aren't html but images
* Include attachments for bar charts
* Move over to in-tree bundle
* Force everything [ci noskip]
trying to ensure that the built jar includes the newer
"resources/frontend_client/app/dist/lib-static-viz.bundle.js"
* Run `yarn build-static-viz` in backend-deps in CI
this js file is now a hard dependency of the backend so it fits in
this tsk. All such things that depend on the backend sources will need
it. Makes me think perhaps we want a checked in version but i'm not
sure yet.
* Look on classpath not filesystem for js bundle [ci noskip]
* Move yarn build-static-viz into the checkout step
* License information for antlr4-runtime
* create attachment for categorical donuts
* add ordinal legend to donuts (#17177)
* set widths of html image and svg image to 1200
* Revert "add ordinal legend to donuts (#17177)"
This reverts commit 1eb81d2ebf.
* Helper functions to render html easily
* readme in dev
* readme ensure that static viz bundle exists
* Cleanup ns after removing proxy
* Donut chart colors and legend (#17251)
* use external color map for fill per dimension
* Add support new color legend for donut
* Ensure text doesn't appear as link
entire thing is actually the body of a link tag for emails but we want
a decent text color rather than a default link color
* use chart colors from https://stats.metabase.com/_internal/colors
* Make checkers happy
- remove unused imports
- add a docstring
- don't shadow fn with a local
* cleanup ns import
* Remove reflective call
* Cleanup ns on correct branch
Co-authored-by: dan sutton <dan@dpsutton.com>
* X-axis: just use (approx) 5 ticks to avoid overlapping labels (#17287)
* increase gap between arcs (#17271)
* Set rendering hints on html->image
* ignore width for now and make them larger
* Ns deprecation and some cleanup
* make namespace checker happy
* Simple tests for detecting chart type
* Rename from poc
* Tests for scalar/smartscalar
* cleanup js svg namespace a bit
* Tests of svg engine
* ns sorting after renaming
* Unify our two different js engine usages
settled on the js context. Has typed returns `(.asString ^Value ...)`
instead of perhaps capturing std out?
https://www.graalvm.org/sdk/javadoc/org/graalvm/polyglot/Value.html
Context is a bit more friendly for getting source into it. One
downside is that the invocable bit isn't quite as nice. The old way
would return a java.util.functionFunction but the difference is
(.apply function (object-array args))
vs
(.execute fn-ref (object-array args))
* Don't io/resource the io/resource
* js engine tests
* Ns cleanup in js-svg
type hints in the js-engine ns mean we don't need as many classes from
polyglot here
* Cleanup of text, ns docstrings, alignment
* Fix fill->fill-opacity with parsed doc, not regex
* Make a single helper that loads a static viz bundle context
* Docstrings and make private in js-svg
* Sort imported classes in js-svg
* Make width passed down through rendering aparatus
- svgs are always rendered at 1200 for quality
- slack images of html are rendered at 1200 so that they can be zoomed
in in the ui but slack automatically scales down
- email sends html and the full svg but includes img width tags so
that is handled appropriately
* docstring
Co-authored-by: dan sutton <dan@dpsutton.com>
Co-authored-by: Kyle Doherty <5248953+kdoh@users.noreply.github.com>
Co-authored-by: Ariya Hidayat <ariya@metabase.com>
* upgrade to @percy/cypress 3.x
@percy/cypress 2.x uses deprecated @percy/agent which includes
an old version of puppeteer. The old version of puppeteer causes
`yarn` command to fail on M1 macOS environment.
Upgrade to @percy/cypress 3.x removes the dependency and thus
solves the issue.
The upgrade was done by running `npx @percy/migrate`.
* remove percy healthcheck task
This no longer exists in 3.x, see
https://github.com/percy/percy-cypress/blob/master/README.md#upgrading
* remove deprecated percy/exec-action
and moved `percy exec` to the correct command
* run 'yarn install' before running percy
Consult any of these keys when trying to determine the "name" portion of the c3p0 data: :db :dbname :sid :catalog
Update existing `c3p0-datasource-name-test` to more robustly check the different portions of the c3p0 dataSourceName
* add moderation events to question timeline
* tweak the height of the DrawerSection header
* replace history modal & control drawer state via redux
* add/fix cy tests
* fix banner unit tests
* prevent leakage of 'null' moderation icon
* rename styled component
* add tests / rmv redundant fn
* remove should('exist') from moderation cy tests
* refactor to make biz logic of a null mod status clearer
* add selector unit test
* Group Badge's icon props into a single `icon` prop
* Allow overwriting Badge's hover color
* Fix CollectionBadge hover for official items
* Fix CollectionBadge size for official items
* Fix QuestionDataSource badge
* Use iconPropTypes in Badge
* Make CollectionBadge icon code more compact
Previously the multifunction for supporting features was always implemented as a trivial true function. However, for mongo we need an actual version number, so here we go
* Move sync executor to bespoke namespace
* Refingerprint fields on type change
* Check if can connect in refingerprint-field!
* docstring update
* Cleanup tests a bit
* Error handling
* Table field values endpoint on sync.concurrent executor
* ns sort api field
* ns cleanup
* Namespace for extensions to clojure.test
if you do a subselection of tests, you might not get the `schema=`
defmethods and then your tests won't compile. Ns docstring to keep the
namespace simple and targeted to global effects. we don't want to load
models and do extra work, just install some test utilities that aren't
ever required directly
* Add aliases
clojure.test had been refer :all
* require the effects in metabase.test.util as well
Namespace cleaner got a bit too aggressive
stupid ns
* Poke github actions empty commit
* Add the whitespace linter
* Fix whitespace linter errors [except for one file]
* Add a line that will intentionally break stuff to verify the linter is working.
* Ok, remove the line that caused the linter to fail.
* Use latest version of the linter
* Fix missing newline
* Include and bump moderation status in search results
* Include limit 1 and order by for moderation review latests
backend does work to ensure there is an invariant that there is 1
most_recent true entry and all the rest are most_recent false. But if
there's some bug this invariant is violated we don't want to blow up
with multiple rows returned in these queries.
* Verification scoring is enterprise now
* wip moving moderation review to enterprise
still need to delete the old non-moved file
* add moderation icon to search results
* update names of icon props in SavedQuestionHeaderButton
* move moderation cypress tests, add test for search results
* Move moderation review api and test to enterprise
* fix props for getVerifiedIcon call
* add moderation status icon to collections
* add/improve moderation cypress tests
Co-authored-by: Dalton Johnson <daltojohnso@users.noreply.github.com>
Since it needs to grab the built JAR anyway, it's faster to obtain the
version.properties from the JAR.
This shaves off approx. 1 minute from Percy CI run.
There's no point of processing files via ESLint loader when building
for production. If there's any linting issue at all, it should be
caught with a different CI run.
This shave off >2 minutes from Uberjar build (front-end step).
Development/hot-reload isn't affected at all.
- Make all specific problems H2-level headings.
- Create a `doc-toc` div at the top for the table of contents.
- Don't use `learn-doc` because we may want to style/show this one independently.
- Link to all problem headings.
Note: can't use metadata in a page header because our CI tools splat their own headers into these pages, so anything we add becomes part of the page body.
Adding new cache to `metabase.driver.sql-jdbc.connection` that tracks the DB details hash values per DB id
Checking if the non-empty hash value of DB details changed in `db->pooled-connection-spec` in order to invalidate an existing pool
Adding test that redefines the log fn to assert that the pool is invalidated on hash change
It's redundant since the job explicitly installs the proper version in
the previous step, and also running that prematurely pulls a bunch of
JARs from Maven Central.
This should shave 20 seconds from the CI runs.
* Bump some more backend dependencies
* Add license info for com.sun.activation/jakarta.activation
* Fix reflection warnings
* Revert bump of MariaDB for now
Remove the `driver/can-connect?` implementation and move the `select-keys` logic instead to the existing place in `sql-jdbc.conn/connection-details->spec`
Update test to also include the `:let-user-control-scheduling` details entry
* Fix JUnit output not correctly stripping ANSI color codes
* Remove find-tests profiling log message
* 1-arg arity of metabase.test-runner/run
* Fix bug with metabase.test-runner/find-tests with a single test name
* Fix location of share icon
* Update 12-public-links.md
Differentiate between dashboard and question sharing
Co-authored-by: Luis Paolini <paoliniluis@gmail.com>
* remove moderation reviews hydration
* Restore hydration of moderation_reviews and handle correctly
nested hydration is (i think) positional. So if you get something
handed to you you must return something in that position. Prevoiusly
was removing nils and then returning a smaller collection of hydrated
items (cards here). But this meant input might look like this
[card1 nil card2] and return [card1 card2] and in the nested hierarchy
things didn't get matched up correctly.
In the real world application it might look like this:
```clojure
{:ordered-cards
[{:card-id 1}
{:card-id nil
:viz-settings {info-for text-card}}
{:card-id 2}]}
```
And the nil card comes into this function and we return them in a
strange manner things get wonky
* Remove `project.clj` from Percy workflows
* Replace `project.clj` with `deps.edn` in `build-for-test` script
* Update reference to the `project.clj` in the comment
We're playing this whackamole because we added the middleware to universally sorta stomp on any `limit` or `offset` params, and they got stomped on. Which, in turn, led to my PR to make the default not limited. Which, in turn, led to the other issue this PR solves which makes the search things we relied upon to not have a limit, have a limit.
* Add some dox
* Rename test-runner & cloverage -> metabase.test-runner & metabase.cloverage-runner
* Move JUnit-related test runner code to separate namespace
* Bespoke JUnit-output code
* Delete unused .lein-classpath file
* Do JUnit writes in a cached thread pool
* Add failing test to make sure JUnit output works as expected...
* Move bad-test to metabase.bad-test so the FAILING tests actually run
* FASTER TEST LOADING 🏎️
* Test CircleCI :system-out
* Final cleanup (hopefully)
* Sort some namespaces; allow initializing :plugins when namespaces get loaded for now.
* Need those [[tags]] in docstrings
* Fix bad docstring
* rmv old bucm icons and remove verified fill color
* add moderation action section to sidebar
* add moderation review icon to the saved question header button
* hide moderation section when is not a moderator
* add UI for ModerationReviewBanner
* Backend for moderation-review
- create table moderation_review. Same as before but also has a
"most_recent" boolean flag for the most recent moderation for easy
lookup
- POST /moderation-review/ . Status can be "verified" or nil
- must be an admin to post
- No PUT or edit route yet. Not sure if this is even
necessary. _MAYBE_ to edit the text, but certainly not for the
status, ids, etc. If there's to be history, let's build some history
- Ensure we never have more than 10 reviews. Adding a new review will
delete the older ones, mark all old ones as not `most_recent`, and
add the newest one as `most_recent true`
- Ensure the card actually exists before creating the mod review
- Since admin only at this time, don't need to check moderate
permission or view permission
- When hydrating ensure reviews are ordered by id desc. Should mimic
the created_at desc
* fix moderation review banner tooltip offset
* disable verification button when already verified
* rmv iconOnly prop because it seems to do nothing
* update getLatestModerationReview to rely on most_recent boolean
* Return 400 on invalid status to post /moderation-review
the schema was using keywords on the left hand side rather than the
symbols. Required a change to the docstring generator, when it made a
docstring for enums, it would call (sort (:vs enum)) and need to
string em.
* Add ModerationReview model to models.clj and copy infra
* hydrate moderation reviews on cards
* clean up + wire up to BE + ensure mod buttons don't show for normal users
* rmv unused moderation redux logic from QuestionDetailsSidebarPanel
* finish writing unit tests for FE
* ensure getIconForReview returns an object
* enable/disable verify button tooltip when unverified/verified
* add e2e tests
* fix tests
* styling tweaks
* more styling on moderationReviewBanner
* add function for abbreviated timestamp
* increase fontsize of timestamp back to 12
* fix tooltip offset
* ensure custom locale is separate from 'en' and not used for other languages
* Deletion moderation reviews when deleting cards
i had actually thought this was a much larger problem. But it turns
out we almost never delete cards (thanks comment!). And so we won't
really generate a lot of garbage.
I was worried that since we aren't using actual foreign keys but just
`moderated_item_type "card"` and `moderated_item_id 2` we would have
deleted cards with these moderation reviews but that is not the case
as the cards aren't deleted.
* hide verify disabled button when a question is verified
* update test to use queryByTestId
* Hydrate moderation reviews on cards on ordered cards
* Handle mysql's lack of offset functionality
mysql cannot handle just a `offset` clause, it also needs a limit
clause grammar from
https://dev.mysql.com/doc/refman/8.0/en/select.html:
[LIMIT {[offset,] row_count | row_count OFFSET offset}]
select id, name from metabase_field offset 5; -- errors
select id, name from metabase_field limit 2 offset 5; -- works
Since our numbers are so small here there is no worry and just do the
offset in memory rather than jump through hoops for different dbs.
* Batch hydrate moderation reviews
* Don't let /api/user/:userId failure conceal moderation banner
* fix moderation cy tests
* work around possible bug in toucan hydration
dashboards hydrate ordered cards
(hydrate [:ordered_cards [:card :moderation_reviews] :series])
Ordered_cards are dashboard_cards which have an optional card_id. But
toucan hydration doesn't filter out the nils as they go down. It seems
toucan returns a nil card, and then when hydrating the
moderation_review, passes the collection of all "cards" including the
nil ones into the hydration function for moderation_reviews
This feels like a bug to me
* Cleanup moderation warnings
* Docstring in moderation review
* include hoisted moderated_status on cards in collections api
* Expect unverified in test 🔧
Co-authored-by: dan sutton <dan@dpsutton.com>
Co-authored-by: Maz Ameli <maz@metabase.com>
Co-authored-by: alxnddr <alxnddr@gmail.com>
* Handle mutation of vizSetting column when syncing columns of a native question
When calling Question's _syncNativeQuerySettings method we need to treat
existing, mutated columns as invalid so that they are filtered out and a
new column takes the existing column's place.
The function we use to find valid columns, findColumnIndexForColumnSetting,
will return true if it finds a matching field_ref OR a matching name, so we
also need to check the addedColumns array we just created in order to remove
any columns from vizSettings that also exist there, since we combine the two
at the end of this method.
* add unit tests for _syncNativeQuerySettings
Updating schema connection property to have proper display name
Updating display name for database to be "Schema"
Fixing `driver/can-connect?` impl in `presto_jdbc.clj` to actually use the correct driver, and dissoc `:engine` before calling the `sql-jdbc.conn` method
Adding test for the `test-database-connection` method, which is what led to the error from the UI
Only build currently active drivers
Update `all-drivers` fn so that it only "finds" drivers that have either a `project.clj` or `deps.edn` file (i.e. some build file), and ignores the directory otherwise. This is to deal with cached artifacts hanging around between builds in a shared executor type environment (ex: CircleCI), where even though the checkout won't bring in some driver directory that doesn't exist on the target branch, some other files (such as `pom.xml` or a `target` directory), might still be hanging around
* Update babel dependencies to latest
* Remove some old babel dependencies
The dependencies in this commit include dependencies that have been
renamed, such as babel-cli which is now @babel/cli. It also includes
dependencies that have been replaced, such as babel-preset-stage-0 and
its ilk, which have been replaced with @babel/preset-env.
* Remove babel-register dependency
This is referenced in our webpack.config.js file. I don't think we need
Babel in our Node envionment, but I could be wrong, so leaving this as a
separate commit.
* Remove @babel/standalone and dependent code
We're using @babel/standalone to support the writing of JSX in our
internal-only ScratchApp editor. Unfortunately, this dependency is
large--1.5mb not gzipped--and our current build does not intelligently
split bundles, meaning our users our burdened by the size of this
dependency whenever they must redownload the Metabase JS bundle.
I'm removing it. We should still be able to write JS in the ScratchApp,
but JSX no longer.
* Add @babel/node to dev deps
Needed by a cypress command
* Add browserslist config
Babel's preset-env dep needs this config to determine how to transform
our JavaScript. By default I believe it ends up defaulting to
"defaults."
"defaults" is equivalent to the following rules:
> 0.5%, last 2 versions, Firefox ESR, not dead
A comma between rules represents a UNION operator.
Per
https://www.metabase.com/docs/latest/faq/general/supported-browsers.html
we have deprecated IE 11 so I am including that explicitly in our
browserslist.
* Update babelrc plugins and presets
- Replaced presets with preset-env and preset-react
- Updated the names of some plugins, like transform-decorators-legacy,
which is now @babel/plugin-proposal-decorators with an explicit
"legacy: true" passed in the config.
- Removed add-react-displayname because it is included in preset-react
- Removed transform-builtin-extend because it is included in preset-env*
- Removed syntax-trailing-function-commas because it is supported by
all browsers we support
* note: There is a "loose" config option for handling the extension of builtin
classes. We may need it, but I am not sure.
* Update babel cli args
Replace -q with --quiet because -q no longer exists
* Fix circular dep in auth.js by making it async
Our app unfortunately contains MANY circular dependencies. Some, like
this one, caused problems when the app instantiates because the
"refreshCurrentUser" method is undefined when auth.js is initially run.
While we should in the future avoid having circular deps in the first
place, making them run async by requiring the dep and function run-time
fixes this particular issue.
* Fix circular dependency in metabase/redux/requests.js
The handleActions and createAction methods imported from
"metabase/lib/redux" are actually just methods from the "redux-actions"
dependency... so importing those methods directly from that dep fixes
this circular dependency problem.
* Remove references to global.services and exports in metabase/services
This line breaks the build. Git blame says the line is 5 years old, and
I wouldn't expect such code to be used with how our app now requires
deps.
* Fix MBQLArrayClause problem caused by Array extension
* Explicitly pass args from flatMap to function via anonymous cb
I think the old Array.prototype.flatMap prototype must've had different
args. They are now "currentValue, index, array" so when we passed it
"enumeratePaths" as a callback, the second arg of enumeratePaths gets
set to a number. This breaks the logic of enumeratePaths because it is a
recursive function that depends on that second arg getting initialized
as an empty array by default.
* Remove code that spreads non-iterable value in array
New Babel is stricter about handling this sort of code that doesn't
match the JS spec. You can't spread an undefined value inside of an
iterator.
* Update jest and babel-jest to latest
Updating Babel breaks our unit tests. Updating nothing causes every test
to fail due to a Babel version mismatch. Updating only babel-jest causes
the following error:
TypeError: Jest: a transform must export a `process` function
There may be a way around updating Jest, but updating it does not appear
to be too difficult.
* Set the testEnvironment to jsdom
This was the default testEnvironment in our old version of jest. At some
point the default was changed to node.
* Remove unfinished "pending" unit tests
I'm unfamiliar with this pending function, but the function no longer
exists, causing the tests to fail.
* don't pass async callback to describe
* Update name of runTimersToTime to advanceTimersByTime
* Fix incorrectly nested unit test
* mock SVGElement.prototype.getBBox method
* Don't return values in describe callback
* Move variable declaration to above usage
* Fix prettier error
* Upgrade `documentation` library to the latest version (13.x.x)
This library was the only one left that was still using Babel 6.
* Return an MBQLClause instance after using Array methods
* remove 'not IE 11' from browserslist
* Remove mock that breaks unit test
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
* Fix some Eastwood failures
* Fix a lot of Eastwood errors now that it runs against test namespaces [ci skip]
* Run Eastwood against test namespaces [WIP]
* Bump Eastwood version to fix some errors in Eastwood itself
* Fix another lint warning
* Fix test failure
* More test fixes 🔧
* Fix all the warnings!
* Test fix 🔧
* Bump Eastwood GH action timeout to 20 minutes (!)
* Add getCollectionIcon function
* Add isRegularCollection function
* Display official badge in collections sidebar
* Don't dim official badge icon in collections sidebar
* Show root collection icon
* Use `CollectionsList.Icon` for "All personal collections"
* Fix CollectionBadge icon color
* Display official collection badge in search results
* Test official badge displayed correctly in sidebar
* Test official badge is shown throughout the app
* Show official badge icon in SavedQuestionPicker
* Use existing isPersonalCollection utility
* Show official badge icon in QuestionPicker
* Show official badge icon in CollectionPicker
* Minor CollectionIcon refactoring
* Use function declaration for isRegularCollection
* Add editCollection test helper
* Add changeCollectionTypeTo helper
* Remove duplicated assertions
* Add helper to create official collection
* Add testOfficialBadgeInSearch helper
* Add badge icon
* Add empty PLUGIN_FORM_WIDGETS
* Add empty PLUGIN_COLLECTIONS
* Extract `getWidgetComponent` from FormWidget
* Allow adding EE-only form widgets
* Add `authority_level` field to collections
* Fix SegmentedControl's `value` prop-type
* Specify icon sizes for SegmentedControl's options
* Add FormCollectionAuthorityLevel form widget
* Add `PLUGIN_COLLECTION_COMPONENTS`
* Add CollectionAuthorityLevelIcon
* Move ToggleMobileSidebarIcon out of PageHeading
* Display official badge on collection page
* Add authority level to collections plugin
* Rename `AUTHORITY_LEVELS` to `EE_AUTHORITY_LEVELS`
* Patch collections plugin with EE authority levels
* Fix authority level value for new collection form
* Add `cy.createCollection` command
* Add `describeWithoutToken` Cypress helper
* Test user can manage collection authority level
* Test collection types available only in EE
* Run prettier
* Keep regular collection opts in AUTHORITY_LEVEL
* Fix test comment
* Translate collection type names
* Avoid aliasing regular authority level
* Minor fixes to performance troubleshooting guide
* Update docs/troubleshooting-guide/performance.md
Co-authored-by: flamber <1447303+flamber@users.noreply.github.com>
* Fixing a few small issues with the performance troubleshooting guide.
* Another fix to performance troubleshooting
Co-authored-by: flamber <1447303+flamber@users.noreply.github.com>
* Improve prop-types, move to the top of the file
* Make FormField a functional component
* Simplify default props assignment
* Remove redundant comparisons
* Add InputContainer component
* Extract ALL_DOT_CHARS regexp for form field id
* Simplify Label
* Extract rootClassNames
* Fix formFieldId
* Fix FormField prop type
* add question details sidebar (code from bucm)
update some cy tests
rmv moderation-related code
rmv padding on description
merge history button logic into label
lint fix
update some clicks in cy tests
update styles to use styled-components
fix more cy tests
rmv unused component
rmv unused reference
rmv unused prop
remove styling code that is no longer necessary
rmv extraneous space
fix translation
rmv added util class
move styled component to own file
* add revision util tests
* rmv some unneeded question panel styling
* move files into query_builder directory
* rearrange stuff in file
* ClampedDescription unit test
* add QuestionActionButtons unit tests
* add SavedQuestionHeaderButton test
* add QuestionActivityTimeline tests
* tweak/improve props/propTypes
* fix cy test referencing question description
* fix revision history cy test
* Question sidebar visual polish (#16942)
* style tweaks
* restyle Revert button
* add a little breadcrumb spacing
* make LastEditInfoLabel use Badge instead of TextButton
* bring back TextButton
* Revision History => History
* fix activity timeline unit test
* rmv unused styled components
Co-authored-by: Dalton Johnson <daltojohnso@users.noreply.github.com>
* hide 'add a description' button when read only
* update permissions cy tests
* update ClampedDescription unit test
* use size enum instead of boolean
* move export to bottom of file
* move modal magic strings to a constants file
* clobber all qb state when reloading
* Fix reversion cy tests
Co-authored-by: Maz Ameli <maz@metabase.com>
Implement JDBC based Presto driver
Adding new Presto JDBC driver using the PrestoDB JDBC driver from `https://github.com/prestodb/presto`
Marking the old Presto driver as being `superseded-by` the new one
Pulling out common Presto code into new presto-common driver (modeled after the relationship between, ex: `googleanalytics` and `google)`
Putting common QP/HoneySQL logic into the new (abstract) :presto-common driver
Updating :presto driver to extend from the new common driver and only adding HTTP/REST API related methods
Adding implementation of Presto JDBC driver, named :presto-jdbc, extending from :presto-common and :sql-jdbc
Using com.facebook.presto/presto-jdbc as underlying JDBC driver dependency (since this is explicitly for Presto clusters, as opposed to Trino)
Adapting code from the existing Presto driver where appropriate
Adding new dependency-satisfied? implementation for :env-var, to allow for a plugin to require an env var be set, and making the Presto JDBC driver depend on that (specifically: `mb-enable-presto-jdbc-driver`)
Adding CircleCI configuration to run against the newer Presto (0.254) Docker image
Adding explicit ordering in a few tests where it was missing
Fixing presto-type->base-type (timestamps were being synced as :type/Time because the regex pattern was wrong)
Add tx/format-name test implementation for :presto-jdbc to lowercase table name
Make modified test Oracle friendly
Fixing bug parsing the `[:zone :time]` case within `metabase.util.date-2.parse/parse-with-formatter`; the offset is nil so it can't be passed directly in this case, so use the `standard-offset` fn (which was moved from `date-2` to `common` to get a standard offset for that zone
Fixing more test failures by adding explicit ordering
Changing sync to check whether the driver supports foreign keys before attempting to sync those (since drivers might throw an exception if attempting to check)
Moving some common test dataset functionality between :presto and :presto-jdbc to a new test.data ns for :presto-common
Adding HoneySQL form for :count-where, since we have to explicitly give a higher precision Decimal in order for Presto to not reduce the precision in results
Put limit within subquery for `expression-using-aggregation-test` (since ordering from subquery is not guaranteed in Presto)
Adding impls for ->prepared-substitution to handle substitutions for native query params
Adding HoneySQL impls for `mod` (to do as "mod(x,y)" and `timestamp` (since it's a function with no parens to invoke) functions
Adding various `sql.qp/date` impls that use the `AT TIME ZONE` operator to account for report tz, and make bucketing tests happy
# Backend changes
Introducing new `superseded-by` property to plugin manifest YAML, which will indicate the driver that is to eventually replace this one (and will drive UI/UX behavior). If a driver declares this property, then it's considered to be deprecated in favor of the specified one.
Adding top level `test_modules` directory (with the same structure as modules) for the sole purpose of module/plugin testing of YAML files, which will not be included with the driver build
Updating `driver-plugin-manifest` to look for the new `test_modules` directory in addition to `modules`, when loading the driver manifest
# Frontend changes
Calculate `supersededBy` and supersedes maps from the "superseded-by" property for each engine
Change the options for the engine field to use a function to dynamically show the legacy driver if allowed by rules (either the new driver is selected, or the legacy driver was already selected for an existing DB, or the driver is not superseded by anything)
Add new `DriverWarning` component to show these warnings based on supersede status
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
Remove regionid from plugin YAML file
Add new driver multimethod to `normalize db-details`, and implementing to handle this particular "migration" within Snowflake
Adding hook from database model post-select to invoke the new multi
Adding test
Update visualization_settings.cljc to handle the table.columns submap
Update dump and load to handle :visualization_settings on a Card (basically identically to how they are handled for a DashboardCard)
Updating test to have a card include :visualization_settings and ensure they fields are properly handled
Updating visualization_settings test case to incorporate table.columns as well as :show_mini_bar (under ::column_settings)
* Update Webpack to the latest version (5.37.0)
* Update `webpack-dev-server` to the latest version (3.11.2)
* Install `webpack-cli`
* Replace `CommonsChunkPlugin` with `splitCunks`
* Remove `NamedModulesPlugin`
* Remove `UnusedFilesWebpackPlugin`
* Replace `extract-text-webpack-plugin` with `mini-css-extract-plugin`
* Upgrade `html-webpack-plugin` to the latest version (5.3.1)
* Upgrade `html-webpack-harddisk-plugin` to the latest version (2.0.0)
* Remove `banner-webpack-plugin`
* Replace deprecated `[hash]` in the config
* Add `mode` to the config
* Upgrade `file-loader` to the latest version (6.2.0)
* Don't include polyfills for `path` and `crypto`
* Remove `uglifyjs-webpack-plugin` and use terser instead
* Workaround CLJS_GLOBAL needs top-level `process` object
* yarn add node-polyfill-webpack-plugin
* Use some polyfills
Apparently, the polyfill for crypto is still necessary since the front-end
code isn't migrated yet to the browser's native Crypto API
(https://www.w3.org/TR/WebCryptoAPI/)
See also https://www.npmjs.com/package/node-polyfill-webpack-plugin
* Admin smoke test: fix getting the empty DB path
* Onboarding setup test: fix getting the empty DB path
* Replace file-loader with type: "asset/resource"
See https://webpack.js.org/guides/asset-modules/
* upgrade react-markdown (#16934)
* Use large CI resource since webpack consumes a lot of memory [ci nocache]
https://circleci.com/docs/2.0/configuration-reference/#resourceclass
* Tidy up `NODE_ENV` references
* Remove Webpack `DefinePlugin`
Co-authored-by: Ariya Hidayat <ariya@metabase.com>
Co-authored-by: Dalton Johnson <daltojohnso@users.noreply.github.com>
Co-authored-by: Alexander Lesnenko <alxnddr@users.noreply.github.com>
* Add icons-only mode to SegmentedControl
* Add `fullWidth` mode to SegmentedControl
* Use SegmentedControl at SchedulePicker
* Use SegmentedControl at EmailAttachmentPicker
* Use SegmentedControl at ChartNestedSettingSeries
* Replace ChartSettingButtonGroup with SegmentedControl
* Remove ButtonGroup component
* Don't show labels for some viz settings
* Increase SegmentedControl's click area
* Add `inactiveColor` prop to SegmentedControl
* Fix icons-only SegmentedControl padding
This is for easier troubleshooting (rerun from failed, separated
recording) since those tests are quite hard to run smoothly with Circle
CI.
Also, move away from using the API (/api/setting/application-colors)
and just use the color picker from the UI to choose invididual custom color.
* offset is a reserved word in mariadb as of 10.6
- https://mariadb.com/kb/en/reserved-words/
- https://github.com/MariaDB/server/commit/299b935320
I think it is related to https://jira.mariadb.org/browse/MDEV-23908
"Implement SELECT ... OFFSET ... FETCH ..."
* Handle new names for utf8 in mariadb
https://mariadb.com/kb/en/unicode/
- utf8: Until MariaDB 10.5, this was a UTF-8 encoding using one to
three bytes per character. Basic Latin letters, numbers and
punctuation use one byte. European and Middle East letters mostly fit
into 2 bytes. Korean, Chinese, and Japanese ideographs use 3-bytes. No
supplementary characters are stored. From MariaDB 10.6, utf8 is an
alias for utf8mb3, but this can changed to ut8mb4 by changing the
default value of the old_mode system variable.
- utf8mb3: UTF-8 encoding using one to three bytes per
character. Basic Latin letters, numbers and punctuation use one
byte. European and Middle East letters mostly fit into 2
bytes. Korean, Chinese, and Japanese ideographs use 3-bytes. No
supplementary characters are stored. Until MariaDB 10.5, this was an
alias for utf8. From MariaDB 10.6, utf8 is by default an alias for
utf8mb3, but this can changed to ut8mb4 by changing the default value
of the old_mode system variable.
Update Oracle driver to set the v$session.program property on the connection, which appears in the PROGRAM column of the v$session table
Update tests accordingly
* Fix Card async-metadata running query even when metadata can be determined programatically
* Update another test
* Adjust affected repros by waiting for the metadata to load (#17068)
* Wait for the `result_metadata` to load
* Reduce the number of bins for the faster execution
* Update repro for #12568
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
* Extract repro for #15279 into a separate file
* Extract repro for #15689 into a separate file
* Extract repro for #15695 into a separate file
* Extract repro for #15119 into a separate file
* Extract repro for #12614 into a separate file
* Extract repro for #12720 into a separate file
* Extract repro for #6660 into a separate file
* Extract repro for #16103 into a separate file
* Extract repro for #13150 into a separate file
* Move existing tests related to the dashboard filters into a new folder
* Create new CI group `dashboard-filters`
This group already takes close to 10 minutes to finish.
We need to have an additional extended coverage for dashboard filters and for query builder filters.
It doesn't make sense to keep them all together under the `filters` CI group because that group on its own could take more than half an hour to finish.
* Add `createNativeQuestionAndDashboard` Cypress custom command
* Apply custom command to the existing test as an example
* Add sane defaults for the dashboard card size
* Apply custom command to 16181 reproduction
I thought (and assured dpsutton, lol) that the default behavior of the pagination spliced in was just not paginating, if there wasn't a limit passed in. Unfortunately, it appears that I tested this on the endpoint without pagination spliced in... so the default behavior isn't that way, it paginates if you don't pass in stuff.
So rip it out of the DB endpoint (only) for now and put it back in when we paginate
Authored-by: Howon Lee <hlee.howon@gmail.com>
* Add `addFilterToDashboard` Cypress custom command (#16964)
* Apply custom command to a few tests as an example
* #16918 Repro: Pulse fails when question has date field with "Month of Year" (#16921)
* Scroll to selected collection and question in the saved question picker
* fixup! Scroll to selected collection and question in the saved question picker
Co-authored-by: Alexander Lesnenko <alxnddr@users.noreply.github.com>
* Handle long collection item titles
* Fix EntityItemMenu's margin-left
* Remove `span` attribute from cols
* Fix "name" column is too close to "type"
* Fix "last edit" columns width
* Hide "last-edit" column at medium breakpoint
* Add test IDs for visualization buttons
* Add `runNativeQuery` helper to e2e-misc-helpers
* Add repro for #16914
* Don't sync native query columns if it failed
* Fix `table.columns: []` treated as "all hidden"
* Remove email quotations which could be deemed controversial. Address part of #11155
* Trigger deploy
Co-authored-by: Bruno Bergher <me@brunobergher.comexport>
The console messages (mostly about React warning on deprecated methods)
are useful when debugging a running application but they are atrocious,
extremely noisy, and barely useful when running the unit tests.
In the CI, those console messages drown the more useful PASS/FAIL status
of each individual test.
* Disable react/prop-types for .info files
* Refactor Checkbox
* Refactor StackedCheckBox
* Add a comment about Checkbox colors
* Remove not used prop
* Don't show outline for focused unchecked checkbox
* Pass custom Checkbox label element
* Refactor EmailAttachmentPicker to use label
* Refactor FieldsPicker to use label
* Refactor FilterOptions to use label
* Refactor DisplayOptionsPane to use label
* Refactor ScratchApp to use label
* Refactor AddSeriesModal to use label
* Add "danger" color to colors
* Refactor DeleteModalWithConfirm to use label
* Add label usage examples for CheckBox
* Add label usage examples for StackedCheckBox
* Add disabled state to CheckBox
* Add disabled state to StackedCheckBox
* Fix label prop-types
* Add space between examples
* Fix StackedCheckBox layout
* Add. StackedCheckBox label
* Fix disabled state for StackedCheckBox
* Migrate FieldsPicker to use label
* Migrate EmailAttachmentPicker to use label
* Add .styled files for checkbox components
* Fix e2e tests
* Fix `uncheckedColor` passed to HTML elements
* Revert eslint overwrite for .info files
* Display focus outline
* Fix home page our analytics section not showing collections
* increase limit for root collection children and pinned dashboards that are loaded on the home page
* fix Saved questions string translation
* fix Everything else folder is not showing items in the saved question picker for other languages
* translate query builder header
* Fix saved questions db on the client side
* i18n on "Saved Questions"
Co-authored-by: dan sutton <dan@dpsutton.com>
* adjust saved question picker layout
* fix saved question picker does not work with collection names that have a colon
* unskip repro
* make saved question picker usable on the mobile device
We defriendlified but the migration running in the first place is not happening for some users. Investigating but it's better just to be safe on this front
* Extract repro for #12228 into a separate file
* Extract repro for #15444 into a separate file
* Extract repro for #15460 into a separate file
* Extract repro for #15700 into a separate file
* Extract repro for #15981 into a separate file
* Extract repro for #16739 into a separate file
* Extract repro for #13961 into a separate file
* Extract repro for #14145 into a separate file
* Extract repro for #12581 into a separate file
* Extract repro for #14302 into a separate file
* Extract repro for #15163 into a separate file
* Extract repro for #11580 into a separate file
* Extract repro for #11480 into a separate file
* Extract repro for #9357 into a separate file
* Add basic segmented control component
* Specify segmented control's selected option color
* Update SegmentedControl's padding
* Use `PropTypes.node` instead of a bunch of types
* Remove `aria-selected` from CSS
* Extract SQL field filter helpers
* Apply field filter helper filters to the `sql-field-filter-string.cy.spec.js`
* Apply field filter helper filters to the `sql-field-filter-number.cy.spec.js`
* Apply field filter helper filters to the `sql-field-filter-date.cy.spec.js`
* hide other personal collections folder when no other users
* expand collections on click but disallow to select all personal collections, show all empty collections
* Move `browse` spec to the "Onboarding" directory
* Reorganize `search` specs
- Rename stray "seach" to "search-pagination" and move it to the new "search" directory under "onboarding"
- Rename existing "search" to "search-typeahead" and move it to the new "search" directory under "onboarding"
- Move old "seach" spec to the new "search" directory under "onboarding"
This happens because StaticParameterWidgetList is used an alternative to
SortableParameterWidget. The sorting handlers make sense for the latter
but not for the former.
* Add new Cypress helper function `openNativeEditor`
- exports the function
- documents the function using JSDoc
* Apply new helper function to the related tests
* Breadth first search on graph for chain filters
- abandons at max-depth and looks in breadth first fashion rather than
descending as far as possible on each possible path
* Add loop detection
not actually fooled by loops since we abandon after a certain path
length. But will remove some false starts before they have to be
killed by the path length check.
* Add seen optimization of nodes
We reach nodes in the shortest path possible, so keep a list of nodes
visited and don't revisit them: we got to them as fast as possible,
and once there, there's no new nodes that we can see from that vantage
point.
This allows a serious speedup and should hopefully fully resolve the
cpu issues seen by some.
* alignment
on java 16 there's a handy (newFilesystem path) but i don't think we
want to require java 16 yet. On previous versions we can use
(newFileSystem Path ClassLoader) which is what it was before. Really
just need to typehint the nil, but nils don't carry
metadata (sensibly) so would have to introduce a local binding
```clojure
(let [^ClassLoader cl nil]
(FileSystems/newFileSystem jar-path cl))
```
feels a bit weird, so just send the proper classloader through.
* Fix SQL saved question reference to Saved Questions with parameters
* Test fixes 🔧
* Don't run test for drivers that don't support FKs
* Test fix 🔧
* Test fix 🔧
When metrics get macro-expanded in xrays, they don't have joins in them. I stick joins in them. Joins! Joins! Joins!
But the underlying problem is that the metric macros have no idea what this "join" business is. Unhappy times!
The temporal units for the time-series footer were hardcoded. This is
now changed to stay consistent with the e.g. the query builder and the
summary sidebar, i.e. by getting it from the breakouts dimension itself.
* Standardize propTypes and defaultProps
* filter possible native question mapping options by parameter operator
* update cy test to ensure contains-->contains title drill works
* add test to ensure you cannot map a parameter to a field filter with differing types
* Add a timeout to the `saveSettings` function
* Update and unskip the test for #15974
* Put `Google` SSO test in its own `describe` block
* Update the test
* Only save successful-query-execution if it wasnt a cached result
the only thing this function does is update the rolling average query
execution time
* Failing tests for cache execution stats
These tests fail, but they shouldn't.
The save-execution information should not be called twice, and the
average execution duration should remain the same. What's happening is
that it is getting called twice just in the cache call, and only one
of those has the new key `:metrics/ignore-execution-time`. This
remains a mystery to me
* Don't call completion arity from step arity
the `(rf acc)` call violated the terms of transduction. If it does
need to be there, perhaps it could be used with reduced
wary of #12207 (which fixes#12165) and seeing if that crops up again.
- i had introduced :metrics/ignore-execution-time but this was because
i thought that we were running a simple query (limit 1) to get col
metadata. Now that i know we were just being a bit careless with `(rf
acc)` this extra information is no longer needed, and we can just
react to cached data.
* hoist closing paren
* wip for licenses
* Lots of changes
- bump clojure.data.xml to prevent illegal access
- had to change navigators because of this
- switch to full text of license
- algo is now:
1. check for a license file in jar
2. check for an override pointing to a license file locally in
resources
3. Check pom file for license information and just use that
can be invoked as:
```bash
% clj -X mb.licenses/process :classpath \"$(cd ../.. && lein classpath)\" :backfill "\"overrides.edn\"" :output-filename "\"backend-licenses.txt\""
License information written to backend-licenses.txt
/Users/dan/.m2/repository/net/jcip/jcip-annotations/1.0/jcip-annotations-1.0.jar : No license information found.
org.clojars.brenton:google-diff-match-patch : No license information found.
% wc backend-licenses.txt
33638 299118 2092948 backend-licenses.txt
```
* xml/parse without the strings
* Add tests
* remove comment form
* namespace comments
* readme
* instructions to create backend license files
* Add version information to the build script
need to either yank it out of the separate project or just use it as a
local root. not sure which
todos: move to s3 once i get access and figure out the proper bucket
copy to a place so it ends up in the jar
generate frontend versions as well
* Build frontend and backend license files
put them in the jar at
META-INF/leiningen/metabase-core/metabase-core/license-frontend-third-party
META-INF/leiningen/metabase-core/metabase-core/license-backend-third-party
need to be named ^license.* for lein uberjar to pick them up
* Rename and reformat
* remove old stuff
* combine with a newline
* Tests ensure all jars have an identifiable license
* Extra information if backfill is malformed or missing
* correct name for apache2_0 backfill
* print out classpaths in CI
* print
* see whole classpath. it's empty on ee for some reason
* assert they are there
* Get deps before attempting classpath
for some reason the first one fails. but subsequent re-runs work.
* loop on deps until satisfied
* Include message if cannot satisfy
* include all drivers when getting deps
* rearrange ns docstring
* Hoist functions on pom inside of `with-open` on the input streams
* Update to inversion of pom parsing
* Move to nio from java.io
* make map visualization sensible for more than 1 column of any type
* actually check if there is at least one metric
* actually check if there is one string in addition to metric
* Wait for the SQL query to load `result_metadata`
* Decrease the number of bins to 50
* Update the default bucket for time series
* Remove comment and the issue reference from "column popover" number test
* Unskip "column popover" longitude test
* Enable number binning test via custom question
* Enable longitude tests
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
* Add initial set of checks for longitude correctness
* Update longitude spec
* Update representative values for "1 degree" bucket size
* Rename function
* Extract x-axis ticks check into a separate function
* Expand and enable all tests
* Add initial set of checks for time series binning correctness
* Fix selected value for "Year"
* Rename the function
* Extract x-axis ticks check into a separate function
* Extract time series footer check into a separate function
* Expand tests
* Move *ignore-cached-results* to middleware
Had tried to adjust the binding to work, but the binding ended before
all of the bound functions and such were submitted to work, so when
that code actually executed the binding was no longer in
place. Annoying and it should be in middleware anyways
* Remove now unused import (cache became keyword in middleware not binding)
* skip some field logic when dealing with ExpressionDimensions
* hide xray drilldowns when there are expression dimensions
* fix tests by limiting filtering to just expression dimensions
* add a cy test
* update comment
* mild cleanup
* Add repro for #14652
* Fix blank screen when DB does not exist
DataSelector assumes `schemas` is an array.
But it's `undefined` if db does not exist
* Fix data model page state when DB does not exist
* Move prop-types to the top
* Fix invalid database request
Previous behavior searched only on columns backend name. This one searches on column frontend name (field name), table name, schema name - but not on all of them at once, just each of them separately
I thought the collections appearing in the list page problem was solved, but this turned out to be a bug with the ttest. Upon actual inspection the lists remained.
To fix this without changing all sorts of crap that depended upon collections api contract (we futzed with the output but the futzing of the output is transparent to the frontend because of the entity thing they got going on), needed a way to elicit null response from collections endpoint. shoved one in. then renamed it because the first name, "dummy", makes it all sound kind of jank, so 'no-model' for explicitly asking for no model without handing in an empty set, which turns into the set of all models.
This is an 80% unjankening. Full unjankening would entail whacking this special case.
This mimics the situation in the notebook (custom question). A minor
difference, hence the omission of the previous implementation, is that
notebook breakout selection only allows one table at a time (the other
tables are "folded" in the accordion list component). In the case of a
simple question, every breakout selection happens in a sidebar, a long
list of all tables and all dimensions. Thus, it needs to support
multiple selection at once, which this commit finally implements.
* Test can't share dashboard without cards
* Fix can't share dashboard with markdown cards only
* Fix public dashboard crash while loading
* Remove duplicated test
* Add `aria-disabled` prop to Triggerable
* Subscribe to dashboards only if it has data cards
* Add initial binning tests for saved SQL (via simple question)
* Add more binning tests for saved SQL (via custom question)
* Add more binning tests for saved SQL (via column popover)
* Simplify the query
* Update visualization for custom question, time series
* Update comments with the references to the created issues
* Unskip the repro
* Add `isPersonalCollectionChild` utility
* Hide permission control for personal collections
* Hide permissions modal for personal collections
* Handle missing `collection.effective_ancestors`
* Use `sidebar` cypress selector
* add units to new columns, add hover state for column headers
* fixup! add units to new columns, add hover state for column headers
* hide loader when filtering or sorting audit table
* fix tests
No tests caught this because of compounding errors on top of a generic
solution that already worked:
for h2, defined unix-timestamp->honeysql on [:h2 :millisecond] but
:millisecond is not a valid dispatch key, it is always
milliseconds (plural). And it was duplicated forgetting to use
:microseconds.
But this all worked because there are defaults in the
sql/query_processor
```clojure
(defmethod unix-timestamp->honeysql [:sql :milliseconds]
[driver _ expr]
(unix-timestamp->honeysql driver :seconds (hx// expr 1000)))
(defmethod unix-timestamp->honeysql [:sql :microseconds]
[driver _ expr]
(unix-timestamp->honeysql driver :seconds (hx// expr 1000000)))
```
so h2 was always using these fallbacks with the division:
```sql
"SELECT \"PUBLIC\".\"INCIDENTS\".\"ID\" AS \"ID\",
\"PUBLIC\".\"INCIDENTS\".\"SEVERITY\" AS \"SEVERITY\",
timestampadd('second', (\"PUBLIC\".\"INCIDENTS\".\"TIMESTAMP\"/1000000),--note division
timestamp '1970-01-01T00:00:00Z') AS \"TIMESTAMP\"
FROM \"PUBLIC\".\"INCIDENTS\" LIMIT 1048575"
```
rather than the h2 optimized form:
```sql
"SELECT \"PUBLIC\".\"INCIDENTS\".\"ID\" AS \"ID\",
\"PUBLIC\".\"INCIDENTS\".\"SEVERITY\" AS \"SEVERITY\",
timestampadd('microsecond', --note microseconds
\"PUBLIC\".\"INCIDENTS\".\"TIMESTAMP\",
timestamp '1970-01-01T00:00:00Z') AS
\"TIMESTAMP\"
FROM \"PUBLIC\".\"INCIDENTS\" LIMIT 1048575"
```
The end result is that the defmethods were never called and the
default was used. So a bug, but not one that was visible. Good catch
jeff
* add seriesIndex back to clicked object
* use transformed series when handled viz click
* unskip repro for related issue
* add additional test for this bug
* Extract largish sort-order function
also, running tests locally could run into timezone issues. only
interested in comparisons (<) so just make sure that its 24 hours in
the past so no issues with time zones
* tests are pretty literal but just satisfying the test checker
* bump coverage from 71 to hopefully 75%
* exclude parameters from isDirty check
* prevent calling methods on null originalQuestion
* rmv Save reliance in e2e test
* code refactor
* unskip repro
* rmv #520 workaround test because bug is fixed
* revert to previous sandbox.cy.spec.js
* skip workaround test
* add explanatory comment
* Don't compute only on non-zero scores
I was removing a helper function when refactoring the scoring. That
had a filter on :score?. I'm not sure why i translated this to only
interpret the denominator of scoring to only include scores that were
positive. This is obviously wrong.
Also, previously, all databases would be returned in the archived
query which seems obviously wrong. Databases cannot be archived and so
should never appear when searching for archived items.
* Add some tests
- had been asserting that ee-score = os-score if not in an official
collection. This is actually hard to do under the current scheme, and
actually not important. The important bit is that the relative
ordering is the same. It had been adding 0 score 2 weight previously,
and removing all scores with a score of 0. This created a bad way to
score, as something with 1/2, 0/3, 0/4, 0/10 (net score 1/2=0.5) would
appear higher than something with 1/2, 1/3, 1/4, 3/10 (net score
6/19=0.315) and it really shouldn't. The former really should
be (1/19=0.05). And since we are adding in a denominator of
2 (official collection weight) all of the scores are actually smaller,
but that's ok, as they are all proportionately smaller.
* Optionaly keep prev list value in EntityListLoader
* Use prop-types in EntityListLoader
* Fix UI flash when sorting collection items
* Fix empty state displayed before items are fetched
* Alias condition statement
* Unify shouldUpdatePrevList condition
* Fix react/display-name
* Rename keepPreviousList —> keepListWhileLoading
* Swap list and previous list internally
* Replace `_.isEqual` check with plain `!==`
* Add sorting params to Search entity
* Remove pinned items sorting by `collection_position`
* Add default sorting to collection items
* Add sorting controls
* Add test IDs to table head and columns
* Move pagination tests into separate file
* Make pagination tests independent from dataset
* Allow creating pinned items via API in Cypress
* Add sorting E2E tests
* Fix sorting by "Last edited at" column
* Add sorting by "Last edited by" column
* Fix BaseItemsTable unit test
* Don't wrap ItemDragSource children with <div />
react-dnd allows wrapping only native DOM elements with DND helpers.
With the new table layout, we now need to drag table rows (td elements).
Rendering a td with ItemDragSource results in invalid HTML as table body's direct children should only include td elements.
* Rename `selected` prop to `selectedItems`
* Pass ItemDragSource props to table item
* Replace styled TableRow with plain <tr />
Table row is wrapped with ItemDragSource, that only accepts native DOM elements as its children
So styled-components can't be used here
* Make table row draggable
* Add headless mode to BaseItemsTable
* Add prop hiding table row's bottom border
* Fix typo and split large comment
* Render full table row when dragging
* Add empty PinnedItemsTable drop area
* Add empty ItemsTable drop area
* Wrap PinnedItemsTable with drop area
* Wrap ItemsTable with drop area
* Extract `<tr />` style
Co-authored-by: Gustavo Saiani <gustavo@poe.ma>
* Fix width style
Co-authored-by: Gustavo Saiani <gustavo@poe.ma>
* Run prettier
* Reference issue about post-pulses work
Co-authored-by: Gustavo Saiani <gustavo@poe.ma>
* Add LastEditInfoLabel component
* Fix Header component's import order
* Display last edit info in Dashboard header
* Display last edit info in Question header
* Show "edited by you" if user is the last editor
* Add prop types for Header component
* Add prop-types for ViewHeader component
* Cleanup from dump-load-entities-test (#16281)
Updating dump-load-entities-test test so that dump dir is actually deleted at the end
* Revamp Cypress custom commands (#16292)
* Extract UI custom commands
* Extract user related custom commands
* Extract overwrites
* Extract permissions related custom commands
* Extract API custom commands
* Extract helpers custom commands
* Extract database related custom commands
* Rename `helpers` group to `visibility`
* Rename custom command `isInViewport` to `isRenderedWithinViewport`
* Unskip repro for #15119 (#16297)
* Create composite Cypress custom command `cy.createQuestionAndDashboard` (#16294)
* Add composite Cypress custom command `createQuestionAndDashboard`
* Refactor the existing test to use new Cypress custom command
* Refactor repro for #13062 using the new Cypress custom command
* Use the `describe` block instead of using `forEach` for complex repro
* Add space between test cases
* #12629 Repro: Human-readable number formatting not working properly (#16231)
* Fix `nosql` user's email address (#16306)
* Generate correct SQL for columns from joins inside other joins or source queries (#16254)
* Fix joining a join inside a nested queries (#12928)
* Code cleanup & dox
* Tests for #13649
* Test fixes 🔧
* Remove unit tests snapshots (#16321)
* Remove snapshots from `Calendar.unit.spec.js`
* Remove snapshots from `DashCard.unit.spec.js`
* Remove snapshots generated for "internal" components
* Remove unused import
* Remove snapshot related command from `scripts`
* Remove `noSnapshotTest` from internal components
* Remove `react-test-renderer`
* filter nulls out of histograms (#16345)
* #15993 Repro: Click behavior with filter pass-thru does not show filters defined on question (#16342)
* #16334 Repro: Click Behavior targeting a question with filter mapped causes the visualization type to change (#16343)
* Add repro for #16334
* Expand test assertion
* #16327 Repro: Double binning menu for date fields when using Saved Question (Native) (#16359)
* #16322 Repro: "Custom mapping" is only available, when "A list of all values" is set (and after a browser refresh) (#16361)
* #16226 Repro: LDAP/Email settings gets cleared if validation fails (#16364)
* Add repro for #16226
* Extract email settings into a separate file
* Add repro for #16226 (Email)
* switch to column settings when sidebar is already open (#16368)
* align labels on non-bars in combo charts (#16369)
* #16378 Repro: Cannot enable JWT authentication (#16384)
* Fix JS error on JWT settings form (#16394)
* fix default updateSettings function in SettingsBatchForm
* unskip repro
* #12128 Repro: Can't change label on empty row value (#16426)
* Post-merge fix
* Post-merge fix 2
I didn't solve this merge conflict properly in the original merge.
Co-authored-by: Jeff Evans <jeff303@users.noreply.github.com>
Co-authored-by: Cam Saul <1455846+camsaul@users.noreply.github.com>
Co-authored-by: Paul Rosenzweig <paulrosenzweig@users.noreply.github.com>
Co-authored-by: Noah Moss <32746338+noahmoss@users.noreply.github.com>
* Display last edit info in Question and Dashboard headers (#16354)
* Add LastEditInfoLabel component
* Fix Header component's import order
* Display last edit info in Dashboard header
* Display last edit info in Question header
* Show "edited by you" if user is the last editor
* Add prop types for Header component
* Add prop-types for ViewHeader component
* Switch the collection page to table-based layout, display last edit info (#16357)
* Extract color functions from EntityItem body
* Refactor EntityItem not to use `switch` statement
* Extract `<EntityItem.Icon />` component
* Extract `<EntityItem.Name />` component
* Extract `<EntityItem.Menu />` component
* Add basic table components for collection items
* Update item name style
* Update `last-edit-info` columns text style
* Make CollectionContent wider
* Highlight pinned items
* Add `renderItem` prop to BaseItemsTable
* Add links to table items' entity pages
* Connect `data-metabase-event` to table items
* Make table items selectable for bulk actions
* Remove filtering E2E test
Collection items filtering is going to be replaced with sorting
* Fix cypress collection item selectors
* Fix cypress pinned items assertions
* Add collection item actions menu
* Fix "Type" column style
* Fix layout when there are no pinned items
* Add CollectionContent's empty state
* Fix typo
* Update table row height
* Add table rows separators
* Remove replaced components
* Remove filtering logic from CollectionContent
* Fix pulses display in collection items table
* Remove note about filtering E2E tests
Collection items filtering is going to be replaced with sorting
* Move BaseTableItem into its own file
* Remove redundant conditional render
* Fix col's span attribute
* Don't use `useCallback` for itemRenderer
* Extract ItemsTable's renderItem
* Extract PinnedItemsTable's renderItem
* Fix `EntityItemIcon` margin prop usage
* Remove not used props from `EntityItem`
* Fix string prop notation
* Add IconButtonWrapper component
* Use IconButtonWrapper for EntityItem's icon
* Update BaseTableItem's EntityIcon usage
* Move EntityItemWrapper to .styled file
* Rename `EntityItemIcon` to `EntityIconCheckBox`
* Sort by last_edited_by (#16388)
for last edits, now can sort by last_edited_at (previously
last_edited) and last_edited_by
* Remove "Pinned items" text assertion
The category title is removed in this PR by design
* Add drag-n-drop to collection items tables (#16360)
* Don't wrap ItemDragSource children with <div />
react-dnd allows wrapping only native DOM elements with DND helpers.
With the new table layout, we now need to drag table rows (td elements).
Rendering a td with ItemDragSource results in invalid HTML as table body's direct children should only include td elements.
* Rename `selected` prop to `selectedItems`
* Pass ItemDragSource props to table item
* Replace styled TableRow with plain <tr />
Table row is wrapped with ItemDragSource, that only accepts native DOM elements as its children
So styled-components can't be used here
* Make table row draggable
* Add headless mode to BaseItemsTable
* Add prop hiding table row's bottom border
* Fix typo and split large comment
* Render full table row when dragging
* Add empty PinnedItemsTable drop area
* Add empty ItemsTable drop area
* Wrap PinnedItemsTable with drop area
* Wrap ItemsTable with drop area
* Extract `<tr />` style
Co-authored-by: Gustavo Saiani <gustavo@poe.ma>
* Fix width style
Co-authored-by: Gustavo Saiani <gustavo@poe.ma>
* Run prettier
* Reference issue about post-pulses work
Co-authored-by: Gustavo Saiani <gustavo@poe.ma>
* Fix last edit query (#16520)
* Correct query for last edits
* tests ensure we get latest revisions
* Add tooltip displaying the exact time of the last collection item edit (#16492)
* Add sorting to collection items (#16385)
* Add sorting params to Search entity
* Remove pinned items sorting by `collection_position`
* Add default sorting to collection items
* Add sorting controls
* Add test IDs to table head and columns
* Move pagination tests into separate file
* Make pagination tests independent from dataset
* Allow creating pinned items via API in Cypress
* Add sorting E2E tests
* Fix sorting by "Last edited at" column
* Add sorting by "Last edited by" column
* Fix BaseItemsTable unit test
* Order by _revision_ id, not _user_ id (#16536)
* Fix UI flash when sorting collection items (#16511)
* Optionaly keep prev list value in EntityListLoader
* Use prop-types in EntityListLoader
* Fix UI flash when sorting collection items
* Fix empty state displayed before items are fetched
* Alias condition statement
* Unify shouldUpdatePrevList condition
* Fix react/display-name
* Rename keepPreviousList —> keepListWhileLoading
* Swap list and previous list internally
* Replace `_.isEqual` check with plain `!==`
* Fix border radius for circle icon
* Rename Header's `showBadge` into `hasBadge`
* Add tests for collections metadata (#16538)
Co-authored-by: dpsutton <dan@dpsutton.com>
Co-authored-by: Gustavo Saiani <gustavo@poe.ma>
If you put in fooBarID, previous behavior on `simple` was to change to Foobarid. New behavior is to keep the caps on non-initial letters, turning it into FooBarID. So camelCaps turns into PascalCaps. ALLCAPS is Titlecapsed, tho.
Friendly names bugfix 1 was a noop because of a typo in the WHERE clause of the DB migration. This one has been manually tested to go and sync on all 4 supported MB db's.
* Fix#16299 MongoDB empty results when filtering against array
* Ok, we still need to use $expr if the VALUE in the expression isn't simple
* More test fixes 🔧
* More fixes 🔧
Repro:
db setup:
```sql
create table wut2 (t bytea, ts text);
insert into wut2 (t, ts) values ('20210421164300'::bytea, '20210421164300');
```
Mark t as :Coercion/YYYYMMDDHHMMSSBytes->Temporal, ts as
:Coercion/YYYYMMDDHHMMSSString->Temporal
Do a simple query, and then aggregate count by the two dates. ts
should succeed and t fails.
We are passing in the UTF8 as a parameter rather than just emitting a
literal.
Before:
```clojure
{:sql
"-- Metabase\nSELECT CAST(to_timestamp(convert_from(\"public\".\"wut\".\"t\", ?), 'YYYYMMDDHH24MISS') AS date) AS \"t\", count(*) AS \"count\" FROM \"public\".\"wut\" GROUP BY CAST(to_timestamp(convert_from(\"public\".\"wut\".\"t\", ?), 'YYYYMMDDHH24MISS') AS date) ORDER BY CAST(to_timestamp(convert_from(\"public\".\"wut\".\"t\", ?), 'YYYYMMDDHH24MISS') AS date) ASC",
:params ("UTF8" "UTF8" "UTF8"),
:type :invalid-query}
```
After:
```sql
SELECT CAST(to_timestamp(convert_from(\"public\".\"wut\".\"t\", 'UTF8'), 'YYYYMMDDHH24MISS') AS date) AS \"t\",
count(*) AS \"count\" FROM \"public\".\"wut\"
GROUP BY CAST(to_timestamp(convert_from(\"public\".\"wut\".\"t\", 'UTF8'), 'YYYYMMDDHH24MISS') AS date)
ORDER BY CAST(to_timestamp(convert_from(\"public\".\"wut\".\"t\", 'UTF8'), 'YYYYMMDDHH24MISS') AS date) ASC
```
Note there's a lot of extra casting that perhaps we can clean up
* adapt ldap test helpers to be able to test AD memberOf attribute
* fix ldap search to handle memberOf attributes with a single group
* use one-or-many instead of if statement
* fix tests
Previously just used JDBC. However, JDBC will merrily execute queries on empty file locations for SQLite, because it'll create a new SQLite file. This one checks that there exists a SQLite file already to be querying in the first place
Folks wanted DB pagination in #11806 and apparently others but it didn't make it to notion. Well, it made it to my attention, anyhow, and here is some DB pagination, but just for the backend.
* adjustments to CSP header in dev environment to make react dev tools work on Firefox
* add test for script-src directive
* fix position of doc string
* fix error in common_test
Peeps want to make questions faster, but we don't give the data about query execution in a very user-friendly way. Here's it in a user-friendly way.
Co-authored-by: Alexander Lesnenko <alxnddr@users.noreply.github.com>
* Add the last-edit-info to the child query
just for cards right now, need to extend to dashboards and then make
sure we clean it up into the proper map.
Have a set of all columns that are required and ensure those are in
each query. Nulls need to be cast to the correct type if they aren't a
text column (at least on postgres, need to investigate h2 and mysql).
* Correct find last-edit-info for cards and dashboards
* Move from group-by max id to self join
* Paginate and search on collection items
- include the metadata information in the query
- to accomplish this, make a backstop to add all columns into each
query
- need types to union the null. in postgres it assumes all nulls are
text. Haven't checked what mysql and h2 do here yet
- sort_column: name, last_edited, model
- sort_direction: asc, desc
* Add default sort column and direction
* handle mysql
mysql doesn't need types on nulls to unify them
* Correct batch post processing for cards and dashboards
* Correct order of expectations
they were alphabetical inside of models
previous order:
- acme products | pulse
- electro-magnetic pulse | pulse
- birthday card | card
- dine & Dashboard | dashboard
and is now ordered solely by name
* Tests and sort on lower name after timestamp
When sorting by last edited, lots of things don't have this, so don't
want arbitrary sorting, but sort by name afterwards
* Ensure nulls go to the end of the list
* Tests
* trying to make h2, mysql, and pg happy
sorting timestamps and always having nulls last is quite difficult
* Make diff bigger
this was going in as a string and losing the timezone information, so
was actually advancing time.
Now: (java.time.OffsetDateTime/now)
"2021-06-01T13:59:33.165483Z"
Minus 2 hours: (.minusHours (java.time.OffsetDateTime/now) 2)
"2021-06-01T11:59:27.528500Z"
As seen in the code:
;; unaltered "now"
"2021-06-01T08:55:28"
;; altered "two hours ago"
"2021-06-01T11:55:29" <- UTC shenanigans. so just bump back 24 hours
* Sort by model ranking
* Remove clj-time and use ZonedDateTime instead of OffsetDateTime
* Warn users about sandboxing and public sharing
* Update docs/enterprise-guide/data-sandboxes.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Add type on collection
* Search with collection type
* Cleanup bounded heap accumulator
* Make search conditionally bump official collection items
* collection api and tests
* Put collection type onto hydrated cards on dashboards
* Move official collection type scoring into ee codebase
* ensure ee and oss agree when not official collection
* Mark Collections tree with type
* Remove unnecessary `and`s when no check for enhancements
* Tests for setting collection tree type
* Include hydration for collection type on dashboards
* Make sure created test collections are cleaned up
* Cleanup tests, don't search on collection_type
looks for all text fields and searches the term against that. official
would bring back all official types. no bueno
* Docstring on protocol and don't shadow comparator
* update to new ee impl var passing style
* Collection model ensures no type change on personal collection
* Check for personal collection when updating type
model checks for personal collection and rejects in the update but
doesn't check for children of personal collection.
* Update checking for unchangeable properties of a personal collection
* Cleanup: type collection tree, batched hydration, combine error checks
* Cleanup
* move bounded-heap accumulator to utils
* switch to test.check testing
* Bad test just to see what our CI will report
* remove purposeful failing test (was for CI)
* collection.type -> collection.authority_level
* Test the actual ee scoring impl, not the wrapped one
locally i had enhancements enabled so the tests were passing, but in
CI it was failing as it did not have enhancements enabled
* Add composite Cypress custom command `createQuestionAndDashboard`
* Refactor the existing test to use new Cypress custom command
* Refactor repro for #13062 using the new Cypress custom command
* Use the `describe` block instead of using `forEach` for complex repro
* Add space between test cases
Serialization refactoring and fixes
*********************************************************
* Support loading linked cards in different collections *
*********************************************************
Architectural changes:
Lay foundation for more generic retry mechanism
Establish a keyword to represent a failed name->id lookup that can happen during load, and map that to information about which part of the card resolution failed (for logging purposes)
Similar to dashboards, add a separate fn for performing a load of cards
Invoking the load-cards fn from the load multimethod implementation when some cards failed name resolution
Test enhancements:
Add new multimethod to perform more specific assertions on the loaded entities
For cards - check that the native SQL of the loaded card matches that of the original (pre-dump) instance
For dashboards - check that the linked card for each dashboard card series matches
Bug fixes:
Apply the mbql-id->fully-qualified-name fn on serialize to filter, breakout, and aggregation clauses
Updating "My Test" card to include filter clause so that fix is exercised
Instead of upserting users with a "changeme" password, simply do not attempt to change the password
Adding :engine to the identity condition of Database (so unrelated test-data instances don't get trashed on load)
Cleanup:
Remove unused specs namespace
****************************************
* Support dashboard card click actions *
****************************************
Architectural changes:
Add visualization settings shared model namespace with utility functions
Adding tests for viz settings
Changes to serialization:
Update visualization_settings in dump to convert column_settings keys from field ID to fully qualified name, and update the click action target with the fully qualified name
Also converting targetId in top level click action to fully qualified name
Doing the reverse in load, using the same utility code
Changes to serialization tests:
Updating the existing dashcard in the "world" roundtrip test to include the click action
Adding assertion for the click action in the assert-loaded-entity implementation for Dashboard
******************************************************************
* Stop nesting cards and check for unresolved query dependencies *
******************************************************************
In names.clj, stop qualifying a card's name by its dataset query's source card ID (this is what caused the "nested" directory structure)
In load.clj, start checking that either the source query [:dataset_query :query] is resolved properly as a card ID or table ID, and if neither, then propagate an ::unresolved-names entry up into the card (causing it to be reloaded on second pass)
Fix logging of unresolved name info
Test changes:
Add two new cards to the "world", one that lives in the root collection but depends on a card within a collection for its query, and one opposite (i.e. lives in a collection but depends on a root card for its query)
Adding new check that collection names match after loading
************************
* Logging improvements *
************************
Include causes when logging errors to aid with diagnosis
Add BEGIN and END messages for both dump and load commands
Add dynamic var to suppress printing of name lookup error on first pass load
*************************************************
* Handle collection namespaces and retry pulses *
*************************************************
Architectural changes:
Considering a collection namespace to be part of its fully qualified name (putting in a subdirectory called :namespace when writing to disk)
Accounting for that change when dumping and loading, and upsert (i.e. namespace is part of unique columns)
Bumping serialization format version to 2 because of this
Changes to load:
Add load-pulses fn that works similarly to others, to facilitate reloading
Add similar logic as other entities to reload a pulse if the :card_id is not found
Model changes:
Add hack fn to compare namespace values for the "namespace does not change" assertion with appropriate comment
Test changes:
Added test for pulse with reload
***********************************************
* Handle dimension entry within template-tags *
***********************************************
Add :field to the recognized clauses under mbql-entity-reference? (since that is how field refs are now represented in MBQL)
Add temp-field helper fn in test_util.clj to reduce some of the "world" verbosity
Adding new test card to world that uses template-tags with a field reference under dimension
************************
* Handle textbox cards *
************************
Add support for passing textbox (virtual) cards through visualization_settings.cljc
Updating load test to add a textbox type card to the dashboard and ensuring it is persisted
***************************************
* Handle dimension parameter mappings *
***************************************
In visualization_settings.cljc:
Adding handling for parameterMapping
Adding new roundtrip test for parameter mappings
Some refactoring to split up larger fns
In serialize.clj, convert all parts of the map that contain the dimension key to use fully qualified names for the respective fields
In load.clj, do the opposite (convert back to new IDs)
Updating test to add parameter mapping to an existing click action, and adding new assertions for that to ensure it is preserved
Fixing serialize test now that the "ids" can themselves be maps (i.e within dimension vector)
In visualization_settings_test.cljc
Adding new roundtrip test for problem dashboard
***********************************
* Handle field literals correctly *
***********************************
On load, only attempt to translate field name to ID if it's actually a fully qualified field name
Adding new fn to names.clj to provide that logic
Adding new test card that uses such a field reference
**************************************
* Accept only-db-ids option for dump *
**************************************
Accept options map for the dump command. Currently, the only supported option is `:only-db-ids`, which is expected to be a set of DB IDs to dump. If set, then only these databases (and their respective tables, fields, and segments) will be dumped to the dump-dir. If not set, then the previous behavior takes effect (all DBs).
Update the load_test.clj to set this to be only the new temporary DB to avoid tramping on existing `sample-dataset` and other instances
*****************************************************
* Don't include personal collections of other users *
*****************************************************
Use the `select-collections` fn to filter only collections that are public, or owned bythe given user (by way of email), plus all collections with those as an ancestor
Updating test to assert a non-owned personal collection is not persisted (including nested), but all nested personal collections are
Adding new wrapped macro to also bind collection/*allow-deleting-personal-collections* true around the with-temp* call so that personal collections can legitimately be deleted (since they're temporary); the name of this macro was chosen to preserve the same amount of spacing within `with-world`
*********************************
* Support native query snippets *
*********************************
Adding support for NativeQuerySnippet:
- in names
- in dump (putting into subdirectory of collection, as cards)
- in upsert (identity condition of :name :collection_id)
- in load (no retry fn necessary since they should not have any dependencies themselves)
Adding a snippet and a card referencing it to roundtrip test (including a deeply nested snippet)
Fixing up handling of snippet namespaced collections and adding relevant name tests
*********************************
* Handle source-table for joins *
*********************************
Adding `mbql-fully-qualified-names->ids*` for recursive calls, which does not normalize its args, keeping mbql-fully-qualified-names->ids` as a wrapper to that
Adding clause to `mbql-fully-qualified-names->ids*` to resolve `:source-table` when it's a fully qualified table name
Adding util fn to names.clj to support that scenario
Updating "My Card" test card to include a join
Skipping two more drivers from `dump-load-entities-test` - SQLite and Spark because they don't support FKs
***************************
* Cleanup and refactoring *
***************************
Responding to PR feedback
Updating upsert_test.clj to check that the identity-condition keys actually exist for the models
Removing legacy MBQL syntax in a couple places
Adding ordering by `:id` to serialization load command for predictability
Lots of fixes and refactorings in visualization_settings.cljc:
- added lots of specs and fdefs
- added test fixture that instruments all the speced fns and required it from both visualization_settings.cljc and load_test.clj
- renamed a bunch of fns for more consistency
- added and fixed some docstrings
- fix implementation of `parse-json-string` for cljs
*************************************************
* Preserve temporal-unit in source-field clause *
*************************************************
Update serialize and match macros to keep the rest of the match, when modifying the source field ID
Adding a bunch of new tables in order to perform a "bucketing on implicit join" type query
Updating test_util.clj to initialize the query processor with all the new temp tables so that queries actually work
**************************
* Misc cleanup and fixes *
**************************
Fixing the implementation of `fully-qualified-name->context` to work properly when the entity name itself is also a model name
***************************
* Handle joining to cards *
***************************
Recognize :source-table pointing to a fully qualified card name in `mbql-fully-qualified-names->ids*`, and adding the unresolved name capture to that lookup
Adding new util fns to capture the unresolved named entries from anywhere within the MBQL query tree: `paths-to-key-in` and `gather-all-unresolved-names`
Updating `resolve-card-dataset-query` to call this new `gather-all-unresolved-names` to pull all unresolved names from the MBQL query tree into card itself
Renaming `fully-qualified-name->id-rec` to `resolve-source-query` for clarity, and removing now unneeded clause
To the test case, adding a card that joins to another card
* Basic `react-gird-layout` usage for Dashboards (#16167)
* Install react-grid-layout
* Add basic NewGridLayout implementation
* Wrap DashboardGrid with WidthProvider
* Bind DashboardGrid's onDrag methods
* Coerce layout item IDs to string
* Add method rendering NewDashboardGrid
* Refactor `onLayoutChange` method to handle new grid
* Use react-grid-layout's CSS classes
* Fix lag while dragging grid items
* Switch back to ExplicitSize HOC
* Fix grid in editing state is too short
* Import missing `react-grid-layout` CSS
* Fix RGL overwrites dragging placeholder color
* Hide default RGL's resize handle
* Fix random dashboard layout rearrangements
Before dashboard cards' parameters (x, y, width, height)
were updated individually for each card
It worked fine because there was only one card moving at a time
Now when cards are automatically moving when a user drags another card,
a race conditon appears, cards' parameters get overwritten, and layout breaks
Fixed by using bulk updates for dashboard cards drag and resizing events
* Fix dashboard background grid
* Align RGL with parent component edges
* Fix grid offset
* Fix dashboard grid height
* Fix no automatic page scroll while dragging
* Fix navbar is stuck when scrolling dashboard
* Remove previous grid component
* Fix grid doesn't respect item min size
* Extract grid background into a separate utils file
* Fix dashboards layout issues (#16236)
* Fix RGL cards are higher than the previous ones
* Return the fixed header in editing mode back
* Add support for responsive dashboard layouts (#16213)
* Fix `gridItemWidth` calculation
* Memoize grid children
* Add utility to adapt layout for breakpoints
* Use responsive react-grid-layout for dashboards
* Don't overwrite main layout with smaller ones
* Pass correct `isMobile` prop to DashCard component
* Extract `renderGridItem` method
* Increase dashboard cards vertical margin for mobile
* Remove dashboard's old renderMobile method
* Add isMobileBreakpoint utility function
* Disable dashboard drag and resize for mobile
* Remove old method disabling mobile drag and resize
* Fix grid background rendering
* Switch to desktop and mobile breakpoints
* Update card heights for dashboard's mobile view
* Add a comment about `!important` in CSS
* Minor code style fix
* Remove not used functions
* Fix#15998: Tooltip only shows first Y-axis value when X-axis is numeric and style is Ordinal/Histogram
* Fix#15998: Tooltip only shows first Y-axis value when X-axis is numeric and style is Ordinal/Histogram
* prettier
Co-authored-by: Roman Abdulmanov <roman.abdulmanov@gmail.com>
When attempting to convert a filter, check first if the filter is a
comparison against a constant. Otherwise, preserve the filter is in its
custom expression (MBQL) form.
Each in the following list can be converted into a standard filter
(since the simple UI for the standard filter only allows literal
comparison):
[Total] < 300
between([Discount], 5, 15)
[Name] = "Joe"
but none of these should be converted
[Discount] <= [Tax]
between([Rating], 0, [MaxRating])
[Reviewer] != UPPER([Name])
* Allow standard filter to be invalid
This is necessary to allow an incomplete filter, i.e. the user hasn't
filled Y in "X < Y" form yet (hence Y is undefined). The filter is not
valid but it is still considered in the standard form.
* Check whether string comparison can be considered a standard filter
* Idiomatic with array destructuring
When attempting to convert a filter, check first if the filter is a
comparison against a constant. Otherwise, preserve the filter is in its
custom expression (MBQL) form.
Each in the following list can be converted into a standard filter
(since the simple UI for the standard filter only allows literal
comparison):
[Total] < 300
between([Discount], 5, 15)
[Name] = "Joe"
but none of these should be converted
[Discount] <= [Tax]
between([Rating], 0, [MaxRating])
[Reviewer] != UPPER([Name])
* Allow standard filter to be invalid
This is necessary to allow an incomplete filter, i.e. the user hasn't
filled Y in "X < Y" form yet (hence Y is undefined). The filter is not
valid but it is still considered in the standard form.
* Check whether string comparison can be considered a standard filter
* Idiomatic with array destructuring
* Add repros for #16198
* Refine the assertion in the second scenario
Guard against the changes in #16174.
* Add repro for the third scenario
* Use full filter name for non-CE expressions
Co-authored-by: flamber <1447303+flamber@users.noreply.github.com>
* Add yyyymmddhhss coercions to type system
* Implementations for h2/mysql/postgres for yyyymmddhhss bytes and strings
* Mongo and oracle
* Adding checksum on migration it said was bad
* import OffsetDateTime
* Redshift and bigquery
* snowflake expectations
* sql server yyyymmddhhmmss. have to format the string then parse
sqlserver lacks a parse function that takes a format string, they just
take an integer that specifies a predefined format string. So we have
to make the string into the right format then parse.
* presto yyyymmddhhmmss
* Test byte conversions
* Remove errant `mt/set-test-drivers!`
* Remove sqlite, change keyword for multiple native types in def
the spec couldn't handle different shapes under the same keyword. so
just use :natives {:postgres "BYTEA"} :native "BYTEA"
* Make schema work with different shape maps
* hx/raw "'foo'" -> hx/literal "foo" and remove checksums
* _coercion_strategy -> _coercion-strategy
* Handle coercion hierarchy for :Coercion/YYYYMMDDHHMMSSBytes->Temporal
* Parse number filter inputs
The frontend feels a bit off on this. It's sending `"string/="`
instead of `"number/="` but the backend code knows that number filters
can ignore it. It's also sending strings from the text inputs when it
should be sending number types.
We've had a convention that the types come across as they should be:
numbers come in as numbers and strings as strings. This seems to break
that convention.
* Unskip the "number" part of the repro for #15981
* use normalized parameter values when diffing for dirtiness
* update tag parameter parsing fns to handle number --> number/= types
* tweak & unskip cypress repro test
* wrap fn in parens to invoke it?
Co-authored-by: Nemanja <31325167+nemanjaglumac@users.noreply.github.com>
Co-authored-by: Dalton Johnson <daltojohnso@users.noreply.github.com>
* Don't case in the outer select when inner select will
Expressions create subselects
```clojure
(-> {:database 1,
:query {:source-table 2
:expressions {:test ["*" 1 1]}
:fields [[:field 2 nil]
[:expression "test"]]}
:type "query"
:parameters []}
qp/process-query
(update-in [:data :rows] count)
pprint)
```
```sql
SELECT "source"."QUANTITY" AS "QUANTITY",
"source"."test" AS "test"
FROM (SELECT timestampadd('second', "PUBLIC"."ORDERS"."QUANTITY", timestamp '1970-01-01T00:00:00Z')
AS "QUANTITY",
(1 * 1) AS "test"
FROM "PUBLIC"."ORDERS") "source"
LIMIT 1048575
```
The bug was that source quantity was marked as a coerced integer to
timestamp, and it coerced in both the inner and outer select.
There's of course a few ways to solve this:
- the easiest way is when fetching the field information, throw away
the coercion strategy. Didn't want to do this as less information gets
confusing. Preferred to add information about _why_ it should be
ignored rather than just suppressing information.
- (solution here) add a marker to just not coerce on the outer
select.
The metadata is a bit strange to me. I'm honestly not sure what it
should be. What is the base type, effective type, and coercion
strategies of the things returned? At the moment they are:
```clojure
{:semantic_type :type/Quantity,
:coercion_strategy :Coercion/UNIXSeconds->DateTime,
:name "QUANTITY",
:field_ref [:field 2 nil],
:effective_type :type/Instant
:base_type :type/DateTime}
```
Should this still have coercion information and a base type of
integer (ie, promote the inner field type's information to the
toplevel) or is this properly the type information now?
* tests
* Add bigquery into test
mongo doesn't support expressions, druid doesn't have a great table
laying around
* dumb oracle is dumb
its probably the only "proper" way to do this but i'm still mad
* clean up tests
* Check for string and number in tests
This ballooned in scope by a bit.
1. FE landed onto this. So both BE and FE
2. Added an offset-pagination middleware
3. Refactored collection in addition to paginating it so it works by a giant UNION ALL
4. Refactored effective collections to work by a moderately complicated query with very many WHERE clauses to execute the effective visibility, instead of doing it in clojureland
* respect `remember` field on backend if session tokens are not enforced via env variable
* modify docs to describe new behavior
* fix indentation
* use get-in to access remember field of request
* add a session api test
* expose session-cookies as a public setting
* hide remember me checkbox if session-cookies is set
* Add async validation of password in account settings, and change schema error message
* run prettier
* add period to error message for consistency
* revert change to api-documentation
* Remove helper type functions from the global scope
We are using `_typeUsingPlaceholder` and `_typeUsingGet` only in `frontend/test/metabase/scenarios/question/custom_column.cy.spec.js`.
There is no point in exporting them and keeping them available in the global scope.
* Create `enterCustomColumnDetails()` helper function
* Use new helper function in repro for 13241
* Refactor existing test with the new helper function
This test is interesting and a good litmus for this new helper function.
It clearly shows that issues with `contentEditable` field were truly fixed in #16157.
Formula `[Quantity] * [Product.Price]` resolves in the UI as `[Quantity] * [Product → Price]` and the test doesn't break, which means race conditions were solved.
* Use new helper function in repro for 12762
* Refactor repro for 13289
* Refactor repro for 12649
* Refactor repro for 15741
* Use helper function in ExpressionEditorTextfield describe block
Confirms #16157 is working as expected.
* Erase `_typeUsingPlaceholder` and `_typeUsingGet` functions
* Refactor repro for 13122
* Refactor repro for 13217
* Refactor repro for 15922
We made fieldname friendliness, people didn't like it, we tried to fix it, people still didn't like it, now we're just making the fieldnames all mildly unfriendly again. Not like, snake_case unfriendly, but Pascal Case With Spaces unfriendly
* Add basic AddDatabaseHelpCard component
* Display database engine on HelpCard
* Display help link for cloud instances
* Display link to database setup docs
* Dispaly help card at `/admin/database/create`
* Test help card changes copy on new db page
* Display db setup help card on Setup
* Display help card only when db engine is selected
* Fix not translated string
* Use styled-components for AddDatabaseHelpCard
* Coercion strategy hierarchies
* Don't rebuild hierarchy on every change
* Emacs config tweaks
* clojurescript-mode is derived from clojure-mode, combine the dir locals
* Remove log statements and improve dir locals
* Use new api from app
- add coercions for text (iso stuff)
- function to get the effective type for a strategy
- extra validation in saving field to ensure that the triple
`[coercion base effective]` is valid
* remove unnecessary `coercions` map
Co-authored-by: dan sutton <dan@dpsutton.com>
* Fix Druid queries with multiple filters if some but not all are temporal
* Fix Druid queries with temporal Fields that don't have a :temporal-unit
* Test fix 🔧
* Fix syntax errors
* Query result metadata should return the :unit from the source query when applicable
* Test fix 🔧
* Clarify comment in test
* More test fixes 🔧
* Test fix 🔧
* Port legacy data type migrations -> Liquibase
* Fix migration IDs
* Field type validation/error handling
* Have semantic type fallback to nil
* Fix semantic-type-migrations-test
* Fix migrations
* Revert accidental changes
* Semantic/* & Relation/* ancestor types
* Fix stray Relation/PK and Relation/FKs
* Semantic/* and Relation/* ancestor types
* cljs test fix
* Fix :require
* FE test fixes 🔧
* Test fixes 🔧
* prettier
* PR f e e d b a c k
* Use medium size CircleCI image for Presto to prevent all the OOMs
* Backport dir-locals tweaks from hierarchy PR
* Redshift: only sync the test schema (faster CI and fix failure)
* Better error handling for sync in tests
* Revert accidental commit
* Redshift test fixes 🔧
* Install slugify
* Add utils to slugify and parse entity URLs
* Add `<Entity.Link />` component
Basically just a `<Entity.Name />`,
but wrapped inside `<Link />`
* Add slugs to `/browse/:databaseId` URLs
* Use constant for "Saved Questions" database ID
* Fix "Saved Questions" not displayed in breadcrumbs
* Fix opening browse page for "Saved Questions"
* Add slugs to `/collection/:id` URLs
* Fix personal collection page slug
* Remove `describe.only`
* Remove unnecessary type coercion
* Fix typo
* Fix collections not highlighted in the sidebar
* Add slug to collection `/archive` URL
* Add slug to collection `/edit` URL
* Add slug to collection `/permissions` URL
* Use slug in `getInitialCollectionId` selector
* Explicitly pass root ID to `Urls.collection`
* Pass prop to EntityLink's Link component
* Fix DashboardMoveToast collection link
* Allow overwriting EntityLink's Link component
* Fix links in QueryBuilder's search
* Fix collection link in search results
* Fix navigating to collection after question archived
* Fix navigating to collection from pulse edit
* Slugify users' personal collection URLs
* Test slugified collection URLs are opened correctly
* Add slugs to `/dashboard/:id` URLs
* Fix dashboard URLs at admin public sharing page
* Fix opening dashboards from main page
* Fix navigation to saved automatic dashboard
* Fix dashboard moving
* Fix dashboard history
* Fix closing dashboard editing modal
* Fix dashboard details form is empty
Reproduced if opening the form via URL
(`/dashboard/:id/details`)
Works correctly if the details modal is opened from a dropdown menu
* Fix dashboard copy form crash
Reproduced if opening the form via URL (`/dashboard/:id/copy`)
* Fix copying dashboard
* Fix closing dashboard copy modal
* Fix navigation once dashboard is created
* Replace hardcoded "root" id with a constant
* Assert pathnames instead of URLs
* Fix duplicate modal not closing automatically
* Fix URLs on Activity page
* Fix navigating to dashboard question is added to
* Fix archiving dashboard
* `dashboardId` —> `slug`
* Add slugs to `/question/:id` URLs
* Fix links to questions on Activity page
* Switch questions to slugified URLs
* Remove not used import
* Update pathname assertion
* Fix collection URL builder fails receiving `null`
* Ensure question URL builder returns valid URL
* Update pathname assertions
* Update pathname assertions
* Update pathname assertions
* Update pathname assertion
* Update pathname assertion
* Add question URL builder unit tests
* Fix collection URL builder
* Fix `null` collection ID when building URL
* Fix repro test issue reference
* Move URLs tests into onboarding folder
* Enable #15256 repro test
* Fix slugs for edge cases
* Delete obsolete Safari favicon
* Set the default favicon path to the existing icon
* Delete fallback png favicons
* Fix placeholder in related Cypress test
log4j includes different class files for different jvm versions. This
means our uberjar includes these versions so we are a multi-release
jar. Need to set this in our manifest.
Fixes the reflection warning seen on startup on jdk 11's
~/p/w/metabase ❯❯❯ java -jar target/uberjar/metabase.jar
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
not-empty so that we don't output a dumb string like
> null DB null connections: null/null (null threads blocked)
Also put the call to `name` behind a some-> so there's no
NPE. Everything else will just output a null if its missing
* fix question level link display settings
* add cypress spec for the question-level link settings on the table visualization
* Address review comments: add a spec for clicn behavior with view_as = link, fix admin column settings
* revert a straightforward fix for hiding irrelevant settings
* fixup! Address review comments: add a spec for clicn behavior with view_as = link, fix admin column settings
* prettier
* handle link_text null value
* Dashboard param value (chain filter) endpoints should return cached FieldValues
* Test fix 🔧
* Test fixes 🔧
* PR feedback/appease linters
* Hydrate :table to avoid lots of DB calls
* Use hash of current user perms set for sandboxed field values cache key
* Fix missing :require
* Test fix 🔧
* Add Azure Guide
* Change links to access the guide
* Add Azure AD SAML SSO guide
* Fix error in pages
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* A couple of changes proposed by Jeff
* Update docs/operations-guide/running-metabase-on-azure.md
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Wrap connection strings in triple backticks
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
- Cherry picks 274be949d5 from `master`, which extracted unit and timezone tests from CircleCi and made them run using GitHub actions
- Synchronizes the "state" between the `master` and `release-x.39.x` branches
Co-authored-by: Ariya Hidayat <ariya@metabase.com>
* add global search results pagination
* fixup! add global search results pagination
* nit fix to allow aleksander test without borking perf too much
* fix incorrect question object in the search spec
* translate pagination
* show filters only for existing types of items for a specified search query
* fixup! show filters only for existing types of items for a specified search query
* fixup! show filters only for existing types of items for a specified search query
* Search pagination models (#15972)
* querying the models by a buncha queries
* up max filtered response because of limits
* archived string and table db id is still a thing cuz of weirdo joins and stuff
* shove models given in the normal api endpoint
* misplaced docstring
* redshift root is actually bonafide different than other roots
Co-authored-by: howon lee <hlee.howon@gmail.com>
* Port legacy data type migrations -> Liquibase
* Fix migration IDs
* Field type validation/error handling
* Have semantic type fallback to nil
* Fix semantic-type-migrations-test
* Fix migrations
* Revert accidental changes
* Add deny list of common passwords
* replace filtered pw list with full list
* switch to using a FileReader to check password line-by-line
* bypass deny list if complexity is set to easy
* fix tests
* more test fixes and remove unneeded ns
* fix typos
* fix reflection warning
* needed to import File
* fix final tests, and update docs to reflect change
* address initial review comments
* fix build after removing import statement
* address more review comments
* use InputStreamReader to read password file in JARs
* Emit events outside of db/transaction
When emitting events (particularly revision events), the event handler
loads the item from the db to ensure that all post-processing is
done. If done from inside a transaction, there's a race between the
event handler looking up the object from the db and the transaction
completing to put the item in the db. When the event handler wins, the
following code fails:
```clojure
(push-revision! :entity Dashboard,
:id id,
:object (Dashboard id), ;; this is nil
:user-id user-id,
:is-creation? (= :dashboard-create topic)
:message revision-message)
```
and then the schema check for `push-revision!` fails `(map? object)`
and there is no event recorded for creation. On edit, it might get the
pre-edit version and record an incorrect message or not create the
revision for the latest change, preventing going back in the future.
I'm only doing this in dashboard but these changes should be audited
and applied everywhere
* Remove `:dashboard-card-update` and `:dashboard-card-create` events
These events were introduced 5 years ago and never have had an event
handler. No worries about race conditions when emitting them from a
transaction if nothing ever cared in the first place.
* Omit dashboard revision history items if no description
* Skip Revert button on topmost visible revision history entry
* Test that topmost revision history entry omits Revert button
* Lint
* Use diff as criterion for omitting revision entry
* Detail comment in shouldRenderRevisionHistory
* Render revision item if either before or after in diff is populated
* Update e2e test
* Use a more durable string to find element in test
* Make it work with MBQL instead
* Update frontend/test/metabase/lib/expressions/typeinferencer.unit.spec.js
Co-authored-by: flamber <1447303+flamber@users.noreply.github.com>
Add diagnostic info for connection pools to Help page
Backend changes:
Add org.clojure/java.jmx dependency to project.clj for use in JMX bean querying
To troubleshooting.clj, add some functions to capture details from the c3p0 connection pool beans to a map
Add new API endpoint to invoke that from util.clj
Add content-disposition header to API endpoint to make it a download
Frontend changes:
In the Help page, add a new paragraph below the existing "Diagnostic Info" JSON blob for advanced details, envisioned as a list of download links (which users can then attach to tickets, etc.)
I don't exactly follow what's going on in #15901. But it seems that
the field information can be kept in the db
> ""field_ref"":[""field"",""count(*)"",{""base-type"":""type/BigInteger""}]
And so we need to handle this base type here. Note that in the
function that calls `update-filter-for-field-type` it checks that the
field has a base_type.
I'm also a bit confused by the base_type vs base-type in the above
snippet. But again the calling code has
```clojure
(and (= param-type :dimension)
(get-in value [:field :base_type]))
(update-filter-for-field-type value)
```
So it seems the field has a base_type rather than a base-type.
* Move unhidden table sync off-thread
* Test bulk hide/unhide syncing
* Check can connect to db and serialize syncs
rather than allowing us to create threads willy nilly, serialize them
and reuse them in a fixed thread pool with a single thread. This helps
if someone "multiple clicks" and launches a few table syncs. It's not
appropriate to use the existing deduplicating stuff in the sync which
prevents later tasks from happening while the current sync step
happens since we want to sync with the most recent changes. So just
serialize them to go one at a time rather than off to the races.
* Clean up ns, make private, and type hints
* Block waiting on table sync to finish
* Use driver.util can connect helper
uses the timeout of 5seconds and catches errors
* disable ToggleHiddenButton during request
Co-authored-by: Dalton Johnson <daltojohnso@users.noreply.github.com>
* Test adding question to dashboard
* Test collections filtering when adding a question
When adding a question to dashboard,
we need to display collections a user has "write" access to.
Collection with "read" access have to be hidden
* Fix adding question to dashboard without access
* Add a note about permissions test suite
* Move question permission tests to collection suite
* Revert initial collections filtering
* Fix adding question to dashboard without access
* Remove redundant state field
* Enable #15281 issue repro test
* Remove requireCollectionWritePermission prop
* Filter items user doesn't have `write` access to
* Fix permission tests
* Fix dashboard test
* Fix part of permission tests disabled for nodata user
* Bring back issue reference to Cypress test
* Remove underscore prefixes for component methods
* Test offers saving dashboard to opened collection
* Fix tests nested incorrectly
* Split dashboard permission test
* Fix suggest saving items to read-only collections
* Fix collection permission filtering
See comment:
https://github.com/metabase/metabase/pull/15613#discussion_r614460504
* Move comment
* Fix test failing due to fixed collection suggestion
* Remove `should("exist")` from Cypress tests
* Merge "adding question to dashboard" tests
* Merge similar permission tests
* Merge similar tests into one
* Use sidebar test ID in permissions test
* Select by .AdminSelect
* Remove redundant search test case
* Revert native query test
* Add React list key to ItemPicker items
* Add collection suggestions tests
* Configure Form's `overwriteOnInitialValuesChange`
* Wrap CollectionSelect with `@Collection.loadList`
When suggesting an initial collection,
we need to check a user has `write` access to it.
For that, collection objects have to be present in Redux store,
so we can retrieve a collection by ID and check the `can_write` flag
* Allow modifying CreateDashboardModal's onSave prop
* Fix collection suggestsions for new dashboard
* Fix collection suggestions when copying dashboards
* Add defaultProps to Form
* Fix SaveQuestionModal unit test
* Fix SaveQuestionModal collection suggestion
* Simplify CollectionSelect wrapper
* Fix dashboard header test selector
* Rename permission tests
* Mock HTTP requests at SaveQuestionModal test
* Pass correct params to initialCollectionId
* Druid Docker image [WIP]
* More Druid Docker stuff [WIP]
* Test fixes 🔧
* Remove maxRowsQueuedForOrdering from query, don't think we need it
* Force test Druid host -> localhost for now
* Update sync tests for updated test data
* Revert unused changes
* Allow multiple `--spec` arguments in Cypress `run` command (#15917)
* Split e2e CI groups (#15924)
* Add option to choose between source-folder and test-files in CCI
* Rearrange Cypress files
* Split e2e workflows
* Split native queries into a separate CI group
* Add option to choose between source-folder and test-files in CCI
* Rearrange Cypress files
* Split e2e workflows
* Split native queries into a separate CI group
* Provide `:last-edit-info` information on collection items
only provided for cards and dashboards. At the moment, collections can
have:
- cards (☑️)
- dashboards (☑️)
- snippets (❌)(not in UI)
- pulses (❌)(deprecated)
- other collection (💪 to be done. needs a table for these
changes. they are revisioned)
* Send edit info along with cards
* Last-edit-info on dashboards
* Move last-edit-info functions into a bespoke namespace under revision
Under revision as the source of changes all come from revisions at the
moment. This might be ill-considered once we add collections into the
mix since they are not good candidates for revisions.
* Renames and docstring update on the last-edit namespace
* Add types
* Add `:last-edit-info` on card and dashboard creation
* Expect last-edit-info in creation repsonse for cards/dashboard
* Docstring on last-edit types
* Make the namespace checker happy
* namespace checker
* Remove moved function
* Add edit info to api/card/ and api/dashboard/
* Move event emission outside of transaction
* Change 400 errors to 401s on failed authentication
* remove TODO comment
* return 401 for expired JWT in enterprise code
* keep existing exception message when JWT is expired
* fix test failures
* remove comment about timeout on CI that's no longer relevant
* Improve Cypress runner command line config (#15854)
* Rename Cypress CI groups to `e2e-tests` (#15877)
* Remove the `node` parameter from Cypress CI job matrix (#15878)
* Merge EE Cypress tests with the OSS ones (#15887)
* Reorganize test folder: Move e2e support files into a separate folder (#15899)
* Refine CI checksum calculation - extract e2e files (#15900)
Add new namespace to manage diagnostic info about the DW DB connection pools
Adding new atom/fn/macro to wrap the capture of diagnostic info, modeled after the call counting functionality of toucan.db
Capturing diagnostic info from a new `datasource-with-diagnostic-info!` fn in the execute namespace, and calling that from the various drivers (instead of `datasource`)
Adding to log message generated for the "stats" portion of the line logged via middleware to output this information
Use the lexical determination of enclosing function, which makes it more
robust agains syntax error or incompleteness. Also, increase the test
coverage.
* Fix testing the operator
* Offer the help text upon a complete field reference (with some tests)
* Merge all enterprise specs together with the OSS ones
* Move whitelabel under `/admin/settings`
* Move specs under `/admin/audit` folder
* Merge `admin/database/add` specs
* Group snippets
* Merge OSS and EE sso specs
* Move sandboxes under `admin/permissions`
* Delete custom drill-throughs spec
* Return boolean value
* Use `Cypress.env` instead of global vars
* Delete empty spec
* Update paths for `eslint` script
* limit and offset work upon hand exercise
* limit added
* add the test
* docstring is wrong make it not wrong
* fix namespace decl
* they were test bugs only
* stick that limit and offset in there
* Permissions imports
* make it work in dbs that are not just h2
* add limit test get
* needs integer
* juice the coverity lol
* just stick querying in the overall dealie
* tests pass
* add counter
* rip out the clauses
* not just if limited
* docstring misplaced
* more linting
* cool tests
* tests pass but lacking test lol
* group id works
* test works
* fix jeff nits
* break that api
* add test for count
* cypress works for some reason lol
* fix capitalizations
* admin people pagination frontend (#15821)
* admin people pagination frontend
* admin people pagination and search specs
* address review comments
* fix rebase
* address review comments
Co-authored-by: Alexander Lesnenko <alxnddr@users.noreply.github.com>
This is carried out via lexical replacement, i.e. finding the token to
be replaced, instead of the rather-brittle and often-incorrect regex for
prefix/postfix.
* Always place the caret after the replacement text
* Only accept suggested "(" when there is no forthcoming "(" already
Make the load multimethod return a function, which serves as a "reload function", to be called after the first entire pass is complete (via cmd)
Updating various places to combine these functions together and invoke at the top level from cmd.clj when load finishes
Updating dashboard load implementation to check for presence of cards referred to by its dashcards, and if missing, reload those dashboards(s) on a second pass
Splitting out a separate load-dashboards fn that contains the bulk of the body from the previous multimethod implementation (which now delegates to it) to support easier reload functionality
* Fix sidebar loading causes UI jump
* Fix sidebar blinks betweed data refreshes
* Replace `...` with `…` character
* Test collection is open immediately after moving
* Update sidebar children cypress selector
* Add comment about sidebar's loadingAndErrorWrapper
* Fix typo
* Add role props to sidebar and its children
It's terrifying to see lots of error messages when the expression is
invalid or not completed yet. To overcome this, we display only one, the
highest-priority and most friendly one. The rank is:
* tokenization error, e.g. mismatched parentheses
* type-checking error, e.g. incorrect count of function arguments
* syntax error (from the parser)
* Fix coding style issue
* Robust against non-Array compileError
* Update .ebextensions to send memory usage to Cloudwatch and also install inspector agent to actively monitor threats. Added full Cloudwatch agent with metrics config, simplified config file. Fix#9021
* Adding a few items to documentation
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
* Add a few more changes to finish this PR
Co-authored-by: Jeff Bruemmer <jeff@metabase.com>
Set c3p0 dataSourceName property to include Metabase specific info
Change data-warehouse-connection-pool-properties multimethod to include database as 2nd param
Setting the dataSourceName c3p0 property to have the format db-<N>-<D>-<DB> where <N> is the DW database ID, <D> is the driver name, and <DB> is the database name from the db details
Adding test
Setting c3p0 dataSourceName for the app DB pool as well
* Revert "Fix tooltip value on unaggregated data (#15472)"
This reverts commit 057b6bbc35.
* Fix wrong tooltip on charts with unaggregated data
* unskip repro for #11907
* temporary set resize throttling to 0 when the app is open in Cypress to reduce flakiness
* Table collection_revision -> collection_permission_graph_revision
in anticipation of getting collection revisions in, need the table
name as the previous table named this was tracking changes to the
permission graph
* rename collection permission graph revision alias
* Remove unnecessary or for precondition
* Rename sequence in sql
liquibase has a renameSequence changeset but it doesn't work for h2
obviously. However, it doesn't allow this constraint to be honored
with a precondition. Just having this changeset on h2 blows up
* move dashboard card action buttons in a popover
* add more top margin for actions menu when editing dashboard, fix tests
* Fix spec broken due to changed height of dashboard cards
* Return "Duplicate" action for read-only dashboards
* Remove findEllipsisMenuFor test utility
* Test can copy dashboard in read-only collection
* Remove test for wrong behavior
* Fix dashboard edit action visibility
Hiding "Change title and description" from users with read-only access to dashboard
* Fix dashboard archive action visibility
Hiding "Archive" action from users with read-only permission to dashboard
* Enable #13229 repro test
* Test user with read permissions can't revert question history
* Fix user with read-only permissions sees revert buttons
* Enable test reproducing #14705
* Test can deselect all items using bulk actions
* Fix bulk select button in collections footer
* Group bulk select tests, use data-testid
* Group bulk actions tests into one
* Add SMTP server for testing
* Try running maildev in the default executor for Cypress
* Add first email-based Cypress test
* Try different ports
* Stop using ENVs for maildev
* Use camelCase for `const`
Co-authored-by: Luis Paolini <paoliniluis@gmail.com>
* Enable test reproducing #14705
* Test can deselect all items using bulk actions
* Fix bulk select button in collections footer
* Group bulk select tests, use data-testid
* Group bulk actions tests into one
* Watch for munge collisions
* Always have to check for munge collisions
was thinking only in the case where munging made a difference did we
have to check, but if the first one defined is foo? -> foo as the
munge, defining foo later will cause a collision
* Prettier fix
* revert popover to before react 16 changes
* move tether/sizeToFit logic to render callback
* set ParameterFieldWidget free
* add debounce to maxheight futzing
* do popover height calc BEFORE updating tether when attached above target
* do height calcs for both top/bottom before tether stuff
* ignore tetherOptions prop
* add string parameter to dashboards instead of replacing location/category
* merge category and string sections
* make location options strings when flag flipped
* update strings in cy tests
* filter location fields when using text/category parameter
* Microcopy tweaks
* update more cy strings
* replace location switch default with isLocation
Co-authored-by: Maz Ameli <maz@metabase.com>
Updated the updateSectionsWithPlugins fn to sort the sections again after reducing by PLUGIN_ADMIN_SETTINGS_UPDATES, because those update functions might change the order of keys
Adding explicit order property for the items in SECTIONS to facilitate that re-sorting
Fix expected property type for SettingsEditorApp sections (which is truly an object, not an array), and fixed the null case accordingly
Incorporate Cypress test written by Nemanja
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
Scrolling gets disabled for the modal and then is supposed to be reenabled when modal is closed. However, SandboxedPortal eats the events that would tell the modal to go and do this. Do it on React lifecycle bits instead.
Does not make it so window scrolling will mutate position of the second popover. However, you will be able to see the actual contents of the second popover.
* Don't add temporal subdimensions to a FieldIDDimension
This restores the behavior just like before MBQL field refactoring (back
when FieldIDDimension still existed).
* Adjust expectation in another test
* Remove type/UNIX* and type/ISO8601* from frontend
* Set effective-type and coercion strategy when syncing
these values will most likely only be available when using tests, as
metadata is very unlikely to have this. As far as I can tell the
toucannery apparatus is the only bit that has this. Its quite
artificial. I'm not sure if this is a good idea.
* :type/UNIXTimestampSeconds and type/ISO8601DateTimeString out of type
remove the coercions from the type hierarchy. This brought up a
strange issue that has been present for a while: all liquidbase
migrations run and then all data migrations run. They are not
interleaved. This allows for the following scenario:
- data migration migrates all X values to Y in version 26
- liquibase migration migrates all Y values to Z in version 28
But these are not run in version order, but all liquibase migrations
are run and then all data migrations are run. If you were on an old
version for a while, you could end up with Y values even though the
liquibase migration which comes "after" the data migration turns all Y
values into Z values.
This impacts this change because a data migration in this way:
- liquibase changesets 287, 288, 289, and 290 remove all 'type/UNIX*'
and 'type/ISO8601*' semantic types. These were in 0.39
- data migration `migrate-field-types` added in 0.20.0 was looking for
special_type's of "timestamp_milliseconds" and migrating them to
"type/UNIXTimestampMilliseconds".
Since the liquibase runs before the migrate-field-types, it's possible
for a 'type/UNIX*' semantic type to reappear. And since these were
removed from the type system it would fail validation and blow up. In
this case it actually can't happen since the field its attempting to
migrate (special_type) no longer exists. But since the migrations are
not interleaved this problem still exists.
* whatever prettier
* Appease the machines
* Restore the unix and iso semantic types to hierarchy
migration tests still use these
`(impl/test-migrations [283 296] [migrate!] ...)`
A few things in tension: the field requires valid semantic_types,
these _were_ valid semantic_types until v39.
* Remove old "coercion" semantic types
* Migrate the v20 version semantic types for unix epoch
* Time travelling migrations
these target v20 and v29 which at the time they existed had a column
special type not semantic type. But these run after all of the schema
migrations, not interleaved, so they will have a semantic_type not
special_type when they actually run even though they target v20 or v29
data. strange world
* add migration's new checksum
* Fix Duplicate action visible with read permission
For `/collection/root` page
* Fix Duplicate action visible with read permission
For `/dashboard/:id` page
* Move dashboard permission test to collection suite
Popover movement is with reference to #15502 but also with reference to #15277 and #15534 and #15502 but not #15445, except for the mutation movement portion of #15445 which is not a whole fix.
Confusingly, the overlap portion of #15502 is almost a completely separate bug but a fix is in here anyways.
* Backend feature flag for new field filters
* Feature flag new parameter options
When the "field-filter-operators-enabled?" flag is disabled we do the following:
1. Replace new operator options with old category and location/city, etc., options
in the PARAMETER_OPTIONS list found in metabase/meta/Parameter.js
2. Hide numbers section in the PARAMETER_SECTIONS list found in
metabase/meta/Dashboard.js
3. Return args as-is in the mapUIParameterToQueryParameter function
found in metabase/meta/Parameter.js
React/UI code handles both old options and new options so doesn't need
to change. Old parameter types like "category" and "location/city" are
treated like "string/=" in the UI but retain their own parameter type
when used to send a new query.
* Fix FE issues caused by meta/Parameter refactor
* mock the field operator param flag to make tests pass
* add/fix cypress tests
* fix import in ParametersPopover
* update widget tag type
* Enable field filter operators for cypress tests
* Question marks are questionable
* Conditionally use category or string/= if field filters are enabled
* rmv mocks where we don't need them
* rmv mock from chained-filters test
* env vars as string in project.clj, alignment
Co-authored-by: dan sutton <dan@dpsutton.com>
* Various improvements to version update checks
***************************
* Release project changes *
***************************
Updating version_info.clj code to build version-info-ee.json file for Enterprise Edition build
Adding test for both oss and ee version info file generation logic
Decoupling GitHub milestone from version and setting it to be the OSS value for :ee release build
Adding nREPL alias for release deps.edn project and usage note
************************
* Core product changes *
************************
Adding new :ee version of :mb-version-info-url (:mb-version-info-ee-url) with default value to match the file name generated by the release code
Making enterprise-edition? public from core.clj, since it's now needed from the upgrade check code to switch between the URLs
Adding new setting to store when updates were last checked (since it's on a 12-hour interval)
Changing update check code to both set this new timestamp setting (always) to the current time, and set version-info itself to nil if the update check fails (so that users won't see "latest and greatest" if the update check succeeded at some point in the past, but then started failing
Frontend changes:
Showing the "last checked" timestamp (in user's browser timezone) when we show the "Sorry" text because of being unable to check for updates, so they know when the last check was made
Removing custom implementation of MetabaseSettings.versionInfo from EE frontend code since it's no longer needed (and never worked), because the EE backend will now be hitting a different URL with the version info specifically for EE releases
Linking to https://www.metabase.com/docs/<newVersion>/operations-guide/upgrading-metabase.html URL instead of simply /start
* Quarantine flaky test in `auditing.cy.spec.js` (#15501)
* Quarantine possible offenders behind CI timeouts (#15504)
* Quarantine possible offenders behind CI timeouts
* Fix brush date filter test (#15505)
* Fix flakiness in `scatter` visualization tests (#15520)
- Fixes a flake in scatter.cy.spec.js that started appearing after throttle was introduced to the ExplicitSize component in #15235
- Also fixes a flake with brush filter that apparently wasn't fully fixed in the previous attempt
## Additional Info:
- Stress testing this fix in isolation with GitHub actions shows 20/20 runs passing ✅
- Stress testing "brush date filter" revealed that it still fails approximately 4-5x out of 20!
- The reason seems to be that `mouseup` event doesn't always happen at the same time so the resulting filter doesn't have the same ending month every time (for example I've seen: "Created At between May, 2016 September, 2016", "Created At between May, 2016 July, 2016" and even "Created At between May, 2016 May, 2016")
- It is more than enough that we assert that this filter exists in the first place so I changed the assertion to: `cy.contains(/^Created At between May, 2016/);` since the beginning of this string is the only thing we care about
NSLog(@"\n\n!!!!! Task terminated with exit code %d !!!!!\n\n",task.terminationStatus);
dispatch_async(dispatch_get_main_queue(),^{
if([[NSAlertalertWithMessageText:@"Fatal Error"
defaultButton:@"Restart"
alternateButton:@"Quit"
otherButton:nil
informativeTextWithFormat:@"The Metabase server terminated unexpectedly.\n\nMessages:\n%@",[weakSelf.lastMessagescomponentsJoinedByString:@""]]//componentsshouldalreadyhavenewlineatend
[[NSAlertalertWithMessageText:@"Saved"defaultButton:@"Done"alternateButton:nilotherButton:nilinformativeTextWithFormat:@"Your data has been saved."]runModal];
[[NSAlertalertWithMessageText:@"Link Copied"defaultButton:@"Ok"alternateButton:nilotherButton:nilinformativeTextWithFormat:@"A link to this page has been copied to your clipboard."]runModal];
<textFieldCell key="cell" enabled="NO" refusesFirstResponder="YES" allowsUndo="NO" sendsActionOnEndEditing="YES" alignment="center" title="Enter the email address you used to create your Metabase account on this computer and we'll get started changing your password." id="Lgy-GU-YOj">
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.