40 Commits

Author SHA1 Message Date
Nicola Mometto
02a1dc223b Restrict notification markdown link schemes and diff pulse recipients by email (#80597)
* Restrict markdown link URI schemes in rendered notifications

Only http/https/mailto links and images survive markdown rendering for
email/Slack notifications; other schemes and unparseable URIs are
dropped instead of being passed through (or, for malformed URIs when a
site URL is set, throwing mid-render).

* Diff pulse recipients by id or email in the subscription-permission check

Key the added-recipients diff the same way pulse-channel keys
recipients (user recipients by :id, external ones by :email) so
external-address changes are covered by the check too.

* Allow inline base64 images in notification markdown

The app's markdown renderer accepts data:image/...;base64 URIs
(DATA_IMAGE_URI_PATTERN); let notification rendering accept the same
pattern so text-card images look the same in email.

* Cross-link DATA_IMAGE_URI_PATTERN to its backend counterpart

* Share the data-image URI pattern between backend and frontend

Lift the pattern into metabase.util.markdown.image (cljc), compile it
into the frontend bundle via shadow-cljs, and re-export it as
DATA_IMAGE_URI_PATTERN so both renderers accept the same inline images
by construction.

* Apply resolve-uri to Slack images too

* Render a dropped link-ref as its text alone instead of restoring brackets
2026-08-26 12:09:26 +00:00
Aleksandr Lesnenko
4897452a08 Metrics Explorer (#68961)
Co-authored-by: Edward Paget <ed.paget@gmail.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Nick Fitzpatrick <nick@metabase.com>
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
Co-authored-by: Kevin Kelleher <kevin.kelleher@metabase.com>
Co-authored-by: Maz Ameli <maz@metabase.com>
2026-03-09 20:18:15 -04:00
Nicola Mometto
09ee4178ca Transforms inspector (#68675)
* wip: transform inspector BE

* WIP: inspect tables

* wip

* format join info as code

* wip: join info

* style

* add join stats type and display match rate

* chore: row

* chore: remove limit

* wip

* handle differerent cases for match rate and row count

* add joins table display and icons

* fix copy

* fix missing join table names

* cleanup

* cleanup

* extend match

* fix missing source table type

* add join tooltip

* chore: with join info

* wip

* wip

* add field statistics dummy component

* small ui adjustments

* small ui adjustments

* add filled rows

* order input columns like joins

* update visualization card styling

* wip: update join calc to be iterative

* add target and source stats

* add target and source stats, a filter by output fields and a tab for unmapped fields

* add target and source stats, a filter by output fields and a tab for unmapped fields

* chore: sort by order

* fix: inverse map

* add cross product warning

* add null join warning

* small QoL UI improvements

* wip: render output_row_count

* wip: native joins

* fix a bug with conditional rendering

* wip: try to pull out useful fields

* WIP: claude'd UI selection of visited_fields

* use default automatic display selection logic

* add transform mock page with mock summary section

* add mock api console log

* fix fetch wrong query metadata

* add mock lenses

* add mock lenses api

* REVERTME: ignore for deploy

* feat: schemas

* require

* feat: degeneracy check

* WIP cljc api

* wip context

* wip lenses

* update comments

* chore: visited-fields

* wip

* wip: base

* wip: triggered lens

* wip: lib

* distinguish drill lenses

* fix: return visited-fields

* wip: v2 api

* WIP: vibe'd FE

* tabs/remove mock

* account for params

* params

* ui improvements

* fix

* show fingerprinting

* fingerprint layout

* use BE fallback

* handle more complex joins

* fallback to interesting fields

* fix trigger via cljc

* remove debug

* different text

* compute-card-results in cljs

* condition trigger in cljc

* semi working

* fix card_id correlation

* remove dead code

* rename BE to just inspector

* switch to new

* ensure transform is deleted

* chore: cleanup schema

* bring back the field fingerprint info

* move cljs out of lib

* fix: schema

* minor cleanup

* appease kondo

* kondo

* remove dead code

* feat: centralize native analysis in query_analisis

* chore: remove #p

* clean up

* Gdgt 1663 move fingerprints out of the lens to data inspect (#68996)

* [transforms inspector] depend on trunk macaw (#69022)

depend on trunk macaw

* fix lint and type issues (#69017)

* remove unused lib function

* fix infinite rendering loop

* fix type lib.interestingFields

* fix

* fix: ui bug

* chore: disable visited fields for now

* chore: better unmatched-rows

* chore: exclude not nulls

* minor tweaks

* minor tweaks

* rename

* chore: tighter contract

* minor optimisation

* Gdgt 1669 make cards clickable (#69116)

* Add card link

* link to hash of question

* add clickable title

* remove tooltip

* Use MBQL info to match columns whe npossible (#69120)

use mbql info for column matching

* Gdgt 1675 disable unmatched rows view details button in inspector (#69152)

* remove row index column

* adapt viz UI, hide row index column

* Unmatched rows lens: return cards for both LHS or RHS null scenarios (#69123)

* feat: handle both cases of no join match

* feat: simple degeneracy

* export function

* remove dead fields

* only propose drill lenses that are applicable (#69169)

* Transforms inspector consistent casing for FE (#69174)

* chore: consistency

* chore: some docstrings updats

* feat: snake_case for the FE

* feat: pass visitedFields to interestingFields, remove ad-hoc logic in FE (#69175)

feat: pass visitedFields to interestingFields, remove ad-hoc logic in the FE

* feat: namespace card-ids with params (#69180)

* add estimated lens complexity (hardcoded for now)  (#69181)

* add estimated lens complexity (hardcoded for now)

* refactor: use with-metadata

* add metadata to triggers, not just cards (#69182)

* appease kondo

* chore: don't depend on git sha macaw

* reinstate metabase-enterprise.transforms.core from merge conflict

* Gdgt 1722 new UI for data summary new (#69220)

* Transforms inspector BE tests  (#69225)

* feat: inspector tests

* fix: impls

* fix: nil safe

* feat: integration tests

* make it work for both FE and BE

* fix card-results for non oute rjoin

* e2e test

* fixes

* fix: quoting

* fix for mysql/bigquery/redshift

* gdgt-1721 rework lenses navigation (#69205)

* Transforms inspector module (#69278)

* split transforms inspector into its own module, introduce transforms-rest

* handle kondo modules warnings

* fix: shadow_cljs

* kondo: fixes

* Transforms inspector be review (#69339)

* chore: align

* remove useless comments

* feat: add schema to context

* feat: simplify column-matching

* fix: visited-fields

* fix test

* remove unused

* tighter schema

* extrac-sources test

* fix: missing metadata-provider

* Gdgt 1744 combine separate averages columns into one for data summary (#69355)

* gdgt-1723 New UI for Joins Analysis (#69325)

* Gdgt 1725 allow running transform from empty screen (#69394)

* chore: move crud functions on transforms, avoid -rest modules dependencies

* fix tests

* appease kondo

* fix modules

* fix: small UI bug

* Transforms orphaned columns (#69497)

* add orphaned columns to unmatched-rows analysis

* chore: update tests

* Transforms inspector analytics (#69484)

* feat: flow discover/lens through transform events

* track transform_id in query execution

* feat: add transform-inspector ids/params to query info

* feat: add inspector specific /query endpoint

* feat: hook FE to new api

* feat: add lens_id/params to view

* feat: add them to ua

* fix: pass params too

* fix: use text.type

* fix test

* chore: remove unused requires

* chore: events

* fix FE part

---------

Co-authored-by: Egor Grushin <egor@metabase.com>

* fix: use lib functions instead of manual mbql construction (#69519)

* fix: use lib functions instead of manual mbql construction

* chore: refactor utils

* cleanup

* more manipulation with lib

* bugfix

* strip fields/filters early

* do it in util function

* fix: bad merge resolution

* kondo

* another bad merge resolution

* chore: indent

* Gdgt 1754 handle slow loading lens indicator (#69334)

* lens loading indicator

# Conflicts:
#	frontend/src/metabase/transforms/pages/TransformInspectPage/TransformInspectPage.tsx
#	frontend/src/metabase/transforms/pages/TransformInspectPage/components/LensContent/LensContent.tsx

* add onCardLoaded handler to context

* Gdgt 1724 new UI column (#69282)

* Gdgt 1783 fix wrong args passed to lens query (#69546)

* fix wrong args being passed to lens queries

* improve naming of query parameters

* remove whitespaces from merge resolution

* add transforms api test

* feat: i18n the inspector strings (#69565)

* fix dark mode color issues (#69570)

fix bg fixed to white

* Gdgt 1737 use card metadata instead of parsing card ids (#69583)

* use card metadata instead of parsing ids

* improve way of getting the table name from cards

* use macaw still, TODO

* port some query-analysis to sql-tools (#69732)

* chore: port some query-analysis to sql-tools

* remove sql-tools from transforms

* chore: vendor macaw functions, appease kondo

* refactor

* chore: refactor

* side agnostic

* revert manual delete, closes https://linear.app/metabase/issue/GDGT-1730/pre-merge-blocker-revert-t2delete-on-transform-once-uxw-3021-is-fixed

* fix: always sync audit db on change

* dead code

* fix: use honeysql instead of building sql manually (#69774)

* GDGT-1791 Maximum depth error in console (#69754)

GDGT-1791 fix and refactor useCardLoadingTracker

* Gdgt 1651 add frontend e2e tests for transform inspector UI (#69677)

* GDGT-1823 make guarantee that LensContent is always stores one lens data (#69800)

* fix: 404 on lens not found (#69838)

* fix: 404 on lens not found

* chore: don't pmap

* fix: remove nil values from statistics

* fix

* Transforms inspector self review be (#69843)

* style

* chore: reuse resolve-join-sides

* add from-table-id for mbql queries too

* split context in mbql/native, don't union

* whitespace

* fix: preprocess native queries

* fix: update tests

* fix

* revert unrelated commit that made in the branch erroneously

* Gdgt 1812 better handle missing sources in lens inspector (#69848)

* Gdgt 1817 add snowplow product events fe (#69801)

* Add analytics events

# Conflicts:
#	frontend/src/metabase/transforms/analytics.ts

* add analytics tests

* simplify and improve lens loaded tracking event

* Move the idempotence check to useLensLoadedTracking

* GDGT-1831 replace unstyled button with a box to get rid of button in button error

* chore: transform lens_params

* GDGT-1786: Parametric lens navigation improvement (#69821)

* GDGT-1757: Refactor Column Comparison utils grouping and scoring logic (#69853)

* Improve types with entity ids (#69891)

* add concrete type declarations and usage for InspectorLensId, InspectorCardId, InspectorSectionId

* add better type to lens params

* cleanup of utils

* improve type for displaySettings

* use entity aliases in more places

* use existing type aliases for source and target

* add comment about InspectorTriggerCondition type

* unify prefix for inspector types

* remove spurious catch

* deduplicate types between metabase-api and lib

* Remove unused query

* fix provided tags for transform inspector

* add alerts and drill buttons test for default lens sections (#69899)

* restrict join analysis to single condition joins

* fix: bad merge reoslution

* fix types in lens test

* use isa function to detect type

* use concrete table id type alias

* use Urls.openInNewTab util

* remove empty mocks file

* use concrete table id type

* move lib for transform inspector to src/transforms/lib

* remove superflous log in e2e tests

* add proper SchemaName type

* remove superflous export

* revert move of isFetchBaseQueryError

* remove obsolete test

* use correct SchemaName type alias

* use correct VisualizationDisplay type

* remove leftover unused type

* remove superflous export

* Use CardId type alias

* discern lens also by based on params when tracking loading

* GDGT-1839 addressing the PR comments

* GDGT-1839 addressing the PR comments

* GDGT-1839 addressing the PR comments

* GDGT-1839 move getLensKey and LensHandle from api/tags directory

---------

Co-authored-by: Egor Grushin <egor@metabase.com>
Co-authored-by: Jakub Chodorowicz <jakub.chodorowicz@metabase.com>
Co-authored-by: Jakub Chodorowicz <jakub@chodorowicz.com>
2026-02-23 10:16:35 +00:00
Oleksandr Yakushev
5880342225 Remove unused metabase.util.formatting.numbers and metabase.util.formatting.time (#63791) 2025-11-03 22:37:47 +02:00
Cam Saul
8ad785fdbb Deprecate legacy MBQL utils (#64595)
* Deprecate legacy MBQL utils

* Deprecate legacy MBQL utils

* cljfmt

* Fix most tests except long native column names

* Kondo

* Test fixes 🔧

* Fix .clj versus .cljc in Lib

* Disable deprecation warnings

* Fix cljs

* Mega normalization improvements

* Kondo

* Another round of test fixes 🔧

* Schema fix 🔧

* PR feedback; test fixes

* Test fixes 🔧

* Fix BigQuery tests

* Test fix 🔧

* Test fixes 🔧

* Fix Card QP parameter normalization AGAIN 😆

* Oops

* Fix Claude code review

* Fix Claude code review

* Fix BigQuery tests
2025-10-21 15:29:36 -07:00
Nemanja Glumac
dbf8855111 Ignore fn-deprecated warnings in cljs logs (#64867) 2025-10-21 13:46:49 +02:00
Cam Saul
8749431e5d [Backend modularization] Move some legacy stuff to .core namespaces (#58185)
* [Backend modularization] Move some legacy stuff to `.core` namespaces

* Kondo linter and rename `metabase.http-client` => `metabase.test.http-client`

* metabase.cloverage-runner => metabase.test.cloverage-runner

* `metabase.types` => `metabase.types.core`

* cljfmt / Kondo

* Fix ns name

* Fix type names

* Add `NumericWrapper?` and `TextWrapper?` helpers

* cljfmt

* `metabase.cmd` => `metabase.cmd.core` (part 2)

* Test fixes 🔧
2025-05-20 22:01:32 +00:00
Cam Saul
c8546e2af3 [Backend modularization] dashboards module (first pass) (#58056)
* [Backend modularization] `dashboards` module

* Move models

* move API namespace
2025-05-16 20:58:47 +00:00
Cam Saul
d7eae1df71 [Backend modularization] parameters module (first pass) (#58043)
* [Backend modularization] `parameters` module (first pass)

* Make sure DB is initialized

* Oops revert the change I made to demo something during BE Guild Sync
2025-05-15 21:15:28 +00:00
Noah Moss
ea73a601b7 Shared FE/BE pivots (#53000)
* POC - moving two pivot functions into CLJC

* Debug logs in frontend pivot impl

* POC continued

* setting isCollapsed consistently

* Working replication of valuesByKey map construction

* sorting logic

* hook up new code to the FE impl

* Fixing small potential bug in valuesByKey

* Fixing bug in clojure vs javascript string key for valuesByKey

* fix valueColumns key name

* Some performance improvements + cleanup

* A couple more attempts at perf improvements

* Fix bugs

* port format-values-in-tree

* WIP add-subtotals

* @tsp - Improved build-values-by-key perf; some debug perf lines in frontend land

* @tsp - in progress conversion of add-subtotals

* @tsp - working converted Pivot.add_subtotals

* @tsp - fixing bug in add-subtotal mapcat recursion

* @tsp - add-subtotal restoring map to mapcat (oops)

* @tsp - adding working Pivot.add_value_column_nodes to replace addValueColumnNodes

* @tsp - removing debug logs & perf timing logs

* Appease whitespace linter

* More formatting code ported, cleanup

* Fix bugs, cleanup

* Fix some tests

* Start hooking up the BE, fix some bugs

* Fix lint/formatting errors

* Fix comparator + i18n

* Fix another module issue

* Test using js-specific i18n function

* Revert "Test using js-specific i18n function"

This reverts commit 9a8604b583885f1c35f48507112e3a142ca0d3a0.

* Bump up timeout on cypress assertion

* WIP - migrate row section getter to cljc

* fix tests

* Fix e2e test

* Switch exports over to fully using CLJC code paths, delete now-unused code

* Delete outdated postprocess tests

* Bug fix

* Fix build

* cleanup, consolidation

* fix function call

* more cleanup

* empty test ns

* misc cleanup/docstrings

* WIP tests

* more tests and refactor

* small fix

* fix getting column title for value columns

* Fix some tests

* Fix another test

* Revert a couple changes

* Fix some more tests

* Fix more tests, fix default values for pivot viz settings

* Ensure headers respect format-rows? settings in exports, and fix more
tests

* Fix lint issues

* fix another test

* Fix a test, a little cleanup

* Try to fix another test by setting format-rows middleware to false for subscription attachments

* Revert "Try to fix another test by setting format-rows middleware to false for subscription attachments"

This reverts commit d860522a6d17c27e67afabccf119039508c96cca.

* Update formatting logic to fix another test

* Fix conditional formatting

* Fix last BE test

* remove test not needed anymore

* Fix drill through + more cleanup

* clean ns

* remove old code from .clj-kondo/config.edn

* add a comment + tweak keywords

* perf improvements and refactoring the CSV export path

* Performance improvements

* Fix bad rebase

* more perf improvements

* fix test

* lint

* oops, fix test

* more optimizations, memoize backend formatting

* fix range-based conditional formatting

* remove unused function

* add some new backend unit tests

* fix FE type checks - caused by an experiment that I didn't revert fully

* address PR review feedback

* fix up tests

---------

Co-authored-by: Tyler Plude <tsplude@gmail.com>
2025-03-14 22:49:11 +00:00
Braden Shepherdson
6f1d9cbdd3 [refs] Generate idents for new clauses in MBQL lib (#50083)
Specifically, for expressions, aggregations, breakouts, and joins.
Adds MBQL lib unit tests for this behaviour.

Currently these idents are discarded during `lib.convert` and don't
survive a round trip to legacy. That will come with a later change.
2024-11-29 11:56:07 -05:00
Cam Saul
6024bf019c Collapse metabase.shared.* namespaces (#48646)
* Collapse `metabase.shared.*` namespaces

* Fix Kondo warnings

* Does updating the stories-data keys fix the failing tests?

* Appease msgcat

* Appease msgcat

* Fix typo

* Make the build happy

* Appease fslint
2024-10-14 10:06:10 -07:00
appleby
52284e622b Ensure that CI checks fail if cljs tests fail (#47192)
* Fix typo in expression-clause-normalization-test

s/time-interval/relative-time-interval/

* Add missing table-id to card def to fix failing legacy-ref-test

* Ensure that if yarn test-cljs fails it fails the CI checks

When configured to autorun, the command

   shadow-cljs compile test

will exit success even if test failures occur in the node test. See

   https://github.com/thheller/shadow-cljs/issues/425

In order to ensure our CI checks fail when cljs tests fail, we need to invoke
node directly. To prevent running the tests twice, remove `:autorun true` from
the `:test` target in shadow-cljs.edn.
2024-08-27 19:15:50 +00:00
metamben
be9740ca70 Enable Clojure logging in the browser (#43959)
Messages at INFO level and higher will appear in the browser console.
2024-06-10 22:39:47 +03:00
Cam Saul
59e7b3d1cb metabase.xrays API namespace (#42491)
* Remove unused `metabase.api.transform`

* Copy the deps graph improvements from my other PR

* New combined X-Rays API namespace

* Fix kondo error
2024-05-10 09:13:59 -07:00
Cam Saul
0a15637f13 Rename metabase.mbql to metabase.legacy-mbql (#40158)
* Wow

* Test fix 🔧

* Fixes

* Actions should use strings for column names (fix :update-row and :create-row normalization)

* MLv2 schema should check against keys for the other query type

* Ok, have I fixed things?

* More fixes 🔧

* Fix indentation

* Another round of test fixes. 🔧

* Hopefully the last few test fixes 🔧

* We need to test normalization for queries that have keyword keys as well.

* Fix Cljs i18n namespaces

* Sort namespaces

* Only test against H2

* Rename `metabase.mbql` to `metabase.legacy-mbql`

* Fix Kondo warnings

* Test fixes 🔧

* Register MBQL clause schemas and test fixes 🔧

* Test fixes and PR feedback

* Test fix

* Remove the normalization tests

* Test fixes 🔧

* Fix kondo

* Fix import

* Another fix 🔧

* Merge

* FIXES

* Add another missing REQUIRE
2024-03-21 05:38:41 +00:00
Mark Bastian
2a017f80dc Add per viz-type card template sizes in x-rays (#38836)
* Add per viz-type card template sizes in x-rays

Previously, any card without a size (width or height) set in their yaml definition was set to a default of `default-card-width` or `default-card-width`, respectively. This PR adds a map of default and min dimensions per viz type (the `:display`) key and fills that in for each template. The template is merged into this result, keeping anything that already existed in the yaml file.

Before merging this we should consider if we want to use the min values and where this data should be stored. It is currently in `metabase.automagic-dashboards.dashboard-templates`, but was lifted from [sizes.ts](https://github.com/metabase/metabase/blob/master/frontend/src/metabase/visualizations/shared/utils/sizes.ts).

* Moving card sizing constants to cljc

The default card sizes were found in `sizes.ts` and were duplicated into `metabase.automagic-dashboards.dashboard-templates`. This PR consolidates those changes into `metabase.shared.automagic-dashboards.constants`.

Note that @kulyk and @JesseSDevaney might want to consider further consolidation of dashboard related constants into this namespace.

* Incorporating Feedback

Changes
- Explicitly defined two constants in the shared file that are clj and js friendly
- Set case to CARD_SIZE_DEFAULTS_JSON
- Updated `metabase.automagic-dashboards.dashboard-templates/set-default-card-dimensions` to use a helper

* Update src/metabase/shared/automagic_dashboards/constants.cljc

Co-authored-by: Jesse Devaney <22608765+JesseSDevaney@users.noreply.github.com>

* Updating ns name per Jesse

---------

Co-authored-by: Jesse Devaney <22608765+JesseSDevaney@users.noreply.github.com>
2024-02-16 20:56:32 +00:00
Braden Shepherdson
c062ab7ff0 Enable source maps for CLJS in release builds (#38819)
We do this for our JS and TS code. This change configures shadow-cljs to
emit the source maps in release mode, and bumps the versions of
shadow-cljs and the transitive dependency `source-maps-js` to enable
webpack to consume these source maps correctly.

(The output source maps have some quirks, and broke some consumers.
Mainly that inlining of CLJS functions can result in empty namespaces,
and while the file contents were correctly inlined,
`sourceContents: ""` looks falsy in JS. Webpack would try to read the
nonexistent source files and fail to build.)
2024-02-16 08:02:13 +00:00
Braden Shepherdson
8e298158a4 Enable ClojureScript REPL and hot reloading of CLJS code (#34007)
This also moves the CLJS output out of `frontend/src/cljs{,_release}` to
`target/cljs_{dev,release}`. If that causes problems for FE tooling I
don't know about, it could be moved back.

Hot reloading:
CLJS changes are hot-reloaded into the running app by Shadow CLJS - no
action is needed. Note that the UI will not refresh; you usually have to
click around so the updated MLv2 code gets executed.

Usage of the CLJS REPL:
- Open the browser (you should see `shadow-cljs: #N ready!` in console)
- Connect to nrepl on 50655
- "Select" the `:app` build;
    - https://shadow-cljs.github.io/docs/UsersGuide.html#_editor_integration
    - Conjure: `:ConjureShadowSelect app`
    - Plain REPL: Eval in CLJ mode `(shadow/repl :app)`
- Consider running `yarn build-hot:cljs` and `yarn build-hot:js` in
  separate terminals; then when `yarn build-hot:js` crashes you don't
  have to reconnect your REPL.
- The REPL works beautifully, even loading files outside the :entries
  of the shadow-cljs build like tests!

"Stale" error in browser:
- This can happen if the webpack bundle is created before the CLJS is
  done building.
- If you ran `yarn build-hot:cljs` and `:js` separately, let `:cljs`
  finish compiling first.
- If you see this otherwise: (a) contact Braden so it can be fixed; and
  (b) either restart `build-hot:js` or force webpack to rebuild by
  hitting http://localhost:8080/webpack-dev-server/invalidate

Yarn commands:
`dev`, `dev-ee`, `build-hot`, and `build-static-viz:watch` all now wait
for the CLJS build to be complete by watching for files to be generated.
They should work the same from a user's POV.

(Since the webpack build no longer watches the CLJS output,
it won't notice the changes and immediately rebuild after the first
build. This avoids errors on the clean build, or "stale build" messages
in the app from Shadow CLJS.)

Fixes #32674.
2023-10-02 21:52:15 +00:00
Anton Kulyk
43bf714dec [MLv2] Add wrappers for column type checks, add getColumnIcon helper (#30396)
* Add `shadow-cljs.edn` entry

* Add TypeScript wrappers for `isa` functions

* Extract `findColumn` test helper

* Add `getColumnIcon` function

* Use `getColumnIcon` in `QueryColumnPicker`

* Deprecate `getIconForField`

* Exclude semantic and effective types

* Remove comment

* Fix test

* Rename test helper to `columnFinder`
2023-05-02 16:06:49 +01:00
Cam Saul
9d0d93225f MLv2 Column grouping utilities (#30018)
* Column grouping [WIP]

* MLv2 Column Grouping

* Oops need to munge name

* Code overhaul

* Test fixes 🔧
2023-04-12 09:15:58 -04:00
Anton Kulyk
34d6f614d1 Use metabase-lib v2 to manage query row limit (#29454)
* Add TypeScript wrapper for limit helpers

* Use v2 limit in `StructuredQuery`

* Add redux action for managing limit in QB

* Switch `QuestionRowCount` to use ML v2

* Add utils to convert v2 queries into v1

* Use `toLegacyQuery`

* Fix CLJS limit functions unavailable in Jest

* Fix tests

* Style tweaks

* Fix type error

* Use opaque type for ML `Query`

* Use `Lib` instead of `MetabaseLib`

* Drop v2 folder

* Fix MBQL in unit tests

* Don't import individual functions from MLv2

* Export `limit` CLJC functions

* Tweak CLJS imports
2023-03-29 17:48:25 +01:00
Cam Saul
7f1531982e MLv2 Move metadata calculation functions into respective namespaces (#29245)
* Query  D E S C R I P T I O N S

* WIP

* MLv2 Query description

* Test fixes

* More robust error handling

* MLv2 more shuffling things around

* Remove unneeded impl

* lib.expressions -> lib.expression

* Address PR feedback
2023-03-20 16:47:41 +00:00
Mark Bastian
289116c334 Cycle 47 minor version bumps for modules (#29181)
Updated module versions along with a few updated main deps.edn libs. Also added the outdated task to the deps file and fixed in place the nrepl and shadow-cljs nrepl ports.
2023-03-14 18:48:23 +00:00
Braden Shepherdson
5e3c29cb9e Fix exclusion of CLJS devtools from release builds
The new approach uses a dev-mode-only CLJC reader conditional
`#?(:cljs-dev ...)` to make `metabase.util.devtools` empty
except in dev builds.

The previous approach for this relied on dead code elimination.
But DCE relies on tracing function calls and `:require`s from the
`:entries` list in shadow-cljs.edn. Top-level namespace code is
handled differently; in particular a namespace listed in `:entries`
like `metabase.util.devtools` will always have its top-level code
in the build.
2023-03-01 09:25:01 -05:00
Braden Shepherdson
4580eab946 [metabase-lib] Dev experience and CLJC infra improvements
Dev experience:
- CLJS now has source maps
- Devtools extension renders CLJS data in Chrome (et al) Devtools
- Use a debug build of CLJS in the tests
- Fixed line numbers and breakpoints being off-by-one in Devtools

CLJC infra:
- More robust direct approach to JS<->CLJS conversions, powered by
  Malli transformers.
- Powerful `define-getters-and-setters` macro to generate the accessors.
2023-02-27 15:40:27 -05:00
Braden Shepherdson
4c15f80c33 [metabase-lib] PoC for wrapping JS objects with cljs-bean in CLJC
A `metabase.domain-entities.malli/defn` macro that (in CLJS) uses the
schemas for the arguments to drive wrapping of vanilla JS objects for
idiomatic use in CLJC code.

See the first two functions in `metabase.domain-entities.queries.util`
which get passed vanilla JS objects but can use them as CLJS maps.
2023-02-14 10:04:34 -05:00
Braden Shepherdson
6c16481269 Port most of metabase.util to CLJC (#28133)
Some JVM-specific functions are now in `metabase.util.jvm`, but they
are re-exported (in CLJ) by `metabase.util`.
2023-02-10 05:08:11 +00:00
Braden Shepherdson
98fecd3e2f Fix metabase.util.log/infof etc., which were not formatting
Includes much-improved testing for the logger in CLJ and CLJS.

**This moves us from explicit `shadow-cljs.edn` dependencies to relying
on deps.edn for everything. It seems to be working nicely.
2023-02-08 14:36:28 -05:00
Braden Shepherdson
dad3d414e5 Standardize on metabase.util.log for logging; use it everywhere (#28050)
This has the same interface as `clojure.tools.logging` but supports CLJS
as well. It was formerly known as `metabase.shared.util.log`.
2023-02-07 21:14:06 +00:00
Braden Shepherdson
335d161f40 [metabase-lib] Port date and time parsing and formatting to CLJC (#27551)
This adds a cross-platform date/time formatting library in CLJC, with
(nearly) identical output in both JVM and JS environments.

The only known difference is week numbers.

CLJS (through Moment.js) has ordinal numbers, and renders the week
number as "34th". Neither the JVM nor Clojure has a handy library for
this (I'm sure it's out there somewhere, but it seems like a silly
dep to add) so it renders week numbers as simply "34".

Both platforms have date formatters that use pattern strings (eg.
`"YYYY-MM-dd"`) to turn date/time objects into strings. There's a strong
resemblance in how these work and what letters stand for what parts of
the date and time, but they are far from identical.

Rather than try to hackily convert one set of strings to another with
regexes or other manipulations, this PR defines a set of names for
fragments of dates (eg. `:year`, `:day-of-week-full`, `:hour-24-dd`)
and includes functions to transform a list of these keys into a
platform-specific format function.

This is portable and transparent, and can be written in Clojure or
JS code:

```clj
[:year "-" :month-dd "-" :day-of-month-dd]   ; 2022-04-08
[:month-full " " :day-of-month-d ", " :year] ; April 8, 2022
[:hour-12-dd ":" :minute-dd " " :am-pm]      ; 7:52 AM
```

```js
[":year", "-", ":month-dd", "-", ":day-of-month-dd"]   ; 2022-04-08
[":month-full", " ", ":day-of-month-d", ", ", ":year"] ; April 8, 2022
[":hour-12-dd", ":", ":minute-dd", " ", ":am-pm"]      ; 7:52 AM
```

Note that the original code allowed an unrecognized `:date-style` string
to be used directly as the formatting string. With the move to
formatting data structures that no longer works. Instead there is a
fixed map of format strings to the above data structures, that contains
all the currently used `:date-style` inputs.

If some caller needs a new format someday, we can either: (a) add the
style to the map; or (b) pass the data structure form directly to the
`:date-format` option, which if provided is used as the format.

It's tempting to go to all the call sites and replace these `:date-style`
strings with the new format structures. However, the strings are still
embedded in the `:visualization_settings` in user appDBs, so we still
need to recognize them.

Most of the existing functions in the TS library have the same API.

The set of allowed `:date-style` and `:time-style` values is turned into
a type using `keyof`, so that we get precise type-checking of these
values rather than simply `string`.

One function has been dropped from the API: `getDateFormatFromStyle`.
There was no practical way to implement it using the new format data
structures. The only caller was the date format column settings, and it
has been rewritten to use the formatted string for its sample date as
its key instead.
2023-01-13 09:08:06 -05:00
Braden Shepherdson
2537f3d084 [metabase-lib] Port number formatting to CLJC; use it from JS (#27114)
This adds a cross-platform number formatting library in CLJC, which has
(nearly) identical output in both JVM and JS environments.

The only known differences are:
- JVM Clojure has `BigDecimal` support, so isn't limited to a `double`'s
  range.
- Exact spelling/capitalization of the full names of currencies differs.
    - JVM: "7.34 US Dollar"
    - JS: "7.34 US dollars"

Some future work is required to fully land this. In particular, the
`currency_in_header` logic should be moved to the table formatters, not
implemented on the JS side of this change.
2022-12-21 15:56:27 -05:00
Noah Moss
dfd93579e7 Bump shadow-cljs to 2.19.6 (#24647)
* Bump shadow-cljs

This lets us use the template tag functionality

```
add experimental support for creating js template strings

used like str but emits native JS `` template

  (js-template "foo" (+ 1 2) "bar")

emits the literal

  `foo${(1 + 2)}bar`
```

But this required a few changes on our side as well.

Required for this:

-- LOGGING

Bumped glogi
```diff
   ;; new stuff
-  [lambdaisland/glogi "1.0.106"]]
-
+  [com.lambdaisland/glogi "1.1.144"]]
```
Annoying because it had a group name change due to Clojars policy. This
had the same error as us below. That's why I'm hopeful that it will
actually just work.

There were some changes with the Google Closure library with things not
included or referenceable. So I just removed these imports and the
typehints that used them. We'll need to verify that this still works but
both compiling for dev and release (with advanced compilation)
work. Still possible there will be some runtime errors but i'm hopeful
there wont be. We can grab a jar from CI and poke around

```shell
❯ npx shadow-cljs compile app
shadow-cljs - config: /Users/dan/projects/work/metabase/shadow-cljs.edn
shadow-cljs - connected to server
[:app] Compiling ...
[:app] Build completed. (100 files, 0 compiled, 0 warnings, 1.65s)

metabase on  nm-all-options-formatting [$!+?] via ☕ v17.30 on ☁️  metabase-query took 2s
❯ npx shadow-cljs release app
shadow-cljs - config: /Users/dan/projects/work/metabase/shadow-cljs.edn
shadow-cljs - connected to server
[:app] Compiling ...
[:app] Build completed. (100 files, 30 compiled, 0 warnings, 28.04s)
```

```diff
             [lambdaisland.glogi.console :as glogi-console])
-  (:require-macros metabase.shared.util.log)
-  (:import goog.debug.Logger
-           goog.debug.Logger.Level))
+  (:require-macros metabase.shared.util.log))
```

```diff
@@ -37,4 +35,4 @@
 (defn is-loggable?
   "Part of the impl for [[metabase.shared.util.log/js-logp]] and [[metabase.shared.util.log/js-logf]]."
   [logger-name level]
-  (.isLoggable ^Logger (log/logger logger-name) ^Level (log/levels level)))
+  (.isLoggable (log/logger logger-name) (log/levels level)))
```

-- WARNINGS ABOUT shadowing `abs`

-  [medley "1.3.0"]
+  [medley "1.4.0"]

Similar to what we bumped for the backend. ClojureScript 1.11.x has an
`abs` function.

* use update-keys and update-vals in cljs

* bump clj-kondo in CI

Co-authored-by: dan sutton <dan@dpsutton.com>
2022-08-05 20:30:08 -04:00
Noah Moss
e136f4307c First pass at parameters in Markdown cards (#23641)
* first pass at parameters in text cards on FE

* trying to get translations working

* relative datetime formatting

* copy changes and 'Text card' header

* default text when no params

* hide header for text cards with height of 1 with params when in param mapping mode

* show UI text in mobile mode

* minor fixes

* enforce that a text card variable can only be mapped to one parameter

* more value formatting

* noop

* fix backend tests

* add back a couple pieces of frontend logic commented out

* misc cleanup

* attempt at adding a FE unit test

* revert unit test, doesn't work

* add a couple of basic cypress tests and fix a couple of bugs

* basic unit tests for cljc

* fix error

* expanded unit tests

* simplify ns

* add cypress test for instance language translation

* basic handling for a couple cases of :date/all-options

* trs docstring clarification

* whitespace tweaks

* fix cypress test

* minor refactor of tag-names

* move cljc file from utils to new parameters dir

* reorder functions

* fix lint

* add test assertion that locale is correctly reset back to english, and add a comment

* fix bug where existing parameter mapping target was not being found

* clojure logic tweaks

* move text card header text to the Text component config

* simplify header logic, and pull out isLoading into a function to reduce complexity

* address alex's css feedback

* fix trs comment
2022-07-12 15:27:00 -04:00
Cam Saul
cf247b6865 Bump backend dependencies (Feb 2022) (#19827)
* Bump backend dependencies (Jan 2022)

* Revert java-time version upgrade for now until https://github.com/dm3/clojure.java-time/issues/77 is fixed

* Add license overrides

* Bump a few more deps (again). Revert Google/BigQuery and Vertica version bumps

* Revert MariaDB and Redshift version changes
2022-02-03 10:57:44 -08:00
Cam Saul
2af2ba952d Bump some more backend dependencies (#17326)
* Bump some more backend dependencies

* Add license info for com.sun.activation/jakarta.activation

* Fix reflection warnings

* Revert bump of MariaDB for now
2021-08-05 11:29:52 -07:00
dpsutton
69bb775885 Remove last of leiningen, update comment in shadow and lsp classpath (#17317)
* Remove last of leiningen, comment in shadow and lsp cp [ci skip]

* empty commit to flush [ci skip]
2021-08-04 10:44:58 -07:00
Noah Moss
042abc928a Move currency list from JS to Clojure (#17243) 2021-07-29 14:13:26 -07:00
Cam Saul
58cfa2e5f6 Shared MBQL lib (#15267)
Port the metabase.mbql utility namespaces to ./shared/ so they can be used on both the frontend and backend.
2021-03-23 17:49:03 -07:00
Cam Saul
cca03d22d4 Shared CLJ/CLJS lib (#14980)
* Shared CLJ/CLJS lib (PoC/WIP)

* PoC 2.0

* Fixes 🔧

* More test fixes 🔧

* Bump shadow-cljs version

* Fix more stuff

* Need to ^:export the exports

* CI fixes 🔧

* Add eslintignore

* Ignore cljs files for FE code coverage

* Try prefixing CLJS -> JS import with goog:

* Revert indentation change

* No goog:

* Add .prettierignore

* Use advanced build for now for JS tests unit we can figure out how to make it work
2021-03-05 13:35:15 -08:00