Compare commits

..

354 Commits

Author SHA1 Message Date
Thomas Schmidt
349f48ce44 Basic sql generation draft 2025-03-26 17:03:56 +01:00
Thomas Schmidt
b08b714b0e Do not convert tool names (#55505) 2025-03-24 07:39:43 +01:00
Tamás Benkő
523cd3812b Always set :queryable-foreign-key-tables for models 2025-03-20 22:45:46 +03:00
Tamás Benkő
d67c024717 Remove unused schema definition 2025-03-20 22:45:09 +03:00
Tamás Benkő
e52be87e3a Convert history sent by the FE 2025-03-20 22:44:39 +03:00
metamben
4f83b5eff3 Merge branch 'master' into metabot-v3-main 2025-03-20 21:42:49 +03:00
metamben
02ef883215 Expose table references for metrics (#55456)
Fixes BOT-66
2025-03-20 21:39:37 +03:00
metamben
a654c4025b Expose field values for metrics, models, and tables (#55413)
Fixes BOT-69
2025-03-20 21:32:33 +03:00
metamben
8c236418de Normalize metadata IDs (#55383)
Fixes BOT-68
2025-03-20 21:26:20 +03:00
metamben
ffb7af68c8 Keep history including the initial user message (#55473) 2025-03-20 21:25:25 +03:00
metamben
5333eeda6b Merge branch 'master' into metabot-v3-main 2025-03-19 20:53:04 +03:00
metamben
9981e3db93 Merge branch 'master' into metabot-v3-main 2025-03-18 22:39:39 +03:00
Tamás Benkő
93ab435a9d Merge remote-tracking branch 'origin/master' into metabot-v3-main
# Conflicts:
#	.github/workflows/uberjar.yml
#	.loki/reference/chrome_laptop_Design_System_Icons_List.png
#	frontend/src/metabase/query_builder/components/view/View/ViewNativeQueryEditor/ViewNativeQueryEditor.jsx
2025-03-18 22:38:59 +03:00
metamben
805f066b8c Add answer source tool (#55172) 2025-03-18 12:36:57 +03:00
Tamás Benkő
8af934fecd fix: Allow strings as tool call arguments 2025-03-17 15:07:51 +03:00
metamben
79a24d5800 Merge branch 'master' into metabot-v3-main 2025-03-17 14:40:56 +03:00
Tamás Benkő
ccc58d6578 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-03-16 18:00:15 +03:00
Tamás Benkő
c06e62576b Merge remote-tracking branch 'origin/master' into metabot-v3-main
# Conflicts:
#	.loki/reference/chrome_laptop_Design_System_Icons_List.png
2025-03-14 10:59:57 +03:00
metamben
2aec76218d Add query model tool (#55089)
Fixes BOT-61
2025-03-14 10:46:52 +03:00
metamben
c14d51b22e Add support for temporal extraction filters (#55037)
* Add support for temporal extraction filters

Fixes BOT-60.

Actually, both extraction and bucketing is supported. The following values can be provided in the filter's `bucket`
property:

```
millisecond
second
minute
hour
day
week
month
quarter
year
second-of-minute
minute-of-hour
hour-of-day
day-of-week
day-of-month
day-of-year
week-of-year
month-of-year
quarter-of-year
year-of-era
```

The values of the form x-of-y result in extraction, the other result in truncation. The extractions `week-of-year` and
`day-of-week` have ISO semantics.

* Placate kondo
2025-03-13 20:41:53 +03:00
metamben
ef69ec3129 Merge branch 'master' into metabot-v3-main 2025-03-12 14:52:36 +03:00
Tamás Benkő
4fee5cd3c4 Update expected native query result text 2025-03-11 23:48:06 +03:00
Tamás Benkő
644f935654 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-03-11 23:11:48 +03:00
Tamás Benkő
2e6734d105 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-03-11 22:55:57 +03:00
metamben
1db581dd1b Merge branch 'master' into metabot-v3-main 2025-03-10 15:18:55 +03:00
metamben
71d913d8d1 Merge branch 'master' into metabot-v3-main 2025-03-09 14:11:37 +03:00
metamben
51ac49727e Merge branch 'master' into metabot-v3-main 2025-03-08 01:56:24 +03:00
metamben
806d623462 Remove dummy tools and Metabot loop related stuff (#54812)
* Remove dummy tools and clean up
2025-03-08 01:56:13 +03:00
metamben
8f62da32cc Merge branch 'master' into metabot-v3-main 2025-03-07 13:49:24 +03:00
metamben
edd7bc6a9c Distinguish temporal types and add semantic type (#54741)
Fixes BOT-56.
2025-03-07 13:49:10 +03:00
metamben
8f62fb33a5 Merge branch 'master' into metabot-v3-main 2025-03-06 17:55:00 +03:00
metamben
15f333d81f Add type agnostic and disjunctive filters (#54731)
Fixes BOT-54.
2025-03-06 17:54:51 +03:00
metamben
2543a6af72 Merge branch 'master' into metabot-v3-main 2025-03-06 15:01:23 +03:00
Tamás Benkő
961a73ff4c Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-03-05 21:36:59 +03:00
Tamás Benkő
383591f75b Update icon list in Loki 2025-03-05 21:36:19 +03:00
Tamás Benkő
2e1c2d7205 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-03-05 17:12:37 +03:00
Tamás Benkő
57ca9f0aac Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-03-05 17:10:36 +03:00
metamben
4c1fe3d767 Merge branch 'master' into metabot-v3-main 2025-03-04 14:27:27 +03:00
Tamás Benkő
8c758e3674 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-03-03 23:56:37 +03:00
Tamás Benkő
2bf18a43ec Send "user_is_viewing" to the agent in the context 2025-03-03 23:55:44 +03:00
metamben
4846e8a94b Merge branch 'master' into metabot-v3-main 2025-03-03 19:44:48 +03:00
metamben
a62e4e10ef Merge branch 'master' into metabot-v3-main 2025-03-03 18:31:54 +03:00
metamben
9778f8c12b Convert dummy tools to normal tools (#54519)
Fixes BOT-30
Fixes BOT-31
Fixes BOT-32
Fixes BOT-33
Fixes BOT-34
Fixes BOT-35
2025-03-03 18:31:43 +03:00
metamben
486d792c9a Remove V1 metabot agent endpoint (#54380)
* Remove V1 metabot agent endpoint

Fixes BOT-36
2025-03-03 17:18:23 +03:00
Tamás Benkő
4975f725a9 Merge remote-tracking branch 'origin/metabot-bot-47-tool-api-endpoint-tests' into metabot-v3-main 2025-03-03 17:09:45 +03:00
Tamás Benkő
b5e38e84b5 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-03-03 17:06:48 +03:00
Tamás Benkő
ce62f30ffc Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-03-03 17:06:16 +03:00
Tamás Benkő
8374ed8450 Fix namespace 2025-02-28 23:37:38 +03:00
metamben
e2660bf3ef Merge branch 'master' into metabot-v3-main 2025-02-28 23:28:36 +03:00
Alexander Polyankin
08ebf5a3e3 Merge remote-tracking branch 'origin/metabot-v3-main' into metabot-v3-main 2025-02-28 15:20:15 -05:00
Alexander Polyankin
485df761d0 Use CSS colors 2025-02-28 15:20:06 -05:00
Alexander Polyankin
f2f4f57866 Use CSS colors 2025-02-28 15:19:45 -05:00
Alexander Polyankin
3abeeb9ab2 Use CSS colors 2025-02-28 15:15:56 -05:00
metamben
cb22a6bcf4 Merge branch 'master' into metabot-v3-main 2025-02-28 23:11:50 +03:00
Tamás Benkő
ad34b73d94 Placate clj-kondo 2025-02-28 22:49:37 +03:00
metamben
351daa8851 Merge branch 'master' into metabot-v3-main 2025-02-28 16:46:47 +03:00
Tamás Benkő
61b153a918 Run tests with the metabot-v3 feature enabled 2025-02-27 18:32:27 +03:00
Tamás Benkő
d1b0bf48fd Add V2 tool API endpoint tests
Fixes BOT-47.
2025-02-27 18:05:43 +03:00
metamben
1671d222cd Merge branch 'master' into metabot-v3-main 2025-02-27 17:09:46 +03:00
metamben
c0ef616c6d Merge branch 'master' into metabot-v3-main 2025-02-26 16:27:15 +03:00
Ngoc Khuat
78743ba645 fix whitespace 🤦 2025-02-26 15:32:03 +07:00
Ngoc Khuat
01e98249a2 fix invalid reducers 2025-02-26 15:13:46 +07:00
Ngoc Khuat
1dabc6ccbb Merge branch 'master' into metabot-v3-main 2025-02-26 14:55:59 +07:00
metamben
752287a56d Merge branch 'master' into metabot-v3-main 2025-02-25 20:23:55 +03:00
metamben
865d8a0d81 Generate site-wide session tokens for the AI service (#54269)
* Generate site-wide session tokens for the AI service

Fixes BOT-38.

This PR also allows the V2 tool API endpoints to be used by other existing authentication methods like API keys and
Metabase sessions.
2025-02-25 20:23:42 +03:00
metamben
26dc17b31d Merge branch 'master' into metabot-v3-main 2025-02-25 16:53:05 +03:00
metamben
2f6a39b40e Merge branch 'master' into metabot-v3-main 2025-02-24 22:10:14 +03:00
Tamás Benkő
dfab957510 Fix tests 2025-02-24 21:15:30 +03:00
metamben
ea3b49aa0a Merge branch 'master' into metabot-v3-main 2025-02-24 18:48:30 +03:00
metamben
5d81a60a64 Introduce V2 tool endpoints (#54192) 2025-02-24 18:48:17 +03:00
metamben
10bd22a3b9 Merge branch 'master' into metabot-v3-main 2025-02-21 16:45:24 +03:00
Ngoc Khuat
1e75309a10 Merge branch 'master' into metabot-v3-main 2025-02-21 11:27:27 +07:00
metamben
68d24b95b4 Merge branch 'master' into metabot-v3-main 2025-02-20 18:46:51 +03:00
metamben
0574137a29 Merge branch 'master' into metabot-v3-main 2025-02-19 17:48:32 +03:00
metamben
455713606e Merge branch 'master' into metabot-v3-main 2025-02-18 17:44:02 +03:00
metamben
fcf86ea731 Skip NOT IN condition if there are no used-ids (#53845) 2025-02-18 17:43:46 +03:00
Mihael Konjević
168c055c8f Keep Metabot palette item stable between query changes (#53827) 2025-02-18 12:41:24 +01:00
Tamás Benkő
1e715f418b Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-02-17 18:46:06 +03:00
Tamás Benkő
5aca4ec11d Fix test 2025-02-17 18:45:41 +03:00
Tamás Benkő
27045cf9a7 Remove .clj-kondo/src/hooks/metabase/models/setting.clj changes 2025-02-17 17:06:36 +03:00
Tamás Benkő
d358286383 Merge remote-tracking branch 'origin/master' into metabot-v3-main
# Conflicts:
#	.clj-kondo/src/hooks/metabase/models/setting.clj
2025-02-17 17:03:06 +03:00
metamben
c19b8310dd Fix module dependencies (#53763)
* Fix module dependencies
2025-02-17 16:03:54 +03:00
metamben
3b535a542c Add find-outliers tool endpoint (#53624)
* Add find-outliers tool endpoint
* Also fix the decoding of find-metric tool API results.
* Add tests for explicit query handling.
* Add context when running the query for finding outliers
2025-02-14 17:00:24 +03:00
metamben
291d9e12e1 Merge branch 'master' into metabot-v3-main 2025-02-14 16:56:41 +03:00
Thomas Schmidt
bd1da7cdb9 Include query in request to ai-service (#53745) 2025-02-14 14:48:37 +01:00
Tamás Benkő
96cd8ba5b7 Appease the module checker even more vigorously 2025-02-14 00:44:10 +03:00
Tamás Benkő
b5e2e66b27 Skip SQL fixer e2e test and mute the module linter for now 2025-02-14 00:17:44 +03:00
Tamás Benkő
5b8a479fe4 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-02-14 00:13:41 +03:00
Tamás Benkő
bf5df2cd38 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-02-13 20:10:58 +03:00
Tamás Benkő
98332749de Address linter issues 2025-02-13 20:09:29 +03:00
metamben
827c0c2eab Merge branch 'master' into metabot-v3-main 2025-02-13 18:21:44 +03:00
Tamás Benkő
e74c254612 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-02-13 18:20:33 +03:00
Tamás Benkő
04b7020aef Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-02-13 18:19:51 +03:00
Romeo Van Snick
edb6b83feb Fix loop in codemirror (#53599)
* Correctly compare selections

* Add reproduction for QUE-614

* Use github issue number in regression

* Unnest issue from other issue reproduction

* Log in before test
2025-02-13 15:16:42 +01:00
Thomas Schmidt
10550112f5 Only send path for xray dashboard not full link (#53594) 2025-02-13 12:34:55 +01:00
Mihael Konjevic
5c99127943 Merge branch 'master' into metabot-v3-main 2025-02-12 20:56:08 +01:00
Mihael Konjević
5b5ab1fa16 Metabot v3 command palette item (#53568)
* Initial implementation of Metabot palette action

* Use default styles, correct ordering of palette items, tests

* Setup tests with metabot_v3 feature enabled

* Recompute "mainReducers" on demand in the renderWithProviders function

This solves a race condition where reducers were computed before the
enterprise plugins were required, which made it impossible to test their
behavior (in this case in PaletteResult tests). makeMainReducers function is
added which computes the reducers object, and the previous behavior is
preserved (mainReducers const points to the output of the makeMainReducers fn)
2025-02-12 19:40:26 +01:00
metamben
76d904ba3a Merge branch 'master' into metabot-v3-main 2025-02-12 19:21:20 +03:00
metamben
ca3698711b SQL fixer: generate schema_ddl (#53381)
* Generate ddl_schema
2025-02-12 19:21:05 +03:00
metamben
271fab9a30 Add find-metric tool endpoint (#53492)
* Add find-metric tool endpoint

* Add missing result conversion functions
2025-02-12 19:20:01 +03:00
metamben
ff0ed9be47 Add filter-records tool endpoint (#53561)
* Add filter-records tool endpoint

* Adapt tests
2025-02-12 18:40:00 +03:00
Tamás Benkő
a69afcb929 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-02-12 17:59:55 +03:00
Tamás Benkő
c4b2f29b0f Remove unused imports 2025-02-11 20:02:45 +03:00
Tamás Benkő
44a15f3a77 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-02-11 19:52:14 +03:00
Tamás Benkő
6630e72403 Stop re-using metabase.api.card/get-card
This function is private now, and it's also doing more than what we need right now.
2025-02-11 19:51:35 +03:00
Tamás Benkő
318e3e6b6f Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-02-11 19:10:06 +03:00
Thomas Schmidt
91948e24ab Redirect user when navigate_to property is present in agent response (#53518) 2025-02-11 16:36:04 +01:00
Tamás Benkő
6c20dc9c17 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-02-11 17:54:38 +03:00
metamben
a6f8cc0919 Fix input encoding for the create-dashboard-subscription tool (#53486) 2025-02-10 23:45:17 +03:00
Alexander Polyankin
3ebda589f5 Merge remote-tracking branch 'refs/remotes/origin/master' into metabot-v3-main
# Conflicts:
#	frontend/src/metabase/query_builder/components/NativeQueryEditor/CodeMirrorEditor/extensions.tsx
2025-02-10 11:05:15 -05:00
Tamás Benkő
e51023ca43 Remove describeEE and onlyOnEE 2025-02-08 02:33:40 +03:00
metamben
44da1f4d67 Merge branch 'master' into metabot-v3-main 2025-02-07 23:24:46 +03:00
metamben
4a1c0c5029 Check table_id parameter in generate-insights tool (#53310) 2025-02-07 20:33:35 +03:00
metamben
f2792cdbfd Merge branch 'master' into metabot-v3-main 2025-02-07 18:47:20 +03:00
metamben
10c042dbdc Add create-dashboard-subscription and generate-insights endpoints (#53253)
* Add create-dashboard-subscription and generate-insights endpoints

* Rename session_id to conversation_id for the tool endpoints
2025-02-07 18:46:40 +03:00
metamben
7badf9005d Merge branch 'master' into metabot-v3-main 2025-02-07 16:39:14 +03:00
metamben
c3d157b9cd Merge branch 'master' into metabot-v3-main 2025-02-07 01:27:05 +03:00
Tamás Benkő
aeaa811281 Fix ns require formatting 2025-02-07 01:26:33 +03:00
metamben
d44226c7fa Merge branch 'master' into metabot-v3-main 2025-02-07 01:24:12 +03:00
Alexander Polyankin
c4578261a8 Fix SQL button tests (#53198) 2025-02-06 11:38:10 -05:00
metamben
ca0352d6ad Merge branch 'master' into metabot-v3-main 2025-02-06 17:03:41 +03:00
metamben
32dc706a39 Merge branch 'master' into metabot-v3-main 2025-02-05 17:40:15 +03:00
metamben
1b44cb8bb9 Implement query-metric tool endpoint (#53195) 2025-02-05 17:40:01 +03:00
Alexander Polyankin
9fa2f59d64 Autofix SQL button (#53153) 2025-02-04 11:32:49 -05:00
metamben
e62bfc2004 Merge branch 'master' into metabot-v3-main 2025-02-04 18:08:04 +03:00
metamben
95726e9d1b Add /api/ee/ai-fix-sql/fix endpoint (#53143)
* Add POST /api/ee/ai-sql-fixer/fix endpoint

* Never prettify the generated SQL
2025-02-04 18:07:06 +03:00
metamben
ec7595557a Merge branch 'master' into metabot-v3-main 2025-02-04 16:20:13 +03:00
metamben
da64908493 Merge branch 'master' into metabot-v3-main 2025-02-04 01:26:31 +03:00
metamben
9bd61f3557 Merge branch 'master' into metabot-v3-main 2025-02-03 21:26:55 +03:00
metamben
49a83e9f41 Merge branch 'master' into metabot-v3-main 2025-02-03 16:38:28 +03:00
metamben
bddc369c0d Merge branch 'master' into metabot-v3-main 2025-02-02 14:31:29 +03:00
metamben
baa6baf893 Merge branch 'master' into metabot-v3-main 2025-02-01 02:43:33 +03:00
metamben
c9fe612a50 Merge branch 'master' into metabot-v3-main 2025-01-31 18:13:39 +03:00
metamben
6480f7b841 Add standalone tests for the invite-user tool (#52984)
* Add standalone tests for the invite-user tool
2025-01-31 18:13:26 +03:00
Tamás Benkő
64fa2f2bc1 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-01-31 18:07:36 +03:00
Tamás Benkő
4c690a44dc Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-01-31 18:07:13 +03:00
metamben
d74631e48b Merge branch 'master' into metabot-v3-main 2025-01-31 00:37:25 +03:00
metamben
c9f5a0e3b6 Merge branch 'master' into metabot-v3-main 2025-01-30 17:57:42 +03:00
metamben
123ec0e340 Add standalone filter-records tests (#52933)
* Add standalone filter-records tests
2025-01-30 17:57:33 +03:00
metamben
3929342ac0 Merge branch 'master' into metabot-v3-main 2025-01-30 00:39:04 +03:00
metamben
ed537dcbc7 Add standalone find-outliers tests (#52906)
* Add standalone find-outliers tests
2025-01-30 00:38:46 +03:00
Tamás Benkő
60b85b61ac Switch to api.macros/defendpoint 2025-01-29 18:23:57 +03:00
metamben
e3347d10a5 Merge branch 'master' into metabot-v3-main 2025-01-29 17:24:25 +03:00
Tamás Benkő
ca0cd2b5a5 Add back React import 2025-01-28 21:53:33 +03:00
metamben
10107fcf5c Merge branch 'master' into metabot-v3-main 2025-01-28 21:39:38 +03:00
Tamás Benkő
42aa717241 Add missing type keyword 2025-01-28 21:37:13 +03:00
Tamás Benkő
cb30ee8a4b Fix wrong merge 2025-01-28 21:30:14 +03:00
metamben
dee6da84cc Merge branch 'master' into metabot-v3-main 2025-01-28 21:16:32 +03:00
metamben
9d6db122ca Add tests for the query-metric tool (#52765)
* Ignore bucketing for non-temporal columns

This was implemented in #52271 but has been reverted by a bad merge.

* Use long names when generating columns

The standard names can easily result in duplicates which will certainly confuse the agent.

* Break long lines

* Add tests for the query-metric tool

* Handle the strange metric-id error cases explicitly
2025-01-28 21:16:14 +03:00
metamben
bdccec2d1f Add tests for the generate-insights tool (#52767) 2025-01-28 21:15:17 +03:00
metamben
653797622b Merge branch 'master' into metabot-v3-main 2025-01-28 16:55:33 +03:00
Tamás Benkő
77fb7c62fa Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-01-27 18:33:01 +03:00
Tamás Benkő
c1eb2559ec Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-01-27 18:32:33 +03:00
Aleksandr Lesnenko
8643fc3aef update import 2025-01-24 16:08:10 -05:00
metamben
67abded9e8 Add default_time_dimension_field_id and adhoc context handling (#52683)
* Add default_time_dimension_field_id and adhoc context handling

* Add show-results-to-user dummy call for adhoc queries in context
2025-01-24 23:33:49 +03:00
Tamás Benkő
dab935167b Fix formatting 2025-01-24 22:54:54 +03:00
metamben
087ad1a2b1 Merge branch 'master' into metabot-v3-main 2025-01-24 22:15:47 +03:00
metamben
c3cbeffc70 Merge branch 'master' into metabot-v3-main 2025-01-24 21:45:31 +03:00
Tamás Benkő
75b33df5bc Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-01-24 20:36:28 +03:00
metamben
b87ce054cb Switch the uberjar build branch to metabot-v3-main (#52678)
* Switch the build branch to metabot-v3-main

* Switch containerize_test_and_push_container too
2025-01-24 20:32:36 +03:00
metamben
89490c9a16 Merge branch 'master' into metabot-v3-main 2025-01-23 22:37:23 +03:00
metamben
9b78a0014f Return a friendly message when the number of round trips is exceeded (#52621)
Fixes #52591.
2025-01-23 22:36:57 +03:00
metamben
74c29161f3 Re-enable the filter-records tool (#52615)
It turns out that a fair number of our test cases rely on this tool, and they worked before, so it might be stable
enough.
2025-01-23 21:01:50 +03:00
metamben
59dcb7f115 Stop checking that parameter names are kebab-case (#52613) 2025-01-23 20:59:15 +03:00
metamben
2f817742e6 Consistently generate and look for query_id and not query-id (#52599) 2025-01-23 18:04:18 +03:00
metamben
be840bb7bd Prevent converting :structured-content to JSON twice (#52556) 2025-01-23 16:54:21 +03:00
metamben
086783dd68 Merge branch 'master' into metabot-v3-main 2025-01-23 16:42:51 +03:00
metamben
3cd107d6d3 Ignore result-columns for saved questions (#52535)
The result-columns of saved pivot questions are a mess, the names and the effective types don't match the values.
Instead of using these columns, we generate the columns from the dataset-query field.

The assumption is that the result-columns for saved questions (but not for models) don't play an important role.
2025-01-23 00:16:29 +03:00
metamben
2d47254583 Disable Metabot's filter-records tool (#52546)
Fixes #52499
2025-01-23 00:16:03 +03:00
Tamás Benkő
ec6033870c Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-01-23 00:05:13 +03:00
metamben
db9e5b1a49 Merge branch 'master' into metabot-v3-main 2025-01-22 23:14:38 +03:00
metamben
96c9346eda Use integers for report_id, strings for table_id (#52528) 2025-01-22 20:28:11 +03:00
metamben
57af6af23a Add standalone test for the create-dashboard-subscription tool (#52519) 2025-01-22 20:27:49 +03:00
metamben
8598fc59d5 Merge branch 'master' into metabot-v3-main 2025-01-22 16:54:50 +03:00
John Swanson
6bcbe2f35a Don't send queries to llm (#51599)
* Don't send queries to the LLM

This introduces `envelope/llm-message`, a private function that formats
messages for the LLM. Currently it does two things: removes `query`s and
json-encodes the structured content.

There's also `envelope/llm-history`, which replaces
`envelope/full-history` by providing the full history appropriate for
the LLM (instead of the full "internal" history for our own use).

* Rework confusing fn

Split this out into a few functions:

- `stringified-content` takes a message and returns the stringified
content (from `structured-content` or `content` as appropriate)

- `stringify-content` takes a message and returns a message with a
stringified `:content` form (and no `:structured-content`)

- `maybe-remove-query` takes a message and removes the `:query` if
necessary

Now `llm-message` just threads the message through `maybe-remove-query`
and then `stringify-content` rather than a confusing `cond->` form.
2025-01-21 21:11:04 +03:00
metamben
57af242b8f Disable invite-user tool (#52450) 2025-01-21 20:35:18 +03:00
metamben
22a3a5fe44 Merge branch 'master' into metabot-v3-main 2025-01-21 18:35:53 +03:00
metamben
8ddeee90f3 Support find-outliers for reports and queries in simple cases (#52414)
When the result has a temporal column, the first in the list is taken as the basis of the analysis.
2025-01-21 17:27:47 +03:00
metamben
74574ed2aa Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-21 00:22:01 +03:00
metamben
d3804ab2bc Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-20 21:11:53 +03:00
metamben
c76e64407b Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-20 19:40:33 +03:00
metamben
7330ca489e Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-20 17:05:33 +03:00
Mihael Konjević
7a0db83be5 Implement redirect reaction for Metabot (#52289)
* Implement redirect reaction for Metabot

Redirect reaction redirects the user to the results page generated by a tool.
Currently `query-metric` and `generate-insights` tools use this reaction.

This also introduces a concept of "terminating" reactions. When a tool returns a
terminating reaction, this tool call result is not sent back to the LLM, instead
it returns the envelope and the frontend handles any sideffects.
2025-01-20 13:26:31 +01:00
metamben
abdfa33583 Ignore archived metrics in the find-metric tool (#52322)
* Ignore archived metrics in the `find-metric` tool

* Use yarn.lock generated by yarn
2025-01-17 22:07:15 +03:00
Tamás Benkő
dee067ffff Merge remote-tracking branch 'origin/master' into metabot-v3-second-try-rebased 2025-01-17 19:33:57 +03:00
Tamás Benkő
8cbea88330 Fix formatting 2025-01-17 18:58:38 +03:00
metamben
d4944b478b Normalize tool error responses (#52230) 2025-01-17 18:02:28 +03:00
metamben
5603565c7c Look for field_granularity instead of field-granularity (#52271)
* Look for field_granularity instead of field-granularity

* Don't report an error when bucketing non-temporal columns

Just ignore the bucket instead.
2025-01-17 17:50:53 +03:00
metamben
fd44da515e Handle dashcards without card when creating subscriptions (#52248) 2025-01-17 17:50:41 +03:00
metamben
56419f6be9 Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-17 16:44:16 +03:00
metamben
43dab70197 Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-16 23:30:08 +03:00
metamben
26bdc4ca18 Distinguish structured and textual tool results (#51698)
* Distinguish structured and textual tool results
2025-01-16 15:58:33 +03:00
metamben
c17bbb8ca3 Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-16 15:57:27 +03:00
metamben
d30c95d30b Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-16 00:41:22 +03:00
metamben
ce8558949a Return metric IDs as plain integers instead of card__<id> (#52059)
See https://metaboat.slack.com/archives/C07SJT1P0ET/p1736237110786299?thread_ts=1735243354.390349&cid=C07SJT1P0ET for context.
2025-01-15 23:52:37 +03:00
Thomas Schmidt
d027f538f1 Update tool spec with examples (#52195)
* Update tool spec with examples

* Support string table_id values in filter-records

---------

Co-authored-by: Tamás Benkő <tamas@metabase.com>
2025-01-15 19:57:43 +03:00
metamben
13a73a6b9a Implement show-results-to-user tool (#52121) 2025-01-14 23:12:42 +03:00
metamben
80c322f8b1 Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-14 16:56:04 +03:00
metamben
acc331c5fb Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-13 21:43:31 +03:00
metamben
4edd922f10 Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-13 21:06:06 +03:00
Tamás Benkő
6108f5d699 Merge remote-tracking branch 'origin/master' into metabot-v3-second-try-rebased 2025-01-13 19:20:40 +03:00
Tamás Benkő
83af40a49d Fix linter errors from wrong merge 2025-01-13 19:19:55 +03:00
Tamás Benkő
71d828be07 Fix require from wrong merge 2025-01-13 18:49:51 +03:00
metamben
129f1ce7f4 Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-13 18:24:23 +03:00
Tamás Benkő
0e0aac3f46 Merge remote-tracking branch 'origin/master' into metabot-v3-second-try-rebased 2025-01-13 18:11:23 +03:00
metamben
474ecbeec5 Properly report unknown users from create-dashboard-subscription (#51662) 2025-01-13 17:41:26 +03:00
metamben
1b3ed70dec Fix typo (#51702) 2024-12-31 15:03:26 +03:00
metamben
91abc51ec5 Merge branch 'master' into metabot-v3-second-try-rebased 2024-12-31 12:37:47 +03:00
John Swanson
1c02aebe07 Find outliers tool (#51534)
* QOL improvements for the REPL namespace

* Find outliers tool

There are also some unrelated changes I threw in here:

- remove a bit of unnecessary serialization of output (we do this
automatically now)

- delete a stale tool that is no longer part of toolbox v1 (I noticed
the LLM trying to call it)

* CR resp

* Userland query

* Userland query *with default constraints*

* kondo
2024-12-27 12:17:41 -08:00
John Swanson
0af736a234 Adjust response format for invite-user (#51536) 2024-12-27 12:17:23 -08:00
metamben
0c3b93131c Generate field_ids by index (#51606)
* Identify fields by their index

* Generate short field IDs

* Truncate columns to day when filtering without time component

* Decode tool history contents when searching for generated queries
2024-12-27 01:04:37 +03:00
metamben
ada0e86826 Merge branch 'master' into metabot-v3-second-try-rebased 2024-12-23 14:33:12 +03:00
Alexander Polyankin
160be30f2e Fix tests 2024-12-22 15:11:14 -05:00
Alexander Polyankin
f6d87b174d Fix tests 2024-12-22 15:10:57 -05:00
metamben
1d8350e6b7 Merge branch 'master' into metabot-v3-second-try-rebased 2024-12-21 02:29:19 +03:00
metamben
88605995e1 Merge branch 'master' into metabot-v3-second-try-rebased 2024-12-20 17:24:51 +03:00
Tamás Benkő
1516add0f2 Handwave FE merge 2024-12-20 00:03:19 +03:00
Tamás Benkő
751c628779 Use metabase.util.json instead of cheshire.core 2024-12-20 00:03:19 +03:00
Tamás Benkő
1c7d841199 Fix bad var ref 2024-12-20 00:03:19 +03:00
Alexander Polyankin
73c8f0e87c Add user_is_viewing to FE->BE context (#51482)
* Add user_is_viewing to FE->BE context

* Use the context provided by the FE

* Fix bad merge

---------

Co-authored-by: Tamás Benkő <tamas@metabase.com>
2024-12-20 00:03:19 +03:00
metamben
5306cc9e37 Send current_user_time in LLM context (#51477) 2024-12-20 00:03:19 +03:00
John Swanson
1545b358dd green CI? (#51459)
* green CI?

* don't swap in test context for dummy tools

* convert non-string tool responses to json

* CR feedback and fix circular dep

* clean up reaction handling in envelope

* fix another test
2024-12-20 00:03:19 +03:00
metamben
2d68c5569e Register generate-insights and invite-user (#51470) 2024-12-20 00:03:19 +03:00
metamben
e0f0fac373 Implement filter-records (#51452)
* Implement filter-data

* Switch dummy tools to new tool parameters

* Rename filter-data to filter-records

* Register find-metric tool

* Select only the fields from table we really need
2024-12-20 00:03:19 +03:00
John Swanson
c98477e607 fix dummy tools (#51457) 2024-12-20 00:03:19 +03:00
John Swanson
974424174c Rework flow to allow for a query store (#51425)
* Rework flow to allow for a query store

Dummy tools need to be able to store queries for later use. Right now
dummy tools work differently than regular tools: they can
only (effectively) return an `output`. They can't change context, they
can't return reactions, etc.

I was imagining that when we wanted to add queries to the query store,
we'd do so in the `envelope`, and probably do this in a functional way
by returning `:queries` or a `:query-id->query` map from the tool call
itself. Then, we'd write a function in `envelope` that would basically
just store this state for later retrieval.

The issue we had is that (before this commit) we don't actually invoke
dummy tools in a way that changes the envelope - we just invoke them as
needed before sending the history to the LLM.

So, this PR reworks things a bit:

- we invoke the dummy tool calls once, immediately after creating the
envelope

- right now, this only means that we store the results of those tool
calls in the env, for later retrieval to send to the LLM when we are
actually making a call. (we don't just store the tool calls in normal
history, because we don't want to return them to the FE when we
respond).

- in the immediate future though, we'll want to *also*, in addition to
adding tool `output` to the response, handle other tool results. One
thing could be changing the context in response to a dummy tool call -
which seems maybe a bit crazy, but on the other hand maybe dummy tool
calls should be able to do anything a regular tool call can do?

- more immediately, we'll probably want to handle a new key from tool
invocations, `:query-id->query-map`, that will store queries for later
retrieval.

* Add a function to find a query

* refactor `invoke-tool` to take entire env

I also changed the response handling so that if you don't return
`:context` it's ignored, rather than associng `nil` onto the existing
env.

* actually use `find-query` in generate-insights
2024-12-20 00:03:19 +03:00
Thomas Schmidt
2df3d84fed Sync tool schemas with ai-service (#51446) 2024-12-20 00:03:19 +03:00
metamben
f1d23fa5d9 Implement run-query (#51420)
* Implement run-query as dummy tool

* Return kebab-cased keys from query-metric tool

The keys will be automatically converted to snake_case in the message.

* Skip context processing of the previous attempt
2024-12-20 00:03:19 +03:00
metamben
de6b854244 Implement query-metrics and get-metric-details tools (#51315)
* Implement ISO day of week extraction (#50475)

* Implement ISO day of week extraction

Fixes #50424

* Support display-name for ISO get-day-of-week exclusion expressions (#50606)

* Support display-name for ISO get-day-of-week exclusion expressions

Fixes #50574

* Support extraction expressions in filter-args-display-name (#50694)

Fixes #50670

* Add get-metric-details and query-metric tools

* Pass history to tool invocations

* Tool implementations (#51244)

* Skeleton for `generate-insights` tool

For now I just implemented the easy things: tables, metrics, and
reports (which I think equal cards on our end?), and totally skipped
`query_id`.

* Pull in all the tools from ai-service

* `get-report-details` and `get-metric-details`

* find metrics using the select metrics endpoint

* reimplement invite-user tool

skipping all confirmation for now

* Update enterprise/backend/src/metabase_enterprise/metabot_v3/tools/generate_insights.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Align table and report details with metric details

* Fix tool JSON schemata

---------

Co-authored-by: John Swanson <john.swanson@metabase.com>
2024-12-20 00:03:19 +03:00
metamben
b6af5b1a2e Implement get-table-details dummy tool (#51143)
* Implement get-table-details dummy tool

* Only return readable tables
2024-12-20 00:03:19 +03:00
metamben
15adc8994c Send auth token via request header instead of the body (#51174)
* Send auth token via request header instead of the body

* Remove instance-info from request body
2024-12-20 00:03:19 +03:00
metamben
da69aaf7d0 Implement dashboard subscription using dummy tools (#51074) 2024-12-20 00:03:19 +03:00
Alexander Polyankin
e1c39f6b5e Add dashboard_id to FE->BE context (#51027) 2024-12-20 00:03:19 +03:00
Sloan Sparger
c76b952c5b [Metabot] Fix e2e failures (#50112)
* force ci run

* test fix

* fix people tests

* revert router changes

* removes force enabling metabot that does not do anything

* disable metabot tests since the token feature was disabled

* change

* change

* fix documentation
2024-12-20 00:03:19 +03:00
Alexander Polyankin
0ad961096b Throw error 2024-12-20 00:03:19 +03:00
Alexander Polyankin
8fac6159ec Hacky multi-tool chain (#50179)
* Hacky multi-tool chain

* Hacky multi-tool chain
2024-12-20 00:03:19 +03:00
Alexander Polyankin
409d7a2251 Add granularity 2024-12-20 00:03:19 +03:00
Alexander Polyankin
325c6a17a1 filter-data and aggregate-data tools (#50168) 2024-12-20 00:03:19 +03:00
Sloan Sparger
6a6312c8e2 [Metabot] UI Access improvements (#49859)
* auto close the metabot input on route changes

* prevents auto-closing metabot if the base path segment is the same

* pr feedback

* makes metabot only visible while the user is signed in

* add test coverage

* fix infinite type instantiation errors
2024-12-20 00:03:19 +03:00
Alexander Polyankin
3a98e2d44f Bring back change-chart-appearance tool (#50104) 2024-12-20 00:03:19 +03:00
Alexander Polyankin
a604ab351b change-query cleanup (#49921) 2024-12-20 00:03:19 +03:00
Sloan Sparger
560f2e1455 [Metabot] Auto-close on route change (#49810)
* auto close the metabot input on route changes

* prevents auto-closing metabot if the base path segment is the same

* pr feedback
2024-12-20 00:03:19 +03:00
Sloan Sparger
a283463d05 [Metabot] Sessions (#49785)
* base e2e tests

* unit tests

* fe implementation for sessions

* BE: Send back a `session_id` if missing

* change to uuid

* send the session id along to ai-service

* pr feedback

* require `session_id` to be passed, and be a UUID

* backend: send a dummy session id to unbreak stats (#49894)

* backend: send a dummy session id to unbreak stats

* preserve session-id across calls

one shortcoming: it does not preserve the session id across FE
invocations. Need to send it back and the FE parrot it back, and the
envelope create function should reuse that when present

```
{
  "messages" : [ {
    "role" : "user",
    "content" : "filter to where the state is CA"
  } ],
  "context" : {...}
  "tools" : [,,,],
  "instance_info" : {,,,},
  "user_id" : 1,
  "session_id" : "6e67dafe-30e6-4b0a-8387-5dc5968e7f28" // <----
}

"LLM -----------------> BE"
{
  "message" : {
    "content" : "",
    "role" : "assistant",
    "tool_calls" : [ {
      "id" : "call_NpA5WEA2gzrRDWt7pC8i0GZR",
      "name" : "change_query",
      "arguments" : "{\"relative_date_filters\":null,\"specific_date_filters\":null,\"boolean_filters\":null,\"breakouts\":null,\"string_filters\":[{\"column\":\"State\",\"operator\":\"=\",\"value\":\"CA\"}],\"order_bys\":null,\"limits\":null,\"number_filters\":null,\"aggregations\":null}"
    } ]
  },
  "metadata" : {
    "usage" : {
      "model" : "gpt-4o-mini-2024-07-18",
      "total" : 7404,
      "prompt" : 7335,
      "completion" : 69,
      "estimated_costs" : 0.0011519999999999998
    }
  }
}

"BE -----------------> LLM"
{
  "messages" : [ {
    "role" : "user",
    "content" : "filter to where the state is CA"
  }, {
    "content" : "",
    "role" : "assistant",
    "tool_calls" : [ {
      "id" : "call_NpA5WEA2gzrRDWt7pC8i0GZR",
      "name" : "change_query",
      "arguments" : "{\"order-bys\":null,\"specific-date-filters\":null,\"breakouts\":null,\"boolean-filters\":null,\"relative-date-filters\":null,\"string-filters\":[{\"column\":\"State\",\"operator\":\"=\",\"value\":\"CA\"}],\"number-filters\":null,\"limits\":null,\"aggregations\":null}"
    } ]
  }, {
    "role" : "tool",
    "tool_call_id" : "call_NpA5WEA2gzrRDWt7pC8i0GZR",
    "content" : "success"
  } ],
  "context" : {},
  "tools" : []..,
  "instance_info" : {,,,}
  },
  "user_id" : 1,
  "session_id" : "6e67dafe-30e6-4b0a-8387-5dc5968e7f28" // <---
}
```

This session_id above is missing in the BE -> FE and then on subsequent FE -> BE payloads

---------

Co-authored-by: dan sutton <dan@dpsutton.com>

* right some wrongs from merge conflict resolutions

* fix

* fix

---------

Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: dan sutton <dan@dpsutton.com>
2024-12-20 00:03:19 +03:00
Sloan Sparger
ecc3e7ba0b [Metabot] Initial e2e + unit tests (#49732)
* base e2e tests

* unit tests

* pr feedback

* run metabot tests

* force enable metabot for testing

* test something in ci

* test something in ci 2

* test something in ci 3

* test something in ci 4

* test something in ci 5

* fix for now
2024-12-20 00:03:19 +03:00
John Swanson
57f58d6009 backend: send a dummy session id to unbreak stats (#49894)
* backend: send a dummy session id to unbreak stats

* preserve session-id across calls

one shortcoming: it does not preserve the session id across FE
invocations. Need to send it back and the FE parrot it back, and the
envelope create function should reuse that when present

```
{
  "messages" : [ {
    "role" : "user",
    "content" : "filter to where the state is CA"
  } ],
  "context" : {...}
  "tools" : [,,,],
  "instance_info" : {,,,},
  "user_id" : 1,
  "session_id" : "6e67dafe-30e6-4b0a-8387-5dc5968e7f28" // <----
}

"LLM -----------------> BE"
{
  "message" : {
    "content" : "",
    "role" : "assistant",
    "tool_calls" : [ {
      "id" : "call_NpA5WEA2gzrRDWt7pC8i0GZR",
      "name" : "change_query",
      "arguments" : "{\"relative_date_filters\":null,\"specific_date_filters\":null,\"boolean_filters\":null,\"breakouts\":null,\"string_filters\":[{\"column\":\"State\",\"operator\":\"=\",\"value\":\"CA\"}],\"order_bys\":null,\"limits\":null,\"number_filters\":null,\"aggregations\":null}"
    } ]
  },
  "metadata" : {
    "usage" : {
      "model" : "gpt-4o-mini-2024-07-18",
      "total" : 7404,
      "prompt" : 7335,
      "completion" : 69,
      "estimated_costs" : 0.0011519999999999998
    }
  }
}

"BE -----------------> LLM"
{
  "messages" : [ {
    "role" : "user",
    "content" : "filter to where the state is CA"
  }, {
    "content" : "",
    "role" : "assistant",
    "tool_calls" : [ {
      "id" : "call_NpA5WEA2gzrRDWt7pC8i0GZR",
      "name" : "change_query",
      "arguments" : "{\"order-bys\":null,\"specific-date-filters\":null,\"breakouts\":null,\"boolean-filters\":null,\"relative-date-filters\":null,\"string-filters\":[{\"column\":\"State\",\"operator\":\"=\",\"value\":\"CA\"}],\"number-filters\":null,\"limits\":null,\"aggregations\":null}"
    } ]
  }, {
    "role" : "tool",
    "tool_call_id" : "call_NpA5WEA2gzrRDWt7pC8i0GZR",
    "content" : "success"
  } ],
  "context" : {},
  "tools" : []..,
  "instance_info" : {,,,}
  },
  "user_id" : 1,
  "session_id" : "6e67dafe-30e6-4b0a-8387-5dc5968e7f28" // <---
}
```

This session_id above is missing in the BE -> FE and then on subsequent FE -> BE payloads

---------

Co-authored-by: dan sutton <dan@dpsutton.com>
2024-12-20 00:03:19 +03:00
Alexander Polyankin
782d167f75 Query tool - no context, fetching metadata on demand (#49812) 2024-12-20 00:03:19 +03:00
John Swanson
a90addd18c Move user_id out of instance_info (#49787) 2024-12-20 00:03:19 +03:00
Aleksandr Lesnenko
9c6a2571d4 merge more viz tools 2024-12-20 00:03:19 +03:00
Aleksandr Lesnenko
2dc9749a64 group into other 2024-12-20 00:03:19 +03:00
Aleksandr Lesnenko
aa4d5180da merge a few viz settings tools into one 2024-12-20 00:03:19 +03:00
Aleksandr Lesnenko
27a1329c00 metabot v3: change series settings tool (#49644)
* change series settings tool

* metabot v3: change column settings tool (#49646)

* change column settings tool

* metabot v3: stacked setting tool (#49649)

* stacked setting tool

* metabot v3: add change goal line tool (#49650)

* add change goal line tool

* y axis range tool (#49651)
2024-12-20 00:03:19 +03:00
Aleksandr Lesnenko
6ed742c176 change axes names tool (#49640) 2024-12-20 00:03:19 +03:00
Aleksandr Lesnenko
888333c654 provide complete visualization settings to llm (#49639) 2024-12-20 00:03:19 +03:00
Alexander Polyankin
b6a9a3d851 Single query tool (#49722)
* MetaBot v3 PoC

* API endpoint

* Fix formatting

* Change tools shape to JSON schema and update API shapes

* initial FE prototype

* FE - update styles, add toggle button for chat, put behind feature flag, PR feedback

* FE - remove previous message history snapshot from being saved on each history entry

* FE - hide behind feature flag

* FE - use history not messages key on agent requests

* Enforce tool naming rules

* Add encoding for reactions

* don't use ::metabot-v3.client/history schema until it exists

put :any in there, we should use the schema when we know what it looks
like

* `defreaction` can be a plain function.

* Remove empty file

* Code cleanup

* Make sure history includes the previous user message

* `:context` should get snake-case encoding for AI Proxy

* Fix encoding of requests => AI Proxy

* Make sure we use :additionalProperties, not :additional_properties

* Fix error with API endpoint and add (mocked) e2e tests for the endpoint

* Some improvements to error handling in client and REPL.

* Add placeholder tool call result messages

* FE - account for schema changes

* Decode api request into internal format

```clojure
api=> (mc/decode [:maybe ::metabot-v3.client.schema/messages]
                 [{:role "user"
                   :content "Invite sloan to metabase"}
                  {:content "Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?"
                   :role "assistant"}]
                 (mtx/transformer
                   {:name :api-request}))
[{:role :user, :content "Invite sloan to metabase"}
 {:content "Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?",
  :role :assistant}]
```

This just ensures that the role gets keywordized.

`:role "user"` -> `:role :user`.

* ensure we decode LLM response

LLM sends back the tool we are using as "invite-user". But our dispatch
stuff needs it to end up as `:metabot.tool/invite-user`. Just a simple
namespace for the keyword.

```shell
❯ echo '{"message":"sloan@metabase.com","context":{},"history":[{"role":"user","content":"Invite sloan to metabase"},{"content":"Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?","role":"assistant"}]}' | http post "localhost:3000/api/ee/metabot-v3/agent" x-api-key:$KEY -pb
{
    "history": [
        {
            "content": "Invite sloan to metabase",
            "role": "user"
        },
        {
            "content": "Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?",
            "role": "assistant"
        },
        {
            "content": "sloan@metabase.com",
            "role": "user"
        },
        {
            "content": "",
            "role": "assistant",
            "tool-calls": [
                {
                    "arguments": {
                        "email": "sloan@metabase.com"
                    },
                    "id": "call_yt9LvJS6bmB2oAekhU94VwqR",
                    "name": "metabot.tool/invite-user"
                }
            ]
        }
    ],
    "reactions": [
        {
            "email": "sloan@metabase.com",
            "type": "metabot.reaction/user-invite-sent"
        }
    ]
}

```

* decode tools

```clojure
api=> (mc/decode ::metabot-v3.client.schema/messages
                 [{:content "",
                   :role "assistant",
                   :tool-calls [{:id "call_Ma3wQEylZIT27t8z4lx8ch2y",
                                 :name "metabot.tool/say-hello",
                                 :arguments {"name" "User", "greeting" "hi"}}]}]
                 (mtx/transformer {:name :api-request}))
[{:content "",
  :role :assistant,
  :tool-calls [{:id "call_Ma3wQEylZIT27t8z4lx8ch2y",
                :name :metabot.tool/say-hello,
                :arguments {"name" "User", "greeting" "hi"}}]}]
```

from the api:

```shell
❯ echo '{"message":"sloan@metabase.com","context":{},"history":[{"role":"user","content":"Invite sloan to metabase"},{"content":"Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?","role":"assistant"}]}' | http post "localhost:3000/api/ee/metabot-v3/agent" x-api-key:$KEY -pb | jq
 '.reactions'
[
  {
    "type": "metabot.reaction/user-invite-sent",
    "email": "sloan@metabase.com"
  }
]
```

and

```
❯ echo '{"message":"hi","context":{},"history":[{"role":"user","content":"hi"},{"content":"","role":"assistant","tool-calls":[{"id":"call_Ma3wQEylZIT27t8z4lx8ch2y","name":"metabot.tool/say-hello","arguments":{"name":"User","greeting":"hi"}}]}]}' | http post "localhost:3000/api/ee/metabot-v3/agent" x-api-key:$KEY -pb | jq '.reactions'
[
  {
    "type": "metabot.reaction/message",
    "message": "Sorry I don't understand that. Could you please provide more context or let me know how I can assist you?",
    "repl/message_color": "green",
    "repl/message_emoji": "🤖"
  }
]
```

* Add tool-call-ids to less recent history items

* Playing around with other tools

* FE - placeholder for demo steps which are ready for their be reactions

* FE - update how metabot can be accessed

* Add `defreactions` for the new reactions

* include comparison operators

* include other tools

* small fixes

* oops - value is a string or number, not an enum

* An actually working schema for filter value

* go to question tool/reaction

* tell the LLM not to change display with filter

* small fixes for malli schemas

* change the question to sales over time

* hardcode the default hosted proxy URL

* correct URL for ai-proxy

* FE - connect fe reaction implementation with real BE reactions

* FE - shorter animation duration

* change Ask Metabot to Metabot request

* hardcode URL in the getter instead of default

oop, `(premium-features/is-hosted?)` isn't available at compile time so
we need to just hardcode this in the getter. This means we can't
actually set the setting now, but that should be fine for now.

* Respond with simple messages after invite/favorite

* Hide any error details for the demo

* simpler error message

* FE - impl loading state

* Improve invite user schema descriptions

* Round trips to the LLM

* Hopefully fix clj-kondo hook test

This is my first time actually touching a hook, so hopefully I didn't
mess it up.

* Fix a few more tests

* Log traffic to `llm-payloads` file

```json
"FE -----------------> BE"
{
  "message" : "invite thomas to metabase",
  "context" : { },
  "history" : [ ]
}

"BE -----------------> LLM"
{
  "messages" : [ {
    "role" : "user",
    "content" : "invite thomas to metabase"
  } ],
  "context" : { },
  "tools" : [ {
    "name" : "apply_visualizations",
    ...
    }]}

"LLM -----------------> BE"
{
  "message" : {
    "content" : "Could you please provide me with Thomas's email address so I can send the invitation to Metabase?",
    "role" : "assistant"
  },
  "metadata" : {
    "usage" : {
      "model" : "gpt-4o-mini-2024-07-18",
      "total" : 1163,
      "prompt" : 1142,
      "completion" : 21,
      "estimated_costs" : 1.8704999999999997E-4
    }
  }
}
...
"BE -----------------> FE"
{
  "reactions" : [ {
    "type" : "metabot.reaction/message",
    "repl/message_color" : "green",
    "repl/message_emoji" : "🤖",
    "message" : "Thomas has been successfully invited to Metabase! If you need anything else, just let me know."
  } ],
  "history" : [ {
    "role" : "user",
    "content" : "invite thomas to metabase"
    ...
```

* Fix the REPL namespace

* FE - multi-line editor, css based animations, error handling, dismissable messages, updated loading state

* max 6 round trips

* Remove unused tools + real invite users

* Oops, fix tool requirements

* fix token feature test

* another small test fix

* oops - eastwood

* delete empty config

* FE - implments context solution so any component or route can set context for metabot, adds handling for change display type tool

* Observability, fix endpoint URL, and fix table viz

* Rework, but turn off, the invite tool

* Don't log tool args/results

* only log paylods in dev

* FE - impl hide/show columns reaction

* check metabot-v3 feature

* add tests for tools on the backend

* FE - make Dan happy

* FE - make Dan happier

* lint

* FE - huge code improvement pass, keep isProcessing state for metabot and await completion of each reaction before processing next, allow reactions to stop further reaction processing in the case of an error

* gate metabot on a different feature for now

* Some documentation

also moved `handle-response` to `handle-envelope`, since that's what
it's doing now.

* add llm-payloads to gitignore (#49505)

* Add back backend support for adding a user

After talking with Sloan, we reworked the reactions here a bit.

The `confirmation` reaction has a `description` and `options` (the
options here means "the options the user can choose between", not
something like "arguments").

The keys of the options are just names for the choices, in this case
"yes" and "no". When the frontend makes the choice, it carries out the
reactions located under that key in the `confirmation` reaction.

* FE - documentation and make context deregistering O(1)

* FE - fix enter key bug

* Include description in column context

* Always send the LLM's messages as reactions

Previously, we would only send the LLM's message as a reaction if there
were no other reactions. I think it makes more sense to send a list
containing:

- all reactions explicitly set by tool call invocations, PLUS

- all messages from the LLM since your last message (formatted as
reaction messages)

* Fix double message on the who's your favorite tool

* Try something with invite user, and fix up setting

* WIP tools testing

* add ? to confirmation description

* fix confirmation flow

* don't send tool call id on tool invocation

* set default ai host to localhost

* exclude metabot-v3 tests from test run

* FE - invite user improvements, implement confirmation + writeback + api call reactions, update users page to use RTKQuery to allow for reactive updates

* oops, fix failing tool tests

* use the real metabot feature

* Add sort-query tool

* Add direction

* Fix formatting

* Fix formatting

* Fix lint

* Add `aggregate_query` tool (#49622)

* aggregate_query tool

* Fix lint

* Add `limit_query` tool (#49629)

* limit_query tool

* Add breakout_query tool (#49631)

* Fix conflicts

* Single tool for queries

* Add grouping

* Add aggregations

* Add aggregations

* Add aggregations

* Add aggregations

* Add filters

* Cleanup

* Cleanup

* Cleanup

* Booleans

* Dates

* Dates

---------

Co-authored-by: Cam Saul <github@camsaul.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: dan sutton <dan@dpsutton.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Maz Ameli <maz@metabase.com>
Co-authored-by: Thomas Schmidt <thomas.heinz.schmidt@gmail.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
2024-12-20 00:03:19 +03:00
Alexander Polyankin
af08a7488c Add sort_query tool (#49619)
* MetaBot v3 PoC

* API endpoint

* Fix formatting

* Change tools shape to JSON schema and update API shapes

* initial FE prototype

* FE - update styles, add toggle button for chat, put behind feature flag, PR feedback

* FE - remove previous message history snapshot from being saved on each history entry

* FE - hide behind feature flag

* FE - use history not messages key on agent requests

* Enforce tool naming rules

* Add encoding for reactions

* don't use ::metabot-v3.client/history schema until it exists

put :any in there, we should use the schema when we know what it looks
like

* `defreaction` can be a plain function.

* Remove empty file

* Code cleanup

* Make sure history includes the previous user message

* `:context` should get snake-case encoding for AI Proxy

* Fix encoding of requests => AI Proxy

* Make sure we use :additionalProperties, not :additional_properties

* Fix error with API endpoint and add (mocked) e2e tests for the endpoint

* Some improvements to error handling in client and REPL.

* Add placeholder tool call result messages

* FE - account for schema changes

* Decode api request into internal format

```clojure
api=> (mc/decode [:maybe ::metabot-v3.client.schema/messages]
                 [{:role "user"
                   :content "Invite sloan to metabase"}
                  {:content "Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?"
                   :role "assistant"}]
                 (mtx/transformer
                   {:name :api-request}))
[{:role :user, :content "Invite sloan to metabase"}
 {:content "Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?",
  :role :assistant}]
```

This just ensures that the role gets keywordized.

`:role "user"` -> `:role :user`.

* ensure we decode LLM response

LLM sends back the tool we are using as "invite-user". But our dispatch
stuff needs it to end up as `:metabot.tool/invite-user`. Just a simple
namespace for the keyword.

```shell
❯ echo '{"message":"sloan@metabase.com","context":{},"history":[{"role":"user","content":"Invite sloan to metabase"},{"content":"Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?","role":"assistant"}]}' | http post "localhost:3000/api/ee/metabot-v3/agent" x-api-key:$KEY -pb
{
    "history": [
        {
            "content": "Invite sloan to metabase",
            "role": "user"
        },
        {
            "content": "Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?",
            "role": "assistant"
        },
        {
            "content": "sloan@metabase.com",
            "role": "user"
        },
        {
            "content": "",
            "role": "assistant",
            "tool-calls": [
                {
                    "arguments": {
                        "email": "sloan@metabase.com"
                    },
                    "id": "call_yt9LvJS6bmB2oAekhU94VwqR",
                    "name": "metabot.tool/invite-user"
                }
            ]
        }
    ],
    "reactions": [
        {
            "email": "sloan@metabase.com",
            "type": "metabot.reaction/user-invite-sent"
        }
    ]
}

```

* decode tools

```clojure
api=> (mc/decode ::metabot-v3.client.schema/messages
                 [{:content "",
                   :role "assistant",
                   :tool-calls [{:id "call_Ma3wQEylZIT27t8z4lx8ch2y",
                                 :name "metabot.tool/say-hello",
                                 :arguments {"name" "User", "greeting" "hi"}}]}]
                 (mtx/transformer {:name :api-request}))
[{:content "",
  :role :assistant,
  :tool-calls [{:id "call_Ma3wQEylZIT27t8z4lx8ch2y",
                :name :metabot.tool/say-hello,
                :arguments {"name" "User", "greeting" "hi"}}]}]
```

from the api:

```shell
❯ echo '{"message":"sloan@metabase.com","context":{},"history":[{"role":"user","content":"Invite sloan to metabase"},{"content":"Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?","role":"assistant"}]}' | http post "localhost:3000/api/ee/metabot-v3/agent" x-api-key:$KEY -pb | jq
 '.reactions'
[
  {
    "type": "metabot.reaction/user-invite-sent",
    "email": "sloan@metabase.com"
  }
]
```

and

```
❯ echo '{"message":"hi","context":{},"history":[{"role":"user","content":"hi"},{"content":"","role":"assistant","tool-calls":[{"id":"call_Ma3wQEylZIT27t8z4lx8ch2y","name":"metabot.tool/say-hello","arguments":{"name":"User","greeting":"hi"}}]}]}' | http post "localhost:3000/api/ee/metabot-v3/agent" x-api-key:$KEY -pb | jq '.reactions'
[
  {
    "type": "metabot.reaction/message",
    "message": "Sorry I don't understand that. Could you please provide more context or let me know how I can assist you?",
    "repl/message_color": "green",
    "repl/message_emoji": "🤖"
  }
]
```

* Add tool-call-ids to less recent history items

* Playing around with other tools

* FE - placeholder for demo steps which are ready for their be reactions

* FE - update how metabot can be accessed

* Add `defreactions` for the new reactions

* include comparison operators

* include other tools

* small fixes

* oops - value is a string or number, not an enum

* An actually working schema for filter value

* go to question tool/reaction

* tell the LLM not to change display with filter

* small fixes for malli schemas

* change the question to sales over time

* hardcode the default hosted proxy URL

* correct URL for ai-proxy

* FE - connect fe reaction implementation with real BE reactions

* FE - shorter animation duration

* change Ask Metabot to Metabot request

* hardcode URL in the getter instead of default

oop, `(premium-features/is-hosted?)` isn't available at compile time so
we need to just hardcode this in the getter. This means we can't
actually set the setting now, but that should be fine for now.

* Respond with simple messages after invite/favorite

* Hide any error details for the demo

* simpler error message

* FE - impl loading state

* Improve invite user schema descriptions

* Round trips to the LLM

* Hopefully fix clj-kondo hook test

This is my first time actually touching a hook, so hopefully I didn't
mess it up.

* Fix a few more tests

* Log traffic to `llm-payloads` file

```json
"FE -----------------> BE"
{
  "message" : "invite thomas to metabase",
  "context" : { },
  "history" : [ ]
}

"BE -----------------> LLM"
{
  "messages" : [ {
    "role" : "user",
    "content" : "invite thomas to metabase"
  } ],
  "context" : { },
  "tools" : [ {
    "name" : "apply_visualizations",
    ...
    }]}

"LLM -----------------> BE"
{
  "message" : {
    "content" : "Could you please provide me with Thomas's email address so I can send the invitation to Metabase?",
    "role" : "assistant"
  },
  "metadata" : {
    "usage" : {
      "model" : "gpt-4o-mini-2024-07-18",
      "total" : 1163,
      "prompt" : 1142,
      "completion" : 21,
      "estimated_costs" : 1.8704999999999997E-4
    }
  }
}
...
"BE -----------------> FE"
{
  "reactions" : [ {
    "type" : "metabot.reaction/message",
    "repl/message_color" : "green",
    "repl/message_emoji" : "🤖",
    "message" : "Thomas has been successfully invited to Metabase! If you need anything else, just let me know."
  } ],
  "history" : [ {
    "role" : "user",
    "content" : "invite thomas to metabase"
    ...
```

* Fix the REPL namespace

* FE - multi-line editor, css based animations, error handling, dismissable messages, updated loading state

* max 6 round trips

* Remove unused tools + real invite users

* Oops, fix tool requirements

* fix token feature test

* another small test fix

* oops - eastwood

* delete empty config

* FE - implments context solution so any component or route can set context for metabot, adds handling for change display type tool

* Observability, fix endpoint URL, and fix table viz

* Rework, but turn off, the invite tool

* Don't log tool args/results

* only log paylods in dev

* FE - impl hide/show columns reaction

* check metabot-v3 feature

* add tests for tools on the backend

* FE - make Dan happy

* FE - make Dan happier

* lint

* FE - huge code improvement pass, keep isProcessing state for metabot and await completion of each reaction before processing next, allow reactions to stop further reaction processing in the case of an error

* gate metabot on a different feature for now

* Some documentation

also moved `handle-response` to `handle-envelope`, since that's what
it's doing now.

* add llm-payloads to gitignore (#49505)

* Add back backend support for adding a user

After talking with Sloan, we reworked the reactions here a bit.

The `confirmation` reaction has a `description` and `options` (the
options here means "the options the user can choose between", not
something like "arguments").

The keys of the options are just names for the choices, in this case
"yes" and "no". When the frontend makes the choice, it carries out the
reactions located under that key in the `confirmation` reaction.

* FE - documentation and make context deregistering O(1)

* FE - fix enter key bug

* Include description in column context

* Always send the LLM's messages as reactions

Previously, we would only send the LLM's message as a reaction if there
were no other reactions. I think it makes more sense to send a list
containing:

- all reactions explicitly set by tool call invocations, PLUS

- all messages from the LLM since your last message (formatted as
reaction messages)

* Fix double message on the who's your favorite tool

* Try something with invite user, and fix up setting

* WIP tools testing

* add ? to confirmation description

* fix confirmation flow

* don't send tool call id on tool invocation

* set default ai host to localhost

* exclude metabot-v3 tests from test run

* FE - invite user improvements, implement confirmation + writeback + api call reactions, update users page to use RTKQuery to allow for reactive updates

* oops, fix failing tool tests

* use the real metabot feature

* Add sort-query tool

* Add direction

* Fix formatting

* Fix formatting

* Fix lint

* Add `aggregate_query` tool (#49622)

* aggregate_query tool

* Fix lint

* Add `limit_query` tool (#49629)

* limit_query tool

* Add breakout_query tool (#49631)

* Fix conflicts

---------

Co-authored-by: Cam Saul <github@camsaul.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: dan sutton <dan@dpsutton.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Maz Ameli <maz@metabase.com>
Co-authored-by: Thomas Schmidt <thomas.heinz.schmidt@gmail.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
2024-12-20 00:03:19 +03:00
John Swanson
8933965e77 Instance info
For now, put `user-id` into the instance info. I'll move this over once
we've added that field to the schema on the `ai-service` side.
2024-12-20 00:03:19 +03:00
Sloan Sparger
d4c0b24558 FE - change error messages 2024-12-20 00:03:19 +03:00
Sloan Sparger
7b34e3c04f FE - copy changes 2024-12-20 00:03:19 +03:00
Sloan Sparger
b498a5ad59 FE - removes the feature from being hardcoded as enabled 2024-12-20 00:03:19 +03:00
John Swanson
4530bb4472 use the real metabot feature 2024-12-20 00:03:19 +03:00
John Swanson
f7bc32c37f oops, fix failing tool tests 2024-12-20 00:03:19 +03:00
Sloan Sparger
b11ace9933 FE - invite user improvements, implement confirmation + writeback + api call reactions, update users page to use RTKQuery to allow for reactive updates 2024-12-20 00:03:19 +03:00
John Swanson
4ac8936dda exclude metabot-v3 tests from test run 2024-12-20 00:03:18 +03:00
John Swanson
da80d576fc set default ai host to localhost 2024-12-20 00:03:18 +03:00
John Swanson
75af27d1ef don't send tool call id on tool invocation 2024-12-20 00:03:18 +03:00
John Swanson
b71f81de3a fix confirmation flow 2024-12-20 00:03:18 +03:00
John Swanson
8ae959af40 add ? to confirmation description 2024-12-20 00:03:18 +03:00
John Swanson
54bea92e33 WIP tools testing 2024-12-20 00:03:18 +03:00
John Swanson
c1cec51039 Try something with invite user, and fix up setting 2024-12-20 00:03:18 +03:00
John Swanson
619faab76a Fix double message on the who's your favorite tool 2024-12-20 00:03:18 +03:00
John Swanson
fa679be3f5 Always send the LLM's messages as reactions
Previously, we would only send the LLM's message as a reaction if there
were no other reactions. I think it makes more sense to send a list
containing:

- all reactions explicitly set by tool call invocations, PLUS

- all messages from the LLM since your last message (formatted as
reaction messages)
2024-12-20 00:03:18 +03:00
Thomas Schmidt
3e151e39ce Include description in column context 2024-12-20 00:03:18 +03:00
Sloan Sparger
2086a5bcee FE - fix enter key bug 2024-12-20 00:03:18 +03:00
Sloan Sparger
94658fdbab FE - documentation and make context deregistering O(1) 2024-12-20 00:03:18 +03:00
John Swanson
c4d52fb574 Add back backend support for adding a user
After talking with Sloan, we reworked the reactions here a bit.

The `confirmation` reaction has a `description` and `options` (the
options here means "the options the user can choose between", not
something like "arguments").

The keys of the options are just names for the choices, in this case
"yes" and "no". When the frontend makes the choice, it carries out the
reactions located under that key in the `confirmation` reaction.
2024-12-20 00:03:18 +03:00
Aleksandr Lesnenko
1444a637e3 add llm-payloads to gitignore (#49505) 2024-12-20 00:03:18 +03:00
John Swanson
fdbbe3e246 Some documentation
also moved `handle-response` to `handle-envelope`, since that's what
it's doing now.
2024-12-20 00:03:18 +03:00
John Swanson
4d722317bf gate metabot on a different feature for now 2024-12-20 00:03:18 +03:00
Sloan Sparger
07fb93c1dd FE - huge code improvement pass, keep isProcessing state for metabot and await completion of each reaction before processing next, allow reactions to stop further reaction processing in the case of an error 2024-12-20 00:03:18 +03:00
John Swanson
3082c28ce6 lint 2024-12-20 00:03:18 +03:00
Sloan Sparger
c43ffdf1f6 FE - make Dan happier 2024-12-20 00:03:18 +03:00
Sloan Sparger
30e1b63230 FE - make Dan happy 2024-12-20 00:03:18 +03:00
John Swanson
35f2719fc7 add tests for tools on the backend 2024-12-20 00:03:18 +03:00
John Swanson
5d2b8b4192 check metabot-v3 feature 2024-12-20 00:03:18 +03:00
Sloan Sparger
dd305e34c2 FE - impl hide/show columns reaction 2024-12-20 00:03:18 +03:00
dan sutton
7c47d4908f only log paylods in dev 2024-12-20 00:03:18 +03:00
John Swanson
718602271b Don't log tool args/results 2024-12-20 00:03:18 +03:00
John Swanson
cbbc868d07 Rework, but turn off, the invite tool 2024-12-20 00:03:18 +03:00
John Swanson
ca5aadc113 Observability, fix endpoint URL, and fix table viz 2024-12-20 00:03:18 +03:00
Sloan Sparger
313f03307a FE - implments context solution so any component or route can set context for metabot, adds handling for change display type tool 2024-12-20 00:03:18 +03:00
John Swanson
e92eba2a34 delete empty config 2024-12-20 00:03:18 +03:00
John Swanson
eefe925be2 oops - eastwood 2024-12-20 00:03:18 +03:00
John Swanson
6ed9935bf0 another small test fix 2024-12-20 00:03:18 +03:00
John Swanson
7032631e79 fix token feature test 2024-12-20 00:03:18 +03:00
John Swanson
b0e5c8f306 Oops, fix tool requirements 2024-12-20 00:03:18 +03:00
John Swanson
06b61e9d11 Remove unused tools + real invite users 2024-12-20 00:03:18 +03:00
John Swanson
74c7533d9f max 6 round trips 2024-12-20 00:03:18 +03:00
Sloan Sparger
eca6d725db FE - multi-line editor, css based animations, error handling, dismissable messages, updated loading state 2024-12-20 00:03:18 +03:00
John Swanson
1ee39a695f Fix the REPL namespace 2024-12-20 00:03:18 +03:00
dan sutton
7d97203f80 Log traffic to llm-payloads file
```json
"FE -----------------> BE"
{
  "message" : "invite thomas to metabase",
  "context" : { },
  "history" : [ ]
}

"BE -----------------> LLM"
{
  "messages" : [ {
    "role" : "user",
    "content" : "invite thomas to metabase"
  } ],
  "context" : { },
  "tools" : [ {
    "name" : "apply_visualizations",
    ...
    }]}

"LLM -----------------> BE"
{
  "message" : {
    "content" : "Could you please provide me with Thomas's email address so I can send the invitation to Metabase?",
    "role" : "assistant"
  },
  "metadata" : {
    "usage" : {
      "model" : "gpt-4o-mini-2024-07-18",
      "total" : 1163,
      "prompt" : 1142,
      "completion" : 21,
      "estimated_costs" : 1.8704999999999997E-4
    }
  }
}
...
"BE -----------------> FE"
{
  "reactions" : [ {
    "type" : "metabot.reaction/message",
    "repl/message_color" : "green",
    "repl/message_emoji" : "🤖",
    "message" : "Thomas has been successfully invited to Metabase! If you need anything else, just let me know."
  } ],
  "history" : [ {
    "role" : "user",
    "content" : "invite thomas to metabase"
    ...
```
2024-12-20 00:03:18 +03:00
John Swanson
53df1cc48f Fix a few more tests 2024-12-20 00:03:18 +03:00
John Swanson
445689bd61 Hopefully fix clj-kondo hook test
This is my first time actually touching a hook, so hopefully I didn't
mess it up.
2024-12-20 00:03:18 +03:00
John Swanson
1afd8db757 Round trips to the LLM 2024-12-20 00:03:18 +03:00
Thomas Schmidt
f36b61d095 Improve invite user schema descriptions 2024-12-20 00:03:18 +03:00
Sloan Sparger
1320442e81 FE - impl loading state 2024-12-20 00:03:18 +03:00
John Swanson
9a5c9f6b43 simpler error message 2024-12-20 00:03:18 +03:00
John Swanson
a77f8d3186 Hide any error details for the demo 2024-12-20 00:03:18 +03:00
John Swanson
d18a41c615 Respond with simple messages after invite/favorite 2024-12-20 00:03:18 +03:00
John Swanson
23e07e333b hardcode URL in the getter instead of default
oop, `(premium-features/is-hosted?)` isn't available at compile time so
we need to just hardcode this in the getter. This means we can't
actually set the setting now, but that should be fine for now.
2024-12-20 00:03:18 +03:00
Maz Ameli
c032343abb change Ask Metabot to Metabot request 2024-12-20 00:03:18 +03:00
Sloan Sparger
85454134b8 FE - shorter animation duration 2024-12-20 00:03:18 +03:00
Sloan Sparger
75a61b3ac1 FE - connect fe reaction implementation with real BE reactions 2024-12-20 00:03:18 +03:00
John Swanson
3855c55a9a correct URL for ai-proxy 2024-12-20 00:03:18 +03:00
John Swanson
7023192f86 hardcode the default hosted proxy URL 2024-12-20 00:03:18 +03:00
John Swanson
e52e7b9ca4 change the question to sales over time 2024-12-20 00:03:18 +03:00
John Swanson
1917280df3 small fixes for malli schemas 2024-12-20 00:03:18 +03:00
John Swanson
6a5ac5a8c5 tell the LLM not to change display with filter 2024-12-20 00:03:18 +03:00
John Swanson
54d966fb98 go to question tool/reaction 2024-12-20 00:03:18 +03:00
John Swanson
89e065a77a An actually working schema for filter value 2024-12-20 00:03:18 +03:00
John Swanson
f46fb69020 oops - value is a string or number, not an enum 2024-12-20 00:03:18 +03:00
John Swanson
e5d0c154e6 small fixes 2024-12-20 00:03:18 +03:00
dan sutton
c937c135db include other tools 2024-12-20 00:03:18 +03:00
dan sutton
0052fdfdec include comparison operators 2024-12-20 00:03:18 +03:00
John Swanson
3847d4067c Add defreactions for the new reactions 2024-12-20 00:03:18 +03:00
Sloan Sparger
ba6d81f561 FE - update how metabot can be accessed 2024-12-20 00:03:18 +03:00
Sloan Sparger
50f3c789b8 FE - placeholder for demo steps which are ready for their be reactions 2024-12-20 00:03:18 +03:00
John Swanson
9c71ce697a Playing around with other tools 2024-12-20 00:03:18 +03:00
John Swanson
ba2c9e9eca Add tool-call-ids to less recent history items 2024-12-20 00:03:18 +03:00
dan sutton
4a61e5a42c decode tools
```clojure
api=> (mc/decode ::metabot-v3.client.schema/messages
                 [{:content "",
                   :role "assistant",
                   :tool-calls [{:id "call_Ma3wQEylZIT27t8z4lx8ch2y",
                                 :name "metabot.tool/say-hello",
                                 :arguments {"name" "User", "greeting" "hi"}}]}]
                 (mtx/transformer {:name :api-request}))
[{:content "",
  :role :assistant,
  :tool-calls [{:id "call_Ma3wQEylZIT27t8z4lx8ch2y",
                :name :metabot.tool/say-hello,
                :arguments {"name" "User", "greeting" "hi"}}]}]
```

from the api:

```shell
❯ echo '{"message":"sloan@metabase.com","context":{},"history":[{"role":"user","content":"Invite sloan to metabase"},{"content":"Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?","role":"assistant"}]}' | http post "localhost:3000/api/ee/metabot-v3/agent" x-api-key:$KEY -pb | jq
 '.reactions'
[
  {
    "type": "metabot.reaction/user-invite-sent",
    "email": "sloan@metabase.com"
  }
]
```

and

```
❯ echo '{"message":"hi","context":{},"history":[{"role":"user","content":"hi"},{"content":"","role":"assistant","tool-calls":[{"id":"call_Ma3wQEylZIT27t8z4lx8ch2y","name":"metabot.tool/say-hello","arguments":{"name":"User","greeting":"hi"}}]}]}' | http post "localhost:3000/api/ee/metabot-v3/agent" x-api-key:$KEY -pb | jq '.reactions'
[
  {
    "type": "metabot.reaction/message",
    "message": "Sorry I don't understand that. Could you please provide more context or let me know how I can assist you?",
    "repl/message_color": "green",
    "repl/message_emoji": "🤖"
  }
]
```
2024-12-20 00:03:18 +03:00
dan sutton
7007095b35 ensure we decode LLM response
LLM sends back the tool we are using as "invite-user". But our dispatch
stuff needs it to end up as `:metabot.tool/invite-user`. Just a simple
namespace for the keyword.

```shell
❯ echo '{"message":"sloan@metabase.com","context":{},"history":[{"role":"user","content":"Invite sloan to metabase"},{"content":"Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?","role":"assistant"}]}' | http post "localhost:3000/api/ee/metabot-v3/agent" x-api-key:$KEY -pb
{
    "history": [
        {
            "content": "Invite sloan to metabase",
            "role": "user"
        },
        {
            "content": "Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?",
            "role": "assistant"
        },
        {
            "content": "sloan@metabase.com",
            "role": "user"
        },
        {
            "content": "",
            "role": "assistant",
            "tool-calls": [
                {
                    "arguments": {
                        "email": "sloan@metabase.com"
                    },
                    "id": "call_yt9LvJS6bmB2oAekhU94VwqR",
                    "name": "metabot.tool/invite-user"
                }
            ]
        }
    ],
    "reactions": [
        {
            "email": "sloan@metabase.com",
            "type": "metabot.reaction/user-invite-sent"
        }
    ]
}

```
2024-12-20 00:03:18 +03:00
dan sutton
d32f8c5940 Decode api request into internal format
```clojure
api=> (mc/decode [:maybe ::metabot-v3.client.schema/messages]
                 [{:role "user"
                   :content "Invite sloan to metabase"}
                  {:content "Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?"
                   :role "assistant"}]
                 (mtx/transformer
                   {:name :api-request}))
[{:role :user, :content "Invite sloan to metabase"}
 {:content "Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?",
  :role :assistant}]
```

This just ensures that the role gets keywordized.

`:role "user"` -> `:role :user`.
2024-12-20 00:03:18 +03:00
Sloan Sparger
a1174e82b1 FE - account for schema changes 2024-12-20 00:03:18 +03:00
Cam Saul
6b500e15f9 Add placeholder tool call result messages 2024-12-20 00:03:18 +03:00
Cam Saul
b02e3c9aa1 Some improvements to error handling in client and REPL. 2024-12-20 00:03:18 +03:00
Cam Saul
6abab13f62 Fix error with API endpoint and add (mocked) e2e tests for the endpoint 2024-12-20 00:03:18 +03:00
Cam Saul
f9919999ff Make sure we use :additionalProperties, not :additional_properties 2024-12-20 00:03:18 +03:00
Cam Saul
22ae538b0c Fix encoding of requests => AI Proxy 2024-12-20 00:03:18 +03:00
Cam Saul
ae093274ff :context should get snake-case encoding for AI Proxy 2024-12-20 00:03:18 +03:00
Cam Saul
ac3a171308 Make sure history includes the previous user message 2024-12-20 00:03:18 +03:00
Cam Saul
de00816b7b Code cleanup 2024-12-20 00:03:18 +03:00
Cam Saul
2c647c0637 Remove empty file 2024-12-20 00:03:18 +03:00
Cam Saul
c4866381a3 defreaction can be a plain function. 2024-12-20 00:03:17 +03:00
dan sutton
749a5e6a56 don't use ::metabot-v3.client/history schema until it exists
put :any in there, we should use the schema when we know what it looks
like
2024-12-20 00:03:17 +03:00
Cam Saul
556acb9b10 Add encoding for reactions 2024-12-20 00:03:17 +03:00
Cam Saul
d60072bcc9 Enforce tool naming rules 2024-12-20 00:03:17 +03:00
Sloan Sparger
d478a79856 FE - use history not messages key on agent requests 2024-12-20 00:03:17 +03:00
Sloan Sparger
6067de5ba7 FE - hide behind feature flag 2024-12-20 00:03:17 +03:00
Sloan Sparger
b98ed219e8 FE - remove previous message history snapshot from being saved on each history entry 2024-12-20 00:03:17 +03:00
Sloan Sparger
d6f4458fba FE - update styles, add toggle button for chat, put behind feature flag, PR feedback 2024-12-20 00:03:17 +03:00
Sloan Sparger
4b5457848f initial FE prototype 2024-12-20 00:03:17 +03:00
Cam Saul
acd4ed01c2 Change tools shape to JSON schema and update API shapes 2024-12-20 00:03:17 +03:00
Cam Saul
06212efee6 Fix formatting 2024-12-20 00:03:17 +03:00
Cam Saul
2c4c399e29 API endpoint 2024-12-20 00:03:17 +03:00
Cam Saul
71ac4f02f1 MetaBot v3 PoC 2024-12-20 00:03:17 +03:00
17599 changed files with 1766355 additions and 2540050 deletions

View File

@@ -1,12 +0,0 @@
Generate a CI failure report for PR $PR_NUM_OR_URL (or current branch if no argument given).
Run `./bin/mage ci-report $PR_NUM_OR_URL` to generate the report, then:
1. **Summarize the failures** - What tests failed and in which jobs?
2. **Categorize** - Are these real failures, flaky tests, or infrastructure issues?
3. **Recommend next steps**:
- If flaky: suggest rerunning with `gh run rerun <run-id> --failed -R metabase/metabase`
- If real: identify the root cause and suggest fixes
- If infrastructure: note the issue type (timeout, OOM, network)
Focus on actionable insights. If all checks are passing, confirm that briefly.

View File

@@ -1 +0,0 @@
Fix flakey test $ARGUMENTS

View File

@@ -1 +0,0 @@
Fix issue $ARGUMENTS

View File

@@ -1 +0,0 @@
Fetch unresolved comments for this branch's PR, then fix them

View File

@@ -1 +0,0 @@
Run mutation testing for $ARGUMENTS

View File

@@ -1 +0,0 @@
Repro issue $ARGUMENTS in a failing test

View File

@@ -1,79 +0,0 @@
## Linting and Formatting
- **Lint PR:** `./bin/mage kondo-updated master` (or whatever target branch)
- Call the command one time at the beginning, record the results, then work through the problems one at a time.
- If the solution is obvious, then please apply the fix. Otherwise skip it.
- If you fix all the issues (and verify by rerunning the kondo-updated command):
- commit the change with a succinct and descriptive commit message
- **Lint File:** `./bin/mage kondo <file or files>`
- Use the linter as a way to know that you are adhering to conventions in place in the codebase
- **Lint Changes:** `./bin/mage kondo-updated HEAD`
- **Format:** `./bin/mage cljfmt-files [path]`
## Testing
Use `./bin/test-agent` to run backend tests. See the top-level CLAUDE.md for usage.
## Code Readability
- **Check Code Readability:** `./bin/mage -check-readable <file> [line-number]`
- Run after every change to Clojure code
- Check specific line first, then entire file if readable
## REPL Usage
> **Note:** If you have `clojure-mcp` tools available (check for tools like `clojure_eval`),
> **always prefer those over `./bin/mage -repl`**. The MCP tools provide better integration,
> richer feedback, and avoid shell escaping issues. Only use `./bin/mage -repl` as a fallback
> when clojure-mcp is not available.
- **Evaluating Clojure Code:** `./bin/mage -repl '<code>'`
- See "Sending Code to the REPL" section for more details
### Sending Code to the REPL
- Send code to the metabase process REPL using: `./bin/mage -repl '(+ 1 1)'` where `(+ 1 1)` is your Clojure code.
- See `./bin/mage -repl -h` for more details.
- If the Metabase backend is not running, you'll see an error message with instructions on how to start it.
#### Working with Files and Namespaces
1. **Load a file and call functions with fully qualified names**:
To call `your.namespace/your-function` on `arg1` and `arg2`:
```
./bin/mage -repl --namespace your.namespace '(your-function arg1 arg2)'
```
DO NOT use "require", "load-file" etc in the code string argument.
#### Understanding the Response
The `./bin/mage -repl` command returns three separate, independent outputs:
- `value`: The return value of the last expression (best for data structures)
- `stdout`: Any printed output from `println` etc. (best for messages)
- `stderr`: Any error messages (best for warnings and errors)
Example call:
```bash
./bin/mage -repl '(println "Hello, world!") '\''({0 1, 1 3, 2 0, 3 2} {0 2, 1 0, 2 3, 3 1})'
```
Example response:
```
ns: user
session: 32a35206-871c-4553-9bc9-f49491173d1c
value: ({0 1, 1 3, 2 0, 3 2} {0 2, 1 0, 2 3, 3 1})
stdout: Hello, world!
stderr:
```
For effective REPL usage:
- Return data structures as function return values
- Use `println` for human-readable messages
- Print errors to stderr

View File

@@ -1,153 +0,0 @@
# Metabase Clojure Style Guide
This guide covers Clojure and ClojureScript coding conventions for Metabase. See also: `CLOJURE_STYLE_GUIDE.adoc` for the Community Clojure Style Guide.
## Naming Conventions
**General Naming:**
- Acceptable abbreviations: `acc`, `i`, `pred`, `coll`, `n`, `s`, `k`, `f`
- Use `kebab-case` for all variables, functions, and constants
**Function Naming:**
- Pure functions should be nouns describing the value they return (e.g., `age` not `calculate-age` or `get-age`)
- Functions with side effects must end with `!`
- Don't repeat namespace alias in function names
**Destructuring:**
- Map destructuring should use kebab-case local bindings even if the map uses `snake_case` keys
## Documentation Standards
**Docstrings:**
- Every public var in `src` or `enterprise/backend/src` must have docstring
- Format using Markdown conventions
- Reference other vars with `[[other-var]]` not backticks
**Comments:**
- `TODO` format: `;; TODO (Name YYYY-MM-DD) -- description`
## Code Organization
**Visibility:**
- Make everything `^:private` unless it is used elsewhere
- Try to organize namespaces to avoid `declare` (put public functions near the end)
**Size and Structure:**
- Break up functions > 20 lines
- Lines ≤ 120 characters
- No blank non-comment lines within definition forms (except pairwise `let`/`cond`)
## Style Conventions
**Keywords and Metadata:**
- Prefer namespaced keywords for internal use: `:query-type/normal` not `:normal`
- Tag variables with `:arglists` metadata if they're functions but wouldn't otherwise have it
## Tests
**Organization:**
- Break large tests into separate `deftest` forms for logically separate test cases
- Test names should end in `-test` or `-test-<number>`
**Performance:**
- Mark pure function tests `^:parallel`
## Modules
**OSS Modules:**
- Follow `metabase.<module>.*` pattern
- Source in `src/metabase/<module>/`
**Enterprise Modules:**
- Follow `metabase-enterprise.<module>.*` pattern
- Source in `enterprise/backend/src/metabase_enterprise/<module>/`
**Module Structure:**
- REST API endpoints go in `<module>.api` or `<module>.api.*` namespaces
- Put module public API in `<module>.core` using Potemkin imports
- Put Toucan models in `<module>.models.*`
- Put settings in `<module>.settings`
- Put schemas in `<module>.schema`
**Module Linters:**
- Do not cheat module linters with `:clj-kondo/ignore [:metabase/modules]`
## REST API Endpoints
**Required Elements:**
- All new endpoints must have response schemas (`:- <schema>` after route string)
- All endpoints need Malli schemas for parameters (detailed and complete)
- All new REST API endpoints MUST HAVE TESTS
**Naming Conventions:**
- Query parameters use kebab-case
- Request bodies use `snake_case`
- Routes use singular nouns (e.g., `/api/dashboard/:id`)
**Behavior:**
- `GET` endpoints should not have side effects (except analytics)
- `defendpoint` forms should be small wrappers around Toucan model code
## MBQL (Metabase Query Language)
**Restrictions:**
- No raw MBQL introspection outside of `lib`, `lib-be`, or `query-processor` modules
- Use Lib and MBQL 5 in new source code; avoid legacy MBQL
## Database and Models
**Naming:**
- Model names and table names should be singular nouns
- Application database uses `snake_case` identifiers
**Best Practices:**
- Use `t2/select-one-fn` instead of fetching entire rows for one column
- Put correct behavior in Toucan methods, not separate helper functions
## Drivers
**Documentation:**
- New driver multimethods must be mentioned in `docs/developers-guide/driver-changelog.md`
**Implementation:**
- Driver implementations should pass `driver` argument to other driver multimethods
- Don't hardcode driver names in implementations
- Minimize logic inside `read-column-thunk` in JDBC-based drivers
## Miscellaneous
**Examples:**
- Example data should be bird-themed if possible
**Linter Suppressions:**
- Use proper format for kondo suppressions
- No `#_:clj-kondo/ignore` (keyword form)
**Configurable Options:**
- Don't define configurable options that can only be set with environment variables
- Use `:internal` `defsetting` instead

View File

@@ -1,8 +0,0 @@
# Autonomous Development Workflow
- Do not attempt to read or edit files outside the project folder
- Add failing tests first, then fix them
- Work autonomously in small, testable increments
- Run targeted tests, and lint continuously during development
- Prioritize understanding existing patterns before implementing
- Don't commit changes, leave it for the user to review and make commits

View File

@@ -1,84 +0,0 @@
# Metabase Writing Style Guide
## Core Principles
Write like you're talking to a colleague. Be conversational, not formal. Get people what they need quickly. Know your audience and match the complexity.
## Tone and Voice
**Do:**
- Use contractions ("can't" not "cannot")
- Say "people" or "companies" instead of "users"
- Be friendly but not peppy
- Acknowledge limitations honestly ("that's on us, not them")
- Jokes and Easter eggs are okay (permit them, don't suggest them)
**Don't:**
- Use exclamation points excessively
- Rely on tired tropes about nerdiness
- Use corporate jargon ("utilize", "offerings", "actionable insights")
- Tell people something is cool (show them instead)
## Structure and Clarity
**Lead with the important stuff:**
- Most important information first
- Lead with the ask, then provide context
- Cut text that adds little value (when in doubt, cut it)
- Each paragraph should have one clear purpose
**Make headings do the work:**
- Convey your actual point, not just the topic
- "Use headings to highlight key points" not "How to write a good heading"
- Use sentence case, no punctuation except question marks
- No links in headings unless entire heading is a link
## Instructions and Examples
**Tell people what to do:**
- Give the action before explaining why
- Put commands in execution order
- Provide context, steps, and reasoning
- Don't describe tasks as "easy" or "simple"
**Answer unasked questions:**
- Include answers to "dumb" questions you had when learning
- Don't literally include the question, just give the answer
## Formatting
**Code vs. UI elements:**
- Backticks for code, variable names, parameters only
- **Bold** for UI elements and labels (e.g., "Click the **Save** button")
- Don't use backticks or quotes for UI elements
## Writing Mechanics
- Limit pronouns when introducing new terms (repeat the term to reinforce it)
- Ampersands only in proper nouns, never as substitute for "and"
## Terminology Preferences
Use familiar terms from tools people already know:
- "Summarize" (like Excel) instead of "aggregate"
- "Take a look at" instead of "reference"
- "Filter" instead of technical database terms when appropriate
## Red Flags
Avoid these patterns:
- Multiple exclamation points
- Linking "here"
- Bullet lists to explain (use prose)
- Numbers that will change (guard against change)
- Describing things as "easy" or "simple"
- Stock photography-worthy content

View File

@@ -1,21 +0,0 @@
## Linting and Formatting
- **Lint:** `bun run lint-eslint-pure`
- Run ESLint on the codebase
- **Format:** `bun run prettier`
- Format code using Prettier
- **Type Check:** `bun run type-check-pure`
- Run TypeScript type checking
## Testing
### JavaScript/TypeScript Tests
- **Test a specific file:** `bun run test-unit-keep-cljs path/to/file.unit.spec.js`
- **Test by pattern:** `bun run test-unit-keep-cljs -t "pattern"`
- Runs tests matching the given pattern
### ClojureScript Tests
- **Test ClojureScript:** `bun run test-cljs`
- Run ClojureScript tests

View File

@@ -1,420 +0,0 @@
---
name: add-malli-schemas
description: Efficiently add Malli schemas to API endpoints in the Metabase codebase with proper patterns, validation timing, and error handling
---
# Add Malli Schemas to API Endpoints
This skill helps you efficiently and uniformly add Malli schemas to API endpoints in the Metabase codebase.
## Reference Files (Best Examples)
- `src/metabase/warehouses/api.clj` - Most comprehensive schemas, custom error messages
- `src/metabase/api_keys/api.clj` - Excellent response schemas
- `src/metabase/collections/api.clj` - Great named schema patterns
- `src/metabase/timeline/api/timeline.clj` - Clean, simple examples
## Quick Checklist
When adding Malli schemas to an endpoint:
- [ ] Route params have schemas
- [ ] Query params have schemas with `:optional true` and `:default` where appropriate
- [ ] Request body has a schema (for POST/PUT)
- [ ] Response schema is defined (using `:-` after route string)
- [ ] Use existing schema types from `ms` namespace when possible
- [ ] Consider creating named schemas for reusable or complex types
- [ ] Add contextual error messages for validation failures
## Basic Structure
### Complete Endpoint Example
```clojure
(mr/def ::Color [:enum "red" "blue" "green"])
(mr/def ::ResponseSchema
[:map
[:id pos-int?]
[:name string?]
[:color ::Color]
[:created_at ms/TemporalString]])
(api.macros/defendpoint :post "/:name" :- ::ResponseSchema
"Create a resource with a given name."
[;; Route Params:
{:keys [name]} :- [:map [:name ms/NonBlankString]]
;; Query Params:
{:keys [include archived]} :- [:map
[:include {:optional true} [:maybe [:= "details"]]]
[:archived {:default false} [:maybe ms/BooleanValue]]]
;; Body Params:
{:keys [color]} :- [:map [:color ::Color]]
]
;; endpoint implementation, ex:
{:id 99
:name (str "mr or mrs " name)
:color ({"red" "blue" "blue" "green" "green" "red"} color)
:created_at (t/format (t/formatter "yyyy-MM-dd'T'HH:mm:ssXXX") (t/zoned-date-time))}
)
```
## Common Schema Patterns
1. Route Params (the 5 in `api/user/id/5`)
2. Query Params (the sort+asc pair in `api/users?sort=asc`)
3. Body Params (the contents of a request body. Almost always decoded from json into edn)
4. The Raw Request map
Of the 4 arguments, deprioritize usage of the raw request unless necessary.
### Route Params
Always required, typically just a map with an ID:
```clojure
[{:keys [id]} :- [:map [:id ms/PositiveInt]]]
```
For multiple route params:
```clojure
[{:keys [id field-id]} :- [:map
[:id ms/PositiveInt]
[:field-id ms/PositiveInt]]]
```
### Query Params
Add properties for `{:optional true ...}` and `:default` values:
```clojure
{:keys [archived include limit offset]} :- [:map
[:archived {:default false} [:maybe ms/BooleanValue]]
[:include {:optional true} [:maybe [:= "tables"]]]
[:limit {:optional true} [:maybe ms/PositiveInt]]
[:offset {:optional true} [:maybe ms/PositiveInt]]]
```
### Request Body (POST/PUT)
```clojure
{:keys [name description parent_id]} :- [:map
[:name ms/NonBlankString]
[:description {:optional true} [:maybe ms/NonBlankString]]
[:parent_id {:optional true} [:maybe ms/PositiveInt]]]
```
### Response Schemas
#### Simple inline response:
```clojure
(api.macros/defendpoint :get "/:id" :- [:map
[:id pos-int?]
[:name string?]]
"Get a thing"
...)
```
#### Named schema for reuse:
```clojure
(mr/def ::Thing
[:map
[:id pos-int?]
[:name string?]
[:description [:maybe string?]]])
(api.macros/defendpoint :get "/:id" :- ::Thing
"Get a thing"
...)
(api.macros/defendpoint :get "/" :- [:sequential ::Thing]
"Get all things"
...)
```
## Common Schema Types
### From `metabase.util.malli.schema` (aliased as `ms`)
Prefer the schemas in the ms/* namespace, since they work better with our api infrastructure.
For example use `ms/PositiveInt` instead of `pos-int?`.
```clojure
ms/PositiveInt ;; Positive integer
ms/NonBlankString ;; Non-empty string
ms/BooleanValue ;; String "true"/"false" or boolean
ms/MaybeBooleanValue ;; BooleanValue or nil
ms/TemporalString ;; ISO-8601 date/time string (for REQUEST params only!)
ms/Map ;; Any map
ms/JSONString ;; JSON-encoded string
ms/PositiveNum ;; Positive number
ms/IntGreaterThanOrEqualToZero ;; 0 or positive
```
**IMPORTANT:** For response schemas, use `:any` for temporal fields, not `ms/TemporalString`!
Response schemas validate BEFORE JSON serialization, so they see Java Time objects.
### Built-in Malli Types
```clojure
:string ;; Any string
:boolean ;; true/false
:int ;; Any integer
:keyword ;; Clojure keyword
pos-int? ;; Positive integer predicate
[:maybe X] ;; X or nil
[:enum "a" "b" "c"] ;; One of these values
[:or X Y] ;; Schema that satisfies X or Y
[:and X Y] ;; Schema that satisfies X and Y
[:sequential X] ;; Sequential of Xs
[:set X] ;; Set of Xs
[:map-of K V] ;; Map with keys w/ schema K and values w/ schema V
[:tuple X Y Z] ;; Fixed-length tuple of schemas X Y Z
```
Avoid using sequence schemas unless completely necessary.
## Step-by-Step: Adding Schemas to an Endpoint
### Example: Adding return schema to `GET /api/field/:id/related`
**Before:**
```clojure
(api.macros/defendpoint :get "/:id/related"
"Return related entities."
[{:keys [id]} :- [:map [:id ms/PositiveInt]]]
(-> (t2/select-one :model/Field :id id) api/read-check xrays/related))
```
**Step 1:** Check what the function returns (look at `xrays/related`)
**Step 2:** Define response schema based on return type:
```clojure
(mr/def ::RelatedEntity
[:map
[:tables [:sequential [:map [:id pos-int?] [:name string?]]]]
[:fields [:sequential [:map [:id pos-int?] [:name string?]]]]])
```
**Step 3:** Add response schema to endpoint:
```clojure
(api.macros/defendpoint :get "/:id/related" :- ::RelatedEntity
"Return related entities."
[{:keys [id]} :- [:map [:id ms/PositiveInt]]]
(-> (t2/select-one :model/Field :id id) api/read-check xrays/related))
```
## Advanced Patterns
### Custom Error Messages
```clojure
(def DBEngineString
"Schema for a valid database engine name."
(mu/with-api-error-message
[:and
ms/NonBlankString
[:fn
{:error/message "Valid database engine"}
#(u/ignore-exceptions (driver/the-driver %))]]
(deferred-tru "value must be a valid database engine.")))
```
### Enum with Documentation
```clojure
(def PinnedState
(into [:enum {:error/message "pinned state must be 'all', 'is_pinned', or 'is_not_pinned'"}]
#{"all" "is_pinned" "is_not_pinned"}))
```
### Complex Nested Response
```clojure
(mr/def ::DashboardQuestionCandidate
[:map
[:id ms/PositiveInt]
[:name ms/NonBlankString]
[:description [:maybe string?]]
[:sole_dashboard_info
[:map
[:id ms/PositiveInt]
[:name ms/NonBlankString]
[:description [:maybe string?]]]]])
(mr/def ::DashboardQuestionCandidatesResponse
[:map
[:data [:sequential ::DashboardQuestionCandidate]]
[:total ms/PositiveInt]])
```
### Paginated Response Pattern
```clojure
(mr/def ::PaginatedResponse
[:map
[:data [:sequential ::Item]]
[:total integer?]
[:limit {:optional true} [:maybe integer?]]
[:offset {:optional true} [:maybe integer?]]])
```
## Common Pitfalls
### Don't: Forget `:maybe` for nullable fields
```clojure
[:description ms/NonBlankString] ;; WRONG - fails if nil
[:description [:maybe ms/NonBlankString]] ;; RIGHT - allows nil
```
### Don't: Forget `:optional true` for optional query params
```clojure
[:limit ms/PositiveInt] ;; WRONG - required but shouldn't be
[:limit {:optional true} [:maybe ms/PositiveInt]] ;; RIGHT
```
### Don't: Forget `:default` values for known params
```clojure
[:limit ms/PositiveInt] ;; WRONG - required but shouldn't be
[:limit {:optional true :default 0} [:maybe ms/PositiveInt]] ;; RIGHT
```
### Don't: Mix up route params, query params, and body
```clojure
;; WRONG - all in one map
[{:keys [id name archived]} :- [:map ...]]
;; RIGHT - separate destructuring
[{:keys [id]} :- [:map [:id ms/PositiveInt]]
{:keys [archived]} :- [:map [:archived {:default false} ms/BooleanValue]]
{:keys [name]} :- [:map [:name ms/NonBlankString]]]
```
### Don't: Use `ms/TemporalString` for Java Time objects in response schemas
```clojure
;; WRONG - Java Time objects aren't strings yet
[:date_joined ms/TemporalString]
;; RIGHT - schemas validate BEFORE JSON serialization
[:date_joined :any] ;; Java Time object, serialized to string by middleware
[:last_login [:maybe :any]] ;; Java Time object or nil
```
**Why:** Response schemas validate the internal Clojure data structures BEFORE they are serialized to JSON. Java Time objects like `OffsetDateTime` get converted to ISO-8601 strings by the JSON middleware, so the schema needs to accept the raw Java objects.
### Don't: Use `[:sequential X]` when the data is actually a set
```clojure
;; WRONG - group_ids is actually a set
[:group_ids {:optional true} [:sequential pos-int?]]
;; RIGHT - matches the actual data structure
[:group_ids {:optional true} [:maybe [:set pos-int?]]]
```
**Why:** Toucan hydration methods often return sets. The JSON middleware will serialize sets to arrays, but the schema validates before serialization.
### Don't: Create anonymous schemas for reused structures
Use `mr/def` for schemas used in multiple places:
```clojure
(mr/def ::User
[:map
[:id pos-int?]
[:email string?]
[:name string?]])
```
## Finding Return Types
1. **Look at the function being called**
```clojure
(api.macros/defendpoint :get "/:id"
[{:keys [id]}]
(t2/select-one :model/Field :id id)) ;; Returns a Field instance
```
2. **Check Toucan models for structure**
Look in `src/metabase/*/models/*.clj` for model definitions.
3. **Use clojure-mcp or REPL to inspect**
```bash
./bin/mage -repl '(require '\''metabase.xrays.core) (doc metabase.xrays.core/related)'
```
4. **Check tests**
Tests often show the expected response structure.
## Understanding Schema Validation Timing
**CRITICAL CONCEPT:** Schemas validate at different points in the request/response lifecycle:
### Request Parameter Schemas (Query/Body/Route)
- Validate AFTER JSON parsing
- Data is already deserialized (strings, numbers, booleans)
- Use `ms/TemporalString` for date/time inputs
- Use `ms/BooleanValue` for boolean query params
### Response Schemas
- Validate BEFORE JSON serialization
- Data is still in Clojure format (Java Time objects, sets, keywords)
- Use `:any` for Java Time objects
- Use `[:set X]` for sets
- Use `[:enum :keyword]` for keyword enums
### Serialization Flow
```
Request: JSON string → Parse → Coerce → Handler
Response: Handler → Schema Check → Encode → Serialize → JSON string
```
## Workflow Summary
1. **Read the endpoint** - understand what it does
2. **Identify params** - route, query, body
3. **Add parameter schemas** - use existing types from `ms`
4. **Determine return type** - check the implementation
5. **Define response schema** - inline or named with `mr/def`
6. **Test** - ensure the endpoint works and validates correctly
## Testing Your Schemas
After adding schemas, verify:
1. **Valid requests work** - test with correct data
2. **Invalid requests fail gracefully** - test with wrong types
3. **Optional params work** - test with/without optional params
4. **Error messages are clear** - check validation error responses
## Tips
- **Start simple** - begin with basic types, refine later
- **Reuse schemas** - if you see the same structure twice, make it a named schema
- **Be specific** - use `ms/PositiveInt` instead of `pos-int?`
- **Document intent** - add docstrings to named schemas
- **Follow conventions** - look at similar endpoints in the same namespace
- **Check the actual data** - use REPL to inspect what's actually returned before serialization
## Additional Resources
- [Malli Documentation](https://github.com/metosin/malli)
- Metabase Malli utilities: `src/metabase/util/malli/schema.clj`
- Metabase schema registry: `src/metabase/util/malli/registry.clj`

View File

@@ -1,357 +0,0 @@
--b-
name: add-tracing
description: Add OpenTelemetry tracing spans to Clojure code following Metabase tracing conventions. Use when instrumenting backend code with trace coverage.
---
# Add Tracing Spans to Clojure Code
This skill helps you add OpenTelemetry (OTel) tracing spans to the Metabase backend codebase using the custom `tracing/with-span` macro.
## Reference Files
- `src/metabase/tracing/core.clj` - `with-span` macro, group registry, SDK lifecycle, `best-effort-sanitize-sql`, Pyroscope integration
- `src/metabase/task/impl.clj` - `defjob` macro that wraps Quartz jobs with root spans
- `.clj-kondo/config/modules/config.edn` - Module boundary configuration
## Module Architecture
The tracing module has a deliberately minimal API surface. **Only 2 namespaces are public** (listed in `:api` in the module config):
| Namespace | Role | Status |
|---|---|---|
| `tracing.core` | Primary API: `with-span`, groups, SDK lifecycle, Pyroscope, MDC, `best-effort-sanitize-sql` | **Public API** |
| `tracing.init` | Side-effect loader for `quartz` and `settings` | **Public API** (init convention) |
| `tracing.attributes` | `best-effort-sanitize-sql` implementation (re-exported via `tracing.core`) | Internal |
| `tracing.settings` | Setting definitions (`MB_TRACING_*` env vars) | Internal |
| `tracing.quartz` | Quartz JDBC proxy + JobListener | Internal |
**Rules:**
- Only require `[metabase.tracing.core :as tracing]` from outside the module. `tracing/best-effort-sanitize-sql` and all other public functions are available from this single namespace.
- Do not add new API namespaces. Add new public functions to `tracing.core` instead.
- Do not require internal namespaces (`tracing.attributes`, `tracing.settings`, `tracing.quartz`) from outside the module.
- `:uses :any` on the `core` module does NOT bypass the target module's `:api` check — internal namespaces are still enforced.
### Cyclic Dependency Avoidance
`tracing/core.clj` is required by many modules across the codebase. It **must NOT** compile-time require `tracing.settings`, as this creates transitive cyclic load dependencies (e.g., `settings/core -> tracing/settings -> tracing/core -> events/impl -> events/core`).
Instead, `tracing/core.clj` uses `requiring-resolve` for settings access:
```clojure
;; CORRECT — lazy runtime resolution, no compile-time dependency
((requiring-resolve 'metabase.tracing.settings/tracing-enabled))
;; WRONG — creates cyclic load dependency
(require '[metabase.tracing.settings :as settings])
(settings/tracing-enabled)
```
External library namespaces (clj-otel API, SDK, exporters) are safe to require normally — they don't participate in Metabase namespace cycles.
**Important:** `requiring-resolve` must use **literal quoted symbols**. Kondo hooks validate that `required-namespaces` are all simple symbols, so dynamic construction fails:
```clojure
;; CORRECT — literal quoted symbol
(requiring-resolve 'metabase.tracing.settings/tracing-endpoint)
;; WRONG — kondo hook rejects this: "Assert failed: (every? simple-symbol? required-namespaces)"
(requiring-resolve (symbol "metabase.tracing.settings" "tracing-endpoint"))
```
## Quick Checklist
When adding tracing spans:
- [ ] Module has `tracing` in its `:uses` set in `.clj-kondo/config/modules/config.edn`
- [ ] Added `[metabase.tracing.core :as tracing]` to ns requires (alphabetically sorted)
- [ ] Span wraps a meaningful I/O boundary (not pure computation)
- [ ] Group matches the domain (check `src/metabase/tracing/core.clj` for registered groups; add a new one if none fit)
- [ ] Span name follows dot-notation convention (`"domain.subsystem.operation"`)
- [ ] Attributes use namespaced keywords (`:search/query-length`, `:db/id`)
- [ ] No sensitive data in attributes (use `best-effort-sanitize-sql` for HoneySQL, never raw SQL)
- [ ] No new tracing namespaces created (add to `tracing.core` instead)
- [ ] No `DO_NOT_ADD_NEW_FILES_HERE.txt` violations in the target directory
- [ ] Run `clj-kondo --lint <files>` to verify 0 errors, 0 warnings
- [ ] Add or update tests in the corresponding `test/` path (see Testing section below)
- [ ] Run tests: `clojure -X:dev:test :only <test-ns>`
## The `with-span` Macro
```clojure
(tracing/with-span group span-name attrs & body)
```
- **group** - A keyword selecting which trace group this span belongs to (e.g., `:tasks`, `:sync`)
- **span-name** - A string identifying the span in traces (e.g., `"search.execute"`)
- **attrs** - A map of span attributes (e.g., `{:db/id 42}`)
- **body** - The code to execute inside the span
**When disabled:** zero overhead -- single atom deref + boolean check, body runs directly.
**When enabled:** creates OTel span AND injects `trace_id`/`span_id` into Log4j2 MDC for log-to-trace correlation.
## Trace Groups
Groups are registered in `src/metabase/tracing/core.clj`. Check that file for the current list. The general rule: **match the group to the domain, not the call site.** If code runs inside a Quartz job but is logically search work, use `:search`, not `:tasks`.
To add a new group:
```clojure
;; In src/metabase/tracing/core.clj
(register-group! :my-domain "Description of what this covers")
```
Users enable groups via `MB_TRACING_GROUPS=tasks,search,sync` (comma-separated, or `"all"`).
## Naming Conventions
### Span Names
Use dot-separated hierarchical names: `"domain.subsystem.operation"`. The domain prefix should match the group name:
```
search.execute -- `:search` group
sync.fingerprint.table -- `:sync` group
task.session-cleanup.delete -- `:tasks` group
db-app.collection-items -- `:db-app` group
```
### Attributes
Use namespaced keywords. The namespace groups related attributes:
```clojure
:db/id -- Database ID (integer)
:db/engine -- Database engine name (string)
:db/statement -- Sanitized SQL (string, via best-effort-sanitize-sql)
:search/engine -- Search engine name (string)
:search/query-length -- Query string length (integer)
:sync/table -- Table name (string)
:sync/step -- Sync step name (string)
:task/name -- Task name (string)
:http/method -- HTTP method (string)
:http/url -- Request URL (string)
```
Invent new namespaced attributes as needed (e.g., `:pulse/id`, `:transform/count`). Keep values as primitives (strings, numbers, booleans) -- no maps or collections.
## Step-by-Step: Adding a Span
### 1. Check module boundaries
Look up the module for your namespace in `.clj-kondo/config/modules/config.edn`. If `tracing` is not in the module's `:uses` set, add it (keep alphabetically sorted):
```edn
my-module
{:team "MyTeam"
:uses #{analytics config tracing util}}
```
### 2. Add the require
```clojure
(ns metabase.my-module.thing
(:require
[metabase.tracing.core :as tracing]
[metabase.util :as u]))
```
`best-effort-sanitize-sql` is available from `tracing.core` — no additional require needed.
### 3. Identify the I/O boundary
Only wrap code at **meaningful I/O boundaries**:
**DO trace:**
- External API calls (embedding APIs, metabot, webhooks)
- Database queries (both app DB and user DB)
- Network requests (HTTP calls to external services)
- Heavy batch processing (batch indexing, batch embedding)
- Top-level orchestration functions that coordinate multiple sub-operations
**DO NOT trace:**
- Pure computation (sorting, filtering, mapping)
- Simple single-row lookups (`t2/select-one :model/Setting :key k`)
- Every function in a call chain (only boundaries matter)
- Trivial operations (string formatting, hash calculations)
### 4. Wrap with `with-span`
```clojure
;; Simple span (no attributes needed)
(tracing/with-span :search "search.init-index" {}
(do-expensive-thing))
;; Span with static attributes
(tracing/with-span :sync "sync.fingerprint.table"
{:db/id (:db_id table)
:sync/table (:name table)}
(fingerprint-fields! table fields))
;; Span with computed attributes
(tracing/with-span :search "search.execute"
{:search/engine (name (:search-engine ctx))
:search/query-length (count (:search-string ctx))}
(search.engine/results ctx))
;; Span with sanitized SQL (for dynamic HoneySQL queries)
(let [hsql {:delete-from [(t2/table-name :model/Session)]
:where [:< :created_at oldest-allowed]}]
(tracing/with-span :tasks "task.session-cleanup.delete"
{:db/statement (tracing/best-effort-sanitize-sql hsql)}
(t2/query-one hsql)))
;; Sub-spans breaking a function into I/O phases
(let [embedding (tracing/with-span :search "search.semantic.embedding"
{:search.semantic/provider (:provider model)}
(get-embedding model search-string))
results (tracing/with-span :search "search.semantic.db-query" {}
(into [] xform reducible))]
(process results))
;; Per-item iteration spans
(doseq [e (search.engine/active-engines)]
(tracing/with-span :search "search.ingestion.update" {:search/engine (name e)}
(search.engine/update! e batch)))
```
### 5. Add tests
Create or update tests in the corresponding `test/` path. Follow the patterns in existing tracing tests:
- **Reference tests:** `test/metabase/tracing/quartz_test.clj`, `test/metabase/server/middleware/trace_test.clj`
- Use `tracing/init-enabled-groups!` / `tracing/shutdown-groups!` with `try`/`finally` to manage group lifecycle
- Test both enabled and disabled paths (verify zero overhead when group is off)
- Use `reify` mocks for Java interfaces (Connection, PreparedStatement, JobListener, etc.)
- Add `(set! *warn-on-reflection* true)` and type-hint proxy/reify calls to avoid reflection warnings
```clojure
(deftest my-span-enabled-test
(testing "when group is enabled, span is created"
(try
(tracing/init-enabled-groups! "my-group" "INFO")
;; ... test that span behavior occurs ...
(finally
(tracing/shutdown-groups!)))))
(deftest my-span-disabled-test
(testing "when group is disabled, code runs without tracing"
(tracing/shutdown-groups!)
;; ... test that code still works, no wrapping applied ...
))
```
### 6. Lint and run tests
```bash
# Lint modified source and test files — expect 0 errors, 0 warnings
clj-kondo --lint path/to/modified/file.clj path/to/test/file.clj
# Run tests (requires Java 21+)
clojure -X:dev:test :only my-ns.test-ns
```
Expect: all tests pass, 0 failures, 0 errors, no reflection warnings from your files.
## Sanitizing SQL for Attributes
When including SQL in span attributes, **always** use `tracing/best-effort-sanitize-sql`. This converts HoneySQL maps to parameterized SQL strings where values become `?` placeholders -- no data leaks.
```clojure
(let [hsql {:delete-from [:core_session]
:where [:< :created_at some-timestamp]}]
(tracing/with-span :tasks "task.cleanup.delete"
{:db/statement (tracing/best-effort-sanitize-sql hsql)}
(t2/query-one hsql)))
;; Trace attribute: db/statement = "DELETE FROM core_session WHERE created_at < ?"
```
**Rules:**
- Never put raw SQL strings or user-provided values in attributes
- Use `best-effort-sanitize-sql` only for app DB (HoneySQL) queries
- For external/user DB queries, trace only timing and counts, not SQL content
## Defjob and Root Spans
The `defjob` macro in `metabase.task.impl` automatically wraps every Quartz job with a `:tasks` root span:
```clojure
(task/defjob ^{DisallowConcurrentExecution true} SessionCleanup [_]
(cleanup-sessions!))
;; Automatically creates span: "task.SessionCleanup" {:task/name "SessionCleanup"}
```
You do NOT need a root span inside `defjob` bodies. Add **child spans** for I/O inside the job.
For code on plain `Thread`s (not Quartz), add the root span manually:
```clojure
(defn init! []
(tracing/with-span :search "search.task.init" {}
(search/init-index!)))
```
## What NOT to Do
### Span Usage Mistakes
```clojure
;; WRONG - pure computation, no I/O
(tracing/with-span :search "search.format-results" {}
(map format-result results))
;; WRONG - trivial single-row lookup
(tracing/with-span :db-app "db-app.get-setting" {}
(t2/select-one :model/Setting :key "my-setting"))
;; WRONG - raw SQL in attributes (data leak)
(tracing/with-span :tasks "task.cleanup" {:db/statement raw-sql-string}
(execute! raw-sql-string))
;; WRONG - wrong group (search work should use :search, not :tasks)
(tracing/with-span :tasks "search.execute" {} ...)
;; WRONG - redundant nesting (do-search already has a span)
(tracing/with-span :search "search.process" {}
(let [results (do-search ctx)]
(tracing/with-span :search "search.format" {}
(format-results results))))
```
### Architecture Mistakes
```clojure
;; WRONG - creating a new tracing namespace
(ns metabase.tracing.my-feature ...)
;; WRONG - requiring internal tracing namespaces from outside the module
(ns metabase.my-module.thing
(:require [metabase.tracing.attributes :as trace-attrs] ;; internal!
[metabase.tracing.settings :as tracing.settings] ;; internal!
[metabase.tracing.quartz :as tracing.quartz])) ;; internal!
;; WRONG - adding compile-time requires to tracing/core.clj for settings or SDK
;; This creates cyclic load dependencies because tracing/core is widely required
(ns metabase.tracing.core
(:require [metabase.tracing.settings :as settings])) ;; causes cycle!
;; WRONG - dynamic symbol construction with requiring-resolve (kondo rejects it)
(requiring-resolve (symbol "metabase.tracing.settings" "tracing-enabled"))
```
## Configuration
All settings are env-var-only (defined in `src/metabase/tracing/settings.clj`):
```bash
# Core
MB_TRACING_ENABLED=true # Enable tracing (default: false)
MB_TRACING_ENDPOINT=host:4317 # OTLP collector endpoint (default: http://localhost:4317)
MB_TRACING_GROUPS=tasks,search,sync # Comma-separated groups or "all" (default: all)
MB_TRACING_SERVICE_NAME=metabase # Service name in traces (default: hostname)
MB_TRACING_LOG_LEVEL=DEBUG # Log threshold for traced threads: TRACE/DEBUG/INFO (default: INFO)
# Batch span processor tuning
MB_TRACING_MAX_QUEUE_SIZE=2048 # Max spans queued for export; drops when full (default: 2048)
MB_TRACING_EXPORT_TIMEOUT_MS=10000 # Max wait for batch export to complete (default: 10000)
MB_TRACING_SCHEDULE_DELAY_MS=5000 # Delay between consecutive batch exports (default: 5000)
```

View File

@@ -1,504 +0,0 @@
---
name: analytics-events
description: Add product analytics events to track user interactions in the Metabase frontend
allowed-tools: Read, Write, Edit, Grep, Glob
---
# Frontend Analytics Events Skill
This skill helps you add product analytics (Snowplow) events to track user interactions in the Metabase frontend codebase.
## Quick Reference
Analytics events in Metabase use Snowplow with typed event schemas. All events must be defined in TypeScript types before use.
**Key Files:**
- `frontend/src/metabase-types/analytics/event.ts` - Event type definitions
- `frontend/src/metabase-types/analytics/schema.ts` - Schema registry
- `frontend/src/metabase/lib/analytics.ts` - Core tracking functions
- Feature-specific `analytics.ts` files - Tracking function wrappers
## Quick Checklist
When adding a new analytics event:
- [ ] Define event type in `frontend/src/metabase-types/analytics/event.ts`
- [ ] Add event to appropriate union type (e.g., `DataStudioEvent`, `SimpleEvent`)
- [ ] Create tracking function in feature's `analytics.ts` file
- [ ] Import and call tracking function at the interaction point
- [ ] Use `trackSimpleEvent()` for basic events (most common)
## Event Schema Types
### 1. Simple Events (Most Common)
Use `SimpleEventSchema` for straightforward tracking. It supports these standard fields:
```typescript
type SimpleEventSchema = {
event: string; // Required: Event name (snake_case)
target_id?: number | null; // Optional: ID of affected entity
triggered_from?: string | null; // Optional: UI location/context
duration_ms?: number | null; // Optional: Duration in milliseconds
result?: string | null; // Optional: Outcome (e.g., "success", "failure")
event_detail?: string | null; // Optional: Additional detail/variant
};
```
**When to use:** 90% of events fit this schema. Use for clicks, opens, closes, creates, deletes, etc.
### 2. Custom Schemas (legacy, no events are being added)
Consider adding new event schema only in very special cases.
**Examples:** `DashboardEventSchema`, `CleanupEventSchema`, `QuestionEventSchema`
## Step-by-Step: Adding a Simple Event
### Example: Track when a user applies filters in a table picker
#### Step 1: Define Event Types
Add event type definitions to `frontend/src/metabase-types/analytics/event.ts`:
```typescript
export type DataStudioTablePickerFiltersAppliedEvent = ValidateEvent<{
event: "data_studio_table_picker_filters_applied";
}>;
export type DataStudioTablePickerFiltersClearedEvent = ValidateEvent<{
event: "data_studio_table_picker_filters_cleared";
}>;
```
#### Step 2: Add to Union Type
Find or create the appropriate union type and add your events:
```typescript
export type DataStudioEvent =
| DataStudioLibraryCreatedEvent
| DataStudioTablePublishedEvent
| DataStudioGlossaryCreatedEvent
| DataStudioGlossaryEditedEvent
| DataStudioGlossaryDeletedEvent
| DataStudioTablePickerFiltersAppliedEvent // <- Add here
| DataStudioTablePickerFiltersClearedEvent; // <- Add here
```
#### Step 3: Create Tracking Functions
In your feature's `analytics.ts` file (e.g., `enterprise/frontend/src/metabase-enterprise/data-studio/analytics.ts`):
```typescript
import { trackSimpleEvent } from "metabase/lib/analytics";
export const trackDataStudioTablePickerFiltersApplied = () => {
trackSimpleEvent({
event: "data_studio_table_picker_filters_applied",
});
};
export const trackDataStudioTablePickerFiltersCleared = () => {
trackSimpleEvent({
event: "data_studio_table_picker_filters_cleared",
});
};
```
#### Step 4: Use in Components
Import and call the tracking function at the interaction point:
```typescript
import {
trackDataStudioTablePickerFiltersApplied,
trackDataStudioTablePickerFiltersCleared,
} from "metabase-enterprise/data-studio/analytics";
function FilterPopover({ filters, onSubmit }) {
const handleReset = () => {
trackDataStudioTablePickerFiltersCleared(); // <- Track here
onSubmit(emptyFilters);
};
return (
<form
onSubmit={(event) => {
event.preventDefault();
trackDataStudioTablePickerFiltersApplied(); // <- Track here
onSubmit(form);
}}
>
{/* form content */}
</form>
);
}
```
## Using SimpleEventSchema Fields
### Example: Event with target_id
```typescript
// Type definition
export type DataStudioLibraryCreatedEvent = ValidateEvent<{
event: "data_studio_library_created";
target_id: number | null;
}>;
// Tracking function
export const trackDataStudioLibraryCreated = (id: CollectionId) => {
trackSimpleEvent({
event: "data_studio_library_created",
target_id: Number(id),
});
};
// Usage
trackDataStudioLibraryCreated(newLibrary.id);
```
### Example: Event with triggered_from
```typescript
// Type definition
export type NewButtonClickedEvent = ValidateEvent<{
event: "new_button_clicked";
triggered_from: "app-bar" | "empty-collection";
}>;
// Tracking function
export const trackNewButtonClicked = (location: "app-bar" | "empty-collection") => {
trackSimpleEvent({
event: "new_button_clicked",
triggered_from: location,
});
};
// Usage
<Button onClick={() => {
trackNewButtonClicked("app-bar");
handleCreate();
}}>
New
</Button>
```
### Example: Event with event_detail
```typescript
// Type definition
export type MetadataEditEvent = ValidateEvent<{
event: "metadata_edited";
event_detail: "type_casting" | "semantic_type_change" | "visibility_change";
triggered_from: "admin" | "data_studio";
}>;
// Tracking function
export const trackMetadataChange = (
detail: "type_casting" | "semantic_type_change" | "visibility_change",
location: "admin" | "data_studio"
) => {
trackSimpleEvent({
event: "metadata_edited",
event_detail: detail,
triggered_from: location,
});
};
// Usage
trackMetadataChange("semantic_type_change", "data_studio");
```
### Example: Event with result and duration
```typescript
// Type definition
export type MoveToTrashEvent = ValidateEvent<{
event: "moved-to-trash";
target_id: number | null;
triggered_from: "collection" | "detail_page" | "cleanup_modal";
duration_ms: number | null;
result: "success" | "failure";
event_detail: "question" | "model" | "metric" | "dashboard";
}>;
// Tracking function
export const trackMoveToTrash = (params: {
targetId: number | null;
triggeredFrom: "collection" | "detail_page" | "cleanup_modal";
durationMs: number | null;
result: "success" | "failure";
itemType: "question" | "model" | "metric" | "dashboard";
}) => {
trackSimpleEvent({
event: "moved-to-trash",
target_id: params.targetId,
triggered_from: params.triggeredFrom,
duration_ms: params.durationMs,
result: params.result,
event_detail: params.itemType,
});
};
// Usage with timing
const startTime = Date.now();
try {
await moveToTrash(item);
trackMoveToTrash({
targetId: item.id,
triggeredFrom: "collection",
durationMs: Date.now() - startTime,
result: "success",
itemType: "question",
});
} catch (error) {
trackMoveToTrash({
targetId: item.id,
triggeredFrom: "collection",
durationMs: Date.now() - startTime,
result: "failure",
itemType: "question",
});
}
```
## Naming Conventions
### Event Names (snake_case)
```typescript
// Good
"data_studio_library_created"
"table_picker_filters_applied"
"metabot_chat_opened"
// Bad
"DataStudioLibraryCreated" // Wrong case
"tablePickerFiltersApplied" // Wrong case
"filters-applied" // Use underscore, not hyphen
```
### Event Type Names (PascalCase with "Event" suffix)
```typescript
// Good
DataStudioLibraryCreatedEvent
TablePickerFiltersAppliedEvent
MetabotChatOpenedEvent
// Bad
dataStudioLibraryCreated // Wrong case
DataStudioLibraryCreated // Missing "Event" suffix
```
### Tracking Function Names (camelCase with "track" prefix)
```typescript
// Good
trackDataStudioLibraryCreated
trackTablePickerFiltersApplied
trackMetabotChatOpened
// Bad
DataStudioLibraryCreated // Missing "track" prefix
track_library_created // Wrong case
logLibraryCreated // Use "track" prefix
```
## Common Patterns
### Pattern 1: Feature-Specific Union Types
Group related events together:
```typescript
export type DataStudioEvent =
| DataStudioLibraryCreatedEvent
| DataStudioTablePublishedEvent
| DataStudioGlossaryCreatedEvent;
export type MetabotEvent =
| MetabotChatOpenedEvent
| MetabotRequestSentEvent
| MetabotFixQueryClickedEvent;
// Then add to SimpleEvent union
export type SimpleEvent =
| /* other events */
| DataStudioEvent
| MetabotEvent
| /* more events */;
```
### Pattern 2: Conditional Tracking
Track different events based on user action:
```typescript
const handleSave = async () => {
if (isNewItem) {
await createItem(data);
trackItemCreated(newItem.id);
} else {
await updateItem(id, data);
trackItemUpdated(id);
}
};
```
## Common Pitfalls
### Don't: Add custom fields to SimpleEvent
```typescript
// WRONG - SimpleEvent doesn't support custom fields
export const trackFiltersApplied = (filters: FilterState) => {
trackSimpleEvent({
event: "filters_applied",
data_layer: filters.dataLayer, // ❌ Not in SimpleEventSchema
data_source: filters.dataSource, // ❌ Not in SimpleEventSchema
with_owner: filters.hasOwner, // ❌ Not in SimpleEventSchema
});
};
// RIGHT - Use only standard SimpleEventSchema fields
export const trackFiltersApplied = () => {
trackSimpleEvent({
event: "filters_applied",
});
};
// Or use event_detail for a single variant
export const trackFilterApplied = (filterType: string) => {
trackSimpleEvent({
event: "filter_applied",
event_detail: filterType, // ✓ "data_layer", "data_source", etc.
});
};
```
### Don't: Forget to add event to union type
```typescript
// Define the event
export type NewFeatureClickedEvent = ValidateEvent<{
event: "new_feature_clicked";
}>;
// ❌ WRONG - Forgot to add to SimpleEvent union
// Event won't be recognized by TypeScript
// ✓ RIGHT - Add to appropriate union
export type SimpleEvent =
| /* other events */
| NewFeatureClickedEvent;
```
### Don't: Mix up event name formats
```typescript
// WRONG
event: "dataStudioLibraryCreated" // camelCase
event: "data-studio-library-created" // kebab-case
event: "Data_Studio_Library_Created" // Mixed case
// RIGHT
event: "data_studio_library_created" // snake_case
```
### Don't: Track PII or sensitive data
```typescript
// WRONG - Don't track user emails, names, or sensitive data
trackSimpleEvent({
event: "user_logged_in",
event_detail: user.email, // ❌ PII
});
// RIGHT - Track non-sensitive identifiers only
trackSimpleEvent({
event: "user_logged_in",
target_id: user.id, // ✓ Just the ID
});
```
### Don't: Forget to track both success and failure
```typescript
// WRONG - Only tracking success
try {
await saveData();
trackDataSaved();
} catch (error) {
// ❌ No tracking for failure case
}
// RIGHT - Track both outcomes
try {
await saveData();
trackDataSaved({ result: "success" });
} catch (error) {
trackDataSaved({ result: "failure" });
}
```
## Testing Analytics Events
While developing, you can verify events are firing:
1. **Check browser console** - When `SNOWPLOW_ENABLED=true` in dev, events are logged
2. **Use shouldLogAnalytics** - Set in `metabase/env` to see all analytics in console
3. **Check Snowplow debugger** - Browser extension for Snowplow events
Example console output:
```
[SNOWPLOW EVENT | event sent:true], data_studio_table_picker_filters_applied
```
## File Organization
### Where to put tracking functions:
```
Feature-specific analytics functions:
frontend/src/metabase/{feature}/analytics.ts
enterprise/frontend/src/metabase-enterprise/{feature}/analytics.ts
Event type definitions (all in one place):
frontend/src/metabase-types/analytics/event.ts
Core tracking utilities:
frontend/src/metabase/lib/analytics.ts
```
## Real-World Examples
See these files for reference:
- **Simple events**: `enterprise/frontend/src/metabase-enterprise/data-studio/analytics.ts`
- **Events with variants**: `frontend/src/metabase/dashboard/analytics.ts`
- **Complex events**: `frontend/src/metabase/query_builder/analytics.js`
- **Event type examples**: `frontend/src/metabase-types/analytics/event.ts`
## Workflow Summary
1. **Identify the user interaction** to track
2. **Decide on event name** (snake_case, descriptive)
3. **Define event type** in `event.ts` using `ValidateEvent`
4. **Add to union type** (create feature union if needed)
5. **Create tracking function** in feature's `analytics.ts`
6. **Import and call** at the interaction point
7. **Test** that events fire correctly
## Tips
- **Be specific** - `filters_applied` is better than `action_performed`
- **Use past tense** - `library_created` not `create_library`
- **Group related events** - Create feature-specific event union types
- **Track meaningful actions** - Not every click needs tracking
- **Consider the data** - What would you want to analyze later?
- **Stay consistent** - Follow existing naming patterns in the codebase
- **Document context** - Use `triggered_from` to track where the action happened

View File

@@ -1,144 +0,0 @@
---
name: clojure-review
description: Review Clojure and ClojureScript code changes for compliance with Metabase coding standards, style violations, and code quality issues. Use when reviewing pull requests or diffs containing Clojure/ClojureScript code.
allowed-tools: Read, Grep, Bash, Glob
---
# Clojure Code Review Skill
@./../_shared/clojure-style-guide.md
@./../_shared/clojure-commands.md
## Review guidelines
**What to flag:**
- Check compliance with the Metabase Clojure style guide (included above)
- If `CLOJURE_STYLE_GUIDE.adoc` exists in the working directory, also check compliance with the community Clojure style guide
- Flag all style guide violations
**What NOT to post:**
- Do not post comments congratulating someone for trivial changes or for following style guidelines
- Do not post comments confirming things "look good" or telling them they did something correctly
- Only post comments about style violations or potential issues
Example bad code review comments to avoid:
> This TODO comment is properly formatted with author and date - nice work!
> Good addition of limit 1 to the query - this makes the test more efficient without changing its behavior.
> The kondo ignore comment is appropriately placed here
> Test name properly ends with -test as required by the style guide.
**Special cases:**
- Do not post comments about missing parentheses (these will be caught by the linter)
## Quick review checklist
Use this to scan through changes efficiently:
### Naming
- [ ] Descriptive names (no `tbl`, `zs'`)
- [ ] Pure functions named as nouns describing their return value
- [ ] `kebab-case` for all variables and functions
- [ ] Side-effect functions end with `!`
- [ ] No namespace-alias repetition in function names
### Documentation
- [ ] Public vars in `src` or `enterprise/backend/src` have useful docstrings
- [ ] Docstrings use Markdown conventions
- [ ] References use `[[other-var]]` not backticks
- [ ] `TODO` comments include author and date: `;; TODO (Name 2025-01-01) -- description`
### Code Organization
- [ ] Everything `^:private` unless used elsewhere
- [ ] No `declare` when avoidable (public functions near end)
- [ ] Functions under 20 lines when possible
- [ ] No blank, non-comment lines within definition forms (except pairwise constructs in `let`/`cond`)
- [ ] Lines ≤ 120 characters
### Tests
- [ ] Separate `deftest` forms for distinct test cases
- [ ] Pure tests marked `^:parallel`
- [ ] Test names end in `-test` or `-test-<number>`
### Modules
- [ ] Correct module patterns (OSS: `metabase.<module>.*`, EE: `metabase-enterprise.<module>.*`)
- [ ] API endpoints in `<module>.api` namespaces
- [ ] Public API in `<module>.core` with Potemkin
- [ ] No cheating module linters with `:clj-kondo/ignore [:metabase/modules]`
### REST API
- [ ] Response schemas present (`:- <schema>`)
- [ ] Query params use kebab-case, bodies use `snake_case`
- [ ] Routes use singular nouns (e.g., `/api/dashboard/:id`)
- [ ] `GET` has no side effects (except analytics)
- [ ] Malli schemas detailed and complete
- [ ] All new endpoints have tests
### MBQL
- [ ] No raw MBQL manipulation outside `lib`, `lib-be`, or `query-processor` modules
- [ ] Uses Lib and MBQL 5, not legacy MBQL
### Database
- [ ] Model and table names are singular nouns
- [ ] Uses `t2/select-one-fn` instead of selecting full rows for one column
- [ ] Logic in Toucan methods, not helper functions
### Drivers
- [ ] New multimethods documented in `docs/developers-guide/driver-changelog.md`
- [ ] Passes `driver` argument to other driver methods (no hardcoded driver names)
- [ ] Minimal logic in `read-column-thunk`
### Miscellaneous
- [ ] Example data is bird-themed when possible
- [ ] Kondo linter suppressions use proper format (not `#_:clj-kondo/ignore` keyword form)
## Pattern matching table
Quick scan for common issues:
| Pattern | Issue |
| -------------------------------------------- | ----------------------------------------------------------- |
| `calculate-age`, `get-user` | Pure functions should be nouns: `age`, `user` |
| `update-db`, `save-model` | Missing `!` for side effects: `update-db!`, `save-model!` |
| `snake_case_var` | Should use kebab-case |
| Public var without docstring | Add docstring explaining purpose |
| `;; TODO fix this` | Missing author/date: `;; TODO (Name 2025-01-01) -- description` |
| `(defn foo ...)` in namespace used elsewhere | Should be `(defn ^:private foo ...)` |
| Function > 20 lines | Consider breaking up into smaller functions |
| `/api/dashboards/:id` | Use singular: `/api/dashboard/:id` |
| Query params with `snake_case` | Use kebab-case for query params |
| New API endpoint without tests | Add tests for the endpoint |
## Feedback format examples
**For style violations:**
> This pure function should be named as a noun describing its return value. Consider `user` instead of `get-user`.
**For missing documentation:**
> This public var needs a docstring explaining its purpose, inputs, and outputs.
**For organization issues:**
> This function is only used in this namespace, so it should be marked `^:private`.
**For API conventions:**
> Query parameters should use kebab-case. Change `user_id` to `user-id`.

View File

@@ -1,69 +0,0 @@
---
name: clojure-write
description: Guide Clojure and ClojureScript development using REPL-driven workflow, coding conventions, and best practices. Use when writing, developing, or refactoring Clojure/ClojureScript code.
---
# Clojure Development Skill
## Tool Preference
When `clojure-mcp` tools are available (e.g., `clojure_eval`, `clojure_edit`), **always use them**
instead of shell commands like `./bin/mage -repl`. The MCP tools provide:
- Direct REPL integration without shell escaping issues
- Better error messages and feedback
- Structural Clojure editing that prevents syntax errors
Only fall back to `./bin/mage` commands when clojure-mcp is not available.
@./../_shared/development-workflow.md
@./../_shared/clojure-style-guide.md
@./../_shared/clojure-commands.md
## REPL-Driven Development Workflow
- Start with small, fundamental functions:
- Identify the core features or functionalities required for your task.
- Break each feature down into the smallest, most basic functions that can be developed and tested independently.
- Write and test in the REPL:
- Write the code for each small function directly in the REPL (Read-Eval-Print Loop).
- Test it thoroughly with a variety of inputs, including typical use cases and relevant edge cases, to ensure it
behaves as expected.
- Integrate into source code:
- Once a function works correctly in the REPL, move it from the REPL environment into your source code files (e.g.,
within appropriate namespaces).
- Gradually increase complexity:
- Build upon tested, basic functions to create more complex functions or components.
- Compose smaller functions together, testing each new composition in the REPL to verify correctness step by step.
- Ensure dependency testing:
- Make sure every function is fully tested in the REPL before it is depended upon by other functions.
- This ensures that each layer of your application is reliable before you build on it.
- Use the REPL fully:
- Use the REPL as your primary tool to experiment with different approaches, iterate quickly, and get immediate
feedback on your code.
- Follow functional programming principles:
- Keep functions small, focused, and composable.
- Use Clojure's functional programming features—like immutability, higher-order functions, and the standard
library—to write concise, effective code.
## How to Evaluate Code
### Bottom-up Dev Loop
1. Write code into a file.
2. Evaluate the file's namespace and make sure it loads correctly with:
```
./bin/mage -repl --namespace metabase.app-db.connection
```
3. Call functions in the namespace with test inputs, and observe that the outputs are correct
Feel free to copy these REPL session trials into actual test cases using `deftest` and `is`.
4. Once you know these functions are good, return to 1, and compose them into the task that you need to build.
## Critical Rules for Editing
- Be careful with parentheses counts when editing Clojure code
- After EVERY change to Clojure code, verify readability with `-check-readable`
- End all files with a newline
- When editing tabular code, where the columns line up, try to keep them aligned
- Spaces on a line with nothing after it is not allowed

View File

@@ -1,163 +0,0 @@
---
name: docs-review
description: Review documentation changes for compliance with the Metabase writing style guide. Use when reviewing pull requests, files, or diffs containing documentation markdown files.
allowed-tools: Read, Grep, Bash, Glob
---
# Documentation Review Skill
@./../_shared/metabase-style-guide.md
## Review mode detection
**IMPORTANT: Before starting the review, determine which mode to use:**
1. **PR review mode**: If the `mcp__github__create_pending_pull_request_review` tool is available, you are reviewing a GitHub PR
- Use the pending review workflow to post all issues as one cohesive review
- Follow the workflow steps in "PR review mode format" below
2. **Local review mode**: If the MCP tool is NOT available, output issues in the conversation
- Format all issues in a numbered markdown list (as described in "Feedback format" below)
## Review process
1. **Detect review mode** - Check if `mcp__github__create_pending_pull_request_review` is available
2. Read the changes through once to understand intent
3. Check all issues that violate style guide or significantly impact readability
4. Only flag issues worth mentioning - if it won't make a material difference to the reader, skip it
5. **REQUIRED: Number ALL feedback sequentially** - Start from Issue 1 and increment for each issue found
## Review checklist
Run through the diff looking for these issues:
**Tone and voice:**
- [ ] Formal/corporate language ("utilize" not "use", "offerings", etc.)
- [ ] "Users" instead of "people" or "companies"
- [ ] Excessive exclamation points or overly peppy tone
- [ ] Telling readers something is cool instead of showing them
**Structure and clarity:**
- [ ] Important information buried instead of leading
- [ ] Verbose text that adds little value
- [ ] Paragraphs without clear purpose
- [ ] Vague headings that don't convey the point
- [ ] Instructions explain "why" before telling "what to do"
- [ ] Tasks described as "easy" or "simple"
**Links and references:**
- [ ] Linking the word "here" instead of descriptive text
- [ ] Links in headings (unless entire heading is a link)
**Formatting:**
- [ ] Ampersands as "and" substitute (except proper nouns)
- [ ] Inconsistent list formatting
**Code and examples:**
- [ ] Code examples that don't work or would error
- [ ] Commands not in execution order
- [ ] Full-width screenshots instead of scoped UI elements
- [ ] Excessive or unnecessary images
**Sentence construction:**
- [ ] Overuse of pronouns when introducing new terms
## Quick scan table
| Pattern | Issue |
| ----------------------------- | --------------------------------------------- |
| we can do X, our feature | Should be "Metabase" or "it" |
| click here, read more here | Need descriptive link text |
| easy, simple, just | Remove condescending qualifiers |
| users | Should be "people" or "companies" if possible |
## Feedback format
**MANDATORY REQUIREMENT: Every single issue MUST be numbered sequentially starting from Issue 1.**
This numbered format is NON-NEGOTIABLE. It allows users to efficiently reference specific issues (e.g., "fix issues 1, 3, and 5") and track which feedback has been addressed.
### Local review mode format
When outputting issues in the conversation (local mode), use this format:
```markdown
## Issues
**Issue 1: [Brief title]**
Line X: Succinct description of the issue
[code or example]
Suggested fix or succinct explanation
**Issue 2: [Brief title]**
Line Y: Description of the issue
Suggested fix or explanation
**Issue 3: [Brief title]**
...
```
**Examples:**
> **Issue 1: Formal tone**
> Line 15: This could be more conversational. Consider: "You can't..." instead of "You cannot..."
> **Issue 2: Vague heading**
> Line 8: The heading could be more specific. Try stating the point directly: "Run migrations before upgrading" vs "Upgrade process"
### PR review mode format
When posting to GitHub (PR mode), use the **pending review workflow**:
**Workflow steps:**
1. **Start a review**: Use `mcp__github__create_pending_pull_request_review` to begin a pending review
- This creates a draft review that won't be visible until submitted
2. **Get diff information**: Use `mcp__github__get_pull_request_diff` to understand the code changes and line numbers
- This helps you determine the correct file paths and line numbers for comments
3. **Identify ALL issues**: Read through all changes and identify every issue worth mentioning
- Collect all issues before posting any comments
- Number them sequentially (Issue 1, Issue 2, Issue 3, etc.)
4. **Add review comments**: Use `mcp__github__add_pull_request_review_comment_to_pending_review` for each issue
- **CRITICAL**: Post ALL comments in a SINGLE response using multiple tool calls in parallel
- Each comment should reference a specific file path and line number from the diff
- Start each comment body with `**Issue N: [Brief title]**`
- Include the description and suggested fix
5. **Submit the review**: Use `mcp__github__submit_pending_pull_request_review` to publish all comments at once
- Use event type `"COMMENT"` (NOT "REQUEST_CHANGES") to make it non-blocking
- **Do NOT include a body message** - Leave the body empty or omit it entirely
- All comments will appear together as one cohesive review
**Comment format example:**
```
**Issue 1: Formal tone**
This could be more conversational. Consider: "You can't..." instead of "You cannot..."
```
**IMPORTANT**:
- Each issue gets its own review comment attached to the pending review
- Number ALL comments sequentially (Issue 1, Issue 2, Issue 3, etc.)
- Always start the comment body with `**Issue N: [Brief title]**`
- **MUST add all comments in parallel in a single response** - Do NOT add them one after another in separate responses
- Do NOT output a summary message to the conversation - only post GitHub review comments
- When submitting the review, do NOT include a body parameter (or leave it empty) to avoid cluttering the PR with summary text
- The review will appear as a single review with multiple comments when submitted
## Final check
1. Remove any issues from your assessment that won't make a material difference to the reader if addressed. Only flag issues worth the author's time to fix.
2. **Verify all issues are numbered sequentially** starting from Issue 1 with no gaps in numbering.
3. Confirm the format exactly matches: `**Issue N: [Brief title]**` where N is the issue number.
4. **In PR mode**: Verify each issue was posted as a separate GitHub comment (not output to conversation).

View File

@@ -1,96 +0,0 @@
---
name: docs-write
description: Write documentation following Metabase's conversational, clear, and user-focused style. Use when creating or editing documentation files (markdown, MDX, etc.).
allowed-tools: Read, Write, Grep, Bash, Glob
---
# Documentation Writing Skill
@./../_shared/metabase-style-guide.md
## When writing documentation
### Start here
1. **Who is this for?** Match complexity to audience. Don't oversimplify hard things or overcomplicate simple ones.
2. **What do they need?** Get them to the answer fast. Nobody wants to be in docs longer than necessary.
3. **What did you struggle with?** Those common questions you had when learning? Answer them (without literally including the question).
### Writing process
**Draft:**
- Write out the steps/explanation as you'd tell a colleague
- Lead with what to do, then explain why
- Use headings that state your point: "Set SAML before adding users" not "SAML configuration timing"
**Edit:**
- Read aloud. Does it sound like you talking? If it's too formal, simplify.
- Cut anything that doesn't directly help the reader
- Check each paragraph has one clear purpose
- Verify examples actually work (don't give examples that error)
**Polish:**
- Make links descriptive (never "here")
- Backticks only for code/variables, **bold** for UI elements
- American spelling, serial commas
- Keep images minimal and scoped tight
**Format:**
- Run prettier on the file after making edits: `bun run prettier --write <file-path>`
- This ensures consistent formatting across all documentation
### Common patterns
**Instructions:**
```markdown
Run:
\`\`\`
command-to-run
\`\`\`
Then:
\`\`\`
next-command
\`\`\`
This ensures you're getting the latest changes.
```
Not: "(remember to run X before Y...)" buried in a paragraph.
**Headings:**
- "Use environment variables for configuration" ✅
- "Environment variables" ❌ (too vague)
- "How to use environment variables for configuration" ❌ (too wordy)
**Links:**
- "Check out the [SAML documentation](link)" ✅
- "Read the docs [here](link)" ❌
### Watch out for
- Describing tasks as "easy" (you don't know the reader's context)
- Using "we" when talking about Metabase features (use "Metabase" or "it")
- Formal language: "utilize", "reference", "offerings"
- Too peppy: multiple exclamation points
- Burying the action in explanation
- Code examples that don't work
- Numbers that will become outdated
### Quick reference
| Write This | Not This |
| -------------------------- | ------------------ |
| people, companies | users |
| summarize | aggregate |
| take a look at | reference |
| can't, don't | cannot, do not |
| **Filter** button | \`Filter\` button |
| Check out [the docs](link) | Click [here](link) |

View File

@@ -1,260 +0,0 @@
---
name: e2e-test-create
description: >
Analyze React component source code to understand UI structure,
then generate idiomatic Cypress E2E tests following Metabase conventions.
Falls back to Playwright MCP browser exploration only when code reading
and screenshot debugging are insufficient.
disable-model-invocation: true
allowed-tools:
- Bash
- Read
- Write
- Grep
- Glob
- Skill
- "mcp__playwright__*"
---
# Code-Reading-First → Generate Cypress Tests (Metabase)
You are writing Cypress E2E tests for the **Metabase** codebase.
Before generating ANY test code, you MUST analyze React component source code
to understand DOM structure, selectors, and user flows.
## Phase 0 — Research
1. Read existing helpers before writing anything:
- `e2e/support/helpers/` — all shared helpers (restore, signInAs, openOrdersTable, etc.)
- `e2e/support/cypress_sample_database.ts` — table/field schema constants (ORDERS, PRODUCTS, etc.)
- `e2e/support/cypress_sample_instance_data.ts` — instance-specific IDs (ORDERS_DASHBOARD_ID, NORMAL_USER_ID, etc.)
2. Glob `e2e/test/scenarios/` to find the closest existing spec to the area under test.
Study its patterns — match them exactly.
3. Glob `frontend/src/metabase/` to find React components for the feature area.
## Phase 1 — Code Analysis
Read React component source to understand DOM structure. No browser needed — source code has everything.
1. **Find relevant components**: Glob and grep `frontend/src/metabase/` for the feature area.
2. **Extract selectors**: Grep for `data-testid` in relevant components.
3. **Note visible text**: Read component JSX for button labels, headings, placeholders.
4. **Note aria attributes**: Grep for `aria-label` in relevant components.
5. **Understand user flows**: Read event handlers (onClick, onSubmit, onChange) to understand interactions.
6. **Find API calls**: Grep for `Api.use`, `fetch`, `useQuery`, endpoint definitions to identify API calls to intercept.
7. **Cross-reference with existing specs**: Find specs in the same area and reuse their proven selectors and `cy.intercept` patterns.
## Phase 2 — Start Backend
Use `MB_EDITION=oss` by default. Only use `MB_EDITION=ee` when the user explicitly asks to write an enterprise test.
Start the backend using `run_in_background: true` (NOT `&`).
`bin/e2e-backend` automatically detects if a backend is already running and reuses it.
```bash
MB_EDITION=oss bin/e2e-backend
```
Do NOT manually generate snapshots by running unrelated test specs.
The `bun test-cypress` runner has `GENERATE_SNAPSHOTS: true` by default and automatically
generates snapshots before running any spec. When running tests in Phase 4 via the `/e2e-test` skill,
snapshots will be generated on the first run if they don't already exist.
Restore clean test data:
```bash
curl -sf -X POST http://localhost:4000/api/testing/restore/default
```
## Phase 3 — Generate Cypress Spec
### File location & naming
Place specs in `e2e/test/scenarios/<area>/` mirroring the URL structure.
Name: `<feature>.cy.spec.js` (NOT `.cy.spec.ts`).
### Metabase conventions (mandatory)
- **Helpers via `cy.H`**: All helpers are accessed via `const { H } = cy;` — NOT via direct imports from `e2e/support/helpers`.
```js
const { H } = cy;
describe("feature name", () => {
beforeEach(() => {
H.restore();
cy.signInAsAdmin();
});
});
```
- **Sample Database schema** (table/field definitions): Import from `cypress_sample_database`.
```js
import { ORDERS, ORDERS_ID, PRODUCTS } from "e2e/support/cypress_sample_database";
```
- **Instance data** (dashboard IDs, user IDs, etc.): Import from `cypress_sample_instance_data`.
```js
import { ORDERS_DASHBOARD_ID } from "e2e/support/cypress_sample_instance_data";
```
- **Selectors** (priority order):
1. `cy.findByText()` / `cy.findByLabelText()` / `cy.findByRole()` — from `@testing-library/cypress`
2. `cy.findByTestId()` — for `data-testid` attributes
3. `cy.get("[data-testid='...']")` — fallback
4. NEVER use positional selectors, CSS class names, or XPaths.
- **Navigation helpers**: Use existing helpers like `H.openOrdersTable()`, `H.openNativeEditor()`,
`H.visitDashboard(id)`, `H.visitQuestion(id)` instead of raw `cy.visit()` chains.
Grep `e2e/support/helpers/` to discover what's available for your area.
- **API setup over UI setup**: Use `cy.request()` or existing API helpers to set up state.
Only use the UI for the flow you're actually testing.
- **Assertions**: Assert on visible text, URL, aria state — not DOM structure.
- **Waits**: Never use `cy.wait(ms)`. Use `cy.intercept()` + `cy.wait("@alias")` for API calls,
or `cy.findByText().should("be.visible")` for DOM readiness.
- **Isolation**: Each `it()` block must be independently runnable.
Don't depend on state from a previous `it()`.
### Spec structure template
```js
const { H } = cy;
import { ORDERS_DASHBOARD_ID } from "e2e/support/cypress_sample_instance_data";
import { ORDERS, ORDERS_ID } from "e2e/support/cypress_sample_database";
describe("area > sub-area > feature (#issue-number)", () => {
beforeEach(() => {
H.restore();
cy.signInAsAdmin();
});
it("should do the primary happy-path thing", () => {
// test
});
it("should handle the edge case", () => {
// test
});
});
```
### Intercepts
When you identified API calls during code analysis, stub or wait on them:
```js
cy.intercept("POST", "/api/dataset").as("dataset");
// ... trigger action ...
cy.wait("@dataset");
```
## Phase 4 — Validate
After generating specs:
1. Check that all imported helpers exist (Grep `e2e/support/helpers/`).
2. **You MUST use the `/e2e-test` skill** to run tests — do NOT run `bun test-cypress` directly.
The `/e2e-test` skill handles edition selection, snapshot management, and correct env vars.
`/e2e-test GREP="should do the thing" --spec e2e/test/scenarios/<path>`
If you created multiple `it()` blocks, run each one individually to isolate failures.
## Phase 5 — Fix Failures (up to 2 attempts)
When a test fails, **try to fix it from Cypress output first**:
1. Read the failure screenshot (path printed under `(Screenshots)`).
2. Read the error message and code frame from the console output.
3. Fix the test and re-run (back to Phase 4, step 2).
If you cannot diagnose the issue after 2 attempts, proceed to Phase 6.
## Phase 6 — Playwright Fallback
Only reach this phase after 2 failed fix attempts from Phase 5. The backend is already running.
Restore clean test data:
```bash
curl -sf -X POST http://localhost:4000/api/testing/restore/default
```
**Bypass CSP** headers before navigating (Metabase serves strict CSP that blocks dev server scripts).
Use `browser_run_code` to set this up:
```js
async (page) => {
// Strip CSP headers so the page loads (mirrors Cypress chromeWebSecurity: false)
await page.context().route('**/*', async (route) => {
const response = await route.fetch();
const headers = { ...response.headers() };
delete headers['content-security-policy'];
delete headers['content-security-policy-report-only'];
await route.fulfill({ response, headers });
});
// Sign in via API
const response = await page.request.post('http://localhost:4000/api/session', {
data: { username: 'admin@metabase.test', password: '12341234' }
});
const session = await response.json();
await page.context().addCookies([{
name: 'metabase.DEVICE',
value: session.id,
domain: 'localhost',
path: '/'
}]);
await page.goto('http://localhost:4000');
await page.waitForLoadState('networkidle');
return 'signed in';
}
```
**Maintain an observation log incrementally.** After EVERY significant Playwright interaction,
IMMEDIATELY append what you observed to the scratch file BEFORE performing the next interaction:
```bash
cat >> /tmp/e2e-observations.md << 'OBSERVATION'
## [Page/Flow name]
- URL: /question/notebook#...
- Clicked: "Box plot" button → visible text "Box plot", role: radio
- Selectors: data-testid="viz-type-button", findByText("Box plot")
- API call: POST /api/dataset (triggered on viz change)
- Key state: after selecting viz type, summary sidebar shows metric picker
OBSERVATION
```
For each page/flow:
- Take an accessibility snapshot (`browser_snapshot`).
- Click through interactive elements, fill forms, trigger modals.
- **Append to the observation log immediately after each step**: URLs, visible text, aria labels, `data-testid` attrs, API calls.
- Screenshot key states.
After exploration:
1. Read back your observation log: `cat /tmp/e2e-observations.md`
2. Fix the test using observed selectors and behavior.
3. Re-run the test (back to Phase 4).
4. Clean up: `rm -f /tmp/e2e-observations.md`
## Phase 7 — Cleanup
After all tests pass (or after giving up on fixing failures), **always kill the backend on port 4000**:
```bash
lsof -ti:4000 | xargs kill 2>/dev/null || true
```
Do NOT use broad `pkill` patterns — there may be other Metabase instances on different ports.
The backend process started in Phase 2 will NOT be killed automatically when the Claude session ends.
Leaving it running wastes resources and can interfere with future sessions. Always clean up.
## What NOT to do
- Do NOT use Playwright as the first step — always analyze source code first.
- Do NOT kill the backend between phases — it stays running throughout.
- Do NOT invent selectors you didn't find in source code or observe in the browser.
- Do NOT hardcode database IDs — import from `cypress_sample_database` or `cypress_sample_instance_data`.
- Do NOT use `cy.wait(1000)` or any numeric wait.
- Do NOT create setup flows through the UI when an API helper exists.
- Do NOT put tests in `cypress/e2e/` — Metabase uses `e2e/test/scenarios/`.
- Do NOT import helpers directly from `e2e/support/helpers` — use `const { H } = cy;`.

View File

@@ -1,78 +0,0 @@
---
name: e2e-test
description: Run Cypress E2E tests, analyze failures including screenshots, and stress test for flakiness
disable-model-invocation: false
---
Run Cypress E2E test: $ARGUMENTS
## Edition selection
**Default: `MB_EDITION=oss`** — runs without enterprise tokens, simpler and faster.
Only use `MB_EDITION=ee` when the user explicitly asks to write or run an enterprise test.
**For `MB_EDITION=ee`:**
These tokens must be exported in the shell before starting Claude Code:
- `CYPRESS_MB_ALL_FEATURES_TOKEN` — all features (most EE tests need this)
- `CYPRESS_MB_PRO_SELF_HOSTED_TOKEN` — pro self-hosted (permissions, SSO, sandboxing)
- `CYPRESS_MB_STARTER_CLOUD_TOKEN` — starter plan (rarely needed)
- `CYPRESS_MB_PRO_CLOUD_TOKEN` — pro cloud (rarely needed)
**NEVER echo, print, or log token values. Only check if they are set:**
```bash
echo "ALL_FEATURES: ${CYPRESS_MB_ALL_FEATURES_TOKEN:+set}" && echo "PRO_SELF_HOSTED: ${CYPRESS_MB_PRO_SELF_HOSTED_TOKEN:+set}" && echo "STARTER: ${CYPRESS_MB_STARTER_CLOUD_TOKEN:+set}" && echo "PRO_CLOUD: ${CYPRESS_MB_PRO_CLOUD_TOKEN:+set}"
```
If tokens are missing, tell the user: "EE tokens are not set. Either export them and restart Claude Code, or add `MB_EDITION=oss` to run OSS-only tests."
## Running
First check if snapshots exist:
```bash
ls e2e/snapshots/default.sql 2>/dev/null && echo "snapshots exist" || echo "snapshots missing"
```
If snapshots exist, skip regeneration for speed:
```bash
MB_EDITION=oss CYPRESS_VIDEO=false CYPRESS_RETRIES=0 CYPRESS_GUI=false GENERATE_SNAPSHOTS=false bun test-cypress $ARGUMENTS
```
If snapshots are missing (first run), let the runner generate them:
```bash
MB_EDITION=oss CYPRESS_VIDEO=false CYPRESS_RETRIES=0 CYPRESS_GUI=false bun test-cypress $ARGUMENTS
```
When running enterprise tests, replace `MB_EDITION=oss` with `MB_EDITION=ee` in the commands above.
If `$ARGUMENTS` is empty, ask the user which spec to run.
If the user provides a test name or fuzzy name instead of a spec path, find the spec file:
```bash
find e2e/test/scenarios -name "*FUZZY_NAME*.cy.spec.*" -type f
```
Then run with `--spec "path/to/matched.cy.spec.js"`.
To filter by test name, use the `GREP` env var (not `--env grep=`, which breaks on commas):
```bash
GREP="test name here" MB_EDITION=oss bun test-cypress --spec path/to/spec.js
```
## Stress test (flaky detection)
When the user asks to verify a test is not flaky:
```bash
MB_EDITION=oss bin/e2e-stress-test $ARGUMENTS
```
Set `E2E_STRESS_RUNS=N` for more iterations (default: 5). Stops on first failure and prints screenshot paths. Read those screenshots to analyze the failure.
## Analyzing failures
When tests fail:
1. Read the **screenshot** for each failure — paths are printed in Cypress output under `(Screenshots)`, typically at `cypress/screenshots/<spec-name>/<test-name> (failed).png`
2. Read the error message and code frame from the console output
3. Identify the root cause: is it a test bug, a product bug, or a timing issue?
4. Suggest a fix with specific code changes

View File

@@ -1,114 +0,0 @@
---
name: mutation-testing
description: Run mutation testing on a Clojure namespace, generate tests to kill surviving mutations, and open draft PRs with Linear issue tracking.
---
# Mutation Testing Skill
This skill runs mutation testing end-to-end: generates a coverage report, groups functions, shells out to `claude -p` to write tests, verifies mutations are killed, and creates draft PRs with Linear issues.
## Prerequisites
- A running nREPL connected to the Metabase dev environment
- `LINEAR_API_KEY` environment variable set with a valid Linear personal API key
- `gh` CLI authenticated with GitHub
- `claude` CLI available on PATH
## Reference Files
- `dev/src/dev/coverage.clj` — mutation testing engine (generates reports, runs mutations)
- `dev/src/dev/mutation_testing.clj` — orchestration, Linear API, PR helpers
## Invocation
The argument is a Clojure namespace, optionally followed by a base branch:
```
/mutation-testing metabase.lib.order-by
/mutation-testing metabase.lib.order-by --base-branch release-x.52.x
/mutation-testing metabase.lib.order-by --project-id abc-123
/mutation-testing metabase.lib.order-by --base-branch release-x.52.x --project-id abc-123
```
- `--base-branch`: defaults to the value in config (set via `set-config!`), or `"master"` if not configured.
- `--project-id`: if provided, issues are added to this existing Linear project instead of creating a new one.
## Steps
### 1. Parse arguments
Parse `$ARGUMENTS` to extract:
- **namespace** (required) — the first positional argument
- **`--base-branch`** (optional) — if present, the next argument is the base branch name
- **`--project-id`** (optional) — if present, the next argument is an existing Linear project ID
### 2. Load and configure
```clojure
(require '[dev.mutation-testing :as mut-test] :reload)
(require '[dev.coverage :as cov] :reload)
```
If this is the first invocation (or the REPL was restarted), set up the Linear team:
```clojure
(mut-test/list-teams!)
(mut-test/set-config! {:team-id "<id>"})
```
### 3. Run
```clojure
(mut-test/run! '<target-ns>)
;; Or with options:
(mut-test/run! '<target-ns> {:base-branch "release-x.52.x"})
(mut-test/run! '<target-ns> {:project-id "abc-123"})
(mut-test/run! '<target-ns> {:base-branch "release-x.52.x" :project-id "abc-123"})
```
Pass the opts map with `:base-branch` and/or `:project-id` if those flags were provided in the arguments.
This will:
1. Generate a baseline mutation testing report
2. Create a Linear project for the namespace
3. Group functions by coverage relationships
4. For each group: create branch → invoke Claude to write tests → verify → retry if needed → commit & push → create Linear issue → create draft PR
5. Print a summary with PR links
### 4. Handle failures
If a group fails mid-processing, `run!` catches the error and continues to the next group. To retry a single group manually:
```clojure
;; Get the parsed namespace info
(def parsed (mut-test/parse-namespace '<target-ns>))
;; Run coverage to get the data
(def coverage-results (cov/test-namespace (:target-ns parsed) [(:test-ns parsed)]))
;; Group and find the one you want
(def groups (mut-test/group-functions coverage-results))
;; Process just that group
(mut-test/process-group! parsed (nth groups <index>))
```
### 5. Verify the prompt (dry run)
To inspect what Claude will see without invoking it:
```clojure
(def parsed (mut-test/parse-namespace '<target-ns>))
(def coverage-results (cov/test-namespace (:target-ns parsed) [(:test-ns parsed)]))
(def groups (mut-test/group-functions coverage-results))
(println (mut-test/build-test-prompt
(merge (select-keys parsed [:target-ns :test-ns :source-path :test-path])
(select-keys (first groups) [:fn-names :mutations]))))
```
## Configuration
- **Team ID**: Set once per REPL session via `(mut-test/list-teams!)` and `(mut-test/set-config! {:team-id "..."})`
- **Project ID**: Set automatically when `run!` calls `create-project-for-namespace!`
- **Base Branch**: Defaults to `"master"`. Override via `(mut-test/set-config! {:base-branch "release-x.52.x"})` or pass `{:base-branch "..."}` as the second arg to `run!`
- **Project ID**: If set (via `set-config!` or `run!` opts), `run!` reuses the existing Linear project instead of creating a new one

View File

@@ -1,20 +0,0 @@
---
name: typescript-review
description: Review TypeScript and JavaScript code changes for compliance with Metabase coding standards, style violations, and code quality issues. Use when reviewing pull requests or diffs containing TypeScript/JavaScript code.
allowed-tools: Read, Grep, Bash, Glob
---
# TypeScript/JavaScript Code Review Skill
@./../_shared/typescript-commands.md
## Code Review Guidelines
Review pull requests with a focus on:
- Readability and maintainability
- Appropriate test coverage
- Compliance with project coding standards and conventions
- Code quality and best practices
- Type safety and proper TypeScript usage
- React best practices (when applicable)

View File

@@ -1,9 +0,0 @@
---
name: typescript-write
description: Write TypeScript and JavaScript code following Metabase coding standards and best practices. Use when developing or refactoring TypeScript/JavaScript code.
---
# TypeScript/JavaScript Development Skill
@./../_shared/development-workflow.md
@./../_shared/typescript-commands.md

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -3,7 +3,7 @@
;; TODO -- seems silly to maintain different blacklists and whitelists here than we use for the deftest `^:parallel`
;; checker...
{:explicitly-safe
#{;; these toucan methods might actually set global values if it's used outside of a transaction,
#{ ;; these toucan methods might actually set global values if it's used outside of a transaction,
;; but since mt/with-temp runs in a transaction, so we'll ignore them in this case.
toucan2.core/delete!
toucan2.core/update!
@@ -13,10 +13,7 @@
toucan2.core/insert-returning-pk!
toucan2.core/insert-returning-pks!
clojure.core.async/<!!
clojure.core.async/<!
clojure.core.async/>!!
clojure.core.async/>!
clojure.core.async/alts!
clojure.core.async/alts!!
clojure.core.async/close!
clojure.core.async/poll!
@@ -52,43 +49,36 @@
metabase.actions.models/insert!
metabase.analytics.snowplow-test/fake-track-event-impl!
metabase.analytics.snowplow/track-event-impl!
metabase.app-db.core/finish-db-setup!
metabase.app-db.core/migrate!
metabase.app-db.core/setup-db!
metabase.app-db.schema-migrations-test.impl/run-migrations-in-range!
metabase.app-db.setup/migrate!
metabase.app-db.setup/setup-db!
metabase.channel.email-test/reset-inbox!
metabase.channel.email/send-email!
metabase.cmd.dump-to-h2/dump-to-h2!
metabase.cmd.load-from-h2/load-from-h2!
metabase.content-verification.models.moderation-review/create-review!
metabase.core.core/ensure-audit-db-installed!
metabase.db.schema-migrations-test.impl/run-migrations-in-range!
metabase.db.setup/migrate!
metabase.db.setup/setup-db!
metabase.db/migrate!
metabase.db/setup-db!
metabase.driver.mongo-test/create-database-from-row-maps!
metabase.driver.postgres-test/create-enums-db!
metabase.driver.postgres-test/drop-if-exists-and-create-db!
metabase.driver.sql-jdbc.execute/execute-statement!
metabase.indexed-entities.models.model-index/add-values!
metabase.driver.sql-jdbc.connection.ssh-tunnel-test/start-mock-servers!
metabase.driver.sql-jdbc.connection.ssh-tunnel-test/stop-mock-servers!
metabase.indexed-entities.task.index-values/job-init!
metabase.model-persistence.models.persisted-info/ready-database!
metabase.model-persistence.task.persist-refresh/job-init!
metabase.model-persistence.task.persist-refresh/refresh-tables!
metabase.model-persistence.task.persist-refresh/schedule-persistence-for-database!
metabase.models.collection.graph-test/clear-graph-revisions!
metabase.models.collection.graph-test/do-with-n-temp-users-with-personal-collections!
metabase.models.field-values/create-or-update-full-field-values!
metabase.models.moderation-review/create-review!
metabase.models.on-demand-test/add-dashcard-with-parameter-mapping!
metabase.permissions.core/grant-application-permissions!
metabase.permissions.core/grant-collection-read-permissions!
metabase.permissions.core/grant-collection-readwrite-permissions!
metabase.permissions.core/grant-full-data-permissions!
metabase.permissions.core/grant-native-readwrite-permissions!
metabase.permissions.core/grant-permissions!
metabase.permissions.core/revoke-application-permissions!
metabase.permissions.core/revoke-data-perms!
metabase.permissions.core/update-data-perms-graph!
metabase.permissions.core/update-group-permissions!
metabase.permissions.models.collection.graph-test/clear-graph-revisions!
metabase.permissions.models.collection.graph-test/do-with-n-temp-users-with-personal-collections!
metabase.models.setting-test/test-user-local-allowed-setting!
metabase.models.setting-test/test-user-local-only-setting!
metabase.models.setting.cache/restore-cache!
metabase.models.setting/set!
metabase.models.setting/validate-settings-formatting!
metabase.permissions.models.permissions/grant-application-permissions!
metabase.permissions.models.permissions/grant-collection-read-permissions!
metabase.permissions.models.permissions/grant-collection-readwrite-permissions!
@@ -100,18 +90,13 @@
metabase.permissions.models.permissions/update-data-perms-graph!
metabase.permissions.models.permissions/update-group-permissions!
metabase.permissions.test-util/with-restored-perms!
metabase.public-sharing-rest.api-test/add-card-to-dashboard!
metabase.public-sharing.api-test/add-card-to-dashboard!
metabase.pulse.send/send-notifications!
metabase.pulse.send/send-pulse!
metabase.query-processor.streaming.interface/begin!
metabase.query-processor.streaming.interface/finish!
metabase.query-processor.streaming.interface/write-row!
metabase.sample-data.impl/try-to-extract-sample-database!
metabase.settings.models.setting-test/test-user-local-allowed-setting!
metabase.settings.models.setting-test/test-user-local-only-setting!
metabase.settings.models.setting.cache/restore-cache!
metabase.settings.models.setting/set!
metabase.settings.models.setting/validate-settings-formatting!
metabase.sample-data/try-to-extract-sample-database!
metabase.setup.core/create-token!
metabase.sso.ldap-test-util/start-ldap-server!
metabase.sync.core/sync-database!
@@ -119,8 +104,7 @@
metabase.sync.sync-metadata/sync-db-metadata!
metabase.sync.util-test/sync-database!
metabase.sync.util/store-sync-summary!
metabase.task.core/delete-task!
metabase.task.impl/delete-task!
metabase.task/delete-task!
metabase.test.data.bigquery-cloud-sdk/execute!
metabase.test.data.impl.get-or-create/set-test-db-permissions!
metabase.test.data.impl/copy-db-tables-and-fields!
@@ -142,6 +126,7 @@
metabase.test/initialize-if-needed!
metabase.test/test-helpers-set-global-values!
metabase.test/with-temp-env-var-value!
metabase.upload.impl-test/set-local-infile!
metabase.upload-test/set-local-infile!
metabase.util.files/create-dir-if-not-exists!
metabase.warehouse-schema.models.field-values/create-or-update-full-field-values!}}}}
metabase.util.ssh-test/start-mock-servers!
metabase.util.ssh-test/stop-mock-servers!}}}}

View File

@@ -0,0 +1,13 @@
(ns macros.metabase.api.card-test)
(defmacro with-ordered-items
[collection model-and-name-syms & body]
`(let ~(into []
(comp (partition-all 2)
(mapcat (fn [[model binding]]
[binding model])))
model-and-name-syms)
~collection
~@(->> (partition-all 2 model-and-name-syms)
(map second))
~@body))

View File

@@ -0,0 +1,24 @@
(ns macros.metabase.api.collection-test)
(defmacro with-collection-hierarchy! [bindings & body]
`(let [~'a nil
~'b nil
~'c nil
~'d nil
~'e nil
~'f nil
~'g nil]
~'a
~'b
~'c
~'d
~'e
~'f
~'g
~bindings
~@body))
(defmacro with-some-children-of-collection! [collection-or-id-or-nil & body]
`(let [~'&ids ~collection-or-id-or-nil]
~'&ids
~@body))

View File

@@ -0,0 +1,13 @@
(ns macros.metabase.api.embed-test)
(defmacro with-embedding-enabled-and-temp-card-referencing!
[table-kw field-kw [card-binding] & body]
`(let [~card-binding [~table-kw ~field-kw]]
~@body))
(defmacro with-embedding-enabled-and-temp-dashcard-referencing!
[table-kw field-kw [dash-binding card-binding dashcard-binding] & body]
`(let [~dash-binding [~table-kw ~field-kw]
~(or card-binding '_) nil
~(or dashcard-binding '_) nil]
~@body))

View File

@@ -1,24 +0,0 @@
(ns macros.metabase.collections-rest.api-test)
(defmacro with-collection-hierarchy! [bindings & body]
`(let [~'a nil
~'b nil
~'c nil
~'d nil
~'e nil
~'f nil
~'g nil]
~'a
~'b
~'c
~'d
~'e
~'f
~'g
~bindings
~@body))
(defmacro with-some-children-of-collection! [collection-or-id-or-nil & body]
`(let [~'&ids ~collection-or-id-or-nil]
~'&ids
~@body))

View File

@@ -1,13 +0,0 @@
(ns macros.metabase.embedding-rest.api.embed-test)
(defmacro with-embedding-enabled-and-temp-card-referencing!
[table-kw field-kw [card-binding] & body]
`(let [~card-binding [~table-kw ~field-kw]]
~@body))
(defmacro with-embedding-enabled-and-temp-dashcard-referencing!
[table-kw field-kw [dash-binding card-binding dashcard-binding] & body]
`(let [~dash-binding [~table-kw ~field-kw]
~(or card-binding '_) nil
~(or dashcard-binding '_) nil]
~@body))

View File

@@ -0,0 +1,4 @@
(ns macros.metabase.models.params.chain-filter-test)
(defmacro chain-filter [& _]
`(metabase.models.params.chain-filter/chain-filter nil nil))

View File

@@ -0,0 +1,12 @@
(ns macros.metabase.models.query-analysis-test)
(defmacro with-test-setup [& body]
`(let [~'card-id 1
~'table-id 2
~'tax-id 3
~'total-id 4]
~'card-id
~'table-id
~'tax-id
~'total-id
~@body))

View File

@@ -0,0 +1,9 @@
(ns macros.metabase.models.user-test)
(defmacro with-groups [bindings & body]
`(let ~(into []
(comp (partition-all 3)
(mapcat (fn [[group-binding properties members]]
[group-binding [properties members]])))
bindings)
~@body))

View File

@@ -1,4 +0,0 @@
(ns macros.metabase.parameters.chain-filter-test)
(defmacro chain-filter [& _]
`(metabase.parameters.chain-filter/chain-filter nil nil))

View File

@@ -0,0 +1,13 @@
(ns macros.metabase.public-sharing.api-test)
(defmacro with-sharing-enabled-and-temp-card-referencing!
[table-kw field-kw [card-binding] & body]
`(let [~card-binding [~table-kw ~field-kw]]
~@body))
(defmacro with-sharing-enabled-and-temp-dashcard-referencing!
[table-kw field-kw [dash-binding card-binding dashcard-binding] & body]
`(let [~dash-binding [~table-kw ~field-kw]
~(or card-binding '_) nil
~(or dashcard-binding '_) nil]
~@body))

View File

@@ -1,13 +0,0 @@
(ns macros.metabase.public-sharing-rest.api-test)
(defmacro with-sharing-enabled-and-temp-card-referencing!
[table-kw field-kw [card-binding] & body]
`(let [~card-binding [~table-kw ~field-kw]]
~@body))
(defmacro with-sharing-enabled-and-temp-dashcard-referencing!
[table-kw field-kw [dash-binding card-binding dashcard-binding] & body]
`(let [~dash-binding [~table-kw ~field-kw]
~(or card-binding '_) nil
~(or dashcard-binding '_) nil]
~@body))

View File

@@ -1,13 +0,0 @@
(ns macros.metabase.queries-rest.api.card-test)
(defmacro with-ordered-items
[collection model-and-name-syms & body]
`(let ~(into []
(comp (partition-all 2)
(mapcat (fn [[model binding]]
[binding model])))
model-and-name-syms)
~collection
~@(->> (partition-all 2 model-and-name-syms)
(map second))
~@body))

View File

@@ -1,12 +0,0 @@
(ns macros.metabase.query-analysis.models.query-analysis-test)
(defmacro with-test-setup [& body]
`(let [~'card-id 1
~'table-id 2
~'tax-id 3
~'total-id 4]
~'card-id
~'table-id
~'tax-id
~'total-id
~@body))

View File

@@ -1,9 +0,0 @@
(ns macros.metabase.users.models.user-test)
(defmacro with-groups! [bindings & body]
`(let ~(into []
(comp (partition-all 3)
(mapcat (fn [[group-binding properties members]]
[group-binding [properties members]])))
bindings)
~@body))

View File

@@ -0,0 +1,16 @@
(ns macros.metabase.xrays.domain-entities.malli)
(defmacro define-getters-and-setters [schema & defs]
`(do
(comment ~schema) ;; Reference the schema
~@(for [[sym _path] (partition 2 defs)]
`(do
(defn ~(vary-meta sym
assoc :export true)
"docs" [_x#] nil)
(defn ~(vary-meta (symbol (str "with-" (name sym)))
assoc :export true)
"docs" [x# _new#] x#)
(defn ~(vary-meta (symbol (str (name sym) "-js"))
assoc :export true)
"docs" [x#] x#)))))

View File

@@ -1,5 +0,0 @@
(ns macros.metabase-enterprise.database-routing.e2e-test)
(defmacro with-temp-dbs! [bindings & body]
`(let [~@(interleave bindings (repeat nil))]
~@body))

View File

@@ -1,38 +0,0 @@
(ns hooks.clojure.core
(:require
[clj-kondo.hooks-api :as hooks]
[clojure.string :as str]))
(defn- all-uppercase? [symb]
(let [s (str symb)]
(= s #_{:clj-kondo/ignore [:discouraged-var]} (str/upper-case s))))
(defn- has-underscores? [symb]
(str/includes? (str symb) "_"))
(defn- check-symbol-is-kebab-case [symbol-node]
(let [symb (hooks/sexpr symbol-node)]
(when (or (all-uppercase? symb)
(has-underscores? symb))
(hooks/reg-finding! (assoc (meta symbol-node)
:message "Use kebab-case for function and variable names; don't use special notation for constants."
:type :metabase/check-def-kebab-case)))))
(defn lint-def* [{:keys [node]}]
(let [[_def & args] (:children node)
name-symbol (some (fn [arg]
(when (and (hooks/token-node? arg)
(symbol? (hooks/sexpr arg)))
arg))
args)]
(check-symbol-is-kebab-case name-symbol)))
(defn lint-def [x]
(lint-def* x)
x)
(comment
(defn x []
(let [node (-> "(def ^:private TYPE->MODEL {\"document\" :model/Document})"
hooks/parse-string)]
(lint-def {:node node}))))

View File

@@ -1,52 +0,0 @@
(ns hooks.clojure.core.def
(:require
[clj-kondo.hooks-api :as hooks]
[clojure.string :as str]))
(defn- uppercase-name?
"Whether the symbol is clearly an uppercase name like `TYPE-CONSTANTS` or `SOME_CONSTANT` or `TYPE->MODEL`."
[s]
(and (re-find #"[A-Z0-9]+(?:[-_]>?[A-Z0-9]+)+" s)
;; ignore stuff like `->SCREAMING_SNAKE_CASE`... this is ok
(not (str/includes? s "SCREAMING_SNAKE_CASE"))))
(defn- has-underscores? [s]
(and (str/includes? s "_")
;; ignore stuff like `->snake_case` and `->SCREAMING_SNAKE_CASE`
(not (str/includes? s "snake_case"))
(not (str/includes? s "SCREAMING_SNAKE_CASE"))))
(defn- check-symbol-is-kebab-case [symbol-node]
(let [symb (hooks/sexpr symbol-node)
s (str symb)]
(cond
(uppercase-name? s)
(hooks/reg-finding!
(assoc (meta symbol-node)
:message "Use lower-case names for functions and variables; don't use special notation for constants. [:metabase/check-def-check-not-all-uppercase]"
:type :metabase/check-def-check-not-uppercase-name))
(has-underscores? s)
(hooks/reg-finding!
(assoc (meta symbol-node)
:message "Use kebab-case names for functions and variables. [:metabase/check-def-no-underscores]"
:type :metabase/check-def-no-underscores)))))
(defn lint-def* [{:keys [node]}]
(let [[_def & args] (:children node)
name-symbol (some (fn [arg]
(when (and (hooks/token-node? arg)
(symbol? (hooks/sexpr arg)))
arg))
args)]
(check-symbol-is-kebab-case name-symbol)))
(defn lint-def [x]
(lint-def* x)
x)
(comment
(defn x []
(let [node (-> "(def ^:private TYPE->MODEL {\"document\" :model/Document})"
hooks/parse-string)]
(lint-def {:node node}))))

View File

@@ -1,7 +0,0 @@
(ns hooks.clojure.core.defmacro
(:require
[hooks.clojure.core.def]))
(defn lint-defmacro [x]
(hooks.clojure.core.def/lint-def* x)
x)

View File

@@ -2,7 +2,6 @@
(:require
[clj-kondo.hooks-api :as hooks]
[clojure.string :as str]
[hooks.clojure.core.def]
[hooks.common]))
(defn- check-arglists [report-node arglists]
@@ -50,5 +49,4 @@
(defn lint-defmulti [x]
(defmulti-check-for-arglists-metadata (:node x))
(hooks.clojure.core.def/lint-def* x)
x)

View File

@@ -2,7 +2,6 @@
(:require
[clj-kondo.hooks-api :as hooks]
[clojure.string :as str]
[hooks.clojure.core.def]
[hooks.common]))
(defn- end-with-exclamation?
@@ -51,11 +50,6 @@
(non-thread-safe-form-should-end-with-exclamation* node config))
{:node node})
(defn lint-defn [x]
(non-thread-safe-form-should-end-with-exclamation x)
(hooks.clojure.core.def/lint-def* x)
x)
(comment
(require '[clj-kondo.core :as clj-kondo])
(def form (str '(defmacro a

View File

@@ -1,7 +1,6 @@
(ns hooks.clojure.core.ns
(:require
[clj-kondo.hooks-api :as hooks]
[clojure.string :as str]
[hooks.common]
[hooks.common.modules :as modules]))
@@ -60,15 +59,12 @@
(printf "Don't know how to figure out what namespace is being required in %s\n" (pr-str node)))))
args)))
(defn- ns-form-node->ns-symb-node [ns-form-node]
(some (fn [node]
(when (and (hooks/token-node? node)
(not= (hooks/sexpr node) 'ns))
node))
(:children ns-form-node)))
(defn- ns-form-node->ns-symb [ns-form-node]
(some-> (ns-form-node->ns-symb-node ns-form-node)
(some-> (some (fn [node]
(when (and (hooks/token-node? node)
(not= (hooks/sexpr node) 'ns))
node))
(:children ns-form-node))
hooks/sexpr))
(defn- lint-modules [ns-form-node config]
@@ -87,69 +83,9 @@
:message error
:type :metabase/modules))))))))
(defn- lint-namespace-name [ns-form-node]
(when-let [ns-symb-node (ns-form-node->ns-symb-node ns-form-node)]
(when-not (contains? (hooks.common/ignored-linters ns-symb-node) :metabase/namespace-name)
(let [parts (str/split (str (hooks/sexpr ns-symb-node)) #"\.")]
(when (and (<= (count parts) 2)
(#{"metabase" "metabase-enterprise"} (first parts))
;; exclude test namespaces for now
(not (str/ends-with? (last parts) "test")))
(hooks/reg-finding! (assoc (meta ns-symb-node)
:message "Metabase namespaces should have the form metabase[-enterprise].<module>.* [:metabase/namespace-name]"
:type :metabase/namespace-name)))))))
(defn- ns-form-node->import-node [ns-form-node]
(some (fn [node]
(when (and (hooks/list-node? node)
(let [first-child (first (:children node))]
(and (hooks/keyword-node? first-child)
(= (hooks/sexpr first-child) :import))))
node))
(:children ns-form-node)))
(defn- jsqlparser-import?
"Check if a symbol or string represents a jsqlparser import."
[s]
(str/starts-with? (str s) "net.sf.jsqlparser"))
(defn- lint-jsqlparser-imports [ns-form-node]
(when-let [import-node (ns-form-node->import-node ns-form-node)]
(doseq [node (rest (:children import-node))
:when (not (contains? (hooks.common/ignored-linters node) :metabase/no-jsqlparser-imports))]
(cond
;; Handle (net.sf.jsqlparser.foo Bar Baz) form
(hooks/list-node? node)
(let [package-node (first (:children node))
package-name (when package-node (str (hooks/sexpr package-node)))]
(when (jsqlparser-import? package-name)
(hooks/reg-finding! (assoc (meta node)
:message "Don't import net.sf.jsqlparser classes directly. This code should be agnostic to the parsing library. [:metabase/no-jsqlparser-imports]"
:type :metabase/no-jsqlparser-imports))))
;; Handle net.sf.jsqlparser.foo.Bar form (single class import)
(hooks/token-node? node)
(when (jsqlparser-import? (hooks/sexpr node))
(hooks/reg-finding! (assoc (meta node)
:message "Don't import net.sf.jsqlparser classes directly. This code should be agnostic to the parsing library. [:metabase/no-jsqlparser-imports]"
:type :metabase/no-jsqlparser-imports)))))))
(defn lint-ns [x]
(doto (:node x)
lint-require-shapes
lint-requires-on-new-lines
(lint-modules (modules/config x))
lint-namespace-name
lint-jsqlparser-imports)
(lint-modules (modules/config x)))
x)
(comment
(lint-ns
{:node (hooks/parse-string
(pr-str
'(ns hooks.clojure.core.ns
(:require
[clj-kondo.hooks-api :as hooks]
[clojure.string :as str]
[hooks.common]
[hooks.common.modules :as modules]))))}))

View File

@@ -52,13 +52,7 @@
(defn- lint-require* [node current-ns config]
(let [[_require & args] (:children node)
required-namespaces (->> (keep unwrap-require args)
;; `require` runs through `cc/load-libs` which allows
;; #{:as :reload :reload-all :require :use :verbose :refer :as-alias}
;; as args. If you use a REPL tool that applies clj-kondo linting (e.g., Clojure MCP)
;; you may wish to `:reload` or `:reload-all` a namespace. So we shouldn't assume all
;; args are keywords:
(remove keyword?))]
required-namespaces (keep unwrap-require args)]
(lint* node current-ns config required-namespaces)))
(defn- lint-requiring-resolve* [node current-ns config]

View File

@@ -2,8 +2,7 @@
(:require
[clj-kondo.hooks-api :as hooks]
[clojure.string :as str]
[hooks.common]
[hooks.common.modules :as modules]))
[hooks.common]))
(defn- warn-about-disallowed-parallel-forms [form config]
(letfn [(error! [form message]
@@ -103,39 +102,7 @@
(:children node))))]
(walk node)))
(defn- test-namespace-lives-outside-of-module-system? [ns-symb]
(some (fn [prefix]
(str/starts-with? (name ns-symb) prefix))
["build-drivers."
"build."
"i18n." ; bin/i18n
"lint-migrations-file-test"
"main-test" ; bin/release-list
"metabase.deps-edn-test"
"metabase.driver."
"metabase.test."
"metabase.test.data."
"metabuild-common."])) ; utils for build scripts
(let [warned-namespaces (atom #{})]
(defn deftest-check-in-valid-module
"Check whether a `deftest` lives in a known module -- important so our system to only run a subset of tests based on
which modules have changes picks them up correctly. (DEV-1125)"
[{:keys [node], ns-symb :ns, :as input}]
(when (and ns-symb
(not (test-namespace-lives-outside-of-module-system? ns-symb))
;; only warn once per namespace
(not (contains? @warned-namespaces ns-symb)))
(swap! warned-namespaces conj ns-symb)
(let [known-modules (set (keys (:metabase/modules (modules/config input))))
current-module (modules/module ns-symb)]
(when (or (not current-module)
(not (contains? known-modules current-module)))
(hooks/reg-finding! (assoc (meta node)
:message (format "All tests must live in a known module; %s is not a known module" (pr-str current-module))
:type :metabase/tests-must-live-in-known-modules)))))))
(defn deftest [{:keys [node cljc lang config], :as input}]
(defn deftest [{:keys [node cljc lang config]}]
;; run [[deftest-check-parallel]] only once... if this is a `.cljc` file only run it for the `:clj` analysis, no point
;; in running it twice.
(when (or (not cljc)
@@ -143,8 +110,7 @@
(deftest-check-parallel node (get-in config [:linters :metabase/validate-deftest])))
(deftest-check-not-horrifically-long node (get-in config [:linters :metabase/i-like-making-cams-eyes-bleed-with-horrifically-long-tests]))
(deftest-check-no-driver-keywords node (get-in config [:linters :metabase/disallow-hardcoded-driver-names-in-tests]))
(deftest-check-in-valid-module input)
input)
{:node node})
;;; this is a hacky way to determine whether these namespaces are required in the `ns` form or not... basically `:ns`
;;; will come back as `nil` if they are not.

View File

@@ -1,6 +1,4 @@
(ns hooks.common.modules
(:require
[clojure.string :as str]))
(ns hooks.common.modules)
(defn ignored-namespace? [config ns-symb]
(some
@@ -19,19 +17,14 @@
(module 'metabase-enterprise.whatever.core) => enterprise/whatever"
[ns-symb]
{:pre [(simple-symbol? ns-symb)]}
;; treat something like `metabase.driver-test` (for a module that hasn't fully been updated to use `.core`
;; namespaces) as being in the `driver` module
(let [ns-symb (if (str/ends-with? (name ns-symb) "-test")
(symbol (str/replace (name ns-symb) #"-test$" ""))
ns-symb)]
(or (some->> (re-find #"^metabase-enterprise\.([^.]+)" (str ns-symb))
second
(symbol "enterprise"))
(some-> (re-find #"^metabase\.([^.]+)" (str ns-symb))
second
symbol))))
(or (some->> (re-find #"^metabase-enterprise\.([^.]+)" (str ns-symb))
second
(symbol "enterprise"))
(some-> (re-find #"^metabase\.([^.]+)" (str ns-symb))
second
symbol)))
(defn- module-api-namespaces
(defn module-api-namespaces
"Set of API namespace symbols for a given module. `:any` means you can use anything, there are no API namespaces for
this module (yet). If unspecified, the default is just the `<module>.core` namespace."
[config module]
@@ -51,12 +44,6 @@
(symbol (str ns-prefix ".core"))
(symbol (str ns-prefix ".init"))}))))
(defn- module-friends
[config module]
"Set of modules that are `:friends` of `module`, i.e. allowed to use *any* namespace from the module, not just the
designated [[module-api-namespaces]]."
(set (get-in config [:metabase/modules module :friends])))
(defn allowed-modules
"Set of namespace symbols that `module` is allowed to use. `:any` means it's allowed to use anything."
[config module]
@@ -67,22 +54,11 @@
(or (= allowed-modules :any)
(contains? (set allowed-modules) required-module))))
(defn- allowed-module-namespace? [config current-module ns-symb]
(defn allowed-module-namespace? [config ns-symb]
(let [module (module ns-symb)
module-api-namespaces (module-api-namespaces config module)
module-friends (module-friends config module)]
module-api-namespaces (module-api-namespaces config module)]
(or (empty? module-api-namespaces)
(contains? module-api-namespaces ns-symb)
(contains? module-friends current-module))))
(defn- rest-module? [module]
(str/ends-with? module "-rest"))
(defn- routes-module? [module]
(str/ends-with? module "-routes"))
(defn- core-module? [module]
(str/ends-with? module "core"))
(contains? module-api-namespaces ns-symb))))
(defn usage-error
"Find usage errors when a `required-namespace` is required in the `current-module`. Returns a string describing the
@@ -98,19 +74,8 @@
current-module
current-module)
(not (allowed-module-namespace? config current-module required-namespace))
(not (allowed-module-namespace? config required-namespace))
(format "Namespace %s is not an allowed external API namespace for the %s module. [:metabase/modules %s :api]"
required-namespace
required-module
required-module)
;; (for now) rest modules are allowed to use one another; `routes` is ok because it collects routes together
;; and `core` is ok because [[metabase.core.init]] might need to init some of the `-routes` modules'
;; namespaces
(and (not ((some-fn rest-module? routes-module? core-module?) current-module))
(rest-module? required-module))
(format "Do not use -rest modules (%s) in non-rest modules (%s) -- move things from %s to %s if needed"
required-module
current-module
required-module
(symbol (str/replace required-module #"-rest$" "")))))))
required-module)))))

View File

@@ -1,67 +1,7 @@
(ns hooks.metabase.api.macros
(:require
[clj-kondo.hooks-api :as api]
[clojure.string :as str]))
(defn- validate-route!
"Make sure the route uses kebab-case rather than snake_case."
[route]
(letfn [(validate-route-string! [route-string]
(when (and (api/string-node? route-string)
(str/includes? (api/sexpr route-string) "_"))
(api/reg-finding! (assoc (meta route-string)
:message "REST API routes and query params should use kebab-case, not snake_case"
:type :metabase/validate-defendpoint-route-uses-kebab-case))))]
(cond
(api/string-node? route)
(validate-route-string! route)
;; otherwise route is a vector like ["/abc/:d/" :d <regex>]
(api/vector-node? route)
(validate-route-string! (first (:children route))))))
(defn- validate-query-params!
"Make sure query params use kebab-case rather than snake_case.
This only works on inline schemas for now. If you define the schema elsewhere then we'd have to implement this
validation in Clojure-land since we can't resolve the schemas in Kondo. It should however still work on complicated
schemas like
[:merge <map-1> <map-2>]"
[query-param-schema-node]
(letfn [(map-schema? [node]
(and (api/vector-node? node)
(when-let [first-child (first (:children node))]
(and (api/keyword-node? first-child)
(= (api/sexpr first-child) :map)))))
(map-schema-keys [{:keys [children], :as _node}]
(let [child-schemas (if (api/map-node? (second children))
(drop 2 children)
(drop 1 children))]
(keep (fn [child-schema]
(when (api/vector-node? child-schema)
(when-let [k (first (:children child-schema))]
(when (api/keyword-node? k)
k))))
child-schemas)))
(validate-map-schema! [form]
;; only warn once per endpoint -- just once on the first bad node
(when-let [bad-node (some (fn [key-node]
(when (str/includes? (str (api/sexpr key-node)) "_")
key-node))
(map-schema-keys form))]
(api/reg-finding! (assoc (meta bad-node)
:message "REST API routes and query params should use kebab-case, not snake_case"
:type :metabase/validate-defendpoint-query-params-use-kebab-case))))
(validate! [node]
(cond
(and (api/vector-node? node)
(map-schema? node))
(validate-map-schema! node)
(api/vector-node? node)
(doseq [child (:children node)]
(validate! child))))]
(validate! query-param-schema-node)))
[hooks.common]))
(defn defendpoint
[arg]
@@ -78,25 +18,16 @@
(cons nil more))
[params & body] more
[bindings schemas] (when (api/vector-node? params)
(loop [bindings [], schemas [], [x y z & more] (:children params)]
(loop [bindings [], schemas [], [x y & more] (:children params)]
(cond
(not x)
[bindings schemas]
(and (api/keyword-node? y)
(= (api/sexpr y) :-))
(recur (conj bindings x) (conj schemas z) more)
(= (api/sexpr x) :-)
(recur bindings (conj schemas y) more)
:else
(recur (conj bindings x) (conj schemas nil) (list* y z more)))))]
(validate-route! route)
(when-not result-schema
(api/reg-finding! (assoc (meta node)
:message "All REST API endpoints should have a detailed response schema"
:type :metabase/validate-defendpoint-has-response-schema)))
(let [[_route-param-schema query-param-schema _body-param-schema] schemas]
(when query-param-schema
(validate-query-params! query-param-schema)))
(recur (conj bindings x) schemas (cons y more)))))]
(-> (api/list-node
(list
(api/token-node 'do)
@@ -123,27 +54,3 @@
body))))
(with-meta (meta node)))))]
(update arg :node update-defendpoint)))
(comment
(defn -defendpoint [form]
(-> {:node (-> form pr-str api/parse-string)}
defendpoint
:node
api/sexpr))
(defn x []
(-defendpoint '(api.macros/defendpoint :get "/:id/syncable_schemas" ; snake_case route should trigger a warning
"Returns a list of all syncable schemas found for the database `id`."
[{:keys [id]} :- [:map
[:id ms/PositiveInt]]
;; snake_case query parameter, should trigger a warning
{:keys [filter_by]} :- [:map
[:filter_by {:optional true} :string]]]
(let [db (get-database id)]
(api/check-403 (or (:is_attached_dwh db)
(and (mi/can-write? db)
(mi/can-read? db))))
(->> db
(driver/syncable-schemas (:engine db))
(vec)
(sort)))))))

View File

@@ -1,14 +0,0 @@
(ns hooks.metabase.app-db.schema-migrations-test.impl
(:require [clj-kondo.hooks-api :as hooks]))
(defn test-migrations
[{{[_ migration-range {[binding] :children} & body] :children} :node}]
(let [node* (hooks/list-node
(list*
(hooks/token-node `let)
(hooks/vector-node
[binding (hooks/list-node (list (hooks/token-node `fn)
(hooks/vector-node [(hooks/token-node '&) (hooks/token-node '_args)])))])
migration-range
body))]
{:node node*}))

View File

@@ -0,0 +1,14 @@
(ns hooks.metabase.db.schema-migrations-test.impl
(:require [clj-kondo.hooks-api :as hooks]))
(defn test-migrations
[{{[_ migration-range {[binding] :children} & body] :children} :node}]
(let [node* (hooks/list-node
(list*
(hooks/token-node `let)
(hooks/vector-node
[binding (hooks/list-node (list (hooks/token-node `fn)
(hooks/vector-node [(hooks/token-node '&) (hooks/token-node '_args)])))])
migration-range
body))]
{:node node*}))

View File

@@ -1,18 +0,0 @@
(ns hooks.metabase.driver.mongo-test
(:require
[clj-kondo.hooks-api :as hooks]
[hooks.common]))
(defn with-describe-table-for-sample
"Forward bindings for results of functions used in mongo's describe-table implementation."
[{{[_ _ & body] :children} :node}]
(let [node* (hooks/list-node
(list*
(hooks/token-node 'let)
(hooks/vector-node
[(hooks/token-node 'dbfields) (hooks/token-node '(delay nil))
(hooks/token-node 'ftree) (hooks/token-node '(delay nil))
(hooks/token-node 'nested-fields) (hooks/token-node '(delay nil))])
body))]
;; Avoid warnings when not all from the dbfields, ftree and nested-fields are used.
{:node (hooks.common/update-ignored-linters node* conj :unused-binding)}))

View File

@@ -33,36 +33,18 @@
(defclause [ag:var var] field-or-expression FieldOrExpressionDef)
=>
(def ag:var [FieldOrExpressionDef])"
[{:keys [node], :as x}]
(let [[_defclause clause-name & arg-specs] (:children node)]
(assoc x :node (-> (hooks/list-node
(list
(hooks/token-node 'do)
(-> (hooks/keyword-node (keyword "metabase.legacy-mbql.schema" (name (hooks/sexpr clause-name))))
(with-meta (meta clause-name))
(hooks/reg-keyword! 'metabase.legacy-mbql.schemas.macros/defclause))
(collect-defclause-body-schemas arg-specs)))
(with-meta (meta node))))))
(defn defclause*
"e.g.
(defclause [ag:var var] field-or-expression FieldOrExpressionDef)
=>
(def ag:var [FieldOrExpressionDef])"
[{:keys [node], :as x}]
(let [[_defclause clause-name schema] (:children node)]
(assoc x :node (-> (hooks/list-node
(list
(hooks/token-node 'do)
(-> (hooks/keyword-node (keyword "metabase.legacy-mbql.schema" (name (hooks/sexpr clause-name))))
(with-meta (meta clause-name))
(hooks/reg-keyword! 'metabase.legacy-mbql.schemas.macros/defclause*))
schema))
(with-meta (meta node))))))
[{:keys [node]}]
(let [[_defclause clause-name-form & arg-specs] (:children node)]
{:node (-> (hooks/list-node
(list
(hooks/token-node 'def)
(unwrap-defclause-clause-name clause-name-form)
(hooks/string-node "Docstring.")
(collect-defclause-body-schemas arg-specs)))
(with-meta (meta node)))}))
(comment
(defn- expand-defclause [form]
(defn- defclause* [form]
(hooks/sexpr
(:node
(defclause
@@ -73,35 +55,15 @@
form)))}))))
(defn- x []
(expand-defclause '(defclause var
field-or-expression FieldOrExpressionDef)))
(defclause* '(defclause [ag:var var]
field-or-expression FieldOrExpressionDef)))
;; =>
(do :metabase.legacy-mbql.schema/var [FieldOrExpressionDef])
(def ag:var "Docstring." [FieldOrExpressionDef])
(defn- y []
(expand-defclause '(defclause var
x (some-list-schema)
y (optional FieldOrExpressionDef)
z (rest (some-other-schema)))))
(defclause* '(defclause var
x (some-list-schema)
y (optional FieldOrExpressionDef)
z (rest (some-other-schema)))))
;; =>
(do :metabase.legacy-mbql.schema/var [(some-list-schema) FieldOrExpressionDef (some-other-schema)]))
(defn one-of [{:keys [node], :as x}]
(let [[_one-of & schema-nodes] (:children node)
node' (-> (hooks/vector-node
(mapv (fn [schema-node]
(if (and (hooks/token-node? schema-node)
(simple-symbol? (hooks/sexpr schema-node)))
(-> (hooks/keyword-node (keyword "mteabase.legacy-mbql.schema" (name (hooks/sexpr schema-node))))
(with-meta (meta schema-node)))
schema-node))
schema-nodes))
(with-meta (meta node)))]
(assoc x :node node')))
(comment
(defn- expand-one-of [form]
(hooks/sexpr (:node (one-of {:node (hooks/parse-string (with-out-str (clojure.pprint/pprint form)))}))))
(defn- z []
(expand-one-of '(one-of absolute-datetime relative-datetime time))))
(def var "Docstring." [(some-list-schema) FieldOrExpressionDef (some-other-schema)]))

View File

@@ -0,0 +1,294 @@
(ns hooks.metabase.models.setting
(:require
[clj-kondo.hooks-api :as hooks]
[hooks.common :as common]))
;;; TODO -- move this into
(def ^:private ignored-implicit-export?
'#{active-users-count
admin-email
analytics-uuid
anon-tracking-enabled
api-key
attachment-table-row-limit
audit-max-retention-days
bcc-enabled?
breakout-bin-width
check-for-updates
cloud-gateway-ips
config-from-file-sync-databases
custom-homepage
custom-homepage-dashboard
database-enable-actions
deprecation-notice-version
dismissed-custom-dashboard-toast
dismissed-onboarding-sidebar-link
email-configured?
email-from-address
email-from-name
email-reply-to
email-smtp-host
email-smtp-password
email-smtp-port
email-smtp-security
email-smtp-username
embedding-app-origin
embedding-secret-key
enable-password-login
enable-public-sharing
enable-query-caching
engines
enum-cardinality-threshold
follow-up-email-sent
google-auth-auto-create-accounts-domain
google-auth-client-id
google-auth-configured
google-auth-enabled
has-sample-database?
has-user-setup
help-link
help-link-custom-destination
instance-creation
is-hosted?
is-metabot-enabled
jdbc-data-warehouse-max-connection-pool-size
jdbc-data-warehouse-unreturned-connection-timeout-seconds
jwt-attribute-email
jwt-attribute-firstname
jwt-attribute-groups
jwt-attribute-lastname
jwt-configured
jwt-enabled
jwt-group-mappings
jwt-group-sync
jwt-identity-provider-uri
jwt-user-provisioning-enabled?
jwt-shared-secret
last-acknowledged-version
last-used-native-database-id
ldap-attribute-email
ldap-attribute-firstname
ldap-attribute-lastname
ldap-bind-dn
ldap-configured?
ldap-enabled
ldap-group-base
ldap-group-mappings
ldap-group-membership-filter
ldap-group-sync
ldap-host
ldap-password
ldap-port
ldap-user-provisioning-enabled?
ldap-security
ldap-sync-user-attributes
ldap-sync-user-attributes-blacklist
ldap-user-base
ldap-user-filter
load-analytics-content
map-tile-server-url
metabot-default-embedding-model
metabot-feedback-url
metabot-get-prompt-templates-url
metabot-prompt-generator-token-limit
multi-setting-read-only
notebook-native-preview-shown
notebook-native-preview-sidebar-width
notification-link-base-url
num-metabot-choices
openai-api-key
openai-available-models
openai-model
openai-organization
other-sso-enabled?
password-complexity
persist-models-enabled
persisted-model-refresh-cron-schedule
premium-embedding-token
prometheus-server-port
query-caching-max-kb
query-caching-max-ttl
query-caching-min-ttl
query-caching-ttl-ratio
redirect-all-requests-to-https
reset-token-ttl-hours
retired-setting
retry-initial-interval
retry-max-attempts
retry-max-interval-millis
retry-multiplier
retry-randomization-factor
saml-application-name
saml-attribute-email
saml-attribute-firstname
saml-attribute-group
saml-attribute-lastname
saml-configured
saml-enabled
saml-group-mappings
saml-group-sync
saml-identity-provider-certificate
saml-identity-provider-issuer
saml-identity-provider-uri
saml-keystore-alias
saml-keystore-password
saml-keystore-path
saml-user-provisioning-enabled?
send-email-on-first-login-from-new-device
send-new-sso-user-admin-email?
session-cookie-samesite
session-cookies
session-timeout
setup-token
show-database-syncing-modal
show-metabase-links
site-url
site-uuid
site-uuid-for-premium-features-token-checks
site-uuid-for-unsubscribing-url
site-uuid-for-version-info-fetching
slack-app-token
slack-cached-channels-and-usernames
slack-channels-and-usernames-last-updated
slack-files-channel
slack-token
slack-token-valid?
snowplow-available
snowplow-enabled
snowplow-url
sql-jdbc-fetch-size
ssh-heartbeat-interval-sec
ssl-certificate-public-key
startup-time-millis
token-features
token-status
toucan-name
trial-banner-dismissal-timestamp
uncached-setting
user-visibility
version
version-info
version-info-last-checked})
(defn- defsetting-lint [node setting-name docstring options-list]
(let [anon-binding (common/with-macro-meta (hooks/token-node '_) node)
;; (defn my-setting [] ...)
getter-node (-> (list
(hooks/token-node 'defn)
setting-name
(hooks/string-node "Docstring.")
(hooks/vector-node []))
hooks/list-node
(with-meta (meta node)))
;; (defn my-setting! [_x] ...)
setter-node (-> (list
(hooks/token-node 'defn)
(with-meta
(hooks/token-node (symbol (str (hooks/sexpr setting-name) \!)))
(meta setting-name))
(hooks/string-node "Docstring.")
(hooks/vector-node [(hooks/token-node '_value-or-nil)]))
hooks/list-node
(with-meta (meta node))
common/add-lsp-ignore-unused-public-var-metadata)]
(when (nil? (second (drop-while (comp not #{[:k :export?]} first) options-list)))
(when-not (contains? ignored-implicit-export? (:value setting-name))
(hooks/reg-finding! (assoc (meta node)
:message "Setting definition must provide an explicit value for :export? indicating whether the setting should be exported or not with serialization."
:type :metabase/defsetting-must-specify-export))))
{:node (-> (hooks/list-node
(list
(hooks/token-node 'let)
;; include description and the options map so they can get validated as well.
(hooks/vector-node
[anon-binding docstring
anon-binding (hooks/map-node options-list)])
(hooks/reg-keyword! (-> (hooks/keyword-node (keyword (hooks/sexpr setting-name)))
(with-meta (meta setting-name)))
'metabase.models.setting/defsetting)
getter-node
setter-node))
(with-meta (meta node)))}))
(defn defsetting
"Rewrite a [[metabase.models.defsetting]] form like
(defsetting my-setting \"Description\" :type :boolean)
as
(let [_ \"Description\"
_ {:type :boolean}]
(defn my-setting \"Docstring.\" [])
(defn my-setting! \"Docstring.\" [_value-or-nil]))
for linting purposes."
[{:keys [node], :as context}]
(let [[setting-name docstring & options] (rest (:children node))]
(merge
context
(defsetting-lint node setting-name docstring options))))
(defn define-multi-setting
"Rewrite a [[metabase.models.define-multi-setting]] form like
(defsetting my-setting \"Description\" :key :type :boolean)
as
(let [_ \"Description\"
_ {:type :boolean, :multi-thunk :key}]
(defn my-setting \"Docstring.\" [])
(defn my-setting! \"Docstring.\" [_value-or-nil]))
for linting purposes."
[{:keys [node], :as context}]
(let [[setting-name docstring thunk & options] (rest (:children node))]
(merge
context
(defsetting-lint node setting-name docstring (concat options [(hooks/token-node :multi-thunk) thunk])))))
(defn define-multi-setting-impl
[context]
(letfn [(update-node [node]
(let [[_define-multi-setting-impl setting-name & more] (:children node)]
(-> (hooks/list-node
(list*
(hooks/token-node 'do)
;; For a qualified setting name like `oss-namespace/setting`, then leave the symbol as is, so the
;; usage of that namespace/var gets recorded. Otherwise for an unqualified name convert it to a
;; keyword so we don't get unresolved var errors.
(if (and (hooks/token-node? setting-name)
(qualified-symbol? (hooks/sexpr setting-name)))
setting-name
(-> (hooks/keyword-node (keyword (hooks/sexpr setting-name)))
(with-meta (meta setting-name))))
more))
(with-meta (meta node)))))]
(update context :node update-node)))
(comment
(defn- defsetting* [form]
(hooks/sexpr
(:node
(defsetting
{:node
(hooks/parse-string
(with-out-str
#_{:clj-kondo/ignore [:unresolved-namespace]}
(clojure.pprint/pprint
form)))}))))
(defn x []
(defsetting*
'(defsetting active-users-count
(deferred-tru "Cached number of active users. Refresh every 5 minutes.")
:visibility :admin
:type :integer
:default 0
:getter (fn []
(if-not ((requiring-resolve 'metabase.db/db-is-set-up?))
0
(cached-active-users-count)))))))

View File

@@ -1,16 +0,0 @@
(ns hooks.metabase.query-processor.expressions-test
(:require
[clj-kondo.hooks-api :as hooks]
[hooks.metabase.test.data :as data]))
(defn calculate-bird-scarcity
[{{[_ formula filter-clause] :children} :node}]
(let [formula* (data/replace-$id-special-tokens formula)
filter-clause* (or (some-> filter-clause data/replace-$id-special-tokens)
(hooks/token-node 'nil))
node* (hooks/list-node
(list
(hooks/token-node 'do)
formula*
filter-clause*))]
{:node node*}))

View File

@@ -0,0 +1,16 @@
(ns hooks.metabase.query-processor-test.expressions-test
(:require
[clj-kondo.hooks-api :as hooks]
[hooks.metabase.test.data :as data]))
(defn calculate-bird-scarcity
[{{[_ formula filter-clause] :children} :node}]
(let [formula* (data/replace-$id-special-tokens formula)
filter-clause* (or (some-> filter-clause data/replace-$id-special-tokens)
(hooks/token-node 'nil))
node* (hooks/list-node
(list
(hooks/token-node 'do)
formula*
filter-clause*))]
{:node node*}))

View File

@@ -1,308 +0,0 @@
(ns hooks.metabase.settings.models.setting
(:require
[clj-kondo.hooks-api :as hooks]
[hooks.common :as common]))
(defn lint-defsetting-namespace [node context]
(when-not (re-matches #"^metabase(?:-enterprise)?\.[^\.]+\.settings$" (name (:ns context)))
(hooks/reg-finding! (assoc (meta node)
:message "All defsettings should live in metabase[-enterprise].<module>.settings namespaces"
:type :metabase/defsetting-namespace))))
;;; TODO -- move this into a Kondo config file in `.clj-kondo/config/`
(def ^:private ignored-implicit-export?
'#{active-users-count
admin-email
analytics-uuid
anon-tracking-enabled
api-key
attachment-table-row-limit
audit-max-retention-days
bcc-enabled?
breakout-bin-width
check-for-updates
cloud-gateway-ips
config-from-file-sync-databases
custom-homepage
custom-homepage-dashboard
database-enable-actions
deprecation-notice-version
dismissed-custom-dashboard-toast
email-configured?
email-from-address
email-from-name
email-reply-to
email-smtp-host
email-smtp-password
email-smtp-port
email-smtp-security
email-smtp-username
embedding-app-origin
embedding-secret-key
enable-password-login
enable-public-sharing
enable-query-caching
engines
enum-cardinality-threshold
follow-up-email-sent
google-auth-auto-create-accounts-domain
google-auth-client-id
google-auth-configured
google-auth-enabled
has-sample-database?
has-user-setup
help-link
help-link-custom-destination
instance-creation
is-hosted?
jdbc-data-warehouse-max-connection-pool-size
jdbc-data-warehouse-unreturned-connection-timeout-seconds
jwt-attribute-email
jwt-attribute-firstname
jwt-attribute-groups
jwt-attribute-lastname
jwt-configured
jwt-enabled
jwt-group-mappings
jwt-group-sync
jwt-identity-provider-uri
jwt-user-provisioning-enabled?
jwt-shared-secret
last-acknowledged-version
last-used-native-database-id
ldap-attribute-email
ldap-attribute-firstname
ldap-attribute-lastname
ldap-bind-dn
ldap-configured?
ldap-enabled
ldap-group-base
ldap-group-mappings
ldap-group-membership-filter
ldap-group-sync
ldap-host
ldap-password
ldap-port
ldap-user-provisioning-enabled?
ldap-security
ldap-sync-user-attributes
ldap-sync-user-attributes-blacklist
ldap-user-base
ldap-user-filter
load-analytics-content
map-tile-server-url
metabot-default-embedding-model
metabot-feedback-url
metabot-get-prompt-templates-url
metabot-prompt-generator-token-limit
multi-setting-read-only
notebook-native-preview-sidebar-width
notification-link-base-url
num-metabot-choices
openai-api-key
openai-available-models
openai-model
openai-organization
other-sso-enabled?
password-complexity
persist-models-enabled
persisted-model-refresh-cron-schedule
premium-embedding-token
prometheus-server-port
query-caching-max-kb
query-caching-max-ttl
query-caching-min-ttl
query-caching-ttl-ratio
redirect-all-requests-to-https
reset-token-ttl-hours
retired-setting
retry-initial-interval
retry-max-retries
retry-max-interval-millis
retry-multiplier
retry-jitter-factor
saml-application-name
saml-attribute-email
saml-attribute-firstname
saml-attribute-group
saml-attribute-lastname
saml-configured
saml-enabled
saml-group-mappings
saml-group-sync
saml-identity-provider-certificate
saml-identity-provider-issuer
saml-identity-provider-uri
saml-keystore-alias
saml-keystore-password
saml-keystore-path
saml-user-provisioning-enabled?
sdk-iframe-embed-setup-settings
send-email-on-first-login-from-new-device
send-new-sso-user-admin-email?
session-cookie-samesite
session-cookies
session-timeout
setup-token
show-database-syncing-modal
show-metabase-links
site-url
site-uuid
site-uuid-for-premium-features-token-checks
site-uuid-for-unsubscribing-url
site-uuid-for-version-info-fetching
slack-app-token
slack-cached-channels-and-usernames
slack-channels-and-usernames-last-updated
slack-files-channel
slack-token
slack-token-valid?
snowplow-available
snowplow-enabled
snowplow-url
sql-jdbc-fetch-size
ssh-heartbeat-interval-sec
ssl-certificate-public-key
startup-time-millis
token-features
token-status
toucan-name
trial-banner-dismissal-timestamp
uncached-setting
user-visibility
version
version-info
version-info-last-checked})
(defn- lint-defsetting-export [node]
(let [[_defsetting setting-name _docstring & options] (:children node)]
(when (nil? (second (drop-while (comp not #{[:k :export?]} first) options)))
(when-not (contains? ignored-implicit-export? (:value setting-name))
(hooks/reg-finding! (assoc (meta node)
:message "Setting definition must provide an explicit value for :export? indicating whether the setting should be exported or not with serialization."
:type :metabase/defsetting-must-specify-export))))))
(defn- defsetting-replacement-node [node]
(let [[_defsetting setting-name docstring & options-list] (:children node)
anon-binding (common/with-macro-meta (hooks/token-node '_) node)
;; (defn my-setting [] ...)
getter-node (-> (list
(hooks/token-node 'defn)
setting-name
(hooks/string-node "Docstring.")
(hooks/vector-node []))
hooks/list-node
(with-meta (meta node)))
;; (defn my-setting! [_x] ...)
setter-node (-> (list
(hooks/token-node 'defn)
(with-meta
(hooks/token-node (symbol (str (hooks/sexpr setting-name) \!)))
(meta setting-name))
(hooks/string-node "Docstring.")
(hooks/vector-node [(hooks/token-node '_value-or-nil)]))
hooks/list-node
(with-meta (meta node))
common/add-lsp-ignore-unused-public-var-metadata)]
(-> (hooks/list-node
(list
(hooks/token-node 'let)
;; include description and the options map so they can get validated as well.
(hooks/vector-node
[anon-binding docstring
anon-binding (hooks/map-node options-list)])
(hooks/reg-keyword! (-> (hooks/keyword-node (keyword (hooks/sexpr setting-name)))
(with-meta (meta setting-name)))
'metabase.settings.models.setting/defsetting)
getter-node
setter-node))
(with-meta (meta node)))))
(defn defsetting
"Rewrite a [[metabase.models.defsetting]] form like
(defsetting my-setting \"Description\" :type :boolean)
as
(let [_ \"Description\"
_ {:type :boolean}]
(defn my-setting \"Docstring.\" [])
(defn my-setting! \"Docstring.\" [_value-or-nil]))
for linting purposes."
[{:keys [node], :as context}]
(lint-defsetting-namespace node context)
(lint-defsetting-export node)
(update context :node defsetting-replacement-node))
(defn define-multi-setting-replacement-node [node]
(let [[defsetting setting-name docstring thunk & options] (:children node)]
(-> (hooks/list-node
(list*
defsetting
setting-name
docstring
(hooks/token-node :multi-thunk) thunk
options))
(with-meta (meta node))
defsetting-replacement-node)))
(defn define-multi-setting
"Rewrite a [[metabase.models.define-multi-setting]] form like
(defsetting my-setting \"Description\" :key :type :boolean)
as
(let [_ \"Description\"
_ {:type :boolean, :multi-thunk :key}]
(defn my-setting \"Docstring.\" [])
(defn my-setting! \"Docstring.\" [_value-or-nil]))
for linting purposes."
[{:keys [node], :as context}]
(update context :node define-multi-setting-replacement-node))
(defn define-multi-setting-impl
[context]
(letfn [(update-node [node]
(let [[_define-multi-setting-impl setting-name & more] (:children node)]
(-> (hooks/list-node
(list*
(hooks/token-node 'do)
;; For a qualified setting name like `oss-namespace/setting`, then leave the symbol as is, so the
;; usage of that namespace/var gets recorded. Otherwise for an unqualified name convert it to a
;; keyword so we don't get unresolved var errors.
(if (and (hooks/token-node? setting-name)
(qualified-symbol? (hooks/sexpr setting-name)))
setting-name
(-> (hooks/keyword-node (keyword (hooks/sexpr setting-name)))
(with-meta (meta setting-name))))
more))
(with-meta (meta node)))))]
(update context :node update-node)))
(comment
(defn- defsetting* [form]
(hooks/sexpr
(:node
(defsetting
{:node
(hooks/parse-string
(with-out-str
#_{:clj-kondo/ignore [:unresolved-namespace]}
(clojure.pprint/pprint
form)))}))))
(defn x []
(defsetting*
'(defsetting active-users-count
(deferred-tru "Cached number of active users. Refresh every 5 minutes.")
:visibility :admin
:type :integer
:default 0
:getter (fn []
(if-not ((requiring-resolve 'metabase.app-db.core/db-is-set-up?))
0
(cached-active-users-count)))))))

View File

@@ -2,8 +2,8 @@
(:require
[clj-kondo.hooks-api :as hooks]))
(defn iterate*
"(iterate {:as limit-spec} [s1 val1 & more] body-form body-form...)"
(defn with-gentest
"(with-gentest {:as limit-spec} [s1 val1 & more] body-form body-form...)"
[{{:keys [children] :as node} :node :as orig}]
(let [[token limit-spec bindings & body-nodes] children
node* (-> (hooks/list-node

View File

@@ -1,6 +1,6 @@
(ns hooks.metabase.test.util
(:require
[clj-kondo.hooks-api :as hooks]))
(:require [clj-kondo.hooks-api :as hooks]
[hooks.common :as common]))
(defn- namespaced-symbol-node? [node]
(when (hooks/token-node? node)

View File

@@ -1,15 +0,0 @@
(ns hooks.metabase.test.util.generators.jvm
(:require
[clj-kondo.hooks-api :as hooks]))
(defn with-random-cards
"(with-random-cards <metadata-provider> <integer-count-value> & body-forms)"
[{{:keys [children] :as node} :node :as orig}]
(let [[token num-cards & body-nodes] children
node* (-> (hooks/list-node
(list token num-cards
(hooks/list-node (list* (hooks/token-node 'let)
(hooks/vector-node [(hooks/token-node '&cards) (hooks/list-node ())])
body-nodes))))
(with-meta (meta node)))]
(assoc orig :node node*)))

View File

@@ -1,16 +0,0 @@
(ns hooks.metabase.util.timing
"Custom CLJ Kondo hooks for timing-related linting rules."
(:require [clj-kondo.hooks-api :as api]))
(defn discourage-millis-duration
"Discourage slow and unreliable duration calculations using System/currentTimeMillis "
[{:keys [node]}]
(let [sexpr (api/sexpr node)]
(when (and (list? sexpr) (= (first sexpr) '-) (>= (count sexpr) 2))
(let [first-arg (second sexpr)]
(when (= first-arg (list 'System/currentTimeMillis))
(api/reg-finding!
(assoc (meta node)
:message "Avoid System/currentTimeMillis for calculating durations. Use u/start-timer and u/since-ms for more reliability."
:type :metabase/discourage-millis-duration
:level :warning)))))))

View File

@@ -20,17 +20,15 @@
(lint-ns form (assoc-in config [:linters :metabase/modules :level] :warning)))
(deftest ^:parallel module-checker-allowed-modules-test
(are [task-namespace] (=? [{:message "Module task should not be used in the api module. [:metabase/modules api :uses]"
:type :metabase/modules}]
(lint-modules
'(ns metabase.api.search
(:require
[metabase.request.core :as request]
[task-namespace :as task]))
'{:metabase/modules {api {:uses #{request}}
request {:api #{metabase.request.core}}}}))
metabase.task
metabase.task.core))
(is (=? [{:message "Module task should not be used in the api module. [:metabase/modules api :uses]"
:type :metabase/modules}]
(lint-modules
'(ns metabase.api.search
(:require
[metabase.request.core :as request]
[metabase.task :as task]))
'{:metabase/modules {api {:uses #{request}}
request {:api #{metabase.request.core}}}}))))
(deftest ^:parallel module-checker-api-namespaces-test
(is (=? [{:message "Namespace metabase.search.config is not an allowed external API namespace for the search module. [:metabase/modules search :api]"
@@ -43,17 +41,6 @@
'{:metabase/modules {api {:uses #{search}}
search {:api #{metabase.search.core}}}}))))
(deftest ^:parallel module-checker-friends-test
(is (= []
(lint-modules
'(ns metabase.search-rest.api
(:require
[metabase.search.config :as search.config]
[metabase.search.core :as search]))
'{:metabase/modules {search {:api #{metabase.search.core}
:friends #{search-rest}}
search-rest {:uses #{search}}}}))))
(deftest ^:parallel module-checker-api-namespaces-ignore-test
(is (empty? (lint-modules
'(ns metabase.api.search

View File

@@ -1,7 +1,6 @@
(ns hooks.metabase.api.macros-test
(:require
[clj-kondo.hooks-api :as api]
[clj-kondo.impl.utils]
[clojure.test :refer :all]
[hooks.metabase.api.macros]))
@@ -70,41 +69,3 @@
(respond :x)
(catch Throwable e
(raise e))))))))
(defn- defendpoint [form]
(hooks.metabase.api.macros/defendpoint
{:node (-> form pr-str api/parse-string)}))
(defn- defendpoint-warnings
[form]
(binding [clj-kondo.impl.utils/*ctx* {:config {:linters {:metabase/validate-defendpoint-route-uses-kebab-case {:level :warning}
:metabase/validate-defendpoint-query-params-use-kebab-case {:level :warning}}}
:ignores (atom nil)
:findings (atom [])
:namespaces (atom {})}]
(defendpoint form)
@(:findings clj-kondo.impl.utils/*ctx*)))
(deftest ^:parallel validate-route-kebab-case-test
(is (=? [{:message "REST API routes and query params should use kebab-case, not snake_case"
:type :metabase/validate-defendpoint-route-uses-kebab-case
:row pos-int?
:col pos-int?}]
(defendpoint-warnings
'(api.macros/defendpoint :get "/:id/syncable_schemas" ; snake_case route should trigger a warning
"Returns a list of all syncable schemas found for the database `id`."
[{:keys [id]}]
id)))))
(deftest ^:parallel validate-query-params-kebab-case-test
(is (=? [{:message "REST API routes and query params should use kebab-case, not snake_case",
:type :metabase/validate-defendpoint-query-params-use-kebab-case
:row pos-int?
:col pos-int?}]
(defendpoint-warnings
'(api.macros/defendpoint :get "/:id/syncable-schemas"
"Returns a list of all syncable schemas found for the database `id`."
[{:keys [id]}
{:keys [filter_by]} :- [:map
[:filter_by {:optional true} :string]]]
id)))))

View File

@@ -0,0 +1,34 @@
(ns hooks.metabase.models.setting-test
(:require
[clj-kondo.hooks-api :as hooks]
[clojure.test :refer :all]
[hooks.metabase.models.setting]))
#_{:clj-kondo/ignore [:metabase/validate-deftest]}
(deftest ^:parallel defsetting-test
(let [node (-> '(defsetting active-users-count
(deferred-tru "Cached number of active users. Refreshed every 5 minutes.")
:visibility :admin
:type :integer
:default 0
:getter (fn []
(if-not ((requiring-resolve 'metabase.db/db-is-set-up?))
0
(cached-active-users-count))))
pr-str
hooks/parse-string)]
(is (= '(let [_ (deferred-tru "Cached number of active users. Refreshed every 5 minutes.")
_ {:default 0
:type :integer
:getter (fn []
(if-not ((requiring-resolve 'metabase.db/db-is-set-up?))
0
(cached-active-users-count)))
:visibility :admin}]
:active-users-count
(defn active-users-count "Docstring." [])
(defn active-users-count! "Docstring." [_value-or-nil]))
(-> {:node node}
hooks.metabase.models.setting/defsetting
:node
hooks/sexpr)))))

View File

@@ -1,34 +0,0 @@
(ns hooks.metabase.settings.models.setting-test
(:require
[clj-kondo.hooks-api :as hooks]
[clojure.test :refer :all]
[hooks.metabase.settings.models.setting]))
#_{:clj-kondo/ignore [:metabase/validate-deftest]}
(deftest ^:parallel defsetting-test
(let [node (-> '(defsetting active-users-count
(deferred-tru "Cached number of active users. Refreshed every 5 minutes.")
:visibility :admin
:type :integer
:default 0
:getter (fn []
(if-not ((requiring-resolve 'metabase.app-db.core/db-is-set-up?))
0
(cached-active-users-count))))
pr-str
hooks/parse-string)]
(is (= '(let [_ (deferred-tru "Cached number of active users. Refreshed every 5 minutes.")
_ {:default 0
:type :integer
:getter (fn []
(if-not ((requiring-resolve 'metabase.app-db.core/db-is-set-up?))
0
(cached-active-users-count)))
:visibility :admin}]
:active-users-count
(defn active-users-count "Docstring." [])
(defn active-users-count! "Docstring." [_value-or-nil]))
(-> {:node node, :ns 'hooks.metabase.settings.models.setting-test}
hooks.metabase.settings.models.setting/defsetting
:node
hooks/sexpr)))))

View File

@@ -38,324 +38,311 @@
;; zero get indented 2 spaces if not on the first line regardless of number of elements on the first line.
:extra-indents
{;; clojure.core stuff
fn* [[:inner 0]]
let* [[:block 1]]
with-meta [[:default]]
fn* [[:inner 0]]
let* [[:block 1]]
with-meta [[:default]]
;; library stuff. Libraries that specify their own `:style/indent` specs are included in the section after this
;; since it is generated automatically.
clojure.core.match/match [[:block 1]]
clojure.test.check.properties/for-all [[:block 1]]
clojure.test.check.properties/for-all* [[:block 1]]
clojure.test.check/quick-check [[:block 1]]
clojure.tools.macro/macrolet [[:block 1] [:inner 2 0]]
clojurewerkz.quartzite.jobs/build [[:default]]
clojurewerkz.quartzite.jobs/with-description [[:default]]
clojurewerkz.quartzite.jobs/with-identity [[:default]]
clojurewerkz.quartzite.schedule.cron/schedule [[:default]]
clojurewerkz.quartzite.schedule.cron/with-misfire-handling-instruction-do-nothing [[:default]]
clojurewerkz.quartzite.triggers/build [[:default]]
clojurewerkz.quartzite.triggers/with-description [[:default]]
clojurewerkz.quartzite.triggers/with-identity [[:default]]
clojurewerkz.quartzite.triggers/with-priority [[:default]]
clojurewerkz.quartzite.triggers/with-schedule [[:default]]
def-map-type [[:block 2] [:inner 1]]
defprotocol+ [[:block 1] [:inner 1]]
defrecord+ [[:block 2] [:inner 1]]
deftype+ [[:block 2] [:inner 1]]
net.cgrand.macrovich/case [[:block 0]]
potemkin.namespaces/import-vars [[:default]]
potemkin.types/def-map-type [[:block 2] [:inner 1]]
potemkin.types/defprotocol+ [[:block 1] [:inner 1]]
potemkin.types/defrecord+ [[:block 2] [:inner 1]]
potemkin.types/deftype+ [[:block 2] [:inner 1]]
potemkin/def-map-type [[:block 2] [:inner 1]]
potemkin/defprotocol+ [[:block 1] [:inner 1]]
potemkin/defrecord+ [[:block 2] [:inner 1]]
potemkin/deftype+ [[:block 2] [:inner 1]]
potemkin/import-vars [[:default]]
taoensso.nippy/extend-freeze [[:inner 0]]
taoensso.nippy/extend-thaw [[:inner 0]]
throttle.core/with-throttling [[:block 1]]
clojure.core.logic/fresh [[:block 1]]
clojure.core.logic/matcha [[:block 1]]
clojure.core.logic/run [[:block 2]]
clojure.core.match/match [[:block 1]]
clojure.test.check.properties/for-all [[:block 1]]
clojure.test.check.properties/for-all* [[:block 1]]
clojure.test.check/quick-check [[:block 1]]
clojure.tools.macro/macrolet [[:block 1] [:inner 2 0]]
clojurewerkz.quartzite.jobs/build [[:default]]
clojurewerkz.quartzite.jobs/with-description [[:default]]
clojurewerkz.quartzite.jobs/with-identity [[:default]]
clojurewerkz.quartzite.schedule.cron/schedule [[:default]]
clojurewerkz.quartzite.schedule.cron/with-misfire-handling-instruction-do-nothing [[:default]]
clojurewerkz.quartzite.triggers/build [[:default]]
clojurewerkz.quartzite.triggers/with-description [[:default]]
clojurewerkz.quartzite.triggers/with-identity [[:default]]
clojurewerkz.quartzite.triggers/with-priority [[:default]]
clojurewerkz.quartzite.triggers/with-schedule [[:default]]
def-map-type [[:block 2] [:inner 1]]
defprotocol+ [[:block 1] [:inner 1]]
defrecord+ [[:block 2] [:inner 1]]
deftype+ [[:block 2] [:inner 1]]
net.cgrand.macrovich/case [[:block 0]]
potemkin.namespaces/import-vars [[:default]]
potemkin.types/def-map-type [[:block 2] [:inner 1]]
potemkin.types/defprotocol+ [[:block 1] [:inner 1]]
potemkin.types/defrecord+ [[:block 2] [:inner 1]]
potemkin.types/deftype+ [[:block 2] [:inner 1]]
potemkin/def-map-type [[:block 2] [:inner 1]]
potemkin/defprotocol+ [[:block 1] [:inner 1]]
potemkin/defrecord+ [[:block 2] [:inner 1]]
potemkin/deftype+ [[:block 2] [:inner 1]]
potemkin/import-vars [[:default]]
taoensso.nippy/extend-freeze [[:inner 0]]
taoensso.nippy/extend-thaw [[:inner 0]]
throttle.core/with-throttling [[:block 1]]
;; the rest of the stuff in this list is generated automatically based on `:style/indent` specs
;; by running [[dev.generate-cljfmt-config/specs]]... use that code to update this list
metabase-enterprise.cache.config-test/with-temp-persist-models [[:block 1]]
metabase-enterprise.impersonation.util-test/with-impersonations! [[:inner 0]]
metabase-enterprise.sandbox.test-util/with-gtaps! [[:inner 0]]
metabase-enterprise.sandbox.test-util/with-gtaps-for-user! [[:inner 0]]
metabase-enterprise.sandbox.test-util/with-user-attributes [[:block 2]]
metabase-enterprise.serialization.test-util/with-dbs [[:inner 0]]
metabase-enterprise.serialization.test-util/with-random-dump-dir [[:block 1]]
metabase-enterprise.serialization.test-util/with-world [[:block 0]]
metabase-enterprise.sso.integrations.jwt-test/with-users-with-email-deleted [[:block 1]]
metabase-enterprise.test/with-gtaps! [[:inner 0]]
metabase-enterprise.test/with-gtaps-for-user! [[:inner 0]]
metabase-enterprise.test/with-user-attributes [[:block 2]]
metabase-enterprise.workspaces.models.workspace-log/track! [[:block 2]]
metabase.actions.actions-test/with-actions-test-data-and-actions-permissively-enabled [[:block 0]]
metabase.actions.test-util/with-actions [[:block 1]]
metabase.actions.test-util/with-actions-disabled [[:block 0]]
metabase.actions.test-util/with-actions-enabled [[:block 0]]
metabase.actions.test-util/with-actions-test-data [[:inner 0]]
metabase.actions.test-util/with-actions-test-data-and-actions-enabled [[:block 0]]
metabase.actions.test-util/with-actions-test-data-tables [[:block 1]]
metabase.actions.test-util/with-empty-db [[:inner 0]]
metabase.actions.test-util/with-temp-test-data [[:inner 0]]
metabase.api.common/check [[:default]]
metabase.api.common/define-routes [[:block 0]]
metabase.api.common/let-404 [[:block 1]]
metabase.app-db.core/with-application-db [[:inner 0]]
metabase.app-db.liquibase/with-liquibase [[:block 1]]
metabase.app-db.liquibase/with-scope-locked [[:block 1]]
metabase.app-db.schema-migrations-test.impl/test-migrations [[:block 2]]
metabase.cmd.copy/step [[:block 1]]
metabase.cmd.copy/with-connection-rollback-only [[:block 1]]
metabase.cmd.copy/with-disabled-db-constraints [[:block 2]]
metabase.collections-rest.api-test/with-collection-hierarchy [[:block 1]]
metabase.collections-rest.api-test/with-french-user-and-personal-collection [[:block 2]]
metabase.collections-rest.api-test/with-some-children-of-collection [[:block 1]]
metabase.collections.models.collection-test/with-collection-hierarchy [[:block 1]]
metabase.collections.models.collection-test/with-current-user-perms-for-collections [[:block 1]]
metabase.collections.models.collection-test/with-personal-and-impersonal-collections [[:block 1]]
metabase.config/build-type-case [[:block 0]]
metabase.dashboards.models.dashboard-test/with-dash-in-collection [[:inner 0]]
metabase.driver-api.core/match [[:inner 0]]
metabase.driver-api.core/match-one [[:inner 0]]
metabase.driver-api.core/match-lite [[:inner 0]]
metabase.driver-api.core/match-many [[:inner 0]]
metabase.driver-api.core/replace [[:inner 0]]
metabase.driver-api.core/replace-lite [[:inner 0]]
metabase.driver-api.core/replace-in [[:inner 0]]
metabase.driver.bigquery-cloud-sdk.query-processor/with-temporal-type [[:default]]
metabase.driver.druid.client/DELETE [[:default]]
metabase.driver.druid.client/GET [[:default]]
metabase.driver.druid.client/POST [[:default]]
metabase.driver.druid.query-processor-test/druid-query [[:block 0]]
metabase.driver.druid.query-processor-test/druid-query-returning-rows [[:block 0]]
metabase.driver.mongo.query-processor/mongo-let [[:block 1]]
metabase.driver.sql-jdbc.actions/with-jdbc-transaction [[:block 1]]
metabase.driver.sql-jdbc.connection/with-connection-spec-for-testing-connection [[:block 1]]
metabase.driver.sql-jdbc.execute.diagnostic/capturing-diagnostic-info [[:block 1]]
metabase.driver.sql.query-processor/apply-top-level-clause [[:default]]
metabase.driver/with-driver [[:block 1]]
metabase.driver/with-swapped-connection-details [[:block 2]]
metabase.embedding-rest.api.embed-test/do-response-formats [[:block 1]]
metabase.embedding-rest.api.embed-test/with-embedding-enabled-and-new-secret-key [[:block 0]]
metabase.embedding-rest.api.embed-test/with-embedding-enabled-and-temp-card-referencing [[:block 3]]
metabase.embedding-rest.api.embed-test/with-embedding-enabled-and-temp-dashcard-referencing [[:block 3]]
metabase.embedding-rest.api.embed-test/with-new-secret-key [[:block 0]]
metabase.embedding-rest.api.embed-test/with-temp-card [[:block 1]]
metabase.embedding-rest.api.embed-test/with-temp-dashcard [[:block 1]]
metabase.embedding-rest.api.preview-embed-test/with-embedding-enabled-and-new-secret-key [[:block 0]]
metabase.embedding-rest.api.preview-embed-test/with-new-secret-key [[:block 0]]
metabase.lib.binning/with-binning [[:default]]
metabase.lib.core/with-binning [[:default]]
metabase.lib.core/with-join-alias [[:default]]
metabase.lib.core/with-join-conditions [[:default]]
metabase.lib.join/with-join-alias [[:default]]
metabase.lib.join/with-join-conditions [[:default]]
metabase.lib.test-util.macros/$ids [[:inner 0]]
metabase.lib.test-util.macros/mbql-5-query [[:inner 0]]
metabase.lib.test-util.macros/mbql-query [[:inner 0]]
metabase.lib.util.match/match [[:inner 0]]
metabase.lib.util.match/match-one [[:inner 0]]
metabase.lib.util.match/match-lite [[:inner 0]]
metabase.lib.util.match/match-many [[:inner 0]]
metabase.lib.util.match/replace [[:inner 0]]
metabase.lib.util.match/replace-lite [[:inner 0]]
metabase.lib.util.match/replace-in [[:inner 0]]
metabase.models.interface/define-batched-hydration-method [[:inner 0]]
metabase.models.interface/define-simple-hydration-method [[:inner 0]]
metabase.parameters.chain-filter-test/with-clean-field-values-for-field [[:block 1]]
metabase.parameters.chain-filter-test/with-fk-field-to-field-remapping! [[:block 0]]
metabase.parameters.chain-filter-test/with-human-readable-values-remapping! [[:block 0]]
metabase.permissions.core/allow-changing-all-users-group-members [[:block 0]]
metabase.permissions.core/disable-perms-cache [[:block 0]]
metabase.public-sharing-rest.api-test/with-sharing-enabled-and-temp-card-referencing [[:block 3]]
metabase.public-sharing-rest.api-test/with-sharing-enabled-and-temp-dashcard-referencing [[:block 3]]
metabase.public-sharing-rest.api-test/with-temp-public-card [[:block 1]]
metabase.public-sharing-rest.api-test/with-temp-public-dashboard [[:block 1]]
metabase.public-sharing-rest.api-test/with-temp-public-dashboard-and-card [[:block 1]]
metabase.pulse.api.alert-test/with-alert-in-collection [[:block 1]]
metabase.pulse.dashboard-subscription-test/with-link-card-fixture-for-dashboard [[:block 2]]
metabase.pulse.models.pulse-test/with-dashboard-subscription-in-collection [[:block 1]]
metabase.pulse.models.pulse-test/with-pulse-in-collection [[:inner 0]]
metabase.queries-rest.api.card-test/with-cards-in-readable-collection [[:block 1]]
metabase.queries-rest.api.card-test/with-cards-in-writeable-collection [[:block 1]]
metabase.queries-rest.api.card-test/with-ordered-items [[:inner 0]]
metabase.queries-rest.api.card-test/with-temp-native-card [[:block 1]]
metabase.queries-rest.api.card-test/with-temp-native-card-with-params [[:block 1]]
metabase.query-processor.debug/debug> [[:block 0]]
metabase.query-processor.error-type/deferror [[:block 1]]
metabase.query-processor.middleware.cache-backend.interface/with-cached-results [[:block 4]]
metabase.query-processor.middleware.process-userland-query-test/with-query-execution [[:block 1]]
metabase.query-processor.pipeline-queries-test/pmbql-query [[:block 1]]
metabase.query-processor.pipeline-queries-test/run-pmbql-query [[:block 1]]
metabase.query-processor.store/cached [[:block 1]]
metabase.query-processor.store/with-metadata-provider [[:inner 0]]
metabase.query-processor.streaming/streaming-response [[:block 1]]
metabase.query-processor.test-util/format-rows-by [[:default]]
metabase.query-processor.timeseries-test.util/test-timeseries-drivers [[:block 0]]
metabase.query-processor.timeseries-test/run-mbql-query [[:inner 0]]
metabase.request.core/as-admin [[:block 0]]
metabase.request.core/with-current-request [[:inner 0]]
metabase.request.core/with-current-user [[:inner 0]]
metabase.request.core/with-limit-and-offset [[:inner 0]]
metabase.request.session/as-admin [[:block 0]]
metabase.request.session/with-current-user [[:inner 0]]
metabase.sample-data.impl-test/with-temp-sample-database-db [[:block 1]]
metabase.search.api-test/do-test-users [[:block 1]]
metabase.server.middleware.session/as-admin [[:block 0]]
metabase.server.middleware.session/with-current-user [[:inner 0]]
metabase.server.streaming-response-test/with-streaming-response-thread-pool [[:block 0]]
metabase.server.streaming-response-test/with-test-driver-db! [[:block 0]]
metabase.server.streaming-response/streaming-response [[:block 2]]
metabase.session.api/http-401-on-error [[:block 0]]
metabase.settings.core/define-multi-setting [[:inner 0]]
metabase.settings.core/defsetting [[:block 1]]
metabase.settings.models.setting.multi-setting/define-multi-setting [[:inner 0]]
metabase.settings.models.setting/defsetting [[:block 1]]
metabase.sync.fetch-metadata/log-if-error [[:block 1]]
metabase.sync.sync-metadata.fields-test/with-test-db [[:block 0]]
metabase.sync.util/do-with-start-and-finish-debug-logging [[:default]]
metabase.sync.util/sum-for [[:block 1]]
metabase.sync.util/sync-operation [[:block 3]]
metabase.sync.util/with-db-logging-disabled [[:default]]
metabase.sync.util/with-duplicate-ops-prevented [[:default]]
metabase.sync.util/with-emoji-progress-bar [[:block 1]]
metabase.sync.util/with-error-handling [[:block 1]]
metabase.sync.util/with-start-and-finish-logging [[:default]]
metabase.sync.util/with-start-and-finish-logging* [[:default]]
metabase.sync.util/with-sync-events [[:default]]
metabase.task-history.models.task-history/with-task-history [[:block 1]]
metabase.task.core/rerun-on-error [[:block 1]]
metabase.task.impl/rerun-on-error [[:block 1]]
metabase.test.data.datasets/test-driver [[:inner 0]]
metabase.test.data.datasets/test-drivers [[:inner 0]]
metabase.test.data.one-off-dbs/with-blank-db [[:block 0]]
metabase.test.data.one-off-dbs/with-blueberries-db [[:block 0]]
metabase.test.data.users/with-test-user [[:inner 0]]
metabase.test.data/$ids [[:inner 0]]
metabase.test.data/dataset [[:inner 0]]
metabase.test.data/mbql-query [[:inner 0]]
metabase.test.data/query [[:block 1]]
metabase.test.data/run-mbql-query [[:inner 0]]
metabase.test.data/with-empty-h2-app-db! [[:block 0]]
metabase.test.data/with-temp-copy-of-db [[:block 0]]
metabase.test.util.public-settings/with-additional-premium-features [[:block 1]]
metabase.test.util.public-settings/with-premium-features [[:block 1]]
metabase.test.util.thread-local/test-helpers-set-global-values! [[:block 0]]
metabase.test.util/discard-setting-changes [[:block 1]]
metabase.test.util/let-url [[:block 1]]
metabase.test.util/throw-if-called! [[:block 1]]
metabase.test.util/with-temp-scheduler [[:block 0]]
metabase.test.util/with-temp-vals-in-db [[:block 3]]
metabase.test.util/with-user-in-groups [[:inner 0]]
metabase.test/$ids [[:inner 0]]
metabase.test/as-admin [[:inner 0]]
metabase.test/dataset [[:inner 0]]
metabase.test/discard-setting-changes [[:block 1]]
metabase.test/format-rows-by [[:default]]
metabase.test/let-url [[:block 1]]
metabase.test/mbql-query [[:inner 0]]
metabase.test/query [[:block 1]]
metabase.test/run-mbql-query [[:inner 0]]
metabase.test/test-driver [[:inner 0]]
metabase.test/test-drivers [[:inner 0]]
metabase.test/test-helpers-set-global-values! [[:block 0]]
metabase.test/throw-if-called! [[:block 1]]
metabase.test/with-actions [[:block 1]]
metabase.test/with-actions-disabled [[:block 0]]
metabase.test/with-actions-enabled [[:block 0]]
metabase.test/with-actions-test-data [[:inner 0]]
metabase.test/with-actions-test-data-and-actions-enabled [[:block 0]]
metabase.test/with-actions-test-data-tables [[:block 1]]
metabase.test/with-additional-premium-features [[:block 1]]
metabase.test/with-current-user [[:inner 0]]
metabase.test/with-driver [[:block 1]]
metabase.test/with-empty-db [[:inner 0]]
metabase.test/with-empty-h2-app-db! [[:block 0]]
metabase.test/with-metadata-provider [[:inner 0]]
metabase.test/with-premium-features [[:block 1]]
metabase.test/with-temp [[:inner 0]]
metabase.test/with-temp-copy-of-db [[:block 0]]
metabase.test/with-temp-scheduler [[:block 0]]
metabase.test/with-temp-test-data [[:inner 0]]
metabase.test/with-temp-vals-in-db [[:block 3]]
metabase.test/with-test-user [[:inner 0]]
metabase.test/with-user-in-groups [[:inner 0]]
metabase.upload.impl-test/with-upload-table! [[:inner 0]]
metabase.upload.types/with-currency [[:default]]
metabase.upload.types/with-parens [[:default]]
metabase.util.async/cancelable-thread [[:block 0]]
metabase.util.date-2/with-time-zone-same-instant [[:default]]
metabase.util.encryption-test/with-secret-key [[:block 1]]
metabase.util.honey-sql-2/with-database-type-info [[:default]]
metabase.util.i18n/deferred-trs [[:default]]
metabase.util.i18n/deferred-tru [[:default]]
metabase.util.i18n/trs [[:default]]
metabase.util.i18n/trsn [[:default]]
metabase.util.i18n/tru [[:default]]
metabase.util.i18n/trun [[:default]]
metabase.util.jvm/auto-retry [[:block 1]]
metabase.util.jvm/varargs [[:block 1]]
metabase.util.jvm/with-us-locale [[:block 0]]
metabase.util.malli/disable-enforcement [[:block 0]]
metabase.util.malli/with [[:default]]
metabase.util.malli/with-api-error-message [[:default]]
metabase.util.namespaces/import-fns [[:default]]
metabase.util.queue-test/await-test-while [[:block 1]]
metabase.util/auto-retry [[:block 1]]
metabase.util/case-enum [[:block 1]]
metabase.util/for-map [[:block 1]]
metabase.util/ignore-exceptions [[:block 0]]
metabase.util/or-with [[:block 1]]
metabase.util/profile [[:block 1]]
metabase.util/prog1 [[:inner 0]]
metabase.util/varargs [[:inner 0]]
metabase.util/with-us-locale [[:block 0]]
metabuild-common.core/exit-when-finished-nonzero-on-exception [[:block 0]]
metabuild-common.core/step [[:block 1]]
metabuild-common.entrypoint/exit-when-finished-nonzero-on-exception [[:block 0]]
metabuild-common.steps/step [[:block 1]]
methodical.core/defmethod [[:inner 0]]
methodical.core/defmulti [[:inner 0]]
methodical.core/with-dispatcher [[:default]]
methodical.core/with-method-table [[:default]]
methodical.core/with-prefers [[:default]]
methodical.interface/with-dispatcher [[:default]]
methodical.interface/with-method-table [[:default]]
methodical.interface/with-prefers [[:default]]
methodical.macros/defmethod [[:inner 0]]
methodical.macros/defmulti [[:inner 0]]
saml20-clj.sp.response/validate-confirmation-datas [[:block 1]]
toucan2.connection/with-transaction [[:block 1]]
toucan2.core/build [[:block 0]]
toucan2.core/compile [[:block 0]]
toucan2.core/debug [[:block 0]]
toucan2.core/define-after-insert [[:inner 0]]
toucan2.core/define-after-select [[:inner 0]]
toucan2.core/define-after-update [[:inner 0]]
toucan2.core/define-before-delete [[:inner 0]]
toucan2.core/define-before-insert [[:inner 0]]
toucan2.core/define-before-select [[:inner 0]]
toucan2.core/define-default-fields [[:inner 0]]
toucan2.core/define-named-query [[:block 1]]
toucan2.core/deftransforms [[:block 1]]
toucan2.core/with-transaction [[:block 1]]
toucan2.tools.after-insert/define-after-insert [[:inner 0]]
toucan2.tools.after-select/define-after-select [[:inner 0]]
toucan2.tools.after-update/define-after-update [[:inner 0]]
toucan2.tools.after/define-after-impl [[:default]]
toucan2.tools.before-delete/define-before-delete [[:inner 0]]
toucan2.tools.before-insert/define-before-insert [[:inner 0]]
toucan2.tools.before-select/define-before-select [[:inner 0]]
toucan2.tools.compile/build [[:block 0]]
toucan2.tools.compile/compile [[:block 0]]
toucan2.tools.compile/resolved [[:block 0]]
toucan2.tools.debug/debug [[:block 0]]
toucan2.tools.default-fields/define-default-fields [[:inner 0]]
toucan2.tools.named-query/define-named-query [[:block 1]]
toucan2.tools.simple-out-transform/define-out-transform [[:inner 0]]
toucan2.tools.transformed/deftransforms [[:block 1]]
toucan2.tools.with-temp/with-temp [[:inner 0]]
toucan2.util/try-with-error-context [[:inner 0]]}}
metabase-enterprise.impersonation.util-test/with-impersonations! [[:inner 0]]
metabase-enterprise.cache.config-test/with-temp-persist-models [[:block 1]]
metabase-enterprise.sandbox.test-util/with-gtaps! [[:inner 0]]
metabase-enterprise.sandbox.test-util/with-gtaps-for-user! [[:inner 0]]
metabase-enterprise.sandbox.test-util/with-user-attributes [[:block 2]]
metabase-enterprise.serialization.test-util/with-dbs [[:inner 0]]
metabase-enterprise.serialization.test-util/with-random-dump-dir [[:block 1]]
metabase-enterprise.serialization.test-util/with-world [[:block 0]]
metabase-enterprise.sso.integrations.jwt-test/with-users-with-email-deleted [[:block 1]]
metabase-enterprise.test/with-gtaps! [[:inner 0]]
metabase-enterprise.test/with-gtaps-for-user! [[:inner 0]]
metabase-enterprise.test/with-user-attributes [[:block 2]]
metabase.actions.actions-test/with-actions-test-data-and-actions-permissively-enabled [[:block 0]]
metabase.actions.test-util/with-actions [[:block 1]]
metabase.actions.test-util/with-actions-disabled [[:block 0]]
metabase.actions.test-util/with-actions-enabled [[:block 0]]
metabase.actions.test-util/with-actions-test-data [[:inner 0]]
metabase.actions.test-util/with-actions-test-data-and-actions-enabled [[:block 0]]
metabase.actions.test-util/with-actions-test-data-tables [[:block 1]]
metabase.actions.test-util/with-empty-db [[:inner 0]]
metabase.actions.test-util/with-temp-test-data [[:inner 0]]
metabase.api.card-test/with-cards-in-readable-collection [[:block 1]]
metabase.api.card-test/with-cards-in-writeable-collection [[:block 1]]
metabase.api.card-test/with-ordered-items [[:inner 0]]
metabase.api.card-test/with-temp-native-card! [[:block 1]]
metabase.api.card-test/with-temp-native-card-with-params! [[:block 1]]
metabase.api.collection-test/with-collection-hierarchy [[:block 1]]
metabase.api.collection-test/with-french-user-and-personal-collection [[:block 2]]
metabase.api.collection-test/with-some-children-of-collection [[:block 1]]
metabase.api.common/check [[:default]]
metabase.api.common/define-routes [[:block 0]]
metabase.api.common/let-404 [[:block 1]]
metabase.api.embed-test/do-response-formats [[:block 1]]
metabase.api.embed-test/with-embedding-enabled-and-new-secret-key [[:block 0]]
metabase.api.embed-test/with-embedding-enabled-and-temp-card-referencing [[:block 3]]
metabase.api.embed-test/with-embedding-enabled-and-temp-dashcard-referencing [[:block 3]]
metabase.api.embed-test/with-new-secret-key [[:block 0]]
metabase.api.embed-test/with-temp-card [[:block 1]]
metabase.api.embed-test/with-temp-dashcard [[:block 1]]
metabase.api.preview-embed-test/with-embedding-enabled-and-new-secret-key [[:block 0]]
metabase.api.preview-embed-test/with-new-secret-key [[:block 0]]
metabase.cmd.copy/step [[:block 1]]
metabase.cmd.copy/with-connection-rollback-only [[:block 1]]
metabase.cmd.copy/with-disabled-db-constraints [[:block 2]]
metabase.config/build-type-case [[:block 0]]
metabase.dashboard-subscription-test/with-link-card-fixture-for-dashboard [[:block 2]]
metabase.db.liquibase/with-liquibase [[:block 1]]
metabase.db.liquibase/with-scope-locked [[:block 1]]
metabase.db.schema-migrations-test.impl/test-migrations [[:block 2]]
metabase.db/with-application-db [[:inner 0]]
metabase.driver.bigquery-cloud-sdk.query-processor/with-temporal-type [[:default]]
metabase.driver.druid.client/DELETE [[:default]]
metabase.driver.druid.client/GET [[:default]]
metabase.driver.druid.client/POST [[:default]]
metabase.driver.druid.query-processor-test/druid-query [[:block 0]]
metabase.driver.druid.query-processor-test/druid-query-returning-rows [[:block 0]]
metabase.driver.mongo.query-processor/mongo-let [[:block 1]]
metabase.driver.sql-jdbc.actions/with-jdbc-transaction [[:block 1]]
metabase.driver.sql-jdbc.connection/with-connection-spec-for-testing-connection [[:block 1]]
metabase.driver.sql-jdbc.execute.diagnostic/capturing-diagnostic-info [[:block 1]]
metabase.driver.sql.query-processor/apply-top-level-clause [[:default]]
metabase.driver/with-driver [[:block 1]]
metabase.lib.binning/with-binning [[:default]]
metabase.lib.core/with-binning [[:default]]
metabase.lib.core/with-join-alias [[:default]]
metabase.lib.core/with-join-conditions [[:default]]
metabase.lib.join/with-join-alias [[:default]]
metabase.lib.join/with-join-conditions [[:default]]
metabase.lib.test-util.macros/$ids [[:inner 0]]
metabase.lib.test-util.macros/mbql-query [[:inner 0]]
metabase.lib.util.match/match [[:inner 0]]
metabase.lib.util.match/match-one [[:inner 0]]
metabase.lib.util.match/replace [[:inner 0]]
metabase.lib.util.match/replace-in [[:inner 0]]
metabase.models.collection-test/with-collection-hierarchy [[:block 1]]
metabase.models.collection-test/with-current-user-perms-for-collections [[:block 1]]
metabase.models.collection-test/with-personal-and-impersonal-collections [[:block 1]]
metabase.models.dashboard-test/with-dash-in-collection [[:inner 0]]
metabase.models.interface/define-batched-hydration-method [[:inner 0]]
metabase.models.interface/define-simple-hydration-method [[:inner 0]]
metabase.models.params.chain-filter-test/with-clean-field-values-for-field [[:block 1]]
metabase.models.params.chain-filter-test/with-fk-field-to-field-remapping [[:block 0]]
metabase.models.params.chain-filter-test/with-human-readable-values-remapping [[:block 0]]
metabase.models.setting.multi-setting/define-multi-setting [[:inner 0]]
metabase.models.setting/defsetting [[:block 1]]
metabase.models.task-history/with-task-history [[:block 1]]
metabase.permissions.core/allow-changing-all-users-group-members [[:block 0]]
metabase.permissions.core/disable-perms-cache [[:block 0]]
metabase.public-sharing.api-test/with-sharing-enabled-and-temp-card-referencing [[:block 3]]
metabase.public-sharing.api-test/with-sharing-enabled-and-temp-dashcard-referencing [[:block 3]]
metabase.public-sharing.api-test/with-temp-public-card [[:block 1]]
metabase.public-sharing.api-test/with-temp-public-dashboard [[:block 1]]
metabase.public-sharing.api-test/with-temp-public-dashboard-and-card [[:block 1]]
metabase.pulse.api.alert-test/with-alert-in-collection [[:block 1]]
metabase.pulse.models.pulse-test/with-dashboard-subscription-in-collection [[:block 1]]
metabase.pulse.models.pulse-test/with-pulse-in-collection [[:inner 0]]
metabase.query-processor-test.pipeline-queries-test/pmbql-query [[:block 1]]
metabase.query-processor-test.pipeline-queries-test/run-pmbql-query [[:block 1]]
metabase.query-processor.debug/debug> [[:block 0]]
metabase.query-processor.error-type/deferror [[:block 1]]
metabase.query-processor.middleware.cache-backend.interface/with-cached-results [[:block 4]]
metabase.query-processor.middleware.process-userland-query-test/with-query-execution [[:block 1]]
metabase.query-processor.store/cached [[:block 1]]
metabase.query-processor.store/with-metadata-provider [[:inner 0]]
metabase.query-processor.streaming/streaming-response [[:block 1]]
metabase.query-processor.test-util/format-rows-by [[:default]]
metabase.request.core/as-admin [[:block 0]]
metabase.request.core/with-current-request [[:inner 0]]
metabase.request.core/with-current-user [[:inner 0]]
metabase.request.core/with-limit-and-offset [[:inner 0]]
metabase.request.session/as-admin [[:block 0]]
metabase.request.session/with-current-user [[:inner 0]]
metabase.sample-data-test/with-temp-sample-database-db [[:block 1]]
metabase.search.api-test/do-test-users [[:block 1]]
metabase.server.middleware.session/as-admin [[:block 0]]
metabase.server.middleware.session/with-current-user [[:inner 0]]
metabase.server.streaming-response-test/with-streaming-response-thread-pool [[:block 0]]
metabase.server.streaming-response-test/with-test-driver-db! [[:block 0]]
metabase.server.streaming-response/streaming-response [[:block 2]]
metabase.session.api/http-401-on-error [[:block 0]]
metabase.sync.fetch-metadata/log-if-error [[:block 1]]
metabase.sync.sync-metadata.fields-test/with-test-db [[:block 0]]
metabase.sync.util/do-with-start-and-finish-debug-logging [[:default]]
metabase.sync.util/sum-for [[:block 1]]
metabase.sync.util/sync-operation [[:block 3]]
metabase.sync.util/with-db-logging-disabled [[:default]]
metabase.sync.util/with-duplicate-ops-prevented [[:default]]
metabase.sync.util/with-emoji-progress-bar [[:block 1]]
metabase.sync.util/with-error-handling [[:block 1]]
metabase.sync.util/with-start-and-finish-logging [[:default]]
metabase.sync.util/with-start-and-finish-logging* [[:default]]
metabase.sync.util/with-sync-events [[:default]]
metabase.task/rerun-on-error [[:block 1]]
metabase.test/$ids [[:inner 0]]
metabase.test/as-admin [[:inner 0]]
metabase.test/dataset [[:inner 0]]
metabase.test/discard-setting-changes [[:block 1]]
metabase.test/format-rows-by [[:default]]
metabase.test/let-url [[:block 1]]
metabase.test/mbql-query [[:inner 0]]
metabase.test/native-query [[:block 0]]
metabase.test/query [[:block 1]]
metabase.test/run-mbql-query [[:inner 0]]
metabase.test/test-driver [[:inner 0]]
metabase.test/test-drivers [[:inner 0]]
metabase.test/test-helpers-set-global-values! [[:block 0]]
metabase.test/throw-if-called! [[:block 1]]
metabase.test/with-actions [[:block 1]]
metabase.test/with-actions-disabled [[:block 0]]
metabase.test/with-actions-enabled [[:block 0]]
metabase.test/with-actions-test-data [[:inner 0]]
metabase.test/with-actions-test-data-and-actions-enabled [[:block 0]]
metabase.test/with-actions-test-data-tables [[:block 1]]
metabase.test/with-additional-premium-features [[:block 1]]
metabase.test/with-current-user [[:inner 0]]
metabase.test/with-driver [[:block 1]]
metabase.test/with-empty-db [[:inner 0]]
metabase.test/with-empty-h2-app-db [[:block 0]]
metabase.test/with-metadata-provider [[:inner 0]]
metabase.test/with-premium-features [[:block 1]]
metabase.test/with-temp [[:inner 0]]
metabase.test/with-temp-copy-of-db [[:block 0]]
metabase.test/with-temp-scheduler [[:block 0]]
metabase.test/with-temp-test-data [[:inner 0]]
metabase.test/with-temp-vals-in-db [[:block 3]]
metabase.test/with-test-user [[:inner 0]]
metabase.test/with-user-in-groups [[:inner 0]]
metabase.test.data/$ids [[:inner 0]]
metabase.test.data/dataset [[:inner 0]]
metabase.test.data/mbql-query [[:inner 0]]
metabase.test.data/native-query [[:block 0]]
metabase.test.data/query [[:block 1]]
metabase.test.data/run-mbql-query [[:inner 0]]
metabase.test.data/with-empty-h2-app-db [[:block 0]]
metabase.test.data/with-temp-copy-of-db [[:block 0]]
metabase.test.data.datasets/test-driver [[:inner 0]]
metabase.test.data.datasets/test-drivers [[:inner 0]]
metabase.test.data.one-off-dbs/with-blank-db [[:block 0]]
metabase.test.data.one-off-dbs/with-blueberries-db [[:block 0]]
metabase.test.data.users/with-test-user [[:inner 0]]
metabase.test.util/discard-setting-changes [[:block 1]]
metabase.test.util/let-url [[:block 1]]
metabase.test.util/throw-if-called! [[:block 1]]
metabase.test.util/with-temp-scheduler [[:block 0]]
metabase.test.util/with-temp-vals-in-db [[:block 3]]
metabase.test.util/with-user-in-groups [[:inner 0]]
metabase.test.util.public-settings/with-additional-premium-features [[:block 1]]
metabase.test.util.public-settings/with-premium-features [[:block 1]]
metabase.test.util.thread-local/test-helpers-set-global-values! [[:block 0]]
metabase.timeseries-query-processor-test.util/test-timeseries-drivers [[:block 0]]
metabase.timeseries-query-processor-test/run-mbql-query [[:inner 0]]
metabase.upload-test/with-upload-table! [[:inner 0]]
metabase.upload.types/with-currency [[:default]]
metabase.upload.types/with-parens [[:default]]
metabase.util/auto-retry [[:block 1]]
metabase.util/case-enum [[:block 1]]
metabase.util/ignore-exceptions [[:block 0]]
metabase.util/or-with [[:block 1]]
metabase.util/profile [[:block 1]]
metabase.util/prog1 [[:inner 0]]
metabase.util/varargs [[:inner 0]]
metabase.util/with-us-locale [[:block 0]]
metabase.util.async/cancelable-thread [[:block 0]]
metabase.util.date-2/with-time-zone-same-instant [[:default]]
metabase.util.encryption-test/with-secret-key [[:block 1]]
metabase.util.honey-sql-2/with-database-type-info [[:default]]
metabase.util.i18n/deferred-trs [[:default]]
metabase.util.i18n/deferred-tru [[:default]]
metabase.util.i18n/trs [[:default]]
metabase.util.i18n/trsn [[:default]]
metabase.util.i18n/tru [[:default]]
metabase.util.i18n/trun [[:default]]
metabase.util.jvm/auto-retry [[:block 1]]
metabase.util.jvm/varargs [[:block 1]]
metabase.util.jvm/with-us-locale [[:block 0]]
metabase.util.malli/disable-enforcement [[:block 0]]
metabase.util.malli/with [[:default]]
metabase.util.malli/with-api-error-message [[:default]]
metabase.util.namespaces/import-fns [[:default]]
metabuild-common.core/exit-when-finished-nonzero-on-exception [[:block 0]]
metabuild-common.core/step [[:block 1]]
metabuild-common.entrypoint/exit-when-finished-nonzero-on-exception [[:block 0]]
metabuild-common.steps/step [[:block 1]]
methodical.core/defmethod [[:inner 0]]
methodical.core/defmulti [[:inner 0]]
methodical.core/with-dispatcher [[:default]]
methodical.core/with-method-table [[:default]]
methodical.core/with-prefers [[:default]]
methodical.interface/with-dispatcher [[:default]]
methodical.interface/with-method-table [[:default]]
methodical.interface/with-prefers [[:default]]
methodical.macros/defmethod [[:inner 0]]
methodical.macros/defmulti [[:inner 0]]
saml20-clj.sp.response/validate-confirmation-datas [[:block 1]]
toucan2.connection/with-transaction [[:block 1]]
toucan2.core/build [[:block 0]]
toucan2.core/compile [[:block 0]]
toucan2.core/debug [[:block 0]]
toucan2.core/define-after-insert [[:inner 0]]
toucan2.core/define-after-select [[:inner 0]]
toucan2.core/define-after-update [[:inner 0]]
toucan2.core/define-before-delete [[:inner 0]]
toucan2.core/define-before-insert [[:inner 0]]
toucan2.core/define-before-select [[:inner 0]]
toucan2.core/define-default-fields [[:inner 0]]
toucan2.core/define-named-query [[:block 1]]
toucan2.core/deftransforms [[:block 1]]
toucan2.core/with-transaction [[:block 1]]
toucan2.tools.after/define-after-impl [[:default]]
toucan2.tools.after-insert/define-after-insert [[:inner 0]]
toucan2.tools.after-select/define-after-select [[:inner 0]]
toucan2.tools.after-update/define-after-update [[:inner 0]]
toucan2.tools.before-delete/define-before-delete [[:inner 0]]
toucan2.tools.before-insert/define-before-insert [[:inner 0]]
toucan2.tools.before-select/define-before-select [[:inner 0]]
toucan2.tools.compile/build [[:block 0]]
toucan2.tools.compile/compile [[:block 0]]
toucan2.tools.compile/resolved [[:block 0]]
toucan2.tools.debug/debug [[:block 0]]
toucan2.tools.default-fields/define-default-fields [[:inner 0]]
toucan2.tools.named-query/define-named-query [[:block 1]]
toucan2.tools.simple-out-transform/define-out-transform [[:inner 0]]
toucan2.tools.transformed/deftransforms [[:block 1]]
toucan2.tools.with-temp/with-temp [[:inner 0]]
toucan2.util/try-with-error-context [[:inner 0]]}}

View File

@@ -1,35 +0,0 @@
afterAll
ags
altho
Brite
cancelation
cancelations
caracteres
copyin
datas
edn
eduction
hasTable
hasTables
horizontalX
horizontalY
juxt
methodd
nd
notIn
optionA
ot
pass-thru
requestor
re-use
re-used
ser
serieses
sortings
titel
trs
trsn
tru
trun
unparseable
withs

View File

@@ -1,24 +0,0 @@
[codespell]
skip =
*.csv,
*.edn,
*.geojson,
*.json,
*.tsv,
*.unit.spec.*,
*.xml,
docs/developers-guide/drivers/driver-tests.md,
docs/developers-guide/e2e-tests.md,
docs/questions/query-builder/expressions-list.md,
docs/questions/visualizations/country-codes.md,
e2e/test/scenarios/admin/i18n/content-translation/constants.ts,
frontend/src/cljs,
frontend/src/metabase/querying/expressions/tokenizer/lezer.js,
frontend/src/metabase/query_builder/components/NativeQueryEditor/CodeMirrorEditor/language.ts,
frontend/src/metabase/static-viz/constants/char-sizes.ts,
frontend/src/metabase/visualizations/lib/mapping_codes.ts,
frontend/*/mocks/*,
modules/drivers/mongo/src/metabase/driver/mongo/operators.clj,
src/metabase/analyze/classifiers/name.clj,
src/metabase/util/currency.cljc,
ignore-words = .codespellignore

View File

@@ -1,11 +1,11 @@
FROM mcr.microsoft.com/vscode/devcontainers/java:21
FROM mcr.microsoft.com/vscode/devcontainers/java:11
# Set up nodesource, install node, bun, fontconfig for static viz, rlwrap for dev ergonomics
# Set up nodesource, install node, yarn, fontconfig for static viz, rlwrap for dev ergonomics
RUN ( curl -fsSL https://deb.nodesource.com/setup_22.x | bash ) \
RUN ( curl -fsSL https://deb.nodesource.com/setup_18.x | bash ) \
&& export DEBIAN_FRONTEND=noninteractive \
&& apt-get update && apt-get -y install --no-install-recommends nodejs rlwrap fontconfig \
&& npm install -g bun
&& apt-key adv --refresh-keys --keyserver keyserver.ubuntu.com \
&& apt-get update && apt-get -y install --no-install-recommends nodejs yarn rlwrap fontconfig
# install Clojure
RUN curl https://download.clojure.org/install/linux-install-1.12.0.1488.sh | bash
RUN curl https://download.clojure.org/install/linux-install-1.11.1.1262.sh | bash

View File

@@ -61,7 +61,6 @@
(eval . (put-clojure-indent 'u/prog1 1))
(eval . (put-clojure-indent 'u/select-keys-when '(:form)))
(eval . (put-clojure-indent 'with-meta '(:form)))
(eval . (put-clojure-indent 'ws.log/track! 2))
;; these ones have to be done with `define-clojure-indent' for now because of upstream bug
;; https://github.com/clojure-emacs/clojure-mode/issues/600 once that's resolved we should use `put-clojure-indent'
;; instead. Please don't add new entries unless they don't work with `put-clojure-indent'

View File

@@ -1,4 +1,5 @@
.babel_cache
docs/*
target/*
.circleci
.cpcache

View File

@@ -1,163 +0,0 @@
# ==============================================================================
# 0. METABASE DEVELOPMENT ENVIRONMENT
# ==============================================================================
#
# Central configuration for local Metabase development.
# Copy to .env and uncomment/modify values as needed.
#
# Usage:
# op inject --in-file .env.example --out-file .env (populate using CLI)
# cp .env.example .env (populate manually)
#
# Hint: Auto-load environment variables with mise or direnv:
# mise - mise.toml: [settings] env_file = ".env" (https://mise.jdx.dev)
# direnv - Create .envrc with: dotenv .env (https://direnv.net)
#
# Full Metabase environment configuration reference:
# https://www.metabase.com/docs/latest/configuring-metabase/environment-variables
#
# ⚠️ Make sure the same var is not exported multiple times in different contexts.
# ==============================================================================
# 1. GLOBAL CONTEXT
# ==============================================================================
#
# Shared credentials and tokens used across multiple dev workflows.
# GitHub - For gh CLI, API access, and CI integrations
# https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens
# GITHUB_TOKEN=your-PAT
# AWS - For S3, Athena, Redshift, and other AWS services
# https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html
# AWS_ACCESS_KEY_ID=your-access-key
# AWS_SECRET_ACCESS_KEY=your-secret-key
# AWS_REGION=us-east-1
# Docker Hub - Avoid rate limits when pulling images
# https://docs.docker.com/security/for-developers/access-tokens/
# DOCKER_USERNAME=your-username
# DOCKER_PASSWORD=your-access-token
# ==============================================================================
# 2. LOCAL METABASE DEVELOPMENT
# ==============================================================================
#
# Settings for running Metabase locally. Most have sensible defaults—only
# configure what you need.
# 2.1 Application Database
# ------------------------------------------------------------------------------
# Where Metabase stores its own data (dashboards, users, settings, etc.).
# https://www.metabase.com/docs/latest/installation-and-operation/configuring-application-database
# Option A: Individual connection parameters
# MB_DB_TYPE="postgres"
# MB_DB_HOST="localhost"
# MB_DB_PORT=5432
# MB_DB_DBNAME="metabase"
# MB_DB_USER="<username>"
# MB_DB_PASS="<password>"
# Option B: Single connection URI (overrides individual params)
# MB_DB_CONNECTION_URI="jdbc:postgresql://localhost:5432/metabase?user=<username>&password=<password>"
# 2.2 Server & Integrations
# ------------------------------------------------------------------------------
# https://www.metabase.com/docs/latest/configuring-metabase/customizing-jetty-webserver
# Jetty server (defaults: localhost:3000)
# MB_JETTY_HOST="localhost"
# MB_JETTY_PORT=3000
# SMTP for testing email notifications (use with maildev/maildev container)
MB_EMAIL_SMTP_HOST="localhost"
MB_EMAIL_SMTP_PORT=1025
MB_EMAIL_SMTP_USERNAME="admin"
MB_EMAIL_SMTP_PASSWORD="admin"
# 2.3 Feature Flags & Tokens
# ------------------------------------------------------------------------------
# Use staging server for license validation (avoids hitting production)
METASTORE_DEV_SERVER_URL="https://token-check.staging.metabase.com"
# Metabase license token
# Uncomment this line only when running local dev and NEVER when running Cypress
# Tests need to start from zero, and set different tokens programmatically (on-demand)
# MB_PREMIUM_EMBEDDING_TOKEN="op://Shared/Metabase License Dev Tokens/Licenses/MB_ALL_FEATURES_TOKEN"
# Skip embedding SDK build for faster frontend compilation
# SKIP_EMBEDDING_SDK="true"
# ==============================================================================
# 3. E2E / CYPRESS TESTING
# ==============================================================================
#
# Configuration for running Cypress end-to-end tests locally.
#
# The whole environment is already orchestrated via deps.edn using :e2e alias.
# Treat it as a source of truth for JVM flags and ENVs whether you decide to run
# backend on your own, or using `bun run test-cypress` (local test runner).
#
# You don't _have to_ export any of these vars (except for the tokens below).
# Things will still work.
# 3.1. E2E Application Database (H2 required)
# ------------------------------------------------------------------------------
# Whether to run the enterprise or the open-source Metabase version
MB_EDITION="ee"
# Which file to use for the ephemeral testing application database?
# MB_DB_FILE="./e2e/tmp/cypress-test.db"
# Where does Cypress go to look for the application? (defaults: localhost:4000)
# MB_JETTY_HOST="localhost"
# MB_JETTY_PORT=4000
# Frontend dev server port (for hot reload). Must match the port used by `yarn build-hot`.
# MB_FRONTEND_DEV_PORT=8080
# 3.2 E2E Local Runner Setup
# ------------------------------------------------------------------------------
# Whether to run true e2e test suite, or SDK component tests (e2e | component)
CYPRESS_TESTING_TYPE="e2e"
# Whether to start Cypress in an interactive (open) mode with full GUI
CYPRESS_GUI="true"
# Whether to generate application database dumps (colloquially "snapshots")
# Keep this enabled unless you're experimenting, or know what you're doing.
GENERATE_SNAPSHOTS="true"
# E2E tests run the backend from source by default.
# To run tests against a JAR instead, set the JAR_PATH variable.
#
# Hint: build the JAR with ./bin/build.sh
# JAR_PATH="target/uberjar/metabase.jar"
# 3.3 E2E Tokens
# ------------------------------------------------------------------------------
#
# These make sense for Metabase core devs only
# op inject --in-file .env.example --out-file .env
CYPRESS_MB_ALL_FEATURES_TOKEN="op://Shared/Metabase License Dev Tokens/Licenses/MB_ALL_FEATURES_TOKEN"
CYPRESS_MB_STARTER_CLOUD_TOKEN="op://Shared/Metabase License Dev Tokens/Licenses/MB_STARTER_CLOUD_TOKEN"
CYPRESS_MB_PRO_CLOUD_TOKEN="op://Shared/Metabase License Dev Tokens/Licenses/MB_PRO_CLOUD_TOKEN"
CYPRESS_MB_PRO_SELF_HOSTED_TOKEN="op://Shared/Metabase License Dev Tokens/Licenses/MB_PRO_SELF_HOSTED_TOKEN"
# 3.4 E2E Misc.
# ------------------------------------------------------------------------------
# To make Node.js match the instance tz
TZ="US/Pacific"
# Export this environment variable when running tests on Apple Silicon.
# DO NOT USE Docker Desktop. Use Orbstack instead. Metabase will fail to connect
# to the dockerized Mongo QA database otherwise.
EXPERIMENTAL_DOCKER_DESKTOP_FORCE_QEMU=1

4
.eslintignore Normal file
View File

@@ -0,0 +1,4 @@
frontend/src/cljs
frontend/src/cljs_release
e2e/support/cypress_sample_database.js
e2e/support/cypress_sample_instance_data.js

292
.eslintrc.js Normal file
View File

@@ -0,0 +1,292 @@
/* eslint-disable import/no-commonjs */
/* eslint-disable no-undef */
// `postcss-modules` lints css modules class names, but it currently crashes
// eslint on vscode. If you use webstorm or want to run the lint for the cli, you
// can use this flag to enable it. This is set to true in CI
const shouldLintCssModules =
process.env.LINT_CSS_MODULES === "true" || process.env.CI;
const plugins = ["react", "no-only-tests"];
if (shouldLintCssModules) {
plugins.push("postcss-modules");
}
module.exports = {
rules: {
strict: [2, "never"],
"no-undef": 2,
"no-var": 1,
"no-unused-vars": [
"error",
{
vars: "all",
args: "none",
varsIgnorePattern: "^_.+$",
ignoreRestSiblings: true,
},
],
"no-empty": [1, { allowEmptyCatch: true }],
// Note: adding this rule to a eslint config file in a subfolder will remove
// *not* carry over the restricted imports from parent folders, you will
// need to copy them over
"no-restricted-imports": [
"error",
{
paths: [
{
name: "moment",
message: "Moment is deprecated, please use dayjs",
},
{
name: "moment-timezone",
message: "Moment is deprecated, please use dayjs",
},
],
},
],
curly: [1, "all"],
eqeqeq: [1, "smart"],
"import/no-default-export": 2,
"import/no-named-as-default": 0,
"import/no-commonjs": 1,
"import/order": [
"error",
{
"newlines-between": "always",
alphabetize: {
order: "asc",
orderImportKind: "asc",
caseInsensitive: false,
},
groups: [
"builtin",
"external",
"internal",
"parent",
"sibling",
"index",
],
warnOnUnassignedImports: false,
},
],
"sort-imports": [
"error",
{
// allows this rule to work with import/order
ignoreDeclarationSort: true,
},
],
"no-console": [2, { allow: ["warn", "error", "errorBuffer"] }],
"react/no-is-mounted": 2,
"react/prefer-es6-class": 2,
"react/display-name": 1,
"react/prop-types": 2,
"react/no-did-mount-set-state": 0,
"react/no-did-update-set-state": 0,
"react/no-find-dom-node": 0,
"react/no-children-prop": 2,
"react/no-string-refs": 2,
"react/no-unescaped-entities": 2,
"react/jsx-no-target-blank": 2,
"react/jsx-key": 2,
"react/forbid-component-props": [2, { forbid: ["sx"] }],
"react-hooks/exhaustive-deps": [
"warn",
{ additionalHooks: "(useSafeAsyncFunction)" },
],
"prefer-const": [1, { destructuring: "all" }],
"no-restricted-globals": ["error", "close"],
"no-useless-escape": 0,
"no-only-tests/no-only-tests": [
"error",
{
block: [
"describe",
"it",
"context",
"test",
"tape",
"fixture",
"serial",
"Feature",
"Scenario",
"Given",
"And",
"When",
"Then",
"describeWithSnowplow",
],
},
],
complexity: ["error", { max: 54 }],
...(shouldLintCssModules
? {
"postcss-modules/no-undef-class": "error",
}
: {}),
},
globals: {
before: true,
cy: true,
Cypress: true,
},
env: {
browser: true,
es6: true,
commonjs: true,
jest: true,
"jest/globals": true,
},
parser: "babel-eslint",
plugins,
extends: [
"eslint:recommended",
"plugin:react/recommended",
"plugin:react/jsx-runtime",
"plugin:react-hooks/recommended",
"plugin:import/errors",
"plugin:import/warnings",
"plugin:import/typescript",
"plugin:depend/recommended",
"plugin:storybook/recommended",
],
settings: {
"import/internal-regex": "^metabase/|^metabase-lib/",
"import/resolver": {
webpack: {
typescript: true,
},
},
"import/ignore": ["\\.css$"],
react: {
version: "detect",
},
"postcss-modules": {
baseDir: "./frontend/src",
},
},
parserOptions: {
ecmaFeatures: {
legacyDecorators: true,
},
},
overrides: [
{
files: ["*.js", "*.jsx", "*.ts", "*.tsx"],
rules: {
"no-unconditional-metabase-links-render": "error",
"no-color-literals": "error",
"no-literal-metabase-strings": "error",
"depend/ban-dependencies": [
"error",
{
allowed: [
"underscore",
"moment",
"lodash.orderby",
"lodash.debounce",
],
},
],
},
},
{
files: [
"*.unit.spec.*",
"frontend/src/metabase/admin/**/*",
"frontend/src/metabase/setup/**/*",
"frontend/lint/**/*",
"*.stories.*",
"stories-data.*",
"e2e/**/*",
"**/tests/*",
"release/**/*",
],
rules: {
"no-color-literals": "off",
"no-unconditional-metabase-links-render": "off",
"no-literal-metabase-strings": "off",
},
},
{
extends: ["plugin:@typescript-eslint/recommended"],
files: ["*.ts", "*.tsx"],
parser: "@typescript-eslint/parser",
plugins: ["@typescript-eslint"],
rules: {
"prefer-rest-params": "off",
"@typescript-eslint/explicit-module-boundary-types": "off",
"@typescript-eslint/no-inferrable-types": "off",
"@typescript-eslint/no-explicit-any": "off",
"@typescript-eslint/no-this-alias": "off",
"@typescript-eslint/consistent-type-imports": [
"error",
{
fixStyle: "inline-type-imports",
},
],
"@typescript-eslint/no-import-type-side-effects": "error",
"@typescript-eslint/no-unused-vars": [
"error",
{
argsIgnorePattern: "^_.+$",
varsIgnorePattern: "^_.+$",
ignoreRestSiblings: true,
destructuredArrayIgnorePattern: "^_.+$",
},
],
// This was introduced in 6.0.0
"@typescript-eslint/no-unsafe-declaration-merging": "off",
},
},
{
extends: [
"plugin:jest/recommended",
"plugin:jest-dom/recommended",
"plugin:testing-library/react",
"plugin:jest-formatting/recommended",
],
plugins: ["jest", "jest-dom", "testing-library", "jest-formatting"],
files: [
"*.unit.spec.ts",
"*.unit.spec.tsx",
"*.unit.spec.js",
"*.unit.spec.jsx",
],
rules: {
"jest/valid-title": ["error", { ignoreTypeOfDescribeName: true }],
},
},
{
// Enable jest formatting for cypress tests too, the plugin logic just works
extends: ["plugin:jest-formatting/recommended"],
files: ["*.cy.spec.ts", "*.cy.spec.js"],
},
{
files: ["frontend/src/**/*"],
rules: {
"no-restricted-syntax": [
"error",
{
selector: "Literal[value=/mb-base-color-/]",
message:
"You may not use base colors in the application, use semantic colors instead. (see colors.module.css)",
},
],
},
},
{
files: ["frontend/src/metabase/**/*"],
rules: {
"no-restricted-syntax": [
"error",
{
selector: "Literal[value=/mb-base-color-/]",
message:
"You may not use base colors in the application, use semantic colors instead. (see colors.module.css)",
}
],
},
},
],
};

9
.gitattributes vendored
View File

@@ -1,9 +0,0 @@
# Custom merge driver for per-release YAML migration files.
# Treats each changeset as an atomic unit to avoid conflicts within changesets.
# Only applies to per-release *_update_migrations.yaml files — directory-based
# migrations (e.g. 060/*.yaml) have one file per migration and don't need this.
# Execute this in your shell to enable custom merge driver:
# echo '\n[include]\npath=../.gitconfig' >> .git/config
#
# TODO (Bryan 12/12/25) -- reenable this when we can completely verify correctness
# resources/migrations/*_update_migrations.yaml merge=yaml-migrations

View File

@@ -1,5 +0,0 @@
# execute this in your shell to enable custom merge driver:
# echo '\n[include]\npath=../.gitconfig' >> .git/config
[merge "yaml-migrations"]
name = Per-release YAML migration changeset-aware merge
driver = bin/mage -merge-yaml-migrations %O %A %B %L %P

273
.github/CODEOWNERS vendored
View File

@@ -1,30 +1,261 @@
# Before changing this file, make sure you have read how we use it:
# https://www.notion.so/metabase/Pull-Request-Code-Review-Guide-374f05df889748b69d67af753f9bc9b8?pvs=4#36550c04ceab40b5a8b8bee6264b090c
# @metabase/core-backend-querying-platform
src/metabase/lib @metabase/core-backend-querying-platform
src/metabase/lib_be @metabase/core-backend-querying-platform
src/metabase/query_processor @metabase/core-backend-querying-platform
# @metabase/graphy
src/metabase/graph @metabase/graphy
enterprise/backend/src/metabase_enterprise/dependencies @metabase/graphy
enterprise/frontend/src/metabase-enterprise/dependencies @metabase/graphy
# @metabase/embedding
enterprise/frontend/src/embedding-sdk-package @metabase/embedding
enterprise/frontend/src/embedding-sdk-bundle @metabase/embedding
enterprise/frontend/src/embedding-sdk-shared @metabase/embedding
enterprise/frontend/src/embedding-sdk @metabase/embedding
frontend/src/metabase/embedding-sdk @metabase/embedding
frontend/src/metabase/public/containers/PublicOrEmbeddedQuestion @metabase/embedding
frontend/src/metabase/public/containers/PublicOrEmbeddedDashboard @metabase/embedding
frontend/src/metabase/ui @metabase/core-frontend-admin-webapp
# snowplow/* @metabase/data
# @metabase/tech-writers
docs @metabase/tech-writers
docs/developers-guide # exclude from tech-writer ownership
# @metabase/devexp
.github @metabase/devexp
.clj-kondo @metabase/devexp
deps.edn @metabase/backend-developers
# @metabase/cloud-ops
src/metabase/analytics/prometheus.clj @metabase/cloud-ops
docs @metabase/tech-writers
#
# Backend modules <=> teams that own them
#
# modules/drivers @metabase/drivers
# enterprise/backend/src/metabase_enterprise/advanced_config @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/advanced_config @metabase/core-backend-admin-webapp
# enterprise/backend/src/metabase_enterprise/advanced_permissions @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/advanced_permissions @metabase/core-backend-admin-webapp
# enterprise/backend/src/metabase_enterprise/airgap @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/airgap @metabase/core-backend-admin-webapp
# enterprise/backend/src/metabase_enterprise/api @metabase/devexp
# enterprise/backend/test/metabase_enterprise/api @metabase/devexp
# enterprise/backend/src/metabase_enterprise/audit_app @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/audit_app @metabase/core-backend-admin-webapp
# enterprise/backend/src/metabase_enterprise/auth_provider @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/auth_provider @metabase/core-backend-admin-webapp
# enterprise/backend/src/metabase_enterprise/billing @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/billing @metabase/core-backend-admin-webapp
# enterprise/backend/src/metabase_enterprise/cache @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/cache @metabase/core-backend-admin-webapp
# enterprise/backend/src/metabase_enterprise/content_verification @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/content_verification @metabase/core-backend-admin-webapp
# enterprise/backend/src/metabase_enterprise/core @metabase/devexp
# enterprise/backend/test/metabase_enterprise/core @metabase/devexp
# enterprise/backend/src/metabase_enterprise/dashboard_subscription_filters @metabase/core-workflows
# enterprise/backend/test/metabase_enterprise/dashboard_subscription_filters @metabase/core-workflows
# enterprise/backend/src/metabase_enterprise/enhancements @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/enhancements @metabase/core-backend-admin-webapp
# # enterprise/backend/src/metabase_enterprise/llm
# # enterprise/backend/test/metabase_enterprise/llm
# enterprise/backend/src/metabase_enterprise/query_reference_validation @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/query_reference_validation @metabase/core-backend-admin-webapp
# enterprise/backend/src/metabase_enterprise/sandbox @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/sandbox @metabase/core-backend-admin-webapp
# enterprise/backend/src/metabase_enterprise/scim @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/scim @metabase/core-backend-admin-webapp
# enterprise/backend/src/metabase_enterprise/search @metabase/core-workflows
# enterprise/backend/test/metabase_enterprise/search @metabase/core-workflows
# enterprise/backend/src/metabase_enterprise/serialization @metabase/core-workflows
# enterprise/backend/test/metabase_enterprise/serialization @metabase/core-workflows
# enterprise/backend/src/metabase_enterprise/snippet_collections @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/snippet_collections @metabase/core-backend-admin-webapp
# enterprise/backend/src/metabase_enterprise/sso @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/sso @metabase/core-backend-admin-webapp
# enterprise/backend/src/metabase_enterprise/stale @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/stale @metabase/core-backend-admin-webapp
# enterprise/backend/src/metabase_enterprise/stats @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/stats @metabase/core-backend-admin-webapp
# enterprise/backend/src/metabase_enterprise/task @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/task @metabase/core-backend-admin-webapp
# enterprise/backend/src/metabase_enterprise/upload_management @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/upload_management @metabase/core-backend-admin-webapp
# src/metabase/actions @metabase/core-backend-admin-webapp
# test/metabase/actions @metabase/core-backend-admin-webapp
# src/metabase/activity_feed @metabase/core-backend-admin-webapp
# test/metabase/activity_feed @metabase/core-backend-admin-webapp
# src/metabase/analytics @metabase/core-backend-admin-webapp
# test/metabase/analytics @metabase/core-backend-admin-webapp
# src/metabase/analyze @metabase/drivers
# test/metabase/analyze @metabase/drivers
# src/metabase/api @metabase/devexp
# test/metabase/api @metabase/devexp
# src/metabase/audit @metabase/core-backend-admin-webapp
# test/metabase/audit @metabase/core-backend-admin-webapp
# src/metabase/auth_provider @metabase/core-backend-admin-webapp
# test/metabase/auth_provider @metabase/core-backend-admin-webapp
# src/metabase/bookmarks @metabase/core-backend-admin-webapp
# test/metabase/bookmarks @metabase/core-backend-admin-webapp
# src/metabase/cmd @metabase/devexp
# test/metabase/cmd @metabase/devexp
# src/metabase/channel @metabase/core-workflows
# test/metabase/channel @metabase/core-workflows
# src/metabase/cloud_migration @metabase/core-workflows
# test/metabase/cloud_migration @metabase/core-workflows
# src/metabase/cmd @metabase/devexp
# test/metabase/cmd @metabase/devexp
# src/metabase/config @metabase/devexp
# test/metabase/config @metabase/devexp
# src/metabase/core @metabase/devexp
# test/metabase/core @metabase/devexp
# src/metabase/db @metabase/core-backend-admin-webapp
# test/metabase/db @metabase/core-backend-admin-webapp
# src/metabase/driver @metabase/drivers
# test/metabase/driver @metabase/drivers
# src/metabase/eid_translation @metabase/core-backend-admin-webapp
# test/metabase/eid_translation @metabase/core-backend-admin-webapp
# src/metabase/embed @metabase/embedding
# test/metabase/embed @metabase/embedding
# src/metabase/events @metabase/devexp
# test/metabase/events @metabase/devexp
# src/metabase/formatter @metabase/core-backend-querying
# test/metabase/formatter @metabase/core-backend-querying
# src/metabase/indexed_entities @metabase/core-backend-querying
# test/metabase/indexed_entities @metabase/core-backend-querying
# src/metabase/integrations @metabase/core-workflows
# test/metabase/integrations @metabase/core-workflows
# src/metabase/legacy_mbql @metabase/core-backend-querying
# test/metabase/legacy_mbql @metabase/core-backend-querying
# src/metabase/lib @metabase/core-backend-querying
# test/metabase/lib @metabase/core-backend-querying
# src/metabase/logger @metabase/devexp
# test/metabase/logger @metabase/devexp
# src/metabase/login_history @metabase/core-backend-admin-webapp
# test/metabase/login_history @metabase/core-backend-admin-webapp
# src/metabase/model_persistence @metabase/core-backend-querying
# test/metabase/model_persistence @metabase/core-backend-querying
# src/metabase/models @metabase/devexp
# test/metabase/models @metabase/devexp
# src/metabase/moderation @metabase/core-backend-admin-webapp
# test/metabase/moderation @metabase/core-backend-admin-webapp
# src/metabase/notification @metabase/core-workflows
# test/metabase/notification @metabase/core-workflows
# src/metabase/permissions @metabase/core-backend-admin-webapp
# test/metabase/permissions @metabase/core-backend-admin-webapp
# src/metabase/plugins @metabase/devexp
# test/metabase/plugins @metabase/devexp
# src/metabase/premium_features @metabase/core-backend-admin-webapp
# test/metabase/premium_features @metabase/core-backend-admin-webapp
# src/metabase/public_settings @metabase/core-backend-admin-webapp
# test/metabase/public_settings @metabase/core-backend-admin-webapp
# src/metabase/public_sharing @metabase/core-backend-admin-webapp
# test/metabase/public_sharing @metabase/core-backend-admin-webapp
# src/metabase/pulse @metabase/core-workflows
# test/metabase/pulse @metabase/core-workflows
# src/metabase/query_analysis @metabase/core-backend-querying
# test/metabase/query_analysis @metabase/core-backend-querying
# src/metabase/query_processor @metabase/core-backend-querying
# test/metabase/query_processor @metabase/core-backend-querying
# src/metabase/request @metabase/devexp
# test/metabase/request @metabase/devexp
# src/metabase/revisions @metabase/core-backend-admin-webapp
# test/metabase/revisions @metabase/core-backend-admin-webapp
# src/metabase/sample_data @metabase/core-backend-admin-webapp
# test/metabase/sample_data @metabase/core-backend-admin-webapp
# src/metabase/search @metabase/core-workflows
# test/metabase/search @metabase/core-workflows
# src/metabase/segments @metabase/core-backend-querying
# test/metabase/segments @metabase/core-backend-querying
# src/metabase/server @metabase/devexp
# test/metabase/server @metabase/devexp
# src/metabase/session @metabase/core-backend-admin-webapp
# test/metabase/session @metabase/core-backend-admin-webapp
# src/metabase/setup @metabase/core-backend-admin-webapp
# test/metabase/setup @metabase/core-backend-admin-webapp
# src/metabase/sso @metabase/core-backend-admin-webapp
# test/metabase/sso @metabase/core-backend-admin-webapp
# src/metabase/sync @metabase/drivers
# test/metabase/sync @metabase/drivers
# src/metabase/task @metabase/devexp
# test/metabase/task @metabase/devexp
# src/metabase/tiles @metabase/core-backend-dashviz
# test/metabase/tiles @metabase/core-backend-dashviz
# src/metabase/timeline @metabase/core-backend-dashviz
# test/metabase/timeline @metabase/core-backend-dashviz
# src/metabase/types @metabase/core-backend-querying
# test/metabase/types @metabase/core-backend-querying
# src/metabase/upload @metabase/core-backend-admin-webapp
# test/metabase/upload @metabase/core-backend-admin-webapp
# src/metabase/user_key_value @metabase/core-backend-admin-webapp
# test/metabase/user_key_value @metabase/core-backend-admin-webapp
# src/metabase/util @metabase/devexp
# test/metabase/util @metabase/devexp
# src/metabase/xrays @metabase/core-backend-querying
# test/metabase/xrays @metabase/core-backend-querying

View File

@@ -63,8 +63,7 @@ body:
attributes:
label: Information about your Metabase installation
description: |
For versions >= 56, copy and paste the Diagnostic info from `Admin -> Tools -> Help`.
For versions < 56, copy and paste the Diagnostic info from `Admin -> Troubleshooting`.
Copy and paste the JSON from `Admin -> Troubleshooting`.
_This will be automatically formatted into code, so no need for backticks._
placeholder: |
{"browser-info": {

23
.github/ISSUE_TEMPLATE/epic.md vendored Normal file
View File

@@ -0,0 +1,23 @@
---
name: Epic (Internal Use)
about: This issue is used to track a feature/project that may span several days
to weeks to implement. This is used by the core Metabase team to project plan
and manage work.
title: "[Epic] Title"
labels: ".Epic"
---
### Links
- [product doc](LINK_TO_PRODUCT_DOC)
- [technical design doc](LINK_TO_TECHNICAL_DESIGN_DOC)
- [testing plan](LINK_TO_TESTING_PLAN)
- feature branch: `BRANCH_NAME`
- issue links: _related issues, if any_
### Implementation Plan
```[tasklist]
#### Milestone 1
- [ ] _Add tasks here_
```

View File

@@ -7,13 +7,16 @@ assignees: ''
---
**What problem will this feature request solve?**
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like.**
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**How does this feature request impact you?**
For example, is it a minor inconvenience, or does it block important work?
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional information**
**How important is this feature to you?**
**Additional context**
Add any other context or screenshots about the feature request here.

14
.github/ISSUE_TEMPLATE/task.md vendored Normal file
View File

@@ -0,0 +1,14 @@
---
name: Task (Internal Use)
about: This issue is used to track a smaller feature/task that does not require
being broken down to milestones. This is used by the core Metabase team to
project plan and manage work.
labels: ".Task"
---
**Context**
- issue links: _related issues if any_

View File

@@ -3,7 +3,7 @@
> **Warning**
>
> If that is your first contribution to Metabase, please sign the [Contributor License Agreement](https://docs.google.com/a/metabase.com/forms/d/1oV38o7b9ONFSwuzwmERRMi9SYrhYeOrkbmNaq9pOJ_E/viewform). Also, if you're attempting to fix a translation issue, please submit your changes to our [Crowdin project](https://crowdin.com/project/metabase-i18n) instead of opening a PR.
> If that is your first contribution to Metabase, please sign the [Contributor License Agreement](https://docs.google.com/a/metabase.com/forms/d/1oV38o7b9ONFSwuzwmERRMi9SYrhYeOrkbmNaq9pOJ_E/viewform) (unless it's a tiny documentation change). Also, if you're attempting to fix a translation issue, please submit your changes to our [Crowdin project](https://crowdin.com/project/metabase-i18n) instead of opening a PR.
Closes https://github.com/metabase/metabase/issues/[issue_number]
@@ -25,4 +25,3 @@ _Upload a demo video or before/after screenshots if sensible or remove the secti
### Checklist
- [ ] Tests have been added/updated to cover changes in this PR
- [ ] If adding new Loki tests: they pass [stress testing](https://github.com/metabase/metabase/actions/workflows/loki-stress-test-flake-fix.yml)

View File

@@ -5,7 +5,7 @@ PENDING CI / PENDING REVIEW / READY _(choose one and update accordingly)_
- Reproduces #XXXXX
### How to test this manually?
- `bun run test-cypress`
- `yarn test-cypress`
- `relative/path/to/the/file`
> _(For still unfixed bug)_
- Replace `it.skip()` with `it.only()` to run the test in isolation

View File

@@ -1,5 +1,4 @@
name: await-port
description: Wait for localhost port to be ready
name: Wait for localhost port to be ready
inputs:
port:

View File

@@ -0,0 +1,116 @@
name: Build ee-extra
description: Build Metabase Enterprise Edition with extra features
inputs:
java-version:
required: true
default: 21
iam-role:
description: "The IAM role to assume"
required: true
runs:
using: composite
steps:
- name: Prepare Java
uses: actions/setup-java@v3
with:
java-version: ${{ inputs.java-version }}
distribution: 'temurin'
- name: Reveal its original version.properties
run: jar xf ./bin/docker/metabase.jar version.properties && cat version.properties
shell: bash
- name: get major version
id: major_version
run: echo "major_version=$(cat version.properties | grep 'tag=' | cut -d'.' -f2)" >> $GITHUB_OUTPUT
shell: bash
- name: get drivers registry
run: |
curl -OL https://raw.githubusercontent.com/metabase/metabase-registry/main/registry.yaml
shell: bash
- uses: actions/setup-node@v4
with:
node-version: lts/Jod # 22.x.x
- run: rm package.json && npm install yaml
shell: bash
- name: parse drivers registry
uses: actions/github-script@v7
id: drivers_registry
with:
script: | # js
const { parse } = require('yaml');
const fs = require('fs');
const registry = parse(fs.readFileSync('registry.yaml', 'utf8'));
const drivers = registry.drivers;
const majorVersion = '${{ steps.major_version.outputs.major_version }}';
const getDriverUrl = (driverName) => {
const driver = drivers.find(driver => driver.name === driverName);
if (!driver) {
throw new Error(`Driver ${driverName} not found in the registry`);
}
const driverUrl = driver.versions[majorVersion] ?? driver.versions.default;
if (!driverUrl) {
throw new Error(`Driver ${driverName} version ${majorVersion} not found in the registry`);
}
return driverUrl;
};
const extraDrivers = {
firebolt: getDriverUrl('firebolt'),
starburst: getDriverUrl('starburst'),
clickhouse: getDriverUrl('clickhouse'),
materialize: getDriverUrl('materialize'),
};
console.log(extraDrivers);
return extraDrivers;
- name: Download Firebolt driver
run: |
curl -OL ${{ fromJson(steps.drivers_registry.outputs.result).firebolt }}
working-directory: modules
shell: bash
- name: Download Starburst driver
run: |
curl -OL ${{ fromJson(steps.drivers_registry.outputs.result).starburst }}
shell: bash
working-directory: modules
- name: Download Clickhouse driver
run: |
curl -OL ${{ fromJson(steps.drivers_registry.outputs.result).clickhouse }}
shell: bash
working-directory: modules
- name: Download Materialize driver
run: |
curl -OL ${{ fromJson(steps.drivers_registry.outputs.result).materialize }}
shell: bash
working-directory: modules
- name: Bundle additional drivers into the Uberjar
run: jar uf bin/docker/metabase.jar modules/*.jar
shell: bash
- name: Launch uberjar
run: >-
java --add-opens java.base/java.nio=ALL-UNNAMED -jar ./bin/docker/metabase.jar &
shell: bash
- name: Wait for Metabase to start
run: while ! curl 'http://localhost:3000/api/health' | grep '{"status":"ok"}'; do sleep 1; done
shell: bash
- name: Kill the java process
run: pkill -f metabase.jar
shell: bash
- name: configure aws credentials
if: ${{ github.ref_name == 'master' }}
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ inputs.iam-role }}
role-session-name: GitHub_to_AWS_via_FederatedOIDC
aws-region: us-east-1
- name: Login to Amazon ECR
if: ${{ github.ref_name == 'master' }}
uses: aws-actions/amazon-ecr-login@v2

View File

@@ -1,36 +0,0 @@
name: Cleanup Python Runner
description: Cleanup python-runner services and upload logs as artifacts
inputs:
upload-logs:
description: 'Whether to upload logs (typically on failure)'
required: false
default: 'false'
logs-artifact-suffix:
description: 'The unique name of the .zip artifact containing the logs'
required: false
default: ''
runs:
using: composite
steps:
- name: Cleanup python-runner services
if: always()
run: |
echo "Stopping python-runner services..."
if [ -d "python-runner" ]; then
cd python-runner
make stop || echo "WARNING: Failed to stop python runner"
fi
shell: bash
- name: Upload python-runner logs
if: ${{inputs.upload-logs == 'true' && inputs.logs-artifact-suffix != ''}}
uses: actions/upload-artifact@v4
with:
name: python-runner-logs-${{inputs.logs-artifact-suffix}}
path: |
python-runner.out.log
python-runner.err.log
retention-days: 1
if-no-files-found: warn

View File

@@ -23,7 +23,7 @@ runs:
GITHUB_TOKEN: ${{ inputs.github-token }}
COMMIT: ${{ inputs.backport-commit }}
shell: bash
run: | # bash
run: |
git config --global user.email "metabase-bot@metabase.com"
git config --global user.name "Metabase bot"
@@ -35,8 +35,6 @@ runs:
git checkout -b ${BACKPORT_BRANCH}
git cherry-pick ${COMMIT} || true
CONFLICT_LIST=$(git ls-files -u)
CONFLICTS=$(git ls-files -u | wc -l)
if [ "$CONFLICTS" -gt 0 ]; then
git cherry-pick --abort
@@ -48,7 +46,7 @@ runs:
echo "git reset HEAD~1" > ./backport.sh
echo "rm ./backport.sh" >> ./backport.sh
echo "git cherry-pick ${COMMIT}" >> ./backport.sh
echo "echo 'Resolve conflicts and force push this branch.\n\nTo backport translations run: bin/i18n/merge-translations <release-branch>'" >> ./backport.sh
echo "echo 'Resolve conflicts and force push this branch'" >> ./backport.sh
chmod +x ./backport.sh
git add ./backport.sh
git commit -m "Add backport resolution script"
@@ -57,14 +55,7 @@ runs:
#${ORIGINAL_PULL_REQUEST_NUMBER}
> [!IMPORTANT]
> Manual conflict resolution is required.
> Checkout the branch and run \`./backport.sh\` script. Force push your changes after cherry-picking. Check the box below when done.
Conflicts:
\`\`\`shell
${CONFLICT_LIST}
\`\`\`
- [ ] Conflicts resolved
Checkout the branch and run \`./backport.sh\` script. Force push your changes after cherry-picking.
END
)
else
@@ -74,7 +65,7 @@ runs:
git push -u origin ${BACKPORT_BRANCH}
BACKPORT_PR_URL=$(gh pr create --base "${TARGET_BRANCH}" --head "${BACKPORT_BRANCH}" --label "was-backported" --assignee "${{ github.event.pull_request.user.login }}" --title "🤖 backported \"${ORIGINAL_TITLE}\"" --body "${PR_BODY}")
BACKPORT_PR_URL=$(gh pr create --base "${TARGET_BRANCH}" --head "${BACKPORT_BRANCH}" --label "was-backported" --assignee "${GITHUB_ACTOR}" --title "🤖 backported \"${ORIGINAL_TITLE}\"" --body "${PR_BODY}")
BACKPORT_PR_NUMBER=${BACKPORT_PR_URL##*/}
echo "backport_pr_number=$BACKPORT_PR_NUMBER" >> $GITHUB_OUTPUT

View File

@@ -1,15 +1,12 @@
name: Cross-branch migration test
inputs:
base-ref: ## Base branch PR came from
required: true
head-ref: ## Source branch of the pull request
base-ref:
required: true
runs:
using: "composite"
steps:
## First we deploy the base branch
- name: Check out base branch
uses: actions/checkout@v4
with:
@@ -17,6 +14,8 @@ runs:
- name: Prepare backend environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: cross-branch-migrations
- name: Run backend on base branch
run: |
@@ -40,11 +39,10 @@ runs:
run: kill $(cat metabase_pid)
shell: bash
## Then we deploy the PR branch to ensure it migrates up successfully
- name: Check out pull request branch
uses: actions/checkout@v4
with:
ref: ${{ inputs.head-ref }}
ref: ${{ github.head_ref }}
- name: Run backend on pull request branch
run: |
@@ -67,38 +65,3 @@ runs:
- name: Stop Metabase on pull request branch
run: kill $(cat metabase_pid)
shell: bash
## Testing that originally deploying this PR allows you to roll back to a previous release
## Reset the checksums so when we go back to the base branch, it's as if the PR branch was what was originally deployed
- name: Reset the checksums as if the PR branch was what was originally deployed
run: |
clojure -M:dev:ee:migrate reset-checksums
shell: bash
- name: Rollback the last deployment
run: |
clojure -M:dev:ee:migrate rollback last-deployment
shell: bash
- name: Re-check out base branch
uses: actions/checkout@v4
with:
ref: ${{ inputs.base-ref }}
- name: Re-run backend on base branch
run: |
clojure -M:run:ee > metabase.log 2>&1 & echo $! > metabase_pid
tail -F metabase.log &
shell: bash
- name: Wait for Metabase to start
run: |
while ! curl -s 'http://localhost:3000/api/health' | grep '{"status":"ok"}' > /dev/null; do
if ! kill -0 $(cat metabase_pid) 2>/dev/null; then
cat metabase.log
echo "Metabase process has exited."
exit 1
fi
sleep 1
done
shell: bash

View File

@@ -8,7 +8,7 @@ inputs:
description: DockerHub password/token
required: true
maildev:
description: Maildev containers
description: Maildev container
required: true
default: false
openldap:
@@ -31,10 +31,6 @@ inputs:
description: Webhook tester container
required: false
default: false
snowplow:
description: Snowplow micro container
required: false
default: false
runs:
using: "composite"
@@ -46,68 +42,56 @@ runs:
password: ${{ inputs.password }}
- name: Start Containers
shell: bash
run: | # bash
run: |
Y='\033[33m'
G='\033[1;32m'
cd e2e/test/scenarios/
docker compose up \
$([ ${{ inputs.postgres }} = true ] && echo "postgres-sample") \
$([ ${{ inputs.mysql }} = true ] && echo "mysql-sample") \
$([ ${{ inputs.mongo }} = true ] && echo "mongo-sample") \
$([ ${{ inputs.maildev }} = true ] && echo "maildev") \
$([ ${{ inputs.maildev }} = true ] && echo "maildev-ssl") \
$([ ${{ inputs.openldap }} = true ] && echo "ldap") \
$([ ${{ inputs.webhook }} = true ] && echo "webhook-tester") \
$([ ${{ inputs.snowplow }} = true ] && echo "snowplow-micro") \
-d
if ${{ inputs.maildev }}; then
echo -e "${Y}checking maildev container..." &&
echo -e "${Y}Starting maildev container..." &&
docker run -d -p 1080:1080 -p 1025:1025 maildev/maildev:2.0.5 &&
while ! nc -z localhost 1080; do sleep 1; done &&
while ! nc -z localhost 1025; do sleep 1; done &&
echo -e "${G}Maildev is up and running!"
echo -e "${Y}checking maildev-ssl container..." &&
while ! nc -z localhost 1081; do sleep 1; done &&
while ! nc -z localhost 465; do sleep 1; done &&
echo -e "${G}Maildev-ssl is up and running!"
fi
if ${{ inputs.openldap }}; then
echo -e "${Y}checking openldap container..." &&
echo -e "${Y}Starting openldap container..." &&
docker run -d -p 389:389 \
--env LDAP_ADMIN_PASSWORD=adminpass \
--env LDAP_USERS=user01@example.org,user02@example.org \
--env LDAP_PASSWORDS=123456,123465 \
--env LDAP_ROOT=dc=example,dc=org \
--env LDAP_PORT_NUMBER=389 \
bitnami/openldap:2.6.4 &&
while ! nc -z localhost 389; do sleep 1; done &&
echo -e "${G}Openldap is up and running!"
fi
if ${{ inputs.postgres }}; then
echo -e "${Y}checking postgres container..." &&
echo -e "${Y}Starting postgres container..." &&
docker run -d -p 5404:5432 -e TZ='US/Pacific' metabase/qa-databases:postgres-sample-12 &&
while ! nc -z localhost 5404; do sleep 1; done &&
echo -e "${G}Postgres is up and running!"
fi
if ${{ inputs.mysql }}; then
echo -e "${Y}checking mysql container..." &&
echo -e "${Y}Starting mysql container..." &&
docker run -d -p 3304:3306 -e TZ='US/Pacific' metabase/qa-databases:mysql-sample-8 &&
while ! nc -z localhost 3304; do sleep 1; done &&
echo -e "${G}MySQL is up and running!"
fi
if ${{ inputs.mongo }}; then
echo -e "${Y}checking mongo container..." &&
echo -e "${Y}Starting mongo container..." &&
docker run -d -p 27004:27017 metabase/qa-databases:mongo-sample-4.4 &&
while ! nc -z localhost 27004; do sleep 1; done &&
echo -e "${G}Mongo is up and running!"
fi
if ${{ inputs.webhook }}; then
echo -e "${Y}checking webhook test container..." &&
echo -e "${Y}Starting webhook test container..." &&
docker run -d -p 9080:8080/tcp tarampampam/webhook-tester:1.1.0 serve --create-session 00000000-0000-0000-0000-000000000000 &&
while ! nc -z localhost 9080; do sleep 1; done &&
echo -e "${G}Webhook tester is up and running!"
fi
if ${{ inputs.snowplow }}; then
echo -e "${Y}checking snowplow micro container..." &&
while ! nc -z localhost 9090; do sleep 1; done &&
echo -e "${G}Snowplow micro is up and running!"
fi
shell: bash

View File

@@ -2,24 +2,9 @@ name: Fetch artifact
description: Fetches a specific artifact and downloads it to the current directory
inputs:
commit:
description: Commit SHA for the artifact
name:
description: The name of the artifact to fetch
required: true
edition:
description: Metabase edition (ee or oss)
required: true
release:
description: Whether this is a release artifact (uses metabase-release- prefix)
required: false
default: "false"
output-name:
description: Optional custom name for the output jar file (defaults to metabase.jar)
required: false
default: "metabase.jar"
extract-path:
description: Directory path where the jar should be extracted (defaults to current directory)
required: false
default: "."
runs:
using: "composite"
@@ -32,100 +17,47 @@ runs:
script: | # js
const fs = require('fs');
const isRelease = "${{ inputs.release }}" === "true";
const prefix = isRelease ? "metabase-release" : "metabase";
const artifactName = `${prefix}-${{ inputs.edition }}-${{ inputs.commit }}-uberjar`;
console.log(`Looking for artifact: ${artifactName}`);
const artifacts = await github.rest.actions.listArtifactsForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
name: artifactName,
name: "${{ inputs.name }}",
per_page: 1,
});
if (!artifacts.data?.artifacts?.[0]?.id) {
core.setFailed(`No artifacts found for ${artifactName}`);
return;
throw new Error(`No artifacts found for ${{ inputs.commit }}`);
}
const artifact_id = artifacts.data.artifacts[0].id;
let download;
const maxAttempts = 3;
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
try {
download = await github.rest.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: artifact_id,
archive_format: 'zip',
});
break;
} catch (err) {
console.log(`Attempt ${attempt}/${maxAttempts} failed: ${err.message}`);
if (attempt === maxAttempts) throw err;
const delay = attempt * 15000; // 15s, 30s
console.log(`Retrying in ${delay / 1000}s...`);
await new Promise(r => setTimeout(r, delay));
}
}
const download = await github.rest.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: artifact_id,
archive_format: 'zip',
});
fs.writeFileSync(`${process.env.GITHUB_WORKSPACE}/mb.zip`, Buffer.from(download.data));
- name: Extract and rename JAR
- name: unzip uberjar artifact
shell: bash
run: unzip mb.zip
- name: Move the Uberjar
shell: bash
run: | # bash
OUTPUT_NAME="${{ inputs.output-name }}"
EXTRACT_PATH="${{ inputs.extract-path }}"
# Create a temporary directory for extraction to avoid conflicts
TEMP_DIR=$(mktemp -d)
# Extract to temporary directory
unzip mb.zip -d "$TEMP_DIR"
# Create target directory
mkdir -p "$EXTRACT_PATH"
# Find the jar and move all files from its directory
if [ -e "$TEMP_DIR/metabase.jar" ]; then
# Move all files to extract path
mv "$TEMP_DIR"/* "$EXTRACT_PATH/" 2>/dev/null || true
# Rename the jar to custom name if different
if [ "$OUTPUT_NAME" != "metabase.jar" ]; then
mv "$EXTRACT_PATH/metabase.jar" "$EXTRACT_PATH/$OUTPUT_NAME"
fi
elif [ -e "$TEMP_DIR/target/uberjar/metabase.jar" ]; then
# Move all files from uberjar directory to extract path
mv "$TEMP_DIR/target/uberjar"/* "$EXTRACT_PATH/" 2>/dev/null || true
# Rename the jar to custom name if different
if [ "$OUTPUT_NAME" != "metabase.jar" ]; then
mv "$EXTRACT_PATH/metabase.jar" "$EXTRACT_PATH/$OUTPUT_NAME"
fi
if [ -e "metabase.jar" ]; then
echo "metabase.jar is already in the right place"
elif [ -e "target/uberjar/metabase.jar" ]; then
echo "moving metabase.jar to current directory"
mv target/uberjar/metabase.jar metabase.jar
else
echo "Could not find metabase.jar in extracted files"
echo "Could not find the metabase.jar file"
exit 1
fi
# Clean up temporary directory
rm -rf "$TEMP_DIR"
# Set the JAR_PATH environment variable
if [ "$EXTRACT_PATH" == "." ]; then
echo "JAR_PATH=$OUTPUT_NAME" >> $GITHUB_ENV
else
echo "JAR_PATH=$EXTRACT_PATH/$OUTPUT_NAME" >> $GITHUB_ENV
fi
- name: Verify that this is a valid JAR file
shell: bash
run: |
echo "Verifying JAR at: $JAR_PATH"
file --mime-type "$JAR_PATH"
run: file --mime-type ./metabase.jar
- name: Reveal its version.properties
shell: bash
run: |
echo "Extracting version.properties from: $JAR_PATH"
jar xf "$JAR_PATH" version.properties && cat version.properties
run: jar xf metabase.jar version.properties && cat version.properties

View File

@@ -1,45 +0,0 @@
name: Get cache keys
description: |
Get deterministic cache keys for Maven and Node.js caches based on the centrally controlled cache prefix,
and the suffix based on the runner OS and the current week of the year.
outputs:
m2-cache-key:
description: "The cache key for the Maven cache."
value: ${{ steps.keys-factory.outputs.m2-cache-key }}
m2-restore-key:
description: "The fallback restore key prefix for the Maven cache."
value: ${{ steps.keys-factory.outputs.m2-restore-key }}
node-cache-key:
description: "The cache key for the Node.js cache."
value: ${{ steps.keys-factory.outputs.node-cache-key }}
node-restore-key:
description: "The fallback restore key prefix for the Node.js cache."
value: ${{ steps.keys-factory.outputs.node-restore-key }}
eslint-cache-key:
description: "The cache key for Eslint cache"
value: ${{ steps.keys-factory.outputs.eslint-cache-key }}
cypress-cache-key:
description: "The cache key for Cypress cache"
value: ${{ steps.keys-factory.outputs.cypress-cache-key }}
runs:
using: "composite"
steps:
- name: Generate and output cache keys
id: keys-factory
run: |
M2_CACHE_PREFIX='M2'
NODE_CACHE_PREFIX='node-modules'
ESLINT_CACHE_PREFIX='eslint'
CYPRESS_CACHE_PREFIX='cypress'
WEEK_OF_YEAR=$(/bin/date "+%G-%V")
CACHE_SUFFIX="${{ runner.os }}-$WEEK_OF_YEAR"
echo "m2-cache-key=$M2_CACHE_PREFIX-$CACHE_SUFFIX" >> $GITHUB_OUTPUT
echo "m2-restore-key=$M2_CACHE_PREFIX-" >> $GITHUB_OUTPUT
echo "node-cache-key=$NODE_CACHE_PREFIX-$CACHE_SUFFIX" >> $GITHUB_OUTPUT
echo "node-restore-key=$NODE_CACHE_PREFIX-" >> $GITHUB_OUTPUT
echo "eslint-cache-key=$ESLINT_CACHE_PREFIX-$CACHE_SUFFIX" >> $GITHUB_OUTPUT
echo "cypress-cache-key=$CYPRESS_CACHE_PREFIX-$CACHE_SUFFIX" >> $GITHUB_OUTPUT
shell: bash

View File

@@ -1,74 +0,0 @@
name: Get Docker Info
description: Determines Docker repository and tag based on branch and edition
inputs:
branch:
description: Branch name (defaults to current branch)
required: false
default: ${{ github.head_ref || github.ref_name }}
edition:
description: Edition (ee or oss)
required: false
default: ee
owner:
description: Repository owner (defaults to current repo owner)
required: false
default: ${{ github.repository_owner }}
outputs:
repo:
description: Docker repository name
value: ${{ steps.docker-info.outputs.repo }}
tag:
description: Docker tag
value: ${{ steps.docker-info.outputs.tag }}
full_image:
description: Full Docker image name (repo:tag)
value: ${{ steps.docker-info.outputs.full_image }}
editions:
description: JSON array of editions to build
value: ${{ steps.docker-info.outputs.editions }}
runs:
using: "composite"
steps:
- name: Determine Docker info
id: docker-info
uses: actions/github-script@v7
with:
script: | # js
const branch = '${{ inputs.branch }}';
const edition = '${{ inputs.edition }}';
const owner = '${{ inputs.owner }}';
console.log(`Determining Docker info for branch: ${branch}, edition: ${edition}`);
let repo, tag, editions;
if (branch === 'master') {
// Master builds both editions with specific repos
editions = ['ee', 'oss'];
tag = 'latest';
if (edition === 'ee') {
repo = `${owner}/metabase-enterprise-head`;
} else {
repo = `${owner}/metabase-head`;
}
} else {
// All other branches only build EE to metabase-dev
editions = ['ee'];
tag = branch.replaceAll('/', '-'); // slashes make docker sad
repo = `${owner}/metabase-dev`;
}
const fullImage = `${repo}:${tag}`;
// Set outputs
core.setOutput('repo', repo);
core.setOutput('tag', tag);
core.setOutput('full_image', fullImage);
core.setOutput('editions', JSON.stringify(editions));
console.log(`Docker image: ${fullImage}`);
console.log(`Editions: ${JSON.stringify(editions)}`);

View File

@@ -1,19 +0,0 @@
name: Maximize disk space
description: Removes unused stuff from GH runner
runs:
using: "composite"
steps:
- name: Maximize build space
shell: bash
run: |
echo "-- Dotnet"
sudo rm -rf /usr/share/dotnet
echo "-- Android"
sudo rm -rf /usr/local/lib/android
echo "-- Haskell"
sudo rm -rf /opt/ghc
echo "-- CodeQL"
sudo rm -rf /opt/hostedtoolcache/CodeQL
df -h

View File

@@ -1,13 +1,5 @@
name: Notify Pull Request
inputs:
remove-previous-comment:
required: false
type: boolean
default: false
add-new-comment:
required: false
type: boolean
default: true
message:
required: true
include-log:
@@ -17,44 +9,7 @@ inputs:
runs:
using: "composite"
steps:
- name: Remove previous pull-request comment
uses: actions/github-script@v7
if: ${{ inputs.remove-previous-comment == 'true' && always() && github.event_name == 'pull_request' }}
with:
script: |
const pr = context.payload.pull_request ?? context.payload.issue;
if (!pr) {
console.error("This action can only be run on pull requests");
return;
}
const { owner, repo } = context.repo;
const { data: comments } = await github.rest.issues.listComments({
owner,
repo,
issue_number: pr.number,
}).catch(console.error);
if (!comments) {
return;
}
const message = "${{ inputs.message }}";
const comment = comments.find((comment) => comment.body.includes(message));
if (comment) {
await github.rest.issues.deleteComment({
owner,
repo,
comment_id: comment.id,
});
}
- name: Notify pull-request
uses: actions/github-script@v7
if: ${{ inputs.add-new-comment == 'true' && always() && github.event_name == 'pull_request' }}
- uses: actions/github-script@v7
id: notify-pull-request
with:
result-encoding: string
@@ -66,7 +21,7 @@ runs:
return;
}
const message = "${{ inputs.message }}";
const message = "${{ inputs.message }}"
const shouldIncludeLog = ${{ inputs.include-log }}.toString() === "true";
const author = context.payload.sender.login;

View File

@@ -1,13 +1,15 @@
name: Prepare back-end environment
description: |
Prepare the backend. Installs jdk, Clojure, and fetches the cache for m2.
inputs:
java-version:
required: true
default: "21"
default: '21'
clojure-version:
required: true
default: "1.12.0.1488"
default: '1.12.0.1488'
m2-cache-key:
description: 'Key to cache M2 packages from Maven Central'
required: true
default: 'm2'
runs:
using: "composite"
@@ -16,39 +18,23 @@ runs:
uses: actions/setup-java@v4
with:
java-version: ${{ inputs.java-version }}
distribution: "temurin"
distribution: 'temurin'
- name: Install Clojure CLI
shell: bash
run: |
for attempt in 1 2 3; do
curl --retry 3 --retry-delay 5 --fail -O https://download.clojure.org/install/linux-install-${{ inputs.clojure-version }}.sh \
&& sudo bash ./linux-install-${{ inputs.clojure-version }}.sh \
&& break
echo "::warning::Clojure install attempt $attempt failed, retrying in 10s..."
sleep 10
done
if ! clojure --version; then
echo "::error::Clojure CLI installation failed after 3 attempts"
exit 1
fi
- name: Get M2 cache key
uses: ./.github/actions/get-cache-keys
id: get-cache-keys
- name: Install uv (for Python package installation)
uses: astral-sh/setup-uv@v7
with:
enable-cache: false
version: "0.10.2" # keep in sync with mise.toml
github-token: ${{ github.token }}
- name: Restore M2 cache
uses: actions/cache/restore@v4
curl -O https://download.clojure.org/install/linux-install-${{ inputs.clojure-version }}.sh &&
sudo bash ./linux-install-${{ inputs.clojure-version }}.sh
- name: Check to see if dependencies should be cached
if: ${{ contains(github.event.head_commit.message, '[ci nocache]') }}
run: echo "Commit message includes [ci nocache]; dependencies will NOT be cached"
shell: bash
- name: Get M2 cache
uses: actions/cache@v4
if: ${{ !contains(github.event.head_commit.message, '[ci nocache]') }}
with:
path: |
~/.m2
~/.gitlibs
key: ${{ steps.get-cache-keys.outputs.m2-cache-key }}
key: ${{ runner.os }}-${{ inputs.m2-cache-key }}-${{ hashFiles('**/deps.edn') }}
restore-keys: |
${{ steps.get-cache-keys.outputs.m2-restore-key }}
${{ runner.os }}-${{ inputs.m2-cache-key }}-

View File

@@ -1,12 +1,6 @@
name: Prepare cypress environment
description: Cypress preparation steps
inputs:
working-directory:
description: "Optional working directory for bun commands"
required: false
default: "."
outputs:
chrome-path:
description: Path to the custom Chrome executable
@@ -15,27 +9,27 @@ outputs:
runs:
using: "composite"
steps:
- name: Install Chrome v144
uses: browser-actions/setup-chrome@v2
- name: Install Chrome v111
uses: browser-actions/setup-chrome@v1
with:
chrome-version: 144
# https://chromium.cypress.io/linux/stable/111.0.5563.146
chrome-version: 1097615
id: setup-chrome
- name: Get cache keys
uses: ./.github/actions/get-cache-keys
id: get-cache-keys
- name: Check to see if dependencies should be cached
if: ${{ contains(github.event.head_commit.message, '[ci nocache]') }}
run: echo "Commit message includes [ci nocache]; dependencies will NOT be cached"
shell: bash
- name: Get Cypress cache
uses: actions/cache/restore@v4
uses: actions/cache@v4
if: ${{ !contains(github.event.head_commit.message, '[ci nocache]') }}
with:
path: ~/.cache/Cypress
key: ${{ steps.get-cache-keys.outputs.cypress-cache-key }}
key: ${{ runner.os }}-Cypress-${{ hashFiles('**/yarn.lock') }}
- name: Ensure that Cypress executable is ready
run: |
bun run cypress install
bun run cypress cache path
bun run cypress cache list
bun run cypress verify
yarn cypress install
yarn cypress cache path
yarn cypress cache list
yarn cypress verify
shell: bash
working-directory: ${{ inputs.working-directory }}

View File

@@ -1,61 +1,37 @@
name: Prepare front-end environment
description: "Prepare the front-end environment for the CI build"
inputs:
node-version:
description: "The version of Node.js to use, overriding .nvmrc"
required: false
apply-patches:
description: "Whether to patch frontend dependencies with patch-package."
required: true
default: "true"
runs:
using: "composite"
steps:
- name: Prepare Node + Bun environment
uses: ./.github/actions/prepare-node-bun
- name: Prepare Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ inputs.node-version }}
node-version: ${{ inputs.node-version }} # this overrides, if present
node-version-file: ".nvmrc"
- name: Get cache keys
uses: ./.github/actions/get-cache-keys
id: get-cache-keys
- name: Restore M2 cache
uses: actions/cache/restore@v4
with:
path: |
~/.m2
~/.gitlibs
key: ${{ steps.get-cache-keys.outputs.m2-cache-key }}
restore-keys: |
${{ steps.get-cache-keys.outputs.m2-restore-key }}
- name: Get bun cache directory
id: bun-cache
- name: Check to see if dependencies should be cached
if: ${{ contains(github.event.head_commit.message, '[ci nocache]') }}
run: |
BUN_CACHE_DIR=$(bun pm cache)
echo "Bun cache directory: $BUN_CACHE_DIR"
echo "dir=$BUN_CACHE_DIR" >> $GITHUB_OUTPUT
echo "Commit message includes [ci nocache]; dependencies will NOT be cached"
yarn cache clean
shell: bash
- name: Restore node_modules cache
uses: actions/cache/restore@v4
id: node-modules-cache
- name: Get M2 cache
uses: actions/cache@v4
if: ${{ !contains(github.event.head_commit.message, '[ci nocache]') }}
with:
path: |
${{ steps.bun-cache.outputs.dir }}
node_modules
key: ${{ steps.get-cache-keys.outputs.node-cache-key }}
restore-keys: |
${{ steps.get-cache-keys.outputs.node-restore-key }}
path: ~/.m2
key: ${{ runner.os }}-cljs-${{ hashFiles('**/shadow-cljs.edn') }}
# postinstall lifecycle hook can be dangerous in CI so we only run it locally (see package.json)
# If a job wants to patch packages, it does so explicitly in the CI. This setting is opt-out.
# For example, we never want to cache node_modules with patches already applied so cache-generator.yml skips that step.
- name: Clean install
run: bun install --frozen-lockfile
shell: bash
- name: Patch frontend dependencies
if: inputs.apply-patches == 'true'
run: bun run patch-package
- name: Get node_modules cache
uses: actions/cache@v4
id: node-modules-cache
if: ${{ !contains(github.event.head_commit.message, '[ci nocache]') }}
with:
path: node_modules
key: ${{ runner.os }}-node-modules-${{ hashFiles('**/yarn.lock', '**/patches/*.patch') }}
- run: yarn install --frozen-lockfile --prefer-offline
shell: bash

Some files were not shown because too many files have changed in this diff Show More