Compare commits

..

354 Commits

Author SHA1 Message Date
Thomas Schmidt
349f48ce44 Basic sql generation draft 2025-03-26 17:03:56 +01:00
Thomas Schmidt
b08b714b0e Do not convert tool names (#55505) 2025-03-24 07:39:43 +01:00
Tamás Benkő
523cd3812b Always set :queryable-foreign-key-tables for models 2025-03-20 22:45:46 +03:00
Tamás Benkő
d67c024717 Remove unused schema definition 2025-03-20 22:45:09 +03:00
Tamás Benkő
e52be87e3a Convert history sent by the FE 2025-03-20 22:44:39 +03:00
metamben
4f83b5eff3 Merge branch 'master' into metabot-v3-main 2025-03-20 21:42:49 +03:00
metamben
02ef883215 Expose table references for metrics (#55456)
Fixes BOT-66
2025-03-20 21:39:37 +03:00
metamben
a654c4025b Expose field values for metrics, models, and tables (#55413)
Fixes BOT-69
2025-03-20 21:32:33 +03:00
metamben
8c236418de Normalize metadata IDs (#55383)
Fixes BOT-68
2025-03-20 21:26:20 +03:00
metamben
ffb7af68c8 Keep history including the initial user message (#55473) 2025-03-20 21:25:25 +03:00
metamben
5333eeda6b Merge branch 'master' into metabot-v3-main 2025-03-19 20:53:04 +03:00
metamben
9981e3db93 Merge branch 'master' into metabot-v3-main 2025-03-18 22:39:39 +03:00
Tamás Benkő
93ab435a9d Merge remote-tracking branch 'origin/master' into metabot-v3-main
# Conflicts:
#	.github/workflows/uberjar.yml
#	.loki/reference/chrome_laptop_Design_System_Icons_List.png
#	frontend/src/metabase/query_builder/components/view/View/ViewNativeQueryEditor/ViewNativeQueryEditor.jsx
2025-03-18 22:38:59 +03:00
metamben
805f066b8c Add answer source tool (#55172) 2025-03-18 12:36:57 +03:00
Tamás Benkő
8af934fecd fix: Allow strings as tool call arguments 2025-03-17 15:07:51 +03:00
metamben
79a24d5800 Merge branch 'master' into metabot-v3-main 2025-03-17 14:40:56 +03:00
Tamás Benkő
ccc58d6578 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-03-16 18:00:15 +03:00
Tamás Benkő
c06e62576b Merge remote-tracking branch 'origin/master' into metabot-v3-main
# Conflicts:
#	.loki/reference/chrome_laptop_Design_System_Icons_List.png
2025-03-14 10:59:57 +03:00
metamben
2aec76218d Add query model tool (#55089)
Fixes BOT-61
2025-03-14 10:46:52 +03:00
metamben
c14d51b22e Add support for temporal extraction filters (#55037)
* Add support for temporal extraction filters

Fixes BOT-60.

Actually, both extraction and bucketing is supported. The following values can be provided in the filter's `bucket`
property:

```
millisecond
second
minute
hour
day
week
month
quarter
year
second-of-minute
minute-of-hour
hour-of-day
day-of-week
day-of-month
day-of-year
week-of-year
month-of-year
quarter-of-year
year-of-era
```

The values of the form x-of-y result in extraction, the other result in truncation. The extractions `week-of-year` and
`day-of-week` have ISO semantics.

* Placate kondo
2025-03-13 20:41:53 +03:00
metamben
ef69ec3129 Merge branch 'master' into metabot-v3-main 2025-03-12 14:52:36 +03:00
Tamás Benkő
4fee5cd3c4 Update expected native query result text 2025-03-11 23:48:06 +03:00
Tamás Benkő
644f935654 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-03-11 23:11:48 +03:00
Tamás Benkő
2e6734d105 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-03-11 22:55:57 +03:00
metamben
1db581dd1b Merge branch 'master' into metabot-v3-main 2025-03-10 15:18:55 +03:00
metamben
71d913d8d1 Merge branch 'master' into metabot-v3-main 2025-03-09 14:11:37 +03:00
metamben
51ac49727e Merge branch 'master' into metabot-v3-main 2025-03-08 01:56:24 +03:00
metamben
806d623462 Remove dummy tools and Metabot loop related stuff (#54812)
* Remove dummy tools and clean up
2025-03-08 01:56:13 +03:00
metamben
8f62da32cc Merge branch 'master' into metabot-v3-main 2025-03-07 13:49:24 +03:00
metamben
edd7bc6a9c Distinguish temporal types and add semantic type (#54741)
Fixes BOT-56.
2025-03-07 13:49:10 +03:00
metamben
8f62fb33a5 Merge branch 'master' into metabot-v3-main 2025-03-06 17:55:00 +03:00
metamben
15f333d81f Add type agnostic and disjunctive filters (#54731)
Fixes BOT-54.
2025-03-06 17:54:51 +03:00
metamben
2543a6af72 Merge branch 'master' into metabot-v3-main 2025-03-06 15:01:23 +03:00
Tamás Benkő
961a73ff4c Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-03-05 21:36:59 +03:00
Tamás Benkő
383591f75b Update icon list in Loki 2025-03-05 21:36:19 +03:00
Tamás Benkő
2e1c2d7205 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-03-05 17:12:37 +03:00
Tamás Benkő
57ca9f0aac Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-03-05 17:10:36 +03:00
metamben
4c1fe3d767 Merge branch 'master' into metabot-v3-main 2025-03-04 14:27:27 +03:00
Tamás Benkő
8c758e3674 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-03-03 23:56:37 +03:00
Tamás Benkő
2bf18a43ec Send "user_is_viewing" to the agent in the context 2025-03-03 23:55:44 +03:00
metamben
4846e8a94b Merge branch 'master' into metabot-v3-main 2025-03-03 19:44:48 +03:00
metamben
a62e4e10ef Merge branch 'master' into metabot-v3-main 2025-03-03 18:31:54 +03:00
metamben
9778f8c12b Convert dummy tools to normal tools (#54519)
Fixes BOT-30
Fixes BOT-31
Fixes BOT-32
Fixes BOT-33
Fixes BOT-34
Fixes BOT-35
2025-03-03 18:31:43 +03:00
metamben
486d792c9a Remove V1 metabot agent endpoint (#54380)
* Remove V1 metabot agent endpoint

Fixes BOT-36
2025-03-03 17:18:23 +03:00
Tamás Benkő
4975f725a9 Merge remote-tracking branch 'origin/metabot-bot-47-tool-api-endpoint-tests' into metabot-v3-main 2025-03-03 17:09:45 +03:00
Tamás Benkő
b5e38e84b5 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-03-03 17:06:48 +03:00
Tamás Benkő
ce62f30ffc Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-03-03 17:06:16 +03:00
Tamás Benkő
8374ed8450 Fix namespace 2025-02-28 23:37:38 +03:00
metamben
e2660bf3ef Merge branch 'master' into metabot-v3-main 2025-02-28 23:28:36 +03:00
Alexander Polyankin
08ebf5a3e3 Merge remote-tracking branch 'origin/metabot-v3-main' into metabot-v3-main 2025-02-28 15:20:15 -05:00
Alexander Polyankin
485df761d0 Use CSS colors 2025-02-28 15:20:06 -05:00
Alexander Polyankin
f2f4f57866 Use CSS colors 2025-02-28 15:19:45 -05:00
Alexander Polyankin
3abeeb9ab2 Use CSS colors 2025-02-28 15:15:56 -05:00
metamben
cb22a6bcf4 Merge branch 'master' into metabot-v3-main 2025-02-28 23:11:50 +03:00
Tamás Benkő
ad34b73d94 Placate clj-kondo 2025-02-28 22:49:37 +03:00
metamben
351daa8851 Merge branch 'master' into metabot-v3-main 2025-02-28 16:46:47 +03:00
Tamás Benkő
61b153a918 Run tests with the metabot-v3 feature enabled 2025-02-27 18:32:27 +03:00
Tamás Benkő
d1b0bf48fd Add V2 tool API endpoint tests
Fixes BOT-47.
2025-02-27 18:05:43 +03:00
metamben
1671d222cd Merge branch 'master' into metabot-v3-main 2025-02-27 17:09:46 +03:00
metamben
c0ef616c6d Merge branch 'master' into metabot-v3-main 2025-02-26 16:27:15 +03:00
Ngoc Khuat
78743ba645 fix whitespace 🤦 2025-02-26 15:32:03 +07:00
Ngoc Khuat
01e98249a2 fix invalid reducers 2025-02-26 15:13:46 +07:00
Ngoc Khuat
1dabc6ccbb Merge branch 'master' into metabot-v3-main 2025-02-26 14:55:59 +07:00
metamben
752287a56d Merge branch 'master' into metabot-v3-main 2025-02-25 20:23:55 +03:00
metamben
865d8a0d81 Generate site-wide session tokens for the AI service (#54269)
* Generate site-wide session tokens for the AI service

Fixes BOT-38.

This PR also allows the V2 tool API endpoints to be used by other existing authentication methods like API keys and
Metabase sessions.
2025-02-25 20:23:42 +03:00
metamben
26dc17b31d Merge branch 'master' into metabot-v3-main 2025-02-25 16:53:05 +03:00
metamben
2f6a39b40e Merge branch 'master' into metabot-v3-main 2025-02-24 22:10:14 +03:00
Tamás Benkő
dfab957510 Fix tests 2025-02-24 21:15:30 +03:00
metamben
ea3b49aa0a Merge branch 'master' into metabot-v3-main 2025-02-24 18:48:30 +03:00
metamben
5d81a60a64 Introduce V2 tool endpoints (#54192) 2025-02-24 18:48:17 +03:00
metamben
10bd22a3b9 Merge branch 'master' into metabot-v3-main 2025-02-21 16:45:24 +03:00
Ngoc Khuat
1e75309a10 Merge branch 'master' into metabot-v3-main 2025-02-21 11:27:27 +07:00
metamben
68d24b95b4 Merge branch 'master' into metabot-v3-main 2025-02-20 18:46:51 +03:00
metamben
0574137a29 Merge branch 'master' into metabot-v3-main 2025-02-19 17:48:32 +03:00
metamben
455713606e Merge branch 'master' into metabot-v3-main 2025-02-18 17:44:02 +03:00
metamben
fcf86ea731 Skip NOT IN condition if there are no used-ids (#53845) 2025-02-18 17:43:46 +03:00
Mihael Konjević
168c055c8f Keep Metabot palette item stable between query changes (#53827) 2025-02-18 12:41:24 +01:00
Tamás Benkő
1e715f418b Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-02-17 18:46:06 +03:00
Tamás Benkő
5aca4ec11d Fix test 2025-02-17 18:45:41 +03:00
Tamás Benkő
27045cf9a7 Remove .clj-kondo/src/hooks/metabase/models/setting.clj changes 2025-02-17 17:06:36 +03:00
Tamás Benkő
d358286383 Merge remote-tracking branch 'origin/master' into metabot-v3-main
# Conflicts:
#	.clj-kondo/src/hooks/metabase/models/setting.clj
2025-02-17 17:03:06 +03:00
metamben
c19b8310dd Fix module dependencies (#53763)
* Fix module dependencies
2025-02-17 16:03:54 +03:00
metamben
3b535a542c Add find-outliers tool endpoint (#53624)
* Add find-outliers tool endpoint
* Also fix the decoding of find-metric tool API results.
* Add tests for explicit query handling.
* Add context when running the query for finding outliers
2025-02-14 17:00:24 +03:00
metamben
291d9e12e1 Merge branch 'master' into metabot-v3-main 2025-02-14 16:56:41 +03:00
Thomas Schmidt
bd1da7cdb9 Include query in request to ai-service (#53745) 2025-02-14 14:48:37 +01:00
Tamás Benkő
96cd8ba5b7 Appease the module checker even more vigorously 2025-02-14 00:44:10 +03:00
Tamás Benkő
b5e2e66b27 Skip SQL fixer e2e test and mute the module linter for now 2025-02-14 00:17:44 +03:00
Tamás Benkő
5b8a479fe4 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-02-14 00:13:41 +03:00
Tamás Benkő
bf5df2cd38 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-02-13 20:10:58 +03:00
Tamás Benkő
98332749de Address linter issues 2025-02-13 20:09:29 +03:00
metamben
827c0c2eab Merge branch 'master' into metabot-v3-main 2025-02-13 18:21:44 +03:00
Tamás Benkő
e74c254612 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-02-13 18:20:33 +03:00
Tamás Benkő
04b7020aef Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-02-13 18:19:51 +03:00
Romeo Van Snick
edb6b83feb Fix loop in codemirror (#53599)
* Correctly compare selections

* Add reproduction for QUE-614

* Use github issue number in regression

* Unnest issue from other issue reproduction

* Log in before test
2025-02-13 15:16:42 +01:00
Thomas Schmidt
10550112f5 Only send path for xray dashboard not full link (#53594) 2025-02-13 12:34:55 +01:00
Mihael Konjevic
5c99127943 Merge branch 'master' into metabot-v3-main 2025-02-12 20:56:08 +01:00
Mihael Konjević
5b5ab1fa16 Metabot v3 command palette item (#53568)
* Initial implementation of Metabot palette action

* Use default styles, correct ordering of palette items, tests

* Setup tests with metabot_v3 feature enabled

* Recompute "mainReducers" on demand in the renderWithProviders function

This solves a race condition where reducers were computed before the
enterprise plugins were required, which made it impossible to test their
behavior (in this case in PaletteResult tests). makeMainReducers function is
added which computes the reducers object, and the previous behavior is
preserved (mainReducers const points to the output of the makeMainReducers fn)
2025-02-12 19:40:26 +01:00
metamben
76d904ba3a Merge branch 'master' into metabot-v3-main 2025-02-12 19:21:20 +03:00
metamben
ca3698711b SQL fixer: generate schema_ddl (#53381)
* Generate ddl_schema
2025-02-12 19:21:05 +03:00
metamben
271fab9a30 Add find-metric tool endpoint (#53492)
* Add find-metric tool endpoint

* Add missing result conversion functions
2025-02-12 19:20:01 +03:00
metamben
ff0ed9be47 Add filter-records tool endpoint (#53561)
* Add filter-records tool endpoint

* Adapt tests
2025-02-12 18:40:00 +03:00
Tamás Benkő
a69afcb929 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-02-12 17:59:55 +03:00
Tamás Benkő
c4b2f29b0f Remove unused imports 2025-02-11 20:02:45 +03:00
Tamás Benkő
44a15f3a77 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-02-11 19:52:14 +03:00
Tamás Benkő
6630e72403 Stop re-using metabase.api.card/get-card
This function is private now, and it's also doing more than what we need right now.
2025-02-11 19:51:35 +03:00
Tamás Benkő
318e3e6b6f Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-02-11 19:10:06 +03:00
Thomas Schmidt
91948e24ab Redirect user when navigate_to property is present in agent response (#53518) 2025-02-11 16:36:04 +01:00
Tamás Benkő
6c20dc9c17 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-02-11 17:54:38 +03:00
metamben
a6f8cc0919 Fix input encoding for the create-dashboard-subscription tool (#53486) 2025-02-10 23:45:17 +03:00
Alexander Polyankin
3ebda589f5 Merge remote-tracking branch 'refs/remotes/origin/master' into metabot-v3-main
# Conflicts:
#	frontend/src/metabase/query_builder/components/NativeQueryEditor/CodeMirrorEditor/extensions.tsx
2025-02-10 11:05:15 -05:00
Tamás Benkő
e51023ca43 Remove describeEE and onlyOnEE 2025-02-08 02:33:40 +03:00
metamben
44da1f4d67 Merge branch 'master' into metabot-v3-main 2025-02-07 23:24:46 +03:00
metamben
4a1c0c5029 Check table_id parameter in generate-insights tool (#53310) 2025-02-07 20:33:35 +03:00
metamben
f2792cdbfd Merge branch 'master' into metabot-v3-main 2025-02-07 18:47:20 +03:00
metamben
10c042dbdc Add create-dashboard-subscription and generate-insights endpoints (#53253)
* Add create-dashboard-subscription and generate-insights endpoints

* Rename session_id to conversation_id for the tool endpoints
2025-02-07 18:46:40 +03:00
metamben
7badf9005d Merge branch 'master' into metabot-v3-main 2025-02-07 16:39:14 +03:00
metamben
c3d157b9cd Merge branch 'master' into metabot-v3-main 2025-02-07 01:27:05 +03:00
Tamás Benkő
aeaa811281 Fix ns require formatting 2025-02-07 01:26:33 +03:00
metamben
d44226c7fa Merge branch 'master' into metabot-v3-main 2025-02-07 01:24:12 +03:00
Alexander Polyankin
c4578261a8 Fix SQL button tests (#53198) 2025-02-06 11:38:10 -05:00
metamben
ca0352d6ad Merge branch 'master' into metabot-v3-main 2025-02-06 17:03:41 +03:00
metamben
32dc706a39 Merge branch 'master' into metabot-v3-main 2025-02-05 17:40:15 +03:00
metamben
1b44cb8bb9 Implement query-metric tool endpoint (#53195) 2025-02-05 17:40:01 +03:00
Alexander Polyankin
9fa2f59d64 Autofix SQL button (#53153) 2025-02-04 11:32:49 -05:00
metamben
e62bfc2004 Merge branch 'master' into metabot-v3-main 2025-02-04 18:08:04 +03:00
metamben
95726e9d1b Add /api/ee/ai-fix-sql/fix endpoint (#53143)
* Add POST /api/ee/ai-sql-fixer/fix endpoint

* Never prettify the generated SQL
2025-02-04 18:07:06 +03:00
metamben
ec7595557a Merge branch 'master' into metabot-v3-main 2025-02-04 16:20:13 +03:00
metamben
da64908493 Merge branch 'master' into metabot-v3-main 2025-02-04 01:26:31 +03:00
metamben
9bd61f3557 Merge branch 'master' into metabot-v3-main 2025-02-03 21:26:55 +03:00
metamben
49a83e9f41 Merge branch 'master' into metabot-v3-main 2025-02-03 16:38:28 +03:00
metamben
bddc369c0d Merge branch 'master' into metabot-v3-main 2025-02-02 14:31:29 +03:00
metamben
baa6baf893 Merge branch 'master' into metabot-v3-main 2025-02-01 02:43:33 +03:00
metamben
c9fe612a50 Merge branch 'master' into metabot-v3-main 2025-01-31 18:13:39 +03:00
metamben
6480f7b841 Add standalone tests for the invite-user tool (#52984)
* Add standalone tests for the invite-user tool
2025-01-31 18:13:26 +03:00
Tamás Benkő
64fa2f2bc1 Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-01-31 18:07:36 +03:00
Tamás Benkő
4c690a44dc Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-01-31 18:07:13 +03:00
metamben
d74631e48b Merge branch 'master' into metabot-v3-main 2025-01-31 00:37:25 +03:00
metamben
c9f5a0e3b6 Merge branch 'master' into metabot-v3-main 2025-01-30 17:57:42 +03:00
metamben
123ec0e340 Add standalone filter-records tests (#52933)
* Add standalone filter-records tests
2025-01-30 17:57:33 +03:00
metamben
3929342ac0 Merge branch 'master' into metabot-v3-main 2025-01-30 00:39:04 +03:00
metamben
ed537dcbc7 Add standalone find-outliers tests (#52906)
* Add standalone find-outliers tests
2025-01-30 00:38:46 +03:00
Tamás Benkő
60b85b61ac Switch to api.macros/defendpoint 2025-01-29 18:23:57 +03:00
metamben
e3347d10a5 Merge branch 'master' into metabot-v3-main 2025-01-29 17:24:25 +03:00
Tamás Benkő
ca0cd2b5a5 Add back React import 2025-01-28 21:53:33 +03:00
metamben
10107fcf5c Merge branch 'master' into metabot-v3-main 2025-01-28 21:39:38 +03:00
Tamás Benkő
42aa717241 Add missing type keyword 2025-01-28 21:37:13 +03:00
Tamás Benkő
cb30ee8a4b Fix wrong merge 2025-01-28 21:30:14 +03:00
metamben
dee6da84cc Merge branch 'master' into metabot-v3-main 2025-01-28 21:16:32 +03:00
metamben
9d6db122ca Add tests for the query-metric tool (#52765)
* Ignore bucketing for non-temporal columns

This was implemented in #52271 but has been reverted by a bad merge.

* Use long names when generating columns

The standard names can easily result in duplicates which will certainly confuse the agent.

* Break long lines

* Add tests for the query-metric tool

* Handle the strange metric-id error cases explicitly
2025-01-28 21:16:14 +03:00
metamben
bdccec2d1f Add tests for the generate-insights tool (#52767) 2025-01-28 21:15:17 +03:00
metamben
653797622b Merge branch 'master' into metabot-v3-main 2025-01-28 16:55:33 +03:00
Tamás Benkő
77fb7c62fa Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-01-27 18:33:01 +03:00
Tamás Benkő
c1eb2559ec Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-01-27 18:32:33 +03:00
Aleksandr Lesnenko
8643fc3aef update import 2025-01-24 16:08:10 -05:00
metamben
67abded9e8 Add default_time_dimension_field_id and adhoc context handling (#52683)
* Add default_time_dimension_field_id and adhoc context handling

* Add show-results-to-user dummy call for adhoc queries in context
2025-01-24 23:33:49 +03:00
Tamás Benkő
dab935167b Fix formatting 2025-01-24 22:54:54 +03:00
metamben
087ad1a2b1 Merge branch 'master' into metabot-v3-main 2025-01-24 22:15:47 +03:00
metamben
c3cbeffc70 Merge branch 'master' into metabot-v3-main 2025-01-24 21:45:31 +03:00
Tamás Benkő
75b33df5bc Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-01-24 20:36:28 +03:00
metamben
b87ce054cb Switch the uberjar build branch to metabot-v3-main (#52678)
* Switch the build branch to metabot-v3-main

* Switch containerize_test_and_push_container too
2025-01-24 20:32:36 +03:00
metamben
89490c9a16 Merge branch 'master' into metabot-v3-main 2025-01-23 22:37:23 +03:00
metamben
9b78a0014f Return a friendly message when the number of round trips is exceeded (#52621)
Fixes #52591.
2025-01-23 22:36:57 +03:00
metamben
74c29161f3 Re-enable the filter-records tool (#52615)
It turns out that a fair number of our test cases rely on this tool, and they worked before, so it might be stable
enough.
2025-01-23 21:01:50 +03:00
metamben
59dcb7f115 Stop checking that parameter names are kebab-case (#52613) 2025-01-23 20:59:15 +03:00
metamben
2f817742e6 Consistently generate and look for query_id and not query-id (#52599) 2025-01-23 18:04:18 +03:00
metamben
be840bb7bd Prevent converting :structured-content to JSON twice (#52556) 2025-01-23 16:54:21 +03:00
metamben
086783dd68 Merge branch 'master' into metabot-v3-main 2025-01-23 16:42:51 +03:00
metamben
3cd107d6d3 Ignore result-columns for saved questions (#52535)
The result-columns of saved pivot questions are a mess, the names and the effective types don't match the values.
Instead of using these columns, we generate the columns from the dataset-query field.

The assumption is that the result-columns for saved questions (but not for models) don't play an important role.
2025-01-23 00:16:29 +03:00
metamben
2d47254583 Disable Metabot's filter-records tool (#52546)
Fixes #52499
2025-01-23 00:16:03 +03:00
Tamás Benkő
ec6033870c Merge remote-tracking branch 'origin/master' into metabot-v3-main 2025-01-23 00:05:13 +03:00
metamben
db9e5b1a49 Merge branch 'master' into metabot-v3-main 2025-01-22 23:14:38 +03:00
metamben
96c9346eda Use integers for report_id, strings for table_id (#52528) 2025-01-22 20:28:11 +03:00
metamben
57af6af23a Add standalone test for the create-dashboard-subscription tool (#52519) 2025-01-22 20:27:49 +03:00
metamben
8598fc59d5 Merge branch 'master' into metabot-v3-main 2025-01-22 16:54:50 +03:00
John Swanson
6bcbe2f35a Don't send queries to llm (#51599)
* Don't send queries to the LLM

This introduces `envelope/llm-message`, a private function that formats
messages for the LLM. Currently it does two things: removes `query`s and
json-encodes the structured content.

There's also `envelope/llm-history`, which replaces
`envelope/full-history` by providing the full history appropriate for
the LLM (instead of the full "internal" history for our own use).

* Rework confusing fn

Split this out into a few functions:

- `stringified-content` takes a message and returns the stringified
content (from `structured-content` or `content` as appropriate)

- `stringify-content` takes a message and returns a message with a
stringified `:content` form (and no `:structured-content`)

- `maybe-remove-query` takes a message and removes the `:query` if
necessary

Now `llm-message` just threads the message through `maybe-remove-query`
and then `stringify-content` rather than a confusing `cond->` form.
2025-01-21 21:11:04 +03:00
metamben
57af242b8f Disable invite-user tool (#52450) 2025-01-21 20:35:18 +03:00
metamben
22a3a5fe44 Merge branch 'master' into metabot-v3-main 2025-01-21 18:35:53 +03:00
metamben
8ddeee90f3 Support find-outliers for reports and queries in simple cases (#52414)
When the result has a temporal column, the first in the list is taken as the basis of the analysis.
2025-01-21 17:27:47 +03:00
metamben
74574ed2aa Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-21 00:22:01 +03:00
metamben
d3804ab2bc Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-20 21:11:53 +03:00
metamben
c76e64407b Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-20 19:40:33 +03:00
metamben
7330ca489e Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-20 17:05:33 +03:00
Mihael Konjević
7a0db83be5 Implement redirect reaction for Metabot (#52289)
* Implement redirect reaction for Metabot

Redirect reaction redirects the user to the results page generated by a tool.
Currently `query-metric` and `generate-insights` tools use this reaction.

This also introduces a concept of "terminating" reactions. When a tool returns a
terminating reaction, this tool call result is not sent back to the LLM, instead
it returns the envelope and the frontend handles any sideffects.
2025-01-20 13:26:31 +01:00
metamben
abdfa33583 Ignore archived metrics in the find-metric tool (#52322)
* Ignore archived metrics in the `find-metric` tool

* Use yarn.lock generated by yarn
2025-01-17 22:07:15 +03:00
Tamás Benkő
dee067ffff Merge remote-tracking branch 'origin/master' into metabot-v3-second-try-rebased 2025-01-17 19:33:57 +03:00
Tamás Benkő
8cbea88330 Fix formatting 2025-01-17 18:58:38 +03:00
metamben
d4944b478b Normalize tool error responses (#52230) 2025-01-17 18:02:28 +03:00
metamben
5603565c7c Look for field_granularity instead of field-granularity (#52271)
* Look for field_granularity instead of field-granularity

* Don't report an error when bucketing non-temporal columns

Just ignore the bucket instead.
2025-01-17 17:50:53 +03:00
metamben
fd44da515e Handle dashcards without card when creating subscriptions (#52248) 2025-01-17 17:50:41 +03:00
metamben
56419f6be9 Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-17 16:44:16 +03:00
metamben
43dab70197 Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-16 23:30:08 +03:00
metamben
26bdc4ca18 Distinguish structured and textual tool results (#51698)
* Distinguish structured and textual tool results
2025-01-16 15:58:33 +03:00
metamben
c17bbb8ca3 Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-16 15:57:27 +03:00
metamben
d30c95d30b Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-16 00:41:22 +03:00
metamben
ce8558949a Return metric IDs as plain integers instead of card__<id> (#52059)
See https://metaboat.slack.com/archives/C07SJT1P0ET/p1736237110786299?thread_ts=1735243354.390349&cid=C07SJT1P0ET for context.
2025-01-15 23:52:37 +03:00
Thomas Schmidt
d027f538f1 Update tool spec with examples (#52195)
* Update tool spec with examples

* Support string table_id values in filter-records

---------

Co-authored-by: Tamás Benkő <tamas@metabase.com>
2025-01-15 19:57:43 +03:00
metamben
13a73a6b9a Implement show-results-to-user tool (#52121) 2025-01-14 23:12:42 +03:00
metamben
80c322f8b1 Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-14 16:56:04 +03:00
metamben
acc331c5fb Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-13 21:43:31 +03:00
metamben
4edd922f10 Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-13 21:06:06 +03:00
Tamás Benkő
6108f5d699 Merge remote-tracking branch 'origin/master' into metabot-v3-second-try-rebased 2025-01-13 19:20:40 +03:00
Tamás Benkő
83af40a49d Fix linter errors from wrong merge 2025-01-13 19:19:55 +03:00
Tamás Benkő
71d828be07 Fix require from wrong merge 2025-01-13 18:49:51 +03:00
metamben
129f1ce7f4 Merge branch 'master' into metabot-v3-second-try-rebased 2025-01-13 18:24:23 +03:00
Tamás Benkő
0e0aac3f46 Merge remote-tracking branch 'origin/master' into metabot-v3-second-try-rebased 2025-01-13 18:11:23 +03:00
metamben
474ecbeec5 Properly report unknown users from create-dashboard-subscription (#51662) 2025-01-13 17:41:26 +03:00
metamben
1b3ed70dec Fix typo (#51702) 2024-12-31 15:03:26 +03:00
metamben
91abc51ec5 Merge branch 'master' into metabot-v3-second-try-rebased 2024-12-31 12:37:47 +03:00
John Swanson
1c02aebe07 Find outliers tool (#51534)
* QOL improvements for the REPL namespace

* Find outliers tool

There are also some unrelated changes I threw in here:

- remove a bit of unnecessary serialization of output (we do this
automatically now)

- delete a stale tool that is no longer part of toolbox v1 (I noticed
the LLM trying to call it)

* CR resp

* Userland query

* Userland query *with default constraints*

* kondo
2024-12-27 12:17:41 -08:00
John Swanson
0af736a234 Adjust response format for invite-user (#51536) 2024-12-27 12:17:23 -08:00
metamben
0c3b93131c Generate field_ids by index (#51606)
* Identify fields by their index

* Generate short field IDs

* Truncate columns to day when filtering without time component

* Decode tool history contents when searching for generated queries
2024-12-27 01:04:37 +03:00
metamben
ada0e86826 Merge branch 'master' into metabot-v3-second-try-rebased 2024-12-23 14:33:12 +03:00
Alexander Polyankin
160be30f2e Fix tests 2024-12-22 15:11:14 -05:00
Alexander Polyankin
f6d87b174d Fix tests 2024-12-22 15:10:57 -05:00
metamben
1d8350e6b7 Merge branch 'master' into metabot-v3-second-try-rebased 2024-12-21 02:29:19 +03:00
metamben
88605995e1 Merge branch 'master' into metabot-v3-second-try-rebased 2024-12-20 17:24:51 +03:00
Tamás Benkő
1516add0f2 Handwave FE merge 2024-12-20 00:03:19 +03:00
Tamás Benkő
751c628779 Use metabase.util.json instead of cheshire.core 2024-12-20 00:03:19 +03:00
Tamás Benkő
1c7d841199 Fix bad var ref 2024-12-20 00:03:19 +03:00
Alexander Polyankin
73c8f0e87c Add user_is_viewing to FE->BE context (#51482)
* Add user_is_viewing to FE->BE context

* Use the context provided by the FE

* Fix bad merge

---------

Co-authored-by: Tamás Benkő <tamas@metabase.com>
2024-12-20 00:03:19 +03:00
metamben
5306cc9e37 Send current_user_time in LLM context (#51477) 2024-12-20 00:03:19 +03:00
John Swanson
1545b358dd green CI? (#51459)
* green CI?

* don't swap in test context for dummy tools

* convert non-string tool responses to json

* CR feedback and fix circular dep

* clean up reaction handling in envelope

* fix another test
2024-12-20 00:03:19 +03:00
metamben
2d68c5569e Register generate-insights and invite-user (#51470) 2024-12-20 00:03:19 +03:00
metamben
e0f0fac373 Implement filter-records (#51452)
* Implement filter-data

* Switch dummy tools to new tool parameters

* Rename filter-data to filter-records

* Register find-metric tool

* Select only the fields from table we really need
2024-12-20 00:03:19 +03:00
John Swanson
c98477e607 fix dummy tools (#51457) 2024-12-20 00:03:19 +03:00
John Swanson
974424174c Rework flow to allow for a query store (#51425)
* Rework flow to allow for a query store

Dummy tools need to be able to store queries for later use. Right now
dummy tools work differently than regular tools: they can
only (effectively) return an `output`. They can't change context, they
can't return reactions, etc.

I was imagining that when we wanted to add queries to the query store,
we'd do so in the `envelope`, and probably do this in a functional way
by returning `:queries` or a `:query-id->query` map from the tool call
itself. Then, we'd write a function in `envelope` that would basically
just store this state for later retrieval.

The issue we had is that (before this commit) we don't actually invoke
dummy tools in a way that changes the envelope - we just invoke them as
needed before sending the history to the LLM.

So, this PR reworks things a bit:

- we invoke the dummy tool calls once, immediately after creating the
envelope

- right now, this only means that we store the results of those tool
calls in the env, for later retrieval to send to the LLM when we are
actually making a call. (we don't just store the tool calls in normal
history, because we don't want to return them to the FE when we
respond).

- in the immediate future though, we'll want to *also*, in addition to
adding tool `output` to the response, handle other tool results. One
thing could be changing the context in response to a dummy tool call -
which seems maybe a bit crazy, but on the other hand maybe dummy tool
calls should be able to do anything a regular tool call can do?

- more immediately, we'll probably want to handle a new key from tool
invocations, `:query-id->query-map`, that will store queries for later
retrieval.

* Add a function to find a query

* refactor `invoke-tool` to take entire env

I also changed the response handling so that if you don't return
`:context` it's ignored, rather than associng `nil` onto the existing
env.

* actually use `find-query` in generate-insights
2024-12-20 00:03:19 +03:00
Thomas Schmidt
2df3d84fed Sync tool schemas with ai-service (#51446) 2024-12-20 00:03:19 +03:00
metamben
f1d23fa5d9 Implement run-query (#51420)
* Implement run-query as dummy tool

* Return kebab-cased keys from query-metric tool

The keys will be automatically converted to snake_case in the message.

* Skip context processing of the previous attempt
2024-12-20 00:03:19 +03:00
metamben
de6b854244 Implement query-metrics and get-metric-details tools (#51315)
* Implement ISO day of week extraction (#50475)

* Implement ISO day of week extraction

Fixes #50424

* Support display-name for ISO get-day-of-week exclusion expressions (#50606)

* Support display-name for ISO get-day-of-week exclusion expressions

Fixes #50574

* Support extraction expressions in filter-args-display-name (#50694)

Fixes #50670

* Add get-metric-details and query-metric tools

* Pass history to tool invocations

* Tool implementations (#51244)

* Skeleton for `generate-insights` tool

For now I just implemented the easy things: tables, metrics, and
reports (which I think equal cards on our end?), and totally skipped
`query_id`.

* Pull in all the tools from ai-service

* `get-report-details` and `get-metric-details`

* find metrics using the select metrics endpoint

* reimplement invite-user tool

skipping all confirmation for now

* Update enterprise/backend/src/metabase_enterprise/metabot_v3/tools/generate_insights.clj

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

* Align table and report details with metric details

* Fix tool JSON schemata

---------

Co-authored-by: John Swanson <john.swanson@metabase.com>
2024-12-20 00:03:19 +03:00
metamben
b6af5b1a2e Implement get-table-details dummy tool (#51143)
* Implement get-table-details dummy tool

* Only return readable tables
2024-12-20 00:03:19 +03:00
metamben
15adc8994c Send auth token via request header instead of the body (#51174)
* Send auth token via request header instead of the body

* Remove instance-info from request body
2024-12-20 00:03:19 +03:00
metamben
da69aaf7d0 Implement dashboard subscription using dummy tools (#51074) 2024-12-20 00:03:19 +03:00
Alexander Polyankin
e1c39f6b5e Add dashboard_id to FE->BE context (#51027) 2024-12-20 00:03:19 +03:00
Sloan Sparger
c76b952c5b [Metabot] Fix e2e failures (#50112)
* force ci run

* test fix

* fix people tests

* revert router changes

* removes force enabling metabot that does not do anything

* disable metabot tests since the token feature was disabled

* change

* change

* fix documentation
2024-12-20 00:03:19 +03:00
Alexander Polyankin
0ad961096b Throw error 2024-12-20 00:03:19 +03:00
Alexander Polyankin
8fac6159ec Hacky multi-tool chain (#50179)
* Hacky multi-tool chain

* Hacky multi-tool chain
2024-12-20 00:03:19 +03:00
Alexander Polyankin
409d7a2251 Add granularity 2024-12-20 00:03:19 +03:00
Alexander Polyankin
325c6a17a1 filter-data and aggregate-data tools (#50168) 2024-12-20 00:03:19 +03:00
Sloan Sparger
6a6312c8e2 [Metabot] UI Access improvements (#49859)
* auto close the metabot input on route changes

* prevents auto-closing metabot if the base path segment is the same

* pr feedback

* makes metabot only visible while the user is signed in

* add test coverage

* fix infinite type instantiation errors
2024-12-20 00:03:19 +03:00
Alexander Polyankin
3a98e2d44f Bring back change-chart-appearance tool (#50104) 2024-12-20 00:03:19 +03:00
Alexander Polyankin
a604ab351b change-query cleanup (#49921) 2024-12-20 00:03:19 +03:00
Sloan Sparger
560f2e1455 [Metabot] Auto-close on route change (#49810)
* auto close the metabot input on route changes

* prevents auto-closing metabot if the base path segment is the same

* pr feedback
2024-12-20 00:03:19 +03:00
Sloan Sparger
a283463d05 [Metabot] Sessions (#49785)
* base e2e tests

* unit tests

* fe implementation for sessions

* BE: Send back a `session_id` if missing

* change to uuid

* send the session id along to ai-service

* pr feedback

* require `session_id` to be passed, and be a UUID

* backend: send a dummy session id to unbreak stats (#49894)

* backend: send a dummy session id to unbreak stats

* preserve session-id across calls

one shortcoming: it does not preserve the session id across FE
invocations. Need to send it back and the FE parrot it back, and the
envelope create function should reuse that when present

```
{
  "messages" : [ {
    "role" : "user",
    "content" : "filter to where the state is CA"
  } ],
  "context" : {...}
  "tools" : [,,,],
  "instance_info" : {,,,},
  "user_id" : 1,
  "session_id" : "6e67dafe-30e6-4b0a-8387-5dc5968e7f28" // <----
}

"LLM -----------------> BE"
{
  "message" : {
    "content" : "",
    "role" : "assistant",
    "tool_calls" : [ {
      "id" : "call_NpA5WEA2gzrRDWt7pC8i0GZR",
      "name" : "change_query",
      "arguments" : "{\"relative_date_filters\":null,\"specific_date_filters\":null,\"boolean_filters\":null,\"breakouts\":null,\"string_filters\":[{\"column\":\"State\",\"operator\":\"=\",\"value\":\"CA\"}],\"order_bys\":null,\"limits\":null,\"number_filters\":null,\"aggregations\":null}"
    } ]
  },
  "metadata" : {
    "usage" : {
      "model" : "gpt-4o-mini-2024-07-18",
      "total" : 7404,
      "prompt" : 7335,
      "completion" : 69,
      "estimated_costs" : 0.0011519999999999998
    }
  }
}

"BE -----------------> LLM"
{
  "messages" : [ {
    "role" : "user",
    "content" : "filter to where the state is CA"
  }, {
    "content" : "",
    "role" : "assistant",
    "tool_calls" : [ {
      "id" : "call_NpA5WEA2gzrRDWt7pC8i0GZR",
      "name" : "change_query",
      "arguments" : "{\"order-bys\":null,\"specific-date-filters\":null,\"breakouts\":null,\"boolean-filters\":null,\"relative-date-filters\":null,\"string-filters\":[{\"column\":\"State\",\"operator\":\"=\",\"value\":\"CA\"}],\"number-filters\":null,\"limits\":null,\"aggregations\":null}"
    } ]
  }, {
    "role" : "tool",
    "tool_call_id" : "call_NpA5WEA2gzrRDWt7pC8i0GZR",
    "content" : "success"
  } ],
  "context" : {},
  "tools" : []..,
  "instance_info" : {,,,}
  },
  "user_id" : 1,
  "session_id" : "6e67dafe-30e6-4b0a-8387-5dc5968e7f28" // <---
}
```

This session_id above is missing in the BE -> FE and then on subsequent FE -> BE payloads

---------

Co-authored-by: dan sutton <dan@dpsutton.com>

* right some wrongs from merge conflict resolutions

* fix

* fix

---------

Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: dan sutton <dan@dpsutton.com>
2024-12-20 00:03:19 +03:00
Sloan Sparger
ecc3e7ba0b [Metabot] Initial e2e + unit tests (#49732)
* base e2e tests

* unit tests

* pr feedback

* run metabot tests

* force enable metabot for testing

* test something in ci

* test something in ci 2

* test something in ci 3

* test something in ci 4

* test something in ci 5

* fix for now
2024-12-20 00:03:19 +03:00
John Swanson
57f58d6009 backend: send a dummy session id to unbreak stats (#49894)
* backend: send a dummy session id to unbreak stats

* preserve session-id across calls

one shortcoming: it does not preserve the session id across FE
invocations. Need to send it back and the FE parrot it back, and the
envelope create function should reuse that when present

```
{
  "messages" : [ {
    "role" : "user",
    "content" : "filter to where the state is CA"
  } ],
  "context" : {...}
  "tools" : [,,,],
  "instance_info" : {,,,},
  "user_id" : 1,
  "session_id" : "6e67dafe-30e6-4b0a-8387-5dc5968e7f28" // <----
}

"LLM -----------------> BE"
{
  "message" : {
    "content" : "",
    "role" : "assistant",
    "tool_calls" : [ {
      "id" : "call_NpA5WEA2gzrRDWt7pC8i0GZR",
      "name" : "change_query",
      "arguments" : "{\"relative_date_filters\":null,\"specific_date_filters\":null,\"boolean_filters\":null,\"breakouts\":null,\"string_filters\":[{\"column\":\"State\",\"operator\":\"=\",\"value\":\"CA\"}],\"order_bys\":null,\"limits\":null,\"number_filters\":null,\"aggregations\":null}"
    } ]
  },
  "metadata" : {
    "usage" : {
      "model" : "gpt-4o-mini-2024-07-18",
      "total" : 7404,
      "prompt" : 7335,
      "completion" : 69,
      "estimated_costs" : 0.0011519999999999998
    }
  }
}

"BE -----------------> LLM"
{
  "messages" : [ {
    "role" : "user",
    "content" : "filter to where the state is CA"
  }, {
    "content" : "",
    "role" : "assistant",
    "tool_calls" : [ {
      "id" : "call_NpA5WEA2gzrRDWt7pC8i0GZR",
      "name" : "change_query",
      "arguments" : "{\"order-bys\":null,\"specific-date-filters\":null,\"breakouts\":null,\"boolean-filters\":null,\"relative-date-filters\":null,\"string-filters\":[{\"column\":\"State\",\"operator\":\"=\",\"value\":\"CA\"}],\"number-filters\":null,\"limits\":null,\"aggregations\":null}"
    } ]
  }, {
    "role" : "tool",
    "tool_call_id" : "call_NpA5WEA2gzrRDWt7pC8i0GZR",
    "content" : "success"
  } ],
  "context" : {},
  "tools" : []..,
  "instance_info" : {,,,}
  },
  "user_id" : 1,
  "session_id" : "6e67dafe-30e6-4b0a-8387-5dc5968e7f28" // <---
}
```

This session_id above is missing in the BE -> FE and then on subsequent FE -> BE payloads

---------

Co-authored-by: dan sutton <dan@dpsutton.com>
2024-12-20 00:03:19 +03:00
Alexander Polyankin
782d167f75 Query tool - no context, fetching metadata on demand (#49812) 2024-12-20 00:03:19 +03:00
John Swanson
a90addd18c Move user_id out of instance_info (#49787) 2024-12-20 00:03:19 +03:00
Aleksandr Lesnenko
9c6a2571d4 merge more viz tools 2024-12-20 00:03:19 +03:00
Aleksandr Lesnenko
2dc9749a64 group into other 2024-12-20 00:03:19 +03:00
Aleksandr Lesnenko
aa4d5180da merge a few viz settings tools into one 2024-12-20 00:03:19 +03:00
Aleksandr Lesnenko
27a1329c00 metabot v3: change series settings tool (#49644)
* change series settings tool

* metabot v3: change column settings tool (#49646)

* change column settings tool

* metabot v3: stacked setting tool (#49649)

* stacked setting tool

* metabot v3: add change goal line tool (#49650)

* add change goal line tool

* y axis range tool (#49651)
2024-12-20 00:03:19 +03:00
Aleksandr Lesnenko
6ed742c176 change axes names tool (#49640) 2024-12-20 00:03:19 +03:00
Aleksandr Lesnenko
888333c654 provide complete visualization settings to llm (#49639) 2024-12-20 00:03:19 +03:00
Alexander Polyankin
b6a9a3d851 Single query tool (#49722)
* MetaBot v3 PoC

* API endpoint

* Fix formatting

* Change tools shape to JSON schema and update API shapes

* initial FE prototype

* FE - update styles, add toggle button for chat, put behind feature flag, PR feedback

* FE - remove previous message history snapshot from being saved on each history entry

* FE - hide behind feature flag

* FE - use history not messages key on agent requests

* Enforce tool naming rules

* Add encoding for reactions

* don't use ::metabot-v3.client/history schema until it exists

put :any in there, we should use the schema when we know what it looks
like

* `defreaction` can be a plain function.

* Remove empty file

* Code cleanup

* Make sure history includes the previous user message

* `:context` should get snake-case encoding for AI Proxy

* Fix encoding of requests => AI Proxy

* Make sure we use :additionalProperties, not :additional_properties

* Fix error with API endpoint and add (mocked) e2e tests for the endpoint

* Some improvements to error handling in client and REPL.

* Add placeholder tool call result messages

* FE - account for schema changes

* Decode api request into internal format

```clojure
api=> (mc/decode [:maybe ::metabot-v3.client.schema/messages]
                 [{:role "user"
                   :content "Invite sloan to metabase"}
                  {:content "Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?"
                   :role "assistant"}]
                 (mtx/transformer
                   {:name :api-request}))
[{:role :user, :content "Invite sloan to metabase"}
 {:content "Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?",
  :role :assistant}]
```

This just ensures that the role gets keywordized.

`:role "user"` -> `:role :user`.

* ensure we decode LLM response

LLM sends back the tool we are using as "invite-user". But our dispatch
stuff needs it to end up as `:metabot.tool/invite-user`. Just a simple
namespace for the keyword.

```shell
❯ echo '{"message":"sloan@metabase.com","context":{},"history":[{"role":"user","content":"Invite sloan to metabase"},{"content":"Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?","role":"assistant"}]}' | http post "localhost:3000/api/ee/metabot-v3/agent" x-api-key:$KEY -pb
{
    "history": [
        {
            "content": "Invite sloan to metabase",
            "role": "user"
        },
        {
            "content": "Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?",
            "role": "assistant"
        },
        {
            "content": "sloan@metabase.com",
            "role": "user"
        },
        {
            "content": "",
            "role": "assistant",
            "tool-calls": [
                {
                    "arguments": {
                        "email": "sloan@metabase.com"
                    },
                    "id": "call_yt9LvJS6bmB2oAekhU94VwqR",
                    "name": "metabot.tool/invite-user"
                }
            ]
        }
    ],
    "reactions": [
        {
            "email": "sloan@metabase.com",
            "type": "metabot.reaction/user-invite-sent"
        }
    ]
}

```

* decode tools

```clojure
api=> (mc/decode ::metabot-v3.client.schema/messages
                 [{:content "",
                   :role "assistant",
                   :tool-calls [{:id "call_Ma3wQEylZIT27t8z4lx8ch2y",
                                 :name "metabot.tool/say-hello",
                                 :arguments {"name" "User", "greeting" "hi"}}]}]
                 (mtx/transformer {:name :api-request}))
[{:content "",
  :role :assistant,
  :tool-calls [{:id "call_Ma3wQEylZIT27t8z4lx8ch2y",
                :name :metabot.tool/say-hello,
                :arguments {"name" "User", "greeting" "hi"}}]}]
```

from the api:

```shell
❯ echo '{"message":"sloan@metabase.com","context":{},"history":[{"role":"user","content":"Invite sloan to metabase"},{"content":"Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?","role":"assistant"}]}' | http post "localhost:3000/api/ee/metabot-v3/agent" x-api-key:$KEY -pb | jq
 '.reactions'
[
  {
    "type": "metabot.reaction/user-invite-sent",
    "email": "sloan@metabase.com"
  }
]
```

and

```
❯ echo '{"message":"hi","context":{},"history":[{"role":"user","content":"hi"},{"content":"","role":"assistant","tool-calls":[{"id":"call_Ma3wQEylZIT27t8z4lx8ch2y","name":"metabot.tool/say-hello","arguments":{"name":"User","greeting":"hi"}}]}]}' | http post "localhost:3000/api/ee/metabot-v3/agent" x-api-key:$KEY -pb | jq '.reactions'
[
  {
    "type": "metabot.reaction/message",
    "message": "Sorry I don't understand that. Could you please provide more context or let me know how I can assist you?",
    "repl/message_color": "green",
    "repl/message_emoji": "🤖"
  }
]
```

* Add tool-call-ids to less recent history items

* Playing around with other tools

* FE - placeholder for demo steps which are ready for their be reactions

* FE - update how metabot can be accessed

* Add `defreactions` for the new reactions

* include comparison operators

* include other tools

* small fixes

* oops - value is a string or number, not an enum

* An actually working schema for filter value

* go to question tool/reaction

* tell the LLM not to change display with filter

* small fixes for malli schemas

* change the question to sales over time

* hardcode the default hosted proxy URL

* correct URL for ai-proxy

* FE - connect fe reaction implementation with real BE reactions

* FE - shorter animation duration

* change Ask Metabot to Metabot request

* hardcode URL in the getter instead of default

oop, `(premium-features/is-hosted?)` isn't available at compile time so
we need to just hardcode this in the getter. This means we can't
actually set the setting now, but that should be fine for now.

* Respond with simple messages after invite/favorite

* Hide any error details for the demo

* simpler error message

* FE - impl loading state

* Improve invite user schema descriptions

* Round trips to the LLM

* Hopefully fix clj-kondo hook test

This is my first time actually touching a hook, so hopefully I didn't
mess it up.

* Fix a few more tests

* Log traffic to `llm-payloads` file

```json
"FE -----------------> BE"
{
  "message" : "invite thomas to metabase",
  "context" : { },
  "history" : [ ]
}

"BE -----------------> LLM"
{
  "messages" : [ {
    "role" : "user",
    "content" : "invite thomas to metabase"
  } ],
  "context" : { },
  "tools" : [ {
    "name" : "apply_visualizations",
    ...
    }]}

"LLM -----------------> BE"
{
  "message" : {
    "content" : "Could you please provide me with Thomas's email address so I can send the invitation to Metabase?",
    "role" : "assistant"
  },
  "metadata" : {
    "usage" : {
      "model" : "gpt-4o-mini-2024-07-18",
      "total" : 1163,
      "prompt" : 1142,
      "completion" : 21,
      "estimated_costs" : 1.8704999999999997E-4
    }
  }
}
...
"BE -----------------> FE"
{
  "reactions" : [ {
    "type" : "metabot.reaction/message",
    "repl/message_color" : "green",
    "repl/message_emoji" : "🤖",
    "message" : "Thomas has been successfully invited to Metabase! If you need anything else, just let me know."
  } ],
  "history" : [ {
    "role" : "user",
    "content" : "invite thomas to metabase"
    ...
```

* Fix the REPL namespace

* FE - multi-line editor, css based animations, error handling, dismissable messages, updated loading state

* max 6 round trips

* Remove unused tools + real invite users

* Oops, fix tool requirements

* fix token feature test

* another small test fix

* oops - eastwood

* delete empty config

* FE - implments context solution so any component or route can set context for metabot, adds handling for change display type tool

* Observability, fix endpoint URL, and fix table viz

* Rework, but turn off, the invite tool

* Don't log tool args/results

* only log paylods in dev

* FE - impl hide/show columns reaction

* check metabot-v3 feature

* add tests for tools on the backend

* FE - make Dan happy

* FE - make Dan happier

* lint

* FE - huge code improvement pass, keep isProcessing state for metabot and await completion of each reaction before processing next, allow reactions to stop further reaction processing in the case of an error

* gate metabot on a different feature for now

* Some documentation

also moved `handle-response` to `handle-envelope`, since that's what
it's doing now.

* add llm-payloads to gitignore (#49505)

* Add back backend support for adding a user

After talking with Sloan, we reworked the reactions here a bit.

The `confirmation` reaction has a `description` and `options` (the
options here means "the options the user can choose between", not
something like "arguments").

The keys of the options are just names for the choices, in this case
"yes" and "no". When the frontend makes the choice, it carries out the
reactions located under that key in the `confirmation` reaction.

* FE - documentation and make context deregistering O(1)

* FE - fix enter key bug

* Include description in column context

* Always send the LLM's messages as reactions

Previously, we would only send the LLM's message as a reaction if there
were no other reactions. I think it makes more sense to send a list
containing:

- all reactions explicitly set by tool call invocations, PLUS

- all messages from the LLM since your last message (formatted as
reaction messages)

* Fix double message on the who's your favorite tool

* Try something with invite user, and fix up setting

* WIP tools testing

* add ? to confirmation description

* fix confirmation flow

* don't send tool call id on tool invocation

* set default ai host to localhost

* exclude metabot-v3 tests from test run

* FE - invite user improvements, implement confirmation + writeback + api call reactions, update users page to use RTKQuery to allow for reactive updates

* oops, fix failing tool tests

* use the real metabot feature

* Add sort-query tool

* Add direction

* Fix formatting

* Fix formatting

* Fix lint

* Add `aggregate_query` tool (#49622)

* aggregate_query tool

* Fix lint

* Add `limit_query` tool (#49629)

* limit_query tool

* Add breakout_query tool (#49631)

* Fix conflicts

* Single tool for queries

* Add grouping

* Add aggregations

* Add aggregations

* Add aggregations

* Add aggregations

* Add filters

* Cleanup

* Cleanup

* Cleanup

* Booleans

* Dates

* Dates

---------

Co-authored-by: Cam Saul <github@camsaul.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: dan sutton <dan@dpsutton.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Maz Ameli <maz@metabase.com>
Co-authored-by: Thomas Schmidt <thomas.heinz.schmidt@gmail.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
2024-12-20 00:03:19 +03:00
Alexander Polyankin
af08a7488c Add sort_query tool (#49619)
* MetaBot v3 PoC

* API endpoint

* Fix formatting

* Change tools shape to JSON schema and update API shapes

* initial FE prototype

* FE - update styles, add toggle button for chat, put behind feature flag, PR feedback

* FE - remove previous message history snapshot from being saved on each history entry

* FE - hide behind feature flag

* FE - use history not messages key on agent requests

* Enforce tool naming rules

* Add encoding for reactions

* don't use ::metabot-v3.client/history schema until it exists

put :any in there, we should use the schema when we know what it looks
like

* `defreaction` can be a plain function.

* Remove empty file

* Code cleanup

* Make sure history includes the previous user message

* `:context` should get snake-case encoding for AI Proxy

* Fix encoding of requests => AI Proxy

* Make sure we use :additionalProperties, not :additional_properties

* Fix error with API endpoint and add (mocked) e2e tests for the endpoint

* Some improvements to error handling in client and REPL.

* Add placeholder tool call result messages

* FE - account for schema changes

* Decode api request into internal format

```clojure
api=> (mc/decode [:maybe ::metabot-v3.client.schema/messages]
                 [{:role "user"
                   :content "Invite sloan to metabase"}
                  {:content "Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?"
                   :role "assistant"}]
                 (mtx/transformer
                   {:name :api-request}))
[{:role :user, :content "Invite sloan to metabase"}
 {:content "Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?",
  :role :assistant}]
```

This just ensures that the role gets keywordized.

`:role "user"` -> `:role :user`.

* ensure we decode LLM response

LLM sends back the tool we are using as "invite-user". But our dispatch
stuff needs it to end up as `:metabot.tool/invite-user`. Just a simple
namespace for the keyword.

```shell
❯ echo '{"message":"sloan@metabase.com","context":{},"history":[{"role":"user","content":"Invite sloan to metabase"},{"content":"Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?","role":"assistant"}]}' | http post "localhost:3000/api/ee/metabot-v3/agent" x-api-key:$KEY -pb
{
    "history": [
        {
            "content": "Invite sloan to metabase",
            "role": "user"
        },
        {
            "content": "Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?",
            "role": "assistant"
        },
        {
            "content": "sloan@metabase.com",
            "role": "user"
        },
        {
            "content": "",
            "role": "assistant",
            "tool-calls": [
                {
                    "arguments": {
                        "email": "sloan@metabase.com"
                    },
                    "id": "call_yt9LvJS6bmB2oAekhU94VwqR",
                    "name": "metabot.tool/invite-user"
                }
            ]
        }
    ],
    "reactions": [
        {
            "email": "sloan@metabase.com",
            "type": "metabot.reaction/user-invite-sent"
        }
    ]
}

```

* decode tools

```clojure
api=> (mc/decode ::metabot-v3.client.schema/messages
                 [{:content "",
                   :role "assistant",
                   :tool-calls [{:id "call_Ma3wQEylZIT27t8z4lx8ch2y",
                                 :name "metabot.tool/say-hello",
                                 :arguments {"name" "User", "greeting" "hi"}}]}]
                 (mtx/transformer {:name :api-request}))
[{:content "",
  :role :assistant,
  :tool-calls [{:id "call_Ma3wQEylZIT27t8z4lx8ch2y",
                :name :metabot.tool/say-hello,
                :arguments {"name" "User", "greeting" "hi"}}]}]
```

from the api:

```shell
❯ echo '{"message":"sloan@metabase.com","context":{},"history":[{"role":"user","content":"Invite sloan to metabase"},{"content":"Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?","role":"assistant"}]}' | http post "localhost:3000/api/ee/metabot-v3/agent" x-api-key:$KEY -pb | jq
 '.reactions'
[
  {
    "type": "metabot.reaction/user-invite-sent",
    "email": "sloan@metabase.com"
  }
]
```

and

```
❯ echo '{"message":"hi","context":{},"history":[{"role":"user","content":"hi"},{"content":"","role":"assistant","tool-calls":[{"id":"call_Ma3wQEylZIT27t8z4lx8ch2y","name":"metabot.tool/say-hello","arguments":{"name":"User","greeting":"hi"}}]}]}' | http post "localhost:3000/api/ee/metabot-v3/agent" x-api-key:$KEY -pb | jq '.reactions'
[
  {
    "type": "metabot.reaction/message",
    "message": "Sorry I don't understand that. Could you please provide more context or let me know how I can assist you?",
    "repl/message_color": "green",
    "repl/message_emoji": "🤖"
  }
]
```

* Add tool-call-ids to less recent history items

* Playing around with other tools

* FE - placeholder for demo steps which are ready for their be reactions

* FE - update how metabot can be accessed

* Add `defreactions` for the new reactions

* include comparison operators

* include other tools

* small fixes

* oops - value is a string or number, not an enum

* An actually working schema for filter value

* go to question tool/reaction

* tell the LLM not to change display with filter

* small fixes for malli schemas

* change the question to sales over time

* hardcode the default hosted proxy URL

* correct URL for ai-proxy

* FE - connect fe reaction implementation with real BE reactions

* FE - shorter animation duration

* change Ask Metabot to Metabot request

* hardcode URL in the getter instead of default

oop, `(premium-features/is-hosted?)` isn't available at compile time so
we need to just hardcode this in the getter. This means we can't
actually set the setting now, but that should be fine for now.

* Respond with simple messages after invite/favorite

* Hide any error details for the demo

* simpler error message

* FE - impl loading state

* Improve invite user schema descriptions

* Round trips to the LLM

* Hopefully fix clj-kondo hook test

This is my first time actually touching a hook, so hopefully I didn't
mess it up.

* Fix a few more tests

* Log traffic to `llm-payloads` file

```json
"FE -----------------> BE"
{
  "message" : "invite thomas to metabase",
  "context" : { },
  "history" : [ ]
}

"BE -----------------> LLM"
{
  "messages" : [ {
    "role" : "user",
    "content" : "invite thomas to metabase"
  } ],
  "context" : { },
  "tools" : [ {
    "name" : "apply_visualizations",
    ...
    }]}

"LLM -----------------> BE"
{
  "message" : {
    "content" : "Could you please provide me with Thomas's email address so I can send the invitation to Metabase?",
    "role" : "assistant"
  },
  "metadata" : {
    "usage" : {
      "model" : "gpt-4o-mini-2024-07-18",
      "total" : 1163,
      "prompt" : 1142,
      "completion" : 21,
      "estimated_costs" : 1.8704999999999997E-4
    }
  }
}
...
"BE -----------------> FE"
{
  "reactions" : [ {
    "type" : "metabot.reaction/message",
    "repl/message_color" : "green",
    "repl/message_emoji" : "🤖",
    "message" : "Thomas has been successfully invited to Metabase! If you need anything else, just let me know."
  } ],
  "history" : [ {
    "role" : "user",
    "content" : "invite thomas to metabase"
    ...
```

* Fix the REPL namespace

* FE - multi-line editor, css based animations, error handling, dismissable messages, updated loading state

* max 6 round trips

* Remove unused tools + real invite users

* Oops, fix tool requirements

* fix token feature test

* another small test fix

* oops - eastwood

* delete empty config

* FE - implments context solution so any component or route can set context for metabot, adds handling for change display type tool

* Observability, fix endpoint URL, and fix table viz

* Rework, but turn off, the invite tool

* Don't log tool args/results

* only log paylods in dev

* FE - impl hide/show columns reaction

* check metabot-v3 feature

* add tests for tools on the backend

* FE - make Dan happy

* FE - make Dan happier

* lint

* FE - huge code improvement pass, keep isProcessing state for metabot and await completion of each reaction before processing next, allow reactions to stop further reaction processing in the case of an error

* gate metabot on a different feature for now

* Some documentation

also moved `handle-response` to `handle-envelope`, since that's what
it's doing now.

* add llm-payloads to gitignore (#49505)

* Add back backend support for adding a user

After talking with Sloan, we reworked the reactions here a bit.

The `confirmation` reaction has a `description` and `options` (the
options here means "the options the user can choose between", not
something like "arguments").

The keys of the options are just names for the choices, in this case
"yes" and "no". When the frontend makes the choice, it carries out the
reactions located under that key in the `confirmation` reaction.

* FE - documentation and make context deregistering O(1)

* FE - fix enter key bug

* Include description in column context

* Always send the LLM's messages as reactions

Previously, we would only send the LLM's message as a reaction if there
were no other reactions. I think it makes more sense to send a list
containing:

- all reactions explicitly set by tool call invocations, PLUS

- all messages from the LLM since your last message (formatted as
reaction messages)

* Fix double message on the who's your favorite tool

* Try something with invite user, and fix up setting

* WIP tools testing

* add ? to confirmation description

* fix confirmation flow

* don't send tool call id on tool invocation

* set default ai host to localhost

* exclude metabot-v3 tests from test run

* FE - invite user improvements, implement confirmation + writeback + api call reactions, update users page to use RTKQuery to allow for reactive updates

* oops, fix failing tool tests

* use the real metabot feature

* Add sort-query tool

* Add direction

* Fix formatting

* Fix formatting

* Fix lint

* Add `aggregate_query` tool (#49622)

* aggregate_query tool

* Fix lint

* Add `limit_query` tool (#49629)

* limit_query tool

* Add breakout_query tool (#49631)

* Fix conflicts

---------

Co-authored-by: Cam Saul <github@camsaul.com>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: dan sutton <dan@dpsutton.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Maz Ameli <maz@metabase.com>
Co-authored-by: Thomas Schmidt <thomas.heinz.schmidt@gmail.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@users.noreply.github.com>
2024-12-20 00:03:19 +03:00
John Swanson
8933965e77 Instance info
For now, put `user-id` into the instance info. I'll move this over once
we've added that field to the schema on the `ai-service` side.
2024-12-20 00:03:19 +03:00
Sloan Sparger
d4c0b24558 FE - change error messages 2024-12-20 00:03:19 +03:00
Sloan Sparger
7b34e3c04f FE - copy changes 2024-12-20 00:03:19 +03:00
Sloan Sparger
b498a5ad59 FE - removes the feature from being hardcoded as enabled 2024-12-20 00:03:19 +03:00
John Swanson
4530bb4472 use the real metabot feature 2024-12-20 00:03:19 +03:00
John Swanson
f7bc32c37f oops, fix failing tool tests 2024-12-20 00:03:19 +03:00
Sloan Sparger
b11ace9933 FE - invite user improvements, implement confirmation + writeback + api call reactions, update users page to use RTKQuery to allow for reactive updates 2024-12-20 00:03:19 +03:00
John Swanson
4ac8936dda exclude metabot-v3 tests from test run 2024-12-20 00:03:18 +03:00
John Swanson
da80d576fc set default ai host to localhost 2024-12-20 00:03:18 +03:00
John Swanson
75af27d1ef don't send tool call id on tool invocation 2024-12-20 00:03:18 +03:00
John Swanson
b71f81de3a fix confirmation flow 2024-12-20 00:03:18 +03:00
John Swanson
8ae959af40 add ? to confirmation description 2024-12-20 00:03:18 +03:00
John Swanson
54bea92e33 WIP tools testing 2024-12-20 00:03:18 +03:00
John Swanson
c1cec51039 Try something with invite user, and fix up setting 2024-12-20 00:03:18 +03:00
John Swanson
619faab76a Fix double message on the who's your favorite tool 2024-12-20 00:03:18 +03:00
John Swanson
fa679be3f5 Always send the LLM's messages as reactions
Previously, we would only send the LLM's message as a reaction if there
were no other reactions. I think it makes more sense to send a list
containing:

- all reactions explicitly set by tool call invocations, PLUS

- all messages from the LLM since your last message (formatted as
reaction messages)
2024-12-20 00:03:18 +03:00
Thomas Schmidt
3e151e39ce Include description in column context 2024-12-20 00:03:18 +03:00
Sloan Sparger
2086a5bcee FE - fix enter key bug 2024-12-20 00:03:18 +03:00
Sloan Sparger
94658fdbab FE - documentation and make context deregistering O(1) 2024-12-20 00:03:18 +03:00
John Swanson
c4d52fb574 Add back backend support for adding a user
After talking with Sloan, we reworked the reactions here a bit.

The `confirmation` reaction has a `description` and `options` (the
options here means "the options the user can choose between", not
something like "arguments").

The keys of the options are just names for the choices, in this case
"yes" and "no". When the frontend makes the choice, it carries out the
reactions located under that key in the `confirmation` reaction.
2024-12-20 00:03:18 +03:00
Aleksandr Lesnenko
1444a637e3 add llm-payloads to gitignore (#49505) 2024-12-20 00:03:18 +03:00
John Swanson
fdbbe3e246 Some documentation
also moved `handle-response` to `handle-envelope`, since that's what
it's doing now.
2024-12-20 00:03:18 +03:00
John Swanson
4d722317bf gate metabot on a different feature for now 2024-12-20 00:03:18 +03:00
Sloan Sparger
07fb93c1dd FE - huge code improvement pass, keep isProcessing state for metabot and await completion of each reaction before processing next, allow reactions to stop further reaction processing in the case of an error 2024-12-20 00:03:18 +03:00
John Swanson
3082c28ce6 lint 2024-12-20 00:03:18 +03:00
Sloan Sparger
c43ffdf1f6 FE - make Dan happier 2024-12-20 00:03:18 +03:00
Sloan Sparger
30e1b63230 FE - make Dan happy 2024-12-20 00:03:18 +03:00
John Swanson
35f2719fc7 add tests for tools on the backend 2024-12-20 00:03:18 +03:00
John Swanson
5d2b8b4192 check metabot-v3 feature 2024-12-20 00:03:18 +03:00
Sloan Sparger
dd305e34c2 FE - impl hide/show columns reaction 2024-12-20 00:03:18 +03:00
dan sutton
7c47d4908f only log paylods in dev 2024-12-20 00:03:18 +03:00
John Swanson
718602271b Don't log tool args/results 2024-12-20 00:03:18 +03:00
John Swanson
cbbc868d07 Rework, but turn off, the invite tool 2024-12-20 00:03:18 +03:00
John Swanson
ca5aadc113 Observability, fix endpoint URL, and fix table viz 2024-12-20 00:03:18 +03:00
Sloan Sparger
313f03307a FE - implments context solution so any component or route can set context for metabot, adds handling for change display type tool 2024-12-20 00:03:18 +03:00
John Swanson
e92eba2a34 delete empty config 2024-12-20 00:03:18 +03:00
John Swanson
eefe925be2 oops - eastwood 2024-12-20 00:03:18 +03:00
John Swanson
6ed9935bf0 another small test fix 2024-12-20 00:03:18 +03:00
John Swanson
7032631e79 fix token feature test 2024-12-20 00:03:18 +03:00
John Swanson
b0e5c8f306 Oops, fix tool requirements 2024-12-20 00:03:18 +03:00
John Swanson
06b61e9d11 Remove unused tools + real invite users 2024-12-20 00:03:18 +03:00
John Swanson
74c7533d9f max 6 round trips 2024-12-20 00:03:18 +03:00
Sloan Sparger
eca6d725db FE - multi-line editor, css based animations, error handling, dismissable messages, updated loading state 2024-12-20 00:03:18 +03:00
John Swanson
1ee39a695f Fix the REPL namespace 2024-12-20 00:03:18 +03:00
dan sutton
7d97203f80 Log traffic to llm-payloads file
```json
"FE -----------------> BE"
{
  "message" : "invite thomas to metabase",
  "context" : { },
  "history" : [ ]
}

"BE -----------------> LLM"
{
  "messages" : [ {
    "role" : "user",
    "content" : "invite thomas to metabase"
  } ],
  "context" : { },
  "tools" : [ {
    "name" : "apply_visualizations",
    ...
    }]}

"LLM -----------------> BE"
{
  "message" : {
    "content" : "Could you please provide me with Thomas's email address so I can send the invitation to Metabase?",
    "role" : "assistant"
  },
  "metadata" : {
    "usage" : {
      "model" : "gpt-4o-mini-2024-07-18",
      "total" : 1163,
      "prompt" : 1142,
      "completion" : 21,
      "estimated_costs" : 1.8704999999999997E-4
    }
  }
}
...
"BE -----------------> FE"
{
  "reactions" : [ {
    "type" : "metabot.reaction/message",
    "repl/message_color" : "green",
    "repl/message_emoji" : "🤖",
    "message" : "Thomas has been successfully invited to Metabase! If you need anything else, just let me know."
  } ],
  "history" : [ {
    "role" : "user",
    "content" : "invite thomas to metabase"
    ...
```
2024-12-20 00:03:18 +03:00
John Swanson
53df1cc48f Fix a few more tests 2024-12-20 00:03:18 +03:00
John Swanson
445689bd61 Hopefully fix clj-kondo hook test
This is my first time actually touching a hook, so hopefully I didn't
mess it up.
2024-12-20 00:03:18 +03:00
John Swanson
1afd8db757 Round trips to the LLM 2024-12-20 00:03:18 +03:00
Thomas Schmidt
f36b61d095 Improve invite user schema descriptions 2024-12-20 00:03:18 +03:00
Sloan Sparger
1320442e81 FE - impl loading state 2024-12-20 00:03:18 +03:00
John Swanson
9a5c9f6b43 simpler error message 2024-12-20 00:03:18 +03:00
John Swanson
a77f8d3186 Hide any error details for the demo 2024-12-20 00:03:18 +03:00
John Swanson
d18a41c615 Respond with simple messages after invite/favorite 2024-12-20 00:03:18 +03:00
John Swanson
23e07e333b hardcode URL in the getter instead of default
oop, `(premium-features/is-hosted?)` isn't available at compile time so
we need to just hardcode this in the getter. This means we can't
actually set the setting now, but that should be fine for now.
2024-12-20 00:03:18 +03:00
Maz Ameli
c032343abb change Ask Metabot to Metabot request 2024-12-20 00:03:18 +03:00
Sloan Sparger
85454134b8 FE - shorter animation duration 2024-12-20 00:03:18 +03:00
Sloan Sparger
75a61b3ac1 FE - connect fe reaction implementation with real BE reactions 2024-12-20 00:03:18 +03:00
John Swanson
3855c55a9a correct URL for ai-proxy 2024-12-20 00:03:18 +03:00
John Swanson
7023192f86 hardcode the default hosted proxy URL 2024-12-20 00:03:18 +03:00
John Swanson
e52e7b9ca4 change the question to sales over time 2024-12-20 00:03:18 +03:00
John Swanson
1917280df3 small fixes for malli schemas 2024-12-20 00:03:18 +03:00
John Swanson
6a5ac5a8c5 tell the LLM not to change display with filter 2024-12-20 00:03:18 +03:00
John Swanson
54d966fb98 go to question tool/reaction 2024-12-20 00:03:18 +03:00
John Swanson
89e065a77a An actually working schema for filter value 2024-12-20 00:03:18 +03:00
John Swanson
f46fb69020 oops - value is a string or number, not an enum 2024-12-20 00:03:18 +03:00
John Swanson
e5d0c154e6 small fixes 2024-12-20 00:03:18 +03:00
dan sutton
c937c135db include other tools 2024-12-20 00:03:18 +03:00
dan sutton
0052fdfdec include comparison operators 2024-12-20 00:03:18 +03:00
John Swanson
3847d4067c Add defreactions for the new reactions 2024-12-20 00:03:18 +03:00
Sloan Sparger
ba6d81f561 FE - update how metabot can be accessed 2024-12-20 00:03:18 +03:00
Sloan Sparger
50f3c789b8 FE - placeholder for demo steps which are ready for their be reactions 2024-12-20 00:03:18 +03:00
John Swanson
9c71ce697a Playing around with other tools 2024-12-20 00:03:18 +03:00
John Swanson
ba2c9e9eca Add tool-call-ids to less recent history items 2024-12-20 00:03:18 +03:00
dan sutton
4a61e5a42c decode tools
```clojure
api=> (mc/decode ::metabot-v3.client.schema/messages
                 [{:content "",
                   :role "assistant",
                   :tool-calls [{:id "call_Ma3wQEylZIT27t8z4lx8ch2y",
                                 :name "metabot.tool/say-hello",
                                 :arguments {"name" "User", "greeting" "hi"}}]}]
                 (mtx/transformer {:name :api-request}))
[{:content "",
  :role :assistant,
  :tool-calls [{:id "call_Ma3wQEylZIT27t8z4lx8ch2y",
                :name :metabot.tool/say-hello,
                :arguments {"name" "User", "greeting" "hi"}}]}]
```

from the api:

```shell
❯ echo '{"message":"sloan@metabase.com","context":{},"history":[{"role":"user","content":"Invite sloan to metabase"},{"content":"Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?","role":"assistant"}]}' | http post "localhost:3000/api/ee/metabot-v3/agent" x-api-key:$KEY -pb | jq
 '.reactions'
[
  {
    "type": "metabot.reaction/user-invite-sent",
    "email": "sloan@metabase.com"
  }
]
```

and

```
❯ echo '{"message":"hi","context":{},"history":[{"role":"user","content":"hi"},{"content":"","role":"assistant","tool-calls":[{"id":"call_Ma3wQEylZIT27t8z4lx8ch2y","name":"metabot.tool/say-hello","arguments":{"name":"User","greeting":"hi"}}]}]}' | http post "localhost:3000/api/ee/metabot-v3/agent" x-api-key:$KEY -pb | jq '.reactions'
[
  {
    "type": "metabot.reaction/message",
    "message": "Sorry I don't understand that. Could you please provide more context or let me know how I can assist you?",
    "repl/message_color": "green",
    "repl/message_emoji": "🤖"
  }
]
```
2024-12-20 00:03:18 +03:00
dan sutton
7007095b35 ensure we decode LLM response
LLM sends back the tool we are using as "invite-user". But our dispatch
stuff needs it to end up as `:metabot.tool/invite-user`. Just a simple
namespace for the keyword.

```shell
❯ echo '{"message":"sloan@metabase.com","context":{},"history":[{"role":"user","content":"Invite sloan to metabase"},{"content":"Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?","role":"assistant"}]}' | http post "localhost:3000/api/ee/metabot-v3/agent" x-api-key:$KEY -pb
{
    "history": [
        {
            "content": "Invite sloan to metabase",
            "role": "user"
        },
        {
            "content": "Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?",
            "role": "assistant"
        },
        {
            "content": "sloan@metabase.com",
            "role": "user"
        },
        {
            "content": "",
            "role": "assistant",
            "tool-calls": [
                {
                    "arguments": {
                        "email": "sloan@metabase.com"
                    },
                    "id": "call_yt9LvJS6bmB2oAekhU94VwqR",
                    "name": "metabot.tool/invite-user"
                }
            ]
        }
    ],
    "reactions": [
        {
            "email": "sloan@metabase.com",
            "type": "metabot.reaction/user-invite-sent"
        }
    ]
}

```
2024-12-20 00:03:18 +03:00
dan sutton
d32f8c5940 Decode api request into internal format
```clojure
api=> (mc/decode [:maybe ::metabot-v3.client.schema/messages]
                 [{:role "user"
                   :content "Invite sloan to metabase"}
                  {:content "Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?"
                   :role "assistant"}]
                 (mtx/transformer
                   {:name :api-request}))
[{:role :user, :content "Invite sloan to metabase"}
 {:content "Sorry, I cant help you with that. Could you please provide the email address of the user you want to invite to Metabase?",
  :role :assistant}]
```

This just ensures that the role gets keywordized.

`:role "user"` -> `:role :user`.
2024-12-20 00:03:18 +03:00
Sloan Sparger
a1174e82b1 FE - account for schema changes 2024-12-20 00:03:18 +03:00
Cam Saul
6b500e15f9 Add placeholder tool call result messages 2024-12-20 00:03:18 +03:00
Cam Saul
b02e3c9aa1 Some improvements to error handling in client and REPL. 2024-12-20 00:03:18 +03:00
Cam Saul
6abab13f62 Fix error with API endpoint and add (mocked) e2e tests for the endpoint 2024-12-20 00:03:18 +03:00
Cam Saul
f9919999ff Make sure we use :additionalProperties, not :additional_properties 2024-12-20 00:03:18 +03:00
Cam Saul
22ae538b0c Fix encoding of requests => AI Proxy 2024-12-20 00:03:18 +03:00
Cam Saul
ae093274ff :context should get snake-case encoding for AI Proxy 2024-12-20 00:03:18 +03:00
Cam Saul
ac3a171308 Make sure history includes the previous user message 2024-12-20 00:03:18 +03:00
Cam Saul
de00816b7b Code cleanup 2024-12-20 00:03:18 +03:00
Cam Saul
2c647c0637 Remove empty file 2024-12-20 00:03:18 +03:00
Cam Saul
c4866381a3 defreaction can be a plain function. 2024-12-20 00:03:17 +03:00
dan sutton
749a5e6a56 don't use ::metabot-v3.client/history schema until it exists
put :any in there, we should use the schema when we know what it looks
like
2024-12-20 00:03:17 +03:00
Cam Saul
556acb9b10 Add encoding for reactions 2024-12-20 00:03:17 +03:00
Cam Saul
d60072bcc9 Enforce tool naming rules 2024-12-20 00:03:17 +03:00
Sloan Sparger
d478a79856 FE - use history not messages key on agent requests 2024-12-20 00:03:17 +03:00
Sloan Sparger
6067de5ba7 FE - hide behind feature flag 2024-12-20 00:03:17 +03:00
Sloan Sparger
b98ed219e8 FE - remove previous message history snapshot from being saved on each history entry 2024-12-20 00:03:17 +03:00
Sloan Sparger
d6f4458fba FE - update styles, add toggle button for chat, put behind feature flag, PR feedback 2024-12-20 00:03:17 +03:00
Sloan Sparger
4b5457848f initial FE prototype 2024-12-20 00:03:17 +03:00
Cam Saul
acd4ed01c2 Change tools shape to JSON schema and update API shapes 2024-12-20 00:03:17 +03:00
Cam Saul
06212efee6 Fix formatting 2024-12-20 00:03:17 +03:00
Cam Saul
2c4c399e29 API endpoint 2024-12-20 00:03:17 +03:00
Cam Saul
71ac4f02f1 MetaBot v3 PoC 2024-12-20 00:03:17 +03:00
11855 changed files with 1551656 additions and 1535903 deletions

View File

@@ -1 +0,0 @@
Fix flakey test $ARGUMENTS

View File

@@ -1 +0,0 @@
Fix issue $ARGUMENTS

View File

@@ -1 +0,0 @@
Fetch unresolved comments for this branch's PR, then fix them

View File

@@ -1 +0,0 @@
Repro issue $ARGUMENTS in a failing test

View File

@@ -11,7 +11,6 @@
:def-fn {:level :warning}
:dynamic-var-not-earmuffed {:level :warning}
:equals-expected-position {:level :warning, :only-in-test-assertion true}
:equals-true {:level :warning}
:keyword-binding {:level :warning}
:main-without-gen-class {:level :warning}
:minus-one {:level :warning}
@@ -43,13 +42,14 @@
#_:docstring-leading-trailing-whitespace
#_:equals-false
#_:equals-true
#_:used-underscored-binding
;;
;; disabled linters
;;
:redundant-ignore {:level :off} ; lots of false positives, see https://github.com/clj-kondo/clj-kondo/issues/2414
:redundant-ignore {:level :off} ; lots of false positives, see https://github.com/clj-kondo/clj-kondo/issues/2414
:unexpected-recur {:level :off} ; TODO (cam): I think we just need to tell it how to handle MBQL match and we can enable this?
;;
@@ -58,17 +58,14 @@
:metabase/check-defmulti-arglists {:level :warning}
:metabase/check-me-humanize {:level :warning}
:metabase/defsetting-must-specify-export {:level :warning}
:metabase/defsetting-namespace {:level :warning}
:metabase/mbql-query-first-arg {:level :warning}
:metabase/missing-test-expr-requires-in-cljs {:level :warning}
:metabase/modules {:level :warning} ; more config in `.clj-kondo/config/modules/config.edn`
:metabase/namespace-name {:level :warning}
:metabase/require-shape-checker {:level :warning}
:metabase/test-helpers-use-non-thread-safe-functions {:level :warning} ; more config in `.clj-kondo/config/test-helpers/config.edn`
:metabase/validate-logging {:level :warning}
:metabase/validate-mb-driver-tests {:level :warning}
:metabase/validate-mb-once {:level :warning}
:metabase/discourage-millis-duration {:level :warning}
:metabase/validate-deftest
{:level :warning
@@ -77,13 +74,8 @@
;;
;; TODO -- these should all be made thread safe if possible or renamed so they end in `!`. Then we can remove them
;; from this list because by default anything ending in `!` is considered to be thread-unsafe
;;
;; TODO (Cam 6/19/25) -- move this into the `test-helpers/config.edn` file so they can live with their friends.
:parallel/unsafe
#{java.util.Locale/setDefault
java.lang.System/setProperty
java.lang.System/setProperties
clojure.core/alter-var-root
#{clojure.core/alter-var-root
clojure.core/with-redefs
clojure.core/with-redefs-fn
metabase.actions.test-util/with-actions
@@ -99,9 +91,11 @@
metabase.test.data.users/with-group-for-user
metabase.test.data/with-empty-h2-app-db
metabase.test.util.log/with-log-level
metabase.test.util.misc/with-single-admin-user
metabase.test.util/with-all-users-permission
metabase.test.util/with-discarded-collections-perms-changes
metabase.test.util/with-env-keys-renamed-by
metabase.test.util/with-locale
metabase.test.util/with-model-cleanup
metabase.test.util/with-non-admin-groups-no-root-collection-for-namespace-perms
metabase.test.util/with-non-admin-groups-no-root-collection-perms
@@ -124,10 +118,12 @@
metabase.test/with-fake-inbox
metabase.test/with-group
metabase.test/with-group-for-user
metabase.test/with-locale
metabase.test/with-log-level
metabase.test/with-model-cleanup
metabase.test/with-non-admin-groups-no-root-collection-for-namespace-perms
metabase.test/with-non-admin-groups-no-root-collection-perms
metabase.test/with-single-admin-user
metabase.test/with-temp-env-var-value
metabase.test/with-temp-vals-in-db
metabase.test/with-temporary-raw-setting-values
@@ -173,6 +169,7 @@
metabase.driver.sql-jdbc.execute/execute-prepared-statement!
metabase.notification.models/create-notification!
metabase.pulse.send/send-pulse!
metabase.query-processor.store/store-database!
;; Safe in tests, where we typically use a private, temporary index per thread.
metabase.search.core/reindex!
metabase.util/run-count!
@@ -193,7 +190,6 @@
;; module drivers
:athena
:bigquery-cloud-sdk
:clickhouse
:databricks
:druid
:druid-jdbc
@@ -207,14 +203,13 @@
:sparksql
:sqlite
:sqlserver
:starburst
:vertica}}
;;
;; misc config for built-in linters
;;
:invalid-arity {:skip-args [metabase.lib.util.match/match]} ; TODO (cam): can we fix these?
:invalid-arity {:skip-args [metabase.lib.util.match/match]} ; TODO (cam): can we fix these?
:refer-all {:level :warning, :exclude [clojure.test]}
:unused-referred-var {:exclude {compojure.core [GET DELETE POST PUT]}}
@@ -235,29 +230,30 @@
(metabase.driver.druid.query-processor-test/druid-query-returning-rows)
(metabase.lib.util.match/match)
(metabase.lib.util.match/match-one)
(metabase.lib.util.match/match-lite)
(metabase.lib.util.match/match-lite-recursive)
(metabase.lib.util.match/replace)
(metabase.lib.util.match/replace-in)
(metabase.driver-api.core/match)
(metabase.driver-api.core/match-one)
(metabase.driver-api.core/replace)
(metabase.driver-api.core/replace-in)
(metabase.query-processor.middleware.cache-backend.interface/with-cached-results)
(metabase.util.regex/rx [opt])
(metabase.util/prog1 [<>])
(taoensso.nippy/extend-freeze)
(taoensso.nippy/extend-thaw)]}
;; TODO (Cam 8/22/25) -- what's the point of marking stuff deprecated if we don't have Kondo enforce it? We should
;; remove all of these and add individual ignores as needed.
:deprecated-var
{:exclude
{metabase.driver.sql-jdbc.execute.old-impl/set-timezone-sql {:namespaces ["metabase\\.driver.*"]}
metabase.driver.sql-jdbc.execute/set-time-zone-if-supported! {:namespaces ["metabase\\.driver.*"]}
metabase.driver/current-db-time {:namespaces ["metabase.*"]}
metabase.driver/supports? {:namespaces ["metabase.*"]}
metabase.query-processor.util/normalize-token {:namespaces ["metabase.*"]}}}
{metabase.cmd/dump {:namespaces ["metabase\\.cmd-test" "metabase-enterprise\\.serialization\\.cmd-test"]}
metabase.cmd/load {:namespaces ["metabase\\.cmd-test" "metabase-enterprise\\.serialization\\.cmd-test"]}
metabase.db.data-migrations/data-migrations {:namespaces ["metabase\\.db\\.data-migrations"]}
metabase.db.data-migrations/defmigration {:namespaces ["metabase\\.db\\.data-migrations"]}
metabase.db.data-migrations/run-migration-if-needed! {:namespaces ["metabase\\.db\\.data-migrations"]}
metabase.driver.sql-jdbc.execute.old-impl/set-timezone-sql {:namespaces ["metabase\\.driver.*"]}
metabase.driver.sql-jdbc.execute/set-time-zone-if-supported! {:namespaces ["metabase\\.driver.*"]}
metabase.driver/current-db-time {:namespaces ["metabase.*"]}
metabase.driver/supports? {:namespaces ["metabase.*"]}
metabase.premium-features.premium-features/enable-enhancements? {:namespaces ["metabase\\.models.*"]}
metabase.query-processor.middleware.resolve-joins/maybe-resolve-source-table {:namespaces ["metabase.*"]}
metabase.query-processor.util/normalize-token {:namespaces ["metabase.*"]}
schema.core/both {:namespaces ["metabase.*"]}
schema.core/either {:namespaces ["metabase.*"]}}}
:discouraged-var
{clojure.core/print {:message "Use metabase.util.log instead of clojure.core/print"}
@@ -277,7 +273,6 @@
compojure.core/context {:message "Use metabase.api.util.handlers/route-map-handler instead of compojure.core/context, it is much faster and supports OpenAPI spec generation"}
compojure.core/defroutes {:message "Use metabase.api.util.handlers/defroutes instead of compojure.core/routes, it supports OpenAPI spec generation."}
compojure.core/routes {:message "Use metabase.api.util.handlers/routes instead of compojure.core/routes, it supports OpenAPI spec generation."}
clojurewerkz.quartzite.jobs/defjob {:message "Use metabase.task/defjob instead of clojurewerkz.quartzite.jobs/defjob"}
fipp.edn/pprint {:message "Use metabase.util.log instead of fipp.edn/pprint"}
honeysql.core/call {:message "Use hx/call instead because it is Honey SQL 2 friendly"}
honeysql.core/raw {:message "Use hx/raw instead because it is Honey SQL 2 friendly"}
@@ -290,17 +285,11 @@
me.flowthing.pp/pprint {:message "Use metabase.util.log instead of me.flowthing.pp/pprint"}
medley.core/random-uuid {:message "Use clojure.core/random-uuid instead of medley.core/random-uuid"}
metabase.driver/database-supports? {:message "Use metabase.driver.util/supports? instead of metabase.driver/database-supports?"}
metabase.lib.convert/->legacy-MBQL {:message "Conversion to legacy MBQL is discouraged, please use Lib to manipulate MBQL queries"}
metabase.lib.core/->legacy-MBQL {:message "Conversion to legacy MBQL is discouraged, please use Lib to manipulate MBQL queries"}
metabase.lib.equality/find-column-indexes-for-refs {:message "Use lib.equality/closest-matches-in-metadata or lib.equality/find-closest-matches-for-refs instead of lib.equality/find-column-indexes-for-refs"}
next.jdbc/active-tx? {:message "Use metabase.app-db.core/in-transaction? to see if we are in a transaction"}
toucan2.core/debug {:message "Don't commit code using t2/debug"}
toucan2.tools.debug/debug {:message "Don't commit code using t2/debug"}}
metabase.lib.equality/find-column-indexes-for-refs {:message "Use lib.equality/closest-matches-in-metadata or lib.equality/find-closest-matches-for-refs instead of lib.equality/find-column-indexes-for-refs"}}
:discouraged-namespace
{camel-snake-kebab.core {:message "CSK is not Turkish-safe, use the versions in metabase.util instead."}
cheshire.core {:message "Use metabase.util.json instead of cheshire.core directly."}
clj-time.core {:message "Use java-time.api or metabase.util.date-2 instead of clj-time"}
clojure.tools.logging {:message "Use metabase.util.log instead of clojure.tools.logging directly"}
grouper.core {:message "Use metabase.util.grouper instead of grouper.core"}
honeysql.core {:message "Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}
@@ -309,7 +298,7 @@
honeysql.types {:message "Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}
honeysql.util {:message "Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}}
:discouraged-tag {p {:message "You can use it, but don't commit it."}}
:discouraged-tag {p {:message "You can use it, but don't commit it." }}
:unresolved-var
{:exclude
@@ -317,90 +306,218 @@
:consistent-alias
{:aliases
{buddy.core.hash buddy-hash
cheshire.generate json.generate
clj-http.client http
clj-ldap.client ldap
clj-time.coerce time.coerce
clj-time.format time.format
clojure.core.async a
clojure.core.match clojure.core.match
clojure.data data
clojure.math.combinatorics math.combo
clojure.pprint pprint
clojure.spec.alpha s
clojure.string str
clojure.test t
clojure.tools.build.api b
clojure.tools.logging log
clojure.tools.namespace.dependency ns.deps
clojure.tools.namespace.find ns.find
clojure.tools.namespace.parse ns.parse
colorize.core colorize
environ.core env
hiccup.core hiccup
hiccup.util hiccup.util
honey.sql sql
honey.sql.helpers sql.helpers
java-time.api t
macaw.core macaw
malli.core mc
malli.error me
malli.experimental mx
malli.generator mg
malli.provider mp
malli.transform mtx
malli.util mut
medley.core m
metabase-enterprise.database-routing.core database-routing
metabase.analyze.classifiers.core classifiers
metabase.channel.core channel
metabase.channel.render.core channel.render
metabase.cloud-migration.core cloud-migration
metabase.eid-translation.core eid-translation
metabase.login-history.core login-history
metabase.model-persistence.core model-persistence
metabase.notification.core notification
metabase.notification.payload.core notification.payload
metabase.permissions.core perms
metabase.premium-features.core premium-features
metabase.public-sharing.core public-sharing
metabase.pulse.core pulse
metabase.query-permissions.core query-perms
metabase.request.core request
metabase.revisions.core revisions
metabase.server.core server
metabase.settings.core setting
metabase.sso.core sso
metabase.timeline.core timeline
metabase.util.cron u.cron
metabase.util.currency currency
metabase.util.date-2 u.date
metabase.util.date-2.common u.date.common
metabase.util.date-2.parse u.date.parse
metabase.util.embed embed
metabase.util.encryption encryption
metabase.util.encryption-test encryption-test
metabase.util.files u.files
metabase.util.grouper grouper
metabase.util.honey-sql-2 h2x
metabase.util.i18n i18n
metabase.util.i18n.impl i18n.impl
metabase.util.jvm u.jvm
metabase.util.log log
metabase.util.macros u.macros
metabase.util.malli mu
metabase.util.malli.schema ms
metabase.util.markdown markdown
metabase.util.password u.password
metabase.util.ui-logic ui-logic
metabase.xrays.core xrays
monger.collection mcoll
ring.mock.request ring.mock
ring.util.codec codec
ring.util.response response
saml20-clj.core saml
toucan2.core t2}}}
{buddy.core.hash buddy-hash
cheshire.generate json.generate
clojure.tools.logging log
clj-http.client http
clj-ldap.client ldap
clj-time.coerce time.coerce
clj-time.format time.format
clojure.core.async a
clojure.core.match clojure.core.match ; Prefer :refer [match]
clojure.data data
clojure.math.combinatorics math.combo
clojure.pprint pprint
clojure.spec.alpha s
clojure.string str
clojure.test t
clojure.tools.build.api b
clojure.tools.namespace.dependency ns.deps
clojure.tools.namespace.find ns.find
clojure.tools.namespace.parse ns.parse
colorize.core colorize
environ.core env
hiccup.core hiccup
hiccup.util hiccup.util
honey.sql sql
honey.sql.helpers sql.helpers
java-time.api t
macaw.core macaw
malli.core mc
malli.error me
malli.experimental mx
malli.generator mg
malli.provider mp
malli.transform mtx
malli.util mut
medley.core m
metabase-enterprise.audit-app.pages.common common
metabase-enterprise.sandbox.api.table table
metabase-enterprise.sandbox.api.util sandbox.api.util
metabase-enterprise.test met
metabase.analytics.sdk sdk
metabase.analytics.stats stats
metabase.analysis.native-query-analyzer nqa
metabase.analysis.native-query-analyzer.parameter-substitution nqa.sub
metabase.analyze.query-results qr
metabase.analyze.classifiers.category classifiers.category
metabase.analyze.classifiers.core classifiers
metabase.analyze.classifiers.name classifiers.name
metabase.analyze.classifiers.no-preview-display classifiers.no-preview-display
metabase.analyze.classifiers.text-fingerprint classifiers.text-fingerprint
metabase.analyze.fingerprint.fingerprinters fingerprinters
metabase.analyze.fingerprint.insights insights
metabase.analyze.fingerprint.schema fingerprint.schema
metabase.api.common api
metabase.channel.core channel
metabase.channel.render.core channel.render
metabase.cloud-migration.core cloud-migration
metabase.cmd.copy.h2 copy.h2
metabase.config config
metabase.config.file config.file
metabase.connection-pool connection-pool
metabase.db.connection mdb.connection
metabase.db.jdbc-protocols mdb.jdbc-protocols
metabase.db.metadata-queries metadata-queries
metabase.db.spec mdb.spec
metabase.db.query mdb.query
metabase.driver.common.parameters params
metabase.driver.common.parameters.dates params.dates
metabase.driver.common.parameters.operators params.ops
metabase.driver.common.parameters.parse params.parse
metabase.driver.common.parameters.values params.values
metabase.driver.druid.client druid.client
metabase.driver.druid.execute druid.execute
metabase.driver.druid.js druid.js
metabase.driver.druid.query-processor druid.qp
metabase.driver.druid.sync druid.sync
metabase.driver.impl driver.impl
metabase.driver.mongo.execute mongo.execute
metabase.driver.mongo.parameters mongo.params
metabase.driver.mongo.query-processor mongo.qp
metabase.driver.mongo.util mongo.util
metabase.driver.sql driver.sql
metabase.driver.sql-jdbc.common sql-jdbc.common
metabase.driver.sql-jdbc.connection sql-jdbc.conn
metabase.driver.sql-jdbc.execute sql-jdbc.execute
metabase.driver.sql-jdbc.execute.diagnostic sql-jdbc.execute.diagnostic
metabase.driver.sql-jdbc.execute.legacy-impl sql-jdbc.legacy
metabase.driver.sql-jdbc.execute.old-impl sql-jdbc.execute.old
metabase.driver.sql-jdbc.sync sql-jdbc.sync
metabase.driver.sql-jdbc.sync.common sql-jdbc.sync.common
metabase.driver.sql-jdbc.sync.describe-database sql-jdbc.describe-database
metabase.driver.sql-jdbc.sync.describe-table sql-jdbc.describe-table
metabase.driver.sql-jdbc.sync.interface sql-jdbc.sync.interface
metabase.driver.sql-jdbc.test-util sql-jdbc.tu
metabase.driver.sql.parameters.substitute sql.params.substitute
metabase.driver.sql.parameters.substitution sql.params.substitution
metabase.eid-translation.core eid-translation
metabase.formatter formatter
metabase.http-client client
metabase.legacy-mbql.normalize mbql.normalize
metabase.legacy-mbql.schema mbql.s
metabase.legacy-mbql.util mbql.u
metabase.lib.util lib.util
metabase.lib.util.match lib.util.match
metabase.login-history.core login-history
metabase.model-persistence.core model-persistence
metabase.models.dependency dependency
metabase.models.interface mi
metabase.models.setting setting
metabase.notification.core notification
metabase.notification.payload.core notification.payload
metabase.notification.test-util notification.tu
metabase.permissions.core perms
metabase.plugins.initialize plugins.init
metabase.public-settings public-settings
metabase.public-sharing.core public-sharing
metabase.premium-features.core premium-features
metabase.pulse.core pulse
metabase.pulse.render render
metabase.pulse.render.body body
metabase.pulse.render.style style
metabase.pulse.test-util pulse.test-util
metabase.pulse.parameters pulse-params
metabase.query-processor-test qp.test
metabase.query-processor.context qp.context
metabase.query-processor.error-type qp.error-type
metabase.query-processor.interface qp.i
metabase.query-processor.middleware.add-dimension-projections qp.add-dimension-projections
metabase.query-processor.middleware.add-implicit-clauses qp.add-implicit-clauses
metabase.query-processor.middleware.add-implicit-joins qp.add-implicit-joins
metabase.query-processor.middleware.auto-bucket-datetimes qp.auto-bucket-datetimes
metabase.query-processor.middleware.constraints qp.constraints
metabase.query-processor.middleware.cumulative-aggregations qp.cumulative-aggregations
metabase.query-processor.middleware.parameters.mbql qp.mbql
metabase.query-processor.middleware.parameters.native qp.native
metabase.query-processor.middleware.permissions qp.perms
metabase.query-processor.middleware.pre-alias-aggregations qp.pre-alias-aggregations
metabase.query-processor.middleware.resolve-fields qp.resolve-fields
metabase.query-processor.middleware.resolve-referenced qp.resolve-referenced
metabase.query-processor.middleware.resolve-source-table qp.resolve-source-table
metabase.query-processor.middleware.wrap-value-literals qp.wrap-value-literals
metabase.query-processor.pivot qp.pivot
metabase.query-processor.streaming qp.streaming
metabase.query-processor.streaming.interface qp.si
metabase.query-processor.streaming.xlsx qp.xlsx
metabase.query-processor.timezone qp.timezone
metabase.query-processor.util qp.util
metabase.request.core request
metabase.revisions.core revisions
metabase.search.config search.config
metabase.search.filter search.filter
metabase.search.in-place.filter search.in-place.filter
metabase.search.in-place.util search.util
metabase.search.in-place.scoring scoring
metabase.server.core server
metabase.server.protocols server.protocols
metabase.sso.core sso
metabase.sync.analyze.fingerprint sync.fingerprint
metabase.sync.field-values sync.field-values
metabase.sync.interface i
metabase.sync.schedules sync.schedules
metabase.sync.sync-metadata.crufty crufty
metabase.sync.sync-metadata.fields.our-metadata fields.our-metadata
metabase.sync.util sync-util
metabase.sync.util-test sync.util-test
metabase.test mt
metabase.test.data.dataset-definitions defs
metabase.test.data.env.impl tx.env.impl
metabase.test.data.impl data.impl
metabase.test.data.users test.users
metabase.test.domain-entities test.de
metabase.test.mock.util mock.util
metabase.test.sync test.sync
metabase.test.redefs test.redefs
metabase.test.util.js test.js
metabase.test.util.timezone test.tz
metabase.timeline.core timeline
metabase.timeseries-query-processor-test.util tqpt
metabase.util.cron u.cron
metabase.util.currency currency
metabase.util.date-2 u.date
metabase.util.date-2.common u.date.common
metabase.util.date-2.parse u.date.parse
metabase.util.embed embed
metabase.util.encryption encryption
metabase.util.encryption-test encryption-test
metabase.util.files u.files
metabase.util.grouper grouper
metabase.util.honey-sql-2 h2x
metabase.util.i18n i18n
metabase.util.i18n.impl i18n.impl
metabase.util.jvm u.jvm
metabase.util.log log
metabase.util.macros u.macros
metabase.util.malli mu
metabase.util.malli.schema ms
metabase.util.markdown markdown
metabase.util.password u.password
metabase.util.schema su
metabase.util.ui-logic ui-logic
metabase.util.urls urls
metabase.xrays.core xrays
metabuild-common.core u
metabuild-common.output out
metabuild-common.shell shell
monger.collection mcoll
ring.mock.request ring.mock
ring.util.codec codec
ring.util.response response
ring.adapter.jetty9.servlet servlet
saml20-clj.core saml
toucan.db db
toucan.models models}}}
:lint-as
{cljs.cache/defcache clojure.core/deftype
@@ -409,43 +526,37 @@
clojure.test.check.clojure-test/defspec clojure.test/deftest
clojure.test.check.properties/for-all clojure.core/let
clojurewerkz.quartzite.jobs/defjob clojure.core/defn
com.climate.claypoole/with-shutdown! clojure.core/let
instaparse.core/defparser clojure.core/def
metabase-enterprise.action-v2.test-util/with-test-tables! clojure.core/let
metabase-enterprise.content-translation.routes-test/with-new-secret-key! clojure.core/fn
metabase-enterprise.serialization.test-util/with-dbs clojure.core/fn
metabase-enterprise.serialization.test-util/with-random-dump-dir clojure.core/let
metabase-enterprise.transforms.api-test/with-transform-tags clojure.core/let
metabase-enterprise.transforms.api-test/with-worker-runs clojure.core/let
metabase-enterprise.transforms.test-util/with-transform-cleanup! clojure.core/let
metabase-enterprise.transforms.execute/with-transform-lifecycle clojure.core/let
metabase.actions.test-util/with-actions clojure.core/let
metabase.api.common/let-404 clojure.core/let
metabase.app-db.custom-migrations.util/with-temp-scheduler! clojure.core/fn
metabase.app-db.custom-migrations/define-migration clj-kondo.lint-as/def-catch-all
metabase.app-db.custom-migrations/define-reversible-migration clj-kondo.lint-as/def-catch-all
metabase.app-db.liquibase/with-liquibase clojure.core/let
metabase.app-db.schema-migrations-test.impl/with-temp-empty-app-db clojure.core/let
metabase.channel.impl.http-test/with-captured-http-requests clojure.core/fn
metabase.channel.impl.http-test/with-server clojure.core/let
metabase.collections.models.collection-test/with-collection-in-location clojure.core/let
metabase.dashboard-subscription-test/with-dashboard-sub-for-card clojure.core/let
metabase.db.custom-migrations/define-migration clj-kondo.lint-as/def-catch-all
metabase.db.custom-migrations/define-reversible-migration clj-kondo.lint-as/def-catch-all
metabase.db.custom-migrations.util/with-temp-scheduler! clojure.core/fn
metabase.db.liquibase/with-liquibase clojure.core/let
metabase.db.schema-migrations-test.impl/with-temp-empty-app-db clojure.core/let
metabase.driver.mongo.connection/with-mongo-client clojure.core/let
metabase.driver.mongo.connection/with-mongo-database clojure.core/let
metabase.driver.mongo.query-processor/mongo-let clojure.core/let
metabase.driver.sql-jdbc.actions/with-jdbc-transaction clojure.core/let
metabase.driver.sql-jdbc.connection.ssh-tunnel/with-ssh-tunnel clojure.core/let
metabase.driver.sql-jdbc.connection/with-connection-spec-for-testing-connection clojure.core/let
metabase.driver.sql-jdbc.execute.diagnostic/capturing-diagnostic-info clojure.core/fn
metabase.driver-api.core/defsetting hooks.metabase.settings.models.setting/defsetting
metabase.models.collection-test/with-collection-in-location clojure.core/let
metabase.models.json-migration/def-json-migration clj-kondo.lint-as/def-catch-all
metabase.models.setting/defsetting clj-kondo.lint-as/def-catch-all
metabase.models.setting.multi-setting/define-multi-setting clojure.core/def
metabase.models.util.spec-update/define-spec clojure.core/def
metabase.notification.test-util/with-temp-notification clojure.core/let
metabase.notification.test-util/with-card-notification clojure.core/let
metabase.notification.test-util/with-system-event-notification! clojure.core/let
metabase.notification.test-util/with-temp-notification clojure.core/let
metabase.public-settings.premium-features/defenterprise-schema clj-kondo.lint-as/def-catch-all
metabase.public-settings.premium-features/define-premium-feature clojure.core/def
metabase.premium-features.core/defenterprise-schema clj-kondo.lint-as/def-catch-all
metabase.premium-features.core/define-premium-feature clojure.core/def
metabase.premium-features.defenterprise/defenterprise-schema clj-kondo.lint-as/def-catch-all
metabase.premium-features.settings/define-premium-feature clojure.core/def
metabase.pulse.dashboard-subscription-test/with-dashboard-sub-for-card clojure.core/let
metabase.pulse.send-test/with-pulse-for-card clojure.core/let
metabase.query-processor-test.pipeline-queries-test/pmbql-query clojure.core/->
metabase.query-processor-test.pipeline-queries-test/run-pmbql-query clojure.core/->
@@ -454,30 +565,23 @@
metabase.query-processor.middleware.process-userland-query-test/with-query-execution! clojure.core/let
metabase.query-processor.setup/with-qp-setup clojure.core/let
metabase.search.api-test/do-test-users clojure.core/let
metabase.settings.core/define-multi-setting clojure.core/def
metabase.settings.core/defsetting clj-kondo.lint-as/def-catch-all
metabase.settings.models.setting.multi-setting/define-multi-setting clojure.core/def
metabase.settings.models.setting/defsetting clj-kondo.lint-as/def-catch-all
metabase.sso.ldap/with-ldap-connection clojure.core/fn
metabase.sync.util/sum-for clojure.core/for
metabase.sync.util/with-emoji-progress-bar clojure.core/let
metabase.task.core/defjob clojure.core/defn
metabase.task.impl/defjob clojure.core/defn
metabase.test.data.interface/defdataset clojure.core/def
metabase.test.data.interface/defdataset-edn clojure.core/def
metabase.test.gentest/defgentest clojure.test/deftest
metabase.test/defdataset clojure.core/def
metabase.test/let-url clojure.core/let
metabase.test/with-actions clojure.core/let
metabase.test/with-grouper-batches! clojure.core/fn
metabase.test/with-open-channels clojure.core/let
metabase.test/with-single-admin-user! clojure.core/fn
metabase.test/with-single-admin-user clojure.core/fn
metabase.test/with-temp-dir clojure.core/let
metabase.test/with-temp-empty-app-db clojure.core/let
metabase.test/with-temp-file clojure.core/let
metabase.test/with-user-in-groups clojure.core/let
metabase.upload.impl-test/with-upload-table! clojure.core/let
metabase.util/for-map clojure.core/for
metabase.upload-test/with-upload-table! clojure.core/let
metabase.util/with-timer-ms clojure.core/fn
metabase.util.files/with-open-path-to-resource clojure.core/let
metabase.util.malli.defn/defn schema.core/defn
metabase.util.malli.defn/defn- schema.core/defn
@@ -487,7 +591,7 @@
metabase.util.malli/defn- schema.core/defn
metabase.util.malli/fn schema.core/fn
metabase.util.namespaces/import-fns potemkin/import-vars
metabase.util/with-timer-ms clojure.core/fn
metabase.util.ssh/with-ssh-tunnel clojure.core/let
metabase.xrays.domain-entities.malli/defn schema.core/defn
metabuild-common.core/with-open-jar-file-system clojure.core/let
metabuild-common.files/with-open-jar-file-system clojure.core/let
@@ -510,7 +614,6 @@
clojure.core/ns hooks.clojure.core.ns/lint-ns
clojure.core/require hooks.clojure.core.require/lint-require
clojure.core/requiring-resolve hooks.clojure.core.require/lint-requiring-resolve
clojure.core/- hooks.metabase.util.timing/discourage-millis-duration
clojure.test/deftest hooks.clojure.test/deftest
clojure.test/is hooks.clojure.test/is
clojure.test/use-fixtures hooks.clojure.test/use-fixtures
@@ -518,47 +621,49 @@
metabase-enterprise.advanced-permissions.models.permissions.application-permissions-test/with-new-group-and-current-graph hooks.common/with-two-top-level-bindings
metabase-enterprise.audit-app.pages-test/with-temp-objects hooks.common/with-one-binding
metabase-enterprise.cache.config-test/with-temp-persist-models hooks.common/with-seven-bindings
metabase-enterprise.semantic-search.db.connection/with-migrate-tx hooks.common/let-one-with-optional-value
metabase-enterprise.serialization.test-util/with-temp-dpc hooks.toucan2.tools.with-temp/with-temp
metabase.api.card-test/with-card-param-values-fixtures hooks.common/let-one-with-optional-value
metabase.api.card-test/with-temp-native-card! hooks.common/with-two-bindings
metabase.api.card-test/with-temp-native-card-with-params! hooks.common/with-two-bindings
metabase.api.collection-test/with-french-user-and-personal-collection! hooks.common/with-two-top-level-bindings
metabase.api.common.internal-test/with-jetty-server hooks.common/with-one-binding
metabase.api.dashboard-test/with-chain-filter-fixtures hooks.common/let-one-with-optional-value
metabase.api.dashboard-test/with-simple-dashboard-with-tabs hooks.common/with-one-binding
metabase.api.embed-test/do-response-formats hooks.common/with-two-bindings
metabase.api.embed-test/with-chain-filter-fixtures! hooks.common/let-one-with-optional-value
metabase.api.embed-test/with-temp-card hooks.common/let-one-with-optional-value
metabase.api.embed-test/with-temp-dashcard hooks.common/let-one-with-optional-value
metabase.api.macros/defendpoint hooks.metabase.api.macros/defendpoint
metabase.app-db.schema-migrations-test.impl/test-migrations hooks.metabase.app-db.schema-migrations-test.impl/test-migrations
metabase.channel.render.js.svg/with-static-viz-context hooks.common/with-one-top-level-binding
metabase.api.user-test/with-temp-user-email! hooks.common/with-one-binding
metabase.channel.template.handlebars-helper/defhelper hooks.metabase.channel.template.handlebars-helper/defhelper
metabase.channel.template.handlebars-test/with-temp-template! hooks.common/with-vec-first-binding
metabase.classloader.core/require hooks.clojure.core.require/lint-require
metabase.classloader.impl/require hooks.clojure.core.require/lint-require
metabase.collections.api-test/with-french-user-and-personal-collection! hooks.common/with-two-top-level-bindings
metabase.collections.models.collection-test/with-collection-hierarchy! hooks.common/let-one-with-optional-value
metabase.collections.models.collection-test/with-personal-and-impersonal-collections hooks.common/with-two-bindings
metabase.dashboards.api-test/with-chain-filter-fixtures hooks.common/let-one-with-optional-value
metabase.dashboards.api-test/with-simple-dashboard-with-tabs hooks.common/with-one-binding
metabase.dashboards.models.dashboard-tab-test/with-dashtab-in-personal-collection hooks.common/with-one-top-level-binding
metabase.dashboards.models.dashboard-test/with-dash-in-collection! hooks.common/with-three-bindings
metabase.dashboard-subscription-test/with-link-card-fixture-for-dashboard hooks.common/let-second
metabase.db.schema-migrations-test.impl/test-migrations hooks.metabase.db.schema-migrations-test.impl/test-migrations
metabase.driver.bigquery-cloud-sdk-test/calculate-bird-scarcity hooks.metabase.query-processor-test.expressions-test/calculate-bird-scarcity
metabase.driver.mongo-test/with-describe-table-for-sample hooks.metabase.driver.mongo-test/with-describe-table-for-sample
metabase.driver.oracle-test/with-temp-user hooks.common/let-one-with-optional-value
metabase.driver/register! hooks.metabase.driver/register!
metabase.driver-api/defenterprise hooks.metabase.premium-features.defenterprise/defenterprise
metabase.embedding.api.embed-test/do-response-formats hooks.common/with-two-bindings
metabase.embedding.api.embed-test/with-chain-filter-fixtures! hooks.common/let-one-with-optional-value
metabase.embedding.api.embed-test/with-temp-card hooks.common/let-one-with-optional-value
metabase.embedding.api.embed-test/with-temp-dashcard hooks.common/let-one-with-optional-value
metabase.legacy-mbql.schema.macros/defclause hooks.metabase.legacy-mbql.schemas.macros/defclause
metabase.lib.schema.mbql-clause/define-catn-mbql-clause hooks.metabase.lib.schema.mbql-clause/define-mbql-clause
metabase.lib.schema.mbql-clause/define-mbql-clause hooks.metabase.lib.schema.mbql-clause/define-mbql-clause
metabase.lib.schema.mbql-clause/define-tuple-mbql-clause hooks.metabase.lib.schema.mbql-clause/define-mbql-clause
metabase.lib.test-util.macros/$ids hooks.metabase.test.data/$ids
metabase.lib.test-util.macros/mbql-query hooks.metabase.test.data/mbql-query
metabase.lib.test-util.macros/mbql-5-query hooks.metabase.test.data/mbql-query
metabase.lib.test-util.macros/with-testing-against-standard-queries hooks.metabase.lib.test-util.macros/with-testing-against-standard-queries
metabase.model-persistence.api-test/with-persistence-setup! hooks.common/with-one-top-level-binding
metabase.model-persistence.api-test/with-setup! hooks.common/with-one-top-level-binding
metabase.model-persistence.test-util/with-persistence-enabled! hooks.metabase.model-persistence.test-util/with-persistence-enabled!
metabase.models.collection-test/with-collection-hierarchy! hooks.common/let-one-with-optional-value
metabase.models.collection-test/with-personal-and-impersonal-collections hooks.common/with-two-bindings
metabase.models.dashboard-tab-test/with-dashtab-in-personal-collection hooks.common/with-one-top-level-binding
metabase.models.dashboard-test/with-dash-in-collection! hooks.common/with-three-bindings
metabase.models.interface/define-batched-hydration-method hooks.metabase.models.interface/define-hydration-method
metabase.models.interface/define-simple-hydration-method hooks.metabase.models.interface/define-hydration-method
metabase.models.setting.multi-setting/define-multi-setting hooks.metabase.models.setting/define-multi-setting
metabase.models.setting.multi-setting/define-multi-setting-impl hooks.metabase.models.setting/define-multi-setting-impl
metabase.models.setting/defsetting hooks.metabase.models.setting/defsetting
metabase.permissions.core/allow-changing-all-users-group-members hooks.common/do*
metabase.permissions.core/disable-perms-cache hooks.common/do*
metabase.plugins.classloader/require hooks.clojure.core.require/lint-require
metabase.premium-features.core/defenterprise hooks.metabase.premium-features.defenterprise/defenterprise
metabase.premium-features.defenterprise/defenterprise hooks.metabase.premium-features.defenterprise/defenterprise
metabase.public-sharing.api-test/with-required-param-card! hooks.common/with-one-binding
@@ -566,29 +671,19 @@
metabase.public-sharing.api-test/with-temp-public-dashboard hooks.common/let-one-with-optional-value
metabase.public-sharing.api-test/with-temp-public-dashboard-and-card hooks.common/with-three-bindings
metabase.pulse.api.alert-test/with-alert-in-collection! hooks.common/with-four-bindings
metabase.pulse.dashboard-subscription-test/with-link-card-fixture-for-dashboard hooks.common/let-second
metabase.pulse.models.pulse-test/with-dashboard-subscription-in-collection! hooks.common/with-four-bindings
metabase.pulse.models.pulse-test/with-pulse-in-collection! hooks.common/with-four-bindings
metabase.pulse.test-util/checkins-query-card hooks.metabase.test.data/$ids
metabase.queries.api.card-test/with-card-param-values-fixtures hooks.common/let-one-with-optional-value
metabase.queries.api.card-test/with-temp-native-card! hooks.common/with-two-bindings
metabase.queries.api.card-test/with-temp-native-card-with-params! hooks.common/with-two-bindings
metabase.query-processor-test.expressions-test/calculate-bird-scarcity hooks.metabase.query-processor-test.expressions-test/calculate-bird-scarcity
metabase.query-processor-test.filter-test/count-with-filter-clause hooks.metabase.test.data/$ids
metabase.query-processor.middleware.cache-test/with-mock-cache! hooks.common/with-two-bindings
metabase.request.core/with-current-request hooks.common/with-used-first-arg
metabase.request.core/with-current-user hooks.common/with-used-first-arg
metabase.request.core/with-limit-and-offset hooks.common/with-used-first-two-args
metabase.sample-data.impl-test/with-temp-sample-database-db hooks.common/with-one-binding
metabase.sample-data-test/with-temp-sample-database-db hooks.common/with-one-binding
metabase.search.api-test/with-search-items-in-collection hooks.metabase.search.api-test/with-search-items-in-collection
metabase.search.api-test/with-search-items-in-root-collection hooks.common/do*
metabase.server.streaming-response-test/with-start-execution-chan hooks.common/with-one-binding
metabase.settings.core/define-multi-setting hooks.metabase.settings.models.setting/define-multi-setting
metabase.settings.core/define-multi-setting-impl hooks.metabase.settings.models.setting/define-multi-setting-impl
metabase.settings.core/defsetting hooks.metabase.settings.models.setting/defsetting
metabase.settings.models.setting.multi-setting/define-multi-setting hooks.metabase.settings.models.setting/define-multi-setting
metabase.settings.models.setting.multi-setting/define-multi-setting-impl hooks.metabase.settings.models.setting/define-multi-setting-impl
metabase.settings.models.setting/defsetting hooks.metabase.settings.models.setting/defsetting
metabase.stale-test/with-stale-items hooks.toucan2.tools.with-temp/with-temp
metabase.test.data.datasets/test-driver hooks.metabase.test.data.datasets/test-driver
metabase.test.data.datasets/test-drivers hooks.metabase.test.data.datasets/test-drivers
@@ -597,9 +692,8 @@
metabase.test.data/dataset hooks.metabase.test.data/dataset
metabase.test.data/mbql-query hooks.metabase.test.data/mbql-query
metabase.test.data/run-mbql-query hooks.metabase.test.data/mbql-query
metabase.test.gentest/iterate hooks.metabase.test.gentest/iterate*
metabase.test.gentest/with-gentest hooks.metabase.test.gentest/with-gentest
metabase.test.util.async/with-open-channels hooks.common/let-with-optional-value-for-last-binding
metabase.test.util.generators.jvm/with-random-cards hooks.metabase.test.util.generators.jvm/with-random-cards
metabase.test.util.log/with-log-level hooks.common/with-ignored-first-arg
metabase.test.util/discard-setting-changes hooks.common/with-ignored-first-arg
metabase.test.util/with-column-remappings hooks.common/with-ignored-first-arg
@@ -621,15 +715,14 @@
metabase.test/with-log-messages-for-level hooks.metabase.util.log.capture/with-log-messages-for-level
metabase.test/with-non-admin-groups-no-root-collection-perms hooks.common/do*
metabase.test/with-persistence-enabled! hooks.metabase.model-persistence.test-util/with-persistence-enabled!
metabase.test/with-prometheus-system! hooks.common/with-two-bindings
metabase.test/with-temp hooks.toucan2.tools.with-temp/with-temp
metabase.test/with-temp-file hooks.metabase.test.util/with-temp-file
metabase.test/with-prometheus-system! hooks.common/with-two-bindings
metabase.test/with-temporary-setting-values hooks.metabase.test.util/with-temporary-setting-values
metabase.users.api-test/with-temp-user-email! hooks.common/with-one-binding
metabase.util.i18n/deferred-trs hooks.metabase.util.i18n/deferred-trs
metabase.util.i18n/tru hooks.metabase.util.i18n/tru
metabase.util.i18n/deferred-tru hooks.metabase.util.i18n/deferred-tru
metabase.util.i18n/trs hooks.metabase.util.i18n/trs
metabase.util.i18n/tru hooks.metabase.util.i18n/tru
metabase.util.i18n/deferred-trs hooks.metabase.util.i18n/deferred-trs
metabase.util.log.capture/with-log-messages-for-level hooks.metabase.util.log.capture/with-log-messages-for-level
metabase.util.log/debug hooks.metabase.util.log/info
metabase.util.log/debugf hooks.metabase.util.log/infof
@@ -649,44 +742,41 @@
metabase.xrays.api.automagic-dashboards-test/with-indexed-model! hooks.metabase.xrays.api.automagic-dashboards-test/with-indexed-model!}
:macroexpand
{clojurewerkz.quartzite.jobs/build macros.quartz/build-job
clojurewerkz.quartzite.schedule.calendar-interval/build macros.quartz/build-job
clojurewerkz.quartzite.schedule.cron/schedule macros.quartz/schedule
clojurewerkz.quartzite.schedule.simple/schedule macros.quartz/simple-schedule
clojurewerkz.quartzite.triggers/build macros.quartz/build-trigger
metabase-enterprise.database-routing.e2e-test/with-temp-dbs! macros.metabase-enterprise.database-routing.e2e-test/with-temp-dbs!
metabase-enterprise.impersonation.util-test/with-impersonations! macros.metabase-enterprise.impersonation.util-test/with-impersonations!
metabase-enterprise.query-reference-validation.api-test/with-test-setup! macros.metabase-enterprise.query-reference-validation.api-test/with-test-setup!
metabase-enterprise.sandbox.test-util/with-gtaps! macros.metabase-enterprise.sandbox.test-util/with-gtaps!
metabase-enterprise.sandbox.test-util/with-gtaps-for-user! macros.metabase-enterprise.sandbox.test-util/with-gtaps!
metabase-enterprise.serialization.test-util/with-world macros.metabase-enterprise.serialization.test-util/with-world
metabase-enterprise.test/with-gtaps! macros.metabase-enterprise.sandbox.test-util/with-gtaps!
metabase-enterprise.test/with-gtaps-for-user! macros.metabase-enterprise.sandbox.test-util/with-gtaps!
metabase.collections.api-test/with-collection-hierarchy! macros.metabase.collections.api-test/with-collection-hierarchy!
metabase.collections.api-test/with-some-children-of-collection! macros.metabase.collections.api-test/with-some-children-of-collection!
metabase.embedding.api.embed-test/with-embedding-enabled-and-temp-card-referencing! macros.metabase.embedding.api.embed-test/with-embedding-enabled-and-temp-card-referencing!
metabase.embedding.api.embed-test/with-embedding-enabled-and-temp-dashcard-referencing! macros.metabase.embedding.api.embed-test/with-embedding-enabled-and-temp-dashcard-referencing!
metabase.lib.common/defop macros.metabase.lib.common/defop
metabase.lib.filter/deffilter macros.metabase.lib.filter/deffilter
metabase.parameters.chain-filter-test/chain-filter macros.metabase.parameters.chain-filter-test/chain-filter
metabase.parameters.chain-filter-test/chain-filter-search macros.metabase.parameters.chain-filter-test/chain-filter
metabase.public-sharing.api-test/with-sharing-enabled-and-temp-card-referencing! macros.metabase.public-sharing.api-test/with-sharing-enabled-and-temp-card-referencing!
metabase.public-sharing.api-test/with-sharing-enabled-and-temp-dashcard-referencing! macros.metabase.public-sharing.api-test/with-sharing-enabled-and-temp-dashcard-referencing!
metabase.queries.api.card-test/with-ordered-items macros.metabase.queries.api.card-test/with-ordered-items
metabase.query-analysis.models.query-analysis-test/with-test-setup macros.metabase.query-analysis.models.query-analysis-test/with-test-setup
metabase.query-analysis.task.test-setup/with-test-setup! macros.metabase.query-analysis.task.test-setup/with-test-setup!
metabase.query-processor.streaming/streaming-response macros.metabase.query-processor.streaming/streaming-response
metabase.server.statistics-handler-test/with-server macros.metabase.server.statistics-handler-test/with-server
metabase.test.data.users/with-group-for-user macros.metabase.test.data.users/with-group-for-user
metabase.test.util/with-temp-env-var-value! macros.metabase.test.util/with-temp-env-var-value!
metabase.test.util/with-temporary-raw-setting-values macros.metabase.test.util/with-temporary-raw-setting-values
metabase.test/with-group-for-user macros.metabase.test.data.users/with-group-for-user
metabase.test/with-temp-env-var-value! macros.metabase.test.util/with-temp-env-var-value!
metabase.test/with-temporary-raw-setting-values macros.metabase.test.util/with-temporary-raw-setting-values
metabase.users.models.user-test/with-groups! macros.metabase.users.models.user-test/with-groups!
metabase.util.namespaces/import-fn macros.metabase.util.namespaces/import-fn
metabase.xrays.domain-entities.malli/define-getters-and-setters macros.metabase.xrays.domain-entities.malli/define-getters-and-setters
metabase.xrays.related-test/with-world macros.metabase.xrays.related-test/with-world}}
{clojurewerkz.quartzite.jobs/build macros.quartz/build-job
clojurewerkz.quartzite.schedule.cron/schedule macros.quartz/schedule
clojurewerkz.quartzite.schedule.simple/schedule macros.quartz/simple-schedule
clojurewerkz.quartzite.triggers/build macros.quartz/build-trigger
metabase-enterprise.impersonation.util-test/with-impersonations! macros.metabase-enterprise.impersonation.util-test/with-impersonations!
metabase-enterprise.query-reference-validation.api-test/with-test-setup! macros.metabase-enterprise.query-reference-validation.api-test/with-test-setup!
metabase-enterprise.sandbox.test-util/with-gtaps! macros.metabase-enterprise.sandbox.test-util/with-gtaps!
metabase-enterprise.sandbox.test-util/with-gtaps-for-user! macros.metabase-enterprise.sandbox.test-util/with-gtaps!
metabase-enterprise.serialization.test-util/with-world macros.metabase-enterprise.serialization.test-util/with-world
metabase-enterprise.test/with-gtaps! macros.metabase-enterprise.sandbox.test-util/with-gtaps!
metabase-enterprise.test/with-gtaps-for-user! macros.metabase-enterprise.sandbox.test-util/with-gtaps!
metabase.api.card-test/with-ordered-items macros.metabase.api.card-test/with-ordered-items
metabase.api.collection-test/with-collection-hierarchy! macros.metabase.api.collection-test/with-collection-hierarchy!
metabase.api.collection-test/with-some-children-of-collection! macros.metabase.api.collection-test/with-some-children-of-collection!
metabase.api.embed-test/with-embedding-enabled-and-temp-card-referencing! macros.metabase.api.embed-test/with-embedding-enabled-and-temp-card-referencing!
metabase.api.embed-test/with-embedding-enabled-and-temp-dashcard-referencing! macros.metabase.api.embed-test/with-embedding-enabled-and-temp-dashcard-referencing!
metabase.lib.common/defop macros.metabase.lib.common/defop
metabase.lib.filter/deffilter macros.metabase.lib.filter/deffilter
metabase.models.params.chain-filter-test/chain-filter macros.metabase.models.params.chain-filter-test/chain-filter
metabase.models.params.chain-filter-test/chain-filter-search macros.metabase.models.params.chain-filter-test/chain-filter
metabase.models.query-analysis-test/with-test-setup macros.metabase.models.query-analysis-test/with-test-setup
metabase.models.user-test/with-groups macros.metabase.models.user-test/with-groups
metabase.public-sharing.api-test/with-sharing-enabled-and-temp-card-referencing! macros.metabase.public-sharing.api-test/with-sharing-enabled-and-temp-card-referencing!
metabase.public-sharing.api-test/with-sharing-enabled-and-temp-dashcard-referencing! macros.metabase.public-sharing.api-test/with-sharing-enabled-and-temp-dashcard-referencing!
metabase.query-analysis.task.test-setup/with-test-setup! macros.metabase.query-analysis.task.test-setup/with-test-setup!
metabase.query-processor.streaming/streaming-response macros.metabase.query-processor.streaming/streaming-response
metabase.test.data.users/with-group-for-user macros.metabase.test.data.users/with-group-for-user
metabase.test.util/with-temp-env-var-value! macros.metabase.test.util/with-temp-env-var-value!
metabase.test.util/with-temporary-raw-setting-values macros.metabase.test.util/with-temporary-raw-setting-values
metabase.test/with-group-for-user macros.metabase.test.data.users/with-group-for-user
metabase.test/with-temp-env-var-value! macros.metabase.test.util/with-temp-env-var-value!
metabase.test/with-temporary-raw-setting-values macros.metabase.test.util/with-temporary-raw-setting-values
metabase.util.namespaces/import-fn macros.metabase.util.namespaces/import-fn
metabase.xrays.domain-entities.malli/define-getters-and-setters macros.metabase.xrays.domain-entities.malli/define-getters-and-setters
metabase.xrays.related-test/with-world macros.metabase.xrays.related-test/with-world}}
:config-in-comment
{:linters {:unresolved-symbol {:level :off}}}
@@ -711,8 +801,6 @@
:name printable-namespaces}
{:pattern "^metabase\\.(?:(?:lib)|(?:legacy-mbql))\\..*"
:name metabase-lib}
{:pattern "^metabase\\.lib\\..*"
:name metabase-lib-excluding-legacy-mbql}
;;
;; QP source namespaces: `metabase.query-processor.*`, `metabase-enterprise.sandbox.query-processor.*`, and
;; `metabase.driver.*`, excluding `-test` namespaces.
@@ -725,19 +813,12 @@
{:pattern "metabase\\.driver.*$"
:name driver-namespaces}
;;
;; Perf-sensitive namespaces are currently the union of metabase.lib.*, driver namespaces, and QP (without tests).
;;
{:pattern "^(?!.*-test(?:\\..*)?$)(?!.*test-util)(?!^metabase\\.test.*)^metabase\\.(lib|driver|legacy-mbql|query-processor).*"
:name performance-sensitive}
;;
;; general QP tests are ones that run across multiple drivers. Anything `metabase.query-processor*-test`,
;; `metabase.driver.sql*-test` (excluding SQLServer and SQLite), or
;; `metabase-enterprise.sandbox.query-processor*-test`
;;
{:pattern "^metabase(?:((?:-enterprise\\.sandbox)?\\.query-processor)|(?:\\.driver\\.sql(?!server)(?!ite))).*-test$"
:name general-qp-and-driver-test-namespaces}
{:pattern "^metabase\\.python-runner\\..*"
:name python-runner-namespaces}]
:name general-qp-and-driver-test-namespaces}]
:config-in-ns
{test-namespaces
@@ -746,12 +827,7 @@
:missing-docstring {:level :off}
:private-call {:level :off}
:metabase/defsetting-must-specify-export {:level :off}
:metabase/defsetting-namespace {:level :off}
:discouraged-var
{metabase.driver/database-supports? {:level :off}
metabase.lib.convert/->legacy-MBQL {:level :off}
metabase.lib.core/->legacy-MBQL {:level :off}}}
:discouraged-var {metabase.driver/database-supports? {:level :off}}}
:hooks
{:analyze-call
@@ -763,14 +839,9 @@
{:linters
{:metabase/validate-escaped-single-quotes-in-i18n {:level :warning}
:metabase/validate-string-or-str-args-to-i18n {:level :warning}
:metabase/discourage-millis-duration {:level :warning}
:discouraged-var
{clojure.core/with-redefs {:message "Don't use with-redefs outside of tests"}
clojure.core/eval {:message "Don't use eval outside of tests"}}
:discouraged-namespace
{metabase.query-processor.store {:message "Please do not use the QP Store outside of legacy QP or driver code. Use MBQL 5/Lib or lib.metadata.jvm/applicaton-database-metadata-provider instead."}}}}
clojure.core/eval {:message "Don't use eval outside of tests"}}}}
printable-namespaces
{:linters
@@ -795,11 +866,9 @@
;; eventually we should do this for the whole codebase, but the args are in the opposite order so switching them
;; all at once isn't trivial, at least we can stop using it in new code.
:discouraged-var
{medley.core/map-keys {:message "Use metabase.util.performance/update-keys"}
medley.core/map-vals {:message "Use clojure.core/update-vals"}
metabase.lib.convert/->legacy-MBQL {:level :off}
metabase.lib.core/->legacy-MBQL {:level :off}
metabase.test/with-temp {:message "Don't use application database inside Lib code"}}
{medley.core/map-keys {:message "Use clojure.core/update-keys"}
medley.core/map-vals {:message "Use clojure.core/update-vals"}
metabase.test/with-temp {:message "Don't use application database inside MLv2 code"}}
:discouraged-namespace
{metabase.driver {:message "MLv2 code should be independent of driver code/methods"}
@@ -811,46 +880,26 @@
toucan.util.test {:message "Don't use application database inside MLv2 code"}
toucan2.core {:message "Don't use application database inside MLv2 code"}}}}
metabase-lib-excluding-legacy-mbql
{:linters
{:discouraged-var
{metabase.legacy-mbql.util/unique-name-generator {:message "Use metabase.lib.util/unique-name-generator or non-truncating-unique-name-generator"}}}}
driver-namespaces
{:linters
{:discouraged-var
{metabase.driver/database-supports? {:level :off}}}}
performance-sensitive
{:linters
{:discouraged-var
{clojure.core/update-keys {:message "Use metabase.util.performance/update-keys"}}}}
qp-and-driver-source-namespaces
{:linters
{:discouraged-namespace
{metabase.query-processor.store {:level :off}
metabase.util.malli.schema {:message "Prefer metabase.lib.schema or metabase.legacy-mbql.schema schemas in QP/driver code"}
toucan.db {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
toucan.models {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
toucan.util.test {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
toucan2.core {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}}
{metabase.util.malli.schema {:message "Prefer metabase.lib.schema or metabase.legacy-mbql.schema schemas in QP/driver code"}
toucan.db {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
toucan.models {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
toucan.util.test {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
toucan2.core {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}}
:discouraged-var
{medley.core/map-keys {:message "Use metabase.util.performance/update-keys"}
medley.core/map-vals {:message "Use clojure.core/update-vals"}
metabase.app-db.core/query {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
metabase.app-db.core/reducible-query {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
metabase.lib.convert/->legacy-MBQL {:level :off}
metabase.lib.core/->legacy-MBQL {:level :off}}}}
{medley.core/map-keys {:message "Use clojure.core/update-keys"}
medley.core/map-vals {:message "Use clojure.core/update-vals"}
metabase.db.query/query {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
metabase.db.query/reducible-query {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}}}}
general-qp-and-driver-test-namespaces
{:linters
{:discouraged-var
{metabase.test/with-column-remappings {:message "Use lib.tu/remap-metadata-provider instead of with-column-remappings in QP tests"}
metabase.test.util/with-column-remappings {:message "Use lib.tu/remap-metadata-provider instead of with-column-remappings in QP tests"}
metabase.test/with-temp {:message "Use lib.tu/mock-metadata-provider in QP tests. Exceptions: tests for code that doesn't use MPs or models not supported by MPs"}
toucan2.tools.with-temp/with-temp {:message "Use lib.tu/mock-metadata-provider in QP tests. Exceptions: tests for code that doesn't use MPs or models not supported by MPs"}}
:metabase/disallow-hardcoded-driver-names-in-tests
{:level :warning}}}}}
{:metabase/disallow-hardcoded-driver-names-in-tests {:level :warning}}}}}

File diff suppressed because it is too large Load Diff

View File

@@ -49,43 +49,36 @@
metabase.actions.models/insert!
metabase.analytics.snowplow-test/fake-track-event-impl!
metabase.analytics.snowplow/track-event-impl!
metabase.app-db.core/finish-db-setup!
metabase.app-db.core/migrate!
metabase.app-db.core/setup-db!
metabase.app-db.schema-migrations-test.impl/run-migrations-in-range!
metabase.app-db.setup/migrate!
metabase.app-db.setup/setup-db!
metabase.channel.email-test/reset-inbox!
metabase.channel.email/send-email!
metabase.cmd.dump-to-h2/dump-to-h2!
metabase.cmd.load-from-h2/load-from-h2!
metabase.content-verification.models.moderation-review/create-review!
metabase.core.core/ensure-audit-db-installed!
metabase.db.schema-migrations-test.impl/run-migrations-in-range!
metabase.db.setup/migrate!
metabase.db.setup/setup-db!
metabase.db/migrate!
metabase.db/setup-db!
metabase.driver.mongo-test/create-database-from-row-maps!
metabase.driver.postgres-test/create-enums-db!
metabase.driver.postgres-test/drop-if-exists-and-create-db!
metabase.driver.sql-jdbc.execute/execute-statement!
metabase.indexed-entities.models.model-index/add-values!
metabase.driver.sql-jdbc.connection.ssh-tunnel-test/start-mock-servers!
metabase.driver.sql-jdbc.connection.ssh-tunnel-test/stop-mock-servers!
metabase.indexed-entities.task.index-values/job-init!
metabase.model-persistence.models.persisted-info/ready-database!
metabase.model-persistence.task.persist-refresh/job-init!
metabase.model-persistence.task.persist-refresh/refresh-tables!
metabase.model-persistence.task.persist-refresh/schedule-persistence-for-database!
metabase.models.collection.graph-test/clear-graph-revisions!
metabase.models.collection.graph-test/do-with-n-temp-users-with-personal-collections!
metabase.models.field-values/create-or-update-full-field-values!
metabase.models.moderation-review/create-review!
metabase.models.on-demand-test/add-dashcard-with-parameter-mapping!
metabase.permissions.core/grant-application-permissions!
metabase.permissions.core/grant-collection-read-permissions!
metabase.permissions.core/grant-collection-readwrite-permissions!
metabase.permissions.core/grant-full-data-permissions!
metabase.permissions.core/grant-native-readwrite-permissions!
metabase.permissions.core/grant-permissions!
metabase.permissions.core/revoke-application-permissions!
metabase.permissions.core/revoke-data-perms!
metabase.permissions.core/update-data-perms-graph!
metabase.permissions.core/update-group-permissions!
metabase.permissions.models.collection.graph-test/clear-graph-revisions!
metabase.permissions.models.collection.graph-test/do-with-n-temp-users-with-personal-collections!
metabase.models.setting-test/test-user-local-allowed-setting!
metabase.models.setting-test/test-user-local-only-setting!
metabase.models.setting.cache/restore-cache!
metabase.models.setting/set!
metabase.models.setting/validate-settings-formatting!
metabase.permissions.models.permissions/grant-application-permissions!
metabase.permissions.models.permissions/grant-collection-read-permissions!
metabase.permissions.models.permissions/grant-collection-readwrite-permissions!
@@ -103,12 +96,7 @@
metabase.query-processor.streaming.interface/begin!
metabase.query-processor.streaming.interface/finish!
metabase.query-processor.streaming.interface/write-row!
metabase.sample-data.impl/try-to-extract-sample-database!
metabase.settings.models.setting-test/test-user-local-allowed-setting!
metabase.settings.models.setting-test/test-user-local-only-setting!
metabase.settings.models.setting.cache/restore-cache!
metabase.settings.models.setting/set!
metabase.settings.models.setting/validate-settings-formatting!
metabase.sample-data/try-to-extract-sample-database!
metabase.setup.core/create-token!
metabase.sso.ldap-test-util/start-ldap-server!
metabase.sync.core/sync-database!
@@ -116,8 +104,7 @@
metabase.sync.sync-metadata/sync-db-metadata!
metabase.sync.util-test/sync-database!
metabase.sync.util/store-sync-summary!
metabase.task.core/delete-task!
metabase.task.impl/delete-task!
metabase.task/delete-task!
metabase.test.data.bigquery-cloud-sdk/execute!
metabase.test.data.impl.get-or-create/set-test-db-permissions!
metabase.test.data.impl/copy-db-tables-and-fields!
@@ -139,6 +126,7 @@
metabase.test/initialize-if-needed!
metabase.test/test-helpers-set-global-values!
metabase.test/with-temp-env-var-value!
metabase.upload.impl-test/set-local-infile!
metabase.upload-test/set-local-infile!
metabase.util.files/create-dir-if-not-exists!
metabase.warehouse-schema.models.field-values/create-or-update-full-field-values!}}}}
metabase.util.ssh-test/start-mock-servers!
metabase.util.ssh-test/stop-mock-servers!}}}}

View File

@@ -0,0 +1,13 @@
(ns macros.metabase.api.card-test)
(defmacro with-ordered-items
[collection model-and-name-syms & body]
`(let ~(into []
(comp (partition-all 2)
(mapcat (fn [[model binding]]
[binding model])))
model-and-name-syms)
~collection
~@(->> (partition-all 2 model-and-name-syms)
(map second))
~@body))

View File

@@ -0,0 +1,24 @@
(ns macros.metabase.api.collection-test)
(defmacro with-collection-hierarchy! [bindings & body]
`(let [~'a nil
~'b nil
~'c nil
~'d nil
~'e nil
~'f nil
~'g nil]
~'a
~'b
~'c
~'d
~'e
~'f
~'g
~bindings
~@body))
(defmacro with-some-children-of-collection! [collection-or-id-or-nil & body]
`(let [~'&ids ~collection-or-id-or-nil]
~'&ids
~@body))

View File

@@ -0,0 +1,13 @@
(ns macros.metabase.api.embed-test)
(defmacro with-embedding-enabled-and-temp-card-referencing!
[table-kw field-kw [card-binding] & body]
`(let [~card-binding [~table-kw ~field-kw]]
~@body))
(defmacro with-embedding-enabled-and-temp-dashcard-referencing!
[table-kw field-kw [dash-binding card-binding dashcard-binding] & body]
`(let [~dash-binding [~table-kw ~field-kw]
~(or card-binding '_) nil
~(or dashcard-binding '_) nil]
~@body))

View File

@@ -1,24 +0,0 @@
(ns macros.metabase.collections.api-test)
(defmacro with-collection-hierarchy! [bindings & body]
`(let [~'a nil
~'b nil
~'c nil
~'d nil
~'e nil
~'f nil
~'g nil]
~'a
~'b
~'c
~'d
~'e
~'f
~'g
~bindings
~@body))
(defmacro with-some-children-of-collection! [collection-or-id-or-nil & body]
`(let [~'&ids ~collection-or-id-or-nil]
~'&ids
~@body))

View File

@@ -1,13 +0,0 @@
(ns macros.metabase.embedding.api.embed-test)
(defmacro with-embedding-enabled-and-temp-card-referencing!
[table-kw field-kw [card-binding] & body]
`(let [~card-binding [~table-kw ~field-kw]]
~@body))
(defmacro with-embedding-enabled-and-temp-dashcard-referencing!
[table-kw field-kw [dash-binding card-binding dashcard-binding] & body]
`(let [~dash-binding [~table-kw ~field-kw]
~(or card-binding '_) nil
~(or dashcard-binding '_) nil]
~@body))

View File

@@ -0,0 +1,4 @@
(ns macros.metabase.models.params.chain-filter-test)
(defmacro chain-filter [& _]
`(metabase.models.params.chain-filter/chain-filter nil nil))

View File

@@ -0,0 +1,12 @@
(ns macros.metabase.models.query-analysis-test)
(defmacro with-test-setup [& body]
`(let [~'card-id 1
~'table-id 2
~'tax-id 3
~'total-id 4]
~'card-id
~'table-id
~'tax-id
~'total-id
~@body))

View File

@@ -0,0 +1,9 @@
(ns macros.metabase.models.user-test)
(defmacro with-groups [bindings & body]
`(let ~(into []
(comp (partition-all 3)
(mapcat (fn [[group-binding properties members]]
[group-binding [properties members]])))
bindings)
~@body))

View File

@@ -1,4 +0,0 @@
(ns macros.metabase.parameters.chain-filter-test)
(defmacro chain-filter [& _]
`(metabase.parameters.chain-filter/chain-filter nil nil))

View File

@@ -1,13 +0,0 @@
(ns macros.metabase.queries.api.card-test)
(defmacro with-ordered-items
[collection model-and-name-syms & body]
`(let ~(into []
(comp (partition-all 2)
(mapcat (fn [[model binding]]
[binding model])))
model-and-name-syms)
~collection
~@(->> (partition-all 2 model-and-name-syms)
(map second))
~@body))

View File

@@ -1,12 +0,0 @@
(ns macros.metabase.query-analysis.models.query-analysis-test)
(defmacro with-test-setup [& body]
`(let [~'card-id 1
~'table-id 2
~'tax-id 3
~'total-id 4]
~'card-id
~'table-id
~'tax-id
~'total-id
~@body))

View File

@@ -1,9 +0,0 @@
(ns macros.metabase.server.statistics-handler-test)
(defmacro with-server
[_ _ [connector chan-start-handle chan-finish-handle chan-done-request] & body]
`(let [~connector nil
~chan-start-handle nil
~chan-finish-handle nil
~chan-done-request nil]
~@body))

View File

@@ -1,9 +0,0 @@
(ns macros.metabase.users.models.user-test)
(defmacro with-groups! [bindings & body]
`(let ~(into []
(comp (partition-all 3)
(mapcat (fn [[group-binding properties members]]
[group-binding [properties members]])))
bindings)
~@body))

View File

@@ -1,5 +0,0 @@
(ns macros.metabase-enterprise.database-routing.e2e-test)
(defmacro with-temp-dbs! [bindings & body]
`(let [~@(interleave bindings (repeat nil))]
~@body))

View File

@@ -1,7 +1,6 @@
(ns hooks.clojure.core.ns
(:require
[clj-kondo.hooks-api :as hooks]
[clojure.string :as str]
[hooks.common]
[hooks.common.modules :as modules]))
@@ -60,15 +59,12 @@
(printf "Don't know how to figure out what namespace is being required in %s\n" (pr-str node)))))
args)))
(defn- ns-form-node->ns-symb-node [ns-form-node]
(some (fn [node]
(when (and (hooks/token-node? node)
(not= (hooks/sexpr node) 'ns))
node))
(:children ns-form-node)))
(defn- ns-form-node->ns-symb [ns-form-node]
(some-> (ns-form-node->ns-symb-node ns-form-node)
(some-> (some (fn [node]
(when (and (hooks/token-node? node)
(not= (hooks/sexpr node) 'ns))
node))
(:children ns-form-node))
hooks/sexpr))
(defn- lint-modules [ns-form-node config]
@@ -87,33 +83,9 @@
:message error
:type :metabase/modules))))))))
(defn- lint-namespace-name [ns-form-node]
(when-let [ns-symb-node (ns-form-node->ns-symb-node ns-form-node)]
(when-not (contains? (hooks.common/ignored-linters ns-symb-node) :metabase/namespace-name)
(let [parts (str/split (str (hooks/sexpr ns-symb-node)) #"\.")]
(when (and (<= (count parts) 2)
(#{"metabase" "metabase-enterprise"} (first parts))
;; exclude test namespaces for now
(not (str/ends-with? (last parts) "test")))
(hooks/reg-finding! (assoc (meta ns-symb-node)
:message "Metabase namespaces should have the form metabase[-enterprise].<module>.* [:metabase/namespace-name]"
:type :metabase/namespace-name)))))))
(defn lint-ns [x]
(doto (:node x)
lint-require-shapes
lint-requires-on-new-lines
(lint-modules (modules/config x))
lint-namespace-name)
(lint-modules (modules/config x)))
x)
(comment
(lint-ns
{:node (hooks/parse-string
(pr-str
'(ns hooks.clojure.core.ns
(:require
[clj-kondo.hooks-api :as hooks]
[clojure.string :as str]
[hooks.common]
[hooks.common.modules :as modules]))))}))

View File

@@ -52,13 +52,7 @@
(defn- lint-require* [node current-ns config]
(let [[_require & args] (:children node)
required-namespaces (->> (keep unwrap-require args)
;; `require` runs through `cc/load-libs` which allows
;; #{:as :reload :reload-all :require :use :verbose :refer :as-alias}
;; as args. If you use a REPL tool that applies clj-kondo linting (e.g., Clojure MCP)
;; you may wish to `:reload` or `:reload-all` a namespace. So we shouldn't assume all
;; args are keywords:
(remove keyword?))]
required-namespaces (keep unwrap-require args)]
(lint* node current-ns config required-namespaces)))
(defn- lint-requiring-resolve* [node current-ns config]

View File

@@ -1,14 +0,0 @@
(ns hooks.metabase.app-db.schema-migrations-test.impl
(:require [clj-kondo.hooks-api :as hooks]))
(defn test-migrations
[{{[_ migration-range {[binding] :children} & body] :children} :node}]
(let [node* (hooks/list-node
(list*
(hooks/token-node `let)
(hooks/vector-node
[binding (hooks/list-node (list (hooks/token-node `fn)
(hooks/vector-node [(hooks/token-node '&) (hooks/token-node '_args)])))])
migration-range
body))]
{:node node*}))

View File

@@ -0,0 +1,14 @@
(ns hooks.metabase.db.schema-migrations-test.impl
(:require [clj-kondo.hooks-api :as hooks]))
(defn test-migrations
[{{[_ migration-range {[binding] :children} & body] :children} :node}]
(let [node* (hooks/list-node
(list*
(hooks/token-node `let)
(hooks/vector-node
[binding (hooks/list-node (list (hooks/token-node `fn)
(hooks/vector-node [(hooks/token-node '&) (hooks/token-node '_args)])))])
migration-range
body))]
{:node node*}))

View File

@@ -1,18 +0,0 @@
(ns hooks.metabase.driver.mongo-test
(:require
[clj-kondo.hooks-api :as hooks]
[hooks.common]))
(defn with-describe-table-for-sample
"Forward bindings for results of functions used in mongo's describe-table implementation."
[{{[_ _ & body] :children} :node}]
(let [node* (hooks/list-node
(list*
(hooks/token-node 'let)
(hooks/vector-node
[(hooks/token-node 'dbfields) (hooks/token-node '(delay nil))
(hooks/token-node 'ftree) (hooks/token-node '(delay nil))
(hooks/token-node 'nested-fields) (hooks/token-node '(delay nil))])
body))]
;; Avoid warnings when not all from the dbfields, ftree and nested-fields are used.
{:node (hooks.common/update-ignored-linters node* conj :unused-binding)}))

View File

@@ -0,0 +1,294 @@
(ns hooks.metabase.models.setting
(:require
[clj-kondo.hooks-api :as hooks]
[hooks.common :as common]))
;;; TODO -- move this into
(def ^:private ignored-implicit-export?
'#{active-users-count
admin-email
analytics-uuid
anon-tracking-enabled
api-key
attachment-table-row-limit
audit-max-retention-days
bcc-enabled?
breakout-bin-width
check-for-updates
cloud-gateway-ips
config-from-file-sync-databases
custom-homepage
custom-homepage-dashboard
database-enable-actions
deprecation-notice-version
dismissed-custom-dashboard-toast
dismissed-onboarding-sidebar-link
email-configured?
email-from-address
email-from-name
email-reply-to
email-smtp-host
email-smtp-password
email-smtp-port
email-smtp-security
email-smtp-username
embedding-app-origin
embedding-secret-key
enable-password-login
enable-public-sharing
enable-query-caching
engines
enum-cardinality-threshold
follow-up-email-sent
google-auth-auto-create-accounts-domain
google-auth-client-id
google-auth-configured
google-auth-enabled
has-sample-database?
has-user-setup
help-link
help-link-custom-destination
instance-creation
is-hosted?
is-metabot-enabled
jdbc-data-warehouse-max-connection-pool-size
jdbc-data-warehouse-unreturned-connection-timeout-seconds
jwt-attribute-email
jwt-attribute-firstname
jwt-attribute-groups
jwt-attribute-lastname
jwt-configured
jwt-enabled
jwt-group-mappings
jwt-group-sync
jwt-identity-provider-uri
jwt-user-provisioning-enabled?
jwt-shared-secret
last-acknowledged-version
last-used-native-database-id
ldap-attribute-email
ldap-attribute-firstname
ldap-attribute-lastname
ldap-bind-dn
ldap-configured?
ldap-enabled
ldap-group-base
ldap-group-mappings
ldap-group-membership-filter
ldap-group-sync
ldap-host
ldap-password
ldap-port
ldap-user-provisioning-enabled?
ldap-security
ldap-sync-user-attributes
ldap-sync-user-attributes-blacklist
ldap-user-base
ldap-user-filter
load-analytics-content
map-tile-server-url
metabot-default-embedding-model
metabot-feedback-url
metabot-get-prompt-templates-url
metabot-prompt-generator-token-limit
multi-setting-read-only
notebook-native-preview-shown
notebook-native-preview-sidebar-width
notification-link-base-url
num-metabot-choices
openai-api-key
openai-available-models
openai-model
openai-organization
other-sso-enabled?
password-complexity
persist-models-enabled
persisted-model-refresh-cron-schedule
premium-embedding-token
prometheus-server-port
query-caching-max-kb
query-caching-max-ttl
query-caching-min-ttl
query-caching-ttl-ratio
redirect-all-requests-to-https
reset-token-ttl-hours
retired-setting
retry-initial-interval
retry-max-attempts
retry-max-interval-millis
retry-multiplier
retry-randomization-factor
saml-application-name
saml-attribute-email
saml-attribute-firstname
saml-attribute-group
saml-attribute-lastname
saml-configured
saml-enabled
saml-group-mappings
saml-group-sync
saml-identity-provider-certificate
saml-identity-provider-issuer
saml-identity-provider-uri
saml-keystore-alias
saml-keystore-password
saml-keystore-path
saml-user-provisioning-enabled?
send-email-on-first-login-from-new-device
send-new-sso-user-admin-email?
session-cookie-samesite
session-cookies
session-timeout
setup-token
show-database-syncing-modal
show-metabase-links
site-url
site-uuid
site-uuid-for-premium-features-token-checks
site-uuid-for-unsubscribing-url
site-uuid-for-version-info-fetching
slack-app-token
slack-cached-channels-and-usernames
slack-channels-and-usernames-last-updated
slack-files-channel
slack-token
slack-token-valid?
snowplow-available
snowplow-enabled
snowplow-url
sql-jdbc-fetch-size
ssh-heartbeat-interval-sec
ssl-certificate-public-key
startup-time-millis
token-features
token-status
toucan-name
trial-banner-dismissal-timestamp
uncached-setting
user-visibility
version
version-info
version-info-last-checked})
(defn- defsetting-lint [node setting-name docstring options-list]
(let [anon-binding (common/with-macro-meta (hooks/token-node '_) node)
;; (defn my-setting [] ...)
getter-node (-> (list
(hooks/token-node 'defn)
setting-name
(hooks/string-node "Docstring.")
(hooks/vector-node []))
hooks/list-node
(with-meta (meta node)))
;; (defn my-setting! [_x] ...)
setter-node (-> (list
(hooks/token-node 'defn)
(with-meta
(hooks/token-node (symbol (str (hooks/sexpr setting-name) \!)))
(meta setting-name))
(hooks/string-node "Docstring.")
(hooks/vector-node [(hooks/token-node '_value-or-nil)]))
hooks/list-node
(with-meta (meta node))
common/add-lsp-ignore-unused-public-var-metadata)]
(when (nil? (second (drop-while (comp not #{[:k :export?]} first) options-list)))
(when-not (contains? ignored-implicit-export? (:value setting-name))
(hooks/reg-finding! (assoc (meta node)
:message "Setting definition must provide an explicit value for :export? indicating whether the setting should be exported or not with serialization."
:type :metabase/defsetting-must-specify-export))))
{:node (-> (hooks/list-node
(list
(hooks/token-node 'let)
;; include description and the options map so they can get validated as well.
(hooks/vector-node
[anon-binding docstring
anon-binding (hooks/map-node options-list)])
(hooks/reg-keyword! (-> (hooks/keyword-node (keyword (hooks/sexpr setting-name)))
(with-meta (meta setting-name)))
'metabase.models.setting/defsetting)
getter-node
setter-node))
(with-meta (meta node)))}))
(defn defsetting
"Rewrite a [[metabase.models.defsetting]] form like
(defsetting my-setting \"Description\" :type :boolean)
as
(let [_ \"Description\"
_ {:type :boolean}]
(defn my-setting \"Docstring.\" [])
(defn my-setting! \"Docstring.\" [_value-or-nil]))
for linting purposes."
[{:keys [node], :as context}]
(let [[setting-name docstring & options] (rest (:children node))]
(merge
context
(defsetting-lint node setting-name docstring options))))
(defn define-multi-setting
"Rewrite a [[metabase.models.define-multi-setting]] form like
(defsetting my-setting \"Description\" :key :type :boolean)
as
(let [_ \"Description\"
_ {:type :boolean, :multi-thunk :key}]
(defn my-setting \"Docstring.\" [])
(defn my-setting! \"Docstring.\" [_value-or-nil]))
for linting purposes."
[{:keys [node], :as context}]
(let [[setting-name docstring thunk & options] (rest (:children node))]
(merge
context
(defsetting-lint node setting-name docstring (concat options [(hooks/token-node :multi-thunk) thunk])))))
(defn define-multi-setting-impl
[context]
(letfn [(update-node [node]
(let [[_define-multi-setting-impl setting-name & more] (:children node)]
(-> (hooks/list-node
(list*
(hooks/token-node 'do)
;; For a qualified setting name like `oss-namespace/setting`, then leave the symbol as is, so the
;; usage of that namespace/var gets recorded. Otherwise for an unqualified name convert it to a
;; keyword so we don't get unresolved var errors.
(if (and (hooks/token-node? setting-name)
(qualified-symbol? (hooks/sexpr setting-name)))
setting-name
(-> (hooks/keyword-node (keyword (hooks/sexpr setting-name)))
(with-meta (meta setting-name))))
more))
(with-meta (meta node)))))]
(update context :node update-node)))
(comment
(defn- defsetting* [form]
(hooks/sexpr
(:node
(defsetting
{:node
(hooks/parse-string
(with-out-str
#_{:clj-kondo/ignore [:unresolved-namespace]}
(clojure.pprint/pprint
form)))}))))
(defn x []
(defsetting*
'(defsetting active-users-count
(deferred-tru "Cached number of active users. Refresh every 5 minutes.")
:visibility :admin
:type :integer
:default 0
:getter (fn []
(if-not ((requiring-resolve 'metabase.db/db-is-set-up?))
0
(cached-active-users-count)))))))

View File

@@ -1,310 +0,0 @@
(ns hooks.metabase.settings.models.setting
(:require
[clj-kondo.hooks-api :as hooks]
[hooks.common :as common]))
(defn lint-defsetting-namespace [node context]
(when-not (re-matches #"^metabase(?:-enterprise)?\.[^\.]+\.settings$" (name (:ns context)))
(hooks/reg-finding! (assoc (meta node)
:message "All defsettings should live in metabase[-enterprise].<module>.settings namespaces"
:type :metabase/defsetting-namespace))))
;;; TODO -- move this into a Kondo config file in `.clj-kondo/config/`
(def ^:private ignored-implicit-export?
'#{active-users-count
admin-email
analytics-uuid
anon-tracking-enabled
api-key
attachment-table-row-limit
audit-max-retention-days
bcc-enabled?
breakout-bin-width
check-for-updates
cloud-gateway-ips
config-from-file-sync-databases
custom-homepage
custom-homepage-dashboard
database-enable-actions
deprecation-notice-version
dismissed-custom-dashboard-toast
email-configured?
email-from-address
email-from-name
email-reply-to
email-smtp-host
email-smtp-password
email-smtp-port
email-smtp-security
email-smtp-username
embedding-app-origin
embedding-secret-key
enable-password-login
enable-public-sharing
enable-query-caching
engines
enum-cardinality-threshold
follow-up-email-sent
google-auth-auto-create-accounts-domain
google-auth-client-id
google-auth-configured
google-auth-enabled
has-sample-database?
has-user-setup
help-link
help-link-custom-destination
instance-creation
is-hosted?
is-metabot-enabled
jdbc-data-warehouse-max-connection-pool-size
jdbc-data-warehouse-unreturned-connection-timeout-seconds
jwt-attribute-email
jwt-attribute-firstname
jwt-attribute-groups
jwt-attribute-lastname
jwt-configured
jwt-enabled
jwt-group-mappings
jwt-group-sync
jwt-identity-provider-uri
jwt-user-provisioning-enabled?
jwt-shared-secret
last-acknowledged-version
last-used-native-database-id
ldap-attribute-email
ldap-attribute-firstname
ldap-attribute-lastname
ldap-bind-dn
ldap-configured?
ldap-enabled
ldap-group-base
ldap-group-mappings
ldap-group-membership-filter
ldap-group-sync
ldap-host
ldap-password
ldap-port
ldap-user-provisioning-enabled?
ldap-security
ldap-sync-user-attributes
ldap-sync-user-attributes-blacklist
ldap-user-base
ldap-user-filter
load-analytics-content
map-tile-server-url
metabot-default-embedding-model
metabot-feedback-url
metabot-get-prompt-templates-url
metabot-prompt-generator-token-limit
multi-setting-read-only
notebook-native-preview-shown
notebook-native-preview-sidebar-width
notification-link-base-url
num-metabot-choices
openai-api-key
openai-available-models
openai-model
openai-organization
other-sso-enabled?
password-complexity
persist-models-enabled
persisted-model-refresh-cron-schedule
premium-embedding-token
prometheus-server-port
query-caching-max-kb
query-caching-max-ttl
query-caching-min-ttl
query-caching-ttl-ratio
redirect-all-requests-to-https
reset-token-ttl-hours
retired-setting
retry-initial-interval
retry-max-attempts
retry-max-interval-millis
retry-multiplier
retry-randomization-factor
saml-application-name
saml-attribute-email
saml-attribute-firstname
saml-attribute-group
saml-attribute-lastname
saml-configured
saml-enabled
saml-group-mappings
saml-group-sync
saml-identity-provider-certificate
saml-identity-provider-issuer
saml-identity-provider-uri
saml-keystore-alias
saml-keystore-password
saml-keystore-path
saml-user-provisioning-enabled?
sdk-iframe-embed-setup-settings
send-email-on-first-login-from-new-device
send-new-sso-user-admin-email?
session-cookie-samesite
session-cookies
session-timeout
setup-token
show-database-syncing-modal
show-metabase-links
site-url
site-uuid
site-uuid-for-premium-features-token-checks
site-uuid-for-unsubscribing-url
site-uuid-for-version-info-fetching
slack-app-token
slack-cached-channels-and-usernames
slack-channels-and-usernames-last-updated
slack-files-channel
slack-token
slack-token-valid?
snowplow-available
snowplow-enabled
snowplow-url
sql-jdbc-fetch-size
ssh-heartbeat-interval-sec
ssl-certificate-public-key
startup-time-millis
token-features
token-status
toucan-name
trial-banner-dismissal-timestamp
uncached-setting
user-visibility
version
version-info
version-info-last-checked})
(defn- lint-defsetting-export [node]
(let [[_defsetting setting-name _docstring & options] (:children node)]
(when (nil? (second (drop-while (comp not #{[:k :export?]} first) options)))
(when-not (contains? ignored-implicit-export? (:value setting-name))
(hooks/reg-finding! (assoc (meta node)
:message "Setting definition must provide an explicit value for :export? indicating whether the setting should be exported or not with serialization."
:type :metabase/defsetting-must-specify-export))))))
(defn- defsetting-replacement-node [node]
(let [[_defsetting setting-name docstring & options-list] (:children node)
anon-binding (common/with-macro-meta (hooks/token-node '_) node)
;; (defn my-setting [] ...)
getter-node (-> (list
(hooks/token-node 'defn)
setting-name
(hooks/string-node "Docstring.")
(hooks/vector-node []))
hooks/list-node
(with-meta (meta node)))
;; (defn my-setting! [_x] ...)
setter-node (-> (list
(hooks/token-node 'defn)
(with-meta
(hooks/token-node (symbol (str (hooks/sexpr setting-name) \!)))
(meta setting-name))
(hooks/string-node "Docstring.")
(hooks/vector-node [(hooks/token-node '_value-or-nil)]))
hooks/list-node
(with-meta (meta node))
common/add-lsp-ignore-unused-public-var-metadata)]
(-> (hooks/list-node
(list
(hooks/token-node 'let)
;; include description and the options map so they can get validated as well.
(hooks/vector-node
[anon-binding docstring
anon-binding (hooks/map-node options-list)])
(hooks/reg-keyword! (-> (hooks/keyword-node (keyword (hooks/sexpr setting-name)))
(with-meta (meta setting-name)))
'metabase.settings.models.setting/defsetting)
getter-node
setter-node))
(with-meta (meta node)))))
(defn defsetting
"Rewrite a [[metabase.models.defsetting]] form like
(defsetting my-setting \"Description\" :type :boolean)
as
(let [_ \"Description\"
_ {:type :boolean}]
(defn my-setting \"Docstring.\" [])
(defn my-setting! \"Docstring.\" [_value-or-nil]))
for linting purposes."
[{:keys [node], :as context}]
(lint-defsetting-namespace node context)
(lint-defsetting-export node)
(update context :node defsetting-replacement-node))
(defn define-multi-setting-replacement-node [node]
(let [[defsetting setting-name docstring thunk & options] (:children node)]
(-> (hooks/list-node
(list*
defsetting
setting-name
docstring
(hooks/token-node :multi-thunk) thunk
options))
(with-meta (meta node))
defsetting-replacement-node)))
(defn define-multi-setting
"Rewrite a [[metabase.models.define-multi-setting]] form like
(defsetting my-setting \"Description\" :key :type :boolean)
as
(let [_ \"Description\"
_ {:type :boolean, :multi-thunk :key}]
(defn my-setting \"Docstring.\" [])
(defn my-setting! \"Docstring.\" [_value-or-nil]))
for linting purposes."
[{:keys [node], :as context}]
(update context :node define-multi-setting-replacement-node))
(defn define-multi-setting-impl
[context]
(letfn [(update-node [node]
(let [[_define-multi-setting-impl setting-name & more] (:children node)]
(-> (hooks/list-node
(list*
(hooks/token-node 'do)
;; For a qualified setting name like `oss-namespace/setting`, then leave the symbol as is, so the
;; usage of that namespace/var gets recorded. Otherwise for an unqualified name convert it to a
;; keyword so we don't get unresolved var errors.
(if (and (hooks/token-node? setting-name)
(qualified-symbol? (hooks/sexpr setting-name)))
setting-name
(-> (hooks/keyword-node (keyword (hooks/sexpr setting-name)))
(with-meta (meta setting-name))))
more))
(with-meta (meta node)))))]
(update context :node update-node)))
(comment
(defn- defsetting* [form]
(hooks/sexpr
(:node
(defsetting
{:node
(hooks/parse-string
(with-out-str
#_{:clj-kondo/ignore [:unresolved-namespace]}
(clojure.pprint/pprint
form)))}))))
(defn x []
(defsetting*
'(defsetting active-users-count
(deferred-tru "Cached number of active users. Refresh every 5 minutes.")
:visibility :admin
:type :integer
:default 0
:getter (fn []
(if-not ((requiring-resolve 'metabase.app-db.core/db-is-set-up?))
0
(cached-active-users-count)))))))

View File

@@ -2,8 +2,8 @@
(:require
[clj-kondo.hooks-api :as hooks]))
(defn iterate*
"(iterate {:as limit-spec} [s1 val1 & more] body-form body-form...)"
(defn with-gentest
"(with-gentest {:as limit-spec} [s1 val1 & more] body-form body-form...)"
[{{:keys [children] :as node} :node :as orig}]
(let [[token limit-spec bindings & body-nodes] children
node* (-> (hooks/list-node

View File

@@ -1,15 +0,0 @@
(ns hooks.metabase.test.util.generators.jvm
(:require
[clj-kondo.hooks-api :as hooks]))
(defn with-random-cards
"(with-random-cards <metadata-provider> <integer-count-value> & body-forms)"
[{{:keys [children] :as node} :node :as orig}]
(let [[token num-cards & body-nodes] children
node* (-> (hooks/list-node
(list token num-cards
(hooks/list-node (list* (hooks/token-node 'let)
(hooks/vector-node [(hooks/token-node '&cards) (hooks/list-node ())])
body-nodes))))
(with-meta (meta node)))]
(assoc orig :node node*)))

View File

@@ -1,16 +0,0 @@
(ns hooks.metabase.util.timing
"Custom CLJ Kondo hooks for timing-related linting rules."
(:require [clj-kondo.hooks-api :as api]))
(defn discourage-millis-duration
"Discourage slow and unreliable duration calculations using System/currentTimeMillis "
[{:keys [node]}]
(let [sexpr (api/sexpr node)]
(when (and (list? sexpr) (= (first sexpr) '-) (>= (count sexpr) 2))
(let [first-arg (second sexpr)]
(when (= first-arg (list 'System/currentTimeMillis))
(api/reg-finding!
(assoc (meta node)
:message "Avoid System/currentTimeMillis for calculating durations. Use u/start-timer and u/since-ms for more reliability."
:type :metabase/discourage-millis-duration
:level :warning)))))))

View File

@@ -20,17 +20,15 @@
(lint-ns form (assoc-in config [:linters :metabase/modules :level] :warning)))
(deftest ^:parallel module-checker-allowed-modules-test
(are [task-namespace] (=? [{:message "Module task should not be used in the api module. [:metabase/modules api :uses]"
:type :metabase/modules}]
(lint-modules
'(ns metabase.api.search
(:require
[metabase.request.core :as request]
[task-namespace :as task]))
'{:metabase/modules {api {:uses #{request}}
request {:api #{metabase.request.core}}}}))
metabase.task
metabase.task.core))
(is (=? [{:message "Module task should not be used in the api module. [:metabase/modules api :uses]"
:type :metabase/modules}]
(lint-modules
'(ns metabase.api.search
(:require
[metabase.request.core :as request]
[metabase.task :as task]))
'{:metabase/modules {api {:uses #{request}}
request {:api #{metabase.request.core}}}}))))
(deftest ^:parallel module-checker-api-namespaces-test
(is (=? [{:message "Namespace metabase.search.config is not an allowed external API namespace for the search module. [:metabase/modules search :api]"

View File

@@ -0,0 +1,34 @@
(ns hooks.metabase.models.setting-test
(:require
[clj-kondo.hooks-api :as hooks]
[clojure.test :refer :all]
[hooks.metabase.models.setting]))
#_{:clj-kondo/ignore [:metabase/validate-deftest]}
(deftest ^:parallel defsetting-test
(let [node (-> '(defsetting active-users-count
(deferred-tru "Cached number of active users. Refreshed every 5 minutes.")
:visibility :admin
:type :integer
:default 0
:getter (fn []
(if-not ((requiring-resolve 'metabase.db/db-is-set-up?))
0
(cached-active-users-count))))
pr-str
hooks/parse-string)]
(is (= '(let [_ (deferred-tru "Cached number of active users. Refreshed every 5 minutes.")
_ {:default 0
:type :integer
:getter (fn []
(if-not ((requiring-resolve 'metabase.db/db-is-set-up?))
0
(cached-active-users-count)))
:visibility :admin}]
:active-users-count
(defn active-users-count "Docstring." [])
(defn active-users-count! "Docstring." [_value-or-nil]))
(-> {:node node}
hooks.metabase.models.setting/defsetting
:node
hooks/sexpr)))))

View File

@@ -1,34 +0,0 @@
(ns hooks.metabase.settings.models.setting-test
(:require
[clj-kondo.hooks-api :as hooks]
[clojure.test :refer :all]
[hooks.metabase.settings.models.setting]))
#_{:clj-kondo/ignore [:metabase/validate-deftest]}
(deftest ^:parallel defsetting-test
(let [node (-> '(defsetting active-users-count
(deferred-tru "Cached number of active users. Refreshed every 5 minutes.")
:visibility :admin
:type :integer
:default 0
:getter (fn []
(if-not ((requiring-resolve 'metabase.app-db.core/db-is-set-up?))
0
(cached-active-users-count))))
pr-str
hooks/parse-string)]
(is (= '(let [_ (deferred-tru "Cached number of active users. Refreshed every 5 minutes.")
_ {:default 0
:type :integer
:getter (fn []
(if-not ((requiring-resolve 'metabase.app-db.core/db-is-set-up?))
0
(cached-active-users-count)))
:visibility :admin}]
:active-users-count
(defn active-users-count "Docstring." [])
(defn active-users-count! "Docstring." [_value-or-nil]))
(-> {:node node, :ns 'hooks.metabase.settings.models.setting-test}
hooks.metabase.settings.models.setting/defsetting
:node
hooks/sexpr)))))

View File

@@ -81,280 +81,268 @@
throttle.core/with-throttling [[:block 1]]
;; the rest of the stuff in this list is generated automatically based on `:style/indent` specs
;; by running [[dev.generate-cljfmt-config/specs]]... use that code to update this list
metabase-enterprise.cache.config-test/with-temp-persist-models [[:block 1]]
metabase-enterprise.impersonation.util-test/with-impersonations! [[:inner 0]]
metabase-enterprise.sandbox.test-util/with-gtaps! [[:inner 0]]
metabase-enterprise.sandbox.test-util/with-gtaps-for-user! [[:inner 0]]
metabase-enterprise.sandbox.test-util/with-user-attributes [[:block 2]]
metabase-enterprise.serialization.test-util/with-dbs [[:inner 0]]
metabase-enterprise.serialization.test-util/with-random-dump-dir [[:block 1]]
metabase-enterprise.serialization.test-util/with-world [[:block 0]]
metabase-enterprise.sso.integrations.jwt-test/with-users-with-email-deleted [[:block 1]]
metabase-enterprise.test/with-gtaps! [[:inner 0]]
metabase-enterprise.test/with-gtaps-for-user! [[:inner 0]]
metabase-enterprise.test/with-user-attributes [[:block 2]]
metabase.actions.actions-test/with-actions-test-data-and-actions-permissively-enabled [[:block 0]]
metabase.actions.test-util/with-actions [[:block 1]]
metabase.actions.test-util/with-actions-disabled [[:block 0]]
metabase.actions.test-util/with-actions-enabled [[:block 0]]
metabase.actions.test-util/with-actions-test-data [[:inner 0]]
metabase.actions.test-util/with-actions-test-data-and-actions-enabled [[:block 0]]
metabase.actions.test-util/with-actions-test-data-tables [[:block 1]]
metabase.actions.test-util/with-empty-db [[:inner 0]]
metabase.actions.test-util/with-temp-test-data [[:inner 0]]
metabase.api.common/check [[:default]]
metabase.api.common/define-routes [[:block 0]]
metabase.api.common/let-404 [[:block 1]]
metabase.app-db.core/with-application-db [[:inner 0]]
metabase.app-db.liquibase/with-liquibase [[:block 1]]
metabase.app-db.liquibase/with-scope-locked [[:block 1]]
metabase.app-db.schema-migrations-test.impl/test-migrations [[:block 2]]
metabase.cmd.copy/step [[:block 1]]
metabase.cmd.copy/with-connection-rollback-only [[:block 1]]
metabase.cmd.copy/with-disabled-db-constraints [[:block 2]]
metabase.collections.api-test/with-collection-hierarchy [[:block 1]]
metabase.collections.api-test/with-french-user-and-personal-collection [[:block 2]]
metabase.collections.api-test/with-some-children-of-collection [[:block 1]]
metabase.collections.models.collection-test/with-collection-hierarchy [[:block 1]]
metabase.collections.models.collection-test/with-current-user-perms-for-collections [[:block 1]]
metabase.collections.models.collection-test/with-personal-and-impersonal-collections [[:block 1]]
metabase.config/build-type-case [[:block 0]]
metabase.dashboards.models.dashboard-test/with-dash-in-collection [[:inner 0]]
metabase.driver-api.core/match [[:inner 0]]
metabase.driver-api.core/match-one [[:inner 0]]
metabase.driver-api.core/replace [[:inner 0]]
metabase.driver-api.core/replace-in [[:inner 0]]
metabase.driver.bigquery-cloud-sdk.query-processor/with-temporal-type [[:default]]
metabase.driver.druid.client/DELETE [[:default]]
metabase.driver.druid.client/GET [[:default]]
metabase.driver.druid.client/POST [[:default]]
metabase.driver.druid.query-processor-test/druid-query [[:block 0]]
metabase.driver.druid.query-processor-test/druid-query-returning-rows [[:block 0]]
metabase.driver.mongo.query-processor/mongo-let [[:block 1]]
metabase.driver.sql-jdbc.actions/with-jdbc-transaction [[:block 1]]
metabase.driver.sql-jdbc.connection/with-connection-spec-for-testing-connection [[:block 1]]
metabase.driver.sql-jdbc.execute.diagnostic/capturing-diagnostic-info [[:block 1]]
metabase.driver.sql.query-processor/apply-top-level-clause [[:default]]
metabase.driver/with-driver [[:block 1]]
metabase.embedding.api.embed-test/do-response-formats [[:block 1]]
metabase.embedding.api.embed-test/with-embedding-enabled-and-new-secret-key [[:block 0]]
metabase.embedding.api.embed-test/with-embedding-enabled-and-temp-card-referencing [[:block 3]]
metabase.embedding.api.embed-test/with-embedding-enabled-and-temp-dashcard-referencing [[:block 3]]
metabase.embedding.api.embed-test/with-new-secret-key [[:block 0]]
metabase.embedding.api.embed-test/with-temp-card [[:block 1]]
metabase.embedding.api.embed-test/with-temp-dashcard [[:block 1]]
metabase.embedding.api.preview-embed-test/with-embedding-enabled-and-new-secret-key [[:block 0]]
metabase.embedding.api.preview-embed-test/with-new-secret-key [[:block 0]]
metabase.lib.binning/with-binning [[:default]]
metabase.lib.core/with-binning [[:default]]
metabase.lib.core/with-join-alias [[:default]]
metabase.lib.core/with-join-conditions [[:default]]
metabase.lib.join/with-join-alias [[:default]]
metabase.lib.join/with-join-conditions [[:default]]
metabase.lib.test-util.macros/$ids [[:inner 0]]
metabase.lib.test-util.macros/mbql-5-query [[:inner 0]]
metabase.lib.test-util.macros/mbql-query [[:inner 0]]
metabase.lib.util.match/match [[:inner 0]]
metabase.lib.util.match/match-one [[:inner 0]]
metabase.lib.util.match/match-lite [[:inner 0]]
metabase.lib.util.match/match-lite-recursive [[:inner 0]]
metabase.lib.util.match/replace [[:inner 0]]
metabase.lib.util.match/replace-in [[:inner 0]]
metabase.models.interface/define-batched-hydration-method [[:inner 0]]
metabase.models.interface/define-simple-hydration-method [[:inner 0]]
metabase.parameters.chain-filter-test/with-clean-field-values-for-field [[:block 1]]
metabase.parameters.chain-filter-test/with-fk-field-to-field-remapping [[:block 0]]
metabase.parameters.chain-filter-test/with-human-readable-values-remapping [[:block 0]]
metabase.permissions.core/allow-changing-all-users-group-members [[:block 0]]
metabase.permissions.core/disable-perms-cache [[:block 0]]
metabase.public-sharing.api-test/with-sharing-enabled-and-temp-card-referencing [[:block 3]]
metabase.public-sharing.api-test/with-sharing-enabled-and-temp-dashcard-referencing [[:block 3]]
metabase.public-sharing.api-test/with-temp-public-card [[:block 1]]
metabase.public-sharing.api-test/with-temp-public-dashboard [[:block 1]]
metabase.public-sharing.api-test/with-temp-public-dashboard-and-card [[:block 1]]
metabase.pulse.api.alert-test/with-alert-in-collection [[:block 1]]
metabase.pulse.dashboard-subscription-test/with-link-card-fixture-for-dashboard [[:block 2]]
metabase.pulse.models.pulse-test/with-dashboard-subscription-in-collection [[:block 1]]
metabase.pulse.models.pulse-test/with-pulse-in-collection [[:inner 0]]
metabase.queries.api.card-test/with-cards-in-readable-collection [[:block 1]]
metabase.queries.api.card-test/with-cards-in-writeable-collection [[:block 1]]
metabase.queries.api.card-test/with-ordered-items [[:inner 0]]
metabase.queries.api.card-test/with-temp-native-card! [[:block 1]]
metabase.queries.api.card-test/with-temp-native-card-with-params! [[:block 1]]
metabase.query-processor-test.pipeline-queries-test/pmbql-query [[:block 1]]
metabase.query-processor-test.pipeline-queries-test/run-pmbql-query [[:block 1]]
metabase.query-processor.debug/debug> [[:block 0]]
metabase.query-processor.error-type/deferror [[:block 1]]
metabase.query-processor.middleware.cache-backend.interface/with-cached-results [[:block 4]]
metabase.query-processor.middleware.process-userland-query-test/with-query-execution [[:block 1]]
metabase.query-processor.store/cached [[:block 1]]
metabase.query-processor.store/with-metadata-provider [[:inner 0]]
metabase.query-processor.streaming/streaming-response [[:block 1]]
metabase.query-processor.test-util/format-rows-by [[:default]]
metabase.request.core/as-admin [[:block 0]]
metabase.request.core/with-current-request [[:inner 0]]
metabase.request.core/with-current-user [[:inner 0]]
metabase.request.core/with-limit-and-offset [[:inner 0]]
metabase.request.session/as-admin [[:block 0]]
metabase.request.session/with-current-user [[:inner 0]]
metabase.sample-data.impl-test/with-temp-sample-database-db [[:block 1]]
metabase.search.api-test/do-test-users [[:block 1]]
metabase.server.middleware.session/as-admin [[:block 0]]
metabase.server.middleware.session/with-current-user [[:inner 0]]
metabase.server.streaming-response-test/with-streaming-response-thread-pool [[:block 0]]
metabase.server.streaming-response-test/with-test-driver-db! [[:block 0]]
metabase.server.streaming-response/streaming-response [[:block 2]]
metabase.session.api/http-401-on-error [[:block 0]]
metabase.settings.core/define-multi-setting [[:inner 0]]
metabase.settings.core/defsetting [[:block 1]]
metabase.settings.models.setting.multi-setting/define-multi-setting [[:inner 0]]
metabase.settings.models.setting/defsetting [[:block 1]]
metabase.sync.fetch-metadata/log-if-error [[:block 1]]
metabase.sync.sync-metadata.fields-test/with-test-db [[:block 0]]
metabase.sync.util/do-with-start-and-finish-debug-logging [[:default]]
metabase.sync.util/sum-for [[:block 1]]
metabase.sync.util/sync-operation [[:block 3]]
metabase.sync.util/with-db-logging-disabled [[:default]]
metabase.sync.util/with-duplicate-ops-prevented [[:default]]
metabase.sync.util/with-emoji-progress-bar [[:block 1]]
metabase.sync.util/with-error-handling [[:block 1]]
metabase.sync.util/with-start-and-finish-logging [[:default]]
metabase.sync.util/with-start-and-finish-logging* [[:default]]
metabase.sync.util/with-sync-events [[:default]]
metabase.task-history.models.task-history/with-task-history [[:block 1]]
metabase.task.core/rerun-on-error [[:block 1]]
metabase.task.impl/rerun-on-error [[:block 1]]
metabase.test.data.datasets/test-driver [[:inner 0]]
metabase.test.data.datasets/test-drivers [[:inner 0]]
metabase.test.data.one-off-dbs/with-blank-db [[:block 0]]
metabase.test.data.one-off-dbs/with-blueberries-db [[:block 0]]
metabase.test.data.users/with-test-user [[:inner 0]]
metabase.test.data/$ids [[:inner 0]]
metabase.test.data/dataset [[:inner 0]]
metabase.test.data/mbql-query [[:inner 0]]
metabase.test.data/native-query [[:block 0]]
metabase.test.data/query [[:block 1]]
metabase.test.data/run-mbql-query [[:inner 0]]
metabase.test.data/with-empty-h2-app-db! [[:block 0]]
metabase.test.data/with-temp-copy-of-db [[:block 0]]
metabase.test.util.public-settings/with-additional-premium-features [[:block 1]]
metabase.test.util.public-settings/with-premium-features [[:block 1]]
metabase.test.util.thread-local/test-helpers-set-global-values! [[:block 0]]
metabase.test.util/discard-setting-changes [[:block 1]]
metabase.test.util/let-url [[:block 1]]
metabase.test.util/throw-if-called! [[:block 1]]
metabase.test.util/with-temp-scheduler [[:block 0]]
metabase.test.util/with-temp-vals-in-db [[:block 3]]
metabase.test.util/with-user-in-groups [[:inner 0]]
metabase.test/$ids [[:inner 0]]
metabase.test/as-admin [[:inner 0]]
metabase.test/dataset [[:inner 0]]
metabase.test/discard-setting-changes [[:block 1]]
metabase.test/format-rows-by [[:default]]
metabase.test/let-url [[:block 1]]
metabase.test/mbql-query [[:inner 0]]
metabase.test/native-query [[:block 0]]
metabase.test/query [[:block 1]]
metabase.test/run-mbql-query [[:inner 0]]
metabase.test/test-driver [[:inner 0]]
metabase.test/test-drivers [[:inner 0]]
metabase.test/test-helpers-set-global-values! [[:block 0]]
metabase.test/throw-if-called! [[:block 1]]
metabase.test/with-actions [[:block 1]]
metabase.test/with-actions-disabled [[:block 0]]
metabase.test/with-actions-enabled [[:block 0]]
metabase.test/with-actions-test-data [[:inner 0]]
metabase.test/with-actions-test-data-and-actions-enabled [[:block 0]]
metabase.test/with-actions-test-data-tables [[:block 1]]
metabase.test/with-additional-premium-features [[:block 1]]
metabase.test/with-current-user [[:inner 0]]
metabase.test/with-driver [[:block 1]]
metabase.test/with-empty-db [[:inner 0]]
metabase.test/with-empty-h2-app-db! [[:block 0]]
metabase.test/with-metadata-provider [[:inner 0]]
metabase.test/with-premium-features [[:block 1]]
metabase.test/with-temp [[:inner 0]]
metabase.test/with-temp-copy-of-db [[:block 0]]
metabase.test/with-temp-scheduler [[:block 0]]
metabase.test/with-temp-test-data [[:inner 0]]
metabase.test/with-temp-vals-in-db [[:block 3]]
metabase.test/with-test-user [[:inner 0]]
metabase.test/with-user-in-groups [[:inner 0]]
metabase.timeseries-query-processor-test.util/test-timeseries-drivers [[:block 0]]
metabase.timeseries-query-processor-test/run-mbql-query [[:inner 0]]
metabase.upload.impl-test/with-upload-table! [[:inner 0]]
metabase.upload.types/with-currency [[:default]]
metabase.upload.types/with-parens [[:default]]
metabase.util/for-map [[:block 1]]
metabase.util.async/cancelable-thread [[:block 0]]
metabase.util.date-2/with-time-zone-same-instant [[:default]]
metabase.util.encryption-test/with-secret-key [[:block 1]]
metabase.util.honey-sql-2/with-database-type-info [[:default]]
metabase.util.i18n/deferred-trs [[:default]]
metabase.util.i18n/deferred-tru [[:default]]
metabase.util.i18n/trs [[:default]]
metabase.util.i18n/trsn [[:default]]
metabase.util.i18n/tru [[:default]]
metabase.util.i18n/trun [[:default]]
metabase.util.jvm/auto-retry [[:block 1]]
metabase.util.jvm/varargs [[:block 1]]
metabase.util.jvm/with-us-locale [[:block 0]]
metabase.util.malli/disable-enforcement [[:block 0]]
metabase.util.malli/with [[:default]]
metabase.util.malli/with-api-error-message [[:default]]
metabase.util.namespaces/import-fns [[:default]]
metabase.util.queue-test/await-test-while [[:block 1]]
metabase.util/auto-retry [[:block 1]]
metabase.util/case-enum [[:block 1]]
metabase.util/ignore-exceptions [[:block 0]]
metabase.util/or-with [[:block 1]]
metabase.util/profile [[:block 1]]
metabase.util/prog1 [[:inner 0]]
metabase.util/varargs [[:inner 0]]
metabase.util/with-us-locale [[:block 0]]
metabuild-common.core/exit-when-finished-nonzero-on-exception [[:block 0]]
metabuild-common.core/step [[:block 1]]
metabuild-common.entrypoint/exit-when-finished-nonzero-on-exception [[:block 0]]
metabuild-common.steps/step [[:block 1]]
methodical.core/defmethod [[:inner 0]]
methodical.core/defmulti [[:inner 0]]
methodical.core/with-dispatcher [[:default]]
methodical.core/with-method-table [[:default]]
methodical.core/with-prefers [[:default]]
methodical.interface/with-dispatcher [[:default]]
methodical.interface/with-method-table [[:default]]
methodical.interface/with-prefers [[:default]]
methodical.macros/defmethod [[:inner 0]]
methodical.macros/defmulti [[:inner 0]]
saml20-clj.sp.response/validate-confirmation-datas [[:block 1]]
toucan2.connection/with-transaction [[:block 1]]
toucan2.core/build [[:block 0]]
toucan2.core/compile [[:block 0]]
toucan2.core/debug [[:block 0]]
toucan2.core/define-after-insert [[:inner 0]]
toucan2.core/define-after-select [[:inner 0]]
toucan2.core/define-after-update [[:inner 0]]
toucan2.core/define-before-delete [[:inner 0]]
toucan2.core/define-before-insert [[:inner 0]]
toucan2.core/define-before-select [[:inner 0]]
toucan2.core/define-default-fields [[:inner 0]]
toucan2.core/define-named-query [[:block 1]]
toucan2.core/deftransforms [[:block 1]]
toucan2.core/with-transaction [[:block 1]]
toucan2.tools.after-insert/define-after-insert [[:inner 0]]
toucan2.tools.after-select/define-after-select [[:inner 0]]
toucan2.tools.after-update/define-after-update [[:inner 0]]
toucan2.tools.after/define-after-impl [[:default]]
toucan2.tools.before-delete/define-before-delete [[:inner 0]]
toucan2.tools.before-insert/define-before-insert [[:inner 0]]
toucan2.tools.before-select/define-before-select [[:inner 0]]
toucan2.tools.compile/build [[:block 0]]
toucan2.tools.compile/compile [[:block 0]]
toucan2.tools.compile/resolved [[:block 0]]
toucan2.tools.debug/debug [[:block 0]]
toucan2.tools.default-fields/define-default-fields [[:inner 0]]
toucan2.tools.named-query/define-named-query [[:block 1]]
toucan2.tools.simple-out-transform/define-out-transform [[:inner 0]]
toucan2.tools.transformed/deftransforms [[:block 1]]
toucan2.tools.with-temp/with-temp [[:inner 0]]
toucan2.util/try-with-error-context [[:inner 0]]}}
metabase-enterprise.impersonation.util-test/with-impersonations! [[:inner 0]]
metabase-enterprise.cache.config-test/with-temp-persist-models [[:block 1]]
metabase-enterprise.sandbox.test-util/with-gtaps! [[:inner 0]]
metabase-enterprise.sandbox.test-util/with-gtaps-for-user! [[:inner 0]]
metabase-enterprise.sandbox.test-util/with-user-attributes [[:block 2]]
metabase-enterprise.serialization.test-util/with-dbs [[:inner 0]]
metabase-enterprise.serialization.test-util/with-random-dump-dir [[:block 1]]
metabase-enterprise.serialization.test-util/with-world [[:block 0]]
metabase-enterprise.sso.integrations.jwt-test/with-users-with-email-deleted [[:block 1]]
metabase-enterprise.test/with-gtaps! [[:inner 0]]
metabase-enterprise.test/with-gtaps-for-user! [[:inner 0]]
metabase-enterprise.test/with-user-attributes [[:block 2]]
metabase.actions.actions-test/with-actions-test-data-and-actions-permissively-enabled [[:block 0]]
metabase.actions.test-util/with-actions [[:block 1]]
metabase.actions.test-util/with-actions-disabled [[:block 0]]
metabase.actions.test-util/with-actions-enabled [[:block 0]]
metabase.actions.test-util/with-actions-test-data [[:inner 0]]
metabase.actions.test-util/with-actions-test-data-and-actions-enabled [[:block 0]]
metabase.actions.test-util/with-actions-test-data-tables [[:block 1]]
metabase.actions.test-util/with-empty-db [[:inner 0]]
metabase.actions.test-util/with-temp-test-data [[:inner 0]]
metabase.api.card-test/with-cards-in-readable-collection [[:block 1]]
metabase.api.card-test/with-cards-in-writeable-collection [[:block 1]]
metabase.api.card-test/with-ordered-items [[:inner 0]]
metabase.api.card-test/with-temp-native-card! [[:block 1]]
metabase.api.card-test/with-temp-native-card-with-params! [[:block 1]]
metabase.api.collection-test/with-collection-hierarchy [[:block 1]]
metabase.api.collection-test/with-french-user-and-personal-collection [[:block 2]]
metabase.api.collection-test/with-some-children-of-collection [[:block 1]]
metabase.api.common/check [[:default]]
metabase.api.common/define-routes [[:block 0]]
metabase.api.common/let-404 [[:block 1]]
metabase.api.embed-test/do-response-formats [[:block 1]]
metabase.api.embed-test/with-embedding-enabled-and-new-secret-key [[:block 0]]
metabase.api.embed-test/with-embedding-enabled-and-temp-card-referencing [[:block 3]]
metabase.api.embed-test/with-embedding-enabled-and-temp-dashcard-referencing [[:block 3]]
metabase.api.embed-test/with-new-secret-key [[:block 0]]
metabase.api.embed-test/with-temp-card [[:block 1]]
metabase.api.embed-test/with-temp-dashcard [[:block 1]]
metabase.api.preview-embed-test/with-embedding-enabled-and-new-secret-key [[:block 0]]
metabase.api.preview-embed-test/with-new-secret-key [[:block 0]]
metabase.cmd.copy/step [[:block 1]]
metabase.cmd.copy/with-connection-rollback-only [[:block 1]]
metabase.cmd.copy/with-disabled-db-constraints [[:block 2]]
metabase.config/build-type-case [[:block 0]]
metabase.dashboard-subscription-test/with-link-card-fixture-for-dashboard [[:block 2]]
metabase.db.liquibase/with-liquibase [[:block 1]]
metabase.db.liquibase/with-scope-locked [[:block 1]]
metabase.db.schema-migrations-test.impl/test-migrations [[:block 2]]
metabase.db/with-application-db [[:inner 0]]
metabase.driver.bigquery-cloud-sdk.query-processor/with-temporal-type [[:default]]
metabase.driver.druid.client/DELETE [[:default]]
metabase.driver.druid.client/GET [[:default]]
metabase.driver.druid.client/POST [[:default]]
metabase.driver.druid.query-processor-test/druid-query [[:block 0]]
metabase.driver.druid.query-processor-test/druid-query-returning-rows [[:block 0]]
metabase.driver.mongo.query-processor/mongo-let [[:block 1]]
metabase.driver.sql-jdbc.actions/with-jdbc-transaction [[:block 1]]
metabase.driver.sql-jdbc.connection/with-connection-spec-for-testing-connection [[:block 1]]
metabase.driver.sql-jdbc.execute.diagnostic/capturing-diagnostic-info [[:block 1]]
metabase.driver.sql.query-processor/apply-top-level-clause [[:default]]
metabase.driver/with-driver [[:block 1]]
metabase.lib.binning/with-binning [[:default]]
metabase.lib.core/with-binning [[:default]]
metabase.lib.core/with-join-alias [[:default]]
metabase.lib.core/with-join-conditions [[:default]]
metabase.lib.join/with-join-alias [[:default]]
metabase.lib.join/with-join-conditions [[:default]]
metabase.lib.test-util.macros/$ids [[:inner 0]]
metabase.lib.test-util.macros/mbql-query [[:inner 0]]
metabase.lib.util.match/match [[:inner 0]]
metabase.lib.util.match/match-one [[:inner 0]]
metabase.lib.util.match/replace [[:inner 0]]
metabase.lib.util.match/replace-in [[:inner 0]]
metabase.models.collection-test/with-collection-hierarchy [[:block 1]]
metabase.models.collection-test/with-current-user-perms-for-collections [[:block 1]]
metabase.models.collection-test/with-personal-and-impersonal-collections [[:block 1]]
metabase.models.dashboard-test/with-dash-in-collection [[:inner 0]]
metabase.models.interface/define-batched-hydration-method [[:inner 0]]
metabase.models.interface/define-simple-hydration-method [[:inner 0]]
metabase.models.params.chain-filter-test/with-clean-field-values-for-field [[:block 1]]
metabase.models.params.chain-filter-test/with-fk-field-to-field-remapping [[:block 0]]
metabase.models.params.chain-filter-test/with-human-readable-values-remapping [[:block 0]]
metabase.models.setting.multi-setting/define-multi-setting [[:inner 0]]
metabase.models.setting/defsetting [[:block 1]]
metabase.models.task-history/with-task-history [[:block 1]]
metabase.permissions.core/allow-changing-all-users-group-members [[:block 0]]
metabase.permissions.core/disable-perms-cache [[:block 0]]
metabase.public-sharing.api-test/with-sharing-enabled-and-temp-card-referencing [[:block 3]]
metabase.public-sharing.api-test/with-sharing-enabled-and-temp-dashcard-referencing [[:block 3]]
metabase.public-sharing.api-test/with-temp-public-card [[:block 1]]
metabase.public-sharing.api-test/with-temp-public-dashboard [[:block 1]]
metabase.public-sharing.api-test/with-temp-public-dashboard-and-card [[:block 1]]
metabase.pulse.api.alert-test/with-alert-in-collection [[:block 1]]
metabase.pulse.models.pulse-test/with-dashboard-subscription-in-collection [[:block 1]]
metabase.pulse.models.pulse-test/with-pulse-in-collection [[:inner 0]]
metabase.query-processor-test.pipeline-queries-test/pmbql-query [[:block 1]]
metabase.query-processor-test.pipeline-queries-test/run-pmbql-query [[:block 1]]
metabase.query-processor.debug/debug> [[:block 0]]
metabase.query-processor.error-type/deferror [[:block 1]]
metabase.query-processor.middleware.cache-backend.interface/with-cached-results [[:block 4]]
metabase.query-processor.middleware.process-userland-query-test/with-query-execution [[:block 1]]
metabase.query-processor.store/cached [[:block 1]]
metabase.query-processor.store/with-metadata-provider [[:inner 0]]
metabase.query-processor.streaming/streaming-response [[:block 1]]
metabase.query-processor.test-util/format-rows-by [[:default]]
metabase.request.core/as-admin [[:block 0]]
metabase.request.core/with-current-request [[:inner 0]]
metabase.request.core/with-current-user [[:inner 0]]
metabase.request.core/with-limit-and-offset [[:inner 0]]
metabase.request.session/as-admin [[:block 0]]
metabase.request.session/with-current-user [[:inner 0]]
metabase.sample-data-test/with-temp-sample-database-db [[:block 1]]
metabase.search.api-test/do-test-users [[:block 1]]
metabase.server.middleware.session/as-admin [[:block 0]]
metabase.server.middleware.session/with-current-user [[:inner 0]]
metabase.server.streaming-response-test/with-streaming-response-thread-pool [[:block 0]]
metabase.server.streaming-response-test/with-test-driver-db! [[:block 0]]
metabase.server.streaming-response/streaming-response [[:block 2]]
metabase.session.api/http-401-on-error [[:block 0]]
metabase.sync.fetch-metadata/log-if-error [[:block 1]]
metabase.sync.sync-metadata.fields-test/with-test-db [[:block 0]]
metabase.sync.util/do-with-start-and-finish-debug-logging [[:default]]
metabase.sync.util/sum-for [[:block 1]]
metabase.sync.util/sync-operation [[:block 3]]
metabase.sync.util/with-db-logging-disabled [[:default]]
metabase.sync.util/with-duplicate-ops-prevented [[:default]]
metabase.sync.util/with-emoji-progress-bar [[:block 1]]
metabase.sync.util/with-error-handling [[:block 1]]
metabase.sync.util/with-start-and-finish-logging [[:default]]
metabase.sync.util/with-start-and-finish-logging* [[:default]]
metabase.sync.util/with-sync-events [[:default]]
metabase.task/rerun-on-error [[:block 1]]
metabase.test/$ids [[:inner 0]]
metabase.test/as-admin [[:inner 0]]
metabase.test/dataset [[:inner 0]]
metabase.test/discard-setting-changes [[:block 1]]
metabase.test/format-rows-by [[:default]]
metabase.test/let-url [[:block 1]]
metabase.test/mbql-query [[:inner 0]]
metabase.test/native-query [[:block 0]]
metabase.test/query [[:block 1]]
metabase.test/run-mbql-query [[:inner 0]]
metabase.test/test-driver [[:inner 0]]
metabase.test/test-drivers [[:inner 0]]
metabase.test/test-helpers-set-global-values! [[:block 0]]
metabase.test/throw-if-called! [[:block 1]]
metabase.test/with-actions [[:block 1]]
metabase.test/with-actions-disabled [[:block 0]]
metabase.test/with-actions-enabled [[:block 0]]
metabase.test/with-actions-test-data [[:inner 0]]
metabase.test/with-actions-test-data-and-actions-enabled [[:block 0]]
metabase.test/with-actions-test-data-tables [[:block 1]]
metabase.test/with-additional-premium-features [[:block 1]]
metabase.test/with-current-user [[:inner 0]]
metabase.test/with-driver [[:block 1]]
metabase.test/with-empty-db [[:inner 0]]
metabase.test/with-empty-h2-app-db [[:block 0]]
metabase.test/with-metadata-provider [[:inner 0]]
metabase.test/with-premium-features [[:block 1]]
metabase.test/with-temp [[:inner 0]]
metabase.test/with-temp-copy-of-db [[:block 0]]
metabase.test/with-temp-scheduler [[:block 0]]
metabase.test/with-temp-test-data [[:inner 0]]
metabase.test/with-temp-vals-in-db [[:block 3]]
metabase.test/with-test-user [[:inner 0]]
metabase.test/with-user-in-groups [[:inner 0]]
metabase.test.data/$ids [[:inner 0]]
metabase.test.data/dataset [[:inner 0]]
metabase.test.data/mbql-query [[:inner 0]]
metabase.test.data/native-query [[:block 0]]
metabase.test.data/query [[:block 1]]
metabase.test.data/run-mbql-query [[:inner 0]]
metabase.test.data/with-empty-h2-app-db [[:block 0]]
metabase.test.data/with-temp-copy-of-db [[:block 0]]
metabase.test.data.datasets/test-driver [[:inner 0]]
metabase.test.data.datasets/test-drivers [[:inner 0]]
metabase.test.data.one-off-dbs/with-blank-db [[:block 0]]
metabase.test.data.one-off-dbs/with-blueberries-db [[:block 0]]
metabase.test.data.users/with-test-user [[:inner 0]]
metabase.test.util/discard-setting-changes [[:block 1]]
metabase.test.util/let-url [[:block 1]]
metabase.test.util/throw-if-called! [[:block 1]]
metabase.test.util/with-temp-scheduler [[:block 0]]
metabase.test.util/with-temp-vals-in-db [[:block 3]]
metabase.test.util/with-user-in-groups [[:inner 0]]
metabase.test.util.public-settings/with-additional-premium-features [[:block 1]]
metabase.test.util.public-settings/with-premium-features [[:block 1]]
metabase.test.util.thread-local/test-helpers-set-global-values! [[:block 0]]
metabase.timeseries-query-processor-test.util/test-timeseries-drivers [[:block 0]]
metabase.timeseries-query-processor-test/run-mbql-query [[:inner 0]]
metabase.upload-test/with-upload-table! [[:inner 0]]
metabase.upload.types/with-currency [[:default]]
metabase.upload.types/with-parens [[:default]]
metabase.util/auto-retry [[:block 1]]
metabase.util/case-enum [[:block 1]]
metabase.util/ignore-exceptions [[:block 0]]
metabase.util/or-with [[:block 1]]
metabase.util/profile [[:block 1]]
metabase.util/prog1 [[:inner 0]]
metabase.util/varargs [[:inner 0]]
metabase.util/with-us-locale [[:block 0]]
metabase.util.async/cancelable-thread [[:block 0]]
metabase.util.date-2/with-time-zone-same-instant [[:default]]
metabase.util.encryption-test/with-secret-key [[:block 1]]
metabase.util.honey-sql-2/with-database-type-info [[:default]]
metabase.util.i18n/deferred-trs [[:default]]
metabase.util.i18n/deferred-tru [[:default]]
metabase.util.i18n/trs [[:default]]
metabase.util.i18n/trsn [[:default]]
metabase.util.i18n/tru [[:default]]
metabase.util.i18n/trun [[:default]]
metabase.util.jvm/auto-retry [[:block 1]]
metabase.util.jvm/varargs [[:block 1]]
metabase.util.jvm/with-us-locale [[:block 0]]
metabase.util.malli/disable-enforcement [[:block 0]]
metabase.util.malli/with [[:default]]
metabase.util.malli/with-api-error-message [[:default]]
metabase.util.namespaces/import-fns [[:default]]
metabuild-common.core/exit-when-finished-nonzero-on-exception [[:block 0]]
metabuild-common.core/step [[:block 1]]
metabuild-common.entrypoint/exit-when-finished-nonzero-on-exception [[:block 0]]
metabuild-common.steps/step [[:block 1]]
methodical.core/defmethod [[:inner 0]]
methodical.core/defmulti [[:inner 0]]
methodical.core/with-dispatcher [[:default]]
methodical.core/with-method-table [[:default]]
methodical.core/with-prefers [[:default]]
methodical.interface/with-dispatcher [[:default]]
methodical.interface/with-method-table [[:default]]
methodical.interface/with-prefers [[:default]]
methodical.macros/defmethod [[:inner 0]]
methodical.macros/defmulti [[:inner 0]]
saml20-clj.sp.response/validate-confirmation-datas [[:block 1]]
toucan2.connection/with-transaction [[:block 1]]
toucan2.core/build [[:block 0]]
toucan2.core/compile [[:block 0]]
toucan2.core/debug [[:block 0]]
toucan2.core/define-after-insert [[:inner 0]]
toucan2.core/define-after-select [[:inner 0]]
toucan2.core/define-after-update [[:inner 0]]
toucan2.core/define-before-delete [[:inner 0]]
toucan2.core/define-before-insert [[:inner 0]]
toucan2.core/define-before-select [[:inner 0]]
toucan2.core/define-default-fields [[:inner 0]]
toucan2.core/define-named-query [[:block 1]]
toucan2.core/deftransforms [[:block 1]]
toucan2.core/with-transaction [[:block 1]]
toucan2.tools.after/define-after-impl [[:default]]
toucan2.tools.after-insert/define-after-insert [[:inner 0]]
toucan2.tools.after-select/define-after-select [[:inner 0]]
toucan2.tools.after-update/define-after-update [[:inner 0]]
toucan2.tools.before-delete/define-before-delete [[:inner 0]]
toucan2.tools.before-insert/define-before-insert [[:inner 0]]
toucan2.tools.before-select/define-before-select [[:inner 0]]
toucan2.tools.compile/build [[:block 0]]
toucan2.tools.compile/compile [[:block 0]]
toucan2.tools.compile/resolved [[:block 0]]
toucan2.tools.debug/debug [[:block 0]]
toucan2.tools.default-fields/define-default-fields [[:inner 0]]
toucan2.tools.named-query/define-named-query [[:block 1]]
toucan2.tools.simple-out-transform/define-out-transform [[:inner 0]]
toucan2.tools.transformed/deftransforms [[:block 1]]
toucan2.tools.with-temp/with-temp [[:inner 0]]
toucan2.util/try-with-error-context [[:inner 0]]}}

View File

@@ -1,11 +1,11 @@
FROM mcr.microsoft.com/vscode/devcontainers/java:21
FROM mcr.microsoft.com/vscode/devcontainers/java:11
# Set up nodesource, install node, yarn, fontconfig for static viz, rlwrap for dev ergonomics
RUN ( curl -fsSL https://deb.nodesource.com/setup_22.x | bash ) \
RUN ( curl -fsSL https://deb.nodesource.com/setup_18.x | bash ) \
&& export DEBIAN_FRONTEND=noninteractive \
&& apt-key adv --refresh-keys --keyserver keyserver.ubuntu.com \
&& apt-get update && apt-get -y install --no-install-recommends nodejs yarn rlwrap fontconfig
# install Clojure
RUN curl https://download.clojure.org/install/linux-install-1.12.0.1488.sh | bash
RUN curl https://download.clojure.org/install/linux-install-1.11.1.1262.sh | bash

View File

@@ -1,20 +1,17 @@
/* eslint-disable import/no-commonjs */
/* eslint-disable no-undef */
const path = require("path");
// `postcss-modules` lints css modules class names, but it currently crashes
// eslint on vscode. If you use webstorm or want to run the lint for the cli, you
// can use this flag to enable it. This is set to true in CI
const shouldLintCssModules =
process.env.LINT_CSS_MODULES === "true" || process.env.CI;
const plugins = ["react", "no-only-tests", "ttag", "i18next"];
const plugins = ["react", "no-only-tests"];
if (shouldLintCssModules) {
plugins.push("postcss-modules");
}
module.exports = {
ignorePatterns: ["!.storybook"],
rules: {
strict: [2, "never"],
"no-undef": 2,
@@ -49,7 +46,6 @@ module.exports = {
],
curly: [1, "all"],
eqeqeq: [1, "smart"],
"import/no-duplicates": ["warn", { considerQueryString: true }],
"import/no-default-export": 2,
"import/no-named-as-default": 0,
"import/no-commonjs": 1,
@@ -122,7 +118,7 @@ module.exports = {
],
},
],
complexity: ["error", { max: 55 }],
complexity: ["error", { max: 54 }],
...(shouldLintCssModules
? {
"postcss-modules/no-undef-class": "error",
@@ -153,13 +149,11 @@ module.exports = {
"plugin:import/typescript",
"plugin:depend/recommended",
"plugin:storybook/recommended",
"plugin:i18next/recommended",
],
settings: {
"import/internal-regex": "^metabase/|^metabase-lib/",
"import/resolver": {
webpack: {
config: path.resolve(__dirname, "./rspack.main.config.js"),
typescript: true,
},
},
@@ -180,7 +174,6 @@ module.exports = {
{
files: ["*.js", "*.jsx", "*.ts", "*.tsx"],
rules: {
"jtag-missing-key": "error",
"no-unconditional-metabase-links-render": "error",
"no-color-literals": "error",
"no-literal-metabase-strings": "error",
@@ -202,18 +195,12 @@ module.exports = {
"*.unit.spec.*",
"frontend/src/metabase/admin/**/*",
"frontend/src/metabase/setup/**/*",
"enterprise/frontend/src/metabase-enterprise/whitelabel/**/*",
"frontend/lint/**/*",
"*.stories.*",
"**/.storybook/*",
"stories-data.*",
"e2e/**/*",
"**/tests/*",
"release/**/*",
"rspack.config.js",
"rspack.main.config.js",
"rspack.embedding-sdk-package.config.js",
"rspack.embedding-sdk-bundle.config.js",
],
rules: {
"no-color-literals": "off",
@@ -221,21 +208,6 @@ module.exports = {
"no-literal-metabase-strings": "off",
},
},
{
files: [
"*.unit.spec.*",
"frontend/lint/**/*",
"*.stories.*",
"stories-data.*",
"e2e/**/*",
"**/tests/*",
"release/**/*",
"rspack.main.config.js",
],
rules: {
"i18next/no-literal-string": "off",
},
},
{
extends: ["plugin:@typescript-eslint/recommended"],
files: ["*.ts", "*.tsx"],
@@ -283,13 +255,6 @@ module.exports = {
],
rules: {
"jest/valid-title": ["error", { ignoreTypeOfDescribeName: true }],
"jest/expect-expect": [
"error",
{
assertFunctionNames: ["expect*", "assert*"],
additionalTestBlockFunctions: [],
},
],
},
},
{
@@ -319,33 +284,9 @@ module.exports = {
selector: "Literal[value=/mb-base-color-/]",
message:
"You may not use base colors in the application, use semantic colors instead. (see colors.module.css)",
},
}
],
},
},
{
files: ["frontend/src/metabase/query_builder/**/*"],
rules: {
"import/no-cycle": "error",
},
},
{
files: ["docs/**/snippets/**/*.{ts,tsx,js,jsx}"],
rules: {
"@typescript-eslint/no-unused-vars": "off",
"@typescript-eslint/no-var-requires": "off",
"import/no-commonjs": "off",
"import/no-default-export": "off",
"import/order": "off",
"import/no-unresolved": "off",
"no-color-literals": "off",
},
},
{
files: ["frontend/build/**/*.js"],
rules: {
"import/no-commonjs": "off",
},
},
],
};

51
.github/CODEOWNERS vendored
View File

@@ -1,29 +1,24 @@
# Before changing this file, make sure you have read how we use it:
# https://www.notion.so/metabase/Pull-Request-Code-Review-Guide-374f05df889748b69d67af753f9bc9b8?pvs=4#36550c04ceab40b5a8b8bee6264b090c
enterprise/frontend/src/embedding-sdk-package @metabase/embedding
enterprise/frontend/src/embedding-sdk-bundle @metabase/embedding
enterprise/frontend/src/embedding-sdk-shared @metabase/embedding
enterprise/frontend/src/embedding-sdk @metabase/embedding
frontend/src/metabase/embedding-sdk @metabase/embedding
frontend/src/metabase/public/containers/PublicOrEmbeddedQuestion @metabase/embedding
frontend/src/metabase/public/containers/PublicOrEmbeddedDashboard @metabase/embedding
frontend/src/metabase/ui @metabase/core-frontend-ux-west
frontend/src/metabase/ui @metabase/core-frontend-admin-webapp
# snowplow/* @metabase/data
src/metabase/analytics/prometheus.clj @metabase/cloud-ops
.github @metabase/devexp
.clj-kondo @metabase/devexp
deps.edn @metabase/backend-developers
docs @metabase/tech-writers
docs/developers-guide # exclude from tech-writer ownership
#
# Backend modules <=> teams that own them
#
# modules/drivers @metabase/drivers
src/metabase/driver/sql_jdbc/actions.clj @metabase/core-ux-east
test/metabase/driver/sql_jdbc/actions_test.clj @metabase/core-ux-east
# enterprise/backend/src/metabase_enterprise/advanced_config @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/advanced_config @metabase/core-backend-admin-webapp
@@ -55,8 +50,8 @@ test/metabase/driver/sql_jdbc/actions_test.clj @metabase/core-ux-east
# enterprise/backend/src/metabase_enterprise/core @metabase/devexp
# enterprise/backend/test/metabase_enterprise/core @metabase/devexp
# enterprise/backend/src/metabase_enterprise/dashboard_subscription_filters @metabase/core-ux-east
# enterprise/backend/test/metabase_enterprise/dashboard_subscription_filters @metabase/core-ux-east
# enterprise/backend/src/metabase_enterprise/dashboard_subscription_filters @metabase/core-workflows
# enterprise/backend/test/metabase_enterprise/dashboard_subscription_filters @metabase/core-workflows
# enterprise/backend/src/metabase_enterprise/enhancements @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/enhancements @metabase/core-backend-admin-webapp
@@ -73,11 +68,11 @@ test/metabase/driver/sql_jdbc/actions_test.clj @metabase/core-ux-east
# enterprise/backend/src/metabase_enterprise/scim @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/scim @metabase/core-backend-admin-webapp
# enterprise/backend/src/metabase_enterprise/search ...
# enterprise/backend/test/metabase_enterprise/search ...
# enterprise/backend/src/metabase_enterprise/search @metabase/core-workflows
# enterprise/backend/test/metabase_enterprise/search @metabase/core-workflows
# enterprise/backend/src/metabase_enterprise/serialization @metabase/core-ux-east
# enterprise/backend/test/metabase_enterprise/serialization @metabase/core-ux-east
# enterprise/backend/src/metabase_enterprise/serialization @metabase/core-workflows
# enterprise/backend/test/metabase_enterprise/serialization @metabase/core-workflows
# enterprise/backend/src/metabase_enterprise/snippet_collections @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/snippet_collections @metabase/core-backend-admin-webapp
@@ -97,8 +92,8 @@ test/metabase/driver/sql_jdbc/actions_test.clj @metabase/core-ux-east
# enterprise/backend/src/metabase_enterprise/upload_management @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/upload_management @metabase/core-backend-admin-webapp
src/metabase/actions @metabase/core-ux-east
test/metabase/actions @metabase/core-ux-east
# src/metabase/actions @metabase/core-backend-admin-webapp
# test/metabase/actions @metabase/core-backend-admin-webapp
# src/metabase/activity_feed @metabase/core-backend-admin-webapp
# test/metabase/activity_feed @metabase/core-backend-admin-webapp
@@ -124,11 +119,11 @@ test/metabase/actions @metabase/core-ux-east
# src/metabase/cmd @metabase/devexp
# test/metabase/cmd @metabase/devexp
# src/metabase/channel @metabase/core-ux-east
# test/metabase/channel @metabase/core-ux-east
# src/metabase/channel @metabase/core-workflows
# test/metabase/channel @metabase/core-workflows
# src/metabase/cloud_migration ...
# test/metabase/cloud_migration ...
# src/metabase/cloud_migration @metabase/core-workflows
# test/metabase/cloud_migration @metabase/core-workflows
# src/metabase/cmd @metabase/devexp
# test/metabase/cmd @metabase/devexp
@@ -160,8 +155,8 @@ test/metabase/actions @metabase/core-ux-east
# src/metabase/indexed_entities @metabase/core-backend-querying
# test/metabase/indexed_entities @metabase/core-backend-querying
# src/metabase/integrations ...
# test/metabase/integrations ...
# src/metabase/integrations @metabase/core-workflows
# test/metabase/integrations @metabase/core-workflows
# src/metabase/legacy_mbql @metabase/core-backend-querying
# test/metabase/legacy_mbql @metabase/core-backend-querying
@@ -184,8 +179,8 @@ test/metabase/actions @metabase/core-ux-east
# src/metabase/moderation @metabase/core-backend-admin-webapp
# test/metabase/moderation @metabase/core-backend-admin-webapp
# src/metabase/notification @metabase/core-ux-east
# test/metabase/notification @metabase/core-ux-east
# src/metabase/notification @metabase/core-workflows
# test/metabase/notification @metabase/core-workflows
# src/metabase/permissions @metabase/core-backend-admin-webapp
# test/metabase/permissions @metabase/core-backend-admin-webapp
@@ -202,8 +197,8 @@ test/metabase/actions @metabase/core-ux-east
# src/metabase/public_sharing @metabase/core-backend-admin-webapp
# test/metabase/public_sharing @metabase/core-backend-admin-webapp
# src/metabase/pulse @metabase/core-ux-east
# test/metabase/pulse @metabase/core-ux-east
# src/metabase/pulse @metabase/core-workflows
# test/metabase/pulse @metabase/core-workflows
# src/metabase/query_analysis @metabase/core-backend-querying
# test/metabase/query_analysis @metabase/core-backend-querying
@@ -220,8 +215,8 @@ test/metabase/actions @metabase/core-ux-east
# src/metabase/sample_data @metabase/core-backend-admin-webapp
# test/metabase/sample_data @metabase/core-backend-admin-webapp
# src/metabase/search ...
# test/metabase/search ...
# src/metabase/search @metabase/core-workflows
# test/metabase/search @metabase/core-workflows
# src/metabase/segments @metabase/core-backend-querying
# test/metabase/segments @metabase/core-backend-querying

View File

@@ -63,8 +63,7 @@ body:
attributes:
label: Information about your Metabase installation
description: |
For versions >= 56, copy and paste the Diagnostic info from `Admin -> Tools -> Help`.
For versions < 56, copy and paste the Diagnostic info from `Admin -> Troubleshooting`.
Copy and paste the JSON from `Admin -> Troubleshooting`.
_This will be automatically formatted into code, so no need for backticks._
placeholder: |
{"browser-info": {

View File

@@ -1,5 +1,4 @@
name: await-port
description: Wait for localhost port to be ready
name: Wait for localhost port to be ready
inputs:
port:
@@ -11,18 +10,6 @@ runs:
- name: Wait for port ${{ inputs.port }} to be ready
shell: bash
run: |
echo "Waiting for port ${{ inputs.port }} to be ready..."
timeout=300 # 5 minutes timeout
elapsed=0
while ! nc -z localhost ${{ inputs.port }}; do
if [ $elapsed -ge $timeout ]; then
echo "❌ Timeout waiting for port ${{ inputs.port }} after ${timeout} seconds"
exit 1
fi
sleep 1
elapsed=$((elapsed + 1))
if [ $((elapsed % 10)) -eq 0 ]; then
echo " Still waiting for port ${{ inputs.port }}... ($elapsed seconds)"
fi
sleep 0.1;
done
echo "✅ Port ${{ inputs.port }} is ready!"

View File

@@ -0,0 +1,116 @@
name: Build ee-extra
description: Build Metabase Enterprise Edition with extra features
inputs:
java-version:
required: true
default: 21
iam-role:
description: "The IAM role to assume"
required: true
runs:
using: composite
steps:
- name: Prepare Java
uses: actions/setup-java@v3
with:
java-version: ${{ inputs.java-version }}
distribution: 'temurin'
- name: Reveal its original version.properties
run: jar xf ./bin/docker/metabase.jar version.properties && cat version.properties
shell: bash
- name: get major version
id: major_version
run: echo "major_version=$(cat version.properties | grep 'tag=' | cut -d'.' -f2)" >> $GITHUB_OUTPUT
shell: bash
- name: get drivers registry
run: |
curl -OL https://raw.githubusercontent.com/metabase/metabase-registry/main/registry.yaml
shell: bash
- uses: actions/setup-node@v4
with:
node-version: lts/Jod # 22.x.x
- run: rm package.json && npm install yaml
shell: bash
- name: parse drivers registry
uses: actions/github-script@v7
id: drivers_registry
with:
script: | # js
const { parse } = require('yaml');
const fs = require('fs');
const registry = parse(fs.readFileSync('registry.yaml', 'utf8'));
const drivers = registry.drivers;
const majorVersion = '${{ steps.major_version.outputs.major_version }}';
const getDriverUrl = (driverName) => {
const driver = drivers.find(driver => driver.name === driverName);
if (!driver) {
throw new Error(`Driver ${driverName} not found in the registry`);
}
const driverUrl = driver.versions[majorVersion] ?? driver.versions.default;
if (!driverUrl) {
throw new Error(`Driver ${driverName} version ${majorVersion} not found in the registry`);
}
return driverUrl;
};
const extraDrivers = {
firebolt: getDriverUrl('firebolt'),
starburst: getDriverUrl('starburst'),
clickhouse: getDriverUrl('clickhouse'),
materialize: getDriverUrl('materialize'),
};
console.log(extraDrivers);
return extraDrivers;
- name: Download Firebolt driver
run: |
curl -OL ${{ fromJson(steps.drivers_registry.outputs.result).firebolt }}
working-directory: modules
shell: bash
- name: Download Starburst driver
run: |
curl -OL ${{ fromJson(steps.drivers_registry.outputs.result).starburst }}
shell: bash
working-directory: modules
- name: Download Clickhouse driver
run: |
curl -OL ${{ fromJson(steps.drivers_registry.outputs.result).clickhouse }}
shell: bash
working-directory: modules
- name: Download Materialize driver
run: |
curl -OL ${{ fromJson(steps.drivers_registry.outputs.result).materialize }}
shell: bash
working-directory: modules
- name: Bundle additional drivers into the Uberjar
run: jar uf bin/docker/metabase.jar modules/*.jar
shell: bash
- name: Launch uberjar
run: >-
java --add-opens java.base/java.nio=ALL-UNNAMED -jar ./bin/docker/metabase.jar &
shell: bash
- name: Wait for Metabase to start
run: while ! curl 'http://localhost:3000/api/health' | grep '{"status":"ok"}'; do sleep 1; done
shell: bash
- name: Kill the java process
run: pkill -f metabase.jar
shell: bash
- name: configure aws credentials
if: ${{ github.ref_name == 'master' }}
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ inputs.iam-role }}
role-session-name: GitHub_to_AWS_via_FederatedOIDC
aws-region: us-east-1
- name: Login to Amazon ECR
if: ${{ github.ref_name == 'master' }}
uses: aws-actions/amazon-ecr-login@v2

View File

@@ -23,7 +23,7 @@ runs:
GITHUB_TOKEN: ${{ inputs.github-token }}
COMMIT: ${{ inputs.backport-commit }}
shell: bash
run: | # bash
run: |
git config --global user.email "metabase-bot@metabase.com"
git config --global user.name "Metabase bot"
@@ -35,8 +35,6 @@ runs:
git checkout -b ${BACKPORT_BRANCH}
git cherry-pick ${COMMIT} || true
CONFLICT_LIST=$(git ls-files -u)
CONFLICTS=$(git ls-files -u | wc -l)
if [ "$CONFLICTS" -gt 0 ]; then
git cherry-pick --abort
@@ -48,7 +46,7 @@ runs:
echo "git reset HEAD~1" > ./backport.sh
echo "rm ./backport.sh" >> ./backport.sh
echo "git cherry-pick ${COMMIT}" >> ./backport.sh
echo "echo 'Resolve conflicts and force push this branch.\n\nTo backport translations run: bin/i18n/merge-translations <release-branch>'" >> ./backport.sh
echo "echo 'Resolve conflicts and force push this branch'" >> ./backport.sh
chmod +x ./backport.sh
git add ./backport.sh
git commit -m "Add backport resolution script"
@@ -57,14 +55,7 @@ runs:
#${ORIGINAL_PULL_REQUEST_NUMBER}
> [!IMPORTANT]
> Manual conflict resolution is required.
> Checkout the branch and run \`./backport.sh\` script. Force push your changes after cherry-picking. Check the box below when done.
Conflicts:
\`\`\`shell
${CONFLICT_LIST}
\`\`\`
- [ ] Conflicts resolved
Checkout the branch and run \`./backport.sh\` script. Force push your changes after cherry-picking.
END
)
else

View File

@@ -1,15 +1,12 @@
name: Cross-branch migration test
inputs:
base-ref: ## Base branch PR came from
required: true
head-ref: ## Source branch of the pull request
base-ref:
required: true
runs:
using: "composite"
steps:
## First we deploy the base branch
- name: Check out base branch
uses: actions/checkout@v4
with:
@@ -42,11 +39,10 @@ runs:
run: kill $(cat metabase_pid)
shell: bash
## Then we deploy the PR branch to ensure it migrates up successfully
- name: Check out pull request branch
uses: actions/checkout@v4
with:
ref: ${{ inputs.head-ref }}
ref: ${{ github.head_ref }}
- name: Run backend on pull request branch
run: |
@@ -69,38 +65,3 @@ runs:
- name: Stop Metabase on pull request branch
run: kill $(cat metabase_pid)
shell: bash
## Testing that originally deploying this PR allows you to roll back to a previous release
## Reset the checksums so when we go back to the base branch, it's as if the PR branch was what was originally deployed
- name: Reset the checksums as if the PR branch was what was originally deployed
run: |
clojure -M:dev:ee:migrate reset-checksums
shell: bash
- name: Rollback the last deployment
run: |
clojure -M:dev:ee:migrate rollback last-deployment
shell: bash
- name: Re-check out base branch
uses: actions/checkout@v4
with:
ref: ${{ inputs.base-ref }}
- name: Re-run backend on base branch
run: |
clojure -M:run:ee > metabase.log 2>&1 & echo $! > metabase_pid
tail -F metabase.log &
shell: bash
- name: Wait for Metabase to start
run: |
while ! curl -s 'http://localhost:3000/api/health' | grep '{"status":"ok"}' > /dev/null; do
if ! kill -0 $(cat metabase_pid) 2>/dev/null; then
cat metabase.log
echo "Metabase process has exited."
exit 1
fi
sleep 1
done
shell: bash

View File

@@ -8,7 +8,7 @@ inputs:
description: DockerHub password/token
required: true
maildev:
description: Maildev containers
description: Maildev container
required: true
default: false
openldap:
@@ -31,10 +31,6 @@ inputs:
description: Webhook tester container
required: false
default: false
snowplow:
description: Snowplow micro container
required: false
default: false
runs:
using: "composite"
@@ -46,68 +42,56 @@ runs:
password: ${{ inputs.password }}
- name: Start Containers
shell: bash
run: | # bash
run: |
Y='\033[33m'
G='\033[1;32m'
cd e2e/test/scenarios/
docker compose up \
$([ ${{ inputs.postgres }} = true ] && echo "postgres-sample") \
$([ ${{ inputs.mysql }} = true ] && echo "mysql-sample") \
$([ ${{ inputs.mongo }} = true ] && echo "mongo-sample") \
$([ ${{ inputs.maildev }} = true ] && echo "maildev") \
$([ ${{ inputs.maildev }} = true ] && echo "maildev-ssl") \
$([ ${{ inputs.openldap }} = true ] && echo "ldap") \
$([ ${{ inputs.webhook }} = true ] && echo "webhook-tester") \
$([ ${{ inputs.snowplow }} = true ] && echo "snowplow-micro") \
-d
if ${{ inputs.maildev }}; then
echo -e "${Y}checking maildev container..." &&
echo -e "${Y}Starting maildev container..." &&
docker run -d -p 1080:1080 -p 1025:1025 maildev/maildev:2.0.5 &&
while ! nc -z localhost 1080; do sleep 1; done &&
while ! nc -z localhost 1025; do sleep 1; done &&
echo -e "${G}Maildev is up and running!"
echo -e "${Y}checking maildev-ssl container..." &&
while ! nc -z localhost 1081; do sleep 1; done &&
while ! nc -z localhost 465; do sleep 1; done &&
echo -e "${G}Maildev-ssl is up and running!"
fi
if ${{ inputs.openldap }}; then
echo -e "${Y}checking openldap container..." &&
echo -e "${Y}Starting openldap container..." &&
docker run -d -p 389:389 \
--env LDAP_ADMIN_PASSWORD=adminpass \
--env LDAP_USERS=user01@example.org,user02@example.org \
--env LDAP_PASSWORDS=123456,123465 \
--env LDAP_ROOT=dc=example,dc=org \
--env LDAP_PORT_NUMBER=389 \
bitnami/openldap:2.6.4 &&
while ! nc -z localhost 389; do sleep 1; done &&
echo -e "${G}Openldap is up and running!"
fi
if ${{ inputs.postgres }}; then
echo -e "${Y}checking postgres container..." &&
echo -e "${Y}Starting postgres container..." &&
docker run -d -p 5404:5432 -e TZ='US/Pacific' metabase/qa-databases:postgres-sample-12 &&
while ! nc -z localhost 5404; do sleep 1; done &&
echo -e "${G}Postgres is up and running!"
fi
if ${{ inputs.mysql }}; then
echo -e "${Y}checking mysql container..." &&
echo -e "${Y}Starting mysql container..." &&
docker run -d -p 3304:3306 -e TZ='US/Pacific' metabase/qa-databases:mysql-sample-8 &&
while ! nc -z localhost 3304; do sleep 1; done &&
echo -e "${G}MySQL is up and running!"
fi
if ${{ inputs.mongo }}; then
echo -e "${Y}checking mongo container..." &&
echo -e "${Y}Starting mongo container..." &&
docker run -d -p 27004:27017 metabase/qa-databases:mongo-sample-4.4 &&
while ! nc -z localhost 27004; do sleep 1; done &&
echo -e "${G}Mongo is up and running!"
fi
if ${{ inputs.webhook }}; then
echo -e "${Y}checking webhook test container..." &&
echo -e "${Y}Starting webhook test container..." &&
docker run -d -p 9080:8080/tcp tarampampam/webhook-tester:1.1.0 serve --create-session 00000000-0000-0000-0000-000000000000 &&
while ! nc -z localhost 9080; do sleep 1; done &&
echo -e "${G}Webhook tester is up and running!"
fi
if ${{ inputs.snowplow }}; then
echo -e "${Y}checking snowplow micro container..." &&
while ! nc -z localhost 9090; do sleep 1; done &&
echo -e "${G}Snowplow micro is up and running!"
fi
shell: bash

View File

@@ -5,14 +5,6 @@ inputs:
name:
description: The name of the artifact to fetch
required: true
output-name:
description: Optional custom name for the output jar file (defaults to metabase.jar)
required: false
default: "metabase.jar"
extract-path:
description: Directory path where the jar should be extracted (defaults to current directory)
required: false
default: "."
runs:
using: "composite"
@@ -33,7 +25,7 @@ runs:
});
if (!artifacts.data?.artifacts?.[0]?.id) {
throw new Error(`No artifacts found for ${{ inputs.name }}`);
throw new Error(`No artifacts found for ${{ inputs.commit }}`);
}
const artifact_id = artifacts.data.artifacts[0].id;
@@ -47,59 +39,25 @@ runs:
fs.writeFileSync(`${process.env.GITHUB_WORKSPACE}/mb.zip`, Buffer.from(download.data));
- name: Extract and rename JAR
- name: unzip uberjar artifact
shell: bash
run: unzip mb.zip
- name: Move the Uberjar
shell: bash
run: | # bash
OUTPUT_NAME="${{ inputs.output-name }}"
EXTRACT_PATH="${{ inputs.extract-path }}"
# Create a temporary directory for extraction to avoid conflicts
TEMP_DIR=$(mktemp -d)
# Extract to temporary directory
unzip mb.zip -d "$TEMP_DIR"
# Create target directory
mkdir -p "$EXTRACT_PATH"
# Find the jar and move all files from its directory
if [ -e "$TEMP_DIR/metabase.jar" ]; then
# Move all files to extract path
mv "$TEMP_DIR"/* "$EXTRACT_PATH/" 2>/dev/null || true
# Rename the jar to custom name if different
if [ "$OUTPUT_NAME" != "metabase.jar" ]; then
mv "$EXTRACT_PATH/metabase.jar" "$EXTRACT_PATH/$OUTPUT_NAME"
fi
elif [ -e "$TEMP_DIR/target/uberjar/metabase.jar" ]; then
# Move all files from uberjar directory to extract path
mv "$TEMP_DIR/target/uberjar"/* "$EXTRACT_PATH/" 2>/dev/null || true
# Rename the jar to custom name if different
if [ "$OUTPUT_NAME" != "metabase.jar" ]; then
mv "$EXTRACT_PATH/metabase.jar" "$EXTRACT_PATH/$OUTPUT_NAME"
fi
if [ -e "metabase.jar" ]; then
echo "metabase.jar is already in the right place"
elif [ -e "target/uberjar/metabase.jar" ]; then
echo "moving metabase.jar to current directory"
mv target/uberjar/metabase.jar metabase.jar
else
echo "Could not find metabase.jar in extracted files"
echo "Could not find the metabase.jar file"
exit 1
fi
# Clean up temporary directory
rm -rf "$TEMP_DIR"
# Set the JAR_PATH environment variable
if [ "$EXTRACT_PATH" == "." ]; then
echo "JAR_PATH=$OUTPUT_NAME" >> $GITHUB_ENV
else
echo "JAR_PATH=$EXTRACT_PATH/$OUTPUT_NAME" >> $GITHUB_ENV
fi
- name: Verify that this is a valid JAR file
shell: bash
run: |
echo "Verifying JAR at: $JAR_PATH"
file --mime-type "$JAR_PATH"
run: file --mime-type ./metabase.jar
- name: Reveal its version.properties
shell: bash
run: |
echo "Extracting version.properties from: $JAR_PATH"
jar xf "$JAR_PATH" version.properties && cat version.properties
run: jar xf metabase.jar version.properties && cat version.properties

View File

@@ -1,74 +0,0 @@
name: Get Docker Info
description: Determines Docker repository and tag based on branch and edition
inputs:
branch:
description: Branch name (defaults to current branch)
required: false
default: ${{ github.head_ref || github.ref_name }}
edition:
description: Edition (ee or oss)
required: false
default: ee
owner:
description: Repository owner (defaults to current repo owner)
required: false
default: ${{ github.repository_owner }}
outputs:
repo:
description: Docker repository name
value: ${{ steps.docker-info.outputs.repo }}
tag:
description: Docker tag
value: ${{ steps.docker-info.outputs.tag }}
full_image:
description: Full Docker image name (repo:tag)
value: ${{ steps.docker-info.outputs.full_image }}
editions:
description: JSON array of editions to build
value: ${{ steps.docker-info.outputs.editions }}
runs:
using: "composite"
steps:
- name: Determine Docker info
id: docker-info
uses: actions/github-script@v7
with:
script: | # js
const branch = '${{ inputs.branch }}';
const edition = '${{ inputs.edition }}';
const owner = '${{ inputs.owner }}';
console.log(`Determining Docker info for branch: ${branch}, edition: ${edition}`);
let repo, tag, editions;
if (branch === 'master') {
// Master builds both editions with specific repos
editions = ['ee', 'oss'];
tag = 'latest';
if (edition === 'ee') {
repo = `${owner}/metabase-enterprise-head`;
} else {
repo = `${owner}/metabase-head`;
}
} else {
// All other branches only build EE to metabase-dev
editions = ['ee'];
tag = branch.replaceAll('/', '-'); // slashes make docker sad
repo = `${owner}/metabase-dev`;
}
const fullImage = `${repo}:${tag}`;
// Set outputs
core.setOutput('repo', repo);
core.setOutput('tag', tag);
core.setOutput('full_image', fullImage);
core.setOutput('editions', JSON.stringify(editions));
console.log(`Docker image: ${fullImage}`);
console.log(`Editions: ${JSON.stringify(editions)}`);

View File

@@ -1,13 +1,5 @@
name: Notify Pull Request
inputs:
remove-previous-comment:
required: false
type: boolean
default: false
add-new-comment:
required: false
type: boolean
default: true
message:
required: true
include-log:
@@ -17,44 +9,7 @@ inputs:
runs:
using: "composite"
steps:
- name: Remove previous pull-request comment
uses: actions/github-script@v7
if: ${{ inputs.remove-previous-comment == 'true' && always() && github.event_name == 'pull_request' }}
with:
script: |
const pr = context.payload.pull_request ?? context.payload.issue;
if (!pr) {
console.error("This action can only be run on pull requests");
return;
}
const { owner, repo } = context.repo;
const { data: comments } = await github.rest.issues.listComments({
owner,
repo,
issue_number: pr.number,
}).catch(console.error);
if (!comments) {
return;
}
const message = "${{ inputs.message }}";
const comment = comments.find((comment) => comment.body.includes(message));
if (comment) {
await github.rest.issues.deleteComment({
owner,
repo,
comment_id: comment.id,
});
}
- name: Notify pull-request
uses: actions/github-script@v7
if: ${{ inputs.add-new-comment == 'true' && always() && github.event_name == 'pull_request' }}
- uses: actions/github-script@v7
id: notify-pull-request
with:
result-encoding: string
@@ -66,7 +21,7 @@ runs:
return;
}
const message = "${{ inputs.message }}";
const message = "${{ inputs.message }}"
const shouldIncludeLog = ${{ inputs.include-log }}.toString() === "true";
const author = context.payload.sender.login;

View File

@@ -1,7 +1,4 @@
name: Prepare back-end environment
description: |
Prepare the backend. Installs jdk, Clojure, and fetches the cache for m2. The CI config fetchs `ci-test-config.json`
from master and lets us ignore tests at the var level if they are flaking.
inputs:
java-version:
required: true
@@ -24,14 +21,9 @@ runs:
distribution: 'temurin'
- name: Install Clojure CLI
shell: bash
continue-on-error: true
run: |
curl -O https://download.clojure.org/install/linux-install-${{ inputs.clojure-version }}.sh &&
sudo bash ./linux-install-${{ inputs.clojure-version }}.sh
- name: Fetch CI config
shell: bash
run: |
curl -o ci-test-config.json https://raw.githubusercontent.com/metabase/metabase/refs/heads/master/ci-test-config.json
- name: Check to see if dependencies should be cached
if: ${{ contains(github.event.head_commit.message, '[ci nocache]') }}
run: echo "Commit message includes [ci nocache]; dependencies will NOT be cached"

View File

@@ -1,12 +1,6 @@
name: Prepare cypress environment
description: Cypress preparation steps
inputs:
working-directory:
description: "Optional working directory for yarn commands"
required: false
default: "."
outputs:
chrome-path:
description: Path to the custom Chrome executable
@@ -39,4 +33,3 @@ runs:
yarn cypress cache list
yarn cypress verify
shell: bash
working-directory: ${{ inputs.working-directory }}

View File

@@ -0,0 +1,13 @@
name: Run Snowplow micro
runs:
using: "composite"
steps:
- name: Download Snowplow micro JAR
run: curl -L -o snowplow-micro.jar https://github.com/snowplow-incubator/snowplow-micro/releases/download/micro-1.2.1/snowplow-micro-1.2.1.jar
shell: bash
- name: Launch Snowplow micro
run: java -cp snowplow-micro.jar:snowplow com.snowplowanalytics.snowplow.micro.Main --collector-config snowplow/micro.conf --iglu snowplow/iglu.json &
shell: bash
- name: Wait until Snowplow micro is ready
run: while ! curl -s 'http://localhost:9090' | grep '404'; do sleep 1; done
shell: bash

View File

@@ -1,5 +1,4 @@
name: Test database driver
description: Test database driver
inputs:
junit-name:
required: true
@@ -24,8 +23,6 @@ runs:
run: yarn build-static-viz
shell: bash
if: ${{ inputs.build-static-viz == 'true' }}
env:
MB_EDITION: ee
- name: Test database driver
run: clojure -X:dev:ci:ee:ee-dev:drivers:drivers-dev:test ${{ inputs.test-args }}
shell: bash
@@ -39,10 +36,3 @@ runs:
list-suites: failed
list-tests: failed
fail-on-error: false
- name: Upload Logs on Failure
if: always() && !success()
uses: actions/upload-artifact@v4
with:
name: test-logs-${{ github.job }}
path: logs/test-log.json
retention-days: 1

View File

@@ -11,7 +11,4 @@ runs:
env:
GITHUB_TOKEN: ${{ inputs.github-token }}
shell: bash
run: |
git config --global user.email "github-automation@metabase.com"
git config --global user.name "Metabase Automation"
bash ${{ github.action_path }}/upgrade-deps.sh
run: bash ${{ github.action_path }}/upgrade-deps.sh

View File

@@ -59,11 +59,6 @@ for upgrade in $UPGRADES; do
# Commit the dependency update, and link to the diff
git add deps.edn modules
# Use an email and name that will pass the CLA check:
git config --global user.email "github-automation@metabase.com"
git config --global user.name "Metabase Automation"
git commit -m ${TITLE} -m ${BODY} --no-verify
git push -u origin $BRANCH_NAME

View File

@@ -18,9 +18,6 @@ inputs:
required: true
trunk-api-token:
required: true
variant:
required: false
default: ''
runs:
using: "composite"
@@ -48,11 +45,10 @@ runs:
aws s3 cp ${OUTPUT_FILE}.zip s3://$BUCKET/$DATE/$GITHUB_RUN_ID/$GITHUB_RUN_ATTEMPT/
- name: Upload results to Trunk
if: ${{ always() && !(github.event_name == 'pull_request' && github.event.pull_request.draft == true) }}
if: ${{ always() }}
uses: trunk-io/analytics-uploader@main
with:
junit-paths: ${{ inputs.input-path }}
variant: ${{ inputs.variant || inputs.output-name }}
org-slug: metabase
token: ${{ inputs.trunk-api-token }}
continue-on-error: true

17
.github/auto_request_review.yml vendored Normal file
View File

@@ -0,0 +1,17 @@
# The reviewers configuration for the auto_request_review workflow
# See https://github.com/necojackarc/auto-request-review/tree/master?tab=readme-ov-file#reviewers-configuration
# for the spec
files:
# migrations related files
'resources/migrations/**':
- team:core-workflows
'src/metabase/db/custom_migrations.clj':
- team:core-workflows
options:
ignore_draft: true
ignored_keywords:
- DO NOT REVIEW
- WIP
enable_group_assignment: false

View File

@@ -1,52 +0,0 @@
# Metabase Developer Assistant Instructions
## Frontend Development
### Repository Structure
```
frontend/src/
├── metabase/ # Main application components and pages
├── metabase-lib/ # Query building and data modeling utilities
├── metabase-types/ # TypeScript type definitions
└── metabase/ui/ # Design system components
e2e/ # Cypress end-to-end tests
frontend/test/ # Jest unit tests
```
### Technology Stack
- **Framework**: React 18 with TypeScript
- **State**: Redux Toolkit
- **Styling**: CSS Modules (preferred) > Emotion styled-components
- **UI**: `metabase/ui` components built with Mantine v8
- **Build**: Rspack (primary), Webpack (legacy)
- **Testing**: Jest + React Testing Library, Cypress
### Coding Standards
#### Component Preferences
- Prefer `metabase/ui`components over `metabase/common/components`
- Use `.tsx` for components, `.ts` for utilities
#### Styling
ALWAYS prefer Mantine style props,then CSS modules. DO NOT suggest styled components, they are deprecated.
#### TypeScript Migration
When heavily editing `.js`/`.jsx` files, create a separate PR to convert to TypeScript first, then implement changes.
#### Enterprise Features
Enterprise functionality MUST use the plugin system. It is very important to not expose enterprise code in the OSS version.
#### Testing Requirements
All PRs should include tests. Prefer Unit tests over E2E tests.
#### Localization
All user-facing strings MUST be localized using the ttag library. Localized strings should be complete phrases, do not concatenate a few separately localized strings. You should add context to strings where the meaning of the string might not be obvious in isolation: e.g. "Home" might have different words in some languages depending on whether you're talking about a dwelling or the landing page for a website.

View File

@@ -16,49 +16,29 @@ frontend_ci: &frontend_ci
- ".github/actions/prepare-backend/**"
- ".github/workflows/frontend.yml"
frontend_embedding_sdk_ci: &frontend_embedding_sdk_ci
- ".github/actions/prepare-frontend/**"
- ".github/actions/prepare-backend/**"
- ".github/workflows/*embedding-sdk*.yml"
frontend_configs: &frontend_configs
frontend_sources: &frontend_sources
- *shared_sources
- "enterprise/frontend/!(test)**/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
- "frontend/!(test)**/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
- "yarn.lock"
- "**/tsconfig*.json"
- "package.json"
- "babel.config.json"
- ".eslintrc.js"
- "postcss.config.js"
- "rspack.config.js"
- "rspack.main.config.js"
- "rspack.embedding-sdk-package.config.js"
- "rspack.embedding-sdk-bundle.config.js"
- "rspack.static-viz.config.js"
- "webpack.config.js"
- "webpack.embedding-sdk.config.js"
- "webpack.static-viz.config.js"
- ".nvmrc"
frontend_sources: &frontend_sources
- *shared_sources
- *frontend_configs
- "enterprise/frontend/!(test)**/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
- "frontend/!(test)**/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
frontend_embedding_sources:
- "enterprise/frontend/src/{embedding,embedding-*}/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
- "enterprise/frontend/src/metabase-enterprise/{embedding,embedding-*}**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
frontend_embedding_sdk_sources:
- *frontend_configs
- "enterprise/frontend/!(test)**/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
frontend_embedding_sdk_package_sources:
- "enterprise/frontend/src/embedding-sdk-package/**/!(*.spec).{js,jsx,ts,tsx,css}"
- "enterprise/frontend/src/embedding-sdk/**"
frontend_specs: &frontend_specs
- *shared_specs
- "frontend/test/**"
- "frontend/**/tests/**"
- "frontend/**/*.unit.*"
- "enterprise/frontend/**/*.unit.*"
- ".github/**/*.unit.spec.*"
- "jest.config.js"
- "jest.tz.unit.conf.json"
@@ -122,49 +102,34 @@ e2e_ci: &e2e_ci
- ".github/actions/prepare-uberjar-artifact/**"
- ".github/actions/e2e-prepare-containers/**"
- ".github/actions/prepare-cypress/**"
- ".github/actions/run-snowplow-micro/**"
- ".github/workflows/e2e-*"
e2e_specs: &e2e_specs
- e2e/!(test-host-app)**/*.cy.*.{js,ts}
- "**/*.cy.*.js"
- "**/*.cy.*.ts"
- "e2e/runner/**"
- "e2e/support/**"
- "e2e/snapshot*/**"
e2e_cross_version: &e2e_cross_version
- ".github/workflows/e2e-cross-version.yml"
- "e2e/test/scenarios/cross-version/**"
e2e_embedding_sdk:
- *frontend_embedding_sdk_ci
- *sources
- "e2e/component/**/*.cy.*.{js,ts}"
e2e_embedding_sdk_compatibility:
- *default
- *frontend_embedding_sdk_ci
- *frontend_sources
- *backend_sources
- "e2e/embedding-sdk-host-apps/**"
- "e2e/runner/embedding-sdk/**"
- "e2e/test-host-app/**/*.cy.*.{js,ts}"
snowplow: &snowplow
- ".github/workflows/snowplow.yml"
- "snowplow/**"
e2e_all:
- *default
- *e2e_ci
- *sources
- *e2e_specs
- "e2e/**"
- *snowplow
snowplow:
- ".github/workflows/snowplow.yml"
- "snowplow/**"
documentation:
- "docs/**"
- "**/*.md"
embedding_documentation:
- ".typedoc/**"
- "docs/embedding/**"
yaml:
- "**/*.yml"
- "**/*.yaml"

View File

@@ -1,84 +0,0 @@
// This script is used in .github/workflows/e2e-matrix-builder.yml
// its aim is to split the e2e test matrix into multiple jobs
// grouping some specific tests together, other tests are split into chunks
const DEFAULT_SPEC_PATTERN = "./e2e/test/scenarios/**/*.cy.spec.*";
const EMBEDDING_SDK_SPEC_PATTERN =
"./e2e/test/scenarios/embedding-sdk/**.cy.spec.*";
const specialTestConfigs = [
{
name: "embedding-sdk",
specs: EMBEDDING_SDK_SPEC_PATTERN,
},
{
name: "oss-subset",
edition: "oss",
tags: "@OSS @smoke+-@EE",
specs: DEFAULT_SPEC_PATTERN,
},
{ name: "mongo", tags: "@mongo", specs: DEFAULT_SPEC_PATTERN },
];
/**
*
* @param {*} inputSpecs - specs, which were changed in the PR, separated by comma
* .e.g e2e/test/scenarios/onboarding/command-palette.cy.spec.js,e2e/test/scenarios/question/document-title.cy.spec.js
* or a pattern like DEFAULT_SPEC_PATTERN
*/
function buildMatrix(options, inputSpecs, inputChunks) {
const { java, defaultRunner } = options;
// number of specs per chunk when running specific specs
const SPECS_PER_CHUNK = 5;
const defaultOptions = {
"java-version": java,
runner: defaultRunner,
edition: "ee",
};
const isDefaultSpecPattern =
inputSpecs === "" || inputSpecs === DEFAULT_SPEC_PATTERN;
console.log("Processed specs value:", inputSpecs);
console.log("Is default pattern:", isDefaultSpecPattern);
let regularChunks;
if (isDefaultSpecPattern) {
regularChunks = inputChunks - specialTestConfigs.length;
} else {
// when pattern is not default, it means we passed some custom list of the changed specs
// so we need to calculate how many chunks we need to run
const matchingSpecsCount = inputSpecs.split(",").length;
regularChunks = Math.max(
1,
Math.ceil(matchingSpecsCount / SPECS_PER_CHUNK),
);
}
const regularTests = new Array(regularChunks).fill(1).map((files, index) => ({
name: `e2e-group-${index + 1}`,
// works when specs less than 5, otherwise seems all chunks will contain
// same specs
...(!isDefaultSpecPattern && {
specs: inputSpecs
.split(",")
.slice(SPECS_PER_CHUNK * index, SPECS_PER_CHUNK * (index + 1))
.join(","),
}),
}));
const testSets = isDefaultSpecPattern
? regularTests.concat(specialTestConfigs)
: regularTests;
const config = testSets.map((options) => ({
...defaultOptions,
...options,
}));
return { config, regularChunks, isDefaultSpecPattern };
}
module.exports = { buildMatrix };

View File

@@ -1,97 +0,0 @@
const { buildMatrix } = require("./build-e2e-matrix");
describe("buildMatrix", () => {
const options = {
java: "11",
defaultRunner: "ubuntu-22.04",
};
const testBuildMatrix = (entry, chunks) =>
buildMatrix(options, entry, chunks);
it("should calculate correct regular chunks with default parameters", () => {
const result = testBuildMatrix("./e2e/test/scenarios/**/*.cy.spec.*", 30);
expect(result.regularChunks).toBe(27);
});
it("should create correct number of configuration objects", () => {
const result = testBuildMatrix("./e2e/test/scenarios/**/*.cy.spec.*", 10);
expect(result.config.length).toBe(10);
});
it("should apply default specs path when empty entry is provided", () => {
const result = testBuildMatrix("", 5);
expect(result.regularChunks).toBe(2);
expect(result.config.length).toBe(5);
});
it("should name regular test groups sequentially", () => {
const result = testBuildMatrix("./e2e/test/scenarios/**/*.cy.spec.*", 10);
const regularTests = result.config.filter((cfg) =>
cfg.name.startsWith("e2e-group-"),
);
regularTests.forEach((test, index) => {
expect(test.name).toBe(`e2e-group-${index + 1}`);
});
});
it("should handle case when input chunks equals special tests count", () => {
const result = testBuildMatrix("./e2e/test/scenarios/**/*.cy.spec.*", 3);
expect(result.regularChunks).toBe(0);
expect(result.config.length).toBe(3);
expect(result.config[0].name).toBe("embedding-sdk");
expect(result.config[1].name).toBe("oss-subset");
expect(result.config[2].name).toBe("mongo");
});
it("should correctly calculate regularChunks for non-default spec pattern", () => {
const entry = "file.cy.spec.js,".repeat(12).slice(0, -1);
const result = testBuildMatrix(entry, 10);
// For 12 matching specs with SPECS_PER_CHUNK=5, we expect ceil(12/5) = 3 chunks
expect(result.regularChunks).toBe(3);
const firstGroupSpecs = result.config[0].specs.split(",").length;
const secondGroupSpecs = result.config[1].specs.split(",").length;
const thirdGroupSpecs = result.config[2].specs.split(",").length;
expect(firstGroupSpecs).toBe(5);
expect(secondGroupSpecs).toBe(5);
expect(thirdGroupSpecs).toBe(2);
});
it("should return isDefaultSpecPattern as true when empty or default value is provided", () => {
let result = testBuildMatrix("", 5);
expect(result.isDefaultSpecPattern).toBe(true);
result = testBuildMatrix("./e2e/test/scenarios/**/*.cy.spec.*", 5);
expect(result.isDefaultSpecPattern).toBe(true);
});
it("should return isDefaultSpecPattern as false when non-default spec pattern is provided", () => {
const result = testBuildMatrix("test1.cy.spec.js,test2.cy.spec.js", 1);
expect(result.isDefaultSpecPattern).toBe(false);
});
it("should return special test configs when default spec pattern is provided", () => {
const result = testBuildMatrix("./e2e/test/scenarios/**/*.cy.spec.*", 50);
expect(result.config[47].name).toBe("embedding-sdk");
expect(result.config[48].name).toBe("oss-subset");
expect(result.config[49].name).toBe("mongo");
});
it("should not return special test configs when non-default spec pattern is provided", () => {
const result = testBuildMatrix("test1.cy.spec.js,test2.cy.spec.js", 50);
expect(result.config.length).toBe(1);
expect(result.config[0].name).toBe("e2e-group-1");
});
});

View File

@@ -1,66 +1,24 @@
const TYPES = new Set([
"feat",
"fix",
"perf",
"docs",
"style",
"refactor",
"test",
"build",
"ci",
]);
/**
* @param title {string}
*/
function isConventionalTitle(title) {
const match = /^(?<type>\w+)\((?<rawScopes>[^)]+)\):/.exec(title);
if (!match) {
return false;
}
const [, type, rawScopes] = match;
const scopes = rawScopes
.split(",")
.map((scope) => scope.trim())
.filter(Boolean);
return TYPES.has(type) && scopes.length > 0;
}
exports.updateComment = updateComment;
/** @param {import('@types/github-script').AsyncFunctionArguments} AsyncFunctionArguments */
async function updateCommentForEmbedding({ github, context }) {
async function updateComment({ github, context }) {
const githubUsername = context.payload.sender.login;
const comment = `${CONVENTIONAL_COMMENT_IDENTIFIER}
@${githubUsername} You have modified embedding code. Please make sure the PR title follows [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) style.
Here are all the supported types:
const comment = `${CONVENTIONAL_COMMIT_REMINDER_COMMENT_IDENTIFIER}
@${githubUsername} You have modified embedding SDK code. Please make sure the PR title follows [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) style.
Here are all the supported types that will be shown in the changelog:
- \`feat\`
- \`fix\`
- \`perf\`
- \`docs\`
- \`style\`
- \`refactor\`
- \`test\`
- \`build\`
- \`ci\`
Please also make sure to include a scope.
These types \`docs\`, \`style\`, \`refactor\`, \`test\`, \`build\`, and \`ci\` are also encouraged for non-changelog related tasks.
Please also make sure to include \`sdk\` scope, otherwise, the changelog will not include this task.
For example, these are valid PR titles:
- \`feat(sdk-bundle): Add interactive dashboard component\`
- \`fix(sdk-package): Fixed MetabaseProviderPropsStore logic\`
- \`fix(sdk-package, embed-js): Fix API\``;
- \`feat(sdk): Add interactive dashboard component\`
- \`feat(sdk): Support theming pie chart\`
- \`fix(sdk): Fix static dashboard crash\``;
return updateComment(comment, { github, context });
}
/**
* @param comment {string}
* @param {import('@types/github-script').AsyncFunctionArguments} AsyncFunctionArguments
*/
async function updateComment(comment, { github, context }) {
const owner = context.repo.owner;
const repo = context.repo.repo;
const pullRequestNumber = context.issue.number;
@@ -91,19 +49,20 @@ async function updateComment(comment, { github, context }) {
}
}
const CONVENTIONAL_COMMENT_IDENTIFIER =
"<!---conventional comment identifier-->";
const CONVENTIONAL_COMMIT_REMINDER_COMMENT_IDENTIFIER =
"<!---conventional commit reminder comment ID-->";
/**
* @param comments {{body: string}[]}
* @typedef Comment
* @property {string} body
*
* @param {Comment[]} comments
*/
function getExistingConventionalCommitReminderComment(comments) {
for (const comment of comments) {
if (comment.body.startsWith(CONVENTIONAL_COMMENT_IDENTIFIER)) {
if (
comment.body.startsWith(CONVENTIONAL_COMMIT_REMINDER_COMMENT_IDENTIFIER)
) {
return comment;
}
}
}
exports.isConventionalTitle = isConventionalTitle;
exports.updateCommentForEmbedding = updateCommentForEmbedding;

View File

@@ -1,45 +0,0 @@
const { isConventionalTitle } = require("./utils");
describe("isConventionalTitle", () => {
it("returns true for a simple conventional commit", () => {
expect(isConventionalTitle("feat: add new feature")).toBe(false);
});
it("returns true for a commit with a scope", () => {
expect(isConventionalTitle("fix(api): correct API bug")).toBe(true);
});
it("returns true for a commit with multiple scopes, comma separated", () => {
expect(isConventionalTitle("feat(build,deps): update dependencies")).toBe(
true,
);
});
it("returns true for a commit with multiple scopes, comma separated with space", () => {
expect(isConventionalTitle("feat(build, deps): update dependencies")).toBe(
true,
);
});
it("returns false for a commit with round brackets in a message", () => {
expect(isConventionalTitle("feat: (something) update dependencies")).toBe(
false,
);
});
it("returns false for missing colon", () => {
expect(isConventionalTitle("feat add feature")).toBe(false);
});
it("returns false for invalid type", () => {
expect(isConventionalTitle("invalidType: some message")).toBe(false);
});
it("returns false for empty message", () => {
expect(isConventionalTitle("feat:")).toBe(false);
});
it("returns false for empty input", () => {
expect(isConventionalTitle("")).toBe(false);
});
});

View File

@@ -1,84 +0,0 @@
const githubSlackMap = require("../../release/github-slack-map.json");
async function getIssueData({ context, github, manualIssueNumber }) {
let issueNumber;
let issueData;
let prReviews = [];
const triggerType = context.eventName;
if (context.eventName === "issues") {
issueNumber = context.payload.issue.number;
issueData = context.payload.issue;
} else if (context.eventName === "workflow_dispatch") {
issueNumber = parseInt(manualIssueNumber);
const { data } = await github.rest.issues.get({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber,
});
issueData = data;
} else if (context.eventName === "pull_request_review") {
const prNumber = context.payload.pull_request.number;
const { data: prData } = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: prNumber,
});
// Extract issue number from branch name (e.g., "claude-fix/issue-12345-some-title")
const branchMatch = prData.head.ref.match(/claude-fix\/issue-(\d+)/);
if (!branchMatch) {
throw new Error(
`Could not extract issue number from branch name: ${prData.head.ref}`,
);
}
issueNumber = parseInt(branchMatch[1]);
const { data } = await github.rest.issues.get({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber,
});
issueData = data;
const { data: reviews } = await github.rest.pulls.listReviews({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: prNumber,
});
prReviews = reviews.filter(
(review) =>
review.state === "CHANGES_REQUESTED" &&
isMetabaseMember(review.user.login),
);
}
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber,
});
const employeeComments = comments.filter((comment) =>
isMetabaseMember(comment.user.login),
);
return {
issueNumber,
issueData,
employeeComments,
prReviews,
triggerType,
isIssueAuthorEmployee: isMetabaseMember(issueData.user.login),
};
}
function isMetabaseMember(username) {
return username in githubSlackMap;
}
module.exports = {
getIssueData,
};

View File

@@ -49,55 +49,6 @@ function httpsRequest(options, data = null) {
});
}
// Makes an HTTPS request with exponential backoff retry logic
async function httpsRequestWithBackoff(options, data = null, backoffOptions = {}) {
// Set default backoff options:
const {
maxRetries = 8,
totalDurationSeconds = 60
} = backoffOptions;
let retries = 0;
let delay = 500; // Start with a .5 second delay
// Calculate a reasonable initial delay based on total duration and max retries
// This helps distribute the retry attempts across the total allowed duration
const totalDurationMs = totalDurationSeconds * 1000;
// Track the start time
const startTime = Date.now();
// Ensure data is stringified if it's an object
const requestData = data ? (typeof data === 'string' ? data : JSON.stringify(data)) : null;
while (true) {
try {
const response = await httpsRequest(options, requestData);
return response; // Success: Return the data
} catch (error) {
retries++;
const elapsedMs = Date.now() - startTime;
const timeRemaining = totalDurationMs - elapsedMs;
// Stop retrying if we've hit max retries or exceeded the total duration
if (retries >= maxRetries || timeRemaining <= 0) {
console.error(`Failed after ${retries} retries (${Math.round(elapsedMs/1000)}s elapsed):`);
throw error;
}
// Calculate the next delay with exponential backoff and jitter
delay = Math.min(delay * 2 * (0.8 + Math.random() * 0.4), timeRemaining);
console.log(`Request failed (attempt ${retries}/${maxRetries}). Retrying in ${Math.round(delay)}ms... (${Math.round(timeRemaining/1000)}s remaining)`);
// Wait for the calculated delay before trying again
await new Promise(resolve => setTimeout(resolve, delay));
}
}
}
async function link_issues(github) {
try {
@@ -124,14 +75,6 @@ async function link_issues(github) {
console.log('PR Info:\n' + JSON.stringify(prInfo, null, 2) + "\n");
if (prInfo &&
prInfo.data &&
prInfo.data.labels &&
prInfo.data.labels.some(label => label.name === "no-auto-issue-links")) {
console.log('PR has "no-auto-issue-links" label, skipping...');
return;
}
const prUrl = prInfo.data.html_url;
console.log(`Processing PR: ${prUrl}`);
@@ -147,11 +90,7 @@ async function link_issues(github) {
attachments {
nodes {url}}}}}}`});
console.log('--- Linear API query ---');
console.log(linearQuery);
console.log('----- End of query -----');
const linearData = await httpsRequestWithBackoff({
const linearData = await httpsRequest({
hostname: 'api.linear.app',
path: '/graphql',
method: 'POST',
@@ -159,9 +98,7 @@ async function link_issues(github) {
'Content-Type': 'application/json',
'Authorization': linearApiKey
}
},
linearQuery,
{ maxRetries: 8, totalDurationSeconds: 60 });
}, linearQuery);
console.log('Linear API response:');
console.log(JSON.stringify(linearData) + "\n");
@@ -216,9 +153,7 @@ async function link_issues(github) {
// Generate closing references
const closingRefs = issueNumbersToAdd.map(num => `closes #${num}`).join(' ');
const messagePreamble = "<!-- Added by 'Add Issue References to PR' GitHub Action. To disable linking, add 'no-auto-issue-links' label to your PR. -->"
newBody = `${messagePreamble} ${closingRefs}\n` + newBody;
newBody = `<!-- Added by 'Add Issue References to PR' GitHub Action -->${closingRefs}\n` + newBody;
// Update PR body with new closing references
await github.rest.issues.update({

97
.github/team.json vendored
View File

@@ -8,9 +8,18 @@
"romeovs",
"bshepherdson",
"metamben",
"appleby",
"uladzimirdev",
"wzimrin",
"ericnormand",
"galdre"
"ericnormand"
]
},
{
"name": "Semantic Layer",
"label": ".Team/SemanticLayer",
"members": [
"kamilmielnik",
"lbrdnk"
]
},
{
@@ -26,10 +35,10 @@
"label": ".Team/Embedding",
"members": [
"WiNloSt",
"oisincoveney",
"npretto",
"heypoom",
"sanex3339",
"lorem--ipsum"
"sanex3339"
]
},
{
@@ -39,18 +48,35 @@
"qnkhuat",
"piranha",
"crisptrutski",
"deniskaber",
"wotbrew",
"spalt08",
"stasgavrylov",
"uladzimirdev",
"kulyk",
"kamilmielnik"
"stasgavrylov"
]
},
{
"name": "Admin Webapp",
"label": ".Team/AdminWebapp",
"members": [
"eliotmetabase"
"npfitz",
"johnswanson",
"rafpaf",
"sloansparger",
"iethree",
"nvoxland",
"fraserdrops",
"edpaget"
]
},
{
"name": "DashViz",
"label": ".Team/DashViz",
"members": [
"alxnddr",
"kulyk",
"tsplude",
"lorem--ipsum",
"noahmoss"
]
},
{
@@ -60,59 +86,6 @@
"camsaul",
"escherize"
]
},
{
"name": "Metabot",
"label": ".Team/Metabot",
"members": [
"appleby",
"noahmoss",
"retro",
"sloansparger",
"somtom",
"wotbrew",
"tsplude",
"lbrdnk",
"bronsa"
]
},
{
"name": "Product Growth",
"label": ".Team/ProductGrowth",
"members": [
"nemanjaglumac",
"chodorowicz",
"andsalves"
]
},
{
"name": "Cloud",
"label": "cloud",
"members": [
"filipesilva"
]
},
{
"name": "DashViz",
"label": ".Team/DashViz",
"members": [
"eliotmetabase"
]
},
{
"name": "UX West",
"label": ".Team/UXWest",
"members": [
"eliotmetabase",
"npfitz",
"johnswanson",
"iethree",
"nvoxland",
"edpaget",
"deniskaber",
"alxnddr",
"bpander"
]
}
]
}

View File

@@ -1,405 +0,0 @@
name: App DB Tests
on:
workflow_call:
inputs:
skip:
type: boolean
default: false
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}-app-db
cancel-in-progress: true
jobs:
be-tests-mariadb:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
version:
- name: MariaDB 10.2
junit-name: be-tests-mariadb-10-2-ee
image: circleci/mariadb:10.2.23
env:
enable-ssl-tests: 'false'
- name: MariaDB Latest
junit-name: be-tests-mariadb-latest-ee
image: circleci/mariadb:latest
env:
enable-ssl-tests: 'false'
job:
- name: Enterprise Tests
build-static-viz: false
test-args: >-
:only '"enterprise/backend/test"'
exclude-tag: ':mb/driver-tests'
- name: EE App DB Tests (Part 1)
build-static-viz: true
test-args: >-
:only '"test"'
:partition/total 2
:partition/index 0
exclude-tag: ':mb/driver-tests'
- name: EE App DB Tests (Part 2)
build-static-viz: true
test-args: >-
:only '"test"'
:partition/total 2
:partition/index 1
exclude-tag: ':mb/driver-tests'
services:
mysql:
image: ${{ matrix.version.image }}
ports:
- "3306:3306"
env:
MYSQL_ALLOW_EMPTY_PASSWORD: true
MYSQL_DATABASE: circle_test
env:
# Only run old migrations tests on pushes to master or release branches. All other branches should skip tests
# with the tag `mb/old-migrations-test`. `__ADDITIONAL_EXCLUDED_TAG__` is not used anywhere outside of splicing
# it in to the command below.
__ADDITIONAL_EXCLUDED_TAG__: >-
${{
(
github.event_name == 'push' &&
(
github.ref == 'master' ||
startsWith(github.ref, 'release-')
) &&
''
) ||
':mb/old-migrations-test'
}}
# actual serious env vars below
CI: 'true'
DRIVERS: mysql
MB_DB_TYPE: mysql
MB_DB_HOST: localhost
MB_DB_PORT: 3306
MB_DB_DBNAME: circle_test
MB_DB_USER: root
MB_MYSQL_TEST_USER: root
#
# SSL config: only for mysql latest tests.
#
# set up env vars for something named "MYSQL_SSL" to run MySQL SSL tests verifying connectivity with PEM cert
# they are deliberately given a different name to prevent them from affecting the regular test run against
# the configured MySQL instance, but there is one particular test (mysql-connect-with-ssl-and-pem-cert-test)
# that overrides the MB_MYSQL_TEST_* values with them
# the MYSQL_RDS_SSL_INSTANCE vars are defined as secrets and can be altered
MB_MYSQL_SSL_TEST_SSL: ${{ matrix.version.env.enable-ssl-tests }}
MB_MYSQL_SSL_TEST_HOST: ${{ secrets.MYSQL_RDS_SSL_INSTANCE_HOST }}
MB_MYSQL_SSL_TEST_ADDITIONAL_OPTIONS: 'verifyServerCertificate=true'
# the contents of the ./resources/certificates/rds-combined-ca-bundle.pem file
MB_MYSQL_SSL_TEST_SSL_CERT: ${{ secrets.MB_MYSQL_SSL_TEST_SSL_CERT }}
MB_MYSQL_SSL_TEST_USER: metabase
MB_MYSQL_SSL_TEST_PASSWORD: ${{ secrets.MYSQL_RDS_SSL_INSTANCE_PASSWORD }}
# for historic reasons (I don't want to mess around with required jobs) the job name should be something like
# "be-tests-mariadb-10-2-ee (0)"
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
steps:
- uses: actions/checkout@v4
- name: Test ${{ matrix.version.name }}
uses: ./.github/actions/test-driver
with:
build-static-viz: ${{ matrix.job.build-static-viz }}
junit-name: ${{ matrix.version.junit-name }}
test-args: >-
${{ matrix.job.test-args }}
:exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-mysql:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
version:
- name: MySQL 8.0
junit-name: be-tests-mysql-8-0-ee
image: cimg/mysql:8.0
env:
enable-ssl-tests: 'false'
- name: MySQL Latest
junit-name: be-tests-mysql-latest-ee
image: mysql:latest
env:
enable-ssl-tests: 'true'
job:
- name: Enterprise Tests
build-static-viz: false
test-args: >-
:only '"enterprise/backend/test"'
exclude-tag: ':mb/driver-tests'
- name: EE App DB Tests (Part 1)
build-static-viz: true
test-args: >-
:only '"test"'
:partition/total 2
:partition/index 0
exclude-tag: ':mb/driver-tests'
- name: EE App DB Tests (Part 2)
build-static-viz: true
test-args: >-
:only '"test"'
:partition/total 2
:partition/index 1
exclude-tag: ':mb/driver-tests'
services:
mysql:
image: ${{ matrix.version.image }}
ports:
- "3306:3306"
env:
MYSQL_ALLOW_EMPTY_PASSWORD: true
MYSQL_DATABASE: circle_test
env:
# Only run old migrations tests on pushes to master or release branches. All other branches should skip tests
# with the tag `mb/old-migrations-test`. `__ADDITIONAL_EXCLUDED_TAG__` is not used anywhere outside of splicing
# it in to the command below.
__ADDITIONAL_EXCLUDED_TAG__: >-
${{
(
github.event_name == 'push' &&
(
github.ref == 'master' ||
startsWith(github.ref, 'release-')
) &&
''
) ||
':mb/old-migrations-test'
}}
# actual serious env vars below
CI: 'true'
DRIVERS: mysql
MB_DB_TYPE: mysql
MB_DB_HOST: localhost
MB_DB_PORT: 3306
MB_DB_DBNAME: circle_test
MB_DB_USER: root
MB_MYSQL_TEST_USER: root
#
# SSL config: only for mysql latest tests.
#
# set up env vars for something named "MYSQL_SSL" to run MySQL SSL tests verifying connectivity with PEM cert
# they are deliberately given a different name to prevent them from affecting the regular test run against
# the configured MySQL instance, but there is one particular test (mysql-connect-with-ssl-and-pem-cert-test)
# that overrides the MB_MYSQL_TEST_* values with them
# the MYSQL_RDS_SSL_INSTANCE vars are defined as secrets and can be altered
MB_MYSQL_SSL_TEST_SSL: ${{ matrix.version.env.enable-ssl-tests }}
MB_MYSQL_SSL_TEST_HOST: ${{ secrets.MYSQL_RDS_SSL_INSTANCE_HOST }}
MB_MYSQL_SSL_TEST_ADDITIONAL_OPTIONS: 'verifyServerCertificate=true'
# the contents of the ./resources/certificates/rds-combined-ca-bundle.pem file
MB_MYSQL_SSL_TEST_SSL_CERT: ${{ secrets.MB_MYSQL_SSL_TEST_SSL_CERT }}
MB_MYSQL_SSL_TEST_USER: metabase
MB_MYSQL_SSL_TEST_PASSWORD: ${{ secrets.MYSQL_RDS_SSL_INSTANCE_PASSWORD }}
# for historic reasons (I don't want to mess around with required jobs) the job name should be something like
# "be-tests-mariadb-10-2-ee (0)"
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
steps:
- uses: actions/checkout@v4
- name: Test ${{ matrix.version.name }}
uses: ./.github/actions/test-driver
with:
build-static-viz: ${{ matrix.job.build-static-viz }}
junit-name: ${{ matrix.version.junit-name }}
test-args: >-
${{ matrix.job.test-args }}
:exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-postgres:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
version:
- name: Postgres 12.x
junit-name: postgres-ee
docker-image: postgres:12-alpine
env:
enable-ssl-tests: 'false'
- name: Postgres Latest
junit-name: postgres-latest-ee
docker-image: postgres:latest
env:
enable-ssl-tests: 'true'
job:
- name: Driver Tests
build-static-viz: false
test-args: >-
:only-tags [:mb/driver-tests]
- name: Enterprise Tests
build-static-viz: false
test-args: >-
:only '"enterprise/backend/test"'
exclude-tag: ':mb/driver-tests'
- name: EE App DB Tests (Part 1)
build-static-viz: true
test-args: >-
:only '"test"'
:exclude-tags [:mb/driver-tests]
:partition/total 2
:partition/index 0
exclude-tag: ':mb/driver-tests'
- name: EE App DB Tests (Part 2)
build-static-viz: true
test-args: >-
:only '"test"'
:partition/total 2
:partition/index 1
exclude-tag: ':mb/driver-tests'
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
env:
# Only run old migrations tests on pushes to master or release branches. All other branches should skip tests
# with the tag `mb/old-migrations-test`. `__ADDITIONAL_EXCLUDED_TAG__` is not used anywhere outside of splicing
# it in to the command below.
__ADDITIONAL_EXCLUDED_TAG__: >-
${{
(
github.event_name == 'push' &&
(
github.ref == 'master' ||
startsWith(github.ref, 'release-')
) &&
''
) ||
':mb/old-migrations-test'
}}
# actual serious env vars below
CI: 'true'
DRIVERS: postgres
MB_DB_TYPE: postgres
MB_DB_PORT: 5432
MB_DB_HOST: localhost
MB_DB_DBNAME: mb_test
MB_DB_USER: mb_test
MB_POSTGRESQL_TEST_USER: mb_test
# SSL tests are only enabled for the postgres-latest job.
MB_POSTGRES_SSL_TEST_SSL: ${{ matrix.version.env.enable-ssl-tests }}
MB_POSTGRES_SSL_TEST_SSL_MODE: verify-full
MB_POSTGRES_SSL_TEST_SSL_ROOT_CERT_PATH: 'test-resources/certificates/us-east-2-bundle.pem'
services:
postgres:
image: ${{ matrix.version.docker-image }}
ports:
- "5432:5432"
env:
POSTGRES_USER: mb_test
POSTGRES_DB: mb_test
POSTGRES_HOST_AUTH_METHOD: trust
steps:
- uses: actions/checkout@v4
- name: Test ${{ matrix.version.name }} (${{ matrix.job.name }})
uses: ./.github/actions/test-driver
with:
build-static-viz: ${{ matrix.job.build-static-viz }}
junit-name: 'be-tests-${{ matrix.version.junit-name }}'
test-args: >-
${{ matrix.job.test-args }}
:exclude-tags '[ ${{ matrix.job.exclude-tag }} ${{ env.__ADDITIONAL_EXCLUDED_TAG__ }}]'
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
app-db-tests-result:
needs:
- be-tests-mariadb
- be-tests-mysql
- be-tests-postgres
runs-on: ubuntu-latest
timeout-minutes: 5
name: app-db-tests-result
if: always() && !cancelled()
steps:
- name: Check job status
uses: actions/github-script@v7
env:
needs: ${{ toJson(needs) }}
with:
script: | # js
const needs = JSON.parse(process.env.needs);
const jobs = Object.entries(needs).map(
([jobName, jobValues]) => ({
name: jobName,
result: jobValues.result
}));
// are all jobs skipped or successful?
if (jobs.every(job => (job.result === 'skipped' || job.result === 'success'))) {
console.log("");
console.log(" _------. ");
console.log(" / , \_ ");
console.log(" / / /{}\ |o\_ ");
console.log(" / \ `--' /-' \ ");
console.log(" | \ \ | ");
console.log(" | |`-, | ");
console.log(" / /__/)/ ");
console.log(" | | ");
console.log("");
console.log("All app db tests have passed (or have been skipped). Cam is very proud of you.");
process.exit(0);
}
// otherwise, something failed
console.log("");
console.log(" .::::::::::. .::::::::::. ");
console.log(" .::::''''''::::. .::::''''''::::. ");
console.log(" .:::' `::::.... ....::::' `:::. ");
console.log(" .::' `:::::::| |:::::::' `::. ");
console.log(" .::| |::::::|_ ___ __|::::::| |::. ");
console.log(" `--' |::::::|_()__()_|::::::| `--' ");
console.log(" ::: |::-o::| |::o-::| ::: ");
console.log(" `::. .|::::::| |::::::|. .::' ");
console.log(" `:::. .::\-----' `-----/::. .:::' ");
console.log(" `::::......::::' `::::......::::' ");
console.log(" `::::::::::' `::::::::::' ");
console.log("");
console.log("App db tests have failed. You have been sentenced to MetaJail.");
jobs.forEach((job) => {
if (job.result !== 'success') {
console.log(`${job.name} - ${job.result}`);
}
});
process.exit(1);

View File

@@ -1,38 +0,0 @@
name: Publish Athena JDBC Driver
on:
workflow_dispatch:
inputs:
version:
description: 'Athena JDBC driver version to publish'
required: true
type: string
jobs:
deploy:
name: Publish Athena JDBC Driver
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Check out the code
uses: actions/checkout@v4
- name: Set up Babashka
uses: turtlequeue/setup-babashka@v1.7.0
with:
babashka-version: 1.12.197
- name: Set up AWS CLI
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_S3_MAVEN_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_S3_MAVEN_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
- name: Deploy Athena JDBC driver
run: ./bin/athena.bb ${{ github.event.inputs.version }} --deploy
- name: Verify deployment
run: |
aws s3 ls s3://metabase-maven-downloads/com/metabase/athena-jdbc/${{ github.event.inputs.version }}/
echo "Athena JDBC driver version ${{ github.event.inputs.version }} has been deployed"

View File

@@ -1,16 +1,18 @@
name: Auto-backport
run-name: Auto Backport ${{ github.event.pull_request.title }}
on:
pull_request:
types: [closed, labeled]
jobs:
create-backport:
if: | # the PR must be merged and have a backport label
github.event.pull_request.merged == true && (
if: | # the PR must be closed and have a backport label
(github.event.pull_request.merged == true || github.event.action == 'closed') && (
contains(github.event.pull_request.labels.*.name, 'backport') ||
contains(github.event.pull_request.labels.*.name, 'double-backport') ||
contains(github.event.pull_request.labels.*.name, 'triple-backport')
contains(github.event.pull_request.labels.*.name, 'triple-backport') ||
github.event.label.name == 'backport' ||
github.event.label.name == 'double-backport' ||
github.event.label.name == 'triple-backport'
)
runs-on: ubuntu-latest
steps:
@@ -74,7 +76,7 @@ jobs:
github-token-2: ${{ secrets.GITHUB_TOKEN }}
- name: Double Backport
if: ${{ always() && fromJson(steps.how-many-backports.outputs.result) >= 2 }}
if: ${{ fromJson(steps.how-many-backports.outputs.result) >= 2 }}
uses: ./.github/actions/create-backport
with:
target-version: ${{ fromJson(steps.get-latest-release-version.outputs.result).secondLatest }}
@@ -83,7 +85,7 @@ jobs:
github-token-2: ${{ secrets.GITHUB_TOKEN }}
- name: Triple Backport
if: ${{ always() && fromJson(steps.how-many-backports.outputs.result) >= 3 }}
if: ${{ fromJson(steps.how-many-backports.outputs.result) >= 3 }}
uses: ./.github/actions/create-backport
with:
target-version: ${{ fromJson(steps.get-latest-release-version.outputs.result).thirdLatest }}

View File

@@ -0,0 +1,21 @@
# Config is in .github/auto_request_review.yml
name: Auto Request Review
on:
pull_request:
types: [opened, ready_for_review, reopened, synchronize]
jobs:
auto-request-review:
# Don't run on forked repos
if: ${{ !github.event.pull_request.head.repo.fork }}
permissions:
contents: read
pull-requests: write
name: Auto Request Review
runs-on: ubuntu-latest
steps:
- name: Request review based on files changes and/or groups the author belongs to
uses: necojackarc/auto-request-review@v0.13.0
with:
token: '${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}'

View File

@@ -19,8 +19,6 @@ jobs:
m2-cache-key: "cloverage"
- name: Build static viz frontend
run: yarn build-static-viz
env:
MB_EDITION: ee
- name: Collect the test coverage
run: clojure -X:dev:ci:ee:ee-dev:test:cloverage
- name: Upload coverage to codecov.io

View File

@@ -24,7 +24,7 @@ jobs:
with:
m2-cache-key: 'kondo'
- name: Run clj-kondo
run: ./bin/mage kondo
run: ./bin/kondo.sh
be-linter-eastwood:
if: ${{ !inputs.skip }}
@@ -40,6 +40,27 @@ jobs:
- run: clojure -X:dev:ee:ee-dev:drivers:drivers-dev:test:eastwood
name: Run Eastwood linter
mage-tests:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
name: Mage
steps:
- uses: actions/checkout@v4
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: 'kondo'
- name: Setup Babashka
uses: turtlequeue/setup-babashka@v1.7.0
with:
babashka-version: 1.12.196
- name: Check bb runs
run: bb --version
- name: bb task list
run: bb tasks
- name: Run mage tests
run: bb ./test/mage/core_test.clj
be-tests:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
@@ -114,8 +135,6 @@ jobs:
- name: Build static-viz frontend
if: ${{ matrix.job.build-static-viz == 'true' && !inputs.skip }}
run: yarn build-static-viz
env:
MB_EDITION: ${{ matrix.job.edition }}
- name: Download Static Viz Bundle Artifact
if: ${{ matrix.job.build-static-viz == 'true' && !inputs.skip }}
uses: actions/download-artifact@v4
@@ -153,14 +172,6 @@ jobs:
list-tests: failed
fail-on-error: false
- name: Upload Logs on Failure
if: always() && !success()
uses: actions/upload-artifact@v4
with:
name: test-logs-${{ matrix.java-version }}-${{ matrix.job.edition }}-${{ github.job }}
path: logs/test-log.json
retention-days: 1
# checks that all the namespaces we actually ship can be compiled, without any dependencies that we don't ship (such
# as `:dev` dependencies). See #27009 for more context.
be-check:
@@ -193,8 +204,7 @@ jobs:
with:
m2-cache-key: 'cljfmt'
- name: Run cljfmt
run: ./bin/mage cljfmt-all --force-check
run: clojure -T:cljfmt check
be-tests-result:
needs:

View File

@@ -5,7 +5,6 @@ on:
types: [auto_merge_enabled, review_requested, labeled, unlabeled, synchronize]
branches:
- "master"
merge_group:
jobs:
backport-reminder:

View File

@@ -1,14 +0,0 @@
name: Block Master Branch PRs
on:
pull_request:
types: [opened, synchronize]
jobs:
check-source-branch:
runs-on: ubuntu-latest
steps:
- name: Block master branch PRs
if: github.head_ref == 'master'
run: |
echo "PRs from `master` branches are not allowed, please use a new branch name for your PR."
exit 1

View File

@@ -3,7 +3,6 @@ name: Prevent backport.sh from being merged
on:
pull_request:
types: [opened, synchronize, reopened]
merge_group:
jobs:
check-backport-script:

View File

@@ -18,6 +18,7 @@ jobs:
timeout-minutes: 3
outputs:
frontend_all: ${{ steps.changes.outputs.frontend_all }}
e2e_specs: ${{ steps.changes.outputs.e2e_specs }}
steps:
- uses: actions/checkout@v4
- name: Test which files changed

View File

@@ -1,327 +0,0 @@
name: "Claude Code Bug Fix"
on:
issues:
types:
- labeled
pull_request_review:
types:
- submitted
workflow_dispatch:
inputs:
issue_number:
description: "Issue number to fix"
required: true
type: string
override_employee_check:
description: "Override employee check for issue creator"
required: false
default: false
type: boolean
jobs:
auto-fix-bug:
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: write
issues: write
pull-requests: write
id-token: write
if: |
(github.event_name == 'issues' && github.event.label.name == 'Claude Bug Fix' && github.event.issue.state == 'open') ||
(github.event_name == 'workflow_dispatch') ||
(github.event_name == 'pull_request_review' && github.event.review.state == 'changes_requested' && startsWith(github.event.pull_request.head.ref, 'claude-fix/'))
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }}
- name: Get issue data and check employee status
id: get_issue
uses: actions/github-script@v7
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { getIssueData } = require('${{ github.workspace }}/.github/scripts/get-issue-data.js');
try {
const result = await getIssueData({
context,
github,
manualIssueNumber: '${{ github.event.inputs.issue_number }}'
});
const { issueNumber, issueData, employeeComments, prReviews, triggerType, isIssueAuthorEmployee } = result;
// Check employee status and fail if not authorized
if (!result.isIssueAuthorEmployee && '${{ github.event.inputs.override_employee_check }}' !== 'true') {
core.setFailed(`User ${result.issueData.user.login} is not a Metabase employee. Skipping bug fix automation.`);
}
core.setOutput('issue_number', issueNumber);
core.setOutput('issue_data', JSON.stringify(issueData));
core.setOutput('issue_comments', JSON.stringify(employeeComments));
core.setOutput('pr_reviews', JSON.stringify(prReviews));
core.setOutput('trigger_type', triggerType);
} catch (error) {
core.setFailed(error.message);
}
- name: Set up git configuration
run: |
git config --global user.name "Claude Code[bot]"
git config --global user.email "noreply@anthropic.com"
- name: Prepare issue context and bug fix prompt
id: prepare_prompt
uses: actions/github-script@v7
env:
ISSUE_DATA: ${{ steps.get_issue.outputs.issue_data }}
ISSUE_COMMENTS: ${{ steps.get_issue.outputs.issue_comments }}
PR_REVIEWS: ${{ steps.get_issue.outputs.pr_reviews }}
TRIGGER_TYPE: ${{ steps.get_issue.outputs.trigger_type }}
with:
script: |
const issueData = JSON.parse(process.env.ISSUE_DATA);
const issueComments = JSON.parse(process.env.ISSUE_COMMENTS);
const prReviews = JSON.parse(process.env.PR_REVIEWS);
const triggerType = process.env.TRIGGER_TYPE;
const issueNumber = '${{ steps.get_issue.outputs.issue_number }}';
// Convert issue title to kebab case and create branch name
const issueTitle = issueData.title || 'untitled';
const kebabTitle = issueTitle
.toLowerCase()
.replace(/[^a-z0-9]+/g, '-')
.replace(/^-+|-+$/g, '')
.replace(/-+/g, '-')
.substring(0, 50);
const branchName = `claude-fix/issue-${issueNumber}-${kebabTitle}`;
// Format data for bash environment
const formattedComments = issueComments.length > 0
? issueComments.map(comment =>
`**Comment by @${comment.user.login} on ${comment.created_at}:**\n${comment.body}`
).join('\n\n---\n\n')
: '';
const formattedReviews = prReviews.length > 0
? prReviews.map(review =>
`**Review by @${review.user.login} on ${review.submitted_at} (Changes Requested):**\n${review.body || 'No review comment provided'}`
).join('\n\n---\n\n')
: '';
core.exportVariable('BRANCH_NAME', branchName);
core.exportVariable('ISSUE_NUMBER', issueNumber);
core.exportVariable('ISSUE_TITLE', issueData.title);
core.exportVariable('ISSUE_AUTHOR', issueData.user.login);
core.exportVariable('ISSUE_CREATED', issueData.created_at);
core.exportVariable('ISSUE_BODY', issueData.body || 'No description provided');
core.exportVariable('ISSUE_COMMENTS_FORMATTED', formattedComments);
core.exportVariable('PR_REVIEWS_FORMATTED', formattedReviews);
core.exportVariable('TRIGGER_TYPE', triggerType);
- name: Generate bug fix prompt
run: |
cat << 'EOF' >> $GITHUB_ENV
BUG_FIX_PROMPT<<PROMPT_END
You are helping to fix a bug reported in GitHub issue #${{ env.ISSUE_NUMBER }}.
## Bug Report Details
**Title:** ${{ env.ISSUE_TITLE }}
**Reporter:** ${{ env.ISSUE_AUTHOR }}
**Created:** ${{ env.ISSUE_CREATED }}
**Description:**
${{ env.ISSUE_BODY }}
**Comments:**
${{ env.ISSUE_COMMENTS_FORMATTED }}
EOF
# Add review feedback section only if this is a review trigger
if [ "${{ env.TRIGGER_TYPE }}" = "pull_request_review" ]; then
cat << 'EOF' >> $GITHUB_ENV
## Previous Fix Attempt - Review Feedback (MUST ADDRESS)
Your previous automated fix attempt received the following feedback that needs to be addressed:
${{ env.PR_REVIEWS_FORMATTED }}
EOF
fi
# Add the rest of the prompt (always the same)
cat << 'EOF' >> $GITHUB_ENV
## Instructions
1. **Analyze the issue description and comments** - Understand what the issue is describing
a. Pay particular attention to reproduction steps and if there's an agreed upon fix.
2. **Search the codebase** - Use Grep and Glob tools to find relevant files
3. **Implement a fix** - Follow existing code patterns and conventions
4. **Test the fix** - Run appropriate tests using the commands in CLAUDE.md:
- JavaScript/TypeScript: `yarn test-unit path/to/file.unit.spec.js`
- Clojure: `clojure -X:dev:test :only namespace/test-name`
- Lint: `yarn lint-eslint-pure` for JS/TS, `./bin/mage kondo <file>` for Clojure
5. **Verify the fix** - Ensure tests pass and code follows conventions
## Important Guidelines
- Follow the development patterns in CLAUDE.md
- Write failing tests first, then fix them
- Work in small, testable increments
- Don't commit changes - leave that for the workflow
- IMPORTANT: Focus only on fixing the reported bug, don't make unrelated changes
## Scratchpad Directory
A `.claude-scratchpad/` directory has been created for your temporary files:
- Use this directory for ALL temporary verification files. However, prefer using production test files rather than adhoc ones.
- Only modify actual production code in the main repository directories
- The scratchpad will be cleaned up automatically - nothing from it will be committed
Start by understanding the bug and then systematically work toward a solution.
PROMPT_END
EOF
- name: Determine branch strategy
run: |
BRANCH_NAME="${{ env.BRANCH_NAME }}"
if git branch -r --list "origin/${BRANCH_NAME}" | grep -q .; then
echo "Branch already exists: $BRANCH_NAME"
echo "Will push to existing branch and skip PR creation"
echo "BRANCH_EXISTS=true" >> $GITHUB_ENV
else
echo "Creating new branch: $BRANCH_NAME"
echo "Will create new PR"
echo "BRANCH_EXISTS=false" >> $GITHUB_ENV
fi
# Export the branch name
echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_ENV
- name: Create scratchpad directory for Claude
run: |
# Create a scratchpad directory for Claude to use for temporary files
mkdir -p .claude-scratchpad
echo "Scratchpad directory created for Claude's temporary verification files"
- name: Generate GitHub App Token for Claude
uses: actions/create-github-app-token@v1
id: claude-app-token
with:
app-id: ${{ secrets.METABASE_BOT_APP_ID }}
private-key: ${{ secrets.METABASE_BOT_APP_PRIVATE_KEY }}
- name: Generate Bug Fix with Claude Code
uses: anthropics/claude-code-action@beta
with:
mode: agent
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
github_token: ${{ steps.claude-app-token.outputs.token }}
direct_prompt: ${{ env.BUG_FIX_PROMPT }}
timeout_minutes: "25"
allowed_tools: "Bash(yarn test-unit*),Bash(yarn lint-eslint-pure),Bash(./bin/mage*),Bash(clojure*),Bash(find*),Bash(ls*),Bash(grep*),Write,Read,Glob,Grep,Edit,MultiEdit"
disallowed_tools: "Bash(git commit*),Bash(git push*),Bash(gh*)"
- name: Clean up scratchpad directory
run: |
# Remove the scratchpad directory so its contents don't get committed
rm -rf .claude-scratchpad/
# Remove output file generated by Claude Code action
rm -f output.txt
echo "Scratchpad directory and action output files cleaned up"
- name: Check for changes and analysis
id: check_changes
run: |
# Now check for actual code changes (excluding analysis file)
if [ -n "$(git status --porcelain)" ]; then
echo "has_changes=true" >> $GITHUB_OUTPUT
echo "Changes detected:"
git status --short
else
echo "has_changes=false" >> $GITHUB_OUTPUT
echo "No changes made by Claude."
fi
- name: Get issue team labels and prepare reviewers
if: steps.check_changes.outputs.has_changes == 'true'
id: prepare_reviewers
uses: actions/github-script@v7
env:
ISSUE_DATA: ${{ steps.get_issue.outputs.issue_data }}
with:
script: |
const issueData = JSON.parse(process.env.ISSUE_DATA);
const issueLabels = issueData.labels.map(label => label.name);
const teamLabels = issueLabels.filter(label => label.startsWith('.Team/'));
const issueCreator = issueData.user.login;
core.setOutput('team_labels', teamLabels.join(','));
core.setOutput('reviewers', issueCreator);
- name: Create Pull Request
if: steps.check_changes.outputs.has_changes == 'true'
uses: peter-evans/create-pull-request@v6
with:
token: ${{ secrets.GITHUB_TOKEN }}
branch: ${{ env.BRANCH_NAME }}
base: master
commit-message: |
Fix: ${{ env.ISSUE_TITLE }}
Fixes #${{ env.ISSUE_NUMBER }}
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
committer: "Claude Code[bot] <noreply@anthropic.com>"
author: "Claude Code[bot] <noreply@anthropic.com>"
title: "Fix: ${{ env.ISSUE_TITLE }}"
body: |
This PR automatically fixes the bug reported in #${{ env.ISSUE_NUMBER }}.
## Bug Report
**Original Issue:** #${{ env.ISSUE_NUMBER }}
**Reporter:** @${{ env.ISSUE_AUTHOR }}
**Issue Title:** ${{ env.ISSUE_TITLE }}
## Automated Fix
🤖 This fix was generated automatically by Claude Code based on the issue description.
**Please review carefully:**
- [ ] The fix addresses the reported issue
- [ ] Tests pass and code follows conventions
- [ ] No unrelated changes were introduced
- [ ] The solution is appropriate and maintainable
## Next Steps
- Review the changes and test thoroughly
- If the fix looks good, approve and merge
- If issues are found, either fix them or close this PR
---
Closes #${{ env.ISSUE_NUMBER }}
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
labels: |
${{ steps.prepare_reviewers.outputs.team_labels }}
automated-fix
ci skip
reviewers: ${{ steps.prepare_reviewers.outputs.reviewers }}
- name: Log existing branch update
if: steps.check_changes.outputs.has_changes == 'true' && env.BRANCH_EXISTS == 'true'
uses: actions/github-script@v7
with:
script: |
console.log('🔄 Updated existing branch: ${{ env.BRANCH_NAME }}');
console.log('The changes have been pushed to the existing PR for this issue.');
- name: Log analysis for no changes
if: steps.check_changes.outputs.has_changes == 'false'
uses: actions/github-script@v7
with:
script: |
console.log('🤖 No code changes were made by Claude.');

View File

@@ -40,5 +40,5 @@ jobs:
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: 'cljs'
- name: Run Cljs tests
- name: Run Cljs tests for shared/ code
run: yarn run test-cljs

View File

@@ -13,11 +13,6 @@ on:
type: string
commit:
type: string
release:
type: string
default: 'true'
build-args:
type: string
workflow_dispatch:
inputs:
artifact-name:
@@ -61,7 +56,7 @@ jobs:
const version = "${{ inputs.tag }}".split('-')[0];
if (!isValidVersionString(version)) {
return "linux/amd64,linux/arm64";
return "linux/amd64";
}
// versions before 53 need to be handled differently
const buildInfo = getBuildRequirements(version);
@@ -71,8 +66,8 @@ jobs:
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ inputs.release == 'true' && secrets.DOCKERHUB_RELEASE_USERNAME || secrets.DOCKERHUB_USERNAME }}
password: ${{ inputs.release == 'true' && secrets.DOCKERHUB_RELEASE_TOKEN || secrets.DOCKERHUB_TOKEN }}
username: ${{ secrets.DOCKERHUB_RELEASE_USERNAME }}
password: ${{ secrets.DOCKERHUB_RELEASE_TOKEN }}
- name: Set up QEMU
if: contains(steps.platforms.outputs.result, 'arm')
uses: docker/setup-qemu-action@v3
@@ -89,7 +84,6 @@ jobs:
tags: ${{ inputs.repo }}:${{ inputs.tag }}
no-cache: true
push: true
build-args: ${{ inputs.build-args }}
# we need to do this to get the nice fallback behavior of the legacy manifest file
- name: Build container (x86 only)
@@ -109,8 +103,8 @@ jobs:
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ inputs.release == 'true' && secrets.DOCKERHUB_RELEASE_USERNAME || secrets.DOCKERHUB_USERNAME }}
password: ${{ inputs.release == 'true' && secrets.DOCKERHUB_RELEASE_TOKEN || secrets.DOCKERHUB_TOKEN }}
username: ${{ secrets.DOCKERHUB_RELEASE_USERNAME }}
password: ${{ secrets.DOCKERHUB_RELEASE_TOKEN }}
- name: Pull the container image
run: docker pull ${{ inputs.repo }}:${{ inputs.tag }}
- name: Launch container
@@ -129,8 +123,8 @@ jobs:
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ inputs.release == 'true' && secrets.DOCKERHUB_RELEASE_USERNAME || secrets.DOCKERHUB_USERNAME }}
password: ${{ inputs.release == 'true' && secrets.DOCKERHUB_RELEASE_TOKEN || secrets.DOCKERHUB_TOKEN }}
username: ${{ secrets.DOCKERHUB_RELEASE_USERNAME }}
password: ${{ secrets.DOCKERHUB_RELEASE_TOKEN }}
- name: Pull the container image
run: docker pull ${{ inputs.repo }}:${{ inputs.tag }}
- name: Launch container

View File

@@ -1,126 +0,0 @@
name: Containerize Uberjar
on:
workflow_call:
workflow_dispatch:
jobs:
get-container-info:
runs-on: ubuntu-22.04
timeout-minutes: 5
name: Get container info
outputs:
editions: ${{ steps.ee-info.outputs.editions }}
ee: ${{ toJson(fromJson(steps.build-matrix.outputs.result).ee) }}
oss: ${{ toJson(fromJson(steps.build-matrix.outputs.result).oss) }}
steps:
- name: Check out the code
uses: actions/checkout@v4
- name: Get EE Docker info
id: ee-info
uses: ./.github/actions/get-docker-info
with:
edition: ee
- name: Get OSS Docker info
id: oss-info
uses: ./.github/actions/get-docker-info
with:
edition: oss
- name: Build matrix info
id: build-matrix
uses: actions/github-script@v7
with:
script: | # js
const editions = JSON.parse('${{ steps.ee-info.outputs.editions }}');
const result = {
editions: editions,
ee: {
repo: '${{ steps.ee-info.outputs.repo }}',
tag: '${{ steps.ee-info.outputs.tag }}'
},
oss: {
repo: '${{ steps.oss-info.outputs.repo }}',
tag: '${{ steps.oss-info.outputs.tag }}'
}
};
console.log('Build matrix:', result);
return result;
containerize:
needs: [get-container-info]
if: needs.get-container-info.result == 'success'
strategy:
matrix:
edition: ${{ fromJson(needs.get-container-info.outputs.editions) }}
uses: ./.github/workflows/containerize-jar.yml
secrets: inherit
with:
artifact-name: metabase-${{ matrix.edition }}-${{ github.event.inputs.commit || github.event.pull_request.head.sha || github.sha }}-uberjar
commit: ${{ github.event.inputs.commit || github.sha }}
repo: ${{ fromJson(needs.get-container-info.outputs[matrix.edition]).repo }}
tag: ${{ fromJson(needs.get-container-info.outputs[matrix.edition]).tag }}
release: "false"
build-args: |
GIT_COMMIT_SHA=${{ github.event.inputs.commit || github.sha }}
run-trivy:
needs: [get-container-info, containerize]
strategy:
matrix:
edition: ${{ fromJson(needs.get-container-info.outputs.editions) }}
runs-on: ubuntu-22.04
name: Run Trivy vulnerability scanner
steps:
- name: Check out the code
uses: actions/checkout@v4
with:
ref: ${{ github.event.inputs.commit || github.sha }}
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@0.29.0
env:
TRIVY_OFFLINE_SCAN: true
repo: ${{ fromJson(needs.get-container-info.outputs[matrix.edition]).repo }}
tag: ${{ fromJson(needs.get-container-info.outputs[matrix.edition]).tag }}
with:
image-ref: docker.io/${{ env.repo }}:${{ env.tag }}
format: sarif
output: trivy-results.sarif
version: "v0.57.1"
- name: Upload Trivy scan results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: "trivy-results.sarif"
post-comment:
needs: [get-container-info, containerize]
if: github.event_name == 'pull_request'
runs-on: ubuntu-22.04
name: Post PR comment
steps:
- name: Check out the code
uses: actions/checkout@v4
- name: Get Docker info for comment
id: docker-info
uses: ./.github/actions/get-docker-info
with:
branch: ${{ github.head_ref || github.ref_name }}
edition: ee
- name: Post PR comment
uses: marocchino/sticky-pull-request-comment@v2
with:
header: docker-build-status
message: |
✅ Docker build completed for commit ${{ github.event.pull_request.head.sha || github.sha }}
The uberjar has been built and containerized successfully.
**Branch:** `${{ github.head_ref || github.ref_name }}`
**Docker Image:** `${{ steps.docker-info.outputs.full_image }}`
**Docker Hub:** [View on Docker Hub](https://hub.docker.com/r/${{ steps.docker-info.outputs.repo }}/tags?name=${{ steps.docker-info.outputs.tag }})
**Triggered by:** @${{ github.actor }}
Pull the image:
```bash
docker pull ${{ steps.docker-info.outputs.full_image }}
```

View File

@@ -4,53 +4,39 @@ on:
pull_request:
branches:
- master
paths:
- enterprise/frontend/src/embedding-sdk/**
jobs:
files-changed:
name: Check which files changed
runs-on: ubuntu-22.04
timeout-minutes: 3
outputs:
frontend_embedding_sources: ${{ steps.changes.outputs.frontend_embedding_sources }}
steps:
- uses: actions/checkout@v4
- name: Test which files changed
uses: dorny/paths-filter@v3.0.0
id: changes
with:
token: ${{ github.token }}
filters: .github/file-paths.yaml
pr-title-reminder:
runs-on: ubuntu-22.04
timeout-minutes: 5
permissions:
pull-requests: write
needs: [ files-changed ]
if: |
!cancelled() &&
needs.files-changed.outputs.frontend_embedding_sources == 'true'
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.head_ref }}
- name: Check if the PR title is already following Conventional Commits style
uses: actions/github-script@v7
id: is-conventional-commit
with:
result-encoding: string
script: | # js
const { isConventionalTitle } = require(".github/scripts/conventional-commit-pr-title-reminder/utils.js")
const pullRequestTitle = context.payload.pull_request.title
const conventionalCommitRegex = /^(?<type>\w+)\((?<scope>\w+)\):/
const match = pullRequestTitle.match(conventionalCommitRegex)
if (match) {
const { type, scope } = match.groups
return ['feat', 'fix', 'perf', 'docs', 'style', 'refactor', 'test', 'build', 'ci'].includes(type) && scope === 'sdk'
}
return isConventionalTitle(pullRequestTitle)
- name: Checkout repository so `gh` CLI works
uses: actions/checkout@v4
with:
ref: ${{ github.head_ref }}
- name: Remind PR authors to use conventional commit in the title
if: ${{ steps.is-conventional-commit.outputs.result != 'true' }}
uses: actions/github-script@v7
with:
script: | #js
const { updateCommentForEmbedding } = require(".github/scripts/conventional-commit-pr-title-reminder/utils.js")
await updateCommentForEmbedding({github, context})
script: |
const { updateComment } = require(".github/scripts/conventional-commit-pr-title-reminder/utils.js")
await updateComment({github, context})

View File

@@ -1,26 +0,0 @@
name: Test coverage
on:
schedule:
- cron: '40 1 * * *' # every day at 1:40
jobs:
fe-test-coverage:
runs-on: ubuntu-22.04
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: "cljs"
- name: Run frontend unit tests
run: yarn run test-unit --silent --coverage
- name: Upload coverage to codecov.io
uses: codecov/codecov-action@v4
with:
files: ./coverage/lcov.info
flags: front-end
token: ${{ secrets.CODECOV_TOKEN }}

View File

@@ -5,19 +5,6 @@ on:
types: [opened, synchronize, reopened, ready_for_review]
paths:
- "resources/migrations/**"
workflow_dispatch:
inputs:
base-ref:
description: 'Base reference for the migration test'
required: true
type: string
head-ref:
description: 'Head reference for the migration test'
required: true
type: string
env:
BASE_REF: ${{ inputs.base-ref || github.base_ref }}
HEAD_REF: ${{ inputs.head-ref || github.head_ref }}
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
@@ -26,21 +13,19 @@ concurrency:
jobs:
migrations-h2-ee:
runs-on: ubuntu-22.04
timeout-minutes: 10
timeout-minutes: 5
env:
CI: 'true'
MB_DB_TYPE: h2
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/cross-branch-migration-test
with:
base-ref: ${{ env.BASE_REF }}
head-ref: ${{ env.HEAD_REF }}
base-ref: ${{ github.base_ref }}
migrations-postgres-12-ee:
runs-on: ubuntu-22.04
timeout-minutes: 10
timeout-minutes: 5
env:
CI: 'true'
MB_DB_TYPE: postgres
@@ -61,12 +46,11 @@ jobs:
- uses: actions/checkout@v4
- uses: ./.github/actions/cross-branch-migration-test
with:
base-ref: ${{ env.BASE_REF }}
head-ref: ${{ env.HEAD_REF }}
base-ref: ${{ github.base_ref }}
migrations-postgres-latest-ee:
runs-on: ubuntu-22.04
timeout-minutes: 10
timeout-minutes: 5
env:
CI: 'true'
MB_DB_TYPE: postgres
@@ -87,12 +71,11 @@ jobs:
- uses: actions/checkout@v4
- uses: ./.github/actions/cross-branch-migration-test
with:
base-ref: ${{ env.BASE_REF }}
head-ref: ${{ env.HEAD_REF }}
base-ref: ${{ github.base_ref }}
migrations-mysql-8-0-ee:
runs-on: ubuntu-22.04
timeout-minutes: 10
timeout-minutes: 5
env:
CI: 'true'
MB_DB_TYPE: mysql
@@ -109,12 +92,11 @@ jobs:
- uses: actions/checkout@v4
- uses: ./.github/actions/cross-branch-migration-test
with:
base-ref: ${{ env.BASE_REF }}
head-ref: ${{ env.HEAD_REF }}
base-ref: ${{ github.base_ref }}
migrations-mysql-latest-ee:
runs-on: ubuntu-22.04
timeout-minutes: 10
timeout-minutes: 5
env:
CI: 'true'
MB_DB_TYPE: mysql
@@ -135,12 +117,11 @@ jobs:
- uses: actions/checkout@v4
- uses: ./.github/actions/cross-branch-migration-test
with:
base-ref: ${{ env.BASE_REF }}
head-ref: ${{ env.HEAD_REF }}
base-ref: ${{ github.base_ref }}
migrations-mariadb-10-2-ee:
runs-on: ubuntu-22.04
timeout-minutes: 10
timeout-minutes: 5
env:
CI: 'true'
MB_DB_TYPE: mysql
@@ -157,12 +138,11 @@ jobs:
- uses: actions/checkout@v4
- uses: ./.github/actions/cross-branch-migration-test
with:
base-ref: ${{ env.BASE_REF }}
head-ref: ${{ env.HEAD_REF }}
base-ref: ${{ github.base_ref }}
migrations-mariadb-latest-ee:
runs-on: ubuntu-22.04
timeout-minutes: 10
timeout-minutes: 5
env:
CI: 'true'
MB_DB_TYPE: mysql
@@ -179,5 +159,4 @@ jobs:
- uses: actions/checkout@v4
- uses: ./.github/actions/cross-branch-migration-test
with:
base-ref: ${{ env.BASE_REF }}
head-ref: ${{ env.HEAD_REF }}
base-ref: ${{ github.base_ref }}

View File

@@ -1,182 +0,0 @@
# .github/workflows/reusable-doc-update.yml
name: Reusable - Update Documentation
# This workflow is designed to be called by other workflows.
on:
workflow_call:
inputs:
target_branch:
description: "The branch to check out and create a PR against"
required: true
type: string
m2_cache_key_suffix:
description: "A unique suffix for the Maven cache key"
required: true
type: string
secrets:
automation_token:
description: "A token with permissions to create branches and PRs"
required: true
permissions:
contents: write
pull-requests: write
jobs:
update-docs:
name: Update documentation for ${{ inputs.target_branch }}
runs-on: ubuntu-22.04
timeout-minutes: 30
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.automation_token }}
ref: ${{ inputs.target_branch }}
- name: Prepare backend environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: "auto-update-docs-${{ inputs.m2_cache_key_suffix }}"
- name: Generate environment variables documentation
id: generate_env_docs
run: |
set -e # Exit on any error
# Ensure we're on the target branch for documentation generation
git checkout ${{ inputs.target_branch }}
echo "Generating environment variables documentation from branch: ${{ inputs.target_branch }}"
clojure -M:ee:doc environment-variables-documentation
continue-on-error: false
- name: Generate config template documentation
id: generate_config_docs
run: |
set -e # Exit on any error
echo "Generating config template documentation..."
clojure -M:ee:doc config-template
continue-on-error: false
- name: Generate API documentation
id: generate_api_docs
run: |
set -e # Exit on any error
echo "Generating API documentation..."
clojure -M:ee:doc api-documentation
continue-on-error: false
- name: Check for changes
id: check_changes
run: |
if git diff --quiet docs/; then
echo "changes=false" >> $GITHUB_OUTPUT
echo "No changes detected in docs directory"
else
echo "changes=true" >> $GITHUB_OUTPUT
echo "Changes detected in docs directory"
fi
- name: Create branch and commit changes
id: commit_and_push
if: steps.check_changes.outputs.changes == 'true'
run: |
git config user.name github-actions
git config user.email github-actions@github.com
BRANCH_NAME="docs/auto-update-${{ inputs.target_branch }}-$(date +%Y-%m-%d-%H%M%S)"
echo "Creating branch: $BRANCH_NAME"
# Create and switch to new branch from current target branch
git checkout -b "$BRANCH_NAME"
# Stage and commit changes
git add docs/
if git diff --cached --quiet; then
echo "No changes to commit"
exit 1
fi
git commit -m "Update generated docs (${{ inputs.target_branch }})" --no-verify
git push -u origin "$BRANCH_NAME"
echo "branch_name=$BRANCH_NAME" >> $GITHUB_OUTPUT
echo "Successfully created and pushed branch"
- name: Create pull request
id: create_pr
if: steps.check_changes.outputs.changes == 'true'
env:
GITHUB_TOKEN: ${{ secrets.automation_token }}
GH_REPO: ${{ github.repository }}
run: |
CHANGES_SUMMARY=$(git diff --name-only HEAD~1 docs/ 2>/dev/null | head -10 | sed 's/^/- /' || echo "- No specific files detected")
TOTAL_CHANGES=$(git diff --name-only HEAD~1 docs/ 2>/dev/null | wc -l || echo "0")
BRANCH_NAME="${{ steps.commit_and_push.outputs.branch_name }}"
echo "Creating PR with branch: $BRANCH_NAME"
echo "Changes detected: $TOTAL_CHANGES files"
for i in {1..3}; do
echo "Attempt $i of 3..."
if gh pr create \
--base "${{ inputs.target_branch }}" \
--head "$BRANCH_NAME" \
--title "Auto-update documentation for ${{ inputs.target_branch }}" \
--body "This PR contains automatically generated documentation updates.
## Changes detected
The following documentation files were updated:
$CHANGES_SUMMARY
$([ "$TOTAL_CHANGES" -gt 10 ] && echo "- ... and $((TOTAL_CHANGES - 10)) more files")
## Generation commands
Documentation was generated using:
- \`clojure -M:ee:doc environment-variables-documentation\`
- \`clojure -M:ee:doc config-template\`
- \`clojure -M:ee:doc api-documentation\`
Please review the changes and merge if they look correct." \
--label "Type:Documentation" \
--label "no-backport" \
--reviewer "metabase/tech-writers"; then
echo "PR created successfully on attempt $i"
break
else
echo "PR creation failed on attempt $i"
if [ $i -eq 3 ]; then
echo "All attempts failed. Exiting with error."
exit 1
fi
echo "Waiting 10 seconds before retry..."
sleep 10
fi
done
- name: Skip PR creation
if: steps.check_changes.outputs.changes == 'false'
run: |
echo "No documentation changes detected, skipping PR creation"
- name: Workflow summary
if: always()
run: |
if [ "${{ steps.check_changes.outputs.changes }}" == "true" ]; then
if [ "${{ steps.commit_and_push.outcome }}" == "success" ]; then
if [ "${{ steps.create_pr.outcome }}" == "success" ]; then
echo "## ✅ Documentation Updated Successfully" >> $GITHUB_STEP_SUMMARY
echo "- Branch: \`${{ steps.commit_and_push.outputs.branch_name }}\`" >> $GITHUB_STEP_SUMMARY
echo "- PR created targeting **${{ inputs.target_branch }}**" >> $GITHUB_STEP_SUMMARY
else
echo "## ⚠️ Branch Created but PR Failed" >> $GITHUB_STEP_SUMMARY
echo "- Branch: \`${{ steps.commit_and_push.outputs.branch_name }}\`" >> $GITHUB_STEP_SUMMARY
echo "- Manual PR creation required" >> $GITHUB_STEP_SUMMARY
fi
else
echo "## ❌ Failed to Create Branch" >> $GITHUB_STEP_SUMMARY
echo "- Check workflow logs for details" >> $GITHUB_STEP_SUMMARY
fi
else
echo "## ℹ️ No Changes Detected" >> $GITHUB_STEP_SUMMARY
echo "- Documentation is up to date on **${{ inputs.target_branch }}**" >> $GITHUB_STEP_SUMMARY
fi

View File

@@ -1,36 +0,0 @@
name: Check doc links
on:
pull_request:
paths:
- "docs/**"
workflow_dispatch:
jobs:
link-checker:
name: Check documentation links
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
sparse-checkout: |
docs
.lychee
- name: Cache lychee
uses: actions/cache@v3
with:
path: ~/.cache/lychee
key: lychee-${{ runner.os }}-${{ hashFiles('.lychee/config.toml') }}
- name: Run link checker
id: link-check
uses: lycheeverse/lychee-action@v2
with:
fail: true
args: docs --config ./.lychee/config.toml --offline
format: markdown

View File

@@ -1,16 +0,0 @@
name: Auto-update documentation (master)
on:
schedule:
- cron: "0 1 * * 1-5" # Every Monday through Friday at 1 AM UTC
workflow_dispatch:
jobs:
update-master-docs:
name: Call documentation update for master
uses: ./.github/workflows/docs-generate-base.yml
with:
target_branch: "master"
m2_cache_key_suffix: "master"
secrets:
automation_token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}

View File

@@ -1,16 +0,0 @@
name: Auto-update documentation (release)
on:
schedule:
- cron: "0 2 * * 1-5" # Every Monday through Friday at 2 AM UTC
workflow_dispatch:
jobs:
update-release-docs:
name: Call documentation update for release branch
uses: ./.github/workflows/docs-generate-base.yml
with:
target_branch: "release-x.56.x"
m2_cache_key_suffix: "release"
secrets:
automation_token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}

View File

@@ -1,44 +0,0 @@
name: Docs Bump Detected
on:
pull_request_target:
types:
- opened
- synchronize
- reopened
branches:
- 'master'
- 'release-x.[0-9]+.x'
paths:
- 'docs/**'
jobs:
push_workflow_to_docs_metabase_github_io_job:
runs-on: ubuntu-latest
env:
SOURCE_BRANCH: ${{ github.head_ref || github.ref }}
TARGET_BRANCH: ${{ github.event.pull_request.base.ref }}
PR_NUMBER: ${{ github.event.pull_request.number }}
steps:
- name: Report changed docs files exist
run: |
echo "A PR with changes in the docs directory was detected."
echo "Sending workflow trigger to create/update the corresponding PR in docs repo."
echo "source branch: $SOURCE_BRANCH"
echo "target branch: $TARGET_BRANCH"
echo "pr number: $PR_NUMBER"
- name: Trigger repo dispatch
uses: actions/github-script@v7
with:
github-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
script: | #js
github.rest.repos.createDispatchEvent({
owner: "${{ github.repository_owner }}",
repo: "docs.metabase.github.io",
event_type: "docs_update",
client_payload: {
dispatch_type: "docs_update",
source_branch: process.env.SOURCE_BRANCH,
target_branch: process.env.TARGET_BRANCH,
pr_number: process.env.PR_NUMBER}});

View File

@@ -1,43 +0,0 @@
name: Docs Close Detected
on:
pull_request:
types:
- closed
branches:
- 'master'
- 'release-x.[0-9]+.x'
paths:
- 'docs/**'
jobs:
push_and_merge_workflow_to_docs_metabase_github_io_job:
if: github.event.pull_request.merged == false
runs-on: ubuntu-latest
env:
SOURCE_BRANCH: ${{ github.head_ref || github.ref }}
TARGET_BRANCH: ${{ github.event.pull_request.base.ref }}
PR_NUMBER: ${{ github.event.pull_request.number }}
steps:
- name: Report changed docs files exist
run: |
echo "A PR with changes to the docs directory was closed."
echo "Sending workflow trigger to close corresponding PR in docs repo."
echo "source branch: $SOURCE_BRANCH"
echo "target branch: $TARGET_BRANCH"
echo "pr number: $PR_NUMBER"
- name: Trigger repo dispatch
uses: actions/github-script@v7
with:
github-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
script: | #js
github.rest.repos.createDispatchEvent({
owner: "${{ github.repository_owner }}",
repo: "docs.metabase.github.io",
event_type: "docs_close",
client_payload: {
dispatch_type: "docs_close",
source_branch: process.env.SOURCE_BRANCH,
target_branch: process.env.TARGET_BRANCH,
pr_number: process.env.PR_NUMBER}});

View File

@@ -1,43 +0,0 @@
name: Docs Merge Detected
on:
pull_request:
types:
- closed
branches:
- 'master'
- 'release-x.[0-9]+.x'
paths:
- 'docs/**'
jobs:
push_and_merge_workflow_to_docs_metabase_github_io_job:
if: github.event.pull_request.merged == true
runs-on: ubuntu-latest
env:
SOURCE_BRANCH: ${{ github.head_ref || github.ref }}
TARGET_BRANCH: ${{ github.event.pull_request.base.ref }}
PR_NUMBER: ${{ github.event.pull_request.number }}
steps:
- name: Report changed docs files exist
run: |
echo "A PR with changes to the docs directory was merged."
echo "Sending workflow trigger to merge corresponding PR in docs repo."
echo "source branch: $SOURCE_BRANCH"
echo "target branch: $TARGET_BRANCH"
echo "pr number: $PR_NUMBER"
- name: Trigger repo dispatch
uses: actions/github-script@v7
with:
github-token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
script: | #js
github.rest.repos.createDispatchEvent({
owner: "${{ github.repository_owner }}",
repo: "docs.metabase.github.io",
event_type: "docs_merge",
client_payload: {
dispatch_type: "docs_merge",
source_branch: process.env.SOURCE_BRANCH,
target_branch: process.env.TARGET_BRANCH,
pr_number: process.env.PR_NUMBER}});

View File

@@ -42,7 +42,6 @@ on:
- SQLite
- SQL Server 2017
- SQL Server 2022
- Starburst
- Vertica
jobs:
@@ -664,50 +663,24 @@ jobs:
junit-name: 'be-tests-sqlserver-2022-ee'
test-args: ":only ${{ inputs.test }} :times ${{ inputs.times }}"
be-tests-starburst-ee:
if: inputs.driver == 'Starburst'
be-tests-vertica-ee:
if: inputs.driver == 'Vertica'
runs-on: ubuntu-22.04
timeout-minutes: 90
env:
CI: 'true'
DRIVERS: starburst
DRIVERS: vertica
services:
starburst:
image: trinodb/trino:latest
env:
CATALOG_MANAGEMENT: dynamic
vertica:
image: vertica/vertica-ce:12.0.2-0
ports:
- "8080:8080"
name: Starburst
- "5433:5433"
steps:
- uses: actions/checkout@v4
- name: Make plugins directory
run: mkdir plugins
- name: Test Starburst driver
- name: Test Vertica driver
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-starburst-ee'
junit-name: 'be-tests-vertica-ee'
test-args: ":only ${{ inputs.test }} :times ${{ inputs.times }}"
# TODO(rileythomp, 2025-08-20): Re-enable vertica tests once the docker image is updated
# be-tests-vertica-ee:
# if: ${{ false && inputs.driver == 'Vertica' }}
# runs-on: ubuntu-22.04
# timeout-minutes: 90
# env:
# CI: 'true'
# DRIVERS: vertica
# services:
# vertica:
# image: vertica/vertica-ce:12.0.2-0
# ports:
# - "5433:5433"
# steps:
# - uses: actions/checkout@v4
# - name: Make plugins directory
# run: mkdir plugins
# - name: Test Vertica driver
# uses: ./.github/actions/test-driver
# with:
# junit-name: 'be-tests-vertica-ee'
# test-args: ":only ${{ inputs.test }} :times ${{ inputs.times }}"

View File

@@ -12,72 +12,44 @@ concurrency:
cancel-in-progress: true
jobs:
# This job will determine if the tier 3 driver tests should be skipped.
# It will always run first.
determine-driver-skips:
runs-on: ubuntu-latest
outputs:
skip: ${{ steps.check_team_and_branch.outputs.skip }}
be-tests-h2:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
env:
CI: 'true'
name: H2
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Fetch base branch
if: github.event_name == 'pull_request' && github.event.pull_request.base.ref != ''
run: |
# Fetch the base branch to ensure its history is available for diffing
git fetch origin ${{ github.event.pull_request.base.ref }}:${{ github.event.pull_request.base.ref }}
# Only run this if it's a pull request and the base ref exists
- name: Check if tier3 drivers should run
id: check_team_and_branch
run: |
IS_MASTER_OR_RELEASE="${{ github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release-') }}"
HAS_TEAM_LABEL="${{ contains(github.event.pull_request.labels.*.name, '.Team/Drivers') || contains(github.event.pull_request.labels.*.name, '.Team/Querying') || contains(github.event.pull_request.labels.*.name, '.Team/SemanticLayer') || contains(github.event.pull_request.labels.*.name, '.Team/DevEx') }}"
- uses: actions/checkout@v4
- name: Test H2 driver
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-h2'
test-args: >-
:only-tags [:mb/driver-tests]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
echo "Is master or release: ${IS_MASTER_OR_RELEASE}"
echo "Has '.Team/Drivers' or '.Team/Querying' or '.Team/SemanticLayer' label: ${HAS_TEAM_LABEL}"
if [[ "${IS_MASTER_OR_RELEASE}" == "true" ]]; then
echo "skip=false" >> $GITHUB_OUTPUT
echo "Merging into master or a release branch. Tier 3 driver tests will run."
elif [[ "${HAS_TEAM_LABEL}" == "true" ]]; then
echo "skip=false" >> $GITHUB_OUTPUT
echo "A relevant team label is present. Tier 3 driver tests will run."
else
echo "Checking for changes in relevant folders..."
CHANGED_FILES=$(git diff --name-only $(git merge-base ${{ github.event.pull_request.base.sha }} ${{ github.sha }}) ${{ github.sha }})
# Initialize an empty array to store folders with changes
CHANGED_FOLDERS=()
for folder in "src/metabase/lib" "src/metabase/query_processor" "modules/drivers" "src/metabase/driver"; do
if echo "$CHANGED_FILES" | grep -q "$folder"; then
CHANGED_FOLDERS+=("$folder")
fi
done
if [[ ${#CHANGED_FOLDERS[@]} -gt 0 ]]; then
echo "skip=false" >> $GITHUB_OUTPUT
echo "Changes detected in the following folders: ${CHANGED_FOLDERS[*]}."
echo "Tier 3 driver tests will run."
else
echo "skip=true" >> $GITHUB_OUTPUT
echo "No relevant team label, not merging into master/release, and no changes in relevant folders. Tier 3 driver tests will be skipped."
fi
fi
shell: bash
be-tests-athena-ee:
needs: determine-driver-skips
if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
env:
CI: 'true'
DRIVERS: athena
MB_ATHENA_TEST_REGION: us-east-1
MB_ATHENA_TEST_REGION_ROUTING: us-east-2
MB_ATHENA_TEST_ACCESS_KEY: ${{ secrets.MB_ATHENA_TEST_ACCESS_KEY }}
MB_ATHENA_TEST_SECRET_KEY: ${{ secrets.MB_ATHENA_TEST_SECRET_KEY }}
MB_ATHENA_TEST_S3_STAGING_DIR: ${{ secrets.MB_ATHENA_TEST_S3_STAGING_DIR }}
MB_ATHENA_TEST_S3_STAGING_DIR_ROUTING: ${{ secrets.MB_ATHENA_TEST_S3_STAGING_DIR_ROUTING }}
# These credentials are used to test the driver when the user does not have the athena:GetTableMetadata permission
MB_ATHENA_TEST_WITHOUT_GET_TABLE_METADATA_ACCESS_KEY: ${{ secrets.MB_ATHENA_TEST_WITHOUT_GET_TABLE_METADATA_ACCESS_KEY }}
MB_ATHENA_TEST_WITHOUT_GET_TABLE_METADATA_SECRET_KEY: ${{ secrets.MB_ATHENA_TEST_WITHOUT_GET_TABLE_METADATA_SECRET_KEY }}
@@ -115,7 +87,6 @@ jobs:
MB_BIGQUERY_TEST_ACCESS_TOKEN: ${{ secrets.MB_BIGQUERY_TEST_ACCESS_TOKEN }}
MB_BIGQUERY_TEST_REFRESH_TOKEN: ${{ secrets.MB_BIGQUERY_TEST_REFRESH_TOKEN }}
MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON: ${{ secrets.MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON }}
MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON_ROUTING: ${{ secrets.MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON_ROUTING }}
name: BigQuery
steps:
- uses: actions/checkout@v4
@@ -137,56 +108,8 @@ jobs:
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-clickhouse-ee:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
env:
CI: 'true'
DRIVERS: clickhouse
MB_CLICKHOUSE_TEST_HOST: localhost
MB_CLICKHOUSE_TEST_PORT: 8123
MB_CLICKHOUSE_TEST_OLD_PORT: 8124
MB_CLICKHOUSE_TEST_NGINX_PORT: 8127
name: Clickhouse
steps:
- uses: actions/checkout@v4
- name: Add ClickHouse TLS instance to /etc/hosts
run: |
sudo echo "127.0.0.1 server.clickhouseconnect.test" | sudo tee -a /etc/hosts
- name: Start ClickHouse in Docker
uses: hoverkraft-tech/compose-action@v2.0.0
with:
compose-file: "modules/drivers/clickhouse/docker-compose.yml"
down-flags: "--volumes"
services: |
clickhouse
clickhouse_tls
clickhouse_older_version
clickhouse_cluster_node1
clickhouse_cluster_node2
nginx
- name: Test Clickhouse driver
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-clickhouse-ee'
test-args: >-
:only-tags [:mb/driver-tests]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-druid-ee:
needs: determine-driver-skips
if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
env:
@@ -220,45 +143,8 @@ jobs:
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-druid-jdbc-ee:
needs: determine-driver-skips
if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
runs-on: ubuntu-22.04
timeout-minutes: 60
env:
CI: 'true'
DRIVERS: druid-jdbc
services:
druid:
image: metabase/druid:29.0.1-json-no-ingestion
ports:
- "8888:8888"
env:
CLUSTER_SIZE: nano-quickstart
name: Druid (JDBC)
steps:
- uses: actions/checkout@v4
- name: Test Druid JDBC driver
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-druid-jdbc-ee'
test-args: >-
:only-tags [:mb/driver-tests]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-databricks-ee:
needs: determine-driver-skips
if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
env:
@@ -268,7 +154,6 @@ jobs:
MB_DATABRICKS_TEST_HTTP_PATH: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HTTP_PATH }}
MB_DATABRICKS_TEST_TOKEN: ${{ secrets.MB_DATABRICKS_JDBC_TEST_TOKEN }}
MB_DATABRICKS_TEST_CATALOG: 'metabase_ci'
MB_DATABRICKS_TEST_CATALOG_ROUTING: 'metabase_routing_catalog'
MB_DATABRICKS_TEST_CLIENT_ID: ${{ secrets.MB_DATABRICKS_JDBC_TEST_CLIENT_ID }}
MB_DATABRICKS_TEST_OAUTH_SECRET: ${{ secrets.MB_DATABRICKS_JDBC_TEST_OAUTH_SECRET }}
name: Databricks
@@ -292,62 +177,27 @@ jobs:
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-databricks-multi-catalog-ee:
needs: determine-driver-skips
if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
runs-on: ubuntu-22.04
timeout-minutes: 60
env:
CI: 'true'
DRIVERS: databricks
MB_DATABRICKS_TEST_MULTI_LEVEL_SCHEMA: true
MB_DATABRICKS_TEST_HOST: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HOST }}
MB_DATABRICKS_TEST_HTTP_PATH: ${{ secrets.MB_DATABRICKS_JDBC_TEST_HTTP_PATH }}
MB_DATABRICKS_TEST_TOKEN: ${{ secrets.MB_DATABRICKS_JDBC_TEST_TOKEN }}
MB_DATABRICKS_TEST_CATALOG: 'metabase_ci'
MB_DATABRICKS_TEST_CATALOG_ROUTING: 'metabase_routing_catalog'
MB_DATABRICKS_TEST_CLIENT_ID: ${{ secrets.MB_DATABRICKS_JDBC_TEST_CLIENT_ID }}
MB_DATABRICKS_TEST_OAUTH_SECRET: ${{ secrets.MB_DATABRICKS_JDBC_TEST_OAUTH_SECRET }}
name: Databricks (Multi-Catalog)
steps:
- uses: actions/checkout@v4
- name: Test Databricks driver with multi-catalog
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-databricks-multi-catalog-ee'
test-args: >-
:only-tags [:mb/driver-tests]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-h2:
be-tests-druid-jdbc-ee:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
env:
CI: 'true'
name: H2
DRIVERS: druid-jdbc
services:
druid:
image: metabase/druid:29.0.1-json-no-ingestion
ports:
- "8888:8888"
env:
CLUSTER_SIZE: nano-quickstart
name: Druid (JDBC)
steps:
- uses: actions/checkout@v4
with:
submodules: true
token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
- name: Start python-runner
working-directory: python-runner
run: make run
- name: Test H2 driver
- name: Test Druid JDBC driver
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-h2'
junit-name: 'be-tests-druid-jdbc-ee'
test-args: >-
:only-tags [:mb/driver-tests]
- name: Upload Test Results
@@ -382,7 +232,7 @@ jobs:
enable-ssl-tests: 'false'
- name: MySQL 8.0
junit-name: be-tests-mysql-8-0-ee
image: mysql:8.0
image: cimg/mysql:8.0
env:
enable-ssl-tests: 'false'
- name: MySQL Latest
@@ -392,8 +242,28 @@ jobs:
enable-ssl-tests: 'true'
job:
- name: Driver Tests
build-static-viz: false
test-args: >-
:only-tags [:mb/driver-tests]
- name: Enterprise Tests
build-static-viz: false
test-args: >-
:only '"enterprise/backend/test"'
exclude-tag: ':mb/driver-tests'
- name: EE App DB Tests (Part 1)
build-static-viz: true
test-args: >-
:only '"test"'
:partition/total 2
:partition/index 0
exclude-tag: ':mb/driver-tests'
- name: EE App DB Tests (Part 2)
build-static-viz: true
test-args: >-
:only '"test"'
:partition/total 2
:partition/index 1
exclude-tag: ':mb/driver-tests'
services:
mysql:
@@ -448,15 +318,10 @@ jobs:
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
steps:
- uses: actions/checkout@v4
with:
submodules: true
token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
- name: Start python-runner
working-directory: python-runner
run: make run
- name: Test ${{ matrix.version.name }}
uses: ./.github/actions/test-driver
with:
build-static-viz: ${{ matrix.job.build-static-viz }}
junit-name: ${{ matrix.version.junit-name }}
test-args: >-
${{ matrix.job.test-args }}
@@ -474,17 +339,19 @@ jobs:
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-mongo:
needs: determine-driver-skips
if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
version:
- name: MongoDB 6.0
junit-name: be-tests-mongo-6-0-ee
image: metabase/qa-databases:mongo-sample-6
- name: MongoDB 4.4
junit-name: be-tests-mongo-4-4-ee
image: metabase/qa-databases:mongo-sample-4.4
- name: MongoDB 5.0
junit-name: be-tests-mongo-5-0-ee
image: metabase/qa-databases:mongo-sample-5.0
- name: MongoDB Latest
junit-name: be-tests-mongo-latest-ee
image: mongo:latest
@@ -524,17 +391,19 @@ jobs:
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-mongo-ssl:
needs: determine-driver-skips
if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
version:
- name: MongoDB 6.0
junit-name: be-tests-mongo-6-0-ssl-ee
image: metabase/qa-databases:mongo-sample-6
- name: MongoDB 4.4
junit-name: be-tests-mongo-4-4-ee
image: metabase/qa-databases:mongo-sample-4.4
- name: MongoDB 5.0
junit-name: be-tests-mongo-5-0-ssl-ee
image: metabase/qa-databases:mongo-sample-5.0
env:
CI: 'true'
DRIVERS: mongo
@@ -589,8 +458,7 @@ jobs:
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-mongo-sharded-cluster-ee:
needs: determine-driver-skips
if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
env:
@@ -623,8 +491,7 @@ jobs:
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-oracle:
needs: determine-driver-skips
if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
strategy:
@@ -714,8 +581,29 @@ jobs:
enable-ssl-tests: 'true'
job:
- name: Driver Tests
build-static-viz: false
test-args: >-
:only-tags [:mb/driver-tests]
- name: Enterprise Tests
build-static-viz: false
test-args: >-
:only '"enterprise/backend/test"'
exclude-tag: ':mb/driver-tests'
- name: EE App DB Tests (Part 1)
build-static-viz: true
test-args: >-
:only '"test"'
:exclude-tags [:mb/driver-tests]
:partition/total 2
:partition/index 0
exclude-tag: ':mb/driver-tests'
- name: EE App DB Tests (Part 2)
build-static-viz: true
test-args: >-
:only '"test"'
:partition/total 2
:partition/index 1
exclude-tag: ':mb/driver-tests'
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
env:
# Only run old migrations tests on pushes to master or release branches. All other branches should skip tests
@@ -757,15 +645,10 @@ jobs:
POSTGRES_HOST_AUTH_METHOD: trust
steps:
- uses: actions/checkout@v4
with:
submodules: true
token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
- name: Start python-runner
working-directory: python-runner
run: make run
- name: Test ${{ matrix.version.name }} (${{ matrix.job.name }})
uses: ./.github/actions/test-driver
with:
build-static-viz: ${{ matrix.job.build-static-viz }}
junit-name: 'be-tests-${{ matrix.version.junit-name }}'
test-args: >-
${{ matrix.job.test-args }}
@@ -783,8 +666,7 @@ jobs:
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-presto-jdbc-ee:
needs: determine-driver-skips
if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
env:
@@ -876,7 +758,6 @@ jobs:
DRIVERS: redshift
MB_REDSHIFT_TEST_USER: metabase_ci
MB_REDSHIFT_TEST_DB: testdb
MB_REDSHIFT_TEST_DB_ROUTING: dev
MB_REDSHIFT_TEST_HOST: ${{ secrets.MB_REDSHIFT_TEST_HOST }}
MB_REDSHIFT_TEST_PASSWORD: ${{ secrets.MB_REDSHIFT_TEST_PASSWORD }}
name: ${{ matrix.job.name }}
@@ -914,10 +795,6 @@ jobs:
MB_SNOWFLAKE_TEST_WAREHOUSE: ${{ secrets.MB_SNOWFLAKE_TEST_WAREHOUSE }}
MB_SNOWFLAKE_TEST_PK_USER: METABASE PK
MB_SNOWFLAKE_TEST_PK_PRIVATE_KEY: ${{ secrets.MB_SNOWFLAKE_TEST_PK_PRIVATE_KEY }}
MB_SNOWFLAKE_TEST_PK_PRIVATE_KEY_2: ${{ secrets.MB_SNOWFLAKE_TEST_PK_PRIVATE_KEY_2 }}
MB_SNOWFLAKE_TEST_PK_PUBLIC_KEY: ${{ secrets.MB_SNOWFLAKE_TEST_PK_PUBLIC_KEY }}
MB_SNOWFLAKE_TEST_PK_PUBLIC_KEY_2: ${{ secrets.MB_SNOWFLAKE_TEST_PK_PUBLIC_KEY_2 }}
# RSA Role testing:
MB_SNOWFLAKE_TEST_RSA_ROLE_TEST_DEFAULT_USER: RSA_ROLE_TEST_DEFAULT_USER
MB_SNOWFLAKE_TEST_RSA_ROLE_TEST_CUSTOM_USER: RSA_ROLE_TEST_CUSTOM_USER
@@ -945,8 +822,7 @@ jobs:
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-sparksql-ee:
needs: determine-driver-skips
if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
env:
@@ -979,8 +855,7 @@ jobs:
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-sqlite-ee:
needs: determine-driver-skips
if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
env:
@@ -1008,8 +883,7 @@ jobs:
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-sqlserver:
needs: determine-driver-skips
if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 30
strategy:
@@ -1058,81 +932,40 @@ jobs:
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
# TODO(dpsutton, 2025-08-20): Re-enable starburst when we figure out why it randomly dies in CI
# be-tests-starburst-ee:
# needs: determine-driver-skips
# if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
# runs-on: ubuntu-22.04
# timeout-minutes: 60
# env:
# CI: 'true'
# DRIVERS: starburst
# services:
# starburst:
# image: trinodb/trino:latest
# env:
# CATALOG_MANAGEMENT: dynamic
# ports:
# - "8080:8080"
# name: Starburst
# steps:
# - uses: actions/checkout@v4
# - name: Make plugins directory
# run: mkdir plugins
# - name: Test Starburst driver
# uses: ./.github/actions/test-driver
# with:
# junit-name: 'be-tests-starburst-ee'
# test-args: >-
# :only-tags [:mb/driver-tests]
# - name: Upload Test Results
# uses: ./.github/actions/upload-test-results
# if: always()
# with:
# input-path: ./target/junit/
# output-name: ${{ github.job }}
# bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
# aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
# aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
# aws-region: ${{ vars.AWS_REGION }}
# trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
# TODO(rileythomp, 2025-08-20): Re-enable vertica tests once the docker image is updated
# be-tests-vertica-ee:
# needs: determine-driver-skips
# if: ${{ false && !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
# runs-on: ubuntu-22.04
# timeout-minutes: 60
# env:
# CI: 'true'
# DRIVERS: vertica
# services:
# vertica:
# image: vertica/vertica-ce:12.0.2-0
# ports:
# - "5433:5433"
# name: Vertica
# steps:
# - uses: actions/checkout@v4
# - name: Make plugins directory
# run: mkdir plugins
# - name: Test Vertica driver
# uses: ./.github/actions/test-driver
# with:
# junit-name: 'be-tests-vertica-ee'
# test-args: >-
# :only-tags [:mb/driver-tests]
# - name: Upload Test Results
# uses: ./.github/actions/upload-test-results
# if: always()
# with:
# input-path: ./target/junit/
# output-name: ${{ github.job }}
# bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
# aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
# aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
# aws-region: ${{ vars.AWS_REGION }}
# trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-vertica-ee:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
env:
CI: 'true'
DRIVERS: vertica
services:
vertica:
image: vertica/vertica-ce:12.0.2-0
ports:
- "5433:5433"
name: Vertica
steps:
- uses: actions/checkout@v4
- name: Make plugins directory
run: mkdir plugins
- name: Test Vertica driver
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-vertica-ee'
test-args: >-
:only-tags [:mb/driver-tests]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
drivers-tests-result:
needs:
@@ -1155,10 +988,7 @@ jobs:
- be-tests-sparksql-ee
- be-tests-sqlite-ee
- be-tests-sqlserver
# - be-tests-starburst-ee
# TODO(rileythomp, 2025-08-20): Re-enable vertica tests once the docker image is updated
# - be-tests-vertica-ee
- be-tests-clickhouse-ee
- be-tests-vertica-ee
runs-on: ubuntu-latest
timeout-minutes: 5
name: drivers-tests-result
@@ -1179,7 +1009,7 @@ jobs:
}));
// are all jobs skipped or successful?
if (jobs.every(job => (job.result === 'skipped' || job.result === 'success' || job.name == 'be-tests-starburst-ee'))) {
if (jobs.every(job => (job.result === 'skipped' || job.result === 'success'))) {
console.log("");
console.log(" _------. ");
console.log(" / , \_ ");

View File

@@ -1,185 +0,0 @@
# Thi workflow does the backward compatibility tests
# between the Embedding SDK package of version N and Embedding SDK Bundle of version N - 1
# N - 1 is a target of a PR
name: Embedding SDK backward-compatibility E2E Component Tests
on:
workflow_call:
inputs:
cached-embedding-sdk-dist-artifact-name:
required: true
type: string
jobs:
files-changed:
name: "SDK backward-compatibility E2E > Check which files changed"
runs-on: ubuntu-22.04
timeout-minutes: 3
outputs:
e2e_embedding_sdk: ${{ steps.changes.outputs.e2e_embedding_sdk }}
steps:
- uses: actions/checkout@v4
- name: Test which files changed
uses: dorny/paths-filter@v3.0.0
id: changes
with:
token: ${{ github.token }}
filters: .github/file-paths.yaml
decide-target-ref:
runs-on: ubuntu-latest
outputs:
target-ref: ${{ steps.get-target-ref.outputs.value }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 2
- name: Decide ref to checkout
id: get-target-ref
run: |
if [[ "${{ github.event_name }}" == "pull_request" ]]; then
echo "value=${{ github.base_ref }}" >> "$GITHUB_OUTPUT"
else
echo "value=$(git rev-parse HEAD^1)" >> "$GITHUB_OUTPUT"
fi
# if this is a test on a release branch, we need to check the build requirements
get-build-requirements:
name: "SDK backward-compatibility E2E > [Backport only] Get build requirements"
needs: [decide-target-ref]
if: |
!cancelled() &&
contains(github.base_ref || github.ref, 'release-x')
runs-on: ubuntu-22.04
timeout-minutes: 10
outputs:
java_version: ${{ fromJson(steps.dependencies.outputs.result).java_version }}
node_version: ${{ fromJson(steps.dependencies.outputs.result).node_version }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ needs.decide-target-ref.outputs.target-ref }}
sparse-checkout: release
- name: Prepare build scripts
run: cd ${{ github.workspace }}/release && yarn && yarn build
- name: Get build dependencies
uses: actions/github-script@v7
id: dependencies
with:
script: | # js
const {
getBuildRequirements,
getVersionFromReleaseBranch,
} = require('${{ github.workspace }}/release/dist/index.cjs');
const targetBranchName = '${{ github.base_ref || github.ref }}';
const version = getVersionFromReleaseBranch(targetBranchName);
const requirements = getBuildRequirements(version);
return {
java_version: requirements.java,
node_version: requirements.node,
};
e2e-tests:
name: "SDK backward-compatibility E2E > Run e2e tests"
needs: [files-changed, decide-target-ref, get-build-requirements]
if: |
!cancelled() &&
needs.files-changed.outputs.e2e_embedding_sdk == 'true'
runs-on: ubuntu-22.04
timeout-minutes: 45
env:
MB_EDITION: ee
DISPLAY: ""
# Any env starting with `CYPRESS_` will be available to all Cypress tests via `Cypress.env()`
# Example: you can get `CYPRESS_FOO` with `Cypress.env("FOO")`
CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.MB_ALL_FEATURES_TOKEN }}
CYPRESS_MB_STARTER_CLOUD_TOKEN: ${{ secrets.MB_STARTER_CLOUD_TOKEN }}
CYPRESS_MB_PRO_CLOUD_TOKEN: ${{ secrets.MB_PRO_CLOUD_TOKEN }}
CYPRESS_MB_PRO_SELF_HOSTED_TOKEN: ${{ secrets.MB_PRO_SELF_HOSTED_TOKEN }}
CYPRESS_PULL_REQUEST_ID: ${{ github.event.pull_request.number || '' }}
COMMIT_INFO_MESSAGE: ${{ github.event.pull_request.title || github.event.head_commit.message || github.event.head.sha }}
TZ: US/Pacific # to make node match the instance tz
CYPRESS_CI: true
steps:
- uses: actions/checkout@v4
with:
ref: ${{ needs.decide-target-ref.outputs.target-ref }}
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: uberjar
java-version: "${{ needs.get-build-requirements.outputs.java_version || 21 }}"
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
with:
node-version: "${{ needs.get-build-requirements.outputs.node_version }}"
- name: Build uberjar with ./bin/build.sh
run: ./bin/build.sh
- name: Retrieve Embedding SDK dist artifact
uses: actions/download-artifact@v4
with:
name: ${{ inputs.cached-embedding-sdk-dist-artifact-name }}
path: ${{ github.workspace }}/resources/embedding-sdk
- name: Prepare Cypress environment
id: cypress-prep
uses: ./.github/actions/prepare-cypress
- name: Run Metabase
run: node e2e/runner/run_cypress_ci.js start
- name: Make app db snapshot
run: node e2e/runner/run_cypress_ci.js snapshot --browser ${{ steps.cypress-prep.outputs.chrome-path }}
- name: Run component tests for Embedding SDK
run: |
node e2e/runner/run_cypress_ci.js component \
--browser ${{ steps.cypress-prep.outputs.chrome-path }} \
--env grepTags="-@skip-backward-compatibility",grepOmitFiltered=true
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit
output-name: e2e-embedding-sdk
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
- name: Upload Cypress Artifacts upon failure
uses: actions/upload-artifact@v4
if: failure()
with:
name: cypress-recording-embedding-sdk-cross-version
path: |
./cypress
./logs/test.log
if-no-files-found: ignore
- name: Publish Summary
if: failure()
uses: actions/github-script@v7
with:
script: | #js
const {
generateReport,
parseReport,
formatSummary
} = require("./.github/scripts/handle-mochawesome-report.js");
const report = await generateReport();
const results = parseReport(report);
const summary = formatSummary(results);
await core.summary.addRaw(summary).write();

View File

@@ -0,0 +1,254 @@
name: E2E Cross-version Component Tests for Embedding SDK
on:
push:
branches:
- "release-**"
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
branches:
- "release-**"
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
cancel-in-progress: true
jobs:
files-changed:
name: Check which files changed
runs-on: ubuntu-22.04
timeout-minutes: 3
outputs:
e2e_all: ${{ steps.changes.outputs.e2e_all }}
steps:
- uses: actions/checkout@v4
- name: Test which files changed
uses: dorny/paths-filter@v3.0.0
id: changes
with:
token: ${{ github.token }}
filters: .github/file-paths.yaml
# if this is a test on a release branch, we need to check the build requirements
get-build-requirements:
if: |
!cancelled()
runs-on: ubuntu-22.04
timeout-minutes: 10
outputs:
java_version: ${{ fromJson(steps.dependencies.outputs.result).java_version }}
node_version: ${{ fromJson(steps.dependencies.outputs.result).node_version }}
steps:
- uses: actions/checkout@v4
with:
sparse-checkout: release
- name: Prepare build scripts
run: cd ${{ github.workspace }}/release && yarn && yarn build
- name: Get build dependencies
uses: actions/github-script@v7
id: dependencies
with:
script: | # js
const {
getBuildRequirements,
getVersionFromReleaseBranch,
} = require('${{ github.workspace }}/release/dist/index.cjs');
const targetBranchName = '${{ github.base_ref || github.ref }}';
const version = getVersionFromReleaseBranch(targetBranchName);
const requirements = getBuildRequirements(version);
return {
java_version: requirements.java,
node_version: requirements.node,
};
build:
needs: [files-changed, get-build-requirements]
if: |
!cancelled() &&
github.event.pull_request.draft == false
runs-on: ubuntu-22.04
timeout-minutes: 25
env:
MB_EDITION: ee
INTERACTIVE: false
# make sure that builds on release branches get licenses, because we use them for releases
SKIP_LICENSES: ${{ github.event_name == 'pull_request' }}
steps:
- uses: actions/checkout@v4
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
with:
node-version: "${{ needs.get-build-requirements.outputs.node_version }}"
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: uberjar
java-version: "${{ needs.get-build-requirements.outputs.java_version || 11 }}"
- name: Build uberjar with ./bin/build.sh
run: ./bin/build.sh
- name: Prepare uberjar artifact
uses: ./.github/actions/prepare-uberjar-artifact
with:
name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
resolve-sdk-version:
if: |
!cancelled()
runs-on: ubuntu-22.04
timeout-minutes: 10
outputs:
sdk_version: ${{ fromJson(steps.dependencies.outputs.result).sdk_version }}
steps:
- uses: actions/checkout@v4
with:
sparse-checkout: release
- name: Prepare build scripts
run: cd ${{ github.workspace }}/release && yarn && yarn build
- name: Get latest SDK release for this branch
uses: actions/github-script@v7
id: dependencies
with:
script: | # js
const { getSdkVersionFromReleaseBranchName } = require('${{ github.workspace }}/release/dist/index.cjs');
const branchName = '${{ github.base_ref || github.ref }}';
const sdk_version = await getSdkVersionFromReleaseBranchName({
github,
owner: context.repo.owner,
repo: context.repo.repo,
branchName,
});
return {
sdk_version,
};
e2e-tests:
needs: [get-build-requirements, build, resolve-sdk-version]
if: |
!cancelled() && needs.build.result == 'success'
runs-on: ubuntu-22.04
timeout-minutes: 45
name: e2e-tests-embedding-sdk
env:
MB_EDITION: ee
DISPLAY: ""
# Any env starting with `CYPRESS_` will be available to all Cypress tests via `Cypress.env()`
# Example: you can get `CYPRESS_FOO` with `Cypress.env("FOO")`
CYPRESS_ALL_FEATURES_TOKEN: ${{ secrets.ENTERPRISE_TOKEN }}
CYPRESS_NO_FEATURES_TOKEN: ${{ secrets.E2E_STARTER_TOKEN }}
CYPRESS_PULL_REQUEST_ID: ${{ github.event.pull_request.number || '' }}
COMMIT_INFO_MESSAGE: ${{ github.event.pull_request.title || github.event.head_commit.message || github.event.head.sha }}
TZ: US/Pacific # to make node match the instance tz
CYPRESS_CI: true
steps:
- uses: actions/checkout@v4
- name: Prepare Docker containers
uses: ./.github/actions/e2e-prepare-containers
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
# Needed to omit newly added tests for functionality that is not yet released
- name: Sparse checkout tests from SDK release commit
uses: actions/checkout@v4
with:
ref: embedding-sdk-${{ needs.resolve-sdk-version.outputs.sdk_version }}
path: sdk-release-e2e
sparse-checkout: e2e
- name: Move tests from the release commit into the repo
run: |
rm -rf ./e2e/support/*
mv ./sdk-release-e2e/e2e/support/* ./e2e/support
rm -rf ./e2e/test-component/*
mv ./sdk-release-e2e/e2e/test-component/* ./e2e/test-component
shell: bash
- name: Retrieve uberjar artifact for ee
uses: actions/download-artifact@v4
with:
name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare JDK ${{ needs.get-build-requirements.outputs.java_version || 11 }}
uses: actions/setup-java@v4
with:
java-version: ${{ needs.get-build-requirements.outputs.java_version || 11 }}
distribution: "temurin"
- name: Prepare Cypress environment
id: cypress-prep
uses: ./.github/actions/prepare-cypress
- name: Install Embedding SDK package v${{ needs.resolve-sdk-version.outputs.sdk_version }}
run: yarn add @metabase/embedding-sdk-react@${{ needs.resolve-sdk-version.outputs.sdk_version }}
- name: Run Metabase
run: node e2e/runner/run_cypress_ci.js start
- name: Make app db snapshot
run: node e2e/runner/run_cypress_ci.js snapshot --browser ${{ steps.cypress-prep.outputs.chrome-path }}
- name: Run component tests for Embedding SDK
run: node e2e/runner/run_cypress_ci.js component --browser ${{ steps.cypress-prep.outputs.chrome-path }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit
output-name: e2e-embedding-sdk
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
- name: Upload Cypress Artifacts upon failure
uses: actions/upload-artifact@v4
if: failure()
with:
name: cypress-recording-embedding-sdk-cross-version
path: |
./cypress
./logs/test.log
if-no-files-found: ignore
- name: Publish Summary
if: failure()
uses: actions/github-script@v7
with:
script: | #js
const {
generateReport,
parseReport,
formatSummary
} = require("./.github/scripts/handle-mochawesome-report.js");
const report = await generateReport();
const results = parseReport(report);
const summary = formatSummary(results);
await core.summary.addRaw(summary).write();
e2e-tests-skipped-stub:
needs: [e2e-tests]
if: |
!cancelled() &&
needs.e2e-tests.result == 'skipped'
runs-on: ubuntu-22.04
timeout-minutes: 5
name: e2e-component-tests-embedding-sdk-cross-version
steps:
- run: |
echo "Didn't run due to conditional filtering"

View File

@@ -2,18 +2,14 @@ name: E2E Component Tests for Embedding SDK
on:
workflow_call:
inputs:
cached-embedding-sdk-dist-artifact-name:
required: true
type: string
jobs:
files-changed:
name: "SDK e2e > Check which files changed"
name: Check which files changed
runs-on: ubuntu-22.04
timeout-minutes: 3
outputs:
e2e_embedding_sdk: ${{ steps.changes.outputs.e2e_embedding_sdk }}
e2e_all: ${{ steps.changes.outputs.e2e_all }}
steps:
- uses: actions/checkout@v4
- name: Test which files changed
@@ -25,7 +21,6 @@ jobs:
# if this is a test on a release branch, we need to check the build requirements
get-build-requirements:
name: "SDK e2e > [Backport only] Get build requirements"
if: |
!cancelled() &&
contains(github.base_ref || github.ref, 'release-x')
@@ -60,11 +55,10 @@ jobs:
};
build-uberjar:
name: "SDK e2e > Build uberjar"
needs: [files-changed, get-build-requirements]
if: |
!cancelled() &&
needs.files-changed.outputs.e2e_embedding_sdk == 'true'
needs.files-changed.outputs.e2e_all == 'true'
runs-on: ubuntu-22.04
timeout-minutes: 25
env:
@@ -92,31 +86,63 @@ jobs:
- name: Prepare uberjar artifact
uses: ./.github/actions/prepare-uberjar-artifact
with:
name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar-sdk-tests
name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
e2e-tests:
name: "SDK e2e > Run e2e tests (React ${{ matrix.react-version }})"
needs: [get-build-requirements, build-uberjar]
build-embedding-sdk:
needs: [files-changed, get-build-requirements]
if: |
!cancelled() &&
needs.build-uberjar.result == 'success'
needs.files-changed.outputs.e2e_all == 'true'
runs-on: ubuntu-22.04
timeout-minutes: 15
env:
MB_EDITION: ee
steps:
- uses: actions/checkout@v4
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: "cljs"
- name: Compile CLJS
run: yarn build-pure:cljs
shell: bash
- name: Build Embedding SDK package
run: yarn build-embedding-sdk
- name: Upload Embedding SDK artifact
uses: actions/upload-artifact@v4
with:
name: embedding-sdk-build
path: resources/embedding-sdk
e2e-tests:
needs: [get-build-requirements, build-uberjar, build-embedding-sdk]
if: |
!cancelled() &&
needs.build-uberjar.result == 'success' &&
needs.build-embedding-sdk.result == 'success'
runs-on: ubuntu-22.04
timeout-minutes: 45
strategy:
matrix:
react-version: [18, 19]
fail-fast: false
name: React ${{ matrix.react-version }}
env:
MB_EDITION: ee
DISPLAY: ""
# Any env starting with `CYPRESS_` will be available to all Cypress tests via `Cypress.env()`
# Example: you can get `CYPRESS_FOO` with `Cypress.env("FOO")`
CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.MB_ALL_FEATURES_TOKEN }}
CYPRESS_MB_STARTER_CLOUD_TOKEN: ${{ secrets.MB_STARTER_CLOUD_TOKEN }}
CYPRESS_MB_PRO_CLOUD_TOKEN: ${{ secrets.MB_PRO_CLOUD_TOKEN }}
CYPRESS_MB_PRO_SELF_HOSTED_TOKEN: ${{ secrets.MB_PRO_SELF_HOSTED_TOKEN }}
CYPRESS_ALL_FEATURES_TOKEN: ${{ secrets.ENTERPRISE_TOKEN }}
CYPRESS_NO_FEATURES_TOKEN: ${{ secrets.E2E_STARTER_TOKEN }}
CYPRESS_PULL_REQUEST_ID: ${{ github.event.pull_request.number || '' }}
CYPRESS_IS_EMBEDDING_SDK: "true"
CYPRESS_IS_EMBEDDING_SDK: true
COMMIT_INFO_MESSAGE: ${{ github.event.pull_request.title || github.event.head_commit.message || github.event.head.sha }}
TZ: US/Pacific # to make node match the instance tz
CYPRESS_CI: true
@@ -124,10 +150,16 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Prepare Docker containers
uses: ./.github/actions/e2e-prepare-containers
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Retrieve uberjar artifact for ee
uses: actions/download-artifact@v4
with:
name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar-sdk-tests
name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
@@ -147,11 +179,11 @@ jobs:
with:
m2-cache-key: "cljs"
- name: Retrieve Embedding SDK dist artifact
- name: Download Embedding SDK artifact
uses: actions/download-artifact@v4
with:
name: ${{ inputs.cached-embedding-sdk-dist-artifact-name }}
path: ${{ github.workspace }}/resources/embedding-sdk
name: embedding-sdk-build
path: resources/embedding-sdk
- name: Run Metabase
run: node e2e/runner/run_cypress_ci.js start
@@ -178,7 +210,7 @@ jobs:
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
- name: "SDK e2e > Upload Cypress Artifacts upon failure"
- name: Upload Cypress Artifacts upon failure
uses: actions/upload-artifact@v4
if: failure()
with:
@@ -188,7 +220,7 @@ jobs:
./logs/test.log
if-no-files-found: ignore
- name: "SDK e2e > Publish Summary"
- name: Publish Summary
if: failure()
uses: actions/github-script@v7
with:

View File

@@ -66,10 +66,8 @@ jobs:
docker run -d \
-v $PWD/my-metabase:/metabase.db \
-p 3000:3000 \
-e CYPRESS_MB_ALL_FEATURES_TOKEN=${{ secrets.MB_ALL_FEATURES_TOKEN }} \
-e CYPRESS_MB_STARTER_CLOUD_TOKEN=${{ secrets.MB_STARTER_CLOUD_TOKEN }} \
-e CYPRESS_MB_PRO_CLOUD_TOKEN=${{ secrets.MB_PRO_CLOUD_TOKEN }} \
-e CYPRESS_MB_PRO_SELF_HOSTED_TOKEN=${{ secrets.MB_PRO_SELF_HOSTED_TOKEN }} \
-e CYPRESS_ALL_FEATURES_TOKEN=${{ secrets.ENTERPRISE_TOKEN }} \
-e CYPRESS_NO_FEATURES_TOKEN=${{ secrets.E2E_STARTER_TOKEN }} \
--name metabase-${{ matrix.version.source }} \
${{ env.DOCKER_SOURCE_IMAGE }}
- name: Wait for Metabase to start on port 3000
@@ -99,10 +97,8 @@ jobs:
docker run -d \
-v $PWD/my-metabase:/metabase.db \
-p 3001:3000 \
-e CYPRESS_MB_ALL_FEATURES_TOKEN=${{ secrets.MB_ALL_FEATURES_TOKEN }} \
-e CYPRESS_MB_STARTER_CLOUD_TOKEN=${{ secrets.MB_STARTER_CLOUD_TOKEN }} \
-e CYPRESS_MB_PRO_CLOUD_TOKEN=${{ secrets.MB_PRO_CLOUD_TOKEN }} \
-e CYPRESS_MB_PRO_SELF_HOSTED_TOKEN=${{ secrets.MB_PRO_SELF_HOSTED_TOKEN }} \
-e CYPRESS_ALL_FEATURES_TOKEN=${{ secrets.ENTERPRISE_TOKEN }} \
-e CYPRESS_NO_FEATURES_TOKEN=${{ secrets.E2E_STARTER_TOKEN }} \
--name metabase-${{ matrix.version.target }} \
${{ env.DOCKER_TARGET_IMAGE }}
- name: Wait for Metabase to start on port 3001

View File

@@ -1,404 +0,0 @@
name: "[NOT REQUIRED] SDK Compatibility tests"
on:
push:
branches:
- "master"
- "release-**"
pull_request:
types: [opened, synchronize, reopened]
branches:
- "master"
- "release-**"
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
cancel-in-progress: true
jobs:
files-changed:
name: Check which files changed
runs-on: ubuntu-22.04
timeout-minutes: 3
outputs:
e2e_embedding_sdk_compatibility: ${{ steps.changes.outputs.e2e_embedding_sdk_compatibility }}
steps:
- uses: actions/checkout@v4
- name: Test which files changed
uses: dorny/paths-filter@v3.0.0
id: changes
with:
token: ${{ github.token }}
filters: .github/file-paths.yaml
# if this is a test on a release branch, we need to check the build requirements
get-build-requirements:
if: |
!cancelled() &&
contains(github.base_ref || github.ref, 'release-x')
runs-on: ubuntu-22.04
timeout-minutes: 10
outputs:
java_version: ${{ fromJson(steps.dependencies.outputs.result).java_version }}
node_version: ${{ fromJson(steps.dependencies.outputs.result).node_version }}
steps:
- uses: actions/checkout@v4
with:
sparse-checkout: release
- name: Prepare build scripts
run: cd ${{ github.workspace }}/release && yarn && yarn build
- name: Get build dependencies
uses: actions/github-script@v7
id: dependencies
with:
script: | # js
const {
getBuildRequirements,
getVersionFromReleaseBranch,
} = require('${{ github.workspace }}/release/dist/index.cjs');
const targetBranchName = '${{ github.base_ref || github.ref }}';
const version = getVersionFromReleaseBranch(targetBranchName);
const requirements = getBuildRequirements(version);
return {
java_version: requirements.java,
node_version: requirements.node,
};
get-sample-apps-data:
if: |
!cancelled()
runs-on: ubuntu-22.04
timeout-minutes: 10
env:
# we must support last two stable releases
SUPPORTED_RELEASES_COUNT: 2
TEST_SUITES: |
[
"metabase-nodejs-react-sdk-embedding-sample-e2e",
"metabase-nextjs-sdk-embedding-sample-e2e",
"shoppy-e2e"
]
TEST_SUITES_TO_RUN_IN_MASTER_ONLY: |
[
"shoppy-e2e"
]
SAMPLE_APP_MAIN_BRANCH_NAME: "main"
SAMPLE_APP_STABLE_RELEASE_BRANCH_NAME_SUFFIX: "stable"
outputs:
branch_name: ${{ fromJson(steps.build-sample-apps-data.outputs.result).branch_name }}
matrix: ${{ toJSON(fromJson(steps.build-sample-apps-data.outputs.result).matrix) }}
steps:
- uses: actions/checkout@v4
- name: Prepare build scripts
run: cd ${{ github.workspace }}/release && yarn && yarn build
- name: Build sample app compatibility data
uses: actions/github-script@v7
id: build-sample-apps-data
with:
script: | # js
const {
isReleaseBranch,
getMajorVersionNumberFromReleaseBranch,
} = require('${{ github.workspace }}/release/dist/index.cjs');
const testSuites = JSON.parse(process.env.TEST_SUITES);
const testSuitesToRunInMasterOnly = JSON.parse(process.env.TEST_SUITES_TO_RUN_IN_MASTER_ONLY);
const ref = '${{ github.base_ref || github.ref }}';
const branchName = ref.replace(/^refs\/heads\//, '');
const isMainBranchRef = branchName === 'master';
const isReleaseBranchRef = isReleaseBranch(branchName);
if (isMainBranchRef) {
return {
branch_name: '${{ env.SAMPLE_APP_MAIN_BRANCH_NAME }}',
matrix: testSuites,
};
}
if (isReleaseBranchRef) {
const supportedReleasesCount = Number(${{ env.SUPPORTED_RELEASES_COUNT }});
const currentReleaseVersion = Number(${{ vars.CURRENT_VERSION }});
const currentBranchVersion = Number(getMajorVersionNumberFromReleaseBranch(branchName));
const sampleAppBranchName = `${currentBranchVersion}-${{ env.SAMPLE_APP_STABLE_RELEASE_BRANCH_NAME_SUFFIX }}`
const shouldRunTestsForVersion = (currentReleaseVersion - currentBranchVersion) <= (supportedReleasesCount - 1);
const releaseBranchTestSuites = testSuites.filter(testSuite => !testSuitesToRunInMasterOnly.includes(testSuite));
return {
branch_name: sampleAppBranchName,
matrix: shouldRunTestsForVersion
? releaseBranchTestSuites
: []
};
}
return {
branch_name: '',
matrix: []
};
build:
needs: [files-changed, get-build-requirements, get-sample-apps-data]
if: |
!cancelled() &&
needs.files-changed.outputs.e2e_embedding_sdk_compatibility == 'true' &&
fromJSON(needs.get-sample-apps-data.outputs.matrix)[0] != null
runs-on: ubuntu-22.04
timeout-minutes: 25
env:
MB_EDITION: ee
INTERACTIVE: false
# make sure that builds on release branches get licenses, because we use them for releases
SKIP_LICENSES: ${{ github.event_name == 'pull_request' }}
steps:
- uses: actions/checkout@v4
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
with:
node-version: "${{ needs.get-build-requirements.outputs.node_version }}"
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: uberjar
java-version: "${{ needs.get-build-requirements.outputs.java_version || 21 }}"
- name: Build uberjar with ./bin/build.sh
run: ./bin/build.sh
- name: Prepare uberjar artifact
uses: ./.github/actions/prepare-uberjar-artifact
with:
name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
- name: Build Embedding SDK package
run: yarn build-embedding-sdk-package
- name: Prepare Embedding SDK artifact
uses: actions/upload-artifact@v4
with:
name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
path: resources/embedding-sdk
# In order to check compatibility between Sample Apps and Embedding SDK, the job for each sample app:
# - pulls a Sample App
# - starts it
# - runs its Cypress tests against the local metabase.jar and local @metabase/embedding-sdk-react package.
# The job is not required, though if it's failing better to be sure that it's not caused by a breaking change.
# For more details see `Sample Apps compatibility with Embedding SDK tests` in `enterprise/frontend/src/embedding-sdk-package/dev.md`
e2e-sample-apps-tests:
needs: [get-build-requirements, get-sample-apps-data, build]
strategy:
matrix:
test_suite: ${{ fromJSON(needs.get-sample-apps-data.outputs.matrix) }}
environment: ["production", "development"]
if: |
!cancelled() && needs.build.result == 'success'
runs-on: ubuntu-22.04
timeout-minutes: 45
name: e2e-sample-app-${{ matrix.test_suite }}-tests-${{ matrix.environment }}-bundle
env:
MB_EDITION: ee
DISPLAY: ""
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number || '' }}
HASH: ${{ github.event.pull_request.head.sha || github.sha }}-${{ github.run_attempt }}
CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.MB_ALL_FEATURES_TOKEN }}
CYPRESS_IS_EMBEDDING_SDK: "true"
TZ: US/Pacific # to make node match the instance tz
CYPRESS_CI: true
SAMPLE_APP_BRANCH_NAME: ${{ needs.get-sample-apps-data.outputs.branch_name }}
SAMPLE_APP_ENVIRONMENT: ${{ matrix.environment }}
permissions:
id-token: write
steps:
- uses: actions/checkout@v4
# We have to use the latest version that does not have this issue https://github.com/docker/compose/pull/12752
- name: Set up Docker Compose
uses: docker/setup-compose-action@v1
with:
version: latest
- name: Get Sample App name
id: get-sample-app-name
shell: bash
run: |
test_suite="${{ matrix.test_suite }}"
echo "sample_app_name=${test_suite%-e2e}" >> "$GITHUB_OUTPUT"
- name: Retrieve uberjar artifact for ee
uses: actions/download-artifact@v4
with:
name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
- name: Retrieve Embedding SDK dist artifact
uses: actions/download-artifact@v4
with:
name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
path: ${{ github.workspace }}/resources/embedding-sdk
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare Cypress environment
id: cypress-prep
uses: ./.github/actions/prepare-cypress
- name: Check if sample app branch exists
id: check-sample-app-branch
run: |
TEST_SUITE="${{ matrix.test_suite }}"
REPO_NAME="${TEST_SUITE%-e2e}"
REPO_URL="https://github.com/metabase/${REPO_NAME}.git"
echo "Checking if branch '${SAMPLE_APP_BRANCH_NAME}' exists in ${REPO_URL}..."
if git ls-remote --exit-code --heads "$REPO_URL" "${SAMPLE_APP_BRANCH_NAME}"; then
echo "sample_app_branch_exists=true" >> $GITHUB_OUTPUT
else
echo "sample_app_branch_exists=false" >> $GITHUB_OUTPUT
fi
- name: Configure AWS credentials
if: ${{ matrix.test_suite == 'shoppy-e2e' }}
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.SHOPPY_IAM_ROLE }}
role-session-name: GitHub_to_AWS_via_FederatedOIDC
aws-region: us-east-1
- name: Download Shoppy's App DB PostgreSQL dump from S3
if: ${{ matrix.test_suite == 'shoppy-e2e' }}
run: |
aws s3 cp s3://${{ secrets.SHOPPY_METABASE_APP_DB_S3_BUCKET }}/${{ secrets.SHOPPY_METABASE_APP_DB_S3_KEY }} ./e2e/tmp/db_dumps/shoppy_metabase_app_db_dump.sql
- name: Prepare and launch Sample Apps for shoppy-e2e test suite
if: ${{ steps.check-sample-app-branch.outputs.sample_app_branch_exists == 'true' && matrix.test_suite == 'shoppy-e2e' }}
env:
METABASE_APP_DB_USER: ${{ secrets.SHOPPY_METABASE_APP_DB_USER }}
METABASE_APP_DB: ${{ secrets.SHOPPY_METABASE_APP_DB_DATABASE }}
run: npx tsx ./e2e/runner/embedding-sdk/sample-apps/start-ci.ts shoppy-e2e
- name: Prepare and launch Sample Apps for other test suites
if: ${{ steps.check-sample-app-branch.outputs.sample_app_branch_exists == 'true' && matrix.test_suite != 'shoppy-e2e' }}
run: npx tsx ./e2e/runner/embedding-sdk/sample-apps/start-ci.ts ${{ matrix.test_suite }}
- name: Run e2e tests for Sample App
id: run-e2e-tests
if: ${{ steps.check-sample-app-branch.outputs.sample_app_branch_exists == 'true' }}
continue-on-error: true
run: node e2e/runner/run_cypress_ci.js ${{ matrix.test_suite }} --browser ${{ steps.cypress-prep.outputs.chrome-path }}
- name: Upload Cypress Artifacts upon failure
uses: actions/upload-artifact@v4
if: ${{ steps.check-sample-app-branch.outputs.sample_app_branch_exists == 'true' && steps.run-e2e-tests.outcome != 'success' }}
with:
name: cypress-recording-sample-app-${{ matrix.test_suite }}-latest
path: |
./e2e/tmp/${{ steps.get-sample-app-name.outputs.sample_app_name }}/cypress
if-no-files-found: ignore
- name: Fail job if tests failed
if: ${{ steps.check-sample-app-branch.outputs.sample_app_branch_exists == 'true' && steps.run-e2e-tests.outcome != 'success' }}
run: |
exit 1
# The job checks integration of the Embedding SDK with consumer's apps that use different frameworks/bundlers,
# or when we want to test some tricky integration cases like conflicting types.
# The job
# - starts an app
# - runs its Cypress tests against the e2e test metabase instance and local @metabase/embedding-sdk-react package.
# The job is not required, though if it's failing better to be sure that it's not caused by a breaking change.
# For more details see `Embedding SDK integration tests with Host Apps` in `enterprise/frontend/src/embedding-sdk-package/dev.md`
e2e-host-apps-tests:
needs: [get-build-requirements, build]
strategy:
matrix:
test_suite:
[
"vite-6-host-app-e2e",
"next-15-app-router-host-app-e2e",
"next-15-pages-router-host-app-e2e",
"angular-20-host-app-e2e",
]
environment: ["production", "development"]
if: |
!cancelled() && needs.build.result == 'success'
runs-on: ubuntu-22.04
timeout-minutes: 45
name: e2e-host-app-${{ matrix.test_suite }}-tests-${{ matrix.environment }}-bundle
env:
MB_EDITION: ee
DISPLAY: ""
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number || '' }}
HASH: ${{ github.event.pull_request.head.sha || github.sha }}-${{ github.run_attempt }}
CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.MB_ALL_FEATURES_TOKEN }}
CYPRESS_IS_EMBEDDING_SDK: "true"
TZ: US/Pacific # to make node match the instance tz
CYPRESS_CI: true
HOST_APP_ENVIRONMENT: ${{ matrix.environment }}
permissions:
id-token: write
steps:
- uses: actions/checkout@v4
- name: Retrieve uberjar artifact for ee
uses: actions/download-artifact@v4
with:
name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
- name: Retrieve Embedding SDK dist artifact
uses: actions/download-artifact@v4
with:
name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
path: ${{ github.workspace }}/resources/embedding-sdk
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare Cypress environment
id: cypress-prep
uses: ./.github/actions/prepare-cypress
- name: Prepare JDK 21
uses: actions/setup-java@v4
with:
java-version: 21
distribution: "temurin"
- name: Run Metabase
run: node e2e/runner/run_cypress_ci.js start
- name: Generate database snapshots
run: node e2e/runner/run_cypress_ci.js snapshot
- name: Prepare and launch Host App
run: npx tsx ./e2e/runner/embedding-sdk/host-apps/start-ci.ts ${{ matrix.test_suite }}
- name: Run e2e tests for Host App
id: run-e2e-tests
continue-on-error: true
run: node e2e/runner/run_cypress_ci.js ${{ matrix.test_suite }} --browser ${{ steps.cypress-prep.outputs.chrome-path }}
- name: Upload Cypress Artifacts upon failure
uses: actions/upload-artifact@v4
if: ${{ steps.run-e2e-tests.outcome != 'success' }}
with:
name: cypress-recording-host-app-${{ matrix.test_suite }}-${{ matrix.environment }}-latest
path: |
./cypress
if-no-files-found: ignore
- name: Fail job if tests failed
if: ${{ steps.run-e2e-tests.outcome != 'success' }}
run: |
exit 1

View File

@@ -7,17 +7,11 @@ on:
required: true
type: number
default: 30
specs:
required: false
type: string
default: "./e2e/test/scenarios/**/*.cy.spec.*"
outputs:
matrix:
value: ${{ jobs.build-matrix.outputs.matrix }}
regularChunks:
value: ${{ jobs.build-matrix.outputs.regularChunks }}
isDefaultSpecPattern:
value: ${{ jobs.build-matrix.outputs.isDefaultSpecPattern }}
jobs:
build-matrix:
@@ -26,38 +20,53 @@ jobs:
outputs:
matrix: ${{ steps.matrix.outputs.config }}
regularChunks: ${{ steps.matrix.outputs.regularChunks }}
isDefaultSpecPattern: ${{ steps.matrix.outputs.isDefaultSpecPattern }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v3
with:
sparse-checkout: |
.github
sparse-checkout-cone-mode: false
- uses: actions/setup-node@v4
with:
node-version: lts/Jod # 22.x.x
- name: Build e2e matrix
uses: actions/github-script@v7
node-version: '22.x'
- uses: actions/github-script@v7
id: matrix
env:
chunks: ${{ inputs.chunks }}
with:
script: | # js
const { buildMatrix } = require('${{ github.workspace }}/.github/scripts/build-e2e-matrix.js');
const inputSpecs = "${{ inputs.specs }}";
const inputChunks = Number(process.env.chunks);
const java = 21;
const defaultRunner = "ubuntu-22.04";
const options = {
java,
defaultRunner,
const defaultOptions = {
"java-version": java,
runner: defaultRunner,
edition: "ee",
};
const { config, regularChunks, isDefaultSpecPattern } = buildMatrix(options, inputSpecs, inputChunks);
const allSpecs = "./e2e/test/scenarios/**/*.cy.spec.*";
// see config.js function getSplittableSpecs for logic that excludes some tests from auto splitting
const specialTests = [
{ name: "embedding-sdk", specs: "./e2e/test/scenarios/embedding-sdk/**.cy.spec.*" },
{ name: "oss-subset", edition: 'oss', tags: "@OSS @smoke+-@EE", specs: allSpecs },
{ name: "mongo", tags: "@mongo", specs: allSpecs },
];
const inputChunks = Number(process.env.chunks);
const regularChunks = inputChunks - specialTests.length;
const regularTests = new Array(regularChunks).fill(1)
.map((files, index) => ({
name: `e2e-group-${index + 1}`,
}));
// regular chunks need to come before special chunks
const testSets = [...regularTests, ...specialTests];
const config = testSets.map((options) => ({
...defaultOptions,
...options,
}));
core.setOutput("regularChunks", JSON.stringify(regularChunks));
core.setOutput("config", JSON.stringify({ include: config }));
core.setOutput("isDefaultSpecPattern", JSON.stringify(isDefaultSpecPattern));
console.log({ regularChunks });
console.log("matrix config", config);
console.log("isDefaultSpecPattern", isDefaultSpecPattern);

View File

@@ -0,0 +1,274 @@
name: E2E Sample Apps tests
on:
push:
branches:
- "master"
- "release-**"
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
branches:
- "master"
- "release-**"
env:
# we must support last two stable releases
SUPPORTED_RELEASES_COUNT: 2
TEST_SUITES: |
[
"metabase-nodejs-react-sdk-embedding-sample-e2e",
"metabase-nextjs-sdk-embedding-sample-e2e"
]
TEST_SUITES_TO_RUN_IN_MASTER_ONLY: |
[
"shoppy-e2e"
]
SAMPLE_APP_MAIN_BRANCH_NAME: 'main'
SAMPLE_APP_STABLE_RELEASE_BRANCH_NAME_SUFFIX: 'stable'
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
cancel-in-progress: true
jobs:
files-changed:
name: Check which files changed
runs-on: ubuntu-22.04
timeout-minutes: 3
outputs:
e2e_all: ${{ steps.changes.outputs.e2e_all }}
steps:
- uses: actions/checkout@v4
- name: Test which files changed
uses: dorny/paths-filter@v3.0.0
id: changes
with:
token: ${{ github.token }}
filters: .github/file-paths.yaml
# if this is a test on a release branch, we need to check the build requirements
get-build-requirements:
if: |
!cancelled() &&
contains(github.base_ref || github.ref, 'release-x')
runs-on: ubuntu-22.04
timeout-minutes: 10
outputs:
java_version: ${{ fromJson(steps.dependencies.outputs.result).java_version }}
node_version: ${{ fromJson(steps.dependencies.outputs.result).node_version }}
steps:
- uses: actions/checkout@v4
with:
sparse-checkout: release
- name: Prepare build scripts
run: cd ${{ github.workspace }}/release && yarn && yarn build
- name: Get build dependencies
uses: actions/github-script@v7
id: dependencies
with:
script: | # js
const {
getBuildRequirements,
getVersionFromReleaseBranch,
} = require('${{ github.workspace }}/release/dist/index.cjs');
const targetBranchName = '${{ github.base_ref || github.ref }}';
const version = getVersionFromReleaseBranch(targetBranchName);
const requirements = getBuildRequirements(version);
return {
java_version: requirements.java,
node_version: requirements.node,
};
get-sample-apps-data:
if: |
!cancelled()
runs-on: ubuntu-22.04
timeout-minutes: 10
outputs:
branch_name: ${{ fromJson(steps.build-sample-apps-data.outputs.result).branch_name }}
matrix: ${{ toJSON(fromJson(steps.build-sample-apps-data.outputs.result).matrix) }}
steps:
- uses: actions/checkout@v4
- name: Prepare build scripts
run: cd ${{ github.workspace }}/release && yarn && yarn build
- name: Build sample app compatibility data
uses: actions/github-script@v7
id: build-sample-apps-data
with:
script: | # js
const {
isReleaseBranch,
getMajorVersionNumberFromReleaseBranch,
} = require('${{ github.workspace }}/release/dist/index.cjs');
const testSuites = JSON.parse(process.env.TEST_SUITES);
const testSuitesToRunInMasterOnly = JSON.parse(process.env.TEST_SUITES_TO_RUN_IN_MASTER_ONLY);
const ref = '${{ github.base_ref || github.ref }}';
const branchName = ref.replace(/^refs\/heads\//, '');
const isMainBranchRef = branchName === 'master';
const isReleaseBranchRef = isReleaseBranch(branchName);
if (isMainBranchRef) {
return {
branch_name: '${{ env.SAMPLE_APP_MAIN_BRANCH_NAME }}',
matrix: testSuites,
};
}
if (isReleaseBranchRef) {
const supportedReleasesCount = Number(${{ env.SUPPORTED_RELEASES_COUNT }});
const currentReleaseVersion = Number(${{ vars.CURRENT_VERSION }});
const currentBranchVersion = Number(getMajorVersionNumberFromReleaseBranch(branchName));
const sampleAppBranchName = `${currentBranchVersion}-${{ env.SAMPLE_APP_STABLE_RELEASE_BRANCH_NAME_SUFFIX }}`
const shouldRunTestsForVersion = (currentReleaseVersion - currentBranchVersion) <= (supportedReleasesCount - 1);
const releaseBranchTestSuites = testSuites.filter(testSuite => !testSuitesToRunInMasterOnly.includes(testSuite));
return {
branch_name: sampleAppBranchName,
matrix: shouldRunTestsForVersion
? releaseBranchTestSuites
: []
};
}
return {
branch_name: '',
matrix: []
};
build:
needs: [files-changed, get-build-requirements, get-sample-apps-data]
if: |
!cancelled() &&
github.event.pull_request.draft == false &&
needs.files-changed.outputs.e2e_all == 'true' &&
fromJSON(needs.get-sample-apps-data.outputs.matrix)[0] != null
runs-on: ubuntu-22.04
timeout-minutes: 25
env:
MB_EDITION: ee
INTERACTIVE: false
# make sure that builds on release branches get licenses, because we use them for releases
SKIP_LICENSES: ${{ github.event_name == 'pull_request' }}
steps:
- uses: actions/checkout@v4
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
with:
node-version: "${{ needs.get-build-requirements.outputs.node_version }}"
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: uberjar
java-version: "${{ needs.get-build-requirements.outputs.java_version || 21 }}"
- name: Build uberjar with ./bin/build.sh
run: ./bin/build.sh
- name: Prepare uberjar artifact
uses: ./.github/actions/prepare-uberjar-artifact
with:
name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
- name: Compile CLJS
run: yarn build-pure:cljs
shell: bash
- name: Build Embedding SDK package
run: yarn build-embedding-sdk
- name: Prepare Embedding SDK artifact
uses: actions/upload-artifact@v4
with:
name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
path: resources/embedding-sdk
e2e-tests:
needs: [get-build-requirements, get-sample-apps-data, build]
strategy:
matrix:
test_suite: ${{ fromJSON(needs.get-sample-apps-data.outputs.matrix) }}
if: |
!cancelled() && needs.build.result == 'success'
runs-on: ubuntu-22.04
timeout-minutes: 45
name: e2e-sample-app-${{ matrix.test_suite }}-tests
env:
MB_EDITION: ee
DISPLAY: ""
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number || '' }}
HASH: ${{ github.event.pull_request.head.sha || github.sha }}-${{ github.run_attempt }}
# Any env starting with `CYPRESS_` will be available to all Cypress tests via `Cypress.env()`
# Example: you can get `CYPRESS_FOO` with `Cypress.env("FOO")`
CYPRESS_ALL_FEATURES_TOKEN: ${{ secrets.ENTERPRISE_TOKEN }}
CYPRESS_NO_FEATURES_TOKEN: ${{ secrets.E2E_STARTER_TOKEN }}
CYPRESS_PULL_REQUEST_ID: ${{ github.event.pull_request.number || '' }}
CYPRESS_IS_EMBEDDING_SDK: true
COMMIT_INFO_MESSAGE: ${{ github.event.pull_request.title || github.event.head_commit.message || github.event.head.sha }}
TZ: US/Pacific # to make node match the instance tz
# Sample Apps testing variables
CYPRESS_CI: true
TEST_SUITE: ${{ matrix.test_suite }}
SAMPLE_APP_BRANCH_NAME: ${{ needs.get-sample-apps-data.outputs.branch_name }}
steps:
- uses: actions/checkout@v4
- name: Retrieve uberjar artifact for ee
uses: actions/download-artifact@v4
with:
name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
- name: Retrieve Embedding SDK dist artifact
uses: actions/download-artifact@v4
with:
name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
path: ${{ github.workspace }}/resources/embedding-sdk
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare Cypress environment
id: cypress-prep
uses: ./.github/actions/prepare-cypress
- name: Check if sample app branch exists
id: check-sample-app-branch
run: |
REPO_NAME="${TEST_SUITE%-e2e}"
REPO_URL="https://github.com/metabase/${REPO_NAME}.git"
echo "Checking if branch '${SAMPLE_APP_BRANCH_NAME}' exists in ${REPO_URL}..."
if git ls-remote --exit-code --heads "$REPO_URL" "${SAMPLE_APP_BRANCH_NAME}"; then
echo "sample_app_branch_exists=true" >> $GITHUB_OUTPUT
else
echo "sample_app_branch_exists=false" >> $GITHUB_OUTPUT
fi
- name: Prepare and launch Sample Apps
if: ${{ steps.check-sample-app-branch.outputs.sample_app_branch_exists == 'true' }}
run: npx tsx ./e2e/runner/start-sample-app-containers/start-ci.ts
- name: Run e2e tests for Sample App
id: run-e2e-tests
if: ${{ steps.check-sample-app-branch.outputs.sample_app_branch_exists == 'true' }}
run: node e2e/runner/run_cypress_ci.js ${{ matrix.test_suite }} --browser ${{ steps.cypress-prep.outputs.chrome-path }}
- name: Upload Cypress Artifacts upon failure
uses: actions/upload-artifact@v4
if: ${{ steps.check-sample-app-branch.outputs.branch_exists == 'true' && steps.run-e2e-tests.outcome != 'success' }}
with:
name: cypress-recording-sample-app-${{ matrix.test_suite }}-latest
path: |
./cypress
./logs/test.log
if-no-files-found: ignore

View File

@@ -30,11 +30,6 @@ on:
- none
- sql
- mongo
enable_network_throttling:
description: "Enable network throttling simulation (4g slow like in chrome dev tools)"
type: boolean
required: false
default: true
jobs:
workflow-summary:
@@ -51,7 +46,6 @@ jobs:
echo '- `burn_in`: ${{ inputs.burn_in }}' >> $GITHUB_STEP_SUMMARY
echo '- `grep`: "${{ inputs.grep }}"' >> $GITHUB_STEP_SUMMARY
echo '- `qa_db`: "${{ inputs.qa_db }}"' >> $GITHUB_STEP_SUMMARY
echo '- `enable_network_throttling`: "${{ inputs.enable_network_throttling }}"' >> $GITHUB_STEP_SUMMARY
echo '' >> $GITHUB_STEP_SUMMARY
echo 'triggered by: @${{ github.event.sender.login }}' >> $GITHUB_STEP_SUMMARY
@@ -63,14 +57,11 @@ jobs:
DISPLAY: ""
QA_DB_ENABLED: ${{ contains(fromJSON('["sql", "mongo"]'), inputs.qa_db) }}
CYPRESS_QA_DB_MONGO: ${{ inputs.qa_db == 'mongo' }}
CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.MB_ALL_FEATURES_TOKEN }}
CYPRESS_MB_STARTER_CLOUD_TOKEN: ${{ secrets.MB_STARTER_CLOUD_TOKEN }}
CYPRESS_MB_PRO_CLOUD_TOKEN: ${{ secrets.MB_PRO_CLOUD_TOKEN }}
CYPRESS_MB_PRO_SELF_HOSTED_TOKEN: ${{ secrets.MB_PRO_SELF_HOSTED_TOKEN }}
CYPRESS_ALL_FEATURES_TOKEN: ${{ secrets.ENTERPRISE_TOKEN }}
CYPRESS_NO_FEATURES_TOKEN: ${{ secrets.E2E_STARTER_TOKEN }}
MB_SNOWPLOW_AVAILABLE: true
MB_SNOWPLOW_URL: "http://localhost:9090" # Snowplow micro
MB_EDITION: ${{ inputs.mb_edition }}
CYPRESS_ENABLE_NETWORK_THROTTLING: ${{ inputs.enable_network_throttling }}
TERM: xterm
TZ: US/Pacific # to make node match the instance tz
steps:
@@ -86,7 +77,6 @@ jobs:
password: ${{ secrets.DOCKERHUB_TOKEN }}
maildev: true
openldap: true
snowplow: true
postgres: ${{ inputs.qa_db == 'sql' }}
mysql: ${{ inputs.qa_db == 'sql' }}
mongo: ${{ inputs.qa_db == 'mongo' }}
@@ -106,6 +96,8 @@ jobs:
- name: Prepare Cypress environment
id: cypress-prep
uses: ./.github/actions/prepare-cypress
- name: Run Snowplow micro
uses: ./.github/actions/run-snowplow-micro
- name: Run Metabase
run: node e2e/runner/run_cypress_ci.js start
- name: Generate database snapshots
@@ -114,12 +106,10 @@ jobs:
# For all options and fine-grained control, take a look at the documentation
# https://github.com/cypress-io/cypress/tree/develop/npm/grep
- name: Stress-test ${{ github.event.inputs.spec }} ${{ github.event.inputs.burn_in }} times
env:
GREP: ${{ github.event.inputs.grep }}
run: |
node e2e/runner/run_cypress_ci.js e2e \
--spec '${{ github.event.inputs.spec }}' \
--env burn=${{ github.event.inputs.burn_in }} \
--env burn=${{ github.event.inputs.burn_in }},grep='${{ github.event.inputs.grep }}' \
--config-file e2e/support/cypress-stress-test.config.js \
--browser ${{ steps.cypress-prep.outputs.chrome-path }}

View File

@@ -44,20 +44,23 @@ jobs:
JOB_NAME: ${{ inputs.name }}
# Any env starting with `CYPRESS_` will be available to all Cypress tests via `Cypress.env()`
# Example: you can get `CYPRESS_FOO` with `Cypress.env("FOO")`
CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.MB_ALL_FEATURES_TOKEN }}
CYPRESS_MB_STARTER_CLOUD_TOKEN: ${{ secrets.MB_STARTER_CLOUD_TOKEN }}
CYPRESS_MB_PRO_CLOUD_TOKEN: ${{ secrets.MB_PRO_CLOUD_TOKEN }}
CYPRESS_MB_PRO_SELF_HOSTED_TOKEN: ${{ secrets.MB_PRO_SELF_HOSTED_TOKEN }}
CYPRESS_ALL_FEATURES_TOKEN: ${{ secrets.ENTERPRISE_TOKEN }}
CYPRESS_NO_FEATURES_TOKEN: ${{ secrets.E2E_STARTER_TOKEN }}
CYPRESS_PULL_REQUEST_ID: ${{ github.event.pull_request.number || '' }}
COMMIT_INFO_MESSAGE: ${{ github.event.pull_request.title || github.event.head_commit.message || github.event.head.sha }}
MB_EXPERIMENTAL_SEARCH_BLOCK_ON_QUEUE: true
MB_SEARCH_ENGINE: appdb
MB_SNOWPLOW_AVAILABLE: true
MB_SNOWPLOW_URL: "http://localhost:9090" # Snowplow micro
TZ: US/Pacific # to make node match the instance tz
CYPRESS_CI: true
CYPRESS_QA_DB_MONGO: ${{ inputs.name == 'mongo' }}
CYPRESS_IS_EMBEDDING_SDK: "${{ inputs.name == 'embedding-sdk' }}"
CYPRESS_IS_EMBEDDING_SDK: ${{ inputs.name == 'embedding-sdk' }}
mongo: ${{ inputs.name == 'mongo' }}
postgres: ${{ inputs.name != 'mongo' }}
mysql: ${{ inputs.name != 'mongo' }}
openldap: true
maildev: true
webhook: true
steps:
- uses: actions/checkout@v4
@@ -67,23 +70,20 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
maildev: true
openldap: true
webhook: true
snowplow: true
postgres: ${{ inputs.name != 'mongo' }}
mysql: ${{ inputs.name != 'mongo' }}
mongo: ${{ inputs.name == 'mongo' }}
openldap: ${{ env.openldap }}
postgres: ${{ env.postgres }}
mysql: ${{ env.mysql }}
mongo: ${{ env.mongo }}
webhook: ${{ env.webhook }}
- name: Run Embedding SDK Storybook
if: inputs.name == 'embedding-sdk'
uses: ./.github/actions/run-embedding-sdk-storybook
- name: Retrieve uberjar artifact for ${{ inputs.edition }}
uses: ./.github/actions/fetch-artifact
uses: actions/download-artifact@v4
with:
name: metabase-${{ inputs.edition }}-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
extract-path: "target/uberjar"
output-name: "metabase.jar"
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
@@ -110,6 +110,9 @@ jobs:
if: github.run_attempt != '1'
run: echo "specs=$(cat failed-specs)" >> $GITHUB_ENV
- name: Run Snowplow micro
uses: ./.github/actions/run-snowplow-micro
- name: Run Metabase
run: node e2e/runner/run_cypress_ci.js start
@@ -131,7 +134,7 @@ jobs:
run: |
echo "Running tests with specs: $SPEC_PATH"
node e2e/runner/run_cypress_ci.js e2e \
--env grepTags="-@mongo+-@flaky+-@OSS+-@skip",grepOmitFiltered=true \
--env grepTags="-@mongo+-@flaky+-@OSS --@quarantine",grepOmitFiltered=true \
--spec "$SPEC_PATH" \
--browser ${{ steps.cypress-prep.outputs.chrome-path }}
@@ -143,7 +146,7 @@ jobs:
run: |
echo "Running tests with specs: $SPEC_PATH"
node e2e/runner/run_cypress_ci.js e2e \
--env grepTags="${{inputs.tags}}+-@skip",grepOmitFiltered=true \
--env grepTags="${{inputs.tags}} --@quarantine",grepOmitFiltered=true \
--spec "$SPEC_PATH" \
--browser ${{ steps.cypress-prep.outputs.chrome-path }}
@@ -153,7 +156,6 @@ jobs:
with:
input-path: ./target/junit
output-name: e2e-${{ inputs.name }}
variant: e2e-tests
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}

View File

@@ -6,10 +6,6 @@ on:
skip:
type: boolean
default: false
specs:
type: string
required: false
default: ""
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}-e2e
@@ -20,7 +16,72 @@ jobs:
uses: ./.github/workflows/e2e-matrix-builder.yml
with:
chunks: 50
specs: ${{ inputs.specs }}
# if this is a test on a release branch, we need to check the build requirements
get-build-requirements:
if: |
!cancelled() &&
contains(github.ref, 'release-x')
runs-on: ubuntu-22.04
timeout-minutes: 10
outputs:
java_version: ${{ fromJson(steps.dependencies.outputs.result).java_version }}
node_version: ${{ fromJson(steps.dependencies.outputs.result).node_version }}
steps:
- uses: actions/checkout@v4
with:
sparse-checkout: release
- name: Prepare build scripts
run: cd ${{ github.workspace }}/release && yarn && yarn build
- name: Get build dependencies
uses: actions/github-script@v7
id: dependencies
with:
script: | # js
const { getBuildRequirements, getVersionFromReleaseBranch } = require('${{ github.workspace }}/release/dist/index.cjs');
const version = getVersionFromReleaseBranch('${{ github.ref }}');
const requirements = getBuildRequirements(version);
return {
java_version: requirements.java,
node_version: requirements.node,
};
build:
needs: [get-build-requirements]
if: ${{ !cancelled() && !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 15
strategy:
matrix:
edition: [oss, ee]
env:
MB_EDITION: ${{ matrix.edition }}
INTERACTIVE: false
# make sure that builds on release branches get licenses, because we use them for releases
SKIP_LICENSES: ${{ github.event_name == 'pull_request' }}
steps:
- uses: actions/checkout@v4
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
with:
node-version: "${{ needs.get-build-requirements.outputs.node_version }}"
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: uberjar
java-version: "${{ needs.get-build-requirements.outputs.java_version || 21 }}"
- name: Build uberjar with ./bin/build.sh
timeout-minutes: 10
run: ./bin/build.sh
shell: bash
- name: Prepare uberjar artifact
uses: ./.github/actions/prepare-uberjar-artifact
with:
name: metabase-${{ matrix.edition }}-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
reset-e2e-test-comment:
if: ${{ github.event_name == 'pull_request' }}
@@ -58,8 +119,9 @@ jobs:
uses: ./.github/workflows/e2e-test.yml
needs:
- e2e-matrix-builder
- build
if: |
!cancelled() && !inputs.skip
!cancelled() && needs.build.result == 'success'
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.e2e-matrix-builder.outputs.matrix) }}
@@ -77,15 +139,14 @@ jobs:
flaky-e2e-tests:
name: Flaky e2e Tests
uses: ./.github/workflows/e2e-test.yml
needs:
- e2e-matrix-builder
# skip flaky test run if only e2e specs changed as an optimization
needs: build
if: |
!cancelled() && !inputs.skip && needs.e2e-matrix-builder.outputs.isDefaultSpecPattern == 'true'
!cancelled() && needs.build.result == 'success'
secrets: inherit
with:
name: Flaky
tags: "@flaky"
specs: "./e2e/test/scenarios/**/*.cy.spec.*"
e2e-tests-result:
needs:
@@ -113,3 +174,11 @@ jobs:
echo "E2E Tests failed"
exit 1
fi
pr-env:
needs: [build]
if: |
!cancelled() &&
contains(github.event.pull_request.labels.*.name, 'PR-Env')
uses: ./.github/workflows/pr-env.yml
secrets: inherit

Some files were not shown because too many files have changed in this diff Show More