Compare commits

..

240 Commits

Author SHA1 Message Date
Nathan Voxland
2935aefc64 only include tables you can use for native query builder 2025-08-12 23:09:01 -05:00
Nathan Voxland
0ecddc9870 Merge remote-tracking branch 'origin/feature/reports' into document-sql 2025-08-12 19:48:05 -05:00
Aleksandr Lesnenko
7ebb0f0000 fix pivot query editing in documents 2025-08-12 20:56:35 -03:00
Denis Berezin
25ce9c5dcf Reports fix fe ci (#62046)
* Fix some CI issues

* Fix some CI issues

* Fix typecheck issues

* Fix lint

* Fix unit tests

* Fix e2e tests
2025-08-13 02:48:09 +03:00
Edward Paget
14a343de12 feat: check for name length 2025-08-12 16:06:07 -05:00
Brad Anderson
ec0c2dba8b Reverts probably-accidental change 2025-08-12 17:02:25 -04:00
Brad Anderson
39afe7fff5 UXW-451 Adds some light dataset error handling 2025-08-12 16:42:00 -04:00
Edward Paget
ab958737a9 fix: remove extraneous precondition migration 2025-08-12 15:33:11 -05:00
Brad Anderson
45a0c85158 UXW-472 Adds ArchivedEntityBanner for documents 2025-08-12 15:49:51 -04:00
Nick Fitzpatrick
d2479a5b95 bury shame 2025-08-12 15:48:11 -04:00
Nick Fitzpatrick
6d13f02bf3 Move, Archive, and Bookmark in documents page 2025-08-12 15:46:27 -04:00
Brad Anderson
3b8161ca75 Fix issue where "Delete permanently" errors out instead of deleting 2025-08-12 15:05:20 -04:00
Brad Anderson
93047a3e2f Updates Document type with newly added properties 2025-08-12 15:04:14 -04:00
Edward Paget
2b902a4ce1 feat: return can_restore, can_delete for docs 2025-08-12 13:43:04 -05:00
eliotmetabase
e03e528f09 Update menu items (#62072)
* Update commands and remove descriptions

* Removes unused code

---------

Co-authored-by: Brad Anderson <brad@metabase.com>
2025-08-12 11:09:29 -07:00
Brad Anderson
6f74d26021 Fix issue where metabot nodes sometimes yield a false positive in hasUnsavedChanges because they updated their own attributes 2025-08-12 13:56:01 -04:00
Nick Fitzpatrick
cb9d5e33e2 more e2e tests 2025-08-12 13:36:11 -04:00
Brad Anderson
1ddff04bf6 Changes how mention text is referenced in the documents metabot prompt serializer. It uses a cache in the documents slice now. It was using a node attribute before but that was hacky and it messed with the "is dirty" calculation. 2025-08-12 13:04:09 -04:00
Edward Paget
640b530563 fix: casing in tests 2025-08-12 11:16:12 -05:00
Nathan Voxland
952a52974c Able to generate sql charts 2025-08-12 11:05:44 -05:00
Edward Paget
915abf29eb fix: token features in recent views test 2025-08-12 10:50:22 -05:00
Edward Paget
67eceb653a fix: whitespace exclusion path 2025-08-12 10:17:59 -05:00
Edward Paget
11260e4d72 fix: whitespace and round-trip serdes test 2025-08-12 10:15:33 -05:00
Edward Paget
11d3fb22d2 Merge branch 'master' into feature/reports 2025-08-12 09:55:58 -05:00
Edward Paget
2627574bce feat: add documents to recents 2025-08-12 09:51:56 -05:00
Brad Anderson
53fcff39ef Updates "Created by Metabot" line based on design/user feedback 2025-08-12 10:38:55 -04:00
Aleksandr Lesnenko
8783c54480 improve state management UXW-463 UXW-450 2025-08-11 22:12:02 -03:00
Brad Anderson
9f8331cb63 UXW-462 Fix issue where you can write markdown in metabot blocks 2025-08-11 18:37:46 -04:00
Aleksandr Lesnenko
65bbe6429e feedback on the command 2025-08-11 19:21:19 -03:00
Aleksandr Lesnenko
7b367b6c0f refactor 2025-08-11 19:21:19 -03:00
Brad Anderson
c328d7f7e0 UXW-462 Fix issue where sometimes custom tiptap node tries to re-render too quickly resulting in an error 2025-08-11 18:09:44 -04:00
Brad Anderson
08cec1d1f5 UXW-462 Adds database picker in metabot block 2025-08-11 18:00:31 -04:00
Nick Fitzpatrick
5c489b9c95 Markdown Loki Test 2025-08-11 17:44:02 -04:00
Aleksandr Lesnenko
4a079f203a bump padding 2025-08-11 18:20:52 -03:00
Aleksandr Lesnenko
1fc9390906 updated styles 2025-08-11 18:15:18 -03:00
Edward Paget
3164d5d866 fix: kondo warning on unused var 2025-08-11 16:13:22 -05:00
Edward Paget
71d915b7d0 feat: support serdes on documents (#62032) 2025-08-11 16:04:13 -05:00
Edward Paget
bbe95ee859 fixup! pinning reconcilation with documents 2025-08-11 15:26:37 -05:00
Edward Paget
6542c95758 feat: support pinning documents 2025-08-11 14:51:17 -05:00
Aleksandr Lesnenko
88ef82938d fix command 2025-08-11 13:23:46 -03:00
Brad Anderson
9477b72420 Fix issue where charts sometimes don't get added via metabot when inserted in the middle of a document 2025-08-11 11:09:51 -04:00
Aleksandr Lesnenko
38bd513835 fix command search 2025-08-09 15:25:14 -03:00
Aleksandr Lesnenko
e4c92ff6e2 placeholder 2025-08-08 18:46:12 -03:00
Aleksandr Lesnenko
4cbb2297a5 mention + command feedback 2025-08-08 18:30:14 -03:00
Edward Paget
54ebe17696 fix: handle inadvertent documents in oss tests 2025-08-08 16:18:44 -05:00
Edward Paget
41674dd1de fix: final in-place search test fixes 2025-08-08 15:12:03 -05:00
Edward Paget
9595d9493d fix: more in-place search fixes for documents 2025-08-08 14:47:04 -05:00
Edward Paget
3c3a8dd127 fix: set type to :question dashboard_id to nil 2025-08-08 14:31:46 -05:00
Edward Paget
4373216910 fix: migration order 2025-08-08 14:25:58 -05:00
Edward Paget
efbfc502f1 fix: in_place search fixes and test fixes 2025-08-08 14:17:42 -05:00
Edward Paget
9bd2243573 fix: emit documentment read on api get 2025-08-08 13:21:52 -05:00
Edward Paget
9c2838341e feat: documents in search
Also supports view_counts and last_viewed_at for documents
2025-08-08 13:21:51 -05:00
Edward Paget
a60e02dd1e fix: remove documents token hack 2025-08-08 13:21:51 -05:00
Edward Paget
92e9d6360d fix: set null on collection cascade for documents
mirrors behavior for dashboards and cards
2025-08-08 13:21:51 -05:00
Nick Fitzpatrick
c6f35e0393 initial e2e tests 2025-08-08 14:58:13 -03:00
Aleksandr Lesnenko
89b4bbe920 fix metadata fetching 2025-08-08 14:29:51 -03:00
Aleksandr Lesnenko
0ab1d4ccb6 no emojis and markdown downloading 2025-08-08 12:44:25 -03:00
Aleksandr Lesnenko
ea64672746 drop emojis 2025-08-08 12:44:25 -03:00
Brad Anderson
b9a446a933 Localizes "Created with Metabot" text 2025-08-08 10:21:15 -04:00
Brad Anderson
7bd57d1d66 MetabotEmbed polish:
* Prevents prompt from highlighting when you hit "Run" (potentially bringing up the editor bubble menu).
* Prevents useEffect hook from running way more than it needs to.
* Adds margin below MetabotEmbed.
* Hitting escape with nothing in the textarea will now remove the MetabotEmbed block.
* Makes layout less janky when Metabot responds with a long error message.
2025-08-08 09:55:47 -04:00
Aleksandr Lesnenko
dfeafc9489 do not show bubble menu on disallowed nodes 2025-08-08 01:42:49 -03:00
Aleksandr Lesnenko
9f0c6d1e2e fix replace question 2025-08-08 01:40:50 -03:00
Aleksandr Lesnenko
e716d1d253 clean 2025-08-08 01:37:28 -03:00
Aleksandr Lesnenko
f7c8221468 improved mention and command 2025-08-07 22:10:42 -03:00
Brad Anderson
4135955236 UXW-461 Save button always shows up for new documents (and user has necessary permissions) 2025-08-07 16:24:52 -04:00
Brad Anderson
b2a0c144ba UXW-461 Prevents jump when Save button appears/disappears. Also adds subtle fade-in transition when it appears. 2025-08-07 16:13:11 -04:00
Brad Anderson
9a56a9e96b UXW-459 Fix issues with LeaveRouteConfirmModal and new documents 2025-08-07 15:25:57 -04:00
Nick Fitzpatrick
436840eac9 hide object detail row in document tables 2025-08-07 16:18:07 -03:00
Brad Anderson
ab0bd1b8ff UXW-459 Fix issue where creating a new document when starting from another document overwrites the existing document (and shows previous document's metadata in new document) 2025-08-07 14:27:13 -04:00
Nick Fitzpatrick
c800fb73d0 Handle URL slugs, remove version code from document page, handle 404 and 403 2025-08-07 14:43:28 -03:00
Brad Anderson
0463d74f76 UXW-453 Fix 404 when clicking draft card title 2025-08-07 13:36:34 -04:00
Nick Fitzpatrick
f8924beba5 add title to untitled docs 2025-08-07 13:54:45 -03:00
Edward Paget
92eb7e62a1 fix: add entity id column to documents 2025-08-07 11:27:27 -05:00
Brad Anderson
6351f1c4ef Replaces hard-coded metabot-generated card title with title from the /generate-content response (or translates a default title if none provided) 2025-08-07 12:25:19 -04:00
Edward Paget
eb8f2d91d0 fix: foreign key indexes 2025-08-07 10:34:07 -05:00
Aleksandr Lesnenko
8c5adc5f85 fixed drills & query editing for metabot cards 2025-08-07 10:37:14 -04:00
Brad Anderson
a792b1706e Updates MetabotEmbed to use most up-to-date api contract 2025-08-07 10:37:14 -04:00
Brad Anderson
268352d02b Stubs out the necessary updates to make MetabotEmbed work with an UnsavedCard from the BE rather than a full Card 2025-08-07 10:37:14 -04:00
Brad Anderson
acafad1b6a Updates MetabotEmbed to use most up-to-date api contract 2025-08-07 10:37:14 -04:00
Brad Anderson
4f95b0ebb7 Updates MetabotEmbed "stop generating" functionality to work with regular (non-streaming) api call 2025-08-07 10:37:14 -04:00
Brad Anderson
92017cfe7c Updates MetabotEmbed to always use draft cards 2025-08-07 10:37:14 -04:00
Brad Anderson
9ee59951ae Updates MetabotEmbed to work with upcoming BE changes 2025-08-07 10:37:14 -04:00
Nick Fitzpatrick
403bf4eafd Show tooltip with precise time in document metadata 2025-08-07 11:12:24 -03:00
Nick Fitzpatrick
010af9dc10 undo moves should work once archived is boolean 2025-08-07 11:12:24 -03:00
Edward Paget
8c1e179da7 fix: bad var reference in tests 2025-08-07 08:36:37 -05:00
Aleksandr Lesnenko
40368f85fb l10n 2025-08-06 22:29:48 -03:00
Aleksandr Lesnenko
978e12c2d6 swap icon 2025-08-06 22:26:38 -03:00
Aleksandr Lesnenko
ea4c347006 UXW-416: replace question should replace only 1 question 2025-08-06 19:57:52 -03:00
Aleksandr Lesnenko
5556654506 UXW-400: formatting floating menu 2025-08-06 19:19:34 -03:00
Nathan Voxland
4ea6c16f19 Merge remote-tracking branch 'origin/feature/reports' into feature/reports 2025-08-06 16:48:59 -05:00
Nathan Voxland
6620040235 New metabot documents API 2025-08-06 16:48:47 -05:00
Edward Paget
f809d7747c fix: module sort 2025-08-06 16:36:22 -05:00
Edward Paget
181e8f31d2 fix: tests and kondos 2025-08-06 16:21:47 -05:00
Edward Paget
83bd4ba534 Merge branch 'master' into feature/reports 2025-08-06 15:50:29 -05:00
Edward Paget
f770d21cdf fix: for OSS tests 2025-08-06 15:48:48 -05:00
Edward Paget
f6f6b2d3a9 fix: ordering of migrations for mysql 2025-08-06 15:36:19 -05:00
Edward Paget
745ab32bde fix: tests and lints 2025-08-06 15:33:44 -05:00
Nick Fitzpatrick
6760658aec App Bar Breadcrumbs for documents 2025-08-06 17:18:47 -03:00
Edward Paget
bad6bccac2 fix: require non-blank string for name 2025-08-06 15:03:50 -05:00
Edward Paget
7448009633 feat: add document delete endpoint 2025-08-06 14:59:33 -05:00
Edward Paget
7418a137c8 fix: always add documents to bookmark lookup 2025-08-06 14:42:37 -05:00
Edward Paget
40269ca9c0 fix: add perms info to documents in collections 2025-08-06 14:41:31 -05:00
Edward Paget
42bee66fba fix: hide documents in collection items if not EE 2025-08-06 14:05:14 -05:00
Edward Paget
76ee0b6f8c feat: allow documents to be bookmarked 2025-08-06 13:52:35 -05:00
Aleksandr Lesnenko
912aca8811 fix UXW-412 again 2025-08-06 15:13:33 -03:00
Brad Anderson
2e9f07ede2 Fix issue where submitting empty metabot prompt (by hitting "Run" or cmd+enter) makes it spin forever 2025-08-06 14:00:11 -04:00
Nick Fitzpatrick
9568b9956d Document Collection Item Caching 2025-08-06 14:23:13 -03:00
Edward Paget
80867e1335 feat: allow documents to be archived 2025-08-06 11:49:50 -05:00
Brad Anderson
7894f1e45e Fix issue where temporary implementation of metabot block can't create questions 2025-08-06 11:20:10 -04:00
Nick Fitzpatrick
fd4a621603 document entity 2025-08-06 12:15:28 -03:00
Edward Paget
0486d547d3 feat: include revision info in collection tree res 2025-08-06 09:08:37 -05:00
Edward Paget
ec7163091a feat: revision history for documents 2025-08-06 08:57:32 -05:00
Aleksandr Lesnenko
287e496a1b wip but working sql query editor in documents 2025-08-05 23:28:00 -03:00
Aleksandr Lesnenko
2ac44861b5 Persist cards when saving document (#61818)
* chore: remove document version model

* persist cards only when saving documents

* chore: remove in_document type

* feat: return document creator

* feat: save cards with parameters pass in as cards

in POST and PUT requests to documents

* chore: exclude doc cards from where shouldn't be

search, recents, collection items

* chore: fixup search and card tests

* contract

* feat: update document ast with new card ids

* fix: report -> document in collection api

* fix: ws

* feat: clone cards not associated with document

* chore: add tests for document permissions

* fix contract

* fetch root collection

* update message

* must be working now

* fix sidebar

---------

Co-authored-by: Edward Paget <ed.paget@gmail.com>
2025-08-05 19:59:20 -03:00
Denis Berezin
47d994182c Merge branch 'master' into feature/reports
# Conflicts:
#	frontend/src/metabase/redux/app.ts
2025-08-06 00:18:15 +03:00
Denis Berezin
17435642ef Fix some CI issues 2025-08-06 00:03:50 +03:00
Denis Berezin
a917ade608 Merge branch 'master' into feature/reports
# Conflicts:
#	resources/migrations/056_update_migrations.yaml
#	test/metabase/test/util/namespace.clj
2025-08-05 22:30:13 +03:00
Nick Fitzpatrick
1799892d0b Document Metadata 2025-08-05 14:45:26 -03:00
Brad Anderson
4f7c22ed53 Adds subtle animation to "Working on it..." message 2025-08-04 17:34:07 -04:00
Brad Anderson
ba806688b6 Fix issues with multiple in-flight requests 2025-08-04 17:17:48 -04:00
Brad Anderson
18898a5fdf Tweaks metabot block styling, wires up "stop generating" functionality 2025-08-04 17:00:10 -04:00
Brad Anderson
1ebe336a00 UXW-404 Improves image formatting 2025-08-04 13:05:36 -04:00
Aleksandr Lesnenko
36790affcf auto focus on title on new documents 2025-08-04 13:08:36 -03:00
Aleksandr Lesnenko
a15154cb84 fix layout 2025-08-04 11:51:08 -03:00
Aleksandr Lesnenko
095cae88af fix links to tables 2025-08-01 22:20:08 -03:00
Aleksandr Lesnenko
5314f9f976 missing import 2025-08-01 22:10:37 -03:00
Aleksandr Lesnenko
7189344bfd paste smartlinks 2025-08-01 22:01:48 -03:00
Aleksandr Lesnenko
11b74e0e4b split mention vs command, allow mentioning other entities 2025-08-01 21:29:55 -03:00
Aleksandr Lesnenko
c5862c278d mention all entities 2025-08-01 20:47:30 -03:00
Aleksandr Lesnenko
0af4b1c0de uxw-366: Navigation back to the report from a drill through can be difficult - can we add the back button like for dashboards 2025-08-01 20:18:35 -03:00
Aleksandr Lesnenko
4cd96fa6a2 fix embedding dashboard-only cards to documents 2025-08-01 19:30:19 -03:00
Aleksandr Lesnenko
b47ccb6123 fix FE usage of documents api 2025-08-01 18:43:27 -03:00
Edward Paget
5271f4092d chore: rename everything to document (#61743)
strip out snapshotting code
2025-08-01 15:04:32 -05:00
Aleksandr Lesnenko
c06ea7d391 do not leak prosemirror styles into charts 2025-08-01 14:17:20 -03:00
Edward Paget
e2e2e5fdf3 keep report cards and collections in sync (#61686) 2025-08-01 10:04:50 -05:00
Ryan Laurie
329b903a8b update metabot question creation parsing 2025-08-01 07:14:18 -06:00
Aleksandr Lesnenko
82acc79f82 fe: reports -> documents 2025-07-31 21:58:28 -03:00
Aleksandr Lesnenko
240862340a icons 2025-07-31 21:00:14 -03:00
Nick Fitzpatrick
7849e0394c cleaning up emoji ui a bit 2025-07-31 19:10:19 -03:00
Nick Fitzpatrick
8fa264cb17 gross emojis 2025-07-31 18:08:02 -03:00
Aleksandr Lesnenko
ca4744dc08 why scroll to bottom when opened 2025-07-31 17:51:25 -03:00
Aleksandr Lesnenko
78e9dda8dd jumping layout when loading 2025-07-31 17:51:25 -03:00
Ryan Laurie
b36f00cf80 stop header bouncing 2025-07-31 14:42:55 -06:00
Nick Fitzpatrick
3f24ba5687 disable metabot sidebar on mob+b 2025-07-31 17:40:18 -03:00
Aleksandr Lesnenko
d7dc44feb2 always clone non in_report cards 2025-07-31 17:34:38 -03:00
Aleksandr Lesnenko
0450ba6a67 always create a card clone when embedding 2025-07-31 17:34:38 -03:00
Aleksandr Lesnenko
b30ecea5ca cleanup 2025-07-31 17:34:38 -03:00
Nick Fitzpatrick
fc8ca9b602 Print Report 2025-07-31 17:01:10 -03:00
Luiz Arakaki
68366d7d37 remove trigger for uberjar 2025-07-31 15:20:36 -03:00
Luiz Arakaki
63a94c8788 run build and docker uberjar on push 2025-07-31 14:47:46 -03:00
Ryan Laurie
10bd9d68ce Insert into reports from metabot (#61642)
* insert from metabot

* Metabot plugin and hook

* make stuff nicer

* cleanup

* no more snapshots
2025-07-31 11:19:18 -06:00
Aleksandr Lesnenko
7221915503 polish 2025-07-31 11:59:02 -03:00
Aleksandr Lesnenko
4b57f82089 saving new report should not show LeaveRouteConfirmModal 2025-07-30 20:19:56 -03:00
Aleksandr Lesnenko
b8d2ac9ebc fix backend not starting 2025-07-30 20:00:53 -03:00
Aleksandr Lesnenko
00e3499af8 remove replace question, drop refresh question 2025-07-30 19:29:04 -03:00
Aleksandr Lesnenko
d81a470a57 drop flushSync error 2025-07-30 19:28:58 -03:00
Aleksandr Lesnenko
c329272406 send used card ids 2025-07-30 19:28:49 -03:00
Aleksandr Lesnenko
75ec895750 less redux 2025-07-30 19:28:42 -03:00
Aleksandr Lesnenko
60ca78c873 remove snapshots 2025-07-30 19:28:35 -03:00
Edward Paget
45ee83044e chore: add enterprise card test 2025-07-30 08:37:43 -05:00
Aleksandr Lesnenko
c43a1c5ca5 switch to prosemirror ast 2025-07-29 23:02:07 -03:00
Aleksandr Lesnenko
a7930e7395 better markdown serde 2025-07-29 22:16:50 -03:00
Edward Paget
ad58a25fc6 feat: associate cards with reports in create 2025-07-29 16:30:49 -05:00
Edward Paget
7c5f3f11db feat: associate in_report cards w/ report_document 2025-07-29 15:04:47 -05:00
Denis Berezin
cff5e706f9 Ditch overall sidebar for now (#61523)
* Remove list of cards sidebar

* Some type fixes

* Review fixes
2025-07-29 01:24:14 +03:00
Nick Fitzpatrick
f889695914 remove references to static questions 2025-07-28 13:28:00 -03:00
Aleksandr Lesnenko
db61438bd8 cleanup 2025-07-28 03:10:45 -03:00
Ryan Laurie
6191275795 more leave confirmation 2025-07-25 15:24:18 -06:00
Ryan Laurie
319cf701dc allow saving in personal collections and our analytics 2025-07-25 15:10:29 -06:00
Brad Anderson
0995bf3526 Fixes issue where dropping images into reports doesn't work because the events are getting swallowed by react-dnd 2025-07-25 13:01:25 -04:00
Aleksandr Lesnenko
00d8e50082 fix lists 2025-07-25 11:39:47 -04:00
Nick Fitzpatrick
606416ba23 Fixing UX jank with Code Jank 2025-07-25 11:42:09 -03:00
John Swanson
86a55c97d9 fix startup more 2025-07-25 05:40:18 -07:00
John Swanson
850afa8366 fix startup
@nvoxland i *think* this was a stray and not necessary for your metabot
changes but we can add it back along with `dev/src/api/routes.clj` if
i'm wrong!
2025-07-25 05:26:58 -07:00
Nick Fitzpatrick
0f1eb823d3 adding column blocks and colmns 2025-07-25 09:05:25 -03:00
Nathan Voxland
64448e927d Added ability to ask metabot about the report you are on 2025-07-24 23:23:27 -05:00
Aleksandr Lesnenko
f2a3859e32 fix editing viz settings when multiple copied entries 2025-07-24 22:34:08 -04:00
Aleksandr Lesnenko
2c85824963 saved images should render as images 2025-07-24 20:36:42 -04:00
Aleksandr Lesnenko
27420b5cb9 fix excessive space 2025-07-24 20:33:37 -04:00
Aleksandr Lesnenko
c3e67fd73e can write 2025-07-24 20:14:06 -04:00
Aleksandr Lesnenko
18ab9af893 hack in_report card type to return just question 2025-07-24 18:18:36 -04:00
Ryan Laurie
d3469efb09 frontend readonly support 2025-07-24 15:58:12 -06:00
Brad Anderson
e00d09cc37 Adds ability to add images via markdown to reports 2025-07-24 17:00:47 -04:00
Ryan Laurie
ef602ae2d0 show and save reports in collections 2025-07-24 14:52:38 -06:00
Aleksandr Lesnenko
65c5af5155 refresh snapshot 2025-07-24 16:34:58 -04:00
Aleksandr Lesnenko
eb0e823d1e title click opens question 2025-07-24 16:28:17 -04:00
Aleksandr Lesnenko
c6eeb8e6ba drill-through 2025-07-24 16:06:52 -04:00
John Swanson
af496e167d check perms on snapshot read/write
to read a snapshot, you need:

- read perms on the collection its report is in, if it has a report
attached, OR

- to be the person who created the snapshot in the first place.

to write a snapshot, you need to be a superuser.
2025-07-24 12:16:36 -07:00
John Swanson
48199ff2f7 add reports to collection items
we'll need to update the frontend to request `report` models, and then
handle them correctly there (with a special icon and links). For now
this won't have any effect, since the FE is selecting specific models
and that doesn't include reports.
2025-07-24 12:16:36 -07:00
Brad Anderson
83ca395ccf Allows links to be pasted into reports 2025-07-24 15:08:45 -04:00
Ryan Laurie
bd83d0691b keyboard shortcut for saving 2025-07-24 13:02:05 -06:00
Ryan Laurie
5b2fa90fdb fix new report navigation 2025-07-24 12:19:43 -06:00
Nick Fitzpatrick
436341ab76 Viz Settings for Static Questions 2025-07-24 14:11:43 -03:00
John Swanson
aabc1d165f Enforce permissions on reports
Add collection_id to reports, optionally allow setting it in API
endpoints (POST and PUT), enforce perms as follows:

- POST and PUT require superuser

- GET endpoints require read perms on the container collection
2025-07-24 09:44:57 -07:00
Edward Paget
a4747ed263 Add snapshotted timestamp 2025-07-24 11:35:21 -05:00
Ryan Laurie
cb5f72cad4 prevent losing unsaved changes 2025-07-24 10:28:54 -06:00
Brad Anderson
6c0b2d81ad UXW-9 Adds smart links. Changes "insert chart" mechanism from "@" to "starting with /" so @'s can be used for aforementioned smart links. 2025-07-24 12:26:13 -04:00
Edward Paget
e0f1ecee2d exclude cards from recent views 2025-07-24 10:11:43 -05:00
Edward Paget
6b8124421a Add in_report card type;
make sure snapshotted cards have it set, exclude it from the collection
view and entity picker.

update the report editor to use the returned card id from the snapshot endpoint
2025-07-24 09:50:37 -05:00
Aleksandr Lesnenko
df462d07d5 ensure question selected 2025-07-23 22:29:19 -04:00
Aleksandr Lesnenko
4f908179ec there has to be something to save to show the save button 2025-07-23 22:27:01 -04:00
Aleksandr Lesnenko
9cbdac4de4 deselect question after embedding so you can type further 2025-07-23 22:19:12 -04:00
Aleksandr Lesnenko
0b4b4dfced padding is too huge 2025-07-23 22:13:19 -04:00
Aleksandr Lesnenko
19f4980a7e fix-hack magical new lines 2025-07-23 22:10:10 -04:00
Aleksandr Lesnenko
38e9f21dfb debloat report page, fix used content sidebar 2025-07-23 21:51:46 -04:00
Aleksandr Lesnenko
2775a7e069 sidebar button 2025-07-23 21:33:24 -04:00
Aleksandr Lesnenko
d92773df06 use snapshots, fix a few bugs, change viz settings sidebar behavior 2025-07-23 21:16:01 -04:00
Ryan Laurie
023864e72f correct ci skip label check (#61370) 2025-07-23 16:52:43 -06:00
Ryan Laurie
9c479154a5 versions frontend 2025-07-23 15:58:42 -06:00
Nathan Voxland
747781ee29 Renamed duplicate constraints in migrations 2025-07-23 16:44:53 -05:00
Nathan Voxland
31503487b3 Merge remote-tracking branch 'origin/feature/reports' into feature/reports 2025-07-23 16:09:37 -05:00
Nathan Voxland
9c7f08c2ae Added ?version argument to GET report-id 2025-07-23 16:09:24 -05:00
Nick Fitzpatrick
8693547be1 Static Quesiton Node 2025-07-23 17:57:43 -03:00
Edward Paget
29786d318c Replace runs with snapshoting endpoints 2025-07-23 15:33:05 -05:00
Nathan Voxland
9307c56d7b Added versions and reports API 2025-07-23 12:12:01 -05:00
Edward Paget
2583528cb3 Convert get run endpoints to stream 2025-07-23 09:53:21 -05:00
Edward Paget
33edd890a3 Working save and fetch of run data 2025-07-23 09:13:08 -05:00
Edward Paget
1e8dbe446b Towards working save of runs 2025-07-23 09:13:08 -05:00
Ryan Laurie
ce0e349f21 picard_facepalm 2025-07-22 16:59:54 -06:00
Ryan Laurie
fc5a6469ba saving fixes 2025-07-22 16:42:51 -06:00
Aleksandr Lesnenko
8523bf0412 ui 2025-07-22 18:11:00 -04:00
Aleksandr Lesnenko
13d11d33cd used content 2025-07-22 18:04:05 -04:00
Aleksandr Lesnenko
bb15f10b44 select chart type 2025-07-22 17:59:07 -04:00
Aleksandr Lesnenko
f9241dd461 show viz settings sidebar for embedded questions (#61341)
* show viz settings sidebar for embedded questions

* some redux

* ensure sidebar open

* settings sidebar edits charts viz settings
2025-07-22 17:27:28 -04:00
Nathan Voxland
a4a135903b Added report crud operations 2025-07-22 16:25:19 -05:00
Brad Anderson
67d31cea38 UXW-20 Adds “Browse all” option to SearchResults and RecentsList dropdowns 2025-07-22 17:20:07 -04:00
Ryan Laurie
b7e77ac43c add reports API Frontend 2025-07-22 13:41:57 -06:00
Kyle Doherty
89fd0b5c2b text fix 2025-07-22 14:58:17 -04:00
Kyle Doherty
5339aaa1ca layout and styling tweaks 2025-07-22 14:58:17 -04:00
Edward Paget
809896ee0a fixup! names on the migrations 2025-07-22 13:26:56 -05:00
Edward Paget
aae3788116 fix: remove namespace test const 2025-07-22 13:07:55 -05:00
Edward Paget
d743daaf20 feat: add migration and models for reports (#61300)
* feat: add migration and models for reports

* chore: add test for pivot card query caching

* Change column name markdown_docuemnt -> document

Add a content_type column to the table

* feat: add report run table and fk on cards

* feat: report run api start

* feat: run report api skeleton fixes for migration issues

* fixup! make qp run work
2025-07-22 12:46:09 -05:00
Aleksandr Lesnenko
853d9b0cf0 collapse sidebar in the reports view 2025-07-22 13:42:52 -04:00
Aleksandr Lesnenko
4e88f438e9 reorganize question embed node files 2025-07-22 13:01:35 -04:00
Ryan Laurie
5b6d472c96 cleanup some llm garbage 2025-07-22 10:38:00 -06:00
Ryan Laurie
60373ead57 add markdown exports
# Conflicts:
#	enterprise/frontend/src/metabase-enterprise/reports/components/ReportPage.tsx
2025-07-22 10:01:30 -06:00
Ryan Laurie
a4a927799d Enable CI skipping on PRs (#61307)
* dont run tests on pr with ci skip

* static viz too

* label instead of title
2025-07-22 09:59:24 -06:00
Aleksandr Lesnenko
8e75cd74f4 Report editing (#61290)
* basic tiptap

* new menu item

* vibecoded embedding

* lint, prettier

* layout

* refactor vibecoded layout

* ui

* useless placeholder

* fix selection

* lint

* ui

* ui

* ux

* ui

* experimental, possibly revert: notebook editor
2025-07-22 11:52:14 -04:00
Aleksandr Lesnenko
8252df95f2 actually tsx 2025-07-21 12:47:54 -04:00
Aleksandr Lesnenko
1cd6a43417 report route 2025-07-21 12:18:23 -04:00
4678 changed files with 578978 additions and 717580 deletions

View File

@@ -1,7 +1,7 @@
;; -*- comment-column: 100; -*-
{:config-paths ["macros" "src" "config"]
:linters
{ ;;
{;;
;; enabled optional linters (these are disabled by default)
;;
@@ -49,7 +49,7 @@
;; disabled linters
;;
:redundant-ignore {:level :off} ; lots of false positives, see https://github.com/clj-kondo/clj-kondo/issues/2414
:redundant-ignore {:level :off} ; lots of false positives, see https://github.com/clj-kondo/clj-kondo/issues/2414
:unexpected-recur {:level :off} ; TODO (cam): I think we just need to tell it how to handle MBQL match and we can enable this?
;;
@@ -173,7 +173,7 @@
metabase.driver.sql-jdbc.execute/execute-prepared-statement!
metabase.notification.models/create-notification!
metabase.pulse.send/send-pulse!
metabase-enterprise.transforms-python.python-runner-test/execute!
metabase.query-processor.store/store-database!
;; Safe in tests, where we typically use a private, temporary index per thread.
metabase.search.core/reindex!
metabase.util/run-count!
@@ -187,7 +187,7 @@
:metabase/disallow-hardcoded-driver-names-in-tests
;; don't include `:h2` since that will probably lead to a mountain of tests where we are specifically testing
;; middleware or compilation behavior with just H2 since it's the default.
{:drivers #{ ;; core drivers
{:drivers #{;; core drivers
#_:h2
:postgres
:mysql
@@ -215,7 +215,7 @@
;; misc config for built-in linters
;;
:invalid-arity {:skip-args [metabase.lib.util.match/match]} ; TODO (cam): can we fix these?
:invalid-arity {:skip-args [metabase.lib.util.match/match]} ; TODO (cam): can we fix these?
:refer-all {:level :warning, :exclude [clojure.test]}
:unused-referred-var {:exclude {compojure.core [GET DELETE POST PUT]}}
@@ -229,7 +229,7 @@
(clojure.core.logic/fresh)
(clojure.core.logic/matcha)
(clojure.core.logic/run)
(clojure.test/is [=? malli= qv=])
(clojure.test/is [query= =? malli= qv=])
(clojure.tools.macro/macrolet)
instaparse.core/transform
(metabase.server.streaming-response/streaming-response)
@@ -250,15 +250,16 @@
(taoensso.nippy/extend-freeze)
(taoensso.nippy/extend-thaw)]}
;; TODO (Cam 8/22/25) -- what's the point of marking stuff deprecated if we don't have Kondo enforce it? We should
;; remove all of these and add individual ignores as needed.
:deprecated-var
{:exclude
{metabase.driver.sql-jdbc.execute.old-impl/set-timezone-sql {:namespaces ["metabase\\.driver.*"]}
metabase.driver.sql-jdbc.execute/set-time-zone-if-supported! {:namespaces ["metabase\\.driver.*"]}
metabase.driver/current-db-time {:namespaces ["metabase.*"]}
metabase.driver/supports? {:namespaces ["metabase.*"]}
metabase.query-processor.util/normalize-token {:namespaces ["metabase.*"]}}}
{metabase.cmd.core/dump {:namespaces ["metabase\\.cmd-test" "metabase-enterprise\\.serialization\\.cmd-test"]}
metabase.cmd.core/load {:namespaces ["metabase\\.cmd-test" "metabase-enterprise\\.serialization\\.cmd-test"]}
metabase.driver.sql-jdbc.execute.old-impl/set-timezone-sql {:namespaces ["metabase\\.driver.*"]}
metabase.driver.sql-jdbc.execute/set-time-zone-if-supported! {:namespaces ["metabase\\.driver.*"]}
metabase.driver/current-db-time {:namespaces ["metabase.*"]}
metabase.driver/supports? {:namespaces ["metabase.*"]}
metabase.query-processor.middleware.resolve-joins/maybe-resolve-source-table {:namespaces ["metabase.*"]}
metabase.query-processor.util/normalize-token {:namespaces ["metabase.*"]}}}
:discouraged-var
{clojure.core/print {:message "Use metabase.util.log instead of clojure.core/print"}
@@ -291,8 +292,6 @@
me.flowthing.pp/pprint {:message "Use metabase.util.log instead of me.flowthing.pp/pprint"}
medley.core/random-uuid {:message "Use clojure.core/random-uuid instead of medley.core/random-uuid"}
metabase.driver/database-supports? {:message "Use metabase.driver.util/supports? instead of metabase.driver/database-supports?"}
metabase.lib.convert/->legacy-MBQL {:message "Conversion to legacy MBQL is discouraged, please use Lib to manipulate MBQL queries"}
metabase.lib.core/->legacy-MBQL {:message "Conversion to legacy MBQL is discouraged, please use Lib to manipulate MBQL queries"}
metabase.lib.equality/find-column-indexes-for-refs {:message "Use lib.equality/closest-matches-in-metadata or lib.equality/find-closest-matches-for-refs instead of lib.equality/find-column-indexes-for-refs"}
next.jdbc/active-tx? {:message "Use metabase.app-db.core/in-transaction? to see if we are in a transaction"}
toucan2.core/debug {:message "Don't commit code using t2/debug"}
@@ -301,7 +300,6 @@
:discouraged-namespace
{camel-snake-kebab.core {:message "CSK is not Turkish-safe, use the versions in metabase.util instead."}
cheshire.core {:message "Use metabase.util.json instead of cheshire.core directly."}
clj-time.core {:message "Use java-time.api or metabase.util.date-2 instead of clj-time"}
clojure.tools.logging {:message "Use metabase.util.log instead of clojure.tools.logging directly"}
grouper.core {:message "Use metabase.util.grouper instead of grouper.core"}
honeysql.core {:message "Use Honey SQL 2. Honey SQL 1 is removed as a dependency in Metabase 49+."}
@@ -410,16 +408,11 @@
clojure.test.check.clojure-test/defspec clojure.test/deftest
clojure.test.check.properties/for-all clojure.core/let
clojurewerkz.quartzite.jobs/defjob clojure.core/defn
com.climate.claypoole/with-shutdown! clojure.core/let
instaparse.core/defparser clojure.core/def
metabase-enterprise.action-v2.test-util/with-test-tables! clojure.core/let
metabase-enterprise.content-translation.routes-test/with-new-secret-key! clojure.core/fn
metabase-enterprise.serialization.test-util/with-dbs clojure.core/fn
metabase-enterprise.serialization.test-util/with-random-dump-dir clojure.core/let
metabase-enterprise.transforms.api-test/with-transform-tags clojure.core/let
metabase-enterprise.transforms.api-test/with-worker-runs clojure.core/let
metabase-enterprise.transforms.test-util/with-transform-cleanup! clojure.core/let
metabase-enterprise.transforms.execute/with-transform-lifecycle clojure.core/let
metabase-enterprise.content-translation.routes-test/with-new-secret-key! clojure.core/fn
metabase.actions.test-util/with-actions clojure.core/let
metabase.api.common/let-404 clojure.core/let
metabase.app-db.custom-migrations.util/with-temp-scheduler! clojure.core/fn
@@ -519,9 +512,7 @@
metabase-enterprise.advanced-permissions.models.permissions.application-permissions-test/with-new-group-and-current-graph hooks.common/with-two-top-level-bindings
metabase-enterprise.audit-app.pages-test/with-temp-objects hooks.common/with-one-binding
metabase-enterprise.cache.config-test/with-temp-persist-models hooks.common/with-seven-bindings
metabase-enterprise.semantic-search.db.connection/with-migrate-tx hooks.common/let-one-with-optional-value
metabase-enterprise.serialization.test-util/with-temp-dpc hooks.toucan2.tools.with-temp/with-temp
metabase-enterprise.transforms-python.drivers-test/with-test-table hooks.common/with-two-bindings
metabase.api.common.internal-test/with-jetty-server hooks.common/with-one-binding
metabase.api.macros/defendpoint hooks.metabase.api.macros/defendpoint
metabase.app-db.schema-migrations-test.impl/test-migrations hooks.metabase.app-db.schema-migrations-test.impl/test-migrations
@@ -652,7 +643,6 @@
:macroexpand
{clojurewerkz.quartzite.jobs/build macros.quartz/build-job
clojurewerkz.quartzite.schedule.calendar-interval/build macros.quartz/build-job
clojurewerkz.quartzite.schedule.cron/schedule macros.quartz/schedule
clojurewerkz.quartzite.schedule.simple/schedule macros.quartz/simple-schedule
clojurewerkz.quartzite.triggers/build macros.quartz/build-trigger
@@ -716,27 +706,17 @@
{:pattern "^metabase\\.lib\\..*"
:name metabase-lib-excluding-legacy-mbql}
;;
;; QP and driver source namespaces: `metabase.query-processor.*`, `metabase-enterprise.sandbox.query-processor.*`,
;; and `metabase.driver.*`, excluding `-test` namespaces.
;; QP source namespaces: `metabase.query-processor.*`, `metabase-enterprise.sandbox.query-processor.*`, and
;; `metabase.driver.*`, excluding `-test` namespaces.
;;
{:pattern "metabase(?:(?:(?:-enterprise\\.sandbox)?\\.query-processor\\.)|(?:\\.driver\\.)).*(?<!-test)$"
:name qp-and-driver-source-namespaces}
;;
;; QP source namespaces
;;
{:pattern "metabase(?:(?:-enterprise\\.sandbox)?\\.query-processor\\.).*(?<!-test)$"
:name qp-source-namespaces}
;;
;; driver namespaces, including tests
;; driver namespaces
;;
{:pattern "metabase\\.driver.*$"
:name driver-namespaces}
;;
;; Perf-sensitive namespaces are currently the union of metabase.lib.*, driver namespaces, and QP (without tests).
;;
{:pattern "^(?!.*-test(?:\\..*)?$)(?!.*test-util)(?!^metabase\\.test.*)^metabase\\.(lib|driver|legacy-mbql|query-processor).*"
:name performance-sensitive}
;;
;; general QP tests are ones that run across multiple drivers. Anything `metabase.query-processor*-test`,
;; `metabase.driver.sql*-test` (excluding SQLServer and SQLite), or
;; `metabase-enterprise.sandbox.query-processor*-test`
@@ -750,13 +730,9 @@
{:inline-def {:level :off}
:missing-docstring {:level :off}
:private-call {:level :off}
:discouraged-var {metabase.driver/database-supports? {:level :off}}
:metabase/defsetting-must-specify-export {:level :off}
:metabase/defsetting-namespace {:level :off}
:discouraged-var
{metabase.driver/database-supports? {:level :off}
metabase.lib.convert/->legacy-MBQL {:level :off}
metabase.lib.core/->legacy-MBQL {:level :off}}}
:metabase/defsetting-namespace {:level :off}}
:hooks
{:analyze-call
@@ -769,13 +745,9 @@
{:metabase/validate-escaped-single-quotes-in-i18n {:level :warning}
:metabase/validate-string-or-str-args-to-i18n {:level :warning}
:metabase/discourage-millis-duration {:level :warning}
:discouraged-var
{clojure.core/with-redefs {:message "Don't use with-redefs outside of tests"}
clojure.core/eval {:message "Don't use eval outside of tests"}}
:discouraged-namespace
{metabase.query-processor.store {:message "Please do not use the QP Store outside of legacy QP or driver code. Use MBQL 5/Lib or lib.metadata.jvm/applicaton-database-metadata-provider instead."}}}}
clojure.core/eval {:message "Don't use eval outside of tests"}}}}
printable-namespaces
{:linters
@@ -787,7 +759,7 @@
clojure.string/lower-case {:level :off}
clojure.string/upper-case {:level :off}}}}
;; enable all the extra linters we haven't enabled in the core app yet in Lib. It's all new code so we can adhere
;; enable all the extra linters we haven't enabled in the core app yet in MLv2. It's all new code so we can adhere
;; to the new stricter rules from day 1
metabase-lib
{:linters
@@ -800,89 +772,50 @@
;; eventually we should do this for the whole codebase, but the args are in the opposite order so switching them
;; all at once isn't trivial, at least we can stop using it in new code.
:discouraged-var
{medley.core/map-keys {:message "Use metabase.util.performance/update-keys"}
medley.core/map-vals {:message "Use clojure.core/update-vals"}
metabase.lib.convert/->legacy-MBQL {:level :off}
metabase.lib.core/->legacy-MBQL {:level :off}
metabase.test/with-temp {:message "Don't use application database inside Lib code"}}
{medley.core/map-keys {:message "Use clojure.core/update-keys"}
medley.core/map-vals {:message "Use clojure.core/update-vals"}
metabase.test/with-temp {:message "Don't use application database inside MLv2 code"}}
:discouraged-namespace
{metabase.driver {:message "Lib code should be independent of driver code/methods"}
metabase.test {:message "Lib tests should only use Lib test helpers"}
metabase.test.data {:message "Lib tests should only use Lib test helpers"}
metabase.util.malli.schema {:message "Prefer metabase.lib.schema schemas in Lib code"}
toucan.db {:message "Don't use application database inside Lib code"}
toucan.models {:message "Don't use application database inside Lib code"}
toucan.util.test {:message "Don't use application database inside Lib code"}
toucan2.core {:message "Don't use application database inside Lib code"}}}}
{metabase.driver {:message "MLv2 code should be independent of driver code/methods"}
metabase.test {:message "MLv2 tests should only use MLv2 test helpers"}
metabase.test.data {:message "MLv2 tests should only use MLv2 test helpers"}
metabase.util.malli.schema {:message "Prefer metabase.lib.schema schemas in MLv2 code"}
toucan.db {:message "Don't use application database inside MLv2 code"}
toucan.models {:message "Don't use application database inside MLv2 code"}
toucan.util.test {:message "Don't use application database inside MLv2 code"}
toucan2.core {:message "Don't use application database inside MLv2 code"}}}}
metabase-lib-excluding-legacy-mbql
{:linters
{:discouraged-var
{metabase.legacy-mbql.util/unique-name-generator {:message "Use metabase.lib.util/unique-name-generator or non-truncating-unique-name-generator"}}}
:discouraged-namespace
{metabase.legacy-mbql.js {:message "Avoid legacy MBQL utils in Lib; move things into Lib if needded."}
metabase.legacy-mbql.jvm-util {:message "Avoid legacy MBQL utils in Lib; move things into Lib if needded."}
metabase.legacy-mbql.predicates {:message "Avoid legacy MBQL utils in Lib; move things into Lib if needded."}
metabase.legacy-mbql.util {:message "Avoid legacy MBQL utils in Lib; move things into Lib if needded."}}}
{metabase.legacy-mbql.util/unique-name-generator {:message "Use metabase.lib.util/unique-name-generator or non-truncating-unique-name-generator"}}}}
driver-namespaces
{:linters
{:discouraged-var
{metabase.driver/database-supports? {:level :off}}}}
performance-sensitive
{:linters
{:discouraged-var
{clojure.core/every? {:message "Use metabase.util.performance/every?"}
clojure.core/mapv {:message "Use metabase.util.performance/mapv"}
clojure.core/run! {:message "Use metabase.util.performance/run!"}
clojure.core/select-keys {:message "Use metabase.util.performance/select-keys"}
clojure.core/some {:message "Use metabase.util.performance/some"}
clojure.core/update-keys {:message "Use metabase.util.performance/update-keys"}
clojure.walk/keywordize-keys {:message "Use metabase.util.performance/keywordize-keys"}
clojure.walk/postwalk {:message "Use metabase.util.performance/postwalk"}
clojure.walk/prewalk {:message "Use metabase.util.performance/prewalk"}
clojure.walk/walk {:message "Use metabase.util.performance/walk"}}}}
qp-and-driver-source-namespaces
{:linters
{:discouraged-namespace
{metabase.query-processor.store {:level :off}
metabase.util.malli.schema {:message "Prefer metabase.lib.schema or metabase.legacy-mbql.schema schemas in QP/driver code"}
toucan.db {:message "Don't use application database directly in QP code; use the MetadataProvider"}
toucan.models {:message "Don't use application database directly in QP code; use the MetadataProvider"}
toucan.util.test {:message "Don't use application database directly in QP code; use the MetadataProvider"}
toucan2.core {:message "Don't use application database directly in QP code; use the MetadataProvider"}}
{metabase.util.malli.schema {:message "Prefer metabase.lib.schema or metabase.legacy-mbql.schema schemas in QP/driver code"}
toucan.db {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
toucan.models {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
toucan.util.test {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
toucan2.core {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}}
:discouraged-var
{medley.core/map-keys {:message "Use metabase.util.performance/update-keys"}
{medley.core/map-keys {:message "Use clojure.core/update-keys"}
medley.core/map-vals {:message "Use clojure.core/update-vals"}
metabase.app-db.core/query {:message "Don't use application database directly in QP code; use the MetadataProvider"}
metabase.app-db.core/reducible-query {:message "Don't use application database directly in QP code; use the MetadataProvider"}
metabase.lib.convert/->legacy-MBQL {:level :off}
metabase.lib.core/->legacy-MBQL {:level :off}}}}
qp-source-namespaces
{:linters
{:discouraged-namespace
{metabase.legacy-mbql.js {:message "Avoid legacy MBQL namespaces in the QP, use Lib instead"}
metabase.legacy-mbql.jvm-util {:message "Avoid legacy MBQL namespaces in the QP, use Lib instead"}
metabase.legacy-mbql.normalize {:message "Avoid legacy MBQL namespaces in the QP, use Lib instead"}
metabase.legacy-mbql.predicates {:message "Avoid legacy MBQL namespaces in the QP, use Lib instead"}
metabase.legacy-mbql.schema {:message "Avoid legacy MBQL namespaces in the QP, use Lib instead"}
metabase.legacy-mbql.schema.helpers {:message "Avoid legacy MBQL namespaces in the QP, use Lib instead"}
metabase.legacy-mbql.schema.macros {:message "Avoid legacy MBQL namespaces in the QP, use Lib instead"}
metabase.legacy-mbql.util {:message "Avoid legacy MBQL namespaces in the QP, use Lib instead"}}}}
metabase.app-db.core/query {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}
metabase.app-db.core/reducible-query {:message "Don't use application database directly in QP code; use QP Store/metadata provider"}}}}
general-qp-and-driver-test-namespaces
{:linters
{:discouraged-var
{metabase.test/with-column-remappings {:message "Use lib.tu/remap-metadata-provider instead of with-column-remappings in QP tests"}
metabase.test.util/with-column-remappings {:message "Use lib.tu/remap-metadata-provider instead of with-column-remappings in QP tests"}
metabase.test/with-temp {:message "Use lib.tu/mock-metadata-provider in QP tests. Exceptions: tests for code that doesn't use MPs or models not supported by MPs"}
toucan2.tools.with-temp/with-temp {:message "Use lib.tu/mock-metadata-provider in QP tests. Exceptions: tests for code that doesn't use MPs or models not supported by MPs"}}
{metabase.test/with-column-remappings {:message "Use metabase.lib.test-util/remap-metadata-provider instead of with-column-remappings in QP tests"}
metabase.test.util/with-column-remappings {:message "Use metabase.lib.test-util/remap-metadata-provider instead of with-column-remappings in QP tests"}}
:metabase/disallow-hardcoded-driver-names-in-tests
{:level :warning}}}}}

File diff suppressed because it is too large Load Diff

View File

@@ -13,10 +13,7 @@
toucan2.core/insert-returning-pk!
toucan2.core/insert-returning-pks!
clojure.core.async/<!!
clojure.core.async/<!
clojure.core.async/>!!
clojure.core.async/>!
clojure.core.async/alts!
clojure.core.async/alts!!
clojure.core.async/close!
clojure.core.async/poll!

View File

@@ -52,13 +52,7 @@
(defn- lint-require* [node current-ns config]
(let [[_require & args] (:children node)
required-namespaces (->> (keep unwrap-require args)
;; `require` runs through `cc/load-libs` which allows
;; #{:as :reload :reload-all :require :use :verbose :refer :as-alias}
;; as args. If you use a REPL tool that applies clj-kondo linting (e.g., Clojure MCP)
;; you may wish to `:reload` or `:reload-all` a namespace. So we shouldn't assume all
;; args are keywords:
(remove keyword?))]
required-namespaces (keep unwrap-require args)]
(lint* node current-ns config required-namespaces)))
(defn- lint-requiring-resolve* [node current-ns config]

View File

@@ -14,7 +14,6 @@ if (shouldLintCssModules) {
}
module.exports = {
ignorePatterns: ["!.storybook"],
rules: {
strict: [2, "never"],
"no-undef": 2,
@@ -49,7 +48,6 @@ module.exports = {
],
curly: [1, "all"],
eqeqeq: [1, "smart"],
"import/no-duplicates": ["warn", { considerQueryString: true }],
"import/no-default-export": 2,
"import/no-named-as-default": 0,
"import/no-commonjs": 1,
@@ -180,7 +178,6 @@ module.exports = {
{
files: ["*.js", "*.jsx", "*.ts", "*.tsx"],
rules: {
"jtag-missing-key": "error",
"no-unconditional-metabase-links-render": "error",
"no-color-literals": "error",
"no-literal-metabase-strings": "error",
@@ -205,14 +202,12 @@ module.exports = {
"enterprise/frontend/src/metabase-enterprise/whitelabel/**/*",
"frontend/lint/**/*",
"*.stories.*",
"**/.storybook/*",
"stories-data.*",
"e2e/**/*",
"**/tests/*",
"release/**/*",
"rspack.config.js",
"rspack.main.config.js",
"rspack.embedding-sdk-package.config.js",
"rspack.embedding-sdk-bundle.config.js",
],
rules: {

48
.github/CODEOWNERS vendored
View File

@@ -1,9 +1,7 @@
# Before changing this file, make sure you have read how we use it:
# https://www.notion.so/metabase/Pull-Request-Code-Review-Guide-374f05df889748b69d67af753f9bc9b8?pvs=4#36550c04ceab40b5a8b8bee6264b090c
enterprise/frontend/src/embedding-sdk-package @metabase/embedding
enterprise/frontend/src/embedding-sdk-bundle @metabase/embedding
enterprise/frontend/src/embedding-sdk-shared @metabase/embedding
enterprise/frontend/src/embedding-sdk @metabase/embedding
frontend/src/metabase/embedding-sdk @metabase/embedding
frontend/src/metabase/public/containers/PublicOrEmbeddedQuestion @metabase/embedding
frontend/src/metabase/public/containers/PublicOrEmbeddedDashboard @metabase/embedding
@@ -22,8 +20,8 @@ docs/developers-guide # exclude from tech-writer ownership
#
# modules/drivers @metabase/drivers
src/metabase/driver/sql_jdbc/actions.clj @metabase/core-ux-east
test/metabase/driver/sql_jdbc/actions_test.clj @metabase/core-ux-east
src/metabase/driver/sql_jdbc/actions.clj @metabase/core-workflows
test/metabase/driver/sql_jdbc/actions_test.clj @metabase/core-workflows
# enterprise/backend/src/metabase_enterprise/advanced_config @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/advanced_config @metabase/core-backend-admin-webapp
@@ -55,8 +53,8 @@ test/metabase/driver/sql_jdbc/actions_test.clj @metabase/core-ux-east
# enterprise/backend/src/metabase_enterprise/core @metabase/devexp
# enterprise/backend/test/metabase_enterprise/core @metabase/devexp
# enterprise/backend/src/metabase_enterprise/dashboard_subscription_filters @metabase/core-ux-east
# enterprise/backend/test/metabase_enterprise/dashboard_subscription_filters @metabase/core-ux-east
# enterprise/backend/src/metabase_enterprise/dashboard_subscription_filters @metabase/core-workflows
# enterprise/backend/test/metabase_enterprise/dashboard_subscription_filters @metabase/core-workflows
# enterprise/backend/src/metabase_enterprise/enhancements @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/enhancements @metabase/core-backend-admin-webapp
@@ -73,11 +71,11 @@ test/metabase/driver/sql_jdbc/actions_test.clj @metabase/core-ux-east
# enterprise/backend/src/metabase_enterprise/scim @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/scim @metabase/core-backend-admin-webapp
# enterprise/backend/src/metabase_enterprise/search ...
# enterprise/backend/test/metabase_enterprise/search ...
# enterprise/backend/src/metabase_enterprise/search @metabase/core-workflows
# enterprise/backend/test/metabase_enterprise/search @metabase/core-workflows
# enterprise/backend/src/metabase_enterprise/serialization @metabase/core-ux-east
# enterprise/backend/test/metabase_enterprise/serialization @metabase/core-ux-east
# enterprise/backend/src/metabase_enterprise/serialization @metabase/core-workflows
# enterprise/backend/test/metabase_enterprise/serialization @metabase/core-workflows
# enterprise/backend/src/metabase_enterprise/snippet_collections @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/snippet_collections @metabase/core-backend-admin-webapp
@@ -97,8 +95,8 @@ test/metabase/driver/sql_jdbc/actions_test.clj @metabase/core-ux-east
# enterprise/backend/src/metabase_enterprise/upload_management @metabase/core-backend-admin-webapp
# enterprise/backend/test/metabase_enterprise/upload_management @metabase/core-backend-admin-webapp
src/metabase/actions @metabase/core-ux-east
test/metabase/actions @metabase/core-ux-east
src/metabase/actions @metabase/core-backend-ux-west
test/metabase/actions @metabase/core-backend-ux-west
# src/metabase/activity_feed @metabase/core-backend-admin-webapp
# test/metabase/activity_feed @metabase/core-backend-admin-webapp
@@ -124,11 +122,11 @@ test/metabase/actions @metabase/core-ux-east
# src/metabase/cmd @metabase/devexp
# test/metabase/cmd @metabase/devexp
# src/metabase/channel @metabase/core-ux-east
# test/metabase/channel @metabase/core-ux-east
# src/metabase/channel @metabase/core-workflows
# test/metabase/channel @metabase/core-workflows
# src/metabase/cloud_migration ...
# test/metabase/cloud_migration ...
# src/metabase/cloud_migration @metabase/core-workflows
# test/metabase/cloud_migration @metabase/core-workflows
# src/metabase/cmd @metabase/devexp
# test/metabase/cmd @metabase/devexp
@@ -160,8 +158,8 @@ test/metabase/actions @metabase/core-ux-east
# src/metabase/indexed_entities @metabase/core-backend-querying
# test/metabase/indexed_entities @metabase/core-backend-querying
# src/metabase/integrations ...
# test/metabase/integrations ...
# src/metabase/integrations @metabase/core-workflows
# test/metabase/integrations @metabase/core-workflows
# src/metabase/legacy_mbql @metabase/core-backend-querying
# test/metabase/legacy_mbql @metabase/core-backend-querying
@@ -184,8 +182,8 @@ test/metabase/actions @metabase/core-ux-east
# src/metabase/moderation @metabase/core-backend-admin-webapp
# test/metabase/moderation @metabase/core-backend-admin-webapp
# src/metabase/notification @metabase/core-ux-east
# test/metabase/notification @metabase/core-ux-east
# src/metabase/notification @metabase/core-workflows
# test/metabase/notification @metabase/core-workflows
# src/metabase/permissions @metabase/core-backend-admin-webapp
# test/metabase/permissions @metabase/core-backend-admin-webapp
@@ -202,8 +200,8 @@ test/metabase/actions @metabase/core-ux-east
# src/metabase/public_sharing @metabase/core-backend-admin-webapp
# test/metabase/public_sharing @metabase/core-backend-admin-webapp
# src/metabase/pulse @metabase/core-ux-east
# test/metabase/pulse @metabase/core-ux-east
# src/metabase/pulse @metabase/core-workflows
# test/metabase/pulse @metabase/core-workflows
# src/metabase/query_analysis @metabase/core-backend-querying
# test/metabase/query_analysis @metabase/core-backend-querying
@@ -220,8 +218,8 @@ test/metabase/actions @metabase/core-ux-east
# src/metabase/sample_data @metabase/core-backend-admin-webapp
# test/metabase/sample_data @metabase/core-backend-admin-webapp
# src/metabase/search ...
# test/metabase/search ...
# src/metabase/search @metabase/core-workflows
# test/metabase/search @metabase/core-workflows
# src/metabase/segments @metabase/core-backend-querying
# test/metabase/segments @metabase/core-backend-querying

View File

@@ -3,7 +3,7 @@
> **Warning**
>
> If that is your first contribution to Metabase, please sign the [Contributor License Agreement](https://docs.google.com/a/metabase.com/forms/d/1oV38o7b9ONFSwuzwmERRMi9SYrhYeOrkbmNaq9pOJ_E/viewform). Also, if you're attempting to fix a translation issue, please submit your changes to our [Crowdin project](https://crowdin.com/project/metabase-i18n) instead of opening a PR.
> If that is your first contribution to Metabase, please sign the [Contributor License Agreement](https://docs.google.com/a/metabase.com/forms/d/1oV38o7b9ONFSwuzwmERRMi9SYrhYeOrkbmNaq9pOJ_E/viewform) (unless it's a tiny documentation change). Also, if you're attempting to fix a translation issue, please submit your changes to our [Crowdin project](https://crowdin.com/project/metabase-i18n) instead of opening a PR.
Closes https://github.com/metabase/metabase/issues/[issue_number]

View File

@@ -74,7 +74,7 @@ runs:
git push -u origin ${BACKPORT_BRANCH}
BACKPORT_PR_URL=$(gh pr create --base "${TARGET_BRANCH}" --head "${BACKPORT_BRANCH}" --label "was-backported" --assignee "${{ github.event.pull_request.user.login }}" --title "🤖 backported \"${ORIGINAL_TITLE}\"" --body "${PR_BODY}")
BACKPORT_PR_URL=$(gh pr create --base "${TARGET_BRANCH}" --head "${BACKPORT_BRANCH}" --label "was-backported" --assignee "${GITHUB_ACTOR}" --title "🤖 backported \"${ORIGINAL_TITLE}\"" --body "${PR_BODY}")
BACKPORT_PR_NUMBER=${BACKPORT_PR_URL##*/}
echo "backport_pr_number=$BACKPORT_PR_NUMBER" >> $GITHUB_OUTPUT

View File

@@ -5,14 +5,6 @@ inputs:
name:
description: The name of the artifact to fetch
required: true
output-name:
description: Optional custom name for the output jar file (defaults to metabase.jar)
required: false
default: "metabase.jar"
extract-path:
description: Directory path where the jar should be extracted (defaults to current directory)
required: false
default: "."
runs:
using: "composite"
@@ -33,7 +25,7 @@ runs:
});
if (!artifacts.data?.artifacts?.[0]?.id) {
throw new Error(`No artifacts found for ${{ inputs.name }}`);
throw new Error(`No artifacts found for ${{ inputs.commit }}`);
}
const artifact_id = artifacts.data.artifacts[0].id;
@@ -47,59 +39,25 @@ runs:
fs.writeFileSync(`${process.env.GITHUB_WORKSPACE}/mb.zip`, Buffer.from(download.data));
- name: Extract and rename JAR
- name: unzip uberjar artifact
shell: bash
run: unzip mb.zip
- name: Move the Uberjar
shell: bash
run: | # bash
OUTPUT_NAME="${{ inputs.output-name }}"
EXTRACT_PATH="${{ inputs.extract-path }}"
# Create a temporary directory for extraction to avoid conflicts
TEMP_DIR=$(mktemp -d)
# Extract to temporary directory
unzip mb.zip -d "$TEMP_DIR"
# Create target directory
mkdir -p "$EXTRACT_PATH"
# Find the jar and move all files from its directory
if [ -e "$TEMP_DIR/metabase.jar" ]; then
# Move all files to extract path
mv "$TEMP_DIR"/* "$EXTRACT_PATH/" 2>/dev/null || true
# Rename the jar to custom name if different
if [ "$OUTPUT_NAME" != "metabase.jar" ]; then
mv "$EXTRACT_PATH/metabase.jar" "$EXTRACT_PATH/$OUTPUT_NAME"
fi
elif [ -e "$TEMP_DIR/target/uberjar/metabase.jar" ]; then
# Move all files from uberjar directory to extract path
mv "$TEMP_DIR/target/uberjar"/* "$EXTRACT_PATH/" 2>/dev/null || true
# Rename the jar to custom name if different
if [ "$OUTPUT_NAME" != "metabase.jar" ]; then
mv "$EXTRACT_PATH/metabase.jar" "$EXTRACT_PATH/$OUTPUT_NAME"
fi
if [ -e "metabase.jar" ]; then
echo "metabase.jar is already in the right place"
elif [ -e "target/uberjar/metabase.jar" ]; then
echo "moving metabase.jar to current directory"
mv target/uberjar/metabase.jar metabase.jar
else
echo "Could not find metabase.jar in extracted files"
echo "Could not find the metabase.jar file"
exit 1
fi
# Clean up temporary directory
rm -rf "$TEMP_DIR"
# Set the JAR_PATH environment variable
if [ "$EXTRACT_PATH" == "." ]; then
echo "JAR_PATH=$OUTPUT_NAME" >> $GITHUB_ENV
else
echo "JAR_PATH=$EXTRACT_PATH/$OUTPUT_NAME" >> $GITHUB_ENV
fi
- name: Verify that this is a valid JAR file
shell: bash
run: |
echo "Verifying JAR at: $JAR_PATH"
file --mime-type "$JAR_PATH"
run: file --mime-type ./metabase.jar
- name: Reveal its version.properties
shell: bash
run: |
echo "Extracting version.properties from: $JAR_PATH"
jar xf "$JAR_PATH" version.properties && cat version.properties
run: jar xf metabase.jar version.properties && cat version.properties

View File

@@ -1,74 +0,0 @@
name: Get Docker Info
description: Determines Docker repository and tag based on branch and edition
inputs:
branch:
description: Branch name (defaults to current branch)
required: false
default: ${{ github.head_ref || github.ref_name }}
edition:
description: Edition (ee or oss)
required: false
default: ee
owner:
description: Repository owner (defaults to current repo owner)
required: false
default: ${{ github.repository_owner }}
outputs:
repo:
description: Docker repository name
value: ${{ steps.docker-info.outputs.repo }}
tag:
description: Docker tag
value: ${{ steps.docker-info.outputs.tag }}
full_image:
description: Full Docker image name (repo:tag)
value: ${{ steps.docker-info.outputs.full_image }}
editions:
description: JSON array of editions to build
value: ${{ steps.docker-info.outputs.editions }}
runs:
using: "composite"
steps:
- name: Determine Docker info
id: docker-info
uses: actions/github-script@v7
with:
script: | # js
const branch = '${{ inputs.branch }}';
const edition = '${{ inputs.edition }}';
const owner = '${{ inputs.owner }}';
console.log(`Determining Docker info for branch: ${branch}, edition: ${edition}`);
let repo, tag, editions;
if (branch === 'master') {
// Master builds both editions with specific repos
editions = ['ee', 'oss'];
tag = 'latest';
if (edition === 'ee') {
repo = `${owner}/metabase-enterprise-head`;
} else {
repo = `${owner}/metabase-head`;
}
} else {
// All other branches only build EE to metabase-dev
editions = ['ee'];
tag = branch.replaceAll('/', '-'); // slashes make docker sad
repo = `${owner}/metabase-dev`;
}
const fullImage = `${repo}:${tag}`;
// Set outputs
core.setOutput('repo', repo);
core.setOutput('tag', tag);
core.setOutput('full_image', fullImage);
core.setOutput('editions', JSON.stringify(editions));
console.log(`Docker image: ${fullImage}`);
console.log(`Editions: ${JSON.stringify(editions)}`);

View File

@@ -1,7 +1,4 @@
name: Prepare back-end environment
description: |
Prepare the backend. Installs jdk, Clojure, and fetches the cache for m2. The CI config fetchs `ci-test-config.json`
from master and lets us ignore tests at the var level if they are flaking.
inputs:
java-version:
required: true
@@ -24,14 +21,9 @@ runs:
distribution: 'temurin'
- name: Install Clojure CLI
shell: bash
continue-on-error: true
run: |
curl -O https://download.clojure.org/install/linux-install-${{ inputs.clojure-version }}.sh &&
sudo bash ./linux-install-${{ inputs.clojure-version }}.sh
- name: Fetch CI config
shell: bash
run: |
curl -o ci-test-config.json https://raw.githubusercontent.com/metabase/metabase/refs/heads/master/ci-test-config.json
- name: Check to see if dependencies should be cached
if: ${{ contains(github.event.head_commit.message, '[ci nocache]') }}
run: echo "Commit message includes [ci nocache]; dependencies will NOT be cached"

View File

@@ -40,7 +40,7 @@ runs:
list-tests: failed
fail-on-error: false
- name: Upload Logs on Failure
if: always() && !success()
if: failure()
uses: actions/upload-artifact@v4
with:
name: test-logs-${{ github.job }}

View File

@@ -30,7 +30,6 @@ frontend_configs: &frontend_configs
- "postcss.config.js"
- "rspack.config.js"
- "rspack.main.config.js"
- "rspack.embedding-sdk-package.config.js"
- "rspack.embedding-sdk-bundle.config.js"
- "rspack.static-viz.config.js"
- ".nvmrc"
@@ -41,17 +40,10 @@ frontend_sources: &frontend_sources
- "enterprise/frontend/!(test)**/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
- "frontend/!(test)**/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
frontend_embedding_sources:
- "enterprise/frontend/src/{embedding,embedding-*}/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
- "enterprise/frontend/src/metabase-enterprise/{embedding,embedding-*}**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
frontend_embedding_sdk_sources:
- *frontend_configs
- "enterprise/frontend/!(test)**/**/{!(*.spec),}.{js,jsx,ts,tsx,css}"
frontend_embedding_sdk_package_sources:
- "enterprise/frontend/src/embedding-sdk-package/**/!(*.spec).{js,jsx,ts,tsx,css}"
frontend_specs: &frontend_specs
- *shared_specs
- "frontend/test/**"
@@ -145,17 +137,16 @@ e2e_embedding_sdk_compatibility:
- "e2e/runner/embedding-sdk/**"
- "e2e/test-host-app/**/*.cy.*.{js,ts}"
snowplow: &snowplow
- ".github/workflows/snowplow.yml"
- "snowplow/**"
e2e_all:
- *default
- *e2e_ci
- *sources
- *e2e_specs
- "e2e/**"
- *snowplow
snowplow:
- ".github/workflows/snowplow.yml"
- "snowplow/**"
documentation:
- "docs/**"

View File

@@ -1,172 +0,0 @@
const fs = require("fs");
const path = require("path");
const { execSync } = require("child_process");
const collectAndAverageTimings = async ({
github,
context,
daysBack = 7,
existingTimingsPath = "e2e/support/timings.json",
}) => {
try {
const artifacts = await findMergedTimingArtifacts({
github,
context,
daysBack,
});
if (artifacts.length === 0) {
throw new Error("No timing artifacts found in the specified time range");
}
const timingData = await downloadAndExtractArtifacts(
artifacts,
token,
github,
context,
);
if (timingData.length === 0) {
throw new Error("No valid timing data extracted from artifacts");
}
const averagedTimings = averageTimings(timingData);
averagedTimings.durations.sort((a, b) => a.spec.localeCompare(b.spec));
fs.writeFileSync(
existingTimingsPath,
JSON.stringify(averagedTimings, null, 2),
);
console.log(`✅ Updated ${existingTimingsPath} with new timing data`);
return { artifactCount: timingData.length };
} catch (error) {
console.error("❌ Error in collectAndAverageTimings:", error.message);
throw error;
}
};
const findMergedTimingArtifacts = async ({ github, context, daysBack }) => {
const cutoffDate = new Date();
cutoffDate.setDate(cutoffDate.getDate() - daysBack);
console.log(
`Searching for merged timing artifacts created after ${cutoffDate.toISOString()}`,
);
const artifacts = [];
let page = 1;
while (true) {
try {
const { data } = await github.rest.actions.listArtifactsForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
name: "merged-e2e-timings",
per_page: 100,
page,
});
const validArtifacts = data.artifacts
.filter((a) => new Date(a.created_at) > cutoffDate && !a.expired)
.map(({ name, created_at, id }) => ({ name, created_at, id }));
artifacts.push(...validArtifacts);
if (
data.artifacts.some((a) => new Date(a.created_at) <= cutoffDate) ||
data.artifacts.length < 100
) {
break;
}
page++;
} catch (error) {
console.log(
`Error fetching artifacts page ${page - 1}: ${error.message}`,
);
break;
}
}
console.log(`Found ${artifacts.length} merged timing artifacts total`);
return artifacts;
};
const downloadAndExtractArtifacts = async (artifacts, github, context) => {
const tempDir = "timing-artifacts-temp";
fs.mkdirSync(tempDir, { recursive: true });
const timingData = [];
for (const artifact of artifacts) {
try {
console.log(`Downloading artifact: ${artifact.name}`);
const zipPath = path.join(tempDir, `${artifact.name}.zip`);
const extractPath = path.join(tempDir, artifact.name);
const { data } = await github.rest.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: artifact.id,
archive_format: "zip",
});
fs.writeFileSync(zipPath, Buffer.from(data));
fs.mkdirSync(extractPath, { recursive: true });
execSync(`unzip -q "${zipPath}" -d "${extractPath}"`);
const timingFilePath = path.join(extractPath, "runTimings.json");
if (fs.existsSync(timingFilePath)) {
const content = JSON.parse(fs.readFileSync(timingFilePath, "utf8"));
if (content.durations?.length) {
timingData.push(content);
console.log(
`✅ Extracted timing data from ${artifact.name} (${content.durations.length} specs)`,
);
} else {
console.log(`⚠ Invalid timing data structure in ${artifact.name}`);
}
} else {
console.log(`⚠ No timing data found in ${artifact.name}`);
}
fs.rmSync(extractPath, { recursive: true });
} catch (error) {
console.log(`❌ Error processing ${artifact.name}: ${error.message}`);
}
}
fs.rmSync(tempDir, { recursive: true });
console.log(
`Successfully extracted timing data from ${timingData.length}/${artifacts.length} artifacts`,
);
return timingData;
};
const averageTimings = (timingDataArray) => {
if (!timingDataArray.length) {
throw new Error("No timing data to process");
}
const specTimings = timingDataArray
.flatMap((data) => data.durations)
.filter((item) => item.spec && typeof item.duration === "number")
.reduce((acc, { spec, duration }) => {
(acc[spec] = acc[spec] || []).push(duration);
return acc;
}, {});
const durations = Object.entries(specTimings).map(([spec, durations]) => ({
spec,
duration: Math.round(
durations.reduce((sum, d) => sum + d, 0) / durations.length,
),
}));
return { durations };
};
module.exports = { collectAndAverageTimings, averageTimings };

View File

@@ -1,132 +0,0 @@
const { averageTimings } = require("./average-e2e-timings");
describe("averageTimings", () => {
it("should throw error when given empty array", () => {
expect(() => averageTimings([])).toThrow("No timing data to process");
});
it("should work for n=1", () => {
const input = [
{
durations: [
{ spec: "test1.cy.js", duration: 1000 },
{ spec: "test2.cy.js", duration: 2000 },
],
},
];
const result = averageTimings(input);
expect(result.durations).toHaveLength(2);
expect(result.durations).toContainEqual({
spec: "test1.cy.js",
duration: 1000,
});
expect(result.durations).toContainEqual({
spec: "test2.cy.js",
duration: 2000,
});
});
it("should calculate average for multiple specs", () => {
const input = [
{
durations: [
{ spec: "test1.cy.js", duration: 1000 },
{ spec: "test2.cy.js", duration: 2000 },
],
},
{
durations: [
{ spec: "test1.cy.js", duration: 2000 },
{ spec: "test2.cy.js", duration: 4000 },
],
},
];
const result = averageTimings(input);
expect(result.durations).toHaveLength(2);
expect(result.durations).toContainEqual({
spec: "test1.cy.js",
duration: 1500,
});
expect(result.durations).toContainEqual({
spec: "test2.cy.js",
duration: 3000,
});
});
it("should round averages to nearest integer", () => {
const input = [
{
durations: [{ spec: "test1.cy.js", duration: 1001 }],
},
{
durations: [{ spec: "test1.cy.js", duration: 1002 }],
},
];
const result = averageTimings(input);
expect(result.durations[0].duration).toBe(1002);
});
it("should ignore invalid entries", () => {
const input = [
{
durations: [
{ spec: "test1.cy.js", duration: 1000 },
{ spec: "test2.cy.js", duration: "invalid" },
{ duration: 2000 }, // missing spec
{ spec: "test3.cy.js", duration: 3000 },
],
},
];
const result = averageTimings(input);
expect(result.durations).toHaveLength(2);
expect(result.durations).toContainEqual({
spec: "test1.cy.js",
duration: 1000,
});
expect(result.durations).toContainEqual({
spec: "test3.cy.js",
duration: 3000,
});
});
it("should handle merging different specs", () => {
const input = [
{
durations: [
{ spec: "test1.cy.js", duration: 1000 },
{ spec: "test2.cy.js", duration: 2000 },
],
},
{
durations: [
{ spec: "test1.cy.js", duration: 3000 },
{ spec: "test3.cy.js", duration: 1000 },
],
},
];
const result = averageTimings(input);
expect(result.durations).toHaveLength(3);
expect(result.durations).toContainEqual({
spec: "test1.cy.js",
duration: 2000,
});
expect(result.durations).toContainEqual({
spec: "test2.cy.js",
duration: 2000,
});
expect(result.durations).toContainEqual({
spec: "test3.cy.js",
duration: 1000,
});
});
});

View File

@@ -1,66 +1,24 @@
const TYPES = new Set([
"feat",
"fix",
"perf",
"docs",
"style",
"refactor",
"test",
"build",
"ci",
]);
/**
* @param title {string}
*/
function isConventionalTitle(title) {
const match = /^(?<type>\w+)\((?<rawScopes>[^)]+)\):/.exec(title);
if (!match) {
return false;
}
const [, type, rawScopes] = match;
const scopes = rawScopes
.split(",")
.map((scope) => scope.trim())
.filter(Boolean);
return TYPES.has(type) && scopes.length > 0;
}
exports.updateComment = updateComment;
/** @param {import('@types/github-script').AsyncFunctionArguments} AsyncFunctionArguments */
async function updateCommentForEmbedding({ github, context }) {
async function updateComment({ github, context }) {
const githubUsername = context.payload.sender.login;
const comment = `${CONVENTIONAL_COMMENT_IDENTIFIER}
@${githubUsername} You have modified embedding code. Please make sure the PR title follows [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) style.
Here are all the supported types:
const comment = `${CONVENTIONAL_COMMIT_REMINDER_COMMENT_IDENTIFIER}
@${githubUsername} You have modified embedding SDK code. Please make sure the PR title follows [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) style.
Here are all the supported types that will be shown in the changelog:
- \`feat\`
- \`fix\`
- \`perf\`
- \`docs\`
- \`style\`
- \`refactor\`
- \`test\`
- \`build\`
- \`ci\`
Please also make sure to include a scope.
These types \`docs\`, \`style\`, \`refactor\`, \`test\`, \`build\`, and \`ci\` are also encouraged for non-changelog related tasks.
Please also make sure to include \`sdk\` scope, otherwise, the changelog will not include this task.
For example, these are valid PR titles:
- \`feat(sdk-bundle): Add interactive dashboard component\`
- \`fix(sdk-package): Fixed MetabaseProviderPropsStore logic\`
- \`fix(sdk-package, embed-js): Fix API\``;
- \`feat(sdk): Add interactive dashboard component\`
- \`feat(sdk): Support theming pie chart\`
- \`fix(sdk): Fix static dashboard crash\``;
return updateComment(comment, { github, context });
}
/**
* @param comment {string}
* @param {import('@types/github-script').AsyncFunctionArguments} AsyncFunctionArguments
*/
async function updateComment(comment, { github, context }) {
const owner = context.repo.owner;
const repo = context.repo.repo;
const pullRequestNumber = context.issue.number;
@@ -91,19 +49,20 @@ async function updateComment(comment, { github, context }) {
}
}
const CONVENTIONAL_COMMENT_IDENTIFIER =
"<!---conventional comment identifier-->";
const CONVENTIONAL_COMMIT_REMINDER_COMMENT_IDENTIFIER =
"<!---conventional commit reminder comment ID-->";
/**
* @param comments {{body: string}[]}
* @typedef Comment
* @property {string} body
*
* @param {Comment[]} comments
*/
function getExistingConventionalCommitReminderComment(comments) {
for (const comment of comments) {
if (comment.body.startsWith(CONVENTIONAL_COMMENT_IDENTIFIER)) {
if (
comment.body.startsWith(CONVENTIONAL_COMMIT_REMINDER_COMMENT_IDENTIFIER)
) {
return comment;
}
}
}
exports.isConventionalTitle = isConventionalTitle;
exports.updateCommentForEmbedding = updateCommentForEmbedding;

View File

@@ -1,45 +0,0 @@
const { isConventionalTitle } = require("./utils");
describe("isConventionalTitle", () => {
it("returns true for a simple conventional commit", () => {
expect(isConventionalTitle("feat: add new feature")).toBe(false);
});
it("returns true for a commit with a scope", () => {
expect(isConventionalTitle("fix(api): correct API bug")).toBe(true);
});
it("returns true for a commit with multiple scopes, comma separated", () => {
expect(isConventionalTitle("feat(build,deps): update dependencies")).toBe(
true,
);
});
it("returns true for a commit with multiple scopes, comma separated with space", () => {
expect(isConventionalTitle("feat(build, deps): update dependencies")).toBe(
true,
);
});
it("returns false for a commit with round brackets in a message", () => {
expect(isConventionalTitle("feat: (something) update dependencies")).toBe(
false,
);
});
it("returns false for missing colon", () => {
expect(isConventionalTitle("feat add feature")).toBe(false);
});
it("returns false for invalid type", () => {
expect(isConventionalTitle("invalidType: some message")).toBe(false);
});
it("returns false for empty message", () => {
expect(isConventionalTitle("feat:")).toBe(false);
});
it("returns false for empty input", () => {
expect(isConventionalTitle("")).toBe(false);
});
});

View File

@@ -1,39 +0,0 @@
const extractChangedTimings = ({ oldTimings, newTimings }) => {
const oldTimingMap =
oldTimings.durations?.reduce((map, item) => {
if (item.spec) {
map[item.spec] = item.duration;
}
return map;
}, {}) || {};
const changedDurations = newTimings.durations
.filter((item) => {
if (!item || !item.spec || typeof item.duration !== "number") {
return false;
}
const newSpec = convertPathFormat(item.spec);
const oldDuration = oldTimingMap[newSpec] || oldTimingMap[item.spec];
// Include if it's a new spec or the duration has changed
return oldDuration === undefined || oldDuration !== item.duration;
})
.map((item) => ({
spec: convertPathFormat(item.spec),
duration: item.duration,
}));
return {
changedTimings: { durations: changedDurations },
hasChanges: changedDurations.length > 0,
};
};
// Convert cypress-split path format to timings.json path format
// from "e2e/test/scenarios/..." to "../test/scenarios/..."
function convertPathFormat(specPath) {
return specPath.startsWith("e2e/test/")
? specPath.replace("e2e/test/", "../test/")
: specPath;
}
module.exports = { extractChangedTimings };

View File

@@ -1,195 +0,0 @@
const { extractChangedTimings } = require("./extract-changed-timings");
describe("extractChangedTimings", () => {
it("should return empty durations when no changes", () => {
const oldTimings = {
durations: [
{ spec: "test1.cy.js", duration: 1000 },
{ spec: "test2.cy.js", duration: 2000 },
],
};
const newTimings = {
durations: [
{ spec: "test1.cy.js", duration: 1000 },
{ spec: "test2.cy.js", duration: 2000 },
],
};
const result = extractChangedTimings({ oldTimings, newTimings });
expect(result.changedTimings.durations).toHaveLength(0);
expect(result.hasChanges).toBe(false);
});
it("should detect new specs", () => {
const oldTimings = {
durations: [{ spec: "test1.cy.js", duration: 1000 }],
};
const newTimings = {
durations: [
{ spec: "test1.cy.js", duration: 1000 },
{ spec: "test2.cy.js", duration: 2000 },
],
};
const result = extractChangedTimings({ oldTimings, newTimings });
expect(result.changedTimings.durations).toHaveLength(1);
expect(result.changedTimings.durations[0]).toEqual({
spec: "test2.cy.js",
duration: 2000,
});
expect(result.hasChanges).toBe(true);
});
it("should detect duration changes", () => {
const oldTimings = {
durations: [
{ spec: "test1.cy.js", duration: 1000 },
{ spec: "test2.cy.js", duration: 2000 },
],
};
const newTimings = {
durations: [
{ spec: "test1.cy.js", duration: 1000 },
{ spec: "test2.cy.js", duration: 2200 },
],
};
const result = extractChangedTimings({ oldTimings, newTimings });
expect(result.changedTimings.durations).toHaveLength(1);
expect(result.changedTimings.durations[0]).toEqual({
spec: "test2.cy.js",
duration: 2200,
});
expect(result.hasChanges).toBe(true);
});
it("should handle missing old timings", () => {
const oldTimings = { durations: [] };
const newTimings = {
durations: [
{ spec: "test1.cy.js", duration: 1000 },
{ spec: "test2.cy.js", duration: 2000 },
],
};
const result = extractChangedTimings({ oldTimings, newTimings });
// All timings should be considered "changed" since old timings is empty
expect(result.changedTimings.durations).toHaveLength(2);
expect(result.changedTimings.durations).toContainEqual({
spec: "test1.cy.js",
duration: 1000,
});
expect(result.changedTimings.durations).toContainEqual({
spec: "test2.cy.js",
duration: 2000,
});
expect(result.hasChanges).toBe(true);
});
it("should handle invalid timing entries", () => {
const oldTimings = {
durations: [{ spec: "test1.cy.js", duration: 1000 }],
};
const newTimings = {
durations: [
{ spec: "test1.cy.js", duration: 1000 },
{ spec: "test2.cy.js", duration: "invalid" },
{ duration: 2000 }, // missing spec
{ spec: "test3.cy.js", duration: 3000 },
],
};
const result = extractChangedTimings({ oldTimings, newTimings });
// Only test3.cy.js should be included (new spec with valid data)
expect(result.changedTimings.durations).toHaveLength(1);
expect(result.changedTimings.durations[0]).toEqual({
spec: "test3.cy.js",
duration: 3000,
});
expect(result.hasChanges).toBe(true);
});
it("should handle timing data with different specs", () => {
const oldTimings = {
durations: [
{ spec: "test1.cy.js", duration: 1000 },
{ spec: "test2.cy.js", duration: 2000 },
],
};
const newTimings = {
durations: [
{ spec: "test1.cy.js", duration: 3000 },
{ spec: "test3.cy.js", duration: 1000 },
],
};
const result = extractChangedTimings({ oldTimings, newTimings });
expect(result.changedTimings.durations).toHaveLength(2);
expect(result.changedTimings.durations).toContainEqual({
spec: "test1.cy.js",
duration: 3000,
});
expect(result.changedTimings.durations).toContainEqual({
spec: "test3.cy.js",
duration: 1000,
});
expect(result.hasChanges).toBe(true);
});
it("should convert cypress-split paths to relative paths", () => {
const oldTimings = {
durations: [
{
spec: "../test/scenarios/dashboard/dashboard.cy.spec.js",
duration: 1000,
},
{
spec: "../test/scenarios/admin/databases.cy.spec.js",
duration: 2000,
},
],
};
const newTimings = {
durations: [
{
spec: "e2e/test/scenarios/dashboard/dashboard.cy.spec.js",
duration: 1200,
},
{
spec: "e2e/test/scenarios/admin/databases.cy.spec.js",
duration: 2000,
},
{
spec: "e2e/test/scenarios/collections/collections.cy.spec.js",
duration: 1500,
},
],
};
const result = extractChangedTimings({ oldTimings, newTimings });
expect(result.changedTimings.durations).toHaveLength(2);
expect(result.changedTimings.durations).toContainEqual({
spec: "../test/scenarios/dashboard/dashboard.cy.spec.js",
duration: 1200,
});
expect(result.changedTimings.durations).toContainEqual({
spec: "../test/scenarios/collections/collections.cy.spec.js",
duration: 1500,
});
expect(result.hasChanges).toBe(true);
});
});

View File

@@ -1,84 +0,0 @@
const githubSlackMap = require("../../release/github-slack-map.json");
async function getIssueData({ context, github, manualIssueNumber }) {
let issueNumber;
let issueData;
let prReviews = [];
const triggerType = context.eventName;
if (context.eventName === "issues") {
issueNumber = context.payload.issue.number;
issueData = context.payload.issue;
} else if (context.eventName === "workflow_dispatch") {
issueNumber = parseInt(manualIssueNumber);
const { data } = await github.rest.issues.get({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber,
});
issueData = data;
} else if (context.eventName === "pull_request_review") {
const prNumber = context.payload.pull_request.number;
const { data: prData } = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: prNumber,
});
// Extract issue number from branch name (e.g., "claude-fix/issue-12345-some-title")
const branchMatch = prData.head.ref.match(/claude-fix\/issue-(\d+)/);
if (!branchMatch) {
throw new Error(
`Could not extract issue number from branch name: ${prData.head.ref}`,
);
}
issueNumber = parseInt(branchMatch[1]);
const { data } = await github.rest.issues.get({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber,
});
issueData = data;
const { data: reviews } = await github.rest.pulls.listReviews({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: prNumber,
});
prReviews = reviews.filter(
(review) =>
review.state === "CHANGES_REQUESTED" &&
isMetabaseMember(review.user.login),
);
}
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber,
});
const employeeComments = comments.filter((comment) =>
isMetabaseMember(comment.user.login),
);
return {
issueNumber,
issueData,
employeeComments,
prReviews,
triggerType,
isIssueAuthorEmployee: isMetabaseMember(issueData.user.login),
};
}
function isMetabaseMember(username) {
return username in githubSlackMap;
}
module.exports = {
getIssueData,
};

View File

@@ -1,30 +0,0 @@
#!/bin/bash
set -e
# Parse command line arguments
SKIP_START=false
while [[ $# -gt 0 ]]; do
case $1 in
--skip-start)
SKIP_START=true
shift
;;
*)
echo "Unknown option: $1"
echo "Usage: $0 [--skip-start]"
exit 1
;;
esac
done
echo "Cloning python-runner repository..."
rm -r python-runner
git clone --depth 1 https://${METABASE_AUTOMATION_USER_TOKEN}@github.com/metabase/python-runner-container.git python-runner
if [ "$SKIP_START" = false ]; then
echo "Starting python-runner..."
cd python-runner
make run-ci
cd ..
fi

View File

@@ -35,21 +35,21 @@ jobs:
build-static-viz: false
test-args: >-
:only '"enterprise/backend/test"'
exclude-tag: ':mb/driver-tests :mb/transforms-python-test'
exclude-tag: ':mb/driver-tests'
- name: EE App DB Tests (Part 1)
build-static-viz: true
test-args: >-
:only '"test"'
:partition/total 2
:partition/index 0
exclude-tag: ':mb/driver-tests :mb/transforms-python-test'
exclude-tag: ':mb/driver-tests'
- name: EE App DB Tests (Part 2)
build-static-viz: true
test-args: >-
:only '"test"'
:partition/total 2
:partition/index 1
exclude-tag: ':mb/driver-tests :mb/transforms-python-test'
exclude-tag: ':mb/driver-tests'
services:
mysql:
@@ -147,21 +147,21 @@ jobs:
build-static-viz: false
test-args: >-
:only '"enterprise/backend/test"'
exclude-tag: ':mb/driver-tests :mb/transforms-python-test'
exclude-tag: ':mb/driver-tests'
- name: EE App DB Tests (Part 1)
build-static-viz: true
test-args: >-
:only '"test"'
:partition/total 2
:partition/index 0
exclude-tag: ':mb/driver-tests :mb/transforms-python-test'
exclude-tag: ':mb/driver-tests'
- name: EE App DB Tests (Part 2)
build-static-viz: true
test-args: >-
:only '"test"'
:partition/total 2
:partition/index 1
exclude-tag: ':mb/driver-tests :mb/transforms-python-test'
exclude-tag: ':mb/driver-tests'
services:
mysql:
@@ -259,13 +259,11 @@ jobs:
build-static-viz: false
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
exclude-tag: ':mb/transforms-python-test'
- name: Enterprise Tests
build-static-viz: false
test-args: >-
:only '"enterprise/backend/test"'
exclude-tag: ':mb/driver-tests :mb/transforms-python-test'
exclude-tag: ':mb/driver-tests'
- name: EE App DB Tests (Part 1)
build-static-viz: true
test-args: >-
@@ -273,14 +271,14 @@ jobs:
:exclude-tags [:mb/driver-tests]
:partition/total 2
:partition/index 0
exclude-tag: ':mb/driver-tests :mb/transforms-python-test'
exclude-tag: ':mb/driver-tests'
- name: EE App DB Tests (Part 2)
build-static-viz: true
test-args: >-
:only '"test"'
:partition/total 2
:partition/index 1
exclude-tag: ':mb/driver-tests :mb/transforms-python-test'
exclude-tag: ':mb/driver-tests'
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
env:
# Only run old migrations tests on pushes to master or release branches. All other branches should skip tests

View File

@@ -6,19 +6,11 @@ on:
jobs:
create-backport:
if:
| # the PR must have a backport label when it's merged, or be labeled with a backport label after merge
if: | # the PR must be merged and have a backport label
github.event.pull_request.merged == true && (
github.event.action == 'closed' && (
contains(github.event.pull_request.labels.*.name, 'backport') ||
contains(github.event.pull_request.labels.*.name, 'double-backport') ||
contains(github.event.pull_request.labels.*.name, 'triple-backport')
) ||
github.event.action == 'labeled' && (
github.event.label.name == 'backport' ||
github.event.label.name == 'double-backport' ||
github.event.label.name == 'triple-backport'
)
contains(github.event.pull_request.labels.*.name, 'backport') ||
contains(github.event.pull_request.labels.*.name, 'double-backport') ||
contains(github.event.pull_request.labels.*.name, 'triple-backport')
)
runs-on: ubuntu-latest
steps:

View File

@@ -154,7 +154,7 @@ jobs:
fail-on-error: false
- name: Upload Logs on Failure
if: always() && !success()
if: failure()
uses: actions/upload-artifact@v4
with:
name: test-logs-${{ matrix.java-version }}-${{ matrix.job.edition }}-${{ github.job }}

View File

@@ -1,14 +0,0 @@
name: Block Master Branch PRs
on:
pull_request:
types: [opened, synchronize]
jobs:
check-source-branch:
runs-on: ubuntu-latest
steps:
- name: Block master branch PRs
if: github.head_ref == 'master'
run: |
echo "PRs from `master` branches are not allowed, please use a new branch name for your PR."
exit 1

View File

@@ -1,327 +0,0 @@
name: "Claude Code Bug Fix"
on:
issues:
types:
- labeled
pull_request_review:
types:
- submitted
workflow_dispatch:
inputs:
issue_number:
description: "Issue number to fix"
required: true
type: string
override_employee_check:
description: "Override employee check for issue creator"
required: false
default: false
type: boolean
jobs:
auto-fix-bug:
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: write
issues: write
pull-requests: write
id-token: write
if: |
(github.event_name == 'issues' && github.event.label.name == 'Claude Bug Fix' && github.event.issue.state == 'open') ||
(github.event_name == 'workflow_dispatch') ||
(github.event_name == 'pull_request_review' && github.event.review.state == 'changes_requested' && startsWith(github.event.pull_request.head.ref, 'claude-fix/'))
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }}
- name: Get issue data and check employee status
id: get_issue
uses: actions/github-script@v7
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { getIssueData } = require('${{ github.workspace }}/.github/scripts/get-issue-data.js');
try {
const result = await getIssueData({
context,
github,
manualIssueNumber: '${{ github.event.inputs.issue_number }}'
});
const { issueNumber, issueData, employeeComments, prReviews, triggerType, isIssueAuthorEmployee } = result;
// Check employee status and fail if not authorized
if (!result.isIssueAuthorEmployee && '${{ github.event.inputs.override_employee_check }}' !== 'true') {
core.setFailed(`User ${result.issueData.user.login} is not a Metabase employee. Skipping bug fix automation.`);
}
core.setOutput('issue_number', issueNumber);
core.setOutput('issue_data', JSON.stringify(issueData));
core.setOutput('issue_comments', JSON.stringify(employeeComments));
core.setOutput('pr_reviews', JSON.stringify(prReviews));
core.setOutput('trigger_type', triggerType);
} catch (error) {
core.setFailed(error.message);
}
- name: Set up git configuration
run: |
git config --global user.name "Claude Code[bot]"
git config --global user.email "noreply@anthropic.com"
- name: Prepare issue context and bug fix prompt
id: prepare_prompt
uses: actions/github-script@v7
env:
ISSUE_DATA: ${{ steps.get_issue.outputs.issue_data }}
ISSUE_COMMENTS: ${{ steps.get_issue.outputs.issue_comments }}
PR_REVIEWS: ${{ steps.get_issue.outputs.pr_reviews }}
TRIGGER_TYPE: ${{ steps.get_issue.outputs.trigger_type }}
with:
script: |
const issueData = JSON.parse(process.env.ISSUE_DATA);
const issueComments = JSON.parse(process.env.ISSUE_COMMENTS);
const prReviews = JSON.parse(process.env.PR_REVIEWS);
const triggerType = process.env.TRIGGER_TYPE;
const issueNumber = '${{ steps.get_issue.outputs.issue_number }}';
// Convert issue title to kebab case and create branch name
const issueTitle = issueData.title || 'untitled';
const kebabTitle = issueTitle
.toLowerCase()
.replace(/[^a-z0-9]+/g, '-')
.replace(/^-+|-+$/g, '')
.replace(/-+/g, '-')
.substring(0, 50);
const branchName = `claude-fix/issue-${issueNumber}-${kebabTitle}`;
// Format data for bash environment
const formattedComments = issueComments.length > 0
? issueComments.map(comment =>
`**Comment by @${comment.user.login} on ${comment.created_at}:**\n${comment.body}`
).join('\n\n---\n\n')
: '';
const formattedReviews = prReviews.length > 0
? prReviews.map(review =>
`**Review by @${review.user.login} on ${review.submitted_at} (Changes Requested):**\n${review.body || 'No review comment provided'}`
).join('\n\n---\n\n')
: '';
core.exportVariable('BRANCH_NAME', branchName);
core.exportVariable('ISSUE_NUMBER', issueNumber);
core.exportVariable('ISSUE_TITLE', issueData.title);
core.exportVariable('ISSUE_AUTHOR', issueData.user.login);
core.exportVariable('ISSUE_CREATED', issueData.created_at);
core.exportVariable('ISSUE_BODY', issueData.body || 'No description provided');
core.exportVariable('ISSUE_COMMENTS_FORMATTED', formattedComments);
core.exportVariable('PR_REVIEWS_FORMATTED', formattedReviews);
core.exportVariable('TRIGGER_TYPE', triggerType);
- name: Generate bug fix prompt
run: |
cat << 'EOF' >> $GITHUB_ENV
BUG_FIX_PROMPT<<PROMPT_END
You are helping to fix a bug reported in GitHub issue #${{ env.ISSUE_NUMBER }}.
## Bug Report Details
**Title:** ${{ env.ISSUE_TITLE }}
**Reporter:** ${{ env.ISSUE_AUTHOR }}
**Created:** ${{ env.ISSUE_CREATED }}
**Description:**
${{ env.ISSUE_BODY }}
**Comments:**
${{ env.ISSUE_COMMENTS_FORMATTED }}
EOF
# Add review feedback section only if this is a review trigger
if [ "${{ env.TRIGGER_TYPE }}" = "pull_request_review" ]; then
cat << 'EOF' >> $GITHUB_ENV
## Previous Fix Attempt - Review Feedback (MUST ADDRESS)
Your previous automated fix attempt received the following feedback that needs to be addressed:
${{ env.PR_REVIEWS_FORMATTED }}
EOF
fi
# Add the rest of the prompt (always the same)
cat << 'EOF' >> $GITHUB_ENV
## Instructions
1. **Analyze the issue description and comments** - Understand what the issue is describing
a. Pay particular attention to reproduction steps and if there's an agreed upon fix.
2. **Search the codebase** - Use Grep and Glob tools to find relevant files
3. **Implement a fix** - Follow existing code patterns and conventions
4. **Test the fix** - Run appropriate tests using the commands in CLAUDE.md:
- JavaScript/TypeScript: `yarn test-unit path/to/file.unit.spec.js`
- Clojure: `clojure -X:dev:test :only namespace/test-name`
- Lint: `yarn lint-eslint-pure` for JS/TS, `./bin/mage kondo <file>` for Clojure
5. **Verify the fix** - Ensure tests pass and code follows conventions
## Important Guidelines
- Follow the development patterns in CLAUDE.md
- Write failing tests first, then fix them
- Work in small, testable increments
- Don't commit changes - leave that for the workflow
- IMPORTANT: Focus only on fixing the reported bug, don't make unrelated changes
## Scratchpad Directory
A `.claude-scratchpad/` directory has been created for your temporary files:
- Use this directory for ALL temporary verification files. However, prefer using production test files rather than adhoc ones.
- Only modify actual production code in the main repository directories
- The scratchpad will be cleaned up automatically - nothing from it will be committed
Start by understanding the bug and then systematically work toward a solution.
PROMPT_END
EOF
- name: Determine branch strategy
run: |
BRANCH_NAME="${{ env.BRANCH_NAME }}"
if git branch -r --list "origin/${BRANCH_NAME}" | grep -q .; then
echo "Branch already exists: $BRANCH_NAME"
echo "Will push to existing branch and skip PR creation"
echo "BRANCH_EXISTS=true" >> $GITHUB_ENV
else
echo "Creating new branch: $BRANCH_NAME"
echo "Will create new PR"
echo "BRANCH_EXISTS=false" >> $GITHUB_ENV
fi
# Export the branch name
echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_ENV
- name: Create scratchpad directory for Claude
run: |
# Create a scratchpad directory for Claude to use for temporary files
mkdir -p .claude-scratchpad
echo "Scratchpad directory created for Claude's temporary verification files"
- name: Generate GitHub App Token for Claude
uses: actions/create-github-app-token@v1
id: claude-app-token
with:
app-id: ${{ secrets.METABASE_BOT_APP_ID }}
private-key: ${{ secrets.METABASE_BOT_APP_PRIVATE_KEY }}
- name: Generate Bug Fix with Claude Code
uses: anthropics/claude-code-action@beta
with:
mode: agent
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
github_token: ${{ steps.claude-app-token.outputs.token }}
direct_prompt: ${{ env.BUG_FIX_PROMPT }}
timeout_minutes: "25"
allowed_tools: "Bash(yarn test-unit*),Bash(yarn lint-eslint-pure),Bash(./bin/mage*),Bash(clojure*),Bash(find*),Bash(ls*),Bash(grep*),Write,Read,Glob,Grep,Edit,MultiEdit"
disallowed_tools: "Bash(git commit*),Bash(git push*),Bash(gh*)"
- name: Clean up scratchpad directory
run: |
# Remove the scratchpad directory so its contents don't get committed
rm -rf .claude-scratchpad/
# Remove output file generated by Claude Code action
rm -f output.txt
echo "Scratchpad directory and action output files cleaned up"
- name: Check for changes and analysis
id: check_changes
run: |
# Now check for actual code changes (excluding analysis file)
if [ -n "$(git status --porcelain)" ]; then
echo "has_changes=true" >> $GITHUB_OUTPUT
echo "Changes detected:"
git status --short
else
echo "has_changes=false" >> $GITHUB_OUTPUT
echo "No changes made by Claude."
fi
- name: Get issue team labels and prepare reviewers
if: steps.check_changes.outputs.has_changes == 'true'
id: prepare_reviewers
uses: actions/github-script@v7
env:
ISSUE_DATA: ${{ steps.get_issue.outputs.issue_data }}
with:
script: |
const issueData = JSON.parse(process.env.ISSUE_DATA);
const issueLabels = issueData.labels.map(label => label.name);
const teamLabels = issueLabels.filter(label => label.startsWith('.Team/'));
const issueCreator = issueData.user.login;
core.setOutput('team_labels', teamLabels.join(','));
core.setOutput('reviewers', issueCreator);
- name: Create Pull Request
if: steps.check_changes.outputs.has_changes == 'true'
uses: peter-evans/create-pull-request@v6
with:
token: ${{ secrets.GITHUB_TOKEN }}
branch: ${{ env.BRANCH_NAME }}
base: master
commit-message: |
Fix: ${{ env.ISSUE_TITLE }}
Fixes #${{ env.ISSUE_NUMBER }}
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
committer: "Claude Code[bot] <noreply@anthropic.com>"
author: "Claude Code[bot] <noreply@anthropic.com>"
title: "Fix: ${{ env.ISSUE_TITLE }}"
body: |
This PR automatically fixes the bug reported in #${{ env.ISSUE_NUMBER }}.
## Bug Report
**Original Issue:** #${{ env.ISSUE_NUMBER }}
**Reporter:** @${{ env.ISSUE_AUTHOR }}
**Issue Title:** ${{ env.ISSUE_TITLE }}
## Automated Fix
🤖 This fix was generated automatically by Claude Code based on the issue description.
**Please review carefully:**
- [ ] The fix addresses the reported issue
- [ ] Tests pass and code follows conventions
- [ ] No unrelated changes were introduced
- [ ] The solution is appropriate and maintainable
## Next Steps
- Review the changes and test thoroughly
- If the fix looks good, approve and merge
- If issues are found, either fix them or close this PR
---
Closes #${{ env.ISSUE_NUMBER }}
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
labels: |
${{ steps.prepare_reviewers.outputs.team_labels }}
automated-fix
ci skip
reviewers: ${{ steps.prepare_reviewers.outputs.reviewers }}
- name: Log existing branch update
if: steps.check_changes.outputs.has_changes == 'true' && env.BRANCH_EXISTS == 'true'
uses: actions/github-script@v7
with:
script: |
console.log('🔄 Updated existing branch: ${{ env.BRANCH_NAME }}');
console.log('The changes have been pushed to the existing PR for this issue.');
- name: Log analysis for no changes
if: steps.check_changes.outputs.has_changes == 'false'
uses: actions/github-script@v7
with:
script: |
console.log('🤖 No code changes were made by Claude.');

View File

@@ -1,126 +0,0 @@
name: Containerize Uberjar
on:
workflow_call:
workflow_dispatch:
jobs:
get-container-info:
runs-on: ubuntu-22.04
timeout-minutes: 5
name: Get container info
outputs:
editions: ${{ steps.ee-info.outputs.editions }}
ee: ${{ toJson(fromJson(steps.build-matrix.outputs.result).ee) }}
oss: ${{ toJson(fromJson(steps.build-matrix.outputs.result).oss) }}
steps:
- name: Check out the code
uses: actions/checkout@v4
- name: Get EE Docker info
id: ee-info
uses: ./.github/actions/get-docker-info
with:
edition: ee
- name: Get OSS Docker info
id: oss-info
uses: ./.github/actions/get-docker-info
with:
edition: oss
- name: Build matrix info
id: build-matrix
uses: actions/github-script@v7
with:
script: | # js
const editions = JSON.parse('${{ steps.ee-info.outputs.editions }}');
const result = {
editions: editions,
ee: {
repo: '${{ steps.ee-info.outputs.repo }}',
tag: '${{ steps.ee-info.outputs.tag }}'
},
oss: {
repo: '${{ steps.oss-info.outputs.repo }}',
tag: '${{ steps.oss-info.outputs.tag }}'
}
};
console.log('Build matrix:', result);
return result;
containerize:
needs: [get-container-info]
if: needs.get-container-info.result == 'success'
strategy:
matrix:
edition: ${{ fromJson(needs.get-container-info.outputs.editions) }}
uses: ./.github/workflows/containerize-jar.yml
secrets: inherit
with:
artifact-name: metabase-${{ matrix.edition }}-${{ github.event.inputs.commit || github.event.pull_request.head.sha || github.sha }}-uberjar
commit: ${{ github.event.inputs.commit || github.sha }}
repo: ${{ fromJson(needs.get-container-info.outputs[matrix.edition]).repo }}
tag: ${{ fromJson(needs.get-container-info.outputs[matrix.edition]).tag }}
release: "false"
build-args: |
GIT_COMMIT_SHA=${{ github.event.inputs.commit || github.sha }}
run-trivy:
needs: [get-container-info, containerize]
strategy:
matrix:
edition: ${{ fromJson(needs.get-container-info.outputs.editions) }}
runs-on: ubuntu-22.04
name: Run Trivy vulnerability scanner
steps:
- name: Check out the code
uses: actions/checkout@v4
with:
ref: ${{ github.event.inputs.commit || github.sha }}
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@0.29.0
env:
TRIVY_OFFLINE_SCAN: true
repo: ${{ fromJson(needs.get-container-info.outputs[matrix.edition]).repo }}
tag: ${{ fromJson(needs.get-container-info.outputs[matrix.edition]).tag }}
with:
image-ref: docker.io/${{ env.repo }}:${{ env.tag }}
format: sarif
output: trivy-results.sarif
version: "v0.57.1"
- name: Upload Trivy scan results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: "trivy-results.sarif"
post-comment:
needs: [get-container-info, containerize]
if: github.event_name == 'pull_request'
runs-on: ubuntu-22.04
name: Post PR comment
steps:
- name: Check out the code
uses: actions/checkout@v4
- name: Get Docker info for comment
id: docker-info
uses: ./.github/actions/get-docker-info
with:
branch: ${{ github.head_ref || github.ref_name }}
edition: ee
- name: Post PR comment
uses: marocchino/sticky-pull-request-comment@v2
with:
header: docker-build-status
message: |
✅ Docker build completed for commit ${{ github.event.pull_request.head.sha || github.sha }}
The uberjar has been built and containerized successfully.
**Branch:** `${{ github.head_ref || github.ref_name }}`
**Docker Image:** `${{ steps.docker-info.outputs.full_image }}`
**Docker Hub:** [View on Docker Hub](https://hub.docker.com/r/${{ steps.docker-info.outputs.repo }}/tags?name=${{ steps.docker-info.outputs.tag }})
**Triggered by:** @${{ github.actor }}
Pull the image:
```bash
docker pull ${{ steps.docker-info.outputs.full_image }}
```

View File

@@ -4,53 +4,39 @@ on:
pull_request:
branches:
- master
paths:
- enterprise/frontend/src/embedding-sdk/**
jobs:
files-changed:
name: Check which files changed
runs-on: ubuntu-22.04
timeout-minutes: 3
outputs:
frontend_embedding_sources: ${{ steps.changes.outputs.frontend_embedding_sources }}
steps:
- uses: actions/checkout@v4
- name: Test which files changed
uses: dorny/paths-filter@v3.0.0
id: changes
with:
token: ${{ github.token }}
filters: .github/file-paths.yaml
pr-title-reminder:
runs-on: ubuntu-22.04
timeout-minutes: 5
permissions:
pull-requests: write
needs: [ files-changed ]
if: |
!cancelled() &&
needs.files-changed.outputs.frontend_embedding_sources == 'true'
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.head_ref }}
- name: Check if the PR title is already following Conventional Commits style
uses: actions/github-script@v7
id: is-conventional-commit
with:
result-encoding: string
script: | # js
const { isConventionalTitle } = require(".github/scripts/conventional-commit-pr-title-reminder/utils.js")
const pullRequestTitle = context.payload.pull_request.title
const conventionalCommitRegex = /^(?<type>\w+)\((?<scope>\w+)\):/
const match = pullRequestTitle.match(conventionalCommitRegex)
if (match) {
const { type, scope } = match.groups
return ['feat', 'fix', 'perf', 'docs', 'style', 'refactor', 'test', 'build', 'ci'].includes(type) && scope === 'sdk'
}
return isConventionalTitle(pullRequestTitle)
- name: Checkout repository so `gh` CLI works
uses: actions/checkout@v4
with:
ref: ${{ github.head_ref }}
- name: Remind PR authors to use conventional commit in the title
if: ${{ steps.is-conventional-commit.outputs.result != 'true' }}
uses: actions/github-script@v7
with:
script: | #js
const { updateCommentForEmbedding } = require(".github/scripts/conventional-commit-pr-title-reminder/utils.js")
await updateCommentForEmbedding({github, context})
script: |
const { updateComment } = require(".github/scripts/conventional-commit-pr-title-reminder/utils.js")
await updateComment({github, context})

View File

@@ -1,182 +0,0 @@
# .github/workflows/reusable-doc-update.yml
name: Reusable - Update Documentation
# This workflow is designed to be called by other workflows.
on:
workflow_call:
inputs:
target_branch:
description: "The branch to check out and create a PR against"
required: true
type: string
m2_cache_key_suffix:
description: "A unique suffix for the Maven cache key"
required: true
type: string
secrets:
automation_token:
description: "A token with permissions to create branches and PRs"
required: true
permissions:
contents: write
pull-requests: write
jobs:
update-docs:
name: Update documentation for ${{ inputs.target_branch }}
runs-on: ubuntu-22.04
timeout-minutes: 30
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.automation_token }}
ref: ${{ inputs.target_branch }}
- name: Prepare backend environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: "auto-update-docs-${{ inputs.m2_cache_key_suffix }}"
- name: Generate environment variables documentation
id: generate_env_docs
run: |
set -e # Exit on any error
# Ensure we're on the target branch for documentation generation
git checkout ${{ inputs.target_branch }}
echo "Generating environment variables documentation from branch: ${{ inputs.target_branch }}"
clojure -M:ee:doc environment-variables-documentation
continue-on-error: false
- name: Generate config template documentation
id: generate_config_docs
run: |
set -e # Exit on any error
echo "Generating config template documentation..."
clojure -M:ee:doc config-template
continue-on-error: false
- name: Generate API documentation
id: generate_api_docs
run: |
set -e # Exit on any error
echo "Generating API documentation..."
clojure -M:ee:doc api-documentation
continue-on-error: false
- name: Check for changes
id: check_changes
run: |
if git diff --quiet docs/; then
echo "changes=false" >> $GITHUB_OUTPUT
echo "No changes detected in docs directory"
else
echo "changes=true" >> $GITHUB_OUTPUT
echo "Changes detected in docs directory"
fi
- name: Create branch and commit changes
id: commit_and_push
if: steps.check_changes.outputs.changes == 'true'
run: |
git config user.name github-actions
git config user.email github-actions@github.com
BRANCH_NAME="docs/auto-update-${{ inputs.target_branch }}-$(date +%Y-%m-%d-%H%M%S)"
echo "Creating branch: $BRANCH_NAME"
# Create and switch to new branch from current target branch
git checkout -b "$BRANCH_NAME"
# Stage and commit changes
git add docs/
if git diff --cached --quiet; then
echo "No changes to commit"
exit 1
fi
git commit -m "Update generated docs (${{ inputs.target_branch }})" --no-verify
git push -u origin "$BRANCH_NAME"
echo "branch_name=$BRANCH_NAME" >> $GITHUB_OUTPUT
echo "Successfully created and pushed branch"
- name: Create pull request
id: create_pr
if: steps.check_changes.outputs.changes == 'true'
env:
GITHUB_TOKEN: ${{ secrets.automation_token }}
GH_REPO: ${{ github.repository }}
run: |
CHANGES_SUMMARY=$(git diff --name-only HEAD~1 docs/ 2>/dev/null | head -10 | sed 's/^/- /' || echo "- No specific files detected")
TOTAL_CHANGES=$(git diff --name-only HEAD~1 docs/ 2>/dev/null | wc -l || echo "0")
BRANCH_NAME="${{ steps.commit_and_push.outputs.branch_name }}"
echo "Creating PR with branch: $BRANCH_NAME"
echo "Changes detected: $TOTAL_CHANGES files"
for i in {1..3}; do
echo "Attempt $i of 3..."
if gh pr create \
--base "${{ inputs.target_branch }}" \
--head "$BRANCH_NAME" \
--title "Auto-update documentation for ${{ inputs.target_branch }}" \
--body "This PR contains automatically generated documentation updates.
## Changes detected
The following documentation files were updated:
$CHANGES_SUMMARY
$([ "$TOTAL_CHANGES" -gt 10 ] && echo "- ... and $((TOTAL_CHANGES - 10)) more files")
## Generation commands
Documentation was generated using:
- \`clojure -M:ee:doc environment-variables-documentation\`
- \`clojure -M:ee:doc config-template\`
- \`clojure -M:ee:doc api-documentation\`
Please review the changes and merge if they look correct." \
--label "Type:Documentation" \
--label "no-backport" \
--reviewer "metabase/tech-writers"; then
echo "PR created successfully on attempt $i"
break
else
echo "PR creation failed on attempt $i"
if [ $i -eq 3 ]; then
echo "All attempts failed. Exiting with error."
exit 1
fi
echo "Waiting 10 seconds before retry..."
sleep 10
fi
done
- name: Skip PR creation
if: steps.check_changes.outputs.changes == 'false'
run: |
echo "No documentation changes detected, skipping PR creation"
- name: Workflow summary
if: always()
run: |
if [ "${{ steps.check_changes.outputs.changes }}" == "true" ]; then
if [ "${{ steps.commit_and_push.outcome }}" == "success" ]; then
if [ "${{ steps.create_pr.outcome }}" == "success" ]; then
echo "## ✅ Documentation Updated Successfully" >> $GITHUB_STEP_SUMMARY
echo "- Branch: \`${{ steps.commit_and_push.outputs.branch_name }}\`" >> $GITHUB_STEP_SUMMARY
echo "- PR created targeting **${{ inputs.target_branch }}**" >> $GITHUB_STEP_SUMMARY
else
echo "## ⚠️ Branch Created but PR Failed" >> $GITHUB_STEP_SUMMARY
echo "- Branch: \`${{ steps.commit_and_push.outputs.branch_name }}\`" >> $GITHUB_STEP_SUMMARY
echo "- Manual PR creation required" >> $GITHUB_STEP_SUMMARY
fi
else
echo "## ❌ Failed to Create Branch" >> $GITHUB_STEP_SUMMARY
echo "- Check workflow logs for details" >> $GITHUB_STEP_SUMMARY
fi
else
echo "## ℹ️ No Changes Detected" >> $GITHUB_STEP_SUMMARY
echo "- Documentation is up to date on **${{ inputs.target_branch }}**" >> $GITHUB_STEP_SUMMARY
fi

View File

@@ -1,36 +0,0 @@
name: Check doc links
on:
pull_request:
paths:
- "docs/**"
workflow_dispatch:
jobs:
link-checker:
name: Check documentation links
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
sparse-checkout: |
docs
.lychee
- name: Cache lychee
uses: actions/cache@v3
with:
path: ~/.cache/lychee
key: lychee-${{ runner.os }}-${{ hashFiles('.lychee/config.toml') }}
- name: Run link checker
id: link-check
uses: lycheeverse/lychee-action@v2
with:
fail: true
args: docs --config ./.lychee/config.toml --offline
format: markdown

View File

@@ -1,16 +0,0 @@
name: Auto-update documentation (master)
on:
schedule:
- cron: "0 1 * * 1-5" # Every Monday through Friday at 1 AM UTC
workflow_dispatch:
jobs:
update-master-docs:
name: Call documentation update for master
uses: ./.github/workflows/docs-generate-base.yml
with:
target_branch: "master"
m2_cache_key_suffix: "master"
secrets:
automation_token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}

View File

@@ -1,16 +0,0 @@
name: Auto-update documentation (release)
on:
schedule:
- cron: "0 2 * * 1-5" # Every Monday through Friday at 2 AM UTC
workflow_dispatch:
jobs:
update-release-docs:
name: Call documentation update for release branch
uses: ./.github/workflows/docs-generate-base.yml
with:
target_branch: "release-x.56.x"
m2_cache_key_suffix: "release"
secrets:
automation_token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}

View File

@@ -689,51 +689,18 @@ jobs:
junit-name: 'be-tests-starburst-ee'
test-args: ":only ${{ inputs.test }} :times ${{ inputs.times }}"
login-to-ecr:
if: inputs.driver == 'Vertica'
runs-on: ubuntu-22.04
timeout-minutes: 15
permissions:
id-token: write
contents: read
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.CI_IAM_ROLE }}
role-session-name: GitHub_to_AWS_via_FederatedOIDC
aws-region: us-east-1
- name: Get ECR password
id: login-ecr
run: |
PASSWORD=$(aws ecr get-login-password --region us-east-1)
REGISTRY="$(aws sts get-caller-identity --query Account --output text).dkr.ecr.us-east-1.amazonaws.com"
echo "docker_username=AWS" >> $GITHUB_OUTPUT
echo "docker_password=$PASSWORD" >> $GITHUB_OUTPUT
echo "registry=$REGISTRY" >> $GITHUB_OUTPUT
outputs:
registry: ${{ steps.login-ecr.outputs.registry }}
docker_username: ${{ steps.login-ecr.outputs.docker_username }}
docker_password: ${{ steps.login-ecr.outputs.docker_password }}
be-tests-vertica-ee:
if: inputs.driver == 'Vertica'
runs-on: ubuntu-22.04
needs: login-to-ecr
timeout-minutes: 90
env:
CI: 'true'
DRIVERS: vertica
services:
vertica:
image: ${{ needs.login-to-ecr.outputs.registry }}/vertica-ce:25.3.0-0
image: vertica/vertica-ce:12.0.2-0
ports:
- "5433:5433"
credentials:
username: ${{ needs.login-to-ecr.outputs.docker_username }}
password: ${{ needs.login-to-ecr.outputs.docker_password }}
steps:
- uses: actions/checkout@v4
- name: Make plugins directory

View File

@@ -44,11 +44,11 @@ jobs:
echo "A relevant team label is present. Tier 3 driver tests will run."
else
echo "Checking for changes in relevant folders..."
CHANGED_FILES=$(git diff --name-only $(git merge-base ${{ github.event.pull_request.base.sha }} ${{ github.sha }}) ${{ github.sha }})
CHANGED_FILES=$(git diff --name-only ${{ github.event.pull_request.base.sha }} ${{ github.sha }})
# Initialize an empty array to store folders with changes
CHANGED_FOLDERS=()
for folder in "src/metabase/lib" "src/metabase/query_processor" "modules/drivers" "src/metabase/driver"; do
for folder in "src/metabase/lib" "src/metabase/query_processor" "modules/drivers"; do
if echo "$CHANGED_FILES" | grep -q "$folder"; then
CHANGED_FOLDERS+=("$folder")
fi
@@ -90,7 +90,6 @@ jobs:
junit-name: 'be-tests-athena-ee'
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -107,19 +106,6 @@ jobs:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
job:
- name: Driver Tests
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
# DS 2025-09-26 temporarily disabled while we diagnose flakes
# - name: Transforms Python Tests
# test-args: >-
# :only-tags [:mb/transforms-python-test]
# needs-python-runner: true
env:
CI: 'true'
DRIVERS: bigquery-cloud-sdk
@@ -130,19 +116,15 @@ jobs:
MB_BIGQUERY_TEST_REFRESH_TOKEN: ${{ secrets.MB_BIGQUERY_TEST_REFRESH_TOKEN }}
MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON: ${{ secrets.MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON }}
MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON_ROUTING: ${{ secrets.MB_BIGQUERY_CLOUD_SDK_TEST_SERVICE_ACCOUNT_JSON_ROUTING }}
name: BigQuery ${{ matrix.job.name }}
name: BigQuery
steps:
- uses: actions/checkout@v4
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
env:
METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
run: ./.github/scripts/setup-python-runner.sh
- name: Test BigQuery Cloud SDK driver
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-bigquery-cloud-sdk-ee'
test-args: ${{ matrix.job.test-args }}
test-args: >-
:only-tags [:mb/driver-tests]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -159,18 +141,6 @@ jobs:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
job:
- name: Driver Tests
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
- name: Transforms Python Tests
test-args: >-
:only-tags [:mb/transforms-python-test]
needs-python-runner: true
env:
CI: 'true'
DRIVERS: clickhouse
@@ -178,14 +148,9 @@ jobs:
MB_CLICKHOUSE_TEST_PORT: 8123
MB_CLICKHOUSE_TEST_OLD_PORT: 8124
MB_CLICKHOUSE_TEST_NGINX_PORT: 8127
name: Clickhouse ${{ matrix.job.name }}
name: Clickhouse
steps:
- uses: actions/checkout@v4
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
env:
METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
run: ./.github/scripts/setup-python-runner.sh
- name: Add ClickHouse TLS instance to /etc/hosts
run: |
sudo echo "127.0.0.1 server.clickhouseconnect.test" | sudo tee -a /etc/hosts
@@ -205,7 +170,8 @@ jobs:
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-clickhouse-ee'
test-args: ${{ matrix.job.test-args }}
test-args: >-
:only-tags [:mb/driver-tests]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -242,7 +208,6 @@ jobs:
junit-name: 'be-tests-druid-ee'
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -279,7 +244,6 @@ jobs:
junit-name: 'be-tests-druid-jdbc-ee'
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -316,7 +280,6 @@ jobs:
junit-name: 'be-tests-databricks-ee'
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -354,7 +317,6 @@ jobs:
junit-name: 'be-tests-databricks-multi-catalog-ee'
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -376,17 +338,12 @@ jobs:
name: H2
steps:
- uses: actions/checkout@v4
- name: Setup python-runner
env:
METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
run: ./.github/scripts/setup-python-runner.sh
- name: Test H2 driver
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-h2'
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -431,11 +388,6 @@ jobs:
- name: Driver Tests
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
- name: Transforms Python Tests
test-args: >-
:only-tags [:mb/transforms-python-test]
needs-python-runner: true
services:
mysql:
@@ -490,11 +442,6 @@ jobs:
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
steps:
- uses: actions/checkout@v4
- name: Setup python-runner
#if: ${{ matrix.job.needs-python-runner == true }}
env:
METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
run: ./.github/scripts/setup-python-runner.sh
- name: Test ${{ matrix.version.name }}
uses: ./.github/actions/test-driver
with:
@@ -529,15 +476,6 @@ jobs:
- name: MongoDB Latest
junit-name: be-tests-mongo-latest-ee
image: mongo:latest
job:
- name: Driver Tests
test-args: >-
:exclude-tags '[:mongo-sharded-cluster-tests :mb/transforms-python-test]'
:only-tags [:mb/driver-tests]
- name: Transforms Python Tests
test-args: >-
:only-tags [:mb/transforms-python-test]
needs-python-runner: true
env:
CI: 'true'
DRIVERS: mongo
@@ -551,19 +489,16 @@ jobs:
env:
MONGO_INITDB_ROOT_USERNAME: metabase
MONGO_INITDB_ROOT_PASSWORD: metasample123
name: ${{ matrix.version.name }} ${{ matrix.job.name }}
name: ${{ matrix.version.name }}
steps:
- uses: actions/checkout@v4
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
env:
METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
run: ./.github/scripts/setup-python-runner.sh
- name: Test ${{ matrix.version.name }}
uses: ./.github/actions/test-driver
with:
junit-name: ${{ matrix.version.junit-name }}
test-args: ${{ matrix.job.test-args }}
test-args: >-
:exclude-tags [:mongo-sharded-cluster-tests]
:only-tags [:mb/driver-tests]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -627,7 +562,7 @@ jobs:
with:
junit-name: ${{ matrix.version.junit-name }}
test-args: >-
:exclude-tags '[:mongo-sharded-cluster-tests :mb/transforms-python-test]'
:exclude-tags [:mongo-sharded-cluster-tests]
:only-tags [:mb/driver-tests]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
@@ -735,7 +670,6 @@ jobs:
junit-name: ${{ matrix.version.junit-name }}
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -770,11 +704,6 @@ jobs:
- name: Driver Tests
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
- name: Transforms Python Tests
test-args: >-
:only-tags [:mb/transforms-python-test]
needs-python-runner: true
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
env:
# Only run old migrations tests on pushes to master or release branches. All other branches should skip tests
@@ -816,11 +745,6 @@ jobs:
POSTGRES_HOST_AUTH_METHOD: trust
steps:
- uses: actions/checkout@v4
- name: Setup python-runner
#if: ${{ matrix.job.needs-python-runner == true }}
env:
METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
run: ./.github/scripts/setup-python-runner.sh
- name: Test ${{ matrix.version.name }} (${{ matrix.job.name }})
uses: ./.github/actions/test-driver
with:
@@ -897,7 +821,6 @@ jobs:
junit-name: 'be-tests-presto-jdbc-ee'
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -927,14 +850,8 @@ jobs:
skip: false
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags '[:mb/upload-tests :mb/transforms-python-test]'
:exclude-tags [:mb/upload-tests]
:fail-fast? true
- name: Redshift Transforms Python Tests
skip: false
test-args: >-
:only-tags [:mb/transforms-python-test]
:fail-fast? true
needs-python-runner: true
env:
CI: 'true'
@@ -947,11 +864,6 @@ jobs:
name: ${{ matrix.job.name }}
steps:
- uses: actions/checkout@v4
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
env:
METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
run: ./.github/scripts/setup-python-runner.sh
- name: Test ${{ matrix.job.name }}
if: ${{ !matrix.job.skip }}
uses: ./.github/actions/test-driver
@@ -975,19 +887,6 @@ jobs:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
job:
- name: Driver Tests
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
# DS 2025-09-26 temporarily disabled while we diagnose flakes
# - name: Transforms Python Tests
# test-args: >-
# :only-tags [:mb/transforms-python-test]
# needs-python-runner: true
env:
CI: 'true'
DRIVERS: snowflake
@@ -1006,19 +905,15 @@ jobs:
MB_SNOWFLAKE_TEST_RSA_ROLE_TEST_CUSTOM_USER: RSA_ROLE_TEST_CUSTOM_USER
MB_SNOWFLAKE_TEST_RSA_ROLE_TEST_ROLE: RSA_ROLE_TEST_ROLE
MB_SNOWFLAKE_TEST_RSA_ROLE_TEST_DB: RSA_ROLE_TEST_DB
name: Snowflake ${{ matrix.job.name }}
name: Snowflake
steps:
- uses: actions/checkout@v4
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
env:
METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
run: ./.github/scripts/setup-python-runner.sh
- name: Test Snowflake driver
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-snowflake-ee'
test-args: ${{ matrix.job.test-args }}
test-args: >-
:only-tags [:mb/driver-tests]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -1053,7 +948,6 @@ jobs:
junit-name: 'be-tests-sparksql-ee'
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -1083,7 +977,6 @@ jobs:
junit-name: 'be-tests-sqlite-ee'
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -1111,15 +1004,6 @@ jobs:
- name: SQL Server 2022
junit-name: be-tests-sqlserver-2022-ee
image: mcr.microsoft.com/mssql/server:2022-latest
job:
- name: Driver Tests
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
- name: Transforms Python Tests
test-args: >-
:only-tags [:mb/transforms-python-test]
needs-python-runner: true
env:
CI: 'true'
DRIVERS: sqlserver
@@ -1135,19 +1019,15 @@ jobs:
ACCEPT_EULA: Y
SA_PASSWORD: 'P@ssw0rd'
MSSQL_MEMORY_LIMIT_MB: 1024
name: ${{ matrix.version.name }} ${{ matrix.job.name }}
name: ${{ matrix.version.name }}
steps:
- uses: actions/checkout@v4
- name: Setup python-runner
if: ${{ matrix.job.needs-python-runner == true }}
env:
METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
run: ./.github/scripts/setup-python-runner.sh
- name: Test ${{ matrix.version.name }}
uses: ./.github/actions/test-driver
with:
junit-name: ${{ matrix.version.junit-name }}
test-args: ${{ matrix.job.test-args }}
test-args: >-
:only-tags [:mb/driver-tests]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -1160,77 +1040,46 @@ jobs:
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
# TODO(dpsutton, 2025-08-20): Re-enable starburst when we figure out why it randomly dies in CI
# be-tests-starburst-ee:
# needs: determine-driver-skips
# if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
# runs-on: ubuntu-22.04
# timeout-minutes: 60
# env:
# CI: 'true'
# DRIVERS: starburst
# services:
# starburst:
# image: trinodb/trino:latest
# env:
# CATALOG_MANAGEMENT: dynamic
# ports:
# - "8080:8080"
# name: Starburst
# steps:
# - uses: actions/checkout@v4
# - name: Make plugins directory
# run: mkdir plugins
# - name: Test Starburst driver
# uses: ./.github/actions/test-driver
# with:
# junit-name: 'be-tests-starburst-ee'
# test-args: >-
# :only-tags [:mb/driver-tests]
# - name: Upload Test Results
# uses: ./.github/actions/upload-test-results
# if: always()
# with:
# input-path: ./target/junit/
# output-name: ${{ github.job }}
# bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
# aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
# aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
# aws-region: ${{ vars.AWS_REGION }}
# trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
login-to-ecr:
be-tests-starburst-ee:
needs: determine-driver-skips
if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
runs-on: ubuntu-22.04
timeout-minutes: 15
permissions:
id-token: write
contents: read
timeout-minutes: 60
env:
CI: 'true'
DRIVERS: starburst
services:
starburst:
image: trinodb/trino:latest
env:
CATALOG_MANAGEMENT: dynamic
ports:
- "8080:8080"
name: Starburst
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
- uses: actions/checkout@v4
- name: Make plugins directory
run: mkdir plugins
- name: Test Starburst driver
uses: ./.github/actions/test-driver
with:
role-to-assume: ${{ secrets.CI_IAM_ROLE }}
role-session-name: GitHub_to_AWS_via_FederatedOIDC
aws-region: us-east-1
- name: Get ECR password
id: login-ecr
run: |
PASSWORD=$(aws ecr get-login-password --region us-east-1)
REGISTRY="$(aws sts get-caller-identity --query Account --output text).dkr.ecr.us-east-1.amazonaws.com"
echo "docker_username=AWS" >> $GITHUB_OUTPUT
echo "docker_password=$PASSWORD" >> $GITHUB_OUTPUT
echo "registry=$REGISTRY" >> $GITHUB_OUTPUT
outputs:
registry: ${{ steps.login-ecr.outputs.registry }}
docker_username: ${{ steps.login-ecr.outputs.docker_username }}
docker_password: ${{ steps.login-ecr.outputs.docker_password }}
junit-name: 'be-tests-starburst-ee'
test-args: >-
:only-tags [:mb/driver-tests]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-vertica-ee:
needs: [determine-driver-skips, login-to-ecr]
needs: determine-driver-skips
if: ${{ !inputs.skip && !(needs.determine-driver-skips.outputs.skip == 'true') }}
runs-on: ubuntu-22.04
timeout-minutes: 60
@@ -1239,12 +1088,9 @@ jobs:
DRIVERS: vertica
services:
vertica:
image: ${{ needs.login-to-ecr.outputs.registry }}/vertica-ce:25.3.0-0
image: vertica/vertica-ce:12.0.2-0
ports:
- "5433:5433"
credentials:
username: ${{ needs.login-to-ecr.outputs.docker_username }}
password: ${{ needs.login-to-ecr.outputs.docker_password }}
name: Vertica
steps:
- uses: actions/checkout@v4
@@ -1256,7 +1102,6 @@ jobs:
junit-name: 'be-tests-vertica-ee'
test-args: >-
:only-tags [:mb/driver-tests]
:exclude-tags [:mb/transforms-python-test]
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -1290,9 +1135,8 @@ jobs:
- be-tests-sparksql-ee
- be-tests-sqlite-ee
- be-tests-sqlserver
# - be-tests-starburst-ee
# TODO(rileythomp, 2025-08-20): Re-enable vertica tests once the docker image is updated
# - be-tests-vertica-ee
- be-tests-starburst-ee
- be-tests-vertica-ee
- be-tests-clickhouse-ee
runs-on: ubuntu-latest
timeout-minutes: 5
@@ -1314,7 +1158,7 @@ jobs:
}));
// are all jobs skipped or successful?
if (jobs.every(job => (job.result === 'skipped' || job.result === 'success' || job.name == 'be-tests-starburst-ee'))) {
if (jobs.every(job => (job.result === 'skipped' || job.result === 'success'))) {
console.log("");
console.log(" _------. ");
console.log(" / , \_ ");

View File

@@ -1,22 +1,15 @@
# Thi workflow does the backward compatibility tests
# between the Embedding SDK package of version N and Embedding SDK Bundle of version N - 1
# N - 1 is a target of a PR
name: Embedding SDK backward-compatibility E2E Component Tests
name: E2E Cross-version Component Tests for Embedding SDK
on:
workflow_call:
inputs:
cached-embedding-sdk-dist-artifact-name:
required: true
type: string
jobs:
files-changed:
name: "SDK backward-compatibility E2E > Check which files changed"
name: "SDK cross-version E2E > Check which files changed"
runs-on: ubuntu-22.04
timeout-minutes: 3
outputs:
e2e_embedding_sdk: ${{ steps.changes.outputs.e2e_embedding_sdk }}
e2e_all: ${{ steps.changes.outputs.e2e_all }}
steps:
- uses: actions/checkout@v4
- name: Test which files changed
@@ -26,31 +19,11 @@ jobs:
token: ${{ github.token }}
filters: .github/file-paths.yaml
decide-target-ref:
runs-on: ubuntu-latest
outputs:
target-ref: ${{ steps.get-target-ref.outputs.value }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 2
- name: Decide ref to checkout
id: get-target-ref
run: |
if [[ "${{ github.event_name }}" == "pull_request" ]]; then
echo "value=${{ github.base_ref }}" >> "$GITHUB_OUTPUT"
else
echo "value=$(git rev-parse HEAD^1)" >> "$GITHUB_OUTPUT"
fi
# if this is a test on a release branch, we need to check the build requirements
get-build-requirements:
name: "SDK backward-compatibility E2E > [Backport only] Get build requirements"
needs: [decide-target-ref]
name: "SDK cross-version E2E > [Backport only] Get build requirements"
if: |
!cancelled() &&
contains(github.base_ref || github.ref, 'release-x')
!cancelled()
runs-on: ubuntu-22.04
timeout-minutes: 10
outputs:
@@ -59,7 +32,6 @@ jobs:
steps:
- uses: actions/checkout@v4
with:
ref: ${{ needs.decide-target-ref.outputs.target-ref }}
sparse-checkout: release
- name: Prepare build scripts
run: cd ${{ github.workspace }}/release && yarn && yarn build
@@ -82,12 +54,76 @@ jobs:
node_version: requirements.node,
};
e2e-tests:
name: "SDK backward-compatibility E2E > Run e2e tests"
needs: [files-changed, decide-target-ref, get-build-requirements]
build:
name: "SDK cross-version E2E > Build uberjar"
needs: [files-changed, get-build-requirements]
if: |
!cancelled() &&
needs.files-changed.outputs.e2e_embedding_sdk == 'true'
!cancelled()
runs-on: ubuntu-22.04
timeout-minutes: 25
env:
MB_EDITION: ee
INTERACTIVE: false
# make sure that builds on release branches get licenses, because we use them for releases
SKIP_LICENSES: ${{ github.event_name == 'pull_request' }}
steps:
- uses: actions/checkout@v4
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
with:
node-version: "${{ needs.get-build-requirements.outputs.node_version }}"
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: uberjar
java-version: "${{ needs.get-build-requirements.outputs.java_version || 11 }}"
- name: Build uberjar with ./bin/build.sh
run: ./bin/build.sh
- name: Prepare uberjar artifact
uses: ./.github/actions/prepare-uberjar-artifact
with:
name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar-cross-version
resolve-sdk-version:
name: "SDK cross-version E2E > Resolve SDK version"
if: |
!cancelled()
runs-on: ubuntu-22.04
timeout-minutes: 10
outputs:
sdk_version: ${{ fromJson(steps.dependencies.outputs.result).sdk_version }}
steps:
- uses: actions/checkout@v4
with:
sparse-checkout: release
- name: Prepare build scripts
run: cd ${{ github.workspace }}/release && yarn && yarn build
- name: Get latest SDK release for this branch
uses: actions/github-script@v7
id: dependencies
with:
script: | # js
const { getSdkVersionFromReleaseBranchName } = require('${{ github.workspace }}/release/dist/index.cjs');
const branchName = '${{ github.base_ref || github.ref }}';
const sdk_version = await getSdkVersionFromReleaseBranchName({
github,
owner: context.repo.owner,
repo: context.repo.repo,
branchName,
});
return {
sdk_version,
};
e2e-tests:
name: "SDK cross-version E2E > Run e2e tests"
needs: [get-build-requirements, build, resolve-sdk-version]
if: |
!cancelled() && needs.build.result == 'success'
runs-on: ubuntu-22.04
timeout-minutes: 45
env:
@@ -106,33 +142,45 @@ jobs:
steps:
- uses: actions/checkout@v4
with:
ref: ${{ needs.decide-target-ref.outputs.target-ref }}
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
# Needed to omit newly added tests for functionality that is not yet released
- name: Sparse checkout tests from SDK release commit
uses: actions/checkout@v4
with:
m2-cache-key: uberjar
java-version: "${{ needs.get-build-requirements.outputs.java_version || 21 }}"
ref: embedding-sdk-${{ needs.resolve-sdk-version.outputs.sdk_version }}
path: sdk-release-e2e
sparse-checkout: e2e
- name: Move tests from the release commit into the repo
run: |
rm -rf ./e2e/support/*
mv ./sdk-release-e2e/e2e/support/* ./e2e/support
rm -rf ./e2e/test-component/*
mv ./sdk-release-e2e/e2e/test-component/* ./e2e/test-component
shell: bash
- name: Retrieve uberjar artifact for ee
uses: actions/download-artifact@v4
with:
name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar-cross-version
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
with:
node-version: "${{ needs.get-build-requirements.outputs.node_version }}"
- name: Build uberjar with ./bin/build.sh
run: ./bin/build.sh
- name: Retrieve Embedding SDK dist artifact
uses: actions/download-artifact@v4
- name: Prepare JDK ${{ needs.get-build-requirements.outputs.java_version || 11 }}
uses: actions/setup-java@v4
with:
name: ${{ inputs.cached-embedding-sdk-dist-artifact-name }}
path: ${{ github.workspace }}/resources/embedding-sdk
java-version: ${{ needs.get-build-requirements.outputs.java_version || 11 }}
distribution: "temurin"
- name: Prepare Cypress environment
id: cypress-prep
uses: ./.github/actions/prepare-cypress
- name: Install Embedding SDK package v${{ needs.resolve-sdk-version.outputs.sdk_version }}
run: yarn add @metabase/embedding-sdk-react@${{ needs.resolve-sdk-version.outputs.sdk_version }}
- name: Run Metabase
run: node e2e/runner/run_cypress_ci.js start
@@ -140,10 +188,7 @@ jobs:
run: node e2e/runner/run_cypress_ci.js snapshot --browser ${{ steps.cypress-prep.outputs.chrome-path }}
- name: Run component tests for Embedding SDK
run: |
node e2e/runner/run_cypress_ci.js component \
--browser ${{ steps.cypress-prep.outputs.chrome-path }} \
--env grepTags="-@skip-backward-compatibility",grepOmitFiltered=true
run: node e2e/runner/run_cypress_ci.js component --browser ${{ steps.cypress-prep.outputs.chrome-path }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results

View File

@@ -0,0 +1,162 @@
name: E2E cross-version for breaking changes
# See `bin/backward-compatibility-test.js` for the documentation of this workflow.
on:
workflow_dispatch:
inputs:
fe-ref:
description: "Frontend git ref to use"
type: string
required: true
default: "master"
be-ref:
description: "Backend git ref to use"
type: string
required: true
default: "master"
total-shards:
description: "Total number of parallel test shards"
type: number
required: false
default: 4
workflow_call:
inputs:
fe-ref:
description: "Frontend git ref to use"
type: string
required: true
be-ref:
description: "Backend git ref to use"
type: string
required: true
total-shards:
description: "Total number of parallel test shards"
type: number
required: false
default: 4
jobs:
generate-matrix:
name: "Breaking changes E2E > Generate matrix"
runs-on: ubuntu-latest
outputs:
matrix_json: ${{ steps.set_matrix.outputs.matrix_json }}
shards_total: ${{ steps.set_matrix.outputs.shards_total }}
steps:
- name: Set up shard matrix
id: set_matrix
uses: actions/github-script@v7
with:
# generate an output like `{"matrixJson": "{\"shard\": [1, 2, 3, 4]}", "shardsTotal": 4}` to be used in the other jobs
script: |
const totalShards = parseInt(process.env.TOTAL_SHARDS_INPUT, 10);
const matrix = { shard: Array.from({ length: totalShards }, (_, i) => i + 1) };
console.log("Generated matrix JSON", matrix);
console.log(`Total shards: ${totalShards}`);
core.setOutput('matrix_json', JSON.stringify(matrix));
core.setOutput('shards_total', totalShards);
env:
TOTAL_SHARDS_INPUT: ${{ inputs.total-shards }}
build-jar:
name: "Breaking changes E2E > Build jar"
runs-on: ubuntu-22.04
timeout-minutes: 45
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Prepare Node.js
# We're not using `actions/prepare-frontend` because we don't want to install deps from the root repo
# nor we want to touch the cache as we'll be potentially be installing different versions of packages
uses: actions/setup-node@v4
with:
node-version-file: ".nvmrc"
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: cross-version-uberjar
- name: Build jar with different frontend and backend version
env:
MB_EDITION: ee
CYPRESS_CI: true
run: |
node bin/backward-compatibility-test.js \
${{ inputs.fe-ref }} ${{ inputs.be-ref }} \
build
- name: Upload Uberjar
uses: actions/upload-artifact@v4
with:
name: metabase-jar-cross-version
path: .tmp/metabase-be/target/uberjar/metabase.jar
if-no-files-found: error
run-breaking-changes-tests:
name: "Breaking changes E2E > Run breaking changes tests"
needs: [build-jar, generate-matrix]
runs-on: ubuntu-22.04
timeout-minutes: 45
env:
TOTAL_SHARDS: ${{ needs.generate-matrix.outputs.shards_total }}
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.generate-matrix.outputs.matrix_json) }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Prepare Node.js
uses: actions/setup-node@v4
with:
node-version-file: ".nvmrc"
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: cross-version-uberjar
- name: Download Uberjar
uses: actions/download-artifact@v4
with:
name: metabase-jar-cross-version
path: .tmp/metabase-be/target/uberjar/
- name: Download FE repo
run: |
git clone --depth 1 -b ${{ inputs.fe-ref }} https://github.com/metabase/metabase.git .tmp/metabase-fe
- name: Install FE dependencies
working-directory: .tmp/metabase-fe
run: |
yarn install
- name: Prepare Cypress environment
id: cypress-prep
uses: ./.github/actions/prepare-cypress
with:
working-directory: .tmp/metabase-fe
- name: Run the tests
env:
MB_EDITION: ee
CYPRESS_MB_ALL_FEATURES_TOKEN: ${{ secrets.MB_ALL_FEATURES_TOKEN }}
CYPRESS_MB_STARTER_CLOUD_TOKEN: ${{ secrets.MB_STARTER_CLOUD_TOKEN }}
CYPRESS_MB_PRO_CLOUD_TOKEN: ${{ secrets.MB_PRO_CLOUD_TOKEN }}
CYPRESS_MB_PRO_SELF_HOSTED_TOKEN: ${{ secrets.MB_PRO_SELF_HOSTED_TOKEN }}
CYPRESS_CI: true
run: |
node bin/backward-compatibility-test.js \
${{ inputs.fe-ref }} ${{ inputs.be-ref }} \
start > /dev/null & \
node bin/backward-compatibility-test.js \
${{ inputs.fe-ref }} ${{ inputs.be-ref }} \
test ${{ matrix.shard}}/${{ env.TOTAL_SHARDS }}

View File

@@ -178,8 +178,12 @@ jobs:
with:
name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
- name: Compile CLJS
run: yarn build-pure:cljs
shell: bash
- name: Build Embedding SDK package
run: yarn build-embedding-sdk-package
run: yarn build-embedding-sdk
- name: Prepare Embedding SDK artifact
uses: actions/upload-artifact@v4
@@ -192,7 +196,7 @@ jobs:
# - starts it
# - runs its Cypress tests against the local metabase.jar and local @metabase/embedding-sdk-react package.
# The job is not required, though if it's failing better to be sure that it's not caused by a breaking change.
# For more details see `Sample Apps compatibility with Embedding SDK tests` in `enterprise/frontend/src/embedding-sdk-package/dev.md`
# For more details see `Sample Apps compatibility with Embedding SDK tests` in `enterprise/frontend/src/embedding-sdk/dev.md`
e2e-sample-apps-tests:
needs: [get-build-requirements, get-sample-apps-data, build]
strategy:
@@ -316,7 +320,7 @@ jobs:
# - starts an app
# - runs its Cypress tests against the e2e test metabase instance and local @metabase/embedding-sdk-react package.
# The job is not required, though if it's failing better to be sure that it's not caused by a breaking change.
# For more details see `Embedding SDK integration tests with Host Apps` in `enterprise/frontend/src/embedding-sdk-package/dev.md`
# For more details see `Embedding SDK integration tests with Host Apps` in `enterprise/frontend/src/embedding-sdk/dev.md`
e2e-host-apps-tests:
needs: [get-build-requirements, build]
strategy:

View File

@@ -60,14 +60,6 @@ jobs:
CYPRESS_IS_EMBEDDING_SDK: "${{ inputs.name == 'embedding-sdk' }}"
steps:
- uses: actions/checkout@v4
with:
submodules: true
token: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
- name: Setup python-runner
env:
METABASE_AUTOMATION_USER_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
run: ./.github/scripts/setup-python-runner.sh
- name: Prepare Docker containers
uses: ./.github/actions/e2e-prepare-containers
@@ -87,11 +79,9 @@ jobs:
uses: ./.github/actions/run-embedding-sdk-storybook
- name: Retrieve uberjar artifact for ${{ inputs.edition }}
uses: ./.github/actions/fetch-artifact
uses: actions/download-artifact@v4
with:
name: metabase-${{ inputs.edition }}-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
extract-path: "target/uberjar"
output-name: "metabase.jar"
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
@@ -131,7 +121,7 @@ jobs:
SPLIT_INDEX: ${{ inputs.split_index }}
SPLIT_FILE: "./e2e/support/timings.json"
SPLIT_OUTPUT_FILE: "newTimes.json"
SPLIT_TIME_THRESHOLD: 0.01 # 1% threshold for changes
SPLIT_TIME_THRESHOLD: 5.0
# do not write cypress-split summary to github
SPLIT_SUMMARY: false
SPEC_PATH: ${{ env.specs || inputs.specs }}
@@ -139,7 +129,7 @@ jobs:
run: |
echo "Running tests with specs: $SPEC_PATH"
node e2e/runner/run_cypress_ci.js e2e \
--env grepTags="-@mongo+-@flaky+-@OSS+-@skip",grepOmitFiltered=true \
--env grepTags="-@mongo+-@flaky+-@OSS --@quarantine",grepOmitFiltered=true \
--spec "$SPEC_PATH" \
--browser ${{ steps.cypress-prep.outputs.chrome-path }}
@@ -151,7 +141,7 @@ jobs:
run: |
echo "Running tests with specs: $SPEC_PATH"
node e2e/runner/run_cypress_ci.js e2e \
--env grepTags="${{inputs.tags && format('{0}+-@skip', inputs.tags) || '-@skip'}}",grepOmitFiltered=true \
--env grepTags="${{inputs.tags}} --@quarantine",grepOmitFiltered=true \
--spec "$SPEC_PATH" \
--browser ${{ steps.cypress-prep.outputs.chrome-path }}
@@ -185,51 +175,6 @@ jobs:
name: failed-tests-${{ inputs.name }}-${{ inputs.edition }}
path: ./cypress/test-results
- name: Extract chunk-specific changed timings
id: extract-timings
if: inputs.split_index >= 0 && github.ref == 'refs/heads/master'
continue-on-error: true
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
const oldTimingsPath = "./e2e/support/timings.json";
const newTimingsPath = "./e2e/support/newTimes.json";
if (!fs.existsSync(newTimingsPath)) {
throw new Error(`newTimes.json not found - cypress-split may not have generated it`);
}
if (!fs.existsSync(oldTimingsPath)) {
throw new Error(`Original timings file ${oldTimingsPath} not found`);
}
const oldTimings = fs.existsSync(oldTimingsPath)
? JSON.parse(fs.readFileSync(oldTimingsPath, "utf8"))
: { durations: [] };
const newTimings = fs.existsSync(newTimingsPath)
? JSON.parse(fs.readFileSync(newTimingsPath, "utf8"))
: { durations: [] };
const { extractChangedTimings } = require('${{ github.workspace }}/.github/scripts/extract-changed-timings.js');
const result = extractChangedTimings({ oldTimings, newTimings });
if (result.hasChanges) {
core.setOutput('has-changes', 'true');
fs.writeFileSync('chunkTimings.json', JSON.stringify(result.changedTimings, null, 2));
console.log(`Extracted ${Object.keys(result.changedTimings).length} changed timings for chunk index ${{ inputs.split_index }}`);
}
- name: Upload Timing Data
uses: actions/upload-artifact@v4
if: steps.extract-timings.outcome == 'success' && steps.extract-timings.outputs.has-changes == 'true'
with:
name: timing-data-${{ inputs.name }}-${{ inputs.split_index }}
path: chunkTimings.json
retention-days: 1 # gets merged immediately by parent workflow
- name: Publish Summary
if: failure()
uses: actions/github-script@v7

View File

@@ -87,67 +87,6 @@ jobs:
name: Flaky
tags: "@flaky"
collect-timings:
name: Collect E2E Timings
runs-on: ubuntu-22.04
timeout-minutes: 10
needs: e2e-tests
if: always() && !cancelled() && github.ref == 'refs/heads/master'
steps:
- uses: actions/checkout@v4
with:
sparse-checkout: |
.github
e2e/support
sparse-checkout-cone-mode: false
- name: Download all timing artifacts
uses: actions/download-artifact@v4
with:
pattern: timing-data-*
path: timing-files
merge-multiple: false
continue-on-error: true
- name: Merge split timings
id: merge
continue-on-error: true
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
const path = require('path');
const chunkFiles = glob.sync('timing-files/*/chunkTimings.json');
const specs = new Map();
chunkFiles.forEach(file => {
const { durations = [] } = JSON.parse(fs.readFileSync(file, 'utf8'));
durations.forEach(({ spec, duration }) => specs.set(spec, duration));
});
if (specs.size === 0) {
throw new Error('No timing data found - no specs had timing changes');
}
const outputPath = 'e2e/support/runTimings.json';
fs.mkdirSync(path.dirname(outputPath), { recursive: true });
const runTimings = {
durations: [...specs].map(([spec, duration]) => ({ spec, duration }))
};
fs.writeFileSync(outputPath, JSON.stringify(runTimings, null, 2));
console.log(`Merged timing data for ${specs.size} specs`);
- name: Upload merged timing files
if: steps.merge.outcome == 'success'
uses: actions/upload-artifact@v4
with:
name: merged-e2e-timings
path: e2e/support/runTimings.json
retention-days: 7
e2e-tests-result:
needs:
- e2e-tests

View File

@@ -1,49 +0,0 @@
name: Embedding SDK Labels Reminder
on:
pull_request:
types: [auto_merge_enabled, review_requested, labeled, unlabeled, synchronize]
branches:
- "master"
merge_group:
jobs:
files-changed:
name: Check which files changed
runs-on: ubuntu-22.04
timeout-minutes: 3
outputs:
frontend_embedding_sdk_package_sources: ${{ steps.changes.outputs.frontend_embedding_sdk_package_sources }}
steps:
- uses: actions/checkout@v4
- name: Test which files changed
uses: dorny/paths-filter@v3.0.0
id: changes
with:
token: ${{ github.token }}
filters: .github/file-paths.yaml
embedding-sdk-package-no-release-notes-reminder:
needs: [ files-changed ]
if: |
!cancelled() &&
(needs.files-changed.outputs.frontend_embedding_sdk_package_sources == 'true')
name: Embedding SDK package release-notes/no-release-notes label reminder
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- uses: jesusvasquez333/verify-pr-label-action@v1.4.0
id: verify-release-notes-labels
with:
github-token: '${{ secrets.GITHUB_TOKEN }}'
valid-labels: 'release-notes, no-release-notes'
disable-reviews: true
- name: Comment on PR if label should be added
if: ${{ always() }}
uses: ./.github/actions/notify-pull-request
with:
remove-previous-comment: true
add-new-comment: ${{ steps.verify-release-notes-labels.outcome == 'failure' }}
message: "⚠️ Changes were made to the <b>Embedding SDK Package</b>. Review this PR and apply one of the following labels.<br><br>- <code>release-notes</code> — select if this PR must be included in the main Metabase changelog release notes.<br>- <code>no-release-notes</code> — select if this PR should not be included in the main Metabase changelog release notes."
include-log: true

View File

@@ -43,7 +43,7 @@ jobs:
with:
m2-cache-key: "cljs"
- name: Build Embedding SDK package
run: yarn build-embedding-sdk-package
run: yarn build-embedding-sdk
- name: Prepare Embedding SDK artifact
uses: actions/upload-artifact@v4
with:
@@ -137,7 +137,7 @@ jobs:
if: ${{ steps.docs-generation.outcome == 'failure' }}
run: exit 1
# See `Component e2e tests` in `enterprise/frontend/src/embedding-sdk-package/dev.md`
# See `Component e2e tests` in `enterprise/frontend/src/embedding-sdk/dev.md`
sdk-e2e-tests:
name: "SDK e2e tests"
uses: ./.github/workflows/e2e-component-tests-embedding-sdk.yml
@@ -146,13 +146,24 @@ jobs:
with:
cached-embedding-sdk-dist-artifact-name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
e2e-component-tests-embedding-sdk-backward-compatibility:
name: "SDK e2e backward compatibility tests"
uses: ./.github/workflows/e2e-component-tests-embedding-sdk-backward-compatibility.yml
needs: [build]
e2e-component-tests-embedding-sdk-cross-version:
name: "[Backport only] SDK e2e cross version tests"
# run on: every push on release branches, and every pr that targets a release branch
if: ${{ startsWith(github.ref, 'refs/heads/release-x.') || (github.event_name == 'pull_request' && startsWith(github.base_ref, 'release-x.')) }}
uses: ./.github/workflows/e2e-component-tests-embedding-sdk-cross-version.yml
secrets: inherit
with:
cached-embedding-sdk-dist-artifact-name: embedding-sdk-${{ github.event.pull_request.head.sha || github.sha }}
# Temporarily skipped because the cypress runner crashes
# See `bin/backward-compatibility-test.js` for the documentation of this workflow.
# cross-version-for-breaking-changes:
# name: "[NOT REQUIRED - Experimental - Backport only] detect breaking changes with cross versions tests"
# # run on: every pr that targets a release branch, it does NOT run for pr targeting master
# if: ${{ github.event_name == 'pull_request' && startsWith(github.base_ref, 'release-x.') }}
# uses: ./.github/workflows/e2e-cross-version-for-breaking-changes.yml
# secrets: inherit
# with:
# fe-ref: ${{ github.base_ref }}
# be-ref: ${{ github.head_ref }}
embedding-sdk-tests-result:
needs:
@@ -160,7 +171,7 @@ jobs:
- embedding-sdk-docs-snippets-type-check
- embedding-sdk-api-documentation-generation-validation
- sdk-e2e-tests
- e2e-component-tests-embedding-sdk-backward-compatibility
- e2e-component-tests-embedding-sdk-cross-version
if: always() && !cancelled()
runs-on: ubuntu-latest
timeout-minutes: 5

View File

@@ -52,8 +52,6 @@ jobs:
run: yarn run lint-prettier-pure
- name: Run ESLint linter
run: yarn run lint-eslint-pure
- name: Run CSS linter
run: yarn run lint-css
fe-type-check:
if: ${{ !inputs.skip }}

32
.github/workflows/links.yml vendored Normal file
View File

@@ -0,0 +1,32 @@
name: Links
on:
repository_dispatch:
workflow_dispatch:
schedule:
- cron: "0 0 * * 0"
jobs:
linkChecker:
runs-on: ubuntu-latest
permissions:
issues: write # required for peter-evans/create-issue-from-file
steps:
- uses: actions/checkout@v4
- name: Link Checker
id: lychee
uses: lycheeverse/lychee-action@v2
with:
fail: false
args: docs --include-fragments --config ./.lychee/config.toml --verbose --no-progress
format: markdown
- name: Create Issue From File
if: steps.lychee.outputs.exit_code != 0
uses: peter-evans/create-issue-from-file@v5
with:
title: Link Checker Report
content-filepath: ./.lychee/out.md
labels: "Type:Documentation"
assignees: jeff-bruemmer

View File

@@ -26,8 +26,26 @@ jobs:
- name: Compile CLJS
run: yarn build-pure:cljs
- name: Build Storybook
# We won't be able to use Storybook play's `userEvent` with production build
# see https://github.com/storybookjs/storybook/issues/19758
run: NODE_ENV=development yarn build-storybook
- name: Serve Storybook
run: yarn http-server storybook-static -p 6006 &
shell: bash
- name: Wait for Storybook to become available
run: |
echo "Waiting for Storybook to be served..."
while ! curl --silent --fail http://localhost:6006; do
sleep 5
echo "Retrying..."
done
echo "Storybook is being served!"
- name: Update Loki Snapshots
run: yarn test-visual:loki:ci || yarn test-visual:loki-approve-diff && yarn test-visual:loki-prune
run: yarn test-visual:loki || yarn test-visual:loki-approve-diff && yarn test-visual:loki-prune
- name: Check for changes
id: check-changes

View File

@@ -53,8 +53,14 @@ jobs:
- name: Compile CLJS
run: NODE_ENV=development yarn build-pure:cljs
- name: Build Storybook
# We won't be able to use Storybook play's `userEvent` with production build
# see https://github.com/storybookjs/storybook/issues/19758
run: NODE_ENV=development yarn build-storybook
- name: Run Loki Visual Tests
run: yarn test-visual:loki:ci
run: |
yarn loki --requireReference --reactUri file:./storybook-static --chromeFlags='--headless --disable-gpu' --verboseRenderer
- name: Generate Visual Report on Failure
if: failure()

View File

@@ -9,13 +9,9 @@ concurrency:
cancel-in-progress: true
jobs:
uberjar:
if: ${{ github.event.label.name == 'Run-perf' }}
uses: ./.github/workflows/uberjar.yml
secrets: inherit
run-perf-test:
needs: [uberjar]
if: ${{ github.event.label.name == 'Run-perf' }}
uses: ./.github/workflows/perf-test.yml
with:
wait_for_uberjar: true
secrets: inherit

View File

@@ -3,6 +3,11 @@ run-name: Run perf test for ${{ github.ref_name }} by @${{ github.actor }}
on:
workflow_call:
inputs:
wait_for_uberjar:
description: "Wait for uberjar build"
required: false
type: boolean
jobs:
build:
runs-on: ubuntu-latest
@@ -26,12 +31,69 @@ jobs:
uses: aws-actions/amazon-ecr-login@v2
with:
registries: "${{ secrets.PR_ENV_AWS_ACCOUNT_ID }}"
- name: Wait for uberjar
id: wait_for_uberjar
if: ${{ inputs.wait_for_uberjar == true }}
run: |
## Get workflow run id for uberjar build
curl -Ls --output e2e-tests.json \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/${{ github.repository }}/actions/workflows/run-tests.yml/runs?head_sha=${{ github.event.pull_request.head.sha || github.sha }}
ID=$(jq -r '.workflow_runs[0].id' e2e-tests.json)
echo "Run ID: ${ID}"
## Get workflow run id for uberjar build
while [ -z "$JOB_ID" ]; do
NEXT_URL="https://api.github.com/repos/${{ github.repository }}/actions/runs/${ID}/jobs?filter=latest"
# Search for build job
echo "Searching for build job..."
while [ -z "$JOB_ID" ] && [ -n "$NEXT_URL" ]; do
# Get one page of jobs
RESPONSE=$(curl -s -i \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
-H "X-GitHub-Api-Version: 2022-11-28" \
"$NEXT_URL")
# Extract headers and body
HEADERS=$(echo "$RESPONSE" | sed -n '1,/^\r$/p')
BODY=$(echo "$RESPONSE" | sed '1,/^\r$/d')
LINK_HEADER=$(echo "$HEADERS" | grep -i '^link:' | sed 's/^link: //')
# Check for next page in Link header
if echo "$LINK_HEADER" | grep -q 'rel="next"'; then
NEXT_URL=$(echo "$LINK_HEADER" | sed -E 's/.*<([^>]+)>; rel="next".*/\1/')
else
NEXT_URL=""
fi
# Check for build job
JOB_ID=$(echo "$BODY" | jq -r '.jobs[] | select(.name | contains("build (ee)")) | .id')
done
done
echo "Job ID: ${JOB_ID}"
## Wait for uberjar build to complete
while [ true ]; do
curl -Ls --output uberjar.json \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/${{ github.repository }}/actions/jobs/${JOB_ID}
jq -r '.steps[] | select(.name == "Prepare uberjar artifact") | .status' uberjar.json | grep -q "completed" && break
echo "Waiting for uberjar build..."
sleep 10
done
echo "run_id=$(jq -r '.workflow_runs[0].id' e2e-tests.json)" >> $GITHUB_OUTPUT
- name: Retrieve uberjar artifact for ee
uses: ./.github/actions/fetch-artifact
uses: actions/download-artifact@v4
with:
name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
extract-path: "target/uberjar"
output-name: "metabase.jar"
github-token: ${{ secrets.GITHUB_TOKEN }}
run-id: ${{ inputs.wait_for_uberjar && steps.wait_for_uberjar.outputs.run_id || github.run_id }}
- name: Move uberjar to bin/docker
run: |
jar xf target/uberjar/metabase.jar

View File

@@ -27,11 +27,9 @@ jobs:
with:
registries: "${{ secrets.PR_ENV_AWS_ACCOUNT_ID }}"
- name: Retrieve uberjar artifact for ee
uses: ./.github/actions/fetch-artifact
uses: actions/download-artifact@v4
with:
name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
extract-path: "target/uberjar"
output-name: "metabase.jar"
- name: Move uberjar to bin/docker
run: |
jar xf target/uberjar/metabase.jar

View File

@@ -5,12 +5,19 @@ on:
workflow_dispatch:
inputs:
branch:
description: "Branch we want to release the SDK from (to release from other branches use the workflow from those branches)"
description: "Branch we want to release the SDK from"
type: choice
required: true
default: "master"
options:
- master
- release-x.51.x
- release-x.52.x
- release-x.53.x
- release-x.54.x
- release-x.55.x
- release-x.56.x
- esbuild-main
concurrency:
# We want to ensure only one job is running at a time per branch because
@@ -171,39 +178,41 @@ jobs:
outputs:
sdk_version: ${{ fromJson(steps.new-sdk-version.outputs.result).version }}
major_version: ${{ fromJson(steps.new-sdk-version.outputs.result).majorVersion }}
pre_release_label: ${{ fromJson(steps.new-sdk-version.outputs.result).preReleaseLabel }}
steps:
- name: Check out the code using the provided branch
uses: actions/checkout@v4
with:
ref: ${{ inputs.branch }}
- name: Prepare build scripts
run: yarn --cwd release --frozen-lockfile && yarn --cwd release build
- name: Get current SDK version
id: current-sdk-version
run: |
VERSION=$(jq -r .version ./enterprise/frontend/src/embedding-sdk-package/package.template.json)
VERSION=$(jq -r .version ./enterprise/frontend/src/embedding-sdk/package.template.json)
echo "sdk_current_version=$VERSION" >> $GITHUB_OUTPUT
- name: Get next SDK patch or pre-release version
- name: Get next SDK patch version
id: new-sdk-version
uses: actions/github-script@v7
with:
script: | #js
const { getNextSdkVersion } = require('${{ github.workspace }}/release/dist/index.cjs');
const branch = "${{ inputs.branch }}";
script: |
const currentVersion = "${{ steps.current-sdk-version.outputs.sdk_current_version }}";
const [currentVersionWithoutSuffix, suffix] = currentVersion.split("-");
const versionParts = currentVersionWithoutSuffix.split(".");
versionParts[versionParts.length - 1] = parseInt(versionParts.at(-1)) + 1;
const newVersion = versionParts.join(".");
const [_, majorVersion] = versionParts
const result = getNextSdkVersion(branch, currentVersion);
if (!suffix) {
return {
version: newVersion,
majorVersion
};
}
return {
version: result.version,
preReleaseLabel: result.preReleaseLabel,
majorVersion: result.majorVersion
};
version: `${newVersion}-${suffix}`,
majorVersion
}
- name: Append workflow summary
run: |
@@ -285,8 +294,12 @@ jobs:
run: |
./bin/embedding-sdk/release_utils.bash update_package_json_template ${{ needs.determine-sdk-version.outputs.sdk_version }}
- name: Build SDK package
run: yarn run build-embedding-sdk-package
- name: Update changelog
run: |
yarn embedding-sdk:generate-changelog -o changelog-diff
- name: Build SDK bundle
run: yarn run build-embedding-sdk
- name: Generate SDK package.json in the build directory
run: yarn run embedding-sdk:generate-package
@@ -297,6 +310,12 @@ jobs:
name: metabase-sdk
path: ./resources/embedding-sdk
- name: Upload changelog diff
uses: actions/upload-artifact@v4
with:
name: sdk-changelog-diff
path: ./changelog-diff
publish-npm:
needs: [build-sdk, determine-sdk-version]
runs-on: ubuntu-22.04
@@ -317,7 +336,6 @@ jobs:
git config --global user.name "Metabase Automation"
- name: Update readme
if: inputs.branch != 'master'
run: |
bash ./bin/embedding-sdk/release_utils.bash update_readme ${{ env.sdk_version }}
@@ -325,26 +343,25 @@ jobs:
run: |
bash ./bin/embedding-sdk/release_utils.bash update_package_json_template ${{ env.sdk_version }}
- name: Create a PR updating readme + published version (using GitHub Metabase Automation account)
- name: Retrieve SDK changelog diff
uses: actions/download-artifact@v4
with:
name: sdk-changelog-diff
- name: Update changelog
run: |
cat changelog-diff enterprise/frontend/src/embedding-sdk/CHANGELOG.md > new-changelog
mv new-changelog enterprise/frontend/src/embedding-sdk/CHANGELOG.md
- name: Create a PR updating readme + published version, and changelog (using GitHub Metabase Automation account)
run: |
git checkout -b update-sdk-version-${{ env.sdk_version }}
if [ "${{ inputs.branch }}" = "master" ]; then
COMMIT_MSG="chore(sdk): Update published npm version to ${{ env.sdk_version }}"
PR_TITLE="chore(sdk): Update SDK version to ${{ env.sdk_version }}"
PR_BODY="Update published npm package version to ${{ env.sdk_version }}"
else
COMMIT_MSG="docs(sdk): Update Readme version references and published npm version to ${{ env.sdk_version }}"
PR_TITLE="docs(sdk): Update SDK version to ${{ env.sdk_version }}"
PR_BODY="Update Readme version references and published npm package version to ${{ env.sdk_version }}"
fi
git commit -a -m "$COMMIT_MSG"
git commit -a -m 'Update Readme version references and published npm version to ${{ env.sdk_version }}'
git push origin HEAD
gh pr create --base "${{ inputs.branch }}" \
--assignee "${GITHUB_ACTOR}" \
--title "$PR_TITLE" \
--body "$PR_BODY"
gh pr create --base ${{ inputs.branch }}\
--assignee "${GITHUB_ACTOR}"\
--title "docs(sdk): Update SDK version to ${{ env.sdk_version }}"\
--body "Update Readme version references and published npm package version to ${{ env.sdk_version }}"
env:
GH_TOKEN: ${{ secrets.METABASE_AUTOMATION_USER_TOKEN }}
@@ -372,36 +389,21 @@ jobs:
name: metabase-sdk
path: sdk
- name: Compute master tag
id: compute-tag
run: |
if [ -n "${{ needs.determine-sdk-version.outputs.pre_release_label }}" ]; then
echo "masterTag=${{ needs.determine-sdk-version.outputs.pre_release_label }}" >> $GITHUB_OUTPUT
else
echo "masterTag=nightly" >> $GITHUB_OUTPUT
fi
- name: Publish to NPM
working-directory: sdk
run: |
echo //registry.npmjs.org/:_authToken=${{ secrets.NPM_RELEASE_TOKEN }} > .npmrc
TAG=$(jq -r --null-input \
--arg branch "${{ inputs.branch }}" \
--arg masterTag "${{ steps.compute-tag.outputs.masterTag }}" '
{
"master": $masterTag,
"release-x.51.x": "51-stable",
"release-x.52.x": "52-stable",
"release-x.53.x": "53-stable",
"release-x.54.x": "54-stable",
"release-x.55.x": "55-stable",
"release-x.56.x": "56-stable"
}[$branch]'
)
echo "Resolved npm tag: $TAG"
npm publish --tag "$TAG"
# Please keep the value in sync with `inputs.branch`'s release branch
npm publish --tag ${{fromJson('{
"master": "nightly",
"release-x.51.x": "51-stable",
"release-x.52.x": "52-stable",
"release-x.53.x": "53-stable",
"release-x.54.x": "54-stable",
"release-x.55.x": "55-stable",
"release-x.56.x": "56-stable",
"esbuild-main": "56-esbuild"
}')[inputs.branch]}}
- name: Add `latest` tag to the latest release branch (`release-x.55.x`) deployment
if: ${{ inputs.branch == 'release-x.56.x' }}

View File

@@ -8,7 +8,6 @@ on:
pull_request:
types: [opened, synchronize, reopened]
merge_group:
workflow_dispatch:
jobs:
files-changed:
@@ -73,22 +72,17 @@ jobs:
if: ${{ !cancelled() && !(needs.files-changed.outputs.documentation == 'true' && needs.files-changed.outputs.backend_all == 'false' && needs.files-changed.outputs.frontend_all == 'false' && needs.files-changed.outputs.e2e_all == 'false') }}
uses: ./.github/workflows/uberjar.yml
secrets: inherit
with:
source_changed: ${{ needs.files-changed.outputs.backend_all == 'true' || needs.files-changed.outputs.frontend_all == 'true' || needs.static-viz-files-changed.outputs.static_viz == 'true' }}
backend-tests:
if: ${{ !cancelled() && needs.files-changed.result == 'success' }}
if: ${{ !cancelled() }}
needs: [files-changed, static-viz-files-changed]
uses: ./.github/workflows/backend.yml
secrets: inherit
with:
skip: ${{ needs.files-changed.outputs.backend_all != 'true' && needs.static-viz-files-changed.outputs.static_viz != 'true' }}
semantic-search-tests:
needs: files-changed
uses: ./.github/workflows/semantic-search.yml
secrets: inherit
with:
skip: ${{ needs.files-changed.outputs.backend_all != 'true' }}
app-db-tests:
needs: files-changed
uses: ./.github/workflows/app-db.yml
@@ -113,7 +107,7 @@ jobs:
e2e-tests:
needs: [files-changed, uberjar]
if: always() && !cancelled() && needs.files-changed.result == 'success'
if: always() && !cancelled()
uses: ./.github/workflows/e2e-tests.yml
secrets: inherit
with:
@@ -136,15 +130,3 @@ jobs:
contains(github.event.pull_request.labels.*.name, 'PR-Env')
uses: ./.github/workflows/pr-env.yml
secrets: inherit
containerize:
needs: [uberjar]
if: |
!cancelled() &&
needs.uberjar.result == 'success' &&
(
(github.event_name == 'push' && contains(fromJson('["master", "metabot-v3-main", "internal-tools"]'), github.ref_name)) ||
(github.event_name == 'pull_request' && contains(github.event.pull_request.labels.*.name, 'build-docker-uberjar'))
)
uses: ./.github/workflows/containerize-uberjar.yml
secrets: inherit

View File

@@ -1,348 +0,0 @@
name: Semantic Search Tests
on:
workflow_call:
inputs:
skip:
type: boolean
default: false
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}-semantic-search
cancel-in-progress: true
jobs:
semantic-search-tests-h2:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
version:
- name: H2 with pgvector pg17
junit-name: semantic-search-tests-h2-pgvector-17
pgvector-image: pgvector/pgvector:pg17
job:
- name: Semantic Search Tests
build-static-viz: false
test-args: >-
:only '"enterprise/backend/test/metabase_enterprise/semantic_search"'
exclude-tag: ':mb/driver-tests'
services:
pgvector:
image: ${{ matrix.version.pgvector-image }}
ports:
- "5432:5432"
env:
POSTGRES_DB: mb_semantic_search
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
env:
MB_PGVECTOR_DB_URL: 'jdbc:postgres://localhost:5432/mb_semantic_search?user=postgres&password=postgres'
CI: 'true'
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
steps:
- uses: actions/checkout@v4
- name: Test ${{ matrix.version.name }}
uses: ./.github/actions/test-driver
with:
build-static-viz: ${{ matrix.job.build-static-viz }}
junit-name: ${{ matrix.version.junit-name }}
test-args: >-
${{ matrix.job.test-args }}
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
semantic-search-tests-mariadb:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
version:
- name: MariaDB 10.2 with pgvector pg17
junit-name: semantic-search-tests-mariadb-10-2
mariadb-image: circleci/mariadb:10.2.23
pgvector-image: pgvector/pgvector:pg17
- name: MariaDB Latest with pgvector pg17
junit-name: semantic-search-tests-mariadb-latest
mariadb-image: circleci/mariadb:latest
pgvector-image: pgvector/pgvector:pg17
job:
- name: Semantic Search Tests
build-static-viz: false
test-args: >-
:only '"enterprise/backend/test/metabase_enterprise/semantic_search"'
exclude-tag: ':mb/driver-tests'
services:
pgvector:
image: ${{ matrix.version.pgvector-image }}
ports:
- "5432:5432"
env:
POSTGRES_DB: mb_semantic_search
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
mysql:
image: ${{ matrix.version.mariadb-image }}
ports:
- "3306:3306"
env:
MYSQL_ALLOW_EMPTY_PASSWORD: true
MYSQL_DATABASE: circle_test
env:
CI: 'true'
DRIVERS: mysql
MB_DB_TYPE: mysql
MB_DB_HOST: localhost
MB_DB_PORT: 3306
MB_DB_DBNAME: circle_test
MB_DB_USER: root
MB_MYSQL_TEST_USER: root
MB_PGVECTOR_DB_URL: 'jdbc:postgres://localhost:5432/mb_semantic_search?user=postgres&password=postgres'
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
steps:
- uses: actions/checkout@v4
- name: Test ${{ matrix.version.name }}
uses: ./.github/actions/test-driver
with:
build-static-viz: ${{ matrix.job.build-static-viz }}
junit-name: ${{ matrix.version.junit-name }}
test-args: >-
${{ matrix.job.test-args }}
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
semantic-search-tests-mysql:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
version:
- name: MySQL 8.0 with pgvector pg17
junit-name: semantic-search-tests-mysql-8-0
mysql-image: cimg/mysql:8.0
pgvector-image: pgvector/pgvector:pg17
- name: MySQL Latest with pgvector pg17
junit-name: semantic-search-tests-mysql-latest
mysql-image: mysql:latest
pgvector-image: pgvector/pgvector:pg17
job:
- name: Semantic Search Tests
build-static-viz: false
test-args: >-
:only '"enterprise/backend/test/metabase_enterprise/semantic_search"'
exclude-tag: ':mb/driver-tests'
services:
pgvector:
image: ${{ matrix.version.pgvector-image }}
ports:
- "5432:5432"
env:
POSTGRES_DB: mb_semantic_search
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
mysql:
image: ${{ matrix.version.mysql-image }}
ports:
- "3306:3306"
env:
MYSQL_ALLOW_EMPTY_PASSWORD: true
MYSQL_DATABASE: circle_test
env:
CI: 'true'
DRIVERS: mysql
MB_DB_TYPE: mysql
MB_DB_HOST: localhost
MB_DB_PORT: 3306
MB_DB_DBNAME: circle_test
MB_DB_USER: root
MB_MYSQL_TEST_USER: root
MB_PGVECTOR_DB_URL: 'jdbc:postgres://localhost:5432/mb_semantic_search?user=postgres&password=postgres'
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
steps:
- uses: actions/checkout@v4
- name: Test ${{ matrix.version.name }}
uses: ./.github/actions/test-driver
with:
build-static-viz: ${{ matrix.job.build-static-viz }}
junit-name: ${{ matrix.version.junit-name }}
test-args: >-
${{ matrix.job.test-args }}
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
semantic-search-tests-postgres:
if: ${{ !inputs.skip }}
runs-on: ubuntu-22.04
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
version:
- name: Postgres 12.x with pgvector pg17
junit-name: semantic-search-tests-postgres-12
postgres-image: postgres:12-alpine
pgvector-image: pgvector/pgvector:pg17
- name: Postgres Latest with pgvector pg17
junit-name: semantic-search-tests-postgres-latest
postgres-image: postgres:latest
pgvector-image: pgvector/pgvector:pg17
job:
- name: Semantic Search Tests
build-static-viz: false
test-args: >-
:only '"enterprise/backend/test/metabase_enterprise/semantic_search"'
exclude-tag: ':mb/driver-tests'
services:
pgvector:
image: ${{ matrix.version.pgvector-image }}
ports:
- "5432:5432"
env:
POSTGRES_DB: mb_semantic_search
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
postgres:
image: ${{ matrix.version.postgres-image }}
ports:
- "5433:5432"
env:
POSTGRES_USER: mb_test
POSTGRES_DB: mb_test
POSTGRES_HOST_AUTH_METHOD: trust
env:
CI: 'true'
DRIVERS: postgres
MB_DB_TYPE: postgres
MB_DB_PORT: 5433
MB_DB_HOST: localhost
MB_DB_DBNAME: mb_test
MB_DB_USER: mb_test
MB_POSTGRESQL_TEST_USER: mb_test
MB_PGVECTOR_DB_URL: 'jdbc:postgres://localhost:5432/mb_semantic_search?user=postgres&password=postgres'
name: "${{ matrix.version.name }} ${{ matrix.job.name }}"
steps:
- uses: actions/checkout@v4
- name: Test ${{ matrix.version.name }}
uses: ./.github/actions/test-driver
with:
build-static-viz: ${{ matrix.job.build-static-viz }}
junit-name: ${{ matrix.version.junit-name }}
test-args: >-
${{ matrix.job.test-args }}
:exclude-tags '[${{ matrix.job.exclude-tag }}]'
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
semantic-search-tests-result:
needs:
- semantic-search-tests-h2
- semantic-search-tests-mariadb
- semantic-search-tests-mysql
- semantic-search-tests-postgres
runs-on: ubuntu-latest
timeout-minutes: 5
name: semantic-search-tests-result
if: always() && !cancelled()
steps:
- name: Check job status
uses: actions/github-script@v7
env:
needs: ${{ toJson(needs) }}
with:
script: | # js
const needs = JSON.parse(process.env.needs);
const jobs = Object.entries(needs).map(
([jobName, jobValues]) => ({
name: jobName,
result: jobValues.result
}));
// are all jobs skipped or successful?
if (jobs.every(job => (job.result === 'skipped' || job.result === 'success'))) {
console.log("");
console.log(" _------. ");
console.log(" / , \_ ");
console.log(" / / /{}\ |o\_ ");
console.log(" / \ `--' /-' \ ");
console.log(" | \ \ | ");
console.log(" | |`-, | ");
console.log(" / /__/)/ ");
console.log(" | | ");
console.log("");
console.log("All semantic search tests have passed (or have been skipped). Cam is very proud of you.");
process.exit(0);
}
// otherwise, something failed
console.log("");
console.log(" .::::::::::. .::::::::::. ");
console.log(" .::::''''''::::. .::::''''''::::. ");
console.log(" .:::' `::::.... ....::::' `:::. ");
console.log(" .::' `:::::::| |:::::::' `::. ");
console.log(" .::| |::::::|_ ___ __|::::::| |::. ");
console.log(" `--' |::::::|_()__()_|::::::| `--' ");
console.log(" ::: |::-o::| |::o-::| ::: ");
console.log(" `::. .|::::::| |::::::|. .::' ");
console.log(" `:::. .::\-----' `-----/::. .:::' ");
console.log(" `::::......::::' `::::......::::' ");
console.log(" `::::::::::' `::::::::::' ");
console.log("");
console.log("Semantic search tests have failed. You have been sentenced to MetaJail.");
jobs.forEach((job) => {
if (job.result !== 'success') {
console.log(`${job.name} - ${job.result}`);
}
});
process.exit(1);

View File

@@ -1,21 +0,0 @@
name: Uberjar Containerize labeled
on:
pull_request:
types: [ labeled, opened, synchronize, reopened ]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
uberjar:
if: ${{ contains(github.event.pull_request.labels.*.name, 'build-docker-uberjar') || github.event.label.name == 'build-docker-uberjar' }}
uses: ./.github/workflows/uberjar.yml
secrets: inherit
containerize:
needs: [uberjar]
if: ${{ contains(github.event.pull_request.labels.*.name, 'build-docker-uberjar') || github.event.label.name == 'build-docker-uberjar' }}
uses: ./.github/workflows/containerize-uberjar.yml
secrets: inherit

View File

@@ -1,16 +1,17 @@
name: Build Uberjar
name: Build + Docker Uberjar
# This workflow builds an uberjar and pushes it to dockerhub for any branch, either on-demand,
# or automatically for the listed branches
concurrency:
group: uberjar-${{ github.event.inputs.commit || github.event.pull_request.head.sha || github.sha }}
group: uberjar-${{ github.ref }}
cancel-in-progress: false
on:
push:
branches:
- master
- metabot-v3-main
- internal-tools
paths-ignore:
# config files
@@ -27,6 +28,11 @@ on:
commit:
description: "Optional full-length commit SHA-1 hash"
workflow_call:
inputs:
source_changed:
description: "Whether source code has changed"
type: boolean
default: true
jobs:
check-existing-artifacts:
@@ -37,41 +43,38 @@ jobs:
oss_exists: ${{ steps.check-oss.outcome == 'success' }}
should_build: ${{ steps.decide.outputs.should_build }}
steps:
- name: Check out the code
uses: actions/checkout@v4
- name: Try to fetch existing EE artifact
- name: Try to download existing EE artifact
id: check-ee
continue-on-error: true
uses: ./.github/actions/fetch-artifact
uses: actions/download-artifact@v4
with:
name: metabase-ee-${{ github.event.inputs.commit || github.event.pull_request.head.sha || github.sha }}-uberjar
output-name: metabase-ee.jar
- name: Try to fetch existing OSS artifact
- name: Try to download existing OSS artifact
id: check-oss
continue-on-error: true
uses: ./.github/actions/fetch-artifact
uses: actions/download-artifact@v4
with:
name: metabase-oss-${{ github.event.inputs.commit || github.event.pull_request.head.sha || github.sha }}-uberjar
output-name: metabase-oss.jar
- name: Decide if build is needed
id: decide
run: |
SOURCE_CHANGED="${{ inputs.source_changed == true }}"
EE_EXISTS="${{ steps.check-ee.outcome == 'success' }}"
OSS_EXISTS="${{ steps.check-oss.outcome == 'success' }}"
echo "Source changed: $SOURCE_CHANGED"
echo "EE exists: $EE_EXISTS"
echo "OSS exists: $OSS_EXISTS"
# Build only if artifacts don't exist for this commit hash
# Since we build by hash, if artifacts exist for this hash, we don't need to rebuild
if [[ "$EE_EXISTS" == "false" ]] || [[ "$OSS_EXISTS" == "false" ]]; then
# Build if source changed OR artifacts don't exist
if [[ "$SOURCE_CHANGED" == "true" ]] || [[ "$EE_EXISTS" == "false" ]] || [[ "$OSS_EXISTS" == "false" ]]; then
SHOULD_BUILD="true"
echo "Will build: ee_exists=$EE_EXISTS, oss_exists=$OSS_EXISTS"
echo "Will build: source_changed=$SOURCE_CHANGED, ee_exists=$EE_EXISTS, oss_exists=$OSS_EXISTS"
else
SHOULD_BUILD="false"
echo "Skipping build: artifacts exist for this commit hash"
echo "Skipping build: artifacts exist and no source changes"
fi
echo "should_build=$SHOULD_BUILD" >> $GITHUB_OUTPUT
@@ -107,17 +110,63 @@ jobs:
node_version: requirements.node,
};
get-container-info:
runs-on: ubuntu-22.04
timeout-minutes: 5
name: Get container info
# Skip if this is a PR without the build-docker-uberjar label
if: github.ref_name == 'master' || contains(github.event.pull_request.labels.*.name, 'build-docker-uberjar')
outputs:
editions: ${{ toJson(fromJson(steps.info.outputs.result).editions) }}
ee: ${{ toJson(fromJson(steps.info.outputs.result).ee) }}
oss: ${{ toJson(fromJson(steps.info.outputs.result).oss) }}
steps:
- name: Get docker repo and tag
id: info
uses: actions/github-script@v7
with:
script: | # js
const branch = '${{ github.head_ref || github.ref_name }}';
const owner = '${{ github.repository_owner }}';
console.log(`Branch: ${branch}`);
if (branch === "master") {
return {
editions: ["ee", "oss"],
ee: {
repo: `${owner}/metabase-enterprise-head`,
tag: "latest"
},
oss: {
repo: `${owner}/metabase-head`,
tag: "latest"
}
};
}
return {
editions: ["ee"],
ee: {
repo: `${owner}/metabase-dev`,
tag: branch.replaceAll("/", "-"), // slashes make docker sad
},
};
- name: Print output
run: |
echo "${{ steps.info.outputs.result }}"
build:
needs:
[check-existing-artifacts, get-build-requirements]
[check-existing-artifacts, get-build-requirements, get-container-info]
if: always() && !cancelled() && needs.check-existing-artifacts.outputs.should_build == 'true' && (needs.get-build-requirements.result == 'success' || needs.get-build-requirements.result == 'skipped')
name: Build MB ${{ matrix.edition }}
runs-on: ubuntu-22.04
timeout-minutes: 15
strategy:
matrix:
edition: ["ee", "oss"]
edition: ${{ needs.get-container-info.result == 'success' && fromJson(needs.get-container-info.outputs.editions) || fromJson('["ee", "oss"]') }}
env:
MB_EDITION: ${{ matrix.edition }}
INTERACTIVE: false
@@ -146,16 +195,14 @@ jobs:
check-jar-health:
runs-on: ubuntu-22.04
name: Is ${{ matrix.edition }} (java ${{ matrix.java-version }}) healthy?
needs: [build, check-existing-artifacts]
needs: [build, check-existing-artifacts, get-container-info]
if: always() && !cancelled() && (needs.build.result == 'success' || (needs.check-existing-artifacts.outputs.should_build == 'false' && needs.check-existing-artifacts.outputs.ee_exists == 'true' && needs.check-existing-artifacts.outputs.oss_exists == 'true'))
timeout-minutes: 10
strategy:
matrix:
edition: ["ee", "oss"]
edition: ${{ needs.get-container-info.result == 'success' && fromJson(needs.get-container-info.outputs.editions) || fromJson('["ee", "oss"]') }}
java-version: [21]
steps:
- name: Check out the code
uses: actions/checkout@v4
- name: Prepare JRE (Java Run-time Environment)
uses: actions/setup-java@v4
with:
@@ -163,12 +210,57 @@ jobs:
java-version: ${{ matrix.java-version }}
distribution: "temurin"
- run: java -version
- name: Retrieve uberjar artifact
uses: ./.github/actions/fetch-artifact
- uses: actions/download-artifact@v4
name: Retrieve uberjar artifact
with:
name: metabase-${{ matrix.edition }}-${{ github.event.inputs.commit || github.event.pull_request.head.sha || github.sha }}-uberjar
- name: Launch uberjar
run: >-
java --add-opens java.base/java.nio=ALL-UNNAMED -jar ./metabase.jar &
java --add-opens java.base/java.nio=ALL-UNNAMED -jar ./target/uberjar/metabase.jar &
- name: Wait for Metabase to start
run: while ! curl 'http://localhost:3000/api/health' | grep '{"status":"ok"}'; do sleep 1; done
containerize:
needs: [check-jar-health, get-container-info, build]
if: needs.get-container-info.result == 'success' && needs.build.result == 'success'
strategy:
matrix:
edition: ${{ fromJson(needs.get-container-info.outputs.editions) }}
uses: ./.github/workflows/containerize-jar.yml
secrets: inherit
with:
artifact-name: metabase-${{ matrix.edition }}-${{ github.event.inputs.commit || github.event.pull_request.head.sha || github.sha }}-uberjar
commit: ${{ github.event.inputs.commit || github.sha }}
repo: ${{ fromJson(needs.get-container-info.outputs[matrix.edition]).repo }}
tag: ${{ fromJson(needs.get-container-info.outputs[matrix.edition]).tag }}
release: "false"
build-args: |
GIT_COMMIT_SHA=${{ github.event.inputs.commit || github.sha }}
run-trivy:
needs: [get-container-info, containerize]
strategy:
matrix:
edition: ${{ fromJson(needs.get-container-info.outputs.editions) }}
runs-on: ubuntu-22.04
name: Run Trivy vulnerability scanner
steps:
- name: Check out the code
uses: actions/checkout@v4
with:
ref: ${{ github.event.inputs.commit || github.sha }}
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@0.29.0
env:
TRIVY_OFFLINE_SCAN: true
repo: ${{ fromJson(needs.get-container-info.outputs[matrix.edition]).repo }}
tag: ${{ fromJson(needs.get-container-info.outputs[matrix.edition]).tag }}
with:
image-ref: docker.io/${{ env.repo }}:${{ env.tag }}
format: sarif
output: trivy-results.sarif
version: "v0.57.1"
- name: Upload Trivy scan results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: "trivy-results.sarif"

View File

@@ -1,92 +0,0 @@
name: Update E2E Timings
permissions:
contents: write
pull-requests: write
on:
workflow_dispatch:
inputs:
days_back:
description: "Number of days to look back for timing artifacts"
type: number
default: 7
required: false
schedule:
# Run weekly on Sundays at 6 AM UTC
- cron: "0 6 * * 0"
jobs:
collect-and-average-timings:
runs-on: ubuntu-22.04
timeout-minutes: 30
env:
DAYS_BACK: ${{ inputs.days_back || 7 }}
steps:
- uses: actions/checkout@v4
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: 22
- name: Collect and average timing data
id: collect-timings
uses: actions/github-script@v7
with:
script: |
const { collectAndAverageTimings } = require('${{ github.workspace }}/.github/scripts/average-e2e-timings.js');
const result = await collectAndAverageTimings({
github,
context,
token: '${{ secrets.GITHUB_TOKEN }}',
daysBack: ${{ env.DAYS_BACK }},
existingTimingsPath: 'e2e/support/timings.json'
});
core.setOutput('artifact-count', result.artifactCount);
- name: Set up git configuration
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
BRANCH_NAME="update-e2e-timings-$(date +%Y%m%d-%H%M%S)"
echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_ENV
echo "Branch name will be: $BRANCH_NAME"
- name: Create Pull Request
uses: peter-evans/create-pull-request@v6
with:
token: ${{ secrets.GITHUB_TOKEN }}
branch: ${{ env.BRANCH_NAME }}
base: master
commit-message: |
Update E2E test timings with averaged data
Collected timing data from ${{ env.DAYS_BACK }} days of test runs.
All specs with timing data were included (simple averages).
🤖 Generated automatically by update-e2e-timings workflow
Co-Authored-By: GitHub Actions <actions@github.com>
committer: "github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>"
author: "github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>"
title: "[chore] Update E2E test timings with recent data"
body: |
## Summary
This PR updates the E2E test timing data with averaged values from recent test runs.
## Details
- Collected data from the last **${{ env.DAYS_BACK }} days** of test runs
- Averaged timing data from **${{ steps.collect-timings.outputs.artifact-count }}** test runs
## Generated automatically
🤖 This PR was created automatically by the `update-e2e-timings` workflow.
The timing data helps cypress-split optimize test distribution across parallel jobs.
Co-Authored-By: GitHub Actions <actions@github.com>
labels: |
.CI & Tests
add-paths: e2e/support/timings.json

11
.gitignore vendored
View File

@@ -61,9 +61,7 @@ cypress.env.json
/resources/frontend_client/app/dist/
/resources/frontend_client/app/locales
/resources/frontend_client/app/embed.js
/resources/frontend_client/app/embedding-sdk.js
/resources/frontend_client/app/embedding-sdk.js.map
/resources/frontend_client/tmp-*/
/resources/frontend_client/tmp-embed-js/
/resources/frontend_client/embed.html
/resources/frontend_client/embed-sdk.html
/resources/frontend_client/index.html
@@ -98,13 +96,6 @@ xcuserdata
dev/src/dev/nocommit/
dev/serialization_deltas/
*~
# TypeScript compilation artifacts
/build/
*.tsbuildinfo
/trace-output/
/temp-types/
**/cypress_sample_database.json
**/cypress_sample_instance_data.json
/frontend/src/cljs

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 35 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 48 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 44 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 43 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 37 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 40 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 88 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 8.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 39 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 39 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 62 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 56 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 56 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 62 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 55 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 60 KiB

Some files were not shown because too many files have changed in this diff Show More