Compare commits

..

55 Commits

Author SHA1 Message Date
Nicolò Pretto
7ca963d0eb hackathon: embedding sdk plugin 2024-03-07 09:20:41 +03:00
Denis Berezin
7eeae6f210 Working solution 2024-03-06 16:25:13 +03:00
Denis Berezin
cb53be1b17 Add api key env 2024-03-06 09:52:34 +03:00
Oisin Coveney
324dd8cf9a Add toggles + switchers for themes, parameters, and styling options 2024-03-01 18:51:43 +03:00
Oisin Coveney
bfb46008a9 remove location from SDK Dashboard 2024-03-01 12:45:00 +03:00
Oisin Coveney
2cb07d3734 Continue static dashboards 2024-02-28 19:43:03 +01:00
Oisin Coveney
3984950693 Continue moving location props out of dashboard components 2024-02-28 12:51:02 +01:00
Oisin Coveney
5e6b42b4ee Move types around on EmbedFrame 2024-02-27 18:52:32 +01:00
Oisin Coveney
e8813d5d09 Add really dirty version of dashboards 2024-02-27 18:16:30 +01:00
Oisin Coveney
1ed70d8c6b save progress on dashboard 2024-02-27 09:53:17 +01:00
Oisin Coveney
da41baa8c5 Formatting and make the style leak flag more useful 2024-02-23 12:38:39 +00:00
Oisin Coveney
ba5940934d Formatting 2024-02-22 12:23:42 +00:00
Oisin Coveney
27007c93ef Quick font adjustment 2024-02-22 10:55:06 +00:00
Oisin Coveney
2c8396318c Add style leak flag to indicate whether a font is leaky 2024-02-21 21:02:11 +00:00
Oisin Coveney
5b14bfa69a Add tailwind and modify app to show off a little more 2024-02-21 20:36:30 +00:00
Oisin Coveney
a03ffb1059 Ensure that public hooks are exposed 2024-02-21 13:39:34 +00:00
Oisin Coveney
1a16919b24 Remove apiUrl from EmbeddingContext 2024-02-21 11:16:34 +00:00
Oisin Coveney
10df8c09f3 Separate hooks and components into public and private folders 2024-02-21 11:04:32 +00:00
Oisin Coveney
a82d370f78 Remove log file from font util API 2024-02-21 10:17:02 +00:00
Oisin Coveney
2228762fa2 Add font selector component to host app and hook for getting and setting fonts within the EmbeddingContext 2024-02-20 18:55:51 +00:00
Oisin Coveney
c129197c6a Reconcile new auth loading, fonts, and config 2024-02-20 09:36:58 +00:00
Oisin Coveney
3502e49eb3 Hide JWT 2024-02-19 20:27:41 +00:00
Denis Berezin
6f37b13e85 Update for default settings 2024-02-15 00:57:25 +07:00
Denis Berezin
96a76bf8d0 Add dynamic api key / jwt usage based on env 2024-02-14 21:27:31 +07:00
Oisin Coveney
66aa4e7da3 Move backend folder to new repo and return files to their rightful place 2024-02-13 17:18:07 +01:00
Oisin Coveney
b7b7d56ab7 Add simple README and redux 2024-02-13 11:06:58 +01:00
Oisin Coveney
69700f56fd Add backend 2024-02-12 17:19:44 +01:00
Oisin Coveney
97d9792dda Add backend 2024-02-12 17:16:20 +01:00
Oisin Coveney
52771825f6 Add embedding check for auth/sso endpoint 2024-02-12 14:39:01 +01:00
Oisin Coveney
51d4b37074 Merge remote-tracking branch 'origin/master' into embedding-sdk-poc-denis 2024-02-12 13:13:42 +01:00
Oisin Coveney
d7a37b178c Move some files around and include the nodejs backend as a new folder 2024-02-09 16:44:50 +01:00
Oisin Coveney
2a0f647c31 Add hooks for managing data and tokens 2024-02-08 20:39:48 +01:00
Oisin Coveney
efa83b6265 Add ability to use session tokens with JWT SSO 2024-02-07 19:05:01 +01:00
Denis Berezin
f9a3177314 Fix user loading, demo prep 2024-02-02 05:10:36 +07:00
Denis Berezin
ac53cf512f Merge branch 'embedding-sdk-poc-denis' of github.com:metabase/metabase into embedding-sdk-poc-denis
# Conflicts:
#	frontend/src/embedding-sdk/QueryVisualization.tsx
#	frontend/src/embedding-sdk/SdkProvider.tsx
#	host-app-example/src/App.css
#	host-app-example/src/App.js
#	host-app-example/src/ChooseQuestionInput.js
2024-02-02 03:50:06 +07:00
Denis Berezin
b10d5dab59 Update demo 2024-02-02 02:11:44 +07:00
Oisin Coveney
ac9b9620d8 Remove unneeded loads for settings + properties 2024-02-01 18:01:27 +00:00
Oisin Coveney
d34503373d Adjust to make sure that fonts are initialized 2024-02-01 17:55:12 +00:00
Oisin Coveney
71952ab1a3 Add question list hook and move MetabaseProvider to root 2024-02-01 17:46:34 +00:00
Oisin Coveney
9bdd7e926c Add question search hook and question list 2024-02-01 16:33:17 +00:00
Oisin Coveney
eb8e199625 Change import order to make things work 2024-02-01 16:31:17 +00:00
Oisin Coveney
3718f2d3ae Remove useApi 2024-02-01 14:13:15 +00:00
Oisin Coveney
b704429325 Remove unneeded files, add hooks, change styles 2024-02-01 14:09:38 +00:00
Oisin Coveney
7653d2fe06 Move css to styled components 2024-02-01 12:51:51 +00:00
Oisin Coveney
27b7ade789 Merge changes 2024-02-01 11:33:25 +00:00
Denis Berezin
7f38319360 Add fonts loading 2024-01-31 23:50:29 +07:00
Denis Berezin
f612a5e5e7 Fix css and emotion global styles 2024-01-30 23:33:39 +07:00
Oisin Coveney
1bb09a7c4c Allow for changes in visualization type 2024-01-29 15:12:51 -06:00
Denis Berezin
470cc7d9b1 Add basic css loading 2024-01-30 03:08:19 +07:00
Denis Berezin
a4599b3432 Fix api data loading 2024-01-30 03:02:25 +07:00
Denis Berezin
17aa273aea Add api token usage 2024-01-29 23:05:50 +07:00
Denis Berezin
95c356ec88 Fixes 2024-01-27 01:02:23 +07:00
Denis Berezin
5330bb6b2f Add bundling and sample app 2024-01-27 00:10:16 +07:00
Denis Berezin
f451285bef Add basic question and visualization bar 2024-01-26 01:48:30 +07:00
Denis Berezin
25cd3c4f54 Draft sdk structure 2024-01-24 22:36:13 +07:00
7654 changed files with 344426 additions and 790873 deletions

File diff suppressed because it is too large Load Diff

View File

@@ -1,7 +1,6 @@
(ns hooks.clojure.core
(:require
[clj-kondo.hooks-api :as hooks]
[clojure.set :as set]
[clojure.string :as str]))
(defn- node->qualified-symbol [node]
@@ -16,7 +15,7 @@
(catch Exception _
nil)))
(def ^:private symbols-allowed-in-fns-not-ending-in-an-exclamation-point
(def ^:private white-card-symbols
'#{;; these toucan methods might actually set global values if it's used outside of a transaction,
;; but since mt/with-temp runs in a transaction, so we'll ignore them in this case.
toucan2.core/delete!
@@ -44,9 +43,6 @@
methodical.core/remove-aux-method-with-unique-key!
next.jdbc/execute!
;; Definitely thread safe
metabase.test.util.dynamic-redefs/patch-vars!
;; TODO: most of these symbols shouldn't be here, we should go through them and
;; find the functions/macros that use them and make sure their names end with !
;; best way to do this is try remove each of these and rely on kondo output to find places where it's used
@@ -69,7 +65,6 @@
metabase.db.schema-migrations-test.impl/run-migrations-in-range!
metabase.db.setup/migrate!
metabase.db.setup/setup-db!
metabase.db/migrate!
metabase.db/setup-db!
metabase.driver.mongo-test/create-database-from-row-maps!
metabase.driver.postgres-test/create-enums-db!
@@ -122,8 +117,7 @@
metabase.test.data.bigquery-cloud-sdk/execute!
metabase.test.data.impl/copy-db-tables-and-fields!
metabase.test.data.impl/get-or-create-database!
metabase.test.data.impl/get-or-create-default-dataset!
metabase.test.data.impl.get-or-create/set-test-db-permissions!
metabase.test.data.impl/get-or-create-test-data-db!
metabase.test.data.interface/create-db!
metabase.test.data.interface/destroy-db!
metabase.test.data.oracle/create-user!
@@ -160,7 +154,7 @@
(walk f child)))]
(walk (fn [form]
(when-let [qualified-symbol (node->qualified-symbol form)]
(when (and (not (contains? symbols-allowed-in-fns-not-ending-in-an-exclamation-point qualified-symbol))
(when (and (not (contains? white-card-symbols qualified-symbol))
(end-with-exclamation? qualified-symbol))
(hooks/reg-finding! (assoc (meta form-name)
:message (format "The name of this %s should end with `!` because it contains calls to non thread safe form `%s`."
@@ -204,126 +198,3 @@
:findings))
(do (non-thread-safe-form-should-end-with-exclamation* (hooks/parse-string form)) nil))
(defn- ns-form-node->require-node [ns-form-node]
(some (fn [node]
(when (and (hooks/list-node? node)
(let [first-child (first (:children node))]
(and (hooks/keyword-node? first-child)
(= (hooks/sexpr first-child) :require))))
node))
(:children ns-form-node)))
(defn- lint-require-shapes [ns-form-node]
(doseq [node (-> ns-form-node
ns-form-node->require-node
:children
rest)]
(cond
(not (hooks/vector-node? node))
(hooks/reg-finding! (assoc (meta node)
:message "All :required namespaces should be wrapped in vectors [:metabase/require-shape-checker]"
:type :metabase/require-shape-checker))
(hooks/vector-node? (second (:children node)))
(hooks/reg-finding! (assoc (meta node)
:message "Don't use prefix forms inside :require [:metabase/require-shape-checker]"
:type :metabase/require-shape-checker)))))
(defn- require-node->namespace-symb-nodes [require-node]
(let [[_ns & args] (:children require-node)]
(into []
;; prefixed namespace forms are NOT SUPPORTED!!!!!!!!1
(keep (fn [node]
(cond
(hooks/vector-node? node)
;; propagate the metadata attached to this vector in case there's a `:clj-kondo/ignore` form.
(vary-meta (first (:children node)) (partial merge (meta require-node) (meta node)))
;; this should also be dead code since we require requires to be vectors
(hooks/token-node? node)
(vary-meta node (partial merge (meta require-node)))
:else
(printf "Don't know how to figure out what namespace is being required in %s\n" (pr-str node)))))
args)))
(defn- ns-form-node->ns-symb [ns-form-node]
(some-> (some (fn [node]
(when (and (hooks/token-node? node)
(not= (hooks/sexpr node) 'ns))
node))
(:children ns-form-node))
hooks/sexpr))
(defn- module
"E.g.
(module 'metabase.qp.middleware.wow) => 'metabase.qp"
[ns-symb]
(some-> (re-find #"^metabase\.[^.]+" (str ns-symb)) symbol))
(defn- ignored-namespace? [ns-symb config]
(some
(fn [pattern-str]
(re-find (re-pattern pattern-str) (str ns-symb)))
(:ignored-namespace-patterns config)))
(defn- module-api-namespaces
"Set API namespaces for a given module. `:any` means you can use anything, there are no API namespaces for this
module (yet). If unspecified, the default is just the namespace with the same name as the module e.g.
`metabase.db`."
[module config]
(let [module-config (get-in config [:api-namespaces module])]
(cond
(= module-config :any)
nil
(set? module-config)
module-config
:else
#{module})))
(defn- lint-modules [ns-form-node config]
(let [ns-symb (ns-form-node->ns-symb ns-form-node)]
(when-not (ignored-namespace? ns-symb config)
(when-let [current-module (module ns-symb)]
(let [allowed-modules (get-in config [:allowed-modules current-module])
required-namespace-symb-nodes (-> ns-form-node
ns-form-node->require-node
require-node->namespace-symb-nodes)]
(doseq [node required-namespace-symb-nodes
:let [clj-kondo-ignore (some-> (meta node) :clj-kondo/ignore hooks/sexpr set)]
:when (not (contains? clj-kondo-ignore :metabase/ns-module-checker))
:let [required-namespace (hooks/sexpr node)
required-module (module required-namespace)]
;; ignore stuff not in a module i.e. non-Metabase stuff.
:when required-module
:let [in-current-module? (= required-module current-module)]
:when (not in-current-module?)
:let [allowed-module? (or (= allowed-modules :any)
(contains? (set allowed-modules) required-module))
module-api-namespaces (module-api-namespaces required-module config)
allowed-module-namespace? (or (empty? module-api-namespaces)
(contains? module-api-namespaces required-namespace))]]
(when-let [error (cond
(not allowed-module?)
(format "Module %s should not be used in the %s module. [:metabase/ns-module-checker :allowed-modules %s]"
required-module
current-module
current-module)
(not allowed-module-namespace?)
(format "Namespace %s is not an allowed external API namespace for the %s module. [:metabase/ns-module-checker :api-namespaces %s]"
required-namespace
required-module
required-module))]
(hooks/reg-finding! (assoc (meta node)
:message error
:type :metabase/ns-module-checker)))))))))
(defn lint-ns [x]
(lint-require-shapes (:node x))
(lint-modules (:node x) (get-in x [:config :linters :metabase/ns-module-checker]))
x)

View File

@@ -1,8 +1,6 @@
(ns hooks.clojure.test
(:require
[clj-kondo.hooks-api :as hooks]
[clojure.string :as str]
[hooks.common]))
(:require [clj-kondo.hooks-api :as hooks]
[clojure.string :as str]))
(def ^:private disallowed-parallel-forms
"Things you should not be allowed to use inside parallel tests. Besides these, anything ending in `!` not whitelisted
@@ -10,11 +8,11 @@
'#{clojure.core/alter-var-root
clojure.core/with-redefs
clojure.core/with-redefs-fn
metabase-enterprise.sandbox.test-util/with-gtaps!
metabase-enterprise.sandbox.test-util/with-gtaps-for-user!
metabase-enterprise.sandbox.test-util/with-gtaps
metabase-enterprise.sandbox.test-util/with-gtaps-for-user
metabase-enterprise.sandbox.test-util/with-user-attributes
metabase-enterprise.test/with-gtaps!
metabase-enterprise.test/with-gtaps-for-user!
metabase-enterprise.test/with-gtaps
metabase-enterprise.test/with-gtaps-for-user
metabase-enterprise.test/with-user-attributes
metabase.actions.test-util/with-actions
metabase.actions.test-util/with-actions-disabled
@@ -30,7 +28,9 @@
metabase.test.util.log/with-log-level
metabase.test.util.log/with-log-messages-for-level
metabase.test.util.misc/with-single-admin-user
metabase.test.util.timezone/with-system-timezone-id
metabase.test.util/with-all-users-permission
metabase.test.util/with-column-remappings
metabase.test.util/with-discarded-collections-perms-changes
metabase.test.util/with-env-keys-renamed-by
metabase.test.util/with-locale
@@ -48,6 +48,7 @@
metabase.test/with-actions-test-data-and-actions-enabled
metabase.test/with-actions-test-data-tables
metabase.test/with-all-users-permission
metabase.test/with-column-remappings
metabase.test/with-discarded-collections-perms-changes
metabase.test/with-env-keys-renamed-by
metabase.test/with-expected-messages
@@ -61,6 +62,7 @@
metabase.test/with-non-admin-groups-no-root-collection-perms
metabase.test/with-persistence-enabled
metabase.test/with-single-admin-user
metabase.test/with-system-timezone-id
metabase.test/with-temp-env-var-value
metabase.test/with-temp-vals-in-db
metabase.test/with-temporary-raw-setting-values
@@ -106,9 +108,20 @@
clojure.core.async/to-chan!
clojure.core.async/to-chan!!
metabase.driver.sql-jdbc.execute/execute-prepared-statement!
metabase.pulse/send-pulse!
metabase.query-processor.store/store-database!
next.jdbc/execute!})
metabase.query-processor/process-query-and-save-execution!
metabase.query-processor/process-query-and-save-with-max-results-constraints!
metabase.query-processor.store/store-database!})
(defn- node->qualified-symbol [node]
(try
(when (hooks/token-node? node)
(let [sexpr (hooks/sexpr node)]
(when (symbol? sexpr)
(when-let [resolved (hooks/resolve {:name sexpr})]
(symbol (name (:ns resolved)) (name (:name resolved)))))))
;; some symbols like `*count/Integer` aren't resolvable.
(catch Exception _
nil)))
(defn- warn-about-disallowed-parallel-forms [form]
(letfn [(error! [form message]
@@ -116,7 +129,7 @@
:message message
:type :metabase/validate-deftest)))
(f [form]
(when-let [qualified-symbol (hooks.common/node->qualified-symbol form)]
(when-let [qualified-symbol (node->qualified-symbol form)]
(cond
(disallowed-parallel-forms qualified-symbol)
(error! form (format "%s is not allowed inside a ^:parallel test or test fixture" qualified-symbol))

View File

@@ -1,7 +1,6 @@
(ns hooks.common
(:require
[clj-kondo.hooks-api :as hooks]
[clojure.pprint]))
(:require [clj-kondo.hooks-api :as hooks]
[clojure.pprint]))
(defn with-macro-meta
"When introducing internal nodes (let, defn, etc) it is important to provide a meta of an existing token
@@ -370,33 +369,3 @@
arg])
body))]
{:node node*}))
(defn node->qualified-symbol [node]
(try
(when (hooks/token-node? node)
(let [sexpr (hooks/sexpr node)]
(when (symbol? sexpr)
(when-let [resolved (hooks/resolve {:name sexpr})]
(symbol (name (:ns resolved)) (name (:name resolved)))))))
;; some symbols like `*count/Integer` aren't resolvable.
(catch Exception _
nil)))
(defn format-string-specifier-count
"Number of things like `%s` in a format string, not counting newlines (`%n`) or escaped percent signs (`%%`). For
checking the number of args to something that takes a format string."
[format-string]
(count (re-seq #"(?<!%)%(?![%n])" format-string)))
(comment
;; should be 1
(format-string-specifier-count "%s %%")
;; should be 2
(format-string-specifier-count "%s %%%n%s")
;; should be 0
(format-string-specifier-count "%n%%%%")
;; should be 1
(format-string-specifier-count "%-02d"))

View File

@@ -0,0 +1,29 @@
(ns hooks.metabase.api.common
(:require
[clj-kondo.hooks-api :as hooks]
[clojure.string :as str]
[hooks.common :as common]))
(defn route-fn-name
"route fn hook"
[method route]
(let [route (if (vector? route) (first route) route)]
(-> (str (name method) route)
(str/replace #"/" "_")
symbol)))
(defn defendpoint
"defendpoint hook"
[{:keys [node]}]
(let [[method route & body] (rest (:children node))]
{:node
(hooks/vector-node [;; register usage of compojure core var
(common/with-macro-meta (hooks/token-node (symbol (str "compojure.core")
(str method)))
node)
;; define function with route-fn-name
(hooks/list-node
(list* (common/with-macro-meta (hooks/token-node 'clojure.core/defn)
node)
(route-fn-name (hooks/sexpr method) (hooks/sexpr route))
body))])}))

View File

@@ -1,18 +0,0 @@
(ns hooks.metabase.driver
(:require
[clj-kondo.hooks-api :as api]))
(defn register!
"Register the driver keyword so LSP can jump to it."
[x]
(letfn [(update-driver-keyword [k]
(api/reg-keyword! k 'metabase.driver/register!))
(update-children [[_register! k :as children]]
(if (api/keyword-node? k)
(update (vec children) 1 update-driver-keyword)
children))
(update-node [node]
(if (api/list-node? node)
(update node :children update-children)
node))]
(update x :node update-node)))

View File

@@ -1,69 +0,0 @@
(ns hooks.metabase.legacy-mbql.schemas.macros
(:require
[clj-kondo.hooks-api :as hooks]))
(defn- unwrap-defclause-clause-name
"The `clause-name-form` can be either a plain symbol, or a vector like `[var-name mbql-clause-name]`. For the vector
form, we are only interested in the `var-name`. For a plain symbol, the two names are the same. Return a token node."
[clause-name-form]
(-> (if (hooks/vector-node? clause-name-form)
(first (:children clause-name-form))
clause-name-form)
(with-meta (meta clause-name-form))))
(defn- collect-defclause-body-schemas
"Collect the argument schemas in the body of a `defclause` form into a vector node.
The body is a sequence of <argument-name> <argument-schema> pairs. We can ignore the argument names.
<argument-schema> can optionally be wrapped be `(optional ...)` or `(rest ...)`... we can ignore these."
[arg-specs]
(hooks/vector-node
(for [[_arg-name arg-schema] (partition-all 2 arg-specs)
:let [arg-schema (if (and (hooks/list-node? arg-schema)
('#{optional rest} (hooks/sexpr (first (:children arg-schema)))))
(-> (second (:children arg-schema))
(with-meta (meta arg-schema)))
arg-schema)]]
arg-schema)))
(defn defclause
"e.g.
(defclause [ag:var var] field-or-expression FieldOrExpressionDef)
=>
(def ag:var [FieldOrExpressionDef])"
[{:keys [node]}]
(let [[_defclause clause-name-form & arg-specs] (:children node)]
{:node (-> (hooks/list-node
(list
(hooks/token-node 'def)
(unwrap-defclause-clause-name clause-name-form)
(hooks/string-node "Docstring.")
(collect-defclause-body-schemas arg-specs)))
(with-meta (meta node)))}))
(comment
(defn- defclause* [form]
(hooks/sexpr
(:node
(defclause
{:node
(hooks/parse-string
(with-out-str
(clojure.pprint/pprint
form)))}))))
(defn- x []
(defclause* '(defclause [ag:var var]
field-or-expression FieldOrExpressionDef)))
;; =>
(def ag:var "Docstring." [FieldOrExpressionDef])
(defn- y []
(defclause* '(defclause var
x (some-list-schema)
y (optional FieldOrExpressionDef)
z (rest (some-other-schema)))))
;; =>
(def var "Docstring." [(some-list-schema) FieldOrExpressionDef (some-other-schema)]))

View File

@@ -1,21 +0,0 @@
(ns hooks.metabase.lib.schema.mbql-clause
(:require
[clj-kondo.hooks-api :as api]))
(defn define-mbql-clause
"Register the schema keyword so LSP can jump to it."
[x]
(letfn [(update-def-keyword [k]
(-> (api/keyword-node (keyword "mbql.clause" (name (api/sexpr k))))
(with-meta (meta k))
(api/reg-keyword! 'metabase.lib.schema.mbql-clause/define-mbql-clause)))
(update-children [[_def k :as children]]
(if (api/keyword-node? k)
(update (vec children) 1 update-def-keyword)
children))
(update-node [node]
(if (api/list-node? node)
(-> (api/list-node (update-children (:children node)))
(with-meta (meta node)))
node))]
(update x :node update-node)))

View File

@@ -0,0 +1,68 @@
(ns hooks.metabase.mbql.schemas.macros
(:require [clj-kondo.hooks-api :as hooks]))
(defn- unwrap-defclause-clause-name
"The `clause-name-form` can be either a plain symbol, or a vector like `[var-name mbql-clause-name]`. For the vector
form, we are only interested in the `var-name`. For a plain symbol, the two names are the same. Return a token node."
[clause-name-form]
(-> (if (hooks/vector-node? clause-name-form)
(first (:children clause-name-form))
clause-name-form)
(with-meta (meta clause-name-form))))
(defn- collect-defclause-body-schemas
"Collect the argument schemas in the body of a `defclause` form into a vector node.
The body is a sequence of <argument-name> <argument-schema> pairs. We can ignore the argument names.
<argument-schema> can optionally be wrapped be `(optional ...)` or `(rest ...)`... we can ignore these."
[arg-specs]
(hooks/vector-node
(for [[_arg-name arg-schema] (partition-all 2 arg-specs)
:let [arg-schema (if (and (hooks/list-node? arg-schema)
('#{optional rest} (hooks/sexpr (first (:children arg-schema)))))
(-> (second (:children arg-schema))
(with-meta (meta arg-schema)))
arg-schema)]]
arg-schema)))
(defn defclause
"e.g.
(defclause [ag:var var] field-or-expression FieldOrExpressionDef)
=>
(def ag:var [FieldOrExpressionDef])"
[{:keys [node]}]
(let [[_defclause clause-name-form & arg-specs] (:children node)]
{:node (-> (hooks/list-node
(list
(hooks/token-node 'def)
(unwrap-defclause-clause-name clause-name-form)
(hooks/string-node "Docstring.")
(collect-defclause-body-schemas arg-specs)))
(with-meta (meta node)))}))
(comment
(defn- defclause* [form]
(hooks/sexpr
(:node
(defclause
{:node
(hooks/parse-string
(with-out-str
(clojure.pprint/pprint
form)))}))))
(defn- x []
(defclause* '(defclause [ag:var var]
field-or-expression FieldOrExpressionDef)))
;; =>
(def ag:var "Docstring." [FieldOrExpressionDef])
(defn- y []
(defclause* '(defclause var
x (some-list-schema)
y (optional FieldOrExpressionDef)
z (rest (some-other-schema)))))
;; =>
(def var "Docstring." [(some-list-schema) FieldOrExpressionDef (some-other-schema)]))

View File

@@ -38,6 +38,8 @@
engines
enum-cardinality-threshold
follow-up-email-sent
ga-code
ga-enabled
google-auth-auto-create-accounts-domain
google-auth-client-id
google-auth-configured
@@ -63,7 +65,6 @@
jwt-user-provisioning-enabled?
jwt-shared-secret
last-acknowledged-version
last-used-native-database-id
ldap-attribute-email
ldap-attribute-firstname
ldap-attribute-lastname
@@ -90,8 +91,6 @@
metabot-get-prompt-templates-url
metabot-prompt-generator-token-limit
multi-setting-read-only
notebook-native-preview-shown
notebook-native-preview-sidebar-width
notification-link-base-url
num-metabot-choices
openai-api-key

View File

@@ -139,7 +139,7 @@
(when-not ((some-fn symbol? nil?) (hooks/sexpr table))
(hooks/reg-finding! (assoc (meta table)
:message "First arg to mbql-query should be either a table name symbol or nil."
:type :metabase/mbql-query-first-arg)))
:type ::mbql-query-first-arg)))
(let [result (replace-$id-special-tokens query)
;; HACK I'm not sure WHY it works but I ran into https://github.com/clj-kondo/clj-kondo/issues/1773 when
;; trying to get this working -- for some magical reason wrapping the whole thing in a `do` form seems to fix

View File

@@ -1,17 +0,0 @@
(ns hooks.metabase.util
(:require
[clj-kondo.hooks-api :as api]
[hooks.common]))
(defn format-color [{:keys [node], :as x}]
(let [[_format-color _color format-string-node & args] (:children node)]
(when (api/string-node? format-string-node)
(let [expected-arg-count (hooks.common/format-string-specifier-count (api/sexpr format-string-node))
actual-arg-count (count args)]
(when-not (= expected-arg-count actual-arg-count)
(api/reg-finding! (assoc (meta node)
:message (format "metabase.util/format-color format string expects %d arguments instead of %d."
expected-arg-count
actual-arg-count)
:type :format))))))
x)

View File

@@ -1,157 +0,0 @@
(ns hooks.metabase.util.log
(:require
[clj-kondo.hooks-api :as api]
[clojure.string :as str]
[clojure.walk :as walk]
[hooks.common]))
;;;; Checks for both log + logf
(defn- disallow-i18n [node]
(walk/postwalk
(fn [x]
(when-let [qualified-symbol (hooks.common/node->qualified-symbol x)]
(when (and (= (namespace qualified-symbol) "metabase.util.i18n")
(or (str/includes? (name qualified-symbol) "tru")
(str/includes? (name qualified-symbol) "trs")))
(api/reg-finding! (assoc (meta x)
:message "do not i18n the logs!"
:type :metabase/validate-logging)))))
node))
(defn- check-position-of-exception-arg
"Tell people to do (info e \"whatever\") instead of (info \"whatever\" e)."
[node]
(let [[f _first-arg & args] (:children node)]
(doseq [arg args]
(when (and (api/token-node? arg)
(= (api/sexpr arg) 'e))
(api/reg-finding! (assoc (meta arg)
:message (format "pass exceptions as the first arg to %s" (api/sexpr f))
:type :metabase/validate-logging))))))
(defn- common-checks [node]
(check-position-of-exception-arg node)
(disallow-i18n node))
(defn- check-for-i18n-format-specifiers
"Make sure people aren't using things like {0} in their log messages."
[node string-node]
(when (re-find #"\{\d+\}" (api/sexpr string-node))
(api/reg-finding! (assoc (meta node)
:message "this looks like an i18n format string. Don't use identifiers like {0} in logging."
:type :metabase/validate-logging))))
;;; log only
(defn- check-not-format-string
"Make sure we're not using something like log/trace with a format string."
[{[f] :children, :as node} string-node]
(when (pos? (hooks.common/format-string-specifier-count (api/sexpr string-node)))
(api/reg-finding! (assoc (meta node)
:message (format "%s used with a format string, use %s instead."
(api/sexpr f)
(str (api/sexpr f) "f"))
:type :metabase/validate-logging))))
(defn- warn-against-log+format-instead-of-logf
"Tell people to use things like infof instead of info + format."
[{[f] :children, :as node} [x :as args]]
(when (and (= (count args) 1)
(api/list-node? x))
(let [[y] (:children x)]
(when (and (api/token-node? y)
(= (hooks.common/node->qualified-symbol y) 'clojure.core/format))
(api/reg-finding! (assoc (meta node)
:message (format "Use %s instead of %s + format"
(str (api/sexpr f) "f")
(api/sexpr f))
:type :metabase/validate-logging))))))
(defn info
"Valid log, debug, etc."
[{:keys [node], :as x}]
(common-checks node)
(let [[_f & args] (:children node)
[_e & [x :as args]] (if (and (api/token-node? (first args))
(api/string-node? (second args)))
args
(cons nil args))]
(when (api/string-node? x)
(check-for-i18n-format-specifiers node x)
(check-not-format-string node x))
(warn-against-log+format-instead-of-logf node args))
x)
;;;; logf only
(defn- check-format-string-arg-count
"Make sure the number of args passed to something like tracef match the number of format string specifiers."
[{[f] :children, :as node} format-string args]
(let [expected-arg-count (hooks.common/format-string-specifier-count (api/sexpr format-string))
actual-arg-count (count args)]
(when (zero? expected-arg-count)
(api/reg-finding! (assoc (meta node)
:message (format "Don't use %s with no format string arguments, use %s instead."
(api/sexpr f)
(str/replace (str (api/sexpr f)) #"f$" ""))
:type :metabase/validate-logging)))
(when-not (= expected-arg-count actual-arg-count)
(api/reg-finding! (assoc (meta node)
:message (format "log format string expects %d arguments instead of %d."
expected-arg-count
actual-arg-count)
:type :metabase/validate-logging)))))
(defn infof
"Valid logf, debugf, etc."
[{:keys [node], :as x}]
(common-checks node)
(let [[_f & args] (:children node)
[_e format-string & args] (if (api/string-node? (first args))
(cons nil args)
args)]
;; TODO -- maybe we can error if you did not pass a format string, e.g. (log/infof 1 2) is obviously incorrect.
(when (api/string-node? format-string)
(check-for-i18n-format-specifiers node format-string)
(check-format-string-arg-count node format-string args)))
x)
(comment
;; should fail, missing a format arg
(infof
{:node (api/parse-string
(str '(log/warnf "Migration lock was acquired after %d retries.")))})
(infof
{:node (api/parse-string
(str '(log/warnf e "Migration lock was acquired after %d retries.")))})
;; should fail, too many format args
(infof
{:node (api/parse-string
(str '(log/warnf "Migration lock was acquired after %d retries." 1 2)))})
(infof
{:node (api/parse-string
(str '(log/warnf e "Migration lock was acquired after %d retries." 1 2)))})
(infof
{:node (api/parse-string
(str '(log/warnf "Migration lock was acquired after {0} retries." 1)))})
;; should fail, has format args but is warn rather than warnf
(info
{:node (api/parse-string
(str '(log/warn e "Migration lock was acquired after %d retries." 1 2)))})
;; should fail -- should not use i18n/tru or i18n/trs
(infof
{:node (api/parse-string
(str '(log/warnf (metabase.util.i18n/tru "Migration lock was acquired after {0} retries." 1))))})
(info
{:node (api/parse-string
(str '(log/warn (metabase.util.i18n/trs "Migration lock was acquired after {0} retries." 1))))}))

View File

@@ -1,26 +1,5 @@
(ns macros.metabase.api.common
(:require
[clojure.string :as str]))
(ns macros.metabase.api.common)
(defmacro define-routes
"Macro for api.common/define-routes"
[& args]
(defmacro define-routes [& args]
`(do (def ~'routes "docstring" nil)
~@args))
(defn route-fn-name
"route fn hook"
[method route]
(let [route (if (vector? route) (first route) route)]
(-> (str (name method) route)
(str/replace #"/" "_")
symbol)))
(defmacro defendpoint
"Macro for api.common/defendpoint*"
[method route & body]
(let [method-fn (symbol "compojure.core" (str method))
route-name (route-fn-name method route)]
`(do
~method-fn
(defn ~route-name ~@body))))

View File

@@ -0,0 +1,16 @@
(ns macros.metabase.domain-entities.malli)
(defmacro define-getters-and-setters [schema & defs]
`(do
(comment ~schema) ;; Reference the schema
~@(for [[sym _path] (partition 2 defs)]
`(do
(defn ~(vary-meta sym
assoc :export true)
"docs" [_x#] nil)
(defn ~(vary-meta (symbol (str "with-" (name sym)))
assoc :export true)
"docs" [x# _new#] x#)
(defn ~(vary-meta (symbol (str (name sym) "-js"))
assoc :export true)
"docs" [x#] x#)))))

View File

@@ -1,9 +0,0 @@
(ns macros.metabase.models.query-field-test
(:require [macros.common]))
(defmacro with-test-setup [& body]
`(let [~(macros.common/ignore-unused 'card-id) 1
~(macros.common/ignore-unused 'table-id) 2
~(macros.common/ignore-unused 'tax-id) 3
~(macros.common/ignore-unused 'total-id) 4]
~@body))

View File

@@ -1,16 +0,0 @@
(ns macros.metabase.xrays.domain-entities.malli)
(defmacro define-getters-and-setters [schema & defs]
`(do
(comment ~schema) ;; Reference the schema
~@(for [[sym _path] (partition 2 defs)]
`(do
(defn ~(vary-meta sym
assoc :export true)
"docs" [_x#] nil)
(defn ~(vary-meta (symbol (str "with-" (name sym)))
assoc :export true)
"docs" [x# _new#] x#)
(defn ~(vary-meta (symbol (str (name sym) "-js"))
assoc :export true)
"docs" [x#] x#)))))

View File

@@ -1,6 +0,0 @@
(ns macros.metabase-enterprise.advanced-permissions.api.util-test
(:require [macros.common]))
(defmacro with-impersonations! [impersonations-and-attributes-map & body]
`(let [~(macros.common/ignore-unused '&group) ~impersonations-and-attributes-map]
~@body))

View File

@@ -1,6 +1,6 @@
(ns macros.metabase-enterprise.sandbox.test-util
(:require [macros.common]))
(defmacro with-gtaps! [gtaps-definition & body]
(defmacro with-gtaps [gtaps-definition & body]
`(let [~(macros.common/ignore-unused '&group) ~gtaps-definition]
~@body))

View File

@@ -7,7 +7,7 @@
defprotocol+ [[:block 1] [:inner 1]]
potemkin/defprotocol+ [[:block 1] [:inner 1]]
let-404 [[:block 1]]
metabase.legacy-mbql.util/match-one [[:inner 0]]
metabase.mbql.util/match-one [[:inner 0]]
merge-with [[:inner 0]]
metabase.test/dataset [[:inner 0]]
metabase.test/formatted-rows [[:inner 0]]

View File

@@ -35,7 +35,6 @@
(eval . (put 'methodical/defmulti 'clojure-doc-string-elt 2))
(eval . (put 'mi/define-simple-hydration-method 'clojure-doc-string-elt 3))
(eval . (put 'mi/define-batched-hydration-method 'clojure-doc-string-elt 3))
(eval . (put 'mr/def 'clojure-doc-string-elt 2))
(eval . (put 'mu/defn 'clojure-doc-string-elt 2))
;; Define custom indentation for functions inside metabase.
;; This list isn't complete; add more forms as we come across them.
@@ -46,26 +45,25 @@
(eval . (put-clojure-indent 'c/step 1))
(eval . (put-clojure-indent 'impl/test-migrations 2))
(eval . (put-clojure-indent 'let-404 0))
(eval . (put-clojure-indent 'lib.schema.match/match 1))
(eval . (put-clojure-indent 'lib.schema.match/match-one 1))
(eval . (put-clojure-indent 'lib.schema.match/replace 1))
(eval . (put-clojure-indent 'lib.schema.match/replace-in 2))
(eval . (put-clojure-indent 'lib.util.match/replace 1))
(eval . (put-clojure-indent 'macros/case 0))
(eval . (put-clojure-indent 'match 1))
(eval . (put-clojure-indent 'mbql.match/match 1))
(eval . (put-clojure-indent 'mbql.match/match-one 1))
(eval . (put-clojure-indent 'mbql.match/replace 1))
(eval . (put-clojure-indent 'mbql.match/replace-in 2))
(eval . (put-clojure-indent 'mbql.u/replace 1))
(eval . (put-clojure-indent 'mi/define-methods '(:form)))
(eval . (put-clojure-indent 'mt/dataset 1))
(eval . (put-clojure-indent 'mt/format-rows-by 1))
(eval . (put-clojure-indent 'mt/query 1))
(eval . (put-clojure-indent 'mt/test-driver 1))
(eval . (put-clojure-indent 'mt/test-drivers 1))
(eval . (put-clojure-indent 'mt/test-driver 1))
(eval . (put-clojure-indent 'prop/for-all 1))
(eval . (put-clojure-indent 'qp.streaming/streaming-response 1))
(eval . (put-clojure-indent 'tc/quick-check 1))
(eval . (put-clojure-indent 'u/profile 1))
(eval . (put-clojure-indent 'u/prog1 1))
(eval . (put-clojure-indent 'u/select-keys-when 1))
(eval . (put-clojure-indent 'with-meta '(:form)))
(eval . (put-clojure-indent 'tc/quick-check 1))
;; these ones have to be done with `define-clojure-indent' for now because of upstream bug
;; https://github.com/clojure-emacs/clojure-mode/issues/600 once that's resolved we should use `put-clojure-indent'
;; instead. Please don't add new entries unless they don't work with `put-clojure-indent'
@@ -85,7 +83,6 @@
(clojure-indent-style . always-align)
(cljr-favor-prefix-notation . nil)
(cljr-insert-newline-after-require . t)
(cljr-print-right-margin . 118)
(clojure-docstring-fill-column . 118)
(cider-preferred-build-tool . clojure-cli)
(cider-default-cljs-repl . shadow-select)

View File

@@ -1,4 +1,2 @@
frontend/src/cljs
frontend/src/cljs_release
e2e/support/cypress_sample_database.js
e2e/support/cypress_sample_instance_data.js

216
.eslintrc Normal file
View File

@@ -0,0 +1,216 @@
{
"rules": {
"strict": [2, "never"],
"no-undef": 2,
"no-var": 1,
"no-unused-vars": [
"error",
{
"vars": "all",
"args": "none",
"varsIgnorePattern": "^_",
"ignoreRestSiblings": true
}
],
"no-empty": [1, { "allowEmptyCatch": true }],
"no-restricted-imports": [
"error",
{
"paths": [
{
"name": "moment",
"message": "Moment is deprecated, please use dayjs"
},
{
"name": "moment-timezone",
"message": "Moment is deprecated, please use dayjs"
}
]
}
],
"curly": [1, "all"],
"eqeqeq": [1, "smart"],
"import/no-default-export": 2,
"import/no-named-as-default": 0,
"import/no-commonjs": 1,
"import/order": [
"error",
{
"pathGroups": [
{
"pattern": "react",
"group": "builtin",
"position": "before"
},
{
"pattern": "metabase/**",
"group": "internal"
},
{
"pattern": "metabase/**/components",
"group": "internal",
"position": "before"
},
{
"pattern": "__**",
"group": "internal",
"position": "before"
},
{
"pattern": "metabase-lib/**",
"group": "internal",
"position": "after"
}
],
"groups": [
"builtin",
"external",
"internal",
"parent",
"sibling",
"index",
"object"
]
}
],
"no-console": [2, { "allow": ["warn", "error"] }],
"react/no-is-mounted": 2,
"react/prefer-es6-class": 2,
"react/display-name": 1,
"react/prop-types": 2,
"react/no-did-mount-set-state": 0,
"react/no-did-update-set-state": 0,
"react/no-find-dom-node": 0,
"react/no-children-prop": 2,
"react/no-string-refs": 2,
"react/no-unescaped-entities": 2,
"react/jsx-no-target-blank": 2,
"react/jsx-key": 2,
"react/forbid-component-props": [2, { "forbid": ["sx"] }],
"react-hooks/exhaustive-deps": [
"warn",
{ "additionalHooks": "(useSyncedQueryString|useSafeAsyncFunction)" }
],
"prefer-const": [1, { "destructuring": "all" }],
"no-useless-escape": 0,
"no-only-tests/no-only-tests": [
"error",
{
"block": [
"describe",
"it",
"context",
"test",
"tape",
"fixture",
"serial",
"Feature",
"Scenario",
"Given",
"And",
"When",
"Then",
"describeWithSnowplow",
"describeEE"
]
}
],
"complexity": ["error", { "max": 54 }]
},
"globals": {
"before": true,
"cy": true,
"Cypress": true
},
"env": {
"browser": true,
"es6": true,
"commonjs": true,
"jest": true,
"jest/globals": true
},
"parser": "babel-eslint",
"plugins": ["react", "no-only-tests"],
"extends": [
"eslint:recommended",
"plugin:react/recommended",
"plugin:react/jsx-runtime",
"plugin:react-hooks/recommended",
"plugin:import/errors",
"plugin:import/warnings",
"plugin:import/typescript"
],
"settings": {
"import/internal-regex": "^metabase/|^metabase-lib/",
"import/resolver": {
"webpack": {
"typescript": true
}
},
"import/ignore": ["\\.css$"],
"react": {
"version": "detect"
}
},
"parserOptions": {
"ecmaFeatures": {
"legacyDecorators": true
}
},
"overrides": [
{
"files": ["*.js", "*.jsx", "*.ts", "*.tsx"],
"rules": {
"no-unconditional-metabase-links-render": "error"
}
},
{
"files": ["*.unit.spec.*", "frontend/src/metabase/admin/**/*"],
"rules": {
"no-unconditional-metabase-links-render": "off"
}
},
{
"extends": ["plugin:@typescript-eslint/recommended"],
"files": ["*.ts", "*.tsx"],
"parser": "@typescript-eslint/parser",
"plugins": ["@typescript-eslint"],
"rules": {
"prefer-rest-params": "off",
"@typescript-eslint/explicit-module-boundary-types": "off",
"@typescript-eslint/no-inferrable-types": "off",
"@typescript-eslint/no-explicit-any": "off",
"@typescript-eslint/no-this-alias": "off",
"@typescript-eslint/consistent-type-imports": "error",
"@typescript-eslint/no-unused-vars": [
"error",
{
"argsIgnorePattern": "^_",
"varsIgnorePattern": "^_",
"ignoreRestSiblings": true,
"destructuredArrayIgnorePattern": "^_"
}
],
// This was introduced in 6.0.0
"@typescript-eslint/no-unsafe-declaration-merging": "off"
}
},
{
"extends": [
"plugin:jest/recommended",
"plugin:jest-dom/recommended",
"plugin:testing-library/react"
],
"plugins": ["jest", "jest-dom", "testing-library"],
"files": [
"*.unit.spec.ts",
"*.unit.spec.tsx",
"*.unit.spec.js",
"*.unit.spec.jsx"
],
"rules": {
"jest/valid-title": ["error", { "ignoreTypeOfDescribeName": true }]
}
}
]
}

View File

@@ -1,224 +0,0 @@
/* eslint-disable import/no-commonjs */
/* eslint-disable no-undef */
// `postcss-modules` lints css modules class names, but it currently crashes
// eslint on vscode. If you use webstorm or want to run the lint for the cli, you
// can use this flag to enable it. This is set to true in CI
const shouldLintCssModules =
process.env.LINT_CSS_MODULES === "true" || process.env.CI;
module.exports = {
rules: {
strict: [2, "never"],
"no-undef": 2,
"no-var": 1,
"no-unused-vars": [
"error",
{
vars: "all",
args: "none",
varsIgnorePattern: "^_",
ignoreRestSiblings: true,
},
],
"no-empty": [1, { allowEmptyCatch: true }],
"no-restricted-imports": [
"error",
{
paths: [
{
name: "moment",
message: "Moment is deprecated, please use dayjs",
},
{
name: "moment-timezone",
message: "Moment is deprecated, please use dayjs",
},
],
},
],
curly: [1, "all"],
eqeqeq: [1, "smart"],
"import/no-default-export": 2,
"import/no-named-as-default": 0,
"import/no-commonjs": 1,
"import/order": [
"error",
{
"newlines-between": "always",
alphabetize: {
order: "asc",
orderImportKind: "asc",
caseInsensitive: false,
},
groups: [
"builtin",
"external",
"internal",
"parent",
"sibling",
"index",
],
warnOnUnassignedImports: false,
},
],
"no-console": [2, { allow: ["warn", "error", "errorBuffer"] }],
"react/no-is-mounted": 2,
"react/prefer-es6-class": 2,
"react/display-name": 1,
"react/prop-types": 2,
"react/no-did-mount-set-state": 0,
"react/no-did-update-set-state": 0,
"react/no-find-dom-node": 0,
"react/no-children-prop": 2,
"react/no-string-refs": 2,
"react/no-unescaped-entities": 2,
"react/jsx-no-target-blank": 2,
"react/jsx-key": 2,
"react/forbid-component-props": [2, { forbid: ["sx"] }],
"react-hooks/exhaustive-deps": [
"warn",
{ additionalHooks: "(useSyncedQueryString|useSafeAsyncFunction)" },
],
"prefer-const": [1, { destructuring: "all" }],
"no-useless-escape": 0,
"no-only-tests/no-only-tests": [
"error",
{
block: [
"describe",
"it",
"context",
"test",
"tape",
"fixture",
"serial",
"Feature",
"Scenario",
"Given",
"And",
"When",
"Then",
"describeWithSnowplow",
"describeEE",
],
},
],
complexity: ["error", { max: 54 }],
...(shouldLintCssModules
? {
"postcss-modules/no-undef-class": "error",
}
: {}),
},
globals: {
before: true,
cy: true,
Cypress: true,
},
env: {
browser: true,
es6: true,
commonjs: true,
jest: true,
"jest/globals": true,
},
parser: "babel-eslint",
plugins: ["react", "no-only-tests", "postcss-modules"],
extends: [
"eslint:recommended",
"plugin:react/recommended",
"plugin:react/jsx-runtime",
"plugin:react-hooks/recommended",
"plugin:import/errors",
"plugin:import/warnings",
"plugin:import/typescript",
],
settings: {
"import/internal-regex": "^metabase/|^metabase-lib/",
"import/resolver": {
webpack: {
typescript: true,
},
},
"import/ignore": ["\\.css$"],
react: {
version: "detect",
},
"postcss-modules": {
baseDir: "./frontend/src",
},
},
parserOptions: {
ecmaFeatures: {
legacyDecorators: true,
},
},
overrides: [
{
files: ["*.js", "*.jsx", "*.ts", "*.tsx"],
rules: {
"no-unconditional-metabase-links-render": "error",
"no-literal-metabase-strings": "error",
},
},
{
files: [
"*.unit.spec.*",
"frontend/src/metabase/admin/**/*",
"frontend/src/metabase/setup/**/*",
"frontend/lint/**/*",
"*.stories.*",
"e2e/**/*",
"**/tests/*",
"release/**/*",
],
rules: {
"no-unconditional-metabase-links-render": "off",
"no-literal-metabase-strings": "off",
},
},
{
extends: ["plugin:@typescript-eslint/recommended"],
files: ["*.ts", "*.tsx"],
parser: "@typescript-eslint/parser",
plugins: ["@typescript-eslint"],
rules: {
"prefer-rest-params": "off",
"@typescript-eslint/explicit-module-boundary-types": "off",
"@typescript-eslint/no-inferrable-types": "off",
"@typescript-eslint/no-explicit-any": "off",
"@typescript-eslint/no-this-alias": "off",
"@typescript-eslint/consistent-type-imports": "error",
"@typescript-eslint/no-unused-vars": [
"error",
{
argsIgnorePattern: "^_",
varsIgnorePattern: "^_",
ignoreRestSiblings: true,
destructuredArrayIgnorePattern: "^_",
},
],
// This was introduced in 6.0.0
"@typescript-eslint/no-unsafe-declaration-merging": "off",
},
},
{
extends: [
"plugin:jest/recommended",
"plugin:jest-dom/recommended",
"plugin:testing-library/react",
],
plugins: ["jest", "jest-dom", "testing-library"],
files: [
"*.unit.spec.ts",
"*.unit.spec.tsx",
"*.unit.spec.js",
"*.unit.spec.jsx",
],
rules: {
"jest/valid-title": ["error", { ignoreTypeOfDescribeName: true }],
},
},
],
};

11
.github/CODEOWNERS vendored
View File

@@ -1,14 +1,3 @@
# Before changing this file, make sure you have read how we use it:
# https://www.notion.so/metabase/Pull-Request-Code-Review-Guide-374f05df889748b69d67af753f9bc9b8?pvs=4#36550c04ceab40b5a8b8bee6264b090c
# TL;DR: if @camsaul hasn't been involved you're probably doing it wrong
src/metabase @camsaul
modules @camsaul
enterprise/backend @camsaul
enterprise/backend/src/metabase_enterprise/advanced_permissions @noahmoss
enterprise/backend/src/metabase_enterprise/audit_app @noahmoss
enterprise/backend/src/metabase_enterprise/sso @noahmoss
enterprise/backend/src/metabase_enterprise/task/truncate_audit_table.clj @noahmoss
enterprise/backend/src/metabase_enterprise/internal_user.clj @noahmoss
src/metabase/**/*permissions* @noahmoss
src/metabase/integrations @noahmoss

View File

@@ -38,7 +38,7 @@ body:
validations:
required: true
- type: textarea
- type: input
id: expected-behavior
attributes:
label: Expected behavior
@@ -82,6 +82,7 @@ body:
label: Severity
description: |
How severe an issue is this bug to you? Is this annoying, blocking some users, blocking an upgrade or blocking your usage of Metabase entirely?
_Note: the more honest and specific you are here the more we will take you seriously._
validations:
required: true

View File

@@ -17,6 +17,7 @@ A clear and concise description of what you want to happen.
A clear and concise description of any alternative solutions or features you've considered.
**How important is this feature to you?**
Note: the more honest and specific you are here the more we will take you seriously.
**Additional context**
Add any other context or screenshots about the feature request here.

View File

@@ -1,9 +1,5 @@
> [!IMPORTANT]
> For those employed by Metabase: if you are merging into master, please add either a `backport` or a `no-backport` label
> to this PR. You will not be able to merge until you do this step. Refer to the section [Do I need to backport this
> PR?](https://www.notion.so/metabase/Metabase-Branching-Strategy-6eb577d5f61142aa960a626d6bbdfeb3?pvs=4#89f80d6f17714a0198aeb66c0efd1b71)
> in the Metabase Branching Strategy document for more details. If you're not employed by Metabase, this section does not
> apply to you, and the label will be taken care of by your reviewer.
> If you are merging into master, please add either a `backport` or a `no-backport` label to this PR. You will not be able to merge until you do this step. Refer to the section [Do I need to backport this PR?](https://www.notion.so/metabase/Metabase-Branching-Strategy-6eb577d5f61142aa960a626d6bbdfeb3?pvs=4#89f80d6f17714a0198aeb66c0efd1b71) in the Metabase Branching Strategy document for more details.
> **Warning**
>

View File

@@ -16,6 +16,7 @@ runs:
const java = 11;
const defaultRunner = "ubuntu-22.04";
const beefierRunner = "buildjet-2vcpu-ubuntu-2204";
const defaultOptions = {
"java-version": java,
@@ -48,8 +49,8 @@ runs:
["visualizations-charts", {} ],
["visualizations-tabular", {} ],
["oss-subset", { edition: 'oss', context: "special" } ],
["slow", { context: "special" } ],
["flaky", { context: "special" } ],
["slow", { runner: beefierRunner, context: "special" } ],
["flaky", { runner: beefierRunner, context: "special" } ],
["mongo", { context: "qa-database" } ],
];

View File

@@ -8,23 +8,23 @@ inputs:
description: DockerHub password/token
required: true
maildev:
description: Maildev container
description: Maildev
required: true
default: 'false'
openldap:
description: OpenLDAP container
description: Maildev
required: true
default: 'false'
postgres:
description: Postgres container
description: Maildev
required: true
default: 'false'
mysql:
description: MySQL container
description: Maildev
required: true
default: 'false'
mongo:
description: Mongo container
description: Maildev
required: true
default: 'false'

View File

@@ -23,12 +23,10 @@ frontend_sources: &frontend_sources
- "**/tsconfig*.json"
- "package.json"
- "babel.config.json"
- ".eslintrc.js"
- ".eslintrc"
- "postcss.config.js"
- "webpack.config.js"
- "webpack.embedding-sdk.config.js"
- "webpack.static-viz.config.js"
- ".nvmrc"
frontend_specs: &frontend_specs
- *shared_specs
@@ -95,7 +93,6 @@ e2e_ci: &e2e_ci
e2e_specs: &e2e_specs
- "**/*.cy.*.js"
- "**/*.cy.*.ts"
- "e2e/runner/**"
- "e2e/support/**"
- "e2e/snapshot*/**"

21
.github/team.json vendored
View File

@@ -7,6 +7,7 @@
"ranquild",
"uladzimirdev",
"kamilmielnik",
"oleggromov",
"nemanjaglumac",
"romeovs"
]
@@ -14,12 +15,16 @@
{
"name": "Embedding",
"label": ".Team/Embedding",
"members": ["WiNloSt", "deniskaber", "oisincoveney", "npretto", "heypoom"]
"members": [
"WiNloSt",
"deniskaber",
"oisincoveney",
"npretto"
]
},
{
"name": "BEC",
"label": ".Team/BackendComponents",
"projectUrl": "https://github.com/orgs/metabase/projects/50/views/1",
"members": [
"tsmacdonald",
"qnkhuat",
@@ -31,8 +36,13 @@
{
"name": "Query Processor",
"label": ".Team/QueryProcessor",
"projectUrl": "https://github.com/orgs/metabase/projects/85/views/1",
"members": ["camsaul", "snoe", "bshepherdson", "metamben", "lbrdnk"]
"members": [
"camsaul",
"snoe",
"bshepherdson",
"metamben",
"lbrdnk"
]
},
{
"name": "Admin Webapp",
@@ -40,8 +50,10 @@
"members": [
"noahmoss",
"escherize",
"qwef",
"iethree",
"npfitz",
"shaunlebron",
"johnswanson",
"rafpaf",
"sloansparger"
@@ -55,6 +67,7 @@
"EmmadUsmani",
"JesseSDevaney",
"kulyk",
"markbastian",
"adam-james-v"
]
}

View File

@@ -30,7 +30,7 @@ jobs:
const prNumber = context.payload.pull_request.number;
if (prAuthor.endsWith('[bot]')) {
return null;
return false;
}
// Assign PR author
@@ -44,7 +44,7 @@ jobs:
const team = teamConfig.teams.find(t => t.members.includes(prAuthor));
if (!team) {
console.log('You are not assigned to any team. If you need one, update .github/team.json');
return null;
return false;
}
try {
@@ -55,7 +55,7 @@ jobs:
});
} catch (e) {
console.log(`The label ${team.label} does not exist, create it first`);
return null;
return false;
}
// Add team label
@@ -66,9 +66,9 @@ jobs:
labels: [team.label]
});
return team.projectUrl || null;
return true;
- uses: actions/add-to-project@v0.5.0
if: ${{ steps.auto-assign.outputs.result != 'null' }}
if: ${{ fromJSON(steps.auto-assign.outputs.result) }}
with:
project-url: ${{ steps.auto-assign.outputs.result }}
project-url: https://github.com/orgs/metabase/projects/50/views/1
github-token: ${{ steps.generate-token.outputs.token }}

View File

@@ -6,7 +6,7 @@ on:
- "master"
- "release-**"
pull_request:
types: [opened, synchronize, reopened, ready_for_review, labeled]
types: [opened, synchronize, reopened, ready_for_review]
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
@@ -65,8 +65,7 @@ jobs:
be-linter-cloverage:
needs: files-changed
if: (github.ref_name == 'master' && needs.files-changed.outputs.backend_all == 'true')
|| contains(github.event.pull_request.labels.*.name, 'run-cloverage')
if: github.ref_name == 'master' && needs.files-changed.outputs.backend_all == 'true'
runs-on: ubuntu-22.04
timeout-minutes: 90
steps:
@@ -80,9 +79,8 @@ jobs:
- name: Collect the test coverage
run: clojure -X:dev:ci:ee:ee-dev:test:cloverage
- name: Upload coverage to codecov.io
uses: codecov/codecov-action@v4
uses: codecov/codecov-action@v3
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./target/coverage/codecov.json
flags: back-end
@@ -208,7 +206,7 @@ jobs:
if: matrix.java-version != 21
run: clojure -X:dev:ci:test:${{ matrix.edition }}:${{ matrix.edition }}-dev
- name: If using Java 21, only run tests on `master`
- name: Run tests using Java 21 on `master` only
id: run-java-tests-21
if: matrix.java-version == 21 && github.ref_name == 'master'
run: clojure -X:dev:ci:test:${{ matrix.edition }}:${{ matrix.edition }}-dev

View File

@@ -13,34 +13,6 @@ on:
required: true
jobs:
start-message:
runs-on: ubuntu-22.04
timeout-minutes: 5
steps:
- uses: actions/checkout@v3
with:
sparse-checkout: release
- name: Prepare build scripts
run: cd ${{ github.workspace }}/release && yarn && yarn build
- name: Send build start message
uses: actions/github-script@v6
env:
SLACK_RELEASE_CHANNEL: ${{ vars.SLACK_RELEASE_CHANNEL }}
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
with:
script: | # js
const { sendReleaseMessage } = require('${{ github.workspace }}/release/dist/index.cjs');
await sendReleaseMessage({
stage: 'build-start',
github,
owner: context.repo.owner,
repo: context.repo.repo,
version: '${{ inputs.version }}',
runId: '${{ github.run_id }}',
releaseSha: '${{ inputs.commit }}',
}).catch(console.error);
check-version:
runs-on: ubuntu-22.04
timeout-minutes: 10

View File

@@ -26,8 +26,8 @@ jobs:
// the github API doesn't expose linked issues, so we have to parse the body ourselves
function getLinkedIssueId(pull) {
// https://regexr.com/7sk94
const match = pull.data.body.match(/(close(s|d)?|fixe?(s|d)?|resolve(s|d)?) (#|https?:\/\/github.com\/.+\/issues\/)(\d+)/i);
// https://regexr.com/7mjnp
const match = pull.data.body.match(/(\/issues\/|#)(\d+)/);
return match && match?.[2];
}

View File

@@ -126,40 +126,6 @@ jobs:
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-druid-jdbc-ee:
needs: files-changed
if: github.event.pull_request.draft == false && needs.files-changed.outputs.backend_all == 'true'
runs-on: ubuntu-22.04
timeout-minutes: 90
env:
CI: 'true'
DRIVERS: druid-jdbc
services:
druid:
image: metabase/druid:29.0.1-json-no-ingestion
ports:
- "8888:8888"
env:
CLUSTER_SIZE: nano-quickstart
steps:
- uses: actions/checkout@v4
- name: Test Druid JDBC driver
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-druid-jdbc-ee'
test-args: ":exclude-tags '[:mb/once]'"
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-googleanalytics-ee:
needs: files-changed
if: github.event.pull_request.draft == false && needs.files-changed.outputs.backend_all == 'true'
@@ -315,7 +281,7 @@ jobs:
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-mongo-4-4-ee'
test-args: ":exclude-tags '[:mb/once :mongo-sharded-cluster-tests]'"
test-args: ":exclude-tags '[:mb/once]'"
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -361,7 +327,7 @@ jobs:
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-mongo-4-4-ee'
test-args: ":exclude-tags '[:mb/once :mongo-sharded-cluster-tests]'"
test-args: ":exclude-tags '[:mb/once]'"
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -395,7 +361,7 @@ jobs:
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-mongo-5-0-ee'
test-args: ":exclude-tags '[:mb/once :mongo-sharded-cluster-tests]'"
test-args: ":exclude-tags '[:mb/once]'"
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -441,7 +407,7 @@ jobs:
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-mongo-5-0-ee'
test-args: ":exclude-tags '[:mb/once :mongo-sharded-cluster-tests]'"
test-args: ":exclude-tags '[:mb/once]'"
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
@@ -478,39 +444,7 @@ jobs:
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-mongo-latest-ee'
test-args: ":exclude-tags '[:mb/once :mongo-sharded-cluster-tests]'"
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()
with:
input-path: ./target/junit/
output-name: ${{ github.job }}
bucket: ${{ vars.AWS_S3_TEST_RESULTS_BUCKET }}
aws-access-key-id: ${{ secrets.AWS_TEST_RESULTS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_TEST_RESULTS_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
trunk-api-token: ${{ secrets.TRUNK_API_TOKEN }}
be-tests-mongo-sharded-cluster-ee:
needs: files-changed
if: github.event.pull_request.draft == false && needs.files-changed.outputs.backend_all == 'true'
runs-on: ubuntu-22.04
timeout-minutes: 90
env:
CI: 'true'
DRIVERS: mongo
services:
mongodb:
image: metabase/mongo-sharded-cluster:6
ports:
- "27017:17017"
steps:
- uses: actions/checkout@v4
- name: Test MongoDB driver (Sharded cluster)
uses: ./.github/actions/test-driver
with:
junit-name: 'be-tests-mongo-sharded-cluster-ee'
test-args: ':only "[metabase.driver.mongo.sharded-cluster-test]"'
test-args: ":exclude-tags '[:mb/once]'"
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()

View File

@@ -11,8 +11,6 @@ jobs:
strategy:
matrix:
version: [
# Major OSS upgrade
{ source: v0.42.2, target: v0.48.7 },
# OSS upgrade
{ source: v0.41.3.1, target: v0.42.2 },
# EE upgrade
@@ -95,63 +93,3 @@ jobs:
path: |
./cypress
if-no-files-found: ignore
notify-on-failure:
needs: cross-version-testing
if: ${{ always() && contains(needs.*.result, 'failure') }}
name: Cross-version test failure
runs-on: ubuntu-22.04
env:
BRANCH_NAME: ${{ github.event.workflow_run.head_branch }}
AUTHOR_NAME: ${{ github.event.workflow_run.head_commit.author.name }}
steps:
- name: Generate job summary
run: |
echo "# ${{ github.event.workflow_run.name }} workflow failed! :x:" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "View the failed run attempt (#${{ github.event.workflow_run.run_attempt }}) using the following link:" >> $GITHUB_STEP_SUMMARY
echo "${{ github.event.workflow_run.html_url }}" >> $GITHUB_STEP_SUMMARY
- uses: actions/setup-node@v4
with:
node-version: lts/Hydrogen # 18.x.x
- run: npm install @slack/web-api
- name: notify slack of failure
uses: actions/github-script@v7
with:
script: | # js
const FAILED_RUN_URL = "${{ github.event.workflow_run.html_url }}";
const FAILED_RUN_NAME = "${{ github.event.workflow_run.name }}";
const BREAKING_COMMIT = "${{ github.event.workflow_run.head_sha }}";
const AUTHOR = process.env.AUTHOR_NAME;
const BRANCH = process.env.BRANCH_NAME;
// notify slack of failure
const { WebClient } = require('@slack/web-api');
const slack = new WebClient('${{ secrets.SLACK_BOT_TOKEN }}');
await slack.chat.postMessage({
channel: 'engineering-ci',
text: 'Cross-version Tests Have Failed',
blocks: [
{
"type": "header",
"text": {
"type": "plain_text",
"text": `:broken_heart: Cross-version tests are failing on ${BRANCH}`,
"emoji": true,
}
},
],
attachments: [{
color: "#BF40BF",
blocks: [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": `Commit <https://github.com/metabase/metabase/commit/${BREAKING_COMMIT}|${BREAKING_COMMIT.slice(0,7)}> has failing <${FAILED_RUN_URL}|${FAILED_RUN_NAME}> tests on the <https://github.com/metabase/metabase/commits/${BRANCH}|\`${BRANCH}\`> branch.`
}
},
]
}]
});

View File

@@ -45,7 +45,6 @@ jobs:
env:
DISPLAY: ""
QA_DB_ENABLED: true
CYPRESS_QA_DB_MONGO: true
CYPRESS_REPLAYIO_ENABLED: 1
CYPRESS_ALL_FEATURES_TOKEN: ${{ secrets.ENTERPRISE_TOKEN }}
CYPRESS_NO_FEATURES_TOKEN: ${{ secrets.E2E_STARTER_TOKEN }}

View File

@@ -4,7 +4,7 @@ on:
push:
branches:
- "master"
- "release-**"
- 'release-**'
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
@@ -87,11 +87,10 @@ jobs:
CYPRESS_ALL_FEATURES_TOKEN: ${{ secrets.ENTERPRISE_TOKEN }}
CYPRESS_NO_FEATURES_TOKEN: ${{ secrets.E2E_STARTER_TOKEN }}
CYPRESS_PULL_REQUEST_ID: ${{ github.event.pull_request.number || '' }}
COMMIT_INFO_MESSAGE: ${{ github.event.pull_request.title || github.event.head_commit.message || github.even.head.sha }}
COMMIT_INFO_MESSAGE : ${{ github.event.pull_request.title || github.event.head_commit.message || github.even.head.sha }}
MB_SNOWPLOW_AVAILABLE: true
MB_SNOWPLOW_URL: "http://localhost:9090" # Snowplow micro
CYPRESS_REPLAYIO_ENABLED: 1
CYPRESS_REPLAYIO_ENABLE_UPLOAD: 1
RECORD_REPLAY_METADATA_FILE: /tmp/replay-metadata.json
RECORD_REPLAY_METADATA_TEST_RUN_ID: ${{ github.run_id }}
REPLAY_API_KEY: ${{ secrets.REPLAY_IO_TOKEN }}
@@ -159,7 +158,7 @@ jobs:
if: matrix.name == 'flaky' && github.event_name != 'schedule'
run: |
yarn run test-cypress-run \
--env grepTags="@flaky --@quarantine",grepOmitFiltered=true \
--env grepTags="@flaky",grepOmitFiltered=true \
--spec './e2e/test/scenarios/**/*.cy.spec.js' \
--browser 'replay-chromium'
@@ -217,3 +216,47 @@ jobs:
steps:
- run: |
echo "Didn't run due to conditional filtering"
visual-regression-tests:
runs-on: ubuntu-22.04
timeout-minutes: 60
needs: [build]
if: |
!cancelled() && needs.build.result == 'success'
name: percy-visual-regression-tests
steps:
- uses: actions/checkout@v4
- name: Prepare Docker containers
uses: ./.github/actions/e2e-prepare-containers
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
maildev: true
- name: Retrieve uberjar artifact for ${{ matrix.edition }}
uses: actions/download-artifact@v4
with:
name: metabase-ee-${{ github.event.pull_request.head.sha || github.sha }}-uberjar
- name: Get the version info
run: |
jar xf target/uberjar/metabase.jar version.properties
mv version.properties resources/
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare JDK 11
uses: actions/setup-java@v4
with:
java-version: 11
distribution: 'temurin'
- name: Prepare Cypress environment
uses: ./.github/actions/prepare-cypress
- name: Percy Test
run: yarn run test-visual-run
env:
PERCY_TOKEN: ${{ github.ref == 'refs/heads/master' && secrets.PERCY_TOKEN || ''}}

View File

@@ -1,39 +0,0 @@
name: Embedding SDK
on:
push:
branches:
- "master"
- "release-**"
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref && github.ref || github.run_id }}
cancel-in-progress: true
jobs:
sdk-build:
if: contains(github.event.pull_request.labels.*.name, 'embedding-sdk-build')
runs-on: ubuntu-22.04
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: "cljs"
- name: Build SDK bundle
run: yarn run build-embedding-sdk
- name: Generate SDK package.json
run: yarn run embedding-sdk:generate-package "$GITHUB_SHA"
- name: Generate .npmrc for Github npm registry
uses: actions/setup-node@v4
with:
registry-url: "https://npm.pkg.github.com"
- name: Publish package
run: yarn run embedding-sdk:publish
env:
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -70,7 +70,7 @@ jobs:
fe-tests-unit:
needs: files-changed
if: github.event.pull_request.draft == false && needs.files-changed.outputs.frontend_all == 'true'
runs-on: ubuntu-22.04
runs-on: buildjet-2vcpu-ubuntu-2204
timeout-minutes: 60
env:
JEST_JUNIT_OUTPUT_DIR: ./target/junit
@@ -91,11 +91,10 @@ jobs:
run: yarn run test-unit --silent
- name: Upload coverage to codecov.io
if: github.ref_name == 'master'
uses: codecov/codecov-action@v4
uses: codecov/codecov-action@v3
with:
files: ./coverage/lcov.info
flags: front-end
token: ${{ secrets.CODECOV_TOKEN }}
- name: Upload Test Results
uses: ./.github/actions/upload-test-results
if: always()

View File

@@ -1,72 +0,0 @@
name: Update Loki Snapshots
on:
pull_request:
types: [labeled]
jobs:
check-label-and-update-snapshot:
if: contains(github.event.pull_request.labels.*.name, 'loki-update')
runs-on: ubuntu-latest
steps:
- name: Checkout Repository
uses: actions/checkout@v3
with:
ref: ${{ github.head_ref }}
- name: Prepare frontend environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: "cljs"
- name: Compile CLJS
run: yarn build-pure:cljs
- name: Build Storybook
run: yarn build-storybook
- name: Serve Storybook
run: yarn http-server storybook-static -p 6006 &
shell: bash
- name: Wait for Storybook to become available
run: |
echo "Waiting for Storybook to be served..."
while ! curl --silent --fail http://localhost:6006; do
sleep 5
echo "Retrying..."
done
echo "Storybook is being served!"
- name: Update Loki Snapshots
run: yarn test-visual:loki-update
- uses: tibdex/github-app-token@v2.1.0
id: generate-token
with:
app_id: ${{ secrets.METABASE_BOT_APP_ID }}
private_key: ${{ secrets.METABASE_BOT_APP_PRIVATE_KEY }}
- name: Commit and Push Changes
run: |
git config --global user.name "Metabase bot"
git config --global user.email "metabase-bot@metabase.com"
git add .loki/**
git commit -m "Update Loki Snapshots"
git push
env:
GITHUB_TOKEN: ${{ steps.generate-token.outputs.token }}
- name: Remove 'loki-update' label
uses: actions/github-script@v3
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
github.issues.removeLabel({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
name: 'loki-update'
})

View File

@@ -1,66 +0,0 @@
name: Loki Visual Regression Testing
on: [pull_request]
jobs:
visual-test:
runs-on: ubuntu-latest
services:
docker:
image: docker:19.03.12
options: --privileged
ports:
- 2376:2376
env:
DOCKER_TLS_CERTDIR: /certs
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /tmp/docker-certs:/certs/client
steps:
- name: Checkout Repository
uses: actions/checkout@v3
- name: Prepare frontend environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: "cljs"
- name: Compile CLJS
run: yarn build-pure:cljs
- name: Build Storybook
run: yarn build-storybook
- name: Serve Storybook
run: yarn http-server storybook-static -p 6006 &
shell: bash
- name: Wait for Storybook to become available
run: |
echo "Waiting for Storybook to be served..."
while ! curl --silent --fail http://localhost:6006; do
sleep 5
echo "Retrying..."
done
echo "Storybook is being served!"
- name: Run Loki Visual Tests
run: |
yarn loki test
- name: Generate Visual Report on Failure
if: failure()
run: yarn test-visual:loki-report
- name: Upload Artifact
id: artifact-upload-step
if: failure()
uses: actions/upload-artifact@v4
with:
name: loki-report
path: .loki/
if-no-files-found: ignore

View File

@@ -1,32 +0,0 @@
name: Update changelog for MBQL library changes
on:
pull_request:
types:
- opened
- reopened
- labeled
- unlabeled
- synchronize
paths:
- src/metabase/lib/js.cljs
jobs:
mbql-library-changelog-check:
runs-on: ubuntu-latest
if: ${{ !contains( github.event.pull_request.labels.*.name, '.metabase-lib/no-change' ) }}
steps:
# - uses: jesusvasquez333/verify-pr-label-action@v1.4.0
# with:
# github-token: '${{ secrets.GITHUB_TOKEN }}'
# valid-labels: '.metabase-lib/no-change'
# disable-reviews: true
- uses: actions/checkout@v4
- name: Check for updated MBQL library changelog
run: |
if [[ $(git show HEAD -- docs/developers-guide/mbql-library-changelog.md) ]]; then
echo "Library and changelog updated together - all good"
else
echo "metabase.lib.js namespace changed but docs/developers/guide/mbql-library-changelog.md did not"
echo "Update the changelog with the changes made to the library's API."
echo "If the edits did not change the API, add the `.metabase-lib/no-change` label."
exit 1
fi

View File

@@ -0,0 +1,91 @@
# Triggers Percy job by "@metabase-bot run visual tests" comment in a PR
name: PercyIssueComment
on:
issue_comment:
types: [created]
jobs:
pr_info:
if: github.event.comment.body == '@metabase-bot run visual tests'
runs-on: ubuntu-22.04
outputs:
pull_request_number: ${{ fromJson(steps.fetch_pr.outputs.data).head.number }}
branch_name: ${{ fromJson(steps.fetch_pr.outputs.data).head.ref }}
commit_sha: ${{ fromJson(steps.fetch_pr.outputs.data).head.sha }}
steps:
- uses: tibdex/github-app-token@v2.1.0
id: generate-token
with:
app_id: ${{ secrets.METABASE_BOT_APP_ID }}
private_key: ${{ secrets.METABASE_BOT_APP_PRIVATE_KEY }}
- uses: actions/checkout@v4
with:
ref: ${{ needs.pr_info.outputs.branch_name }}
token: ${{ secrets.GITHUB_TOKEN }}
- name: Verify Membership
id: verify-membership
uses: ./.github/actions/verify-membership
with:
github_token: ${{ steps.generate-token.outputs.token }}
organization_name: metabase
username: ${{ github.event.comment.user.login }}
- run: |
result=${{ steps.verify-membership.outputs.is_member }}
if [ $result == false ]; then
echo User ${{ github.event.comment.user.login }} is not a member of Metabase organization
exit 1
fi
- name: Fetch issue
uses: octokit/request-action@v2.x
id: fetch_issue
with:
route: GET ${{ github.event.issue.url }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Fetch PR
uses: octokit/request-action@v2.x
id: fetch_pr
with:
route: GET ${{ fromJson(steps.fetch_issue.outputs.data).pull_request.url }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
percy:
timeout-minutes: 30
runs-on: buildjet-4vcpu-ubuntu-2004
needs: [pr_info]
env:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
MB_EDITION: ee
CYPRESS_ALL_FEATURES_TOKEN: ${{ secrets.ENTERPRISE_TOKEN }}
CYPRESS_NO_FEATURES_TOKEN: ${{ secrets.E2E_STARTER_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ needs.pr_info.outputs.branch_name }}
token: ${{ secrets.GITHUB_TOKEN }}
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: percy-issue-comment
- name: Prepare cypress environment
uses: ./.github/actions/prepare-cypress
- run: ./bin/build.sh
- name: Get the version info
run: |
jar xf target/uberjar/metabase.jar version.properties
mv version.properties resources/
- name: Run maildev
run: docker run -d -p 1080:1080 -p 1025:1025 maildev/maildev:2.0.5
- name: Percy Test
run: yarn run test-visual-run
env:
PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}
PERCY_COMMIT: ${{ needs.pr_info.outputs.commit_sha }}
PERCY_BRANCH: ${{ needs.pr_info.outputs.branch_name }}
PERCY_PULL_REQUEST: ${{ needs.pr_info.outputs.pull_request_number }}

View File

@@ -0,0 +1,39 @@
# Triggers Percy job on pull requests that contain "visual" label
name: Percy visual tests
on:
pull_request:
types: [synchronize, labeled]
jobs:
percy:
if: contains(github.event.pull_request.labels.*.name, 'visual')
timeout-minutes: 30
runs-on: buildjet-4vcpu-ubuntu-2004
env:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
MB_EDITION: ee
CYPRESS_ALL_FEATURES_TOKEN: ${{ secrets.ENTERPRISE_TOKEN }}
CYPRESS_NO_FEATURES_TOKEN: ${{ secrets.E2E_STARTER_TOKEN }}
steps:
- uses: actions/checkout@v4
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: percy-visual-label
- name: Prepare cypress environment
uses: ./.github/actions/prepare-cypress
- run: ./bin/build.sh
- name: Get the version info
run: |
jar xf target/uberjar/metabase.jar version.properties
mv version.properties resources/
- name: Run maildev
run: docker run -d -p 1080:1080 -p 1025:1025 maildev/maildev:2.0.5
- name: Percy Test
run: yarn run test-visual-run
env:
PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}

View File

@@ -5,11 +5,11 @@ on:
workflow_dispatch:
inputs:
version:
description: "Metabase version (e.g. v0.46.3)"
description: 'Metabase version (e.g. v0.46.3)'
type: string
required: true
commit:
description: "A full-length commit SHA-1 hash"
description: 'A full-length commit SHA-1 hash'
required: true
jobs:
@@ -91,21 +91,21 @@ jobs:
edition: [oss, ee]
java-version: [11, 17]
steps:
- name: Prepare JRE (Java Run-time Environment)
uses: actions/setup-java@v4
with:
java-package: jre
java-version: ${{ matrix.java-version }}
distribution: "temurin"
- run: java -version
- uses: actions/download-artifact@v4
name: Retrieve uberjar artifact
with:
name: metabase-${{ matrix.edition }}-uberjar
- name: Launch uberjar (and keep it running)
run: java -jar ./metabase.jar &
- name: Wait for Metabase to start
run: while ! curl -s 'http://localhost:3000/api/health' | grep '{"status":"ok"}'; do sleep 1; done
- name: Prepare JRE (Java Run-time Environment)
uses: actions/setup-java@v4
with:
java-package: jre
java-version: ${{ matrix.java-version }}
distribution: 'temurin'
- run: java -version
- uses: actions/download-artifact@v4
name: Retrieve uberjar artifact
with:
name: metabase-${{ matrix.edition }}-uberjar
- name: Launch uberjar (and keep it running)
run: java -jar ./metabase.jar &
- name: Wait for Metabase to start
run: while ! curl -s 'http://localhost:3000/api/health' | grep '{"status":"ok"}'; do sleep 1; done
run-sanity-check:
runs-on: ubuntu-22.04
@@ -119,9 +119,6 @@ jobs:
DISPLAY: ""
CYPRESS_ALL_FEATURES_TOKEN: ${{ secrets.ENTERPRISE_TOKEN }}
CYPRESS_NO_FEATURES_TOKEN: ${{ secrets.E2E_STARTER_TOKEN }}
# disabled because of out of memory issues
# probably related to https://github.com/cypress-io/cypress/issues/27415
CYPRESS_NO_COMMAND_LOG: 1
MB_SNOWPLOW_AVAILABLE: true
MB_SNOWPLOW_URL: "http://localhost:9090" # Snowplow micro
TERM: xterm
@@ -135,43 +132,43 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
steps:
- name: Check out the code
uses: actions/checkout@v4
with:
ref: ${{ needs.release-artifact.outputs.commit }}
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare JDK 11
uses: actions/setup-java@v4
with:
java-version: 11
distribution: "temurin"
- name: Prepare Cypress environment
uses: ./.github/actions/prepare-cypress
- name: Run Snowplow micro
uses: ./.github/actions/run-snowplow-micro
- uses: actions/download-artifact@v4
name: Retrieve uberjar artifact for ${{ matrix.edition }}
with:
name: metabase-${{ matrix.edition }}-uberjar
- name: Get the version info
run: |
jar xf metabase.jar version.properties
mv version.properties resources/
- name: Run a few important Cypress tests as sanity check
run: |
mkdir -p ./target/uberjar && cp metabase.jar ./target/uberjar/metabase.jar
yarn test-cypress-run --folder onboarding
- name: Check out the code
uses: actions/checkout@v4
with:
ref: ${{ needs.release-artifact.outputs.commit }}
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare JDK 11
uses: actions/setup-java@v4
with:
java-version: 11
distribution: "temurin"
- name: Prepare Cypress environment
uses: ./.github/actions/prepare-cypress
- name: Run Snowplow micro
uses: ./.github/actions/run-snowplow-micro
- uses: actions/download-artifact@v4
name: Retrieve uberjar artifact for ${{ matrix.edition }}
with:
name: metabase-${{ matrix.edition }}-uberjar
- name: Get the version info
run: |
jar xf metabase.jar version.properties
mv version.properties resources/
- name: Run a few important Cypress tests as sanity check
run: |
mkdir -p ./target/uberjar && cp metabase.jar ./target/uberjar/metabase.jar
yarn test-cypress-run --folder onboarding
- name: Upload Cypress Artifacts upon failure
uses: actions/upload-artifact@v4
if: failure()
with:
name: cypress-failed-tests-recording
path: |
./cypress
./logs/test.log
if-no-files-found: ignore
- name: Upload Cypress Artifacts upon failure
uses: actions/upload-artifact@v4
if: failure()
with:
name: cypress-failed-tests-recording
path: |
./cypress
./logs/test.log
if-no-files-found: ignore
containerize:
needs: [release-artifact, run-sanity-check]
@@ -188,71 +185,71 @@ jobs:
outputs:
image_tag: ${{ steps.target.outputs.image_tag }}
steps:
- name: Get the version
run: |
if [[ "${{ matrix.edition }}" == "ee" ]]; then
echo "VERSION=${{ needs.release-artifact.outputs.ee_version }}" >> $GITHUB_ENV
else
echo "VERSION=${{ needs.release-artifact.outputs.oss_version }}" >> $GITHUB_ENV
fi
- name: Check out the code
uses: actions/checkout@v4
with:
ref: ${{ needs.release-artifact.outputs.commit }}
- uses: actions/download-artifact@v4
name: Retrieve uberjar artifact
with:
name: metabase-${{ matrix.edition }}-uberjar
- name: Move the Uberjar to the context dir
run: mv metabase.jar bin/docker/.
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v2
with:
driver-opts: network=host
- name: Build ${{ matrix.edition }} container
uses: docker/build-push-action@v3
with:
context: bin/docker/.
platforms: linux/amd64
network: host
tags: localhost:5000/local-metabase:${{ env.VERSION }}
no-cache: true
push: true
- name: Get the version
run: |
if [[ "${{ matrix.edition }}" == "ee" ]]; then
echo "VERSION=${{ needs.release-artifact.outputs.ee_version }}" >> $GITHUB_ENV
else
echo "VERSION=${{ needs.release-artifact.outputs.oss_version }}" >> $GITHUB_ENV
fi
- name: Check out the code
uses: actions/checkout@v4
with:
ref: ${{ needs.release-artifact.outputs.commit }}
- uses: actions/download-artifact@v4
name: Retrieve uberjar artifact
with:
name: metabase-${{ matrix.edition }}-uberjar
- name: Move the Uberjar to the context dir
run: mv metabase.jar bin/docker/.
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v2
with:
driver-opts: network=host
- name: Build ${{ matrix.edition }} container
uses: docker/build-push-action@v3
with:
context: bin/docker/.
platforms: linux/amd64
network: host
tags: localhost:5000/local-metabase:${{ env.VERSION }}
no-cache: true
push: true
- name: Launch container
run: docker run --rm -dp 3000:3000 localhost:5000/local-metabase:${{ env.VERSION }}
timeout-minutes: 5
- name: Wait for Metabase to start
run: while ! curl -s 'http://localhost:3000/api/health' | grep '{"status":"ok"}'; do sleep 1; done
timeout-minutes: 3
- name: Launch container
run: docker run --rm -dp 3000:3000 localhost:5000/local-metabase:${{ env.VERSION }}
timeout-minutes: 5
- name: Wait for Metabase to start
run: while ! curl -s 'http://localhost:3000/api/health' | grep '{"status":"ok"}'; do sleep 1; done
timeout-minutes: 3
- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_RELEASE_USERNAME }}
password: ${{ secrets.DOCKERHUB_RELEASE_TOKEN }}
- name: Retag and push container image to ${{ vars.DOCKERHUB_OWNER }} staging Docker Hub repo
id: target
run: |
# SHA-1 truncated to 7 digits should be enough
SHORT_HASH=${COMMIT:0:7}
- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_RELEASE_USERNAME }}
password: ${{ secrets.DOCKERHUB_RELEASE_TOKEN }}
- name: Retag and push container image to ${{ vars.DOCKERHUB_OWNER }} staging Docker Hub repo
id: target
run: |
# SHA-1 truncated to 7 digits should be enough
SHORT_HASH=${COMMIT:0:7}
IMAGE_TAG=$VERSION-$SHORT_HASH
CONTAINER_IMAGE=${{ vars.DOCKERHUB_OWNER }}/${{ vars.DOCKERHUB_STAGING_REPO }}:$IMAGE_TAG
IMAGE_TAG=$VERSION-$SHORT_HASH
CONTAINER_IMAGE=${{ vars.DOCKERHUB_OWNER }}/${{ vars.DOCKERHUB_STAGING_REPO }}:$IMAGE_TAG
echo "image_tag=$IMAGE_TAG" >> $GITHUB_OUTPUT
echo "image_tag=$IMAGE_TAG" >> $GITHUB_OUTPUT
echo "Retag local image for staging"
docker tag localhost:5000/local-metabase:$VERSION $CONTAINER_IMAGE
echo "Retag local image for staging"
docker tag localhost:5000/local-metabase:$VERSION $CONTAINER_IMAGE
echo "Pushing container image $CONTAINER_IMAGE ..."
docker push $CONTAINER_IMAGE
echo "Pushing container image $CONTAINER_IMAGE ..."
docker push $CONTAINER_IMAGE
echo "Finished!"
shell: bash
env:
COMMIT: ${{ needs.release-artifact.outputs.commit }}
echo "Finished!"
shell: bash
env:
COMMIT: ${{ needs.release-artifact.outputs.commit }}
verify-docker-pull:
runs-on: ubuntu-22.04
@@ -264,22 +261,22 @@ jobs:
env:
IMAGE_TAG: ${{ needs.containerize.outputs.image_tag }}
steps:
- name: Login to Docker Hub # authenticated, to avoid being rate-throttled
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_RELEASE_USERNAME }}
password: ${{ secrets.DOCKERHUB_RELEASE_TOKEN }}
- name: Pull the container image
run: |
CONTAINER_IMAGE=${{ vars.DOCKERHUB_OWNER }}/${{ vars.DOCKERHUB_STAGING_REPO }}:$IMAGE_TAG
echo "Pulling container image $CONTAINER_IMAGE ..."
docker pull $CONTAINER_IMAGE
echo "Successful!"
- name: Launch container
run: |
CONTAINER_IMAGE=${{ vars.DOCKERHUB_OWNER }}/${{ vars.DOCKERHUB_STAGING_REPO }}:$IMAGE_TAG
docker run --rm -dp 3000:3000 $CONTAINER_IMAGE
timeout-minutes: 5
- name: Wait for Metabase to start
run: while ! curl -s 'http://localhost:3000/api/health' | grep '{"status":"ok"}'; do sleep 1; done
timeout-minutes: 3
- name: Login to Docker Hub # authenticated, to avoid being rate-throttled
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_RELEASE_USERNAME }}
password: ${{ secrets.DOCKERHUB_RELEASE_TOKEN }}
- name: Pull the container image
run: |
CONTAINER_IMAGE=${{ vars.DOCKERHUB_OWNER }}/${{ vars.DOCKERHUB_STAGING_REPO }}:$IMAGE_TAG
echo "Pulling container image $CONTAINER_IMAGE ..."
docker pull $CONTAINER_IMAGE
echo "Successful!"
- name: Launch container
run: |
CONTAINER_IMAGE=${{ vars.DOCKERHUB_OWNER }}/${{ vars.DOCKERHUB_STAGING_REPO }}:$IMAGE_TAG
docker run --rm -dp 3000:3000 $CONTAINER_IMAGE
timeout-minutes: 5
- name: Wait for Metabase to start
run: while ! curl -s 'http://localhost:3000/api/health' | grep '{"status":"ok"}'; do sleep 1; done
timeout-minutes: 3

View File

@@ -1,193 +0,0 @@
name: Release Metabase Embedding SDK for React
on:
workflow_dispatch:
inputs:
# TODO: Add a version validation, so the workflow won't fail when publishing to npm
sdk_version:
description: 'SDK version (e.g. 0.1.3)'
type: string
required: true
git_ref:
description: 'Valid git reference e.g. branches, tags, hashes'
type: string
required: true
concurrency:
# We want to ensure only one job is running at a time because
# there could be a conflict when updating the readme file.
group: ${{ github.workflow }}
cancel-in-progress: true
jobs:
git-tag:
runs-on: ubuntu-22.04
timeout-minutes: 20
env:
tag: embedding-sdk-${{ inputs.sdk_version }}
steps:
- name: Check out the code using the provided ref
uses: actions/checkout@v4
with:
ref: ${{ inputs.git_ref }}
- name: Setup git user
run: |
git config --global user.email "metabase-bot@metabase.com"
git config --global user.name "Metabase bot"
- name: Create a new git tag
run: |
git tag ${{ env.tag }}
- name: Push the new tag
id : push-tag
run: |
git push origin ${{ env.tag }}
- if: ${{ failure() && steps.push-tag.outcome == 'failure' }}
run: echo "Tag '${{ env.tag }}' already exists. If you expect to run this workflow with the same tag, please remove the tag first and rerun this workflow again."
test:
needs: git-tag
runs-on: ubuntu-22.04
timeout-minutes: 20
steps:
- name: Check out the code using the provided ref
uses: actions/checkout@v4
with:
ref: ${{ inputs.git_ref }}
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: "release-sdk"
- name: Run unit tests
run: yarn embedding-sdk:test-unit
build-sdk:
needs: test
runs-on: ubuntu-22.04
timeout-minutes: 20
steps:
- name: Check out the code using the provided ref
uses: actions/checkout@v4
with:
ref: ${{ inputs.git_ref }}
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: "release-sdk"
- name: Bump published npm package version
run: |
sed -i -E 's/("version": ").*"/\1${{ inputs.sdk_version }}"/' enterprise/frontend/src/embedding-sdk/package.template.json
- name: Build SDK bundle
run: yarn run build-embedding-sdk
- name: Generate SDK package.json in the build directory
run: yarn run embedding-sdk:generate-package
- name: Upload built SDK package as artifact
uses: actions/upload-artifact@v4
with:
name: metabase-sdk
path: ./resources/embedding-sdk
build-jar:
needs: test
uses: ./.github/workflows/uberjar-sdk.yml
with:
git_ref: ${{ inputs.git_ref }}
image_name: embedding-sdk-${{ inputs.sdk_version }}
secrets: inherit
upload-jar:
needs: build-jar
runs-on: ubuntu-22.04
timeout-minutes: 20
steps:
- uses: actions/download-artifact@v4
name: Retrieve uberjar artifact
with:
name: metabase-ee-uberjar
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_S3_RELEASE_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_S3_RELEASE_SECRET_ACCESS_KEY }}
aws-region: ${{ vars.AWS_REGION }}
- name: Upload uberjar to S3
env:
BUCKET: ${{ vars.AWS_S3_DOWNLOADS_BUCKET }}
BUCKET_PATH: sdk/v${{ inputs.sdk_version }}/metabase.jar
FILE: ./target/uberjar/metabase.jar
shell: bash
run: |
aws s3 cp $FILE s3://$BUCKET/$BUCKET_PATH
publish-npm:
needs: [build-sdk, upload-jar]
runs-on: ubuntu-22.04
timeout-minutes: 20
steps:
- name: Check out master to prepare for a SDK version bump PR
uses: actions/checkout@v4
with:
# when we created version update PR, it would only have diff from 2 files SDK readme and SDK package.json template
ref: master
- name: Setup git user
run: |
git config --global user.email "metabase-bot@metabase.com"
git config --global user.name "Metabase bot"
- name: Update readme
run: |
sed -i -E 's|(embedding-sdk-)[0-9.]+|\1${{ inputs.sdk_version }}|' enterprise/frontend/src/embedding-sdk/README.md
sed -i -E 's|(http://downloads.metabase.com/sdk/v)[0-9.]+|\1${{ inputs.sdk_version }}|' enterprise/frontend/src/embedding-sdk/README.md
- name: Bump published npm package version
run: |
sed -i -E 's/("version": ").*"/\1${{ inputs.sdk_version }}"/' enterprise/frontend/src/embedding-sdk/package.template.json
- uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.METABASE_BOT_APP_ID }}
private-key: ${{ secrets.METABASE_BOT_APP_PRIVATE_KEY }}
- name: Create a PR updating readme + published version
run: |
git checkout -b update-sdk-version-${{ inputs.sdk_version }}
git commit -a -m 'Update Readme version references and published npm version to ${{ inputs.sdk_version }}'
git push origin HEAD
gh pr create --base master\
--assignee "${GITHUB_ACTOR}"\
--title "Update SDK version to ${{ inputs.sdk_version }}"\
--body "Update Readme version references and published npm package version to ${{ inputs.sdk_version }}"
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
- name: Retrieve build SDK package artifact
uses: actions/download-artifact@v4
with:
name: metabase-sdk
path: sdk
- name: Publish to NPM
run: |
cd sdk
echo //registry.npmjs.org/:_authToken=${{ secrets.NPM_RELEASE_TOKEN }} > .npmrc
npm publish

View File

@@ -3,7 +3,7 @@ name: Release Status Check
on:
workflow_dispatch:
schedule:
- cron: '0 11 * * 1-5' # every weekday at 6am US eastern time
- cron: '0 13 * * *' # every day at 8am US eastern time
jobs:
check-release-status:

View File

@@ -0,0 +1,44 @@
name: Require Percy specs
on:
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
jobs:
files-changed:
if: github.event.pull_request.draft == false && !contains(github.event.pull_request.labels.*.name, 'visual')
name: Check which files changed
runs-on: ubuntu-22.04
timeout-minutes: 3
outputs:
visualizations: ${{ steps.changes.outputs.visualizations }}
steps:
- uses: actions/checkout@v4
- name: Test which files changed
uses: dorny/paths-filter@v3.0.0
id: changes
with:
token: ${{ github.token }}
filters: .github/file-paths.yaml
add-label:
needs: files-changed
if: needs.files-changed.outputs.visualizations == 'true'
runs-on: ubuntu-22.04
timeout-minutes: 20
steps:
- uses: tibdex/github-app-token@v2.1.0
id: generate-token
with:
app_id: ${{ secrets.METABASE_BOT_APP_ID }}
private_key: ${{ secrets.METABASE_BOT_APP_PRIVATE_KEY }}
- uses: actions/github-script@v7
with:
github-token: ${{ steps.generate-token.outputs.token }}
script: |
github.rest.issues.addLabels({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
labels: ["visual"]
})

View File

@@ -45,7 +45,7 @@ jobs:
repo: context.repo.repo,
run_id: ${{ github.event.workflow_run.id }},
});
} else if (!BRANCH.includes('backport-')) {
} else {
// notify slack of repeated failure
const { WebClient } = require('@slack/web-api');
const slack = new WebClient('${{ secrets.SLACK_BOT_TOKEN }}');

View File

@@ -1,157 +0,0 @@
name: Build + Docker Uberjar for SDK
on:
# This workflow will be called from "release-embedding-sdk.yml"
workflow_call:
inputs:
git_ref:
description: 'Valid git reference e.g. branches, tags, hashes'
type: string
required: true
image_name:
description: Docker image used to upload to Dockerhub
type: string
required: true
jobs:
build:
name: Build MB ${{ matrix.edition }}
runs-on: ubuntu-22.04
timeout-minutes: 40
strategy:
matrix:
edition: [ee]
env:
MB_EDITION: ${{ matrix.edition }}
INTERACTIVE: false
steps:
- name: Check out the code
uses: actions/checkout@v4
with:
ref: ${{ inputs.git_ref }}
- name: Prepare front-end environment
uses: ./.github/actions/prepare-frontend
- name: Prepare back-end environment
uses: ./.github/actions/prepare-backend
with:
m2-cache-key: uberjar
- name: Build
run: ./bin/build.sh
- name: Prepare uberjar artifact
uses: ./.github/actions/prepare-uberjar-artifact
with:
name: metabase-ee-uberjar
check_jar_health:
runs-on: ubuntu-22.04
name: Is ${{ matrix.edition }} (java ${{ matrix.java-version }}) healthy?
needs: build
timeout-minutes: 10
strategy:
matrix:
edition: [ee]
java-version: [11, 17]
steps:
- name: Prepare JRE (Java Run-time Environment)
uses: actions/setup-java@v4
with:
java-package: jre
java-version: ${{ matrix.java-version }}
distribution: 'temurin'
- run: java -version
- uses: actions/download-artifact@v4
name: Retrieve uberjar artifact
with:
name: metabase-ee-uberjar
- name: Launch uberjar
run: java -jar ./target/uberjar/metabase.jar &
- name: Wait for Metabase to start
run: while ! curl 'http://localhost:3000/api/health' | grep '{"status":"ok"}'; do sleep 1; done
containerize_test_and_push_container:
runs-on: ubuntu-22.04
name: Containerize ${{ matrix.edition }}
needs: check_jar_health
strategy:
matrix:
edition: [ee]
services:
registry:
image: registry:2
ports:
- 5000:5000
steps:
- name: Extract and clean branch name
shell: bash
run: echo "branch=$(echo $GITHUB_REF_NAME | sed 's/[^-._a-zA-Z0-9]/-/g')" >> $GITHUB_OUTPUT
id: extract_branch
- name: Set image name to env
run: echo "image_name=${{ inputs.image_name || steps.extract_branch.outputs.branch }}" >> "$GITHUB_ENV"
- name: Verify the intended tag of the container image
run: echo "Container image will be tagged as ${{ env.image_name }}-${{ matrix.edition }}"
- name: Check out the code (Dockerfile needed)
uses: actions/checkout@v4
with:
ref: ${{ inputs.git_ref }}
- name: Download uploaded artifacts to insert into container
uses: actions/download-artifact@v4
with:
name: metabase-ee-uberjar
path: bin/docker/
- name: Move the ${{ matrix.edition }} uberjar to the context dir
run: mv bin/docker/target/uberjar/metabase.jar bin/docker/.
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v2.5.0
with:
driver-opts: network=host
- name: Build ${{ matrix.edition }} container
uses: docker/build-push-action@v3
with:
context: bin/docker/.
platforms: linux/amd64
network: host
tags: localhost:5000/metabase-dev:${{ env.image_name }}-${{ matrix.edition }}
no-cache: true
push: true
- name: Launch ${{ matrix.edition }} container
run: docker run --rm -dp 3000:3000 localhost:5000/metabase-dev:${{ env.image_name }}-${{ matrix.edition }}
timeout-minutes: 5
- name: Is Docker running?
run: docker ps
- name: Wait for Metabase to start and reach 100% health
run: while ! curl -s 'http://localhost:3000/api/health' | grep '{"status":"ok"}'; do sleep 1; done
timeout-minutes: 3
- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Retag and push images
run: docker tag localhost:5000/metabase-dev:${{ env.image_name }}-ee ${{ github.repository_owner }}/metabase-dev:${{ env.image_name }} && docker push ${{ github.repository_owner }}/metabase-dev:${{ env.image_name }}
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
env:
TRIVY_OFFLINE_SCAN: true
with:
image-ref: docker.io/${{ github.repository_owner }}/metabase-dev:${{ env.image_name }}
format: sarif
output: trivy-results.sarif
- name: Upload Trivy scan results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v2
with:
sarif_file: 'trivy-results.sarif'

9
.gitignore vendored
View File

@@ -1,5 +1,4 @@
*.class
*.*.rej
*.iml
*.jar
*.log
@@ -27,7 +26,7 @@
/.lein-repl-history
/.sqlite.db
/backend-checksums.txt
/backend-docs
/backend-docs/uberdoc.html
/build.xml
/checkouts
/classes
@@ -51,7 +50,8 @@
/resources/frontend_client/embed.html
/resources/frontend_client/index.html
/resources/frontend_client/public.html
/resources/embedding-sdk
/resources/frontend_client/embedding-example.html
/resources/embedding-sdk/dist/
/resources/i18n/*.edn
/resources/instance_analytics.zip
/resources/license-backend-third-party.txt
@@ -109,5 +109,4 @@ modules/drivers/*/.lsp/*
.github/static-viz-sources.yaml
# airgap
/resources/airgap/
/frontend/src/embedding-sdk/example-nodejs-backend/node_modules/

4
.loki/.gitignore vendored
View File

@@ -1,4 +0,0 @@
current
difference
report.html
report.json

Binary file not shown.

Before

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.0 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 63 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.9 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 28 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.6 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 9.1 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 8.7 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 8.6 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.9 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 37 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 44 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 16 KiB

Some files were not shown because too many files have changed in this diff Show More