Strengthen assertions across 5 test files by replacing weak `some?`/`string?`/
`contains?` existence checks with `=?` pattern assertions that validate actual
values. Consolidate scattered single-field `(is ...)` calls into single map
assertions.
Add new security-critical tests: expired OAuth bearer tokens, scope enforcement
on tool calls, PKCE S256 flow (happy + missing verifier), expired authorization
codes, and revoked refresh tokens.
Fix invalid-bearer-token test to set site-url and reset the OAuth provider.
Fold addColumn changesets (client_type, expiry) into their original
createTable changesets since these tables haven't shipped yet. Remove
leftover #p debug macros from the OAuth API handlers.
Replace the temporary superuser fallback in the MCP handler with proper
OAuth 2.0 bearer token validation. Add RFC 9728 protected resource
metadata discovery, thread token scopes through MCP→Agent API dispatch,
and filter tools/list by granted scopes.
Key changes:
- Emit :scope from endpoint metadata into tools manifest definitions
- Add /.well-known/oauth-protected-resource endpoint (RFC 9728)
- Fix resource parameter forwarding through OAuth consent form
- Validate bearer tokens against the OAuth token store
- Store user-id and scopes on MCP sessions for subsequent requests
- Filter tools/list visibility based on token scopes
- Fix JSON deserialization of resource field (list→vector)
- Add OAuth/.well-known routes to request logging middleware
- Add tests for auth, scope filtering, and manifest generation
Replace the JSON response from GET /oauth/authorize with a Hiccup-rendered
HTML consent page that shows the client name, requested scopes, and
Allow/Deny buttons. The decision endpoint now accepts form-encoded POST
params instead of JSON.
The consent page respects Metabase appearance settings (application-font,
application-colors brand, application-logo-url) and includes the CSP nonce
on the style tag so inline styles work with Metabase's Content-Security-Policy.
Also includes: frontend route guard for /oauth/ paths to prevent SPA
interception, refresh token expiry column migration, and store protocol
updates for the new expiry parameter.
The OIDC discovery endpoint must be served at /.well-known/openid-configuration
relative to the issuer URL, not under /oauth/. Extract well-known-routes as a
separate top-level route handler and update tests accordingly.
Implement POST /oauth/token endpoint to exchange authorization codes for
access/refresh/ID tokens, completing the core OAuth authorization code flow.
Also fix user-id handling: convert to string for oidc-provider library
compatibility and parse back to integer for DB storage.
Implement GET /oauth/authorize (validates request, returns consent data)
and POST /oauth/authorize/decision (processes approve/deny, redirects with
code or error). Add client_type column to oauth_client so the oidc-provider
library can distinguish confidential from public clients without relying on
the presence of a client-secret field.
Adds /api/ee/oauth-server/clients endpoints for admin-only static client
registration, listing, reading, updating, and deleting. Uses defendpoint
with Malli response schemas, PBKDF2 secret hashing via oidc-provider, and
metabot-v3 feature gating.
Implement RFC 7591 dynamic client registration (POST /oauth/register) and
RFC 7592 client configuration read (GET /oauth/register/:client-id) for the
embedded OIDC provider. Registration access tokens are hashed via PBKDF2
before storage, with hash verification used for client read authentication.
Also simplify store key conversion helpers to use update-keys with
u/->kebab-case-en and u/->snake_case_en instead of manual field-by-field
mapping.
Replace in-memory stores with persistent DB implementations so OAuth
state survives restarts and works across multiple app instances.
Adds Liquibase migrations for four tables (oauth_client,
oauth_authorization_code, oauth_access_token, oauth_refresh_token),
Toucan 2 models, protocol implementations (ClientStore,
AuthorizationCodeStore, TokenStore, ClaimsProvider), and wires them
into the provider construction.
Expose /oauth/.well-known/openid-configuration and /oauth/jwks using the
defenterprise pattern: OSS route handlers return 404 when the enterprise
functions return nil; EE implementations serve responses from the
oidc-provider library gated behind :metabot-v3.
Wire oidc-provider as a local dependency and create the
metabase-enterprise.oauth-server module skeleton gated behind
the :metabot-v3 feature flag. The provider boots lazily on first
access, generates and persists an RSA signing key via settings,
and uses in-memory stores (database-backed stores deferred to
Phase 5). No routes or endpoints yet — just the foundation.
Phased plan for embedding an OAuth/OIDC server in Metabase Enterprise,
with database-backed storage prioritized early (Phase 3) so all
subsequent client registration and auth flows build on production-grade
persistence from the start.
- Return 404 (not 400) for invalid/expired sessions per MCP spec
- Add Origin header validation to prevent DNS rebinding attacks
- Log client info during initialize handshake
- Use JSON-RPC error format for 405 responses
- Omit Content-Type on 202 notification-only responses
Replace the soft TODO comment on the MCP superuser fallback with an
explicit warning block so automated reviewers stop suggesting its
removal.
Also adds query param support for GET/DELETE tool calls and
unifies routing through dispatch-via-agent-api.
Add a new ee-ai-metabot-internal-tasks-enabled? setting to allow disabling metabot internal "bookkeeping" tasks that might make additional llm calls, like user intent classification.
* BOT-1054 Add snowplow events for user intent classification
* Move run-agent-loop :conversation-id into :tracking-opts :session-id
It does make some sense as a top-level argument, but it's currently only used for snowplow tracking, so push it down
into the tracking-opts.
* Move run-agent-loop :track-user-intent? into :tracking-opts
* Add ee-ai-metabot-provider-lite defsetting and use it in classify-user-intent!
This allows using a lighter weight model for simple tasks like user intent and/or agent response
classification. Defaults to openrouter/openai/gpt-oss-20b which is the same model used by ai-service for these tasks.
* Add run-agent-loop! test helper to run the agent for side effects
* whitespace: fix alignment
* Extract :tracking-opts schema into an mr/def similar to other agent opts
* Rename :profile-name -> :profile-id in tracking-opts
* BOT-1039 Create metabase.analytics.util
And move some func there from metabase.analytics.llm-token-usage that will also be needed for agent_used_tool events.
* Add ai_service_event snowplow schema
In preparation for tracking the following events:
- agent_tool_used
- user_intent
- agent_action_summary
* Track snowplow agent_used_tool events in metabase-enterprise.metabot-v3.self/call-llm
- add agent loop :iteration to tracking-opts
- add :duration-ms to :tool-output parts
- add report-tool-usage-xf to transducer chain in metabase-enterprise.metabot-v3.self/call-llm to track
:snowplow/ai_service_event with :event "agent_used_tool"
* BOT-1052 Move token-usage funcs into analytics module
So they can be shared between oss sql gen and metabot.
* Add some snowplot tests for self/call-llm and call-llm-structured
* Add prometheus tests for call-llm-structured
* Add basic snowplow and prometheus tests for example question generation
* Modify agent loop snowplot test to use with-fake-snowplow-collector
* Modify native-generate-content-snowplow-test to use with-fake-snowplow-collector
* Fix whitespace
* BOT-1039 Include the profile :name in the profile map in register-profile!
Sometimes it's useful to be able to exract the profile name when you only
have a reference to the profile.
* Add token_usage snowplow event tracking to the clojure agent
- refactor register-profile! to single-arg form with :name now stored in the profile map and returned by get-profile
- report-token-usage-xf now emits a :snowplow/token_usage event per LLM call alongside existing prometheus metrics
- pass tracking-opts from loop-step through call-llm into the token-usage transducer
- likewise for callers of call-llm-structured
- init-agent generates a per-run request-id UUID
- run-agent-loop accepts an optional conversation-id that turns into the snowplow token_usage session_id
* Add :model to tracking-opts and drop separate source arg
The prometheus source label should always be the same as the snowplow :tag (that's how it works in ai-service), so
drop the separate `source` arg from various functions and just take the source label from the tracking-opts :tag.
* Destructure token-opts in report-token-usage-xf
* Allow passing tracking-opts to run-agent-loop
And use this to override the :source in metabot-v3.api.document/native-generate-content
* Improve docstring for tracking-opts
* Provide model and source defaults in report-aisdk-errors-xf
* Fallback to analytics-uuid if premium-embedding-token is unset
* Fix typo: snowplot -> snowplow
* Compute hashed-token-or-uuid once rather than for each usage part in report-token-usage-xf
After #70061 call-llm-structured now expects the model to include a
provider prefix, so "anthropic/claude-haiku-4-5" was getting parsed as
:provider "anthropic" :model "claude-haiku-4-5".
Update metabase-enterprise.metabot-v3.example-question-generator/call-llm to
use the provider + model configured via llm/ee-ai-metabot-provider
* BOT-866 Add prometheus metrics to metabot clojure agent
* BOT-866 Also count inline aisdk errors
* BOT-866 push prometheus token counting down into call-llm
This is more in keeping with how current ai-service reports the metric, i.e. per llm call rather than a single batched
token usage per agent request.
This also has the advantage of keeping most prometheus metrics localized to the llm layer.
* Ensure that :metabase-metabot/llm-requests counts retries
This is more consistent with the existing ai-service llm_invocations_total metric
* Make :metabase-metabot/llm-duration-ms per llm invocation rather than cumulative
This more closely matches the existing ai-service llm_duration_ms metric
* Add :metabase-metabot/agent-duration-ms prometheus metric
Records the full duration of the outer agent loop, now that :metabase-metabot/llm-duration-ms records individual llm calls.
* Add comment on :metabase-metabot/llm-duration-ms buckets
And extend the low end of the range down to 100ms
* fixup! Add :metabase-metabot/agent-duration-ms prometheus metric
* Fix whitespace
* Add prometheus metrics tracking to call-llm-structured
* Allow llm callers to specificy the source label for prometheus metrics
And pass "example-question-generation" for question generation.
* Fix call-llm-prometheus-test
After rebasing on #70061 these tests need to include the "openrouter/" provider prefix along with the model.
Key changes:
- SQL tools (edit_sql_query, replace_sql_query) now operate on in-memory
query state instead of persisting to database
- Convert tools from reactions to data-parts pattern (navigate_to)
- Add semantic_queries and keyword_queries to search tool schema
- Make edit_sql_query and replace_sql_query state-dependent tools
- Support string query IDs (in addition to int) for memory-based queries
- Extract SQL dialect only from native query context type
- Use streaming.navigate-to-part instead of direct reactions
- Add instructions pattern to SQL tools for better LLM guidance
This aligns the SQL tools with the AI Service's transient query model
where queries exist in conversation state until explicitly saved.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Key changes:
- Add markdown_link_buffer.clj for handling links split across streaming
text chunks (matches Python AI Service MarkdownLinkBuffer behavior)
- Seed agent state with queries from user_is_viewing context
- Support both snake_case and kebab-case keys in context for prompts
- Normalize entity types (keyword/string) in message formatting
- Decode tool arguments from JSON strings when needed
- Add table link resolution (metabase://table/{id})
- Preserve system-instructions when wrapping state-dependent tools
- Fix iteration boundary check (use inc iteration for comparison)
- Gracefully handle unresolvable links by falling back to link text
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit adds feature parity between the Clojure native agent and
Python AI service, implementing all phases from the parity plan.
New Tools:
- todo_write / todo_read: Task management with todo_list data parts
- navigate_user: Page and entity navigation with redirect reactions
- ask_for_sql_clarification: SQL clarification with final-response flag
- write_transform_sql / write_transform_python: Transform code editing
- create_autogenerated_dashboard: X-ray dashboard generation
- list_snippets / get_snippet_details: SQL snippet access
Core Changes:
- State data part format: {type: data, data-type: "state", version: 1}
- Finish message format: {type: finish, finish-reason: "stop"|"error"}
- New data part types: todo_list, code_edit, transform_suggestion
- Data part streaming from tool results via :data-parts key
- Final response pattern: tools return :final-response? true to stop loop
- Capability-based tool filtering with tool-capabilities map
Profile Updates:
- internal: Added navigate_user, create_autogenerated_dashboard
- transforms-codegen: Added todo tools, transform write tools, snippets
- sql-only: Added ask_for_sql_clarification, :required-tool-call? flag
- nlq-only: Added navigate_user, create_chart, edit_chart
Tests:
- Comprehensive tests for all new tools (29 new tests, 200+ assertions)
- Updated existing tests for new formats and capabilities
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add redirect reactions to SQL query tools (create, edit, replace) for
auto-navigation after query operations
- Add streaming module to convert reactions to AI SDK data parts
- Fix structured output extraction to handle both :structured-output and
:structured_output key formats (Clojure vs JSON style)
- Add role normalization in message formatting to handle both keyword
and string roles from frontend
- Include :query key in SQL tool returns for memory storage
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Align Clojure native agent tool outputs with Python AI Service patterns
by wrapping results in <result>/<instructions> XML format for consistent
LLM guidance.
- Add llm_representations.clj with entity-to-XML formatting utilities
- Add instructions.clj with LLM guidance constants for each tool type
- Update messages.clj to use format-with-instructions wrapper
- Add type-specific formatters for search, entity, query, chart results
- Include comprehensive tests for LLM representation functions
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Key changes:
- Format tool results as XML for LLM consumption (messages.clj)
- Query results now show query ID clearly: <query id="..." type="query">
- Chart results include chart ID and reference link
- This matches Python ai-service format so LLM can extract IDs properly
- Add fallback for chart link resolution (links.clj)
- When LLM uses metabase://chart/ with a query ID, fall back to
treating it as a query link
- Handles common LLM mistake of using wrong link type
- Fix chart extraction logic (core.clj)
- Charts identified by :chart-id + :query-id (without :query)
- Distinguishes charts from query results correctly
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The native agent now resolves metabase:// URIs in text parts before
streaming to clients. This mirrors the Python AI service's
MarkdownLinkBuffer behavior.
Link types supported:
- metabase://query/{uuid} - Resolves to /question# using stored query
- metabase://chart/{uuid} - Resolves to /question# via chart's query
- metabase://model/{id} - Resolves to /model/{id}
- metabase://metric/{id} - Resolves to /metric/{id}
- metabase://dashboard/{id} - Resolves to /dashboard/{id}
- metabase://question/{id} - Resolves to /question/{id}
- metabase://transform/{id} - Resolves to /admin/transforms/{id}
Changes:
- links.clj: New module for link resolution with markdown parsing
- core.clj: Integrate link processing in stream-parts-to-output!
- Tests for link resolution and streaming integration
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
State-dependent tools (show_results_to_user, create_chart, edit_chart) now
look up queries and charts from agent memory by ID instead of expecting raw
query objects. This mirrors the Python AI service pattern.
Key changes:
- memory.clj: Add query/chart storage functions (remember-query, find-query,
store-chart, find-chart, load-queries-from-state, load-charts-from-state)
- core.clj: Extract queries/charts from tool outputs and store in memory
- tools.clj: Wrap state-dependent tools with memory atom access
- claude.clj: Update schema to accept wrapped tool maps with :fn/:doc/:schema
- self/core.clj: Handle wrapped tool maps in tool-executor-rff
- show_results_to_user.clj: Accept query-id and look up from queries-state
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Key fixes:
- Fix premature loop exit: Continue when tool calls present, regardless of text
- Fix Claude API integration: Extract system message separately, pass via :system param
- Fix tool name mismatch: Pass tools as [name, var] pairs so Claude sees registry keys
- Fix message format for tool_use: Use content array format for assistant messages
- Handle START_STEP/FINISH_STEP types in aisdk->messages
- Rename profiles to simpler keys (:embedding, :internal, etc.)
- Add /native-agent-streaming endpoint for testing (bypasses feature flag)
- Add debug logging for message building
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The Python ai-service outputs a specific line protocol format that the
frontend expects. This adds the conversion layer to output the same format
from the native Clojure agent.
Format examples:
- 0:"text content" - Text (JSON-encoded string)
- 2:{"type":...} - Data parts
- 3:"error" - Errors
- 9:{"toolCallId":..., "toolName":..., "args":...} - Tool calls
- a:{"toolCallId":..., "result":...} - Tool results
- d:{"finishReason":"stop", "usage":{...}} - Finish message
- f:{"messageId":...} - Start message
Changes:
- Add format-* functions for each part type in self/core.clj
- Add aisdk-line-xf transducer for batch conversion
- Export new functions via self.clj
- Update api.clj to stream parts in line protocol format
- Add comprehensive tests for all format functions
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Phase 2C - Chart Tools:
- Add create_chart tool to create visualizations from queries
- Add edit_chart tool to modify existing chart types
- Register chart tools in agent tools registry
Phase 2F - Profiles & Templates:
- Port sql-querying-only.selmer from Jinja2 (fixes broken :metabot-sql-only)
- Port metabot-next.selmer template
- Port natural-language-querying-only.selmer template
- Add :metabot-next, :metabot-nlq-only, :metabot-beta-customer profiles
Async Blocking Fix:
- Use a/thread instead of a/pipeline-blocking in run-llm-with-tools
- tool-executor-rff uses a/<!! in completion arity which cannot run in go-blocks
- Synchronous transducer application in thread context is now aligned with self code
- Fix stream-parts-to-output! and finalize-stream! to use a/>! in go blocks
- Fix api.clj syntax error with metadata on string literal
- Update tests to generate proper Claude raw SSE format
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implement the foundational read_resource tool that provides unified
interface for fetching detailed information about Metabase resources
using URI patterns. This tool implements the "Context Engineering with
Links" pattern where URIs serve as lightweight, token-efficient
references that can be fetched on-demand at the appropriate detail level.
Supported URI patterns:
- metabase://table/{id} - Basic table info
- metabase://table/{id}/fields - Table structure with fields
- metabase://table/{id}/fields/{field_id} - Specific field details
- metabase://model/{id}/fields - Model structure with fields
- metabase://metric/{id}/dimensions - Metric dimensions
- metabase://transform/{id} - Transform details (not yet implemented)
- metabase://dashboard/{id} - Dashboard details (not yet implemented)
Key features:
- Parses metabase:// URIs and routes to appropriate handlers
- Supports fetching multiple resources in parallel (max 5)
- Handles field IDs with slashes (e.g., c75/17)
- Returns formatted XML-like output for LLM consumption
- Provides detailed error messages for invalid URIs
This tool is critical for:
- Enabling agents to get more context about resources on demand
- Supporting workflows where search returns URIs that agents fetch later
- Reducing token usage by deferring detail fetching until needed
Changes:
- Created read_resource.clj with URI parsing and resource handlers
- Added read-resource-tool wrapper in tools.clj with Malli schema
- Updated all 4 profiles to include read_resource tool
- Added comprehensive tests for URI parsing and resource fetching
Tests: 24 assertions, 0 failures
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Fixed multiple issues with SQL tools not handling the transformed
lib/query format when Card models are read from the database:
## Issues Fixed
1. **extract-sql-content not finding SQL**: Card dataset_query transforms
from {:native {:query "SQL"}} to {:stages [{:native "SQL"}]} when read.
Updated extract-sql-content in all SQL tools to handle both formats.
2. **OffsetDateTime vs Date**: The rank-result function in sql_search
called .getTime on timestamps, but Toucan2 returns OffsetDateTime.
Added instanceof check to convert appropriately.
3. **Missing visualization_settings**: Test code using t2/insert! needed
:visualization_settings {} field (database constraint).
4. **Test permissions**: Wrapped all test code with mt/with-current-user
to provide proper permission context.
5. **Fixed typo in sql_search**: Changed accessible-cards variable reference
from undefined to all-cards.
## Files Changed
- edit_sql_query.clj: Handle both lib/query and legacy formats
- replace_sql_query.clj: Handle both lib/query and legacy formats
- sql_search.clj: Handle both formats + OffsetDateTime conversion
- sql_tools_test.clj: Add visualization_settings, fix permissions,
add clojure.string alias, fix indentation
## Test Results
All 19 assertions passing (4 test functions):
- edit-sql-query-test: 4 edit types tested
- replace-sql-query-test: Replace with metadata preservation
- sql-search-test: Search, filter by database, limit parameter
- integration-create-edit-search-test: Full workflow
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add four SQL query manipulation tools to enable the agent to create, edit,
and search SQL queries:
New Tools:
- create_sql_query: Create new SQL queries with validation
- edit_sql_query: Apply structured edits (replace, append, insert, etc.)
- replace_sql_query: Replace SQL content entirely while preserving metadata
- sql_search: Search for SQL queries by content with relevance ranking
Features:
- Direct Toucan2 Card model access (no HTTP round-trips)
- Database and permissions validation
- Structured edit operations (replace, replace-all, append, prepend, insert-after, insert-before)
- Search with ranking by name match, description match, SQL position, and recency
- Integration with existing agent tool registry
- Added to metabot-internal profile and new metabot-sql-only profile
Testing:
- Comprehensive test coverage for all SQL tools
- Integration tests for full workflow (create → edit → search)
- All tests passing (18 assertions, 0 failures)
Architecture:
- Follows existing tool pattern (mu/defn with Malli schemas)
- Returns structured output for agent state tracking
- Error handling with agent-error? flag for LLM-friendly messages
Phase 2B complete. Ready for Phase 2C: Visualization Tools.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The tool-executor-rff reducing function blocks in its completion arity
to wait for tool results. Using a/pipeline causes 'Invalid blocking call
in dispatch thread' errors because the regular pipeline uses a fixed
thread pool that doesn't allow blocking.
Solution: Use a/pipeline-blocking instead, which uses a separate thread
pool designed for blocking operations. This allows the tool executor's
completion arity to safely block while waiting for tool results.
Fixes the core-test integration test failures.
Selmer v1.12.70 provides Jinja-like templating for Clojure,
used by the Phase 2A prompts system for rendering system messages
with dynamic context variables.
This commit completes Phase 2A of the native Clojure agent implementation,
integrating the templated prompt system with context enrichment into the
main agent execution loop.
## New Features
### Prompt System (prompts.clj)
- Selmer template loading with caching
- SQL dialect instruction injection (14 dialects supported)
- Tool instruction extraction from docstrings
- System message template rendering with context variables
### Context Enrichment (user_context.clj)
- Timezone-aware time formatting
- SQL dialect extraction from context
- Viewing context formatting for all types:
- Adhoc queries (notebook editor)
- Native SQL queries with error context
- Transforms with source information
- Code editor with buffer/cursor/selection details
- Entity details (tables, models, questions, metrics, dashboards)
- Recent views formatting with relevance hints
- Full context enrichment pipeline for template variable injection
### Resource Files
- 3 system prompt templates (embedding, internal, transform-codegen)
- 14 SQL dialect instruction files
- All templates support dynamic context injection
## Integration Changes
### Agent Core (core.clj)
- Updated build-messages-for-llm to accept context, profile, and tools
- Integrated system message building with enriched context
- Fixed pre-existing async blocking issue in run-llm-with-tools
(changed from blocking a/<!! to returning channel for non-blocking a/<!)
- Agent now builds fresh system messages with context on each iteration
### Messages (messages.clj)
- Updated build-system-message to use templated prompts
- Accepts context, profile, and tools parameters
- Calls prompts/build-system-message-content for rendering
### Profiles (profiles.clj)
- Added :prompt-template mapping to all three profiles
- Maps profile IDs to template files in resources/metabot/prompts/system/
## Tests
All Phase 2A functionality is comprehensively tested:
- prompts-test: 6 tests, 46 assertions ✅
- user-context-test: 5 tests, 81 assertions ✅
- messages-test: 3 tests, 47 assertions ✅
- profiles-test: 1 test, 36 assertions ✅
Total: 15 tests, 210 assertions passing
Note: core-test integration tests have pre-existing failures due to blocking
calls in self/core.clj tool-executor-rff (line 116), which existed before
this change. The async pipeline architecture issue is tracked separately.
## Architecture
The agent now follows this flow on each iteration:
1. Enrich raw context (user_context/enrich-context-for-template)
2. Load and render system prompt template (prompts/build-system-message-content)
3. Build full message array with system message + history
4. Execute LLM call with enriched context
5. Process tool calls and responses
6. Repeat until completion
This matches the Python implementation's prompt and context system while
leveraging Clojure's functional architecture.
This commit implements a production-ready native Clojure agent loop that
replaces the external Python AI Service with an in-process implementation,
leveraging the existing llm-inside streaming infrastructure.
Key Components:
- agent/core.clj: Main agent loop with iteration control and streaming
- agent/memory.clj: Conversation state and step tracking
- agent/messages.clj: Message formatting and LLM communication
- agent/profiles.clj: Profile configurations (embedding, internal, transforms)
- agent/tools.clj: Tool registry with 13 wrapped tools
Integration:
- api.clj: Feature-flagged routing via use-native-agent setting
- settings.clj: New use-native-agent boolean setting (default: false)
Benefits:
- Direct Toucan2 database access (no HTTP round-trips)
- Single JVM process deployment
- Lower latency for tool execution
- Unified logging and error handling
- Reuses 80%+ of existing streaming infrastructure
Test Coverage:
- 5 test files with comprehensive coverage
- All tests passing (133 total assertions)
- Unit tests for memory, messages, profiles, tools
- Integration tests for agent loop with mocked LLM
The implementation is production-ready and can be enabled via feature flag
for gradual rollout and validation against the Python service.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Handle 403 errors in rerun-workflows.yml
Wrap the reRunWorkflowFailedJobs API call in a try/catch that
gracefully handles HTTP 403 errors. This prevents false-positive CI
failures when the target run is cancelled, a PR run, or otherwise
ineligible for retry.
Also adds missing await to the API call so errors are properly caught.
Update E2E test timings with averaged data
Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).
🤖 Generated automatically by update-e2e-timings workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
- Adds same-database validation for `fk_target_field_id` in `PUT /api/field/:id` (resolves existing TODO)
- Adds existence and same-database validation for `human_readable_field_id` in `POST /api/field/:id/dimension`
- Returns 400 if the target field belongs to a different database than the source field
Updated the documentation to clarify that the HAR file contains sensitive data and revised the instructions for saving the HAR file accordingly. Also added a screenshot of the settings in Chrome.
---------
Co-authored-by: Rodrigo López Dato <rodrigo@metabase.com>
- The `:soft?` marking in `find-bad-refs-with-source` was too broad — it marked **all** bad refs in `:fields` clauses as soft, even when the field was completely missing from the source (not just inactive).
- Added `(not (:active column true))` guard so only **inactive** fields in `:fields` are marked `:soft?`. Missing/unresolvable fields remain hard errors and are properly surfaced by dependency analysis.
- This preserves the GHY-3044 fix (inactive fields the QP can drop are still soft) while fixing GHY-3157 (genuinely missing fields are reported).
During startup, the database isn't migrated yet, so we can't check the
token hash.
If the database hasn't been started yet, skip checking the token hash in
the DB and just delegate to the lower-level checker that actually hits
the store.
* test: agent-specific test runner
Adds an "agent" test runner that filters the output of running `clojure
-X:...:test` commands to make it more relevant for agents preventing
them from needing to use tail or grep to find information and reducing
context pollution.
Also implements randomized ports for test web-servers allowing multiple
agents to run tests in parallel (or to run multiple commands in
parallel). This removes the directive to use the ./bin/mage-based test
runner, to direct agents to use the agent-specific runner.
* fixup! missing import
* add the test agent script
* move additional options to clickhouse settings
* add test
* add test description
* add prefix and keep in subname
* just ignore unknown settings
* fix test
* wrap in case
* wrap case in cast, cleanup
* comments
* rename
* add test
* move comparison operators to lib.schema.filter
* Add boolean ops
* add and, or, not. rename comparison to predicate
* clean up tests
Babashka task that starts up an isolated dev environment per worktree. Expands on @edpaget's excellent work, with interactive fzf prompts for edition, token, appdb, and warehouse services. Optional `--run` flag to launch backend + frontend in the background with PID tracking. Persistent config in `.dev-env-state.edn` for reuse across invocations.
* CssVarsDeclarationPlugin: Add plugin and documentation to generate .d.css files at build step
* CssVarsDeclarationPlugin: Add recommended .vscode settings to support CSS vars autocomplete
* CssVarsDeclarationPlugin: Fix nested unions parsing
* CssVarsDeclarationPlugin: Add unit tests
* Update vscode settings for css vars to include Mantine
* Fix can't click on the time dropdown on subscription sidebar in StrictMode
* Add tests
* update e2e test
* Fix unit tests
* improve code clarity
* Fix E2E tests
* Simplify code
* Make default value consistent
* Make the new Select button have dynamic size
* Fix unit tests
* Fix E2E tests
Adds health-check-only cross-version migration tests using Docker Compose
with PostgreSQL. Tests verify that Metabase can upgrade/downgrade between
versions by checking the `/api/health` endpoint.
Usage:
```
./cross-version/test.sh --source v1.58.6 --target v1.58.7
```
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Upgrade mariadb images to 10.4.0
* Update job names to match the new version
* Advance minimum mariadb version to 10.4.6 to exclude unstable early versions of 10.4
* Don't specify patch version
* Upgrade images to MariaDB 10.6
* Upgrade CI actions to use pg14
* WIP: start updating MySQL 8.0 -> 8.4
* Advance minimum required app db versions
* Revert "Upgrade CI actions to use pg14"
This reverts commit 82f1aec823.
* Revert "WIP: start updating MySQL 8.0 -> 8.4"
This reverts commit c1c78d4298.
* Revert "Upgrade images to MariaDB 10.6"
This reverts commit f99d3c4f04.
* Revert just about all the changes to run CI
* Use newer MariaDB in semantic search tests
* Use newer mysql in semantic search tests
* Use newer postgres in semantic search tests
* Update app db version
* Update postgres version
* Trigger mysql driver tests
* Update unit tests to expect the new required versions
* Update mariadb and mysql used in drivers tests
* Fix expected supported mysql versions
* Remove dirt
* DRY the tests
* Dirt to trigger some CI tests
* Dirt to trigger some CI tests
* Remove dirt and update failing CI job
* Remove dirt
* Update another missed CI definition
* Add retry + checksum to Clojure install, remove continue-on-error
Remove continue-on-error: true from the Install Clojure CLI step so failures
surface immediately instead of silently propagating to downstream steps.
Add retry logic (3 attempts with curl --retry and --fail) and a post-install
clojure --version gate to handle transient download failures from
download.clojure.org.
* Add explicit error message when all Clojure install retries fail
* Convert frontend/src/metabase/visualizations/lib/timeseries.js to TypeScript
* Convert frontend/src/metabase/visualizations/lib/timeseries.unit.spec.js to TypeScript
* Convert frontend/src/metabase/visualizations/lib/errors.js to TypeScript
* Convert frontend/src/metabase/visualizations/lib/mapping.js to TypeScript
* Convert frontend/src/metabase/visualizations/lib/warnings.js to TypeScript
* Convert frontend/src/metabase/visualizations/lib/trends.js to TypeScript
* Convert frontend/src/metabase/visualizations/lib/numeric.js to TypeScript
* Convert frontend/src/metabase/visualizations/lib/table.js to TypeScript
* Convert frontend/src/metabase/visualizations/lib/table.unit.spec.js to TypeScript
* Convert frontend/src/metabase/visualizations/lib/action.js to TypeScript
* Convert frontend/src/metabase/visualizations/lib/renderer_utils.js to TypeScript
* Convert frontend/src/metabase/visualizations/lib/renderer_utils.unit.spec.js to TypeScript
* Convert frontend/src/metabase/visualizations/lib/table_format.js to TypeScript
* Convert frontend/src/metabase/visualizations/lib/table_format.unit.spec.js to TypeScript
* Convert frontend/src/metabase/visualizations/lib/utils.js to TypeScript
Add retry loop to fetch-artifact download step
Wrap the downloadArtifact API call with a retry loop (3 attempts,
exponential backoff at 15s/30s) to handle transient GitHub API 500
errors that have been causing e2e and health-check job failures.
Co-authored-by: mendral-app[bot] <233154221+mendral-app[bot]@users.noreply.github.com>
Fix codespell workflow: use prompt_file instead of shell expansion in YAML
The prompt field contained $(cat codespell_output.txt) which YAML passes
as a literal string — shell expansion does not occur in YAML values.
This caused Claude Code to receive the literal text instead of the actual
codespell output, resulting in exit code 1 on every nightly run.
Fix by writing the prompt to a file in a dedicated build step, then
passing it via the prompt_file input which claude-code-action supports.
Co-authored-by: mendral-app[bot] <233154221+mendral-app[bot]@users.noreply.github.com>
* Extract shared upsell components into common
Moved to common/components/upsells/:
- components/ subdir — the building blocks: UpsellBanner, UpsellBigCard, UpsellCta, UpsellGem, UpsellPill, UpsellWrapper, UpgradeModal, analytics, use-upsell-link
- constants.ts — shared URLs and constants
- 4 composed upsell pages used outside admin: UpsellEmbedHomepage, UpsellMetabaseBanner, UpsellStorage, UpsellUsageAnalytics
- UpsellCard (merged in from its old location at common/components/UpsellCard/)
Kept in admin/upsells/:
- 13 admin-only upsell pages (UpsellBetterSupport, UpsellCacheConfig, UpsellCloud, UpsellDevInstances, UpsellEmailWhitelabel, UpsellHosting, UpsellPerformanceTools, UpsellPermissions, UpsellSSO, UpsellSdkLink,
UpsellSemanticSearch, UpsellUploads, UpsellWhitelabel)
- These now import primitives from metabase/common/components/upsells/ instead of ./components
Updated ~30 import paths across the codebase. Reduced shared/common → feature/admin violations from 12 to 2.
There are 30 instances on stats with groups named "Data Analysts". We
need to handle this case.
First, we modify the migration:
- set `validCheckSum` to `ANY` to handle the case where this migration
already ran
- skip if there's already a `data-analyst` magic group
- insert a group named "Data Analysts", if one doesn't already exist
- otherwise name it "Metabase Data Analysts" (there are 0 groups on
with this name in cloud, theoretically it's possible but someone would
have to have BOTH `Metabase Data Analysts` and `Data Analysts` for this
to be an issue).
Similarly, we create a "Data Analysts" group on startup in OSS if one
doesn't already exist. This group is an invisible placeholder, that will
become visible if the instance ever upgrades. Adopt similar logic here:
if the existing group for data analysts is called "Metabase Data
Analysts", rename it to "Metabase Data Analysts (converted)" and create
the new placeholder group as "Metabase Data Analysts".
* Add worktree dev environment tooling
Add `mage dev-env` task for isolated worktree dev environments with
Docker containers and deterministic per-worktree ports. Includes workmux
layout with nREPL dev server, frontend build, and nvim panes. Adds a
custom Claude Code Docker image with Clojure tooling via mise, and
a bin/claude-dangerous helper script.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Move dev-env mage task to personal -edpaget-dev-env task
Move the worktree dev environment tooling from the shared mage task
namespace (mage.dev-env) to a personal task namespace (mage.edpaget.dev-env),
remove the sample .workmux.example.yaml, and update .workmux.yaml hooks
to use the new command name.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fixes
* test parser changes
* schema test
* fix linter
* implicit join error
* fe tests
* fix linter
* test fixes
* more tests
* tests
* upgrade field refs for dashboardcard_series and dashboards in transitive deps
- Add dashboard-upgrade-field-refs! for Phase 1b: upgrades parameter_mappings
on dashboards found in transitive deps (cards are upgraded first in Phase 1)
- Fix dashboard-card-update-field-refs! to find dashcards via
DashboardCardSeries, not just by primary card_id
- Split dashboard-ids out of transitive in runner, pass separately to run-swap*
- Add tests for series card parameter mapping swap and dedup
* fixup some tests, add promise to progress instance
useful for test so we don't have to loop on the db row
```clojure
(reify replacement.protocols/IRunnerProgress
(set-total! [_ total] ...)
(advance! [this] ...)
(advance! [this n]
(let [c (swap! completed* + n)
t @total*
prev (- c n)
crossed-boundary? (or (= c t)
(not= (quot prev progress-batch-size)
(quot c progress-batch-size)))]
(when crossed-boundary?
(let [progress (if (pos? t) (double (/ c t)) 0.0)]
(update-progress! run-id progress)
(when (replacement.protocols/canceled? this)
(log/infof "Replacement run %d was canceled" run-id)
(when on-complete
(deliver on-complete :run/cancelled)) ;; <-----------
(throw (ex-info "Run canceled" {:run-id run-id})))))))
(canceled? [_] ...)
(start-run! [_] ...)
(succeed-run! [_]
(log/infof "Replacement run %d succeeded." run-id)
(succeed-run! run-id)
(when on-complete
(deliver on-complete :run/success))) ;; <--------
(fail-run! [_ throwable]
(log/errorf throwable "Replacement run %d failed" run-id)
(fail-run! run-id (ex-message throwable))
(when on-complete
(deliver on-complete :run/fail)))) :<-----
```
and in tests we can do like so:
```clojure
(deftest advance!-progress-write-boundaries-test
(testing "advance! writes progress at batch boundaries and on final item"
(mt/with-premium-features #{:dependencies}
(mt/with-model-cleanup [:model/ReplacementRun]
(let [writes (atom [])
done? (promise)]
(with-redefs [replacement-run/update-progress!
(fn [_run-id progress]
(swap! writes conj progress))]
(let [record (replacement-run/create-run! :card 1 :card 2 (mt/user->id :rasta))
progress (replacement-run/run-row->progress record done?)]
(execute/execute-async!
(fn [progress]
(replacement.protocols/set-total! progress 120)
;; advance one-at-a-time for 120 items
(dotimes [_ 120]
(replacement.protocols/advance! progress)))
progress)
;; wait for virtual thread
(is (= :run/success (u/deref-with-timeout done? 500))) ;; <------------
;; Writes at items 50, 100, 120 (final) → progress 50/120, 100/120, 120/120
(is (= [(double (/ 50 120)) (double (/ 100 120)) 1.0]
@writes))
(is (=? {:is_active nil :progress 1.0 :status :succeeded}
(t2/select-one :model/ReplacementRun :id (:id record)))))))))))
```
* fixes
* add new flag to resolution
* fix
* fix sandboxing
* fix
* wip be
* wip be
* wip be
* wip
* wip be
* wip be
* missing join-alias
* wip be
* more field ref cases
* wip be
* wip be
* wip be
* remove healing
* implicit join second level
* swap refactoring
* both loops process all
[GHY-3182: swap all dependencies](https://linear.app/metabase/issue/GHY-3182/swap-all-dependencies)
* field refs
* fix types
* fix ref types
* short curcuit when swapping
* base-type test
* fix order by
* test
* Viz settings in cards
* handle field->expression conversion
* base-type fix
* unique fix
* deduplicate test
* only upgrade/swap when well-formed query
`(-> card :dataset_query :stages seq)`
degenerate cases are `{}` and `{:lib/type :lib/query}`
* remove tests with broken schema
* test that :field -> :expression works in parameters, too
* viz_settings!
* legacy->pMBQL
* preserve native queries metadata
* fix typo in a comment
* Tests for field_refs.clj
* swap function order
* remove transitive card ids, add usages tests
we don't need to keep a set of all card ids. it's huge, and we're gonna
upgrade cards either way. So no need to not upgrade card references
* Fix upgrade-field-ref-to-name crash on :aggregation refs
Viz-settings locations (pivot_table.column_split etc.) can contain
[:aggregation 0] refs, not just [:field ...]. Wrap ->pMBQL in
with-aggregation-list so it can resolve legacy index-based aggregation
refs to UUID-based pMBQL. Add try/catch as safety net for other
unexpected ref types.
Fixes 13 cards failing with "no aggregation at index 0" error.
* fix expressions
* add a test
* fix the test
* add a proper test
* Skip non-field refs in upgrade-field-ref-to-name
Per Alexander's feedback: since we only resolve :field refs downstream,
skip :aggregation/:expression refs at the gate instead of setting up
aggregation context for ->pMBQL.
* remove log call in source-swap
* add schemas
* add schemas
* repro test
* guard against missing tables
* more fixes
* wip tests
* add tests for resolution with missing metadata
* Mutation testing improvements
* Some more tests
* revertme: adds comilation tracker for statsdb swap+upgrade analysis
* add checks for wrong queries
* fixes
* test fixes
* fix
* remove old code
* temp test fixes
* fix viz
* fixes
* One more tests.
* Singular mutation testing run
* tests
* fix tests
* remove not needed column checks
* fix tests
* run-swap throws with old-source new-source and check-result
* fix dependencies check test
```clojure
source-swap-test=> (clojure.test/run-test swap-native-card-source!-updates-dependencies-test)
Testing metabase-enterprise.replacement.source-swap-test
Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
{:test 1, :pass 1, :fail 0, :error 0, :type :summary}
```
So an interesting bit:
we must update dependencies regardless. It's important that the dep
graph is updated. Imagine if Card A is swapped for Card B. All cards
that use Card A will be listed as deps of Card A even if they don't
reference it and don't require changes. We are still swapping the source
so we need to update the graph.
We don't have a real api for dependencies. So to ensure the cards are in
the graph we call `(events/publish-event! :event/card-create {:object
card :user-id (mt/user->id :rasta)})` and when swapping we prefer to use
`(models.dependency/swap-dependency! :measure entity-id old-source
new-source)`. But for dashboards we'll just do the full recompute
accomplished with
```clojure
(events/publish-event! :event/dashboard-update
{:object (t2/select-one :model/Dashboard :id dashboard-id)})
```
* working field-refs tests
* Fix logic
* Strong logic (but is it possible?)
* fix ::entity-id schema to include all entity types
* refactor
* fixes
* fixes
* fixes
* fixes
* fixes
* wip
* fix
* fixes
* fixes
* fixes
* fixes
* fixes
* fixes
* wip be
* wip be
* wip be
* remove unused
* wip be
* remove invalid queries
* wip fixes
* wip walk function
* Measure and segment tests
* fixes
* aliases
* fixes
* fix tests
* make source_swap accept the preloaded entity
* renaming
* unify sources
* database id fixes
* fix table id check
* deps fixes
* reflection warning
* fixes
* fixes
* fixes
* fix team
* fix CI
* check for admin
* disable plugin
* Disable api endpoints in prod
* wip tests
* fire events; namespace doc
```clojure
clj -X:dev:test:ee:ee-dev :only metabase-enterprise.replacement.source-swap-test
Ran 29 tests in 29.144 seconds
219 assertions, 0 failures, 0 errors.
{:test 29, :pass 219, :fail 0, :error 0, :type :summary, :duration 29144.393083, :single-threaded 29}
Ran 0 tests in parallel, 29 single-threaded.
Finding and running tests took 40.1 s.
```
All tests passing
Fire event only when things change, swap sources before. That's faster
and must happen regardless, change event only on change.
Namespace docstring indicating why
* fix constraint error if dependency already exists:
```
custom-swap=> (source-swap/swap-source! [:card 27452] [:card 6759] [:table 52728])
Execution error (PSQLException) at org.postgresql.core.v3.QueryExecutorImpl/receiveErrorResponse (QueryExecutorImpl.java:2736).
ERROR: duplicate key value violates unique constraint "idx_unique_dependency"
Detail: Key (from_entity_type, from_entity_id, to_entity_type, to_entity_id)=(card, 27452, table, 52728) already exists.
```
it seems to have already have a dependency. So now we just remove
dependency if the one we are adding already exists, otherwise we swap.
swap card A: dep on B -> dep on C
if it already depends on C, just delete the dep on B
otherwise swap where card A dep B -> dep C
* simplify check
* upgrade parameter value_source_config
* field refs
* tests
* viz tests
* parameters
* tests
* fixes
* tests
* check sources fixes
* fix tests
* tests
* wip tests
* fixes
* wip tests
* wip tests
* wip tests
* wip cleanup
* cleanup
* move test
* lint
* fix
* fix
* fixes
* fixes
* fixes
* fix
* fixes
* fix
* fix
* fixes
* fix
* FE cache fixes
* fix checks
* fix
* polish: remove tracing code, update docstring, add :pending to ::run-status enum
* Add test for table->card in check-column-mappings
* Remove unneeded argument.
* Remove unused function.
* Test that all api endpoints are gated behind dependencies feature flag
* use api/*current-user-id* instead of (:id @api/*current-user*) in source swap
Avoids unnecessarily forcing the *current-user* delay (which loads the
full User record from the DB) when only the integer ID is needed.
* stop swapping card_id in parameters
* throw on partial swap failures so the run is marked as failed
Previously run-swap* returned {:failures [...]} which execute-async!
ignored, marking the run as succeeded. Now it throws with a descriptive
message (count + first 10 failures) that gets persisted in the run's
message column.
* Check that all routes are protected behind superuser
* Add test for runner.run-swap
* FE checks
* Remove source-error schema (unused)
* Reuse lib-be.schema.source-swap/column-error schema instead of remaking it
* Kondo error
* move code to lib-be
* Better mutation testing
* fixes
* More tests for field-refs
* Better test
* More source swap tests.
* suppress dependency event handlers during source replacement
Source replacement manages dependencies itself via swap-dependency!, so
the event-driven dependency recalculation, stale-marking, and metadata
propagation triggered by entity update events is redundant work.
Adds *suppress-dependency-events?* dynamic var, checked by all dependency
event handlers. Bound to true in run-swap* phase 2.
* Fewer deps! Yay!
* Revert "suppress dependency event handlers during source replacement"
This reverts commit 03625e9c26.
* Clearer code
* update 2 docstrings
* publish dashboard-update event only when a dashcard changed
* simplify implementation
* simplify
* bulk mbql replacement
* Revert "bulk mbql replacement"
This reverts commit cf12636c42.
* simplify
* fix reseting selection when the graph changes
* fix
* add source_swap module
* fix lint
* fix comment
* fix
* fixes
* fix module order
* fix module order
* fix order
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
Co-authored-by: dan sutton <dan@dpsutton.com>
Co-authored-by: Eric Normand <ericwnormand@gmail.com>
* Add configurable CSV field separator setting
- Adds csv-field-separator setting for configuring CSV export delimiter
- Supports comma (default), semicolon, tab, and other delimiters
- Useful for European locales where semicolon is standard
- Configurable via MB_CSV_FIELD_SEPARATOR environment variable
- Includes tests for different separator types and quoting behavior
Fixes#10256
* Split CSV separator tests into smaller deftests and clean up comments
Break the monolithic csv-field-separator-test into four focused tests:
- csv-field-separator-default-test
- csv-field-separator-semicolon-test
- csv-field-separator-tab-test
- csv-separator-quoting-test (already separate, just removed extra nesting)
Remove the "Changed from hardcoded" comment in csv.clj and the unused
metabase.test.util require.
* fixup! eastwood and serialization baseline
* fixup! whitespace
---------
Co-authored-by: kevin raymond <kevray39@gmail.com>
### Stats
- **Surviving mutations before:** 6
- **Tests added:** 2
- **Mutations killed by this PR:** 4
- **Mutations remaining:** 2
### Mutations killed
- Replace -1 with 0
- Replace field-ref with nil
- Replace :aggregation with :aggregation__
- Replace lib.metadata.calculation/returned-columns with nil
### Mutations not killed (with rationale)
- Replace some? with nil — Schema-only mutation — changing the Malli schema annotation does not affect runtime behavior
- Replace query with nil (in find-matching-column) — lib.equality/find-matching-column does not use the query arg for field ref matching in these test cases
* Add v3 for modular embedding analytics
* Add implementation
* Improve test readability
* Improve code format
* Make the code autocomplete-able to prevent human-error
* Add missing default value tests
* Format
* Ensure we assert all properties
* fix "id_new" and "id_new_native" tracking
* Update the implementation so it count 0 like before
* Fix logic
* Fix snowplow schema lint
* Remove too much linear ticket number references
* make LeafletMap generic component
* convert leaflet grid heatmap
* use filter guard to make sure we're dealing with L.Rectangle
* remove param which doesn't exist
* add guard helper for data set column
* create LeafletMapPoint variadic generic type
# Conflicts:
# frontend/src/metabase/visualizations/components/LeafletMap.tsx
* extact sync map from props methods
* migrate LeafletMarkerPinMap to TS
* don't force assert existence of pin dependencies
* use 3 tuple point everywhere in LeafletMarkerPinMap
* use ClickObject type
* migrate LeafletTilePinMap to TS
* convert LeafletChoropleth to TS
* remove type assertions in LeafletMarkerPinMap
* remove TileLayerWithInternals casting
* improve tile layer typings
* add casting comment
* improve embedding context types
* improve LeafletGridHeatMap types
* don't hardcode default scale for d3 scale linear
* call zoom control also when prevProps are missing
* improve HoveredObject, use HoveredObject in LeafletGridHeatMap
* remove unneeded nullish coalescence
* simplify getTileUrl by using just parameters as arguments
* remove unneeded assertion
* remove unneeded this param
fix: prevent script injection via PR title in embedding-sdk workflow
Move user-controlled PR context values (title, number, URL) and secrets
from direct GitHub Actions expression interpolation into env vars.
This prevents a crafted PR title from breaking out of the shell string
and executing arbitrary commands on the runner.
Co-authored-by: mendral-app[bot] <233154221+mendral-app[bot]@users.noreply.github.com>
* quarantined driver notifications
* temp on push to bot-testing channel
* prettier
* more compact
* add back a bit of emphasis
* remove push edge and restore channel
* Clarify that SSO provisioning also allows reactivating accounts
* Update wording in docs
* ditto LDAP
* Use radio buttons instead of a toggle
* lint/wording
* lint
* Revert changes to autogenerated docs
* Fix eslint rule + improve wording
* Make OIDC provisioning setting consistent with other providers
* Add detailed descriptions only to env var docs, not UI
* Fix E2E test
* fix: update ESLint configuration file path from .eslintrc.js to eslint.config.mjs
* run lint when we change component tests specs
* fixing linting
* ignore linting issue in embedding test
---------
Co-authored-by: Nicolò Pretto <info@npretto.com>
Co-authored-by: Nick Fitzpatrick <nickfitz.582@gmail.com>
Fixes QUE2-394.
This issue was caused by losing the thread of a column between stages in
the case where it has a different `:inherited-temporal-unit` than its
proper `:temporal-unit`.
`lib.equality/=` was checking the `:temporal-unit`s, which match, and
then returning `false` because only one column had
`:inherited-temporal-unit` for a different, finer unit. That shouldn't
fail the match, since the `:inherited-temporal-unit` is obsolete when
the column has just been bucketed differently.
Although there was an early return if isLoading is true, the useState had already run, and set the state to the DEFAULT_FORMATTING_SETTINGS. This would then not get replaced when isLoading became false.
Instead, use the same idiom as SessionTimeoutSetting - install a useEffect hook that depends on the setting value and applies that setting value to the local state.
* fix: allow dashboard caching in read-only mode
* fix: split out cache permissions from can-write?
We were denying changing caching policy for dashboards on read-only
remote-synced instances because we set can-write? to false for
dashboards when we are in read-only mode.
we also need to create a new boolean flag we can return to the frontend
to indicate that caching is still allowed even when editing a dashboard
is otherwise disabled.
* fixup! PR feedback
* add another feature flag for transforms
* fix linting error from master
* add debug info
* enable-transforms-basic -> enable-basic-transforms
* remove logs
* Add "configure data permissions and enable tenants" skeleton for onboarding (#68895)
* use onboarding stepper
* add TODO for EMB-1266
* add e2e test for skeleton
* use gap=sm for steppers
* align with design
* Add completion checks for data segregation and tenants (#68890)
* add completion information for segregation and tenants
* add data segregation checks
* add data segregation config test
* add the combined data-permissions-and-enable-tenants step
* separate main and sub checklist
* explain the purely ui step
* add schema for checklist response
* update checklist completion states
* add test cases
* address review on test permutations and using is_active for tests
* remove stale configure data permissions unit test
* remove stale test on user strategy card
* Add enable tenants and create shared collection button (#69131)
Add enable tenants and create shared collection button (#69131)
* Add data segregation strategy selection step for embedding onboarding (#69127)
Add data segregation strategy selection step for embedding onboarding (#69127)
* Lock steps 2 to 5 in data permissions setup checklist (#69135)
Lock steps 2 to 5 in data permissions setup checklist (#69135)
* Add link to documentation for database routing in the onboarding step (#69138)
Add link to documentation for database routing in the onboarding step (#69138)
* Add tenant creation and tenant summary step for tenants onboarding (#69454)
Add tenant creation and tenant summary step for tenants onboarding (#69454)
use index as key instead
update failing unit test
refactor: use slugify instead
add assertion on configure data permissions step should be done
cleanup summary components
adjust font size
use related setting card for summary
refactor: cleanup card components
add e2e test verifying tenant errors are shown
add e2e test verifying tenants are created
fix: prevent going to previous incomplete step
fix: skip locked steps
fix: consider locked steps when auto-advancing
fix: failing e2e by enabling tenants via ui
refactor: cleanup tenants import
add fallback logic for tenants listing
refactor: move onboarding step to tenants plugin
scaffold tenant summary step
scaffold tenant creation step
use paper from mantine-ui instead
scaffold tenant creation step
* Onboarding stepper should navigate to next steps correctly even when all steps are done (#69653)
Onboarding stepper should navigate to next steps correctly even when all steps are done (#69653)
cleanup: remove redundant test
fix: cleanup broken tests
remove inapplicable test case
apply onboarding stepper fixes
* Add row and column level security step for onboarding (#69622)
fix failing e2e tests
wait for permission graphs to update before checking error toasts
add test assertion on hiding selected tables
hide tables that are selected by other cards
add unit test for building permissions graph
address review on type-safe sandbox policy
extract error and success toasts into UI consumer
add clarification on why we assert GTAP
extract MiniPicker hard-coded margins to the data picker call site
address review on using design system units and rem
address review on "All tenant users" permission update hook
Add row and column level security step for onboarding
revert all changes to OnboardingStepper
update the permission graph as well
refactor: renamings
lock step "Select data to make available" if strategy not chosen
cleanup: remove button
use metabase/ui for select style
refactor: extract table column card
simplify trigger styles
revert MiniPicker change
undo test name change
refactor: simplify stepper
update e2e test assertions
add step completion assertion to e2e tests
refactor: extract to step components
allow going to next step
fix onboarding stepper re-rendering
chore: cleanup sandboxing creation test
add e2e tests
add sandbox on clicking next
clicking next on rls selector should advance to next step
refactor: rename to selections and onSelectionsChange
display options on the bottom
position dropdown directly below the trigger
fix: trigger height
fix: do not hide title on RLS step
do not collapse current step when all steps are complete
make it clearer what is goToNextIncompleteStep
simplify logic for going to next incomplete step
add data selector for row and column security
* Create a multi-select database selector component (#69635)
Create a multi-select database selector component (#69635)
chore: remove fluff tests
allow disabling rightSection with null
add data fetching to DatabaseMultiSelect
use multi autocomplete
add storybook story and unit test for disabled options
allow disabling database options
update multi select tests
chore: formatting
cleanup dropdown style
show placeholder always
apply brand focus color
remove inline-end
limit combo box width
update styles to match figma design
update option icon
scaffold database multi select
* Add connection impersonation onboarding step (#69644)
Add connection impersonation onboarding step (#69644)
add permission graph and impersonation assertion
add e2e test
use useUpdateTenantGroupPermissions for permission updates
add connection impersonation creation hook
add edge case for database does not support impersonation
disable dbs without connection impersonation
scaffold ui for connection impersonation
* Add JWT authentication onboarding for embedding hub (#69823)
test: e2e fail due to lock icons found in related settings section
replace e2e test on jwt completion
update e2e tests
scaffold e2e tests
refactor: cleanup sso stepper component
add help url and complete step button
scaffold "Test that JWT authentication is working correctly" step
scaffold "Add a new endpoint to your app" step
add ui to configure identity provider
scaffold sso setup
* Mark get embed snippet step as done when guest embed is published (#69889)
* mark guest embedding onboarding test step as done
* add e2e test for guest embed
* simplify e2e test
* fix invalid imports due to merge conflicts
* fix guest embed snippet copy incorrectly marking production embed step as done
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Embed wizard should default to newly created dashboards (#69907)
Embed wizard should default to newly created dashboards (#69907)
scaffold defaulting to newly created dashboards
add a comment on why newly created dashboards are prioritized
chore: update inline comments
de-duplicate recent items at call site
refactor: cleanup implementation
add created_at time filter
address review feedback on variable naming
address review by going through actual x-ray flow
remove invalid unit test
update duplicated and failing tests
fix failing tests
fix failing select-embed-experience e2e test
fix failing select-embed-entity e2e test
* Remove create models step from embedding onboarding checklist (#70256)
remove create models step
* Show tenant_identifier autocompletion from column values in RLS (#69829)
Show tenant_identifier autocompletion from column values in RLS (#69829)
scaffold basic tenant_identifier autocompletion
make placeholder dynamic to tenant identifier input
fix: update broken types
document why we use ad-hoc query
refactor: small cleanup
cleanup docstring
chore: update hook tests
add e2e tests for autocompletion
explicitly pass tenant attributes
simplify CreateTenantInput
Suppress ad-hoc query errors in tenant identifier autocomplete
Autocomplete suggestions are best-effort during onboarding. Query
failures (permissions, timeout, etc.) should not surface global
error UI noise.
Make attributes optional in CreateTenantInput to match API contract
The backend marks attributes as optional; the frontend type was
overly strict and forced callers to always send attributes: {}.
Derive selectedFieldIds from sandboxedTables, not tableColumnSelections
Previously, field IDs were extracted from all tableColumnSelections,
including entries skipped because the table failed to fetch. Now we
only include IDs from policies that were actually applied, deduplicated.
Use stable placeholder and aria-label in TenantIdentifierInput
The old dynamic placeholder (first suggestion value) looked pre-selected
to users. Now prefix with "e.g." using c().t for localization context,
falling back to "e.g. acme-corp" when no suggestions are loaded. Also
adds aria-label for accessible and stable test targeting.
Update E2E tests to use "e.g. 1" placeholder for tenant_identifier input
The placeholder is now prefixed with "e.g." to clarify it's an example.
This also implicitly validates that autocomplete suggestions were fetched
successfully, since the placeholder is dynamic when data is available.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Tenant creation should differ the data isolation field by strategy (#70258)
Tenant creation should differ the data isolation field by strategy (#70258)
scaffold implementation on tenant isolation fields
refactor: use ts-pattern for isolation field config and label lookups
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
update checklist endpoint to return checklist separate from isolation strategy field
refactor: use dataIsolationFieldValue field for creating tenants
fix: use activeStrategy for reload-aware strategy picker and step content
On page reload, selectedStrategy is null but activeStrategy is populated
from the backend. Switch the data segregation picker value, select-data
step lock condition, title hide flag, and match expression to use
activeStrategy so the correct state is reflected on reload.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
tweak tenant name styling
always require data isolation strategy field value to create tenants
update embedding hub checklist backend test
add e2e tests for reload and isolation strategy
fix: remove hard-coded string in tenant identifier input
refactor: move DataSegregationStrategy type to embedding-hub types module
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
refactor: remove single-letter variables
* Show segregation strategy specific data permissions in tenant summary (#70268)
Show segregation strategy specific data permissions in tenant summary (#70268)
show segregation strategy specific summary
test: add unit tests for getDataPermissionsDescription
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: add selectedFieldIds to TenantsSummaryOnboardingStep plugin interface
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
add e2e test assertions and test cases
simplify row-level security summary logic
fix rls imports
update data permission description span
add bold text to description summaries
fix failing e2e tests
fix starter test failing
remove focused test
update comment on the rls e2e test
fix: use singular 'table' for single-table RLS data permissions description
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
add a comment on why RLS needs the same column
use conjunct method for string formatting
simplify textFromReactNode
simplify getDataPermissionsDescription unit test
chore: code formatting
remove route title
fix: failing test due to placeholder update
remove the starter test as it does not apply
fix: reset hiddenParameters when switching from guest to SSO embed mode for dashboards
When recentlyCreatedDashboards pre-populates the dashboardId, the resource loads
in guest mode and initializes hiddenParameters with all parameters as disabled.
Switching to SSO now resets hiddenParameters to [] for dashboard embeds, matching
the existing behavior for question embeds.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix ignore_error flag
fix: incorrect test assertion for database selector
fix: incorrect placeholder assertion
* Tenant users are unable to query database after onboarding flow (#70476)
Tenant users are unable to query database after onboarding flow (#70476)
fix(embedding): set create-queries permission for RLS tenant users
Tenant users with RLS could view dashboards but were unable to query
the database because the permissions graph only set `view-data` without
`create-queries`. Add `"create-queries": "query-builder"` at the
database level for both sandboxed and impersonated configurations.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: update unit tests on embed settings
fix failing tests due to parameters reset
* Polish tenant creation and SSO for embedding onboarding (#70477)
Polish tenant creation and SSO for embedding onboarding (#70477)
polish permissions setup and create tenants onboarding
update copy for tenant identifier
* fix(type): add rlsColumnName type to plugin defs
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
There was a typo in a transducer where the cat and remove were swapped. The cat was meant to combine collections (`#{x})` and the remove was looking for `x`, but since it happened in the wrong order, it never removed them. This led to self-dependent entities, and therefore cycles which caused infinite loops.
* should fix clj files imported from sdk component tests
* reduce noise when not finding the sdk package
* remove mantine from tests that don't need it
* eslint rule
* test ci
* fix
* fix e2e
* make linters happy?
* more ci fixes
* more ci fixes
* maybe fix flake
* more ci things?
* more things including a test that we do fallback to old auth
* remove un-needed changes
* more cleaning
* make tests more stable
* clean up debug comments
* remove ! that was not needed
* fix broken build
* cleanup
* more cleaning up
* more cleanup
* clean up consts
* more cleaning up and saml bail out tests + logic
* revert uneeded change in copy js plugin
* better comments and renamed chunk auth function
* merge test files, fix double auth requests done due to missing redux dispatch
* fix typo in backend comment
* fixed version of renaming param to useLegacyMonolithicBundle
* remove cy wait
* remove wait + fix matcher on request for dev mode to not match map file
* clean up tests
* remove empty lines from gitignore
* remove unrelated gitignore change
* r emove imports
* cleanup tests from redundant tests
* remove other un-needed code
* comments
* add date to a TODO as suggested by Kelvin
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* rename checkForAuthConfig -> checkForAuthConfigAndStartAuth
* startAuth -> startJwtAuth
* use new URL to construct baseUrl
* use shared auth logic
* use the real version as we can actually
* add comment on numTestsKeptInMemory usage
* add test about path traversal
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Fix duplicate alerts/subscriptions firing in multi-pod clusters
Set acquireTriggersWithinLock=true to force pessimistic locking during
trigger acquisition. Without this, Quartz uses an optimistic UPDATE that
is vulnerable to an ABA race condition where multiple cluster nodes can
acquire and fire the same trigger simultaneously.
* Add *warn-on-reflection* to custom_migrations/util
Fixes clj-kondo warning triggered by Java interop (System/setProperty,
System/clearProperty) added in the acquireTriggersWithinLock fix.
* Convert `admin/people/selectors` to TypeScript
Part of the Accelerate TypeScript Migration project.
Closes DEV-1509
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add people property to AdminState type
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* [Mutation Testing] Add tests for metabase.lib.field/fieldable-columns
* test: mutation tests for column-metadata->field-ref source-alias propagation
Tests the MEGA HACK source-alias-to-join-alias propagation in column-metadata->field-ref:
- Verifies source-alias becomes :join-alias for table-default columns
- Verifies inherited columns do not propagate source-alias
- Verifies inherited columns do not get non-inherited propagated keys
Kills 6 more mutations (51/60 total, 9 unkillable due to schema constraints).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
The previous check used `.includes("openstreetmap.org")` which could be
bypassed by malicious domains like `openstreetmap.org.evil.com` or
`fake-openstreetmap.org`.
This fix validates that the hostname is either exactly `openstreetmap.org`
or a legitimate subdomain ending with `.openstreetmap.org`.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Fixes QUE2-351.
I added a new exported function `lib/ignore-case`, since some drivers,
test, and Metabot were using `lib.options/update-options` to set
`:case-sensitive false` on `:contains` et al clauses.
`lib.options` is only used by friends now (mostly QP).
* do not put embeddings in metadata json for semantic search
* semantic search: filter personal_owner_id right in the database
* simplify batch-update code
* Wrap title and buttons earlier when nav bar is open
* Wrap title in header at wider breakpoint
* Left align ad-hoc questions too
* Add gap when question title collapses
* Increase breakpoint
* Fix padding on ViewSection
* Use container query for query builder breakpoints
* Make padding more consistent with other headers on small screens
* Tweak question action buttons on small screens
* Add separator for actions
* Fix padding on filter panel
* Fix padding on back button
* Fix padding to match header
* Break sooner
* Do not break subheader as soon
* Fix test
* Remove reliance on query-builder from FilterPanel component
* Remove spurious margin
* Fix gap between header items
* Move container to header and footer
* Rename confusingly named setParameterAttribute
* Remove unnecessary check
* Add options to disable parameter settings
* Disable parameters settings in transform sidebar
* Add e2e test for parameter field settings
* Rename disableParamaterSettings to parametersAreUserVisible
* Make parametersAreUserVisible configurable all the way up to QueryEditor
* Update copy when parameters aren't visible to user
* Fix test
Previously, password-reset-disabled? blocked password reset for all
SSO users unconditionally (except Google when SSO was globally
disabled). After a license downgrade, EE SSO users were locked out —
they couldn't use SSO (no license) and couldn't reset their password
either.
Introduce sso-source-enabled? which checks whether a user's specific
SSO provider is currently active, using setting/get to respect the
:feature guards on EE settings. password-reset-disabled? now delegates
to this function, so password reset is only blocked when the provider
is actually usable.
Fixes QUE2-352.
- `lib.util/strip-id` is newly exported with the more self-documenting
name `lib/display-name-without-id`.
- `lib.util/clause?` and `clause-of-type?` were already exported
- `lib.util/source-card-id` (and `source-table-id`) are exported as
`lib/primary-source-card-id` (and likewise for tables).
This isn't all the escaped usages of `lib.util` functions, but other PRs
will handle those.
* chore: convert viz tests to typescript
* convert more tests
* more tests
* more tests
* more tests
* use global helpers
* more tests
* more tests
* convert settings definition
* more files to typescript
* more files to typescript
* fixup
* fixup
* Update v59 sample Examples visualizations and dashboard. (#70075)
Adds Box Plot viz and a Scalar viz with conditional colors to the Example dashboard.
The resources/sample-content.edn was regenerated from the live instance content.
* Pin patched libraries to their patched version (#70143)
* address review
* add a guard
---------
Co-authored-by: Katya Voropaieva <110612633+KorowkaMu@users.noreply.github.com>
Co-authored-by: Nemanja Glumac <31325167+nemanjaglumac@users.noreply.github.com>
* Convert Legend components to TypeScript (DEV-1502)
Convert Legend, LegendItem, LegendItemDot, and LegendLayout.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* GDGT-1859: Fix Python transforms not enabled after adding to the trial
* GDGT-1859: Add page refresh after enabling Advanced Transforms
* GDGT-1859: Update unit tests
* chore: convert dashcard related redux components to typescript
* fixup
* address review
* another review iteration
* add a missing import
* chore: split into components and hooks ClickMappings
* fix imports
* delete old file
* make hook code human readable
Update E2E test timings with averaged data
Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).
🤖 Generated automatically by update-e2e-timings workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
Fixes QUE2-383.
Uses `:lib/card-id` in metadata to check if a column came from a card at any point.
If so, it should use a name-based `:field` ref rather than an ID-based one.
This makes the refs to joined columns consistent with those to main source columns,
based on whether the source is a table or card.
* Tenant attributes and collections improvements
First round of edits to add the ability to create tenant attributes using SSO.
* Revise tenant-specific access for non-admin internal users
* Edits based on Alberto's comments
* Move tenant claims info to claims section, link to it from attributes
* Apply suggestion from @alexyarosh
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
* Moved tenant attribute logic and simplified sentence
* Fix a sentence, improve code example, and clean up permissions bullet points
---------
Co-authored-by: Alex Yarosh <alexandra@metabase.com>
As of Cypress 15.10.0, `Cypress.env()` is deprecated. This commit migrates
all usages to the new recommended APIs:
- Non-sensitive config values → `Cypress.expose()` (synchronous)
- Sensitive token values → `cy.env()` (asynchronous)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* revert sorting by id on parameters serialization, parameters ordering is significant and must be preserved
* revert, we're storing position
* feat: add :position to parameters, backfill
* feat: backfill position on imported parameters
* chore: tests
* FE: update position
* chore: gitignore python-sources.lock
* chore: ensure we get position
* restrict to dashboard params
* tests
* simpler approach: just sort by position on serialization export/import
The legacy .insertAll API (and newer Storage API) do not work well with
tables that are created/dropped frequently. This is because their
metadata caches can take a long time to up date.
Python transforms use the drop-recreate fallback strategy for bigquery
so we observe this inconsistency for every run after the first, even
with absurd multi minute timeouts.
The SQL API does not have a problem, so we'll use that. We expect a
performance regression but its better to work slowly than not work at
all.
Other changes:
- Removed now redundant bigquery retry loop
- Dropped dictionary support, we cannot support it easily and it cannot be
considered higher prio than bigquery working at all.
- Added correct parameter typing for JSON.
- Fixed ordering issues in tests (assumed row order when it is
non-deterministic)
* fix: real error codes in streaming responses
* fixup! try faking old errors deeper in the stack
* fixup! wrap fetch endpoint in embedding too
* fixup! treat string only errors differently
* fixup! handle errors in a couple other places
* fixup! hard to figure out what kind of error it is
* fixup! typecasts
* fixup! test arity
* fixup! remove blanket 403 page from dataset
* fixup! simplify backend 403 responses
* fixup! a couple missed tests
* fixup! add mockresponse for mockclient
* fixup! other failing tests
* fixup! last failing test
* fixup! add another OSS check
* fixup! one more oss test
* fixup! update status codes on some more tests
* fixup! use committed function
- Was using wrong project / creds
- Using two different sha names only worked when shas were not used as
qualifiers in the query (they are for bigquery)
Added the ability to override the hash-key used for tx/hash-dataset to
support forcing a hash collision (as is required by routing).
Many uses of `lib.util/source-card-id` and `source-table-id` were
followed by calling `lib.metadata/card` or `lib.metadata/table`
respectively.
This PR adds two new functions `lib.metadata.calculation/primary-source-card`
and `primary-source-table` which look up the card/table directly.
It also adds `lib.metadata.calculation/primary-source` for the handful
of places where whichever of `primary-source-card` or `primary-source-table`
returns non-nil gets used, such as in some drill-thrus.
- Adds a fully automated `run!` entry point to `dev.mutation-testing` that orchestrates the entire mutation testing workflow: generate report → create Linear project → group functions → write tests via `claude -p` → verify → retry →
commit → create draft PRs
- Replaces the previous manual, step-by-step SKILL.md instructions (which required the operator to execute each step) with Clojure functions that handle the full pipeline automatically
- Key new functions: `parse-namespace`, `group-functions`, `build-test-prompt`, `invoke-claude!`, `verify-and-retry!`, `process-group!`, `print-summary!`
## Motivation
The mutation testing workflow previously required ~20 manual steps per namespace, with the Claude operator interpreting skill instructions and running each step. This was slow, error-prone, and required constant supervision. Now you just
call `(mut-test/run! 'metabase.lib.order-by)` and walk away.
## How it works
1. **`parse-namespace`** — derives source/test paths, short names, auto-detects `.cljc` vs `.clj`
2. **`group-functions`** — clusters private functions with their public entry points based on test overlap, skips fully-covered functions
3. **`build-test-prompt`** — assembles a prompt with function source, existing tests, and surviving mutations
4. **`invoke-claude!`** — shells out to `claude -p` with scoped tool access (`Edit`, `Read`, `Bash(clj-nrepl-eval*)`)
5. **`verify-and-retry!`** — checks which mutations were killed, retries up to 2x with targeted prompts for survivors
6. **`process-group!`** — full per-group pipeline: branch → invoke Claude → verify → commit & push → Linear issue → draft PR
7. **`run!`** — ties it all together, catches errors per group so one failure doesn't stop the batch
Given browser or API access to a Metabase instance, the tools in this PR
can recursively fetch cards and all the metadata (tables, fields, etc.)
required to run them through QP preprocessing and compilation.
Of course this cannot actually _execute_ the compiled form, since you
don't have access to the user's DWH, but it enables debugging of
hard-to-reproduce user bug reports in your local REPL.
This works by making API calls like `/api/card` and
`/api/card/query_metadata` until a complete picture of the metadata is
acquired. This is poured into a `MetadataProvider`, which is all the QP
needs to analyze and compile the query.
Update E2E test timings with averaged data
Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).
🤖 Generated automatically by update-e2e-timings workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
Adds Box Plot viz and a Scalar viz with conditional colors to the Example dashboard.
The resources/sample-content.edn was regenerated from the live instance content.
* chore: convert dashboard reducer to typescript
* convert to rtk
* use immer when possible
* fixup
* Apply suggestion from @kamilmielnik
I checked it locally and it is indeed unnecessary guard
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
---------
Co-authored-by: Kamil Mielnik <kamil@kamilmielnik.com>
Part of the mutation testing project for metabase.lib.field.
Closes QUE2-338
Functions
Stats
Surviving mutations before: 16
Tests added: 1
Mutations killed by this PR: 16
Mutations remaining: 0
Mutations killed
Replace fingerprint with nil
Replace :type with :type__
Replace :type/DateTime with :DateTime__
Replace let with comment
Replace earliest with nil
Replace latest with nil
Replace when-not with when
Replace days with nil
Replace > with >=
Replace > with nil
Replace days with nil
Replace 1 with 0
Replace :minute with :minute__
Replace 31 with 0
Replace 365 with 0
Replace :week with :week__
Mutations not killed (with rationale)
Replace :day with :day__ — available-temporal-buckets-for-type has its own internal default of :day for datetime types, so the :day return value from fingerprint-based-default-unit is redundant with the fallback
* [Mutation Testing] Add tests for metabase.lib.field/fieldable-columns
* test: add mutation tests for column-metadata-effective-type and nest-display-name
Adds tests that exercise:
- type-of fallback to :base-type when :effective-type is absent
- type-of preference for :effective-type over :base-type
- temporal extraction returning :type/Integer
- temporal truncation preserving effective type
- nested field with nil parent display-name graceful fallback
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Fixes QUE2-350.
There are two changes here:
- Exporting some legit public functions from `lib.core`
- "Befriending" the `enterprise/sandbox` and `lib-be` modules, which are
deeply coupled to lib and MBQL in any case.
That allows us to remove several exported namespaces:
- `lib.binning`
- `lib.convert`
- `lib.field`
- `lib.metadata.cached-provider`
- `lib.metadata.column`
- `lib.metadata.invocation-tracker`
- `lib.metadata.result-metadata`
- `lib.normalize`
- `lib.query`
- `lib.temporal-bucket`
- `lib.walk`
The functions newly exported from `lib.core` are:
- `lib.field/infer-has-field-values`, called from some BE API handlers
and AppDB readers, producing a "cooked" value the FE wants rather than
exposing details of how `:model/FieldValues` is stored. Single-use,
but a legitimate export.
- `lib.field/json-field?`, checks if the passed in field is a JSON field,
that's a valid export used in a few places.
- `lib.query/native?`, checks if the first stage of this query is native,
that's another legit export used in a few places.
`lib.normalize/normalize` and `lib.query/query` were already exported, I
updated some usages of those functions to `lib/foo` instead.
* Convert `admin/permissions/utils/urls` to TypeScript
Part of the Accelerate TypeScript Migration project.
Closes DEV-1510
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* improve type guard
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Uladzimir Havenchyk <uladzimir@metabase.com>
Move some logging-only computations into the `(log/* ...)` call so it won't be executed
unless the log level is sufficiently verbose for the log to actually be output.
* GDGT-1755: Migrate Text to TS
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* GDGT-1755: Migrate Label to TS
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* GDGT-1755: Migrate Subhead to TS
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* GDGT-1755: Migrate IconWrapper to TS
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* GDGT-1755: Migrate LabelIcon to TS
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* GDGT-1755: Migrate use-scroll-on-mount to TS
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* GDGT-1755: Missed TS fixes
* GDGT-1755: Migrate QueryButton to TS
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* GDGT-1755: Migrate IconBorder to TS
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* GDGT-1755: Remove Text.tsx wrapper, import from metabase/ui directly
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* GDGT-1755: Adjust line height in places using new Text component
* GDGT-1755: Fix styling inconsistencies
* Update frontend Claude guidelines for typecheck/linting to use bun
* GDGT-1755: Address PR comments
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
These tests were temporarily disabled on 2025-09-26 due to flakes.
Re-enabling them now to restore full test coverage.
Lets monitor and see what happens
* wip: transform inspector BE
* WIP: inspect tables
* wip
* format join info as code
* wip: join info
* style
* add join stats type and display match rate
* chore: row
* chore: remove limit
* wip
* handle differerent cases for match rate and row count
* add joins table display and icons
* fix copy
* fix missing join table names
* cleanup
* cleanup
* extend match
* fix missing source table type
* add join tooltip
* chore: with join info
* wip
* wip
* add field statistics dummy component
* small ui adjustments
* small ui adjustments
* add filled rows
* order input columns like joins
* update visualization card styling
* wip: update join calc to be iterative
* add target and source stats
* add target and source stats, a filter by output fields and a tab for unmapped fields
* add target and source stats, a filter by output fields and a tab for unmapped fields
* chore: sort by order
* fix: inverse map
* add cross product warning
* add null join warning
* small QoL UI improvements
* wip: render output_row_count
* wip: native joins
* fix a bug with conditional rendering
* wip: try to pull out useful fields
* WIP: claude'd UI selection of visited_fields
* use default automatic display selection logic
* add transform mock page with mock summary section
* add mock api console log
* fix fetch wrong query metadata
* add mock lenses
* add mock lenses api
* REVERTME: ignore for deploy
* feat: schemas
* require
* feat: degeneracy check
* WIP cljc api
* wip context
* wip lenses
* update comments
* chore: visited-fields
* wip
* wip: base
* wip: triggered lens
* wip: lib
* distinguish drill lenses
* fix: return visited-fields
* wip: v2 api
* WIP: vibe'd FE
* tabs/remove mock
* account for params
* params
* ui improvements
* fix
* show fingerprinting
* fingerprint layout
* use BE fallback
* handle more complex joins
* fallback to interesting fields
* fix trigger via cljc
* remove debug
* different text
* compute-card-results in cljs
* condition trigger in cljc
* semi working
* fix card_id correlation
* remove dead code
* rename BE to just inspector
* switch to new
* ensure transform is deleted
* chore: cleanup schema
* bring back the field fingerprint info
* move cljs out of lib
* fix: schema
* minor cleanup
* appease kondo
* kondo
* remove dead code
* feat: centralize native analysis in query_analisis
* chore: remove #p
* clean up
* Gdgt 1663 move fingerprints out of the lens to data inspect (#68996)
* [transforms inspector] depend on trunk macaw (#69022)
depend on trunk macaw
* fix lint and type issues (#69017)
* remove unused lib function
* fix infinite rendering loop
* fix type lib.interestingFields
* fix
* fix: ui bug
* chore: disable visited fields for now
* chore: better unmatched-rows
* chore: exclude not nulls
* minor tweaks
* minor tweaks
* rename
* chore: tighter contract
* minor optimisation
* Gdgt 1669 make cards clickable (#69116)
* Add card link
* link to hash of question
* add clickable title
* remove tooltip
* Use MBQL info to match columns whe npossible (#69120)
use mbql info for column matching
* Gdgt 1675 disable unmatched rows view details button in inspector (#69152)
* remove row index column
* adapt viz UI, hide row index column
* Unmatched rows lens: return cards for both LHS or RHS null scenarios (#69123)
* feat: handle both cases of no join match
* feat: simple degeneracy
* export function
* remove dead fields
* only propose drill lenses that are applicable (#69169)
* Transforms inspector consistent casing for FE (#69174)
* chore: consistency
* chore: some docstrings updats
* feat: snake_case for the FE
* feat: pass visitedFields to interestingFields, remove ad-hoc logic in FE (#69175)
feat: pass visitedFields to interestingFields, remove ad-hoc logic in the FE
* feat: namespace card-ids with params (#69180)
* add estimated lens complexity (hardcoded for now) (#69181)
* add estimated lens complexity (hardcoded for now)
* refactor: use with-metadata
* add metadata to triggers, not just cards (#69182)
* appease kondo
* chore: don't depend on git sha macaw
* reinstate metabase-enterprise.transforms.core from merge conflict
* Gdgt 1722 new UI for data summary new (#69220)
* Transforms inspector BE tests (#69225)
* feat: inspector tests
* fix: impls
* fix: nil safe
* feat: integration tests
* make it work for both FE and BE
* fix card-results for non oute rjoin
* e2e test
* fixes
* fix: quoting
* fix for mysql/bigquery/redshift
* gdgt-1721 rework lenses navigation (#69205)
* Transforms inspector module (#69278)
* split transforms inspector into its own module, introduce transforms-rest
* handle kondo modules warnings
* fix: shadow_cljs
* kondo: fixes
* Transforms inspector be review (#69339)
* chore: align
* remove useless comments
* feat: add schema to context
* feat: simplify column-matching
* fix: visited-fields
* fix test
* remove unused
* tighter schema
* extrac-sources test
* fix: missing metadata-provider
* Gdgt 1744 combine separate averages columns into one for data summary (#69355)
* gdgt-1723 New UI for Joins Analysis (#69325)
* Gdgt 1725 allow running transform from empty screen (#69394)
* chore: move crud functions on transforms, avoid -rest modules dependencies
* fix tests
* appease kondo
* fix modules
* fix: small UI bug
* Transforms orphaned columns (#69497)
* add orphaned columns to unmatched-rows analysis
* chore: update tests
* Transforms inspector analytics (#69484)
* feat: flow discover/lens through transform events
* track transform_id in query execution
* feat: add transform-inspector ids/params to query info
* feat: add inspector specific /query endpoint
* feat: hook FE to new api
* feat: add lens_id/params to view
* feat: add them to ua
* fix: pass params too
* fix: use text.type
* fix test
* chore: remove unused requires
* chore: events
* fix FE part
---------
Co-authored-by: Egor Grushin <egor@metabase.com>
* fix: use lib functions instead of manual mbql construction (#69519)
* fix: use lib functions instead of manual mbql construction
* chore: refactor utils
* cleanup
* more manipulation with lib
* bugfix
* strip fields/filters early
* do it in util function
* fix: bad merge resolution
* kondo
* another bad merge resolution
* chore: indent
* Gdgt 1754 handle slow loading lens indicator (#69334)
* lens loading indicator
# Conflicts:
# frontend/src/metabase/transforms/pages/TransformInspectPage/TransformInspectPage.tsx
# frontend/src/metabase/transforms/pages/TransformInspectPage/components/LensContent/LensContent.tsx
* add onCardLoaded handler to context
* Gdgt 1724 new UI column (#69282)
* Gdgt 1783 fix wrong args passed to lens query (#69546)
* fix wrong args being passed to lens queries
* improve naming of query parameters
* remove whitespaces from merge resolution
* add transforms api test
* feat: i18n the inspector strings (#69565)
* fix dark mode color issues (#69570)
fix bg fixed to white
* Gdgt 1737 use card metadata instead of parsing card ids (#69583)
* use card metadata instead of parsing ids
* improve way of getting the table name from cards
* use macaw still, TODO
* port some query-analysis to sql-tools (#69732)
* chore: port some query-analysis to sql-tools
* remove sql-tools from transforms
* chore: vendor macaw functions, appease kondo
* refactor
* chore: refactor
* side agnostic
* revert manual delete, closes https://linear.app/metabase/issue/GDGT-1730/pre-merge-blocker-revert-t2delete-on-transform-once-uxw-3021-is-fixed
* fix: always sync audit db on change
* dead code
* fix: use honeysql instead of building sql manually (#69774)
* GDGT-1791 Maximum depth error in console (#69754)
GDGT-1791 fix and refactor useCardLoadingTracker
* Gdgt 1651 add frontend e2e tests for transform inspector UI (#69677)
* GDGT-1823 make guarantee that LensContent is always stores one lens data (#69800)
* fix: 404 on lens not found (#69838)
* fix: 404 on lens not found
* chore: don't pmap
* fix: remove nil values from statistics
* fix
* Transforms inspector self review be (#69843)
* style
* chore: reuse resolve-join-sides
* add from-table-id for mbql queries too
* split context in mbql/native, don't union
* whitespace
* fix: preprocess native queries
* fix: update tests
* fix
* revert unrelated commit that made in the branch erroneously
* Gdgt 1812 better handle missing sources in lens inspector (#69848)
* Gdgt 1817 add snowplow product events fe (#69801)
* Add analytics events
# Conflicts:
# frontend/src/metabase/transforms/analytics.ts
* add analytics tests
* simplify and improve lens loaded tracking event
* Move the idempotence check to useLensLoadedTracking
* GDGT-1831 replace unstyled button with a box to get rid of button in button error
* chore: transform lens_params
* GDGT-1786: Parametric lens navigation improvement (#69821)
* GDGT-1757: Refactor Column Comparison utils grouping and scoring logic (#69853)
* Improve types with entity ids (#69891)
* add concrete type declarations and usage for InspectorLensId, InspectorCardId, InspectorSectionId
* add better type to lens params
* cleanup of utils
* improve type for displaySettings
* use entity aliases in more places
* use existing type aliases for source and target
* add comment about InspectorTriggerCondition type
* unify prefix for inspector types
* remove spurious catch
* deduplicate types between metabase-api and lib
* Remove unused query
* fix provided tags for transform inspector
* add alerts and drill buttons test for default lens sections (#69899)
* restrict join analysis to single condition joins
* fix: bad merge reoslution
* fix types in lens test
* use isa function to detect type
* use concrete table id type alias
* use Urls.openInNewTab util
* remove empty mocks file
* use concrete table id type
* move lib for transform inspector to src/transforms/lib
* remove superflous log in e2e tests
* add proper SchemaName type
* remove superflous export
* revert move of isFetchBaseQueryError
* remove obsolete test
* use correct SchemaName type alias
* use correct VisualizationDisplay type
* remove leftover unused type
* remove superflous export
* Use CardId type alias
* discern lens also by based on params when tracking loading
* GDGT-1839 addressing the PR comments
* GDGT-1839 addressing the PR comments
* GDGT-1839 addressing the PR comments
* GDGT-1839 move getLensKey and LensHandle from api/tags directory
---------
Co-authored-by: Egor Grushin <egor@metabase.com>
Co-authored-by: Jakub Chodorowicz <jakub.chodorowicz@metabase.com>
Co-authored-by: Jakub Chodorowicz <jakub@chodorowicz.com>
* Convert `app-embed` to TypeScript
Part of the Accelerate TypeScript Migration project.
Closes DEV-1516
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Show Python upsell on self hosted
* Add e2e tests
* Fix unit test
* Simplify TransformsUpsellPage a bit more, fix tests
* Simplify logic
* Fix e2e and unit test
* Update self-hosted python upsell as per new design / Code clean up
* Fix unit test and also add a new one for useTransformsBilling
* Remove e2e tests to be moved to a new PR
* Fix cta link and bullet points icon alignment
Our frontend code that prevents sending unchanged collections missed
tenant collections being added to the payload. This fixes both the
frontend to make sure we don't send when there are no changed
collections and it updates the backend actually check if collections are
being changed rather than just checking if the collections map exists.
* fix: transforms collections set setting on import
Also turns off the transforms sync setting if there are not transform
related models in the import
* fix: transform/snippet settings on collection sync
Updates our check for 'did transforms or snippets change on remote' to
examine collections in the respective namespaces so an import with only
transform collections (or snippet collection) creates the correct
settings on the importing instance.
* fixup! better handling for custom vs builtin tags
* fixup! better conflict checking of namespaced cols
* fixup! explicit connectionb for token cache
* fixup! PR feedback
* Adding dataPicker to embedding-data-picker redux store
* Refactor: move `dataPicker` redux
Move from interactive embedding store to embedding data picker store
* Add `dataPicker` type for the SDK
* Fix storybook id value
* Accept `dataPicker` in modular embedding SDK
* Make the doc more suitable for markdown table in the docs site
* Add test for InteractiveQuestion
* Fix drillThroughQuestionProps type
* Add tests for drillThroughQuestionProps.dataPicker
* Fix SDK components can't specify different `entityTypes` or `dataPicker`
* Fix doc error
* Fix comment grammar
* Fix wrong file name
* Clarify redux slice usage with a comment
* Make context more specific to avoid misuse
* Add `question` back to SDK's `entityTypes` on the API doc
* feat(dashboard): move subscriptions to action menu
* fix unit tests
* fix e2e tests
* fix console warning about uncontrolled component
* fix more tests
* fix remaining test
* bring test back
* Validate transform queries can be preprocessed
* remove prn
* update backend error messages
* Add support for custom toggleTemplateTagEditor
* Allow passing setTemplateTagConfig to TagEditorParam
* Add support for storing parameter values when question changes
* Add template-tags tab
* Add template tag sidebar
* Allow saving optional template tags or tags with a default value
* Show error message on save
* Add new spec for transform template tags
* Avoid sharing state when navigating
* Add combination test for template tag types in transforms
* Add definitionTab helper
* Add test for multiple template tags
* Fix error message
* Fix tooltips in test
* Remove tooltip check in tests
* Use getErrorMessage to get error message in toast
* Only return message if it is not empty
* Wrap error message in form
* Use H.createTestNativeQuery helper
* Move variableTemplateTags and isVariableTemplateTag to Lib
* Use more conventional key to reset uistate
* update backend error messages again
* Revert "Wrap error message in form"
This is now handled in the api
This reverts commit 9efecaf261.
* Fix test
---------
Co-authored-by: William Zimrin <william@metabase.com>
When an artifact doesn't exist (expected for new commits), the
fetch-artifact action was using throw new Error() which causes
GitHub Actions to report "Unhandled error" - confusing and noisy.
Using core.setFailed() cleanly marks the step as failed without
the scary error message.
Fixes: DEV-1581
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Prints the mage command and args before execution so we can quickly
see what labels/flags were passed when debugging CI driver issues.
Closes: DEV-1577
Fixes QUE2-353.
`mi/normalize-result-metadata-column` called some internals to modernize
a possibly-lib, possibly-legacy column from `:result_metadata` on a
card. Pull that logic into Lib as a new exported function.
* Remove dead code for actions
* Disable HTTP action type at API level
* clarify thrown exceptions
* Remove HTTP action tests
HTTP actions are now blocked at the API level; these tests cover
functionality that's no longer accessible.
* Remove remaining HTTP action references from api_test
The previous commit missed several HTTP action usages:
- all-actions-default helper function
- list-actions-test action-1 binding
- action-parameters-test (now uses query action)
We want to support transforms with template tags, as long as those tags aren't required or have default values. However, if we do that, the backend should actually verify that we do in fact have values for all relevant template tags. This pr adds basic validation logic to transform creation and update routes -- essentially, we just run qp.preprocess/preprocess and see if we get an error.
* Fix downloading currency values in narrowSymbol format (#69779)
* Add tests for narrowSymbol currency style in downloads
---------
Co-authored-by: Will Bryant <will.bryant@gmail.com>
Problem: When a database is configured for both SQL access and Transforms, any feature with SQL access (like raw SQL questions) inherits the elevated permissions (CREATE, DROP, INSERT, etc.) that Transforms need.
Solution: Admins can _optionally_ configure write credentials alongside existing read credentials. One logical database, two connection types. Specific code paths opt in to using write credentials; by default everything uses a read-only connection.
How: This PR adds a new `write_data_details` column on `Database`. If `write_data_details` isn't configured, everything will still use `details` as before. When it is configured, it acts as a set of overrides on top of `details`, but only in code paths that require write permissions. There should be no behavioral change for existing deployments.
* Explain how DB routing and query caching work together
* Update docs/configuring-metabase/caching.md
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
It makes sense to use strings in http tests because that is what the
http-client will accept as json.
If a string target :type is supplied, we convert it to a keyword to
ensure we can dispatch to the intended method.
Fixes QUE2-354.
`lib/with-card-clean-hook` can be used to bind a callback function to
any material changes made by the `lib.convert/clean` process, which
generally indicates errors.
* fix: cancelling remote sync tasks dismisses modal
Previously we didn't close this modal until the task reached a
'cancelled' state which could take several seconds.
* fixup! PR feedback
Closes QUE2-285
The original idea was making the :strategy key required and leaving the setting of the value to the normalization, but
I decided not to go down that road. Here's why: there are multiple places on the path of constructing a join where we
can specify the join strategy but don't have to. These steps are not going through normalization and introducing
normalization would be inefficient and feel strange. Also, it would need to be done at several points, so it doesn't
seem a gain in the end. Because of this, all this PR does is naming the default strategy, so that changing it should
be possible at a single point. (Doing this would be a bad idea, so this PR is meant to be a documentation
improvement.)
* Convert Heading to TypeScript
Closes DEV-1409
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Delete heading and replace with title
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Convert `lib/schema_metadata` to TypeScript
Part of the Accelerate TypeScript Migration project.
Closes DEV-1506
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Map Snowflake big integers to :type/BigInteger (fix#63600)
* Test fixes 🔧
* Test fixes 🔧
* Test fix 🔧
* More test fixes 🔧
* Another round of test fixes 🔧
* (Hopefully) final test fix 🔧
* Code cleanup
* Revert changes that broke Clickhouse test
* Test fix 🔧
* Convert `common/hooks/use-scroll-on-mount` to TypeScript
Part of the Accelerate TypeScript Migration project.
Closes DEV-1518
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Convert `lib/encoding` to TypeScript
Part of the Accelerate TypeScript Migration project.
Closes DEV-1490
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add events for transform run tags and dependency diagnostics row selection
* Fix e2e test
* Adjust event as per review / Fix e2e tests
* Add tracking events for when Dependency Graph is opened for an entity
* Add remaining e2e tests
* Update simple_event/jsonschema/1-0-0
Update E2E test timings with averaged data
Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).
🤖 Generated automatically by update-e2e-timings workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
Fixes QUE-3044.
The QP will ignore refs in `:fields` clauses which cannot be resolved, since they're output only and not critical to the meaning of the query. This change makes the deps analysis treat these "soft" refs the same way - they are not considered to break the queries.
I added some commented tests that are now captured as part of QUE-3081 rather than part of this PR.
Fixes QUE2-349.
This removes several namespaces from the export list for `lib` which were only used by `legacy-mbql`.
- `metabase.lib.filter.desugar.jvm`
- `metabase.lib.hierarchy`
- `metabase.lib.schema.binning`
- `metabase.lib.schema.constraints`
- `metabase.lib.schema.middleware-options`
- `metabase.lib.schema.settings`
- `metabase.lib.schema.window`
* Ensure race free waiting for overlapping transforms
Previously transforms could contend for the is_active slot.
A common outcome (observed with two overlapping schedules) was that one transform would crash (due to the duplicate key violation).
This commit ensures a collision will fallback to waiting to try again rather than crashing (and alerting the user).
Introduces a unified `sql-tools` API for SQL parsing with swappable backends:
- Macaw (Java, existing) — moved from `metabase.driver.sql`, behavior preserved
- SQLGlot (Python, new) — broader dialect support via GraalVM Polyglot
Architecture: callers go through `sql-tools` which dispatches to either backend
via the `MB_SQL_TOOLS_PARSER_BACKEND` setting (default: sqlglot). The old
Macaw calls (parsed-query -> ast -> field-references) are replaced
with a `sql-tools` API (e.g. `sql-tools/referenced-fields`,
`sql-tools/validate-query`).
SQLGlot runs Python inside GraalVM Polyglot contexts managed by a reusable pool
to amortize initialization cost. The Python source and SQLGlot library are
bundled into the uberjar via a dedicated build step.
* record group membership removals in activity log
Previously the `PUT /api/membership/:group-id/clear` and
`DELETE /api/membership/:id` endpoints did not emit group membership
deletion events.
In the long run it would be better if `t2` were the interface for
PermissionsGroupMembership, so users didn't need to call these "blessed"
functions. But for now, let's go along with the existing pattern here,
have some blessed functions for doing things, and throw exceptions if
someone does a direct `t2/delete!` on a `:model/PermissionsGroupMembership`.
* use macro for more obvious workaround
The ci-test-config.json uses driver directory names (e.g., "bigquery-cloud-sdk")
but the mage code uses internal driver keywords (e.g., :bigquery). This mismatch
meant quarantined drivers like bigquery were not being recognized.
Now uses the existing driver-directory->drivers mapping to translate config names
to internal keywords, with fallback to direct keyword conversion for names that
match (like "databricks").
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add schema for test-native-query-spec
* Implement test-native-query helper
* Add js wrapper for test-native-query
* Add ts wrapper for test-native-query
* Make common parts optional
* Fix types
* Fix temporal-unit test spec
* Handle snippet tags
* Fix temporal-unit tags
* Fix card tag spec
* Simplify schema
* Add unit tests for createNativeQuery
* Add tests for test-native-query
* Add missing options
* Fix types
* Actually resolve field ref
* Code style
* Add native-query api endpoint
* Add h.createTestNativeQuery helper
* Add test for required, widget-type and default options
* Move parsing into test-spec module as it is part of the implementation
* Add example e2e test use of H.createTestNativeQuery
* Use coercer instead of decoder to validate spec too
* Fix lint
* Move adjust to update-vals
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* Match threading style
* Add test for optional template tagsfg
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* show both transforms tiers even when on trial
* Show upgrade modal in place when clicking Upgrade to Pro
* Improve comments
* Show upsell transforms trial variant when available for instance not on trial
* Fix ts
* Say 'Start trial' when trial is available
* Use correct type for order-bys
* Move parser for test-query-spec into implementation
* Be more strict in test
* Convert another e2e test to prove the parsing works
* Make args optional in test-query-spec
* Use threading instead of partial
Co-authored-by: Braden Shepherdson <braden@metabase.com>
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
* move app switcher into Dependency Graph
* adjusting unit test
* remove z-index nonsense
* moving the app switcher up to DependencyGraphPage, small change to SelectItem
* switch function for metabot fetching field values
* refactor tests
* fix lint
* fix tests - keep existing behavior of auto-creating field values if needed
* [Lib] remove deprecated/confusing `:source-alias` key from col metadata
* Metadata schema cleanup
* Test fixes 🔧
* New test fix 🔧
* Ban legacy keys in column metadata
* Fix most tests
* More test fixes 🔧
* Kondo
* Test fix 🔧
* Add TypeDoc-friendly interfaces for EAJS web component documentation
* Add TypeDoc config for EAJS web component docs generation
* Generate EAJS web component props tables via TypeDoc
* Replace manual parameter lists with auto-generated props tables
* Fixed file references in components.md and enforced types in embed.ts
* .
* .
* .
* added detail on a prop and uncommited *.md files
* added info about attributes needing to be stringified
* .
Jest v30 validates that snapshot files have the current documentation URL.
Updated the header from the old shortened URL to the new official URL.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Fix virtualized list items rendering
* Add a repro for GDGT-1776
* Fix GDGT-1776
* Remove isHidden prop from MiniPickerItem and instead just do not render items that should be hidden
* Clean up
* Fix var access
Use prepare-release-scripts action in release-log workflow
Simplify the workflow by using the shared action instead of
manually setting up bun and building release scripts.
This works because the workflow always checks out master,
so the action from master (which uses bun) is always used.
Fixes: DEV-1544
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Make ci-report work without GNU timeout
* Destroy process on timeout
* Cherry-pick b9ff360230
---------
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
The previous fix ran `cd release && bun install --yarn` which changed the
working directory for subsequent commands. The sed command then tried to
access `release/yarn.lock` from within the `release/` directory, resulting
in "sed: can't read release/yarn.lock: No such file or directory".
Split into separate steps with explicit working-directory for clarity.
The sed commands now continue on error to let Snyk report parsing issues
directly rather than failing on the workaround.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Premium feature token checks are cached per-instance in memory with a 12h TTL. During rolling deploys after a plan upgrade, different instances can disagree about which features are available:
1. Request hits instance A (old cache) → reports old features
2. Request hits instance B (freshly started, no cache) → reports new features
3. Frontend says "upgrade successful!"
4. Request hits instance A again → old features are back
This flapping means the frontend can't reliably detect when an upgrade has landed.
## Solution
### DB-hash-aware cross-instance cache
Replace the per-instance in-memory TTL cache with a two-layer caching scheme coordinated through the DB:
- A **local in-memory cache** holds the full token status (features, validity, etc.)
- A **DB table** (`premium_features_token_cache`) stores only a SHA-256 hash of the token status, keyed by a hash of the token itself. The DB never stores actual features — just enough to coordinate freshness across instances.
When a token check comes in:
- If the local cache matches the DB hash and is < 12h old: serve it immediately
- If it matches the DB hash and is < 36h old: serve the stale data and refresh asynchronously in the background
- If the DB hash doesn't match (another instance refreshed), or the cache is > 36h old, or this is a cold start: synchronous MetaStore fetch
This ensures all instances converge quickly after a plan change while keeping MetaStore traffic low. Cold-starting instances always hit the MetaStore — there's no way to bootstrap features from the DB alone, since it only stores hashes.
### Checker stack (inner to outer)
1. **`store-and-airgap-token-checker`** — hits MetaStore or validates airgap token
2. **`circuit-breaker-token-checker`** — circuit breaking + timeout
3. **`db-hash-aware-token-checker`** — the core caching layer (described above)
4. **`local-cached-token-checker`** — 5s TTL memoization to avoid hitting the DB on every request
5. **`error-catching-token-checker`** — wraps exceptions into `{:valid false}` responses
The 5s local cache means we hit the DB at most once every 5s, even during the grace period.
### Force-refresh endpoint
`POST /api/premium-features/token/refresh` clears all caches and synchronously re-checks against the MetaStore. The frontend uses this when polling for new features after a purchase (`useTokenRefreshUntil` in `UpgradeModalLoading`, `MetabotSettingUpModal`, `TransformsSettingUpModal`). A cookie-based mechanism propagates the invalidation to other instances so they clear their local caches on the next request.
Root Cause:
Bun's `bun install --yarn` command has a bug where it normalizes version
ranges inconsistently in the generated yarn.lock:
- Alias entries normalize ranges: "safer-buffer@>= 2.1.2 < 3"
(converts "< 3.0.0" to "< 3")
- Dependency specs keep original: safer-buffer ">= 2.1.2 < 3.0.0"
(preserves the ".0.0")
Snyk performs exact string matching when resolving dependencies to their
lock file entries. When it sees `safer-buffer@>= 2.1.2 < 3.0.0` as a
dependency, it looks for that exact string in the alias list, but only
finds `safer-buffer@>= 2.1.2 < 3`, causing the error:
"Dependency safer-buffer@>= 2.1.2 < 3.0.0 was not found in yarn.lock"
Solution:
Add a post-processing sed command that normalizes version ranges in the
dependency specifications to match the aliases. This transforms:
- "< 3.0.0" → "< 3"
- "< 3.0" → "< 3"
This ensures Snyk can match dependencies to their resolutions.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Sorts parameters by their uuid to ensure that we maintain an order for
parameters across exports no matter the order they are serialized to the
database in.
Luckily a bug saved us from *actually* creating revisions here (the
`before-update` hook doesn't get an actual instance in this case). But
... we shouldn't be attempting it.
Offer to install essential build tools during setup, which are required
for compiling native extensions in development dependencies.
macOS:
- Detects Xcode Command Line Tools via xcode-select -p
- Triggers native installer dialog and polls for completion
- 10-minute timeout with manual installation fallback
Linux (multi-distro support):
- Debian/Ubuntu/Mint/Pop: build-essential
- Fedora/RHEL/CentOS/Rocky/Alma: Development Tools group
- Arch/Manjaro/EndeavourOS: base-devel
- openSUSE/SLES: devel_basis pattern
- Unknown distros: manual installation guidance
The check runs as part of step 1 (prerequisites) and is idempotent -
existing installations are detected and skipped with a success message.
* Add progress indicator during Xcode CLT installation wait
Show dots every 10 seconds while polling for installation completion,
so users know the script is still running during the wait.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
This completes the removal of `@peculiar/webcrypto` from the codebase,
following the Jest setup cleanup in #69466.
Changes:
- Update `polyfills.cljc` to use Node's native `crypto.webcrypto`
- Remove `@peculiar/webcrypto` from package.json
The previous comment about needing Node v23+ was outdated:
- `crypto.webcrypto` available since Node 15
- `globalThis.crypto` available since Node 19
- Node 22 (current LTS) fully supports it
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Use native NodeJS cryppolyfill in fe unit tests
* Force jose library to use Node.js runtime in Jest tests
The jose library's browser runtime expects CryptoKey to be globally
available, which jsdom doesn't provide. This caused tests using JWT
signing (via jose) to fail with "CryptoKey is not defined" after
switching from @peculiar/webcrypto to Node's native webcrypto.
By mapping jose to its Node.js runtime via moduleNameMapper, we avoid
the CryptoKey dependency since the Node runtime uses require('crypto')
directly.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Remove old and unused props
* Support `targetCollection` in the CreateDashboardModal
* Add tests
* Handle special collection name e.g. `root`, `personal`
* Address review
* Update test names
* Report conflicts on transforms/snippets/lib
[UXW-2743: Warn before remote sync overwrites local library/transforms](https://linear.app/metabase/issue/UXW-2743/warn-before-remote-sync-overwrites-local-librarytransforms)
Co-authored-by: edpaget<ed@metabase.com>
* fixup! store conflicts in remote-sync-task table
* fixup! make conflict checking spec driven
* Restore initial implementation (implement conflict modal when pulling for the first time)
* Fix modal not opening when in data studio
* Don't set version in task when it fails with conflict / Adjust conflict options / Start e2e tests
* Add e2e tests
* Add some unit tests
* Fix remote sync settings update when creating new branch
* Isolate git config for initial import conflict test cases
* Report conflicts on transforms/snippets/lib
[UXW-2743: Warn before remote sync overwrites local library/transforms](https://linear.app/metabase/issue/UXW-2743/warn-before-remote-sync-overwrites-local-librarytransforms)
Co-authored-by: edpaget<ed@metabase.com>
* fixup! store conflicts in remote-sync-task table
* fixup! make conflict checking spec driven
* fixup! tests failures
* fixup! lints
* fixup! exclude tags from conflict
* Fix unintended changes from merge in RemoteSyncSettingsForm.tsx
* Address PR comments
---------
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Edward Paget <ed.paget@gmail.com>
* Do not allow publishing tables without query perms
Publishing a table changes query permissions. So you shouldn't be able
to publish a table that you don't have query permissions on, and then be
able to query it.
* fns for `can-publish?`, `can-publish-all-table?`
* switch to `library` ff
* Unify table visibility with can-query semantics
Table visibility checks had drifted across different APIs.
- `(can-query? table)` checked whether you had view access
and create-queries access OR the table was published. This is incorrect:
the table is queryable if you have view access AND either the table is
published OR you have create-queries.
- the collections API included tables that were not queryable.
- the search API allowed you to view tables if a) you had data
permissions on the table, AND b) the table was unpublished or published
in a collection you had access to. This is incorrect, since publishing a
table shouldn't hide a previously visible table from search results.
Search should return tables for which you have view-data permissions
PLUS the table is either published or you have create-queries
permissions.
Fundamentally this change:
- correctly defines what `(can-query? table)` means, and
- uses that new definition of `can-query` to determine table visibility
across the collection, search, and table APIs.
Note that I did not change some existing usages of
`(mi/visible-filter-clause :model/Table ...)`, which will continue to
exclude tables for which the user does not have explicit data
permissions set (but could otherwise query due to being published). That
should be a simple followup PR.
- align SQL table visibility with mi/can-query? semantics for search paths
- add enterprise `published-table-visible-clause` and wire optional
- `published-via-collection` support into `(mi/visible-filter-clause :model/Table ...)`
- update search table fallback permission check to use canonical `mi/can-query?`
- fix collection table children filtering to include `published-via-collection` queryability
- make `(mi/can-query? :model/Table)` return explicit booleans
- add/adjust EE tests for:
- published-table SQL clause behavior
- collection items visibility under view/query perms
- search visibility when user lacks query perms
- search visibility for unpublished tables when user has data perms
- stabilize appdb negation test by avoiding brittle exact count for "customer" in shared index data
* don't use `published-tables` directly
and fix kondo config
* correct feature for library
* kondo
* ensure we bind `*current-user-id*` in search
fallbacks rely on functions that use dynamic vars to check permissions.
Search is designed to use only the search context (with its
`current-user-id` and `current-user-perms`) and binds those dynamic vars
accordingly.
Previously we just needed to bind `current-user-permissions-set` but now
we should also bind `current-user-id` as well.
* fix tests
* correct feature for library
two changes:
- chris changed a file in a workspaces test file and no backend tests
ran at all:
```
=== Module Analysis ===
Changed modules: #{}
Driver module affected: false
Important file changed: false
Drivers with file changes: #{}
=== Driver Decisions ===
h2 RUN - H2/Postgres always run
athena SKIP - no relevant changes for cloud driver
bigquery SKIP - no relevant changes for cloud driver
clickhouse SKIP - driver module not affected
databricks SKIP - no relevant changes for cloud driver
druid SKIP - driver module not affected
druid-jdbc SKIP - driver module not affected
mongo SKIP - driver module not affected
mongo-ssl SKIP - driver module not affected
mongo-sharded-cluster SKIP - driver module not affected
mysql-mariadb SKIP - driver module not affected
oracle SKIP - driver module not affected
postgres RUN - H2/Postgres always run
presto-jdbc SKIP - driver module not affected
redshift SKIP - no relevant changes for cloud driver
snowflake SKIP - no relevant changes for cloud driver
sparksql SKIP - driver module not affected
sqlite SKIP - driver module not affected
sqlserver SKIP - driver module not affected
vertica SKIP - driver module not affected
```
So we want the workspace module to be marked dirty by this change. It's
currently filtered out by `remove-non-driver-test-namespaces`.
But a follow-on: workspaces run against different dbs and we therefore
want to ensure this not only triggers driver tests, but also cloud db
tests. So a few tests change to ensure this happens, consequent of
adding `enterprise/workspaces` to `modules-triggering-cloud-drivers`.
Better display for transform sync changes where we treat toggling the
`remote-sync-transforms` as though we are toggling the
`is_remote_synced` setting on Root transforms collection. This then
creates a 'collection' typed remote sync object entry for the Root
collection and uses that to drive dirty state display (or non-display)
on the frontend.
This object also drives how items are deleted from the sync so when this
'collection' is in a deleted state it removes all transforms related
content from the sync but preserves it locally.
* ease how we generate nano-ids
Fixes https://github.com/metabase/metabase/issues/68130
nano-ids are not needed to be secure, they just need to be
distinct. They hide no secrets but assign identity.
In https://github.com/metabase/metabase/pull/66413 we replaced our usage
of https://github.com/zelark/nano-id with encore. These are almost the
same except that encore uses `secure?` to be true by default. In
practice this is:
```clojure
(if secure?
(com.taoensso.encore.Ids/genNanoId (.get com.taoensso.encore.Ids/SRNG_STRONG) len)
(com.taoensso.encore.Ids/genNanoId len))
```
And that `com.taoensso.encore.Ids/SRNG_STRONG` is
```java
public static final ThreadLocal<SecureRandom> SRNG = ThreadLocal.withInitial(SecureRandom::new);
public static final ThreadLocal<SecureRandom> SRNG_STRONG = ThreadLocal.withInitial(() -> {
try { return SecureRandom.getInstanceStrong(); }
catch (java.security.NoSuchAlgorithmException e) {
throw new RuntimeException("Failed to initialize strong SecureRandom", e);
}});
```
the implmentation in the old library was
```clojure
(defonce *secure-random (delay (SecureRandom.)))
```
So passing `false` for the encore version of the nanoid restores the
previous behavior.
CALLOUT:
One thing to call out is that this did slightly change how we generate
nano-ids based on a seed. We used this behavior to backfill
nano-ids. That let us try to ensure that the same thing in multiple
systems would have the same nano-id, thus identifying them across
systems.
We are now generating different nanoids for the same inputs.
```clojure
❯ java -cp $JARS/1.45.4.jar clojure.main
Clojure 1.11.1
user=> (doto 'metabase.util require in-ns)
metabase.util
metabase.util=> (generate-nano-id "cafebabe")
"VlXdQMzunVywNw_SArkKX"
metabase.util=>
```
This is a consistent nanoid all the way up to 57:
```clojure
❯ java -cp $JARS/1.57.10.jar clojure.main
Clojure 1.12.2
user=> (doto 'metabase.util require in-ns)
metabase.util
metabase.util=> (generate-nano-id "cafebabe")
"VlXdQMzunVywNw_SArkKX"
metabase.util=>
```
And now is different in 58:
```clojure
❯ java -cp $JARS/1.58.3.jar clojure.main
Clojure 1.12.3
user=> (doto 'metabase.util require in-ns)
metabase.util
metabase.util=> (generate-nano-id "cafebabe")
"xNzFsobWPxaYpY0ucTMmz"
```
This is not a problem because we backfilled these nanoids in ~41 or so
and no longer allow anyone to save a model without one. But is worth a
callout that 58 changed this.
* fix drivers team attribution (#69323)
causes module config tests to fail
* Use bun in package.json
* Update workflows
* Update more workflows
* Kick ci
* Fix yaml
* Fix type errors
* Compat with snyk
* Fix jest resolutions
* Add another esm package
* Fix aliases and next js missing deps
* Update next js embed tests
* Try fix embed test config
* Fix
* Fix requires in e2e support
* Use bun not yarn in embed bin
* Revert config changes
* Use bun in more scripts and use prepare-release action
* Replace more yarn usages
* Remove overrides from bunfig move to package json
* Revert e2e support changes
* Add back comment
* [ci skip] comment typo
* Update bun lock
* Update to reference bun
* Update to reference bun
* Update license disclaimer
* Tidy tests
* Update to use bun in workflows
* Change aliases
* Update codemirrort selector
* Tidy license generation
* Remove .only
* Fix yarn references
* Checkout .github for release scripts
* Update lockfile
* Re-add package
* update release bunlock
* fix checkout check
---------
Co-authored-by: Ryan Laurie <iethree@gmail.com>
Fixes#67075.
`:starts-with`, `:contains` and `:ends-with`, which use SQL `LIKE`
clauses on SQL databases, were previously using the literal string from
the user directly, without escaping the `LIKE` metacharacters `%`, `_`
and backslash.
This change adds a new `sql.qp` multimethod to the driver API, with a
basic implementation that uses backslashes and works for Postgres,
MySQL/Maria and H2. SQL Server supports escaping via regex-style `[%]`
character classes.
Oracle and SQLite need an explicitly set `LIKE '...' ESCAPE '\'` clause.
I'm leaving that for a future PR since it requires refactoring how the
`LIKE` clauses are constructed more deeply.
Updates the driver changelog, though fortunately the default case should
work for most drivers. Not to be backported, because of the driver API
change.
The workflow was using `github.sha` (the merge commit SHA) when fetching
artifacts, but the uberjar workflow uploads artifacts using the PR head
commit SHA (`github.event.pull_request.head.sha`). This mismatch caused
the containerize workflow to fail with "artifact not found" errors.
Added `github.event.pull_request.head.sha` to the fallback chain in three
places: the artifact fetch commit, the GIT_COMMIT_SHA build arg, and the
Trivy scanner checkout ref.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* wip
* Revert "wip"
This reverts commit a4dbdef16c.
* cleanup
* fix bug when clicking on breadcrumbs
* clean up old type
* Update frontend/src/embedding-sdk-bundle/components/private/SdkInternalNavigation/SdkInternalNavigationProvider.tsx
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* jsdoc for virtual
---------
Co-authored-by: Phoomparin Mano <poom@metabase.com>
* Update Claude instructions
* Some DashboardSubscription type usage refactoring
* Migrate SendTestPulse to TS. Introduce DashboardSubscriptionData. Related refactoring
* Migrate ArchivePulseModal to TS
* Migrate UnsubscribePulseModal to TS
* Migrate ArchiveModal and test
* Migrate to TS: UnsubscribeModal. Related refactoring
* Migrate to TS: Notifications actions and routes
* Migrate to TS: ModalRoute refinement
* Fix eslint
* Adjust types of ModalRoutecomponents
* Fixes following bot code review
* Some more type refinement
* Address PR comments
* More fixes/refactoring
* Update route files. Convert to components
* Rename ArchiveNotificationModal to avoid confusion with collections. Fix export
* Fix more exports. Remove type casting. Add Response type guard
* Refactor routes back to getter functions
* Add early return for modals rendering. Add isObject in type-guard
* Refine Response type-guard
* Add auto-detection of changed Loki stories for stress testing
Enable the Loki stress test workflow to automatically trigger on PRs
that modify .stories.tsx files and detect which stories to test based
on git diff. Manual workflow_dispatch still works with optional filter
override.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* enable page
* update permission functions, add transformsEnabled check
* hook up button
* temp
* add unit tests for transform layout page
* adjusting unit tests for permissions graph
* adjust e2e tests
* temp
* updating checks in e2e tests
* adding missing route mocks to unit test
* now the test should work
* do not export transforms-enabled
* adjust transforms e2e test
* update selectors to be a bit more clear
* additional tests
Update E2E test timings with averaged data
Collected timing data from 7 days of test runs.
All specs with timing data were included (simple averages).
🤖 Generated automatically by update-e2e-timings workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
* Guest Embedding should use the primary database for queries
* make sure user is set to nil for guest embeds
* test
* test
* .
* .
* fixed existing clj test
* fixed existing clj test
* lint
* address comments on unit tests
* using with-database-routing-off
* kondo
* whitespace
* modules order
* addressed comments on tests
* SDK internal navigation: main implementation with context (#68900)
* squashed everything
* Trigger CI
* test ci
* fix accidentally enabling navigation by default
* fix tests
* clean up
* clean up and revert
* fix eslint rule after v9 migration
* idon'trememberanymore
* fix stale closure
* fix button position and more cleanup
* rename to virtual
* fix virtual
* more renaming
* more tests
* more fixes
* more fixes
* more reverts and cleaning up
* remove unused type
* internal navigation: change back button cover on hover (#69122)
* add support for ct to sdk back button (#69219)
add support for tc in back button
* support enable-entity-navigation in EAJS (#69185)
* tests, fixes
* do a real test for guest on a dashboard
* add "enableEntityNavigation" to types for browser
* Update frontend/src/embedding-sdk-bundle/components/private/SdkInternalNavigation/SdkInternalNavigationProvider.tsx
Co-authored-by: Sébastien <sebastien@metabase.com>
---------
Co-authored-by: Sébastien <sebastien@metabase.com>
* export back button, test that it shows up with title=false, align button (#69222)
* export back button and test it, also test that it shows up with
title={false}
* EMB-1275 map question props to question
* EMB-1278 add last remaining test
* EMB-1299: align back button with save button when drilling
* revert back button changes to keep backward compatibilty
* removed test
---------
Co-authored-by: Sébastien <sebastien@metabase.com>
* feat(eajs): support handleLink function
* fix unit tests
* rename, remove timeout which caused bugs if host app used alert() or other blocking things
* nest in pluginsConfig as is in the sdk
* put token feature guard
* Revert " put token feature guard"
This reverts commit 4f368ab41e.
* remove isWithinIframe
* Make eslint rule easier to read
* Prevent future misuse of the Metabase logo in the SDK
* Fix wrong Metabase icon on the SDK when using EE build with whitelabel feature
* fix eslint error message
* Allow internal users to have perms on tenant colls
Notably, this required allowing internal users to list tenants. I think
this is acceptable?
* CR response
* more cr response
* access
* sneak in some other edits
* Apply suggestions from code review
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
---------
Co-authored-by: Jeff Bruemmer <jeff.bruemmer@gmail.com>
* trigger cloud driver tests on certain modules
Motivating example:
https://github.com/metabase/metabase/pull/68256/changes added some
schemas to transform apis. Transforms run when drivers run. But it
didn't trigger cloud dbs. And of course, there was an issue with
redshift transform tests.
https://github.com/metabase/metabase/actions/runs/21718378450/job/62645866080
is the determine driver skips link:
```
=== Module Analysis ===
Changed modules: #{models transforms}
Driver module affected: true
Important file changed: false
Drivers with file changes: #{}
=== Driver Decisions ===
h2 RUN - H2/Postgres always run
athena SKIP - no relevant changes for cloud driver
bigquery SKIP - no relevant changes for cloud driver
clickhouse RUN - driver module affected by shared code changes
databricks SKIP - no relevant changes for cloud driver
druid RUN - driver module affected by shared code changes
druid-jdbc RUN - driver module affected by shared code changes
mongo RUN - driver module affected by shared code changes
mongo-ssl RUN - driver module affected by shared code changes
mongo-sharded-cluster RUN - driver module affected by shared code changes
mysql-mariadb RUN - driver module affected by shared code changes
oracle RUN - driver module affected by shared code changes
postgres RUN - H2/Postgres always run
presto-jdbc RUN - driver module affected by shared code changes
redshift SKIP - no relevant changes for cloud driver
snowflake SKIP - no relevant changes for cloud driver
sparksql RUN - driver module affected by shared code changes
sqlite RUN - driver module affected by shared code changes
sqlserver RUN - driver module affected by shared code changes
vertica RUN - driver module affected by shared code changes
```
Important part is `Changed modules: #{models transforms}` and `redshift
SKIP - no relevant changes for cloud driver`. This intends to change that.
* update comment
If you have an expression with an if statement inside an if statement, the outer if would get desugared to a case
statement, but the inner if would be left untouched. This makes desugaring also desugar the inner if.
Fixes QUE-3070.
This splits the vague, dual-purpose `supported-entities` into two:
- those entities which can be depended on, whose dependents get marked
stale when they're updated (eg. a table during sync)
- those entities which we can analyze, and should check for staleness
I added `:table` to the old `supported-entities` since they're a valid
*upstream* dependency that might needs its downstream dependents to be
marked as stale, and it failed the schema.
The trouble is that we don't run analysis on tables, but the same set of
"supported" types was used to check for what entities need reanalysis.
That led to a chain like this:
- Something changed, and the entity-check job is scheduled immediately.
- It runs, and checks each `supported-entities` for any entity with
a missing, `stale` or outdated analysis finding.
- That list now includes tables, and since tables are not analyzed none
of them have entries - so there's apparently lots of work to do!
- On trying to analyze tables, it fails because they're not supported,
and it silently does nothing.
- But the batch draining logic still thinks the whole batch was used, so
it reschedules.
- No real progress is getting made, since the tables cannot really be
analyzed, therefore they never get marked as done.
- Infinite loop spinning forever!
* Create a metabase_table.transform_id column instead of dynamically populating it from the dependencies
* Fixing tests
* Include transform in table dependencies
* Moved test after merge
* Moved test after merge
* transforms are now oss
* Fixed failing tests
* Review fixes
* Hide transforms from metabot if source is not readable
* test simplification
* fix test nesting
* dedupe transform-ids earlier
* Fix module interface
* Fix rebase - transforms were moved to OSS
seeing error in CI:
```
ERROR in metabase.upload.db-test/current-database-test (DbException.java:502)
Uncaught exception, not in assertion.
clojure.lang.ExceptionInfo: Table "METABASE_DATABASE" not found (this database is empty); SQL statement:
SELECT * FROM "METABASE_DATABASE" WHERE "UPLOADS_ENABLED" = TRUE [42104-214]
```
https://github.com/metabase/metabase/actions/runs/21711162702/job/62620032674
we just need to ensure that the db is initialized first
On Apple Silicon Macs, some webpack dependencies (webpack-notifier)
require Rosetta 2 to run x86_64 binaries. Without it, the webpack dev
server fails with "Error: spawn Unknown system error -86".
This adds a check during prerequisites that detects if Rosetta is
missing and offers to install it via softwareupdate --install-rosetta.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Our migrations can race with writers, this leads to situations
where we might still see values for e.g. an enum according to some prior
schema.
We saw on stats a sync occurred concurrently with the migrations and caused a record with medallion data_layer values to appear _after_ the migration.
Should follow up with a longer term fix without leaving such cruft in the code.
* Remove link from broken dashboard notification
* Remove links from alerts setup email
* Add tests
* Remove links from unsubscribe email
* Remove links from card change or archived emails
* add tests for card change email notifications
* add tests to the broken dashboard email
* Format
* Remove unused comments
* Improve test util names
* Format again
* Fix name
* Fixing tests
* Fixing tests
* Fixing tests
* Fixing tests
* Fixing tests
* Fixing tests
* Fixing test
* Added source_readable to transform APIs
* Fixing tests
* Starts implementing read-only state for job detail page
* Stubs out potential solution for preventing non-admins from putting jobs into a state where they can no longer edit them
* Troubleshooting failing test
* Shows permissions error on transform page when user doesn't have source-read permissions. Disables run button when user lacks write access.
* Troubleshooting failing test
* Makes TagSection readOnly on job page if user lacks write access on any transforms that'd be run by the job
* Removes unnecessarily duplicated code from ScheduleSection
* Troubleshooting failing test
* Troubleshooting failing test
* Troubleshooting failing test
* Troubleshooting failing test
* Cleanup
* Fix unit tests
* Updates e2e test (group-15)
* Updates e2e tests (transforms-permissions.cy.spec.ts)
* Fixed transform-job permission check
* Only require transform write access for tag selector on job page
* Tweak tag tooltip text
* Prevents deleting/editing tag when it runs a job the user doesn't have transform write access on
* Adds a couple more transforms-permissions e2e tests
* Build transform permission off table can-query
* Added :can_run to transform-tag API
* Updates FE to use can_run on TransformTag
* Code cleanup
* Updates transforms list to disable transform rows the user doesn't have read access for
* Code cleanup
* Fixing test failures
* Fixing test failures
* Fixing test failures
* Fixing test failures
* Merged from master
* Test troubleshooting
* Resolved conflicts
* fixup: Reimplements row disabling in TransformListPage
* fixup: Adds readOnly prop to new IncrementalTransformSettingsWrapper component
* fixup: Reimplements readOnly state for PythonTransformEditor
* Adds read-only state for transform owner section
* Updates recently-moved canAccessDataStudio with canAccessTransforms check
* Updates permission-management unit tests
* Adds tooltip to transform column in permissions table
* Hides tooltip in Transforms column in permissions table if selected group is admin or analysts
* Fix unit test
* Updates some e2e tests that deal with non-admins using transforms to make sure they belong to the analyst group
* Adjusting backend permissions for new logic: all analysists can see transforms they have db access to, transform-permission is for writes
* Code improvements
* Code improvements
* Fixed test
* Fixing tests
* Troubleshooting tests
* Troubleshooting tests
* Fixing tests
* Update e2e tests
* Brought back can_access_transforms
* Fixing tests
* Better python permission checking
* Start, WIP
* Moved Library back to EE
# Conflicts:
# enterprise/frontend/src/metabase-enterprise/data-studio/library/LibrarySectionLayout/PublishTableModal.tsx
# frontend/src/metabase/data-studio/data-model/components/TableSection/TableAttributesEditBulk.tsx
* Continue moving Library back to EE
* Hotfix
* Fixing permissions
* Fixing permissions
* Fixing tests
* Fixing tests
* move out transforms api
* Fixing tests
* move transforms to oss
* actually it is true
* cleanup
* Fixing tests
* Fix post-merge ts error
* Close library functionality in OSS, fix some units
* Move oss table api endpoints for data structure, fix more unit tests
* Move oss table api endpoints for data structure, fix more unit tests
* Updates transforms list page so "New" is hidden and "Python library" is disabled if user has no transform permission for any database
* Fix typecheck issues
* Move non-python transform APIs to OSS
* Fix kondo
* Fix EE BE
* Code review fixes
* Code review fixes
* Merge remote-tracking branch 'origin/master' into transforms-permissions
# Conflicts:
# .clj-kondo/config/modules/config.edn
# enterprise/backend/test/metabase_enterprise/remote_sync/transforms_test.clj
# enterprise/frontend/src/metabase-enterprise/transforms/pages/TransformListPage/TransformListPage.tsx
* Code review fixes
* Fixed merge conflict
* Fixed merge conflict
* Fix lint and TS error from merge
* Fix merge issues
* Fix be issues
* Fix some unit tests
* Fix be tests issues
* Fix be tests issues
* Code review fixes
* Allow people with transform permission to see syncable schemas on a db
* code cleanup
* Fix be tests issues
* Fixing permissions
* Fix be tests issues
* Fix prettier
* wip: oss transforms
* fix module order
* fix tests for OSS transforms
* cleanup
* cleanup
* Fix some unit tests
* Fix library for OSS
* fix some more tests
* cljfmt
* add missing placeholder file
* fix more tests
* more tests
* more tests
* deftest-oss into `metabase.test`
* Fix some unit tests
* Fix some unit tests
* Fix transforms upsell for oss
* fix search filter
* fix transforms tables
* fix cycles
* feat: transform setting into collection list (#68758)
* feat: transform setting into collection list
Move the setting for syncing transforms into the list of collections
available to remote sync, so conceptually the transforms root collection
looks like a standard syncable collection.
* Clean up FE a little bit
* undo unneeded changes
---------
Co-authored-by: Anderson <anderson@metabase.com>
* Remove Security and Appearance from embedding settings related links on OSS (#68151)
* feat: enable transform setting on first import (#68807)
* feat: enable transform setting on first import
When a we import a remote that has transforms set the
`remote-sync-transforms` setting to true.
Update the remote sync object table clean up to be driven by the spec
configuration.
Add tests showing that transforms are consistently synced and erased
according to the following rules:
1. if the instance has `remote-sync-transforms` off and local transforms
and the remote does not have transforms content they are preserved after
an import
2. if the instance has `remote-sync-transforms` off and local transforms
and the remote has transforms content, the local transforms are
completely replaced by the remote.
3. if the instance has `remote-sync-transforms` on and the remote has
transforms content any local transforms are completely replaced by the
remote.
4. if the instance has `remote-sync-transforms` on has local transforms,
but the remote has no transforms, the local transforms are removed on import.
* fixup! relocate settings update
* fixup! don't let python libraries clash on import
* fixup! transactions for mysql
* fix: y axis not shown even if it would fit (#68861)
* Deps breaking broken (#68866)
* Add rename /broken to /breaking and add new /broken
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
* Analysts can do crud on segments and measures (#68501)
* Data Analyst CRUD on measures/segments
Data analysts with unrestricted view-data permissions on a table can now
create, update, and delete segments and measures for that table
(previously superuser-only).
`can-read?` was already granted by the `manage-table-metadata` permission.
Add permission tests for `can-write?` and `can-create?` methods.
* Remove `brand-light` and `brand-lighter` (#68290)
* brand key adjustments
* loki tests
* handle the forgotten brand-dark
* Update Loki Snapshots
* removing brand-hover from dynamic-css-vars
* Adjustments to calendar picker colors
* small adjustments on the permissions pages
* sdk: remove background-hover tests from sdk as dynamic css derives are replaced with color-mix
* stupid spelling
* building shared resources
* Adjusting unit test, loki images
* rebasing adjustments
* Update Loki Snapshots
* small color fixes while I wait for loki
* re-adding highlighted class
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
* Fix empty Python Library not showing placeholder/comment (#68826)
* CI: query-processor module should trigger driver tests (#68877)
* Delete fuzzing workflow (#68880)
* fix(sdk): fix failing tests for white-labeled colors (#68790)
* Dependency diagnostics sidebar updates (#68825)
* Use getter function to move element (#68714)
* Fix cloud implicit join tests (#68762)
A previous pr broke some tests on master, and this fixes them.
* feat(sdk, embed-js): Add an alerts support to modular embeddings (#68777)
* Add an alerts button on the SDK (#67796)
* Prepare to add alerts to SDK
* Remove unrelated prop from storybook
* Add basic alerts button
* Only show alerts button with the proper flag
* Prevent download button from being shown without proper flag
* use CSS to hide things instead of JS
* Properly position all question buttons
* Use JS to hide empty divs as pure CSS is impossible
* Render alert modal ✌️
* Fix type
* Add todo
* Structure the components better
* Control alert modal state
* Add interactive question storybook
* Make interactive and static question button styles consistent
* Add alerts modal to interactive question
* Make alert button works with custom layouts
* Fix tests
* Fix unit tests
* Fix import order issue
* Remove outdated comment
* Add the same condition to display alert button from the core app
* Polish
* Modify question storybooks to show a question instead of a model
* Hide empty elements more reliably
* Add more conditions to display alert button
* Remove unused `classNames` prop
* Use the same style as the interactive question one
* Remove outdated diff
* Fix alerts button not working in custom layouts
* Remove another unused code
* Add a basic test
* Add unit tests
* fix visibility logic
* Fix E2E tests
* Make assertion on static question tests better
* Improve assertions
* Add interactive question unit tests
* Add e2e test for static question
* Add interactive question e2e tests
* Fix subcomponent
* clarify comment
* Hide other channels than email on the alert modal (#68184)
* Hide channel options on alerts modal
* Add unit tests
* Make the test name clearer
* Fix can't view created notifications the first time (#68197)
* Fix can't view created notifications
* Add tests
* Remove unnecessary return type
* Add a safety guard
* Hide Alert (and download) button on edit mode (#68189)
Hide some buttons on edit mode as it makes no sense there
* Alerts polishing (#68477)
* Misc. improvements
* Make it typesafe
* Add alerts to modular embedding (#68286)
* Add the option in modular embedding wizard
* Support `withAlerts` in modular embedding
* Close the modal after hiding the alert button
* Add E2E tests
* Fix type
* Fix `withAlerts` not working on modular embedding with `drills: false`
* It's 2026!
* 1 plugin to rule 'em all!
* Fix E2E after merging master
* Fix test name
* Add alerts analytics (#68531)
* Add alert setup analytics
* Track withAlerts on modular embedding
* Update snowplow schema
* Fix E2E tests
* Fix schema version
* Remove links from alerts test email (#68547)
* Remove links from alerts template
* Remove links from alert email computed part
* Add tests
* Make embedding util easier to use
* Make alerts and subscriptions payload lighter
* Remove unexpected pprint
* Remove links from alert email via tasks (#68610)
* Save notification card's disable_links when creating
* Add tests
* make bin/mage run-tests able to run a single test (#67450)
* Add content translations for aggregations (#68733)
* fix: typecheck
* oops, add missing file
* fix upload impl tests
* Move upsells to ee for transforms
* fix search tests
* fix mage tests
* fix audit perms
* fix `user->tenant-collection-and-descendant-ids`
* reorganize transform permissions
* fix cyclic dep
* add forgotten test file
* fix modules tests / kondo
* another module config fix
* Fix python transforms upsell in oss, fix unit tests
* fix more tests
* one more test
* one more
* Fix e2e tests
* one more
* and a few more
* oops, syntax error
* fix more tests
* kondoooooo
* dumb test: try increasing latch timeout
* hopefully actually fix h2 oss test
* Fix flaky e2e tests
* Fix lint
* Trigger rebuild
* Fix CI
* Fix CI
* Fix failing e2e test
* fix check for is-data-analyst
* Gate transforms based on `:hosting` flag
* move noop events
* Remove unnecessary `check-permissions-for-model`
* use transform feature gating in `can-read?`
* cr resp: moving tests around
* add `enabled-source-types-for-user` and use it
* remove last `deftest-oss` usage
also clean up kondo config:
- no longer define custom expansions for old macros that no longer exist
- remove `deftest-oss`
* change owner to UX west
* use `is-data-analyst?` and `check-data-analyst`
... in a few more places, instead of manual `(or api/*is-superuser?* api/*is-data-analyst?*)`
* Fixes for BE after merge
* Integrate Workspaces code
* Fixes for BE
* Fixes for BE
* Fixes
* remove unnecessary `when-ee-evailable`
also that typo is driving me up the wall and I'm going to need to fix
it :grimace:
* transforms doesn't use config
* missing import
* fix tests
* kondoooo
* last be test failure?
* fix feature gating for workspaces transforms API
why do we have a separate workspaces transforms API
* kondo
* two more tests
* tests
* couple more tests
* almost... there
* fix a flaky transform run test
this was failing locally because I already had enough transform runs to
fill up the page. simple helper to clean up the transform runs that
exist before running the test, then restore them.
* fix data-model.shared.cy.spec.ts
---------
Co-authored-by: Nathan Voxland <nathan@metabase.com>
Co-authored-by: Brad Anderson <brad@metabase.com>
Co-authored-by: Aleksandr Lesnenko <alxnddr@gmail.com>
Co-authored-by: John Swanson <john.swanson@metabase.com>
Co-authored-by: Edward Paget <edpaget@users.noreply.github.com>
Co-authored-by: Anderson <anderson@metabase.com>
Co-authored-by: kevinkelleher12 <kevin.kelleher@metabase.com>
Co-authored-by: Maksym Yakubych <undrfined@gmail.com>
Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
Co-authored-by: Nick Fitzpatrick <nick@metabase.com>
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
Co-authored-by: Anderson <ands.alves.nunes@gmail.com>
Co-authored-by: bryan <bryan.maass@gmail.com>
Co-authored-by: Fraser <fraser@metabase.com>
Co-authored-by: Phoomparin Mano <poom@metabase.com>
Co-authored-by: Romeo Van Snick <romeo@romeovansnick.be>
Co-authored-by: William <william@metabase.com>
Co-authored-by: Mahatthana (Kelvin) Nomsawadi <me@bboykelvin.dev>
Co-authored-by: Oleksandr Solovyov <alexander@solovyov.net>
Co-authored-by: Timofey Kachalov <timofei@metabase.com>
I think this permission makes more sense and is probably what we
intended. Note that users without `manage-table-metadata` but with table
permission can already read this information from the
`/api/query_metadata` endpoint.
* module test runner
support for `:module` and `:modules` syntax for our test runner
When using multiple with `:modules` need to put them in a vector, and
for shell reasons need to quote it. So there's a convenience method
where you just pass an individual:
```shell
clj -X:dev:ee:ee-dev:test :modules '[tiles enterprise/agent-api]'
clj -X:dev:ee:ee-dev:test :module enterprise/agent-api
```
REPL
===
```clojure
test-runner=> (metabase.test-runner/find-and-run-tests-repl
{:modules '[tiles enterprise/agent-api]})
Running tests with options {:mode :repl, :namespace-pattern #"^(?:(?:metabase.*)|(?:hooks\..*))", :exclude-directories [".clj-kondo/src" "classes" "dev" "enterprise/backend/src" "local" "resources" "resources-ee" "src" "target" "test_config" "test_resources"], :test-warn-time 3000, :only ("test/metabase/tiles" "enterprise/backend/test/metabase_enterprise/agent_api")}
Running tests in ("test/metabase/tiles" "enterprise/backend/test/metabase_enterprise/agent_api")
Looking for test namespaces in directory test/metabase/tiles
Looking for test namespaces in directory enterprise/backend/test/metabase_enterprise/agent_api
Finding tests took 159.0 ms.
Running 32 tests
Running before-run hooks...
LONG TEST in metabase-enterprise.agent-api.api-test/get-metric-details-test
Test took 3.447 seconds seconds to run
Ran 32 tests in 6.506 seconds
129 assertions, 0 failures, 0 errors.
{:test 32, :pass 129, :fail 0, :error 0, :type :summary, :duration 6505.825875, :single-threaded 20, :parallel 12}
Ran 12 tests in parallel, 20 single-threaded.
Finding and running tests took 6.7 s.
All tests passed.
Running after-run hooks...
Running QP tests against these drivers: #{:h2}
{:test 32,
:pass 129,
:fail 0,
:error 0,
:type :summary,
:duration 6505.825875,
:single-threaded 20,
:parallel 12}
```
CLI
===
multiple:
```shell
❯ clj -X:dev:ee:ee-dev:test :modules '[tiles enterprise/agent-api]'
Warning: environ value /Users/dan/.sdkman/candidates/java/current for key :java-home has been overwritten with /Users/dan/.sdkman/candidates/java/21.0.2-tem
Reflection warning, clojurewerkz/quartzite/triggers.clj:85:6 - call to method forJob on org.quartz.TriggerBuilder can't be resolved (argument types: unknown).
Reflection warning, clojurewerkz/quartzite/scheduler.clj:254:3 - call to method checkExists on org.quartz.Scheduler can't be resolved (argument types: unknown).
Reflection warning, macaw/walk.clj:47:5 - call to method walk on com.metabase.macaw.AstWalker can't be resolved (argument types: unknown).
Running tests with options {:mode :cli/local, :namespace-pattern #"^(?:(?:metabase.*)|(?:hooks\..*))", :exclude-directories [".clj-kondo/src" "classes" "dev" "enterprise/backend/src" "local" "resources" "resources-ee" "src" "target" "test_config" "test_resources"], :test-warn-time 3000, :exclude-tags [:metabot-v3/e2e], :only ("test/metabase/tiles" "enterprise/backend/test/metabase_enterprise/agent_api")}
Running tests in ("test/metabase/tiles" "enterprise/backend/test/metabase_enterprise/agent_api")
Looking for test namespaces in directory test/metabase/tiles
Looking for test namespaces in directory enterprise/backend/test/metabase_enterprise/agent_api
Finding tests took 11.8 s.
Running 32 tests
Running before-run hooks...
0/32 0% [ ] ETA: --:--Placeholder until other changesets
LONG TEST in metabase-enterprise.agent-api.api-test/get-metric-details-test
Test took 10.536 seconds seconds to run
32/32 100% [==================================================] ETA: 00:00
Ran 32 tests in 18.556 seconds
129 assertions, 0 failures, 0 errors.
{:test 32, :pass 129, :fail 0, :error 0, :type :summary, :duration 18555.859333, :single-threaded 20, :parallel 12}
Ran 12 tests in parallel, 20 single-threaded.
Finding and running tests took 30.4 s.
All tests passed.
Running after-run hooks...
Running QP tests against these drivers: #{:h2}
```
individual:
```
❯ clj -X:dev:ee:ee-dev:test :module enterprise/agent-api
Warning: environ value /Users/dan/.sdkman/candidates/java/current for key :java-home has been overwritten with /Users/dan/.sdkman/candidates/java/21.0.2-tem
Reflection warning, clojurewerkz/quartzite/triggers.clj:85:6 - call to method forJob on org.quartz.TriggerBuilder can't be resolved (argument types: unknown).
Reflection warning, clojurewerkz/quartzite/scheduler.clj:254:3 - call to method checkExists on org.quartz.Scheduler can't be resolved (argument types: unknown).
Reflection warning, macaw/walk.clj:47:5 - call to method walk on com.metabase.macaw.AstWalker can't be resolved (argument types: unknown).
Running tests with options {:mode :cli/local, :namespace-pattern #"^(?:(?:metabase.*)|(?:hooks\..*))", :exclude-directories [".clj-kondo/src" "classes" "dev" "enterprise/backend/src" "local" "resources" "resources-ee" "src" "target" "test_config" "test_resources"], :test-warn-time 3000, :exclude-tags [:metabot-v3/e2e], :module enterprise/agent-api, :only ("enterprise/backend/test/metabase_enterprise/agent_api")}
Running tests in ("enterprise/backend/test/metabase_enterprise/agent_api")
Looking for test namespaces in directory enterprise/backend/test/metabase_enterprise/agent_api
Finding tests took 12.6 s.
Running 19 tests
Running before-run hooks...
0/19 0% [ ] ETA: --:--Placeholder until other changesets
LONG TEST in metabase-enterprise.agent-api.api-test/get-metric-details-test
Test took 11.252 seconds seconds to run
19/19 100% [==================================================] ETA: 00:00
Ran 19 tests in 19.122 seconds
105 assertions, 0 failures, 0 errors.
{:test 19, :pass 105, :fail 0, :error 0, :type :summary, :duration 19122.404791, :single-threaded 19}
Ran 0 tests in parallel, 19 single-threaded.
Finding and running tests took 31.8 s.
All tests passed.
Running after-run hooks...
Running QP tests against these drivers: #{:h2}
```
* very opinionated
* add onboarding accordion
* add storybook for onboarding accordion
* fix the hover color of accordion
* chore: remove unused scrollListItemIntoView
* cleanup accordion
* add onboarding stepper instead
* retain checked icon in selected steps
* remove active step
* add unit tests for accordion
* rename step label to step number
* remove stepper root
* remove explicit ref
* refactor: simplify styles
* simplify storybook
* auto advance stepper
* support locked steps
* auto-infer step number
* remove forwardRef
* do not provide defaults for completed and locked steps
* add horizontal padding to match 64px in design
* style fixes to match design
* address reviews on scroll refs
* fix inconsistent scroll state
Athena's JDBC driver returns type names with the same capitalization as
the user types into the admin interface, but the
`sql-jdbc.sync/database-type->base-type` implementation for Athena only
recognized lower-case names (which are the defaults, I gather).
I think most databases don't have this problem, since they ignore the
case you typed in e.g. `CREATE TABLE` statements and have the JDBC
driver always return the canonical spelling of the type names.
Fixes QUE-3000.
When either the table metadata or field metadata is edited, this runs
the analysis. Since only some metadata changes can impact the analysis,
this is over-sensitive and runs sometimes when it doesn't need to.
Update llms.txt and llms-full.txt (master)
Regenerated llms.txt to include all markdown files from the docs/ directory.
Also generated llms-full.txt files for the embedding section.
🤖 Generated automatically by llms-txt-generate workflow
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <actions@github.com>
Change medallion values to hidden/internal/published. Hidden inherits visbility_type = hidden behaviour, internal/final are label only for now.
---------
Co-authored-by: Dan Stone <wotbrew@gmail.com>
Jest's haste map uses filenames (without extensions) as identifiers for
manual mocks. Both ExplicitSize and NativeQueryEditor had `__mocks__/index.ts`
barrel files, causing a collision on the name "index".
Fix by removing the barrel files and using explicit require() calls in
register-visualizations.js to point directly to the uniquely-named mock
files (ExplicitSize.tsx and NativeQueryEditor.jsx).
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
The fix was changing the mock from returning a function directly (which
would be a default export) to returning an object with a Link property
(named export), matching how the actual module exports the component.
* fix kerberos tests
in CI we're running:
```
[2/3] RUN ( curl -fsSL https://deb.nodesource.com/setup_22.x | bash ) && export DEBIAN_FRONTEND=noninteractive && apt-get update && apt-get -y install --no-install-recommends nodejs yarn rlwrap fontconfig
...
3.579 Hit:1 http://deb.debian.org/debian trixie InRelease
(#6 3.579 Hit:2 http://deb.debian.org/debian trixie-updates InRelease
(#6 3.588 Hit:3 http://deb.debian.org/debian-security trixie-security InRelease
(#6 3.651 Hit:4 https://deb.nodesource.com/node_22.x nodistro InRelease
(#6 3.661 Reading package lists...
(#6 4.185 Reading package lists...
(#6 4.691 Building dependency tree...
(#6 4.834 Reading state information...
Dockerfile:5
--------------------
4 |
5 | >>> RUN ( curl -fsSL https://deb.nodesource.com/setup_22.x | bash ) \
6 | >>> && export DEBIAN_FRONTEND=noninteractive \
7 | >>> && apt-get update && apt-get -y install --no-install-recommends nodejs yarn rlwrap fontconfig
8 |
--------------------
failed to solve: process "/bin/sh -c ( curl -fsSL https://deb.nodesource.com/setup_22.x | bash ) && export DEBIAN_FRONTEND=noninteractive && apt-get update && apt-get -y install --no-install-recommends nodejs yarn rlwrap fontconfig" did not complete successfully: exit code: 100
(#6 4.850 E: Unable to locate package yarn
(#6 ERROR: process "/bin/sh -c ( curl -fsSL https://deb.nodesource.com/setup_22.x | bash ) && export DEBIAN_FRONTEND=noninteractive && apt-get update && apt-get -y install --no-install-recommends nodejs yarn rlwrap fontconfig" did not complete successfully: exit code: 100
------
> [2/3] RUN ( curl -fsSL https://deb.nodesource.com/setup_22.x | bash ) && export DEBIAN_FRONTEND=noninteractive && apt-get update && apt-get -y install --no-install-recommends nodejs yarn rlwrap fontconfig:
3.517 2026-02-03 18:01:51 - To install N|solid Runtime, run: apt install nsolid -y
3.517
3.579 Hit:1 http://deb.debian.org/debian trixie InRelease
3.579 Hit:2 http://deb.debian.org/debian trixie-updates InRelease
3.588 Hit:3 http://deb.debian.org/debian-security trixie-security InRelease
3.651 Hit:4 https://deb.nodesource.com/node_22.x nodistro InRelease
4.850 E: Unable to locate package yarn
```
and it seems like debian doesn't host yarn (anymore?) which is
understandable. So we just do:
```
&& npm install --global yarn
```
And we should be good.
LLM diagnosed this. The setup here is a bit strange to me but it cats
it's resulting dockerfile:
```yaml
- name: Print compose config
run: cat mba-compose.yml
```
which was helpful. That dockerfile references:
```yaml
metabase:
build:
context: /home/runner/work/metabase/metabase/.devcontainer/
dockerfile: Dockerfile
```
And it's that devcontainer we are fixing here
* trigger kerberos tests
it needs files to be changed:
```yaml
run-presto-kerberos-test:
needs: files-changed
if: needs.files-changed.outputs.backend_presto_kerberos == 'true'
```
which is configured in `.github/file-paths.yaml`
```yaml
backend_presto_kerberos:
- "**/presto_jdbc/**"
- "**/presto_jdbc.clj"
- ".github/workflows/presto-kerberos-integration-test.yml"
```
so i add a `^:synchronized` teset to a file that should trigger this
files changed check and we get CI
* fix how we detect presto changes
usually hyphens are underscores, so it was looking for
`"**/presto_jdbc/**"` but the driver module is not a classpath root and
it uses a hyphen. Also it excluded the test file. so just manually add
that in, although it would be hit by the hyphen rule that i added
* correct typo in test filename
* touch test file
* Update oss sqlgen snowplow tests to use snowplow-test/with-fake-snowplow-collector
To follow existing snowplow test patterns from other modules.
* Move chat-completion-returns-usage-test next to other chat-completion tests
* Rename estimate-sanity-test to estimate-simplified-models-test
And use "raw" model names by default
* Update model-pricing with all models returned by list-models
Also add a comment with code for updating the model-pricing table.
* Ensure estimated_costs_usd is both number? and pos?
to ensure you still get a readable diff when the predicate fails rather than a hard to debug stacktrace.
* Use mt/with-random-premium-token! to get valid test token
* feat: send current user email on cloud reqs
* feat: upgrade modal
* refactor: simplify upsell banner integration & tests
* test: metabase-enterprise.cloud-proxy.api-test
* fix: error stats for upgrade modal
* etc: whitespace
* fix: upgrade modal should only show up for upgrade url
* feat: refresh page after upsell flow for module reload
* refactor: use global modals instead of localized jsx
* refactor: remove unnecessary mock data and convert to localized strings
* test: add hm.client interceptor tests
* constants to types
---------
Co-authored-by: Gabriel Moreno <gabriel@metabase.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>
* Misc agent API improvements + llms-agent-api.txt
* add llms-agent-api.txt
* Add more specific query execution response schema, fix some tests, and update llms.txt again
* Simplify count aggregation schema
* address comments, some other tweaks
* rename & move agent API llms.txt
* bump ci
* bump ci
* small change: ensure we record query execution properly with :agent context
* fix test
BigQuery can't do case statements with different types in each branch. Other dbs can. This fixes a test that ran a case statement with different types to skip BigQuery, but it adds a new assertion to check correct behavior without it.
This also corrects a fault in the SnowFlake driver that incorrectly assumed that convertTimezone() returned a timestamp_ntz but it returns a timestamp_tz. This wraps it in a cast back to timestamp_ntz.
* Add `autoRefreshInterval` to the SDK (#68729)
* Fix new callback don't get registered with our own `useInterval`
* wip
* Handle negative number
* Add autoRefreshInterval to StaticDashboard storybook
* Add autoRefreshInvertal to EditableDashboard storybook
* Add tests
* Make test order more sensible
* Add more test coverage
* Update types
* Update test format for consistency
* Make the condition clearer
* Fix wrong user-facing comment
* Fix wrong test name
* use the same jest fake timer configuration for all tests
* Address review
* Add support for `auto-refresh-interval` in modular embedding (#68830)
* Support `auto-refresh-interval` in modular embedding
* Make the test names more sensible
* Add tests
* Add an auto-refresh indicator (#68847)
* Add a dummy refresh indicator
* Add the actual icon
* Pass tests
* Add tests
* Fix another test
Move reindex connection management to use a connection only around the writes, not try to use a connection across all writes, which may commit and therefore close server-side cursors still being used to read the next batch.
* Add tests for source-model-card
source-model-card is only called from one function, card-returned-columns.
So we have to test source-model-card (which is private) through card-returned-columns.
This kills all mutants but 3, which have to do with the self-reference check in source-model-card. Is it really
necessary? It is an impossible branch to reach.
* new pickers
implement all pickers
remove test code
remove dead code
type fixes
reorganize
type fixes
data picker plus plus
more fixes
type fixes
fix column test ids
type and testfixes
some e2e test updates and kick off ci again
improved default states
more e2e test updates
test updates
better loading state
lint fixes
better root namespace handling
fix more root collection nonsense
improve backparsing logic
fix closing multiple modals on esc or enter
more e2e test updates
e2e test updates 4
fix (some) backparsing and add tests
fix spacing
e2e test updates 5
more backparse fixes
weird snippet deal
more tenant shenanigans
sweet sweet unit tests
add back NewDashboardDialog
more e2e tests and start NewDashboardDialog unit test
lint fixes
fix infinite render loop 😅
various fixes
bug fixes
search improvements
better cache hitting
dialog unit tests
test fixes
mucho test fixes
* fix conflict resolution errors
* test and type fixes
* bug and test fixes
* New folder button for snippets and transforms, e2e tests
* fix some dashboard cases
* test fixes
* entity picker e2e tests
* much fixo
* more e2e fixes
* fix some flakes
* prettier
* fixes
* more e2e test updates
* fix sdk component tests
* some stabilization
* fix dashboard saving bug
* prettier
* address review comments
* test fixes
* fix sandbox tests
* obey the linter
* cant pass null namespace
* oh types
* timeline
* fix default expanded 😫
---------
Co-authored-by: Kevin Kelleher <kevin.kelleher@metabase.com>
* Experiment with communicating with LLMs in Clojure
* Libor comments
* switch to new openai /v1/responses api, check for structured output support
* support for claude to show how different adapters are
* linter fixes
* update llm config edn
* init chat completion endpoint and env vars
* basic sqlgen BE endpoint
* BE test script
* basic table schema context extraction
* temp hack EE inline entrypoint to use OSS sql gen
* BE OSS async & streaming support
* BE OSS async & streaming support
* structured json response
* dialect instructions and sys prompt mustache
* hacky disable file logging via env var
* improved dialect mapping
* improved table metadata
* fix merge conflict
* abstract out sql generation to a seperate hook
* move MetabotPromptInput and MetabotInlineSQLPrompt components into OSS land
* always register metabot context rather than doing it conditionally in enterprise
* move OSS implementation into a seperate file
* oss implementation first pass
* add setting to know if the feature is enabled or not, optionally show ui based on if it is enabled, consolidate state into a single hook call to fix OSS issues, gets everything working
* clean up
* add test coverage
* revert metabot context changes, they did not end up being needed
* restrict @mentions to tables in OSS and models, tables, questions in EE
* send current sql value and no longer send buffer id
* add anthropic provider
* dumb module resolution workaround to get unit tests working correctly
* if there is only one search model provided preselect it
* tweak sys prompt; add table descriptions
* improved table & field metadata
* tweak model env var
* add openai settings
* fix CSP tiptap nonce console error
* fix oss not having access to default import of use-metabot-sql-suggestion hook in prod builds
* add backend support for source_sql
* OSS SQL Generation v1.6 (#68584)
* update layout
* impl table input bar
* persist state on open / close of inline sql prompt input
* ui improvements, referenced_entities now in req/res body
* support referenced_entities param in /generate-sql; add extract-tables endpoint
* use search, fix some ux/ui issue with dropdown
* fix sizing issue
* simplify generate-sql response format; properly union the referenced entities
* return table model spec in response
* updated backend tests
* improve table selection logic using new endpoint changes w/ richer data
* rate limiting; throttling; http timeouts; max resp tokens
---------
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
* small spacing / sizing corrections
* fix ee extracting tables when it should not, add test cases
* handle disabeld state and enter press better
* cleanup
* fix modules
* add init; fix kondo
* test fix
* fix modules
* Add snowplow events to oss sqlgen endpoints (#68822)
* Copy token_usage snowplow schema from ai-service
* Add :snowplow/token_usage to metabase.analytics.snowplow/schema->version
* Add basic anthropic cost estimation based on static tables
* Return :usage and :duration-ms from llm.anthropic/chat-completion
* Add snowplow token_usage tracking to oss /generate-sql endpoint
* Add snowplow token_usage tracking to oss /generate-sql-streaming endpoint
* Add snowplow simple_event tracking for sqlgen success and failure events
* Consolidate snowplow tracking tests
* Appease whitespace linter
* Don't use non-api analytics.setting namespace
* Ensure duration_ms is an integer in snowplow events
* Add note about oss instances to hashed_metabase_license_token schema description
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
* Fix snowplow lint by adding old versions of token_usage schemas
The snowplow linter was complaining that these older schemas were missing. These versions had been deleted from the
ai-service repo, so weren't copied over. Restore them from the last version in ai-service prior to being deleted.
---------
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
* Add README reminder to token_usage snowplow schema
See also https://github.com/metabase/ai-service/pull/605
* cleanup streaming
* rm stream test
* pr feedback & test fixes
* test fix
* [Metabot] OSS SQL generation admin UI changes (#68824)
* wip
* load models async, move model input to a select, show error if deprecated model
* move to password input for api key
* handle case where env vars have been set via env vars and can not be changed
* clean up oss view
* remove token checks for deciding to upsell metabot
* be tests for new models endpoint
* fix bug
* fix broken api tests
* ui feedback, token validation, style fixes
* unit tests
* fix cypress
* fix e2e test tag
* type fixes
* condo / eastwood fixes
* fix unit tests
* fix e2e test
* fix stale css var reference
* pr feedback
* fix var
* fix n+1 query; improve metadata fetching
* driver multimethod for dialect instruction resource; review PR cleanups & fixes
* pr feedback
* fix added in driver multimethod
* whitespace lint
* address my own comments
---------
Co-authored-by: Oleksandr Solovyov <o@solovyov.net>
Co-authored-by: Sloan Sparger <sloansparger@gmail.com>
Co-authored-by: appleby <86076+appleby@users.noreply.github.com>
Co-authored-by: Thomas Schmidt <thomas@metabase.com>
Co-authored-by: Sloan Sparger <sloansparger@users.noreply.github.com>
Co-authored-by: Noah Moss <noahbmoss@gmail.com>
* replace moment with dayjs in cljs
* Fix CLJS compilation with DayJS port of metabase.util.time
* fix some tests
* fix time parsing in drill chart
* formatting
* make a better fix
* kondo
* fix sdk test
* indent
* drop moment from locales
* reorder imports
* ran bin/mage cljfmt-all to indent
* Add an ignore for 1 unsorted namespace
- cljfmt makes the namespaces unordered -> kondo complains
* typo
* attampt to move logic to clojurescript
* one more thing to remove business logic from metabase-lib
* address cljs review
---------
Co-authored-by: Braden Shepherdson <braden@metabase.com>
Co-authored-by: Bryan Maass <bryan.maass@gmail.com>
* feat: add library/transforms toggles to modal
In the 'Set up git sync' modal in data studio this adds pre-selected
toggles to sync the library and transforms. This also creates the
library if you select it to sync, but it has not yet been created.
* fixup! push the library/transforms behavior into CollectionList
* fixup! better types to reduce casting
* fixup! missing mock setup
* Update enterprise/frontend/src/metabase-enterprise/remote_sync/components/RemoteSyncAdminSettings/RemoteSyncSettingsForm.tsx
Co-authored-by: Anderson <ands.alves.nunes@gmail.com>
* fixup! separate library sync component
* fixup! prettier
---------
Co-authored-by: Anderson <ands.alves.nunes@gmail.com>
* add fix and test
* dont drop bigquery tables and add test
* use atomic renames when possible
* use helper functions
* use existing wait-for-table
* use shared temp-table-name
* small change
* remove bigquery hack, make more similar to python impl
* address comments
* use random suffix and table name for temp tables
* use table hash
* update temp-table-name
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* mocked test to verify strategy selection
* fix test
* test description
* remove test
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Adds new spec-driven methods to detect writability of remote-sync
controlled methods. These use the existing sync-eligibility settings so
in the future we can straightforwardly apply these settings to
remote-sync controlled models (TODO: follow up to build them for the
other models).
Adds batched-versions of these checks to avoid n+1 queries when loading
can_write for models.
Adds tests of remote-sync checks for snippets.
* Split out more granular 400 vs 404 error codes for metabot tools
* a couple new tests
* fix test
* fix tests, cleanup
* discrete error handling in resolve-datasource
* refactor
* undo some refactor to fix tests
* Add an alerts button on the SDK (#67796)
* Prepare to add alerts to SDK
* Remove unrelated prop from storybook
* Add basic alerts button
* Only show alerts button with the proper flag
* Prevent download button from being shown without proper flag
* use CSS to hide things instead of JS
* Properly position all question buttons
* Use JS to hide empty divs as pure CSS is impossible
* Render alert modal ✌️
* Fix type
* Add todo
* Structure the components better
* Control alert modal state
* Add interactive question storybook
* Make interactive and static question button styles consistent
* Add alerts modal to interactive question
* Make alert button works with custom layouts
* Fix tests
* Fix unit tests
* Fix import order issue
* Remove outdated comment
* Add the same condition to display alert button from the core app
* Polish
* Modify question storybooks to show a question instead of a model
* Hide empty elements more reliably
* Add more conditions to display alert button
* Remove unused `classNames` prop
* Use the same style as the interactive question one
* Remove outdated diff
* Fix alerts button not working in custom layouts
* Remove another unused code
* Add a basic test
* Add unit tests
* fix visibility logic
* Fix E2E tests
* Make assertion on static question tests better
* Improve assertions
* Add interactive question unit tests
* Add e2e test for static question
* Add interactive question e2e tests
* Fix subcomponent
* clarify comment
* Hide other channels than email on the alert modal (#68184)
* Hide channel options on alerts modal
* Add unit tests
* Make the test name clearer
* Fix can't view created notifications the first time (#68197)
* Fix can't view created notifications
* Add tests
* Remove unnecessary return type
* Add a safety guard
* Hide Alert (and download) button on edit mode (#68189)
Hide some buttons on edit mode as it makes no sense there
* Alerts polishing (#68477)
* Misc. improvements
* Make it typesafe
* Add alerts to modular embedding (#68286)
* Add the option in modular embedding wizard
* Support `withAlerts` in modular embedding
* Close the modal after hiding the alert button
* Add E2E tests
* Fix type
* Fix `withAlerts` not working on modular embedding with `drills: false`
* It's 2026!
* 1 plugin to rule 'em all!
* Fix E2E after merging master
* Fix test name
* Add alerts analytics (#68531)
* Add alert setup analytics
* Track withAlerts on modular embedding
* Update snowplow schema
* Fix E2E tests
* Fix schema version
* Remove links from alerts test email (#68547)
* Remove links from alerts template
* Remove links from alert email computed part
* Add tests
* Make embedding util easier to use
* Make alerts and subscriptions payload lighter
* Remove unexpected pprint
* Remove links from alert email via tasks (#68610)
* Save notification card's disable_links when creating
* Add tests
* brand key adjustments
* loki tests
* handle the forgotten brand-dark
* Update Loki Snapshots
* removing brand-hover from dynamic-css-vars
* Adjustments to calendar picker colors
* small adjustments on the permissions pages
* sdk: remove background-hover tests from sdk as dynamic css derives are replaced with color-mix
* stupid spelling
* building shared resources
* Adjusting unit test, loki images
* rebasing adjustments
* Update Loki Snapshots
* small color fixes while I wait for loki
* re-adding highlighted class
---------
Co-authored-by: Metabase Automation <github-automation@metabase.com>
Co-authored-by: Phoomparin Mano <phoomparin@gmail.com>
* Data Analyst CRUD on measures/segments
Data analysts with unrestricted view-data permissions on a table can now
create, update, and delete segments and measures for that table
(previously superuser-only).
`can-read?` was already granted by the `manage-table-metadata` permission.
Add permission tests for `can-write?` and `can-create?` methods.
* feat: enable transform setting on first import
When a we import a remote that has transforms set the
`remote-sync-transforms` setting to true.
Update the remote sync object table clean up to be driven by the spec
configuration.
Add tests showing that transforms are consistently synced and erased
according to the following rules:
1. if the instance has `remote-sync-transforms` off and local transforms
and the remote does not have transforms content they are preserved after
an import
2. if the instance has `remote-sync-transforms` off and local transforms
and the remote has transforms content, the local transforms are
completely replaced by the remote.
3. if the instance has `remote-sync-transforms` on and the remote has
transforms content any local transforms are completely replaced by the
remote.
4. if the instance has `remote-sync-transforms` on has local transforms,
but the remote has no transforms, the local transforms are removed on import.
* fixup! relocate settings update
* fixup! don't let python libraries clash on import
* fixup! transactions for mysql
* feat: transform setting into collection list
Move the setting for syncing transforms into the list of collections
available to remote sync, so conceptually the transforms root collection
looks like a standard syncable collection.
* Clean up FE a little bit
* undo unneeded changes
---------
Co-authored-by: Anderson <anderson@metabase.com>
* temp
* hook up mode switcher in other apps
* Ci issues
* adding get help, fixing e2e tests, other adjustments
* most unit tests
* rename to profile link?
* adjusting unit tests with routing
* partial feedback
* rename profile link
* base current app off routes array
* adjustments
* add non-admin user state, fix unit tests
* PR feedback
* fixing test
* fixing e2e
* only 1 app switcher in admin navbar
* e2e adjustments
In the case statement, the default case did not TIMEZONE() to alter the timezone. The cases with TIMEZONE() returned a timestamp. The default case without TIMEZONE() returned timestamptz. Postgres reconciled the different types by converting all of them to timestamptz. So the timestamp was altered (by TIMEZONE()), but a new UTC timestamp was added (by the cast).
When the post-processor formats the rows, it checks that the metadata for the column has :converted_timezone. If it has that, it is formatted in the converted timezone, and since it's UTC, it will convert it again, a second time, which is incorrect.
The fix here is to check if the returned column is a timestamptz and strip it before formatting. We assume the database has already done the conversion. This avoids the second conversion.
* data studio upsell
* refactor: fix route guards
* fix: lint comments
* fix: data studio layout tests
* fix: remove local stub for api
* fix: moar lint errors
* fix: comments
---------
Co-authored-by: Gabriel Moreno <gabriel@metabase.com>
* fix: clean up style issues in new remote-sync code
Fixes a grab bag of small style/docstring issues I found after the
parent PRs were merged.
* fixup! lint
* fixup! keep the newsnippetpage as always dirty
* Fix migration for data studio
We usually auto-populate some other fields in permissions based on the value of object - perm_value, perm_type, and collection_id.
This obviously doesn't happen in a migration! So set them manually here.
* just roll forward
* Push the analysis of the dependents to the job
Fixes QUE-3001
The analysis findings of dependents are marked as stale (if they exists) and the analysis job picks them up with
higher priority. Also, the job is triggered right after the analysis of the dependency and doesn't stop until all
stale findings are refreshed.
* Fix boolean type in migration
* Set *warn-on-reflection* to true
* Stick to SQL change
* Fix tests
* Remove race condition
* Process review feedback
* fixes
* fix tests
---------
Co-authored-by: Alexander Polyankin <alexander.polyankin@metabase.com>
The revert operation triggers a chain: revert API -> reload card page.
Test was only waiting for the revert API, then sometimes it would manage
to click the History tab *before* the page reload. Then the page reloads
and we're not on the History tab anymore.
Waiting for the card query request to finish means that we've reloaded
the page and ready to roll.
* New endpoint implementations
* add new schemas for new endpoints
* new field values endpoints
* simple search endpoint + bastic test
* WIP - query construction endpoint
* Query construction tests, convert :output errors to real 404
* Basic query execution endpoint that just wraps qp/process-query
* Add query response schema
* Add descriptions to malli specs
* Add more tests, simplify some schemas
* Misc fixes, malli description improvements, try to fix decoding/encoding stuff but will need to revisit
* fix whitespace
* Greatly simplify schema sharing w/existing tool endpoints
* cleanup ns
* Fix some tests
* Fix up metric field values test
* add agent-client test helper, and new agent-api-user-binding-test
* simplify base64 query encoding
* small cleanups
* Address feedback
* add comment
* bump ci
* bump ci
* fix modules
* switch to agent-api feature flag
* Ensure we catch semantic search errors before RRF + add a new prometheus metric
* Revert "Ensure we catch semantic search errors before RRF + add a new prometheus metric"
This reverts commit d972fdf6df.
* Move fallback logic to semantic-search.core to not be metabot-specific, add test
* off grid detection
* woof
* fixing bad counts
* adjusting no data points check
* expose initalUiState in visualizer
* cleaning up dashcard functions
* adjust broken behavior for existing tests
* e2e tests
* PR feedback
* remove buttons to open settings and auto adjust. Adding tests for pinned cards and documents
* removing unneeded code, pr feedback, and adding loki screenshots
* fix silly thing
Add support for syncing PythonLibrary entities (Python transform libraries)
with a Git repository when remote-sync-transforms setting is enabled.
Implements serdes serialization for pythonlibraries using `path` as the
entity id.
Adds FE changes to refresh on dirty state and show changes in the AllChangesView
* Add support for versioned endpoints to deftool
* Create new API namespace + add custom jwt middleware
* add kondo module dependency
* revert the deftool stuff which isn't needed now
* fix lint issues
* fix jwt case insensitivity issue; more structured errors for auth issues
* small jwt cleanups
* Add support for session token exchange
* cleanup, lint, tests
* more test cases, cleanup
* bump ci
* Restructure session logic to use normal middleware, add test for session expiration
* fix up module config
* share more code with jwt sso module + remove some now-redundent tests
* try to fix tests by using existing SSO JWT test setup
* fix whitespace
* fix up module config
1. if the `log-artifact-suffix` is empty, just do
`test-logs-{github-job}`.
2. include a `log-artifact-suffix` argument for the driver tests
This way we don't get 409 conflict errors when uploading logs on test
failures for driver tests.
* refactor: remove avatars and "no owner" fallback from transforms list
* refactor: right-align rows column on data structure
* fix: search_check icon fill color
Say you are doing an implicit join from orders to products in order to summarize by products.category. Previously, with implicit joins, we join with the actual products table instead of a select blah from products wrapper, and so the various fields returned by the join were returned under their original name. However, the code would abbreviate excessively long column names from the joined table and break the query.
That difference was due to add-implicit-fields running before add-implicit-joins. Explicit joins got implicit fields added, while implicit joins didn't get those fields. This pr runs add-implicit-fields a second time after add-implicit-joins, which makes implicit and explicit joins compile to the same (correct) sql.
The test was missing an explicit database selection step which sometimes
resulted in the Save button being disabled.
Also removed the unnecessary sync_schema call — the syncable_schemas
endpoint queries Postgres directly via the driver, so no metadata
sync is needed.
* Do not delete built-in transform tags on sync
I found that some tests were failing intermittently - sometimes
TransformTags were present, and sometimes they were missing. I tracked
it down to this test, which enables and then disables remote syncing
transforms.
The issue is that if we sync transforms (... and transform tags), then
unsync them, then we delete everything that was synced - including the
built-in tags.
We should exclude these from deletion.
* Fix serialization API tests
logs:
```
2026-01-27 15:02:23,442 INFO util.queue :: Starting listener field-value-invalidate with 1 threads 🎧
2026-01-27 15:02:23,448 INFO util.queue :: Starting listener search-index-update with 1 threads 🎧
2026-01-27 15:02:23,448 INFO core.core :: Metabase Initialization COMPLETE
2026-01-27 15:02:23,453 INFO core.core :: Metabase Initialization COMPLETE in 4.8 s (JVM uptime: 57.8 s)
```
metrics:
```
❯ http get :9191/metrics | grep startup
metabase_startup_jvm_to_complete_millis 57764.0
metabase_startup_init_duration_millis 4816.0
```
Note that there two different notions of startup time: from jvm
instantiation, and from before init! was called. This maintains the two
separately so we can diagnose if class load time gets longer
independently of amount of work we do during startup itself
* feat: support remote sync for NativeQuerySnippets
When the Library collection is remote-synced, all NativeQuerySnippets
and their collections (snippets namespace) are automatically tracked
for remote sync. No separate setting is needed - syncing is tied
directly to the Library collection's is_remote_synced status.
Update the FE to use the declarative approach to refresh dirty state
when snippets are modified and update the library dirty state indicator
to turn on when any snippet or snippet collection is modified.
* fix: deletion of snippets/transforms collections
Fixes a bug where collections in the special namespaces were not cleaned
up after changing branches.
* fixup! don't double create entries
* fixup! transaction scope for test on mysql
* Show empty schemas when picking a transform target
* Use listSyncableDatabaseSchemas endpoint in data structure
* Handle empty schemas
* Add repro for metabase#68378
* Move transforms tests to new folder
* Split off transforms reproductions into separate specs
* Better logging around token checks
We used to just log we were checking but not give error responses. So it
would look like a ton of token checks for no real reason.
Now includes error responses if any, and also include logs when engaging
the token check circuit breaker, and when it is "half open", when it
lets one real token check happen every 30 seconds to see if the service
is back.
```
2026-01-23 12:26:47,175 INFO premium-features.token-check :: Checking with the MetaStore to see whether token 'f325...d01a' is valid...
2026-01-23 12:26:47,186 INFO premium-features.token-check :: Reporting Metabase stats: {:embedding-dashboard-count 0, :enabled-embedding-sdk false, :internal-users 1, :embedding-question-count 0, :query_executions_static_embed 0, :query_executions_public_link 0, :domains 1, :metabot-usage-date 2026-01-22, :enabled-embedding-static false, :enabled-embedding-simple false, :metabot-queries 0, :external-users 0, :metabot-tokens 0, :query_executions_simple_embed 0, :enabled-embedding-interactive false, :query_executions_internal 0, :metabot-users 0, :query_executions_sdk_embed 0, :query_executions_interactive_embed 0, :users 1}
2026-01-23 12:26:49,190 INFO premium-features.token-check :: Error checking token: Connect timed out †
2026-01-23 12:26:49,194 INFO premium-features.token-check :: Checking with the MetaStore to see whether token 'f325...d01a' is valid...
2026-01-23 12:26:49,201 INFO premium-features.token-check :: Reporting Metabase stats: {:embedding-dashboard-count 0, :enabled-embedding-sdk false, :internal-users 1, :embedding-question-count 0, :query_executions_static_embed 0, :query_executions_public_link 0, :domains 1, :metabot-usage-date 2026-01-22, :enabled-embedding-static false, :enabled-embedding-simple false, :metabot-queries 0, :external-users 0, :metabot-tokens 0, :query_executions_simple_embed 0, :enabled-embedding-interactive false, :query_executions_internal 0, :metabot-users 0, :query_executions_sdk_embed 0, :query_executions_interactive_embed 0, :users 1}
2026-01-23 12:26:51,205 INFO premium-features.token-check :: Engaging circuit breaker in token check ††
2026-01-23 12:26:51,206 INFO premium-features.token-check :: Error checking token: Connect timed out
2026-01-23 12:27:44,548 INFO premium-features.token-check :: In token check circuit breaker but attempting check
2026-01-23 12:27:44,551 INFO premium-features.token-check :: Checking with the MetaStore to see whether token 'f325...d01a' is valid...
2026-01-23 12:27:44,565 INFO premium-features.token-check :: Reporting Metabase stats: {:embedding-dashboard-count 0, :enabled-embedding-sdk false, :internal-users 1, :embedding-question-count 0, :query_executions_static_embed 0, :query_executions_public_link 0, :domains 1, :metabot-usage-date 2026-01-22, :enabled-embedding-static false, :enabled-embedding-simple false, :metabot-queries 0, :external-users 0, :metabot-tokens 0, :query_executions_simple_embed 0, :enabled-embedding-interactive false, :query_executions_internal 0, :metabot-users 0, :query_executions_sdk_embed 0, :query_executions_interactive_embed 0, :users 1}
2026-01-23 12:27:46,574 INFO premium-features.token-check :: Engaging circuit breaker in token check
2026-01-23 12:27:46,575 INFO premium-features.token-check :: Error checking token: Connect timed out
...
2026-01-23 12:32:50,432 INFO premium-features.token-check :: In token check circuit breaker but attempting check †††
2026-01-23 12:32:50,433 INFO premium-features.token-check :: Checking with the MetaStore to see whether token 'f325...d01a' is valid...
2026-01-23 12:32:50,444 INFO premium-features.token-check :: Reporting Metabase stats: {:embedding-dashboard-count 0, :enabled-embedding-sdk false, :internal-users 1, :embedding-question-count 0, :query_executions_static_embed 0, :query_executions_public_link 0, :domains 1, :metabot-usage-date 2026-01-22, :enabled-embedding-static false, :enabled-embedding-simple false, :metabot-queries 0, :external-users 0, :metabot-tokens 0, :query_executions_simple_embed 0, :enabled-embedding-interactive false, :query_executions_internal 0, :metabot-users 0, :query_executions_sdk_embed 0, :query_executions_interactive_embed 0, :users 1}
2026-01-23 12:32:52,450 INFO premium-features.token-check :: Engaging circuit breaker in token check
2026-01-23 12:32:52,450 INFO premium-features.token-check :: Error checking token: Connect timed out
```
The highlights are now we include:
- † `Error checkign token: Connect timed out` the error response when
appropriate
- †† `Engaging circuit breaker in token check` when we engage the
circuit breaker to no longer do real token checks
- ††† `In token check circuit breaker but attempting check` indicating
we are doing a real one from the circuit breaker to determine if the
service is restored
* Whew. Lots of stuff
add a namespace to the circuit breaker keyword:
`:token-check/circuit-breaker`. Change the keyword holding
`:token-check/app-db-not-ready` to cause. I had used reason but nothing
checked that.
this is because want to log when a token check fails, but not when it is
- failure due to circuit breaker
- failure due to appdb not ready yet.
token checks can happen anywhere and with an enormous frequency. they
fail before the appdb is setup (we can't check settigns yet) and of
course they fail during the circuit breaker opening. that's the point of
the circuit breaker. So we don't have to log these.
We _do_ want to log other failures.
We also want to do some prometheus metrics. Here's a journey:
You previously couldn't import the prometheus analytics namespace
because analytics.settings required appearance.settings. It does so
because a docstring in analytics settings used to include Metabase in
the string. I removed that so now it says "in order to help us
improve". And then things would be smooth sailing.
Except we had an issue.
```
026-01-23 20:50:03,514 INFO analytics.prometheus :: Starting prometheus metrics collector
2026-01-23 20:50:03,539 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
thread229238 we made the system!
thread229238 and now we are done with the lock!
2026-01-23 20:50:03,607 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,611 ERROR premium-features.token-check :: Error validating token: Failed to initialize Prometheus on port 9,191
2026-01-23 20:50:03,611 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,612 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,614 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,616 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,617 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,618 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,619 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,619 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
2026-01-23 20:50:03,653 INFO core.core :: Setting up prometheus metrics
```
We have locking around the system but it is starting tons of times! Lots
of debugging, swearing that this can't happen. but then i finally threw
in my favorite trick: `(clojure.repl/pst (Exception. "how did we get
here"))`. And it turns out we are doing this:
1. token check fails, so we call `prometheus/inc!`
2. inc is helpful, sees system isn't yet setup, and sets it up for us
3. Setup metric system
4. setup metrics in system
5. we hit the following code which gets initial values for labels
```clojure
(defn- initial-labelled-metric-values []
(for [metric (keys (methods known-labels))
labels (known-labels metric)]
{:metric metric
:labels labels
:value (initial-value metric labels)}))
```
6. We hit the following search code about which search we have
```clojure
(defmethod analytics/initial-value :metabase-search/engine-active
[_ {:keys [engine]}]
(if (search.engine/supported-engine? (keyword "search.engine" engine))
1 0))
```
7. This takes us to semantic search:
```clojure
(defmethod search.engine/supported-engine? :search.engine/semantic [_]
(try
(supported?)
(catch Exception e
(log/warn e "Semantic search engine not supported")
false)))
```
8. `supported?` here checks to see if you support a particular search
impelementation, which of course depends on _YOUR FEATURES_.
9. Checking features fails, so we should increment our prometheus
counter
10. the system is not yet initialized yet. Let's initialize it. (this is
also step 2)
So I have added a simple `inc-if-initialized!` function. It's not
important if the system is down. It's ok if these events are lost. The
system will be initialized in core anyways.
```
❯ http get :9191/metrics | grep metabase_token
HELP metabase_token_check_attempt_total Total number of token checks. Includes a status label.
TYPE metabase_token_check_attempt_total counter
metabase_token_check_attempt_total{status="failure",} 8.0
```
logs:
```
2026-01-23 22:58:26,668 INFO premium-features.token-check :: Checking with the MetaStore to see whether token '82bf...c3fe' is valid...
2026-01-23 22:58:28,706 INFO premium-features.token-check :: Error checking token: Connect timed out
2026-01-23 22:58:28,750 INFO core.core :: Setting up prometheus metrics
2026-01-23 22:58:28,751 INFO analytics.prometheus :: Starting prometheus metrics collector
2026-01-23 22:58:28,789 INFO premium-features.token-check :: Checking with the MetaStore to see whether token '82bf...c3fe' is valid...
2026-01-23 22:58:30,793 INFO premium-features.token-check :: Error checking token: Connect timed out
2026-01-23 22:58:30,798 INFO analytics.prometheus :: Starting prometheus metrics web-server on port 9191
...
2026-01-23 22:58:53,441 INFO premium-features.token-check :: Checking with the MetaStore to see whether token '82bf...c3fe' is valid... {mb-quartz-job-type=SyncAndAnalyzeDatabase}
...
2026-01-23 22:58:55,445 INFO premium-features.token-check :: Engaging circuit breaker in token check {mb-quartz-job-type=SyncAndAnalyzeDatabase}
2026-01-23 22:58:55,445 INFO premium-features.token-check :: Error checking token: Connect timed out {mb-quartz-job-type=SyncAndAnalyzeDatabase}
```
```
Exception how here??
metabase.analytics.prometheus/setup! (prometheus.clj:584)
metabase.analytics.prometheus/setup! (prometheus.clj:576)
metabase.analytics.prometheus/inc! (prometheus.clj:668)
metabase.analytics.prometheus/inc! (prometheus.clj:658)
metabase.analytics.prometheus/inc! (prometheus.clj:665)
metabase.analytics.prometheus/inc! (prometheus.clj:658)
metabase.premium-features.token-check/error-catching-token-checker/reify--98262 (token_check.clj:431)
metabase.premium-features.token-check/check-token (token_check.clj:488)
metabase.premium-features.token-check/check-token (token_check.clj:484)
metabase.premium-features.token-check/check-token (token_check.clj:486)
metabase.premium-features.token-check/check-token (token_check.clj:484)
metabase.premium-features.token-check/eval98325/*token-features*98348--98349 (token_check.clj:534)
metabase.premium-features.token-check/eval98325/fn--98353 (token_check.clj:529)
metabase.premium-features.token-check/has-feature? (token_check.clj:575)
metabase.premium-features.token-check/has-feature? (token_check.clj:569)
clojure.lang.Var.invoke (Var.java:386)
metabase.premium-features.defenterprise/check-feature (defenterprise.clj:41)
metabase.premium-features.defenterprise/check-feature (defenterprise.clj:35)
metabase.premium-features.defenterprise/dynamic-ee-oss-fn/fn--71956 (defenterprise.clj:53)
metabase.search.semantic.core/eval180600/fn--180601 (core.clj:58)
clojure.lang.MultiFn.invoke (MultiFn.java:229)
metabase.search.core/eval139302/fn--139304 (core.clj:77)
clojure.lang.MultiFn.invoke (MultiFn.java:234)
metabase.analytics.prometheus/initial-labelled-metric-values/iter--97802--97808/fn--97809/iter--97804--97810/fn--97811 (prometheus.clj:521)
clojure.lang.LazySeq.force (LazySeq.java:50)
clojure.lang.LazySeq.realize (LazySeq.java:89)
clojure.lang.LazySeq.seq (LazySeq.java:106)
clojure.lang.RT.seq (RT.java:555)
clojure.core/seq--5488 (core.clj:139)
clojure.core/concat/fn--5579 (core.clj:727)
clojure.lang.LazySeq.force (LazySeq.java:50)
clojure.lang.LazySeq.realize (LazySeq.java:89)
clojure.lang.LazySeq.seq (LazySeq.java:106)
clojure.lang.Cons.next (Cons.java:41)
clojure.lang.RT.next (RT.java:733)
clojure.core/next--5472 (core.clj:64)
clojure.core/next--5472 (core.clj:64)
metabase.analytics.prometheus/setup-metrics! (prometheus.clj:546)
metabase.analytics.prometheus/setup-metrics! (prometheus.clj:533)
metabase.analytics.prometheus/make-prometheus-system (prometheus.clj:60)
metabase.analytics.prometheus/make-prometheus-system (prometheus.clj:55)
metabase.analytics.prometheus/setup!/fn--97866 (prometheus.clj:591)
metabase.analytics.prometheus/setup! (prometheus.clj:590)
metabase.analytics.prometheus/setup! (prometheus.clj:576)
metabase.analytics.prometheus/inc! (prometheus.clj:668)
metabase.analytics.prometheus/inc! (prometheus.clj:658)
metabase.analytics.prometheus/inc! (prometheus.clj:665)
metabase.analytics.prometheus/inc! (prometheus.clj:658)
metabase.premium-features.token-check/error-catching-token-checker/reify--98262 (token_check.clj:431)
metabase.premium-features.token-check/check-token (token_check.clj:488)
```
* module config fixup
There's apparently an instance on cloud that is trying to remove 70k queries from the cache at once, and postgres
doesn't allow 70k params to a query. Batching the deletes in groups of 1000 should fix that.
description: Add OpenTelemetry tracing spans to Clojure code following Metabase tracing conventions. Use when instrumenting backend code with trace coverage.
---
# Add Tracing Spans to Clojure Code
This skill helps you add OpenTelemetry (OTel) tracing spans to the Metabase backend codebase using the custom `tracing/with-span` macro.
- Only require `[metabase.tracing.core :as tracing]` from outside the module. `tracing/best-effort-sanitize-sql` and all other public functions are available from this single namespace.
- Do not add new API namespaces. Add new public functions to `tracing.core` instead.
- Do not require internal namespaces (`tracing.attributes`, `tracing.settings`, `tracing.quartz`) from outside the module.
-`:uses :any` on the `core` module does NOT bypass the target module's `:api` check — internal namespaces are still enforced.
### Cyclic Dependency Avoidance
`tracing/core.clj` is required by many modules across the codebase. It **must NOT** compile-time require `tracing.settings`, as this creates transitive cyclic load dependencies (e.g., `settings/core -> tracing/settings -> tracing/core -> events/impl -> events/core`).
Instead, `tracing/core.clj` uses `requiring-resolve` for settings access:
```clojure
;; CORRECT — lazy runtime resolution, no compile-time dependency
External library namespaces (clj-otel API, SDK, exporters) are safe to require normally — they don't participate in Metabase namespace cycles.
**Important:**`requiring-resolve` must use **literal quoted symbols**. Kondo hooks validate that `required-namespaces` are all simple symbols, so dynamic construction fails:
- [ ] Span wraps a meaningful I/O boundary (not pure computation)
- [ ] Group matches the domain (check `src/metabase/tracing/core.clj` for registered groups; add a new one if none fit)
- [ ] Span name follows dot-notation convention (`"domain.subsystem.operation"`)
- [ ] Attributes use namespaced keywords (`:search/query-length`, `:db/id`)
- [ ] No sensitive data in attributes (use `best-effort-sanitize-sql` for HoneySQL, never raw SQL)
- [ ] No new tracing namespaces created (add to `tracing.core` instead)
- [ ] No `DO_NOT_ADD_NEW_FILES_HERE.txt` violations in the target directory
- [ ] Run `clj-kondo --lint <files>` to verify 0 errors, 0 warnings
- [ ] Add or update tests in the corresponding `test/` path (see Testing section below)
- [ ] Run tests: `clojure -X:dev:test :only <test-ns>`
## The `with-span` Macro
```clojure
(tracing/with-spangroupspan-nameattrs&body)
```
- **group** - A keyword selecting which trace group this span belongs to (e.g., `:tasks`, `:sync`)
- **span-name** - A string identifying the span in traces (e.g., `"search.execute"`)
- **attrs** - A map of span attributes (e.g., `{:db/id 42}`)
- **body** - The code to execute inside the span
**When disabled:** zero overhead -- single atom deref + boolean check, body runs directly.
**When enabled:** creates OTel span AND injects `trace_id`/`span_id` into Log4j2 MDC for log-to-trace correlation.
## Trace Groups
Groups are registered in `src/metabase/tracing/core.clj`. Check that file for the current list. The general rule: **match the group to the domain, not the call site.** If code runs inside a Quartz job but is logically search work, use `:search`, not `:tasks`.
To add a new group:
```clojure
;; In src/metabase/tracing/core.clj
(register-group!:my-domain"Description of what this covers")
```
Users enable groups via `MB_TRACING_GROUPS=tasks,search,sync` (comma-separated, or `"all"`).
## Naming Conventions
### Span Names
Use dot-separated hierarchical names: `"domain.subsystem.operation"`. The domain prefix should match the group name:
```
search.execute -- `:search` group
sync.fingerprint.table -- `:sync` group
task.session-cleanup.delete -- `:tasks` group
db-app.collection-items -- `:db-app` group
```
### Attributes
Use namespaced keywords. The namespace groups related attributes:
Invent new namespaced attributes as needed (e.g., `:pulse/id`, `:transform/count`). Keep values as primitives (strings, numbers, booleans) -- no maps or collections.
## Step-by-Step: Adding a Span
### 1. Check module boundaries
Look up the module for your namespace in `.clj-kondo/config/modules/config.edn`. If `tracing` is not in the module's `:uses` set, add it (keep alphabetically sorted):
```edn
my-module
{:team"MyTeam"
:uses#{analyticsconfigtracingutil}}
```
### 2. Add the require
```clojure
(ns metabase.my-module.thing
(:require
[metabase.tracing.core:astracing]
[metabase.util:asu]))
```
`best-effort-sanitize-sql` is available from `tracing.core` — no additional require needed.
### 3. Identify the I/O boundary
Only wrap code at **meaningful I/O boundaries**:
**DO trace:**
- External API calls (embedding APIs, metabot, webhooks)
- Database queries (both app DB and user DB)
- Network requests (HTTP calls to external services)
- Heavy batch processing (batch indexing, batch embedding)
- Top-level orchestration functions that coordinate multiple sub-operations
Expect: all tests pass, 0 failures, 0 errors, no reflection warnings from your files.
## Sanitizing SQL for Attributes
When including SQL in span attributes, **always** use `tracing/best-effort-sanitize-sql`. This converts HoneySQL maps to parameterized SQL strings where values become `?` placeholders -- no data leaks.
description: Evaluate Clojure code via nREPL using clj-nrepl-eval. Use this when you need to test code, check if edited files compile, verify function behavior, or interact with a running REPL session.
---
# Clojure REPL Evaluation
## When to Use This Skill
Use this skill when you need to:
- **Verify that edited Clojure files compile and load correctly**
- Test function behavior interactively
- Check the current state of the REPL
- Debug code by evaluating expressions
- Require or load namespaces for testing
- Validate that code changes work before committing
## How It Works
The `clj-nrepl-eval` command evaluates Clojure code against an nREPL server. **Session state persists between evaluations**, so you can require a namespace in one evaluation and use it in subsequent calls. Each host:port combination maintains its own session file.
## Instructions
### 0. Discover and select nREPL server
First, discover what nREPL servers are running in the current directory:
```bash
clj-nrepl-eval --discover-ports
```
This will show all nREPL servers (Clojure, Babashka, shadow-cljs, etc.) running in the current project directory.
**Then use the AskUserQuestion tool:**
- **If ports are discovered:** Prompt user to select which nREPL port to use:
- **question:** "Which nREPL port would you like to use?"
- **header:** "nREPL Port"
- **options:** Present each discovered port as an option with:
- **label:** The port number
- **description:** The server type and status (e.g., "Clojure nREPL server in current directory")
- Include up to 4 discovered ports as options
- The user can select "Other" to enter a custom port number
- **If no ports are discovered:** Prompt user how to start an nREPL server:
- **question:** "No nREPL servers found. How would you like to start one?"
- **header:** "Start nREPL"
- **options:**
- **label:** "deps.edn alias", **description:** "Find and use an nREPL alias in deps.edn"
- **label:** "Leiningen", **description:** "Start nREPL using 'lein repl'"
- The user can select "Other" for alternative methods or if they already have a server running on a specific port
IMPORTANT: IF you start a REPL do not supply a port let the nREPL start and return the port that it was started on.
### 1. Evaluate Clojure Code
> Evaluation automatically connects to the given port
Use the `-p` flag to specify the port and pass your Clojure code.
**Recommended: Pass code as a command-line argument:**
```bash
clj-nrepl-eval -p <PORT> "(+ 1 2 3)"
```
**For multiple expressions (single line):**
```bash
clj-nrepl-eval -p <PORT> "(def x 10) (+ x 20)"
```
**Alternative: Using heredoc (may require permission approval for multiline commands):**
```bash
clj-nrepl-eval -p <PORT> <<'EOF'
(def x 10)
(+ x 20)
EOF
```
**Alternative: Via stdin pipe:**
```bash
echo"(+ 1 2 3)"| clj-nrepl-eval -p <PORT>
```
### 2. Display nREPL Sessions
**Discover all nREPL servers in current directory:**
```bash
clj-nrepl-eval --discover-ports
```
Shows all running nREPL servers in the current project directory, including their type (clj/bb/basilisp) and whether they match the current working directory.
**Check previously connected sessions:**
```bash
clj-nrepl-eval --connected-ports
```
Shows only connections you have made before (appears after first evaluation on a port).
### 3. Common Patterns
**Require a namespace (always use :reload to pick up changes):**
-`-r, --reset-session` - Reset the persistent nREPL session
-`-c, --connected-ports` - List previously connected nREPL sessions
-`-d, --discover-ports` - Discover nREPL servers in current directory
-`-h, --help` - Show help message
## Important Notes
- **Prefer command-line arguments:** Pass code as quoted strings: `clj-nrepl-eval -p <PORT> "(+ 1 2 3)"` - works with existing permissions
- **Heredoc for complex code:** Use heredoc (`<<'EOF' ... EOF`) for truly multiline code, but note it may require permission approval
- **Sessions persist:** State (vars, namespaces, loaded libraries) persists across invocations until the nREPL server restarts or `--reset-session` is used
- **Automatic delimiter repair:** The tool automatically repairs missing or mismatched parentheses
- **Always use :reload:** When requiring namespaces, use `:reload` to pick up recent changes
If tokens are missing, tell the user: "EE tokens are not set. Either export them and restart Claude Code, or add `MB_EDITION=oss` to run OSS-only tests."
## Running
First check if snapshots exist:
```bash
ls e2e/snapshots/default.sql 2>/dev/null &&echo"snapshots exist"||echo"snapshots missing"
```
If snapshots exist, skip regeneration for speed:
```bash
MB_EDITION=oss CYPRESS_VIDEO=falseCYPRESS_RETRIES=0CYPRESS_GUI=falseGENERATE_SNAPSHOTS=false bun test-cypress $ARGUMENTS
```
If snapshots are missing (first run), let the runner generate them:
```bash
MB_EDITION=oss CYPRESS_VIDEO=falseCYPRESS_RETRIES=0CYPRESS_GUI=false bun test-cypress $ARGUMENTS
```
When running enterprise tests, replace `MB_EDITION=oss` with `MB_EDITION=ee` in the commands above.
If `$ARGUMENTS` is empty, ask the user which spec to run.
If the user provides a test name or fuzzy name instead of a spec path, find the spec file:
```bash
find e2e/test/scenarios -name "*FUZZY_NAME*.cy.spec.*" -type f
```
Then run with `--spec "path/to/matched.cy.spec.js"`.
To filter by test name, use the `GREP` env var (not `--env grep=`, which breaks on commas):
```bash
GREP="test name here"MB_EDITION=oss bun test-cypress --spec path/to/spec.js
```
## Stress test (flaky detection)
When the user asks to verify a test is not flaky:
```bash
MB_EDITION=oss bin/e2e-stress-test $ARGUMENTS
```
Set `E2E_STRESS_RUNS=N` for more iterations (default: 5). Stops on first failure and prints screenshot paths. Read those screenshots to analyze the failure.
## Analyzing failures
When tests fail:
1. Read the **screenshot** for each failure — paths are printed in Cypress output under `(Screenshots)`, typically at `cypress/screenshots/<spec-name>/<test-name> (failed).png`
2. Read the error message and code frame from the console output
3. Identify the root cause: is it a test bug, a product bug, or a timing issue?
description: Run mutation testing on a Clojure namespace, generate tests to kill surviving mutations, and open draft PRs with Linear issue tracking.
---
# Mutation Testing Skill
This skill runs mutation testing end-to-end: generates a coverage report, groups functions, shells out to `claude -p` to write tests, verifies mutations are killed, and creates draft PRs with Linear issues.
## Prerequisites
- A running nREPL connected to the Metabase dev environment
-`LINEAR_API_KEY` environment variable set with a valid Linear personal API key
- **Team ID**: Set once per REPL session via `(mut-test/list-teams!)` and `(mut-test/set-config! {:team-id "..."})`
- **Project ID**: Set automatically when `run!` calls `create-project-for-namespace!`
- **Base Branch**: Defaults to `"master"`. Override via `(mut-test/set-config! {:base-branch "release-x.52.x"})` or pass `{:base-branch "..."}` as the second arg to `run!`
- **Project ID**: If set (via `set-config!` or `run!` opts), `run!` reuses the existing Linear project instead of creating a new one
metabase.lib.convert/->legacy-MBQL{:message"Conversion to legacy MBQL is discouraged, please use Lib to manipulate MBQL queries"}
metabase.lib.core/->legacy-MBQL{:message"Conversion to legacy MBQL is discouraged, please use Lib to manipulate MBQL queries"}
metabase.lib.equality/find-column-indexes-for-refs{:message"Use lib.equality/closest-matches-in-metadata or lib.equality/find-closest-matches-for-refs instead of lib.equality/find-column-indexes-for-refs"}
metabase.lib.util.match/match{:message"Use lib.util.match/match-many instead of lib.util.match/match."}
metabase.lib.util.match/match-one{:message"Use lib.util.match/match-lite instead of lib.util.match/match-one."}
metabase.lib.util.match/replace{:message"Use lib.util.match/replace-lite instead of lib.util.match/replace."}
metabase-enterprise.transforms.interface/execute!{:message"Use metabase-enterprise.transforms.execute/execute! instead of metabase-enterprise.transforms.interface/execute!"}
next.jdbc/active-tx?{:message"Use metabase.app-db.core/in-transaction? to see if we are in a transaction"}
toucan2.core/debug{:message"Don't commit code using t2/debug"}
toucan2.tools.debug/debug{:message"Don't commit code using t2/debug"}
macaw.core/parsed-query{:message"Use metabase.driver.util/parsed-query instead of macaw.core/parsed-query"}}
}
:discouraged-namespace
{camel-snake-kebab.core{:message"CSK is not Turkish-safe, use the versions in metabase.util instead."}
-**UI**: `metabase/ui` components built with Mantine v8
-**Build**: Rspack (primary), Webpack (legacy)
- **Testing**: Jest + React Testing Library, Cypress
-Check for hardcoded secrets, API keys, or credentials
-Look for SQL injection and XSS vulnerabilities
-Verify proper input validation and sanitization
-Review authentication and authorization logic
-Ensure enterprise features use the plugin system (no enterprise code in OSS)
### Coding Standards
### Performance Red Flags
#### Component Preferences
- Identify N+1 database query problems
- Spot inefficient loops and algorithmic issues
- Check for memory leaks and resource cleanup
- Review caching opportunities for expensive operations
- Prefer `metabase/ui`components over `metabase/common/components`
- Use `.tsx` for components, `.ts` for utilities
### Code Quality Essentials
#### Styling
- Functions should be focused and appropriately sized
- Use clear, descriptive naming conventions
- Ensure proper error handling throughout
- Remove dead code and unused imports
ALWAYS prefer Mantine style props,then CSS modules. DO NOT suggest styled components, they are deprecated.
### Review Style
#### TypeScript Migration
- Only flag actual issues worth mentioning
- Do not post "looks good" comments or congratulate following conventions
- Be specific and actionable in feedback
- Explain the "why" behind recommendations
- Ask clarifying questions when code intent is unclear
When heavily editing `.js`/`.jsx` files, create a separate PR to convert to TypeScript first, then implement changes.
### Review Format
#### Enterprise Features
- Condense feedback into as few comments as possible
- Prefer a single summary comment over dozens of inline comments
- Only use inline comments for critical issues that require immediate attention
- For non-critical issues, link to the specific line from the summary instead
- Delete all stale comments and summaries on each new push or CI run
Enterprise functionality MUST use the plugin system. It is very important to not expose enterprise code in the OSS version.
#### Testing Requirements
All PRs should include tests. Prefer Unit tests over E2E tests.
#### Localization
All user-facing strings MUST be localized using the ttag library. Localized strings should be complete phrases, do not concatenate a few separately localized strings. You should add context to strings where the meaning of the string might not be obvious in isolation: e.g. "Home" might have different words in some languages depending on whether you're talking about a dwelling or the landing page for a website.
Language-specific rules are in `.github/instructions/*.instructions.md`.
> Watch out for these deprecated props and gotchas for Metabase 57 onwards, for modular embedding:
>
> 1. \`config\` prop on MetabaseProvider no longer exist - it is replaced by \`authConfig\`.
> 2. \`authProviderUri\` field no longer exist.
> 3. \`jwtProviderUri\` optional field only exists in v58+. This is used to make JWT auth faster by skipping the \`GET /auth/sso\` discovery request. Not needed for initial implementation.
> 4. \`fetchRequestToken\` is not needed by default. This is only used to customize how the SDK fetches the request token.
> 5. Numeric IDs must be integers not strings, e.g. \`dashboardId={1}\`. When the ID is retrieved from the URL and it is numeric, convert it to an integer via \`parseInt\` before passing it to the SDK. IDs can also be strings for entity ids, so you should not parse all IDs as numbers if entity ids are also to be expected.
`;
}
/**
* Generate llms-full.txt file for a specific section
* @param {string} branch - The branch name
* @param {Object} config - Section config object with section name and optional getPreamble function
* @param {Array} files - Array of all markdown file objects
* @param {string} baseUrl - Base URL for raw GitHub links
--allowedTools "Bash(bun run test-unit*),Bash(bun run lint-eslint-pure),Bash(./bin/mage*),Bash(clojure*),Bash(find*),Bash(ls*),Bash(grep*),Write,Read,Glob,Grep,Edit,MultiEdit"
bun run embedding-sdk:docs:generate:pure --emit none 2>&1 \
| tee /dev/stdout \
| tee >( tail -n 10 \
| sed '1s/^/output<<EOF\n/' \
@@ -98,14 +102,14 @@ jobs:
with:
remove-previous-comment:true
add-new-comment:${{ steps.docs-generation.outcome == 'failure' && contains(steps.docs-generation.outputs.output, 'not included in the documentation') }}
message:"📝 Some exported types from `@metabase/embedding-sdk-react` reference other types that are not exported, i.e. internal types.<br><br>The internal type reference breaks API documentation generation for the SDK and also may unexpectedly increase the output size of the type declaration file.<br><br>Please check the CI logs to find the name of such type.<br><br>- If a type should be publicly available in the SDK, add a description to it or a TypeDoc annotation and export it from SDK.<br>- If a type is internal and should _not_ be publicly exposed, refactor types to prevent referencing other types that are not exported from SDK.<br><br>You can run `yarn embedding-sdk:docs:generate` locally to verify that the issue has been resolved."
message:"📝 Some exported types from `@metabase/embedding-sdk-react` reference other types that are not exported, i.e. internal types.<br><br>The internal type reference breaks API documentation generation for the SDK and also may unexpectedly increase the output size of the type declaration file.<br><br>Please check the CI logs to find the name of such type.<br><br>- If a type should be publicly available in the SDK, add a description to it or a TypeDoc annotation and export it from SDK.<br>- If a type is internal and should _not_ be publicly exposed, refactor types to prevent referencing other types that are not exported from SDK.<br><br>You can run `bun run embedding-sdk:docs:generate` locally to verify that the issue has been resolved."
include-log:true
- name:Comment on PR if 'Failed to resolve link' error happens
message:"📝 A typedoc `{@link <TYPE_NAME>}` annotation is found that can't be resolved.<br><br>Most likely it happens when the referenced type is not exported. Please check the CI logs to find the name of such type.<br><br>- If a referenced in `{@link}` type should be publicly available in the SDK, add a description to it or a TypeDoc annotation and export it from SDK.<br>- If a type is internal and should _not_ be publicly exposed, change types to prevent unexported type reference in `{@link}` annotation.<br><br>You can run `yarn embedding-sdk:docs:generate` locally to verify that the issue has been resolved."
message:"📝 A typedoc `{@link <TYPE_NAME>}` annotation is found that can't be resolved.<br><br>Most likely it happens when the referenced type is not exported. Please check the CI logs to find the name of such type.<br><br>- If a referenced in `{@link}` type should be publicly available in the SDK, add a description to it or a TypeDoc annotation and export it from SDK.<br>- If a type is internal and should _not_ be publicly exposed, change types to prevent unexported type reference in `{@link}` annotation.<br><br>You can run `bun run embedding-sdk:docs:generate` locally to verify that the issue has been resolved."
run:yarn --cwd release --frozen-lockfile && npm i -g tsx
- name:Build release scripts
run:yarn --cwd release build
- name:Prepare release scripts
uses:./.github/actions/prepare-release-scripts
- name:Install tsx
run:npm i -g tsx
- name:Get version number
uses:actions/github-script@v7
with:
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.