docs(api): sync trigger workflow error contract

(cherry picked from commit f5aaa50a30)

.agents/skills/frontend-code-review/SKILL.md

@@ -27,7 +27,7 @@ Read only the packs matched by the diff:
 - Stable Dify runtime invariants in the named paths: [`references/dify-invariants.md`][dify-invariants]
 - General TypeScript or styling quality not owned above: [`references/code-quality.md`][code-quality]
 
-Read `packages/dify-ui/README.md`, `packages/dify-ui/AGENTS.md`, `web/docs/overlay.md`, or `web/docs/test.md` only when the reviewed code falls under that contract. Check current official documentation when local code and bundled references do not settle a framework, browser, or accessibility behavior.
+Read `packages/dify-ui/README.md`, `packages/dify-ui/AGENTS.md`, `packages/dify-ui/docs/overlays.md`, or `web/docs/test.md` only when the reviewed code falls under that contract. Check current official documentation when local code and bundled references do not settle a framework, browser, or accessibility behavior.
 
 ## Severity And Output
 

.agents/skills/frontend-code-review/references/dify-ui.md

@@ -1,141 +1,29 @@
-# Dify UI Rules
+# Dify UI Review Routing
 
-Use these rules whenever a review touches `packages/dify-ui/` or code consuming `@langgenius/dify-ui/*`.
+Use this reference when a review touches `packages/dify-ui/` or consumes
+`@langgenius/dify-ui/*`. It routes to owner documentation; it does not redefine package
+contracts.
 
-Before finalizing findings for those files, read the current local docs that apply:
+Read `packages/dify-ui/AGENTS.md` and the primitive implementation first, then only the matching
+owner:
 
-- `packages/dify-ui/README.md`
-- `packages/dify-ui/AGENTS.md`
-- `web/docs/overlay.md` for floating UI
-- `packages/dify-ui/src/<primitive>/index.tsx` for the primitive being changed or consumed
+| Review area                                | Canonical owner                                                     |
+| ------------------------------------------ | ------------------------------------------------------------------- |
+| Package boundary and document routing      | `packages/dify-ui/AGENTS.md`                                        |
+| Imports, exports, public types, and anatomy | `packages/dify-ui/docs/authoring.md`                                 |
+| Button or icon-only actions                | `packages/dify-ui/src/button/README.md`, `packages/dify-ui/src/icon-button/README.md` |
+| Compound inputs                            | `packages/dify-ui/src/input-group/README.md`                         |
+| Forms and field semantics                  | `packages/dify-ui/docs/forms.md`                                     |
+| Selection and typed values                 | `packages/dify-ui/docs/selection.md`                                 |
+| Portals, layers, and floating surfaces     | `packages/dify-ui/docs/overlays.md`                                  |
+| Tailwind and radius tokens                 | `packages/dify-ui/docs/styling.md`                                   |
+| Package tests and Storybook                | `packages/dify-ui/docs/testing.md`                                   |
 
-## Package Boundary
+For consumer code under `web/`, also read `web/AGENTS.md` for application-owned reuse policy and
+`packages/dify-ui/README.md` for the available public subpaths.
 
-Flag in `packages/dify-ui`:
-
-- Imports from `web/`.
-- Dependencies on Next.js, i18n, ky, Jotai, Zustand, TanStack Query, oRPC, or business APIs.
-- Business-specific component behavior that belongs in `web/`.
-- Multiple unrelated primitives in one component folder.
-
-`packages/dify-ui` is a primitive layer: Base UI headless components + `cva` + `cn` + Dify design tokens.
-
-## Imports And Exports
-
-Flag:
-
-- Consumer imports from `@langgenius/dify-ui` without a subpath.
-- Missing `package.json#exports` entry for a new primitive.
-- Internal package imports using workspace subpaths instead of relative paths.
-- Exported props using internal-only types that consumers cannot import from the component subpath.
-- Canonical primitive boundaries or their associated public types using a redundant `Root` suffix when no higher-level convenience component exists in the same subpath.
-
-Consumers use subpath exports such as `@langgenius/dify-ui/button`.
-
-Canonical boundaries use the primitive name and matching public types (`Select` / `SelectProps`). Keep `Root` only to distinguish a low-level anatomy root from a higher-level convenience component (`CheckboxRoot` / `Checkbox`); implementation aliases should still show their Base UI source (`BaseSelect.Root.Props`).
-
-## Props And State
-
-Flag:
-
-- Flattened props where related values need a discriminated union, such as `value` / `defaultValue`, `multiple` / `value`, or `clearable` / `onChange`.
-- React state used only to mirror Base UI state for class names.
-- JavaScript conditional class logic for visual states that the Dify UI/Base UI primitive already exposes through `data-*` attributes or CSS variables.
-- Controlled props added when uncontrolled DOM state or CSS variables would be enough.
-- Thin wrappers that rename Base UI parts without adding semantics.
-- Generic Base UI selection primitives wrapped without preserving their value generics, such as `Select.Root<Value, Multiple>`, `RadioGroup<Value>`, or `Radio.Root<Value>`.
-- Shared select/radio option components that type selected values as `string` while callers pass enums, unions, booleans, numbers, objects, or nullable placeholder values.
-
-Prefer Base UI/Dify UI data attributes and CSS variables for visual state: `data-open`, `data-checked`, `data-disabled`, `data-highlighted`, `data-popup-open`, `group-data-*`, `peer-data-*`, `has-[:focus-visible]`, and primitive CSS variables such as anchor width or transform origin. Use JS conditional classes for product/business state that the primitive does not expose.
-
-For non-string `Select` and `RadioGroup` values, prefer explicit domain generics at the root and at child value carriers. JSX children do not inherit the parent generic, so `RadioGroup<PromptMode>` should compose with `Radio<PromptMode>`, `RadioItem<PromptMode>`, or option values from a typed collection. For `Select`, prefer the Base UI `items` collection pattern for typed value-to-label rendering, and flag string coercion helpers used only to recover display labels.
-
-## Forms
-
-Flag:
-
-- Form-like UI using unrelated `Input` and `Button` pieces without a submit boundary.
-- Text-like fields not composed through `Field`, `FieldLabel`, and `FieldControl` when using Dify UI form semantics.
-- Select fields using `FieldLabel` instead of `SelectLabel`.
-- Slider fields using a generic label instead of `SliderLabel`.
-- Checkbox/radio groups missing `Fieldset` and `FieldsetLegend`.
-- Field errors or descriptions rendered without `FieldDescription` / `FieldError` relationships.
-
-`Form` is the submit boundary. Dify UI form primitives are not a form state-management framework; business validation and schema-driven behavior belong in `web/`.
-
-## Overlay Contract
-
-Flag:
-
-- Legacy web overlay imports in new or modified code.
-- Manual portals around Dify UI overlay primitives.
-- Call-site `z-*` overrides on overlays.
-- Missing root `isolation: isolate` assumptions when debugging overlay stacking.
-- Repeated backdrop, z-index, or portal chrome at call sites.
-- Tooltip used for infotips, long text, or interactive content.
-
-All Dify UI body-portalled overlays use `z-50`. Toast uses `z-60`. DOM order handles stacking between overlays.
-
-## Primitive Selection
-
-Flag:
-
-- `Tabs` used for simple mode/filter/view selection where `SegmentedControl` is the semantic primitive.
-- `SegmentedControl` used where `tablist` / `tabpanel` semantics are required.
-- `Select` used for searchable or free-form input.
-- `Combobox` used for unrestricted search text where no selected option is remembered.
-- `Autocomplete` used for closed-list selection.
-- Tooltip or PreviewCard used for content that must be reachable on touch or by screen readers.
-
-Use:
-
-- `Autocomplete` for free-form text with optional suggestions.
-- `Combobox` for searchable selected values from a collection.
-- `Select` for closed, scannable option sets.
-- `Popover` for infotips, help text, rich content, or interactions.
-
-## Bad Usage Patterns To Flag
-
-Flag:
-
-- Manually recreating UI behavior or chrome already owned by `@langgenius/dify-ui/*` or `web/app/components/base/*`, such as buttons, inputs, toggle groups, popovers, dropdown menus, alert dialogs, switches, avatars, scroll areas, toasts, borders, focus states, disabled states, segmented controls, or existing feature components.
-- Styling a raw Base UI primitive directly in `web/` when a Dify UI primitive exists.
-- Wrapping a Dify UI primitive in a feature component that hides its label, error, disabled, or focus contract.
-- Replacing a semantic primitive with a generic `div` plus classes to match a screenshot.
-- Using `Tooltip` because it is visually convenient when the content is actually help text or needs touch access.
-- Adding a `z-*` override to make a child popup appear over a parent dialog.
-- Adding a new app-level wrapper around Dialog, Drawer, Popover, Select, or Combobox that repeats portal/backdrop/positioner logic.
-- Using dify-ui `Input` as a drop-in replacement for legacy inputs that include search, clear, copy, unit, localized placeholder, or number normalization behavior.
-- Building a form row from loose text and controls instead of the matching Field/Form primitives.
-- Adding component state only to style `data-open`, `data-checked`, `data-disabled`, or highlighted states that Base UI already exposes.
-- Passing booleans down only so children can toggle classes already expressible with primitive `data-*` selectors.
-
-## Tokens, Radius, And Styling
-
-Flag:
-
-- `radius-*` class names.
-- Custom Tailwind `borderRadius` extension for Figma radius values.
-- Generic colors where semantic Dify tokens exist.
-- Hardcoded design values where Dify tokens, component variants, or documented Figma radius mappings exist.
-- `!` important modifiers used to fight primitive styles instead of fixing the variant, selector, or component composition.
-- Manual class strings that duplicate primitive variants.
-- `min-w-(--anchor-width)` on picker popups when it defeats viewport clamping.
-
-Use the Figma radius mapping from `packages/dify-ui/README.md`; for example `--radius/sm` maps to `rounded-md`, and `--radius/md` maps to `rounded-lg`.
-
-Use `!` only for a tightly scoped compatibility override after confirming the primitive API, data attributes, and selector structure cannot express the state.
-
-## Focus Details
-
-Flag focus rings attached to the wrong element. For example, Base UI `Slider.Thumb` focuses an internal `input[type=range]`, so the visible thumb wrapper needs `has-[:focus-visible]` rather than direct wrapper `focus-visible`.
-
-## Custom SVG Icons
-
-Flag:
-
-- New generated React icon components or JSON files under `web/app/components/base/icons/src/...` for custom SVG icons.
-- Custom SVG icons consumed outside the Tailwind `i-custom-*` icon class pipeline.
-- Generated `packages/iconify-collections/custom-*/icons.json` diffs where unrelated existing icons lost or changed intrinsic `width` or `height`.
-
-New custom SVG icons belong in `packages/iconify-collections/assets/...`. Regenerate with `pnpm --filter @dify/iconify-collections generate`, validate with `pnpm --filter @dify/iconify-collections check:dimensions`, and consume the generated icon with Tailwind `i-custom-*` classes.
+Treat the implementation, public types, tests, and stories as evidence for the documented
+contract. If they disagree, identify the actual owner before reporting a finding. Read current
+official Base UI documentation and installed type declarations for upstream-derived behavior.
+Report only a reproducible contract violation or observable defect, not a preference inferred from
+this routing file.

.agents/skills/frontend-testing/SKILL.md

@@ -13,4 +13,4 @@ description: Use when writing or changing Vitest or React Testing Library tests
 4. Run the focused spec before the affected suite and relevant static checks.
 5. Report the behavior verified and any remaining browser, visual, or end-to-end risk.
 
-Recommend deleting low-value tests as readily as adding missing behavior coverage. Use `web/docs/test.md` for policy and Web commands; use the `packages/dify-ui/README.md` Development section for Dify UI commands.
+Recommend deleting low-value tests as readily as adding missing behavior coverage. Use `web/docs/test.md` for policy and Web commands; use `packages/dify-ui/docs/testing.md` for Dify UI commands.

.agents/skills/how-to-write-component/SKILL.md

@@ -32,7 +32,7 @@ Use this skill to route component architecture decisions to its bundled referenc
 1. Identify the behavior owner and the public contract being changed.
 2. Read the nearby implementation, tests, and only the routed skill references.
 3. Implement one coherent vertical slice. Do not expand into equivalent patterns elsewhere unless the current contract cannot be completed without them.
-4. Verify observable behavior at the narrowest sufficient boundary, then run the checks documented by the owning package: `web/docs/test.md` or `web/docs/lint.md` for Web, and the `packages/dify-ui/README.md` Development section for Dify UI.
+4. Verify observable behavior at the narrowest sufficient boundary, then run the checks documented by the owning package: `web/docs/test.md` or `web/docs/lint.md` for Web, and `packages/dify-ui/docs/testing.md` for Dify UI.
 
 [data]: references/data.md
 [interactions]: references/interactions.md

.agents/skills/how-to-write-component/references/interactions.md

@@ -1,6 +1,6 @@
 # Component Interactions And Overlays
 
-Read this document when a change involves application hotkeys, focus, dialogs, menus, popovers, or other secondary surfaces. Overlay primitive selection and layering are owned by the [overlay guide].
+Read this document when a change involves application hotkeys, focus, dialogs, menus, popovers, or other secondary surfaces. Overlay primitive selection and layering are owned by the [overlay contract].
 
 ## Focus And Semantics
 
@@ -20,12 +20,11 @@ Read this document when a change involves application hotkeys, focus, dialogs, m
 
 ## Secondary Surfaces
 
-- Follow `web/docs/overlay.md` for primitive choice. Dify UI primitives are the default, with package-approved Web wrappers such as `Infotip` where the overlay guide allows them.
+- Follow the [overlay contract] for primitive choice and shared mechanics. The nearest consumer `AGENTS.md` owns application-specific composite reuse policy.
 - Separate behavior ownership from placement ownership: the action may own trigger, open state, and menu content while the caller owns slots, offsets, and alignment.
 - Keep menu and dialog surfaces as siblings when a menu command opens a dialog. Mount the dialog outside popup content.
 - Mount controlled overlays unconditionally unless unmounting is required for performance or reset semantics. Prefer keyed or owner-local reset over conditional wrappers.
 - Put query and mutation work inside dialog or alert-dialog content when it should mount only after opening.
 - Prefer uncontrolled roots when the primitive can own open state. Use controlled state only for business coordination, analytics, cleanup, or explicit reset behavior.
-- Do not add manual portals or call-site z-index escalation. Fix ownership and stacking structure at the shared boundary.
 
-[overlay guide]: ../../../../web/docs/overlay.md
+[overlay contract]: ../../../../packages/dify-ui/docs/overlays.md

.agents/skills/how-to-write-component/references/runtime.md

@@ -4,6 +4,7 @@ Read this document when a change introduces Effects, navigation side effects, me
 
 ## Effects
 
+- Keep render pure: do not read or write `ref.current` during render except for predictable null-guarded lazy initialization. Update interaction-owned refs in event handlers, synchronize external-system refs after commit, and use state or derivation for rendered values.
 - Use Effects only to synchronize with a named external system such as a browser API, subscription, timer, analytics integration, non-React widget, or imperative DOM API.
 - Do not use Effects to transform render state, handle user actions, copy query data, reset state from props, or fetch data owned by framework APIs or TanStack Query.
 - Initialize query-backed forms with keyed remounts or surface-entry hydration instead of copying data through Effects.

.editorconfig

@@ -37,3 +37,7 @@ indent_size = 2
 [package.json]
 indent_style = space
 indent_size = 2
+
+# Oxlint writes its generated suppression baseline without a final newline.
+[oxlint-suppressions.json]
+insert_final_newline = false

.gemini/config.yaml

@@ -1,13 +0,0 @@
-have_fun: false
-memory_config:
-  disabled: false
-code_review:
-  disable: true
-  comment_severity_threshold: MEDIUM
-  max_review_comments: -1
-  pull_request_opened:
-    help: false
-    summary: false
-    code_review: false
-    include_drafts: false
-ignore_patterns: []

.github/CODEOWNERS

@@ -245,7 +245,7 @@
 
 # Frontend - Billing and Education
 /web/app/components/billing/ @iamjoel @zxhlyh
-/web/app/education-apply/ @iamjoel @zxhlyh
+/web/app/education/ @iamjoel @zxhlyh
 
 # Frontend - Workspace
 /web/app/components/header/account-dropdown/workplace-selector/ @iamjoel @zxhlyh
@@ -267,5 +267,9 @@
 /web/app/auth/refresh/route.ts @iamjoel @lyzno1
 /web/service/server.ts @iamjoel @lyzno1
 
+# Frontend - Browser Mode Tests
+/web/app/**/*.browser.spec.ts @lyzno1
+/web/app/**/*.browser.spec.tsx @lyzno1
+
 # Docker
 /docker/* @laipz8200

.github/actions/setup-web/action.yml

@@ -9,7 +9,7 @@ runs:
       with:
         run_install: false
     - name: Setup Vite+
-      uses: voidzero-dev/setup-vp@143f5f385f39b1b753ffed1a01ad443811855c8b # v1.16.1
+      uses: voidzero-dev/setup-vp@1b32467adbe183473499fd9d5d372c3ed9641754 # v1.18.0
       with:
         node-version-file: .nvmrc
         cache: true

.github/linters/.yaml-lint.yml

@@ -4,6 +4,8 @@ extends: default
 rules:
   brackets:
     max-spaces-inside: 1
+  comments:
+    min-spaces-from-content: 1
   comments-indentation: disable
   document-start: disable
   line-length: disable

.github/pull_request_template.md

@@ -21,4 +21,4 @@
 - [ ] I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
 - [ ] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
 - [ ] I've updated the documentation accordingly.
-- [ ] I ran `make lint && make type-check` (backend) and `cd web && pnpm exec vp staged` (frontend) to appease the lint gods
+- [ ] I ran `make lint && make type-check` (backend) and `vp staged` (frontend) to appease the lint gods

.github/workflows/accessibility-e2e.yml

@@ -36,7 +36,7 @@ concurrency:
 jobs:
   accessibility:
     name: WCAG Level ${{ inputs.level == 'a' && 'A' || 'AA' }} · ${{ inputs.page }}
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: ubuntu-24.04
     timeout-minutes: 120
     defaults:
       run:
@@ -52,7 +52,7 @@ jobs:
         uses: ./.github/actions/setup-web
 
       - name: Setup UV and Python
-        uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+        uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
         with:
           enable-cache: true
           python-version: '3.12'

.github/workflows/api-tests.yml

@@ -35,7 +35,7 @@ jobs:
           persist-credentials: false
 
       - name: Setup UV and Python
-        uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+        uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
         with:
           enable-cache: true
           python-version: ${{ matrix.python-version }}
@@ -94,7 +94,7 @@ jobs:
           persist-credentials: false
 
       - name: Setup UV and Python
-        uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+        uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
         with:
           enable-cache: true
           python-version: ${{ matrix.python-version }}
@@ -145,7 +145,7 @@ jobs:
           persist-credentials: false
 
       - name: Setup UV and Python
-        uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+        uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
         with:
           enable-cache: true
           python-version: '3.12'

.github/workflows/autofix.yml

@@ -50,6 +50,8 @@ jobs:
             vite.config.ts
             lint.config.ts
             eslint.config.mjs
+            knip.config.ts
+            scripts/check-web-production-unused-after-knip-fix.mjs
             oxlint-suppressions.json
             eslint-suppressions.json
             .vscode/**
@@ -88,7 +90,7 @@ jobs:
           python-version: '3.11'
 
       - if: github.event_name != 'merge_group'
-        uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+        uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
 
       - name: Generate Docker Compose
         if: github.event_name != 'merge_group' && steps.docker-compose-changes.outputs.any_changed == 'true'

.github/workflows/build-push.yml

@@ -103,7 +103,7 @@ jobs:
           password: ${{ env.DOCKERHUB_TOKEN }}
 
       - name: Set up Depot CLI
-        uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.7.1
+        uses: depot/setup-action@91bc8495a33ebfc504ffc89e5674379ccf23c29c # v1.7.2
 
       - name: Extract metadata for Docker
         id: meta
@@ -160,7 +160,7 @@ jobs:
             file: 'docker/Dockerfile'
     steps:
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0
+        uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
 
       - name: Validate Docker image
         uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
@@ -229,3 +229,32 @@ jobs:
           IMAGE_VERSION: ${{ steps.meta.outputs.version }}
         run: |
           docker buildx imagetools inspect "$IMAGE_NAME:$IMAGE_VERSION"
+
+  sync-e2b-template:
+    needs: create-manifest
+    runs-on: ubuntu-24.04
+    if: github.repository == 'langgenius/dify' && startsWith(github.ref, 'refs/tags/')
+    strategy:
+      matrix:
+        include:
+          - project: dev
+            api_key_secret: E2B_API_KEY_DEV
+          - project: prod
+            api_key_secret: E2B_API_KEY_PROD
+    steps:
+      - name: Checkout
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
+
+      - name: Login to Docker Hub
+        uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
+        with:
+          username: ${{ env.DOCKERHUB_USER }}
+          password: ${{ env.DOCKERHUB_TOKEN }}
+
+      - name: Sync E2B Template (${{ matrix.project }})
+        env:
+          E2B_API_KEY: ${{ secrets[matrix.api_key_secret] }}
+        run: ./dify-agent-runtime/docker/sync-e2b-template.sh

.github/workflows/db-migration-test.yml

@@ -19,7 +19,7 @@ jobs:
           persist-credentials: false
 
       - name: Setup UV and Python
-        uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+        uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
         with:
           enable-cache: true
           python-version: '3.12'
@@ -40,7 +40,7 @@ jobs:
           cp envs/middleware.env.example middleware.env
 
       - name: Set up Middlewares
-        uses: hoverkraft-tech/compose-action@11beaa1c2dae4e8ed7b1665aa074723b6cecb0e4 # v3.0.0
+        uses: hoverkraft-tech/compose-action@ee6af68587292d72db67743171809c19787df4c9 # v3.1.0
         with:
           compose-file: |
             docker/docker-compose.middleware.yaml
@@ -69,7 +69,7 @@ jobs:
           persist-credentials: false
 
       - name: Setup UV and Python
-        uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+        uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
         with:
           enable-cache: true
           python-version: '3.12'
@@ -94,7 +94,7 @@ jobs:
           sed -i 's/DB_USERNAME=postgres/DB_USERNAME=mysql/' middleware.env
 
       - name: Set up Middlewares
-        uses: hoverkraft-tech/compose-action@11beaa1c2dae4e8ed7b1665aa074723b6cecb0e4 # v3.0.0
+        uses: hoverkraft-tech/compose-action@ee6af68587292d72db67743171809c19787df4c9 # v3.1.0
         with:
           compose-file: |
             docker/docker-compose.middleware.yaml

.github/workflows/deploy-hitl-im-dev.yml

@@ -0,0 +1,25 @@
+name: Deploy HITL IM Dev
+
+on:
+  workflow_run:
+    workflows: ['Build and Push API & Web']
+    branches:
+      - 'build/feat/hitl-im-dev'
+    types:
+      - completed
+
+jobs:
+  deploy:
+    runs-on: depot-ubuntu-24.04
+    if: |
+      github.event.workflow_run.conclusion == 'success' &&
+      github.event.workflow_run.head_branch == 'build/feat/hitl-im-dev'
+    steps:
+      - name: Deploy to server
+        uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2 # v1.2.5
+        with:
+          host: ${{ secrets.HITL_SSH_HOST }}
+          username: ${{ secrets.SSH_USER }}
+          key: ${{ secrets.SSH_PRIVATE_KEY }}
+          script: |
+            ${{ vars.SSH_SCRIPT || secrets.SSH_SCRIPT }}

.github/workflows/docker-build.yml

@@ -65,7 +65,7 @@ jobs:
             file: 'docker/Dockerfile'
     steps:
       - name: Set up Depot CLI
-        uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.7.1
+        uses: depot/setup-action@91bc8495a33ebfc504ffc89e5674379ccf23c29c # v1.7.2
 
       - name: Build Docker Image
         uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.18.0
@@ -96,7 +96,7 @@ jobs:
             file: 'docker/Dockerfile'
     steps:
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0
+        uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
 
       - name: Build Docker Image
         uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0

.github/workflows/main-ci.yml

@@ -15,8 +15,10 @@ permissions:
   statuses: write
 
 concurrency:
-  group: main-ci-${{ github.event.pull_request.number || github.event.merge_group.head_sha || github.run_id }}
-  cancel-in-progress: true
+  # Pull request deduplication is owned by skip-duplicate-actions so pre_job can
+  # publish should_skip before required status finalizers run.
+  group: main-ci-${{ github.event.merge_group.head_sha || github.run_id }}
+  cancel-in-progress: ${{ github.event_name == 'merge_group' }}
 
 jobs:
   pre_job:
@@ -394,10 +396,10 @@ jobs:
   style-check:
     name: Style Check
     needs: pre_job
-    if: needs.pre_job.outputs.should_skip != 'true'
     uses: ./.github/workflows/style.yml
     with:
       base-rev: ${{ github.event.pull_request.base.sha || github.event.merge_group.base_sha }}
+      skip-checks: ${{ needs.pre_job.outputs.should_skip == 'true' }}
 
   vdb-tests-run:
     name: Run VDB Tests

.github/workflows/pyrefly-diff.yml

@@ -22,7 +22,7 @@ jobs:
           fetch-depth: 0
 
       - name: Setup Python & UV
-        uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+        uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
         with:
           enable-cache: true
 

.github/workflows/pyrefly-type-coverage-comment.yml

@@ -24,7 +24,7 @@ jobs:
         uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
 
       - name: Setup Python & UV
-        uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+        uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
         with:
           enable-cache: true
 

.github/workflows/pyrefly-type-coverage.yml

@@ -22,7 +22,7 @@ jobs:
           fetch-depth: 0
 
       - name: Setup Python & UV
-        uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+        uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
         with:
           enable-cache: true
 

.github/workflows/style.yml

@@ -6,6 +6,11 @@ on:
       base-rev:
         required: true
         type: string
+      skip-checks:
+        description: Create the required check runs without repeating previously successful work.
+        required: false
+        type: boolean
+        default: false
 
 permissions:
   checks: write
@@ -15,6 +20,7 @@ permissions:
 jobs:
   python-style:
     name: Python Style
+    if: ${{ !inputs.skip-checks }}
     runs-on: depot-ubuntu-24.04
 
     steps:
@@ -41,7 +47,7 @@ jobs:
 
       - name: Setup UV and Python
         if: steps.changed-files.outputs.any_changed == 'true'
-        uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+        uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
         with:
           enable-cache: false
           python-version: '3.12'
@@ -79,6 +85,7 @@ jobs:
 
   web-style:
     name: Web Style
+    if: ${{ !inputs.skip-checks }}
     runs-on: depot-ubuntu-24.04
     defaults:
       run:
@@ -138,6 +145,7 @@ jobs:
 
   ts-common-style:
     name: TS Common
+    if: ${{ !inputs.skip-checks }}
     runs-on: depot-ubuntu-24.04-4
     permissions:
       checks: write
@@ -166,6 +174,8 @@ jobs:
             vite.config.ts
             lint.config.ts
             eslint.config.mjs
+            knip.config.ts
+            scripts/check-web-production-unused-after-knip-fix.mjs
             oxlint-suppressions.json
             eslint-suppressions.json
             .vscode/**
@@ -181,6 +191,7 @@ jobs:
 
   superlinter:
     name: SuperLinter
+    if: ${{ !inputs.skip-checks }}
     runs-on: depot-ubuntu-24.04
 
     steps:
@@ -201,6 +212,7 @@ jobs:
             **Dockerfile
             dev/**
             .editorconfig
+            .vite-hooks/**
 
       - name: Super-linter
         uses: super-linter/super-linter/slim@4ce20838b8ab83717e78138c5b3a1407148e0918 # v8.7.0
@@ -209,10 +221,11 @@ jobs:
           BASH_SEVERITY: warning
           DEFAULT_BRANCH: origin/main
           EDITORCONFIG_FILE_NAME: editorconfig-checker.json
-          FILTER_REGEX_INCLUDE: pnpm-lock.yaml
+          ENABLE_GITHUB_PULL_REQUEST_SUMMARY_COMMENT: false
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           IGNORE_GENERATED_FILES: true
           IGNORE_GITIGNORED_FILES: true
+          VALIDATE_ALL_CODEBASE: false
           VALIDATE_BASH: true
           VALIDATE_BASH_EXEC: true
           # FIXME: temporarily disabled until api-docker.yaml's run script is fixed for shellcheck

.github/workflows/translate-i18n-claude.yml

@@ -8,15 +8,19 @@ on:
   workflow_dispatch:
     inputs:
       files:
-        description: 'Specific files to translate (space-separated, e.g., "app common"). Required for full mode; leave empty in incremental mode to use en-US files changed since HEAD~1.'
+        description: 'Specific files to translate (space-separated, e.g., "app common").
+          Required for full mode; leave empty in incremental mode to use en-US
+          files changed since HEAD~1.'
         required: false
         type: string
       languages:
-        description: 'Specific languages to translate (space-separated, e.g., "zh-Hans ja-JP"). Leave empty for all supported target languages except en-US.'
+        description: 'Specific languages to translate (space-separated, e.g., "zh-Hans
+          ja-JP"). Leave empty for all supported target languages except en-US.'
         required: false
         type: string
       mode:
-        description: 'Sync mode: incremental (compare with previous en-US revision) or full (sync all keys in scope)'
+        description: 'Sync mode: incremental (compare with previous en-US revision) or
+          full (sync all keys in scope)'
         required: false
         default: incremental
         type: choice
@@ -158,7 +162,7 @@ jobs:
 
       - name: Run Claude Code for Translation Sync
         if: steps.context.outputs.CHANGED_FILES != ''
-        uses: anthropics/claude-code-action@c038e4dcdedfbbca18dfb17df35a17e40ded4ddc # v1.0.186
+        uses: anthropics/claude-code-action@dcb57747bfceeaa1fa72638cae52295d1d853d4a # v1.0.199
         with:
           anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
           github_token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/vdb-tests-full.yml

@@ -29,14 +29,14 @@ jobs:
           persist-credentials: false
 
       - name: Free Disk Space
-        uses: endersonmenezes/free-disk-space@7901478139cff6e9d44df5972fd8ab8fcade4db1 # v3.2.2
+        uses: endersonmenezes/free-disk-space@2a22f8c59cae9cddae7194b33e6b92d023b2b4b8 # v4.0.0
         with:
           remove_dotnet: true
           remove_haskell: true
           remove_tool_cache: true
 
       - name: Setup UV and Python
-        uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+        uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
         with:
           enable-cache: true
           python-version: ${{ matrix.python-version }}

.github/workflows/vdb-tests.yml

@@ -26,14 +26,14 @@ jobs:
           persist-credentials: false
 
       - name: Free Disk Space
-        uses: endersonmenezes/free-disk-space@7901478139cff6e9d44df5972fd8ab8fcade4db1 # v3.2.2
+        uses: endersonmenezes/free-disk-space@2a22f8c59cae9cddae7194b33e6b92d023b2b4b8 # v4.0.0
         with:
           remove_dotnet: true
           remove_haskell: true
           remove_tool_cache: true
 
       - name: Setup UV and Python
-        uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+        uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
         with:
           enable-cache: true
           python-version: ${{ matrix.python-version }}

.github/workflows/web-e2e.yml

@@ -14,7 +14,7 @@ permissions:
 jobs:
   test:
     name: Web Full-Stack E2E
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: ubuntu-24.04
     timeout-minutes: 120
     defaults:
       run:
@@ -26,16 +26,11 @@ jobs:
         with:
           persist-credentials: false
 
-      - name: Switch Ubuntu mirror
-        run: |
-          sudo sed -i 's|http://us-east-1.ec2.archive.ubuntu.com/ubuntu|http://azure.archive.ubuntu.com/ubuntu|g' /etc/apt/sources.list.d/ubuntu.sources
-          sudo apt-get update
-
       - name: Setup web dependencies
         uses: ./.github/actions/setup-web
 
       - name: Setup UV and Python
-        uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+        uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
         with:
           enable-cache: true
           python-version: '3.12'

.github/workflows/web-tests.yml

@@ -33,7 +33,7 @@ jobs:
         uses: ./.github/actions/setup-web
 
       - name: Run tests
-        run: vp test run --reporter=blob --reporter=minimal --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --coverage
+        run: vp test run --project unit --reporter=blob --reporter=minimal --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --coverage
 
       - name: Upload blob report
         if: ${{ !cancelled() }}
@@ -44,6 +44,41 @@ jobs:
           include-hidden-files: true
           retention-days: 1
 
+  browser-test:
+    name: Web Browser Tests
+    runs-on: ubuntu-24.04
+    timeout-minutes: 20
+    defaults:
+      run:
+        shell: bash
+        working-directory: ./web
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
+
+      - name: Setup web environment
+        uses: ./.github/actions/setup-web
+
+      - name: Install Chromium for Browser Mode
+        timeout-minutes: 15
+        run: vp exec playwright install --with-deps --only-shell chromium
+
+      - name: Run browser tests
+        run: vp test run --project browser --silent=passed-only
+
+      - name: Upload browser failure artifacts
+        if: ${{ failure() && !cancelled() }}
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: browser-test-failures
+          path: web/.vitest-browser/
+          if-no-files-found: ignore
+          include-hidden-files: true
+          retention-days: 7
+
   merge-reports:
     name: Merge Test Reports
     if: ${{ !cancelled() }}
@@ -86,7 +121,7 @@ jobs:
 
   dify-ui-test:
     name: dify-ui Tests
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: ubuntu-24.04
     timeout-minutes: 20
     env:
       CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
@@ -101,11 +136,6 @@ jobs:
         with:
           persist-credentials: false
 
-      - name: Switch Ubuntu mirror
-        run: |
-          sudo sed -i 's|http://us-east-1.ec2.archive.ubuntu.com/ubuntu|http://azure.archive.ubuntu.com/ubuntu|g' /etc/apt/sources.list.d/ubuntu.sources
-          sudo apt-get update
-
       - name: Setup web environment
         uses: ./.github/actions/setup-web
 
@@ -116,6 +146,16 @@ jobs:
       - name: Run dify-ui tests
         run: vp test run --project unit --coverage --silent=passed-only
 
+      - name: Upload dify-ui test failure artifacts
+        if: ${{ failure() && !cancelled() }}
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: dify-ui-test-failures
+          path: packages/dify-ui/.vitest-browser/
+          if-no-files-found: ignore
+          include-hidden-files: true
+          retention-days: 7
+
       - name: Report coverage
         if: ${{ env.CODECOV_TOKEN != '' }}
         uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
@@ -127,7 +167,7 @@ jobs:
 
   dify-ui-storybook-test:
     name: dify-ui Storybook Tests
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: ubuntu-24.04
     timeout-minutes: 20
     defaults:
       run:
@@ -140,11 +180,6 @@ jobs:
         with:
           persist-credentials: false
 
-      - name: Switch Ubuntu mirror
-        run: |
-          sudo sed -i 's|http://us-east-1.ec2.archive.ubuntu.com/ubuntu|http://azure.archive.ubuntu.com/ubuntu|g' /etc/apt/sources.list.d/ubuntu.sources
-          sudo apt-get update
-
       - name: Setup web environment
         uses: ./.github/actions/setup-web
 
@@ -154,3 +189,13 @@ jobs:
 
       - name: Run dify-ui Storybook tests
         run: vp run test:storybook
+
+      - name: Upload dify-ui Storybook test failure artifacts
+        if: ${{ failure() && !cancelled() }}
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: dify-ui-storybook-test-failures
+          path: packages/dify-ui/.vitest-browser/
+          if-no-files-found: ignore
+          include-hidden-files: true
+          retention-days: 7

.gitignore

@@ -263,6 +263,7 @@ scripts/stress-test/reports/
 .qoder/*
 .context/
 # Vitest local reports
+web/.vitest-browser/
 web/.vitest-reports/
 
 # dify-agent-runtime

.vite-hooks/pre-commit

@@ -5,7 +5,6 @@ files=$(git diff --cached --name-only)
 # check if api or web directory is modified
 
 api_modified=false
-web_modified=false
 skip_web_checks=false
 
 git_path() {
@@ -29,10 +28,6 @@ do
             # set api_modified flag to true
             api_modified=true
             ;;
-        web/*)
-            # set web_modified flag to true
-            web_modified=true
-            ;;
     esac
 done
 

.vscode/settings.example.json

@@ -4,7 +4,6 @@
 
   "tailwindCSS.experimental.configFile": "web/app/styles/globals.css",
 
-  // Format
   "[javascript]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
   "[javascriptreact]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
   "[typescript]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
@@ -25,9 +24,9 @@
   "[handlebars]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
   "editor.formatOnSave": true,
   "editor.formatOnSaveMode": "file",
-  "oxc.fmt.configPath": "./vite.config.ts",
+  "oxc.disableNestedConfig": true,
+  "oxc.fmt.disableNestedConfig": true,
 
-  // Lint fix
   "eslint.useFlatConfig": true,
   "eslint.validate": ["json", "jsonc", "markdown", "yaml", "toml"],
   "editor.codeActionsOnSave": {

README.md

@@ -129,83 +129,35 @@ All of Dify's offerings come with corresponding APIs, so you could effortlessly
 
 Star Dify on GitHub and be instantly notified of new releases.
 
-![star-us](https://github.com/langgenius/dify/assets/13230914/b823edc1-6388-4e25-ad45-2f6b187adbb4)
+<img width="1344" height="720" alt="star" src="https://github.com/user-attachments/assets/dcd086d1-af0f-471b-ae52-1ad2fa040595" />
 
 ## Advanced Setup
 
-### Custom configurations
-
-If you need to customize the configuration, edit `docker/.env`. The essential startup defaults live in [`docker/.env.example`](docker/.env.example), and optional advanced variables are split under `docker/envs/` by theme. After making any changes, re-run `docker compose up -d` from the `docker` directory. You can find the full list of available environment variables [here](https://docs.dify.ai/getting-started/install-self-hosted/environments).
-
-### Metrics Monitoring with Grafana
-
-Import the dashboard to Grafana, using Dify's PostgreSQL database as data source, to monitor metrics in granularity of apps, tenants, messages, and more.
-
-- [Grafana Dashboard by @bowenliang123](https://github.com/bowenliang123/dify-grafana-dashboard)
-
-### Deployment with Kubernetes
-
-If you'd like to configure a highly available setup, there are community-contributed [Helm Charts](https://helm.sh/) and YAML files which allow Dify to be deployed on Kubernetes.
-
-- [Helm Chart by @LeoQuote](https://github.com/douban/charts/tree/master/charts/dify)
-- [Helm Chart by @BorisPolonsky](https://github.com/BorisPolonsky/dify-helm)
-- [Helm Chart by @magicsong](https://github.com/magicsong/ai-charts)
-- [YAML file by @Winson-030](https://github.com/Winson-030/dify-kubernetes)
-- [YAML file by @wyy-holding](https://github.com/wyy-holding/dify-k8s)
-- [🚀 NEW! YAML files (Supports Dify v1.6.0) by @Zhoneym](https://github.com/Zhoneym/DifyAI-Kubernetes)
-
-#### Using Terraform for Deployment
-
-Deploy Dify to Cloud Platform with a single click using [terraform](https://www.terraform.io/)
-
-##### Azure Global
-
-- [Azure Terraform by @nikawang](https://github.com/nikawang/dify-azure-terraform)
-
-##### Google Cloud
-
-- [Google Cloud Terraform by @sotazum](https://github.com/DeNA/dify-google-cloud-terraform)
-
-#### Using AWS CDK for Deployment
-
-Deploy Dify to AWS with [CDK](https://aws.amazon.com/cdk/)
-
-##### AWS
-
-- [AWS CDK by @KevinZhao (EKS based)](https://github.com/aws-samples/solution-for-deploying-dify-on-aws)
-- [AWS CDK by @tmokmss (ECS based)](https://github.com/aws-samples/dify-self-hosted-on-aws)
-
-#### Using Alibaba Cloud Computing Nest
-
-Quickly deploy Dify to Alibaba cloud with [Alibaba Cloud Computing Nest](https://computenest.console.aliyun.com/service/instance/create/default?type=user&ServiceName=Dify%E7%A4%BE%E5%8C%BA%E7%89%88)
-
-#### Using Alibaba Cloud Data Management
-
-One-Click deploy Dify to Alibaba Cloud with [Alibaba Cloud Data Management](https://www.alibabacloud.com/help/en/dms/dify-in-invitational-preview/)
-
-#### Deploy to AKS with Azure Devops Pipeline
-
-One-Click deploy Dify to AKS with [Azure Devops Pipeline Helm Chart by @LeoZhang](https://github.com/Ruiruiz30/Dify-helm-chart-AKS)
+For custom configuration, observability, and deployment options, see [Advanced Setup](docs/ADVANCED_SETUP.md).
 
 ## Contributing
 
-For those who'd like to contribute code, see our [Contribution Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md).
-At the same time, please consider supporting Dify by sharing it on social media and at events and conferences.
+Dify welcomes contributions of all kinds:
 
-> We are looking for contributors to help translate Dify into languages other than Mandarin or English. If you are interested in helping, please see the [i18n README](https://github.com/langgenius/dify/blob/main/web/i18n-config/README.md) for more information, and leave us a comment in the `global-users` channel of our [Discord Community Server](https://discord.gg/8Tpq4AcN9c).
+- **Code**: Read the [Contribution Guide](CONTRIBUTING.md), then browse [good first issues](https://github.com/langgenius/dify/issues?q=is%3Aissue%20state%3Aopen%20label%3A%22good%20first%20issue%22).
+- **Ideas and feedback**: Start or join a [GitHub Discussion](https://github.com/langgenius/dify/discussions).
+- **Translations**: Follow the [internationalization guide](web/i18n-config/README.md) to add or update a locale.
+- **Community**: Share the apps you build, help other users, and spread the word about Dify.
+
+### Contributors
+
+<a href="https://github.com/langgenius/dify/graphs/contributors">
+  <img alt="Dify contributors" src="https://contrib.rocks/image?repo=langgenius/dify" />
+</a>
 
 ## Community & contact
 
-- [GitHub Discussion](https://github.com/langgenius/dify/discussions). Best for: sharing feedback and asking questions.
-- [GitHub Issues](https://github.com/langgenius/dify/issues). Best for: bugs you encounter using Dify.AI, and feature proposals. See our [Contribution Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md).
-- [Discord](https://discord.gg/FngNHpbcY7). Best for: sharing your applications and hanging out with the community.
-- [X(Twitter)](https://twitter.com/dify_ai). Best for: sharing your applications and hanging out with the community.
+Choose the channel that best fits your question:
 
-**Contributors**
-
-<a href="https://github.com/langgenius/dify/graphs/contributors">
-  <img src="https://contrib.rocks/image?repo=langgenius/dify" />
-</a>
+- [GitHub Discussions](https://github.com/langgenius/dify/discussions): Get help, share feedback, and propose ideas.
+- [GitHub Issues](https://github.com/langgenius/dify/issues): Report reproducible bugs and track engineering work. Read the [Contribution Guide](CONTRIBUTING.md) before opening one.
+- [Discord](https://discord.gg/FngNHpbcY7): Chat in real time, share your apps, and connect with other Dify users.
+- [X](https://x.com/dify_ai): Follow Dify for release news and project updates.
 
 ## Star History
 

api/.env.example

@@ -98,9 +98,11 @@ REDIS_KEEPALIVE=true
 CELERY_BROKER_URL=redis://:difyai123456@localhost:${REDIS_PORT}/1
 CELERY_BACKEND=redis
 
-# Ops trace retry configuration
-OPS_TRACE_RETRYABLE_DISPATCH_MAX_RETRIES=60
+# Ops trace configuration
+OPS_TRACE_UNIFIED_ENABLED=false
+OPS_TRACE_RETRYABLE_DISPATCH_MAX_RETRIES=780
 OPS_TRACE_RETRYABLE_DISPATCH_DELAY_SECONDS=5
+OPS_TRACE_PARENT_CONTEXT_TTL_SECONDS=3900
 
 # Database configuration
 DB_TYPE=postgresql
@@ -488,6 +490,13 @@ SENDGRID_API_KEY=
 # Sentry configuration
 SENTRY_DSN=
 
+# Cloudflare Turnstile server-side verification for Dify Cloud sign-in
+TURNSTILE_SECRET_KEY=
+# Comma-separated parent or exact hostnames, for example: dify.ai,staging.dify.dev
+TURNSTILE_ALLOWED_HOSTNAMES=
+# Enable only after the compatible web client has been fully deployed.
+TURNSTILE_EMAIL_CODE_VERIFY_REQUIRED=false
+
 # DEBUG
 DEBUG=false
 ENABLE_REQUEST_LOGGING=False
@@ -578,7 +587,7 @@ INDEXING_MAX_SEGMENTATION_TOKENS_LENGTH=4000
 
 # Workflow runtime configuration
 WORKFLOW_MAX_EXECUTION_STEPS=500
-WORKFLOW_MAX_EXECUTION_TIME=1200
+WORKFLOW_MAX_EXECUTION_TIME=3600
 WORKFLOW_CALL_MAX_DEPTH=5
 MAX_VARIABLE_SIZE=204800
 # Maximum concurrent node-builder LLM calls per workflow generation request
@@ -623,7 +632,7 @@ WORKFLOW_LOG_CLEANUP_BATCH_SIZE=100
 WORKFLOW_LOG_CLEANUP_SPECIFIC_WORKFLOW_IDS=
 
 # App configuration
-APP_MAX_EXECUTION_TIME=1200
+APP_MAX_EXECUTION_TIME=3600
 APP_DEFAULT_ACTIVE_REQUESTS=0
 APP_MAX_ACTIVE_REQUESTS=0
 
@@ -698,11 +707,12 @@ INNER_API_KEY_FOR_PLUGIN=QaHbTe77CtuXmsfyhR7+vRjI/+XbV1AaFy691iy+kGDv2Jvy0/eAh8Y
 
 # Dify Agent backend
 AGENT_BACKEND_BASE_URL=http://localhost:5050
-# Bearer token sent to the Agent backend /runs API. Must match DIFY_AGENT_API_TOKEN on the server side.
+# Bearer token sent to the Agent backend control-plane API. Must match DIFY_AGENT_API_TOKEN on the server side.
 AGENT_BACKEND_API_TOKEN=dify-agent-run-token-for-dev-only
 AGENT_BACKEND_STREAM_READ_TIMEOUT_SECONDS=30
 AGENT_BACKEND_STREAM_MAX_RECONNECTS=3
-AGENT_BACKEND_RUN_TIMEOUT_SECONDS=1200
+# Client deadline for converting a Binding file to a ToolFile through the Agent backend.
+AGENT_BACKEND_BINDING_FILE_DOWNLOAD_TIMEOUT_SECONDS=240
 
 # KnowledgeFS (Dataset 2.0)
 KNOWLEDGE_FS_ENABLED=false
@@ -729,6 +739,8 @@ RESET_PASSWORD_TOKEN_EXPIRY_MINUTES=5
 EMAIL_REGISTER_TOKEN_EXPIRY_MINUTES=5
 CHANGE_EMAIL_TOKEN_EXPIRY_MINUTES=5
 OWNER_TRANSFER_TOKEN_EXPIRY_MINUTES=5
+EMAIL_CODE_LOGIN_TOKEN_EXPIRY_MINUTES=5
+EMAIL_CODE_LOGIN_MAX_ATTEMPTS=5
 
 CREATE_TIDB_SERVICE_JOB_ENABLED=false
 
@@ -825,6 +837,12 @@ ENABLE_HUMAN_INPUT_TIMEOUT_TASK=true
 # Human input timeout check interval in minutes
 HUMAN_INPUT_TIMEOUT_TASK_INTERVAL=1
 
+# Whether to recover soft-deleted conversation cleanup jobs periodically
+ENABLE_CONVERSATION_CLEANUP_TASK=true
+# Recovery interval in minutes and maximum conversations dispatched per sweep
+CONVERSATION_CLEANUP_TASK_INTERVAL=5
+CONVERSATION_CLEANUP_BATCH_SIZE=100
+
 # Nacos remote settings source HTTP timeouts (seconds).
 # Bound how long requests to the Nacos endpoint wait before failing, so a slow or
 # unresponsive Nacos server cannot stall API startup or token refresh.
@@ -832,3 +850,6 @@ HUMAN_INPUT_TIMEOUT_TASK_INTERVAL=1
 DIFY_ENV_NACOS_REQUEST_TIMEOUT=10.0
 # Connect timeout for Nacos requests (default: 3.0)
 DIFY_ENV_NACOS_CONNECT_TIMEOUT=3.0
+
+# skill entry
+ENABLE_SKILL=true

api/.importlinter

@@ -15,6 +15,117 @@ root_packages =
     services
 include_external_packages = True
 
+[importlinter:contract:backend-layers]
+name = Backend modules follow the transport, application, domain, and shared-library layers
+type = layers
+layers =
+    controllers
+    services
+    core
+    libs
+# Migration baseline: keep every exception exact. The default unmatched-import
+# error makes this list shrink whenever a legacy dependency is removed.
+unmatched_ignore_imports_alerting = error
+ignore_imports =
+    core.app.app_config.easy_ui_based_app.dataset.manager -> services.dataset_service
+    core.app.apps.advanced_chat.app_generator -> controllers.console.app.workflow
+    core.app.apps.advanced_chat.app_generator -> services.conversation_service
+    core.app.apps.advanced_chat.app_generator -> services.errors.conversation
+    core.app.apps.advanced_chat.app_generator -> services.workflow_draft_variable_service
+    core.app.apps.advanced_chat.app_runner -> services.conversation_variable_updater
+    core.app.apps.agent_app.app_generator -> services.agent.composer_service
+    core.app.apps.agent_app.app_generator -> services.agent.workspace_service
+    core.app.apps.agent_app.app_generator -> services.conversation_service
+    core.app.apps.agent_app.runtime_request_builder -> services.agent.prompt_mentions
+    core.app.apps.agent_app.session_store -> services.agent.workspace_service
+    core.app.apps.agent_chat.app_generator -> services.conversation_service
+    core.app.apps.base_app_generator -> services.workflow_draft_variable_service
+    core.app.apps.chat.app_generator -> services.conversation_service
+    core.app.apps.common.workflow_response_converter -> services.variable_truncator
+    core.app.apps.completion.app_generator -> services.errors.app
+    core.app.apps.completion.app_generator -> services.errors.message
+    core.app.apps.message_based_app_generator -> services.errors.app_model_config
+    core.app.apps.message_based_app_generator -> services.errors.conversation
+    core.app.apps.message_based_app_generator -> services.errors.message
+    core.app.apps.pipeline.pipeline_generator -> services
+    core.app.apps.pipeline.pipeline_generator -> services.dataset_service
+    core.app.apps.pipeline.pipeline_generator -> services.datasource_provider_service
+    core.app.apps.pipeline.pipeline_generator -> services.feature_service
+    core.app.apps.pipeline.pipeline_generator -> services.workflow_draft_variable_service
+    core.app.apps.pipeline.pipeline_runner -> services.dataset_ref_service
+    core.app.apps.workflow.app_generator -> controllers.console.app.workflow
+    core.app.apps.workflow.app_generator -> services.snippet_generate_service
+    core.app.apps.workflow.app_generator -> services.workflow_draft_variable_service
+    core.app.apps.workflow_app_runner -> tasks.mail_human_input_delivery_task
+    core.app.features.annotation_reply.annotation_reply -> services.annotation_service
+    core.app.features.annotation_reply.annotation_reply -> services.dataset_service
+    core.app.llm.quota -> services.credit_pool_service
+    core.app.task_pipeline.message_cycle_manager -> services.account_service
+    core.app.task_pipeline.message_cycle_manager -> services.annotation_service
+    core.app.workflow.layers.persistence -> services.workflow.inspector_events
+    core.datasource.datasource_manager -> services.datasource_provider_service
+    core.helper.creators -> services.oauth_server
+    core.helper.credential_utils -> services.enterprise.plugin_manager_service
+    core.helper.credential_utils -> services.feature_service
+    core.indexing_runner -> services.vector_space_admission_service
+    core.mcp.auth_client -> services
+    core.provider_manager -> services.credential_permission_service
+    core.provider_manager -> services.credit_pool_service
+    core.provider_manager -> services.feature_service
+    core.rag.datasource.retrieval_service -> services.external_knowledge_service
+    core.rag.index_processor.index_processor -> services.vector_space_admission_service
+    core.rag.index_processor.index_processor_base -> services.file_service
+    core.rag.index_processor.processor.paragraph_index_processor -> services.account_service
+    core.rag.index_processor.processor.paragraph_index_processor -> services.summary_index_service
+    core.rag.index_processor.processor.parent_child_index_processor -> services.account_service
+    core.rag.index_processor.processor.parent_child_index_processor -> services.summary_index_service
+    core.rag.index_processor.processor.qa_index_processor -> services.summary_index_service
+    core.rag.retrieval.dataset_retrieval -> services.external_knowledge_service
+    core.rag.retrieval.dataset_retrieval -> services.feature_service
+    core.rag.summary_index.summary_index -> services.summary_index_service
+    core.rag.summary_index.summary_index -> tasks.generate_summary_index_task
+    core.repositories.sqlalchemy_workflow_node_execution_repository -> services.file_service
+    core.repositories.sqlalchemy_workflow_node_execution_repository -> services.variable_truncator
+    core.tools.tool_manager -> services
+    core.workflow.node_runtime -> services
+    core.workflow.nodes.agent.message_transformer -> services
+    core.workflow.nodes.agent_v2.agent_node -> services.agent.prompt_mentions
+    core.workflow.nodes.agent_v2.agent_node -> services.agent.workspace_service
+    core.workflow.nodes.agent_v2.dify_tools_builder -> services
+    core.workflow.nodes.agent_v2.runtime_feature_manifest -> services.agent.knowledge_datasets
+    core.workflow.nodes.agent_v2.runtime_request_builder -> services.agent.prompt_mentions
+    core.workflow.nodes.agent_v2.runtime_request_builder -> services.skill_management_service
+    core.workflow.nodes.agent_v2.session_store -> services.agent.workspace_service
+    core.workflow.nodes.agent_v2.validators -> services.agent.knowledge_datasets
+    core.workflow.nodes.agent_v2.workspace_retirement_layer -> tasks.collect_agent_resources_task
+    libs.device_flow_security -> controllers.openapi._models
+    libs.device_flow_security -> services.entities.feature_entities
+    libs.device_flow_security -> services.feature_service
+    libs.email_i18n -> services.entities.feature_entities
+    libs.email_i18n -> services.feature_service
+    libs.external_api -> core
+    libs.external_api -> core.errors.error
+    libs.external_api -> extensions.ext_logging
+    libs.flask_utils -> models
+    libs.helper -> core.app.features.rate_limiting.rate_limit
+    libs.helper -> models
+    libs.helper -> models.model
+    libs.json_in_md_parser -> core.llm_generator.output_parser.errors
+    libs.login -> extensions.ext_login
+    libs.login -> models
+    libs.login -> models.model
+    libs.oauth -> core.helper.http_client_pooling
+    libs.oauth_bearer -> models
+    libs.oauth_data_source -> core.db.session_factory
+    libs.oauth_data_source -> core.helper.http_client_pooling
+    libs.rsa -> extensions.ext_storage
+    libs.workspace_permission -> services.enterprise.enterprise_service
+    libs.workspace_permission -> services.feature_service
+    services.account_service -> controllers
+    services.account_service -> controllers.console.error
+    services.app_generate_service -> controllers.console.app.workflow
+    services.billing_service -> controllers.console.error
+
 [importlinter:contract:no-direct-rsa-imports]
 # Note: `libs` itself is deliberately excluded from source_modules -- import-linter's
 # `forbidden` contract cannot check a package against its own descendant (libs.rsa lives
@@ -65,6 +176,110 @@ forbidden_modules =
     sqlalchemy
     werkzeug
 
+[importlinter:contract:init-validation-service-boundary]
+name = Initialization validation application service is framework and persistence neutral
+type = forbidden
+source_modules =
+    services.init_validation_service
+forbidden_modules =
+    controllers
+    extensions
+    flask
+    models
+    repositories
+    sqlalchemy
+    werkzeug
+
+[importlinter:contract:explore-banner-query-service-boundary]
+name = Explore banner query application service is framework and persistence neutral
+type = forbidden
+source_modules =
+    services.explore_banner_query_service
+forbidden_modules =
+    controllers
+    extensions
+    flask
+    models
+    repositories
+    sqlalchemy
+    werkzeug
+
+[importlinter:contract:account-application-boundary]
+name = Account application services and contracts are framework and persistence neutral
+type = forbidden
+source_modules =
+    services.account_avatar_service
+    services.account_change_email_ports
+    services.account_change_email_service
+    services.account_deletion_service
+    services.account_deletion_feedback_service
+    services.account_education_service
+    services.account_errors
+    services.account_initialization_service
+    services.account_integration_service
+    services.account_password_service
+    services.account_ports
+    services.account_profile_service
+    services.entities.account_entities
+forbidden_modules =
+    configs
+    controllers
+    extensions
+    flask
+    models
+    repositories
+    sqlalchemy
+    werkzeug
+
+[importlinter:contract:app-definition-query-service-boundary]
+name = App definition query application service is framework and persistence neutral
+type = forbidden
+source_modules =
+    services.app_definition_query_service
+forbidden_modules =
+    controllers
+    extensions
+    flask
+    models
+    repositories
+    sqlalchemy
+    werkzeug
+
+[importlinter:contract:webapp-access-query-service-boundary]
+name = Web app access query application service is framework and persistence neutral
+type = forbidden
+source_modules =
+    services.webapp_access_query_service
+forbidden_modules =
+    configs
+    controllers
+    extensions
+    flask
+    models
+    repositories
+    services.enterprise
+    services.feature_service
+    sqlalchemy
+    werkzeug
+
+[importlinter:contract:web-app-runtime-query-service-boundary]
+name = Web app runtime query application service does not directly depend on configuration, feature implementation, transport, or ORM modules
+type = forbidden
+source_modules =
+    services.web_app_runtime_query_service
+forbidden_modules =
+    configs
+    controllers
+    extensions
+    flask
+    models
+    repositories
+    services.feature_service
+    services.feature_service_gateway
+    sqlalchemy
+    werkzeug
+allow_indirect_imports = True
+
 [importlinter:contract:feature-query-service-boundary]
 name = Feature query application service is framework and persistence neutral
 type = forbidden
@@ -125,3 +340,76 @@ forbidden_modules =
     repositories
     sqlalchemy
     werkzeug
+
+[importlinter:contract:account-activation-service-boundary]
+name = Account activation application service is framework and persistence neutral
+type = forbidden
+source_modules =
+    services.account_activation_service
+    services.entities.account_activation_entities
+forbidden_modules =
+    configs
+    controllers
+    extensions
+    flask
+    models
+    repositories
+    sqlalchemy
+    werkzeug
+
+[importlinter:contract:tag-application-service-boundary]
+name = Tag application service is framework and persistence neutral
+type = forbidden
+source_modules =
+    services.tag_application_service
+forbidden_modules =
+    configs
+    controllers
+    extensions
+    flask
+    models
+    repositories
+    sqlalchemy
+    werkzeug
+
+[importlinter:contract:recommended-app-query-service-boundary]
+name = Recommended app query application service is framework and persistence neutral
+type = forbidden
+source_modules =
+    services.recommended_app_query_service
+forbidden_modules =
+    configs
+    controllers
+    extensions
+    flask
+    models
+    repositories
+    services.feature_service
+    services.recommended_app_catalog_gateway
+    sqlalchemy
+    werkzeug
+
+[importlinter:contract:recommended-app-catalog-gateway-boundary]
+name = Recommended app catalog gateway does not depend on Flask
+type = forbidden
+source_modules =
+    services.recommended_app_catalog_gateway
+forbidden_modules =
+    flask
+
+[importlinter:contract:billing-application-boundary]
+name = Billing application services and ports are framework and infrastructure neutral
+type = forbidden
+source_modules =
+    services.billing_portal_service
+    services.partner_tenant_binding_service
+forbidden_modules =
+    configs
+    controllers
+    extensions
+    flask
+    models
+    repositories
+    services.billing_service
+    sqlalchemy
+    werkzeug

api/Dockerfile

@@ -33,7 +33,6 @@ RUN uv sync --frozen --no-dev --no-editable
 FROM base AS production
 
 ENV FLASK_APP=app.py
-ENV EDITION=SELF_HOSTED
 ENV DEPLOY_ENV=PRODUCTION
 ENV CONSOLE_API_URL=http://127.0.0.1:5001
 ENV CONSOLE_WEB_URL=http://127.0.0.1:3000

api/app_factory.py

@@ -14,6 +14,7 @@ from contexts.wrapper import RecyclableContextVar
 from controllers.console.error import UnauthorizedAndForceLogout
 from core.logging.context import init_request_context
 from dify_app import DifyApp
+from enums import DeploymentEdition
 from extensions.ext_socketio import sio
 from services.enterprise.enterprise_service import EnterpriseService
 from services.entities.feature_entities import LicenseStatus
@@ -112,7 +113,7 @@ def create_flask_app_with_configs() -> DifyApp:
         init_request_context()
         RecyclableContextVar.increment_thread_recycles()
 
-        if dify_config.ENTERPRISE_ENABLED:
+        if dify_config.DEPLOYMENT_EDITION == DeploymentEdition.ENTERPRISE:
             surface = _match_license_gated_surface(request.path)
             if surface is not None:
                 try:

api/clients/agent_backend/client.py

@@ -23,6 +23,7 @@ from dify_agent.protocol import (
     CancelRunResponse,
     CreateRunRequest,
     CreateRunResponse,
+    RunCancelledEvent,
     RunEvent,
     RunStatusResponse,
 )
@@ -45,6 +46,15 @@ class AgentBackendRunClient(Protocol):
     def cancel_run(self, run_id: str, request: CancelRunRequest | None = None) -> CancelRunResponse:
         """Request explicit cancellation for one Agent backend run."""
 
+    def cancel_run_and_wait(
+        self,
+        run_id: str,
+        request: CancelRunRequest | None = None,
+        *,
+        after: str | None = None,
+    ) -> RunCancelledEvent:
+        """Request cancellation and wait for runner cleanup to finish."""
+
     def stream_events(
         self,
         run_id: str,
@@ -67,6 +77,15 @@ class _DifyAgentSyncClient(Protocol):
     def cancel_run_sync(self, run_id: str, request: CancelRunRequest | None = None) -> CancelRunResponse:
         """Cancel one run synchronously."""
 
+    def cancel_run_and_wait_sync(
+        self,
+        run_id: str,
+        request: CancelRunRequest | None = None,
+        *,
+        after: str | None = None,
+    ) -> RunCancelledEvent:
+        """Cancel one run and wait for its terminal event synchronously."""
+
     def stream_events_sync(
         self,
         run_id: str,
@@ -92,11 +111,9 @@ class DifyAgentBackendRunClient:
         client: _DifyAgentSyncClient,
         *,
         stream_max_reconnects: int = 3,
-        stream_timeout_seconds: float = 1200,
     ) -> None:
         self.client = client
         self._stream_max_reconnects = stream_max_reconnects
-        self._stream_timeout_seconds = stream_timeout_seconds
 
     def create_run(self, request: CreateRunRequest) -> CreateRunResponse:
         """Create one run through ``POST /runs`` and normalize client exceptions."""
@@ -112,6 +129,19 @@ class DifyAgentBackendRunClient:
         except Exception as exc:
             raise _normalize_dify_agent_error(exc) from exc
 
+    def cancel_run_and_wait(
+        self,
+        run_id: str,
+        request: CancelRunRequest | None = None,
+        *,
+        after: str | None = None,
+    ) -> RunCancelledEvent:
+        """Cancel one run, then wait for the cleanup-complete terminal event."""
+        try:
+            return self.client.cancel_run_and_wait_sync(run_id, request=request, after=after)
+        except Exception as exc:
+            raise _normalize_dify_agent_error(exc) from exc
+
     def stream_events(
         self,
         run_id: str,
@@ -125,7 +155,6 @@ class DifyAgentBackendRunClient:
                 run_id,
                 after=after,
                 max_reconnects=self._stream_max_reconnects,
-                timeout_seconds=self._stream_timeout_seconds,
                 should_stop=should_stop,
             )
         except Exception as exc:

api/clients/agent_backend/errors.py

@@ -8,11 +8,40 @@ classes.
 
 from __future__ import annotations
 
+from http import HTTPStatus
 from typing import Any
 
+from dify_agent.client import DifyAgentHTTPError
 from dify_agent.protocol import RunFailureType
 
 
+def backend_reported_failure(exc: DifyAgentHTTPError) -> bool:
+    """Whether the backend itself reported the failure.
+
+    ``DifyAgentValidationError`` is a ``DifyAgentHTTPError`` that the client also
+    raises when a *successful* response fails DTO validation. Those carry a 2xx
+    status and a Pydantic error list, not a backend-authored error envelope.
+    """
+    return exc.status_code >= HTTPStatus.BAD_REQUEST
+
+
+def backend_error_detail(
+    exc: DifyAgentHTTPError,
+    *,
+    default_code: str = "agent_backend_error",
+) -> tuple[str, str]:
+    """Split an Agent backend HTTP error into the code and message the backend itself sent."""
+    detail = exc.detail
+    if isinstance(detail, dict):
+        code = detail.get("code")
+        message = detail.get("message")
+        return (
+            code if isinstance(code, str) and code else default_code,
+            message if isinstance(message, str) and message else str(exc),
+        )
+    return default_code, str(detail)
+
+
 class AgentBackendError(Exception):
     """Base error for API-side Agent backend integration failures."""
 

api/clients/agent_backend/event_adapter.py

@@ -99,6 +99,8 @@ class AgentBackendRunFailedInternalEvent(AgentBackendInternalEventBase):
     error: str
     error_type: RunFailureType | None = None
     reason: str | None = None
+    session_snapshot: CompositorSessionSnapshot | None = None
+    usage: dict[str, JsonValue] | None = None
 
 
 class AgentBackendRunCancelledInternalEvent(AgentBackendInternalEventBase):
@@ -107,6 +109,8 @@ class AgentBackendRunCancelledInternalEvent(AgentBackendInternalEventBase):
     type: Literal[AgentBackendInternalEventType.RUN_CANCELLED] = AgentBackendInternalEventType.RUN_CANCELLED
     reason: str | None = None
     message: str | None = None
+    session_snapshot: CompositorSessionSnapshot | None = None
+    usage: dict[str, JsonValue] | None = None
 
 
 type AgentBackendInternalEvent = Annotated[
@@ -184,6 +188,8 @@ class AgentBackendRunEventAdapter:
                         error=event.data.error,
                         error_type=event.data.error_type,
                         reason=event.data.reason,
+                        session_snapshot=event.data.session_snapshot,
+                        usage=_agent_run_usage(event.data.usage),
                     )
                 ]
             case RunCancelledEvent():
@@ -193,6 +199,8 @@ class AgentBackendRunEventAdapter:
                         source_event_id=event.id,
                         reason=event.data.reason,
                         message=event.data.message,
+                        session_snapshot=event.data.session_snapshot,
+                        usage=_agent_run_usage(event.data.usage),
                     )
                 ]
         raise TypeError(f"unsupported agent backend run event: {type(event).__name__}")

api/clients/agent_backend/factory.py

@@ -8,6 +8,25 @@ from clients.agent_backend.client import AgentBackendRunClient, DifyAgentBackend
 from clients.agent_backend.fake_client import FakeAgentBackendRunClient, FakeAgentBackendScenario
 
 
+def create_agent_backend_client(
+    *,
+    base_url: str,
+    api_token: str | None = None,
+    stream_timeout: float = 30,
+    timeout: float = 30.0,
+    binding_file_download_timeout: float = 240,
+) -> Client:
+    api_token = api_token.strip() if api_token else None
+    headers = {"Authorization": f"Bearer {api_token}"} if api_token else None
+    return Client(
+        base_url=base_url,
+        stream_timeout=stream_timeout,
+        timeout=timeout,
+        binding_file_download_timeout=binding_file_download_timeout,
+        headers=headers,
+    )
+
+
 def create_agent_backend_run_client(
     *,
     base_url: str | None = None,
@@ -16,18 +35,17 @@ def create_agent_backend_run_client(
     fake_scenario: str | FakeAgentBackendScenario = FakeAgentBackendScenario.SUCCESS,
     stream_read_timeout_seconds: float = 30,
     stream_max_reconnects: int = 3,
-    stream_run_timeout_seconds: float = 1200,
 ) -> AgentBackendRunClient:
     """Create the API-side run client without hiding the ``dify-agent`` protocol."""
     if use_fake:
         return FakeAgentBackendRunClient(scenario=FakeAgentBackendScenario(fake_scenario))
     if base_url is None:
         raise ValueError("base_url is required when creating a real Agent backend client")
-    headers: dict[str, str] = {}
-    if api_token:
-        headers["Authorization"] = f"Bearer {api_token}"
     return DifyAgentBackendRunClient(
-        Client(base_url=base_url, stream_timeout=stream_read_timeout_seconds, headers=headers),
+        create_agent_backend_client(
+            base_url=base_url,
+            api_token=api_token,
+            stream_timeout=stream_read_timeout_seconds,
+        ),
         stream_max_reconnects=stream_max_reconnects,
-        stream_timeout_seconds=stream_run_timeout_seconds,
     )

api/clients/agent_backend/fake_client.py

@@ -18,6 +18,8 @@ from dify_agent.protocol import (
     CreateRunRequest,
     CreateRunResponse,
     DeferredToolCallPayload,
+    RunCancelledEvent,
+    RunCancelledEventData,
     RunEvent,
     RunFailedEvent,
     RunFailedEventData,
@@ -69,6 +71,28 @@ class FakeAgentBackendRunClient:
         del request
         return CancelRunResponse(run_id=run_id, status="cancelled")
 
+    def cancel_run_and_wait(
+        self,
+        run_id: str,
+        request: CancelRunRequest | None = None,
+        *,
+        after: str | None = None,
+    ) -> RunCancelledEvent:
+        """Return a deterministic cleanup-complete cancellation event."""
+        del after
+        request = request or CancelRunRequest()
+        _ = self.cancel_run(run_id, request)
+        return RunCancelledEvent(
+            id="cancel-0",
+            run_id=run_id,
+            created_at=_FIXED_TIME,
+            data=RunCancelledEventData(
+                reason=request.reason,
+                message=request.message,
+                session_snapshot=CompositorSessionSnapshot(layers=[]),
+            ),
+        )
+
     def stream_events(
         self,
         run_id: str,
@@ -133,7 +157,11 @@ class FakeAgentBackendRunClient:
                         id="2-0",
                         run_id=run_id,
                         created_at=_FIXED_TIME,
-                        data=RunFailedEventData(error="fake failure", reason="unit_test"),
+                        data=RunFailedEventData(
+                            error="fake failure",
+                            reason="unit_test",
+                            session_snapshot=CompositorSessionSnapshot(layers=[]),
+                        ),
                     ),
                 )
             case FakeAgentBackendScenario.PAUSED:

api/clients/agent_backend/request_builder.py

@@ -25,11 +25,9 @@ from dify_agent.layers.dify_core_tools import DIFY_CORE_TOOLS_LAYER_TYPE_ID, Dif
 from dify_agent.layers.dify_plugin import (
     DIFY_PLUGIN_LLM_LAYER_TYPE_ID,
     DIFY_PLUGIN_TOOLS_LAYER_TYPE_ID,
-    DifyPluginCredentialValue,
     DifyPluginLLMLayerConfig,
     DifyPluginToolsLayerConfig,
 )
-from dify_agent.layers.drive import DIFY_DRIVE_LAYER_TYPE_ID, DifyDriveLayerConfig
 from dify_agent.layers.execution_context import (
     DIFY_EXECUTION_CONTEXT_LAYER_TYPE_ID,
     DifyExecutionContextLayerConfig,
@@ -57,7 +55,6 @@ AGENT_APP_USER_PROMPT_LAYER_ID = "agent_app_user_prompt"
 DIFY_EXECUTION_CONTEXT_LAYER_ID = "execution_context"
 DIFY_RUNTIME_LAYER_ID = "runtime"
 DIFY_CONFIG_LAYER_ID = "config"
-DIFY_DRIVE_LAYER_ID = "drive"
 DIFY_PLUGIN_TOOLS_LAYER_ID = "tools"
 DIFY_CORE_TOOLS_LAYER_ID = "core_tools"
 DIFY_KNOWLEDGE_BASE_LAYER_ID = "knowledge"
@@ -73,24 +70,10 @@ def _shell_layer_deps() -> dict[str, str]:
     }
 
 
-def _drive_layer_deps() -> dict[str, str]:
-    return {"shell": DIFY_SHELL_LAYER_ID}
-
-
 def _config_layer_deps() -> dict[str, str]:
     return {"shell": DIFY_SHELL_LAYER_ID}
 
 
-def _shell_config_with_drive_ref(
-    shell_config: DifyShellLayerConfig | None,
-    drive_config: DifyDriveLayerConfig | None,
-) -> DifyShellLayerConfig:
-    config = shell_config or DifyShellLayerConfig()
-    if drive_config is None:
-        return config
-    return config.model_copy(update={"agent_stub_drive_ref": drive_config.drive_ref})
-
-
 def _markdown_backtick_fence(text: str) -> str:
     """Choose a fence that will not terminate inside the prompt body."""
     longest_backtick_run = max((len(match.group(0)) for match in re.finditer(r"`+", text)), default=0)
@@ -150,8 +133,8 @@ class AgentBackendModelConfig(BaseModel):
     plugin_id: str
     model_provider: str
     model: str
-    credentials: dict[str, DifyPluginCredentialValue] = Field(default_factory=dict)
     model_settings: dict[str, JsonValue] = Field(default_factory=dict)
+    context_window_tokens: int | None = Field(default=None, gt=0)
 
     model_config: ClassVar[ConfigDict] = ConfigDict(extra="forbid")
 
@@ -225,9 +208,6 @@ class AgentBackendWorkflowNodeRunInput(BaseModel):
     core_tools: DifyCoreToolsLayerConfig | None = None
     knowledge: DifyKnowledgeBaseLayerConfig | None = None
     config_layer_config: DifyConfigLayerConfig | None = None
-    # Drive Skills & Files declaration (dify.drive) — an index the agent pulls
-    # through the back proxy, never inline content.
-    drive_config: DifyDriveLayerConfig | None = None
     # Human-in-the-loop ask_human deferred tool (dify.ask_human). Present only when
     # the Agent Soul configures human involvement; a deferred call ends the run and
     # the workflow pauses via the existing HITL form mechanism (ENG-635).
@@ -274,9 +254,6 @@ class AgentBackendAgentAppRunInput(BaseModel):
     core_tools: DifyCoreToolsLayerConfig | None = None
     knowledge: DifyKnowledgeBaseLayerConfig | None = None
     config_layer_config: DifyConfigLayerConfig | None = None
-    # Drive Skills & Files declaration (dify.drive) — an index the agent pulls
-    # through the back proxy, never inline content.
-    drive_config: DifyDriveLayerConfig | None = None
     # Human-in-the-loop ask_human deferred tool (dify.ask_human). Present only when
     # the Agent Soul configures human involvement (ENG-635).
     ask_human_config: DifyAskHumanLayerConfig | None = None
@@ -308,7 +285,7 @@ class AgentBackendRunRequestBuilder:
         """Build an Agent App conversation-turn run request.
 
         Layer graph: optional Agent Soul system prompt → user prompt →
-        execution context → optional shell / config / drive / history
+        execution context → optional shell / config / history
         (multi-turn) → LLM → optional plugin-direct tools / core-routed tools /
         knowledge search / ask_human / structured output. Mirrors the
         workflow-node layer ordering minus the workflow-job / previous-node
@@ -346,9 +323,7 @@ class AgentBackendRunRequestBuilder:
             ]
         )
 
-        include_shell = (
-            run_input.include_shell or run_input.config_layer_config is not None or run_input.drive_config is not None
-        )
+        include_shell = run_input.include_shell or run_input.config_layer_config is not None
         if include_shell:
             layers.append(
                 RunLayerSpec(
@@ -358,16 +333,15 @@ class AgentBackendRunRequestBuilder:
                     config=DifyRuntimeLayerConfig(backend_binding_ref=run_input.backend_binding_ref),
                 )
             )
-            # Sandboxed bash workspace (dify.shell). It enters before config/drive
-            # so eager pulls materialize content in the same filesystem used by
-            # model commands.
+            # Sandboxed bash workspace (dify.shell). It enters before config so
+            # eager pulls materialize content in the same filesystem used by model commands.
             layers.append(
                 RunLayerSpec(
                     name=DIFY_SHELL_LAYER_ID,
                     type=DIFY_SHELL_LAYER_TYPE_ID,
                     deps=_shell_layer_deps(),
                     metadata=run_input.metadata,
-                    config=_shell_config_with_drive_ref(run_input.shell_config, run_input.drive_config),
+                    config=run_input.shell_config or DifyShellLayerConfig(),
                 )
             )
 
@@ -382,19 +356,6 @@ class AgentBackendRunRequestBuilder:
                 )
             )
 
-        if run_input.drive_config is not None:
-            # Drive Skills & Files declaration (dify.drive): the catalog plus
-            # prompt-mentioned entries eagerly pulled through the shell layer.
-            layers.append(
-                RunLayerSpec(
-                    name=DIFY_DRIVE_LAYER_ID,
-                    type=DIFY_DRIVE_LAYER_TYPE_ID,
-                    deps=_drive_layer_deps(),
-                    metadata=run_input.metadata,
-                    config=run_input.drive_config,
-                )
-            )
-
         if run_input.include_history:
             layers.append(
                 RunLayerSpec(
@@ -414,8 +375,8 @@ class AgentBackendRunRequestBuilder:
                     plugin_id=run_input.model.plugin_id,
                     model_provider=run_input.model.model_provider,
                     model=run_input.model.model,
-                    credentials=run_input.model.credentials,
                     model_settings=_agent_model_settings(run_input.model.model_settings),
+                    context_window_tokens=run_input.model.context_window_tokens,
                 ),
             )
         )
@@ -496,7 +457,7 @@ class AgentBackendRunRequestBuilder:
         """Build a workflow Agent Node run request without defining another wire schema.
 
         Layer graph mirrors the workflow surface: prompts → execution context →
-        optional shell / config / drive / history → LLM → optional
+        optional shell / config / history → LLM → optional
         plugin-direct tools / core-routed tools / knowledge search /
         ask_human / structured output.
         """
@@ -538,9 +499,7 @@ class AgentBackendRunRequestBuilder:
             ]
         )
 
-        include_shell = (
-            run_input.include_shell or run_input.config_layer_config is not None or run_input.drive_config is not None
-        )
+        include_shell = run_input.include_shell or run_input.config_layer_config is not None
         if include_shell:
             layers.append(
                 RunLayerSpec(
@@ -550,16 +509,15 @@ class AgentBackendRunRequestBuilder:
                     config=DifyRuntimeLayerConfig(backend_binding_ref=run_input.backend_binding_ref),
                 )
             )
-            # Sandboxed bash workspace (dify.shell). It enters before drive so
-            # drive can materialize mentioned targets with `dify-agent drive pull`
-            # in the same shell-visible filesystem used by model commands.
+            # Sandboxed bash workspace (dify.shell). It enters before config so
+            # eager pulls materialize content in the same filesystem used by model commands.
             layers.append(
                 RunLayerSpec(
                     name=DIFY_SHELL_LAYER_ID,
                     type=DIFY_SHELL_LAYER_TYPE_ID,
                     deps=_shell_layer_deps(),
                     metadata=run_input.metadata,
-                    config=_shell_config_with_drive_ref(run_input.shell_config, run_input.drive_config),
+                    config=run_input.shell_config or DifyShellLayerConfig(),
                 )
             )
 
@@ -574,19 +532,6 @@ class AgentBackendRunRequestBuilder:
                 )
             )
 
-        if run_input.drive_config is not None:
-            # Drive Skills & Files declaration (dify.drive): the catalog plus
-            # prompt-mentioned entries eagerly pulled through the shell layer.
-            layers.append(
-                RunLayerSpec(
-                    name=DIFY_DRIVE_LAYER_ID,
-                    type=DIFY_DRIVE_LAYER_TYPE_ID,
-                    deps=_drive_layer_deps(),
-                    metadata=run_input.metadata,
-                    config=run_input.drive_config,
-                )
-            )
-
         if run_input.include_history:
             layers.append(
                 RunLayerSpec(
@@ -607,8 +552,8 @@ class AgentBackendRunRequestBuilder:
                         plugin_id=run_input.model.plugin_id,
                         model_provider=run_input.model.model_provider,
                         model=run_input.model.model,
-                        credentials=run_input.model.credentials,
                         model_settings=_agent_model_settings(run_input.model.model_settings),
+                        context_window_tokens=run_input.model.context_window_tokens,
                     ),
                 ),
             ]

api/commands/account.py

@@ -8,6 +8,7 @@ from constants.languages import languages
 from extensions.ext_database import db
 from libs.helper import email as email_validate
 from libs.password import hash_password, password_pattern, valid_password
+from services.account_email import normalize_email
 from services.account_service import AccountService, RegisterService, TenantService
 
 
@@ -20,7 +21,7 @@ def reset_password(email, new_password, password_confirm):
     Reset password of owner account
     Only available in SELF_HOSTED mode
     """
-    if str(new_password).strip() != str(password_confirm).strip():
+    if new_password.strip() != password_confirm.strip():
         click.echo(click.style("Passwords do not match.", fg="red"))
         return
     normalized_email = email.strip().lower()
@@ -33,7 +34,7 @@ def reset_password(email, new_password, password_confirm):
 
     try:
         valid_password(new_password)
-    except:
+    except ValueError:
         click.echo(click.style(f"Invalid password. Must match {password_pattern}", fg="red"))
         return
 
@@ -62,7 +63,7 @@ def reset_email(email, new_email, email_confirm):
     Replace account email
     :return:
     """
-    if str(new_email).strip() != str(email_confirm).strip():
+    if new_email.strip() != email_confirm.strip():
         click.echo(click.style("New emails do not match.", fg="red"))
         return
     normalized_new_email = new_email.strip().lower()
@@ -75,13 +76,14 @@ def reset_email(email, new_email, email_confirm):
 
     try:
         email_validate(normalized_new_email)
-    except:
+    except ValueError:
         click.echo(click.style(f"Invalid email: {new_email}", fg="red"))
         return
 
     with Session(db.engine) as session:
         account = session.merge(account)
         account.email = normalized_new_email
+        account.normalized_email = normalize_email(normalized_new_email)
         session.commit()
     click.echo(click.style("Email updated successfully.", fg="green"))
 

api/commands/retention.py

@@ -10,6 +10,8 @@ import click
 import sqlalchemy as sa
 from sqlalchemy.orm import Session, sessionmaker
 
+from configs import dify_config
+from enums import CloudPlan, DeploymentEdition
 from extensions.ext_database import db
 from libs.datetime_utils import naive_utc_now
 from services.clear_free_plan_tenant_expired_logs import ClearFreePlanTenantExpiredLogs
@@ -126,14 +128,12 @@ def _get_archive_candidate_tenant_ids_by_prefix(
 
 
 def _filter_paid_workflow_archive_tenant_ids(tenant_ids: list[str]) -> tuple[list[str], list[str]]:
-    from configs import dify_config
-    from enums.cloud_plan import CloudPlan
     from services.billing_service import BillingService
 
     tenant_ids = sorted(set(tenant_ids))
     if not tenant_ids:
         return [], []
-    if not dify_config.BILLING_ENABLED:
+    if dify_config.DEPLOYMENT_EDITION != DeploymentEdition.CLOUD:
         return tenant_ids, []
 
     plans = BillingService.get_plan_bulk_with_cache(tenant_ids)
@@ -1462,7 +1462,7 @@ def cleanup_orphaned_draft_variables(
     "--graceful-period",
     default=21,
     show_default=True,
-    help="Graceful period in days after subscription expiration, will be ignored when billing is disabled.",
+    help="Graceful period in days after subscription expiration; ignored outside the Cloud edition.",
 )
 @click.option("--dry-run", is_flag=True, default=False, help="Show messages logs would be cleaned without deleting")
 def clean_expired_messages(
@@ -1514,8 +1514,8 @@ def clean_expired_messages(
                 if from_days_ago <= before_days:
                     raise click.UsageError("--from-days-ago must be greater than --before-days.")
 
-        # Create policy based on billing configuration
-        # NOTE: graceful_period will be ignored when billing is disabled.
+        # Create the policy for the configured deployment edition.
+        # NOTE: graceful_period is ignored outside the Cloud edition.
         policy = create_message_clean_policy(graceful_period_days=graceful_period)
 
         if from_days_ago is not None and before_days is not None:

api/commands/system.py

@@ -6,7 +6,7 @@ from sqlalchemy import delete, select, update
 from sqlalchemy.orm import sessionmaker
 
 from configs import dify_config
-from enums.deployment_edition import DeploymentEdition
+from enums import DeploymentEdition
 from events.app_event import app_was_created
 from extensions.ext_database import db
 from extensions.ext_redis import redis_client

api/configs/app_config.py

@@ -5,7 +5,6 @@ from typing import Any, override
 from pydantic.fields import FieldInfo
 from pydantic_settings import BaseSettings, PydanticBaseSettingsSource, SettingsConfigDict, TomlConfigSettingsSource
 
-from enums.deployment_edition import DeploymentEdition
 from libs.file_utils import search_file_upwards
 
 from .deploy import DeploymentConfig
@@ -117,11 +116,3 @@ class DifyConfig(
                 ),
             ),
         )
-
-    @property
-    def DEPLOYMENT_EDITION(self) -> DeploymentEdition:
-        if self.EDITION == "CLOUD":
-            return DeploymentEdition.CLOUD
-        if self.ENTERPRISE_ENABLED:
-            return DeploymentEdition.ENTERPRISE
-        return DeploymentEdition.COMMUNITY

api/configs/deploy/__init__.py

@@ -1,8 +1,8 @@
-from typing import Literal
-
 from pydantic import Field
 from pydantic_settings import BaseSettings
 
+from enums import DeploymentEdition
+
 
 class DeploymentConfig(BaseSettings):
     """
@@ -25,9 +25,14 @@ class DeploymentConfig(BaseSettings):
         default=False,
     )
 
-    EDITION: Literal["SELF_HOSTED", "CLOUD"] = Field(
-        description="Deployment edition of the application (e.g., 'SELF_HOSTED', 'CLOUD')",
-        default="SELF_HOSTED",
+    DEPLOYMENT_EDITION: DeploymentEdition = Field(
+        description="Product edition of the application.",
+        default=DeploymentEdition.COMMUNITY,
+    )
+
+    INIT_PASSWORD: str = Field(
+        description="Password required before initializing a self-hosted deployment",
+        default="",
     )
 
     DEPLOY_ENV: str = Field(

api/configs/enterprise/__init__.py

@@ -8,12 +8,6 @@ class EnterpriseFeatureConfig(BaseSettings):
     **Before using, please contact business@dify.ai by email to inquire about licensing matters.**
     """
 
-    ENTERPRISE_ENABLED: bool = Field(
-        description="Enable or disable enterprise-level features."
-        "Before using, please contact business@dify.ai by email to inquire about licensing matters.",
-        default=False,
-    )
-
     WEBAPP_PUBLIC_ACCESS_ENABLED: bool = Field(
         description="Whether admins are allowed to set a webapp's access mode to public (anyone with the link, "
         "no auth). Disable in security-sensitive on-prem deployments.",
@@ -59,7 +53,7 @@ class EnterpriseTelemetryConfig(BaseSettings):
     """
 
     ENTERPRISE_TELEMETRY_ENABLED: bool = Field(
-        description="Enable enterprise telemetry collection (also requires ENTERPRISE_ENABLED=true).",
+        description="Enable enterprise telemetry collection for enterprise deployments.",
         default=False,
     )
 
@@ -89,11 +83,6 @@ class EnterpriseTelemetryConfig(BaseSettings):
         default=False,
     )
 
-    ENTERPRISE_SERVICE_NAME: str = Field(
-        description="Service name for OTEL resource.",
-        default="dify",
-    )
-
     ENTERPRISE_OTEL_SAMPLING_RATE: float = Field(
         description="Sampling rate for enterprise traces (0.0 to 1.0, default 1.0 = 100%).",
         default=1.0,

api/configs/extra/__init__.py

@@ -1,8 +1,10 @@
 from configs.extra.agent_backend_config import AgentBackendConfig
 from configs.extra.archive_config import ArchiveStorageConfig
 from configs.extra.knowledge_fs_config import KnowledgeFSConfig
+from configs.extra.logstore_config import LogStoreConfig
 from configs.extra.notion_config import NotionConfig
 from configs.extra.sentry_config import SentryConfig
+from configs.extra.turnstile_config import TurnstileConfig
 
 
 class ExtraServiceConfig(
@@ -10,7 +12,9 @@ class ExtraServiceConfig(
     AgentBackendConfig,
     ArchiveStorageConfig,
     KnowledgeFSConfig,
+    LogStoreConfig,
     NotionConfig,
     SentryConfig,
+    TurnstileConfig,
 ):
     pass

api/configs/extra/agent_backend_config.py

@@ -13,7 +13,7 @@ class AgentBackendConfig(BaseSettings):
     )
 
     AGENT_BACKEND_API_TOKEN: str | None = Field(
-        description="Bearer token for authenticating with the Agent backend /runs API.",
+        description="Bearer token for authenticating with the Agent backend control-plane API.",
         default=None,
     )
 
@@ -37,9 +37,17 @@ class AgentBackendConfig(BaseSettings):
         default=3,
     )
 
-    AGENT_BACKEND_RUN_TIMEOUT_SECONDS: PositiveFloat = Field(
-        description="Total deadline for one Agent backend run event stream.",
-        default=1200,
+    AGENT_BACKEND_HOME_SNAPSHOT_TIMEOUT_SECONDS: PositiveFloat = Field(
+        description=(
+            "Client timeout for Agent backend calls that may carry a Home Snapshot transfer: "
+            "snapshot capture and delete, and Execution Binding creation that restores one. "
+        ),
+        default=45.0,
+    )
+
+    AGENT_BACKEND_BINDING_FILE_DOWNLOAD_TIMEOUT_SECONDS: PositiveFloat = Field(
+        description="Client timeout for converting a Binding file to a ToolFile through the Agent backend.",
+        default=240,
     )
 
     AGENT_SHELL_ENABLED: bool = Field(

api/configs/extra/logstore_config.py

@@ -0,0 +1,11 @@
+from pydantic_settings import BaseSettings
+
+
+class LogStoreConfig(BaseSettings):
+    """Migration controls for repositories backed by Aliyun LogStore."""
+
+    LOGSTORE_DUAL_WRITE_ENABLED: bool = False
+
+    # Keep workflow graphs in LogStore by default. Deployments may disable this
+    # while migrating large graph payloads to another persistence owner.
+    LOGSTORE_ENABLE_PUT_GRAPH_FIELD: bool = True

api/configs/extra/turnstile_config.py

@@ -0,0 +1,41 @@
+from pydantic import Field, SecretStr, field_validator
+from pydantic_settings import BaseSettings
+
+
+class TurnstileConfig(BaseSettings):
+    """Server-side Cloudflare Turnstile settings for Cloud sign-in."""
+
+    TURNSTILE_SECRET_KEY: SecretStr | None = Field(
+        default=None,
+        description="Secret key used to validate Cloudflare Turnstile tokens.",
+    )
+    TURNSTILE_ALLOWED_HOSTNAMES: str = Field(
+        default="",
+        description="Comma-separated parent or exact hostnames accepted from Turnstile.",
+    )
+    TURNSTILE_EMAIL_CODE_VERIFY_REQUIRED: bool = Field(
+        default=False,
+        description=(
+            "Require a separate Turnstile challenge when verifying email login codes on Dify Cloud. "
+            "Enable after the compatible web client has been deployed."
+        ),
+    )
+
+    @field_validator("TURNSTILE_SECRET_KEY", mode="before")
+    @classmethod
+    def normalize_secret_key(cls, value: object) -> object:
+        if isinstance(value, SecretStr):
+            normalized = value.get_secret_value().strip()
+            return SecretStr(normalized) if normalized else None
+        if isinstance(value, str):
+            normalized = value.strip()
+            return normalized or None
+        return value
+
+    @property
+    def TURNSTILE_ALLOWED_HOSTNAME_SET(self) -> frozenset[str]:
+        return frozenset(
+            hostname.strip().lower().strip(".")
+            for hostname in self.TURNSTILE_ALLOWED_HOSTNAMES.split(",")
+            if hostname.strip().strip(".")
+        )

api/configs/feature/__init__.py

@@ -12,6 +12,7 @@ from pydantic import (
     PositiveInt,
     computed_field,
     field_validator,
+    model_validator,
 )
 from pydantic_settings import BaseSettings
 
@@ -83,7 +84,7 @@ class AppExecutionConfig(BaseSettings):
 
     APP_MAX_EXECUTION_TIME: PositiveInt = Field(
         description="Maximum allowed execution time for the application in seconds",
-        default=1200,
+        default=3600,
     )
     APP_DEFAULT_ACTIVE_REQUESTS: NonNegativeInt = Field(
         description="Default number of concurrent active requests per app (0 for unlimited)",
@@ -810,17 +811,6 @@ class ModelLoadBalanceConfig(BaseSettings):
     )
 
 
-class BillingConfig(BaseSettings):
-    """
-    Configuration for platform billing features
-    """
-
-    BILLING_ENABLED: bool = Field(
-        description="Enable or disable billing functionality",
-        default=False,
-    )
-
-
 class UpdateConfig(BaseSettings):
     """
     Configuration for application update checks
@@ -900,7 +890,7 @@ class WorkflowConfig(BaseSettings):
 
     WORKFLOW_MAX_EXECUTION_TIME: PositiveInt = Field(
         description="Maximum execution time in seconds for a single workflow",
-        default=1200,
+        default=3600,
     )
 
     WORKFLOW_CALL_MAX_DEPTH: PositiveInt = Field(
@@ -1301,6 +1291,13 @@ class DataSetConfig(BaseSettings):
     )
 
 
+class SkillConfig(BaseSettings):
+    ENABLE_SKILL: bool = Field(
+        description="Enable or disable Skill feature entry points",
+        default=True,
+    )
+
+
 class WorkspaceConfig(BaseSettings):
     """
     Configuration for workspace management
@@ -1335,10 +1332,28 @@ class MultiModalTransferConfig(BaseSettings):
     )
 
 
+class NewAgentBetaConfig(BaseSettings):
+    NEW_AGENT_BETA_ACTIVITY_START_AT: datetime | None = Field(
+        description="New Agent Beta Publish window start in RFC3339 UTC (inclusive)",
+        default=None,
+    )
+    NEW_AGENT_BETA_ACTIVITY_END_AT: datetime | None = Field(
+        description="New Agent Beta Publish window end in RFC3339 UTC (exclusive)",
+        default=None,
+    )
+
+
 class OpsTraceConfig(BaseSettings):
+    OPS_TRACE_UNIFIED_ENABLED: bool = Field(
+        description="Enable unified ops tracing for providers registered in the unified registry.",
+        default=False,
+    )
+
+    # Include scheduling and export grace after the parent workflow's maximum execution time.
+    # Recommended: max_retries >= ceil((WORKFLOW_MAX_EXECUTION_TIME + grace_seconds) / delay_seconds).
     OPS_TRACE_RETRYABLE_DISPATCH_MAX_RETRIES: PositiveInt = Field(
         description="Maximum retry attempts for transient ops trace provider dispatch failures.",
-        default=60,
+        default=780,
     )
 
     OPS_TRACE_RETRYABLE_DISPATCH_DELAY_SECONDS: PositiveInt = Field(
@@ -1346,6 +1361,20 @@ class OpsTraceConfig(BaseSettings):
         default=5,
     )
 
+    OPS_TRACE_PARENT_CONTEXT_TTL_SECONDS: PositiveInt = Field(
+        description="Retention in seconds for unified tracing parent contexts.",
+        default=3900,
+    )
+
+    @model_validator(mode="after")
+    def validate_parent_context_retention(self) -> "OpsTraceConfig":
+        if not self.OPS_TRACE_UNIFIED_ENABLED:
+            return self
+        retry_window = self.OPS_TRACE_RETRYABLE_DISPATCH_MAX_RETRIES * self.OPS_TRACE_RETRYABLE_DISPATCH_DELAY_SECONDS
+        if retry_window > self.OPS_TRACE_PARENT_CONTEXT_TTL_SECONDS:
+            raise ValueError("OPS_TRACE_PARENT_CONTEXT_TTL_SECONDS must cover the retry window")
+        return self
+
 
 class CeleryBeatConfig(BaseSettings):
     CELERY_BEAT_SCHEDULER_TIME: int = Field(
@@ -1355,6 +1384,18 @@ class CeleryBeatConfig(BaseSettings):
 
 
 class CeleryScheduleTasksConfig(BaseSettings):
+    ENABLE_CONVERSATION_CLEANUP_TASK: bool = Field(
+        description="Enable periodic recovery of soft-deleted conversation cleanup",
+        default=True,
+    )
+    CONVERSATION_CLEANUP_TASK_INTERVAL: PositiveInt = Field(
+        description="Soft-deleted conversation cleanup recovery interval in minutes",
+        default=5,
+    )
+    CONVERSATION_CLEANUP_BATCH_SIZE: PositiveInt = Field(
+        description="Maximum soft-deleted conversations dispatched per cleanup sweep",
+        default=100,
+    )
     ENABLE_CLEAN_EMBEDDING_CACHE_TASK: bool = Field(
         description="Enable clean embedding cache task",
         default=False,
@@ -1537,6 +1578,10 @@ class LoginConfig(BaseSettings):
         description="expiry time in minutes for email code login token",
         default=5,
     )
+    EMAIL_CODE_LOGIN_MAX_ATTEMPTS: PositiveInt = Field(
+        description="maximum number of verification attempts for an email code login challenge",
+        default=5,
+    )
     ALLOW_REGISTER: bool = Field(
         description="whether to enable register",
         default=False,
@@ -1621,7 +1666,6 @@ class FeatureConfig(
     # place the configs in alphabet order
     AppExecutionConfig,
     AuthConfig,  # Changed from OAuthConfig to AuthConfig
-    BillingConfig,
     CodeExecutionSandboxConfig,
     CreatorsPlatformConfig,
     TriggerConfig,
@@ -1641,12 +1685,14 @@ class FeatureConfig(
     ModelLoadBalanceConfig,
     ModerationConfig,
     MultiModalTransferConfig,
+    NewAgentBetaConfig,
     OpsTraceConfig,
     PositionConfig,
     RagEtlConfig,
     RepositoryConfig,
     SandboxExpiredRecordsCleanConfig,
     SecurityConfig,
+    SkillConfig,
     TenantIsolatedTaskQueueConfig,
     ToolConfig,
     UpdateConfig,

api/configs/feature/hosted_service/__init__.py

@@ -451,6 +451,12 @@ class HostedFetchAppTemplateConfig(BaseSettings):
         default="https://tmpl.dify.ai",
     )
 
+    HOSTED_FETCH_APP_TEMPLATES_CACHE_TTL: int = Field(
+        description="TTL in seconds for caching remote app template HTTP responses. 0 disables caching.",
+        default=600,
+        ge=0,
+    )
+
 
 class HostedFetchPipelineTemplateConfig(BaseSettings):
     """

api/configs/middleware/__init__.py

@@ -335,22 +335,6 @@ class CeleryConfig(DatabaseConfig):
         return self.CELERY_BROKER_URL.startswith("rediss://") if self.CELERY_BROKER_URL else False
 
 
-class InternalTestConfig(BaseSettings):
-    """
-    Configuration settings for Internal Test
-    """
-
-    AWS_SECRET_ACCESS_KEY: str | None = Field(
-        description="Internal test AWS secret access key",
-        default=None,
-    )
-
-    AWS_ACCESS_KEY_ID: str | None = Field(
-        description="Internal test AWS access key ID",
-        default=None,
-    )
-
-
 class DatasetQueueMonitorConfig(BaseSettings):
     """
     Configuration settings for Dataset Queue Monitor
@@ -416,7 +400,6 @@ class MiddlewareConfig(
     WeaviateConfig,
     ElasticsearchConfig,
     CouchbaseConfig,
-    InternalTestConfig,
     VikingDBConfig,
     UpstashConfig,
     TidbOnQdrantConfig,

api/configs/remote_settings_sources/apollo/client.py

@@ -167,8 +167,8 @@ class ApolloClient:
                 old_value = old_kv.get(key)
                 if old_value is None:
                     self._change_listener("add", namespace, key, new_value)
-        except BaseException as e:
-            logger.warning(str(e))
+        except BaseException:
+            logger.warning("Apollo config change listener failed", exc_info=True)
 
     def _path_checker(self) -> None:
         if not os.path.isdir(self._cache_file_path):
@@ -235,8 +235,8 @@ class ApolloClient:
                     return
             else:
                 logger.warning("Sleep...")
-        except Exception as e:
-            logger.warning(str(e))
+        except Exception:
+            logger.warning("Apollo long polling failed", exc_info=True)
 
     def _get_net_and_set_local(self, namespace: str, n_id: int, call_change: bool = False) -> None:
         namespace_data = self.get_json_from_net(namespace)

api/constants/__init__.py

@@ -36,6 +36,7 @@ _UNSTRUCTURED_DOCUMENT_EXTENSION_BASE: frozenset[str] = frozenset(
         "pptx",
         "xml",
         "epub",
+        "odt",
     )
 )
 _DEFAULT_DOCUMENT_EXTENSION_BASE: frozenset[str] = frozenset(
@@ -53,6 +54,7 @@ _DEFAULT_DOCUMENT_EXTENSION_BASE: frozenset[str] = frozenset(
         "csv",
         "vtt",
         "properties",
+        "odt",
     )
 )
 

api/constants/recommended_apps.json

@@ -166,6 +166,7 @@
                     "categories": ["Workflow"],
                     "copyright": null,
                     "description": "Based on users' choice, retrieve external knowledge to more accurately summarize articles.",
+                    "is_learn_dify": true,
                     "is_listed": true,
                     "position": 5,
                     "privacy_policy": null
@@ -294,6 +295,7 @@
                     "categories": ["Workflow"],
                     "copyright": null,
                     "description": "Basic Workflow Template, a chatbot capable of identifying intents alongside with a knowledge base.",
+                    "is_learn_dify": true,
                     "is_listed": true,
                     "position": 4,
                     "privacy_policy": null
@@ -326,6 +328,7 @@
                     "categories": ["Workflow"],
                     "copyright": null,
                     "description": "Basic Workflow Template, A chatbot with a knowledge base. ",
+                    "is_learn_dify": true,
                     "is_listed": true,
                     "position": 4,
                     "privacy_policy": null

api/constants/tts_auto_play_timeout.py

@@ -1,4 +0,0 @@
-TTS_AUTO_PLAY_TIMEOUT = 5
-
-# sleep 20 ms ( 40ms => 1280 byte audio file,20ms => 640 byte audio file)
-TTS_AUTO_PLAY_YIELD_CPU_TIME = 0.02

api/controllers/common/agent_app_parameters.py

@@ -1,60 +0,0 @@
-from typing import Any
-
-from sqlalchemy import select
-from sqlalchemy.orm import Session
-
-from core.agent.publish_visibility import agent_has_workflow_callable_active_snapshot
-from core.app.apps.agent_app.app_feature_projection import merge_agent_app_features
-from core.app.apps.agent_app.app_variable_projection import agent_app_variables_to_user_input_form
-from core.app.apps.agent_app.errors import AgentAppGeneratorError, AgentAppNotPublishedError
-from models.agent import Agent, AgentConfigSnapshot, AgentStatus
-from models.agent_config_entities import AgentSoulConfig
-from models.model import App, load_annotation_reply_config
-
-
-def get_published_agent_app_feature_dict_and_user_input_form(
-    app_model: App,
-    *,
-    session: Session,
-) -> tuple[dict[str, Any], list[dict[str, Any]]]:
-    """Return public Agent App parameters backed by the published Agent Soul."""
-    app_model_config = app_model.app_model_config_with_session(session=session)
-
-    agent_id = app_model.bound_agent_id
-    if not agent_id:
-        raise AgentAppGeneratorError("Agent App has no bound Agent")
-
-    agent = session.scalar(
-        select(Agent)
-        .where(
-            Agent.tenant_id == app_model.tenant_id,
-            Agent.id == agent_id,
-            Agent.status == AgentStatus.ACTIVE,
-        )
-        .limit(1)
-    )
-    if agent is None:
-        raise AgentAppGeneratorError("Agent App has no bound Agent")
-    if not agent_has_workflow_callable_active_snapshot(session=session, agent=agent):
-        raise AgentAppNotPublishedError("Agent has not been published")
-
-    snapshot = session.scalar(
-        select(AgentConfigSnapshot)
-        .where(
-            AgentConfigSnapshot.tenant_id == app_model.tenant_id,
-            AgentConfigSnapshot.agent_id == agent.id,
-            AgentConfigSnapshot.id == agent.active_config_snapshot_id,
-        )
-        .limit(1)
-    )
-    if snapshot is None:
-        raise AgentAppGeneratorError("Agent published version not found")
-
-    agent_soul = AgentSoulConfig.model_validate(snapshot.config_snapshot_dict)
-    annotation_reply = load_annotation_reply_config(session, app_model.id) if app_model_config else None
-    features_dict = merge_agent_app_features(
-        agent_soul=agent_soul,
-        app_model_config=app_model_config,
-        annotation_reply=annotation_reply,
-    )
-    return features_dict, agent_app_variables_to_user_input_form(agent_soul.app_variables)

api/controllers/common/fields.py

@@ -126,12 +126,6 @@ class UsageCountResponse(ResponseModel):
     count: int
 
 
-class IndexInfoResponse(ResponseModel):
-    welcome: str
-    api_version: str
-    server_version: str
-
-
 class AvatarUrlResponse(ResponseModel):
     avatar_url: str
 

api/controllers/common/wraps.py

@@ -152,7 +152,7 @@ def _extract_resource_id(
     if resource_type == RBACResourceScope.APP:
         app_id = matched_args.get("app_id")
         if app_id:
-            return str(app_id)
+            return str(app_id)  # pyrefly: ignore[unnecessary-type-conversion]
 
         agent_id = matched_args.get("agent_id")
         if agent_id:
@@ -161,7 +161,7 @@ def _extract_resource_id(
 
         resource_id = matched_args.get("resource_id")
         if resource_id:
-            return str(resource_id)
+            return str(resource_id)  # pyrefly: ignore[unnecessary-type-conversion]
         raise ValueError("Missing app_id in request path")
 
     if resource_type == RBACResourceScope.DATASET:
@@ -171,9 +171,11 @@ def _extract_resource_id(
 
         pipeline_id = matched_args.get("pipeline_id")
         if pipeline_id:
-            dataset = db.session.scalar(select(Dataset).where(Dataset.pipeline_id == str(pipeline_id)))
+            dataset = db.session.scalar(
+                select(Dataset).where(Dataset.pipeline_id == str(pipeline_id), Dataset.tenant_id == tenant_id)
+            )
             if not dataset:
                 raise NotFound("Dataset not found for pipeline")
-            return str(dataset.id)
+            return str(dataset.id)  # pyrefly: ignore[unnecessary-type-conversion]
         raise ValueError("Missing dataset_id or pipeline_id in request path")
     raise ValueError(f"Unknown resource_type: {resource_type}")

api/controllers/console/__init__.py

@@ -1,9 +1,11 @@
+from http import HTTPStatus
 from importlib import import_module
 
-from flask import Blueprint
+from flask import Blueprint, current_app, got_request_exception
 from flask_restx import Namespace
 
 from libs.external_api import ExternalApi
+from machinery.errors import ActiveWorkspaceRequiredError
 
 bp = Blueprint("console", __name__, url_prefix="/console/api")
 
@@ -14,6 +16,19 @@ api = ExternalApi(
     description="Console management APIs for app configuration, monitoring, and administration",
 )
 
+
+@api.errorhandler(ActiveWorkspaceRequiredError)
+def _handle_active_workspace_required_error(error: ActiveWorkspaceRequiredError):
+    """Map a broken Console admission invariant without exposing internal details."""
+    got_request_exception.send(current_app, exception=error)
+    status = HTTPStatus.INTERNAL_SERVER_ERROR
+    return {
+        "code": error.error_code,
+        "message": status.phrase,
+        "status": status.value,
+    }, status.value
+
+
 console_ns = Namespace("console", description="Console management API operations", path="/")
 
 RESOURCE_MODULES = (
@@ -41,10 +56,9 @@ from . import (
     knowledge_fs_proxy,
     notification,
     onboarding,
-    ping,
     setup,
     spec,
-    version,
+    system,
     workflow_run_archive,
 )
 from .agent import composer as agent_composer
@@ -58,7 +72,6 @@ from .app import (
     agent_app_feature,
     agent_app_sandbox,
     agent_config_inspector,
-    agent_drive_inspector,
     annotation,
     app,
     audio,
@@ -144,6 +157,7 @@ from .workspace import (
     models,
     plugin,
     rbac,
+    skills,
     snippets,
     tool_providers,
     trigger_providers,
@@ -162,7 +176,6 @@ __all__ = [
     "agent_app_sandbox",
     "agent_composer",
     "agent_config_inspector",
-    "agent_drive_inspector",
     "agent_providers",
     "agent_roster",
     "annotation",
@@ -213,7 +226,6 @@ __all__ = [
     "onboarding",
     "ops_trace",
     "parameter",
-    "ping",
     "plugin",
     "rag_pipeline",
     "rag_pipeline_datasets",
@@ -225,17 +237,18 @@ __all__ = [
     "saved_message",
     "setup",
     "site",
+    "skills",
     "snippet_workflow",
     "snippet_workflow_draft_variable",
     "snippets",
     "socketio_workflow",
     "spec",
     "statistic",
+    "system",
     "tags",
     "tool_providers",
     "trial",
     "trigger_providers",
-    "version",
     "website",
     "workflow",
     "workflow_app_log",

api/controllers/console/agent/composer.py

@@ -182,14 +182,7 @@ class WorkflowAgentComposerValidateApi(Resource):
         AgentComposerService.validate_knowledge_datasets(
             session=session, tenant_id=tenant_id, agent_soul=req_data.agent_soul
         )
-        findings = AgentComposerService.collect_validation_findings(
-            session=session,
-            tenant_id=tenant_id,
-            payload=req_data,
-            agent_id=AgentComposerService.resolve_workflow_node_agent_id(
-                session=session, tenant_id=tenant_id, app_id=app_model.id, node_id=node_id
-            ),
-        )
+        findings = AgentComposerService.collect_validation_findings(payload=req_data)
         return dump_response(AgentComposerValidateResponse, {"result": "success", "errors": [], **findings})
 
 
@@ -284,7 +277,7 @@ class WorkflowAgentComposerSaveToRosterApi(Resource):
 
 def _require_snippet_app_id(*, session: Session, tenant_id: str, snippet_id: UUID) -> str:
     snippet = SnippetService(session=session).get_snippet_by_id(
-        snippet_id=str(snippet_id),
+        snippet_id=str(snippet_id),  # pyrefly: ignore[unnecessary-type-conversion]
         tenant_id=tenant_id,
     )
     if snippet is None:
@@ -413,22 +406,12 @@ class SnippetAgentComposerValidateApi(Resource):
     @with_session(write=False)
     @model_validate(ComposerSavePayload)
     def post(self, req_data: ComposerSavePayload, session: Session, tenant_id: str, snippet_id: UUID, node_id: str):
-        app_id = _require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id)
+        _require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id)
         ComposerConfigValidator.validate_publish_payload(req_data)
         AgentComposerService.validate_knowledge_datasets(
             session=session, tenant_id=tenant_id, agent_soul=req_data.agent_soul
         )
-        findings = AgentComposerService.collect_validation_findings(
-            session=session,
-            tenant_id=tenant_id,
-            payload=req_data,
-            agent_id=AgentComposerService.resolve_workflow_node_agent_id(
-                session=session,
-                tenant_id=tenant_id,
-                app_id=app_id,
-                node_id=node_id,
-            ),
-        )
+        findings = AgentComposerService.collect_validation_findings(payload=req_data)
         return dump_response(AgentComposerValidateResponse, {"result": "success", "errors": [], **findings})
 
 
@@ -580,12 +563,7 @@ class AgentComposerValidateApi(Resource):
         AgentComposerService.validate_knowledge_datasets(
             session=session, tenant_id=tenant_id, agent_soul=req_data.agent_soul
         )
-        findings = AgentComposerService.collect_validation_findings(
-            session=session,
-            tenant_id=tenant_id,
-            payload=req_data,
-            agent_id=str(agent_id),
-        )
+        findings = AgentComposerService.collect_validation_findings(payload=req_data)
         return dump_response(AgentComposerValidateResponse, {"result": "success", "errors": [], **findings})
 
 

api/controllers/console/agent/roster.py

@@ -3,7 +3,7 @@ from uuid import UUID
 from flask import abort, request
 from flask_restx import Resource
 from pydantic import AliasChoices, BaseModel, Field, field_validator
-from sqlalchemy import func, select
+from sqlalchemy import func, or_, select
 from sqlalchemy.orm import Session
 
 from controllers.common.schema import (
@@ -484,12 +484,13 @@ def _resolve_agent_runtime_app_model(session: Session, *, tenant_id: str, agent_
     return _agent_roster_service(session).get_agent_runtime_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
 
 
-def _agent_api_key_count(session: Session, app_id: str) -> int:
+def _agent_api_key_count(session: Session, app_model: App) -> int:
     return (
         session.scalar(
             select(func.count(ApiToken.id)).where(
+                or_(ApiToken.tenant_id == app_model.tenant_id, ApiToken.tenant_id.is_(None)),
                 ApiToken.type == ApiTokenType.APP,
-                ApiToken.app_id == app_id,
+                ApiToken.app_id == app_model.id,
             )
         )
         or 0
@@ -521,7 +522,7 @@ def _serialize_agent_api_access(session: Session, app_model: App) -> dict:
         meta_endpoint=f"{base_url}/meta",
         api_rpm=app_model.api_rpm or 0,
         api_rph=app_model.api_rph or 0,
-        api_key_count=_agent_api_key_count(session, str(app_model.id)),
+        api_key_count=_agent_api_key_count(session, app_model),
     )
     return response.model_dump(mode="json")
 
@@ -598,6 +599,7 @@ class AgentAppListApi(Resource):
 
     @console_ns.expect(console_ns.models[AgentAppCreatePayload.__name__])
     @console_ns.response(201, "Agent app created successfully", console_ns.models[AgentAppDetailWithSite.__name__])
+    @console_ns.response(409, "Agent name already exists")
     @console_ns.response(403, "Insufficient permissions")
     @console_ns.response(400, "Invalid request parameters")
     @setup_required
@@ -940,6 +942,8 @@ class AgentApiKeyListApi(BaseApiKeyListResource):
     @console_ns.response(200, "Agent service API keys", console_ns.models[ApiKeyList.__name__])
     @rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
     @with_current_tenant_id
+    @edit_permission_required
+    @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
     @with_session(write=False)
     def get(self, session: Session, tenant_id: str, agent_id: UUID) -> dict[str, object]:
         app_model = _resolve_agent_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
@@ -1069,7 +1073,7 @@ class AgentLogMessagesApi(Resource):
             payload = _agent_observability_service(session).list_log_messages(
                 app=app_model,
                 agent_id=str(agent_id),
-                conversation_id=str(conversation_id),
+                conversation_id=str(conversation_id),  # pyrefly: ignore[unnecessary-type-conversion]
                 params=AgentLogQueryParams(
                     page=query.page,
                     limit=query.limit,

api/controllers/console/apikey.py

@@ -5,7 +5,7 @@ import flask_restx
 from flask_restx import Resource
 from flask_restx._http import HTTPStatus
 from pydantic import field_validator
-from sqlalchemy import delete, func, select
+from sqlalchemy import func, or_, select
 from sqlalchemy.orm import Session
 from werkzeug.exceptions import Forbidden
 
@@ -89,7 +89,9 @@ class BaseApiKeyListResource(Resource):
         _get_resource(resource_id, current_tenant_id, self.resource_model, session=session)
         keys = session.scalars(
             select(ApiToken).where(
-                ApiToken.type == self.resource_type, getattr(ApiToken, self.resource_id_field) == resource_id
+                or_(ApiToken.tenant_id == current_tenant_id, ApiToken.tenant_id.is_(None)),
+                ApiToken.type == self.resource_type,
+                getattr(ApiToken, self.resource_id_field) == resource_id,
             )
         ).all()
         return ApiKeyList.model_validate({"data": keys}, from_attributes=True)
@@ -110,7 +112,9 @@ class BaseApiKeyListResource(Resource):
         current_key_count: int = (
             session.scalar(
                 select(func.count(ApiToken.id)).where(
-                    ApiToken.type == self.resource_type, getattr(ApiToken, self.resource_id_field) == resource_id
+                    or_(ApiToken.tenant_id == current_tenant_id, ApiToken.tenant_id.is_(None)),
+                    ApiToken.type == self.resource_type,
+                    getattr(ApiToken, self.resource_id_field) == resource_id,
                 )
             )
             or 0
@@ -172,6 +176,7 @@ class BaseApiKeyResource(Resource):
         key = session.scalar(
             select(ApiToken)
             .where(
+                or_(ApiToken.tenant_id == current_tenant_id, ApiToken.tenant_id.is_(None)),
                 getattr(ApiToken, self.resource_id_field) == resource_id,
                 ApiToken.type == self.resource_type,
                 ApiToken.id == api_key_id,
@@ -187,7 +192,7 @@ class BaseApiKeyResource(Resource):
         assert key is not None  # nosec - for type checker only
         ApiTokenCache.delete(key.token, key.type)
 
-        session.execute(delete(ApiToken).where(ApiToken.id == api_key_id))
+        session.delete(key)
         session.commit()
 
 
@@ -198,13 +203,15 @@ class AppApiKeyListResource(BaseApiKeyListResource):
     @console_ns.doc(params={"resource_id": "App ID"})
     @console_ns.response(200, "API keys retrieved successfully", console_ns.models[ApiKeyList.__name__])
     @with_current_tenant_id
+    @edit_permission_required
+    @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
     @agent_manage_required_for_agent_app
     @with_session(write=False)
     def get(self, session: Session, current_tenant_id: str, resource_id: UUID) -> dict[str, object]:
         """Get all API keys for an app"""
         return dump_response(
             ApiKeyList,
-            self._get_api_key_list(str(resource_id), current_tenant_id, session=session),
+            self._get_api_key_list(str(resource_id), current_tenant_id, session=session),  # pyrefly: ignore[unnecessary-type-conversion]
         )
 
     @console_ns.doc("create_app_api_key")
@@ -221,7 +228,7 @@ class AppApiKeyListResource(BaseApiKeyListResource):
         """Create a new API key for an app"""
         return dump_response(
             ApiKeyItem,
-            self._create_api_key(str(resource_id), current_tenant_id, session=session),
+            self._create_api_key(str(resource_id), current_tenant_id, session=session),  # pyrefly: ignore[unnecessary-type-conversion]
         ), 201
 
     resource_type = ApiTokenType.APP
@@ -251,7 +258,7 @@ class AppApiKeyResource(BaseApiKeyResource):
     ) -> tuple[str, int]:
         """Delete an API key for an app"""
         self._delete_api_key(
-            str(resource_id),
+            str(resource_id),  # pyrefly: ignore[unnecessary-type-conversion]
             str(api_key_id),
             current_tenant_id,
             current_user,
@@ -271,12 +278,14 @@ class DatasetApiKeyListResource(BaseApiKeyListResource):
     @console_ns.doc(params={"resource_id": "Dataset ID"})
     @console_ns.response(200, "API keys retrieved successfully", console_ns.models[ApiKeyList.__name__])
     @with_current_tenant_id
+    @edit_permission_required
+    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_API_KEY_MANAGE)
     @with_session(write=False)
     def get(self, session: Session, current_tenant_id: str, resource_id: UUID) -> dict[str, object]:
         """Get all API keys for a dataset"""
         return dump_response(
             ApiKeyList,
-            self._get_api_key_list(str(resource_id), current_tenant_id, session=session),
+            self._get_api_key_list(str(resource_id), current_tenant_id, session=session),  # pyrefly: ignore[unnecessary-type-conversion]
         )
 
     @console_ns.doc("create_dataset_api_key")
@@ -292,7 +301,7 @@ class DatasetApiKeyListResource(BaseApiKeyListResource):
         """Create a new API key for a dataset"""
         return dump_response(
             ApiKeyItem,
-            self._create_api_key(str(resource_id), current_tenant_id, session=session),
+            self._create_api_key(str(resource_id), current_tenant_id, session=session),  # pyrefly: ignore[unnecessary-type-conversion]
         ), 201
 
     resource_type = ApiTokenType.DATASET
@@ -321,7 +330,7 @@ class DatasetApiKeyResource(BaseApiKeyResource):
     ) -> tuple[str, int]:
         """Delete an API key for a dataset"""
         self._delete_api_key(
-            str(resource_id),
+            str(resource_id),  # pyrefly: ignore[unnecessary-type-conversion]
             str(api_key_id),
             current_tenant_id,
             current_user,

api/controllers/console/app/agent.py

@@ -1,21 +1,12 @@
 from typing import Any
-from uuid import UUID
 
-from flask import request
 from flask_restx import Resource
 from pydantic import BaseModel, Field, field_validator
-from sqlalchemy import select
 from sqlalchemy.orm import Session
 
-from controllers.common.schema import (
-    query_params_from_model,
-    query_params_from_request,
-    register_response_schema_models,
-    register_schema_models,
-)
+from controllers.common.schema import query_params_from_model, register_response_schema_models
 from controllers.common.session import with_session
 from controllers.console import console_ns
-from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
 from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import (
     RBACPermission,
@@ -24,41 +15,13 @@ from controllers.console.wraps import (
     model_validate,
     rbac_permission_required,
     setup_required,
-    with_current_tenant_id,
-    with_current_user,
 )
 from fields.base import ResponseModel
 from libs.helper import uuid_value
 from libs.login import login_required
-from models import Account
-from models.model import App, AppMode, UploadFile
-from services.agent.composer_service import AgentComposerService
-from services.agent.skill_package_service import SkillManifest, SkillPackageError
-from services.agent.skill_standardize_service import SkillStandardizeService
-from services.agent.skill_tool_inference_service import (
-    SkillToolInferenceError,
-    SkillToolInferenceResult,
-    SkillToolInferenceService,
-)
-from services.agent_drive_service import (
-    AgentDriveError,
-    AgentDriveService,
-    DriveCommitItem,
-    DriveFileRef,
-    normalize_drive_key,
-)
+from models.model import App, AppMode
 from services.agent_service import AgentService
 
-_WORKFLOW_AGENT_DRIVE_APP_MODES = [AppMode.WORKFLOW, AppMode.ADVANCED_CHAT]
-_AGENT_SKILL_UPLOAD_PARAMS = {
-    "file": {
-        "in": "formData",
-        "type": "file",
-        "required": True,
-        "description": "Skill package (.zip or .skill).",
-    }
-}
-
 
 class AgentLogQuery(BaseModel):
     message_id: str = Field(..., description="Message UUID")
@@ -70,27 +33,6 @@ class AgentLogQuery(BaseModel):
         return uuid_value(value)
 
 
-class AgentDriveFilePayload(BaseModel):
-    upload_file_id: str = Field(..., description="UploadFile UUID from POST /console/api/files/upload")
-
-    @field_validator("upload_file_id")
-    @classmethod
-    def validate_upload_file_id(cls, value: str) -> str:
-        return uuid_value(value)
-
-
-class AgentDriveMutationQuery(BaseModel):
-    node_id: str | None = Field(default=None, description="Workflow node ID (workflow composer variant)")
-
-
-class AgentDriveDeleteFileQuery(AgentDriveMutationQuery):
-    key: str = Field(min_length=1, description="Drive key, e.g. files/sample.pdf")
-
-
-class AgentDriveDeleteFileByAgentQuery(BaseModel):
-    key: str = Field(min_length=1, description="Drive key, e.g. files/sample.pdf")
-
-
 class AgentLogMetaResponse(ResponseModel):
     status: str
     executor: str
@@ -128,204 +70,7 @@ class AgentLogResponse(ResponseModel):
     files: list[Any] = Field(default_factory=list)
 
 
-class AgentUploadedSkillResponse(ResponseModel):
-    name: str
-    description: str
-    path: str
-    skill_md_key: str
-    archive_key: str | None = None
-
-
-class AgentSkillUploadResponse(ResponseModel):
-    skill: AgentUploadedSkillResponse
-    manifest: SkillManifest
-
-
-class AgentDriveFileResponse(ResponseModel):
-    name: str
-    drive_key: str
-    file_id: str
-    size: int | None = None
-    mime_type: str | None = None
-
-
-class AgentDriveFileCommitResponse(ResponseModel):
-    file: AgentDriveFileResponse
-
-
-class AgentDriveDeleteResponse(ResponseModel):
-    result: str
-    removed_keys: list[str] = Field(default_factory=list)
-
-
-register_schema_models(console_ns, AgentLogQuery, AgentDriveFilePayload, AgentDriveDeleteFileByAgentQuery)
-register_response_schema_models(
-    console_ns,
-    AgentDriveDeleteResponse,
-    AgentDriveFileCommitResponse,
-    AgentDriveFileResponse,
-    AgentLogResponse,
-    AgentUploadedSkillResponse,
-    AgentSkillUploadResponse,
-    SkillToolInferenceResult,
-)
-
-
-def _resolve_agent_id(session: Session, app_model: App, node_id: str | None) -> str | None:
-    if node_id and app_model.mode != AppMode.AGENT:
-        return AgentComposerService.resolve_workflow_node_agent_id(
-            session=session, tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id
-        )
-    return app_model.bound_agent_id_with_session(session=session)
-
-
-def _agent_not_bound() -> tuple[dict[str, str], int]:
-    return {"code": "agent_not_bound", "message": "no agent is bound for this app/node"}, 400
-
-
-def _upload_skill_for_app(*, session: Session, current_user: Account, app_model: App):
-    """Upload one skill package and commit its normalized files into the agent drive."""
-
-    query = query_params_from_request(AgentDriveMutationQuery)
-    agent_id = _resolve_agent_id(session, app_model, query.node_id)
-    if not agent_id:
-        return _agent_not_bound()
-    if "file" not in request.files:
-        return {"code": "no_file", "message": "no skill file uploaded"}, 400
-    if len(request.files) > 1:
-        return {"code": "too_many_files", "message": "only one skill file is allowed"}, 400
-
-    upload = request.files["file"]
-    content = upload.stream.read()
-    try:
-        result = SkillStandardizeService().standardize(
-            content=content,
-            filename=upload.filename or "",
-            tenant_id=app_model.tenant_id,
-            user_id=current_user.id,
-            agent_id=agent_id,
-            session=session,
-        )
-    except (SkillPackageError, AgentDriveError) as exc:
-        return {"code": exc.code, "message": exc.message}, exc.status_code
-    return result, 201
-
-
-def _commit_drive_file_for_app(*, session: Session, current_user: Account, app_model: App, allow_node_id: bool = True):
-    payload = AgentDriveFilePayload.model_validate(console_ns.payload or {})
-    query = query_params_from_request(AgentDriveMutationQuery)
-    node_id = query.node_id if allow_node_id else None
-    agent_id = _resolve_agent_id(session, app_model, node_id)
-    if not agent_id:
-        return _agent_not_bound()
-
-    upload_file = session.scalar(
-        select(UploadFile).where(
-            UploadFile.id == payload.upload_file_id,
-            UploadFile.tenant_id == app_model.tenant_id,
-        )
-    )
-    if upload_file is None:
-        return {"code": "upload_file_not_found", "message": "upload file not found in this workspace"}, 404
-
-    try:
-        key = normalize_drive_key(f"files/{upload_file.name}")
-        committed = AgentDriveService().commit(
-            tenant_id=app_model.tenant_id,
-            user_id=current_user.id,
-            agent_id=agent_id,
-            items=[
-                DriveCommitItem(
-                    key=key,
-                    file_ref=DriveFileRef(kind="upload_file", id=upload_file.id),
-                    # ADD FILE uploads exist solely to live in the drive, so the
-                    # drive owns (and physically cleans) the value on delete.
-                    value_owned_by_drive=True,
-                )
-            ],
-            session=session,
-        )
-    except AgentDriveError as exc:
-        return {"code": exc.code, "message": exc.message}, exc.status_code
-
-    row = committed[0]
-    return {
-        "file": {
-            "name": upload_file.name,
-            "drive_key": row["key"],
-            "file_id": upload_file.id,
-            "size": row.get("size"),
-            "mime_type": row.get("mime_type"),
-        },
-    }, 201
-
-
-def _delete_drive_file_for_app(*, session: Session, current_user: Account, app_model: App, allow_node_id: bool = True):
-    query = query_params_from_request(AgentDriveDeleteFileQuery)
-    node_id = query.node_id if allow_node_id else None
-    agent_id = _resolve_agent_id(session, app_model, node_id)
-    if not agent_id:
-        return _agent_not_bound()
-    try:
-        key = normalize_drive_key(query.key)
-    except AgentDriveError as exc:
-        return {"code": exc.code, "message": exc.message}, exc.status_code
-
-    try:
-        result = AgentDriveService().commit(
-            tenant_id=app_model.tenant_id,
-            user_id=current_user.id,
-            agent_id=agent_id,
-            items=[DriveCommitItem(key=key, file_ref=None)],
-            session=session,
-        )
-    except AgentDriveError as exc:
-        return {"code": exc.code, "message": exc.message}, exc.status_code
-    removed_keys = [item["key"] for item in result if item.get("removed")]
-    return {"result": "success", "removed_keys": removed_keys}
-
-
-def _delete_skill_for_app(
-    *, session: Session, current_user: Account, app_model: App, slug: str, allow_node_id: bool = True
-):
-    query = query_params_from_request(AgentDriveMutationQuery)
-    node_id = query.node_id if allow_node_id else None
-    agent_id = _resolve_agent_id(session, app_model, node_id)
-    if not agent_id:
-        return _agent_not_bound()
-    if "/" in slug or not slug.strip():
-        return {"code": "drive_key_invalid", "message": "skill slug must be a single path segment"}, 400
-
-    try:
-        result = AgentDriveService().commit(
-            tenant_id=app_model.tenant_id,
-            user_id=current_user.id,
-            agent_id=agent_id,
-            items=[
-                DriveCommitItem(key=f"{slug}/SKILL.md", file_ref=None),
-                DriveCommitItem(key=f"{slug}/.DIFY-SKILL-FULL.zip", file_ref=None),
-            ],
-            session=session,
-        )
-    except AgentDriveError as exc:
-        return {"code": exc.code, "message": exc.message}, exc.status_code
-    removed_keys = [item["key"] for item in result if item.get("removed")]
-    return {"result": "success", "removed_keys": removed_keys}
-
-
-def _infer_skill_tools_for_app(*, session: Session, app_model: App, slug: str):
-    query = query_params_from_request(AgentDriveMutationQuery)
-    agent_id = _resolve_agent_id(session, app_model, query.node_id)
-    if not agent_id:
-        return _agent_not_bound()
-    if "/" in slug or not slug.strip():
-        return {"code": "drive_key_invalid", "message": "skill slug must be a single path segment"}, 400
-    try:
-        return SkillToolInferenceService().infer(
-            tenant_id=app_model.tenant_id, agent_id=agent_id, slug=slug, session=session
-        )
-    except SkillToolInferenceError as exc:
-        return {"code": exc.code, "message": exc.message}, exc.status_code
+register_response_schema_models(console_ns, AgentLogResponse)
 
 
 @console_ns.route("/apps/<uuid:app_id>/agent/logs")
@@ -344,209 +89,6 @@ class AgentLogApi(Resource):
     @get_app_model(mode=[AppMode.AGENT_CHAT])
     @model_validate(AgentLogQuery)
     def get(self, req_data: AgentLogQuery, session: Session, app_model: App):
-        """Get agent logs"""
+        """Get agent logs."""
 
         return AgentService.get_agent_logs(app_model, req_data.conversation_id, req_data.message_id, session)
-
-
-@console_ns.route("/agent/<uuid:agent_id>/skills/upload")
-class AgentSkillUploadByAgentApi(Resource):
-    @console_ns.doc("upload_agent_skill_by_agent")
-    @console_ns.doc(description="Upload + standardize a Skill into an Agent App drive")
-    @console_ns.doc(consumes=["multipart/form-data"], params={"agent_id": "Agent ID", **_AGENT_SKILL_UPLOAD_PARAMS})
-    @console_ns.response(201, "Skill uploaded into drive", console_ns.models[AgentSkillUploadResponse.__name__])
-    @console_ns.response(400, "Invalid skill package or no bound agent")
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_current_user
-    @with_current_tenant_id
-    @with_session
-    def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
-        return _upload_skill_for_app(session=session, current_user=current_user, app_model=app_model)
-
-
-@console_ns.route("/apps/<uuid:app_id>/agent/skills/upload")
-class AgentSkillUploadApi(Resource):
-    @console_ns.doc("upload_agent_skill")
-    @console_ns.doc(description="Upload + standardize a Skill into the agent drive")
-    @console_ns.doc(
-        consumes=["multipart/form-data"],
-        params={
-            "app_id": "Application ID",
-            **query_params_from_model(AgentDriveMutationQuery),
-            **_AGENT_SKILL_UPLOAD_PARAMS,
-        },
-    )
-    @console_ns.response(201, "Skill uploaded into drive", console_ns.models[AgentSkillUploadResponse.__name__])
-    @console_ns.response(400, "Invalid skill package or no bound agent")
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_current_user
-    @with_session
-    @get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
-    def post(self, session: Session, current_user: Account, app_model: App):
-        """Upload a Skill, validate it, and commit drive-backed skill files."""
-        return _upload_skill_for_app(session=session, current_user=current_user, app_model=app_model)
-
-
-@console_ns.route("/agent/<uuid:agent_id>/files")
-class AgentDriveFilesByAgentApi(Resource):
-    @console_ns.doc("commit_agent_drive_file_by_agent")
-    @console_ns.doc(description="Commit an uploaded file into the Agent App drive under files/<name>")
-    @console_ns.doc(params={"agent_id": "Agent ID"})
-    @console_ns.expect(console_ns.models[AgentDriveFilePayload.__name__])
-    @console_ns.response(
-        201, "File committed into the agent drive", console_ns.models[AgentDriveFileCommitResponse.__name__]
-    )
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_current_user
-    @with_current_tenant_id
-    @with_session
-    def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
-        return _commit_drive_file_for_app(
-            session=session, current_user=current_user, app_model=app_model, allow_node_id=False
-        )
-
-    @console_ns.doc("delete_agent_drive_file_by_agent")
-    @console_ns.doc(description="Delete one Agent App drive file by key")
-    @console_ns.doc(params={"agent_id": "Agent ID", **query_params_from_model(AgentDriveDeleteFileByAgentQuery)})
-    @console_ns.response(200, "File removed", console_ns.models[AgentDriveDeleteResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_current_user
-    @with_current_tenant_id
-    @with_session
-    def delete(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
-        return _delete_drive_file_for_app(
-            session=session, current_user=current_user, app_model=app_model, allow_node_id=False
-        )
-
-
-@console_ns.route("/apps/<uuid:app_id>/agent/files")
-class AgentDriveFilesApi(Resource):
-    @console_ns.doc("commit_agent_drive_file")
-    @console_ns.doc(description="Commit an uploaded file into the agent drive under files/<name> (ENG-625 D3)")
-    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(AgentDriveMutationQuery)})
-    @console_ns.expect(console_ns.models[AgentDriveFilePayload.__name__])
-    @console_ns.response(
-        201, "File committed into the agent drive", console_ns.models[AgentDriveFileCommitResponse.__name__]
-    )
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_current_user
-    @with_session
-    @get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
-    def post(self, session: Session, current_user: Account, app_model: App):
-        """ADD FILE: commit one uploaded file into the bound agent's drive."""
-        return _commit_drive_file_for_app(session=session, current_user=current_user, app_model=app_model)
-
-    @console_ns.doc("delete_agent_drive_file")
-    @console_ns.doc(description="Delete one drive file by key via drive commit-null semantics")
-    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(AgentDriveDeleteFileQuery)})
-    @console_ns.response(200, "File removed", console_ns.models[AgentDriveDeleteResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_current_user
-    @with_session
-    @get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
-    def delete(self, session: Session, current_user: Account, app_model: App):
-        return _delete_drive_file_for_app(session=session, current_user=current_user, app_model=app_model)
-
-
-@console_ns.route("/agent/<uuid:agent_id>/skills/<string:slug>")
-class AgentSkillByAgentApi(Resource):
-    @console_ns.doc("delete_agent_skill_by_agent")
-    @console_ns.doc(description="Delete a standardized skill from an Agent App drive")
-    @console_ns.doc(params={"agent_id": "Agent ID", "slug": "Skill slug (single path segment)"})
-    @console_ns.response(200, "Skill removed", console_ns.models[AgentDriveDeleteResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_current_user
-    @with_current_tenant_id
-    @with_session
-    def delete(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, slug: str):
-        app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
-        return _delete_skill_for_app(
-            session=session, current_user=current_user, app_model=app_model, slug=slug, allow_node_id=False
-        )
-
-
-@console_ns.route("/apps/<uuid:app_id>/agent/skills/<string:slug>")
-class AgentSkillApi(Resource):
-    @console_ns.doc("delete_agent_skill")
-    @console_ns.doc(description="Delete a standardized skill by removing its known drive keys via commit-null")
-    @console_ns.doc(
-        params={
-            "app_id": "Application ID",
-            "slug": "Skill slug (single path segment)",
-            **query_params_from_model(AgentDriveMutationQuery),
-        }
-    )
-    @console_ns.response(200, "Skill removed", console_ns.models[AgentDriveDeleteResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_current_user
-    @with_session
-    @get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
-    def delete(self, session: Session, current_user: Account, app_model: App, slug: str):
-        return _delete_skill_for_app(session=session, current_user=current_user, app_model=app_model, slug=slug)
-
-
-@console_ns.route("/agent/<uuid:agent_id>/skills/<string:slug>/infer-tools")
-class AgentSkillInferToolsByAgentApi(Resource):
-    @console_ns.doc("infer_agent_skill_tools_by_agent")
-    @console_ns.doc(description="Infer CLI tool + ENV suggestions from a standardized Agent App skill")
-    @console_ns.doc(params={"agent_id": "Agent ID", "slug": "Skill slug (single path segment)"})
-    @console_ns.response(
-        200,
-        "Inference result (draft suggestions, nothing persisted)",
-        console_ns.models[SkillToolInferenceResult.__name__],
-    )
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_current_tenant_id
-    @with_session(write=False)
-    def post(self, session: Session, tenant_id: str, agent_id: UUID, slug: str):
-        app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
-        return _infer_skill_tools_for_app(session=session, app_model=app_model, slug=slug)
-
-
-@console_ns.route("/apps/<uuid:app_id>/agent/skills/<string:slug>/infer-tools")
-class AgentSkillInferToolsApi(Resource):
-    @console_ns.doc("infer_agent_skill_tools")
-    @console_ns.doc(
-        description="Infer CLI tool + ENV suggestions from a standardized skill's SKILL.md (draft only, ENG-371)"
-    )
-    @console_ns.doc(
-        params={
-            "app_id": "Application ID",
-            "slug": "Skill slug (single path segment)",
-            **query_params_from_model(AgentDriveMutationQuery),
-        }
-    )
-    @console_ns.response(
-        200,
-        "Inference result (draft suggestions, nothing persisted)",
-        console_ns.models[SkillToolInferenceResult.__name__],
-    )
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_session(write=False)
-    @get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
-    def post(self, session: Session, app_model: App, slug: str):
-        """Suggest CLI tools/env for a skill. Saving still goes through composer validation."""
-        return _infer_skill_tools_for_app(session=session, app_model=app_model, slug=slug)

api/controllers/console/app/agent_app_sandbox.py

@@ -10,10 +10,11 @@ from __future__ import annotations
 from typing import Literal
 from uuid import UUID
 
-from dify_agent.client import DifyAgentClientError, DifyAgentHTTPError, DifyAgentTimeoutError
+from dify_agent.client import DifyAgentClientError, DifyAgentHTTPError
 from flask_restx import Resource
 from pydantic import BaseModel, Field
 
+from clients.agent_backend.errors import backend_error_detail, backend_reported_failure
 from controllers.common.schema import (
     query_params_from_model,
     query_params_from_request,
@@ -24,8 +25,11 @@ from controllers.console import console_ns
 from controllers.console.app.error import AppNotFoundError
 from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import (
+    RBACPermission,
+    RBACResourceScope,
     account_initialization_required,
     model_validate,
+    rbac_permission_required,
     setup_required,
     with_current_tenant_id,
     with_current_user,
@@ -132,15 +136,10 @@ register_response_schema_models(
 def _handle(exc: Exception) -> tuple[dict[str, object], int]:
     if isinstance(exc, AgentSandboxInspectorError):
         return {"code": exc.code, "message": exc.message}, exc.status_code
-    if isinstance(exc, DifyAgentHTTPError):
-        detail = exc.detail
-        if isinstance(detail, dict):
-            return {
-                "code": detail.get("code", "agent_backend_error"),
-                "message": detail.get("message", str(exc)),
-            }, exc.status_code
-        return {"code": "agent_backend_error", "message": str(detail)}, exc.status_code
-    if isinstance(exc, DifyAgentTimeoutError | DifyAgentClientError):
+    if isinstance(exc, DifyAgentHTTPError) and backend_reported_failure(exc):
+        code, message = backend_error_detail(exc)
+        return {"code": code, "message": message}, exc.status_code
+    if isinstance(exc, DifyAgentClientError):
         return {"code": "agent_backend_unreachable", "message": str(exc)}, 502
     raise exc
 
@@ -154,6 +153,7 @@ class AgentAppSandboxInfoResource(Resource):
     @setup_required
     @login_required
     @account_initialization_required
+    @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
     @with_current_tenant_id
     @with_current_user
     def get(self, current_user: Account, tenant_id: str, agent_id: UUID):
@@ -183,6 +183,7 @@ class AgentAppSandboxListResource(Resource):
     @setup_required
     @login_required
     @account_initialization_required
+    @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
     @with_current_tenant_id
     @with_current_user
     def get(self, current_user: Account, tenant_id: str, agent_id: UUID):
@@ -213,6 +214,7 @@ class AgentAppSandboxReadResource(Resource):
     @setup_required
     @login_required
     @account_initialization_required
+    @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
     @with_current_tenant_id
     @with_current_user
     def get(self, current_user: Account, tenant_id: str, agent_id: UUID):
@@ -243,6 +245,7 @@ class AgentAppSandboxDownloadResource(Resource):
     @setup_required
     @login_required
     @account_initialization_required
+    @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
     @with_current_tenant_id
     @with_current_user
     @model_validate(AgentSandboxDownloadPayload)
@@ -286,6 +289,7 @@ class WorkflowAgentSandboxListResource(Resource):
     @setup_required
     @login_required
     @account_initialization_required
+    @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
     @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW])
     @with_current_tenant_id
     def get(self, tenant_id: str, app_model: App, workflow_run_id: UUID, node_id: str):
@@ -323,6 +327,7 @@ class WorkflowAgentSandboxReadResource(Resource):
     @setup_required
     @login_required
     @account_initialization_required
+    @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
     @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW])
     @with_current_tenant_id
     def get(self, tenant_id: str, app_model: App, workflow_run_id: UUID, node_id: str):
@@ -353,6 +358,7 @@ class WorkflowAgentSandboxDownloadResource(Resource):
     @setup_required
     @login_required
     @account_initialization_required
+    @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
     @with_current_user
     @with_current_tenant_id
     @model_validate(WorkflowAgentSandboxDownloadPayload)

api/controllers/console/app/agent_drive_inspector.py

@@ -1,434 +0,0 @@
-"""Console read-only inspector for the agent drive (ENG-624).
-
-``agent-drive`` looks at the *static* drive assets (standardized skills and
-committed files); the sibling ``agent-sandbox`` routes look at a *runtime*
-sandbox workspace. Unlike the sandbox routes this never proxies to the agent
-backend — drive data lives in the API's own DB/storage, served straight from
-``AgentDriveService``. Download hands the browser an **external** signed URL
-(the inner manifest hands agents internal ones — the two must never mix).
-"""
-
-from __future__ import annotations
-
-import json
-from collections.abc import Mapping
-from typing import Any
-from uuid import UUID
-
-from flask import Response
-from flask_restx import Resource
-from pydantic import BaseModel, Field
-from sqlalchemy.orm import Session
-
-from controllers.common.schema import (
-    query_params_from_model,
-    query_params_from_request,
-    register_response_schema_models,
-)
-from controllers.common.session import with_session
-from controllers.console import console_ns
-from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
-from controllers.console.app.wraps import get_app_model
-from controllers.console.wraps import account_initialization_required, setup_required, with_current_tenant_id
-from fields.base import ResponseModel
-from libs.login import login_required
-from models.model import App, AppMode
-from services.agent.composer_service import AgentComposerService
-from services.agent_drive_service import AgentDriveError, AgentDriveService
-
-
-class AgentDriveListQuery(BaseModel):
-    prefix: str = Field(default="", description="Key prefix filter: '<slug>/' for one skill, 'files/' for files")
-    node_id: str | None = Field(default=None, description="Workflow node ID (workflow composer variant)")
-
-
-class AgentDriveListByAgentQuery(BaseModel):
-    prefix: str = Field(default="", description="Key prefix filter: '<slug>/' for one skill, 'files/' for files")
-
-
-class AgentDriveFileQuery(BaseModel):
-    key: str = Field(min_length=1, description="Drive key, e.g. tender-analyzer/SKILL.md")
-    node_id: str | None = Field(default=None, description="Workflow node ID (workflow composer variant)")
-
-
-class AgentDriveFileByAgentQuery(BaseModel):
-    key: str = Field(min_length=1, description="Drive key, e.g. tender-analyzer/SKILL.md")
-
-
-class AgentDriveSkillInspectQuery(BaseModel):
-    node_id: str | None = Field(default=None, description="Workflow node ID (workflow composer variant)")
-
-
-class AgentDriveItemResponse(ResponseModel):
-    key: str
-    size: int | None = None
-    mime_type: str | None = None
-    hash: str | None = None
-    file_kind: str
-    created_at: int | None = None
-    is_skill: bool | None = None
-    skill_metadata: str | None = None
-
-
-class AgentDriveListResponse(ResponseModel):
-    items: list[AgentDriveItemResponse] = Field(default_factory=list)
-
-
-class AgentDriveSkillItemResponse(ResponseModel):
-    path: str
-    skill_md_key: str
-    archive_key: str | None = None
-    name: str
-    description: str
-    size: int | None = None
-    mime_type: str | None = None
-    hash: str | None = None
-    created_at: int | None = None
-
-
-class AgentDriveSkillListResponse(ResponseModel):
-    items: list[AgentDriveSkillItemResponse] = Field(default_factory=list)
-
-
-class AgentDriveSkillFileResponse(ResponseModel):
-    path: str
-    name: str
-    type: str
-    drive_key: str | None = None
-    available_in_drive: bool
-
-
-class AgentDriveSkillMarkdownResponse(ResponseModel):
-    key: str
-    size: int | None = None
-    truncated: bool
-    binary: bool
-    text: str | None = None
-
-
-class AgentDriveSkillInspectResponse(ResponseModel):
-    path: str
-    skill_md_key: str
-    archive_key: str | None = None
-    name: str
-    description: str
-    size: int | None = None
-    mime_type: str | None = None
-    hash: str | None = None
-    created_at: int | None = None
-    source: str
-    files: list[AgentDriveSkillFileResponse] = Field(default_factory=list)
-    file_tree: list[dict[str, Any]] = Field(default_factory=list)
-    skill_md: AgentDriveSkillMarkdownResponse
-    warnings: list[str] = Field(default_factory=list)
-
-
-class AgentDrivePreviewResponse(ResponseModel):
-    key: str
-    size: int | None = None
-    truncated: bool
-    binary: bool
-    text: str | None = None
-
-
-class AgentDriveDownloadResponse(ResponseModel):
-    url: str
-
-
-register_response_schema_models(
-    console_ns,
-    AgentDriveDownloadResponse,
-    AgentDriveListResponse,
-    AgentDrivePreviewResponse,
-    AgentDriveSkillInspectResponse,
-    AgentDriveSkillListResponse,
-)
-
-
-def _resolve_agent_id(session: Session, app_model: App, node_id: str | None) -> str | None:
-    """Agent identity for the drive: app-bound agent, or the workflow node binding."""
-    if node_id:
-        return AgentComposerService.resolve_workflow_node_agent_id(
-            session=session, tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id
-        )
-    return app_model.bound_agent_id_with_session(session=session)
-
-
-def _agent_not_bound() -> tuple[dict[str, object], int]:
-    return {"code": "agent_not_bound", "message": "no agent is bound for this app/node"}, 400
-
-
-def _handle(exc: AgentDriveError) -> tuple[dict[str, object], int]:
-    return {"code": exc.code, "message": exc.message}, exc.status_code
-
-
-def _json_response(data: Mapping[str, Any]):
-    return Response(
-        response=json.dumps(data, ensure_ascii=False, separators=(",", ":")),
-        content_type="application/json; charset=utf-8",
-    )
-
-
-_WORKFLOW_APP_MODES = [AppMode.WORKFLOW, AppMode.ADVANCED_CHAT]
-
-
-@console_ns.route("/agent/<uuid:agent_id>/drive/files")
-class AgentDriveListByAgentApi(Resource):
-    @console_ns.doc("list_agent_drive_files_by_agent")
-    @console_ns.doc(description="List agent drive entries for an Agent App")
-    @console_ns.doc(params={"agent_id": "Agent ID", **query_params_from_model(AgentDriveListByAgentQuery)})
-    @console_ns.response(200, "Drive entries", console_ns.models[AgentDriveListResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_current_tenant_id
-    @with_session(write=False)
-    def get(self, session: Session, tenant_id: str, agent_id: UUID):
-        query = query_params_from_request(AgentDriveListByAgentQuery)
-        resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
-        try:
-            items = AgentDriveService().manifest(
-                tenant_id=tenant_id, agent_id=str(agent_id), prefix=query.prefix, session=session
-            )
-        except AgentDriveError as exc:
-            return _handle(exc)
-        return {"items": [{k: v for k, v in item.items() if k != "file_id"} for item in items]}
-
-
-@console_ns.route("/agent/<uuid:agent_id>/drive/skills")
-class AgentDriveSkillListByAgentApi(Resource):
-    @console_ns.doc("list_agent_drive_skills_by_agent")
-    @console_ns.doc(description="List drive-backed skills for an Agent App")
-    @console_ns.doc(params={"agent_id": "Agent ID"})
-    @console_ns.response(200, "Drive skills", console_ns.models[AgentDriveSkillListResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_current_tenant_id
-    @with_session(write=False)
-    def get(self, session: Session, tenant_id: str, agent_id: UUID):
-        resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
-        try:
-            items = AgentDriveService().list_skills(tenant_id=tenant_id, agent_id=str(agent_id), session=session)
-        except AgentDriveError as exc:
-            return _handle(exc)
-        return {"items": items}
-
-
-@console_ns.route("/agent/<uuid:agent_id>/drive/skills/<path:skill_path>/inspect")
-class AgentDriveSkillInspectByAgentApi(Resource):
-    @console_ns.doc("inspect_agent_drive_skill_by_agent")
-    @console_ns.doc(description="Inspect one drive-backed skill for slash-menu hover/detail UI")
-    @console_ns.doc(params={"agent_id": "Agent ID", "skill_path": "Skill path/slug, e.g. tender-analyzer"})
-    @console_ns.response(200, "Drive skill inspect view", console_ns.models[AgentDriveSkillInspectResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_current_tenant_id
-    @with_session(write=False)
-    def get(self, session: Session, tenant_id: str, agent_id: UUID, skill_path: str):
-        resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
-        try:
-            return _json_response(
-                AgentDriveService().inspect_skill(
-                    tenant_id=tenant_id,
-                    agent_id=str(agent_id),
-                    skill_path=skill_path,
-                    session=session,
-                )
-            )
-        except AgentDriveError as exc:
-            return _handle(exc)
-
-
-@console_ns.route("/agent/<uuid:agent_id>/drive/files/preview")
-class AgentDrivePreviewByAgentApi(Resource):
-    @console_ns.doc("preview_agent_drive_file_by_agent")
-    @console_ns.doc(description="Truncated text preview of one Agent App drive value")
-    @console_ns.doc(params={"agent_id": "Agent ID", **query_params_from_model(AgentDriveFileByAgentQuery)})
-    @console_ns.response(200, "Preview", console_ns.models[AgentDrivePreviewResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_current_tenant_id
-    @with_session(write=False)
-    def get(self, session: Session, tenant_id: str, agent_id: UUID):
-        query = query_params_from_request(AgentDriveFileByAgentQuery)
-        resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
-        try:
-            return AgentDriveService().preview(
-                tenant_id=tenant_id, agent_id=str(agent_id), key=query.key, session=session
-            )
-        except AgentDriveError as exc:
-            return _handle(exc)
-
-
-@console_ns.route("/agent/<uuid:agent_id>/drive/files/download")
-class AgentDriveDownloadByAgentApi(Resource):
-    @console_ns.doc("download_agent_drive_file_by_agent")
-    @console_ns.doc(description="Time-limited external signed URL for one Agent App drive value")
-    @console_ns.doc(params={"agent_id": "Agent ID", **query_params_from_model(AgentDriveFileByAgentQuery)})
-    @console_ns.response(200, "Signed URL", console_ns.models[AgentDriveDownloadResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_current_tenant_id
-    @with_session(write=False)
-    def get(self, session: Session, tenant_id: str, agent_id: UUID):
-        query = query_params_from_request(AgentDriveFileByAgentQuery)
-        resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
-        try:
-            url = AgentDriveService().download_url(
-                tenant_id=tenant_id, agent_id=str(agent_id), key=query.key, session=session
-            )
-        except AgentDriveError as exc:
-            return _handle(exc)
-        return {"url": url}
-
-
-@console_ns.route("/apps/<uuid:app_id>/agent/drive/files")
-class AgentDriveListApi(Resource):
-    @console_ns.doc("list_agent_drive_files")
-    @console_ns.doc(description="List agent drive entries (read-only inspector; one endpoint for both tabs)")
-    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(AgentDriveListQuery)})
-    @console_ns.response(200, "Drive entries", console_ns.models[AgentDriveListResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_session(write=False)
-    @get_app_model(mode=_WORKFLOW_APP_MODES)
-    def get(self, session: Session, app_model: App):
-        query = query_params_from_request(AgentDriveListQuery)
-        agent_id = _resolve_agent_id(session, app_model, query.node_id)
-        if not agent_id:
-            return _agent_not_bound()
-        try:
-            items = AgentDriveService().manifest(
-                tenant_id=app_model.tenant_id, agent_id=agent_id, prefix=query.prefix, session=session
-            )
-        except AgentDriveError as exc:
-            return _handle(exc)
-        # the inner manifest exposes file_id for agent-side pulls; the console
-        # inspector is a pure read surface and does not need value pointers
-        return {"items": [{k: v for k, v in item.items() if k != "file_id"} for item in items]}
-
-
-@console_ns.route("/apps/<uuid:app_id>/agent/drive/skills")
-class AgentDriveSkillListApi(Resource):
-    @console_ns.doc("list_agent_drive_skills")
-    @console_ns.doc(description="List drive-backed skills for the bound agent")
-    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(AgentDriveListQuery)})
-    @console_ns.response(200, "Drive skills", console_ns.models[AgentDriveSkillListResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_session(write=False)
-    @get_app_model(mode=_WORKFLOW_APP_MODES)
-    def get(self, session: Session, app_model: App):
-        query = query_params_from_request(AgentDriveListQuery)
-        agent_id = _resolve_agent_id(session, app_model, query.node_id)
-        if not agent_id:
-            return _agent_not_bound()
-        try:
-            items = AgentDriveService().list_skills(tenant_id=app_model.tenant_id, agent_id=agent_id, session=session)
-        except AgentDriveError as exc:
-            return _handle(exc)
-        return {"items": items}
-
-
-@console_ns.route("/apps/<uuid:app_id>/agent/drive/skills/<path:skill_path>/inspect")
-class AgentDriveSkillInspectApi(Resource):
-    @console_ns.doc("inspect_agent_drive_skill")
-    @console_ns.doc(description="Inspect one drive-backed skill for slash-menu hover/detail UI")
-    @console_ns.doc(
-        params={
-            "app_id": "Application ID",
-            "skill_path": "Skill path/slug, e.g. tender-analyzer",
-            **query_params_from_model(AgentDriveSkillInspectQuery),
-        }
-    )
-    @console_ns.response(200, "Drive skill inspect view", console_ns.models[AgentDriveSkillInspectResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_session(write=False)
-    @get_app_model(mode=_WORKFLOW_APP_MODES)
-    def get(self, session: Session, app_model: App, skill_path: str):
-        query = query_params_from_request(AgentDriveSkillInspectQuery)
-        agent_id = _resolve_agent_id(session, app_model, query.node_id)
-        if not agent_id:
-            return _agent_not_bound()
-        try:
-            return _json_response(
-                AgentDriveService().inspect_skill(
-                    tenant_id=app_model.tenant_id,
-                    agent_id=agent_id,
-                    skill_path=skill_path,
-                    session=session,
-                )
-            )
-        except AgentDriveError as exc:
-            return _handle(exc)
-
-
-@console_ns.route("/apps/<uuid:app_id>/agent/drive/files/preview")
-class AgentDrivePreviewApi(Resource):
-    @console_ns.doc("preview_agent_drive_file")
-    @console_ns.doc(description="Truncated text preview of one drive value (binary-safe; SKILL.md is the main case)")
-    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(AgentDriveFileQuery)})
-    @console_ns.response(200, "Preview", console_ns.models[AgentDrivePreviewResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_session(write=False)
-    @get_app_model(mode=_WORKFLOW_APP_MODES)
-    def get(self, session: Session, app_model: App):
-        query = query_params_from_request(AgentDriveFileQuery)
-        agent_id = _resolve_agent_id(session, app_model, query.node_id)
-        if not agent_id:
-            return _agent_not_bound()
-        try:
-            return AgentDriveService().preview(
-                tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key, session=session
-            )
-        except AgentDriveError as exc:
-            return _handle(exc)
-
-
-@console_ns.route("/apps/<uuid:app_id>/agent/drive/files/download")
-class AgentDriveDownloadApi(Resource):
-    @console_ns.doc("download_agent_drive_file")
-    @console_ns.doc(description="Time-limited external signed URL for one drive value (no streaming proxy)")
-    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(AgentDriveFileQuery)})
-    @console_ns.response(200, "Signed URL", console_ns.models[AgentDriveDownloadResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_session(write=False)
-    @get_app_model(mode=_WORKFLOW_APP_MODES)
-    def get(self, session: Session, app_model: App):
-        query = query_params_from_request(AgentDriveFileQuery)
-        agent_id = _resolve_agent_id(session, app_model, query.node_id)
-        if not agent_id:
-            return _agent_not_bound()
-        try:
-            url = AgentDriveService().download_url(
-                tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key, session=session
-            )
-        except AgentDriveError as exc:
-            return _handle(exc)
-        return {"url": url}
-
-
-__all__ = [
-    "AgentDriveDownloadApi",
-    "AgentDriveDownloadByAgentApi",
-    "AgentDriveListApi",
-    "AgentDriveListByAgentApi",
-    "AgentDrivePreviewApi",
-    "AgentDrivePreviewByAgentApi",
-    "AgentDriveSkillInspectApi",
-    "AgentDriveSkillInspectByAgentApi",
-    "AgentDriveSkillListApi",
-    "AgentDriveSkillListByAgentApi",
-]

api/controllers/console/app/completion.py

@@ -16,6 +16,7 @@ from controllers.common.schema import register_response_schema_models, register_
 from controllers.console import console_ns
 from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
 from controllers.console.app.error import (
+    AgentSessionConfigurationChangedError,
     AppUnavailableError,
     CompletionRequestError,
     ConversationCompletedError,
@@ -620,6 +621,10 @@ def _raise_agent_stream_error_before_response(response):
             if isinstance(response, _ClosableStream):
                 response.close()
             message = error_payload.get("message")
+            if error_payload.get("code") == AgentSessionConfigurationChangedError.error_code:
+                raise AgentSessionConfigurationChangedError(
+                    str(message or AgentSessionConfigurationChangedError.description)
+                )
             raise CompletionRequestError(str(message or "Agent App chat failed."))
 
         return _prepend_stream_chunks(buffered, chunk, iterator)

api/controllers/console/app/error.py

@@ -1,3 +1,7 @@
+from core.app.apps.agent_app.errors import (
+    AGENT_SESSION_CONFIGURATION_CHANGED_ERROR_CODE,
+    AGENT_SESSION_CONFIGURATION_CHANGED_MESSAGE,
+)
 from libs.exception import BaseHTTPException
 
 
@@ -49,6 +53,12 @@ class CompletionRequestError(BaseHTTPException):
     code = 400
 
 
+class AgentSessionConfigurationChangedError(BaseHTTPException):
+    error_code = AGENT_SESSION_CONFIGURATION_CHANGED_ERROR_CODE
+    description = AGENT_SESSION_CONFIGURATION_CHANGED_MESSAGE
+    code = 409
+
+
 class AppMoreLikeThisDisabledError(BaseHTTPException):
     error_code = "app_more_like_this_disabled"
     description = "The 'More like this' feature is disabled. Please refresh your page."

api/controllers/console/app/generator.py

@@ -364,6 +364,7 @@ class InstructionGenerateApi(Resource):
                     case "llm":
                         return LLMGenerator.generate_rule_config(
                             current_tenant_id,
+                            app_id=req_data.flow_id,
                             args=RuleGeneratePayload(
                                 instruction=req_data.instruction,
                                 model_config=req_data.model_config_data,
@@ -373,6 +374,7 @@ class InstructionGenerateApi(Resource):
                     case "agent":
                         return LLMGenerator.generate_rule_config(
                             current_tenant_id,
+                            app_id=req_data.flow_id,
                             args=RuleGeneratePayload(
                                 instruction=req_data.instruction,
                                 model_config=req_data.model_config_data,
@@ -382,6 +384,7 @@ class InstructionGenerateApi(Resource):
                     case "code":
                         return LLMGenerator.generate_code(
                             tenant_id=current_tenant_id,
+                            app_id=req_data.flow_id,
                             args=RuleCodeGeneratePayload(
                                 instruction=req_data.instruction,
                                 model_config=req_data.model_config_data,

api/controllers/console/app/mcp_server.py

@@ -190,7 +190,7 @@ class AppMCPServerRefreshController(Resource):
     @login_required
     @account_initialization_required
     @edit_permission_required
-    @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
+    @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
     @with_current_tenant_id
     @get_app_model
     def post(self, current_tenant_id: str, app_model: App):

api/controllers/console/app/workflow.py

@@ -20,6 +20,8 @@ from sqlalchemy.orm import Session, sessionmaker
 from werkzeug.exceptions import BadRequest, Forbidden, InternalServerError, NotFound
 
 import services
+from configs import dify_config
+from controllers.common.app_access import resolve_app_access_filter
 from controllers.common.controller_schemas import DefaultBlockConfigQuery, WorkflowListQuery, WorkflowUpdatePayload
 from controllers.common.errors import InvalidArgumentError
 from controllers.common.fields import GeneratedAppResponse, NewAppResponse, SimpleResultResponse
@@ -53,6 +55,7 @@ from core.app.apps.base_app_queue_manager import AppQueueManager
 from core.app.apps.workflow.app_generator import SKIP_PREPARE_USER_INPUTS_KEY
 from core.app.entities.app_invoke_entities import InvokeFrom
 from core.app.file_access import DatabaseFileAccessController
+from core.db.session_factory import session_factory
 from core.helper import encrypter
 from core.helper.trace_id_helper import get_external_trace_id
 from core.plugin.impl.exc import PluginInvokeError
@@ -95,7 +98,6 @@ from services.errors.app import IsDraftWorkflowError, WorkflowHashNotEqualError,
 from services.errors.llm import InvokeRateLimitError
 from services.workflow_ref_service import WorkflowRefService
 from services.workflow_service import DraftWorkflowDeletionError, WorkflowInUseError, WorkflowService
-from tasks.collect_agent_resources_task import enqueue_agent_resource_collection
 
 logger = logging.getLogger(__name__)
 
@@ -315,6 +317,9 @@ class WorkflowResponse(ResponseModel):
     )
     hash: str = Field(validation_alias=AliasChoices("unique_hash", "hash"))
     version: str
+    # NULL for drafts and for versions published before numbering was introduced; those
+    # render as "Untitled Version" instead of `#N`. Never 0, so clients must test for null.
+    version_number: int | None = None
     marked_name: str
     marked_comment: str
     created_by: SimpleAccount | None = Field(
@@ -1300,7 +1305,7 @@ class PublishedWorkflowApi(Resource):
 
         workflow_service = WorkflowService()
         with sessionmaker(db.engine).begin() as session:
-            workflow, retirement_candidates = workflow_service.publish_workflow(
+            workflow = workflow_service.publish_workflow(
                 session=session,
                 app_model=app_model,
                 account=current_user,
@@ -1317,16 +1322,6 @@ class PublishedWorkflowApi(Resource):
 
             workflow_created_at = TimestampField().format(workflow.created_at)
 
-        binding_ids, home_snapshot_ids = WorkflowAgentRetirementService.retire_unowned(
-            tenant_id=app_model.tenant_id,
-            agent_ids=retirement_candidates,
-            account_id=current_user.id,
-        )
-        enqueue_agent_resource_collection(
-            tenant_id=app_model.tenant_id,
-            binding_ids=binding_ids,
-            home_snapshot_ids=home_snapshot_ids,
-        )
         return {
             "result": "success",
             "created_at": workflow_created_at,
@@ -1617,10 +1612,11 @@ class WorkflowByIdApi(Resource):
     @login_required
     @account_initialization_required
     @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW])
+    @with_current_user
     @edit_permission_required
     @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
     @console_ns.response(204, "Workflow deleted successfully")
-    def delete(self, app_model: App, workflow_id: str):
+    def delete(self, current_user: Account, app_model: App, workflow_id: str):
         """
         Delete workflow
         """
@@ -1630,7 +1626,7 @@ class WorkflowByIdApi(Resource):
         # Create a session and manage the transaction
         with sessionmaker(db.engine).begin() as session:
             try:
-                workflow_service.delete_workflow(
+                retirement_candidates = workflow_service.delete_workflow(
                     session=session,
                     workflow_ref=workflow_ref,
                 )
@@ -1641,6 +1637,11 @@ class WorkflowByIdApi(Resource):
             except ValueError as e:
                 raise NotFound(str(e))
 
+        WorkflowAgentRetirementService.retire_unowned(
+            tenant_id=app_model.tenant_id,
+            agent_ids=retirement_candidates,
+            account_id=current_user.id,
+        )
         return None, 204
 
 
@@ -1926,8 +1927,9 @@ class WorkflowOnlineUsersApi(Resource):
     @setup_required
     @login_required
     @account_initialization_required
+    @with_current_user
     @with_current_tenant_id
-    def post(self, current_tenant_id: str):
+    def post(self, current_tenant_id: str, current_user: Account):
         args = WorkflowOnlineUsersPayload.model_validate(console_ns.payload or {})
 
         app_ids = args.app_ids
@@ -1937,8 +1939,20 @@ class WorkflowOnlineUsersApi(Resource):
         if not app_ids:
             return {"data": []}
 
+        access_filter = None
         workflow_service = WorkflowService()
-        accessible_app_ids = workflow_service.get_accessible_app_ids(app_ids, current_tenant_id, session=db.session())
+        with session_factory.create_session() as session:
+            if dify_config.RBAC_ENABLED:
+                access_filter = resolve_app_access_filter(current_tenant_id, current_user.id, session=session)
+            app_maintainers = workflow_service.get_tenant_app_maintainers(app_ids, current_tenant_id, session=session)
+
+        accessible_app_ids = set(app_maintainers)
+        if access_filter is not None:
+            accessible_app_ids = {
+                app_id
+                for app_id, maintainer in app_maintainers.items()
+                if access_filter.is_app_accessible(app_id, maintainer, current_user.id)
+            }
         ordered_accessible_app_ids = [app_id for app_id in app_ids if app_id in accessible_app_ids]
 
         users_json_by_app_id: dict[str, Any] = {}

api/controllers/console/app/workflow_comment.py

@@ -3,6 +3,7 @@ from datetime import datetime
 
 from flask_restx import Resource
 from pydantic import BaseModel, Field, TypeAdapter, computed_field, field_validator
+from sqlalchemy.orm import Session
 
 from controllers.common.schema import register_response_schema_models, register_schema_models
 from controllers.console import console_ns
@@ -21,6 +22,15 @@ from fields.member_fields import AccountWithRole
 from libs.helper import build_avatar_url, dump_response, to_timestamp
 from libs.login import login_required
 from models import Account, App
+from models.comment import (
+    WorkflowComment as WorkflowCommentModel,
+)
+from models.comment import (
+    WorkflowCommentMention as WorkflowCommentMentionModel,
+)
+from models.comment import (
+    WorkflowCommentReply as WorkflowCommentReplyModel,
+)
 from services.account_service import TenantService
 from services.workflow_comment_service import WorkflowCommentService
 
@@ -133,6 +143,73 @@ class WorkflowCommentDetail(ResponseModel):
         return to_timestamp(value)
 
 
+def _workflow_comment_reply_response(reply: WorkflowCommentReplyModel, *, session: Session) -> WorkflowCommentReply:
+    return WorkflowCommentReply.model_validate(
+        {
+            "id": reply.id,
+            "content": reply.content,
+            "created_by": reply.created_by,
+            "created_by_account": reply.created_by_account(session),
+            "created_at": reply.created_at,
+        }
+    )
+
+
+def _workflow_comment_mention_response(
+    mention: WorkflowCommentMentionModel, *, session: Session
+) -> WorkflowCommentMention:
+    return WorkflowCommentMention.model_validate(
+        {
+            "mentioned_user_id": mention.mentioned_user_id,
+            "mentioned_user_account": mention.mentioned_user_account(session),
+            "reply_id": mention.reply_id,
+        }
+    )
+
+
+def _workflow_comment_basic_response(comment: WorkflowCommentModel, *, session: Session) -> WorkflowCommentBasic:
+    return WorkflowCommentBasic.model_validate(
+        {
+            "id": comment.id,
+            "position_x": comment.position_x,
+            "position_y": comment.position_y,
+            "content": comment.content,
+            "created_by": comment.created_by,
+            "created_by_account": comment.created_by_account(session),
+            "created_at": comment.created_at,
+            "updated_at": comment.updated_at,
+            "resolved": comment.resolved,
+            "resolved_at": comment.resolved_at,
+            "resolved_by": comment.resolved_by,
+            "resolved_by_account": comment.resolved_by_account(session),
+            "reply_count": comment.reply_count,
+            "mention_count": comment.mention_count,
+            "participants": comment.participants(session),
+        }
+    )
+
+
+def _workflow_comment_detail_response(comment: WorkflowCommentModel, *, session: Session) -> WorkflowCommentDetail:
+    return WorkflowCommentDetail.model_validate(
+        {
+            "id": comment.id,
+            "position_x": comment.position_x,
+            "position_y": comment.position_y,
+            "content": comment.content,
+            "created_by": comment.created_by,
+            "created_by_account": comment.created_by_account(session),
+            "created_at": comment.created_at,
+            "updated_at": comment.updated_at,
+            "resolved": comment.resolved,
+            "resolved_at": comment.resolved_at,
+            "resolved_by": comment.resolved_by,
+            "resolved_by_account": comment.resolved_by_account(session),
+            "replies": [_workflow_comment_reply_response(reply, session=session) for reply in comment.replies],
+            "mentions": [_workflow_comment_mention_response(mention, session=session) for mention in comment.mentions],
+        }
+    )
+
+
 class WorkflowCommentCreate(ResponseModel):
     id: str
     created_at: int | None = None
@@ -225,8 +302,11 @@ class WorkflowCommentListApi(Resource):
     def get(self, current_tenant_id: str, app_model: App):
         """Get all comments for a workflow."""
         comments = WorkflowCommentService.get_comments(tenant_id=current_tenant_id, app_id=app_model.id)
+        session = db.session()
 
-        return WorkflowCommentBasicList.model_validate({"data": comments}).model_dump(mode="json")
+        return WorkflowCommentBasicList.model_validate(
+            {"data": [_workflow_comment_basic_response(comment, session=session) for comment in comments]}
+        ).model_dump(mode="json")
 
     @console_ns.doc("create_workflow_comment")
     @console_ns.doc(description="Create a new workflow comment")
@@ -281,8 +361,9 @@ class WorkflowCommentDetailApi(Resource):
         comment = WorkflowCommentService.get_comment(
             tenant_id=current_tenant_id, app_id=app_model.id, comment_id=comment_id
         )
+        session = db.session()
 
-        return dump_response(WorkflowCommentDetail, comment)
+        return dump_response(WorkflowCommentDetail, _workflow_comment_detail_response(comment, session=session))
 
     @console_ns.doc("update_workflow_comment")
     @console_ns.doc(description="Update a workflow comment")

api/controllers/console/app/wraps.py

@@ -1,8 +1,9 @@
 """Controller decorators for console app resources.
 
 `get_app_model` still supports legacy handlers backed by Flask-SQLAlchemy's
-scoped session. Trial app handlers compose `get_app_model_with_trial` under
-`controllers.common.session.with_session` and always reuse that request session.
+scoped session. Preview handlers compose `get_previewable_app_model` under
+`controllers.common.session.with_session`; preview admission finishes before
+the request Session loads the accepted App.
 """
 
 from collections.abc import Callable
@@ -16,16 +17,17 @@ from configs import dify_config
 from controllers.common.session import with_session
 from controllers.common.wraps import RBACPermission, RBACResourceScope, enforce_rbac_access
 from controllers.console.app.error import AppNotFoundError
+from extensions.ext_application_services import application_services
 from extensions.ext_database import db
 from libs.login import current_account_with_tenant
-from models import App, AppMode, TrialApp
+from models import App, AppMode
 from models.agent import AgentScope
-from services.recommended_app_service import RecommendedAppService
+from services.app_service import AppService
 
 __all__ = [
     "agent_manage_required_for_agent_app",
     "get_app_model",
-    "get_app_model_with_trial",
+    "get_previewable_app_model",
     "with_session",
 ]
 
@@ -48,12 +50,11 @@ def _load_app_model_from_scoped_session(app_id: str) -> App | None:
     return app_model
 
 
-def _load_app_model_with_trial(session: Session, app_id: str) -> App | None:
-    """Load a normal app through its trial registration without applying current-tenant scope."""
-    app_model = session.scalar(
-        select(App).join(TrialApp, TrialApp.app_id == App.id).where(App.id == app_id, App.status == "normal").limit(1)
-    )
-    return app_model
+def _load_previewable_app_model(session: Session, app_id: str) -> App | None:
+    """Load a normal App after preview admission completes outside the request Session."""
+    if not application_services().recommended_app_queries.is_previewable(app_id):
+        return None
+    return AppService.get_normal_app_by_id(app_id, session)
 
 
 def agent_manage_required_for_agent_app[**P, R](view: Callable[P, R]) -> Callable[P, R]:
@@ -183,7 +184,7 @@ def get_app_model[**P, R](
 
 
 @overload
-def get_app_model_with_trial[**P, R](
+def get_previewable_app_model[**P, R](
     view: Callable[P, R],
     *,
     mode: AppMode | list[AppMode] | None = None,
@@ -191,19 +192,25 @@ def get_app_model_with_trial[**P, R](
 
 
 @overload
-def get_app_model_with_trial[**P, R](
+def get_previewable_app_model[**P, R](
     view: None = None,
     *,
     mode: AppMode | list[AppMode] | None = None,
 ) -> Callable[[Callable[P, R]], Callable[P, R]]: ...
 
 
-def get_app_model_with_trial[**P, R](
+def get_previewable_app_model[**P, R](
     view: Callable[P, R] | None = None,
     *,
     mode: AppMode | list[AppMode] | None = None,
 ) -> Callable[P, R] | Callable[[Callable[P, R]], Callable[P, R]]:
-    """Inject a trial-registered or recommended App using the Session supplied by `with_session`."""
+    """Inject an App authorized for read-only template preview.
+
+    Preview reads accept either an explicit TrialApp registration or membership
+    in the recommended catalog. This does not grant trial execution, which is
+    separately protected by TrialAppResource's feature, registration, and quota
+    checks.
+    """
 
     def decorator(view_func: Callable[P, R]) -> Callable[P, R]:
         @wraps(view_func)
@@ -218,10 +225,8 @@ def get_app_model_with_trial[**P, R](
 
             session = _get_injected_session(args)
             if session is None:
-                raise RuntimeError("get_app_model_with_trial requires @with_session")
-            app_model = _load_app_model_with_trial(session, app_id)
-            if app_model is None:
-                app_model = RecommendedAppService.get_app(app_id, session=session)
+                raise RuntimeError("get_previewable_app_model requires @with_session")
+            app_model = _load_previewable_app_model(session, app_id)
 
             if not app_model:
                 raise AppNotFoundError()

api/controllers/console/auth/activate.py

@@ -1,24 +1,25 @@
 from flask import request
 from flask_restx import Resource
 from pydantic import BaseModel, Field, field_validator
-from sqlalchemy import select
 
-from configs import dify_config
 from constants.languages import supported_language
 from controllers.common.schema import query_params_from_model, register_schema_models
 from controllers.console import console_ns
-from controllers.console.auth.error import InvitationAccountMismatchError
-from controllers.console.error import AccountInFreezeError, AlreadyActivateError
-from controllers.console.wraps import model_validate
-from extensions.ext_database import db
-from libs.datetime_utils import naive_utc_now
-from libs.helper import EmailStr, timezone
+from controllers.console.auth.error import InvitationAccountMismatchError as InvitationAccountMismatchHTTPError
+from controllers.console.error import AccountInFreezeError, AlreadyActivateError, EmailDomainSuspendedError
+from extensions.ext_application_services import application_services
+from libs.helper import EmailStr, dump_response, timezone
 from libs.login import current_account_with_tenant
 from libs.token import extract_access_token
-from models import AccountStatus
-from models.account import TenantAccountJoin, TenantAccountRole
-from services.account_service import RegisterService, TenantService
-from services.billing_service import BillingService
+from services.account_activation_service import (
+    EmailDomainSuspendedError as EmailDomainSuspendedRegistrationError,
+)
+from services.account_activation_service import (
+    FrozenAccountError,
+    InvalidInvitationError,
+    InvitationAccountMismatchError,
+)
+from services.entities.account_activation_entities import ActivationCommand, InvitationLookup
 
 
 class ActivateCheckQuery(BaseModel):
@@ -87,45 +88,19 @@ class ActivateCheckApi(Resource):
         "Success",
         console_ns.models[ActivationCheckResponse.__name__],
     )
-    @model_validate(ActivateCheckQuery)
-    def get(self, req_data: ActivateCheckQuery):
-
-        workspaceId = req_data.workspace_id
-        token = req_data.token
-
-        invitation = RegisterService.get_invitation_with_case_fallback(
-            workspaceId, req_data.email, token, session=db.session()
+    def get(self):
+        args = ActivateCheckQuery.model_validate(request.args.to_dict(flat=True))
+        result = application_services().account_activation.check(
+            InvitationLookup(
+                workspace_id=args.workspace_id,
+                email=args.email,
+                token=args.token,
+            ),
+        )
+        return ActivationCheckResponse.model_validate(result, from_attributes=True).model_dump(
+            mode="json",
+            exclude_none=True,
         )
-        if invitation:
-            data = invitation.get("data", {})
-            tenant = invitation.get("tenant", None)
-
-            # Check workspace permission
-            if tenant:
-                from libs.workspace_permission import check_workspace_member_invite_permission
-
-                check_workspace_member_invite_permission(tenant.id)
-
-            workspace_name = tenant.name if tenant else None
-            workspace_id = tenant.id if tenant else None
-            invitee_email = data.get("email") if data else None
-            account = invitation.get("account")
-            account_status = account.status if account else None
-            requires_setup = data.get("requires_setup")
-            if requires_setup is None:
-                requires_setup = account_status == AccountStatus.PENDING
-            return {
-                "is_valid": invitation is not None,
-                "data": {
-                    "workspace_name": workspace_name,
-                    "workspace_id": workspace_id,
-                    "email": invitee_email,
-                    "account_status": account_status,
-                    "requires_setup": requires_setup,
-                },
-            }
-        else:
-            return {"is_valid": False}
 
 
 @console_ns.route("/activate")
@@ -139,70 +114,39 @@ class ActivateApi(Resource):
         console_ns.models[ActivationResponse.__name__],
     )
     @console_ns.response(400, "Already activated or invalid token")
-    @model_validate(ActivatePayload)
-    def post(self, req_data: ActivatePayload):
+    def post(self):
         """Accept an invitation without letting an existing session act for another account.
 
         Token-only activation remains available for legacy clients. When the request already
         carries a console session, that session must belong to the account encoded in the
         invitation before the token is consumed or tenant membership is changed.
         """
+        args = ActivatePayload.model_validate(console_ns.payload or {})
+        authenticated_account_id: str | None = None
+        if extract_access_token(request) is not None:
+            authenticated_account_id = current_account_with_tenant().account.id
 
-        normalized_request_email = req_data.email.lower() if req_data.email else None
-        invitation = RegisterService.get_invitation_with_case_fallback(
-            req_data.workspace_id, req_data.email, req_data.token, session=db.session()
-        )
-        if invitation is None:
-            raise AlreadyActivateError()
-
-        account = invitation["account"]
-        if extract_access_token(request):
-            current_account, _ = current_account_with_tenant()
-            if current_account.id != account.id:
-                raise InvitationAccountMismatchError()
-
-        if dify_config.BILLING_ENABLED and BillingService.is_email_in_freeze(account.email):
-            raise AccountInFreezeError()
-
-        tenant = invitation["tenant"]
-        raw_role = invitation["data"].get("role")
         try:
-            role = TenantAccountRole(raw_role) if raw_role else TenantAccountRole.NORMAL
-        except ValueError:
-            role = TenantAccountRole.NORMAL
-        if not TenantAccountRole.is_non_owner_role(role):
-            role = TenantAccountRole.NORMAL
-
-        membership_id = db.session.scalar(
-            select(TenantAccountJoin.id).where(
-                TenantAccountJoin.tenant_id == tenant.id,
-                TenantAccountJoin.account_id == account.id,
+            application_services().account_activation.activate(
+                ActivationCommand(
+                    invitation=InvitationLookup(
+                        workspace_id=args.workspace_id,
+                        email=args.email,
+                        token=args.token,
+                    ),
+                    name=args.name,
+                    interface_language=args.interface_language,
+                    timezone=args.timezone,
+                ),
+                authenticated_account_id=authenticated_account_id,
             )
-        )
+        except InvalidInvitationError:
+            raise AlreadyActivateError() from None
+        except InvitationAccountMismatchError:
+            raise InvitationAccountMismatchHTTPError() from None
+        except EmailDomainSuspendedRegistrationError:
+            raise EmailDomainSuspendedError() from None
+        except FrozenAccountError:
+            raise AccountInFreezeError() from None
 
-        requires_setup = invitation["data"].get("requires_setup")
-        if requires_setup is None:
-            requires_setup = account.status == AccountStatus.PENDING
-
-        setup_fields: tuple[str, str, str] | None = None
-        if requires_setup:
-            if not req_data.name or not req_data.interface_language or not req_data.timezone:
-                raise AlreadyActivateError()
-            setup_fields = (req_data.name, req_data.interface_language, req_data.timezone)
-
-        RegisterService.revoke_token(req_data.workspace_id, normalized_request_email, req_data.token)
-
-        if membership_id is None:
-            TenantService.create_tenant_member(tenant, account, db.session(), role=role)
-
-        if setup_fields:
-            account.name = setup_fields[0]
-            account.interface_language = setup_fields[1]
-            account.timezone = setup_fields[2]
-            account.interface_theme = "light"
-            account.status = AccountStatus.ACTIVE
-            account.initialized_at = naive_utc_now()
-
-        TenantService.switch_tenant(account, tenant.id, session=db.session())
-
-        return {"result": "success"}
+        return dump_response(ActivationResponse, {"result": "success"})

api/controllers/console/auth/data_source_bearer_auth.py

@@ -1,33 +1,52 @@
+from http import HTTPStatus
 from uuid import UUID
 
 from flask_restx import Resource
-from pydantic import BaseModel, Field
+from pydantic import BaseModel, ConfigDict, Field
 
 from controllers.common.fields import SimpleResultResponse
 from controllers.common.schema import register_response_schema_models, register_schema_models
-from extensions.ext_database import db
+from controllers.console import console_ns
+from controllers.console.auth.error import (
+    DataSourceApiKeyAuthCredentialsRejectedRequestError,
+    DataSourceApiKeyAuthProviderNotSupportedError,
+    DataSourceApiKeyAuthProviderUnavailableRequestError,
+    InvalidDataSourceApiKeyAuthCredentialsRequestError,
+)
+from controllers.console.flask_admission import console_account_admission
+from controllers.console.wraps import RBACPermission, RBACResourceScope, model_validate
+from extensions.ext_application_services import application_services
 from fields.base import ResponseModel
-from libs.login import login_required
-from services.auth.api_key_auth_service import ApiKeyAuthService
-
-from .. import console_ns
-from ..auth.error import ApiKeyAuthFailedError
-from ..wraps import (
-    RBACPermission,
-    RBACResourceScope,
-    account_initialization_required,
-    is_admin_or_owner_required,
-    model_validate,
-    rbac_permission_required,
-    setup_required,
-    with_current_tenant_id,
+from libs.helper import dump_response
+from machinery.context import RequestContext
+from models.account import TenantAccountRole
+from services.auth.errors import (
+    DataSourceApiKeyAuthCredentialValidationError,
+    DataSourceApiKeyAuthProviderUnavailableError,
+    InvalidDataSourceApiKeyAuthCredentialsError,
+    UnsupportedDataSourceApiKeyAuthProviderError,
+)
+from services.entities.data_source_api_key_auth_entities import (
+    DataSourceApiKeyAuthBindingCreate,
+    DataSourceApiKeyAuthCredentials,
 )
 
 
+class ApiKeyAuthConfigPayload(BaseModel):
+    model_config = ConfigDict(extra="allow")
+
+    api_key: str = Field(min_length=1)
+
+
+class ApiKeyAuthCredentialsPayload(BaseModel):
+    auth_type: str = Field(min_length=1)
+    config: ApiKeyAuthConfigPayload
+
+
 class ApiKeyAuthBindingPayload(BaseModel):
-    category: str = Field(...)
-    provider: str = Field(...)
-    credentials: dict = Field(...)
+    category: str = Field(min_length=1)
+    provider: str = Field(min_length=1)
+    credentials: ApiKeyAuthCredentialsPayload
 
 
 class ApiKeyAuthDataSourceItem(ResponseModel):
@@ -52,65 +71,77 @@ register_response_schema_models(
 )
 
 
+_ADMIN_OR_OWNER_ROLES = frozenset({TenantAccountRole.ADMIN, TenantAccountRole.OWNER})
+
+
 @console_ns.route("/api-key-auth/data-source")
 class ApiKeyAuthDataSource(Resource):
-    @console_ns.response(200, "Success", console_ns.models[ApiKeyAuthDataSourceListResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_current_tenant_id
-    def get(self, current_tenant_id: str):
-        data_source_api_key_bindings = ApiKeyAuthService.get_provider_auth_list(current_tenant_id, session=db.session())
-        if data_source_api_key_bindings:
-            return {
+    @console_ns.response(HTTPStatus.OK, "Success", console_ns.models[ApiKeyAuthDataSourceListResponse.__name__])
+    @console_account_admission()
+    def get(self, request_context: RequestContext):
+        bindings = application_services().data_source_api_key_auth.list_bindings(request_context)
+        return dump_response(
+            ApiKeyAuthDataSourceListResponse,
+            {
                 "sources": [
                     {
-                        "id": data_source_api_key_binding.id,
-                        "category": data_source_api_key_binding.category,
-                        "provider": data_source_api_key_binding.provider,
-                        "disabled": data_source_api_key_binding.disabled,
-                        "created_at": int(data_source_api_key_binding.created_at.timestamp()),
-                        "updated_at": int(data_source_api_key_binding.updated_at.timestamp()),
+                        "id": binding.id,
+                        "category": binding.category,
+                        "provider": binding.provider,
+                        "disabled": binding.disabled,
+                        "created_at": int(binding.created_at.timestamp()),
+                        "updated_at": int(binding.updated_at.timestamp()),
                     }
-                    for data_source_api_key_binding in data_source_api_key_bindings
+                    for binding in bindings
                 ]
-            }
-        return {"sources": []}
+            },
+        )
 
 
 @console_ns.route("/api-key-auth/data-source/binding")
 class ApiKeyAuthDataSourceBinding(Resource):
-    @console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @is_admin_or_owner_required
-    @rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.CREDENTIAL_CREATE, resource_required=False)
+    @console_ns.response(HTTPStatus.OK, "Success", console_ns.models[SimpleResultResponse.__name__])
     @console_ns.expect(console_ns.models[ApiKeyAuthBindingPayload.__name__])
-    @with_current_tenant_id
+    @console_account_admission(
+        allowed_roles=_ADMIN_OR_OWNER_ROLES,
+        rbac_resource_scope=RBACResourceScope.WORKSPACE,
+        rbac_permission=RBACPermission.CREDENTIAL_CREATE,
+        rbac_resource_required=False,
+    )
     @model_validate(ApiKeyAuthBindingPayload)
-    def post(self, req_data: ApiKeyAuthBindingPayload, current_tenant_id: str):
-        # The role of the current user in the table must be admin or owner
-        data = req_data.model_dump()
-        ApiKeyAuthService.validate_api_key_auth_args(data)
+    def post(self, req_data: ApiKeyAuthBindingPayload, request_context: RequestContext):
+        credential_options = req_data.credentials.config.model_dump(exclude={"api_key"})
+        command = DataSourceApiKeyAuthBindingCreate(
+            category=req_data.category,
+            provider=req_data.provider,
+            credentials=DataSourceApiKeyAuthCredentials(
+                auth_type=req_data.credentials.auth_type,
+                api_key=req_data.credentials.config.api_key,
+                options=credential_options,
+            ),
+        )
         try:
-            ApiKeyAuthService.create_provider_auth(current_tenant_id, data, session=db.session())
-        except Exception as e:
-            raise ApiKeyAuthFailedError(str(e))
-        return {"result": "success"}, 200
+            application_services().data_source_api_key_auth.create_binding(request_context, command)
+        except UnsupportedDataSourceApiKeyAuthProviderError as exc:
+            raise DataSourceApiKeyAuthProviderNotSupportedError() from exc
+        except InvalidDataSourceApiKeyAuthCredentialsError as exc:
+            raise InvalidDataSourceApiKeyAuthCredentialsRequestError(description=str(exc)) from exc
+        except DataSourceApiKeyAuthCredentialValidationError as exc:
+            raise DataSourceApiKeyAuthCredentialsRejectedRequestError(description=str(exc)) from exc
+        except DataSourceApiKeyAuthProviderUnavailableError as exc:
+            raise DataSourceApiKeyAuthProviderUnavailableRequestError() from exc
+        return dump_response(SimpleResultResponse, {"result": "success"}), HTTPStatus.OK
 
 
 @console_ns.route("/api-key-auth/data-source/<uuid:binding_id>")
 class ApiKeyAuthDataSourceBindingDelete(Resource):
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @is_admin_or_owner_required
-    @rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.CREDENTIAL_MANAGE, resource_required=False)
-    @console_ns.response(204, "Binding deleted successfully")
-    @with_current_tenant_id
-    def delete(self, current_tenant_id: str, binding_id: UUID):
-        # The role of the current user in the table must be admin or owner
-        ApiKeyAuthService.delete_provider_auth(current_tenant_id, str(binding_id), session=db.session())
-
-        return "", 204
+    @console_ns.response(HTTPStatus.NO_CONTENT, "Binding deleted successfully")
+    @console_account_admission(
+        allowed_roles=_ADMIN_OR_OWNER_ROLES,
+        rbac_resource_scope=RBACResourceScope.WORKSPACE,
+        rbac_permission=RBACPermission.CREDENTIAL_MANAGE,
+        rbac_resource_required=False,
+    )
+    def delete(self, request_context: RequestContext, binding_id: UUID):
+        application_services().data_source_api_key_auth.delete_binding(request_context, str(binding_id))
+        return "", HTTPStatus.NO_CONTENT

api/controllers/console/auth/email_register.py

@@ -13,8 +13,10 @@ from controllers.console.auth.error import (
     EmailRegisterLimitError,
     InvalidEmailError,
     InvalidTokenError,
+    NormalizedEmailAlreadyInUseError,
     PasswordMismatchError,
 )
+from enums import DeploymentEdition
 from extensions.ext_database import db
 from fields.base import ResponseModel
 from libs.helper import EmailStr, extract_remote_ip
@@ -23,9 +25,16 @@ from libs.password import valid_password
 from models import Account
 from services.account_service import AccountService
 from services.billing_service import BillingService
-from services.errors.account import AccountRegisterError, SeatsLimitExceededError
+from services.errors.account import (
+    AccountNormalizedEmailAlreadyInUseError,
+    AccountRegisterError,
+    SeatsLimitExceededError,
+)
+from services.errors.account import (
+    EmailDomainSuspendedError as EmailDomainSuspendedRegistrationError,
+)
 
-from ..error import AccountInFreezeError, EmailSendIpLimitError, SeatsLimitExceeded
+from ..error import AccountInFreezeError, EmailDomainSuspendedError, EmailSendIpLimitError, SeatsLimitExceeded
 from ..wraps import email_password_login_enabled, email_register_enabled, model_validate, setup_required
 
 
@@ -98,8 +107,12 @@ class EmailRegisterSendEmailApi(Resource):
         if req_data.language is not None and req_data.language in languages:
             language = req_data.language
 
-        if dify_config.BILLING_ENABLED and BillingService.is_email_in_freeze(normalized_email):
-            raise AccountInFreezeError()
+        if dify_config.DEPLOYMENT_EDITION == DeploymentEdition.CLOUD:
+            freeze_type = BillingService.get_email_freeze_type(normalized_email)
+            if freeze_type:
+                if freeze_type == "email_domain_suspended":
+                    raise EmailDomainSuspendedError()
+                raise AccountInFreezeError()
 
         account = AccountService.get_account_by_email_with_case_fallback(req_data.email, session=db.session())
         token = AccountService.send_email_register_email(email=normalized_email, account=account, language=language)
@@ -181,13 +194,15 @@ class EmailRegisterResetApi(Resource):
         if account:
             raise EmailAlreadyInUseError()
 
+        ip_address = extract_remote_ip(request)
         account = self._create_new_account(
             email=normalized_email,
             password=req_data.password_confirm,
             timezone=req_data.timezone,
             language=req_data.language,
+            ip_address=ip_address,
         )
-        token_pair = AccountService.login(account=account, session=db.session(), ip_address=extract_remote_ip(request))
+        token_pair = AccountService.login(account=account, session=db.session(), ip_address=ip_address)
         AccountService.reset_login_error_rate_limit(normalized_email)
 
         return {"result": "success", "data": token_pair.model_dump()}
@@ -198,6 +213,7 @@ class EmailRegisterResetApi(Resource):
         password: str,
         timezone: str | None = None,
         language: str | None = None,
+        ip_address: str | None = None,
     ) -> Account:
         try:
             return AccountService.create_account_and_tenant(
@@ -206,9 +222,15 @@ class EmailRegisterResetApi(Resource):
                 password=password,
                 interface_language=get_valid_language(language),
                 timezone=timezone,
+                ip_address=ip_address,
+                check_normalized_email=True,
                 session=db.session(),
             )
         except SeatsLimitExceededError:
             raise SeatsLimitExceeded()
-        except AccountRegisterError:
-            raise AccountInFreezeError()
+        except EmailDomainSuspendedRegistrationError as exc:
+            raise EmailDomainSuspendedError() from exc
+        except AccountNormalizedEmailAlreadyInUseError as exc:
+            raise NormalizedEmailAlreadyInUseError() from exc
+        except AccountRegisterError as exc:
+            raise AccountInFreezeError() from exc

api/controllers/console/auth/error.py

@@ -1,10 +1,28 @@
 from libs.exception import BaseHTTPException
 
 
-class ApiKeyAuthFailedError(BaseHTTPException):
-    error_code = "auth_failed"
-    description = "{message}"
-    code = 500
+class DataSourceApiKeyAuthProviderNotSupportedError(BaseHTTPException):
+    error_code = "unsupported_data_source_api_key_auth_provider"
+    description = "The data-source API-key authentication provider is not supported."
+    code = 400
+
+
+class InvalidDataSourceApiKeyAuthCredentialsRequestError(BaseHTTPException):
+    error_code = "invalid_data_source_api_key_auth_credentials"
+    description = "The data-source API-key authentication credentials are invalid."
+    code = 400
+
+
+class DataSourceApiKeyAuthCredentialsRejectedRequestError(BaseHTTPException):
+    error_code = "data_source_api_key_auth_credentials_rejected"
+    description = "The data-source provider rejected the API-key authentication credentials."
+    code = 400
+
+
+class DataSourceApiKeyAuthProviderUnavailableRequestError(BaseHTTPException):
+    error_code = "data_source_api_key_auth_provider_unavailable"
+    description = "The data-source API-key authentication provider is temporarily unavailable."
+    code = 502
 
 
 class InvalidEmailError(BaseHTTPException):
@@ -77,6 +95,12 @@ class EmailCodeError(BaseHTTPException):
     code = 400
 
 
+class EmailCodeLoginServiceUnavailableError(BaseHTTPException):
+    error_code = "email_code_login_service_unavailable"
+    description = "Email code verification is temporarily unavailable. Please try again later."
+    code = 503
+
+
 class EmailOrPasswordMismatchError(BaseHTTPException):
     error_code = "email_or_password_mismatch"
     description = "The email or password is mismatched."
@@ -95,6 +119,18 @@ class EmailPasswordLoginLimitError(BaseHTTPException):
     code = 429
 
 
+class TurnstileVerificationFailedError(BaseHTTPException):
+    error_code = "turnstile_verification_failed"
+    description = "Turnstile verification failed. Please try again."
+    code = 400
+
+
+class TurnstileServiceUnavailableError(BaseHTTPException):
+    error_code = "turnstile_service_unavailable"
+    description = "Turnstile verification is temporarily unavailable. Please try again later."
+    code = 503
+
+
 class EmailCodeLoginRateLimitExceededError(BaseHTTPException):
     error_code = "email_code_login_rate_limit_exceeded"
     description = "Too many login emails have been sent. Please try again in {minutes} minutes."
@@ -139,6 +175,11 @@ class EmailAlreadyInUseError(BaseHTTPException):
     code = 400
 
 
+class NormalizedEmailAlreadyInUseError(EmailAlreadyInUseError):
+    error_code = "normalized_email_already_in_use"
+    description = "An account with an equivalent email address already exists."
+
+
 class OwnerTransferLimitError(BaseHTTPException):
     error_code = "owner_transfer_limit"
     description = "Too many failed owner transfer attempts. Please try again in 24 hours."

api/controllers/console/auth/login.py

@@ -1,4 +1,5 @@
 import logging
+from uuid import UUID
 
 import flask_login
 from flask import make_response, request
@@ -22,14 +23,19 @@ from controllers.console import console_ns
 from controllers.console.auth.error import (
     AuthenticationFailedError,
     EmailCodeError,
+    EmailCodeLoginServiceUnavailableError,
     EmailPasswordLoginLimitError,
     InvalidEmailError,
     InvalidTokenError,
+    NormalizedEmailAlreadyInUseError,
+    TurnstileServiceUnavailableError,
+    TurnstileVerificationFailedError,
 )
 from controllers.console.error import (
     AccountBannedError,
     AccountInFreezeError,
     AccountNotFound,
+    EmailDomainSuspendedError,
     EmailSendIpLimitError,
     NotAllowedCreateWorkspace,
     SeatsLimitExceeded,
@@ -43,6 +49,7 @@ from controllers.console.wraps import (
     setup_required,
     with_current_user,
 )
+from enums import DeploymentEdition
 from extensions.ext_database import db
 from libs.helper import EmailStr, extract_remote_ip
 from libs.helper import timezone as validate_timezone_string
@@ -58,15 +65,29 @@ from libs.token import (
 from models.account import Account
 from services.account_service import AccountService, InvitationDetailDict, RegisterService, TenantService
 from services.billing_service import BillingService
+from services.email_code_login_challenge import (
+    EmailCodeLoginChallengeStatus,
+    EmailCodeLoginChallengeUnavailableError,
+)
 from services.entities.auth_entities import LoginFailureReason, LoginPayloadBase
 from services.errors.account import (
+    AccountNormalizedEmailAlreadyInUseError,
     AccountRegisterError,
     RefreshTokenAccountNotFoundError,
     RefreshTokenNotFoundError,
     SeatsLimitExceededError,
 )
+from services.errors.account import (
+    EmailDomainSuspendedError as EmailDomainSuspendedRegistrationError,
+)
 from services.errors.workspace import WorkSpaceNotAllowedCreateError, WorkspacesLimitExceededError
 from services.feature_service import FeatureService
+from services.turnstile_service import (
+    EMAIL_CODE_VERIFY_ACTION,
+    TurnstileChallengeRejectedError,
+    TurnstileService,
+    TurnstileUpstreamError,
+)
 
 logger = logging.getLogger(__name__)
 
@@ -81,10 +102,23 @@ class EmailPayload(BaseModel):
     language: str | None = Field(default=None)
 
 
+class EmailCodeSendPayload(EmailPayload):
+    turnstile_token: str | None = Field(
+        default=None,
+        max_length=2048,
+        description="Cloudflare Turnstile token. Required at runtime for Dify Cloud.",
+    )
+
+
 class EmailCodeLoginPayload(BaseModel):
     email: EmailStr = Field(...)
-    code: str = Field(...)
-    token: str = Field(...)
+    code: str
+    token: UUID
+    turnstile_token: str | None = Field(
+        default=None,
+        max_length=2048,
+        description="Cloudflare Turnstile token for email-code verification.",
+    )
     language: str | None = Field(default=None)
     timezone: str | None = Field(default=None)
 
@@ -96,7 +130,7 @@ class EmailCodeLoginPayload(BaseModel):
         return validate_timezone_string(value)
 
 
-register_schema_models(console_ns, LoginPayload, EmailPayload, EmailCodeLoginPayload)
+register_schema_models(console_ns, LoginPayload, EmailPayload, EmailCodeSendPayload, EmailCodeLoginPayload)
 register_response_schema_models(
     console_ns,
     SimpleResultDataResponse,
@@ -121,9 +155,13 @@ class LoginApi(Resource):
         request_email = req_data.email
         normalized_email = request_email.lower()
 
-        if dify_config.BILLING_ENABLED and BillingService.is_email_in_freeze(normalized_email):
-            _log_console_login_failure(email=normalized_email, reason=LoginFailureReason.ACCOUNT_IN_FREEZE)
-            raise AccountInFreezeError()
+        if dify_config.DEPLOYMENT_EDITION == DeploymentEdition.CLOUD:
+            freeze_type = BillingService.get_email_freeze_type(normalized_email)
+            if freeze_type:
+                _log_console_login_failure(email=normalized_email, reason=LoginFailureReason.ACCOUNT_IN_FREEZE)
+                if freeze_type == "email_domain_suspended":
+                    raise EmailDomainSuspendedError()
+                raise AccountInFreezeError()
 
         is_login_error_rate_limit = AccountService.is_login_error_rate_limit(normalized_email)
         if is_login_error_rate_limit:
@@ -160,7 +198,6 @@ class LoginApi(Resource):
             AccountService.add_login_error_rate_limit(normalized_email)
             _log_console_login_failure(email=normalized_email, reason=LoginFailureReason.INVALID_CREDENTIALS)
             raise AuthenticationFailedError() from exc
-        # SELF_HOSTED only have one workspace
         tenants = TenantService.get_join_tenants(account, session=db.session())
         if len(tenants) == 0:
             if (
@@ -226,8 +263,10 @@ class ResetPasswordSendEmailApi(Resource):
             language = "en-US"
         try:
             account = _get_account_with_case_fallback(req_data.email)
-        except AccountRegisterError:
-            raise AccountInFreezeError()
+        except EmailDomainSuspendedRegistrationError as exc:
+            raise EmailDomainSuspendedError() from exc
+        except AccountRegisterError as exc:
+            raise AccountInFreezeError() from exc
 
         token = AccountService.send_reset_password_email(
             email=normalized_email,
@@ -242,24 +281,36 @@ class ResetPasswordSendEmailApi(Resource):
 @console_ns.route("/email-code-login")
 class EmailCodeLoginSendEmailApi(Resource):
     @setup_required
-    @console_ns.expect(console_ns.models[EmailPayload.__name__])
+    @console_ns.expect(console_ns.models[EmailCodeSendPayload.__name__])
     @console_ns.response(200, "Success", console_ns.models[SimpleResultDataResponse.__name__])
-    @model_validate(EmailPayload)
-    def post(self, req_data: EmailPayload):
+    @model_validate(EmailCodeSendPayload)
+    def post(self, req_data: EmailCodeSendPayload):
         normalized_email = req_data.email.lower()
 
         ip_address = extract_remote_ip(request)
         if AccountService.is_email_send_ip_limit(ip_address):
             raise EmailSendIpLimitError()
 
+        if dify_config.DEPLOYMENT_EDITION == DeploymentEdition.CLOUD:
+            try:
+                TurnstileService.verify(token=req_data.turnstile_token, remote_ip=ip_address)
+            except TurnstileChallengeRejectedError as exc:
+                logger.info("Turnstile rejected an email-code login challenge")
+                raise TurnstileVerificationFailedError() from exc
+            except TurnstileUpstreamError as exc:
+                logger.warning("Turnstile verification is unavailable", exc_info=True)
+                raise TurnstileServiceUnavailableError() from exc
+
         if req_data.language is not None and req_data.language == "zh-Hans":
             language = "zh-Hans"
         else:
             language = "en-US"
         try:
             account = _get_account_with_case_fallback(req_data.email)
-        except AccountRegisterError:
-            raise AccountInFreezeError()
+        except EmailDomainSuspendedRegistrationError as exc:
+            raise EmailDomainSuspendedError() from exc
+        except AccountRegisterError as exc:
+            raise AccountInFreezeError() from exc
 
         if account is None:
             if FeatureService.get_system_features().is_allow_register:
@@ -284,31 +335,66 @@ class EmailCodeLoginApi(Resource):
         original_email = req_data.email
         user_email = original_email.lower()
         language = req_data.language
+        ip_address = extract_remote_ip(request)
 
-        token_data = AccountService.get_email_code_login_data(req_data.token)
-        if token_data is None:
-            _log_console_login_failure(email=user_email, reason=LoginFailureReason.INVALID_EMAIL_CODE_TOKEN)
-            raise InvalidTokenError()
-
-        token_email = token_data.get("email")
-        normalized_token_email = token_email.lower() if isinstance(token_email, str) else token_email
-        if normalized_token_email != user_email:
-            _log_console_login_failure(email=user_email, reason=LoginFailureReason.EMAIL_CODE_EMAIL_MISMATCH)
-            raise InvalidEmailError()
-
-        if token_data["code"] != req_data.code:
+        # ``code`` is Base64 on the wire and is decoded by
+        # ``decrypt_code_field`` before model validation reaches this handler.
+        if len(req_data.code) != 6 or not req_data.code.isascii() or not req_data.code.isdigit():
+            _log_console_login_failure(email=user_email, reason=LoginFailureReason.INVALID_EMAIL_CODE)
+            raise EmailCodeError()
+
+        if dify_config.DEPLOYMENT_EDITION == DeploymentEdition.CLOUD and (
+            dify_config.TURNSTILE_EMAIL_CODE_VERIFY_REQUIRED or req_data.turnstile_token
+        ):
+            try:
+                TurnstileService.verify(
+                    token=req_data.turnstile_token,
+                    remote_ip=ip_address,
+                    expected_action=EMAIL_CODE_VERIFY_ACTION,
+                )
+            except TurnstileChallengeRejectedError as exc:
+                logger.info("Turnstile rejected an email-code verification challenge")
+                raise TurnstileVerificationFailedError() from exc
+            except TurnstileUpstreamError as exc:
+                logger.warning("Turnstile verification is unavailable", exc_info=True)
+                raise TurnstileServiceUnavailableError() from exc
+
+        try:
+            verification = AccountService.verify_email_code_login_challenge(
+                email=user_email,
+                code=req_data.code,
+                token=str(req_data.token),
+            )
+        except EmailCodeLoginChallengeUnavailableError as exc:
+            logger.warning("Email-code challenge verification is unavailable", exc_info=True)
+            raise EmailCodeLoginServiceUnavailableError() from exc
+
+        if verification.status == EmailCodeLoginChallengeStatus.INVALID_TOKEN:
+            _log_console_login_failure(email=user_email, reason=LoginFailureReason.INVALID_EMAIL_CODE_TOKEN)
+            raise InvalidTokenError()
+
+        if verification.status == EmailCodeLoginChallengeStatus.EMAIL_MISMATCH:
+            _log_console_login_failure(email=user_email, reason=LoginFailureReason.EMAIL_CODE_EMAIL_MISMATCH)
+            raise InvalidEmailError()
+
+        if verification.status in {
+            EmailCodeLoginChallengeStatus.INVALID_CODE,
+            EmailCodeLoginChallengeStatus.EXHAUSTED,
+        }:
             _log_console_login_failure(email=user_email, reason=LoginFailureReason.INVALID_EMAIL_CODE)
             raise EmailCodeError()
 
-        AccountService.revoke_email_code_login_token(req_data.token)
         try:
             account = _get_account_with_case_fallback(original_email)
         except Unauthorized as exc:
             _log_console_login_failure(email=user_email, reason=LoginFailureReason.ACCOUNT_BANNED)
             raise AccountBannedError() from exc
-        except AccountRegisterError:
+        except EmailDomainSuspendedRegistrationError as exc:
             _log_console_login_failure(email=user_email, reason=LoginFailureReason.ACCOUNT_IN_FREEZE)
-            raise AccountInFreezeError()
+            raise EmailDomainSuspendedError() from exc
+        except AccountRegisterError as exc:
+            _log_console_login_failure(email=user_email, reason=LoginFailureReason.ACCOUNT_IN_FREEZE)
+            raise AccountInFreezeError() from exc
         if account:
             tenants = TenantService.get_join_tenants(account, session=db.session())
             if not tenants:
@@ -327,18 +413,25 @@ class EmailCodeLoginApi(Resource):
                     name=user_email,
                     interface_language=get_valid_language(language),
                     timezone=req_data.timezone,
+                    ip_address=ip_address,
+                    check_normalized_email=True,
                     session=db.session(),
                 )
             except WorkSpaceNotAllowedCreateError:
                 raise NotAllowedCreateWorkspace()
             except SeatsLimitExceededError:
                 raise SeatsLimitExceeded()
-            except AccountRegisterError:
+            except EmailDomainSuspendedRegistrationError as exc:
                 _log_console_login_failure(email=user_email, reason=LoginFailureReason.ACCOUNT_IN_FREEZE)
-                raise AccountInFreezeError()
+                raise EmailDomainSuspendedError() from exc
+            except AccountNormalizedEmailAlreadyInUseError as exc:
+                raise NormalizedEmailAlreadyInUseError() from exc
+            except AccountRegisterError as exc:
+                _log_console_login_failure(email=user_email, reason=LoginFailureReason.ACCOUNT_IN_FREEZE)
+                raise AccountInFreezeError() from exc
             except WorkspacesLimitExceededError:
                 raise WorkspacesLimitExceeded()
-        token_pair = AccountService.login(account, session=db.session(), ip_address=extract_remote_ip(request))
+        token_pair = AccountService.login(account, session=db.session(), ip_address=ip_address)
         AccountService.reset_login_error_rate_limit(user_email)
 
         # Create response with cookies instead of returning tokens in body

api/controllers/console/auth/oauth.py

@@ -12,6 +12,8 @@ from configs import dify_config
 from constants.languages import languages
 from controllers.common.fields import RedirectResponse
 from controllers.common.schema import query_params_from_model, register_response_schema_model, register_schema_models
+from controllers.console.error import AccountInFreezeError, EmailDomainSuspendedError
+from enums import DeploymentEdition
 from extensions.ext_database import db
 from libs.datetime_utils import naive_utc_now
 from libs.helper import extract_remote_ip
@@ -25,7 +27,14 @@ from libs.token import (
 from models import Account, AccountStatus
 from services.account_service import AccountService, RegisterService, TenantService
 from services.billing_service import BillingService
-from services.errors.account import AccountNotFoundError, AccountRegisterError, SeatsLimitExceededError
+from services.errors.account import (
+    AccountNotFoundError,
+    AccountRegisterError,
+    SeatsLimitExceededError,
+)
+from services.errors.account import (
+    EmailDomainSuspendedError as EmailDomainSuspendedRegistrationError,
+)
 from services.errors.workspace import WorkSpaceNotAllowedCreateError, WorkSpaceNotFoundError
 from services.feature_service import FeatureService
 
@@ -232,7 +241,13 @@ class OAuthCallback(Resource):
             return _redirect_with_console_session(account, target_url)
 
         try:
-            account, oauth_new_user = _generate_account(provider, user_info, timezone=timezone, language=language)
+            account, oauth_new_user = _generate_account(
+                provider,
+                user_info,
+                timezone=timezone,
+                language=language,
+                ip_address=extract_remote_ip(request),
+            )
         except AccountNotFoundError:
             return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Account not found.")
         except (WorkSpaceNotFoundError, WorkSpaceNotAllowedCreateError):
@@ -242,8 +257,10 @@ class OAuthCallback(Resource):
             )
         except SeatsLimitExceededError:
             return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Licensed seats limit exceeded.")
-        except AccountRegisterError as e:
-            return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message={e.description}")
+        except EmailDomainSuspendedRegistrationError:
+            return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message={EmailDomainSuspendedError.description}")
+        except AccountRegisterError as exc:
+            return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message={exc.description}")
 
         # Check account status
         if account.status == AccountStatus.BANNED:
@@ -284,6 +301,7 @@ def _generate_account(
     user_info: OAuthUserInfo,
     timezone: str | None = None,
     language: str | None = None,
+    ip_address: str | None = None,
 ) -> tuple[Account, bool]:
     # Get account by openid or email.
     account = _get_account_by_openid_or_email(provider, user_info)
@@ -301,13 +319,12 @@ def _generate_account(
         normalized_email = user_info.email.lower()
         oauth_new_user = True
         if not FeatureService.get_system_features().is_allow_register:
-            if dify_config.BILLING_ENABLED and BillingService.is_email_in_freeze(normalized_email):
-                raise AccountRegisterError(
-                    description=(
-                        "This email account has been deleted within the past "
-                        "30 days and is temporarily unavailable for new account registration"
-                    )
-                )
+            if dify_config.DEPLOYMENT_EDITION == DeploymentEdition.CLOUD:
+                freeze_type = BillingService.get_email_freeze_type(normalized_email)
+                if freeze_type:
+                    if freeze_type == "email_domain_suspended":
+                        raise EmailDomainSuspendedRegistrationError()
+                    raise AccountRegisterError(description=AccountInFreezeError.description or "")
             raise AccountRegisterError(description=("Invalid email or password"))
         account_name = user_info.name or "Dify"
         interface_language = _preferred_interface_language(language)
@@ -319,6 +336,7 @@ def _generate_account(
             provider=provider,
             language=interface_language,
             timezone=timezone,
+            ip_address=ip_address,
             session=db.session(),
         )
 

api/controllers/console/auth/oauth_server.py

@@ -42,6 +42,7 @@ class OAuthProviderAppResponse(BaseModel):
     app_icon: str
     app_label: dict[str, Any]
     scope: str
+    auto_authorize: bool
 
 
 class OAuthProviderAuthorizeResponse(BaseModel):
@@ -167,6 +168,7 @@ class OAuthServerAppApi(Resource):
                 "app_icon": oauth_provider_app.app_icon,
                 "app_label": oauth_provider_app.app_label,
                 "scope": oauth_provider_app.scope,
+                "auto_authorize": oauth_provider_app.auto_authorize,
             }
         )
 

api/controllers/console/billing/billing.py

@@ -7,20 +7,23 @@ from werkzeug.exceptions import BadRequest
 
 from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
 from controllers.console import console_ns
-from controllers.console.wraps import (
-    account_initialization_required,
-    model_validate,
-    only_edition_cloud,
-    setup_required,
-    with_current_tenant_id,
-    with_current_user,
+from controllers.console.billing.error import (
+    BillingOperationFailedErrorResponse,
+    BillingUnavailableErrorResponse,
+    BillingUnprocessableEntityErrorResponse,
+    to_billing_request_error,
 )
-from enums.cloud_plan import CloudPlan
-from extensions.ext_database import db
+from controllers.console.flask_admission import console_account_admission
+from controllers.console.wraps import model_validate
+from enums import CloudPlan, DeploymentEdition
+from extensions.ext_application_services import application_services
 from fields.base import ResponseModel
-from libs.login import login_required
-from models import Account
-from services.billing_service import BillingService
+from libs.helper import dump_response
+from machinery.context import RequestContext
+from models.account import TenantAccountRole
+from services.errors.billing import BillingError
+
+_BILLING_PORTAL_ALLOWED_ROLES = frozenset({TenantAccountRole.OWNER, TenantAccountRole.ADMIN})
 
 
 class SubscriptionQuery(BaseModel):
@@ -40,38 +43,83 @@ class BillingInvoiceResponse(ResponseModel):
     url: str
 
 
+class BillingSubscriptionResponse(ResponseModel):
+    url: str
+
+
 register_schema_models(console_ns, SubscriptionQuery, PartnerTenantsPayload)
-register_response_schema_models(console_ns, BillingResponse, BillingInvoiceResponse)
+register_response_schema_models(
+    console_ns,
+    BillingOperationFailedErrorResponse,
+    BillingUnprocessableEntityErrorResponse,
+    BillingUnavailableErrorResponse,
+    BillingResponse,
+    BillingInvoiceResponse,
+    BillingSubscriptionResponse,
+)
 
 
 @console_ns.route("/billing/subscription")
 class Subscription(Resource):
     @console_ns.doc(params=query_params_from_model(SubscriptionQuery))
-    @console_ns.response(200, "Success", console_ns.models[BillingResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @only_edition_cloud
-    @with_current_user
-    @with_current_tenant_id
+    @console_ns.response(200, "Success", console_ns.models[BillingSubscriptionResponse.__name__])
+    @console_ns.response(403, "Forbidden")
+    @console_ns.response(
+        422,
+        "Invalid subscription query",
+        console_ns.models[BillingUnprocessableEntityErrorResponse.__name__],
+    )
+    @console_ns.response(
+        502,
+        "Billing operation failed",
+        console_ns.models[BillingOperationFailedErrorResponse.__name__],
+    )
+    @console_ns.response(
+        503,
+        "Billing unavailable",
+        console_ns.models[BillingUnavailableErrorResponse.__name__],
+    )
+    @console_account_admission(
+        editions=frozenset({DeploymentEdition.CLOUD}),
+        allowed_roles=_BILLING_PORTAL_ALLOWED_ROLES,
+    )
     @model_validate(SubscriptionQuery)
-    def get(self, req_data: SubscriptionQuery, current_tenant_id: str, current_user: Account):
-        BillingService.is_tenant_owner_or_admin(current_user, session=db.session())
-        return BillingService.get_subscription(req_data.plan, req_data.interval, current_user.email, current_tenant_id)
+    def get(self, req_data: SubscriptionQuery, request_context: RequestContext):
+        try:
+            data = application_services().billing_portal.get_subscription(
+                request_context,
+                plan=req_data.plan,
+                interval=req_data.interval,
+            )
+        except BillingError as error:
+            raise to_billing_request_error(error) from error
+        return dump_response(BillingSubscriptionResponse, data)
 
 
 @console_ns.route("/billing/invoices")
 class Invoices(Resource):
     @console_ns.response(200, "Success", console_ns.models[BillingInvoiceResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @only_edition_cloud
-    @with_current_user
-    @with_current_tenant_id
-    def get(self, current_tenant_id: str, current_user: Account):
-        BillingService.is_tenant_owner_or_admin(current_user, session=db.session())
-        return BillingService.get_invoices(current_user.email, current_tenant_id)
+    @console_ns.response(403, "Forbidden")
+    @console_ns.response(
+        502,
+        "Billing operation failed",
+        console_ns.models[BillingOperationFailedErrorResponse.__name__],
+    )
+    @console_ns.response(
+        503,
+        "Billing unavailable",
+        console_ns.models[BillingUnavailableErrorResponse.__name__],
+    )
+    @console_account_admission(
+        editions=frozenset({DeploymentEdition.CLOUD}),
+        allowed_roles=_BILLING_PORTAL_ALLOWED_ROLES,
+    )
+    def get(self, request_context: RequestContext):
+        try:
+            data = application_services().billing_portal.get_invoices(request_context)
+        except BillingError as error:
+            raise to_billing_request_error(error) from error
+        return dump_response(BillingInvoiceResponse, data)
 
 
 @console_ns.route("/billing/partners/<string:partner_key>/tenants")
@@ -82,20 +130,20 @@ class PartnerTenants(Resource):
     @console_ns.expect(console_ns.models[PartnerTenantsPayload.__name__])
     @console_ns.response(200, "Tenants synced to partner successfully", console_ns.models[BillingResponse.__name__])
     @console_ns.response(400, "Invalid partner information")
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @only_edition_cloud
-    @with_current_user
+    @console_account_admission(editions=frozenset({DeploymentEdition.CLOUD}))
     @model_validate(PartnerTenantsPayload)
-    def put(self, req_data: PartnerTenantsPayload, current_user: Account, partner_key: str):
+    def put(self, req_data: PartnerTenantsPayload, request_context: RequestContext, partner_key: str):
         try:
             click_id = req_data.click_id
             decoded_partner_key = base64.b64decode(partner_key).decode("utf-8")
-        except Exception:
-            raise BadRequest("Invalid partner_key")
+        except Exception as e:
+            raise BadRequest("Invalid partner_key") from e
 
-        if not click_id or not decoded_partner_key or not current_user.id:
+        if not click_id or not decoded_partner_key:
             raise BadRequest("Invalid partner information")
 
-        return BillingService.sync_partner_tenants_bindings(current_user.id, decoded_partner_key, click_id)
+        return application_services().partner_tenant_bindings.sync(
+            account_id=request_context.account_id,
+            partner_key=decoded_partner_key,
+            click_id=click_id,
+        )

api/controllers/console/billing/error.py

@@ -0,0 +1,47 @@
+from typing import Literal
+
+from fields.base import ResponseModel
+from libs.exception import BaseHTTPException
+from services.errors.billing import (
+    BillingError,
+    BillingUpstreamInvalidResponseError,
+    BillingUpstreamUnavailableError,
+)
+
+
+class BillingUnprocessableEntityErrorResponse(ResponseModel):
+    code: Literal["unprocessable_entity"]
+    message: str
+    status: Literal[422]
+
+
+class BillingOperationFailedErrorResponse(ResponseModel):
+    code: Literal["billing_operation_failed"]
+    message: str
+    status: Literal[502]
+
+
+class BillingUnavailableErrorResponse(ResponseModel):
+    code: Literal["billing_unavailable"]
+    message: str
+    status: Literal[503]
+
+
+class BillingOperationFailedError(BaseHTTPException):
+    error_code = "billing_operation_failed"
+    description = "We couldn't complete this request. Please try again. If the problem persists, contact support."
+    code = 502
+
+
+class BillingUnavailableError(BaseHTTPException):
+    error_code = "billing_unavailable"
+    description = "This operation is temporarily unavailable. Please try again later."
+    code = 503
+
+
+def to_billing_request_error(error: BillingError) -> BaseHTTPException:
+    if isinstance(error, BillingUpstreamInvalidResponseError):
+        return BillingOperationFailedError()
+    if isinstance(error, BillingUpstreamUnavailableError):
+        return BillingUnavailableError()
+    raise TypeError(f"Unsupported billing error: {type(error).__name__}")

api/controllers/console/datasets/data_source.py

@@ -36,6 +36,7 @@ from ..wraps import (
     RBACPermission,
     RBACResourceScope,
     account_initialization_required,
+    is_admin_or_owner_required,
     model_validate,
     rbac_permission_required,
     setup_required,
@@ -138,6 +139,8 @@ register_response_schema_models(
 class DataSourceApi(Resource):
     @setup_required
     @login_required
+    @is_admin_or_owner_required
+    @rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.CREDENTIAL_MANAGE, resource_required=False)
     @account_initialization_required
     @console_ns.response(200, "Success", console_ns.models[DataSourceIntegrateListResponse.__name__])
     @with_current_tenant_id
@@ -187,6 +190,8 @@ class DataSourceApi(Resource):
 
     @setup_required
     @login_required
+    @is_admin_or_owner_required
+    @rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.CREDENTIAL_MANAGE, resource_required=False)
     @account_initialization_required
     @console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
     @with_current_tenant_id

api/controllers/console/datasets/datasets.py

@@ -53,6 +53,7 @@ from models.enums import ApiTokenType, SegmentStatus
 from models.provider_ids import ModelProviderID
 from services.api_token_service import ApiTokenCache
 from services.app_service import AppService
+from services.dataset_ref_service import DatasetRefService
 from services.dataset_service import DatasetPermissionService, DatasetService, DocumentService
 from services.enterprise import rbac_service as enterprise_rbac_service
 from services.enterprise.rbac_service import RBACResourceWhitelistScope, ReplaceMemberBindings
@@ -67,6 +68,18 @@ def _has_dataset_list_permission(permission_keys: list[str]) -> bool:
     return any(permission_key in DATASET_LIST_PERMISSION_KEYS for permission_key in permission_keys)
 
 
+def _get_accessible_dataset(dataset_id: UUID, tenant_id: str, current_user: Account, session: Session) -> Dataset:
+    dataset = DatasetService.get_dataset_for_tenant(str(dataset_id), tenant_id, session=session)
+    if dataset is None:
+        raise NotFound("Dataset not found.")
+    if not dify_config.RBAC_ENABLED:
+        try:
+            DatasetService.check_dataset_permission(dataset, current_user, session)
+        except services.errors.account.NoPermissionError as e:
+            raise Forbidden(str(e))
+    return dataset
+
+
 def _validate_indexing_technique(value: str | None) -> str | None:
     if value is None:
         return value
@@ -655,10 +668,12 @@ class DatasetApi(Resource):
         dataset = DatasetService.get_dataset(dataset_id_str, session)
         if dataset is None:
             raise NotFound("Dataset not found.")
-        try:
-            DatasetService.check_dataset_permission(dataset, current_user, session)
-        except services.errors.account.NoPermissionError as e:
-            raise Forbidden(str(e))
+
+        if not dify_config.RBAC_ENABLED:
+            try:
+                DatasetService.check_dataset_permission(dataset, current_user, session)
+            except services.errors.account.NoPermissionError as e:
+                raise Forbidden(str(e))
         permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
             current_tenant_id,
             current_user.id,
@@ -811,12 +826,13 @@ class DatasetUseCheckApi(Resource):
     @setup_required
     @login_required
     @account_initialization_required
+    @with_current_user
+    @with_current_tenant_id
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
     @with_session(write=False)
-    def get(self, session: Session, dataset_id: UUID):
-        dataset_id_str = str(dataset_id)
-
-        dataset_is_using = DatasetService.dataset_use_check(dataset_id_str, session)
+    def get(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
+        dataset = _get_accessible_dataset(dataset_id, current_tenant_id, current_user, session)
+        dataset_is_using = DatasetService.dataset_use_check(DatasetRefService.create_dataset_ref(dataset), session)
         return UsageCheckResponse(is_using=dataset_is_using).model_dump(mode="json"), 200
 
 
@@ -1027,13 +1043,15 @@ class DatasetIndexingStatusApi(Resource):
     @setup_required
     @login_required
     @account_initialization_required
+    @with_current_user
     @with_current_tenant_id
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
     @with_session(write=False)
-    def get(self, session: Session, current_tenant_id: str, dataset_id: UUID):
-        dataset_id_str = str(dataset_id)
+    def get(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
+        dataset = _get_accessible_dataset(dataset_id, current_tenant_id, current_user, session)
+
         documents = session.scalars(
-            select(Document).where(Document.dataset_id == dataset_id_str, Document.tenant_id == current_tenant_id)
+            select(Document).where(Document.dataset_id == dataset.id, Document.tenant_id == dataset.tenant_id)
         ).all()
         documents_status = []
         for document in documents:
@@ -1041,6 +1059,8 @@ class DatasetIndexingStatusApi(Resource):
                 session.scalar(
                     select(func.count(DocumentSegment.id)).where(
                         DocumentSegment.completed_at.isnot(None),
+                        DocumentSegment.tenant_id == dataset.tenant_id,
+                        DocumentSegment.dataset_id == dataset.id,
                         DocumentSegment.document_id == str(document.id),
                         DocumentSegment.status != SegmentStatus.RE_SEGMENT,
                     )
@@ -1050,6 +1070,8 @@ class DatasetIndexingStatusApi(Resource):
             total_segments = (
                 session.scalar(
                     select(func.count(DocumentSegment.id)).where(
+                        DocumentSegment.tenant_id == dataset.tenant_id,
+                        DocumentSegment.dataset_id == dataset.id,
                         DocumentSegment.document_id == str(document.id),
                         DocumentSegment.status != SegmentStatus.RE_SEGMENT,
                     )
@@ -1086,6 +1108,8 @@ class DatasetApiKeyApi(Resource):
     @console_ns.response(200, "API keys retrieved successfully", console_ns.models[ApiKeyList.__name__])
     @setup_required
     @login_required
+    @is_admin_or_owner_required
+    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_API_KEY_MANAGE, resource_required=False)
     @account_initialization_required
     @with_current_tenant_id
     @with_session(write=False)
@@ -1177,12 +1201,16 @@ class DatasetEnableApiApi(Resource):
     @login_required
     @account_initialization_required
     @console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
+    @with_current_user
+    @with_current_tenant_id
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     @with_session
-    def post(self, session: Session, dataset_id: UUID, status: str):
-        dataset_id_str = str(dataset_id)
+    def post(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID, status: str):
+        dataset = _get_accessible_dataset(dataset_id, current_tenant_id, current_user, session)
+        if not current_user.is_dataset_editor:
+            raise Forbidden()
 
-        DatasetService.update_dataset_api_status(dataset_id_str, status == "enable", session)
+        DatasetService.update_dataset_api_status(dataset, status == "enable", current_user, session)
 
         return SimpleResultResponse(result="success").model_dump(mode="json"), 200
 
@@ -1248,14 +1276,15 @@ class DatasetErrorDocs(Resource):
     @setup_required
     @login_required
     @account_initialization_required
+    @with_current_user
+    @with_current_tenant_id
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
     @with_session(write=False)
-    def get(self, session: Session, dataset_id: UUID):
-        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, session)
-        if dataset is None:
-            raise NotFound("Dataset not found.")
-        results = DocumentService.get_error_documents_by_dataset_id(dataset_id_str, session)
+    def get(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
+        dataset = _get_accessible_dataset(dataset_id, current_tenant_id, current_user, session)
+        results = DocumentService.get_error_documents_by_dataset_ref(
+            DatasetRefService.create_dataset_ref(dataset), session
+        )
 
         return dump_response(ErrorDocsResponse, {"data": results, "total": len(results)}), 200
 
@@ -1307,12 +1336,13 @@ class DatasetAutoDisableLogApi(Resource):
     @setup_required
     @login_required
     @account_initialization_required
+    @with_current_user
+    @with_current_tenant_id
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
     @with_session(write=False)
-    def get(self, session: Session, dataset_id: UUID):
-        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, session)
-        if dataset is None:
-            raise NotFound("Dataset not found.")
-        auto_disable_logs = DatasetService.get_dataset_auto_disable_logs(dataset_id_str, session)
+    def get(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
+        dataset = _get_accessible_dataset(dataset_id, current_tenant_id, current_user, session)
+        auto_disable_logs = DatasetService.get_dataset_auto_disable_logs(
+            DatasetRefService.create_dataset_ref(dataset), session
+        )
         return dump_response(AutoDisableLogsResponse, auto_disable_logs), 200

api/controllers/console/datasets/datasets_document.py

@@ -16,6 +16,7 @@ from sqlalchemy.orm import Session
 from werkzeug.exceptions import Forbidden, NotFound
 
 import services
+from configs import dify_config
 from controllers.common.controller_schemas import DocumentBatchDownloadZipPayload
 from controllers.common.fields import SimpleResultMessageResponse, SimpleResultResponse, UrlResponse
 from controllers.common.schema import register_response_schema_models, register_schema_models
@@ -54,14 +55,17 @@ from libs.helper import dump_response, to_timestamp
 from libs.login import login_required
 from libs.pagination import paginate_query
 from models import Account, Document, DocumentSegment, UploadFile
-from models.dataset import DocumentPipelineExecutionLog
+from models.dataset import DatasetPermissionEnum, DocumentPipelineExecutionLog
 from models.enums import IndexingStatus, ProcessRuleMode, SegmentStatus
 from services.dataset_ref_service import DatasetRefService
 from services.dataset_service import DatasetService, DocumentService
+from services.enterprise import rbac_service as enterprise_rbac_service
+from services.enterprise.rbac_service import RBACResourceWhitelistScope, ReplaceMemberBindings
 from services.entities.knowledge_entities.knowledge_entities import KnowledgeConfig, ProcessRule, RetrievalModel
 from services.file_service import FileService
 from services.vector_space_admission_service import get_vector_space_admission_error_fields
 from tasks.generate_summary_index_task import generate_summary_index_task
+from tasks.initialize_created_app_rbac_access_task import initialize_created_app_rbac_access_task
 
 from ..app.error import (
     ProviderModelCurrentlyNotSupportError,
@@ -78,6 +82,7 @@ from ..datasets.error import (
 )
 from ..wraps import (
     account_initialization_required,
+    check_knowledge_rate_limit,
     cloud_edition_billing_rate_limit_check,
     cloud_edition_billing_resource_check,
     setup_required,
@@ -294,23 +299,23 @@ class DocumentResource(Resource):
     def get_document(
         self, session: Session, dataset_id: str, document_id: str, current_user: Account, current_tenant_id: str
     ) -> Document:
-        dataset = DatasetService.get_dataset(dataset_id, session)
+        dataset = DatasetService.get_dataset_for_tenant(dataset_id, current_tenant_id, session=session)
         if not dataset:
             raise NotFound("Dataset not found.")
 
-        try:
-            DatasetService.check_dataset_permission(dataset, current_user, session)
-        except services.errors.account.NoPermissionError as e:
-            raise Forbidden(str(e))
+        if not dify_config.RBAC_ENABLED:
+            try:
+                DatasetService.check_dataset_permission(dataset, current_user, session)
+            except services.errors.account.NoPermissionError as e:
+                raise Forbidden(str(e))
 
-        document = DocumentService.get_document(dataset_id, document_id, session=session)
+        dataset_ref = DatasetRefService.create_dataset_ref(dataset)
+        document_ref = DatasetRefService.create_document_ref_from_id(dataset_ref, document_id)
+        document = DatasetRefService.get_document_by_ref(document_ref, session=session)
 
         if not document:
             raise NotFound("Document not found.")
 
-        if document.tenant_id != current_tenant_id:
-            raise Forbidden("No permission.")
-
         return document
 
     def get_batch_documents(
@@ -578,18 +583,33 @@ class DatasetDocumentListApi(Resource):
     @setup_required
     @login_required
     @account_initialization_required
-    @cloud_edition_billing_rate_limit_check("knowledge")
     @console_ns.response(204, "Documents deleted successfully")
+    @with_current_user
+    @with_current_tenant_id
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     @with_session
-    def delete(self, session: Session, dataset_id: UUID):
+    def delete(
+        self,
+        session: Session,
+        current_tenant_id: str,
+        current_user: Account,
+        dataset_id: UUID,
+    ):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, session)
+        dataset = DatasetService.get_dataset_for_tenant(dataset_id_str, current_tenant_id, session=session)
         if dataset is None:
             raise NotFound("Dataset not found.")
-        # check user's model setting
-        DatasetService.check_dataset_model_setting(dataset)
 
+        if not current_user.is_dataset_editor:
+            raise Forbidden()
+
+        if not dify_config.RBAC_ENABLED:
+            try:
+                DatasetService.check_dataset_permission(dataset, current_user, session)
+            except services.errors.account.NoPermissionError as e:
+                raise Forbidden(str(e))
+
+        check_knowledge_rate_limit()
         try:
             document_ids = request.args.getlist("document_id")
             dataset_ref = DatasetRefService.create_dataset_ref(dataset)
@@ -662,6 +682,21 @@ class DatasetInitApi(Resource):
         except ModelCurrentlyNotSupportError:
             raise ProviderModelCurrentlyNotSupportError()
 
+        if dify_config.RBAC_ENABLED:
+            dataset.permission = DatasetPermissionEnum.ALL_TEAM
+        else:
+            dataset.permission = DatasetPermissionEnum.ONLY_ME
+        session.flush()
+
+        if dify_config.RBAC_ENABLED:
+            enterprise_rbac_service.RBACService.DatasetAccess.replace_whitelist(
+                current_tenant_id,
+                current_user.id,
+                dataset.id,
+                ReplaceMemberBindings(scope=RBACResourceWhitelistScope.ALL),
+            )
+            initialize_created_app_rbac_access_task.delay(current_tenant_id, current_user.id, dataset_id=dataset.id)
+
         return dump_response(
             DatasetAndDocumentResponse,
             {"dataset": dataset, "documents": document_responses(documents, session=session), "batch": batch},
@@ -1367,29 +1402,32 @@ class DocumentPauseApi(DocumentResource):
     @setup_required
     @login_required
     @account_initialization_required
-    @cloud_edition_billing_rate_limit_check("knowledge")
     @console_ns.response(204, "Document paused successfully")
+    @with_current_user
+    @with_current_tenant_id
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     @with_session
-    def patch(self, session: Session, dataset_id: UUID, document_id: UUID):
+    def patch(
+        self,
+        session: Session,
+        current_tenant_id: str,
+        current_user: Account,
+        dataset_id: UUID,
+        document_id: UUID,
+    ):
         """pause document."""
         dataset_id_str = str(dataset_id)
         document_id_str = str(document_id)
 
-        dataset = DatasetService.get_dataset(dataset_id_str, session)
-        if not dataset:
-            raise NotFound("Dataset not found.")
-
-        document = DocumentService.get_document(dataset.id, document_id_str, session=session)
-
-        # 404 if document not found
-        if document is None:
-            raise NotFound("Document Not Exists.")
+        document = self.get_document(session, dataset_id_str, document_id_str, current_user, current_tenant_id)
+        if not current_user.is_dataset_editor:
+            raise Forbidden()
 
         # 403 if document is archived
         if DocumentService.check_archived(document):
             raise ArchivedDocumentImmutableError()
 
+        check_knowledge_rate_limit()
         try:
             # pause document
             DocumentService.pause_document(document, session)
@@ -1404,26 +1442,31 @@ class DocumentRecoverApi(DocumentResource):
     @setup_required
     @login_required
     @account_initialization_required
-    @cloud_edition_billing_rate_limit_check("knowledge")
     @console_ns.response(204, "Document resumed successfully")
+    @with_current_user
+    @with_current_tenant_id
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     @with_session
-    def patch(self, session: Session, dataset_id: UUID, document_id: UUID):
+    def patch(
+        self,
+        session: Session,
+        current_tenant_id: str,
+        current_user: Account,
+        dataset_id: UUID,
+        document_id: UUID,
+    ):
         """recover document."""
         dataset_id_str = str(dataset_id)
         document_id_str = str(document_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, session)
-        if not dataset:
-            raise NotFound("Dataset not found.")
-        document = DocumentService.get_document(dataset.id, document_id_str, session=session)
 
-        # 404 if document not found
-        if document is None:
-            raise NotFound("Document Not Exists.")
+        document = self.get_document(session, dataset_id_str, document_id_str, current_user, current_tenant_id)
+        if not current_user.is_dataset_editor:
+            raise Forbidden()
 
         # 403 if document is archived
         if DocumentService.check_archived(document):
             raise ArchivedDocumentImmutableError()
+        check_knowledge_rate_limit()
         try:
             # pause document
             DocumentService.recover_document(document, session)
@@ -1438,22 +1481,42 @@ class DocumentRetryApi(DocumentResource):
     @setup_required
     @login_required
     @account_initialization_required
-    @cloud_edition_billing_rate_limit_check("knowledge")
     @console_ns.expect(console_ns.models[DocumentRetryPayload.__name__])
     @console_ns.response(204, "Documents retry started successfully")
+    @with_current_user
+    @with_current_tenant_id
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     @with_session
-    def post(self, session: Session, dataset_id: UUID):
+    @model_validate(DocumentRetryPayload)
+    def post(
+        self,
+        req_data: DocumentRetryPayload,
+        session: Session,
+        current_tenant_id: str,
+        current_user: Account,
+        dataset_id: UUID,
+    ):
         """retry document."""
-        payload = DocumentRetryPayload.model_validate(console_ns.payload or {})
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, session)
-        retry_documents = []
+        dataset = DatasetService.get_dataset_for_tenant(dataset_id_str, current_tenant_id, session=session)
         if not dataset:
             raise NotFound("Dataset not found.")
-        documents = DocumentService.get_documents_by_ids(dataset.id, payload.document_ids, session)
+
+        if not current_user.is_dataset_editor:
+            raise Forbidden()
+
+        if not dify_config.RBAC_ENABLED:
+            try:
+                DatasetService.check_dataset_permission(dataset, current_user, session)
+            except services.errors.account.NoPermissionError as e:
+                raise Forbidden(str(e))
+
+        documents = DocumentService.get_documents_by_ids(
+            DatasetRefService.create_dataset_ref(dataset), req_data.document_ids, session
+        )
         documents_by_id = {document.id: document for document in documents}
-        for document_id in payload.document_ids:
+        retry_documents = []
+        for document_id in req_data.document_ids:
             try:
                 document = documents_by_id.get(document_id)
 
@@ -1473,6 +1536,7 @@ class DocumentRetryApi(DocumentResource):
                 logger.exception("Failed to retry document, document id: %s", document_id)
                 continue
         # retry document
+        check_knowledge_rate_limit()
         DocumentService.retry_document(dataset_id_str, retry_documents, session)
 
         return "", 204
@@ -1488,7 +1552,15 @@ class DocumentRenameApi(DocumentResource):
     @with_current_user
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     @with_session
-    def post(self, session: Session, current_user: Account, dataset_id: UUID, document_id: UUID):
+    @model_validate(DocumentRenamePayload)
+    def post(
+        self,
+        req_data: DocumentRenamePayload,
+        session: Session,
+        current_user: Account,
+        dataset_id: UUID,
+        document_id: UUID,
+    ):
         # The role of the current user in the ta table must be admin, owner, editor, or dataset_operator
         if not current_user.is_dataset_editor:
             raise Forbidden()
@@ -1496,10 +1568,9 @@ class DocumentRenameApi(DocumentResource):
         if not dataset:
             raise NotFound("Dataset not found.")
         DatasetService.check_dataset_operator_permission(current_user, dataset, session=session)
-        payload = DocumentRenamePayload.model_validate(console_ns.payload or {})
 
         try:
-            document = DocumentService.rename_document(str(dataset_id), str(document_id), payload.name, session)
+            document = DocumentService.rename_document(str(dataset_id), str(document_id), req_data.name, session)
         except services.errors.document.DocumentIndexingError:
             raise DocumentIndexingError("Cannot delete document during indexing.")
 
@@ -1512,28 +1583,34 @@ class WebsiteDocumentSyncApi(DocumentResource):
     @login_required
     @account_initialization_required
     @console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
+    @with_current_user
     @with_current_tenant_id
-    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
+    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     @with_session
-    def get(self, session: Session, current_tenant_id: str, dataset_id: UUID, document_id: UUID):
+    def get(
+        self,
+        session: Session,
+        current_tenant_id: str,
+        current_user: Account,
+        dataset_id: UUID,
+        document_id: UUID,
+    ):
         """sync website document."""
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, session)
+        dataset = DatasetService.get_dataset_for_tenant(dataset_id_str, current_tenant_id, session=session)
         if not dataset:
             raise NotFound("Dataset not found.")
+        if not current_user.is_dataset_editor:
+            raise Forbidden()
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset.id, document_id_str, session=session)
-        if not document:
-            raise NotFound("Document not found.")
-        if document.tenant_id != current_tenant_id:
-            raise Forbidden("No permission.")
+        document = self.get_document(session, dataset.id, document_id_str, current_user, current_tenant_id)
         if document.data_source_type != "website_crawl":
             raise ValueError("Document is not a website document.")
         # 403 if document is archived
         if DocumentService.check_archived(document):
             raise ArchivedDocumentImmutableError()
         # sync document
-        DocumentService.sync_website_document(dataset_id_str, document, session)
+        DocumentService.sync_website_document(dataset, document, session)
 
         return SimpleResultResponse(result="success").model_dump(mode="json"), 200
 
@@ -1548,21 +1625,25 @@ class DocumentPipelineExecutionLogApi(DocumentResource):
     @setup_required
     @login_required
     @account_initialization_required
+    @with_current_user
+    @with_current_tenant_id
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
     @with_session(write=False)
-    def get(self, session: Session, dataset_id: UUID, document_id: UUID):
+    def get(
+        self,
+        session: Session,
+        current_tenant_id: str,
+        current_user: Account,
+        dataset_id: UUID,
+        document_id: UUID,
+    ):
         dataset_id_str = str(dataset_id)
         document_id_str = str(document_id)
 
-        dataset = DatasetService.get_dataset(dataset_id_str, session)
-        if not dataset:
-            raise NotFound("Dataset not found.")
-        document = DocumentService.get_document(dataset.id, document_id_str, session=session)
-        if not document:
-            raise NotFound("Document not found.")
+        document = self.get_document(session, dataset_id_str, document_id_str, current_user, current_tenant_id)
         log = session.scalar(
             select(DocumentPipelineExecutionLog)
-            .where(DocumentPipelineExecutionLog.document_id == document_id_str)
+            .where(DocumentPipelineExecutionLog.document_id == document.id)
             .order_by(DocumentPipelineExecutionLog.created_at.desc())
             .limit(1)
         )
@@ -1600,7 +1681,8 @@ class DocumentGenerateSummaryApi(Resource):
     @with_current_user
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     @with_session
-    def post(self, session: Session, current_user: Account, dataset_id: UUID):
+    @model_validate(GenerateSummaryPayload)
+    def post(self, req_data: GenerateSummaryPayload, session: Session, current_user: Account, dataset_id: UUID):
         """
         Generate summary index for specified documents.
 
@@ -1624,9 +1706,7 @@ class DocumentGenerateSummaryApi(Resource):
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
-        # Validate request payload
-        payload = GenerateSummaryPayload.model_validate(console_ns.payload or {})
-        document_list = payload.document_list
+        document_list = req_data.document_list
 
         if not document_list:
             from werkzeug.exceptions import BadRequest
@@ -1645,7 +1725,9 @@ class DocumentGenerateSummaryApi(Resource):
             raise ValueError("Summary index is not enabled for this dataset. Please enable it in the dataset settings.")
 
         # Verify all documents exist and belong to the dataset
-        documents = DocumentService.get_documents_by_ids(dataset_id_str, document_list, session)
+        documents = DocumentService.get_documents_by_ids(
+            DatasetRefService.create_dataset_ref(dataset), document_list, session
+        )
 
         if len(documents) != len(document_list):
             found_ids = {doc.id for doc in documents}

api/controllers/console/datasets/datasets_segments.py

@@ -631,12 +631,16 @@ class DatasetDocumentSegmentBatchImportApi(Resource):
     ):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, session)
+        dataset = DatasetService.get_dataset_for_tenant(dataset_id_str, current_tenant_id, session=session)
         if not dataset:
             raise NotFound("Dataset not found.")
+
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
+        document_ref = DatasetRefService.create_document_ref_from_id(
+            DatasetRefService.create_dataset_ref(dataset), document_id_str
+        )
+        document = DatasetRefService.get_document_by_ref(document_ref, session=session)
         if not document:
             raise NotFound("Document not found.")
 

api/controllers/console/datasets/external.py

@@ -42,7 +42,6 @@ from services.enterprise import rbac_service as enterprise_rbac_service
 from services.entities.external_knowledge_entities.external_knowledge_entities import ExternalDatasetCreatePayload
 from services.external_knowledge_service import ExternalDatasetService
 from services.hit_testing_service import HitTestingService
-from services.knowledge_service import BedrockRetrievalSetting, ExternalDatasetTestService
 
 
 class ExternalKnowledgeApiPayload(BaseModel):
@@ -56,12 +55,6 @@ class ExternalHitTestingPayload(BaseModel):
     metadata_filtering_conditions: dict[str, Any] | None = None
 
 
-class BedrockRetrievalPayload(BaseModel):
-    retrieval_setting: BedrockRetrievalSetting
-    query: str
-    knowledge_id: str
-
-
 class ExternalApiTemplateListQuery(BaseModel):
     page: int = Field(default=1, description="Page number")
     limit: int = Field(default=20, description="Number of items per page")
@@ -137,23 +130,11 @@ class ExternalHitTestingResponse(ResponseModel):
     records: list[ExternalHitTestingRecordResponse]
 
 
-class BedrockRetrievalRecordResponse(ResponseModel):
-    metadata: dict[str, Any] | None = None
-    score: float
-    title: str | None = None
-    content: str | None = None
-
-
-class BedrockRetrievalResponse(ResponseModel):
-    records: list[BedrockRetrievalRecordResponse]
-
-
 register_schema_models(
     console_ns,
     ExternalKnowledgeApiPayload,
     ExternalDatasetCreatePayload,
     ExternalHitTestingPayload,
-    BedrockRetrievalPayload,
     ExternalApiTemplateListQuery,
 )
 register_response_schema_models(
@@ -166,8 +147,6 @@ register_response_schema_models(
     ExternalHitTestingQueryResponse,
     ExternalHitTestingRecordResponse,
     ExternalHitTestingResponse,
-    BedrockRetrievalRecordResponse,
-    BedrockRetrievalResponse,
 )
 
 
@@ -451,20 +430,3 @@ class ExternalKnowledgeHitTestingApi(Resource):
             return dump_response(ExternalHitTestingResponse, response)
         except Exception as e:
             raise InternalServerError(str(e))
-
-
-@console_ns.route("/test/retrieval")
-class BedrockRetrievalApi(Resource):
-    # this api is only for internal testing
-    @console_ns.doc("bedrock_retrieval_test")
-    @console_ns.doc(description="Bedrock retrieval test (internal use only)")
-    @console_ns.expect(console_ns.models[BedrockRetrievalPayload.__name__])
-    @console_ns.response(200, "Bedrock retrieval test completed", console_ns.models[BedrockRetrievalResponse.__name__])
-    @model_validate(BedrockRetrievalPayload)
-    def post(self, req_data: BedrockRetrievalPayload):
-
-        # Call the knowledge retrieval service
-        result = ExternalDatasetTestService.knowledge_retrieval(
-            req_data.retrieval_setting, req_data.query, req_data.knowledge_id
-        )
-        return dump_response(BedrockRetrievalResponse, result), 200

api/controllers/console/datasets/metadata.py

@@ -3,8 +3,10 @@ from uuid import UUID
 
 from flask_restx import Resource
 from sqlalchemy.orm import Session
-from werkzeug.exceptions import NotFound
+from werkzeug.exceptions import Forbidden, NotFound
 
+import services
+from configs import dify_config
 from controllers.common.controller_schemas import MetadataUpdatePayload
 from controllers.common.schema import register_response_schema_models, register_schema_models
 from controllers.common.session import with_session
@@ -35,6 +37,7 @@ from services.entities.knowledge_entities.knowledge_entities import (
     MetadataDetail,
     MetadataOperationData,
 )
+from services.errors.metadata import MetadataResourceNotFoundError
 from services.metadata_service import MetadataService
 
 register_schema_models(
@@ -87,13 +90,20 @@ class DatasetMetadataCreateApi(Resource):
     @console_ns.response(
         200, "Metadata retrieved successfully", console_ns.models[DatasetMetadataListResponse.__name__]
     )
+    @with_current_user
+    @with_current_tenant_id
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
     @with_session(write=False)
-    def get(self, session: Session, dataset_id: UUID):
+    def get(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, session)
+        dataset = DatasetService.get_dataset_for_tenant(dataset_id_str, current_tenant_id, session=session)
         if dataset is None:
             raise NotFound("Dataset not found.")
+        if not dify_config.RBAC_ENABLED:
+            try:
+                DatasetService.check_dataset_permission(dataset, current_user, session)
+            except services.errors.account.NoPermissionError as e:
+                raise Forbidden(str(e))
         metadata = MetadataService.get_dataset_metadatas(dataset, session)
         return dump_response(DatasetMetadataListResponse, metadata), 200
 
@@ -124,14 +134,12 @@ class DatasetMetadataApi(Resource):
 
         dataset_id_str = str(dataset_id)
         metadata_id_str = str(metadata_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, session)
+        dataset = DatasetService.get_dataset_for_tenant(dataset_id_str, current_tenant_id, session=session)
         if dataset is None:
             raise NotFound("Dataset not found.")
         DatasetService.check_dataset_permission(dataset, current_user, session)
 
-        metadata = MetadataService.update_metadata_name(
-            dataset_id_str, metadata_id_str, name, current_user, current_tenant_id, session=session
-        )
+        metadata = MetadataService.update_metadata_name(dataset, metadata_id_str, name, current_user, session=session)
         return dump_response(DatasetMetadataResponse, metadata), 200
 
     @setup_required
@@ -140,17 +148,25 @@ class DatasetMetadataApi(Resource):
     @enterprise_license_required
     @console_ns.response(204, "Metadata deleted successfully")
     @with_current_user
+    @with_current_tenant_id
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     @with_session
-    def delete(self, session: Session, current_user: Account, dataset_id: UUID, metadata_id: UUID):
+    def delete(
+        self,
+        session: Session,
+        current_tenant_id: str,
+        current_user: Account,
+        dataset_id: UUID,
+        metadata_id: UUID,
+    ):
         dataset_id_str = str(dataset_id)
         metadata_id_str = str(metadata_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, session)
+        dataset = DatasetService.get_dataset_for_tenant(dataset_id_str, current_tenant_id, session=session)
         if dataset is None:
             raise NotFound("Dataset not found.")
         DatasetService.check_dataset_permission(dataset, current_user, session)
 
-        MetadataService.delete_metadata(dataset_id_str, metadata_id_str, session)
+        MetadataService.delete_metadata(dataset, metadata_id_str, session)
         # Frontend callers only await success and invalidate metadata caches; no response body is consumed.
         return "", 204
 
@@ -208,18 +224,29 @@ class DocumentMetadataEditApi(Resource):
         204,
         "Documents metadata updated successfully",
     )
+    @console_ns.response(404, "Dataset, document, or metadata not found")
     @with_current_user
+    @with_current_tenant_id
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     @with_session
     @model_validate(MetadataOperationData)
-    def post(self, req_data: MetadataOperationData, session: Session, current_user: Account, dataset_id: UUID):
-        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, session)
+    def post(
+        self,
+        req_data: MetadataOperationData,
+        session: Session,
+        current_tenant_id: str,
+        current_user: Account,
+        dataset_id: UUID,
+    ):
+        dataset = DatasetService.get_dataset_for_tenant(str(dataset_id), current_tenant_id, session=session)
         if dataset is None:
             raise NotFound("Dataset not found.")
         DatasetService.check_dataset_permission(dataset, current_user, session)
 
-        MetadataService.update_documents_metadata(dataset, req_data, current_user, session=session)
+        try:
+            MetadataService.update_documents_metadata(dataset, req_data, current_user, session=session)
+        except MetadataResourceNotFoundError as exc:
+            raise NotFound(str(exc)) from exc
 
         # Frontend callers only await success and invalidate caches; no response body is consumed.
         return "", 204

api/controllers/console/datasets/rag_pipeline/datasource_auth.py

@@ -30,6 +30,7 @@ from graphon.model_runtime.errors.validate import CredentialsValidateFailedError
 from libs.helper import dump_response
 from libs.login import login_required
 from models import Account
+from models.enums import PermissionEnum
 from models.provider_ids import DatasourceProviderID
 from services.datasource_provider_service import DatasourceProviderService
 from services.plugin.oauth_service import OAuthProxyService
@@ -74,6 +75,13 @@ class DatasourceUpdateNamePayload(BaseModel):
 
 class DatasourceOAuthAuthorizationQuery(BaseModel):
     credential_id: str | None = Field(default=None, description="Credential ID to reauthorize")
+    visibility: str | None = Field(
+        default=None,
+        description=(
+            "Visibility for the credential to be created. Accepts 'only_me' or 'all_team_members'; "
+            "any other value falls back to 'only_me'. Ignored on reauthorization (credential_id set)."
+        ),
+    )
 
 
 class DatasourceOAuthCallbackQuery(BaseModel):
@@ -175,12 +183,27 @@ class DatasourcePluginOAuthAuthorizationUrl(Resource):
         if not oauth_config:
             raise ValueError(f"No OAuth Client Config for {provider_id}")
 
+        # Visibility is chosen by the user in the frontend before the redirect,
+        # then read back in the callback below when the credential is created.
+        # Only ONLY_ME / ALL_TEAM are accepted; anything else falls back to
+        # ONLY_ME (OAuth tokens are personal by nature).
+        # For reauthorization (credential_id set), visibility is ignored — we
+        # keep whatever the credential was created with.
+        raw_visibility = request.args.get("visibility")
+        try:
+            requested_visibility = PermissionEnum(raw_visibility) if raw_visibility else PermissionEnum.ONLY_ME
+        except ValueError:
+            requested_visibility = PermissionEnum.ONLY_ME
+        if requested_visibility not in (PermissionEnum.ONLY_ME, PermissionEnum.ALL_TEAM):
+            requested_visibility = PermissionEnum.ONLY_ME
+
         context_id = OAuthProxyService.create_proxy_context(
             user_id=current_user.id,
             tenant_id=tenant_id,
             plugin_id=plugin_id,
             provider=provider_name,
             credential_id=credential_id,
+            extra_data={"visibility": requested_visibility.value},
         )
         oauth_handler = OAuthHandler()
         redirect_uri = f"{dify_config.CONSOLE_API_URL}/console/api/oauth/plugin/{provider_id}/datasource/callback"
@@ -253,6 +276,17 @@ class DatasourceOAuthCallback(Resource):
                 credential_id=credential_id,
             )
         else:
+            # Visibility was chosen by the user before the redirect and stashed
+            # in the proxy context. Fall back to ONLY_ME for older cookies (or
+            # anything that somehow wrote an unexpected value) — OAuth tokens
+            # are personal by default.
+            stored_visibility = context.get("visibility")
+            try:
+                visibility = PermissionEnum(stored_visibility) if stored_visibility else PermissionEnum.ONLY_ME
+            except ValueError:
+                visibility = PermissionEnum.ONLY_ME
+            if visibility not in (PermissionEnum.ONLY_ME, PermissionEnum.ALL_TEAM):
+                visibility = PermissionEnum.ONLY_ME
             datasource_provider_service.add_datasource_oauth_provider(
                 tenant_id=tenant_id,
                 provider_id=datasource_provider_id,
@@ -260,6 +294,8 @@ class DatasourceOAuthCallback(Resource):
                 name=oauth_response.metadata.get("name") or None,
                 expire_at=oauth_response.expires_at,
                 credentials=dict(oauth_response.credentials),
+                user_id=user_id,
+                visibility=visibility,
             )
         return redirect(f"{dify_config.CONSOLE_WEB_URL}/oauth-callback")
 
@@ -300,6 +336,8 @@ class DatasourceAuth(Resource):
     @setup_required
     @login_required
     @account_initialization_required
+    @edit_permission_required
+    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.CREDENTIAL_MANAGE, resource_required=False)
     @with_current_user
     @with_current_tenant_id
     def get(self, current_tenant_id: str, user: Account, provider_id: str):
@@ -381,11 +419,12 @@ class DatasourceAuthListApi(Resource):
     @setup_required
     @login_required
     @account_initialization_required
+    @with_current_user
     @with_current_tenant_id
-    def get(self, current_tenant_id: str):
+    def get(self, current_tenant_id: str, user: Account):
         datasource_provider_service = DatasourceProviderService()
         datasources = datasource_provider_service.get_all_datasource_credentials(
-            tenant_id=current_tenant_id, session=db.session()
+            tenant_id=current_tenant_id, session=db.session(), user=user
         )
         return dump_response(DatasourceProviderAuthListResponse, {"result": datasources}), 200
 
@@ -400,11 +439,12 @@ class DatasourceHardCodeAuthListApi(Resource):
     @setup_required
     @login_required
     @account_initialization_required
+    @with_current_user
     @with_current_tenant_id
-    def get(self, current_tenant_id: str):
+    def get(self, current_tenant_id: str, user: Account):
         datasource_provider_service = DatasourceProviderService()
         datasources = datasource_provider_service.get_hard_code_datasource_credentials(
-            tenant_id=current_tenant_id, session=db.session()
+            tenant_id=current_tenant_id, session=db.session(), user=user
         )
         return dump_response(DatasourceProviderAuthListResponse, {"result": datasources}), 200
 
@@ -436,6 +476,8 @@ class DatasourceAuthOauthCustomClient(Resource):
     @setup_required
     @login_required
     @account_initialization_required
+    @edit_permission_required
+    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.CREDENTIAL_MANAGE, resource_required=False)
     @console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
     @with_current_tenant_id
     def delete(self, current_tenant_id: str, provider_id: str):

api/controllers/console/datasets/rag_pipeline/rag_pipeline.py

@@ -3,10 +3,10 @@ from typing import Any
 
 from flask_restx import Resource
 from pydantic import BaseModel, Field
-from sqlalchemy import select
-from sqlalchemy.orm import Session, sessionmaker
-from werkzeug.exceptions import NotFound
+from sqlalchemy.orm import Session
+from werkzeug.exceptions import Forbidden, NotFound
 
+from configs import dify_config
 from controllers.common.fields import SimpleDataResponse
 from controllers.common.schema import (
     JsonResponseWithStatus,
@@ -16,11 +16,15 @@ from controllers.common.schema import (
 )
 from controllers.console import console_ns
 from controllers.console.app.wraps import with_session
+from controllers.console.datasets.wraps import get_rag_pipeline
 from controllers.console.wraps import (
+    RBACPermission,
+    RBACResourceScope,
     account_initialization_required,
     enterprise_license_required,
     knowledge_pipeline_publish_enabled,
     model_validate,
+    rbac_permission_required,
     setup_required,
     with_current_tenant_id,
     with_current_user,
@@ -30,8 +34,11 @@ from fields.base import ResponseModel
 from libs.helper import dump_response
 from libs.login import login_required
 from models.account import Account
-from models.dataset import PipelineCustomizedTemplate
+from models.dataset import Pipeline
+from services.dataset_service import DatasetService
 from services.entities.knowledge_entities.rag_pipeline_entities import IconInfo, PipelineTemplateInfoEntity
+from services.errors.account import NoPermissionError
+from services.errors.rag_pipeline import RagPipelineResourceNotFoundError
 from services.rag_pipeline.rag_pipeline import RagPipelineService
 
 logger: logging.Logger = logging.getLogger(__name__)
@@ -125,15 +132,24 @@ class PipelineTemplateListApi(Resource):
 class PipelineTemplateDetailApi(Resource):
     @console_ns.doc(params=query_params_from_model(PipelineTemplateDetailQuery))
     @console_ns.response(200, "Pipeline template", console_ns.models[PipelineTemplateDetailResponse.__name__])
+    @console_ns.response(404, "Pipeline template not found")
     @setup_required
     @login_required
     @account_initialization_required
     @enterprise_license_required
-    @with_session
+    @with_current_tenant_id
+    @with_session(write=False)
     @model_validate(PipelineTemplateDetailQuery)
-    def get(self, req_data: PipelineTemplateDetailQuery, session: Session, template_id: str) -> JsonResponseWithStatus:
+    def get(
+        self,
+        req_data: PipelineTemplateDetailQuery,
+        session: Session,
+        current_tenant_id: str,
+        template_id: str,
+    ) -> JsonResponseWithStatus:
         pipeline_template = RagPipelineService.get_pipeline_template_detail(
             template_id,
+            current_tenant_id,
             type=req_data.type,
             session=session,
         )
@@ -181,38 +197,57 @@ class CustomizedPipelineTemplateApi(Resource):
     @account_initialization_required
     @enterprise_license_required
     @console_ns.response(200, "Success", console_ns.models[SimpleDataResponse.__name__])
-    def post(self, template_id: str) -> JsonResponseWithStatus:
-        with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
-            template = session.scalar(
-                select(PipelineCustomizedTemplate).where(PipelineCustomizedTemplate.id == template_id).limit(1)
+    @console_ns.response(404, "Customized pipeline template not found")
+    @with_current_tenant_id
+    @with_session(write=False)
+    def post(self, session: Session, current_tenant_id: str, template_id: str) -> JsonResponseWithStatus:
+        try:
+            yaml_content = RagPipelineService.get_customized_pipeline_template_yaml(
+                template_id, current_tenant_id, session=session
             )
-            if not template:
-                raise ValueError("Customized pipeline template not found.")
+        except RagPipelineResourceNotFoundError as exc:
+            raise NotFound(str(exc)) from exc
 
-        return dump_response(SimpleDataResponse, {"data": template.yaml_content}), 200
+        return dump_response(SimpleDataResponse, {"data": yaml_content}), 200
 
 
 @console_ns.route("/rag/pipelines/<string:pipeline_id>/customized/publish")
 class PublishCustomizedPipelineTemplateApi(Resource):
     @console_ns.expect(console_ns.models[CustomizedPipelineTemplatePayload.__name__])
     @console_ns.response(204, "Pipeline template published")
+    @console_ns.response(404, "Pipeline, workflow, or dataset not found")
     @setup_required
     @login_required
     @account_initialization_required
     @enterprise_license_required
     @knowledge_pipeline_publish_enabled
     @with_current_user
-    @with_current_tenant_id
+    @get_rag_pipeline
+    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_PIPELINE_RELEASE)
     @model_validate(CustomizedPipelineTemplatePayload)
     def post(
         self,
         req_data: CustomizedPipelineTemplatePayload,
-        current_tenant_id: str,
         current_user: Account,
-        pipeline_id: str,
+        pipeline: Pipeline,
     ) -> tuple[str, int]:
-        rag_pipeline_service = RagPipelineService(db.session())
-        rag_pipeline_service.publish_customized_pipeline_template(
-            pipeline_id, req_data.model_dump(), current_user, current_tenant_id, session=db.session()
-        )
+        session = db.session()
+        dataset = pipeline.retrieve_dataset(session=session)
+        if dataset is None:
+            raise NotFound("Dataset not found")
+
+        if not dify_config.RBAC_ENABLED:
+            if not current_user.is_dataset_editor:
+                raise Forbidden()
+            try:
+                DatasetService.check_dataset_permission(dataset, current_user, session)
+            except NoPermissionError as exc:
+                raise Forbidden(str(exc)) from exc
+
+        try:
+            RagPipelineService.publish_customized_pipeline_template(
+                pipeline, dataset, req_data.model_dump(), current_user, session=session
+            )
+        except RagPipelineResourceNotFoundError as exc:
+            raise NotFound(str(exc)) from exc
         return "", 204

api/controllers/console/datasets/rag_pipeline/rag_pipeline_draft_variable.py

@@ -8,7 +8,6 @@ from flask import Response
 from flask_restx import Resource, marshal, marshal_with
 from pydantic import BaseModel, Field
 from sqlalchemy.orm import sessionmaker
-from werkzeug.exceptions import Forbidden
 
 from controllers.common.errors import InvalidArgumentError, NotFoundError
 from controllers.common.schema import query_params_from_model, register_schema_models
@@ -19,12 +18,22 @@ from controllers.console.app.error import (
 from controllers.console.app.workflow_draft_variable import (
     _WORKFLOW_DRAFT_VARIABLE_FIELDS,  # type: ignore[private-usage]
     EnvironmentVariableListResponse,
+    ensure_variable_access,
     workflow_draft_variable_list_model,
     workflow_draft_variable_list_without_value_model,
     workflow_draft_variable_model,
 )
 from controllers.console.datasets.wraps import get_rag_pipeline
-from controllers.console.wraps import account_initialization_required, model_validate, setup_required, with_current_user
+from controllers.console.wraps import (
+    RBACPermission,
+    RBACResourceScope,
+    account_initialization_required,
+    edit_permission_required,
+    model_validate,
+    rbac_permission_required,
+    setup_required,
+    with_current_user,
+)
 from core.app.file_access import DatabaseFileAccessController
 from core.workflow.llm_environment_variable import LLMEnvironmentVariable, environment_variable_value_type
 from core.workflow.variable_prefixes import CONVERSATION_VARIABLE_NODE_ID, SYSTEM_VARIABLE_NODE_ID
@@ -62,21 +71,21 @@ def _api_prerequisite[T, **P, R](
 
     It ensures the following conditions are satisfied:
 
-    - Dify has been property setup.
+    - Dify has been properly set up.
     - The request user has logged in and initialized.
-    - The requested app is a workflow or a chat flow.
-    - The request user has the edit permission for the app.
+    - The requested RAG pipeline belongs to the current tenant.
+    - The request user has legacy edit permission or Dataset edit RBAC access.
     """
 
     @setup_required
     @login_required
     @account_initialization_required
     @get_rag_pipeline
+    @edit_permission_required
+    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     @with_current_user
     @wraps(f)
     def wrapper(self: T, current_user: Account, *args: P.args, **kwargs: P.kwargs) -> R | Response:
-        if not current_user.has_edit_permission:
-            raise Forbidden()
         return f(self, current_user, *args, **kwargs)
 
     return wrapper
@@ -180,16 +189,17 @@ class RagPipelineVariableApi(Resource):
     @console_ns.response(200, "Variable retrieved successfully", workflow_draft_variable_model)
     @_api_prerequisite
     @marshal_with(workflow_draft_variable_model)
-    def get(self, _current_user: Account, pipeline: Pipeline, variable_id: UUID):
+    def get(self, current_user: Account, pipeline: Pipeline, variable_id: UUID):
         draft_var_srv = WorkflowDraftVariableService(
             session=db.session(),
         )
         variable_id_str = str(variable_id)
-        variable = draft_var_srv.get_variable(variable_id=variable_id_str)
-        if variable is None:
-            raise NotFoundError(description=f"variable not found, id={variable_id_str}")
-        if variable.app_id != pipeline.id:
-            raise NotFoundError(description=f"variable not found, id={variable_id_str}")
+        variable = ensure_variable_access(
+            variable=draft_var_srv.get_variable(variable_id=variable_id_str),
+            app_id=pipeline.id,
+            variable_id=variable_id_str,
+            current_user_id=current_user.id,
+        )
         return variable
 
     @console_ns.response(200, "Variable updated successfully", workflow_draft_variable_model)
@@ -200,7 +210,7 @@ class RagPipelineVariableApi(Resource):
     def patch(
         self,
         req_data: WorkflowDraftVariablePatchPayload,
-        _current_user: Account,
+        current_user: Account,
         pipeline: Pipeline,
         variable_id: UUID,
     ):
@@ -231,11 +241,12 @@ class RagPipelineVariableApi(Resource):
         args = req_data.model_dump(exclude_none=True)
 
         variable_id_str = str(variable_id)
-        variable = draft_var_srv.get_variable(variable_id=variable_id_str)
-        if variable is None:
-            raise NotFoundError(description=f"variable not found, id={variable_id_str}")
-        if variable.app_id != pipeline.id:
-            raise NotFoundError(description=f"variable not found, id={variable_id_str}")
+        variable = ensure_variable_access(
+            variable=draft_var_srv.get_variable(variable_id=variable_id_str),
+            app_id=pipeline.id,
+            variable_id=variable_id_str,
+            current_user_id=current_user.id,
+        )
 
         new_name = args.get(self._PATCH_NAME_FIELD, None)
         raw_value = args.get(self._PATCH_VALUE_FIELD, None)
@@ -272,16 +283,17 @@ class RagPipelineVariableApi(Resource):
 
     @console_ns.response(204, "Variable deleted successfully")
     @_api_prerequisite
-    def delete(self, _current_user: Account, pipeline: Pipeline, variable_id: UUID):
+    def delete(self, current_user: Account, pipeline: Pipeline, variable_id: UUID):
         draft_var_srv = WorkflowDraftVariableService(
             session=db.session(),
         )
         variable_id_str = str(variable_id)
-        variable = draft_var_srv.get_variable(variable_id=variable_id_str)
-        if variable is None:
-            raise NotFoundError(description=f"variable not found, id={variable_id_str}")
-        if variable.app_id != pipeline.id:
-            raise NotFoundError(description=f"variable not found, id={variable_id_str}")
+        variable = ensure_variable_access(
+            variable=draft_var_srv.get_variable(variable_id=variable_id_str),
+            app_id=pipeline.id,
+            variable_id=variable_id_str,
+            current_user_id=current_user.id,
+        )
         draft_var_srv.delete_variable(variable)
         db.session.commit()
         return Response("", 204)
@@ -292,7 +304,7 @@ class RagPipelineVariableResetApi(Resource):
     @console_ns.response(200, "Variable reset successfully", workflow_draft_variable_model)
     @console_ns.response(204, "Variable reset (no content)")
     @_api_prerequisite
-    def put(self, _current_user: Account, pipeline: Pipeline, variable_id: UUID):
+    def put(self, current_user: Account, pipeline: Pipeline, variable_id: UUID):
         draft_var_srv = WorkflowDraftVariableService(
             session=db.session(),
         )
@@ -304,11 +316,12 @@ class RagPipelineVariableResetApi(Resource):
                 f"Draft workflow not found, pipeline_id={pipeline.id}",
             )
         variable_id_str = str(variable_id)
-        variable = draft_var_srv.get_variable(variable_id=variable_id_str)
-        if variable is None:
-            raise NotFoundError(description=f"variable not found, id={variable_id_str}")
-        if variable.app_id != pipeline.id:
-            raise NotFoundError(description=f"variable not found, id={variable_id_str}")
+        variable = ensure_variable_access(
+            variable=draft_var_srv.get_variable(variable_id=variable_id_str),
+            app_id=pipeline.id,
+            variable_id=variable_id_str,
+            current_user_id=current_user.id,
+        )
 
         resetted = draft_var_srv.reset_variable(draft_workflow, variable)
         db.session.commit()

api/controllers/console/datasets/rag_pipeline/rag_pipeline_workflow.py

@@ -10,6 +10,7 @@ from sqlalchemy.orm import Session, sessionmaker
 from werkzeug.exceptions import BadRequest, Forbidden, InternalServerError, NotFound
 
 import services
+from configs import dify_config
 from controllers.common.controller_schemas import DefaultBlockConfigQuery, WorkflowListQuery, WorkflowUpdatePayload
 from controllers.common.fields import SimpleResultResponse
 from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
@@ -60,8 +61,11 @@ from models import Account
 from models.dataset import Pipeline
 from models.model import EndUser
 from models.workflow import Workflow
+from services.agent.retirement_service import WorkflowAgentRetirementService
+from services.dataset_service import DatasetService
 from services.errors.app import IsDraftWorkflowError, WorkflowHashNotEqualError, WorkflowNotFoundError
 from services.errors.llm import InvokeRateLimitError
+from services.errors.rag_pipeline import RagPipelineResourceNotFoundError
 from services.rag_pipeline.pipeline_generate_service import PipelineGenerateService
 from services.rag_pipeline.rag_pipeline import RagPipelineService
 from services.rag_pipeline.rag_pipeline_manage_service import RagPipelineManageService
@@ -767,8 +771,9 @@ class RagPipelineByIdApi(Resource):
     @account_initialization_required
     @edit_permission_required
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
+    @with_current_user
     @get_rag_pipeline
-    def delete(self, pipeline: Pipeline, workflow_id: str):
+    def delete(self, current_user: Account, pipeline: Pipeline, workflow_id: str):
         """
         Delete a published workflow version that is not currently active on the pipeline.
         """
@@ -780,7 +785,7 @@ class RagPipelineByIdApi(Resource):
 
         with sessionmaker(db.engine).begin() as session:
             try:
-                workflow_service.delete_workflow(
+                retirement_candidates = workflow_service.delete_workflow(
                     session=session,
                     workflow_ref=workflow_ref,
                 )
@@ -791,6 +796,11 @@ class RagPipelineByIdApi(Resource):
             except ValueError as e:
                 raise NotFound(str(e))
 
+        WorkflowAgentRetirementService.retire_unowned(
+            tenant_id=pipeline.tenant_id,
+            agent_ids=retirement_candidates,
+            account_id=current_user.id,
+        )
         return None, 204
 
 
@@ -1016,19 +1026,31 @@ class RagPipelineWorkflowLastRunApi(Resource):
 @console_ns.route("/rag/pipelines/transform/datasets/<uuid:dataset_id>")
 class RagPipelineTransformApi(Resource):
     @console_ns.response(200, "Success", console_ns.models[RagPipelineOpaqueResponse.__name__])
+    @console_ns.response(404, "Dataset or pipeline not found")
     @setup_required
     @login_required
     @account_initialization_required
     @with_current_user
+    @with_current_tenant_id
+    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     @with_session
-    def post(self, session: Session, current_user: Account, dataset_id: UUID):
-        if not (current_user.has_edit_permission or current_user.is_dataset_operator):
-            raise Forbidden()
+    def post(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
+        dataset = DatasetService.get_dataset_for_tenant(str(dataset_id), current_tenant_id, session=session)
+        if dataset is None:
+            raise NotFound("Dataset not found.")
 
-        dataset_id_str = str(dataset_id)
-        rag_pipeline_transform_service = RagPipelineTransformService()
-        result = rag_pipeline_transform_service.transform_dataset(dataset_id_str, session)
-        return result
+        if not dify_config.RBAC_ENABLED:
+            if not (current_user.has_edit_permission or current_user.is_dataset_operator):
+                raise Forbidden()
+            try:
+                DatasetService.check_dataset_permission(dataset, current_user, session)
+            except services.errors.account.NoPermissionError as exc:
+                raise Forbidden(str(exc)) from exc
+
+        try:
+            return RagPipelineTransformService().transform_dataset(dataset, current_user.id, session)
+        except RagPipelineResourceNotFoundError as exc:
+            raise NotFound(str(exc)) from exc
 
 
 @console_ns.route("/rag/pipelines/<uuid:pipeline_id>/workflows/draft/datasource/variables-inspect")

api/controllers/console/error.py

@@ -92,11 +92,17 @@ class AccountInFreezeError(BaseHTTPException):
     error_code = "account_in_freeze"
     code = 400
     description = (
-        "This email account has been deleted within the past 30 days"
+        "This email account has been deleted within the past 30 days "
         "and is temporarily unavailable for new account registration."
     )
 
 
+class EmailDomainSuspendedError(BaseHTTPException):
+    error_code = "email_domain_suspended"
+    code = 400
+    description = "This email domain has been suspended."
+
+
 class EducationVerifyLimitError(BaseHTTPException):
     error_code = "education_verify_limit"
     description = "Rate limit exceeded"

api/controllers/console/explore/banner.py

@@ -3,17 +3,14 @@ from typing import cast
 
 from flask_restx import Namespace, Resource
 from pydantic import BaseModel, Field, RootModel, field_validator
-from sqlalchemy import select
 
 from controllers.common.schema import query_params_from_model, register_response_schema_models
 from controllers.console import api
 from controllers.console.wraps import model_validate
-from extensions.ext_database import db
+from extensions.ext_application_services import application_services
 from fields.base import ResponseModel
 from libs.helper import dump_response
 from models.enums import BannerStatus
-from models.model import ExporleBanner
-from services.feature_service import FeatureService
 
 
 class BannerListQuery(BaseModel):
@@ -67,23 +64,7 @@ class BannerApi(Resource):
     @model_validate(BannerListQuery)
     def get(self, req_data: BannerListQuery):
         """Get banner list."""
-        if not FeatureService.is_explore_banner_enabled():
-            return dump_response(BannerListResponse, [])
-
-        # Build base query for enabled banners
-        base_query = select(ExporleBanner).where(ExporleBanner.status == BannerStatus.ENABLED)
-
-        # Try to get banners in the requested language
-        banners = db.session.scalars(
-            base_query.where(ExporleBanner.language == req_data.language).order_by(ExporleBanner.sort)
-        ).all()
-
-        # Fallback to en-US if no banners found and language is not en-US
-        if not banners and req_data.language != "en-US":
-            banners = db.session.scalars(
-                base_query.where(ExporleBanner.language == "en-US").order_by(ExporleBanner.sort)
-            ).all()
-
+        banners = application_services().explore_banner_queries.list_for_language(req_data.language)
         return dump_response(BannerListResponse, banners)
 
 

api/controllers/console/explore/error.py

@@ -31,6 +31,12 @@ class AppAccessDeniedError(BaseHTTPException):
     code = 403
 
 
+class RecommendedAppNotFoundError(BaseHTTPException):
+    error_code = "recommended_app_not_found"
+    description = "Recommended app not found."
+    code = 404
+
+
 class TrialAppNotAllowed(BaseHTTPException):
     """*403* `Trial App Not Allowed`
 
@@ -51,3 +57,9 @@ class TrialAppLimitExceeded(BaseHTTPException):
     error_code = "trial_app_limit_exceeded"
     code = 403
     description = "The user has exceeded the trial app limit."
+
+
+class TrialAppFeatureDisabledError(BaseHTTPException):
+    error_code = "trial_app_feature_disabled"
+    code = 403
+    description = "Trial app feature is not enabled."

api/controllers/console/explore/parameter.py

@@ -1,16 +1,16 @@
-from typing import Any, cast
+from typing import Any
 
 from pydantic import BaseModel, Field
 
-from controllers.common import fields
+from controllers.common.fields import Parameters
 from controllers.common.schema import register_response_schema_models
 from controllers.console import console_ns
 from controllers.console.app.error import AppUnavailableError
 from controllers.console.explore.wraps import InstalledAppResource
-from core.app.app_config.common.parameters_mapping import get_parameters_from_feature_dict
-from extensions.ext_database import db
-from models.model import AppMode, InstalledApp, load_annotation_reply_config
-from services.app_service import AppService
+from extensions.ext_application_services import application_services
+from libs.helper import dump_response
+from models.model import InstalledApp
+from services.app_definition_query_service import AppDefinitionUnavailableError
 
 
 class ExploreAppMetaResponse(BaseModel):
@@ -22,44 +22,22 @@ class ExploreAppMetaResponse(BaseModel):
     tool_icons: dict[str, str | dict[str, Any]] = Field(default_factory=dict)
 
 
-register_response_schema_models(console_ns, fields.Parameters, ExploreAppMetaResponse)
+register_response_schema_models(console_ns, Parameters, ExploreAppMetaResponse)
 
 
 @console_ns.route("/installed-apps/<uuid:installed_app_id>/parameters", endpoint="installed_app_parameters")
 class AppParameterApi(InstalledAppResource):
     """Resource for app variables."""
 
-    @console_ns.response(200, "Success", console_ns.models[fields.Parameters.__name__])
+    @console_ns.response(200, "Success", console_ns.models[Parameters.__name__])
     def get(self, installed_app: InstalledApp):
         """Retrieve app parameters."""
-        session = db.session()
-        app_model = installed_app.app_with_session(session=session)
+        try:
+            parameters = application_services().app_definitions.get_parameters(installed_app.app_id)
+        except AppDefinitionUnavailableError:
+            raise AppUnavailableError() from None
 
-        if app_model is None:
-            raise AppUnavailableError()
-
-        if app_model.mode in {AppMode.ADVANCED_CHAT, AppMode.WORKFLOW}:
-            workflow = app_model.workflow_with_session(session=session)
-            if workflow is None:
-                raise AppUnavailableError()
-
-            features_dict: dict[str, Any] = workflow.features_dict
-            user_input_form = workflow.user_input_form(to_old_structure=True)
-        else:
-            app_model_config = app_model.app_model_config_with_session(session=session)
-            if app_model_config is None:
-                raise AppUnavailableError()
-
-            annotation_reply = load_annotation_reply_config(session, app_model.id)
-            features_dict = cast(
-                dict[str, Any],
-                app_model_config.to_dict(annotation_reply=annotation_reply),
-            )
-
-            user_input_form = features_dict.get("user_input_form", [])
-
-        parameters = get_parameters_from_feature_dict(features_dict=features_dict, user_input_form=user_input_form)
-        return fields.Parameters.model_validate(parameters).model_dump(mode="json")
+        return dump_response(Parameters, parameters)
 
 
 @console_ns.route("/installed-apps/<uuid:installed_app_id>/meta", endpoint="installed_app_meta")
@@ -67,7 +45,12 @@ class ExploreAppMetaApi(InstalledAppResource):
     @console_ns.response(200, "Success", console_ns.models[ExploreAppMetaResponse.__name__])
     def get(self, installed_app: InstalledApp):
         """Get app meta"""
-        app_model = installed_app.app_with_session(session=db.session())
-        if not app_model:
-            raise ValueError("App not found")
-        return AppService().get_app_meta(app_model, session=db.session())
+        try:
+            tool_icons = application_services().app_definitions.get_tool_icons(installed_app.app_id)
+        except AppDefinitionUnavailableError:
+            raise AppUnavailableError() from None
+
+        return dump_response(
+            ExploreAppMetaResponse,
+            {"tool_icons": tool_icons},
+        )