Initial plan

.agents/skills/e2e-cucumber-playwright/SKILL.md

@@ -1,79 +0,0 @@
----
-name: e2e-cucumber-playwright
-description: Write, update, or review Dify end-to-end tests under `e2e/` that use Cucumber, Gherkin, and Playwright. Use when the task involves `.feature` files, `features/step-definitions/`, `features/support/`, `DifyWorld`, scenario tags, locator/assertion choices, or E2E testing best practices for this repository.
----
-
-# Dify E2E Cucumber + Playwright
-
-Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS.md`](../../../e2e/AGENTS.md) as the canonical guide for local architecture and conventions, then apply Playwright/Cucumber best practices only where they fit the current suite.
-
-## Scope
-
-- Use this skill for `.feature` files, Cucumber step definitions, `DifyWorld`, hooks, tags, and E2E review work under `e2e/`.
-- Do not use this skill for Vitest or React Testing Library work under `web/`; use `frontend-testing` instead.
-- Do not use this skill for backend test or API review tasks under `api/`.
-
-## Read Order
-
-1. Read [`e2e/AGENTS.md`](../../../e2e/AGENTS.md) first.
-2. Read only the files directly involved in the task:
-   - target `.feature` files under `e2e/features/`
-   - related step files under `e2e/features/step-definitions/`
-   - `e2e/features/support/hooks.ts` and `e2e/features/support/world.ts` when session lifecycle or shared state matters
-   - `e2e/scripts/run-cucumber.ts` and `e2e/cucumber.config.ts` when tags or execution flow matter
-3. Read [`references/playwright-best-practices.md`](references/playwright-best-practices.md) only when locator, assertion, isolation, or waiting choices are involved.
-4. Read [`references/cucumber-best-practices.md`](references/cucumber-best-practices.md) only when scenario wording, step granularity, tags, or expression design are involved.
-5. Re-check official docs with Context7 before introducing a new Playwright or Cucumber pattern.
-
-## Local Rules
-
-- `e2e/` uses Cucumber for scenarios and Playwright as the browser layer.
-- `DifyWorld` is the per-scenario context object. Type `this` as `DifyWorld` and use `async function`, not arrow functions.
-- Keep glue organized by capability under `e2e/features/step-definitions/`; use `common/` only for broadly reusable steps.
-- Browser session behavior comes from `features/support/hooks.ts`:
-  - default: authenticated session with shared storage state
-  - `@unauthenticated`: clean browser context
-  - `@authenticated`: readability/selective-run tag only unless implementation changes
-  - `@fresh`: only for `e2e:full*` flows
-- Do not import Playwright Test runner patterns that bypass the current Cucumber + `DifyWorld` architecture unless the task is explicitly about changing that architecture.
-
-## Workflow
-
-1. Rebuild local context.
-   - Inspect the target feature area.
-   - Reuse an existing step when wording and behavior already match.
-   - Add a new step only for a genuinely new user action or assertion.
-   - Keep edits close to the current capability folder unless the step is broadly reusable.
-2. Write behavior-first scenarios.
-   - Describe user-observable behavior, not DOM mechanics.
-   - Keep each scenario focused on one workflow or outcome.
-   - Keep scenarios independent and re-runnable.
-3. Write step definitions in the local style.
-   - Keep one step to one user-visible action or one assertion.
-   - Prefer Cucumber Expressions such as `{string}` and `{int}`.
-   - Scope locators to stable containers when the page has repeated elements.
-   - Avoid page-object layers or extra helper abstractions unless repeated complexity clearly justifies them.
-4. Use Playwright in the local style.
-   - Prefer user-facing locators: `getByRole`, `getByLabel`, `getByPlaceholder`, `getByText`, then `getByTestId` for explicit contracts.
-   - Use web-first `expect(...)` assertions.
-   - Do not use `waitForTimeout`, manual polling, or raw visibility checks when a locator action or retrying assertion already expresses the behavior.
-5. Validate narrowly.
-   - Run the narrowest tagged scenario or flow that exercises the change.
-   - Run `pnpm -C e2e check`.
-   - Broaden verification only when the change affects hooks, tags, setup, or shared step semantics.
-
-## Review Checklist
-
-- Does the scenario describe behavior rather than implementation?
-- Does it fit the current session model, tags, and `DifyWorld` usage?
-- Should an existing step be reused instead of adding a new one?
-- Are locators user-facing and assertions web-first?
-- Does the change introduce hidden coupling across scenarios, tags, or instance state?
-- Does it document or implement behavior that differs from the real hooks or configuration?
-
-Lead findings with correctness, flake risk, and architecture drift.
-
-## References
-
-- [`references/playwright-best-practices.md`](references/playwright-best-practices.md)
-- [`references/cucumber-best-practices.md`](references/cucumber-best-practices.md)

.agents/skills/e2e-cucumber-playwright/agents/openai.yaml

@@ -1,4 +0,0 @@
-interface:
-  display_name: "E2E Cucumber + Playwright"
-  short_description: "Write and review Dify E2E scenarios."
-  default_prompt: "Use $e2e-cucumber-playwright to write or review a Dify E2E scenario under e2e/."

.agents/skills/e2e-cucumber-playwright/references/cucumber-best-practices.md

@@ -1,93 +0,0 @@
-# Cucumber Best Practices For Dify E2E
-
-Use this reference when writing or reviewing Gherkin scenarios, step definitions, parameter expressions, and step reuse in Dify's `e2e/` suite.
-
-Official sources:
-
-- https://cucumber.io/docs/guides/10-minute-tutorial/
-- https://cucumber.io/docs/cucumber/step-definitions/
-- https://cucumber.io/docs/cucumber/cucumber-expressions/
-
-## What Matters Most
-
-### 1. Treat scenarios as executable specifications
-
-Cucumber scenarios should describe examples of behavior, not test implementation recipes.
-
-Apply it like this:
-
-- write what the user does and what should happen
-- avoid UI-internal wording such as selector details, DOM structure, or component names
-- keep language concrete enough that the scenario reads like living documentation
-
-### 2. Keep scenarios focused
-
-A scenario should usually prove one workflow or business outcome. If a scenario wanders across several unrelated behaviors, split it.
-
-In Dify's suite, this means:
-
-- one capability-focused scenario per feature path
-- no long setup chains when existing bootstrap or reusable steps already cover them
-- no hidden dependency on another scenario's side effects
-
-### 3. Reuse steps, but only when behavior really matches
-
-Good reuse reduces duplication. Bad reuse hides meaning.
-
-Prefer reuse when:
-
-- the user action is genuinely the same
-- the expected outcome is genuinely the same
-- the wording stays natural across features
-
-Write a new step when:
-
-- the behavior is materially different
-- reusing the old wording would make the scenario misleading
-- a supposedly generic step would become an implementation-detail wrapper
-
-### 4. Prefer Cucumber Expressions
-
-Use Cucumber Expressions for parameters unless regex is clearly necessary.
-
-Common examples:
-
-- `{string}` for labels, names, and visible text
-- `{int}` for counts
-- `{float}` for decimal values
-- `{word}` only when the value is truly a single token
-
-Keep expressions readable. If a step needs complicated parsing logic, first ask whether the scenario wording should be simpler.
-
-### 5. Keep step definitions thin and meaningful
-
-Step definitions are glue between Gherkin and automation, not a second abstraction language.
-
-For Dify:
-
-- type `this` as `DifyWorld`
-- use `async function`
-- keep each step to one user-visible action or assertion
-- rely on `DifyWorld` and existing support code for shared context
-- avoid leaking cross-scenario state
-
-### 6. Use tags intentionally
-
-Tags should communicate run scope or session semantics, not become ad hoc metadata.
-
-In Dify's current suite:
-
-- capability tags group related scenarios
-- `@unauthenticated` changes session behavior
-- `@authenticated` is descriptive/selective, not a behavior switch by itself
-- `@fresh` belongs to reset/full-install flows only
-
-If a proposed tag implies behavior, verify that hooks or runner configuration actually implement it.
-
-## Review Questions
-
-- Does the scenario read like a real example of product behavior?
-- Are the steps behavior-oriented instead of implementation-oriented?
-- Is a reused step still truthful in this feature?
-- Is a new tag documenting real behavior, or inventing semantics that the suite does not implement?
-- Would a new reader understand the outcome without opening the step-definition file?

.agents/skills/e2e-cucumber-playwright/references/playwright-best-practices.md

@@ -1,96 +0,0 @@
-# Playwright Best Practices For Dify E2E
-
-Use this reference when writing or reviewing locator, assertion, isolation, or synchronization logic for Dify's Cucumber-based E2E suite.
-
-Official sources:
-
-- https://playwright.dev/docs/best-practices
-- https://playwright.dev/docs/locators
-- https://playwright.dev/docs/test-assertions
-- https://playwright.dev/docs/browser-contexts
-
-## What Matters Most
-
-### 1. Keep scenarios isolated
-
-Playwright's model is built around clean browser contexts so one test does not leak into another. In Dify's suite, that principle maps to per-scenario session setup in `features/support/hooks.ts` and `DifyWorld`.
-
-Apply it like this:
-
-- do not depend on another scenario having run first
-- do not persist ad hoc scenario state outside `DifyWorld`
-- do not couple ordinary scenarios to `@fresh` behavior
-- when a flow needs special auth/session semantics, express that through the existing tag model or explicit hook changes
-
-### 2. Prefer user-facing locators
-
-Playwright recommends built-in locators that reflect what users perceive on the page.
-
-Preferred order in this repository:
-
-1. `getByRole`
-2. `getByLabel`
-3. `getByPlaceholder`
-4. `getByText`
-5. `getByTestId` when an explicit test contract is the most stable option
-
-Avoid raw CSS/XPath selectors unless no stable user-facing contract exists and adding one is not practical.
-
-Also remember:
-
-- repeated content usually needs scoping to a stable container
-- exact text matching is often too brittle when role/name or label already exists
-- `getByTestId` is acceptable when semantics are weak but the contract is intentional
-
-### 3. Use web-first assertions
-
-Playwright assertions auto-wait and retry. Prefer them over manual state inspection.
-
-Prefer:
-
-- `await expect(page).toHaveURL(...)`
-- `await expect(locator).toBeVisible()`
-- `await expect(locator).toBeHidden()`
-- `await expect(locator).toBeEnabled()`
-- `await expect(locator).toHaveText(...)`
-
-Avoid:
-
-- `expect(await locator.isVisible()).toBe(true)`
-- custom polling loops for DOM state
-- `waitForTimeout` as synchronization
-
-If a condition genuinely needs custom retry logic, use Playwright's polling/assertion tools deliberately and keep that choice local and explicit.
-
-### 4. Let actions wait for actionability
-
-Locator actions already wait for the element to be actionable. Do not preface every click/fill with extra timing logic unless the action needs a specific visible/ready assertion for clarity.
-
-Good pattern:
-
-- assert a meaningful visible state when that is part of the behavior
-- then click/fill/select via locator APIs
-
-Bad pattern:
-
-- stack arbitrary waits before every action
-- wait on unstable implementation details instead of the visible state the user cares about
-
-### 5. Match debugging to the current suite
-
-Playwright's wider ecosystem supports traces and rich debugging tools. Dify's current suite already captures:
-
-- full-page screenshots
-- page HTML
-- console errors
-- page errors
-
-Use the existing artifact flow by default. If a task is specifically about improving diagnostics, confirm the change fits the current Cucumber architecture before importing broader Playwright tooling.
-
-## Review Questions
-
-- Would this locator survive DOM refactors that do not change user-visible behavior?
-- Is this assertion using Playwright's retrying semantics?
-- Is any explicit wait masking a real readiness problem?
-- Does this code preserve per-scenario isolation?
-- Is a new abstraction really needed, or does it bypass the existing `DifyWorld` + step-definition model?

.claude/skills/e2e-cucumber-playwright

@@ -1 +0,0 @@
-../../.agents/skills/e2e-cucumber-playwright

.github/dependabot.yml

@@ -1,6 +1,106 @@
 version: 2
 
 updates:
+  - package-ecosystem: "pip"
+    directory: "/api"
+    open-pull-requests-limit: 10
+    schedule:
+      interval: "weekly"
+    groups:
+      flask:
+        patterns:
+          - "flask"
+          - "flask-*"
+          - "werkzeug"
+          - "gunicorn"
+      google:
+        patterns:
+          - "google-*"
+          - "googleapis-*"
+      opentelemetry:
+        patterns:
+          - "opentelemetry-*"
+      pydantic:
+        patterns:
+          - "pydantic"
+          - "pydantic-*"
+      llm:
+        patterns:
+          - "langfuse"
+          - "langsmith"
+          - "litellm"
+          - "mlflow*"
+          - "opik"
+          - "weave*"
+          - "arize*"
+          - "tiktoken"
+          - "transformers"
+      database:
+        patterns:
+          - "sqlalchemy"
+          - "psycopg2*"
+          - "psycogreen"
+          - "redis*"
+          - "alembic*"
+      storage:
+        patterns:
+          - "boto3*"
+          - "botocore*"
+          - "azure-*"
+          - "bce-*"
+          - "cos-python-*"
+          - "esdk-obs-*"
+          - "google-cloud-storage"
+          - "opendal"
+          - "oss2"
+          - "supabase*"
+          - "tos*"
+      vdb:
+        patterns:
+          - "alibabacloud*"
+          - "chromadb"
+          - "clickhouse-*"
+          - "clickzetta-*"
+          - "couchbase"
+          - "elasticsearch"
+          - "opensearch-py"
+          - "oracledb"
+          - "pgvect*"
+          - "pymilvus"
+          - "pymochow"
+          - "pyobvector"
+          - "qdrant-client"
+          - "intersystems-*"
+          - "tablestore"
+          - "tcvectordb"
+          - "tidb-vector"
+          - "upstash-*"
+          - "volcengine-*"
+          - "weaviate-*"
+          - "xinference-*"
+          - "mo-vector"
+          - "mysql-connector-*"
+      dev:
+        patterns:
+          - "coverage"
+          - "dotenv-linter"
+          - "faker"
+          - "lxml-stubs"
+          - "basedpyright"
+          - "ruff"
+          - "pytest*"
+          - "types-*"
+          - "boto3-stubs"
+          - "hypothesis"
+          - "pandas-stubs"
+          - "scipy-stubs"
+          - "import-linter"
+          - "celery-types"
+          - "mypy*"
+          - "pyrefly"
+      python-packages:
+        patterns:
+          - "*"
   - package-ecosystem: "uv"
     directory: "/api"
     open-pull-requests-limit: 10

.github/labeler.yml

@@ -1,10 +1,3 @@
 web:
   - changed-files:
-      - any-glob-to-any-file:
-          - 'web/**'
-          - 'packages/**'
-          - 'package.json'
-          - 'pnpm-lock.yaml'
-          - 'pnpm-workspace.yaml'
-          - '.npmrc'
-          - '.nvmrc'
+      - any-glob-to-any-file: 'web/**'

.github/pull_request_template.md

@@ -7,7 +7,6 @@
 ## Summary
 
 <!-- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change. -->
-<!-- If this PR was created by an automated agent, add `From <Tool Name>` as the final line of the description. Example: `From Codex`. -->
 
 ## Screenshots
 
@@ -18,7 +17,7 @@
 ## Checklist
 
 - [ ] This change requires a documentation update, included: [Dify Document](https://github.com/langgenius/dify-docs)
-- [ ] I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
-- [ ] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
-- [ ] I've updated the documentation accordingly.
-- [ ] I ran `make lint && make type-check` (backend) and `cd web && pnpm exec vp staged` (frontend) to appease the lint gods
+- [x] I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
+- [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
+- [x] I've updated the documentation accordingly.
+- [x] I ran `make lint` and `make type-check` (backend) and `cd web && pnpm exec vp staged` (frontend) to appease the lint gods

.github/scripts/generate-i18n-changes.mjs

@@ -1,82 +0,0 @@
-import { execFileSync } from 'node:child_process'
-import fs from 'node:fs'
-import path from 'node:path'
-
-const repoRoot = process.cwd()
-const baseSha = process.env.BASE_SHA || ''
-const headSha = process.env.HEAD_SHA || ''
-const files = (process.env.CHANGED_FILES || '').split(/\s+/).filter(Boolean)
-const outputPath = process.env.I18N_CHANGES_OUTPUT_PATH || '/tmp/i18n-changes.json'
-
-const englishPath = fileStem => path.join(repoRoot, 'web', 'i18n', 'en-US', `${fileStem}.json`)
-
-const readCurrentJson = (fileStem) => {
-  const filePath = englishPath(fileStem)
-  if (!fs.existsSync(filePath))
-    return null
-
-  return JSON.parse(fs.readFileSync(filePath, 'utf8'))
-}
-
-const readBaseJson = (fileStem) => {
-  if (!baseSha)
-    return null
-
-  try {
-    const relativePath = `web/i18n/en-US/${fileStem}.json`
-    const content = execFileSync('git', ['show', `${baseSha}:${relativePath}`], { encoding: 'utf8' })
-    return JSON.parse(content)
-  }
-  catch {
-    return null
-  }
-}
-
-const compareJson = (beforeValue, afterValue) => JSON.stringify(beforeValue) === JSON.stringify(afterValue)
-
-const changes = {}
-
-for (const fileStem of files) {
-  const currentJson = readCurrentJson(fileStem)
-  const beforeJson = readBaseJson(fileStem) || {}
-  const afterJson = currentJson || {}
-  const added = {}
-  const updated = {}
-  const deleted = []
-
-  for (const [key, value] of Object.entries(afterJson)) {
-    if (!(key in beforeJson)) {
-      added[key] = value
-      continue
-    }
-
-    if (!compareJson(beforeJson[key], value)) {
-      updated[key] = {
-        before: beforeJson[key],
-        after: value,
-      }
-    }
-  }
-
-  for (const key of Object.keys(beforeJson)) {
-    if (!(key in afterJson))
-      deleted.push(key)
-  }
-
-  changes[fileStem] = {
-    fileDeleted: currentJson === null,
-    added,
-    updated,
-    deleted,
-  }
-}
-
-fs.writeFileSync(
-  outputPath,
-  JSON.stringify({
-    baseSha,
-    headSha,
-    files,
-    changes,
-  })
-)

.github/workflows/api-tests.yml

@@ -54,7 +54,7 @@ jobs:
         run: uv run --project api bash dev/pytest/pytest_unit_tests.sh
 
       - name: Upload unit coverage data
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
         with:
           name: api-coverage-unit
           path: coverage-unit
@@ -129,7 +129,7 @@ jobs:
             api/tests/test_containers_integration_tests
 
       - name: Upload integration coverage data
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
         with:
           name: api-coverage-integration
           path: coverage-integration

.github/workflows/autofix.yml

@@ -39,11 +39,9 @@ jobs:
         with:
           files: |
             web/**
-            packages/**
             package.json
             pnpm-lock.yaml
             pnpm-workspace.yaml
-            .npmrc
             .nvmrc
       - name: Check api inputs
         if: github.event_name != 'merge_group'

.github/workflows/build-push.yml

@@ -81,7 +81,7 @@ jobs:
 
       - name: Build Docker image
         id: build
-        uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
+        uses: docker/build-push-action@d08e5c354a6adb9ed34480a06d141179aa583294 # v7.0.0
         with:
           context: ${{ matrix.build_context }}
           file: ${{ matrix.file }}
@@ -101,7 +101,7 @@ jobs:
           touch "/tmp/digests/${sanitized_digest}"
 
       - name: Upload digest
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
         with:
           name: digests-${{ matrix.artifact_context }}-${{ env.PLATFORM_PAIR }}
           path: /tmp/digests/*

.github/workflows/docker-build.yml

@@ -6,7 +6,12 @@ on:
       - "main"
     paths:
       - api/Dockerfile
+      - web/docker/**
       - web/Dockerfile
+      - package.json
+      - pnpm-lock.yaml
+      - pnpm-workspace.yaml
+      - .nvmrc
 
 concurrency:
   group: docker-build-${{ github.head_ref || github.run_id }}
@@ -43,7 +48,7 @@ jobs:
         uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
 
       - name: Build Docker Image
-        uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
+        uses: docker/build-push-action@d08e5c354a6adb9ed34480a06d141179aa583294 # v7.0.0
         with:
           push: false
           context: ${{ matrix.context }}

.github/workflows/main-ci.yml

@@ -65,11 +65,9 @@ jobs:
               - 'docker/volumes/sandbox/conf/**'
             web:
               - 'web/**'
-              - 'packages/**'
               - 'package.json'
               - 'pnpm-lock.yaml'
               - 'pnpm-workspace.yaml'
-              - '.npmrc'
               - '.nvmrc'
               - '.github/workflows/web-tests.yml'
               - '.github/actions/setup-web/**'
@@ -79,11 +77,9 @@ jobs:
               - 'api/uv.lock'
               - 'e2e/**'
               - 'web/**'
-              - 'packages/**'
               - 'package.json'
               - 'pnpm-lock.yaml'
               - 'pnpm-workspace.yaml'
-              - '.npmrc'
               - '.nvmrc'
               - 'docker/docker-compose.middleware.yaml'
               - 'docker/middleware.env.example'
@@ -92,7 +88,6 @@ jobs:
             vdb:
               - 'api/core/rag/datasource/**'
               - 'api/tests/integration_tests/vdb/**'
-              - 'api/providers/vdb/*/tests/**'
               - '.github/workflows/vdb-tests.yml'
               - '.github/workflows/expose_service_ports.sh'
               - 'docker/.env.example'

.github/workflows/pyrefly-diff-comment.yml

@@ -21,7 +21,7 @@ jobs:
     if: ${{ github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.pull_requests[0].head.repo.full_name != github.repository }}
     steps:
       - name: Download pyrefly diff artifact
-        uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+        uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
         with:
           github-token: ${{ secrets.GITHUB_TOKEN }}
           script: |
@@ -49,7 +49,7 @@ jobs:
         run: unzip -o pyrefly_diff.zip
 
       - name: Post comment
-        uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+        uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
         with:
           github-token: ${{ secrets.GITHUB_TOKEN }}
           script: |
@@ -79,11 +79,10 @@ jobs:
             const body = diff.trim()
               ? '### Pyrefly Diff\n<details>\n<summary>base → PR</summary>\n\n```diff\n' + diff + '\n```\n</details>'
               : '### Pyrefly Diff\nNo changes detected.';
-            if (diff.trim()) {
-              await github.rest.issues.createComment({
-                issue_number: prNumber,
-                owner: context.repo.owner,
-                repo: context.repo.repo,
-                body,
-              });
-            }
+
+            await github.rest.issues.createComment({
+              issue_number: prNumber,
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              body,
+            });

.github/workflows/pyrefly-diff.yml

@@ -66,7 +66,7 @@ jobs:
           echo ${{ github.event.pull_request.number }} > pr_number.txt
 
       - name: Upload pyrefly diff
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
         with:
           name: pyrefly_diff
           path: |
@@ -75,7 +75,7 @@ jobs:
 
       - name: Comment PR with pyrefly diff
         if: ${{ github.event.pull_request.head.repo.full_name == github.repository && steps.line_count_check.outputs.same == 'false' }}
-        uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+        uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
         with:
           github-token: ${{ secrets.GITHUB_TOKEN }}
           script: |

.github/workflows/pyrefly-type-coverage-comment.yml

@@ -1,118 +0,0 @@
-name: Comment with Pyrefly Type Coverage
-
-on:
-  workflow_run:
-    workflows:
-      - Pyrefly Type Coverage
-    types:
-      - completed
-
-permissions: {}
-
-jobs:
-  comment:
-    name: Comment PR with type coverage
-    runs-on: ubuntu-latest
-    permissions:
-      actions: read
-      contents: read
-      issues: write
-      pull-requests: write
-    if: ${{ github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.pull_requests[0].head.repo.full_name != github.repository }}
-    steps:
-      - name: Checkout default branch (trusted code)
-        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
-
-      - name: Setup Python & UV
-        uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
-        with:
-          enable-cache: true
-
-      - name: Install dependencies
-        run: uv sync --project api --dev
-
-      - name: Download type coverage artifact
-        uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-          script: |
-            const fs = require('fs');
-            const artifacts = await github.rest.actions.listWorkflowRunArtifacts({
-              owner: context.repo.owner,
-              repo: context.repo.repo,
-              run_id: ${{ github.event.workflow_run.id }},
-            });
-            const match = artifacts.data.artifacts.find((artifact) =>
-              artifact.name === 'pyrefly_type_coverage'
-            );
-            if (!match) {
-              throw new Error('pyrefly_type_coverage artifact not found');
-            }
-            const download = await github.rest.actions.downloadArtifact({
-              owner: context.repo.owner,
-              repo: context.repo.repo,
-              artifact_id: match.id,
-              archive_format: 'zip',
-            });
-            fs.writeFileSync('pyrefly_type_coverage.zip', Buffer.from(download.data));
-
-      - name: Unzip artifact
-        run: unzip -o pyrefly_type_coverage.zip
-
-      - name: Render coverage markdown from structured data
-        id: render
-        run: |
-          comment_body="$(uv run --directory api python libs/pyrefly_type_coverage.py \
-            --base base_report.json \
-            < pr_report.json)"
-
-          {
-            echo "### Pyrefly Type Coverage"
-            echo ""
-            echo "$comment_body"
-          } > /tmp/type_coverage_comment.md
-
-      - name: Post comment
-        uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-          script: |
-            const fs = require('fs');
-            const body = fs.readFileSync('/tmp/type_coverage_comment.md', { encoding: 'utf8' });
-            let prNumber = null;
-            try {
-              prNumber = parseInt(fs.readFileSync('pr_number.txt', { encoding: 'utf8' }), 10);
-            } catch (err) {
-              const prs = context.payload.workflow_run.pull_requests || [];
-              if (prs.length > 0 && prs[0].number) {
-                prNumber = prs[0].number;
-              }
-            }
-            if (!prNumber) {
-              throw new Error('PR number not found in artifact or workflow_run payload');
-            }
-
-            // Update existing comment if one exists, otherwise create new
-            const { data: comments } = await github.rest.issues.listComments({
-              issue_number: prNumber,
-              owner: context.repo.owner,
-              repo: context.repo.repo,
-            });
-            const marker = '### Pyrefly Type Coverage';
-            const existing = comments.find(c => c.body.startsWith(marker));
-
-            if (existing) {
-              await github.rest.issues.updateComment({
-                comment_id: existing.id,
-                owner: context.repo.owner,
-                repo: context.repo.repo,
-                body,
-              });
-            } else {
-              await github.rest.issues.createComment({
-                issue_number: prNumber,
-                owner: context.repo.owner,
-                repo: context.repo.repo,
-                body,
-              });
-            }

.github/workflows/pyrefly-type-coverage.yml

@@ -1,120 +0,0 @@
-name: Pyrefly Type Coverage
-
-on:
-  pull_request:
-    paths:
-      - 'api/**/*.py'
-
-permissions:
-  contents: read
-
-jobs:
-  pyrefly-type-coverage:
-    runs-on: ubuntu-latest
-    permissions:
-      contents: read
-      issues: write
-      pull-requests: write
-    steps:
-      - name: Checkout PR branch
-        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
-        with:
-          fetch-depth: 0
-
-      - name: Setup Python & UV
-        uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
-        with:
-          enable-cache: true
-
-      - name: Install dependencies
-        run: uv sync --project api --dev
-
-      - name: Run pyrefly report on PR branch
-        run: |
-          uv run --directory api --dev pyrefly report 2>/dev/null > /tmp/pyrefly_report_pr.tmp && \
-            mv /tmp/pyrefly_report_pr.tmp /tmp/pyrefly_report_pr.json || \
-            echo '{}' > /tmp/pyrefly_report_pr.json
-
-      - name: Save helper script from base branch
-        run: |
-          git show ${{ github.event.pull_request.base.sha }}:api/libs/pyrefly_type_coverage.py > /tmp/pyrefly_type_coverage.py 2>/dev/null \
-            || cp api/libs/pyrefly_type_coverage.py /tmp/pyrefly_type_coverage.py
-
-      - name: Checkout base branch
-        run: git checkout ${{ github.base_ref }}
-
-      - name: Run pyrefly report on base branch
-        run: |
-          uv run --directory api --dev pyrefly report 2>/dev/null > /tmp/pyrefly_report_base.tmp && \
-            mv /tmp/pyrefly_report_base.tmp /tmp/pyrefly_report_base.json || \
-            echo '{}' > /tmp/pyrefly_report_base.json
-
-      - name: Generate coverage comparison
-        id: coverage
-        run: |
-          comment_body="$(uv run --directory api python /tmp/pyrefly_type_coverage.py \
-            --base /tmp/pyrefly_report_base.json \
-            < /tmp/pyrefly_report_pr.json)"
-
-          {
-            echo "### Pyrefly Type Coverage"
-            echo ""
-            echo "$comment_body"
-          } | tee -a "$GITHUB_STEP_SUMMARY" > /tmp/type_coverage_comment.md
-
-          # Save structured data for the fork-PR comment workflow
-          cp /tmp/pyrefly_report_pr.json pr_report.json
-          cp /tmp/pyrefly_report_base.json base_report.json
-
-      - name: Save PR number
-        run: |
-          echo ${{ github.event.pull_request.number }} > pr_number.txt
-
-      - name: Upload type coverage artifact
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
-        with:
-          name: pyrefly_type_coverage
-          path: |
-            pr_report.json
-            base_report.json
-            pr_number.txt
-
-      - name: Comment PR with type coverage
-        if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
-        uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-          script: |
-            const fs = require('fs');
-            const marker = '### Pyrefly Type Coverage';
-            let body;
-            try {
-              body = fs.readFileSync('/tmp/type_coverage_comment.md', { encoding: 'utf8' });
-            } catch {
-              body = `${marker}\n\n_Coverage report unavailable._`;
-            }
-            const prNumber = context.payload.pull_request.number;
-
-            // Update existing comment if one exists, otherwise create new
-            const { data: comments } = await github.rest.issues.listComments({
-              issue_number: prNumber,
-              owner: context.repo.owner,
-              repo: context.repo.repo,
-            });
-            const existing = comments.find(c => c.body.startsWith(marker));
-
-            if (existing) {
-              await github.rest.issues.updateComment({
-                comment_id: existing.id,
-                owner: context.repo.owner,
-                repo: context.repo.repo,
-                body,
-              });
-            } else {
-              await github.rest.issues.createComment({
-                issue_number: prNumber,
-                owner: context.repo.owner,
-                repo: context.repo.repo,
-                body,
-              });
-            }

.github/workflows/stale.yml

@@ -23,8 +23,8 @@ jobs:
           days-before-issue-stale: 15
           days-before-issue-close: 3
           repo-token: ${{ secrets.GITHUB_TOKEN }}
-          stale-issue-message: "Closed due to inactivity. If you have any questions, you can reopen it."
-          stale-pr-message: "Closed due to inactivity. If you have any questions, you can reopen it."
+          stale-issue-message: "Close due to it's no longer active, if you have any questions, you can reopen it."
+          stale-pr-message: "Close due to it's no longer active, if you have any questions, you can reopen it."
           stale-issue-label: 'no-issue-activity'
           stale-pr-label: 'no-pr-activity'
-          any-of-labels: '🌚 invalid,🙋‍♂️ question,wont-fix,no-issue-activity,no-pr-activity,💪 enhancement,🤔 cant-reproduce,🙏 help wanted'
+          any-of-labels: 'duplicate,question,invalid,wontfix,no-issue-activity,no-pr-activity,enhancement,cant-reproduce,help-wanted'

.github/workflows/style.yml

@@ -77,11 +77,9 @@ jobs:
         with:
           files: |
             web/**
-            packages/**
             package.json
             pnpm-lock.yaml
             pnpm-workspace.yaml
-            .npmrc
             .nvmrc
             .github/workflows/style.yml
             .github/actions/setup-web/**

.github/workflows/tool-test-sdks.yaml

@@ -9,7 +9,6 @@ on:
       - package.json
       - pnpm-lock.yaml
       - pnpm-workspace.yaml
-      - .npmrc
 
 concurrency:
   group: sdk-tests-${{ github.head_ref || github.run_id }}

.github/workflows/translate-i18n-claude.yml

@@ -68,7 +68,89 @@ jobs:
           " web/i18n-config/languages.ts | sed 's/[[:space:]]*$//')
 
           generate_changes_json() {
-            node .github/scripts/generate-i18n-changes.mjs
+            node <<'NODE'
+            const { execFileSync } = require('node:child_process')
+            const fs = require('node:fs')
+            const path = require('node:path')
+
+            const repoRoot = process.cwd()
+            const baseSha = process.env.BASE_SHA || ''
+            const headSha = process.env.HEAD_SHA || ''
+            const files = (process.env.CHANGED_FILES || '').split(/\s+/).filter(Boolean)
+
+            const englishPath = fileStem => path.join(repoRoot, 'web', 'i18n', 'en-US', `${fileStem}.json`)
+
+            const readCurrentJson = (fileStem) => {
+              const filePath = englishPath(fileStem)
+              if (!fs.existsSync(filePath))
+                return null
+
+              return JSON.parse(fs.readFileSync(filePath, 'utf8'))
+            }
+
+            const readBaseJson = (fileStem) => {
+              if (!baseSha)
+                return null
+
+              try {
+                const relativePath = `web/i18n/en-US/${fileStem}.json`
+                const content = execFileSync('git', ['show', `${baseSha}:${relativePath}`], { encoding: 'utf8' })
+                return JSON.parse(content)
+              }
+              catch (error) {
+                return null
+              }
+            }
+
+            const compareJson = (beforeValue, afterValue) => JSON.stringify(beforeValue) === JSON.stringify(afterValue)
+
+            const changes = {}
+
+            for (const fileStem of files) {
+              const currentJson = readCurrentJson(fileStem)
+              const beforeJson = readBaseJson(fileStem) || {}
+              const afterJson = currentJson || {}
+              const added = {}
+              const updated = {}
+              const deleted = []
+
+              for (const [key, value] of Object.entries(afterJson)) {
+                if (!(key in beforeJson)) {
+                  added[key] = value
+                  continue
+                }
+
+                if (!compareJson(beforeJson[key], value)) {
+                  updated[key] = {
+                    before: beforeJson[key],
+                    after: value,
+                  }
+                }
+              }
+
+              for (const key of Object.keys(beforeJson)) {
+                if (!(key in afterJson))
+                  deleted.push(key)
+              }
+
+              changes[fileStem] = {
+                fileDeleted: currentJson === null,
+                added,
+                updated,
+                deleted,
+              }
+            }
+
+            fs.writeFileSync(
+              '/tmp/i18n-changes.json',
+              JSON.stringify({
+                baseSha,
+                headSha,
+                files,
+                changes,
+              })
+            )
+            NODE
           }
 
           if [ "${{ github.event_name }}" = "repository_dispatch" ]; then
@@ -158,7 +240,7 @@ jobs:
 
       - name: Run Claude Code for Translation Sync
         if: steps.context.outputs.CHANGED_FILES != ''
-        uses: anthropics/claude-code-action@b47fd721da662d48c5680e154ad16a73ed74d2e0 # v1.0.93
+        uses: anthropics/claude-code-action@6e2bd52842c65e914eba5c8badd17560bd26b5de # v1.0.89
         with:
           anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
           github_token: ${{ secrets.GITHUB_TOKEN }}
@@ -188,7 +270,7 @@ jobs:
             Tool rules:
             - Use Read for repository files.
             - Use Edit for JSON updates.
-            - Use Bash only for `vp`.
+            - Use Bash only for `pnpm`.
             - Do not use Bash for `git`, `gh`, or branch management.
 
             Required execution plan:
@@ -210,7 +292,7 @@ jobs:
                - Read the current English JSON file for any file that still exists so wording, placeholders, and surrounding terminology stay accurate.
                - If `Structured change set available` is `false`, treat this as a scoped full sync and use the current English files plus scoped checks as the source of truth.
             4. Run a scoped pre-check before editing:
-               - `vp run dify-web#i18n:check ${{ steps.context.outputs.FILE_ARGS }} ${{ steps.context.outputs.LANG_ARGS }}`
+               - `pnpm --dir ${{ github.workspace }}/web run i18n:check ${{ steps.context.outputs.FILE_ARGS }} ${{ steps.context.outputs.LANG_ARGS }}`
                - Use this command as the source of truth for missing and extra keys inside the current scope.
             5. Apply translations.
                - For every target language and scoped file:
@@ -218,19 +300,19 @@ jobs:
                  - If the locale file does not exist yet, create it with `Write` and then continue with `Edit` as needed.
                  - ADD missing keys.
                  - UPDATE stale translations when the English value changed.
-                 - DELETE removed keys. Prefer `vp run dify-web#i18n:check ${{ steps.context.outputs.FILE_ARGS }} ${{ steps.context.outputs.LANG_ARGS }} --auto-remove` for extra keys so deletions stay in scope.
+                 - DELETE removed keys. Prefer `pnpm --dir ${{ github.workspace }}/web run i18n:check ${{ steps.context.outputs.FILE_ARGS }} ${{ steps.context.outputs.LANG_ARGS }} --auto-remove` for extra keys so deletions stay in scope.
                - Preserve placeholders exactly: `{{variable}}`, `${variable}`, HTML tags, component tags, and variable names.
                - Match the existing terminology and register used by each locale.
                - Prefer one Edit per file when stable, but prioritize correctness over batching.
             6. Verify only the edited files.
-               - Run `vp run dify-web#lint:fix --quiet -- <relative edited i18n file paths under web/>`
-               - Run `vp run dify-web#i18n:check ${{ steps.context.outputs.FILE_ARGS }} ${{ steps.context.outputs.LANG_ARGS }}`
+               - Run `pnpm --dir ${{ github.workspace }}/web lint:fix --quiet -- <relative edited i18n file paths>`
+               - Run `pnpm --dir ${{ github.workspace }}/web run i18n:check ${{ steps.context.outputs.FILE_ARGS }} ${{ steps.context.outputs.LANG_ARGS }}`
                - If verification fails, fix the remaining problems before continuing.
             7. Stop after the scoped locale files are updated and verification passes.
                - Do not create branches, commits, or pull requests.
           claude_args: |
             --max-turns 120
-            --allowedTools "Read,Write,Edit,Bash(vp *),Bash(vp:*),Glob,Grep"
+            --allowedTools "Read,Write,Edit,Bash(pnpm *),Bash(pnpm:*),Glob,Grep"
 
       - name: Prepare branch metadata
         id: pr_meta
@@ -272,7 +354,6 @@ jobs:
       - name: Create or update translation PR
         if: steps.pr_meta.outputs.has_changes == 'true'
         env:
-          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           BRANCH_NAME: ${{ steps.pr_meta.outputs.branch_name }}
           FILES_IN_SCOPE: ${{ steps.context.outputs.CHANGED_FILES }}
           TARGET_LANGS: ${{ steps.context.outputs.TARGET_LANGS }}
@@ -321,8 +402,8 @@ jobs:
             '',
             '## Verification',
             '',
-            `- \`vp run dify-web#i18n:check --file ${process.env.FILES_IN_SCOPE} --lang ${process.env.TARGET_LANGS}\``,
-            `- \`vp run dify-web#lint:fix --quiet -- <edited i18n files under web/>\``,
+            `- \`pnpm --dir web run i18n:check --file ${process.env.FILES_IN_SCOPE} --lang ${process.env.TARGET_LANGS}\``,
+            `- \`pnpm --dir web lint:fix --quiet -- <edited i18n files>\``,
             '',
             '## Notes',
             '',

.github/workflows/trigger-i18n-sync.yml

@@ -42,7 +42,88 @@ jobs:
           fi
 
           export BASE_SHA HEAD_SHA CHANGED_FILES
-          node .github/scripts/generate-i18n-changes.mjs
+          node <<'NODE'
+          const { execFileSync } = require('node:child_process')
+          const fs = require('node:fs')
+          const path = require('node:path')
+
+          const repoRoot = process.cwd()
+          const baseSha = process.env.BASE_SHA || ''
+          const headSha = process.env.HEAD_SHA || ''
+          const files = (process.env.CHANGED_FILES || '').split(/\s+/).filter(Boolean)
+
+          const englishPath = fileStem => path.join(repoRoot, 'web', 'i18n', 'en-US', `${fileStem}.json`)
+
+          const readCurrentJson = (fileStem) => {
+            const filePath = englishPath(fileStem)
+            if (!fs.existsSync(filePath))
+              return null
+
+            return JSON.parse(fs.readFileSync(filePath, 'utf8'))
+          }
+
+          const readBaseJson = (fileStem) => {
+            if (!baseSha)
+              return null
+
+            try {
+              const relativePath = `web/i18n/en-US/${fileStem}.json`
+              const content = execFileSync('git', ['show', `${baseSha}:${relativePath}`], { encoding: 'utf8' })
+              return JSON.parse(content)
+            }
+            catch (error) {
+              return null
+            }
+          }
+
+          const compareJson = (beforeValue, afterValue) => JSON.stringify(beforeValue) === JSON.stringify(afterValue)
+
+          const changes = {}
+
+          for (const fileStem of files) {
+            const beforeJson = readBaseJson(fileStem) || {}
+            const afterJson = readCurrentJson(fileStem) || {}
+            const added = {}
+            const updated = {}
+            const deleted = []
+
+            for (const [key, value] of Object.entries(afterJson)) {
+              if (!(key in beforeJson)) {
+                added[key] = value
+                continue
+              }
+
+              if (!compareJson(beforeJson[key], value)) {
+                updated[key] = {
+                  before: beforeJson[key],
+                  after: value,
+                }
+              }
+            }
+
+            for (const key of Object.keys(beforeJson)) {
+              if (!(key in afterJson))
+                deleted.push(key)
+            }
+
+            changes[fileStem] = {
+              fileDeleted: readCurrentJson(fileStem) === null,
+              added,
+              updated,
+              deleted,
+            }
+          }
+
+          fs.writeFileSync(
+            '/tmp/i18n-changes.json',
+            JSON.stringify({
+              baseSha,
+              headSha,
+              files,
+              changes,
+            })
+          )
+          NODE
 
           if [ -n "$CHANGED_FILES" ]; then
             echo "has_changes=true" >> "$GITHUB_OUTPUT"
@@ -56,7 +137,7 @@ jobs:
 
       - name: Trigger i18n sync workflow
         if: steps.detect.outputs.has_changes == 'true'
-        uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
+        uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
         env:
           BASE_SHA: ${{ steps.detect.outputs.base_sha }}
           HEAD_SHA: ${{ steps.detect.outputs.head_sha }}

.github/workflows/vdb-tests-full.yml

@@ -89,7 +89,7 @@ jobs:
           cp api/tests/integration_tests/.env.example api/tests/integration_tests/.env
 
 #      - name: Check VDB Ready (TiDB)
-#        run: uv run --project api python api/providers/vdb/tidb-vector/tests/integration_tests/check_tiflash_ready.py
+#        run: uv run --project api python api/tests/integration_tests/vdb/tidb_vector/check_tiflash_ready.py
 
       - name: Test Vector Stores
         run: uv run --project api bash dev/pytest/pytest_vdb.sh

.github/workflows/vdb-tests.yml

@@ -81,12 +81,12 @@ jobs:
           cp api/tests/integration_tests/.env.example api/tests/integration_tests/.env
 
 #      - name: Check VDB Ready (TiDB)
-#        run: uv run --project api python api/providers/vdb/tidb-vector/tests/integration_tests/check_tiflash_ready.py
+#        run: uv run --project api python api/tests/integration_tests/vdb/tidb_vector/check_tiflash_ready.py
 
       - name: Test Vector Stores
         run: |
           uv run --project api pytest --timeout "${PYTEST_TIMEOUT:-180}" \
-            api/providers/vdb/vdb-chroma/tests/integration_tests \
-            api/providers/vdb/vdb-pgvector/tests/integration_tests \
-            api/providers/vdb/vdb-qdrant/tests/integration_tests \
-            api/providers/vdb/vdb-weaviate/tests/integration_tests
+            api/tests/integration_tests/vdb/chroma \
+            api/tests/integration_tests/vdb/pgvector \
+            api/tests/integration_tests/vdb/qdrant \
+            api/tests/integration_tests/vdb/weaviate

.github/workflows/web-e2e.yml

@@ -53,7 +53,7 @@ jobs:
 
       - name: Upload Cucumber report
         if: ${{ !cancelled() }}
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
         with:
           name: cucumber-report
           path: e2e/cucumber-report
@@ -61,7 +61,7 @@ jobs:
 
       - name: Upload E2E logs
         if: ${{ !cancelled() }}
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
         with:
           name: e2e-logs
           path: e2e/.logs

.github/workflows/web-tests.yml

@@ -43,7 +43,7 @@ jobs:
 
       - name: Upload blob report
         if: ${{ !cancelled() }}
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
         with:
           name: blob-report-${{ matrix.shardIndex }}
           path: web/.vitest-reports/*

.vite-hooks/pre-commit

@@ -81,8 +81,8 @@ if $web_modified; then
 
     if $web_ts_modified; then
         echo "Running TypeScript type-check:tsgo"
-        if ! npm run type-check:tsgo; then
-            echo "Type check failed. Please run 'npm run type-check:tsgo' to fix the errors."
+        if ! pnpm run type-check:tsgo; then
+            echo "Type check failed. Please run 'pnpm run type-check:tsgo' to fix the errors."
             exit 1
         fi
     else
@@ -90,8 +90,8 @@ if $web_modified; then
     fi
 
     echo "Running knip"
-    if ! npm run knip; then
-        echo "Knip check failed. Please run 'npm run knip' to fix the errors."
+    if ! pnpm run knip; then
+        echo "Knip check failed. Please run 'pnpm run knip' to fix the errors."
         exit 1
     fi
 

CONTRIBUTING.md

@@ -97,3 +97,8 @@ Feel free to reach out if you encounter any issues during the setup process.
 ## Getting Help
 
 If you ever get stuck or get a burning question while contributing, simply shoot your queries our way via the related GitHub issue, or hop onto our [Discord](https://discord.gg/8Tpq4AcN9c) for a quick chat.
+
+## Automated Agent Contributions
+
+> [!NOTE]
+> If you are an automated agent， just add 🤖🤖🤖 to the end of the PR title to opt-in.

api/.env.example

@@ -57,9 +57,6 @@ REDIS_SSL_CERTFILE=
 REDIS_SSL_KEYFILE=
 # Path to client private key file for SSL authentication
 REDIS_DB=0
-# Optional global prefix for Redis keys, topics, streams, and Celery Redis transport artifacts.
-# Leave empty to preserve current unprefixed behavior.
-REDIS_KEY_PREFIX=
 
 # redis Sentinel configuration.
 REDIS_USE_SENTINEL=false
@@ -74,13 +71,6 @@ REDIS_USE_CLUSTERS=false
 REDIS_CLUSTERS=
 REDIS_CLUSTERS_PASSWORD=
 
-REDIS_RETRY_RETRIES=3
-REDIS_RETRY_BACKOFF_BASE=1.0
-REDIS_RETRY_BACKOFF_CAP=10.0
-REDIS_SOCKET_TIMEOUT=5.0
-REDIS_SOCKET_CONNECT_TIMEOUT=5.0
-REDIS_HEALTH_CHECK_INTERVAL=30
-
 # celery configuration
 CELERY_BROKER_URL=redis://:difyai123456@localhost:${REDIS_PORT}/1
 CELERY_BACKEND=redis
@@ -112,7 +102,6 @@ S3_BUCKET_NAME=your-bucket-name
 S3_ACCESS_KEY=your-access-key
 S3_SECRET_KEY=your-secret-key
 S3_REGION=your-region
-S3_ADDRESS_STYLE=auto
 
 # Workflow run and Conversation archive storage (S3-compatible)
 ARCHIVE_STORAGE_ENABLED=false

api/.ruff.toml

@@ -69,6 +69,8 @@ ignore = [
     "FURB152", # math-constant
     "UP007",   # non-pep604-annotation
     "UP032",   # f-string
+    "UP045",   # non-pep604-annotation-optional
+    "B005",    # strip-with-multi-characters
     "B006",    # mutable-argument-default
     "B007",    # unused-loop-control-variable
     "B026",    # star-arg-unpacking-after-keyword-arg
@@ -82,6 +84,7 @@ ignore = [
     "SIM102",  # collapsible-if
     "SIM103",  # needless-bool
     "SIM105",  # suppressible-exception
+    "SIM107",  # return-in-try-except-finally
     "SIM108",  # if-else-block-instead-of-if-exp
     "SIM113",  # enumerate-for-loop
     "SIM117",  # multiple-with-statements
@@ -90,16 +93,29 @@ ignore = [
 ]
 
 [lint.per-file-ignores]
+"__init__.py" = [
+    "F401", # unused-import
+    "F811", # redefined-while-unused
+]
 "configs/*" = [
     "N802", # invalid-function-name
 ]
+"graphon/model_runtime/callbacks/base_callback.py" = ["T201"]
+"core/workflow/callbacks/workflow_logging_callback.py" = ["T201"]
 "libs/gmpy2_pkcs10aep_cipher.py" = [
     "N803", # invalid-argument-name
 ]
 "tests/*" = [
+    "F811", # redefined-while-unused
     "T201", # allow print in tests,
     "S110", # allow ignoring exceptions in tests code (currently)
+
 ]
+"controllers/console/explore/trial.py" = ["TID251"]
+"controllers/console/human_input_form.py" = ["TID251"]
+"controllers/web/human_input_form.py" = ["TID251"]
+
+[lint.flake8-tidy-imports]
 
 [lint.flake8-tidy-imports.banned-api."flask_restx.reqparse"]
 msg = "Use Pydantic payload/query models instead of reqparse."

api/Dockerfile

@@ -21,9 +21,8 @@ RUN apt-get update \
           # for building gmpy2
           libmpfr-dev libmpc-dev
 
-# Install Python dependencies (workspace members under providers/vdb/)
+# Install Python dependencies
 COPY pyproject.toml uv.lock ./
-COPY providers ./providers
 RUN uv sync --locked --no-dev
 
 # production stage

api/commands/account.py

@@ -2,6 +2,7 @@ import base64
 import secrets
 
 import click
+from sqlalchemy.orm import sessionmaker
 
 from constants.languages import languages
 from extensions.ext_database import db
@@ -24,31 +25,30 @@ def reset_password(email, new_password, password_confirm):
         return
     normalized_email = email.strip().lower()
 
-    account = AccountService.get_account_by_email_with_case_fallback(email.strip())
+    with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
+        account = AccountService.get_account_by_email_with_case_fallback(email.strip(), session=session)
 
-    if not account:
-        click.echo(click.style(f"Account not found for email: {email}", fg="red"))
-        return
+        if not account:
+            click.echo(click.style(f"Account not found for email: {email}", fg="red"))
+            return
 
-    try:
-        valid_password(new_password)
-    except:
-        click.echo(click.style(f"Invalid password. Must match {password_pattern}", fg="red"))
-        return
+        try:
+            valid_password(new_password)
+        except:
+            click.echo(click.style(f"Invalid password. Must match {password_pattern}", fg="red"))
+            return
 
-    # generate password salt
-    salt = secrets.token_bytes(16)
-    base64_salt = base64.b64encode(salt).decode()
+        # generate password salt
+        salt = secrets.token_bytes(16)
+        base64_salt = base64.b64encode(salt).decode()
 
-    # encrypt password with salt
-    password_hashed = hash_password(new_password, salt)
-    base64_password_hashed = base64.b64encode(password_hashed).decode()
-    account = db.session.merge(account)
-    account.password = base64_password_hashed
-    account.password_salt = base64_salt
-    db.session.commit()
-    AccountService.reset_login_error_rate_limit(normalized_email)
-    click.echo(click.style("Password reset successfully.", fg="green"))
+        # encrypt password with salt
+        password_hashed = hash_password(new_password, salt)
+        base64_password_hashed = base64.b64encode(password_hashed).decode()
+        account.password = base64_password_hashed
+        account.password_salt = base64_salt
+        AccountService.reset_login_error_rate_limit(normalized_email)
+        click.echo(click.style("Password reset successfully.", fg="green"))
 
 
 @click.command("reset-email", help="Reset the account email.")
@@ -65,22 +65,21 @@ def reset_email(email, new_email, email_confirm):
         return
     normalized_new_email = new_email.strip().lower()
 
-    account = AccountService.get_account_by_email_with_case_fallback(email.strip())
+    with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
+        account = AccountService.get_account_by_email_with_case_fallback(email.strip(), session=session)
 
-    if not account:
-        click.echo(click.style(f"Account not found for email: {email}", fg="red"))
-        return
+        if not account:
+            click.echo(click.style(f"Account not found for email: {email}", fg="red"))
+            return
 
-    try:
-        email_validate(normalized_new_email)
-    except:
-        click.echo(click.style(f"Invalid email: {new_email}", fg="red"))
-        return
+        try:
+            email_validate(normalized_new_email)
+        except:
+            click.echo(click.style(f"Invalid email: {new_email}", fg="red"))
+            return
 
-    account = db.session.merge(account)
-    account.email = normalized_new_email
-    db.session.commit()
-    click.echo(click.style("Email updated successfully.", fg="green"))
+        account.email = normalized_new_email
+        click.echo(click.style("Email updated successfully.", fg="green"))
 
 
 @click.command("create-tenant", help="Create account and tenant.")

api/commands/vector.py

@@ -341,10 +341,11 @@ def add_qdrant_index(field: str):
             click.echo(click.style("No dataset collection bindings found.", fg="red"))
             return
         import qdrant_client
-        from dify_vdb_qdrant.qdrant_vector import PathQdrantParams, QdrantConfig
         from qdrant_client.http.exceptions import UnexpectedResponse
         from qdrant_client.http.models import PayloadSchemaType
 
+        from core.rag.datasource.vdb.qdrant.qdrant_vector import PathQdrantParams, QdrantConfig
+
         for binding in bindings:
             if dify_config.QDRANT_URL is None:
                 raise ValueError("Qdrant URL is required.")

api/configs/middleware/__init__.py

@@ -1,5 +1,5 @@
 import os
-from typing import Any, Literal, TypedDict
+from typing import Any, Literal
 from urllib.parse import parse_qsl, quote_plus
 
 from pydantic import Field, NonNegativeFloat, NonNegativeInt, PositiveFloat, PositiveInt, computed_field
@@ -107,17 +107,6 @@ class KeywordStoreConfig(BaseSettings):
     )
 
 
-class SQLAlchemyEngineOptionsDict(TypedDict):
-    pool_size: int
-    max_overflow: int
-    pool_recycle: int
-    pool_pre_ping: bool
-    connect_args: dict[str, str]
-    pool_use_lifo: bool
-    pool_reset_on_return: None
-    pool_timeout: int
-
-
 class DatabaseConfig(BaseSettings):
     # Database type selector
     DB_TYPE: Literal["postgresql", "mysql", "oceanbase", "seekdb"] = Field(
@@ -160,16 +149,6 @@ class DatabaseConfig(BaseSettings):
         default="",
     )
 
-    DB_SESSION_TIMEZONE_OVERRIDE: str = Field(
-        description=(
-            "PostgreSQL session timezone override injected via startup options."
-            " Default is 'UTC' for out-of-the-box consistency."
-            " Set to empty string to disable app-level timezone injection, for example when using RDS Proxy"
-            " together with a database-side default timezone."
-        ),
-        default="UTC",
-    )
-
     @computed_field  # type: ignore[prop-decorator]
     @property
     def SQLALCHEMY_DATABASE_URI_SCHEME(self) -> str:
@@ -230,22 +209,21 @@ class DatabaseConfig(BaseSettings):
 
     @computed_field  # type: ignore[prop-decorator]
     @property
-    def SQLALCHEMY_ENGINE_OPTIONS(self) -> SQLAlchemyEngineOptionsDict:
+    def SQLALCHEMY_ENGINE_OPTIONS(self) -> dict[str, Any]:
         # Parse DB_EXTRAS for 'options'
         db_extras_dict = dict(parse_qsl(self.DB_EXTRAS))
         options = db_extras_dict.get("options", "")
-        connect_args: dict[str, str] = {}
+        connect_args = {}
         # Use the dynamic SQLALCHEMY_DATABASE_URI_SCHEME property
         if self.SQLALCHEMY_DATABASE_URI_SCHEME.startswith("postgresql"):
-            merged_options = options.strip()
-            session_timezone_override = self.DB_SESSION_TIMEZONE_OVERRIDE.strip()
-            if session_timezone_override:
-                timezone_opt = f"-c timezone={session_timezone_override}"
-                merged_options = f"{merged_options} {timezone_opt}".strip() if merged_options else timezone_opt
-            if merged_options:
-                connect_args = {"options": merged_options}
+            timezone_opt = "-c timezone=UTC"
+            if options:
+                merged_options = f"{options} {timezone_opt}"
+            else:
+                merged_options = timezone_opt
+            connect_args = {"options": merged_options}
 
-        result: SQLAlchemyEngineOptionsDict = {
+        return {
             "pool_size": self.SQLALCHEMY_POOL_SIZE,
             "max_overflow": self.SQLALCHEMY_MAX_OVERFLOW,
             "pool_recycle": self.SQLALCHEMY_POOL_RECYCLE,
@@ -255,7 +233,6 @@ class DatabaseConfig(BaseSettings):
             "pool_reset_on_return": None,
             "pool_timeout": self.SQLALCHEMY_POOL_TIMEOUT,
         }
-        return result
 
 
 class CeleryConfig(DatabaseConfig):

api/configs/middleware/cache/redis_config.py

@@ -32,11 +32,6 @@ class RedisConfig(BaseSettings):
         default=0,
     )
 
-    REDIS_KEY_PREFIX: str = Field(
-        description="Optional global prefix for Redis keys, topics, and transport artifacts",
-        default="",
-    )
-
     REDIS_USE_SSL: bool = Field(
         description="Enable SSL/TLS for the Redis connection",
         default=False,
@@ -122,37 +117,6 @@ class RedisConfig(BaseSettings):
         default=None,
     )
 
-    REDIS_RETRY_RETRIES: NonNegativeInt = Field(
-        description="Maximum number of retries per Redis command on "
-        "transient failures (ConnectionError, TimeoutError, socket.timeout)",
-        default=3,
-    )
-
-    REDIS_RETRY_BACKOFF_BASE: PositiveFloat = Field(
-        description="Base delay in seconds for exponential backoff between retries",
-        default=1.0,
-    )
-
-    REDIS_RETRY_BACKOFF_CAP: PositiveFloat = Field(
-        description="Maximum backoff delay in seconds between retries",
-        default=10.0,
-    )
-
-    REDIS_SOCKET_TIMEOUT: PositiveFloat | None = Field(
-        description="Socket timeout in seconds for Redis read/write operations",
-        default=5.0,
-    )
-
-    REDIS_SOCKET_CONNECT_TIMEOUT: PositiveFloat | None = Field(
-        description="Socket timeout in seconds for Redis connection establishment",
-        default=5.0,
-    )
-
-    REDIS_HEALTH_CHECK_INTERVAL: NonNegativeInt = Field(
-        description="Interval in seconds between Redis connection health checks (0 to disable)",
-        default=30,
-    )
-
     @field_validator("REDIS_MAX_CONNECTIONS", mode="before")
     @classmethod
     def _empty_string_to_none_for_max_conns(cls, v):

api/configs/middleware/vdb/hologres_config.py

@@ -1,3 +1,4 @@
+from holo_search_sdk.types import BaseQuantizationType, DistanceType, TokenizerType
 from pydantic import Field
 from pydantic_settings import BaseSettings
 
@@ -41,17 +42,17 @@ class HologresConfig(BaseSettings):
         default="public",
     )
 
-    HOLOGRES_TOKENIZER: str = Field(
+    HOLOGRES_TOKENIZER: TokenizerType = Field(
         description="Tokenizer for full-text search index (e.g., 'jieba', 'ik', 'standard', 'simple').",
         default="jieba",
     )
 
-    HOLOGRES_DISTANCE_METHOD: str = Field(
+    HOLOGRES_DISTANCE_METHOD: DistanceType = Field(
         description="Distance method for vector index (e.g., 'Cosine', 'Euclidean', 'InnerProduct').",
         default="Cosine",
     )
 
-    HOLOGRES_BASE_QUANTIZATION_TYPE: str = Field(
+    HOLOGRES_BASE_QUANTIZATION_TYPE: BaseQuantizationType = Field(
         description="Base quantization type for vector index (e.g., 'rabitq', 'sq8', 'fp16', 'fp32').",
         default="rabitq",
     )

api/configs/middleware/vdb/iris_config.py

@@ -1,7 +1,5 @@
 """Configuration for InterSystems IRIS vector database."""
 
-from typing import Any
-
 from pydantic import Field, PositiveInt, model_validator
 from pydantic_settings import BaseSettings
 
@@ -66,7 +64,7 @@ class IrisVectorConfig(BaseSettings):
 
     @model_validator(mode="before")
     @classmethod
-    def validate_config(cls, values: dict[str, Any]) -> dict[str, Any]:
+    def validate_config(cls, values: dict) -> dict:
         """Validate IRIS configuration values.
 
         Args:

api/controllers/common/controller_schemas.py

@@ -1,104 +0,0 @@
-from typing import Any, Literal
-from uuid import UUID
-
-from pydantic import BaseModel, Field, model_validator
-
-from libs.helper import UUIDStrOrEmpty
-
-# --- Conversation schemas ---
-
-
-class ConversationRenamePayload(BaseModel):
-    name: str | None = None
-    auto_generate: bool = False
-
-    @model_validator(mode="after")
-    def validate_name_requirement(self):
-        if not self.auto_generate:
-            if self.name is None or not self.name.strip():
-                raise ValueError("name is required when auto_generate is false")
-        return self
-
-
-# --- Message schemas ---
-
-
-class MessageListQuery(BaseModel):
-    conversation_id: UUIDStrOrEmpty = Field(description="Conversation UUID")
-    first_id: UUIDStrOrEmpty | None = Field(default=None, description="First message ID for pagination")
-    limit: int = Field(default=20, ge=1, le=100, description="Number of messages to return (1-100)")
-
-
-class MessageFeedbackPayload(BaseModel):
-    rating: Literal["like", "dislike"] | None = None
-    content: str | None = None
-
-
-# --- Saved message schemas ---
-
-
-class SavedMessageListQuery(BaseModel):
-    last_id: UUIDStrOrEmpty | None = None
-    limit: int = Field(default=20, ge=1, le=100)
-
-
-class SavedMessageCreatePayload(BaseModel):
-    message_id: UUIDStrOrEmpty
-
-
-# --- Workflow schemas ---
-
-
-class DefaultBlockConfigQuery(BaseModel):
-    q: str | None = None
-
-
-class WorkflowListQuery(BaseModel):
-    page: int = Field(default=1, ge=1, le=99999)
-    limit: int = Field(default=10, ge=1, le=100)
-    user_id: str | None = None
-    named_only: bool = False
-
-
-class WorkflowRunPayload(BaseModel):
-    inputs: dict[str, Any]
-    files: list[dict[str, Any]] | None = None
-
-
-class WorkflowUpdatePayload(BaseModel):
-    marked_name: str | None = Field(default=None, max_length=20)
-    marked_comment: str | None = Field(default=None, max_length=100)
-
-
-# --- Dataset schemas ---
-
-
-DOCUMENT_BATCH_DOWNLOAD_ZIP_MAX_DOCS = 100
-
-
-class ChildChunkCreatePayload(BaseModel):
-    content: str
-
-
-class ChildChunkUpdatePayload(BaseModel):
-    content: str
-
-
-class DocumentBatchDownloadZipPayload(BaseModel):
-    """Request payload for bulk downloading documents as a zip archive."""
-
-    document_ids: list[UUID] = Field(..., min_length=1, max_length=DOCUMENT_BATCH_DOWNLOAD_ZIP_MAX_DOCS)
-
-
-class MetadataUpdatePayload(BaseModel):
-    name: str
-
-
-# --- Audio schemas ---
-
-
-class TextToAudioPayload(BaseModel):
-    message_id: str | None = Field(default=None, description="Message ID")
-    voice: str | None = Field(default=None, description="Voice to use for TTS")
-    text: str | None = Field(default=None, description="Text to convert to audio")
-    streaming: bool | None = Field(default=None, description="Enable streaming response")

api/controllers/console/apikey.py

@@ -1,16 +1,12 @@
-from datetime import datetime
-
 import flask_restx
-from flask_restx import Resource
+from flask_restx import Resource, fields, marshal_with
 from flask_restx._http import HTTPStatus
-from pydantic import field_validator
 from sqlalchemy import delete, func, select
 from sqlalchemy.orm import sessionmaker
 from werkzeug.exceptions import Forbidden
 
-from controllers.common.schema import register_schema_models
 from extensions.ext_database import db
-from fields.base import ResponseModel
+from libs.helper import TimestampField
 from libs.login import current_account_with_tenant, login_required
 from models.dataset import Dataset
 from models.enums import ApiTokenType
@@ -20,31 +16,21 @@ from services.api_token_service import ApiTokenCache
 from . import console_ns
 from .wraps import account_initialization_required, edit_permission_required, setup_required
 
+api_key_fields = {
+    "id": fields.String,
+    "type": fields.String,
+    "token": fields.String,
+    "last_used_at": TimestampField,
+    "created_at": TimestampField,
+}
 
-def _to_timestamp(value: datetime | int | None) -> int | None:
-    if isinstance(value, datetime):
-        return int(value.timestamp())
-    return value
+api_key_item_model = console_ns.model("ApiKeyItem", api_key_fields)
 
+api_key_list = {"data": fields.List(fields.Nested(api_key_item_model), attribute="items")}
 
-class ApiKeyItem(ResponseModel):
-    id: str
-    type: str
-    token: str
-    last_used_at: int | None = None
-    created_at: int | None = None
-
-    @field_validator("last_used_at", "created_at", mode="before")
-    @classmethod
-    def _normalize_timestamp(cls, value: datetime | int | None) -> int | None:
-        return _to_timestamp(value)
-
-
-class ApiKeyList(ResponseModel):
-    data: list[ApiKeyItem]
-
-
-register_schema_models(console_ns, ApiKeyItem, ApiKeyList)
+api_key_list_model = console_ns.model(
+    "ApiKeyList", {"data": fields.List(fields.Nested(api_key_item_model), attribute="items")}
+)
 
 
 def _get_resource(resource_id, tenant_id, resource_model):
@@ -68,6 +54,7 @@ class BaseApiKeyListResource(Resource):
     token_prefix: str | None = None
     max_keys = 10
 
+    @marshal_with(api_key_list_model)
     def get(self, resource_id):
         assert self.resource_id_field is not None, "resource_id_field must be set"
         resource_id = str(resource_id)
@@ -79,8 +66,9 @@ class BaseApiKeyListResource(Resource):
                 ApiToken.type == self.resource_type, getattr(ApiToken, self.resource_id_field) == resource_id
             )
         ).all()
-        return ApiKeyList.model_validate({"data": keys}, from_attributes=True).model_dump(mode="json")
+        return {"items": keys}
 
+    @marshal_with(api_key_item_model)
     @edit_permission_required
     def post(self, resource_id):
         assert self.resource_id_field is not None, "resource_id_field must be set"
@@ -112,7 +100,7 @@ class BaseApiKeyListResource(Resource):
         api_token.type = self.resource_type
         db.session.add(api_token)
         db.session.commit()
-        return ApiKeyItem.model_validate(api_token, from_attributes=True).model_dump(mode="json"), 201
+        return api_token, 201
 
 
 class BaseApiKeyResource(Resource):
@@ -159,7 +147,7 @@ class AppApiKeyListResource(BaseApiKeyListResource):
     @console_ns.doc("get_app_api_keys")
     @console_ns.doc(description="Get all API keys for an app")
     @console_ns.doc(params={"resource_id": "App ID"})
-    @console_ns.response(200, "API keys retrieved successfully", console_ns.models[ApiKeyList.__name__])
+    @console_ns.response(200, "Success", api_key_list_model)
     def get(self, resource_id):  # type: ignore
         """Get all API keys for an app"""
         return super().get(resource_id)
@@ -167,7 +155,7 @@ class AppApiKeyListResource(BaseApiKeyListResource):
     @console_ns.doc("create_app_api_key")
     @console_ns.doc(description="Create a new API key for an app")
     @console_ns.doc(params={"resource_id": "App ID"})
-    @console_ns.response(201, "API key created successfully", console_ns.models[ApiKeyItem.__name__])
+    @console_ns.response(201, "API key created successfully", api_key_item_model)
     @console_ns.response(400, "Maximum keys exceeded")
     def post(self, resource_id):  # type: ignore
         """Create a new API key for an app"""
@@ -199,7 +187,7 @@ class DatasetApiKeyListResource(BaseApiKeyListResource):
     @console_ns.doc("get_dataset_api_keys")
     @console_ns.doc(description="Get all API keys for a dataset")
     @console_ns.doc(params={"resource_id": "Dataset ID"})
-    @console_ns.response(200, "API keys retrieved successfully", console_ns.models[ApiKeyList.__name__])
+    @console_ns.response(200, "Success", api_key_list_model)
     def get(self, resource_id):  # type: ignore
         """Get all API keys for a dataset"""
         return super().get(resource_id)
@@ -207,7 +195,7 @@ class DatasetApiKeyListResource(BaseApiKeyListResource):
     @console_ns.doc("create_dataset_api_key")
     @console_ns.doc(description="Create a new API key for a dataset")
     @console_ns.doc(params={"resource_id": "Dataset ID"})
-    @console_ns.response(201, "API key created successfully", console_ns.models[ApiKeyItem.__name__])
+    @console_ns.response(201, "API key created successfully", api_key_item_model)
     @console_ns.response(400, "Maximum keys exceeded")
     def post(self, resource_id):  # type: ignore
         """Create a new API key for a dataset"""

api/controllers/console/app/advanced_prompt_template.py

@@ -5,7 +5,7 @@ from pydantic import BaseModel, Field
 from controllers.console import console_ns
 from controllers.console.wraps import account_initialization_required, setup_required
 from libs.login import login_required
-from services.advanced_prompt_template_service import AdvancedPromptTemplateArgs, AdvancedPromptTemplateService
+from services.advanced_prompt_template_service import AdvancedPromptTemplateService
 
 
 class AdvancedPromptTemplateQuery(BaseModel):
@@ -35,10 +35,5 @@ class AdvancedPromptTemplateList(Resource):
     @account_initialization_required
     def get(self):
         args = AdvancedPromptTemplateQuery.model_validate(request.args.to_dict(flat=True))  # type: ignore
-        prompt_args: AdvancedPromptTemplateArgs = {
-            "app_mode": args.app_mode,
-            "model_mode": args.model_mode,
-            "model_name": args.model_name,
-            "has_context": args.has_context,
-        }
-        return AdvancedPromptTemplateService.get_prompt(prompt_args)
+
+        return AdvancedPromptTemplateService.get_prompt(args.model_dump())

api/controllers/console/app/annotation.py

@@ -25,13 +25,7 @@ from fields.annotation_fields import (
 )
 from libs.helper import uuid_value
 from libs.login import login_required
-from services.annotation_service import (
-    AppAnnotationService,
-    EnableAnnotationArgs,
-    UpdateAnnotationArgs,
-    UpdateAnnotationSettingArgs,
-    UpsertAnnotationArgs,
-)
+from services.annotation_service import AppAnnotationService
 
 DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
 
@@ -126,12 +120,7 @@ class AnnotationReplyActionApi(Resource):
         args = AnnotationReplyPayload.model_validate(console_ns.payload)
         match action:
             case "enable":
-                enable_args: EnableAnnotationArgs = {
-                    "score_threshold": args.score_threshold,
-                    "embedding_provider_name": args.embedding_provider_name,
-                    "embedding_model_name": args.embedding_model_name,
-                }
-                result = AppAnnotationService.enable_app_annotation(enable_args, app_id)
+                result = AppAnnotationService.enable_app_annotation(args.model_dump(), app_id)
             case "disable":
                 result = AppAnnotationService.disable_app_annotation(app_id)
         return result, 200
@@ -172,8 +161,7 @@ class AppAnnotationSettingUpdateApi(Resource):
 
         args = AnnotationSettingUpdatePayload.model_validate(console_ns.payload)
 
-        setting_args: UpdateAnnotationSettingArgs = {"score_threshold": args.score_threshold}
-        result = AppAnnotationService.update_app_annotation_setting(app_id, annotation_setting_id, setting_args)
+        result = AppAnnotationService.update_app_annotation_setting(app_id, annotation_setting_id, args.model_dump())
         return result, 200
 
 
@@ -249,16 +237,8 @@ class AnnotationApi(Resource):
     def post(self, app_id):
         app_id = str(app_id)
         args = CreateAnnotationPayload.model_validate(console_ns.payload)
-        upsert_args: UpsertAnnotationArgs = {}
-        if args.answer is not None:
-            upsert_args["answer"] = args.answer
-        if args.content is not None:
-            upsert_args["content"] = args.content
-        if args.message_id is not None:
-            upsert_args["message_id"] = args.message_id
-        if args.question is not None:
-            upsert_args["question"] = args.question
-        annotation = AppAnnotationService.up_insert_app_annotation_from_message(upsert_args, app_id)
+        data = args.model_dump(exclude_none=True)
+        annotation = AppAnnotationService.up_insert_app_annotation_from_message(data, app_id)
         return Annotation.model_validate(annotation, from_attributes=True).model_dump(mode="json")
 
     @setup_required
@@ -335,12 +315,9 @@ class AnnotationUpdateDeleteApi(Resource):
         app_id = str(app_id)
         annotation_id = str(annotation_id)
         args = UpdateAnnotationPayload.model_validate(console_ns.payload)
-        update_args: UpdateAnnotationArgs = {}
-        if args.answer is not None:
-            update_args["answer"] = args.answer
-        if args.question is not None:
-            update_args["question"] = args.question
-        annotation = AppAnnotationService.update_app_annotation_directly(update_args, app_id, annotation_id)
+        annotation = AppAnnotationService.update_app_annotation_directly(
+            args.model_dump(exclude_none=True), app_id, annotation_id
+        )
         return Annotation.model_validate(annotation, from_attributes=True).model_dump(mode="json")
 
     @setup_required

api/controllers/console/app/app.py

@@ -7,7 +7,7 @@ from flask import request
 from flask_restx import Resource
 from graphon.enums import WorkflowExecutionStatus
 from graphon.file import helpers as file_helpers
-from pydantic import AliasChoices, BaseModel, Field, computed_field, field_validator
+from pydantic import AliasChoices, BaseModel, ConfigDict, Field, computed_field, field_validator
 from sqlalchemy import select
 from sqlalchemy.orm import sessionmaker
 from werkzeug.exceptions import BadRequest
@@ -26,25 +26,25 @@ from controllers.console.wraps import (
     setup_required,
 )
 from core.ops.ops_trace_manager import OpsTraceManager
-from core.rag.entities import PreProcessingRule, Rule, Segmentation
 from core.rag.retrieval.retrieval_methods import RetrievalMethod
 from core.trigger.constants import TRIGGER_NODE_TYPES
 from extensions.ext_database import db
-from fields.base import ResponseModel
 from libs.login import current_account_with_tenant, login_required
 from models import App, DatasetPermissionEnum, Workflow
 from models.model import IconType
-from services.app_dsl_service import AppDslService
+from services.app_dsl_service import AppDslService, ImportMode
 from services.app_service import AppService
 from services.enterprise.enterprise_service import EnterpriseService
-from services.entities.dsl_entities import ImportMode
 from services.entities.knowledge_entities.knowledge_entities import (
     DataSource,
     InfoList,
     NotionIcon,
     NotionInfo,
     NotionPage,
+    PreProcessingRule,
     RerankingModel,
+    Rule,
+    Segmentation,
     WebsiteInfo,
     WeightKeywordSetting,
     WeightModel,
@@ -155,6 +155,16 @@ class AppTracePayload(BaseModel):
 type JSONValue = Any
 
 
+class ResponseModel(BaseModel):
+    model_config = ConfigDict(
+        from_attributes=True,
+        extra="ignore",
+        populate_by_name=True,
+        serialize_by_alias=True,
+        protected_namespaces=(),
+    )
+
+
 def _to_timestamp(value: datetime | int | None) -> int | None:
     if isinstance(value, datetime):
         return int(value.timestamp())

api/controllers/console/app/app_import.py

@@ -1,8 +1,7 @@
-from flask_restx import Resource
+from flask_restx import Resource, fields, marshal_with
 from pydantic import BaseModel, Field
-from sqlalchemy.orm import sessionmaker
+from sqlalchemy.orm import Session, sessionmaker
 
-from controllers.common.schema import register_schema_models
 from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import (
     account_initialization_required,
@@ -11,15 +10,34 @@ from controllers.console.wraps import (
     setup_required,
 )
 from extensions.ext_database import db
+from fields.app_fields import (
+    app_import_check_dependencies_fields,
+    app_import_fields,
+    leaked_dependency_fields,
+)
 from libs.login import current_account_with_tenant, login_required
 from models.model import App
-from services.app_dsl_service import AppDslService, Import
+from services.app_dsl_service import AppDslService, ImportStatus
 from services.enterprise.enterprise_service import EnterpriseService
-from services.entities.dsl_entities import CheckDependenciesResult, ImportStatus
 from services.feature_service import FeatureService
 
 from .. import console_ns
 
+# Register models for flask_restx to avoid dict type issues in Swagger
+# Register base model first
+leaked_dependency_model = console_ns.model("LeakedDependency", leaked_dependency_fields)
+
+app_import_model = console_ns.model("AppImport", app_import_fields)
+
+# For nested models, need to replace nested dict with registered model
+app_import_check_dependencies_fields_copy = app_import_check_dependencies_fields.copy()
+app_import_check_dependencies_fields_copy["leaked_dependencies"] = fields.List(fields.Nested(leaked_dependency_model))
+app_import_check_dependencies_model = console_ns.model(
+    "AppImportCheckDependencies", app_import_check_dependencies_fields_copy
+)
+
+DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
+
 
 class AppImportPayload(BaseModel):
     mode: str = Field(..., description="Import mode")
@@ -33,18 +51,18 @@ class AppImportPayload(BaseModel):
     app_id: str | None = Field(None)
 
 
-register_schema_models(console_ns, AppImportPayload, Import, CheckDependenciesResult)
+console_ns.schema_model(
+    AppImportPayload.__name__, AppImportPayload.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0)
+)
 
 
 @console_ns.route("/apps/imports")
 class AppImportApi(Resource):
     @console_ns.expect(console_ns.models[AppImportPayload.__name__])
-    @console_ns.response(200, "Import completed", console_ns.models[Import.__name__])
-    @console_ns.response(202, "Import pending confirmation", console_ns.models[Import.__name__])
-    @console_ns.response(400, "Import failed", console_ns.models[Import.__name__])
     @setup_required
     @login_required
     @account_initialization_required
+    @marshal_with(app_import_model)
     @cloud_edition_billing_resource_check("apps")
     @edit_permission_required
     def post(self):
@@ -53,7 +71,7 @@ class AppImportApi(Resource):
         args = AppImportPayload.model_validate(console_ns.payload)
 
         # Create service with session
-        with sessionmaker(db.engine).begin() as session:
+        with Session(db.engine) as session:
             import_service = AppDslService(session)
             # Import app
             account = current_user
@@ -74,22 +92,19 @@ class AppImportApi(Resource):
             EnterpriseService.WebAppAuth.update_app_access_mode(result.app_id, "private")
         # Return appropriate status code based on result
         status = result.status
-        match status:
-            case ImportStatus.FAILED:
-                return result.model_dump(mode="json"), 400
-            case ImportStatus.PENDING:
-                return result.model_dump(mode="json"), 202
-            case ImportStatus.COMPLETED | ImportStatus.COMPLETED_WITH_WARNINGS:
-                return result.model_dump(mode="json"), 200
+        if status == ImportStatus.FAILED:
+            return result.model_dump(mode="json"), 400
+        elif status == ImportStatus.PENDING:
+            return result.model_dump(mode="json"), 202
+        return result.model_dump(mode="json"), 200
 
 
 @console_ns.route("/apps/imports/<string:import_id>/confirm")
 class AppImportConfirmApi(Resource):
-    @console_ns.response(200, "Import confirmed", console_ns.models[Import.__name__])
-    @console_ns.response(400, "Import failed", console_ns.models[Import.__name__])
     @setup_required
     @login_required
     @account_initialization_required
+    @marshal_with(app_import_model)
     @edit_permission_required
     def post(self, import_id):
         # Check user role first
@@ -110,11 +125,11 @@ class AppImportConfirmApi(Resource):
 
 @console_ns.route("/apps/imports/<string:app_id>/check-dependencies")
 class AppImportCheckDependenciesApi(Resource):
-    @console_ns.response(200, "Dependencies checked", console_ns.models[CheckDependenciesResult.__name__])
     @setup_required
     @login_required
     @get_app_model
     @account_initialization_required
+    @marshal_with(app_import_check_dependencies_model)
     @edit_permission_required
     def get(self, app_model: App):
         with sessionmaker(db.engine).begin() as session:

api/controllers/console/app/mcp_server.py

@@ -1,68 +1,39 @@
 import json
-from datetime import datetime
-from typing import Any
 
-from flask_restx import Resource
-from pydantic import BaseModel, Field, field_validator
+from flask_restx import Resource, marshal_with
+from pydantic import BaseModel, Field
 from sqlalchemy import select
 from werkzeug.exceptions import NotFound
 
-from controllers.common.schema import register_schema_models
 from controllers.console import console_ns
 from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required
 from extensions.ext_database import db
-from fields.base import ResponseModel
+from fields.app_fields import app_server_fields
 from libs.login import current_account_with_tenant, login_required
 from models.enums import AppMCPServerStatus
 from models.model import AppMCPServer
 
+DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
 
-def _to_timestamp(value: datetime | int | None) -> int | None:
-    if isinstance(value, datetime):
-        return int(value.timestamp())
-    return value
+# Register model for flask_restx to avoid dict type issues in Swagger
+app_server_model = console_ns.model("AppServer", app_server_fields)
 
 
 class MCPServerCreatePayload(BaseModel):
     description: str | None = Field(default=None, description="Server description")
-    parameters: dict[str, Any] = Field(..., description="Server parameters configuration")
+    parameters: dict = Field(..., description="Server parameters configuration")
 
 
 class MCPServerUpdatePayload(BaseModel):
     id: str = Field(..., description="Server ID")
     description: str | None = Field(default=None, description="Server description")
-    parameters: dict[str, Any] = Field(..., description="Server parameters configuration")
+    parameters: dict = Field(..., description="Server parameters configuration")
     status: str | None = Field(default=None, description="Server status")
 
 
-class AppMCPServerResponse(ResponseModel):
-    id: str
-    name: str
-    server_code: str
-    description: str
-    status: str
-    parameters: dict[str, Any] | list[Any] | str
-    created_at: int | None = None
-    updated_at: int | None = None
-
-    @field_validator("parameters", mode="before")
-    @classmethod
-    def _parse_json_string(cls, value: Any) -> Any:
-        if isinstance(value, str):
-            try:
-                return json.loads(value)
-            except (json.JSONDecodeError, TypeError):
-                return value
-        return value
-
-    @field_validator("created_at", "updated_at", mode="before")
-    @classmethod
-    def _normalize_timestamp(cls, value: datetime | int | None) -> int | None:
-        return _to_timestamp(value)
-
-
-register_schema_models(console_ns, MCPServerCreatePayload, MCPServerUpdatePayload, AppMCPServerResponse)
+for model in (MCPServerCreatePayload, MCPServerUpdatePayload):
+    console_ns.schema_model(model.__name__, model.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0))
 
 
 @console_ns.route("/apps/<uuid:app_id>/server")
@@ -70,27 +41,27 @@ class AppMCPServerController(Resource):
     @console_ns.doc("get_app_mcp_server")
     @console_ns.doc(description="Get MCP server configuration for an application")
     @console_ns.doc(params={"app_id": "Application ID"})
-    @console_ns.response(200, "Server configuration", console_ns.models[AppMCPServerResponse.__name__])
+    @console_ns.response(200, "MCP server configuration retrieved successfully", app_server_model)
     @login_required
     @account_initialization_required
     @setup_required
     @get_app_model
+    @marshal_with(app_server_model)
     def get(self, app_model):
         server = db.session.scalar(select(AppMCPServer).where(AppMCPServer.app_id == app_model.id).limit(1))
-        if server is None:
-            return {}
-        return AppMCPServerResponse.model_validate(server, from_attributes=True).model_dump(mode="json")
+        return server
 
     @console_ns.doc("create_app_mcp_server")
     @console_ns.doc(description="Create MCP server configuration for an application")
     @console_ns.doc(params={"app_id": "Application ID"})
     @console_ns.expect(console_ns.models[MCPServerCreatePayload.__name__])
-    @console_ns.response(200, "Server created", console_ns.models[AppMCPServerResponse.__name__])
+    @console_ns.response(201, "MCP server configuration created successfully", app_server_model)
     @console_ns.response(403, "Insufficient permissions")
     @account_initialization_required
     @get_app_model
     @login_required
     @setup_required
+    @marshal_with(app_server_model)
     @edit_permission_required
     def post(self, app_model):
         _, current_tenant_id = current_account_with_tenant()
@@ -111,19 +82,20 @@ class AppMCPServerController(Resource):
         )
         db.session.add(server)
         db.session.commit()
-        return AppMCPServerResponse.model_validate(server, from_attributes=True).model_dump(mode="json")
+        return server
 
     @console_ns.doc("update_app_mcp_server")
     @console_ns.doc(description="Update MCP server configuration for an application")
     @console_ns.doc(params={"app_id": "Application ID"})
     @console_ns.expect(console_ns.models[MCPServerUpdatePayload.__name__])
-    @console_ns.response(200, "Server updated", console_ns.models[AppMCPServerResponse.__name__])
+    @console_ns.response(200, "MCP server configuration updated successfully", app_server_model)
     @console_ns.response(403, "Insufficient permissions")
     @console_ns.response(404, "Server not found")
     @get_app_model
     @login_required
     @setup_required
     @account_initialization_required
+    @marshal_with(app_server_model)
     @edit_permission_required
     def put(self, app_model):
         payload = MCPServerUpdatePayload.model_validate(console_ns.payload or {})
@@ -146,7 +118,7 @@ class AppMCPServerController(Resource):
             except ValueError:
                 raise ValueError("Invalid status")
         db.session.commit()
-        return AppMCPServerResponse.model_validate(server, from_attributes=True).model_dump(mode="json")
+        return server
 
 
 @console_ns.route("/apps/<uuid:server_id>/server/refresh")
@@ -154,12 +126,13 @@ class AppMCPServerRefreshController(Resource):
     @console_ns.doc("refresh_app_mcp_server")
     @console_ns.doc(description="Refresh MCP server configuration and regenerate server code")
     @console_ns.doc(params={"server_id": "Server ID"})
-    @console_ns.response(200, "Server refreshed", console_ns.models[AppMCPServerResponse.__name__])
+    @console_ns.response(200, "MCP server refreshed successfully", app_server_model)
     @console_ns.response(403, "Insufficient permissions")
     @console_ns.response(404, "Server not found")
     @setup_required
     @login_required
     @account_initialization_required
+    @marshal_with(app_server_model)
     @edit_permission_required
     def get(self, server_id):
         _, current_tenant_id = current_account_with_tenant()
@@ -172,4 +145,4 @@ class AppMCPServerRefreshController(Resource):
             raise NotFound()
         server.server_code = AppMCPServer.generate_server_code(16)
         db.session.commit()
-        return AppMCPServerResponse.model_validate(server, from_attributes=True).model_dump(mode="json")
+        return server

api/controllers/console/app/message.py

@@ -8,7 +8,6 @@ from pydantic import BaseModel, Field, field_validator
 from sqlalchemy import exists, func, select
 from werkzeug.exceptions import InternalServerError, NotFound
 
-from controllers.common.controller_schemas import MessageFeedbackPayload as _MessageFeedbackPayloadBase
 from controllers.common.schema import register_schema_models
 from controllers.console import console_ns
 from controllers.console.app.error import (
@@ -60,8 +59,10 @@ class ChatMessagesQuery(BaseModel):
         return uuid_value(value)
 
 
-class MessageFeedbackPayload(_MessageFeedbackPayloadBase):
+class MessageFeedbackPayload(BaseModel):
     message_id: str = Field(..., description="Message ID")
+    rating: Literal["like", "dislike"] | None = Field(default=None, description="Feedback rating")
+    content: str | None = Field(default=None, description="Feedback content")
 
     @field_validator("message_id")
     @classmethod

api/controllers/console/app/model_config.py

@@ -1,11 +1,9 @@
 import json
-from typing import Any, cast
+from typing import cast
 
 from flask import request
-from flask_restx import Resource
-from pydantic import BaseModel, Field
+from flask_restx import Resource, fields
 
-from controllers.common.schema import register_schema_models
 from controllers.console import console_ns
 from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required
@@ -20,30 +18,30 @@ from models.model import AppMode, AppModelConfig
 from services.app_model_config_service import AppModelConfigService
 
 
-class ModelConfigRequest(BaseModel):
-    provider: str | None = Field(default=None, description="Model provider")
-    model: str | None = Field(default=None, description="Model name")
-    configs: dict[str, Any] | None = Field(default=None, description="Model configuration parameters")
-    opening_statement: str | None = Field(default=None, description="Opening statement")
-    suggested_questions: list[str] | None = Field(default=None, description="Suggested questions")
-    more_like_this: dict[str, Any] | None = Field(default=None, description="More like this configuration")
-    speech_to_text: dict[str, Any] | None = Field(default=None, description="Speech to text configuration")
-    text_to_speech: dict[str, Any] | None = Field(default=None, description="Text to speech configuration")
-    retrieval_model: dict[str, Any] | None = Field(default=None, description="Retrieval model configuration")
-    tools: list[dict[str, Any]] | None = Field(default=None, description="Available tools")
-    dataset_configs: dict[str, Any] | None = Field(default=None, description="Dataset configurations")
-    agent_mode: dict[str, Any] | None = Field(default=None, description="Agent mode configuration")
-
-
-register_schema_models(console_ns, ModelConfigRequest)
-
-
 @console_ns.route("/apps/<uuid:app_id>/model-config")
 class ModelConfigResource(Resource):
     @console_ns.doc("update_app_model_config")
     @console_ns.doc(description="Update application model configuration")
     @console_ns.doc(params={"app_id": "Application ID"})
-    @console_ns.expect(console_ns.models[ModelConfigRequest.__name__])
+    @console_ns.expect(
+        console_ns.model(
+            "ModelConfigRequest",
+            {
+                "provider": fields.String(description="Model provider"),
+                "model": fields.String(description="Model name"),
+                "configs": fields.Raw(description="Model configuration parameters"),
+                "opening_statement": fields.String(description="Opening statement"),
+                "suggested_questions": fields.List(fields.String(), description="Suggested questions"),
+                "more_like_this": fields.Raw(description="More like this configuration"),
+                "speech_to_text": fields.Raw(description="Speech to text configuration"),
+                "text_to_speech": fields.Raw(description="Text to speech configuration"),
+                "retrieval_model": fields.Raw(description="Retrieval model configuration"),
+                "tools": fields.List(fields.Raw(), description="Available tools"),
+                "dataset_configs": fields.Raw(description="Dataset configurations"),
+                "agent_mode": fields.Raw(description="Agent mode configuration"),
+            },
+        )
+    )
     @console_ns.response(200, "Model configuration updated successfully")
     @console_ns.response(400, "Invalid configuration")
     @console_ns.response(404, "App not found")

api/controllers/console/app/site.py

@@ -1,12 +1,11 @@
 from typing import Literal
 
-from flask_restx import Resource
+from flask_restx import Resource, marshal_with
 from pydantic import BaseModel, Field, field_validator
 from sqlalchemy import select
 from werkzeug.exceptions import NotFound
 
 from constants.languages import supported_language
-from controllers.common.schema import register_schema_models
 from controllers.console import console_ns
 from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import (
@@ -16,11 +15,13 @@ from controllers.console.wraps import (
     setup_required,
 )
 from extensions.ext_database import db
-from fields.base import ResponseModel
+from fields.app_fields import app_site_fields
 from libs.datetime_utils import naive_utc_now
 from libs.login import current_account_with_tenant, login_required
 from models import Site
 
+DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
+
 
 class AppSiteUpdatePayload(BaseModel):
     title: str | None = Field(default=None)
@@ -48,26 +49,13 @@ class AppSiteUpdatePayload(BaseModel):
         return supported_language(value)
 
 
-class AppSiteResponse(ResponseModel):
-    app_id: str
-    access_token: str | None = Field(default=None, validation_alias="code")
-    code: str | None = None
-    title: str
-    icon: str | None = None
-    icon_background: str | None = None
-    description: str | None = None
-    default_language: str
-    customize_domain: str | None = None
-    copyright: str | None = None
-    privacy_policy: str | None = None
-    custom_disclaimer: str | None = None
-    customize_token_strategy: str
-    prompt_public: bool
-    show_workflow_steps: bool
-    use_icon_as_answer_icon: bool
+console_ns.schema_model(
+    AppSiteUpdatePayload.__name__,
+    AppSiteUpdatePayload.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0),
+)
 
-
-register_schema_models(console_ns, AppSiteUpdatePayload, AppSiteResponse)
+# Register model for flask_restx to avoid dict type issues in Swagger
+app_site_model = console_ns.model("AppSite", app_site_fields)
 
 
 @console_ns.route("/apps/<uuid:app_id>/site")
@@ -76,7 +64,7 @@ class AppSite(Resource):
     @console_ns.doc(description="Update application site configuration")
     @console_ns.doc(params={"app_id": "Application ID"})
     @console_ns.expect(console_ns.models[AppSiteUpdatePayload.__name__])
-    @console_ns.response(200, "Site configuration updated successfully", console_ns.models[AppSiteResponse.__name__])
+    @console_ns.response(200, "Site configuration updated successfully", app_site_model)
     @console_ns.response(403, "Insufficient permissions")
     @console_ns.response(404, "App not found")
     @setup_required
@@ -84,6 +72,7 @@ class AppSite(Resource):
     @edit_permission_required
     @account_initialization_required
     @get_app_model
+    @marshal_with(app_site_model)
     def post(self, app_model):
         args = AppSiteUpdatePayload.model_validate(console_ns.payload or {})
         current_user, _ = current_account_with_tenant()
@@ -117,7 +106,7 @@ class AppSite(Resource):
         site.updated_at = naive_utc_now()
         db.session.commit()
 
-        return AppSiteResponse.model_validate(site, from_attributes=True).model_dump(mode="json")
+        return site
 
 
 @console_ns.route("/apps/<uuid:app_id>/site/access-token-reset")
@@ -125,7 +114,7 @@ class AppSiteAccessTokenReset(Resource):
     @console_ns.doc("reset_app_site_access_token")
     @console_ns.doc(description="Reset access token for application site")
     @console_ns.doc(params={"app_id": "Application ID"})
-    @console_ns.response(200, "Access token reset successfully", console_ns.models[AppSiteResponse.__name__])
+    @console_ns.response(200, "Access token reset successfully", app_site_model)
     @console_ns.response(403, "Insufficient permissions (admin/owner required)")
     @console_ns.response(404, "App or site not found")
     @setup_required
@@ -133,6 +122,7 @@ class AppSiteAccessTokenReset(Resource):
     @is_admin_or_owner_required
     @account_initialization_required
     @get_app_model
+    @marshal_with(app_site_model)
     def post(self, app_model):
         current_user, _ = current_account_with_tenant()
         site = db.session.scalar(select(Site).where(Site.app_id == app_model.id).limit(1))
@@ -145,4 +135,4 @@ class AppSiteAccessTokenReset(Resource):
         site.updated_at = naive_utc_now()
         db.session.commit()
 
-        return AppSiteResponse.model_validate(site, from_attributes=True).model_dump(mode="json")
+        return site

api/controllers/console/app/workflow.py

@@ -14,7 +14,6 @@ from sqlalchemy.orm import sessionmaker
 from werkzeug.exceptions import BadRequest, Forbidden, InternalServerError, NotFound
 
 import services
-from controllers.common.controller_schemas import DefaultBlockConfigQuery, WorkflowListQuery, WorkflowUpdatePayload
 from controllers.console import console_ns
 from controllers.console.app.error import ConversationCompletedError, DraftWorkflowNotExist, DraftWorkflowNotSync
 from controllers.console.app.workflow_run import workflow_run_node_execution_model
@@ -143,6 +142,10 @@ class PublishWorkflowPayload(BaseModel):
     marked_comment: str | None = Field(default=None, max_length=100)
 
 
+class DefaultBlockConfigQuery(BaseModel):
+    q: str | None = None
+
+
 class ConvertToWorkflowPayload(BaseModel):
     name: str | None = None
     icon_type: str | None = None
@@ -150,6 +153,18 @@ class ConvertToWorkflowPayload(BaseModel):
     icon_background: str | None = None
 
 
+class WorkflowListQuery(BaseModel):
+    page: int = Field(default=1, ge=1, le=99999)
+    limit: int = Field(default=10, ge=1, le=100)
+    user_id: str | None = None
+    named_only: bool = False
+
+
+class WorkflowUpdatePayload(BaseModel):
+    marked_name: str | None = Field(default=None, max_length=20)
+    marked_comment: str | None = Field(default=None, max_length=100)
+
+
 class DraftWorkflowTriggerRunPayload(BaseModel):
     node_id: str
 

api/controllers/console/app/workflow_app_log.py

@@ -87,7 +87,7 @@ class WorkflowAppLogApi(Resource):
 
         # get paginate workflow app logs
         workflow_app_service = WorkflowAppService()
-        with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
+        with sessionmaker(db.engine).begin() as session:
             workflow_app_log_pagination = workflow_app_service.get_paginate_workflow_app_logs(
                 session=session,
                 app_model=app_model,
@@ -124,7 +124,7 @@ class WorkflowArchivedLogApi(Resource):
         args = WorkflowAppLogQuery.model_validate(request.args.to_dict(flat=True))  # type: ignore
 
         workflow_app_service = WorkflowAppService()
-        with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
+        with sessionmaker(db.engine).begin() as session:
             workflow_app_log_pagination = workflow_app_service.get_paginate_workflow_archive_logs(
                 session=session,
                 app_model=app_model,

api/controllers/console/app/workflow_draft_variable.py

@@ -1,7 +1,7 @@
 import logging
 from collections.abc import Callable
 from functools import wraps
-from typing import Any, TypedDict
+from typing import Any
 
 from flask import Response, request
 from flask_restx import Resource, fields, marshal, marshal_with
@@ -86,14 +86,7 @@ def _serialize_variable_type(workflow_draft_var: WorkflowDraftVariable) -> str:
     return value_type.exposed_type().value
 
 
-class FullContentDict(TypedDict):
-    size_bytes: int | None
-    value_type: str
-    length: int | None
-    download_url: str
-
-
-def _serialize_full_content(variable: WorkflowDraftVariable) -> FullContentDict | None:
+def _serialize_full_content(variable: WorkflowDraftVariable) -> dict | None:
     """Serialize full_content information for large variables."""
     if not variable.is_truncated():
         return None
@@ -101,13 +94,12 @@ def _serialize_full_content(variable: WorkflowDraftVariable) -> FullContentDict
     variable_file = variable.variable_file
     assert variable_file is not None
 
-    result: FullContentDict = {
+    return {
         "size_bytes": variable_file.size,
         "value_type": variable_file.value_type.exposed_type().value,
         "length": variable_file.length,
         "download_url": file_helpers.get_signed_file_url(variable_file.upload_file_id, as_attachment=True),
     }
-    return result
 
 
 def _ensure_variable_access(
@@ -392,27 +384,24 @@ class VariableApi(Resource):
 
         new_value = None
         if raw_value is not None:
-            match variable.value_type:
-                case SegmentType.FILE:
-                    if not isinstance(raw_value, dict):
-                        raise InvalidArgumentError(description=f"expected dict for file, got {type(raw_value)}")
-                    raw_value = build_from_mapping(
-                        mapping=raw_value,
-                        tenant_id=app_model.tenant_id,
-                        access_controller=_file_access_controller,
-                    )
-                case SegmentType.ARRAY_FILE:
-                    if not isinstance(raw_value, list):
-                        raise InvalidArgumentError(description=f"expected list for files, got {type(raw_value)}")
-                    if len(raw_value) > 0 and not isinstance(raw_value[0], dict):
-                        raise InvalidArgumentError(description=f"expected dict for files[0], got {type(raw_value)}")
-                    raw_value = build_from_mappings(
-                        mappings=raw_value,
-                        tenant_id=app_model.tenant_id,
-                        access_controller=_file_access_controller,
-                    )
-                case _:
-                    pass
+            if variable.value_type == SegmentType.FILE:
+                if not isinstance(raw_value, dict):
+                    raise InvalidArgumentError(description=f"expected dict for file, got {type(raw_value)}")
+                raw_value = build_from_mapping(
+                    mapping=raw_value,
+                    tenant_id=app_model.tenant_id,
+                    access_controller=_file_access_controller,
+                )
+            elif variable.value_type == SegmentType.ARRAY_FILE:
+                if not isinstance(raw_value, list):
+                    raise InvalidArgumentError(description=f"expected list for files, got {type(raw_value)}")
+                if len(raw_value) > 0 and not isinstance(raw_value[0], dict):
+                    raise InvalidArgumentError(description=f"expected dict for files[0], got {type(raw_value)}")
+                raw_value = build_from_mappings(
+                    mappings=raw_value,
+                    tenant_id=app_model.tenant_id,
+                    access_controller=_file_access_controller,
+                )
             new_value = build_segment_with_type(variable.value_type, raw_value)
         draft_var_srv.update_variable(variable, name=new_name, value=new_value)
         db.session.commit()

api/controllers/console/app/workflow_run.py

@@ -36,7 +36,7 @@ from models import Account, App, AppMode, EndUser, WorkflowArchiveLog, WorkflowR
 from models.workflow import WorkflowRun
 from repositories.factory import DifyAPIRepositoryFactory
 from services.retention.workflow_run.constants import ARCHIVE_BUNDLE_NAME
-from services.workflow_run_service import WorkflowRunListArgs, WorkflowRunService
+from services.workflow_run_service import WorkflowRunService
 
 
 def _build_backstage_input_url(form_token: str | None) -> str | None:
@@ -214,11 +214,7 @@ class AdvancedChatAppWorkflowRunListApi(Resource):
         Get advanced chat app workflow run list
         """
         args_model = WorkflowRunListQuery.model_validate(request.args.to_dict(flat=True))  # type: ignore
-        args: WorkflowRunListArgs = {"limit": args_model.limit}
-        if args_model.last_id is not None:
-            args["last_id"] = args_model.last_id
-        if args_model.status is not None:
-            args["status"] = args_model.status
+        args = args_model.model_dump(exclude_none=True)
 
         # Default to DEBUGGING if not specified
         triggered_from = (
@@ -360,11 +356,7 @@ class WorkflowRunListApi(Resource):
         Get workflow run list
         """
         args_model = WorkflowRunListQuery.model_validate(request.args.to_dict(flat=True))  # type: ignore
-        args: WorkflowRunListArgs = {"limit": args_model.limit}
-        if args_model.last_id is not None:
-            args["last_id"] = args_model.last_id
-        if args_model.status is not None:
-            args["status"] = args_model.status
+        args = args_model.model_dump(exclude_none=True)
 
         # Default to DEBUGGING for workflow if not specified (backward compatibility)
         triggered_from = (

api/controllers/console/app/workflow_trigger.py

@@ -64,7 +64,7 @@ class WebhookTriggerApi(Resource):
 
         node_id = args.node_id
 
-        with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
+        with sessionmaker(db.engine).begin() as session:
             # Get webhook trigger for this app and node
             webhook_trigger = session.scalar(
                 select(WorkflowWebhookTrigger)
@@ -95,7 +95,7 @@ class AppTriggersApi(Resource):
         assert isinstance(current_user, Account)
         assert current_user.current_tenant_id is not None
 
-        with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
+        with sessionmaker(db.engine).begin() as session:
             # Get all triggers for this app using select API
             triggers = (
                 session.execute(

api/controllers/console/auth/activate.py

@@ -1,9 +1,8 @@
 from flask import request
-from flask_restx import Resource
+from flask_restx import Resource, fields
 from pydantic import BaseModel, Field, field_validator
 
 from constants.languages import supported_language
-from controllers.common.schema import register_schema_models
 from controllers.console import console_ns
 from controllers.console.error import AlreadyActivateError
 from extensions.ext_database import db
@@ -12,6 +11,8 @@ from libs.helper import EmailStr, timezone
 from models import AccountStatus
 from services.account_service import RegisterService
 
+DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
+
 
 class ActivateCheckQuery(BaseModel):
     workspace_id: str | None = Field(default=None)
@@ -38,16 +39,8 @@ class ActivatePayload(BaseModel):
         return timezone(value)
 
 
-class ActivationCheckResponse(BaseModel):
-    is_valid: bool = Field(description="Whether token is valid")
-    data: dict | None = Field(default=None, description="Activation data if valid")
-
-
-class ActivationResponse(BaseModel):
-    result: str = Field(description="Operation result")
-
-
-register_schema_models(console_ns, ActivateCheckQuery, ActivatePayload, ActivationCheckResponse, ActivationResponse)
+for model in (ActivateCheckQuery, ActivatePayload):
+    console_ns.schema_model(model.__name__, model.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0))
 
 
 @console_ns.route("/activate/check")
@@ -58,7 +51,13 @@ class ActivateCheckApi(Resource):
     @console_ns.response(
         200,
         "Success",
-        console_ns.models[ActivationCheckResponse.__name__],
+        console_ns.model(
+            "ActivationCheckResponse",
+            {
+                "is_valid": fields.Boolean(description="Whether token is valid"),
+                "data": fields.Raw(description="Activation data if valid"),
+            },
+        ),
     )
     def get(self):
         args = ActivateCheckQuery.model_validate(request.args.to_dict(flat=True))  # type: ignore
@@ -96,7 +95,12 @@ class ActivateApi(Resource):
     @console_ns.response(
         200,
         "Account activated successfully",
-        console_ns.models[ActivationResponse.__name__],
+        console_ns.model(
+            "ActivationResponse",
+            {
+                "result": fields.String(description="Operation result"),
+            },
+        ),
     )
     @console_ns.response(400, "Already activated or invalid token")
     def post(self):

api/controllers/console/auth/email_register.py

@@ -1,6 +1,7 @@
 from flask import request
 from flask_restx import Resource
 from pydantic import BaseModel, Field, field_validator
+from sqlalchemy.orm import sessionmaker
 
 from configs import dify_config
 from constants.languages import languages
@@ -13,6 +14,7 @@ from controllers.console.auth.error import (
     InvalidTokenError,
     PasswordMismatchError,
 )
+from extensions.ext_database import db
 from libs.helper import EmailStr, extract_remote_ip
 from libs.password import valid_password
 from models import Account
@@ -71,7 +73,8 @@ class EmailRegisterSendEmailApi(Resource):
         if dify_config.BILLING_ENABLED and BillingService.is_email_in_freeze(normalized_email):
             raise AccountInFreezeError()
 
-        account = AccountService.get_account_by_email_with_case_fallback(args.email)
+        with sessionmaker(db.engine).begin() as session:
+            account = AccountService.get_account_by_email_with_case_fallback(args.email, session=session)
         token = AccountService.send_email_register_email(email=normalized_email, account=account, language=language)
         return {"result": "success", "data": token}
 
@@ -142,16 +145,17 @@ class EmailRegisterResetApi(Resource):
         email = register_data.get("email", "")
         normalized_email = email.lower()
 
-        account = AccountService.get_account_by_email_with_case_fallback(email)
+        with sessionmaker(db.engine).begin() as session:
+            account = AccountService.get_account_by_email_with_case_fallback(email, session=session)
 
-        if account:
-            raise EmailAlreadyInUseError()
-        else:
-            account = self._create_new_account(normalized_email, args.password_confirm)
-            if not account:
-                raise AccountNotFoundError()
-            token_pair = AccountService.login(account=account, ip_address=extract_remote_ip(request))
-            AccountService.reset_login_error_rate_limit(normalized_email)
+            if account:
+                raise EmailAlreadyInUseError()
+            else:
+                account = self._create_new_account(normalized_email, args.password_confirm)
+                if not account:
+                    raise AccountNotFoundError()
+                token_pair = AccountService.login(account=account, ip_address=extract_remote_ip(request))
+                AccountService.reset_login_error_rate_limit(normalized_email)
 
         return {"result": "success", "data": token_pair.model_dump()}
 

api/controllers/console/auth/forgot_password.py

@@ -3,7 +3,8 @@ import secrets
 
 from flask import request
 from flask_restx import Resource
-from pydantic import BaseModel, Field
+from pydantic import BaseModel, Field, field_validator
+from sqlalchemy.orm import sessionmaker
 
 from controllers.common.schema import register_schema_models
 from controllers.console import console_ns
@@ -19,18 +20,35 @@ from controllers.console.wraps import email_password_login_enabled, setup_requir
 from events.tenant_event import tenant_was_created
 from extensions.ext_database import db
 from libs.helper import EmailStr, extract_remote_ip
-from libs.password import hash_password
+from libs.password import hash_password, valid_password
 from services.account_service import AccountService, TenantService
-from services.entities.auth_entities import (
-    ForgotPasswordCheckPayload,
-    ForgotPasswordResetPayload,
-    ForgotPasswordSendPayload,
-)
 from services.feature_service import FeatureService
 
 DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
 
 
+class ForgotPasswordSendPayload(BaseModel):
+    email: EmailStr = Field(...)
+    language: str | None = Field(default=None)
+
+
+class ForgotPasswordCheckPayload(BaseModel):
+    email: EmailStr = Field(...)
+    code: str = Field(...)
+    token: str = Field(...)
+
+
+class ForgotPasswordResetPayload(BaseModel):
+    token: str = Field(...)
+    new_password: str = Field(...)
+    password_confirm: str = Field(...)
+
+    @field_validator("new_password", "password_confirm")
+    @classmethod
+    def validate_password(cls, value: str) -> str:
+        return valid_password(value)
+
+
 class ForgotPasswordEmailResponse(BaseModel):
     result: str = Field(description="Operation result")
     data: str | None = Field(default=None, description="Reset token")
@@ -84,7 +102,8 @@ class ForgotPasswordSendEmailApi(Resource):
         else:
             language = "en-US"
 
-        account = AccountService.get_account_by_email_with_case_fallback(args.email)
+        with sessionmaker(db.engine).begin() as session:
+            account = AccountService.get_account_by_email_with_case_fallback(args.email, session=session)
 
         token = AccountService.send_reset_password_email(
             account=account,
@@ -182,18 +201,17 @@ class ForgotPasswordResetApi(Resource):
         password_hashed = hash_password(args.new_password, salt)
 
         email = reset_data.get("email", "")
-        account = AccountService.get_account_by_email_with_case_fallback(email)
+        with sessionmaker(db.engine).begin() as session:
+            account = AccountService.get_account_by_email_with_case_fallback(email, session=session)
 
-        if account:
-            account = db.session.merge(account)
-            self._update_existing_account(account, password_hashed, salt)
-            db.session.commit()
-        else:
-            raise AccountNotFound()
+            if account:
+                self._update_existing_account(account, password_hashed, salt, session)
+            else:
+                raise AccountNotFound()
 
         return {"result": "success"}
 
-    def _update_existing_account(self, account, password_hashed, salt):
+    def _update_existing_account(self, account, password_hashed, salt, session):
         # Update existing account credentials
         account.password = base64.b64encode(password_hashed).decode()
         account.password_salt = base64.b64encode(salt).decode()

api/controllers/console/auth/login.py

@@ -1,10 +1,7 @@
-import logging
-
 import flask_login
 from flask import make_response, request
 from flask_restx import Resource
 from pydantic import BaseModel, Field
-from werkzeug.exceptions import Unauthorized
 
 import services
 from configs import dify_config
@@ -45,16 +42,16 @@ from libs.token import (
 )
 from services.account_service import AccountService, InvitationDetailDict, RegisterService, TenantService
 from services.billing_service import BillingService
-from services.entities.auth_entities import LoginFailureReason, LoginPayloadBase
 from services.errors.account import AccountRegisterError
 from services.errors.workspace import WorkSpaceNotAllowedCreateError, WorkspacesLimitExceededError
 from services.feature_service import FeatureService
 
 DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
-logger = logging.getLogger(__name__)
 
 
-class LoginPayload(LoginPayloadBase):
+class LoginPayload(BaseModel):
+    email: EmailStr = Field(..., description="Email address")
+    password: str = Field(..., description="Password")
     remember_me: bool = Field(default=False, description="Remember me flag")
     invite_token: str | None = Field(default=None, description="Invitation token")
 
@@ -95,12 +92,10 @@ class LoginApi(Resource):
         normalized_email = request_email.lower()
 
         if dify_config.BILLING_ENABLED and BillingService.is_email_in_freeze(normalized_email):
-            _log_console_login_failure(email=normalized_email, reason=LoginFailureReason.ACCOUNT_IN_FREEZE)
             raise AccountInFreezeError()
 
         is_login_error_rate_limit = AccountService.is_login_error_rate_limit(normalized_email)
         if is_login_error_rate_limit:
-            _log_console_login_failure(email=normalized_email, reason=LoginFailureReason.LOGIN_RATE_LIMITED)
             raise EmailPasswordLoginLimitError()
 
         invite_token = args.invite_token
@@ -116,20 +111,14 @@ class LoginApi(Resource):
                 invitee_email = data.get("email") if data else None
                 invitee_email_normalized = invitee_email.lower() if isinstance(invitee_email, str) else invitee_email
                 if invitee_email_normalized != normalized_email:
-                    _log_console_login_failure(
-                        email=normalized_email,
-                        reason=LoginFailureReason.INVALID_INVITATION_EMAIL,
-                    )
                     raise InvalidEmailError()
             account = _authenticate_account_with_case_fallback(
                 request_email, normalized_email, args.password, invite_token
             )
         except services.errors.account.AccountLoginError:
-            _log_console_login_failure(email=normalized_email, reason=LoginFailureReason.ACCOUNT_BANNED)
             raise AccountBannedError()
         except services.errors.account.AccountPasswordError as exc:
             AccountService.add_login_error_rate_limit(normalized_email)
-            _log_console_login_failure(email=normalized_email, reason=LoginFailureReason.INVALID_CREDENTIALS)
             raise AuthenticationFailedError() from exc
         # SELF_HOSTED only have one workspace
         tenants = TenantService.get_join_tenants(account)
@@ -252,27 +241,20 @@ class EmailCodeLoginApi(Resource):
 
         token_data = AccountService.get_email_code_login_data(args.token)
         if token_data is None:
-            _log_console_login_failure(email=user_email, reason=LoginFailureReason.INVALID_EMAIL_CODE_TOKEN)
             raise InvalidTokenError()
 
         token_email = token_data.get("email")
         normalized_token_email = token_email.lower() if isinstance(token_email, str) else token_email
         if normalized_token_email != user_email:
-            _log_console_login_failure(email=user_email, reason=LoginFailureReason.EMAIL_CODE_EMAIL_MISMATCH)
             raise InvalidEmailError()
 
         if token_data["code"] != args.code:
-            _log_console_login_failure(email=user_email, reason=LoginFailureReason.INVALID_EMAIL_CODE)
             raise EmailCodeError()
 
         AccountService.revoke_email_code_login_token(args.token)
         try:
             account = _get_account_with_case_fallback(original_email)
-        except Unauthorized as exc:
-            _log_console_login_failure(email=user_email, reason=LoginFailureReason.ACCOUNT_BANNED)
-            raise AccountBannedError() from exc
         except AccountRegisterError:
-            _log_console_login_failure(email=user_email, reason=LoginFailureReason.ACCOUNT_IN_FREEZE)
             raise AccountInFreezeError()
         if account:
             tenants = TenantService.get_join_tenants(account)
@@ -298,7 +280,6 @@ class EmailCodeLoginApi(Resource):
             except WorkSpaceNotAllowedCreateError:
                 raise NotAllowedCreateWorkspace()
             except AccountRegisterError:
-                _log_console_login_failure(email=user_email, reason=LoginFailureReason.ACCOUNT_IN_FREEZE)
                 raise AccountInFreezeError()
             except WorkspacesLimitExceededError:
                 raise WorkspacesLimitExceeded()
@@ -356,12 +337,3 @@ def _authenticate_account_with_case_fallback(
         if original_email == normalized_email:
             raise
         return AccountService.authenticate(normalized_email, password, invite_token)
-
-
-def _log_console_login_failure(*, email: str, reason: LoginFailureReason) -> None:
-    logger.warning(
-        "Console login failed: email=%s reason=%s ip_address=%s",
-        email,
-        reason,
-        extract_remote_ip(request),
-    )

api/controllers/console/auth/oauth.py

@@ -4,6 +4,7 @@ import urllib.parse
 import httpx
 from flask import current_app, redirect, request
 from flask_restx import Resource
+from sqlalchemy.orm import sessionmaker
 from werkzeug.exceptions import Unauthorized
 
 from configs import dify_config
@@ -179,7 +180,8 @@ def _get_account_by_openid_or_email(provider: str, user_info: OAuthUserInfo) ->
     account: Account | None = Account.get_by_openid(provider, user_info.id)
 
     if not account:
-        account = AccountService.get_account_by_email_with_case_fallback(user_info.email)
+        with sessionmaker(db.engine).begin() as session:
+            account = AccountService.get_account_by_email_with_case_fallback(user_info.email, session=session)
 
     return account
 

api/controllers/console/billing/billing.py

@@ -2,17 +2,18 @@ import base64
 from typing import Literal
 
 from flask import request
-from flask_restx import Resource
+from flask_restx import Resource, fields
 from pydantic import BaseModel, Field
 from werkzeug.exceptions import BadRequest
 
-from controllers.common.schema import register_schema_models
 from controllers.console import console_ns
 from controllers.console.wraps import account_initialization_required, only_edition_cloud, setup_required
 from enums.cloud_plan import CloudPlan
 from libs.login import current_account_with_tenant, login_required
 from services.billing_service import BillingService
 
+DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
+
 
 class SubscriptionQuery(BaseModel):
     plan: Literal[CloudPlan.PROFESSIONAL, CloudPlan.TEAM] = Field(..., description="Subscription plan")
@@ -23,7 +24,8 @@ class PartnerTenantsPayload(BaseModel):
     click_id: str = Field(..., description="Click Id from partner referral link")
 
 
-register_schema_models(console_ns, SubscriptionQuery, PartnerTenantsPayload)
+for model in (SubscriptionQuery, PartnerTenantsPayload):
+    console_ns.schema_model(model.__name__, model.model_json_schema(ref_template=DEFAULT_REF_TEMPLATE_SWAGGER_2_0))
 
 
 @console_ns.route("/billing/subscription")
@@ -56,7 +58,12 @@ class PartnerTenants(Resource):
     @console_ns.doc("sync_partner_tenants_bindings")
     @console_ns.doc(description="Sync partner tenants bindings")
     @console_ns.doc(params={"partner_key": "Partner key"})
-    @console_ns.expect(console_ns.models[PartnerTenantsPayload.__name__])
+    @console_ns.expect(
+        console_ns.model(
+            "SyncPartnerTenantsBindingsRequest",
+            {"click_id": fields.String(required=True, description="Click Id from partner referral link")},
+        )
+    )
     @console_ns.response(200, "Tenants synced to partner successfully")
     @console_ns.response(400, "Invalid partner information")
     @setup_required

api/controllers/console/datasets/data_source.py

@@ -162,9 +162,7 @@ class DataSourceApi(Resource):
         binding_id = str(binding_id)
         with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
             data_source_binding = session.execute(
-                select(DataSourceOauthBinding).where(
-                    DataSourceOauthBinding.id == binding_id, DataSourceOauthBinding.tenant_id == current_tenant_id
-                )
+                select(DataSourceOauthBinding).filter_by(id=binding_id, tenant_id=current_tenant_id)
             ).scalar_one_or_none()
         if data_source_binding is None:
             raise NotFound("Data source binding not found.")
@@ -224,11 +222,11 @@ class DataSourceNotionListApi(Resource):
                     raise ValueError("Dataset is not notion type.")
 
                 documents = session.scalars(
-                    select(Document).where(
-                        Document.dataset_id == query.dataset_id,
-                        Document.tenant_id == current_tenant_id,
-                        Document.data_source_type == "notion_import",
-                        Document.enabled.is_(True),
+                    select(Document).filter_by(
+                        dataset_id=query.dataset_id,
+                        tenant_id=current_tenant_id,
+                        data_source_type="notion_import",
+                        enabled=True,
                     )
                 ).all()
                 if documents:

api/controllers/console/datasets/datasets.py

@@ -11,7 +11,10 @@ import services
 from configs import dify_config
 from controllers.common.schema import get_or_create_model, register_schema_models
 from controllers.console import console_ns
-from controllers.console.apikey import ApiKeyItem, ApiKeyList
+from controllers.console.apikey import (
+    api_key_item_model,
+    api_key_list_model,
+)
 from controllers.console.app.error import ProviderNotInitializeError
 from controllers.console.datasets.error import DatasetInUseError, DatasetNameDuplicateError, IndexingEstimateError
 from controllers.console.wraps import (
@@ -782,23 +785,23 @@ class DatasetApiKeyApi(Resource):
 
     @console_ns.doc("get_dataset_api_keys")
     @console_ns.doc(description="Get dataset API keys")
-    @console_ns.response(200, "API keys retrieved successfully", console_ns.models[ApiKeyList.__name__])
+    @console_ns.response(200, "API keys retrieved successfully", api_key_list_model)
     @setup_required
     @login_required
     @account_initialization_required
+    @marshal_with(api_key_list_model)
     def get(self):
         _, current_tenant_id = current_account_with_tenant()
         keys = db.session.scalars(
             select(ApiToken).where(ApiToken.type == self.resource_type, ApiToken.tenant_id == current_tenant_id)
         ).all()
-        return ApiKeyList.model_validate({"data": keys}, from_attributes=True).model_dump(mode="json")
+        return {"items": keys}
 
-    @console_ns.response(200, "API key created successfully", console_ns.models[ApiKeyItem.__name__])
-    @console_ns.response(400, "Maximum keys exceeded")
     @setup_required
     @login_required
     @is_admin_or_owner_required
     @account_initialization_required
+    @marshal_with(api_key_item_model)
     def post(self):
         _, current_tenant_id = current_account_with_tenant()
 
@@ -825,7 +828,7 @@ class DatasetApiKeyApi(Resource):
         api_token.type = self.resource_type
         db.session.add(api_token)
         db.session.commit()
-        return ApiKeyItem.model_validate(api_token, from_attributes=True).model_dump(mode="json"), 200
+        return api_token, 200
 
 
 @console_ns.route("/datasets/api-keys/<uuid:api_key_id>")

api/controllers/console/datasets/datasets_document.py

@@ -4,6 +4,7 @@ from argparse import ArgumentTypeError
 from collections.abc import Sequence
 from contextlib import ExitStack
 from typing import Any, Literal, cast
+from uuid import UUID
 
 import sqlalchemy as sa
 from flask import request, send_file
@@ -15,7 +16,6 @@ from sqlalchemy import asc, desc, func, select
 from werkzeug.exceptions import Forbidden, NotFound
 
 import services
-from controllers.common.controller_schemas import DocumentBatchDownloadZipPayload
 from controllers.common.schema import get_or_create_model, register_schema_models
 from controllers.console import console_ns
 from core.errors.error import (
@@ -71,6 +71,9 @@ from ..wraps import (
 
 logger = logging.getLogger(__name__)
 
+# NOTE: Keep constants near the top of the module for discoverability.
+DOCUMENT_BATCH_DOWNLOAD_ZIP_MAX_DOCS = 100
+
 
 # Register models for flask_restx to avoid dict type issues in Swagger
 dataset_model = get_or_create_model("Dataset", dataset_fields)
@@ -107,6 +110,12 @@ class GenerateSummaryPayload(BaseModel):
     document_list: list[str]
 
 
+class DocumentBatchDownloadZipPayload(BaseModel):
+    """Request payload for bulk downloading documents as a zip archive."""
+
+    document_ids: list[UUID] = Field(..., min_length=1, max_length=DOCUMENT_BATCH_DOWNLOAD_ZIP_MAX_DOCS)
+
+
 class DocumentDatasetListParam(BaseModel):
     page: int = Field(1, title="Page", description="Page number.")
     limit: int = Field(20, title="Limit", description="Page size.")
@@ -271,7 +280,7 @@ class DatasetDocumentListApi(Resource):
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
-        query = select(Document).where(Document.dataset_id == str(dataset_id), Document.tenant_id == current_tenant_id)
+        query = select(Document).filter_by(dataset_id=str(dataset_id), tenant_id=current_tenant_id)
 
         if status:
             query = DocumentService.apply_display_status_filter(query, status)

api/controllers/console/datasets/datasets_segments.py

@@ -10,7 +10,6 @@ from werkzeug.exceptions import Forbidden, NotFound
 
 import services
 from configs import dify_config
-from controllers.common.controller_schemas import ChildChunkCreatePayload, ChildChunkUpdatePayload
 from controllers.common.schema import register_schema_models
 from controllers.console import console_ns
 from controllers.console.app.error import ProviderNotInitializeError
@@ -83,6 +82,14 @@ class BatchImportPayload(BaseModel):
     upload_file_id: str
 
 
+class ChildChunkCreatePayload(BaseModel):
+    content: str
+
+
+class ChildChunkUpdatePayload(BaseModel):
+    content: str
+
+
 class ChildChunkBatchUpdatePayload(BaseModel):
     chunks: list[ChildChunkUpdateArgs]
 

api/controllers/console/datasets/external.py

@@ -227,11 +227,10 @@ class ExternalApiUseCheckApi(Resource):
     @login_required
     @account_initialization_required
     def get(self, external_knowledge_api_id):
-        _, current_tenant_id = current_account_with_tenant()
         external_knowledge_api_id = str(external_knowledge_api_id)
 
         external_knowledge_api_is_using, count = ExternalDatasetService.external_knowledge_api_use_check(
-            external_knowledge_api_id, current_tenant_id
+            external_knowledge_api_id
         )
         return {"is_using": external_knowledge_api_is_using, "count": count}, 200
 

api/controllers/console/datasets/metadata.py

@@ -1,9 +1,9 @@
 from typing import Literal
 
 from flask_restx import Resource, marshal_with
+from pydantic import BaseModel
 from werkzeug.exceptions import NotFound
 
-from controllers.common.controller_schemas import MetadataUpdatePayload
 from controllers.common.schema import register_schema_models
 from controllers.console import console_ns
 from controllers.console.wraps import account_initialization_required, enterprise_license_required, setup_required
@@ -18,6 +18,11 @@ from services.entities.knowledge_entities.knowledge_entities import (
 )
 from services.metadata_service import MetadataService
 
+
+class MetadataUpdatePayload(BaseModel):
+    name: str
+
+
 register_schema_models(
     console_ns, MetadataArgs, MetadataOperationData, MetadataUpdatePayload, DocumentMetadataOperation, MetadataDetail
 )

api/controllers/console/datasets/rag_pipeline/rag_pipeline_draft_variable.py

@@ -223,27 +223,24 @@ class RagPipelineVariableApi(Resource):
 
         new_value = None
         if raw_value is not None:
-            match variable.value_type:
-                case SegmentType.FILE:
-                    if not isinstance(raw_value, dict):
-                        raise InvalidArgumentError(description=f"expected dict for file, got {type(raw_value)}")
-                    raw_value = build_from_mapping(
-                        mapping=raw_value,
-                        tenant_id=pipeline.tenant_id,
-                        access_controller=_file_access_controller,
-                    )
-                case SegmentType.ARRAY_FILE:
-                    if not isinstance(raw_value, list):
-                        raise InvalidArgumentError(description=f"expected list for files, got {type(raw_value)}")
-                    if len(raw_value) > 0 and not isinstance(raw_value[0], dict):
-                        raise InvalidArgumentError(description=f"expected dict for files[0], got {type(raw_value)}")
-                    raw_value = build_from_mappings(
-                        mappings=raw_value,
-                        tenant_id=pipeline.tenant_id,
-                        access_controller=_file_access_controller,
-                    )
-                case _:
-                    pass
+            if variable.value_type == SegmentType.FILE:
+                if not isinstance(raw_value, dict):
+                    raise InvalidArgumentError(description=f"expected dict for file, got {type(raw_value)}")
+                raw_value = build_from_mapping(
+                    mapping=raw_value,
+                    tenant_id=pipeline.tenant_id,
+                    access_controller=_file_access_controller,
+                )
+            elif variable.value_type == SegmentType.ARRAY_FILE:
+                if not isinstance(raw_value, list):
+                    raise InvalidArgumentError(description=f"expected list for files, got {type(raw_value)}")
+                if len(raw_value) > 0 and not isinstance(raw_value[0], dict):
+                    raise InvalidArgumentError(description=f"expected dict for files[0], got {type(raw_value)}")
+                raw_value = build_from_mappings(
+                    mappings=raw_value,
+                    tenant_id=pipeline.tenant_id,
+                    access_controller=_file_access_controller,
+                )
             new_value = build_segment_with_type(variable.value_type, raw_value)
         draft_var_srv.update_variable(variable, name=new_name, value=new_value)
         db.session.commit()

api/controllers/console/datasets/rag_pipeline/rag_pipeline_import.py

@@ -19,7 +19,7 @@ from fields.rag_pipeline_fields import (
 )
 from libs.login import current_account_with_tenant, login_required
 from models.dataset import Pipeline
-from services.entities.dsl_entities import ImportStatus
+from services.app_dsl_service import ImportStatus
 from services.rag_pipeline.rag_pipeline_dsl_service import RagPipelineDslService
 
 
@@ -83,13 +83,11 @@ class RagPipelineImportApi(Resource):
 
         # Return appropriate status code based on result
         status = result.status
-        match status:
-            case ImportStatus.FAILED:
-                return result.model_dump(mode="json"), 400
-            case ImportStatus.PENDING:
-                return result.model_dump(mode="json"), 202
-            case ImportStatus.COMPLETED | ImportStatus.COMPLETED_WITH_WARNINGS:
-                return result.model_dump(mode="json"), 200
+        if status == ImportStatus.FAILED:
+            return result.model_dump(mode="json"), 400
+        elif status == ImportStatus.PENDING:
+            return result.model_dump(mode="json"), 202
+        return result.model_dump(mode="json"), 200
 
 
 @console_ns.route("/rag/pipelines/imports/<string:import_id>/confirm")

api/controllers/console/datasets/rag_pipeline/rag_pipeline_workflow.py

@@ -10,7 +10,6 @@ from sqlalchemy.orm import sessionmaker
 from werkzeug.exceptions import BadRequest, Forbidden, InternalServerError, NotFound
 
 import services
-from controllers.common.controller_schemas import DefaultBlockConfigQuery, WorkflowListQuery, WorkflowUpdatePayload
 from controllers.common.schema import register_schema_models
 from controllers.console import console_ns
 from controllers.console.app.error import (
@@ -95,6 +94,22 @@ class PublishedWorkflowRunPayload(DraftWorkflowRunPayload):
     original_document_id: str | None = None
 
 
+class DefaultBlockConfigQuery(BaseModel):
+    q: str | None = None
+
+
+class WorkflowListQuery(BaseModel):
+    page: int = Field(default=1, ge=1, le=99999)
+    limit: int = Field(default=10, ge=1, le=100)
+    user_id: str | None = None
+    named_only: bool = False
+
+
+class WorkflowUpdatePayload(BaseModel):
+    marked_name: str | None = Field(default=None, max_length=20)
+    marked_comment: str | None = Field(default=None, max_length=100)
+
+
 class NodeIdQuery(BaseModel):
     node_id: str
 
@@ -346,6 +361,89 @@ class PublishedRagPipelineRunApi(Resource):
             raise InvokeRateLimitHttpError(ex.description)
 
 
+# class RagPipelinePublishedDatasourceNodeRunStatusApi(Resource):
+#     @setup_required
+#     @login_required
+#     @account_initialization_required
+#     @get_rag_pipeline
+#     def post(self, pipeline: Pipeline, node_id: str):
+#         """
+#         Run rag pipeline datasource
+#         """
+#         # The role of the current user in the ta table must be admin, owner, or editor
+#         if not current_user.has_edit_permission:
+#             raise Forbidden()
+#
+#         if not isinstance(current_user, Account):
+#             raise Forbidden()
+#
+#         parser = (reqparse.RequestParser()
+#             .add_argument("job_id", type=str, required=True, nullable=False, location="json")
+#             .add_argument("datasource_type", type=str, required=True, location="json")
+#         )
+#         args = parser.parse_args()
+#
+#         job_id = args.get("job_id")
+#         if job_id == None:
+#             raise ValueError("missing job_id")
+#         datasource_type = args.get("datasource_type")
+#         if datasource_type == None:
+#             raise ValueError("missing datasource_type")
+#
+#         rag_pipeline_service = RagPipelineService()
+#         result = rag_pipeline_service.run_datasource_workflow_node_status(
+#             pipeline=pipeline,
+#             node_id=node_id,
+#             job_id=job_id,
+#             account=current_user,
+#             datasource_type=datasource_type,
+#             is_published=True
+#         )
+#
+#         return result
+
+
+# class RagPipelineDraftDatasourceNodeRunStatusApi(Resource):
+#     @setup_required
+#     @login_required
+#     @account_initialization_required
+#     @get_rag_pipeline
+#     def post(self, pipeline: Pipeline, node_id: str):
+#         """
+#         Run rag pipeline datasource
+#         """
+#         # The role of the current user in the ta table must be admin, owner, or editor
+#         if not current_user.has_edit_permission:
+#             raise Forbidden()
+#
+#         if not isinstance(current_user, Account):
+#             raise Forbidden()
+#
+#         parser = (reqparse.RequestParser()
+#             .add_argument("job_id", type=str, required=True, nullable=False, location="json")
+#             .add_argument("datasource_type", type=str, required=True, location="json")
+#         )
+#         args = parser.parse_args()
+#
+#         job_id = args.get("job_id")
+#         if job_id == None:
+#             raise ValueError("missing job_id")
+#         datasource_type = args.get("datasource_type")
+#         if datasource_type == None:
+#             raise ValueError("missing datasource_type")
+#
+#         rag_pipeline_service = RagPipelineService()
+#         result = rag_pipeline_service.run_datasource_workflow_node_status(
+#             pipeline=pipeline,
+#             node_id=node_id,
+#             job_id=job_id,
+#             account=current_user,
+#             datasource_type=datasource_type,
+#             is_published=False
+#         )
+#
+#         return result
+#
 @console_ns.route("/rag/pipelines/<uuid:pipeline_id>/workflows/published/datasource/nodes/<string:node_id>/run")
 class RagPipelinePublishedDatasourceNodeRunApi(Resource):
     @console_ns.expect(console_ns.models[DatasourceNodeRunPayload.__name__])

api/controllers/console/explore/audio.py

@@ -2,10 +2,10 @@ import logging
 
 from flask import request
 from graphon.model_runtime.errors.invoke import InvokeError
+from pydantic import BaseModel, Field
 from werkzeug.exceptions import InternalServerError
 
 import services
-from controllers.common.controller_schemas import TextToAudioPayload
 from controllers.common.schema import register_schema_model
 from controllers.console.app.error import (
     AppUnavailableError,
@@ -32,6 +32,14 @@ from .. import console_ns
 
 logger = logging.getLogger(__name__)
 
+
+class TextToAudioPayload(BaseModel):
+    message_id: str | None = None
+    voice: str | None = None
+    text: str | None = None
+    streaming: bool | None = Field(default=None, description="Enable streaming response")
+
+
 register_schema_model(console_ns, TextToAudioPayload)
 
 

api/controllers/console/explore/conversation.py

@@ -1,11 +1,10 @@
 from typing import Any
 
 from flask import request
-from pydantic import BaseModel, Field, TypeAdapter
+from pydantic import BaseModel, Field, TypeAdapter, model_validator
 from sqlalchemy.orm import sessionmaker
 from werkzeug.exceptions import NotFound
 
-from controllers.common.controller_schemas import ConversationRenamePayload
 from controllers.common.schema import register_schema_models
 from controllers.console.explore.error import NotChatAppError
 from controllers.console.explore.wraps import InstalledAppResource
@@ -33,6 +32,18 @@ class ConversationListQuery(BaseModel):
     pinned: bool | None = None
 
 
+class ConversationRenamePayload(BaseModel):
+    name: str | None = None
+    auto_generate: bool = False
+
+    @model_validator(mode="after")
+    def validate_name_requirement(self):
+        if not self.auto_generate:
+            if self.name is None or not self.name.strip():
+                raise ValueError("name is required when auto_generate is false")
+        return self
+
+
 register_schema_models(console_ns, ConversationListQuery, ConversationRenamePayload)
 
 

api/controllers/console/explore/message.py

@@ -3,10 +3,9 @@ from typing import Literal
 
 from flask import request
 from graphon.model_runtime.errors.invoke import InvokeError
-from pydantic import BaseModel, TypeAdapter
+from pydantic import BaseModel, Field, TypeAdapter
 from werkzeug.exceptions import InternalServerError, NotFound
 
-from controllers.common.controller_schemas import MessageFeedbackPayload, MessageListQuery
 from controllers.common.schema import register_schema_models
 from controllers.console.app.error import (
     AppMoreLikeThisDisabledError,
@@ -26,6 +25,7 @@ from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotIni
 from fields.conversation_fields import ResultResponse
 from fields.message_fields import MessageInfiniteScrollPagination, MessageListItem, SuggestedQuestionsResponse
 from libs import helper
+from libs.helper import UUIDStrOrEmpty
 from libs.login import current_account_with_tenant
 from models.enums import FeedbackRating
 from models.model import AppMode
@@ -44,6 +44,17 @@ from .. import console_ns
 logger = logging.getLogger(__name__)
 
 
+class MessageListQuery(BaseModel):
+    conversation_id: UUIDStrOrEmpty
+    first_id: UUIDStrOrEmpty | None = None
+    limit: int = Field(default=20, ge=1, le=100)
+
+
+class MessageFeedbackPayload(BaseModel):
+    rating: Literal["like", "dislike"] | None = None
+    content: str | None = None
+
+
 class MoreLikeThisQuery(BaseModel):
     response_mode: Literal["blocking", "streaming"]
 

api/controllers/console/explore/saved_message.py

@@ -1,18 +1,28 @@
 from flask import request
-from pydantic import TypeAdapter
+from pydantic import BaseModel, Field, TypeAdapter
 from werkzeug.exceptions import NotFound
 
-from controllers.common.controller_schemas import SavedMessageCreatePayload, SavedMessageListQuery
 from controllers.common.schema import register_schema_models
 from controllers.console import console_ns
 from controllers.console.explore.error import NotCompletionAppError
 from controllers.console.explore.wraps import InstalledAppResource
 from fields.conversation_fields import ResultResponse
 from fields.message_fields import SavedMessageInfiniteScrollPagination, SavedMessageItem
+from libs.helper import UUIDStrOrEmpty
 from libs.login import current_account_with_tenant
 from services.errors.message import MessageNotExistsError
 from services.saved_message_service import SavedMessageService
 
+
+class SavedMessageListQuery(BaseModel):
+    last_id: UUIDStrOrEmpty | None = None
+    limit: int = Field(default=20, ge=1, le=100)
+
+
+class SavedMessageCreatePayload(BaseModel):
+    message_id: UUIDStrOrEmpty
+
+
 register_schema_models(console_ns, SavedMessageListQuery, SavedMessageCreatePayload)
 
 

api/controllers/console/explore/workflow.py

@@ -1,10 +1,11 @@
 import logging
+from typing import Any
 
 from graphon.graph_engine.manager import GraphEngineManager
 from graphon.model_runtime.errors.invoke import InvokeError
+from pydantic import BaseModel
 from werkzeug.exceptions import InternalServerError
 
-from controllers.common.controller_schemas import WorkflowRunPayload
 from controllers.common.schema import register_schema_model
 from controllers.console.app.error import (
     CompletionRequestError,
@@ -33,6 +34,12 @@ from .. import console_ns
 
 logger = logging.getLogger(__name__)
 
+
+class WorkflowRunPayload(BaseModel):
+    inputs: dict[str, Any]
+    files: list[dict[str, Any]] | None = None
+
+
 register_schema_model(console_ns, WorkflowRunPayload)
 
 

api/controllers/console/human_input_form.py

@@ -7,8 +7,7 @@ import logging
 from collections.abc import Generator
 
 from flask import Response, jsonify, request
-from flask_restx import Resource
-from pydantic import BaseModel
+from flask_restx import Resource, reqparse
 from sqlalchemy import select
 from sqlalchemy.orm import Session, sessionmaker
 
@@ -34,11 +33,6 @@ from services.workflow_event_snapshot_service import build_workflow_event_stream
 logger = logging.getLogger(__name__)
 
 
-class HumanInputFormSubmitPayload(BaseModel):
-    inputs: dict
-    action: str
-
-
 def _jsonify_form_definition(form: Form) -> Response:
     payload = form.get_definition().model_dump()
     payload["expiration_time"] = int(form.expiration_time.timestamp())
@@ -90,7 +84,10 @@ class ConsoleHumanInputFormApi(Resource):
             "action": "Approve"
         }
         """
-        payload = HumanInputFormSubmitPayload.model_validate(request.get_json())
+        parser = reqparse.RequestParser()
+        parser.add_argument("inputs", type=dict, required=True, location="json")
+        parser.add_argument("action", type=str, required=True, location="json")
+        args = parser.parse_args()
         current_user, _ = current_account_with_tenant()
 
         service = HumanInputService(db.engine)
@@ -110,8 +107,8 @@ class ConsoleHumanInputFormApi(Resource):
         service.submit_form_by_token(
             recipient_type=recipient_type,
             form_token=form_token,
-            selected_action_id=payload.action,
-            form_data=payload.inputs,
+            selected_action_id=args["action"],
+            form_data=args["inputs"],
             submission_user_id=current_user.id,
         )
 
@@ -171,13 +168,12 @@ class ConsoleWorkflowEventsApi(Resource):
         else:
             msg_generator = MessageGenerator()
             generator: BaseAppGenerator
-            match app.mode:
-                case AppMode.ADVANCED_CHAT:
-                    generator = AdvancedChatAppGenerator()
-                case AppMode.WORKFLOW:
-                    generator = WorkflowAppGenerator()
-                case _:
-                    raise InvalidArgumentError(f"cannot subscribe to workflow run, workflow_run_id={workflow_run.id}")
+            if app.mode == AppMode.ADVANCED_CHAT:
+                generator = AdvancedChatAppGenerator()
+            elif app.mode == AppMode.WORKFLOW:
+                generator = WorkflowAppGenerator()
+            else:
+                raise InvalidArgumentError(f"cannot subscribe to workflow run, workflow_run_id={workflow_run.id}")
 
             include_state_snapshot = request.args.get("include_state_snapshot", "false").lower() == "true"
 

api/controllers/console/notification.py

@@ -1,4 +1,3 @@
-from collections.abc import Mapping
 from typing import TypedDict
 
 from flask import request
@@ -14,14 +13,6 @@ from services.billing_service import BillingService
 _FALLBACK_LANG = "en-US"
 
 
-class NotificationLangContent(TypedDict, total=False):
-    lang: str
-    title: str
-    subtitle: str
-    body: str
-    titlePicUrl: str
-
-
 class NotificationItemDict(TypedDict):
     notification_id: str | None
     frequency: str | None
@@ -37,11 +28,9 @@ class NotificationResponseDict(TypedDict):
     notifications: list[NotificationItemDict]
 
 
-def _pick_lang_content(contents: Mapping[str, NotificationLangContent], lang: str) -> NotificationLangContent:
+def _pick_lang_content(contents: dict, lang: str) -> dict:
     """Return the single LangContent for *lang*, falling back to English."""
-    return (
-        contents.get(lang) or contents.get(_FALLBACK_LANG) or next(iter(contents.values()), NotificationLangContent())
-    )
+    return contents.get(lang) or contents.get(_FALLBACK_LANG) or next(iter(contents.values()), {})
 
 
 class DismissNotificationPayload(BaseModel):
@@ -82,7 +71,7 @@ class NotificationApi(Resource):
 
         notifications: list[NotificationItemDict] = []
         for notification in result.get("notifications") or []:
-            contents: Mapping[str, NotificationLangContent] = notification.get("contents") or {}
+            contents: dict = notification.get("contents") or {}
             lang_content = _pick_lang_content(contents, lang)
             item: NotificationItemDict = {
                 "notification_id": notification.get("notificationId"),

api/controllers/console/workspace/__init__.py

@@ -35,24 +35,22 @@ def plugin_permission_required(
                     return view(*args, **kwargs)
 
                 if install_required:
-                    match permission.install_permission:
-                        case TenantPluginPermission.InstallPermission.NOBODY:
+                    if permission.install_permission == TenantPluginPermission.InstallPermission.NOBODY:
+                        raise Forbidden()
+                    if permission.install_permission == TenantPluginPermission.InstallPermission.ADMINS:
+                        if not user.is_admin_or_owner:
                             raise Forbidden()
-                        case TenantPluginPermission.InstallPermission.ADMINS:
-                            if not user.is_admin_or_owner:
-                                raise Forbidden()
-                        case TenantPluginPermission.InstallPermission.EVERYONE:
-                            pass
+                    if permission.install_permission == TenantPluginPermission.InstallPermission.EVERYONE:
+                        pass
 
                 if debug_required:
-                    match permission.debug_permission:
-                        case TenantPluginPermission.DebugPermission.NOBODY:
+                    if permission.debug_permission == TenantPluginPermission.DebugPermission.NOBODY:
+                        raise Forbidden()
+                    if permission.debug_permission == TenantPluginPermission.DebugPermission.ADMINS:
+                        if not user.is_admin_or_owner:
                             raise Forbidden()
-                        case TenantPluginPermission.DebugPermission.ADMINS:
-                            if not user.is_admin_or_owner:
-                                raise Forbidden()
-                        case TenantPluginPermission.DebugPermission.EVERYONE:
-                            pass
+                    if permission.debug_permission == TenantPluginPermission.DebugPermission.EVERYONE:
+                        pass
 
             return view(*args, **kwargs)
 

api/controllers/console/workspace/account.py

@@ -1,13 +1,14 @@
 from __future__ import annotations
 
 from datetime import datetime
-from typing import Any, Literal
+from typing import Literal
 
 import pytz
 from flask import request
 from flask_restx import Resource, fields, marshal_with
 from pydantic import BaseModel, Field, field_validator, model_validator
 from sqlalchemy import select
+from sqlalchemy.orm import sessionmaker
 
 from configs import dify_config
 from constants.languages import supported_language
@@ -174,7 +175,7 @@ reg(CheckEmailUniquePayload)
 register_schema_models(console_ns, AccountResponse)
 
 
-def _serialize_account(account) -> dict[str, Any]:
+def _serialize_account(account) -> dict:
     return AccountResponse.model_validate(account, from_attributes=True).model_dump(mode="json")
 
 
@@ -561,7 +562,8 @@ class ChangeEmailSendEmailApi(Resource):
 
             user_email = current_user.email
         else:
-            account = AccountService.get_account_by_email_with_case_fallback(args.email)
+            with sessionmaker(db.engine).begin() as session:
+                account = AccountService.get_account_by_email_with_case_fallback(args.email, session=session)
             if account is None:
                 raise AccountNotFound()
             email_for_sending = account.email

api/controllers/console/workspace/workspace.py

@@ -28,7 +28,7 @@ from enums.cloud_plan import CloudPlan
 from extensions.ext_database import db
 from libs.helper import TimestampField
 from libs.login import current_account_with_tenant, login_required
-from models.account import Tenant, TenantCustomConfigDict, TenantStatus
+from models.account import Tenant, TenantStatus
 from services.account_service import TenantService
 from services.billing_service import BillingService, SubscriptionPlan
 from services.enterprise.enterprise_service import EnterpriseService
@@ -240,10 +240,8 @@ class CustomConfigWorkspaceApi(Resource):
         args = WorkspaceCustomConfigPayload.model_validate(payload)
         tenant = db.get_or_404(Tenant, current_tenant_id)
 
-        custom_config_dict: TenantCustomConfigDict = {
-            "remove_webapp_brand": args.remove_webapp_brand
-            if args.remove_webapp_brand is not None
-            else tenant.custom_config_dict.get("remove_webapp_brand", False),
+        custom_config_dict = {
+            "remove_webapp_brand": args.remove_webapp_brand,
             "replace_webapp_logo": args.replace_webapp_logo
             if args.replace_webapp_logo is not None
             else tenant.custom_config_dict.get("replace_webapp_logo"),

api/controllers/console/wraps.py

@@ -20,7 +20,7 @@ from models.account import AccountStatus
 from models.dataset import RateLimitLog
 from models.model import DifySetup
 from services.feature_service import FeatureService, LicenseStatus
-from services.operation_service import OperationService, UtmInfo
+from services.operation_service import OperationService
 
 from .error import NotInitValidateError, NotSetupError, UnauthorizedAndForceLogout
 
@@ -205,7 +205,7 @@ def cloud_utm_record[**P, R](view: Callable[P, R]) -> Callable[P, R]:
                 utm_info = request.cookies.get("utm_info")
 
                 if utm_info:
-                    utm_info_dict: UtmInfo = json.loads(utm_info)
+                    utm_info_dict: dict = json.loads(utm_info)
                     OperationService.record_utm(current_tenant_id, utm_info_dict)
 
         return view(*args, **kwargs)

api/controllers/inner_api/app/dsl.py

@@ -9,7 +9,7 @@ from flask import request
 from flask_restx import Resource
 from pydantic import BaseModel, Field
 from sqlalchemy import select
-from sqlalchemy.orm import sessionmaker
+from sqlalchemy.orm import Session
 
 from controllers.common.schema import register_schema_model
 from controllers.console.wraps import setup_required
@@ -18,8 +18,7 @@ from controllers.inner_api.wraps import enterprise_inner_api_only
 from extensions.ext_database import db
 from models import Account, App
 from models.account import AccountStatus
-from services.app_dsl_service import AppDslService
-from services.entities.dsl_entities import ImportMode, ImportStatus
+from services.app_dsl_service import AppDslService, ImportMode, ImportStatus
 
 
 class InnerAppDSLImportPayload(BaseModel):
@@ -56,7 +55,7 @@ class EnterpriseAppDSLImport(Resource):
 
         account.set_tenant_id(workspace_id)
 
-        with sessionmaker(db.engine).begin() as session:
+        with Session(db.engine) as session:
             dsl_service = AppDslService(session)
             result = dsl_service.import_app(
                 account=account,
@@ -65,6 +64,7 @@ class EnterpriseAppDSLImport(Resource):
                 name=args.name,
                 description=args.description,
             )
+            session.commit()
 
         if result.status == ImportStatus.FAILED:
             return result.model_dump(mode="json"), 400

api/controllers/inner_api/plugin/wraps.py

@@ -94,9 +94,10 @@ def get_user_tenant[**P, R](view_func: Callable[P, R]) -> Callable[P, R]:
 
 
 def plugin_data[**P, R](
+    view: Callable[P, R] | None = None,
     *,
     payload_type: type[BaseModel],
-) -> Callable[[Callable[P, R]], Callable[P, R]]:
+) -> Callable[P, R] | Callable[[Callable[P, R]], Callable[P, R]]:
     def decorator(view_func: Callable[P, R]) -> Callable[P, R]:
         @wraps(view_func)
         def decorated_view(*args: P.args, **kwargs: P.kwargs) -> R:
@@ -115,4 +116,7 @@ def plugin_data[**P, R](
 
         return decorated_view
 
-    return decorator
+    if view is None:
+        return decorator
+    else:
+        return decorator(view)

api/controllers/mcp/mcp.py

@@ -2,7 +2,7 @@ from typing import Any, Union
 
 from flask import Response
 from flask_restx import Resource
-from graphon.variables.input_entities import VariableEntity, VariableEntityType
+from graphon.variables.input_entities import VariableEntity
 from pydantic import BaseModel, Field, ValidationError
 from sqlalchemy import select
 from sqlalchemy.orm import Session, sessionmaker
@@ -158,20 +158,14 @@ class MCPAppApi(Resource):
         except ValidationError as e:
             raise MCPRequestError(mcp_types.INVALID_PARAMS, f"Invalid user_input_form: {str(e)}")
 
-    def _convert_user_input_form(self, raw_form: list[dict[str, Any]]) -> list[VariableEntity]:
+    def _convert_user_input_form(self, raw_form: list[dict]) -> list[VariableEntity]:
         """Convert raw user input form to VariableEntity objects"""
         return [self._create_variable_entity(item) for item in raw_form]
 
-    def _create_variable_entity(self, item: dict[str, Any]) -> VariableEntity:
+    def _create_variable_entity(self, item: dict) -> VariableEntity:
         """Create a single VariableEntity from raw form item"""
-        variable_type_raw: str = item.get("type", "") or list(item.keys())[0]
-        try:
-            variable_type = VariableEntityType(variable_type_raw)
-        except ValueError as e:
-            raise MCPRequestError(
-                mcp_types.INVALID_PARAMS, f"Invalid user_input_form variable type: {variable_type_raw}"
-            ) from e
-        variable = item[variable_type_raw]
+        variable_type = item.get("type", "") or list(item.keys())[0]
+        variable = item[variable_type]
 
         return VariableEntity(
             type=variable_type,
@@ -184,7 +178,7 @@ class MCPAppApi(Resource):
             json_schema=variable.get("json_schema"),
         )
 
-    def _parse_mcp_request(self, args: dict[str, Any]) -> mcp_types.ClientRequest | mcp_types.ClientNotification:
+    def _parse_mcp_request(self, args: dict) -> mcp_types.ClientRequest | mcp_types.ClientNotification:
         """Parse and validate MCP request"""
         try:
             return mcp_types.ClientRequest.model_validate(args)

api/controllers/service_api/app/annotation.py

@@ -12,12 +12,7 @@ from controllers.service_api.wraps import validate_app_token
 from extensions.ext_redis import redis_client
 from fields.annotation_fields import Annotation, AnnotationList
 from models.model import App
-from services.annotation_service import (
-    AppAnnotationService,
-    EnableAnnotationArgs,
-    InsertAnnotationArgs,
-    UpdateAnnotationArgs,
-)
+from services.annotation_service import AppAnnotationService
 
 
 class AnnotationCreatePayload(BaseModel):
@@ -51,15 +46,10 @@ class AnnotationReplyActionApi(Resource):
     @validate_app_token
     def post(self, app_model: App, action: Literal["enable", "disable"]):
         """Enable or disable annotation reply feature."""
-        payload = AnnotationReplyActionPayload.model_validate(service_api_ns.payload or {})
+        args = AnnotationReplyActionPayload.model_validate(service_api_ns.payload or {}).model_dump()
         match action:
             case "enable":
-                enable_args: EnableAnnotationArgs = {
-                    "score_threshold": payload.score_threshold,
-                    "embedding_provider_name": payload.embedding_provider_name,
-                    "embedding_model_name": payload.embedding_model_name,
-                }
-                result = AppAnnotationService.enable_app_annotation(enable_args, app_model.id)
+                result = AppAnnotationService.enable_app_annotation(args, app_model.id)
             case "disable":
                 result = AppAnnotationService.disable_app_annotation(app_model.id)
         return result, 200
@@ -145,9 +135,8 @@ class AnnotationListApi(Resource):
     @validate_app_token
     def post(self, app_model: App):
         """Create a new annotation."""
-        payload = AnnotationCreatePayload.model_validate(service_api_ns.payload or {})
-        insert_args: InsertAnnotationArgs = {"question": payload.question, "answer": payload.answer}
-        annotation = AppAnnotationService.insert_app_annotation_directly(insert_args, app_model.id)
+        args = AnnotationCreatePayload.model_validate(service_api_ns.payload or {}).model_dump()
+        annotation = AppAnnotationService.insert_app_annotation_directly(args, app_model.id)
         response = Annotation.model_validate(annotation, from_attributes=True)
         return response.model_dump(mode="json"), HTTPStatus.CREATED
 
@@ -175,9 +164,8 @@ class AnnotationUpdateDeleteApi(Resource):
     @edit_permission_required
     def put(self, app_model: App, annotation_id: str):
         """Update an existing annotation."""
-        payload = AnnotationCreatePayload.model_validate(service_api_ns.payload or {})
-        update_args: UpdateAnnotationArgs = {"question": payload.question, "answer": payload.answer}
-        annotation = AppAnnotationService.update_app_annotation_directly(update_args, app_model.id, annotation_id)
+        args = AnnotationCreatePayload.model_validate(service_api_ns.payload or {}).model_dump()
+        annotation = AppAnnotationService.update_app_annotation_directly(args, app_model.id, annotation_id)
         response = Annotation.model_validate(annotation, from_attributes=True)
         return response.model_dump(mode="json")
 

api/controllers/service_api/app/audio.py

@@ -3,10 +3,10 @@ import logging
 from flask import request
 from flask_restx import Resource
 from graphon.model_runtime.errors.invoke import InvokeError
+from pydantic import BaseModel, Field
 from werkzeug.exceptions import InternalServerError
 
 import services
-from controllers.common.controller_schemas import TextToAudioPayload
 from controllers.common.schema import register_schema_model
 from controllers.service_api import service_api_ns
 from controllers.service_api.app.error import (
@@ -86,6 +86,13 @@ class AudioApi(Resource):
             raise InternalServerError()
 
 
+class TextToAudioPayload(BaseModel):
+    message_id: str | None = Field(default=None, description="Message ID")
+    voice: str | None = Field(default=None, description="Voice to use for TTS")
+    text: str | None = Field(default=None, description="Text to convert to audio")
+    streaming: bool | None = Field(default=None, description="Enable streaming response")
+
+
 register_schema_model(service_api_ns, TextToAudioPayload)
 
 

api/controllers/service_api/app/conversation.py

@@ -2,12 +2,11 @@ from typing import Any, Literal
 
 from flask import request
 from flask_restx import Resource
-from pydantic import BaseModel, Field, TypeAdapter, field_validator
+from pydantic import BaseModel, Field, TypeAdapter, field_validator, model_validator
 from sqlalchemy.orm import sessionmaker
 from werkzeug.exceptions import BadRequest, NotFound
 
 import services
-from controllers.common.controller_schemas import ConversationRenamePayload
 from controllers.common.schema import register_schema_models
 from controllers.service_api import service_api_ns
 from controllers.service_api.app.error import NotChatAppError
@@ -35,6 +34,18 @@ class ConversationListQuery(BaseModel):
     )
 
 
+class ConversationRenamePayload(BaseModel):
+    name: str | None = Field(default=None, description="New conversation name (required if auto_generate is false)")
+    auto_generate: bool = Field(default=False, description="Auto-generate conversation name")
+
+    @model_validator(mode="after")
+    def validate_name_requirement(self):
+        if not self.auto_generate:
+            if self.name is None or not self.name.strip():
+                raise ValueError("name is required when auto_generate is false")
+        return self
+
+
 class ConversationVariablesQuery(BaseModel):
     last_id: UUIDStrOrEmpty | None = Field(default=None, description="Last variable ID for pagination")
     limit: int = Field(default=20, ge=1, le=100, description="Number of variables to return")

api/controllers/service_api/app/message.py

@@ -1,4 +1,5 @@
 import logging
+from typing import Literal
 
 from flask import request
 from flask_restx import Resource
@@ -6,7 +7,6 @@ from pydantic import BaseModel, Field, TypeAdapter
 from werkzeug.exceptions import BadRequest, InternalServerError, NotFound
 
 import services
-from controllers.common.controller_schemas import MessageFeedbackPayload, MessageListQuery
 from controllers.common.schema import register_schema_models
 from controllers.service_api import service_api_ns
 from controllers.service_api.app.error import NotChatAppError
@@ -14,6 +14,7 @@ from controllers.service_api.wraps import FetchUserArg, WhereisUserArg, validate
 from core.app.entities.app_invoke_entities import InvokeFrom
 from fields.conversation_fields import ResultResponse
 from fields.message_fields import MessageInfiniteScrollPagination, MessageListItem
+from libs.helper import UUIDStrOrEmpty
 from models.enums import FeedbackRating
 from models.model import App, AppMode, EndUser
 from services.errors.message import (
@@ -26,6 +27,17 @@ from services.message_service import MessageService
 logger = logging.getLogger(__name__)
 
 
+class MessageListQuery(BaseModel):
+    conversation_id: UUIDStrOrEmpty
+    first_id: UUIDStrOrEmpty | None = None
+    limit: int = Field(default=20, ge=1, le=100, description="Number of messages to return")
+
+
+class MessageFeedbackPayload(BaseModel):
+    rating: Literal["like", "dislike"] | None = Field(default=None, description="Feedback rating")
+    content: str | None = Field(default=None, description="Feedback content")
+
+
 class FeedbackListQuery(BaseModel):
     page: int = Field(default=1, ge=1, description="Page number")
     limit: int = Field(default=20, ge=1, le=101, description="Number of feedbacks per page")

api/controllers/service_api/app/workflow.py

@@ -1,5 +1,5 @@
 import logging
-from typing import Literal
+from typing import Any, Literal
 
 from dateutil.parser import isoparse
 from flask import request
@@ -11,7 +11,6 @@ from pydantic import BaseModel, Field
 from sqlalchemy.orm import sessionmaker
 from werkzeug.exceptions import BadRequest, InternalServerError, NotFound
 
-from controllers.common.controller_schemas import WorkflowRunPayload as WorkflowRunPayloadBase
 from controllers.common.schema import register_schema_models
 from controllers.service_api import service_api_ns
 from controllers.service_api.app.error import (
@@ -47,7 +46,9 @@ from services.workflow_app_service import WorkflowAppService
 logger = logging.getLogger(__name__)
 
 
-class WorkflowRunPayload(WorkflowRunPayloadBase):
+class WorkflowRunPayload(BaseModel):
+    inputs: dict[str, Any]
+    files: list[dict[str, Any]] | None = None
     response_mode: Literal["blocking", "streaming"] | None = None
 
 

api/controllers/service_api/dataset/document.py

@@ -10,7 +10,6 @@ from sqlalchemy import desc, func, select
 from werkzeug.exceptions import Forbidden, NotFound
 
 import services
-from controllers.common.controller_schemas import DocumentBatchDownloadZipPayload
 from controllers.common.errors import (
     FilenameNotExistsError,
     FileTooLargeError,
@@ -32,7 +31,6 @@ from controllers.service_api.wraps import (
     cloud_edition_billing_resource_check,
 )
 from core.errors.error import ProviderTokenNotInitError
-from core.rag.entities import PreProcessingRule, Rule, Segmentation
 from core.rag.retrieval.retrieval_methods import RetrievalMethod
 from extensions.ext_database import db
 from fields.document_fields import document_fields, document_status_fields
@@ -42,8 +40,11 @@ from models.enums import SegmentStatus
 from services.dataset_service import DatasetService, DocumentService
 from services.entities.knowledge_entities.knowledge_entities import (
     KnowledgeConfig,
+    PreProcessingRule,
     ProcessRule,
     RetrievalModel,
+    Rule,
+    Segmentation,
 )
 from services.file_service import FileService
 from services.summary_index_service import SummaryIndexService
@@ -101,6 +102,15 @@ class DocumentListQuery(BaseModel):
     status: str | None = Field(default=None, description="Document status filter")
 
 
+DOCUMENT_BATCH_DOWNLOAD_ZIP_MAX_DOCS = 100
+
+
+class DocumentBatchDownloadZipPayload(BaseModel):
+    """Request payload for bulk downloading uploaded documents as a ZIP archive."""
+
+    document_ids: list[UUID] = Field(..., min_length=1, max_length=DOCUMENT_BATCH_DOWNLOAD_ZIP_MAX_DOCS)
+
+
 register_enum_models(service_api_ns, RetrievalMethod)
 
 register_schema_models(
@@ -519,7 +529,7 @@ class DocumentListApi(DatasetApiResource):
         if not dataset:
             raise NotFound("Dataset not found.")
 
-        query = select(Document).where(Document.dataset_id == dataset_id, Document.tenant_id == tenant_id)
+        query = select(Document).filter_by(dataset_id=str(dataset_id), tenant_id=tenant_id)
 
         if query_params.status:
             query = DocumentService.apply_display_status_filter(query, query_params.status)

api/controllers/service_api/dataset/metadata.py

@@ -2,9 +2,9 @@ from typing import Literal
 
 from flask_login import current_user
 from flask_restx import marshal
+from pydantic import BaseModel
 from werkzeug.exceptions import NotFound
 
-from controllers.common.controller_schemas import MetadataUpdatePayload
 from controllers.common.schema import register_schema_model, register_schema_models
 from controllers.service_api import service_api_ns
 from controllers.service_api.wraps import DatasetApiResource, cloud_edition_billing_rate_limit_check
@@ -18,6 +18,11 @@ from services.entities.knowledge_entities.knowledge_entities import (
 )
 from services.metadata_service import MetadataService
 
+
+class MetadataUpdatePayload(BaseModel):
+    name: str
+
+
 register_schema_model(service_api_ns, MetadataUpdatePayload)
 register_schema_models(
     service_api_ns,

api/controllers/service_api/dataset/segment.py

@@ -8,7 +8,6 @@ from sqlalchemy import select
 from werkzeug.exceptions import NotFound
 
 from configs import dify_config
-from controllers.common.controller_schemas import ChildChunkCreatePayload, ChildChunkUpdatePayload
 from controllers.common.schema import register_schema_models
 from controllers.service_api import service_api_ns
 from controllers.service_api.app.error import ProviderNotInitializeError
@@ -33,25 +32,25 @@ from services.errors.chunk import ChildChunkIndexingError as ChildChunkIndexingS
 from services.summary_index_service import SummaryIndexService
 
 
-def _marshal_segment_with_summary(segment, dataset_id: str) -> dict[str, Any]:
+def _marshal_segment_with_summary(segment, dataset_id: str) -> dict:
     """Marshal a single segment and enrich it with summary content."""
-    segment_dict: dict[str, Any] = dict(marshal(segment, segment_fields))  # type: ignore[arg-type]
+    segment_dict = dict(marshal(segment, segment_fields))  # type: ignore[arg-type]
     summary = SummaryIndexService.get_segment_summary(segment_id=segment.id, dataset_id=dataset_id)
     segment_dict["summary"] = summary.summary_content if summary else None
     return segment_dict
 
 
-def _marshal_segments_with_summary(segments, dataset_id: str) -> list[dict[str, Any]]:
+def _marshal_segments_with_summary(segments, dataset_id: str) -> list[dict]:
     """Marshal multiple segments and enrich them with summary content (batch query)."""
     segment_ids = [segment.id for segment in segments]
-    summaries: dict[str, str | None] = {}
+    summaries: dict = {}
     if segment_ids:
         summary_records = SummaryIndexService.get_segments_summaries(segment_ids=segment_ids, dataset_id=dataset_id)
         summaries = {chunk_id: record.summary_content for chunk_id, record in summary_records.items()}
 
-    result: list[dict[str, Any]] = []
+    result = []
     for segment in segments:
-        segment_dict: dict[str, Any] = dict(marshal(segment, segment_fields))  # type: ignore[arg-type]
+        segment_dict = dict(marshal(segment, segment_fields))  # type: ignore[arg-type]
         segment_dict["summary"] = summaries.get(segment.id)
         result.append(segment_dict)
     return result
@@ -70,12 +69,20 @@ class SegmentUpdatePayload(BaseModel):
     segment: SegmentUpdateArgs
 
 
+class ChildChunkCreatePayload(BaseModel):
+    content: str
+
+
 class ChildChunkListQuery(BaseModel):
     limit: int = Field(default=20, ge=1)
     keyword: str | None = None
     page: int = Field(default=1, ge=1)
 
 
+class ChildChunkUpdatePayload(BaseModel):
+    content: str
+
+
 register_schema_models(
     service_api_ns,
     SegmentCreatePayload,

api/controllers/web/audio.py

@@ -3,11 +3,10 @@ import logging
 from flask import request
 from flask_restx import fields, marshal_with
 from graphon.model_runtime.errors.invoke import InvokeError
-from pydantic import field_validator
+from pydantic import BaseModel, field_validator
 from werkzeug.exceptions import InternalServerError
 
 import services
-from controllers.common.controller_schemas import TextToAudioPayload as TextToAudioPayloadBase
 from controllers.web import web_ns
 from controllers.web.error import (
     AppUnavailableError,
@@ -35,7 +34,12 @@ from services.errors.audio import (
 from ..common.schema import register_schema_models
 
 
-class TextToAudioPayload(TextToAudioPayloadBase):
+class TextToAudioPayload(BaseModel):
+    message_id: str | None = None
+    voice: str | None = None
+    text: str | None = None
+    streaming: bool | None = None
+
     @field_validator("message_id")
     @classmethod
     def validate_message_id(cls, value: str | None) -> str | None:

api/controllers/web/conversation.py

@@ -1,11 +1,10 @@
 from typing import Literal
 
 from flask import request
-from pydantic import BaseModel, Field, TypeAdapter, field_validator
+from pydantic import BaseModel, Field, TypeAdapter, field_validator, model_validator
 from sqlalchemy.orm import sessionmaker
 from werkzeug.exceptions import NotFound
 
-from controllers.common.controller_schemas import ConversationRenamePayload
 from controllers.common.schema import register_schema_models
 from controllers.web import web_ns
 from controllers.web.error import NotChatAppError
@@ -38,6 +37,18 @@ class ConversationListQuery(BaseModel):
         return uuid_value(value)
 
 
+class ConversationRenamePayload(BaseModel):
+    name: str | None = None
+    auto_generate: bool = False
+
+    @model_validator(mode="after")
+    def validate_name_requirement(self):
+        if not self.auto_generate:
+            if self.name is None or not self.name.strip():
+                raise ValueError("name is required when auto_generate is false")
+        return self
+
+
 register_schema_models(web_ns, ConversationListQuery, ConversationRenamePayload)
 
 

api/controllers/web/forgot_password.py

@@ -3,6 +3,8 @@ import secrets
 
 from flask import request
 from flask_restx import Resource
+from pydantic import BaseModel, Field, field_validator
+from sqlalchemy.orm import sessionmaker
 
 from controllers.common.schema import register_schema_models
 from controllers.console.auth.error import (
@@ -17,15 +19,33 @@ from controllers.console.error import EmailSendIpLimitError
 from controllers.console.wraps import email_password_login_enabled, only_edition_enterprise, setup_required
 from controllers.web import web_ns
 from extensions.ext_database import db
-from libs.helper import extract_remote_ip
-from libs.password import hash_password
+from libs.helper import EmailStr, extract_remote_ip
+from libs.password import hash_password, valid_password
 from models.account import Account
 from services.account_service import AccountService
-from services.entities.auth_entities import (
-    ForgotPasswordCheckPayload,
-    ForgotPasswordResetPayload,
-    ForgotPasswordSendPayload,
-)
+
+
+class ForgotPasswordSendPayload(BaseModel):
+    email: EmailStr
+    language: str | None = None
+
+
+class ForgotPasswordCheckPayload(BaseModel):
+    email: EmailStr
+    code: str
+    token: str = Field(min_length=1)
+
+
+class ForgotPasswordResetPayload(BaseModel):
+    token: str = Field(min_length=1)
+    new_password: str
+    password_confirm: str
+
+    @field_validator("new_password", "password_confirm")
+    @classmethod
+    def validate_password(cls, value: str) -> str:
+        return valid_password(value)
+
 
 register_schema_models(web_ns, ForgotPasswordSendPayload, ForgotPasswordCheckPayload, ForgotPasswordResetPayload)
 
@@ -61,7 +81,9 @@ class ForgotPasswordSendEmailApi(Resource):
         else:
             language = "en-US"
 
-        account = AccountService.get_account_by_email_with_case_fallback(request_email)
+        with sessionmaker(db.engine).begin() as session:
+            account = AccountService.get_account_by_email_with_case_fallback(request_email, session=session)
+        token = None
         if account is None:
             raise AuthenticationFailedError()
         else:
@@ -158,14 +180,13 @@ class ForgotPasswordResetApi(Resource):
 
         email = reset_data.get("email", "")
 
-        account = AccountService.get_account_by_email_with_case_fallback(email)
+        with sessionmaker(db.engine).begin() as session:
+            account = AccountService.get_account_by_email_with_case_fallback(email, session=session)
 
-        if account:
-            account = db.session.merge(account)
-            self._update_existing_account(account, password_hashed, salt)
-            db.session.commit()
-        else:
-            raise AuthenticationFailedError()
+            if account:
+                self._update_existing_account(account, password_hashed, salt)
+            else:
+                raise AuthenticationFailedError()
 
         return {"result": "success"}
 

api/controllers/web/human_input_form.py

@@ -5,11 +5,9 @@ Web App Human Input Form APIs.
 import json
 import logging
 from datetime import datetime
-from typing import Any, NotRequired, TypedDict
 
 from flask import Response, request
-from flask_restx import Resource
-from pydantic import BaseModel
+from flask_restx import Resource, reqparse
 from sqlalchemy import select
 from werkzeug.exceptions import Forbidden
 
@@ -25,12 +23,6 @@ from services.human_input_service import Form, FormNotFoundError, HumanInputServ
 
 logger = logging.getLogger(__name__)
 
-
-class HumanInputFormSubmitPayload(BaseModel):
-    inputs: dict
-    action: str
-
-
 _FORM_SUBMIT_RATE_LIMITER = RateLimiter(
     prefix="web_form_submit_rate_limit",
     max_attempts=dify_config.WEB_FORM_SUBMIT_RATE_LIMIT_MAX_ATTEMPTS,
@@ -59,19 +51,10 @@ def _to_timestamp(value: datetime) -> int:
     return int(value.timestamp())
 
 
-class FormDefinitionPayload(TypedDict):
-    form_content: Any
-    inputs: Any
-    resolved_default_values: dict[str, str]
-    user_actions: Any
-    expiration_time: int
-    site: NotRequired[dict]
-
-
 def _jsonify_form_definition(form: Form, site_payload: dict | None = None) -> Response:
     """Return the form payload (optionally with site) as a JSON response."""
     definition_payload = form.get_definition().model_dump()
-    payload: FormDefinitionPayload = {
+    payload = {
         "form_content": definition_payload["rendered_content"],
         "inputs": definition_payload["inputs"],
         "resolved_default_values": _stringify_default_values(definition_payload["default_values"]),
@@ -102,7 +85,7 @@ class HumanInputFormApi(Resource):
         _FORM_ACCESS_RATE_LIMITER.increment_rate_limit(ip_address)
 
         service = HumanInputService(db.engine)
-        # TODO(QuantumGhost): forbid submission for form tokens
+        # TODO(QuantumGhost): forbid submision for form tokens
         # that are only for console.
         form = service.get_form_by_token(form_token)
 
@@ -129,7 +112,10 @@ class HumanInputFormApi(Resource):
             "action": "Approve"
         }
         """
-        payload = HumanInputFormSubmitPayload.model_validate(request.get_json())
+        parser = reqparse.RequestParser()
+        parser.add_argument("inputs", type=dict, required=True, location="json")
+        parser.add_argument("action", type=str, required=True, location="json")
+        args = parser.parse_args()
 
         ip_address = extract_remote_ip(request)
         if _FORM_SUBMIT_RATE_LIMITER.is_rate_limited(ip_address):
@@ -149,8 +135,8 @@ class HumanInputFormApi(Resource):
             service.submit_form_by_token(
                 recipient_type=recipient_type,
                 form_token=form_token,
-                selected_action_id=payload.action,
-                form_data=payload.inputs,
+                selected_action_id=args["action"],
+                form_data=args["inputs"],
                 submission_end_user_id=None,
                 # submission_end_user_id=_end_user.id,
             )

api/controllers/web/login.py

@@ -1,10 +1,7 @@
-import logging
-
 from flask import make_response, request
 from flask_restx import Resource
 from jwt import InvalidTokenError
 from pydantic import BaseModel, Field, field_validator
-from werkzeug.exceptions import Unauthorized
 
 import services
 from configs import dify_config
@@ -23,7 +20,7 @@ from controllers.console.wraps import (
 )
 from controllers.web import web_ns
 from controllers.web.wraps import decode_jwt_token
-from libs.helper import EmailStr, extract_remote_ip
+from libs.helper import EmailStr
 from libs.passport import PassportService
 from libs.password import valid_password
 from libs.token import (
@@ -32,13 +29,13 @@ from libs.token import (
 )
 from services.account_service import AccountService
 from services.app_service import AppService
-from services.entities.auth_entities import LoginFailureReason, LoginPayloadBase
 from services.webapp_auth_service import WebAppAuthService
 
-logger = logging.getLogger(__name__)
 
+class LoginPayload(BaseModel):
+    email: EmailStr
+    password: str
 
-class LoginPayload(LoginPayloadBase):
     @field_validator("password")
     @classmethod
     def validate_password(cls, value: str) -> str:
@@ -81,18 +78,14 @@ class LoginApi(Resource):
     def post(self):
         """Authenticate user and login."""
         payload = LoginPayload.model_validate(web_ns.payload or {})
-        normalized_email = payload.email.lower()
 
         try:
             account = WebAppAuthService.authenticate(payload.email, payload.password)
         except services.errors.account.AccountLoginError:
-            _log_web_login_failure(email=normalized_email, reason=LoginFailureReason.ACCOUNT_BANNED)
             raise AccountBannedError()
         except services.errors.account.AccountPasswordError:
-            _log_web_login_failure(email=normalized_email, reason=LoginFailureReason.INVALID_CREDENTIALS)
             raise AuthenticationFailedError()
         except services.errors.account.AccountNotFoundError:
-            _log_web_login_failure(email=normalized_email, reason=LoginFailureReason.ACCOUNT_NOT_FOUND)
             raise AuthenticationFailedError()
 
         token = WebAppAuthService.login(account=account)
@@ -221,30 +214,21 @@ class EmailCodeLoginApi(Resource):
 
         token_data = WebAppAuthService.get_email_code_login_data(payload.token)
         if token_data is None:
-            _log_web_login_failure(email=user_email, reason=LoginFailureReason.INVALID_EMAIL_CODE_TOKEN)
             raise InvalidTokenError()
 
         token_email = token_data.get("email")
         if not isinstance(token_email, str):
-            _log_web_login_failure(email=user_email, reason=LoginFailureReason.EMAIL_CODE_EMAIL_MISMATCH)
             raise InvalidEmailError()
         normalized_token_email = token_email.lower()
         if normalized_token_email != user_email:
-            _log_web_login_failure(email=user_email, reason=LoginFailureReason.EMAIL_CODE_EMAIL_MISMATCH)
             raise InvalidEmailError()
 
         if token_data["code"] != payload.code:
-            _log_web_login_failure(email=user_email, reason=LoginFailureReason.INVALID_EMAIL_CODE)
             raise EmailCodeError()
 
         WebAppAuthService.revoke_email_code_login_token(payload.token)
-        try:
-            account = WebAppAuthService.get_user_through_email(token_email)
-        except Unauthorized as exc:
-            _log_web_login_failure(email=user_email, reason=LoginFailureReason.ACCOUNT_BANNED)
-            raise AccountBannedError() from exc
+        account = WebAppAuthService.get_user_through_email(token_email)
         if not account:
-            _log_web_login_failure(email=user_email, reason=LoginFailureReason.ACCOUNT_NOT_FOUND)
             raise AuthenticationFailedError()
 
         token = WebAppAuthService.login(account=account)
@@ -252,12 +236,3 @@ class EmailCodeLoginApi(Resource):
         response = make_response({"result": "success", "data": {"access_token": token}})
         # set_access_token_to_cookie(request, response, token, samesite="None", httponly=False)
         return response
-
-
-def _log_web_login_failure(*, email: str, reason: LoginFailureReason) -> None:
-    logger.warning(
-        "Web login failed: email=%s reason=%s ip_address=%s",
-        email,
-        reason,
-        extract_remote_ip(request),
-    )

api/controllers/web/message.py

@@ -3,10 +3,9 @@ from typing import Literal
 
 from flask import request
 from graphon.model_runtime.errors.invoke import InvokeError
-from pydantic import BaseModel, Field, TypeAdapter
+from pydantic import BaseModel, Field, TypeAdapter, field_validator
 from werkzeug.exceptions import InternalServerError, NotFound
 
-from controllers.common.controller_schemas import MessageFeedbackPayload, MessageListQuery
 from controllers.common.schema import register_schema_models
 from controllers.web import web_ns
 from controllers.web.error import (
@@ -25,6 +24,7 @@ from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotIni
 from fields.conversation_fields import ResultResponse
 from fields.message_fields import SuggestedQuestionsResponse, WebMessageInfiniteScrollPagination, WebMessageListItem
 from libs import helper
+from libs.helper import uuid_value
 from models.enums import FeedbackRating
 from models.model import AppMode
 from services.app_generate_service import AppGenerateService
@@ -40,6 +40,24 @@ from services.message_service import MessageService
 logger = logging.getLogger(__name__)
 
 
+class MessageListQuery(BaseModel):
+    conversation_id: str = Field(description="Conversation UUID")
+    first_id: str | None = Field(default=None, description="First message ID for pagination")
+    limit: int = Field(default=20, ge=1, le=100, description="Number of messages to return (1-100)")
+
+    @field_validator("conversation_id", "first_id")
+    @classmethod
+    def validate_uuid(cls, value: str | None) -> str | None:
+        if value is None:
+            return value
+        return uuid_value(value)
+
+
+class MessageFeedbackPayload(BaseModel):
+    rating: Literal["like", "dislike"] | None = Field(default=None, description="Feedback rating")
+    content: str | None = Field(default=None, description="Feedback content")
+
+
 class MessageMoreLikeThisQuery(BaseModel):
     response_mode: Literal["blocking", "streaming"] = Field(
         description="Response mode",

api/controllers/web/passport.py

@@ -1,6 +1,5 @@
 import uuid
 from datetime import UTC, datetime, timedelta
-from typing import Any
 
 from flask import make_response, request
 from flask_restx import Resource
@@ -104,23 +103,21 @@ class PassportResource(Resource):
         return response
 
 
-def decode_enterprise_webapp_user_id(jwt_token: str | None) -> dict[str, Any] | None:
+def decode_enterprise_webapp_user_id(jwt_token: str | None):
     """
     Decode the enterprise user session from the Authorization header.
     """
     if not jwt_token:
         return None
 
-    decoded: dict[str, Any] = PassportService().verify(jwt_token)
+    decoded = PassportService().verify(jwt_token)
     source = decoded.get("token_source")
     if not source or source != "webapp_login_token":
         raise Unauthorized("Invalid token source. Expected 'webapp_login_token'.")
     return decoded
 
 
-def exchange_token_for_existing_web_user(
-    app_code: str, enterprise_user_decoded: dict[str, Any], auth_type: WebAppAuthType
-):
+def exchange_token_for_existing_web_user(app_code: str, enterprise_user_decoded: dict, auth_type: WebAppAuthType):
     """
     Exchange a token for an existing web user session.
     """
@@ -141,15 +138,12 @@ def exchange_token_for_existing_web_user(
     if not app_model or app_model.status != "normal" or not app_model.enable_site:
         raise NotFound()
 
-    match auth_type:
-        case WebAppAuthType.PUBLIC:
-            return _exchange_for_public_app_token(app_model, site, enterprise_user_decoded)
-        case WebAppAuthType.EXTERNAL:
-            if user_auth_type != "external":
-                raise WebAppAuthRequiredError("Please login as external user.")
-        case WebAppAuthType.INTERNAL:
-            if user_auth_type != "internal":
-                raise WebAppAuthRequiredError("Please login as internal user.")
+    if auth_type == WebAppAuthType.PUBLIC:
+        return _exchange_for_public_app_token(app_model, site, enterprise_user_decoded)
+    elif auth_type == WebAppAuthType.EXTERNAL and user_auth_type != "external":
+        raise WebAppAuthRequiredError("Please login as external user.")
+    elif auth_type == WebAppAuthType.INTERNAL and user_auth_type != "internal":
+        raise WebAppAuthRequiredError("Please login as internal user.")
 
     end_user = None
     if end_user_id:

api/controllers/web/saved_message.py

@@ -1,17 +1,27 @@
 from flask import request
-from pydantic import TypeAdapter
+from pydantic import BaseModel, Field, TypeAdapter
 from werkzeug.exceptions import NotFound
 
-from controllers.common.controller_schemas import SavedMessageCreatePayload, SavedMessageListQuery
 from controllers.common.schema import register_schema_models
 from controllers.web import web_ns
 from controllers.web.error import NotCompletionAppError
 from controllers.web.wraps import WebApiResource
 from fields.conversation_fields import ResultResponse
 from fields.message_fields import SavedMessageInfiniteScrollPagination, SavedMessageItem
+from libs.helper import UUIDStrOrEmpty
 from services.errors.message import MessageNotExistsError
 from services.saved_message_service import SavedMessageService
 
+
+class SavedMessageListQuery(BaseModel):
+    last_id: UUIDStrOrEmpty | None = None
+    limit: int = Field(default=20, ge=1, le=100)
+
+
+class SavedMessageCreatePayload(BaseModel):
+    message_id: UUIDStrOrEmpty
+
+
 register_schema_models(web_ns, SavedMessageListQuery, SavedMessageCreatePayload)
 
 

api/controllers/web/site.py

@@ -1,4 +1,4 @@
-from typing import Any, cast
+from typing import cast
 
 from flask_restx import fields, marshal, marshal_with
 from sqlalchemy import select
@@ -113,12 +113,12 @@ class AppSiteInfo:
             }
 
 
-def serialize_site(site: Site) -> dict[str, Any]:
+def serialize_site(site: Site) -> dict:
     """Serialize Site model using the same schema as AppSiteApi."""
-    return cast(dict[str, Any], marshal(site, AppSiteApi.site_fields))
+    return cast(dict, marshal(site, AppSiteApi.site_fields))
 
 
-def serialize_app_site_payload(app_model: App, site: Site, end_user_id: str | None) -> dict[str, Any]:
+def serialize_app_site_payload(app_model: App, site: Site, end_user_id: str | None) -> dict:
     can_replace_logo = FeatureService.get_features(app_model.tenant_id).can_replace_logo
     app_site_info = AppSiteInfo(app_model.tenant, app_model, site, end_user_id, can_replace_logo)
-    return cast(dict[str, Any], marshal(app_site_info, AppSiteApi.app_fields))
+    return cast(dict, marshal(app_site_info, AppSiteApi.app_fields))

api/controllers/web/workflow.py

@@ -1,10 +1,11 @@
 import logging
+from typing import Any
 
 from graphon.graph_engine.manager import GraphEngineManager
 from graphon.model_runtime.errors.invoke import InvokeError
+from pydantic import BaseModel, Field
 from werkzeug.exceptions import InternalServerError
 
-from controllers.common.controller_schemas import WorkflowRunPayload
 from controllers.common.schema import register_schema_models
 from controllers.web import web_ns
 from controllers.web.error import (
@@ -29,6 +30,12 @@ from models.model import App, AppMode, EndUser
 from services.app_generate_service import AppGenerateService
 from services.errors.llm import InvokeRateLimitError
 
+
+class WorkflowRunPayload(BaseModel):
+    inputs: dict[str, Any] = Field(description="Input variables for the workflow")
+    files: list[dict[str, Any]] | None = Field(default=None, description="Files to be processed by the workflow")
+
+
 logger = logging.getLogger(__name__)
 
 register_schema_models(web_ns, WorkflowRunPayload)