GareArc
8c3ef87f90
fix(otel): resolve OTEL-related type errors
...
- Add trace_manager parameter to _handle_advanced_chat_message_end_event
- Add missing _lookup_app_and_workspace_names helper function
- Import Tenant model for workspace name lookup
2026-03-01 01:08:38 -08:00
GareArc
d0a373d534
feat(telemetry): add missing OTEL infrastructure files
...
- Add core telemetry events module
- Add enterprise telemetry contracts, draft trace, ID generator
- Add enterprise telemetry tasks and event handlers
- Add OTEL configuration (app_config, enterprise config)
- Add OTEL semantic conventions
- Add feedback events and app event updates
- Add comprehensive test coverage for all modules
2026-03-01 00:47:37 -08:00
GareArc
81f44c746f
feat(telemetry): add missing ID fields for name attributes
...
- Add dify.credential.id to node execution events
- Add dify.event.id to all telemetry events (APP_CREATED, APP_UPDATED, APP_DELETED, FEEDBACK_CREATED)
This ensures all .name fields have corresponding .id fields for reliable aggregation and deduplication.
2026-03-01 00:45:30 -08:00
GareArc
20b36d984c
fix(telemetry): add resolved_trace_id property to eliminate trace_id inconsistencies
...
Add computed property to BaseTraceInfo that provides intelligent fallback:
1. External trace_id (from X-Trace-Id header)
2. workflow_run_id (for workflow-related traces)
3. message_id (as final fallback)
This ensures attribute dify.trace_id always matches log-level trace_id,
eliminating inconsistencies where attribute was null but log-level had value.
Changes:
- Add resolved_trace_id property to BaseTraceInfo (trace_entity.py)
- Replace 4 direct trace_id attribute assignments with resolved_trace_id
- Add trace_id_source parameter to 5 emit_metric_only_event calls
Fixes trace_id inconsistency found in MESSAGE_RUN, TOOL_EXECUTION,
MODERATION_CHECK, SUGGESTED_QUESTION_GENERATION, GENERATE_NAME_EXECUTION,
DATASET_RETRIEVAL, and PROMPT_GENERATION_EXECUTION events.
All 78 telemetry tests passing.
2026-03-01 00:44:36 -08:00
GareArc
5e8226fddd
refactor(telemetry): move gateway to core as stateless module-level functions
...
Move routing table, emit(), and is_enterprise_telemetry_enabled() from
enterprise/telemetry/gateway.py into core/telemetry/gateway.py so both
CE and EE share one code path. The ce_eligible flag in CASE_ROUTING
controls which events flow in CE — flipping it is the only change needed
to enable an event in community edition.
- Delete enterprise/telemetry/gateway.py (class-based singleton)
- Create core/telemetry/gateway.py (stateless functions, no shared state)
- Simplify core/telemetry/__init__.py to thin facade over gateway
- Remove TelemetryGateway class and get_gateway() from ext_enterprise_telemetry
- Single-source is_enterprise_telemetry_enabled in core.telemetry.gateway
- Fix pre-existing test bugs (missing dify.event.id in metric handler tests)
- Update all imports and mock paths across 7 test files
2026-03-01 00:44:36 -08:00
GareArc
e1e599efc1
feat(telemetry): add model provider and name tags to all trace metrics
...
Add comprehensive model tracking across all OTEL metrics and logs:
- Node execution metrics now include model_name for LLM operations
- Suggested question metrics include model_provider and model_name
- Dataset retrieval captures both embedding and rerank model info
- Updated DATA_DICTIONARY.md with complete metric label documentation
This enables granular cost tracking, performance analysis, and usage monitoring per model across all operation types.
2026-03-01 00:44:22 -08:00
GareArc
366a889977
docs(enterprise): split telemetry docs into README and data dictionary
...
Separate background/configuration instructions from the data dictionary:
- README.md: Overview, configuration, correlation model, content gating
- DATA_DICTIONARY.md: Pure reference format with signals and attributes
The data dictionary is now concise (465 lines vs 911) and focuses on
attribute types and relationships without verbose explanations.
2026-03-01 00:44:10 -08:00
GareArc
e6a68123c7
docs(enterprise): add telemetry data dictionary for OTEL signals
...
- Comprehensive reference for all enterprise telemetry signals
- Documents 3 span types, 10 counters, 6 histograms, 13 log events
- Includes trace correlation model with ASCII diagrams
- Configuration reference for all 8 ENTERPRISE_* variables
- Per-emission-site label tables for metrics
- Full JSON schemas for structured log events
- Content gating behavior and token double-counting warnings
2026-03-01 00:44:10 -08:00
GareArc
f55a34c4a5
feat(enterprise-telemetry): wire bearer token auth and configurable insecure flag into OTEL exporter
2026-03-01 00:44:04 -08:00
GareArc
99722f3088
feat(telemetry): unify token metric label structure with Pydantic enforcement
...
- Add TokenMetricLabels BaseModel to enforce consistent label structure
- All dify.token.* metrics now use identical 6-label structure:
* tenant_id, app_id, operation_type, model_provider, model_name, node_type
- Pydantic validation ensures runtime enforcement (extra='forbid', frozen=True)
- Enables filtering by operation_type to avoid double-counting:
* workflow: aggregated workflow-level tokens
* node_execution: individual node-level tokens
* message: direct message tokens
* rule_generate/code_generate: prompt generation tokens
Previously, inconsistent label cardinality made aggregation impossible:
- WORKFLOW: 3 labels
- NODE_EXECUTION: 6 labels
- MESSAGE: 5 labels
- PROMPT_GENERATION: 5 labels
Now all use the same 6-label structure for consistent querying.
2026-03-01 00:44:04 -08:00
GareArc
85391d93d3
feat: add dedicated app event counters and convert event names to StrEnum
...
- Add APP_CREATED, APP_UPDATED, APP_DELETED counters to EnterpriseTelemetryCounter
- Create EnterpriseTelemetryEvent StrEnum for type-safe event names
- Update metric_handler to use new app-specific counters with labels (tenant_id, app_id, mode)
- Convert all event_name strings to EnterpriseTelemetryEvent enum values
- Update exporter to create OTEL meters for new app counters (dify.app.created.total, etc.)
- Update tests to verify new counter behavior and enum usage
2026-03-01 00:44:03 -08:00
GareArc
eb9f1c9780
feat(telemetry): add operation_type labels for token metrics
...
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode )
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai >
2026-03-01 00:44:03 -08:00
GareArc
3daa6ea2f5
docs(telemetry): clarify enterprise_telemetry queue is EE-only
2026-03-01 00:43:46 -08:00
GareArc
945d01dd08
feat(telemetry): add enterprise OTEL telemetry with gateway, traces, metrics, and logs
2026-03-01 00:43:46 -08:00
99
00e52796e6
refactor(workflow): remove code node helper imports ( #32759 )
...
Co-authored-by: -LAN- <laipz8200@outlook.com >
2026-03-01 16:31:45 +08:00
tda
4b8a02cf25
fix: add return type annotation to auth decorators ( #32699 )
...
Co-authored-by: root <root@DESKTOP-KQLO90N>
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai >
2026-03-01 17:19:33 +09:00
Br1an
cfdf16c49e
fix(api): resolve type errors in BaseTraceInstance and OpsTraceManager ( #32773 )
2026-03-01 17:14:37 +09:00
Br1an
d4c508cf8e
fix(api): add explicit return type annotations to clean() methods ( #32772 )
2026-03-01 17:09:43 +09:00
99
9e9e617e09
fix(workflow): decouple http request node external dependencies ( #32762 )
2026-03-01 15:42:57 +08:00
FFXN
a7789f2c91
fix: some Qwen3 models only support streaming output. ( #32766 )
...
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-03-01 15:42:44 +08:00
FFXN
de4dac89ae
fix: Fixed the adaptation issue of response formats for different mod… ( #32326 )
autofix.ci / autofix (push) Has been cancelled
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/amd64, build-api-amd64) (push) Has been cancelled
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/arm64, build-api-arm64) (push) Has been cancelled
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/amd64, build-web-amd64) (push) Has been cancelled
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/arm64, build-web-arm64) (push) Has been cancelled
Build and Push API & Web / create-manifest (api, DIFY_API_IMAGE_NAME, merge-api-images) (push) Has been cancelled
Build and Push API & Web / create-manifest (web, DIFY_WEB_IMAGE_NAME, merge-web-images) (push) Has been cancelled
Main CI Pipeline / Check Changed Files (push) Has been cancelled
Main CI Pipeline / API Tests (push) Has been cancelled
Main CI Pipeline / Web Tests (push) Has been cancelled
Main CI Pipeline / Style Check (push) Has been cancelled
Main CI Pipeline / VDB Tests (push) Has been cancelled
Main CI Pipeline / DB Migration Test (push) Has been cancelled
2026-03-01 10:23:17 +08:00
-LAN-
20fcc95db9
test(api): add autospec to MagicMock-based patch usage ( #32752 )
autofix.ci / autofix (push) Has been cancelled
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/amd64, build-api-amd64) (push) Has been cancelled
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/arm64, build-api-arm64) (push) Has been cancelled
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/amd64, build-web-amd64) (push) Has been cancelled
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/arm64, build-web-arm64) (push) Has been cancelled
Build and Push API & Web / create-manifest (api, DIFY_API_IMAGE_NAME, merge-api-images) (push) Has been cancelled
Build and Push API & Web / create-manifest (web, DIFY_WEB_IMAGE_NAME, merge-web-images) (push) Has been cancelled
Main CI Pipeline / Check Changed Files (push) Has been cancelled
Main CI Pipeline / API Tests (push) Has been cancelled
Main CI Pipeline / Web Tests (push) Has been cancelled
Main CI Pipeline / Style Check (push) Has been cancelled
Main CI Pipeline / VDB Tests (push) Has been cancelled
Main CI Pipeline / DB Migration Test (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
2026-03-01 04:30:45 +08:00
-LAN-
c034eb036c
refactor: inject memory interface into LLMNode ( #32754 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-03-01 04:05:18 +08:00
-LAN-
1f0fca89a8
refactor(workflow): move variables package into core.workflow ( #32750 )
2026-03-01 03:15:09 +08:00
-LAN-
962df17a15
refactor: consolidate LLM runtime model state on ModelInstance ( #32746 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
2026-03-01 02:29:32 +08:00
dependabot[bot]
48d8667c4f
chore(deps): bump pypdf from 6.7.1 to 6.7.4 in /api ( #32736 )
...
autofix.ci / autofix (push) Has been cancelled
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/amd64, build-api-amd64) (push) Has been cancelled
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/arm64, build-api-arm64) (push) Has been cancelled
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/amd64, build-web-amd64) (push) Has been cancelled
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/arm64, build-web-arm64) (push) Has been cancelled
Build and Push API & Web / create-manifest (api, DIFY_API_IMAGE_NAME, merge-api-images) (push) Has been cancelled
Build and Push API & Web / create-manifest (web, DIFY_WEB_IMAGE_NAME, merge-web-images) (push) Has been cancelled
Main CI Pipeline / Check Changed Files (push) Has been cancelled
Main CI Pipeline / API Tests (push) Has been cancelled
Main CI Pipeline / Web Tests (push) Has been cancelled
Main CI Pipeline / Style Check (push) Has been cancelled
Main CI Pipeline / VDB Tests (push) Has been cancelled
Main CI Pipeline / DB Migration Test (push) Has been cancelled
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-28 16:42:03 +09:00
Tyson Cung
91dfdd87e3
fix: replace unreachable yield expression with yield from () ( #32727 )
2026-02-28 15:27:32 +09:00
木之本澪
33242697ce
test: migrate document_service_status SQL tests to testcontainers ( #32536 )
...
autofix.ci / autofix (push) Has been cancelled
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/amd64, build-api-amd64) (push) Has been cancelled
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/arm64, build-api-arm64) (push) Has been cancelled
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/amd64, build-web-amd64) (push) Has been cancelled
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/arm64, build-web-arm64) (push) Has been cancelled
Build and Push API & Web / create-manifest (api, DIFY_API_IMAGE_NAME, merge-api-images) (push) Has been cancelled
Build and Push API & Web / create-manifest (web, DIFY_WEB_IMAGE_NAME, merge-web-images) (push) Has been cancelled
Main CI Pipeline / Check Changed Files (push) Has been cancelled
Main CI Pipeline / API Tests (push) Has been cancelled
Main CI Pipeline / Web Tests (push) Has been cancelled
Main CI Pipeline / Style Check (push) Has been cancelled
Main CI Pipeline / VDB Tests (push) Has been cancelled
Main CI Pipeline / DB Migration Test (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
Co-authored-by: KinomotoMio <200703522+KinomotoMio@users.noreply.github.com >
2026-02-28 01:50:55 +09:00
木之本澪
ad600f0827
test: migrate test_dataset_service SQL tests to testcontainers ( #32535 )
...
Co-authored-by: KinomotoMio <200703522+KinomotoMio@users.noreply.github.com >
2026-02-27 22:40:20 +09:00
tda
71ff135927
fix: add return type to abstract _publish method ( #32701 )
...
Co-authored-by: root <root@DESKTOP-KQLO90N>
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai >
2026-02-27 21:52:49 +09:00
木之本澪
f9196f7bea
test: migrate document_indexing_sync_task SQL tests to testcontainers ( #32534 )
...
Co-authored-by: KinomotoMio <200703522+KinomotoMio@users.noreply.github.com >
2026-02-27 21:36:32 +09:00
Varun Chawla
233e12e631
fix: correct mock return type in CodeBasedExtension test ( #32058 )
2026-02-27 20:40:51 +09:00
wangxiaolei
eccb67d5b6
refactor: decouple the business logic from datasource_node ( #32515 )
...
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-02-27 18:49:14 +08:00
-LAN-
1e6de0e6ad
docs(api): simplify setup README and worker guidance ( #32704 )
2026-02-27 18:12:52 +08:00
dependabot[bot]
6c66e11cac
chore(deps-dev): bump nltk from 3.9.2 to 3.9.3 in /api ( #32691 )
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-27 17:20:55 +09:00
-LAN-
149a7870bc
test: align file preview mimetype expectation ( #32688 )
2026-02-27 15:27:30 +08:00
-LAN-
a694533fc9
refactor(workflow): inject credential/model access ports into LLM nodes ( #32569 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com >
2026-02-27 14:36:41 +08:00
-LAN-
d20880d102
revert: "fix: image preview triggers binary download" ( #32683 )
2026-02-27 14:28:30 +08:00
-LAN-
eea1cf17ef
refactor(workflow): inject redis into graph engine manager ( #32622 )
2026-02-27 13:29:52 +08:00
-LAN-
700a4029c6
refactor(api): inject code executor from node factory ( #32618 )
2026-02-27 13:29:00 +08:00
不做了睡大觉
349d2d8e4e
fix: replace deprecated SpanAttributes and ResourceAttributes with new semconv imports ( #32661 )
...
Co-authored-by: User <user@example.com >
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-02-27 08:53:45 +09:00
edvatar
2eefb585f9
fix: add type annotations to BaseStorage.exists and BaseStorage.download ( #32652 )
...
Signed-off-by: edvatar <88481784+toroleapinc@users.noreply.github.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-02-27 07:35:30 +09:00
木之本澪
5cb1b53b47
test: migrate dataset service update-dataset SQL tests to testcontainers ( #32533 )
...
Co-authored-by: KinomotoMio <200703522+KinomotoMio@users.noreply.github.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-02-27 07:10:15 +09:00
edvatar
b48f36a4e5
fix: replace dict() merge with dict unpacking to resolve overload error ( #32653 )
...
Signed-off-by: edvatar <88481784+toroleapinc@users.noreply.github.com >
2026-02-27 06:15:17 +09:00
木之本澪
0bf5f4df3b
test: migrate dataset_indexing_task SQL tests to testcontainers ( #32531 )
...
Co-authored-by: KinomotoMio <200703522+KinomotoMio@users.noreply.github.com >
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-02-27 06:06:42 +09:00
木之本澪
56759c03b7
test: migrate clean_dataset_task SQL tests to testcontainers ( #32529 )
...
autofix.ci / autofix (push) Has been cancelled
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/amd64, build-api-amd64) (push) Has been cancelled
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/arm64, build-api-arm64) (push) Has been cancelled
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/amd64, build-web-amd64) (push) Has been cancelled
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/arm64, build-web-arm64) (push) Has been cancelled
Build and Push API & Web / create-manifest (api, DIFY_API_IMAGE_NAME, merge-api-images) (push) Has been cancelled
Build and Push API & Web / create-manifest (web, DIFY_WEB_IMAGE_NAME, merge-web-images) (push) Has been cancelled
Main CI Pipeline / Check Changed Files (push) Has been cancelled
Main CI Pipeline / API Tests (push) Has been cancelled
Main CI Pipeline / Web Tests (push) Has been cancelled
Main CI Pipeline / Style Check (push) Has been cancelled
Main CI Pipeline / VDB Tests (push) Has been cancelled
Main CI Pipeline / DB Migration Test (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
Co-authored-by: KinomotoMio <200703522+KinomotoMio@users.noreply.github.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-02-26 18:59:36 +09:00
不做了睡大觉
cec6d82650
fix: add None checks for tenant.id in dataset vector index tests ( #32603 )
...
Co-authored-by: User <user@example.com >
2026-02-26 17:15:45 +09:00
Asuka Minato
33e0dae2b2
ci: try from main repo ( #32620 )
...
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-02-26 16:30:10 +09:00
木之本澪
5d927b413f
test: migrate workflow_node_execution_service_repository SQL tests to testcontainers ( #32591 )
...
autofix.ci / autofix (push) Has been cancelled
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/amd64, build-api-amd64) (push) Has been cancelled
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/arm64, build-api-arm64) (push) Has been cancelled
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/amd64, build-web-amd64) (push) Has been cancelled
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/arm64, build-web-arm64) (push) Has been cancelled
Build and Push API & Web / create-manifest (api, DIFY_API_IMAGE_NAME, merge-api-images) (push) Has been cancelled
Build and Push API & Web / create-manifest (web, DIFY_WEB_IMAGE_NAME, merge-web-images) (push) Has been cancelled
Main CI Pipeline / Check Changed Files (push) Has been cancelled
Main CI Pipeline / API Tests (push) Has been cancelled
Main CI Pipeline / Web Tests (push) Has been cancelled
Main CI Pipeline / Style Check (push) Has been cancelled
Main CI Pipeline / VDB Tests (push) Has been cancelled
Main CI Pipeline / DB Migration Test (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
Co-authored-by: KinomotoMio <200703522+KinomotoMio@users.noreply.github.com >
2026-02-26 03:42:08 +09:00
木之本澪
39de931555
test: migrate restore_archived_workflow_run SQL tests to testcontainers ( #32590 )
...
Co-authored-by: KinomotoMio <200703522+KinomotoMio@users.noreply.github.com >
2026-02-26 03:24:58 +09:00
