Commit Graph

83 Commits

Author SHA1 Message Date
jyong
efce1b04e0 fix style check 2025-09-17 22:34:11 +08:00
QuantumGhost
04919195cc Merge remote-tracking branch 'upstream/feat/queue-based-graph-engine' into feat/rag-2 2025-09-17 18:00:48 +08:00
-LAN-
73a7756350 feat(graph_engine): allow to dumps and loads RSC 2025-09-17 12:45:51 +08:00
-LAN-
02d15ebd5a feat(graph_engine): support dumps and loads in GraphExecution 2025-09-16 19:38:10 +08:00
QuantumGhost
dd34002db2 Merge remote-tracking branch 'upstream/main' into feat/rag-2 2025-09-16 14:59:35 +08:00
-LAN-
da87fce751 feat(graph_engine): dump and load ready queue 2025-09-16 04:19:46 +08:00
-LAN-
d5342927d0 chore: change _outputs type to dict[str, object] 2025-09-16 01:53:25 +08:00
-LAN-
bab4975809 chore: add ast-grep rule to convert Optional[T] to T | None (#25560)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-09-15 13:06:33 +08:00
-LAN-
b4ef1de30f feat(graph_engine): add ready_queue state persistence to GraphRuntimeState
- Add ReadyQueueState TypedDict for type-safe queue serialization
- Add ready_queue attribute to GraphRuntimeState for initializing with pre-existing queue state
- Update GraphEngine to load ready_queue from GraphRuntimeState on initialization
- Implement proper type hints using ReadyQueueState for better type safety
- Add comprehensive tests for ready_queue loading functionality

The ready_queue is read-only after initialization and allows resuming workflow
execution with a pre-populated queue of nodes ready to execute.
2025-09-15 03:05:10 +08:00
-LAN-
b4c1766932 fix: type errors
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-09-10 21:48:05 +08:00
-LAN-
23cd615489 Merge branch 'feat/queue-based-graph-engine' into feat/rag-2 2025-09-08 14:30:43 +08:00
-LAN-
abb53f11ad Merge remote-tracking branch 'origin/main' into feat/queue-based-graph-engine 2025-09-06 16:05:13 +08:00
Asuka Minato
a78339a040 remove bare list, dict, Sequence, None, Any (#25058)
Some checks are pending
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/amd64, build-api-amd64) (push) Waiting to run
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/arm64, build-api-arm64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/amd64, build-web-amd64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/arm64, build-web-arm64) (push) Waiting to run
Build and Push API & Web / create-manifest (api, DIFY_API_IMAGE_NAME, merge-api-images) (push) Blocked by required conditions
Build and Push API & Web / create-manifest (web, DIFY_WEB_IMAGE_NAME, merge-web-images) (push) Blocked by required conditions
Main CI Pipeline / Check Changed Files (push) Waiting to run
Main CI Pipeline / API Tests (push) Blocked by required conditions
Main CI Pipeline / Web Tests (push) Blocked by required conditions
Main CI Pipeline / Style Check (push) Waiting to run
Main CI Pipeline / VDB Tests (push) Blocked by required conditions
Main CI Pipeline / DB Migration Test (push) Blocked by required conditions
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
2025-09-06 03:32:23 +08:00
-LAN-
7ba1f0a046 chore: improve typing
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-09-05 20:57:11 +08:00
jyong
b9394d542c Merge branch 'feat/queue-based-graph-engine' into feat/rag-2
# Conflicts:
#	api/core/app/apps/advanced_chat/generate_task_pipeline.py
#	api/pyproject.toml
#	api/uv.lock
#	docker/docker-compose-template.yaml
#	docker/docker-compose.yaml
#	web/package.json
2025-09-04 20:30:08 +08:00
-LAN-
16e9cd5ac5 feat(graph_runtime_state): prevent to set variable pool after initialized. 2025-09-04 02:20:19 +08:00
-LAN-
fe3f03e50a feat: add property-based access control to GraphRuntimeState
- Replace direct field access with private attributes and property decorators
- Implement deep copy protection for mutable objects (dict, LLMUsage)
- Add helper methods: set_output(), get_output(), update_outputs()
- Add increment_node_run_steps() and add_tokens() convenience methods
- Update loop_node and event_handlers to use new accessor methods
- Add comprehensive unit tests for immutability and validation
- Ensure backward compatibility with existing property access patterns
2025-09-04 02:08:58 +08:00
QuantumGhost
e9e7d4f3cd Merge branch 'feat/rag-2' into feat/workflow-draft-var-optimize 2025-08-31 15:17:23 +08:00
QuantumGhost
6b9d2e98b9 feat(api): Implement truncation for WorkflowNodeExecution 2025-08-31 13:44:39 +08:00
jyong
90d72f5ddf merge new graph engine 2025-08-27 17:46:46 +08:00
jyong
392514fa13 Merge branch 'feat/queue-based-graph-engine' into feat/rag-2
# Conflicts:
#	api/commands.py
#	api/core/app/apps/common/workflow_response_converter.py
#	api/core/llm_generator/llm_generator.py
#	api/core/plugin/entities/plugin.py
#	api/core/plugin/impl/tool.py
#	api/core/rag/index_processor/index_processor_base.py
#	api/core/workflow/entities/workflow_execution.py
#	api/core/workflow/entities/workflow_node_execution.py
#	api/core/workflow/enums.py
#	api/core/workflow/graph_engine/entities/graph.py
#	api/core/workflow/graph_engine/graph_engine.py
#	api/core/workflow/nodes/enums.py
#	api/services/dataset_service.py
2025-08-27 16:05:59 +08:00
-LAN-
8c35663220 feat: queue-based graph engine
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-08-27 15:33:28 +08:00
twwu
85fd97e090 Merge branch 'main' into feat/rag-2 2025-08-25 15:30:18 +08:00
-LAN-
da9af7b547 [Chore/Refactor] Use centralized naive_utc_now for UTC datetime operations (#24352)
Some checks are pending
autofix.ci / autofix (push) Waiting to run
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/amd64, build-api-amd64) (push) Waiting to run
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/arm64, build-api-arm64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/amd64, build-web-amd64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/arm64, build-web-arm64) (push) Waiting to run
Build and Push API & Web / create-manifest (api, DIFY_API_IMAGE_NAME, merge-api-images) (push) Blocked by required conditions
Build and Push API & Web / create-manifest (web, DIFY_WEB_IMAGE_NAME, merge-web-images) (push) Blocked by required conditions
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-08-22 23:53:05 +08:00
autofix-ci[bot]
ff76adc88a [autofix.ci] apply automated fixes 2025-08-15 04:17:51 +00:00
jyong
5a2618d002 fix file 2025-08-14 18:07:52 +08:00
jyong
69a821db02 fix file 2025-08-14 18:01:29 +08:00
jyong
72ea3b4d01 fix variable_pool 2025-08-13 17:38:14 +08:00
jyong
2b7243dbc7 add credential id 2025-08-12 11:13:47 +08:00
jyong
22b3933cc3 Merge branch 'main' into feat/rag-2
# Conflicts:
#	api/core/workflow/entities/variable_pool.py
2025-08-12 11:13:04 +08:00
-LAN-
577062b93a refactor: simplify variable pool key structure and improve type safety (#23732)
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-08-11 18:10:04 +08:00
zxhlyh
de47b56ca4 merge main 2025-07-17 16:49:22 +08:00
zxhlyh
01566035e3 merge main 2025-07-17 16:48:43 +08:00
NeatGuyCoding
c3037c5491 minor code fix: remove duplicate type check branch (#22536) 2025-07-17 13:20:31 +08:00
yihong
d2933c2bfe fix: drop dead code phase2 unused class (#22042)
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
2025-07-17 09:33:07 +08:00
QuantumGhost
2c1ab4879f refactor(api): Separate SegmentType for Integer/Float to Enable Pydantic Serialization (#22025)
refactor(api): Separate SegmentType for Integer/Float to Enable Pydantic Serialization (#22025)

This PR addresses serialization issues in the VariablePool model by separating the `value_type` tags for `IntegerSegment`/`FloatSegment` and `IntegerVariable`/`FloatVariable`. Previously, both Integer and Float types shared the same `SegmentType.NUMBER` tag, causing conflicts during serialization.

Key changes:
- Introduce distinct `value_type` tags for Integer and Float segments/variables
- Add `VariableUnion` and `SegmentUnion` types for proper type discrimination
- Leverage Pydantic's discriminated union feature for seamless serialization/deserialization
- Enable accurate serialization of data structures containing these types

Closes #22024.
2025-07-16 12:31:37 +08:00
jyong
832bef053f Merge branch 'main' into feat/r2
# Conflicts:
#	docker/docker-compose.middleware.yaml
#	web/app/components/workflow-app/components/workflow-main.tsx
#	web/app/components/workflow-app/hooks/index.ts
#	web/app/components/workflow/hooks-store/store.ts
#	web/app/components/workflow/hooks/index.ts
#	web/app/components/workflow/nodes/_base/components/variable/var-reference-picker.tsx
2025-07-02 18:20:05 +08:00
jyong
81b07dc3be r2 2025-07-02 18:15:23 +08:00
QuantumGhost
c2e599cd85 fix(api): Fix resetting sys var causing internal server error (#21604)
and sorts draft variables by their creation time, ensures a consist order.
2025-07-02 13:36:35 +08:00
jyong
efccbe4039 r2 2025-06-25 17:32:26 +08:00
jyong
540096a8d8 Merge branch 'main' into feat/r2
# Conflicts:
#	api/core/plugin/impl/oauth.py
#	api/core/workflow/entities/variable_pool.py
#	api/models/workflow.py
#	api/services/dataset_service.py
2025-06-25 14:35:23 +08:00
QuantumGhost
10b738a296 feat: Persist Variables for Enhanced Debugging Workflow (#20699)
This pull request introduces a feature aimed at improving the debugging experience during workflow editing. With the addition of variable persistence, the system will automatically retain the output variables from previously executed nodes. These persisted variables can then be reused when debugging subsequent nodes, eliminating the need for repetitive manual input.

By streamlining this aspect of the workflow, the feature minimizes user errors and significantly reduces debugging effort, offering a smoother and more efficient experience.

Key highlights of this change:

- Automatic persistence of output variables for executed nodes.
- Reuse of persisted variables to simplify input steps for nodes requiring them (e.g., `code`, `template`, `variable_assigner`).
- Enhanced debugging experience with reduced friction.

Closes #19735.
2025-06-24 09:05:29 +08:00
jyong
9cdd2cbb27 r2 2025-06-03 19:02:57 +08:00
jyong
309fffd1e4 Merge branch 'main' into feat/r2
# Conflicts:
#	api/core/repositories/sqlalchemy_workflow_node_execution_repository.py
#	api/core/workflow/entities/node_entities.py
#	api/core/workflow/enums.py
2025-06-03 18:56:49 +08:00
-LAN-
f7fb10635f refactor(workflow): Rename workflow node execution models (#20458)
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-05-30 04:56:37 +08:00
-LAN-
32e779eef3 refactor(workflow): Rename NodeRunMetadataKey to WorkflowNodeExecutionMetadataKey (#20457)
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-05-30 04:47:56 +08:00
-LAN-
482e50aae9 Refactor/remove db from cycle manager (#20455)
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-05-30 04:34:13 +08:00
jyong
7f59ffe7af r2 2025-05-28 17:56:04 +08:00
jyong
35be8721b9 Merge branch 'main' into feat/r2 2025-05-26 14:50:33 +08:00
-LAN-
d31235ca13 feat: Introduce WorkflowExecution Domain Entity and Repository, Replace WorkflowRun Direct Usage, and Unify Stream Response Logic (#20067)
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-05-21 22:01:53 +08:00