Commit Graph

491 Commits

Author SHA1 Message Date
QuantumGhost
e2ae89e08d fix(tests): fix broken tests and linter issues 2025-09-01 14:55:35 +08:00
QuantumGhost
e9e7d4f3cd Merge branch 'feat/rag-2' into feat/workflow-draft-var-optimize 2025-08-31 15:17:23 +08:00
jyong
c2afb84884 fix chunk format 2025-08-29 17:10:18 +08:00
jyong
3c0adfb48a fix chunk format 2025-08-29 16:27:22 +08:00
jyong
d0dd728e6c Merge branch 'main' into feat/rag-2
# Conflicts:
#	api/core/app/entities/queue_entities.py
#	api/core/workflow/graph_engine/entities/event.py
2025-08-29 11:29:51 +08:00
Bowen Liang
39064197da chore: cleanup unnecessary mypy suppressions on imports (#24712)
Some checks are pending
autofix.ci / autofix (push) Waiting to run
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/amd64, build-api-amd64) (push) Waiting to run
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/arm64, build-api-arm64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/amd64, build-web-amd64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/arm64, build-web-arm64) (push) Waiting to run
Build and Push API & Web / create-manifest (api, DIFY_API_IMAGE_NAME, merge-api-images) (push) Blocked by required conditions
Build and Push API & Web / create-manifest (web, DIFY_WEB_IMAGE_NAME, merge-web-images) (push) Blocked by required conditions
Check i18n Files and Create PR / check-and-update (push) Waiting to run
2025-08-28 23:17:25 +08:00
Asuka Minato
4adf85d7d4 example for rm extra cast (#24646) 2025-08-28 09:37:39 +08:00
Asuka Minato
d2f234757b example try rm ignore (#24649) 2025-08-28 09:36:16 +08:00
jyong
90d72f5ddf merge new graph engine 2025-08-27 17:46:46 +08:00
jyong
392514fa13 Merge branch 'feat/queue-based-graph-engine' into feat/rag-2
# Conflicts:
#	api/commands.py
#	api/core/app/apps/common/workflow_response_converter.py
#	api/core/llm_generator/llm_generator.py
#	api/core/plugin/entities/plugin.py
#	api/core/plugin/impl/tool.py
#	api/core/rag/index_processor/index_processor_base.py
#	api/core/workflow/entities/workflow_execution.py
#	api/core/workflow/entities/workflow_node_execution.py
#	api/core/workflow/enums.py
#	api/core/workflow/graph_engine/entities/graph.py
#	api/core/workflow/graph_engine/graph_engine.py
#	api/core/workflow/nodes/enums.py
#	api/services/dataset_service.py
2025-08-27 16:05:59 +08:00
-LAN-
8c35663220 feat: queue-based graph engine
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-08-27 15:33:28 +08:00
Yongtao Huang
826f19e968 Chore : rm dead code detected by pylance (#24588) 2025-08-27 13:19:40 +08:00
twwu
da48e54778 Merge branch 'main' into feat/rag-2 2025-08-27 11:16:27 +08:00
Petrus Han
d9e26eba65 fix: rag/milvus clarify full-text search warning with actionable guidance (#24570) 2025-08-26 23:32:26 +08:00
Yongtao Huang
fa753239ad Refactor: use logger = logging.getLogger(__name__) in logging (#24515)
Co-authored-by: Yongtao Huang <99629139+hyongtao-db@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2025-08-26 18:10:31 +08:00
GuanMu
47f480c0dc fix: unify log format, use placeholders instead of string concatenation (#24544) 2025-08-26 15:45:16 +08:00
huangzhuo1949
98473e9d4f fix:external dataset weight rerank bug (#24533)
Co-authored-by: huangzhuo <huangzhuo1@xiaomi.com>
2025-08-26 14:54:40 +08:00
-LAN-
04954918a5 Merge commit from fork
Some checks are pending
autofix.ci / autofix (push) Waiting to run
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/amd64, build-api-amd64) (push) Waiting to run
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/arm64, build-api-arm64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/amd64, build-web-amd64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/arm64, build-web-arm64) (push) Waiting to run
Build and Push API & Web / create-manifest (api, DIFY_API_IMAGE_NAME, merge-api-images) (push) Blocked by required conditions
Build and Push API & Web / create-manifest (web, DIFY_WEB_IMAGE_NAME, merge-web-images) (push) Blocked by required conditions
* fix(oraclevector): SQL Injection

Signed-off-by: -LAN- <laipz8200@outlook.com>

* fix(oraclevector): Remove bind variables from FETCH FIRST clause

Oracle doesn't support bind variables in the FETCH FIRST clause.
Fixed by using validated integers directly in the SQL string while
maintaining proper input validation to prevent SQL injection.

- Updated search_by_vector method to use validated top_k directly
- Updated search_by_full_text method to use validated top_k directly
- Adjusted parameter numbering for document_ids_filter placeholders

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-26 13:51:23 +08:00
twwu
85fd97e090 Merge branch 'main' into feat/rag-2 2025-08-25 15:30:18 +08:00
huayaoyue6
23dcb2dc1b fix(vector): use semantic version comparison for version check (#24409) 2025-08-24 21:04:33 +08:00
-LAN-
da9af7b547 [Chore/Refactor] Use centralized naive_utc_now for UTC datetime operations (#24352)
Some checks are pending
autofix.ci / autofix (push) Waiting to run
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/amd64, build-api-amd64) (push) Waiting to run
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/arm64, build-api-arm64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/amd64, build-web-amd64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/arm64, build-web-arm64) (push) Waiting to run
Build and Push API & Web / create-manifest (api, DIFY_API_IMAGE_NAME, merge-api-images) (push) Blocked by required conditions
Build and Push API & Web / create-manifest (web, DIFY_WEB_IMAGE_NAME, merge-web-images) (push) Blocked by required conditions
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-08-22 23:53:05 +08:00
Asuka Minato
51cc2bf429 example of next(, None) (#24345) 2025-08-22 18:32:22 +08:00
twwu
8c44151e6f Merge branch 'main' into feat/rag-2 2025-08-22 17:40:34 +08:00
willzhao
5ab6bc283c [CHORE]: x: T = None to x: Optional[T] = None (#24217)
Some checks are pending
autofix.ci / autofix (push) Waiting to run
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/amd64, build-api-amd64) (push) Waiting to run
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/arm64, build-api-arm64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/amd64, build-web-amd64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/arm64, build-web-arm64) (push) Waiting to run
Build and Push API & Web / create-manifest (api, DIFY_API_IMAGE_NAME, merge-api-images) (push) Blocked by required conditions
Build and Push API & Web / create-manifest (web, DIFY_WEB_IMAGE_NAME, merge-web-images) (push) Blocked by required conditions
2025-08-21 21:58:39 +08:00
Yongtao Huang
106ab7f2a8 Fix: safe defaults for BaseModel dict fields (#24098)
Co-authored-by: Yongtao Huang <99629139+hyongtao-db@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-21 21:38:55 +08:00
Guangdong Liu
1abf1240b2 refactor: replace try-except blocks with contextlib.suppress for cleaner exception handling (#24284)
Some checks failed
autofix.ci / autofix (push) Waiting to run
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/amd64, build-api-amd64) (push) Waiting to run
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/arm64, build-api-arm64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/amd64, build-web-amd64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/arm64, build-web-arm64) (push) Waiting to run
Build and Push API & Web / create-manifest (api, DIFY_API_IMAGE_NAME, merge-api-images) (push) Blocked by required conditions
Build and Push API & Web / create-manifest (web, DIFY_WEB_IMAGE_NAME, merge-web-images) (push) Blocked by required conditions
Check i18n Files and Create PR / check-and-update (push) Has been cancelled
2025-08-21 18:18:49 +08:00
8bitpd
a183b2affb fix: rollback when AnalyticDB create zhparser failed (#24260)
Co-authored-by: xiaozeyu <xiaozeyu.xzy@alibaba-inc.com>
2025-08-21 15:00:26 +08:00
Amy
738aaee101 fix(api):Fix the issue of empty and not empty operations failing in k… (#24276)
Co-authored-by: liumin <min.liu@tongdun.net>
2025-08-21 14:43:08 +08:00
twwu
dfd33b3d84 Merge branch 'main' into feat/rag-2 2025-08-21 09:43:51 +08:00
8bitpd
6b1606f4f4 fix: keep idempotent when init AnalyticdbVectorBySql (#24239)
Co-authored-by: xiaozeyu <xiaozeyu.xzy@alibaba-inc.com>
2025-08-20 23:22:27 +08:00
yihong
4c1ad40f8e docs: format all md files (#24195)
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
2025-08-20 13:49:11 +08:00
twwu
a1666fe058 Merge branch 'main' into feat/rag-2 2025-08-19 14:59:06 +08:00
He Wang
670d479e32 Bump pyobvector to 0.2.15 (#24120)
Some checks are pending
autofix.ci / autofix (push) Waiting to run
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/amd64, build-api-amd64) (push) Waiting to run
Build and Push API & Web / build (api, DIFY_API_IMAGE_NAME, linux/arm64, build-api-arm64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/amd64, build-web-amd64) (push) Waiting to run
Build and Push API & Web / build (web, DIFY_WEB_IMAGE_NAME, linux/arm64, build-web-arm64) (push) Waiting to run
Build and Push API & Web / create-manifest (api, DIFY_API_IMAGE_NAME, merge-api-images) (push) Blocked by required conditions
Build and Push API & Web / create-manifest (web, DIFY_WEB_IMAGE_NAME, merge-web-images) (push) Blocked by required conditions
Check i18n Files and Create PR / check-and-update (push) Waiting to run
2025-08-18 17:36:27 +08:00
crazywoola
8288b1dcab Revert "fix pg_vector extension requires SUPERUSER, but not availabl… (#24108) 2025-08-18 16:46:15 +08:00
Elvis_LEE
16d1289a0a fix pg_vector extension requires SUPERUSER, but not available on Huawei Cloud RDS (#24093)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-18 16:29:36 +08:00
Bo Wu
790a6ec203 fix: return empty list instead of raising exception for qdrant search when score_threshold is 1 (#24032) 2025-08-18 12:44:05 +08:00
twwu
386614951f Merge branch 'main' into feat/rag-2 2025-08-18 11:16:18 +08:00
-LAN-
e340fccafb feat: integrate flask-orjson for improved JSON serialization performance (#23935) 2025-08-14 19:50:59 +08:00
jyong
ae3addb922 add credential id 2025-08-12 15:43:11 +08:00
jyong
22b3933cc3 Merge branch 'main' into feat/rag-2
# Conflicts:
#	api/core/workflow/entities/variable_pool.py
2025-08-12 11:13:04 +08:00
engchina
7566d90dfe fix issue #23758 (#23764)
Co-authored-by: root <root@thinkpad-pc.localdomain>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-12 10:26:13 +08:00
Harry
543f80ad5d refactor: replace get_real_credential_by_id with get_datasource_credentials in multiple services for consistency 2025-08-11 20:04:04 +08:00
twwu
fc779d00df Merge branch 'main' into feat/rag-2 2025-08-11 11:15:58 +08:00
yunqiqiliang
14e1c16cf2 Fix ClickZetta stability and reduce logging noise (#23632)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-08 22:57:47 +08:00
湛露先生
fd536a943a word extractor cleans. (#20926)
Signed-off-by: zhanluxianshen <zhanluxianshen@163.com>
2025-08-08 09:37:51 +08:00
yunqiqiliang
62772e8871 fix: ensure vector database cleanup on dataset deletion regardless of document presence (affects all 33 vector databases) (#23574)
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-08-08 09:18:43 +08:00
Qiang Lee
e9045a8838 Fix: Apply Metadata Filters Correctly in Full-Text Search Mode for Tencent Cloud Vector Database (#23564) 2025-08-07 05:36:06 -07:00
twwu
1b3860d012 Merge branch 'main' into feat/rag-2 2025-08-07 16:27:20 +08:00
yunqiqiliang
e01510e2a6 feat: Add Clickzetta Lakehouse vector database integration (#22551)
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-07 14:21:46 +08:00
twwu
40f3524cfe Merge branch 'main' into feat/rag-2 2025-08-06 14:23:51 +08:00