jandres/GenAIExamples
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
698 Commits 43 Branches 9 Tags
enable_persistent_db
Commit Graph

197 Commits

Author SHA1 Message Date
XinyuYe-Intel
28f9c31129 textgen ollama code refactor. (#1158) 
Remove Ollama folder since default openai API is able to consume Ollama service, modified Ollama readme and added UT.
#998
Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>
 2025-01-17 16:49:31 +08:00
Letong Han
8e48849c49 Refactor Vectorstores to E-RAG Style (#1159) 
Move DBs with yaml/Dockerfile into third_parties folder and delete vectorstores folder.
Modify related components for change of vectordb folder path.
Remained vectorestores:
- thirdparties
    - elasticsearch
    - milvus
    - opensearch
    - pathway
    - pgvector
    - redis
    - vdms

Fix issue https://github.com/opea-project/GenAIComps/issues/1008.

Signed-off-by: letonghan <letong.han@intel.com>
 2025-01-16 19:46:11 +08:00
XinyaoWa
d6312a73b9 Fix vllm hpu to a stable release (#1156) 
There exist risks with vllm-fork main branch, change to latest stable release v0.6.4.post2+Gaudi-1.19.0

Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
 2025-01-16 17:48:06 +08:00
Yao Qing
3802cd84a7 Refactor chathistory based on E-RAG code structure. (#1154) 
Previously, the code for implementing microservice and functionalities was placed in one folder. Now the microservice and functionalities code have been separated.

Fix [Issue 989](https://github.com/opea-project/GenAIComps/issues/989)

Signed-off-by: Yao, Qing <qing.yao@intel.com>
 2025-01-16 14:17:16 +08:00
Letong Han
c49db2b037 Refactor Retrievers (#1138) 
Refactor retrievers into E-RAG style.
The folder structure after refactored is as below:

- retrievers
    - deployment
        - docker_compose
        - kubernetes
    - src
        - integrations
            - redis.py
            - milvus.py
            - ...
        - Dockerfile
        - opea_retrievers_microservice.py
        - README.md
        - requirements.txt
        - set_env.sh

#1004

Signed-off-by: letonghan <letong.han@intel.com>
 2025-01-16 14:12:30 +08:00
XinyaoWa
6d07a064f8 Refactor LLM text generation native comps (#1151) 
Part work of code refactor to combine different text generation backends, remove duplcated native langchain and llama_index folder, consice the optimum habana implementation as a native integration OPEATextGen_Native.

Add feature for issue https://github.com/opea-project/GenAIComps/issues/998

Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
 2025-01-16 13:21:46 +08:00
XinyaoWa
5098e82040 Refactor vllm openvino to third parties (#1141) 
vllm-openvino is a dependency for text generation comps, move it to third-parties folder, add UT for both cpu and gpu.

Related to feature issue#998

Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
 2025-01-16 09:38:18 +08:00
qgao007
4db13298cd Build guardrail "Hallucination Detection" microservice. (#1108) 
* Build guardrail "Hallucination Detection" microservice.

Signed-off-by: Qun Gao <qun.gao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update comps/guardrails/hallucination_detection/README.md

Co-authored-by: Daniel De León <111013930+daniel-de-leon-user293@users.noreply.github.com>

* - register Dockerfile

- rename file for hpu

- update endpoints to be consistent

Signed-off-by: Qun Gao <qun.gao@intel.com>

* Update repo structure

Signed-off-by: Qun Gao <qun.gao@intel.com>

* refactor

Signed-off-by: Qun Gao <qun.gao@intel.com>

* Refactored Hallucination Guardrail to wrap code under new OpeaComponent Class and leaverage OpeaComponentLoader Class for serving inference.

Signed-off-by: Qun Gao <qun.gao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Updated Copyright year to reflect right date

Signed-off-by: Qun Gao <qun.gao@intel.com>

---------

Signed-off-by: Qun Gao <qun.gao@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Abolfazl Shahbazi <12436063+ashahba@users.noreply.github.com>
Co-authored-by: Daniel De León <111013930+daniel-de-leon-user293@users.noreply.github.com>
Co-authored-by: ZePan110 <ze.pan@intel.com>
 2025-01-15 10:31:08 -08:00
lkk
3e6c5cd8bd remove ragas. (#1149) 
* remove ragas.

* remove ci docker compose.
 2025-01-15 13:28:45 +08:00
XinyaoWa
4c21738ad5 Refactor llm predictionguard (#1143) 
* refactor llm predictionguard

Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>

* refine predictionguard ut

Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>

* remove duplicated dockerfile path

Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>

* fix bug

Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix bug

Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>

---------

Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: ZePan110 <ze.pan@intel.com>
 2025-01-15 10:13:42 +08:00
Neo Zhang Jianyu
89dd6285d8 Update check-online-doc-build.yml (#1145) 2025-01-15 09:07:14 +08:00
XinyaoWa
afb8a28b2f remove vllm hpu commit id limit (#1139) 
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
 2025-01-14 11:05:05 +08:00
XinyaoWa
88f93733b0 Refactor llm Docsum (#1101) 
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
 2025-01-13 15:24:43 +08:00
Sihan Chen
feef30b0ea Refactor lvms (#1096) 
Co-authored-by: ZePan110 <ze.pan@intel.com>
 2025-01-13 13:06:59 +08:00
XinyaoWa
ea72c943bd Refactor FaqGen (#1093) 
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2025-01-13 11:30:59 +08:00
Liang Lv
b91911a543 Refine embedding naming and move dependency to 3rd_party (#1125) 
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
 2025-01-10 14:44:25 +08:00
Yao Qing
4f9f95574b Rename folder name integration to integrations in image2image and animation (#1130) 
Signed-off-by: Yao, Qing <qing.yao@intel.com>
Co-authored-by: ZePan110 <ze.pan@intel.com>
 2025-01-10 10:35:21 +08:00
chen, suyue
e7b8d25bfc Update action token (#1131) 
Signed-off-by: chensuyue <suyue.chen@intel.com>
 2025-01-09 17:19:04 +08:00
XinyuYe-Intel
efd95780fd Finetuning code refactor (#1081) 
Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>
 2025-01-09 11:50:15 +08:00
XinyuYe-Intel
2587a2978a Text2image code refactor (#1054) 
Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>
 2025-01-09 11:45:15 +08:00
Liang Lv
179b5da06b Refactor prompt registry microservice (#1124) 
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
 2025-01-09 11:26:03 +08:00
Liang Lv
ec66b91c51 Feedback management microservice refactor (#1057) 
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
 2025-01-09 11:18:15 +08:00
Sihan Chen
962e097893 Refactor web retriever (#1102) 2025-01-08 15:24:08 +08:00
Lianhao Lu
4480d807a3 CI: Automatically create issue in CIInfra (#1123) 
Automatically create issue in CIInfra to track the changes of docker compose
files for correspdonding helm charts.

Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
 2025-01-08 15:09:27 +08:00
Liang Lv
631b570481 Refactor guardrails microservice (#1116) 
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
 2025-01-08 13:29:23 +08:00
WenjiaoYue
267cad1f44 Refactor reranking (#1113) 
Signed-off-by: WenjiaoYue <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>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: ZePan110 <ze.pan@intel.com>
 2025-01-08 10:19:04 +08:00
ZePan110
ca21633a73 Change license template 2024 to 2025 (#1119) 
Signed-off-by: ZePan110 <ze.pan@intel.com>
 2025-01-07 19:29:45 +08:00
lkk
cf90932fef refine agent directories. (#1109) 2025-01-06 17:35:39 +08:00
ZePan110
b933b66f15 Check duplicated dockerfile (#1073) 
Signed-off-by: ZePan110 <ze.pan@intel.com>
 2025-01-06 17:27:59 +08:00
chen, suyue
e96b5d91a9 Update code owner list (#1110) 
Signed-off-by: chensuyue <suyue.chen@intel.com>
 2025-01-06 14:53:58 +08:00
chen, suyue
6419ace56c Fix changed file detect issue (#1103) 
Signed-off-by: chensuyue <suyue.chen@intel.com>
 2025-01-03 11:24:19 +08:00
chen, suyue
f57e30dde6 GenAIComps microservices refactor (#1072) 
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Signed-off-by: letonghan <letong.han@intel.com>
Signed-off-by: Wang, Kai Lawrence <kai.lawrence.wang@intel.com>
Signed-off-by: Wang, Xigui <xigui.wang@intel.com>
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: WenjiaoYue <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>
 2025-01-02 16:31:01 +08:00
Yao Qing
2cfd014b3b Refactor text2sql based on ERAG (#1080) 
Signed-off-by: Yao, Qing <qing.yao@intel.com>
 2025-01-02 10:09:10 +08:00
XinyuYe-Intel
90a86345c5 Image2video code refactor (#1075) 
* image2video code refactor.

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix spell error.

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

* Update opea_image2video_microservice.py

* changed naming

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

---------

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-12-31 13:18:15 +08:00
Sihan Chen
a19c222636 Refactor asr/tts components (#1083) 
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-12-31 12:03:10 +08:00
Yao Qing
1040875055 Refactor image2image (#1076) 
* Refactor image2image

Signed-off-by: Yao, Qing <qing.yao@intel.com>




---------

Signed-off-by: Yao, Qing <qing.yao@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-12-27 17:05:57 +08:00
Yao Qing
a7888ab299 Refactor Animation based on ERAG (#1079) 
Signed-off-by: Yao, Qing <qing.yao@intel.com>
 2024-12-27 15:26:49 +08:00
Sihan Chen
f006a3ee6c remove dataprep/multimedia2text (#1065) 2024-12-26 14:42:33 +08:00
Cameron Morin
8d6b4b0ac7 Add opensearch integration for OPEA (#1024) 
* Add opensearch integration for OPEA

Signed-off-by: Cameron Morin <cammorin@amazon.com>

* Update docker compose yaml workflows files

Signed-off-by: Cameron Morin <cammorin@amazon.com>

* Fix empty files

Signed-off-by: Cameron Morin <cammorin@amazon.com>

* Address PR comments

Signed-off-by: Cameron Morin <cammorin@amazon.com>

---------

Signed-off-by: Cameron Morin <cammorin@amazon.com>
 2024-12-26 11:09:59 +08:00
Letong Han
f3aaaebf5a [Reorg] Remove redundant file in retrievers/redis (#1016) 
Signed-off-by: letonghan <letong.han@intel.com>
 2024-12-17 12:01:13 +08:00
kkrishTa
5ed041bded Feature/elasticsearch vector store integration - Infosys (#972) 
* Feature/elastic

Elasticsearch vectorstore, dataprep and retriever

---------

Co-authored-by: Adarsh <reachaadi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Liang Lv <liang1.lv@intel.com>
 2024-12-10 09:40:44 +08:00
VincyZhang
766c757f3b add dangerous cmd check (#955) 
* add dangerous cmd check

Signed-off-by: Wenxin Zhang <wenxin.zhang@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* clean code

Signed-off-by: Wenxin Zhang <wenxin.zhang@intel.com>

---------

Signed-off-by: Wenxin Zhang <wenxin.zhang@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-12-03 22:33:26 +08:00
Neo Zhang Jianyu
0772494f62 add label automatically when create issue (#960) 
Co-authored-by: ZhangJianyu <zhang.jianyu@outlook.com>
 2024-12-02 14:48:17 +08:00
ZePan110
9d6d7b8195 Check image and service names in compose.yaml (#951) 
* WIP

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Chack image and service names in compose.yaml

Signed-off-by: ZePan110 <ze.pan@intel.com>

* merge pr-check-duplicated-image.yml to pr-dockerfile-path-scan.yaml

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Remove .github/workflows/pr-check-duplicated-image.yml

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Unblocking txt files from push-image-build.yml

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Fix name error

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Split pr-dockerfile-path-scan.yaml to .github/workflows/pr-dockerfile-path-scan.yaml and .github/workflows/pr-link-path-scan.yaml and change the mask of .github/workflows/push-image-build.yml

Signed-off-by: ZePan110 <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: ZePan110 <ze.pan@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-11-28 13:39:52 +08:00
Lianhao Lu
0e94eecbb7 CI: Add check for conflict image build definition (#944) 
Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
 2024-11-27 11:28:47 +08:00
chen, suyue
6c2b466372 Fix image scan _get-image-list.yml (#941) 
* fix corner issue

Signed-off-by: chensuyue <suyue.chen@intel.com>

* for test

Signed-off-by: chensuyue <suyue.chen@intel.com>

* fix issue

Signed-off-by: chensuyue <suyue.chen@intel.com>

* fix images

Signed-off-by: chensuyue <suyue.chen@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: chensuyue <suyue.chen@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-11-22 09:46:33 +08:00
Isaac Ng
3dcc1e0a78 fix issue template bug (#939) 
Signed-off-by: isaacncz <isaac.ng@intel.com>
 2024-11-21 17:09:15 +08:00
chen, suyue
9b1e322624 fix the image name (#918) 
Signed-off-by: chensuyue <suyue.chen@intel.com>
 2024-11-20 11:00:01 +08:00
ZePan110
85818c97af Fix CD issues (#917) 
* Add PREDICTIONGUARD_API_KEY

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Increase timeout

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Fix log name issue

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Remove useless code.

Signed-off-by: ZePan110 <ze.pan@intel.com>

---------

Signed-off-by: ZePan110 <ze.pan@intel.com>
 2024-11-20 09:34:40 +08:00
ZePan110
f19cf083d1 Rename image names XXX-hpu to XXX-gaudi (#911) 
Signed-off-by: ZePan110 <ze.pan@intel.com>
 2024-11-19 22:06:55 +08:00