Commit Graph

431 Commits

Author SHA1 Message Date
Letong Han
3c29fb4779 Support SearchedDoc input type in LLM for No Rerank Pipeline (#541)
Signed-off-by: letonghan <letong.han@intel.com>
2024-08-21 21:44:48 +08:00
Letong Han
04ff8bfbc4 Fix Dataprep Potential Error in get_file (#540)
* fix get file error & refine logs

Signed-off-by: letonghan <letong.han@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: letonghan <letong.han@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-08-21 15:43:13 +08:00
Sun, Xuehao
945b9e4a0c fix freeze workflow (#522)
Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>
2024-08-21 15:37:03 +08:00
Chendi.Xue
f3a8935354 Add Assistant API for agent (#490)
* Add assistants api support

Signed-off-by: Chendi.Xue <chendi.xue@intel.com>

* update UT to match new port definition

Signed-off-by: Chendi.Xue <chendi.xue@intel.com>

* rebase

Signed-off-by: Chendi.Xue <chendi.xue@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix gramma in Test

Signed-off-by: Chendi.Xue <chendi.xue@intel.com>

---------

Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-08-20 13:33:59 -05:00
ZePan110
5dedd04365 Add CD workflow. (#519)
* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* for test.

Signed-off-by: zepan <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add func.

Signed-off-by: zepan <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* for test.

Signed-off-by: zepan <ze.pan@intel.com>

* For test2.

Signed-off-by: zepan <ze.pan@intel.com>

* add function.

Signed-off-by: zepan <ze.pan@intel.com>

* Modify get matrix.

Signed-off-by: zepan <ze.pan@intel.com>

* modify get matrix.

Signed-off-by: zepan <ze.pan@intel.com>

* modify input.

Signed-off-by: zepan <ze.pan@intel.com>

* Add comps workflow.
Signed-off-by: zepan <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add node.

Signed-off-by: zepan <ze.pan@intel.com>

* Add images build.

Signed-off-by: zepan <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix issue node.

Signed-off-by: zepan <ze.pan@intel.com>

* Add images build.

Signed-off-by: zepan <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix issue of yaml name.

Signed-off-by: zepan <ze.pan@intel.com>

* Fix issue of service value.

Signed-off-by: zepan <ze.pan@intel.com>

* test

Signed-off-by: zepan <ze.pan@intel.com>

* test

Signed-off-by: zepan <ze.pan@intel.com>

* restore build Image.

Signed-off-by: zepan <ze.pan@intel.com>

* modify Build Image.

Signed-off-by: zepan <ze.pan@intel.com>

* test

Signed-off-by: zepan <ze.pan@intel.com>

* modify

Signed-off-by: zepan <ze.pan@intel.com>

* modify call service.

Signed-off-by: zepan <ze.pan@intel.com>

* Fix issue work_dir

Signed-off-by: zepan <ze.pan@intel.com>

* Fix issue docker_compose_path

Signed-off-by: zepan <ze.pan@intel.com>

* Fix tag.

Signed-off-by: zepan <ze.pan@intel.com>

* Change Work_dir.

Signed-off-by: zepan <ze.pan@intel.com>

* Change image-build branch

Signed-off-by: zepan <ze.pan@intel.com>

* Change work_dir.

Signed-off-by: zepan <ze.pan@intel.com>

* Chaange work_dir and image_build branch

Signed-off-by: zepan <ze.pan@intel.com>

* Add test-service-compose.

Signed-off-by: zepan <ze.pan@intel.com>

* Remove Unrelated variables.

Signed-off-by: zepan <ze.pan@intel.com>

* for test

Signed-off-by: zepan <ze.pan@intel.com>

* test2

Signed-off-by: zepan <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* for test.

Signed-off-by: zepan <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add func.

Signed-off-by: zepan <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* for test.

Signed-off-by: zepan <ze.pan@intel.com>

* For test2.

Signed-off-by: zepan <ze.pan@intel.com>

* add function.

Signed-off-by: zepan <ze.pan@intel.com>

* Modify get matrix.

Signed-off-by: zepan <ze.pan@intel.com>

* modify get matrix.

Signed-off-by: zepan <ze.pan@intel.com>

* modify input.

Signed-off-by: zepan <ze.pan@intel.com>

* Add comps workflow.
Signed-off-by: zepan <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add node.

Signed-off-by: zepan <ze.pan@intel.com>

* Add images build.

Signed-off-by: zepan <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix issue node.

Signed-off-by: zepan <ze.pan@intel.com>

* Add images build.

Signed-off-by: zepan <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix issue of yaml name.

Signed-off-by: zepan <ze.pan@intel.com>

* Fix issue of service value.

Signed-off-by: zepan <ze.pan@intel.com>

* test

Signed-off-by: zepan <ze.pan@intel.com>

* test

Signed-off-by: zepan <ze.pan@intel.com>

* restore build Image.

Signed-off-by: zepan <ze.pan@intel.com>

* modify Build Image.

Signed-off-by: zepan <ze.pan@intel.com>

* test

Signed-off-by: zepan <ze.pan@intel.com>

* modify

Signed-off-by: zepan <ze.pan@intel.com>

* modify call service.

Signed-off-by: zepan <ze.pan@intel.com>

* Fix issue work_dir

Signed-off-by: zepan <ze.pan@intel.com>

* Fix issue docker_compose_path

Signed-off-by: zepan <ze.pan@intel.com>

* Fix tag.

Signed-off-by: zepan <ze.pan@intel.com>

* Change Work_dir.

Signed-off-by: zepan <ze.pan@intel.com>

* Change image-build branch

Signed-off-by: zepan <ze.pan@intel.com>

* Change work_dir.

Signed-off-by: zepan <ze.pan@intel.com>

* Chaange work_dir and image_build branch

Signed-off-by: zepan <ze.pan@intel.com>

* Add test-service-compose.

Signed-off-by: zepan <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove Unrelated variables.

Signed-off-by: zepan <ze.pan@intel.com>

* for test

Signed-off-by: zepan <ze.pan@intel.com>

* test2

Signed-off-by: zepan <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix issue of run-test not be run.

Signed-off-by: zepan <ze.pan@intel.com>

* for test

Signed-off-by: zepan <ze.pan@intel.com>

* Change input.

Signed-off-by: zepan <ze.pan@intel.com>

* Fix issue of input.service

Signed-off-by: zepan <ze.pan@intel.com>

* Fix output issue.

Signed-off-by: zepan <ze.pan@intel.com>

* Fix " issue.

Signed-off-by: zepan <ze.pan@intel.com>

* Change the trigger method to manual.

Signed-off-by: zepan <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix issue of run-test not be run.

Signed-off-by: zepan <ze.pan@intel.com>

* for test

Signed-off-by: zepan <ze.pan@intel.com>

* Change input.

Signed-off-by: zepan <ze.pan@intel.com>

* Fix issue of input.service

Signed-off-by: zepan <ze.pan@intel.com>

* Fix output issue.

Signed-off-by: zepan <ze.pan@intel.com>

* Fix " issue.

Signed-off-by: zepan <ze.pan@intel.com>

* Change the trigger method to manual.

Signed-off-by: zepan <ze.pan@intel.com>

* Rename yaml.

Signed-off-by: zepan <ze.pan@intel.com>

---------

Signed-off-by: zepan <ze.pan@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-08-20 22:12:43 +08:00
Abolfazl Shahbazi
5deb383adb Using habana docker 1.16.1 everywhere (#528)
Signed-off-by: Abolfazl Shahbazi <abolfazl.shahbazi@intel.com>
2024-08-20 22:09:32 +08:00
XinyaoWa
fab1fbdfe9 Add logging for unified debug (#521)
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
2024-08-20 22:09:04 +08:00
Letong Han
0584b45ef1 refine llm parameter (#532)
Signed-off-by: letonghan <letong.han@intel.com>
2024-08-20 21:27:09 +08:00
Lianhao Lu
8b9dcddca2 Fix tts image push error (#530)
Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
2024-08-20 14:26:27 +08:00
Abolfazl Shahbazi
4a7b8f4248 adding entrypoint.sh to faq-generation comp (#525)
Signed-off-by: Abolfazl Shahbazi <abolfazl.shahbazi@intel.com>
2024-08-20 09:38:32 +08:00
Letong Han
69f989535f Refine Nginx Component (#523)
Signed-off-by: letonghan <letong.han@intel.com>
2024-08-20 09:13:29 +08:00
Chendi.Xue
56daf95b7d Add Retrieval gateway in core to support IndexRetrivel Megaservice (#314)
* Add Retrieval gateway

Signed-off-by: Chendi.Xue <chendi.xue@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update gateway to latest protocal

Signed-off-by: Chendi.Xue <chendi.xue@intel.com>

* tested with DocIndexer, rebased gateway is now workable

Signed-off-by: Chendi.Xue <chendi.xue@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-08-19 11:38:32 -05:00
minmin-intel
368c833762 Add RagAgentDocGrader to agent comp (#480)
* set hf_hub to 0.24.0

* add docgrader to agent strategy openai llm code passed

* add nonstreaming output for agent

* add react langgraph and tests

* fix react langchain bug

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix test script

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* fix bug in test script

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* update readme and rm old agentic-rag strategy

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update test and docgrader readme

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix bug in test

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update test

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update rag agent strategy name and update readme

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update test

Signed-off-by: minmin-intel <minmin.hou@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: minmin-intel <minmin.hou@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-08-19 09:22:33 -07:00
Sun, Xuehao
61dba72c0d Freeze base images (#509)
Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>
2024-08-19 15:33:54 +08:00
Letong Han
60cc0b0266 Add Nginx Component for Service Forwarding (#506)
Signed-off-by: letonghan <letong.han@intel.com>
2024-08-19 10:38:44 +08:00
Chun Tao
20fc8ca4da Revise TTS, SpeechT5Model to end the last audio chunk at the correct punctuation mark location (#513)
* Revise SpeechT5Model to end the last audio chunk at the correct punctuation mark

Signed-off-by: Chun Tao <chun.tao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* still append the last chunk if it contains no punctuation

Signed-off-by: Chun Tao <chun.tao@intel.com>

---------

Signed-off-by: Chun Tao <chun.tao@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-08-19 10:36:59 +08:00
chen, suyue
cd83854f93 Fix CI test changed file detect issue (#512)
* update wf

Signed-off-by: chensuyue <suyue.chen@intel.com>

* for test

Signed-off-by: chensuyue <suyue.chen@intel.com>

* remove test code

Signed-off-by: chensuyue <suyue.chen@intel.com>

---------

Signed-off-by: chensuyue <suyue.chen@intel.com>
2024-08-18 19:30:51 +08:00
lvliang-intel
cc8cd702ae Remove unnecessary langsmith dependency (#369)
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
2024-08-18 17:11:13 +08:00
David Kinder
77e0e7be1c doc: fix multiple H1 headings (#481)
Only one H1 heading (first for the title) allowed.

Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>
2024-08-17 23:43:12 +08:00
Harsha Ramayanam
b271739838 Update README.md (#482)
Co-authored-by: Abolfazl Shahbazi <abolfazl.shahbazi@intel.com>
2024-08-17 23:38:46 +08:00
ZePan110
2b14c637f4 Fix waring. (#504)
Signed-off-by: zepan <ze.pan@intel.com>
2024-08-17 23:35:16 +08:00
Wang, Kai Lawrence
8eb8b6a44e Add cmds to restart ollama service and add proxy settings while launching docker (#438)
Signed-off-by: Wang, Kai Lawrence <kai.lawrence.wang@intel.com>
2024-08-17 23:33:24 +08:00
David Kinder
5bd8bda946 doc: remove use of unknown highlight language (#489)
This file use ```log  but there's no such syntax highlight language.
Leave it empty instead.

Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
2024-08-17 11:54:51 +08:00
Harsha Ramayanam
72a2553036 Fix image in docker compose yaml to use the built docker image tag from the README (#498)
* Update README.md

The `ray_serve:habana` doesn't exist (yet) in docker hub

* Fixed the image in docker compose yaml

Signed-off-by: Harsha Ramayanam <harsha.ramayanam@intel.com>

---------

Signed-off-by: Harsha Ramayanam <harsha.ramayanam@intel.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>
2024-08-16 09:14:09 -07:00
Sun, Xuehao
22075037a0 Fix requirement actions (#500)
Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>
2024-08-16 20:53:49 +08:00
chen, suyue
10433364bb add sudo in wf remove (#511)
Signed-off-by: chensuyue <suyue.chen@intel.com>
2024-08-16 20:11:30 +08:00
chen, suyue
7ffaf247f7 adapt GenAIExample test structure refine (#510)
Signed-off-by: chensuyue <suyue.chen@intel.com>
2024-08-16 19:58:57 +08:00
Sihan Chen
9580298a22 fix asr ut (#501) 2024-08-16 11:46:13 +08:00
Sun, Xuehao
fecf4acc1d Fix line endings to LF (#497)
Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>
2024-08-16 11:18:38 +08:00
ZePan110
8a6079d1bf Modify validate result check. (#503)
Signed-off-by: zepan <ze.pan@intel.com>
2024-08-16 11:17:48 +08:00
Letong Han
b16b14a93b Refine LLM Native Microservice (#477)
* refine test script of embedding llama_index

Signed-off-by: letonghan <letong.han@intel.com>

* update

Signed-off-by: letonghan <letong.han@intel.com>

* update code of llm-native

Signed-off-by: letonghan <letong.han@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix llm native issue

Signed-off-by: letonghan <letong.han@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* udpate param & add readme

Signed-off-by: letonghan <letong.han@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: letonghan <letong.han@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-08-16 10:15:49 +08:00
ZePan110
cf15b913e6 Add validate result detection. (#492)
Signed-off-by: zepan <ze.pan@intel.com>
2024-08-15 21:21:10 +08:00
ZePan110
a56a847c36 Standardize image build. (#472)
Signed-off-by: zepan <ze.pan@intel.com>
2024-08-15 21:20:56 +08:00
Sun, Xuehao
5d9a8557f4 Freeze requirements workflow for release tag (#485)
Signed-off-by: Sun, Xuehao <xuehao.sun@intel.com>
Signed-off-by: NeuralChatBot <grp_neural_chat_bot@intel.com>
2024-08-15 14:57:05 +08:00
ZePan110
5159aacaaa Check build fail and change port 8008 to 5025/5026. (#487)
Signed-off-by: zepan <ze.pan@intel.com>
2024-08-15 11:04:29 +08:00
ZePan110
4a67d427bd Change /root/ to /home/user/. (#475)
* Change /root/ to /hmoe/user/.

Signed-off-by: zepan <ze.pan@intel.com>

* Fix issue.

Signed-off-by: zepan <ze.pan@intel.com>

---------

Signed-off-by: zepan <ze.pan@intel.com>
2024-08-14 15:46:28 +08:00
ZePan110
0bd821532f Fix issue (#483)
* Standardize image names and labels.

Signed-off-by: zepan <ze.pan@intel.com>

* Remove test cases that are no longer in use.

Signed-off-by: zepan <ze.pan@intel.com>

---------

Signed-off-by: zepan <ze.pan@intel.com>
2024-08-14 14:47:26 +08:00
Sihan Chen
daec6803aa support whisper long-form generation (#469)
* fix long form asr accuracy

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix test input pad issue

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>
2024-08-14 11:23:07 +08:00
Letong Han
7aee7e4689 fix empty content issue (#476)
Signed-off-by: letonghan <letong.han@intel.com>
2024-08-14 10:14:41 +08:00
lvliang-intel
f2497c5638 Revert max_new_tokens to 1024 (#479)
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
2024-08-14 01:05:17 +08:00
Tyler W
62ca5bcf55 Add output evaluation for guardrails (#332)
* add single query input/output guardrails

Signed-off-by: Tyler Wilbers <tyler.wilbers@intel.com>

* removed comment

Signed-off-by: Tyler Wilbers <tyler.wilbers@intel.com>

---------

Signed-off-by: Tyler Wilbers <tyler.wilbers@intel.com>
2024-08-13 08:21:05 -07:00
ZePan110
ab8ebc4ce7 update test scripts and bug fix (#464)
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: zepan <ze.pan@intel.com>
2024-08-13 18:14:01 +08:00
rbrugaro
2891cc621c fix HuggingFaceEmbedding deprecated in favor of HuggingFaceInferenceAPIEmbedding (#466)
* Fix HuggingFaceEmbedding deprecated in favor of HuggingFaceInferenceAPIEmbedding

Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: rbrugaro <rita.brugarolas.brufau@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-08-13 16:28:21 +08:00
Wang, Kai Lawrence
eb51018f65 Refine the instructions to run the retriever example with qdrant (#442)
Signed-off-by: Wang, Kai Lawrence <kai.lawrence.wang@intel.com>
2024-08-13 15:59:01 +08:00
Letong Han
a851abfee5 Fix Dataprep qdrant issues and add Test Script (#474)
Signed-off-by: letonghan <letong.han@intel.com>
2024-08-13 15:50:01 +08:00
ganesanintel
075e84f517 Address test case failure (#462)
Signed-off-by: V, Ganesan <ganesan.v@intel.com>
2024-08-13 15:16:24 +08:00
ZePan110
87905ad00d Fix embeddings_langchain-mosec issue. (#473)
Signed-off-by: zepan <ze.pan@intel.com>
2024-08-13 15:10:45 +08:00
chen, suyue
f36629a037 clean local images before test (#471)
* clean test images before test

Signed-off-by: chensuyue <suyue.chen@intel.com>

* add --no-cache for image build

Signed-off-by: chensuyue <suyue.chen@intel.com>

---------

Signed-off-by: chensuyue <suyue.chen@intel.com>
2024-08-13 13:23:21 +08:00
Letong Han
04986c17b2 Refine robustness of Dataprep Redis (#463)
Signed-off-by: letonghan <letong.han@intel.com>
2024-08-13 12:14:56 +08:00
Letong Han
d3eefea4b0 Update Milvus docker-compose.yaml (#459)
Signed-off-by: letonghan <letong.han@intel.com>
2024-08-13 12:13:38 +08:00