jandres/GenAIExamples
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity
698 Commits 43 Branches 9 Tags
enable_persistent_db
Commit Graph

698 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sharan Shirodkar
391c4a5801 Prediction Guard LLM component (#674) 
Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>
 2024-09-17 21:57:06 +08:00
Sharan Shirodkar
191061b642 Prediction Guard embeddings component (#675) 
* added files for PG embeddingso component

Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>

* added package

Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>

* fixed dockerfile link

Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>

* Fix pre-commit issues: end-of-file, requirements.txt, trailing whitespace, imports, and formatting

Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>

* added package

Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>

* added package

Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>

* fixed embedoc call

Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>

* file structure updated to latest

Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>

* Fix pre-commit issues: end-of-file, requirements.txt, trailing whitespace, imports, and formatting

Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>

* added package

Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>

---------

Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>
 2024-09-17 21:33:19 +08:00
chen, suyue
b4a7f261fe update action (#696) 
Signed-off-by: chensuyue <suyue.chen@intel.com>
 2024-09-17 12:53:07 +08:00
qgao007
812c85cefb Add Bias Detection Microservice (#659) 
* Add Bias Detection Microservice

Signed-off-by: Qun Gao <qun.gao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Adding Bias Detection Container to CI (#695)

Signed-off-by: Abolfazl Shahbazi <abolfazl.shahbazi@intel.com>

---------

Signed-off-by: Qun Gao <qun.gao@intel.com>
Signed-off-by: Abolfazl Shahbazi <abolfazl.shahbazi@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Abolfazl Shahbazi <abolfazl.shahbazi@intel.com>
 2024-09-17 10:01:22 +08:00
XuhuiRen
29fe569d34 Enable GraphRAG with Neo4J (#682) 
* add graphrag for neo4j

Signed-off-by: XuhuiRen <xuhui.ren@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add

Signed-off-by: XuhuiRen <xuhui.ren@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add

Signed-off-by: XuhuiRen <xuhui.ren@intel.com>

* add

Signed-off-by: XuhuiRen <xuhui.ren@intel.com>

* fix ut

Signed-off-by: XuhuiRen <xuhui.ren@intel.com>

* fix

Signed-off-by: XuhuiRen <xuhui.ren@intel.com>

* add

Signed-off-by: XuhuiRen <xuhui.ren@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update retriever_neo4j.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add

Signed-off-by: XuhuiRen <xuhui.ren@intel.com>

* Update test_retrievers_neo4j_langchain.sh

* add

Signed-off-by: XuhuiRen <xuhui.ren@intel.com>

* Update test_retrievers_neo4j_langchain.sh

* Update test_retrievers_neo4j_langchain.sh

* Update test_retrievers_neo4j_langchain.sh

* add docker

Signed-off-by: XuhuiRen <xuhui.ren@intel.com>

* Update retrievers-compose-cd.yaml

* Update test_retrievers_neo4j_langchain.sh

* Update config.py

* Update test_retrievers_neo4j_langchain.sh

* Update test_retrievers_neo4j_langchain.sh

* Update config.py

* Update test_retrievers_neo4j_langchain.sh

* Update requirements.txt

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update requirements.txt

* Update requirements.txt

* Update requirements.txt

---------

Signed-off-by: XuhuiRen <xuhui.ren@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: lvliang-intel <liang1.lv@intel.com>
 2024-09-15 18:12:29 +08:00
Harsha Ramayanam
18092f3dec Changes to comps/llms/text-generation/README (#678) 
* Added changes per dbkinder's review

Signed-off-by: Harsha Ramayanam <harsha.ramayanam@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Harsha Ramayanam <harsha.ramayanam@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-09-14 13:52:56 +08:00
David Kinder
f8f8854f9c doc: fix heading levels (#690) 
Only one H1 for the title is allowed

Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
 2024-09-14 09:36:41 +08:00
chen, suyue
3c5fc80570 update cd workflow (#689) 
Signed-off-by: ZePan110 <ze.pan@intel.com>
Signed-off-by: chensuyue <suyue.chen@intel.com>
 2024-09-13 23:01:14 +08:00
ZePan110
2794abd2c0 Update manual test. (#684) 
* Update manual test.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Update script folder.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Change Matching Rules

Signed-off-by: ZePan110 <ze.pan@intel.com>

---------

Signed-off-by: ZePan110 <ze.pan@intel.com>
 2024-09-13 21:59:58 +08:00
Hoong Tee, Yeoh
0f2c2b1ed7 vllm langchain: Add Document Retriever Support (#687) 
* vllm langchain: Add Document Retriever Support

Include SearchedDoc in /v1/chat/completions endpoint to accept document
data retreived from retriever service to parse into LLM for answer
generation.

Signed-off-by: Yeoh, Hoong Tee <hoong.tee.yeoh@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* vllm: Update README documentation

Signed-off-by: Yeoh, Hoong Tee <hoong.tee.yeoh@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Yeoh, Hoong Tee <hoong.tee.yeoh@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-09-13 20:08:56 +08:00
lvliang-intel
574fecfe77 Update embedding svc test port number (#688) 
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
 2024-09-13 17:35:15 +08:00
Tiep Le
2ca56f3ed9 Update example name into MultimodalQnA and update image names (#686) 
* update image names

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update image names for dataprep service

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update image name for retriever

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update image name for lvm

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update lvm include caption in output

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update gateway name into MultimodalQnAGateway

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update test gateway for rename

Signed-off-by: Tiep Le <tiep.le@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Tiep Le <tiep.le@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-09-13 17:07:38 +08:00
Huiling Bao
2b68323c2c Rename videoragqna to videoqna to align with other examples (#680) 
Signed-off-by: BaoHuiling <huiling.bao@intel.com>
 2024-09-12 22:12:39 +08:00
chen, suyue
0458443baf update torch cpu installation (#681) 
Signed-off-by: chensuyue <suyue.chen@intel.com>
 2024-09-12 22:01:38 +08:00
XinyaoWa
ca94c60fb7 fix vllm llamaindex stream bug (#679) 
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
 2024-09-12 14:05:14 +08:00
chen, suyue
6ce65514e1 replace langchain/langchain:latest with python:3.11-slim (#669) 
* replace langchain/langchain:latest with python:3.11-slim

Signed-off-by: chensuyue <suyue.chen@intel.com>

* fix test

Signed-off-by: chensuyue <suyue.chen@intel.com>

---------

Signed-off-by: chensuyue <suyue.chen@intel.com>
 2024-09-12 13:24:50 +08:00
ZePan110
887ca7581c Fix dockerfile check logical (#671) 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: ZePan110 <ze.pan@intel.com>
 2024-09-12 00:35:36 +08:00
lvliang-intel
1b6989797f Add OpenAI client access OPEA microservice UT cases (#653) 
* Add OpeanAI access microservice UT cases

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add the absolute path

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* refactor code

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* add openai denpendency

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* install openai in scripts

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* fix ci issue

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

---------

Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-09-12 00:08:14 +08:00
XuhuiRen
a672569551 Fix security issue (#673) 
Signed-off-by: XuhuiRen <xuhui.ren@intel.com>
 2024-09-12 00:01:58 +08:00
lvliang-intel
cff0a4dfbb Support export megaservice yaml to docker compose file (#642) 
* Support export megaservice yaml tp docker compose file

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add ut case

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix issue

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add code spell ignore

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* fix ut issue

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* disable cli case

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

---------

Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-09-11 22:26:43 +08:00
Zhenzhong1
3ac391a1d2 Fixed CI yaml (#672) 
* fixed CI yaml

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-09-11 21:40:00 +08:00
Zhenzhong1
af4e0f8775 [ChatQnA] Support K8S Python Client to export ChatQnA E2E manifests (#634) 
* initialize the exporter

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update manifests_exporter

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update manifests_exporter

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* baseline

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refactor

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove HF_TOKEN

* E2E pass

* gloden manifests & remove duplicated configs

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* gloden manifests & remove duplicated configs

* golden scripts

* rename

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* added replicas and removed useless6008 6009

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add the yaml load func

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* added UT

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* cleancode

* added kubernetes

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* added UT

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>
 2024-09-11 21:05:11 +08:00
Steve Zhang
161c338782 fix vllm components images building. (#667) 
Signed-off-by: zhlsunshine <huailong.zhang@intel.com>
 2024-09-11 20:50:45 +08:00
chen, suyue
2d5130fc15 temp remove dockerfile check (#670) 
Signed-off-by: chensuyue <suyue.chen@intel.com>
 2024-09-11 18:32:02 +08:00
ZePan110
61b8fa9b51 Opt filecheck (#663) 
* update description

Signed-off-by: chensuyue <suyue.chen@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Optimize path-scan.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* For test.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix bug.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* test1

Signed-off-by: ZePan110 <ze.pan@intel.com>

* test2.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* test3

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Restore test file.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* test4

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Fix syntax error

Signed-off-by: ZePan110 <ze.pan@intel.com>

* test

Signed-off-by: ZePan110 <ze.pan@intel.com>

* test

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Fix issue.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Remove debug output

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Restore test file and remove debug output.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Remove deprecated code.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Modify output.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* modify output

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Fix file path error.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix issue

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Fix error

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Fix error.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* Fix error.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix error

Signed-off-by: ZePan110 <ze.pan@intel.com>

* resolve confilct.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix error.

Signed-off-by: ZePan110 <ze.pan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: ZePan110 <ze.pan@intel.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-09-11 18:05:35 +08:00
Letong Han
7e40475b6d Refine READMEs after reorg (#666) 
* update dataprep readme

Signed-off-by: letonghan <letong.han@intel.com>

* update vectorstore readme

Signed-off-by: letonghan <letong.han@intel.com>

* udpate retriever readme

Signed-off-by: letonghan <letong.han@intel.com>

* update retriever readme

Signed-off-by: letonghan <letong.han@intel.com>

* udpate embedding readme

Signed-off-by: letonghan <letong.han@intel.com>

* update guardrails readme

Signed-off-by: letonghan <letong.han@intel.com>

* update other readmes

Signed-off-by: letonghan <letong.han@intel.com>

* update reranks readme

Signed-off-by: letonghan <letong.han@intel.com>

* update llm&lvms readme

Signed-off-by: letonghan <letong.han@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: letonghan <letong.han@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-09-11 17:23:40 +08:00
Tiep Le
7664578c97 Support for UI of MultimodalRAGWithVideos in GenAIExamples (#656) 
* using png instead of jpg due to tgi-gaudi. return message include video_id maps

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update lvm and lvm_tgi to handle empty retrieval results

Signed-off-by: Tiep Le <tiep.le@intel.com>

* handle no retrieval results

Signed-off-by: Tiep Le <tiep.le@intel.com>

* add metadata to textdoc returned by LVM

Signed-off-by: Tiep Le <tiep.le@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add metadatatextdoc

Signed-off-by: Tiep Le <tiep.le@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add test

Signed-off-by: Tiep Le <tiep.le@intel.com>

* add metadata to Chatcompletionresponsechoice

Signed-off-by: Tiep Le <tiep.le@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix bug

Signed-off-by: Tiep Le <tiep.le@intel.com>

* fix bug

Signed-off-by: Tiep Le <tiep.le@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix bug

Signed-off-by: Tiep Le <tiep.le@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* minor fix for more consice

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update port for test

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update test

Signed-off-by: Tiep Le <tiep.le@intel.com>

---------

Signed-off-by: Tiep Le <tiep.le@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: kevinintel <hanwen.chang@intel.com>
Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>
 2024-09-11 17:22:59 +08:00
XinyaoWa
8e3f553a55 Support Llama index for vLLM (#665) 
Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-09-11 16:59:47 +08:00
Letong Han
267fb02c76 Move fintuning test script path (#664) 
Signed-off-by: letonghan <letong.han@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-09-11 16:46:03 +08:00
lvliang-intel
98c62a016d Move neuralspeed embedding rerank and vllm-xft to catalog (#647) 
* Move neuralspeed embedding rerank and vllm-xft to catalog

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* delete xft code

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* update d yaml

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

---------

Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>
 2024-09-11 14:51:38 +08:00
Steve Zhang
1d761fa0a6 Fix the Retriever README error (#655) 
* change based on comments for missing places.
Signed-off-by: zhlsunshine <huailong.zhang@intel.com>
 2024-09-11 14:36:21 +08:00
sri-intel
f84d91a3b0 adding dataprep support for CLIP based models for VideoRAGQnA example for v1.0 (#621) 
* dataprep service

Signed-off-by: srinarayan-srikanthan <srinarayan.srikanthan@intel.com>

* dataprep updates

Signed-off-by: srinarayan-srikanthan <srinarayan.srikanthan@intel.com>

* rearranged dirs

Signed-off-by: srinarayan-srikanthan <srinarayan.srikanthan@intel.com>

* added readme

Signed-off-by: srinarayan-srikanthan <srinarayan.srikanthan@intel.com>

* removed checks

Signed-off-by: srinarayan-srikanthan <srinarayan.srikanthan@intel.com>

* added features

Signed-off-by: srinarayan-srikanthan <srinarayan.srikanthan@intel.com>

* added get method

Signed-off-by: srinarayan-srikanthan <srinarayan.srikanthan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add dim at init, rm unused

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add wait after connect DB

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove unused

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* Update comps/dataprep/vdms/README.md

Co-authored-by: XinyuYe-Intel <xinyu.ye@intel.com>
Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* add test script for mm case

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* add return value and update readme

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* check bug

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* fix mm-script

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* add into dataprep workflow

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* rm whitespace

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* updated readme and added test script

Signed-off-by: srinarayan-srikanthan <srinarayan.srikanthan@intel.com>

* removed unused file

Signed-off-by: srinarayan-srikanthan <srinarayan.srikanthan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* move test script

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* restructured repo

Signed-off-by: srinarayan-srikanthan <srinarayan.srikanthan@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* updates path in test script

Signed-off-by: srinarayan-srikanthan <srinarayan.srikanthan@intel.com>

* add name for build

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

---------

Signed-off-by: srinarayan-srikanthan <srinarayan.srikanthan@intel.com>
Signed-off-by: BaoHuiling <huiling.bao@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: BaoHuiling <huiling.bao@intel.com>
Co-authored-by: XinyuYe-Intel <xinyu.ye@intel.com>
 2024-09-11 13:19:04 +08:00
chen, suyue
4165c7d8c1 optimize ci test scope (#662) 
Signed-off-by: chensuyue <suyue.chen@intel.com>
 2024-09-11 12:04:01 +08:00
minmin-intel
42cde68dd6 fix ragagent text generator bug (#660) 
* fix ragagent text generator bug

Signed-off-by: minmin-intel <minmin.hou@intel.com>
 2024-09-11 08:16:25 +08:00
chen, suyue
1785149ecf minor fix for CI detect (#658) 
Signed-off-by: chensuyue <suyue.chen@intel.com>
 2024-09-11 00:45:34 +08:00
Letong Han
a3da7c1e3f Fix Reorg Issues (#657) 
Signed-off-by: letonghan <letong.han@intel.com>
 2024-09-11 00:45:12 +08:00
Letong Han
bea9bb0242 [Reorg] Reorg Folder to Support Different Vendors (#625) 
Signed-off-by: letonghan <letong.han@intel.com>
Signed-off-by: chensuyue <suyue.chen@intel.com>
 2024-09-10 23:24:09 +08:00
ZePan110
2a91903627 Fix Dockerfile detect issue (#652) 
Signed-off-by: ZePan110 <ze.pan@intel.com>
 2024-09-10 17:22:35 +08:00
lkk
c718602abf add resume finetuning checkpoint ut. (#646) 
* add resume finetuning checkpoint ut.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add final tuned model.

---------

Co-authored-by: root <root@idc708073.jf.intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-09-10 17:35:23 +09:00
chen, suyue
94eb60f168 add PREDICTIONGUARD_API_KEY for CI (#650) 
Signed-off-by: chensuyue <suyue.chen@intel.com>
 2024-09-10 15:56:47 +08:00
XinyuYe-Intel
29311479bb added finetuned model deployment tutorial in readme. (#648) 
Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>
 2024-09-10 15:52:15 +08:00
XuhuiRen
2fbce3e53d Fix the loading error of jsonl file (#644) 
* fixjsonl

Signed-off-by: XuhuiRen <xuhui.ren@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: XuhuiRen <xuhui.ren@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-09-10 15:50:40 +08:00
chen, suyue
48d4e53396 unify default reranking model with BAAI/bge-reranker-base (#623) 
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: ZePan110 <ze.pan@intel.com>
 2024-09-10 15:23:10 +08:00
ZePan110
8a1141314d Optimize the content of the alerts. (#640) 
Signed-off-by: ZePan110 <ze.pan@intel.com>
Signed-off-by: chensuyue <suyue.chen@intel.com>
 2024-09-10 13:43:06 +08:00
XinyuYe-Intel
58e9972d82 Add LLM pretraining support (#622) 
* support rerank model finetuning.

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

* adapt rerank model to transformers' scheme.

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix typo.

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

* refined readme.

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add LLM pretraining support.

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* added readme.

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

* added rerank finetuning test.

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* added LLM pretraining test.

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>

---------

Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-09-10 13:18:35 +08:00
Dong, Bo
d76751ad81 optimize rerank with backend ref (#579) 
* add rerank with neural speed

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* add the code

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* add the code

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* fix mismatched response format w/wo streaming guardrails (#568)

* fix mismatched response format w/wo streaming  guardrails

* fix & debug

* fix & rm debug

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* Fix guardrails out handle logics for space linebreak and quote (#571)

* fix mismatched response format w/wo streaming  guardrails

* fix & debug

* fix & rm debug

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* debug

* debug

* debug

* fix pre-space and linebreak

* fix pre-space and linebreak

* fix single/double quote

* fix single/double quote

* remove debug

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* BUG FIX: LVM security fix (#572)

* add url validator

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add validation for video_url

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

---------

Signed-off-by: BaoHuiling <huiling.bao@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* Modify output messages. (#569)

* Reduced output.

Signed-off-by: zepan <ze.pan@intel.com>

* Output the location where the modified Dockerfile file is referenced.

Signed-off-by: zepan <ze.pan@intel.com>

* for test

Signed-off-by: zepan <ze.pan@intel.com>

* Restore test file.

Signed-off-by: zepan <ze.pan@intel.com>

---------

Signed-off-by: zepan <ze.pan@intel.com>
Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* refine logging code. (#559)

* add ut and refine logging code.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update microservice port.

---------

Co-authored-by: root <root@idc708073.jf.intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* adding lancedb to langchain vectorstores (#291)

* adding lancedb to langchain vectorstores

Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: lvliang-intel <liang1.lv@intel.com>
Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* Refine Dataprep Milvus MS (#570)

Signed-off-by: letonghan <letong.han@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* final version

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* update the readme

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* add the sign

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* fix error for pre ci

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* add the ut

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* update docker file

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* update CI test log achieve (#577)

Signed-off-by: chensuyue <suyue.chen@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* Multimodal dataprep (#575)

* multimodal embedding for MM RAG for videos

Signed-off-by: Tiep Le <tiep.le@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* develop data prep first commit

Signed-off-by: Tiep Le <tiep.le@intel.com>

* develop dataprep microservice for multimodal data

Signed-off-by: Tiep Le <tiep.le@intel.com>

* multimodal langchain for dataprep

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update README

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update README

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update README

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update README

Signed-off-by: Tiep Le <tiep.le@intel.com>

* cosmetic

Signed-off-by: Tiep Le <tiep.le@intel.com>

* test for multimodal dataprep

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update test

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update test

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update test

Signed-off-by: Tiep Le <tiep.le@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* cosmetic update

Signed-off-by: Tiep Le <tiep.le@intel.com>

* remove langsmith

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update API to remove /dataprep from API names and remove langsmith

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update test

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update the error message per PR reviewer

Signed-off-by: Tiep Le <tiep.le@intel.com>

---------

Signed-off-by: Tiep Le <tiep.le@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* add: Pathway vector store and retriever as LangChain component (#342)

* nb

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* init changes

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* docker

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* example data

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* docs(readme): update, add commands

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* fix: formatting, data sources

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* docs(readme): update instructions, add comments

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* fix: rm unused parts

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* fix: image name, compose env vars

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* fix: rm unused part

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* fix: logging name

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* fix: env var

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* fix: rename pw docker

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* docs(readme): update input sources

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* nb

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* init changes

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* fix: formatting, data sources

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* docs(readme): update instructions, add comments

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* fix: rm unused part

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* fix: rename pw docker

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* feat: mv vector store, naming, clarify instructions, improve ingestion components

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* tests: add pw retriever test
fix: update docker to include libmagic

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* implement suggestions from review, entrypoint, reqs, comments, https_proxy.

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: update docker tags in test and readme

Signed-off-by: Berke <berkecanrizai1@gmail.com>

* tests: add separate pathway vectorstore test

Signed-off-by: Berke <berkecanrizai1@gmail.com>

---------

Signed-off-by: Berke <berkecanrizai1@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com>
Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* Add local Rerank microservice for VideoRAGQnA (#496)

* initial commit

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* save

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* add readme, test script, fix bug

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* update video URL

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* use default

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update core dependency

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* use p 5000

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* use 5037

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* update ctnr name

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* remove langsmith

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add rerank algo desc in readme

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: BaoHuiling <huiling.bao@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>
Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* Add Scan Container. (#560)

Signed-off-by: zepan <ze.pan@intel.com>
Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* fix SearchedMultimodalDoc in docarray (#583)

Signed-off-by: BaoHuiling <huiling.bao@intel.com>
Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* update image build yaml (#529)

Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: zepan <ze.pan@intel.com>
Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* add microservice for intent detection (#131)

* add microservice for intent detection

Signed-off-by: Liangyx2 <yuxiang.liang@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update license copyright

Signed-off-by: Liangyx2 <yuxiang.liang@intel.com>

* add ut

Signed-off-by: Liangyx2 <yuxiang.liang@intel.com>

* refine

Signed-off-by: Liangyx2 <yuxiang.liang@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update folder

Signed-off-by: Liangyx2 <yuxiang.liang@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix test

Signed-off-by: Liangyx2 <yuxiang.liang@intel.com>

---------

Signed-off-by: Liangyx2 <yuxiang.liang@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* Make the scanning method optional. (#580)

Signed-off-by: zepan <ze.pan@intel.com>
Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* add code owners (#586)

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* remove revision for tei (#584)

Signed-off-by: letonghan <letong.han@intel.com>
Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* Bug fix (#591)

* Check if the document exists.

Signed-off-by: zepan <ze.pan@intel.com>

* Add flag output.

Signed-off-by: zepan <ze.pan@intel.com>

* Modify nginx readme.

Signed-off-by: zepan <ze.pan@intel.com>

* Modify document detection logic

Signed-off-by: zepan <ze.pan@intel.com>

---------

Signed-off-by: zepan <ze.pan@intel.com>
Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* fix ut issue

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* merge the main

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* align with new pipeline

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* align with newest pipeline

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* upload code

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* update the ut

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* add docker path

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

* add the docker path

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>

---------

Signed-off-by: Dong, Bo1 <bo1.dong@intel.com>
Signed-off-by: BaoHuiling <huiling.bao@intel.com>
Signed-off-by: zepan <ze.pan@intel.com>
Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>
Signed-off-by: letonghan <letong.han@intel.com>
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: Tiep Le <tiep.le@intel.com>
Signed-off-by: Berke <berkecanrizai1@gmail.com>
Signed-off-by: Liangyx2 <yuxiang.liang@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com>
Co-authored-by: Huiling Bao <huiling.bao@intel.com>
Co-authored-by: ZePan110 <ze.pan@intel.com>
Co-authored-by: lkk <33276950+lkk12014402@users.noreply.github.com>
Co-authored-by: root <root@idc708073.jf.intel.com>
Co-authored-by: Sharan Shirodkar <91109427+sharanshirodkar7@users.noreply.github.com>
Co-authored-by: lvliang-intel <liang1.lv@intel.com>
Co-authored-by: Letong Han <106566639+letonghan@users.noreply.github.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>
Co-authored-by: Tiep Le <97980157+tileintel@users.noreply.github.com>
Co-authored-by: berkecanrizai <63911408+berkecanrizai@users.noreply.github.com>
Co-authored-by: Liangyx2 <yuxiang.liang@intel.com>
Co-authored-by: kevinintel <hanwen.chang@intel.com>
 2024-09-10 12:08:42 +08:00
Huiling Bao
2c48bc80c7 Add Megaservice support for MMRAG VideoRAGQnA usecase (#603) 
* add videoragqna gateway

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add test script for gateway

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* rm ip

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix exist bug

Signed-off-by: BaoHuiling <huiling.bao@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: BaoHuiling <huiling.bao@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: kevinintel <hanwen.chang@intel.com>
 2024-09-09 23:37:17 +09:00
Tiep Le
23cc3ea5ee Add schema to Redis initialization & Improve LVM-TGI For Multimodal Retriever Microservice (#638) 
* add schema to Redis initialization

Signed-off-by: Tiep Le <tiep.le@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update lvm_tgi

Signed-off-by: Tiep Le <tiep.le@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Tiep Le <tiep.le@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-09-09 11:59:05 +08:00
lvliang-intel
fb4b8d27e3 Fix LVM streaming issue (#637) 
* Fix LVM streaming issue

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-09-09 08:34:44 +09:00
lkk
7e1a2e57fd enable embedding finetuning (#639) 
* add embedding finetuning.
* enhance training on hpu.
* add embedding training ut.
 2024-09-09 07:29:19 +08:00