Commit Graph

7 Commits

Author SHA1 Message Date
Sihan Chen
feef30b0ea Refactor lvms (#1096)
Co-authored-by: ZePan110 <ze.pan@intel.com>
2025-01-13 13:06:59 +08:00
Melanie Hart Buehler
29ef64269a MultimodalQnA Image and Audio Support Phase 1 (#852)
* Adds an endpoint for image ingestion

Signed-off-by: Melanie Buehler <melanie.h.buehler@intel.com>

* Combined image and video endpoint

Signed-off-by: Melanie Buehler <melanie.h.buehler@intel.com>

* Add test and update README

Signed-off-by: Melanie Buehler <melanie.h.buehler@intel.com>

* fixed variable name for embedding model (#1)

Signed-off-by: okhleif-IL <omar.khleif@intel.com>

* Fixed test script

Signed-off-by: Melanie Buehler <melanie.h.buehler@intel.com>

* Remove redundant function

Signed-off-by: Melanie Buehler <melanie.h.buehler@intel.com>

* get_videos, delete_videos --> get_files, delete_files (#3)

Signed-off-by: okhleif-IL <omar.khleif@intel.com>

* Updates test per review feedback

Signed-off-by: Melanie Buehler <melanie.h.buehler@intel.com>

* Fixed test

Signed-off-by: Melanie Buehler <melanie.h.buehler@intel.com>

* Add support for audio files multimodal data ingestion (#4)

* Add support for audio files multimodal data ingestion

Signed-off-by: dmsuehir <dina.s.jones@intel.com>

* Update function name

Signed-off-by: dmsuehir <dina.s.jones@intel.com>

---------

Signed-off-by: dmsuehir <dina.s.jones@intel.com>

* Change videos_with_transcripts to ingest_with_text

Signed-off-by: Melanie Buehler <melanie.h.buehler@intel.com>

* Add image support to video ingestion with transcript functionality

Signed-off-by: Melanie Buehler <melanie.h.buehler@intel.com>

* Update test and README

Signed-off-by: Melanie Buehler <melanie.h.buehler@intel.com>

* Updated for review suggestions

Signed-off-by: Melanie Buehler <melanie.h.buehler@intel.com>

* Add two tests for ingest_with_text

Signed-off-by: Melanie Buehler <melanie.h.buehler@intel.com>

* LVM TGI Gaudi update for prompts without images (#7)

* LVM Gaudi TGI update for prompts without images

Signed-off-by: dmsuehir <dina.s.jones@intel.com>

* Wording

Signed-off-by: dmsuehir <dina.s.jones@intel.com>

* Add a test

Signed-off-by: dmsuehir <dina.s.jones@intel.com>

---------

Signed-off-by: dmsuehir <dina.s.jones@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Change dummy image to be b64 encoded instead of the url (#9)

Signed-off-by: dmsuehir <dina.s.jones@intel.com>

* Updates based on review feedback (#10)

Signed-off-by: dmsuehir <dina.s.jones@intel.com>

* Test fix (#11)

Signed-off-by: dmsuehir <dina.s.jones@intel.com>

---------

Signed-off-by: Melanie Buehler <melanie.h.buehler@intel.com>
Signed-off-by: okhleif-IL <omar.khleif@intel.com>
Signed-off-by: dmsuehir <dina.s.jones@intel.com>
Co-authored-by: dmsuehir <dina.s.jones@intel.com>
Co-authored-by: Omar Khleif <omar.khleif@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Abolfazl Shahbazi <12436063+ashahba@users.noreply.github.com>
2024-11-08 10:19:46 +08:00
lvliang-intel
534c227a6e Support Llama3.2 vision and vision guard model (#753)
* Support llama3.2 models

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* fix issues

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* update code and doc

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* add llama vision guard support

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* add llama guard prompt format utils

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* add tp support

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* add wheel

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* fix accuracy issue

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* update tp service code

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* update dockerfile

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* support lvm tp serving

Signed-off-by: letonghan <letong.han@intel.com>

* update dockerfile

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* add run tp script

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* fix max_new_tokens

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* update run tp script

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* refine code and doc

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* install transformers from local wheel

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* update  code using official transformers

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* remove unnecessary code

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove blank line

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* fix precommit issues

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* fix cd issue

Signed-off-by: lvliang-intel <liang1.lv@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Signed-off-by: letonghan <letong.han@intel.com>
Co-authored-by: letonghan <letong.han@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-09-30 17:27:41 +09:00
Sharan Shirodkar
1249c4fe72 Prediction Guard LVM component (#676)
* added files for PG lvm component

Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>

* Fix pre-commit issues: end-of-file, requirements.txt, trailing whitespace, imports, and formatting

Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>

* added package

Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>

---------

Signed-off-by: sharanshirodkar7 <ssharanshirodkar7@gmail.com>
2024-09-18 10:32:16 +08:00
Tiep Le
2ca56f3ed9 Update example name into MultimodalQnA and update image names (#686)
* update image names

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update image names for dataprep service

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update image name for retriever

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update image name for lvm

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update lvm include caption in output

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update gateway name into MultimodalQnAGateway

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update test gateway for rename

Signed-off-by: Tiep Le <tiep.le@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Tiep Le <tiep.le@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-09-13 17:07:38 +08:00
Tiep Le
7664578c97 Support for UI of MultimodalRAGWithVideos in GenAIExamples (#656)
* using png instead of jpg due to tgi-gaudi. return message include video_id maps

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update lvm and lvm_tgi to handle empty retrieval results

Signed-off-by: Tiep Le <tiep.le@intel.com>

* handle no retrieval results

Signed-off-by: Tiep Le <tiep.le@intel.com>

* add metadata to textdoc returned by LVM

Signed-off-by: Tiep Le <tiep.le@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add metadatatextdoc

Signed-off-by: Tiep Le <tiep.le@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add test

Signed-off-by: Tiep Le <tiep.le@intel.com>

* add metadata to Chatcompletionresponsechoice

Signed-off-by: Tiep Le <tiep.le@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix bug

Signed-off-by: Tiep Le <tiep.le@intel.com>

* fix bug

Signed-off-by: Tiep Le <tiep.le@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix bug

Signed-off-by: Tiep Le <tiep.le@intel.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* minor fix for more consice

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update port for test

Signed-off-by: Tiep Le <tiep.le@intel.com>

* update test

Signed-off-by: Tiep Le <tiep.le@intel.com>

---------

Signed-off-by: Tiep Le <tiep.le@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: kevinintel <hanwen.chang@intel.com>
Co-authored-by: Sihan Chen <39623753+Spycsh@users.noreply.github.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>
2024-09-11 17:22:59 +08:00
Letong Han
bea9bb0242 [Reorg] Reorg Folder to Support Different Vendors (#625)
Signed-off-by: letonghan <letong.han@intel.com>
Signed-off-by: chensuyue <suyue.chen@intel.com>
2024-09-10 23:24:09 +08:00