jandres/GenAIExamples
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

Refactor lvm related examples (#1333)

Browse Source
This commit is contained in:
Sihan Chen
2025-01-13 13:42:06 +08:00
committed by GitHub
parent f48bd8e74f
commit ca15fe9bdb
25 changed files with 162 additions and 155 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
MultimodalQnA/docker_compose/amd/gpu/rocm/README.md
View File

@@ -39,7 +39,7 @@ docker build --no-cache -t opea/embedding:latest --build-arg https_proxy=$https_
Build lvm-llava image
```bash
docker build --no-cache -t opea/lvm-llava:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/llava/dependency/Dockerfile .
docker build --no-cache -t opea/lvm-llava:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/src/integrations/dependency/llava/Dockerfile .
```
### 3. Build retriever-multimodal-redis Image
@@ -85,7 +85,7 @@ Then run the command `docker images`, you will have the following 8 Docker Image
1. `opea/dataprep-multimodal-redis:latest`
2. `ghcr.io/huggingface/text-generation-inference:2.4.1-rocm`
3. `opea/lvm-tgi:latest`
3. `opea/lvm:latest`
4. `opea/retriever-multimodal-redis:latest`
5. `opea/embedding:latest`
6. `opea/embedding-multimodal-bridgetower:latest`
@@ -193,7 +193,7 @@ curl http://${host_ip}:${LLAVA_SERVER_PORT}/generate \
-d '{"prompt":"Describe the image please.", "img_b64_str": "iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAAFUlEQVR42mP8/5+hnoEIwDiqkL4KAcT9GO0U4BxoAAAAAElFTkSuQmCC"}'
```
5. lvm-llava-svc
5. lvm
```bash
curl http://${host_ip}:9399/v1/lvm \

11
MultimodalQnA/docker_compose/amd/gpu/rocm/compose.yaml
View File

@@ -24,7 +24,7 @@ services:
container_name: dataprep-multimodal-redis
depends_on:
- redis-vector-db
- lvm-tgi
- lvm
ports:
- "6007:6007"
environment:
@@ -116,9 +116,9 @@ services:
ipc: host
command: --model-id ${LVM_MODEL_ID} --max-input-tokens 3048 --max-total-tokens 4096
restart: unless-stopped
lvm-tgi:
image: ${REGISTRY:-opea}/lvm-tgi:${TAG:-latest}
container_name: lvm-tgi
lvm:
image: ${REGISTRY:-opea}/lvm:${TAG:-latest}
container_name: lvm
depends_on:
- tgi-rocm
ports:
@@ -128,6 +128,7 @@ services:
no_proxy: ${no_proxy}
http_proxy: ${http_proxy}
https_proxy: ${https_proxy}
LVM_COMPONENT_NAME: "OPEA_TGI_LLAVA_LVM"
LVM_ENDPOINT: ${LVM_ENDPOINT}
HF_HUB_DISABLE_PROGRESS_BARS: 1
HF_HUB_ENABLE_HF_TRANSFER: 0
@@ -140,7 +141,7 @@ services:
- dataprep-multimodal-redis
- embedding
- retriever-redis
- lvm-tgi
- lvm
ports:
- "8888:8888"
environment:

12
MultimodalQnA/docker_compose/intel/cpu/xeon/README.md
View File

@@ -36,7 +36,7 @@ lvm-llava
================
Port 8399 - Open to 0.0.0.0/0
lvm-llava-svc
lvm
===
Port 9399 - Open to 0.0.0.0/0
@@ -132,13 +132,13 @@ docker build --no-cache -t opea/retriever-redis:latest --build-arg https_proxy=$
Build lvm-llava image
```bash
docker build --no-cache -t opea/lvm-llava:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/llava/dependency/Dockerfile .
docker build --no-cache -t opea/lvm-llava:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/src/integrations/dependency/llava/Dockerfile .
```
Build lvm-llava-svc microservice image
Build lvm microservice image
```bash
docker build --no-cache -t opea/lvm-llava-svc:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/llava/Dockerfile .
docker build --no-cache -t opea/lvm:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/src/Dockerfile .
```
### 4. Build dataprep-multimodal-redis Image
@@ -179,7 +179,7 @@ cd ../../../
Then run the command `docker images`, you will have the following 11 Docker Images:
1. `opea/dataprep-multimodal-redis:latest`
2. `opea/lvm-llava-svc:latest`
2. `opea/lvm:latest`
3. `opea/lvm-llava:latest`
4. `opea/retriever-multimodal-redis:latest`
5. `opea/whisper:latest`
@@ -271,7 +271,7 @@ curl http://${host_ip}:${LLAVA_SERVER_PORT}/generate \
-d '{"prompt":"Describe the image please.", "img_b64_str": "iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAAFUlEQVR42mP8/5+hnoEIwDiqkL4KAcT9GO0U4BxoAAAAAElFTkSuQmCC"}'
```
6. lvm-llava-svc
6. lvm
```bash
curl http://${host_ip}:9399/v1/lvm \

9
MultimodalQnA/docker_compose/intel/cpu/xeon/compose.yaml
View File

@@ -100,9 +100,9 @@ services:
https_proxy: ${https_proxy}
entrypoint: ["python", "llava_server.py", "--device", "cpu", "--model_name_or_path", $LVM_MODEL_ID]
restart: unless-stopped
lvm-llava-svc:
image: ${REGISTRY:-opea}/lvm-llava-svc:${TAG:-latest}
container_name: lvm-llava-svc
lvm:
image: ${REGISTRY:-opea}/lvm:${TAG:-latest}
container_name: lvm
depends_on:
- lvm-llava
ports:
@@ -112,6 +112,7 @@ services:
no_proxy: ${no_proxy}
http_proxy: ${http_proxy}
https_proxy: ${https_proxy}
LVM_COMPONENT_NAME: "OPEA_LLAVA_LVM"
LVM_ENDPOINT: ${LVM_ENDPOINT}
restart: unless-stopped
multimodalqna:
@@ -122,7 +123,7 @@ services:
- dataprep-multimodal-redis
- embedding
- retriever-redis
- lvm-llava-svc
- lvm
ports:
- "8888:8888"
environment:

10
MultimodalQnA/docker_compose/intel/hpu/gaudi/README.md
View File

@@ -86,10 +86,10 @@ Build TGI Gaudi image
docker pull ghcr.io/huggingface/tgi-gaudi:2.0.6
```
Build lvm-tgi microservice image
Build lvm microservice image
```bash
docker build --no-cache -t opea/lvm-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/tgi-llava/Dockerfile .
docker build --no-cache -t opea/lvm:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/src/Dockerfile .
```
### 4. Build dataprep-multimodal-redis Image
@@ -128,7 +128,7 @@ docker build --no-cache -t opea/multimodalqna-ui:latest --build-arg https_proxy=
Then run the command `docker images`, you will have the following 11 Docker Images:
1. `opea/dataprep-multimodal-redis:latest`
2. `opea/lvm-tgi:latest`
2. `opea/lvm:latest`
3. `ghcr.io/huggingface/tgi-gaudi:2.0.6`
4. `opea/retriever-multimodal-redis:latest`
5. `opea/whisper:latest`
@@ -220,7 +220,7 @@ curl http://${host_ip}:${LLAVA_SERVER_PORT}/generate \
-H 'Content-Type: application/json'
```
6. lvm-tgi
6. lvm
```bash
curl http://${host_ip}:9399/v1/lvm \
@@ -274,7 +274,7 @@ curl --silent --write-out "HTTPSTATUS:%{http_code}" \
-F "files=@./${audio_fn}"
```
Also, test dataprep microservice with generating an image caption using lvm-tgi
Also, test dataprep microservice with generating an image caption using lvm
```bash
curl --silent --write-out "HTTPSTATUS:%{http_code}" \

11
MultimodalQnA/docker_compose/intel/hpu/gaudi/compose.yaml
View File

@@ -24,7 +24,7 @@ services:
container_name: dataprep-multimodal-redis
depends_on:
- redis-vector-db
- lvm-tgi
- lvm
ports:
- "6007:6007"
environment:
@@ -115,9 +115,9 @@ services:
ipc: host
command: --model-id ${LVM_MODEL_ID} --max-input-tokens 3048 --max-total-tokens 4096
restart: unless-stopped
lvm-tgi:
image: ${REGISTRY:-opea}/lvm-tgi:${TAG:-latest}
container_name: lvm-tgi
lvm:
image: ${REGISTRY:-opea}/lvm:${TAG:-latest}
container_name: lvm
depends_on:
- tgi-gaudi
ports:
@@ -127,6 +127,7 @@ services:
no_proxy: ${no_proxy}
http_proxy: ${http_proxy}
https_proxy: ${https_proxy}
LVM_COMPONENT_NAME: "OPEA_TGI_LLAVA_LVM"
LVM_ENDPOINT: ${LVM_ENDPOINT}
HF_HUB_DISABLE_PROGRESS_BARS: 1
HF_HUB_ENABLE_HF_TRANSFER: 0
@@ -139,7 +140,7 @@ services:
- dataprep-multimodal-redis
- embedding
- retriever-redis
- lvm-tgi
- lvm
ports:
- "8888:8888"
environment:

14
MultimodalQnA/docker_image_build/build.yaml
View File

@@ -38,21 +38,15 @@ services:
lvm-llava:
build:
context: GenAIComps
dockerfile: comps/lvms/llava/dependency/Dockerfile
dockerfile: comps/lvms/src/integrations/dependency/llava/Dockerfile
extends: multimodalqna
image: ${REGISTRY:-opea}/lvm-llava:${TAG:-latest}
lvm-llava-svc:
lvm:
build:
context: GenAIComps
dockerfile: comps/lvms/llava/Dockerfile
dockerfile: comps/lvms/src/Dockerfile
extends: multimodalqna
image: ${REGISTRY:-opea}/lvm-llava-svc:${TAG:-latest}
lvm-tgi:
build:
context: GenAIComps
dockerfile: comps/lvms/tgi-llava/Dockerfile
extends: multimodalqna
image: ${REGISTRY:-opea}/lvm-tgi:${TAG:-latest}
image: ${REGISTRY:-opea}/lvm:${TAG:-latest}
dataprep-multimodal-redis:
build:
context: GenAIComps

8
MultimodalQnA/tests/test_compose_on_gaudi.sh
View File

@@ -22,7 +22,7 @@ function build_docker_images() {
cd $WORKPATH/docker_image_build
git clone https://github.com/opea-project/GenAIComps.git && cd GenAIComps && git checkout "${opea_branch:-"main"}" && cd ../
echo "Build all the images with --no-cache, check docker_image_build.log for details..."
service_list="multimodalqna multimodalqna-ui embedding-multimodal-bridgetower embedding retriever-redis lvm-tgi dataprep-multimodal-redis whisper"
service_list="multimodalqna multimodalqna-ui embedding-multimodal-bridgetower embedding retriever-redis lvm dataprep-multimodal-redis whisper"
docker compose -f build.yaml build ${service_list} --no-cache > ${LOG_PATH}/docker_image_build.log
docker pull ghcr.io/huggingface/tgi-gaudi:2.0.6
@@ -214,12 +214,12 @@ function validate_microservices() {
'{"inputs":"![](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/rabbit.png)What is this a picture of?\n\n","parameters":{"max_new_tokens":16, "seed": 42}}'
# lvm
echo "Evaluating lvm-tgi"
echo "Evaluating lvm"
validate_service \
"http://${host_ip}:9399/v1/lvm" \
'"text":"' \
"lvm-tgi" \
"lvm-tgi" \
"lvm" \
"lvm" \
'{"retrieved_docs": [], "initial_query": "What is this?", "top_n": 1, "metadata": [{"b64_img_str": "iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAAFUlEQVR42mP8/5+hnoEIwDiqkL4KAcT9GO0U4BxoAAAAAElFTkSuQmCC", "transcript_for_inference": "yellow image", "video_id": "8c7461df-b373-4a00-8696-9a2234359fe0", "time_of_frame_ms":"37000000", "source_video":"WeAreGoingOnBullrun_8c7461df-b373-4a00-8696-9a2234359fe0.mp4"}], "chat_template":"The caption of the image is: '\''{context}'\''. {question}"}'
# data prep requiring lvm

8
MultimodalQnA/tests/test_compose_on_rocm.sh
View File

@@ -23,7 +23,7 @@ function build_docker_images() {
git clone https://github.com/opea-project/GenAIComps.git && cd GenAIComps && git checkout "${opea_branch:-"main"}" && cd ../
echo "Build all the images with --no-cache, check docker_image_build.log for details..."
service_list="multimodalqna multimodalqna-ui embedding-multimodal-bridgetower embedding retriever-redis lvm-tgi lvm-llava-svc dataprep-multimodal-redis whisper"
service_list="multimodalqna multimodalqna-ui embedding-multimodal-bridgetower embedding retriever-redis lvm dataprep-multimodal-redis whisper"
docker compose -f build.yaml build ${service_list} --no-cache > ${LOG_PATH}/docker_image_build.log
docker images && sleep 1m
@@ -220,12 +220,12 @@ function validate_microservices() {
'{"inputs":"![](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/rabbit.png)What is this a picture of?\n\n","parameters":{"max_new_tokens":16, "seed": 42}}'
# lvm
echo "Evaluating lvm-llava-svc"
echo "Evaluating lvm"
validate_service \
"http://${host_ip}:9399/v1/lvm" \
'"text":"' \
"lvm-tgi" \
"lvm-tgi" \
"lvm" \
"lvm" \
'{"retrieved_docs": [], "initial_query": "What is this?", "top_n": 1, "metadata": [{"b64_img_str": "iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAAFUlEQVR42mP8/5+hnoEIwDiqkL4KAcT9GO0U4BxoAAAAAElFTkSuQmCC", "transcript_for_inference": "yellow image", "video_id": "8c7461df-b373-4a00-8696-9a2234359fe0", "time_of_frame_ms":"37000000", "source_video":"WeAreGoingOnBullrun_8c7461df-b373-4a00-8696-9a2234359fe0.mp4"}], "chat_template":"The caption of the image is: '\''{context}'\''. {question}"}'
# data prep requiring lvm

8
MultimodalQnA/tests/test_compose_on_xeon.sh
View File

@@ -22,7 +22,7 @@ function build_docker_images() {
cd $WORKPATH/docker_image_build
git clone https://github.com/opea-project/GenAIComps.git && cd GenAIComps && git checkout "${opea_branch:-"main"}" && cd ../
echo "Build all the images with --no-cache, check docker_image_build.log for details..."
service_list="multimodalqna multimodalqna-ui embedding-multimodal-bridgetower embedding retriever-redis lvm-llava lvm-llava-svc dataprep-multimodal-redis whisper"
service_list="multimodalqna multimodalqna-ui embedding-multimodal-bridgetower embedding retriever-redis lvm-llava lvm dataprep-multimodal-redis whisper"
docker compose -f build.yaml build ${service_list} --no-cache > ${LOG_PATH}/docker_image_build.log
docker images && sleep 1m
@@ -212,12 +212,12 @@ function validate_microservices() {
'{"prompt":"Describe the image please.", "img_b64_str": "iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAAFUlEQVR42mP8/5+hnoEIwDiqkL4KAcT9GO0U4BxoAAAAAElFTkSuQmCC"}'
# lvm
echo "Evaluating lvm-llava-svc"
echo "Evaluating lvm"
validate_service \
"http://${host_ip}:9399/v1/lvm" \
'"text":"' \
"lvm-llava-svc" \
"lvm-llava-svc" \
"lvm" \
"lvm" \
'{"retrieved_docs": [], "initial_query": "What is this?", "top_n": 1, "metadata": [{"b64_img_str": "iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAAFUlEQVR42mP8/5+hnoEIwDiqkL4KAcT9GO0U4BxoAAAAAElFTkSuQmCC", "transcript_for_inference": "yellow image", "video_id": "8c7461df-b373-4a00-8696-9a2234359fe0", "time_of_frame_ms":"37000000", "source_video":"WeAreGoingOnBullrun_8c7461df-b373-4a00-8696-9a2234359fe0.mp4"}], "chat_template":"The caption of the image is: '\''{context}'\''. {question}"}'
# data prep requiring lvm

17
VideoQnA/docker_compose/intel/cpu/xeon/README.md
View File

@@ -71,10 +71,10 @@ docker build -t opea/reranking:latest --build-arg https_proxy=$https_proxy --bui
### 4. Build LVM Image (Xeon)
```bash
docker build -t opea/video-llama-lvm-server:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/video-llama/dependency/Dockerfile .
docker build -t opea/lvm-video-llama:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/src/integrations/dependency/video-llama/Dockerfile .
# LVM Service Image
docker build -t opea/lvm-video-llama:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/video-llama/Dockerfile .
docker build -t opea/lvm:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/src/Dockerfile .
```
### 5. Build Dataprep Image
@@ -109,11 +109,16 @@ Then run the command `docker images`, you will have the following 8 Docker Image
1. `opea/dataprep-multimodal-vdms:latest`
2. `opea/embedding-multimodal-clip:latest`
3. `opea/retriever-vdms:latest`
<<<<<<< HEAD
4. `opea/reranking:latest`
5. `opea/video-llama-lvm-server:latest`
6. `opea/lvm-video-llama:latest`
7. `opea/videoqna:latest`
8. `opea/videoqna-ui:latest`
6. # `opea/lvm-video-llama:latest`
7. `opea/reranking-tei:latest`
8. `opea/lvm-video-llama:latest`
9. `opea/lvm:latest`
> > > > > > > d93597cbfd9da92b956adb3673c9e5d743c181af
10. `opea/videoqna:latest`
11. `opea/videoqna-ui:latest`
## 🚀 Start Microservices
@@ -275,7 +280,7 @@ docker compose up -d
In first startup, this service will take times to download the LLM file. After it's finished, the service will be ready.
Use `docker logs video-llama-lvm-server` to check if the download is finished.
Use `docker logs lvm-video-llama` to check if the download is finished.
```bash
curl -X POST \

9
VideoQnA/docker_compose/intel/cpu/xeon/compose.yaml
View File

@@ -75,8 +75,8 @@ services:
DATAPREP_GET_VIDEO_LIST_ENDPOINT: ${DATAPREP_GET_VIDEO_LIST_ENDPOINT}
restart: unless-stopped
lvm-video-llama:
image: ${REGISTRY:-opea}/video-llama-lvm-server:${TAG:-latest}
container_name: video-llama-lvm-server
image: ${REGISTRY:-opea}/lvm-video-llama:${TAG:-latest}
container_name: lvm-video-llama
ports:
- "9009:9009"
ipc: host
@@ -90,8 +90,8 @@ services:
- video-llama-model:/home/user/model
restart: unless-stopped
lvm:
image: ${REGISTRY:-opea}/lvm-video-llama:${TAG:-latest}
container_name: lvm-video-llama
image: ${REGISTRY:-opea}/lvm:${TAG:-latest}
container_name: lvm
ports:
- "9000:9000"
ipc: host
@@ -99,6 +99,7 @@ services:
http_proxy: ${http_proxy}
https_proxy: ${https_proxy}
no_proxy: ${no_proxy}
LVM_COMPONENT_NAME: "OPEA_VIDEO_LLAMA_LVM"
LVM_ENDPOINT: ${LVM_ENDPOINT}
restart: unless-stopped
depends_on:

14
VideoQnA/docker_image_build/build.yaml
View File

@@ -41,15 +41,15 @@ services:
dockerfile: comps/rerankings/src/Dockerfile
extends: videoqna
image: ${REGISTRY:-opea}/reranking:${TAG:-latest}
video-llama-lvm-server:
build:
context: GenAIComps
dockerfile: comps/lvms/video-llama/dependency/Dockerfile
extends: videoqna
image: ${REGISTRY:-opea}/video-llama-lvm-server:${TAG:-latest}
lvm-video-llama:
build:
context: GenAIComps
dockerfile: comps/lvms/video-llama/Dockerfile
dockerfile: comps/lvms/src/integrations/dependency/video-llama/Dockerfile
extends: videoqna
image: ${REGISTRY:-opea}/lvm-video-llama:${TAG:-latest}
lvm:
build:
context: GenAIComps
dockerfile: comps/lvms/src/Dockerfile
extends: videoqna
image: ${REGISTRY:-opea}/lvm:${TAG:-latest}

2
VideoQnA/tests/test_compose_on_xeon.sh
View File

@@ -51,7 +51,7 @@ function start_services() {
sleep 1m
# List of containers running uvicorn
list=("dataprep-vdms-server" "embedding-multimodal-server" "retriever-vdms-server" "reranking-tei-server" "video-llama-lvm-server" "lvm-video-llama" "videoqna-xeon-backend-server")
list=("dataprep-vdms-server" "embedding-multimodal-server" "retriever-vdms-server" "reranking-tei-server" "lvm-video-llama" "lvm-video-llama" "videoqna-xeon-backend-server")
# Define the maximum time limit in seconds
TIME_LIMIT=5400

4
VisualQnA/docker_compose/amd/gpu/rocm/README.md
View File

@@ -11,7 +11,7 @@ First of all, you need to build Docker Images locally and install the python pac
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
docker build --no-cache -t opea/lvm-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/tgi-llava/Dockerfile .
docker build --no-cache -t opea/lvm:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/src/Dockerfile .
docker build --no-cache -t opea/nginx:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/third_parties/nginx/src/Dockerfile .
```
@@ -43,7 +43,7 @@ docker pull ghcr.io/huggingface/text-generation-inference:2.4.1-rocm
Then run the command `docker images`, you will have the following 5 Docker Images:
1. `ghcr.io/huggingface/text-generation-inference:2.4.1-rocm`
2. `opea/lvm-tgi:latest`
2. `opea/lvm:latest`
3. `opea/visualqna:latest`
4. `opea/visualqna-ui:latest`
5. `opea/nginx`

9
VisualQnA/docker_compose/amd/gpu/rocm/compose.yaml
View File

@@ -28,9 +28,9 @@ services:
- seccomp:unconfined
ipc: host
command: --model-id ${LVM_MODEL_ID} --max-input-length 4096 --max-total-tokens 8192
lvm-tgi:
image: ${REGISTRY:-opea}/lvm-tgi:${TAG:-latest}
container_name: lvm-tgi-server
lvm:
image: ${REGISTRY:-opea}/lvm:${TAG:-latest}
container_name: lvm-server
depends_on:
- visualqna-llava-tgi-service
ports:
@@ -41,6 +41,7 @@ services:
http_proxy: ${http_proxy}
https_proxy: ${https_proxy}
LVM_ENDPOINT: ${LVM_ENDPOINT}
LVM_COMPONENT_NAME: "OPEA_TGI_LLAVA_LVM"
HF_HUB_DISABLE_PROGRESS_BARS: 1
HF_HUB_ENABLE_HF_TRANSFER: 0
restart: unless-stopped
@@ -49,7 +50,7 @@ services:
container_name: visualqna-rocm-backend-server
depends_on:
- visualqna-llava-tgi-service
- lvm-tgi
- lvm
ports:
- "${BACKEND_SERVICE_PORT:-8888}:8888"
environment:

4
VisualQnA/docker_compose/intel/cpu/xeon/README.md
View File

@@ -41,7 +41,7 @@ First of all, you need to build Docker Images locally and install the python pac
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
docker build --no-cache -t opea/lvm-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/tgi-llava/Dockerfile .
docker build --no-cache -t opea/lvm:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/src/Dockerfile .
docker build --no-cache -t opea/nginx:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/third_parties/nginx/src/Dockerfile .
```
@@ -73,7 +73,7 @@ docker pull ghcr.io/huggingface/text-generation-inference:2.4.0-intel-cpu
Then run the command `docker images`, you will have the following 5 Docker Images:
1. `ghcr.io/huggingface/text-generation-inference:2.4.0-intel-cpu`
2. `opea/lvm-tgi:latest`
2. `opea/lvm:latest`
3. `opea/visualqna:latest`
4. `opea/visualqna-ui:latest`
5. `opea/nginx`

9
VisualQnA/docker_compose/intel/cpu/xeon/compose.yaml
View File

@@ -23,9 +23,9 @@ services:
timeout: 10s
retries: 60
command: --model-id ${LVM_MODEL_ID} --max-input-length 4096 --max-total-tokens 8192 --cuda-graphs 0
lvm-tgi:
image: ${REGISTRY:-opea}/lvm-tgi:${TAG:-latest}
container_name: lvm-tgi-xeon-server
lvm:
image: ${REGISTRY:-opea}/lvm:${TAG:-latest}
container_name: lvm-xeon-server
depends_on:
llava-tgi-service:
condition: service_healthy
@@ -37,6 +37,7 @@ services:
http_proxy: ${http_proxy}
https_proxy: ${https_proxy}
LVM_ENDPOINT: ${LVM_ENDPOINT}
LVM_COMPONENT_NAME: "OPEA_TGI_LLAVA_LVM"
HF_HUB_DISABLE_PROGRESS_BARS: 1
HF_HUB_ENABLE_HF_TRANSFER: 0
restart: unless-stopped
@@ -45,7 +46,7 @@ services:
container_name: visualqna-xeon-backend-server
depends_on:
- llava-tgi-service
- lvm-tgi
- lvm
ports:
- "8888:8888"
environment:

4
VisualQnA/docker_compose/intel/hpu/gaudi/README.md
View File

@@ -11,7 +11,7 @@ First of all, you need to build Docker Images locally. This step can be ignored
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
docker build --no-cache -t opea/lvm-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/tgi-llava/Dockerfile .
docker build --no-cache -t opea/lvm:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/src/Dockerfile .
docker build --no-cache -t opea/nginx:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/third_parties/nginx/src/Dockerfile .
```
@@ -44,7 +44,7 @@ docker build --no-cache -t opea/visualqna-ui:latest --build-arg https_proxy=$htt
Then run the command `docker images`, you will have the following 5 Docker Images:
1. `ghcr.io/huggingface/tgi-gaudi:2.0.6`
2. `opea/lvm-tgi:latest`
2. `opea/lvm:latest`
3. `opea/visualqna:latest`
4. `opea/visualqna-ui:latest`
5. `opea/nginx`

9
VisualQnA/docker_compose/intel/hpu/gaudi/compose.yaml
View File

@@ -27,9 +27,9 @@ services:
- SYS_NICE
ipc: host
command: --model-id ${LVM_MODEL_ID} --max-input-length 4096 --max-total-tokens 8192
lvm-tgi:
image: ${REGISTRY:-opea}/lvm-tgi:${TAG:-latest}
container_name: lvm-tgi-gaudi-server
lvm:
image: ${REGISTRY:-opea}/lvm:${TAG:-latest}
container_name: lvm-gaudi-server
depends_on:
- llava-tgi-service
ports:
@@ -40,6 +40,7 @@ services:
http_proxy: ${http_proxy}
https_proxy: ${https_proxy}
LVM_ENDPOINT: ${LVM_ENDPOINT}
LVM_COMPONENT_NAME: "OPEA_TGI_LLAVA_LVM"
HF_HUB_DISABLE_PROGRESS_BARS: 1
HF_HUB_ENABLE_HF_TRANSFER: 0
restart: unless-stopped
@@ -48,7 +49,7 @@ services:
container_name: visualqna-gaudi-backend-server
depends_on:
- llava-tgi-service
- lvm-tgi
- lvm
ports:
- "8888:8888"
environment:

6
VisualQnA/docker_image_build/build.yaml
View File

@@ -17,12 +17,12 @@ services:
dockerfile: ./docker/Dockerfile
extends: visualqna
image: ${REGISTRY:-opea}/visualqna-ui:${TAG:-latest}
lvm-tgi:
lvm:
build:
context: GenAIComps
dockerfile: comps/lvms/tgi-llava/Dockerfile
dockerfile: comps/lvms/src/Dockerfile
extends: visualqna
image: ${REGISTRY:-opea}/lvm-tgi:${TAG:-latest}
image: ${REGISTRY:-opea}/lvm:${TAG:-latest}
nginx:
build:
context: GenAIComps

6
VisualQnA/tests/test_compose_on_gaudi.sh
View File

@@ -49,7 +49,7 @@ function start_services() {
n=0
until [[ "$n" -ge 100 ]]; do
docker logs lvm-tgi-gaudi-server > ${LOG_PATH}/lvm_tgi_service_start.log
docker logs lvm-gaudi-server > ${LOG_PATH}/lvm_tgi_service_start.log
if grep -q Connected ${LOG_PATH}/lvm_tgi_service_start.log; then
break
fi
@@ -93,8 +93,8 @@ function validate_microservices() {
validate_services \
"${ip_address}:9399/v1/lvm" \
"The image" \
"lvm-tgi" \
"lvm-tgi-gaudi-server" \
"lvm" \
"lvm-gaudi-server" \
'{"image": "iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAAFUlEQVR42mP8/5+hnoEIwDiqkL4KAcT9GO0U4BxoAAAAAElFTkSuQmCC", "prompt":"What is this?"}'
}

2
VisualQnA/tests/test_compose_on_rocm.sh
View File

@@ -99,7 +99,7 @@ function validate_microservices() {
validate_services \
"${ip_address}:9399/v1/lvm" \
"The image" \
"lvm-tgi" \
"lvm" \
"visualqna-tgi-service" \
'{"image": "iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAAFUlEQVR42mP8/5+hnoEIwDiqkL4KAcT9GO0U4BxoAAAAAElFTkSuQmCC", "prompt":"What is this?"}'
}

6
VisualQnA/tests/test_compose_on_xeon.sh
View File

@@ -50,7 +50,7 @@ function start_services() {
n=0
until [[ "$n" -ge 200 ]]; do
docker logs lvm-tgi-xeon-server > ${LOG_PATH}/lvm_tgi_service_start.log
docker logs lvm-xeon-server > ${LOG_PATH}/lvm_tgi_service_start.log
if grep -q Connected ${LOG_PATH}/lvm_tgi_service_start.log; then
break
fi
@@ -94,8 +94,8 @@ function validate_microservices() {
validate_services \
"${ip_address}:9399/v1/lvm" \
"The image" \
"lvm-tgi" \
"lvm-tgi-xeon-server" \
"lvm" \
"lvm-xeon-server" \
'{"image": "iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAAFUlEQVR42mP8/5+hnoEIwDiqkL4KAcT9GO0U4BxoAAAAAElFTkSuQmCC", "prompt":"What is this?"}'
}

17
docker_images_list.md
View File

@@ -41,7 +41,7 @@ Take ChatQnA for example. ChatQnA is a chatbot application service based on the
## Microservice images
| Microservice Images | Dockerfile | Description |
| ------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| ------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [opea/agent]() | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/agent/src/Dockerfile) | The docker image exposed the OPEA agent microservice for GenAI application use |
| [opea/asr](https://hub.docker.com/r/opea/asr) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/asr/src/Dockerfile) | The docker image exposed the OPEA Audio-Speech-Recognition microservice for GenAI application use |
| [opea/chathistory-mongo-server](https://hub.docker.com/r/opea/chathistory-mongo-server) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/chathistory/mongo/Dockerfile) | The docker image exposes OPEA Chat History microservice which based on MongoDB database, designed to allow user to store, retrieve and manage chat conversations |
@@ -69,13 +69,14 @@ Take ChatQnA for example. ChatQnA is a chatbot application service based on the
| [opea/guardrails-toxicity-detection](https://hub.docker.com/r/opea/guardrails-toxicity-detection) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/guardrails/src/toxicity_detection/Dockerfile) | The docker image exposed the OPEA guardrail microservice to provide toxicity detection for GenAI application use |
| [opea/guardrails-pii-detection](https://hub.docker.com/r/opea/guardrails-pii-detection) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/guardrails/src/pii_detection/Dockerfile) | The docker image exposed the OPEA guardrail microservice to provide PII detection for GenAI application use |
| [opea/llm-docsum-tgi](https://hub.docker.com/r/opea/llm-docsum-tgi) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/llms/summarization/tgi/langchain/Dockerfile) | This docker image is designed to build a document summarization microservice using the HuggingFace Text Generation Inference(TGI) framework. The microservice accepts document input and generates a document summary. |
| [opea/llm-faqgen]() | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/llms/src/faq-generation/Dockerfile) | This docker image is designed to build a frequently asked questions microservice using the HuggingFace Text Generation Inference(TGI) framework. The microservice accepts document input and generates a FAQ. |
| [opea/llm-faqgen]() | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/llms/src/faq-generation/Dockerfile) | This docker image is designed to build a frequently asked questions microservice using the HuggingFace Text Generation Inference(TGI) framework. The microservice accepts document input and generates a FAQ. |
| [opea/llm-textgen](https://hub.docker.com/r/opea/llm-textgen) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/llms/src/text-generation/Dockerfile) | The docker image exposed the OPEA LLM microservice upon TGI docker image for GenAI application use |
| [opea/llava-gaudi](https://hub.docker.com/r/opea/llava-hpu) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/lvms/llava/dependency/Dockerfile.intel_hpu) | The docker image exposed the OPEA microservice running LLaVA as a large visual model (LVM) service for GenAI application use on the Gaudi |
| [opea/lvm-tgi](https://hub.docker.com/r/opea/lvm-tgi) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/lvms/tgi-llava/Dockerfile) | This docker image is designed to build a large visual model (LVM) microservice using the HuggingFace Text Generation Inference(TGI) framework. The microservice accepts document input and generates a answer to question. |
| [opea/lvm-llava](https://hub.docker.com/r/opea/lvm-llava) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/lvms/llava/dependency/Dockerfile) | The docker image exposed the OPEA microservice running LLaVA as a large visual model (LVM) server for GenAI application use |
| [opea/lvm-llava-svc](https://hub.docker.com/r/opea/lvm-llava-svc) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/lvms/llava/Dockerfile) | The docker image exposed the OPEA microservice running LLaVA as a large visual model (LVM) service for GenAI application use |
| [opea/lvm-video-llama](https://hub.docker.com/r/opea/lvm-video-llama) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/lvms/video-llama/Dockerfile) | The docker image exposed the OPEA microservice running Video-Llama as a large visual model (LVM) for GenAI application use |
| [opea/llava-gaudi](https://hub.docker.com/r/opea/llava-hpu) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/lvms/src/integrations/dependency/llava/Dockerfile.intel_hpu) | The docker image exposed the OPEA microservice running LLaVA as a large visual model (LVM) service for GenAI application use on the Gaudi |
| [opea/lvm]() | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/lvms/src/Dockerfile) | This docker image is designed to build a large visual model (LVM) unified wrapper service |
| [opea/lvm-llava](https://hub.docker.com/r/opea/lvm-llava) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/lvms/src/integrations/dependency/llava/Dockerfile) | The docker image exposed the OPEA microservice running LLaVA as a large visual model (LVM) server for GenAI application use |
| [opea/lvm-video-llama](https://hub.docker.com/r/opea/lvm-video-llama) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/lvms/src/integrations/dependency/video-llama/Dockerfile) | The docker image exposed the OPEA microservice running Video LLaMA as a large visual model (LVM) server for GenAI application use |
| [opea/lvm-llama-vision]() | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/lvms/src/integrations/dependency/llama-vision/Dockerfile) | The docker image exposed the OPEA microservice running LLaMA Vision as a large visual model (LVM) server for GenAI application use |
| [opea/lvm-predictionguard](https://hub.docker.com/r/opea/lvm-predictionguard) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/lvms/src/integrations/dependency/predictionguard/Dockerfile) | The docker image exposed the OPEA microservice running PredictionGuard as a large visual model (LVM) server for GenAI application use |
| [opea/nginx](https://hub.docker.com/r/opea/nginx) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/third_parties/nginx/src/Dockerfile) | The docker image exposed the OPEA nginx microservice for GenAI application use |
| [opea/promptregistry-mongo-server](https://hub.docker.com/r/opea/promptregistry-mongo-server) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/prompt_registry/src/Dockerfile) | The docker image exposes the OPEA Prompt Registry microservices which based on MongoDB database, designed to store and retrieve user's preferred prompts |
| [opea/reranking]() | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/rerankings/src/Dockerfile) | The docker image exposed the OPEA reranking microservice based on tei docker image for GenAI application use |
@@ -91,7 +92,7 @@ Take ChatQnA for example. ChatQnA is a chatbot application service based on the
| [opea/speecht5-gaudi](https://hub.docker.com/r/opea/speecht5-gaudi) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/tts/src/integrations/dependency/speecht5/Dockerfile.intel_hpu) | The docker image exposed the OPEA SpeechT5 service on Gaudi2 for GenAI application use |
| [opea/tei-gaudi](https://hub.docker.com/r/opea/tei-gaudi/tags) | [Link](https://github.com/huggingface/tei-gaudi/blob/habana-main/Dockerfile-hpu) | The docker image powered by HuggingFace Text Embedding Inference (TEI) on Gaudi2 for deploying and serving Embedding Models |
| [opea/vectorstore-pathway](https://hub.docker.com/r/opea/vectorstore-pathway) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/vectorstores/pathway/Dockerfile) | The docker image exposed the OPEA Vectorstores microservice with Pathway for GenAI application use |
| [opea/video-llama-lvm-server](https://hub.docker.com/r/opea/video-llama-lvm-server) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/lvms/video-llama/dependency/Dockerfile) | The docker image exposed the OPEA microservice running Video-Llama as a large visual model (LVM) server for GenAI application use |
| [opea/lvm-video-llama](https://hub.docker.com/r/opea/lvm-video-llama) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/lvms/src/integrations/dependency/video-llama/Dockerfile) | The docker image exposed the OPEA microservice running Video-Llama as a large visual model (LVM) server for GenAI application use |
| [opea/tts](https://hub.docker.com/r/opea/tts) | [Link](https://github.com/opea-project/GenAIComps/blob/main/comps/tts/src/Dockerfile) | The docker image exposed the OPEA Text-To-Speech microservice for GenAI application use |
| [opea/vllm](https://hub.docker.com/r/opea/vllm) | [Link](https://github.com/vllm-project/vllm/blob/main/Dockerfile.cpu) | The docker image powered by vllm-project for deploying and serving vllm Models |
| [opea/vllm-gaudi]() | [Link](https://github.com/HabanaAI/vllm-fork/blob/habana_main/Dockerfile.hpu) | The docker image powered by vllm-fork for deploying and serving vllm-gaudi Models |