jandres/GenAIExamples
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

Fix README issues (#817)

Browse Source 
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
lvliang-intel
2024-09-18 09:50:17 +08:00
committed by GitHub
parent 375ea7a90c
commit bceacdc804
24 changed files with 106 additions and 243 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
ChatQnA/docker_compose/intel/cpu/xeon/README.md
View File

@@ -61,14 +61,11 @@ Port 5173 - Open to 0.0.0.0/0
First of all, you need to build Docker Images locally and install the python package of it.
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 1. Build Embedding Image
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
docker build --no-cache -t opea/embedding-tei:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/embeddings/tei/langchain/Dockerfile .
```
@@ -128,7 +125,6 @@ cd ..
git clone https://github.com/opea-project/GenAIExamples.git
cd GenAIExamples/ChatQnA
docker build --no-cache -t opea/chatqna:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
cd ../../..
```
2. MegaService without Rerank
@@ -139,7 +135,6 @@ cd ..
git clone https://github.com/opea-project/GenAIExamples.git
cd GenAIExamples/ChatQnA
docker build --no-cache -t opea/chatqna-without-rerank:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile.without_rerank .
cd ../../..
```
### 7. Build UI Docker Image
@@ -149,7 +144,6 @@ Build frontend Docker image via below command:
```bash
cd GenAIExamples/ChatQnA/ui
docker build --no-cache -t opea/chatqna-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f ./docker/Dockerfile .
cd ../../../..
```
### 8. Build Conversational React UI Docker Image (Optional)
@@ -161,7 +155,6 @@ Build frontend Docker image that enables Conversational experience with ChatQnA
```bash
cd GenAIExamples/ChatQnA/ui
docker build --no-cache -t opea/chatqna-conversation-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f ./docker/Dockerfile.react .
cd ../../../..
```
Then run the command `docker images`, you will have the following 7 Docker Images:
@@ -188,7 +181,7 @@ By default, the embedding, reranking and LLM models are set to a default value a
Change the `xxx_MODEL_ID` below for your needs.
For customers with proxy issues, the models from [ModelScope](https://www.modelscope.cn/models) are also supported in ChatQnA with TGI serving. ModelScope models are supported in two ways for TGI:
For users in China who are unable to download models directly from Huggingface, you can use [ModelScope](https://www.modelscope.cn/models) or a Huggingface mirror to download models. TGI can load the models either online or offline as described below:
1. Online
@@ -196,7 +189,7 @@ For customers with proxy issues, the models from [ModelScope](https://www.models
export HF_TOKEN=${your_hf_token}
export HF_ENDPOINT="https://hf-mirror.com"
model_name="Intel/neural-chat-7b-v3-3"
docker run -p 8008:80 -v ./data:/data --name tgi-service -e HF_ENDPOINT=$HF_ENDPOINT -e http_proxy=$http_proxy -e https_proxy=$https_proxy --shm-size 1g ghcr.io/huggingface/text-generation-inference:2.1.0 --model-id $model_name
docker run -p 8008:80 -v ./data:/data --name tgi-service -e HF_ENDPOINT=$HF_ENDPOINT -e http_proxy=$http_proxy -e https_proxy=$https_proxy --shm-size 1g ghcr.io/huggingface/text-generation-inference:2.2.0 --model-id $model_name
```
2. Offline
@@ -210,7 +203,7 @@ For customers with proxy issues, the models from [ModelScope](https://www.models
```bash
export HF_TOKEN=${your_hf_token}
export model_path="/path/to/model"
docker run -p 8008:80 -v $model_path:/data --name tgi_service --shm-size 1g ghcr.io/huggingface/text-generation-inference:2.1.0 --model-id /data
docker run -p 8008:80 -v $model_path:/data --name tgi_service --shm-size 1g ghcr.io/huggingface/text-generation-inference:2.2.0 --model-id /data
```
### Setup Environment Variables

60
ChatQnA/docker_compose/intel/hpu/gaudi/README.md
View File

@@ -6,26 +6,21 @@ This document outlines the deployment process for a ChatQnA application utilizin
First of all, you need to build Docker Images locally. This step can be ignored after the Docker images published to Docker hub.
### 1. Source Code install GenAIComps
### 1. Build Embedding Image
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 2. Build Embedding Image
```bash
docker build --no-cache -t opea/embedding-tei:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/embeddings/tei/langchain/Dockerfile .
```
### 3. Build Retriever Image
### 2. Build Retriever Image
```bash
docker build --no-cache -t opea/retriever-redis:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/retrievers/redis/langchain/Dockerfile .
```
### 4. Build Rerank Image
### 3. Build Rerank Image
> Skip for ChatQnA without Rerank pipeline
@@ -33,17 +28,17 @@ docker build --no-cache -t opea/retriever-redis:latest --build-arg https_proxy=$
docker build --no-cache -t opea/reranking-tei:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/reranks/tei/Dockerfile .
```
### 5. Build LLM Image
### 4. Build LLM Image
You can use different LLM serving solutions, choose one of following four options.
#### 5.1 Use TGI
#### 4.1 Use TGI
```bash
docker build --no-cache -t opea/llm-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/llms/text-generation/tgi/Dockerfile .
```
#### 5.2 Use VLLM
#### 4.2 Use VLLM
Build vllm docker.
@@ -57,7 +52,7 @@ Build microservice docker.
docker build --no-cache -t opea/llm-vllm:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/llms/text-generation/vllm/langchain/Dockerfile .
```
#### 5.3 Use VLLM-on-Ray
#### 4.3 Use VLLM-on-Ray
Build vllm-on-ray docker.
@@ -71,24 +66,21 @@ Build microservice docker.
docker build --no-cache -t opea/llm-vllm-ray:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/llms/text-generation/vllm/ray/Dockerfile .
```
### 6. Build Dataprep Image
### 5. Build Dataprep Image
```bash
docker build --no-cache -t opea/dataprep-redis:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/dataprep/redis/langchain/Dockerfile .
```
### 7. Build TEI Gaudi Image
### 6. Build Guardrails Docker Image (Optional)
Since a TEI Gaudi Docker image hasn't been published, we'll need to build it from the [tei-gaudi](https://github.com/huggingface/tei-gaudi) repository.
To fortify AI initiatives in production, Guardrails microservice can secure model inputs and outputs, building Trustworthy, Safe, and Secure LLM-based Applications.
```bash
git clone https://github.com/huggingface/tei-gaudi
cd tei-gaudi/
docker build --no-cache -f Dockerfile-hpu -t opea/tei-gaudi:latest .
cd ../..
docker build -t opea/guardrails-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/guardrails/llama_guard/langchain/Dockerfile .
```
### 8. Build MegaService Docker Image
### 7. Build MegaService Docker Image
1. MegaService with Rerank
@@ -98,7 +90,6 @@ cd ../..
git clone https://github.com/opea-project/GenAIExamples.git
cd GenAIExamples/ChatQnA/docker
docker build --no-cache -t opea/chatqna:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
cd ../../..
```
2. MegaService with Guardrails
@@ -109,7 +100,6 @@ cd ../..
git clone https://github.com/opea-project/GenAIExamples.git
cd GenAIExamples/ChatQnA/
docker build --no-cache -t opea/chatqna-guardrails:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile.guardrails .
cd ../../..
```
3. MegaService without Rerank
@@ -120,20 +110,18 @@ cd ../..
git clone https://github.com/opea-project/GenAIExamples.git
cd GenAIExamples/ChatQnA/docker
docker build --no-cache -t opea/chatqna-without-rerank:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile.without_rerank .
cd ../../..
```
### 9. Build UI Docker Image
### 8. Build UI Docker Image
Construct the frontend Docker image using the command below:
```bash
cd GenAIExamples/ChatQnA/ui
docker build --no-cache -t opea/chatqna-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f ./docker/Dockerfile .
cd ../../../..
```
### 10. Build Conversational React UI Docker Image (Optional)
### 9. Build Conversational React UI Docker Image (Optional)
Build frontend Docker image that enables Conversational experience with ChatQnA megaservice via below command:
@@ -142,26 +130,14 @@ Build frontend Docker image that enables Conversational experience with ChatQnA
```bash
cd GenAIExamples/ChatQnA/ui
docker build --no-cache -t opea/chatqna-conversation-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f ./docker/Dockerfile.react .
cd ../../../..
```
### 11. Build Guardrails Docker Image (Optional)
To fortify AI initiatives in production, Guardrails microservice can secure model inputs and outputs, building Trustworthy, Safe, and Secure LLM-based Applications.
```bash
cd GenAIComps
docker build -t opea/guardrails-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/guardrails/llama_guard/langchain/Dockerfile .
cd ../../..
```
Then run the command `docker images`, you will have the following 8 Docker Images:
Then run the command `docker images`, you will have the following 7 Docker Images:
- `opea/embedding-tei:latest`
- `opea/retriever-redis:latest`
- `opea/reranking-tei:latest`
- `opea/llm-tgi:latest` or `opea/llm-vllm:latest` or `opea/llm-vllm-ray:latest`
- `opea/tei-gaudi:latest`
- `opea/dataprep-redis:latest`
- `opea/chatqna:latest` or `opea/chatqna-guardrails:latest` or `opea/chatqna-without-rerank:latest`
- `opea/chatqna-ui:latest`
@@ -188,7 +164,7 @@ By default, the embedding, reranking and LLM models are set to a default value a
Change the `xxx_MODEL_ID` below for your needs.
For customers with proxy issues, the models from [ModelScope](https://www.modelscope.cn/models) are also supported in ChatQnA with TGI serving. ModelScope models are supported in two ways for TGI:
For users in China who are unable to download models directly from Huggingface, you can use [ModelScope](https://www.modelscope.cn/models) or a Huggingface mirror to download models. TGI can load the models either online or offline as described below:
1. Online
@@ -196,7 +172,7 @@ For customers with proxy issues, the models from [ModelScope](https://www.models
export HF_TOKEN=${your_hf_token}
export HF_ENDPOINT="https://hf-mirror.com"
model_name="Intel/neural-chat-7b-v3-3"
docker run -p 8008:80 -v ./data:/data --name tgi-service -e HF_ENDPOINT=$HF_ENDPOINT -e http_proxy=$http_proxy -e https_proxy=$https_proxy --shm-size 1g ghcr.io/huggingface/text-generation-inference:2.1.0 --model-id $model_name
docker run -p 8008:80 -v ./data:/data --name tgi-service -e HF_ENDPOINT=$HF_ENDPOINT -e http_proxy=$http_proxy -e https_proxy=$https_proxy --runtime=habana -e HABANA_VISIBLE_DEVICES=all -e OMPI_MCA_btl_vader_single_copy_mechanism=none -e HUGGING_FACE_HUB_TOKEN=$HF_TOKEN -e ENABLE_HPU_GRAPH=true -e LIMIT_HPU_GRAPH=true -e USE_FLASH_ATTENTION=true -e FLASH_ATTENTION_RECOMPUTE=true --cap-add=sys_nice --ipc=host ghcr.io/huggingface/tgi-gaudi:2.0.5 --model-id $model_name --max-input-tokens 1024 --max-total-tokens 2048
```
2. Offline
@@ -210,7 +186,7 @@ For customers with proxy issues, the models from [ModelScope](https://www.models
```bash
export HF_TOKEN=${your_hf_token}
export model_path="/path/to/model"
docker run -p 8008:80 -v $model_path:/data --name tgi_service --shm-size 1g ghcr.io/huggingface/text-generation-inference:2.1.0 --model-id /data
docker run -p 8008:80 -v $model_path:/data --name tgi_service --runtime=habana -e HABANA_VISIBLE_DEVICES=all -e OMPI_MCA_btl_vader_single_copy_mechanism=none -e HUGGING_FACE_HUB_TOKEN=$HF_TOKEN -e ENABLE_HPU_GRAPH=true -e LIMIT_HPU_GRAPH=true -e USE_FLASH_ATTENTION=true -e FLASH_ATTENTION_RECOMPUTE=true --cap-add=sys_nice --ipc=host ghcr.io/huggingface/tgi-gaudi:2.0.5 --model-id /data --max-input-tokens 1024 --max-total-tokens 2048
```
### Setup Environment Variables

13
CodeGen/docker_compose/intel/cpu/xeon/README.md
View File

@@ -14,20 +14,15 @@ After launching your instance, you can connect to it using SSH (for Linux instan
Should the Docker image you seek not yet be available on Docker Hub, you can build the Docker image locally.
### 1. Git Clone GenAIComps
### 1. Build the LLM Docker Image
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 2. Build the LLM Docker Image
```bash
docker build -t opea/llm-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/llms/text-generation/tgi/Dockerfile .
```
### 3. Build the MegaService Docker Image
### 2. Build the MegaService Docker Image
To construct the Mega Service, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `codegen.py` Python script. Build MegaService Docker image via the command below:
@@ -37,7 +32,7 @@ cd GenAIExamples/CodeGen
docker build -t opea/codegen:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
```
### 4. Build the UI Docker Image
### 3. Build the UI Docker Image
Build the frontend Docker image via the command below:
@@ -52,7 +47,7 @@ Then run the command `docker images`, you will have the following 3 Docker Image
- `opea/codegen:latest`
- `opea/codegen-ui:latest`
### 8. Build CodeGen React UI Docker Image (Optional)
### 4. Build CodeGen React UI Docker Image (Optional)
Build react frontend Docker image via below command:

13
CodeGen/docker_compose/intel/hpu/gaudi/README.md
View File

@@ -6,20 +6,15 @@ This document outlines the deployment process for a CodeGen application utilizin
First of all, you need to build the Docker images locally. This step can be ignored after the Docker images published to the Docker Hub.
### 1. Git Clone GenAIComps
### 1. Build the LLM Docker Image
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 2. Build the LLM Docker Image
```bash
docker build -t opea/llm-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/llms/text-generation/tgi/Dockerfile .
```
### 3. Build the MegaService Docker Image
### 2. Build the MegaService Docker Image
To construct the Mega Service, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `codegen.py` Python script. Build the MegaService Docker image via the command below:
@@ -29,7 +24,7 @@ cd GenAIExamples/CodeGen
docker build -t opea/codegen:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
```
### 4. Build the UI Docker Image
### 3. Build the UI Docker Image
Construct the frontend Docker image via the command below:
@@ -38,7 +33,7 @@ cd GenAIExamples/CodeGen/ui
docker build -t opea/codegen-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f ./docker/Dockerfile .
```
### 8. Build CodeGen React UI Docker Image (Optional)
### 4. Build CodeGen React UI Docker Image (Optional)
Build react frontend Docker image via below command:

13
CodeTrans/docker_compose/intel/cpu/xeon/README.md
View File

@@ -14,20 +14,15 @@ After launching your instance, you can connect to it using SSH (for Linux instan
First of all, you need to build Docker Images locally and install the python package of it. This step can be ignored after the Docker images published to Docker hub.
### 1. Install GenAIComps from Source Code
### 1. Build the LLM Docker Image
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 2. Build the LLM Docker Image
```bash
docker build -t opea/llm-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/llms/text-generation/tgi/Dockerfile .
```
### 3. Build MegaService Docker Image
### 2. Build MegaService Docker Image
```bash
git clone https://github.com/opea-project/GenAIExamples.git
@@ -35,14 +30,14 @@ cd GenAIExamples/CodeTrans
docker build -t opea/codetrans:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
```
### 4. Build UI Docker Image
### 3. Build UI Docker Image
```bash
cd GenAIExamples/CodeTrans/ui
docker build -t opea/codetrans-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f ./docker/Dockerfile .
```
### 5. Build Nginx Docker Image
### 4. Build Nginx Docker Image
```bash
cd GenAIComps

13
CodeTrans/docker_compose/intel/hpu/gaudi/README.md
View File

@@ -6,20 +6,15 @@ This document outlines the deployment process for a CodeTrans application utiliz
First of all, you need to build Docker Images locally and install the python package of it. This step can be ignored after the Docker images published to Docker hub.
### 1. Source Code install GenAIComps
### 1. Build the LLM Docker Image
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 2. Build the LLM Docker Image
```bash
docker build -t opea/llm-tgi:latest --no-cache --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/llms/text-generation/tgi/Dockerfile .
```
### 3. Build MegaService Docker Image
### 2. Build MegaService Docker Image
```bash
git clone https://github.com/opea-project/GenAIExamples.git
@@ -27,14 +22,14 @@ cd GenAIExamples/CodeTrans
docker build -t opea/codetrans:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
```
### 4. Build UI Docker Image
### 3. Build UI Docker Image
```bash
cd GenAIExamples/CodeTrans/ui
docker build -t opea/codetrans-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f docker/Dockerfile .
```
### 5. Build Nginx Docker Image
### 4. Build Nginx Docker Image
```bash
cd GenAIComps

7
DocSum/docker_compose/intel/cpu/xeon/README.md
View File

@@ -14,14 +14,11 @@ After launching your instance, you can connect to it using SSH (for Linux instan
First of all, you need to build Docker Images locally and install the python package of it.
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 1. Build LLM Image
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
docker build -t opea/llm-docsum-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/llms/summarization/tgi/langchain/Dockerfile .
```

11
DocSum/docker_compose/intel/hpu/gaudi/README.md
View File

@@ -6,22 +6,19 @@ This document outlines the deployment process for a Document Summarization appli
First of all, you need to build Docker Images locally. This step can be ignored once the Docker images are published to Docker hub.
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 1. Pull TGI Gaudi Image
As TGI Gaudi has been officially published as a Docker image, we simply need to pull it:
```bash
docker pull ghcr.io/huggingface/tgi-gaudi:2.0.1
docker pull ghcr.io/huggingface/tgi-gaudi:2.0.5
```
### 2. Build LLM Image
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
docker build -t opea/llm-docsum-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/llms/summarization/tgi/langchain/Dockerfile .
```
@@ -56,7 +53,7 @@ docker build -t opea/docsum-react-ui:latest --build-arg BACKEND_SERVICE_ENDPOINT
Then run the command `docker images`, you will have the following Docker Images:
1. `ghcr.io/huggingface/tgi-gaudi:2.0.1`
1. `ghcr.io/huggingface/tgi-gaudi:2.0.5`
2. `opea/llm-docsum-tgi:latest`
3. `opea/docsum:latest`
4. `opea/docsum-ui:latest`

7
FaqGen/docker_compose/intel/cpu/xeon/README.md
View File

@@ -14,14 +14,11 @@ After launching your instance, you can connect to it using SSH (for Linux instan
First of all, you need to build Docker Images locally. This step can be ignored once the Docker images are published to Docker hub.
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 1. Build LLM Image
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
docker build -t opea/llm-faqgen-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/llms/faq-generation/tgi/langchain/Dockerfile .
```

11
FaqGen/docker_compose/intel/hpu/gaudi/README.md
View File

@@ -6,22 +6,19 @@ This document outlines the deployment process for a FAQ Generation application u
First of all, you need to build Docker Images locally. This step can be ignored once the Docker images are published to Docker hub.
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 1. Pull TGI Gaudi Image
As TGI Gaudi has been officially published as a Docker image, we simply need to pull it:
```bash
docker pull ghcr.io/huggingface/tgi-gaudi:2.0.1
docker pull ghcr.io/huggingface/tgi-gaudi:2.0.5
```
### 2. Build LLM Image
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
docker build -t opea/llm-faqgen-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/llms/faq-generation/tgi/langchain/Dockerfile .
```
@@ -56,7 +53,7 @@ docker build -t opea/faqgen-react-ui:latest --build-arg https_proxy=$https_proxy
Then run the command `docker images`, you will have the following Docker Images:
1. `ghcr.io/huggingface/tgi-gaudi:2.0.1`
1. `ghcr.io/huggingface/tgi-gaudi:2.0.5`
2. `opea/llm-faqgen-tgi:latest`
3. `opea/faqgen:latest`
4. `opea/faqgen-ui:latest`

13
InstructionTuning/docker_compose/intel/cpu/xeon/README.md
View File

@@ -6,23 +6,18 @@ This document outlines the deployment process for a Instruction Tuning Service u
First of all, you need to build Docker Images locally. This step can be ignored after the Docker images published to Docker hub.
### 1. Source Code install GenAIComps
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 2. Build Docker Image
### 1. Build Docker Image
Build docker image with below command:
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
export HF_TOKEN=${your_huggingface_token}
docker build -t opea/finetuning:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy --build-arg HF_TOKEN=$HF_TOKEN -f comps/finetuning/Dockerfile .
```
### 3. Run Docker with CLI
### 2. Run Docker with CLI
Start docker container with below command:

13
InstructionTuning/docker_compose/intel/hpu/gaudi/README.md
View File

@@ -6,22 +6,17 @@ This document outlines the deployment process for a Instruction Tuning Service u
First of all, you need to build Docker Images locally. This step can be ignored after the Docker images published to Docker hub.
### 1. Source Code install GenAIComps
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 2. Build Docker Image
### 1. Build Docker Image
Build docker image with below command:
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
docker build -t opea/finetuning-gaudi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/finetuning/Dockerfile.intel_hpu .
```
### 3. Run Docker with CLI
### 2. Run Docker with CLI
Start docker container with below command:

13
MultimodalQnA/docker_compose/intel/cpu/xeon/README.md
View File

@@ -100,18 +100,13 @@ Note: Please replace with `host_ip` with you external IP address, do not use loc
## 🚀 Build Docker Images
First of all, you need to build Docker Images locally and install the python package of it.
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 1. Build embedding-multimodal-bridgetower Image
Build embedding-multimodal-bridgetower docker image
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
docker build --no-cache -t opea/embedding-multimodal-bridgetower:latest --build-arg EMBEDDER_PORT=$EMBEDDER_PORT --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/embeddings/multimodal/bridgetower/Dockerfile .
```
@@ -340,6 +335,6 @@ curl http://${host_ip}:8888/v1/multimodalqna \
```bash
curl http://${host_ip}:8888/v1/multimodalqna \
-H "Content-Type: application/json" \
-d '{"messages": [{"role": "user", "content": [{"type": "text", "text": "hello, "}, {"type": "image_url", "image_url": {"url": "https://www.ilankelman.org/stopsigns/australia.jpg"}}]}, {"role": "assistant", "content": "opea project! "}, {"role": "user", "content": "chao, "}], "max_tokens": 10}'
-H "Content-Type: application/json" \
-d '{"messages": [{"role": "user", "content": [{"type": "text", "text": "hello, "}, {"type": "image_url", "image_url": {"url": "https://www.ilankelman.org/stopsigns/australia.jpg"}}]}, {"role": "assistant", "content": "opea project! "}, {"role": "user", "content": "chao, "}], "max_tokens": 10}'
```

15
MultimodalQnA/docker_compose/intel/hpu/gaudi/README.md
View File

@@ -1,4 +1,4 @@
# Build Mega Service of MultimodalRAGWithVideos on Gaudi
# Build Mega Service of MultimodalQnA on Gaudi
This document outlines the deployment process for a MultimodalQnA application utilizing the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline on Intel Gaudi server. The steps include Docker image creation, container deployment via Docker Compose, and service execution to integrate microservices such as `multimodal_embedding` that employs [BridgeTower](https://huggingface.co/BridgeTower/bridgetower-large-itm-mlm-gaudi) model as embedding model, `multimodal_retriever`, `lvm`, and `multimodal-data-prep`. We will publish the Docker images to Docker Hub soon, it will simplify the deployment process for this service.
@@ -52,16 +52,13 @@ Note: Please replace with `host_ip` with you external IP address, do not use loc
First of all, you need to build Docker Images locally and install the python package of it.
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 1. Build embedding-multimodal-bridgetower Image
Build embedding-multimodal-bridgetower docker image
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
docker build --no-cache -t opea/embedding-multimodal-bridgetower:latest --build-arg EMBEDDER_PORT=$EMBEDDER_PORT --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/embeddings/multimodal/bridgetower/Dockerfile .
```
@@ -82,7 +79,7 @@ docker build --no-cache -t opea/retriever-multimodal-redis:latest --build-arg ht
Build TGI Gaudi image
```bash
docker pull ghcr.io/huggingface/tgi-gaudi:2.0.4
docker pull ghcr.io/huggingface/tgi-gaudi:2.0.5
```
Build lvm-tgi microservice image
@@ -105,7 +102,6 @@ To construct the Mega Service, we utilize the [GenAIComps](https://github.com/op
git clone https://github.com/opea-project/GenAIExamples.git
cd GenAIExamples/MultimodalQnA
docker build --no-cache -t opea/multimodalqna:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
cd ../..
```
### 6. Build UI Docker Image
@@ -115,14 +111,13 @@ Build frontend Docker image via below command:
```bash
cd GenAIExamples/MultimodalQnA/ui/
docker build --no-cache -t opea/multimodalqna-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f ./docker/Dockerfile .
cd ../../../
```
Then run the command `docker images`, you will have the following 8 Docker Images:
1. `opea/dataprep-multimodal-redis:latest`
2. `opea/lvm-tgi:latest`
3. `ghcr.io/huggingface/tgi-gaudi:2.0.4`
3. `ghcr.io/huggingface/tgi-gaudi:2.0.5`
4. `opea/retriever-multimodal-redis:latest`
5. `opea/embedding-multimodal:latest`
6. `opea/embedding-multimodal-bridgetower:latest`

12
ProductivitySuite/docker_compose/intel/cpu/xeon/README.md
View File

@@ -6,14 +6,11 @@ This document outlines the deployment process for OPEA Productivity Suite utiliz
First of all, you need to build Docker Images locally and install the python package of it.
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 1. Build Embedding Image
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
docker build --no-cache -t opea/embedding-tei:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/embeddings/tei/langchain/Dockerfile .
```
@@ -69,7 +66,6 @@ The Productivity Suite is composed of multiple GenAIExample reference solutions
git clone https://github.com/opea-project/GenAIExamples.git
cd GenAIExamples/ChatQnA/
docker build --no-cache -t opea/chatqna:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
cd ../../..
```
#### 8.2 Build DocSum Megaservice Docker Images
@@ -77,7 +73,6 @@ cd ../../..
```bash
cd GenAIExamples/DocSum
docker build --no-cache -t opea/docsum:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
cd ../../..
```
#### 8.3 Build CodeGen Megaservice Docker Images
@@ -85,7 +80,6 @@ cd ../../..
```bash
cd GenAIExamples/CodeGen
docker build --no-cache -t opea/codegen:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
cd ../../..
```
#### 8.4 Build FAQGen Megaservice Docker Images
@@ -93,7 +87,6 @@ cd ../../..
```bash
cd GenAIExamples/FaqGen
docker build --no-cache -t opea/faqgen:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
cd ../../..
```
### 9. Build UI Docker Image
@@ -105,7 +98,6 @@ Build frontend Docker image that enables via below command:
```bash
cd GenAIExamples/ProductivitySuite/ui
docker build --no-cache -t ProductivitySuite/docker_compose/intel/cpu/xeon/compose.yaml docker/Dockerfile.react .
cd ../../../..
```
## 🚀 Start Microservices

13
RerankFinetuning/docker_compose/intel/cpu/xeon/README.md
View File

@@ -6,23 +6,18 @@ This document outlines the deployment process for a rerank model finetuning serv
First of all, you need to build Docker Images locally. This step can be ignored after the Docker images published to Docker hub.
### 1. Source Code install GenAIComps
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 2. Build Docker Image
### 1. Build Docker Image
Build docker image with below command:
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
export HF_TOKEN=${your_huggingface_token}
docker build -t opea/finetuning:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy --build-arg HF_TOKEN=$HF_TOKEN -f comps/finetuning/Dockerfile .
```
### 3. Run Docker with CLI
### 2. Run Docker with CLI
Start docker container with below command:

13
RerankFinetuning/docker_compose/intel/hpu/gaudi/README.md
View File

@@ -6,22 +6,17 @@ This document outlines the deployment process for a rerank model finetuning serv
First of all, you need to build Docker Images locally. This step can be ignored after the Docker images published to Docker hub.
### 1. Source Code install GenAIComps
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 2. Build Docker Image
### 1. Build Docker Image
Build docker image with below command:
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
docker build -t opea/finetuning-gaudi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/finetuning/Dockerfile.intel_hpu .
```
### 3. Run Docker with CLI
### 2. Run Docker with CLI
Start docker container with below command:

19
SearchQnA/docker_compose/intel/cpu/xeon/README.md
View File

@@ -4,38 +4,33 @@ This document outlines the deployment process for a SearchQnA application utiliz
## 🚀 Build Docker images
### 1. Source Code install GenAIComps
### 1. Build Embedding Image
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 2. Build Embedding Image
```bash
docker build --no-cache -t opea/embedding-tei:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/embeddings/tei/langchain/Dockerfile .
```
### 3. Build Retriever Image
### 2. Build Retriever Image
```bash
docker build --no-cache -t opea/web-retriever-chroma:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/web_retrievers/chroma/langchain/Dockerfile .
```
### 4. Build Rerank Image
### 3. Build Rerank Image
```bash
docker build --no-cache -t opea/reranking-tei:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/reranks/tei/Dockerfile .
```
### 5. Build LLM Image
### 4. Build LLM Image
```bash
docker build --no-cache -t opea/llm-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/llms/text-generation/tgi/Dockerfile .
```
### 6. Build MegaService Docker Image
### 5. Build MegaService Docker Image
To construct the Mega Service, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `searchqna.py` Python script. Build the MegaService Docker image using the command below:
@@ -43,17 +38,15 @@ To construct the Mega Service, we utilize the [GenAIComps](https://github.com/op
git clone https://github.com/opea-project/GenAIExamples.git
cd GenAIExamples/SearchQnA
docker build --no-cache -t opea/searchqna:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
cd ../../..
```
### 7. Build UI Docker Image
### 6. Build UI Docker Image
Build frontend Docker image via below command:
```bash
cd GenAIExamples/SearchQnA/ui
docker build --no-cache -t opea/opea/searchqna-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f ./docker/Dockerfile .
cd ../../../..
```
Then run the command `docker images`, you will have following images ready:

18
SearchQnA/docker_compose/intel/hpu/gaudi/README.md
View File

@@ -6,38 +6,33 @@ This document outlines the deployment process for a SearchQnA application utiliz
First of all, you need to build Docker Images locally. This step can be ignored after the Docker images published to Docker hub.
### 1. Source Code install GenAIComps
### 1. Build Embedding Image
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 2. Build Embedding Image
```bash
docker build --no-cache -t opea/embedding-tei:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/embeddings/tei/langchain/Dockerfile .
```
### 3. Build Retriever Image
### 2. Build Retriever Image
```bash
docker build --no-cache -t opea/web-retriever-chroma:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/web_retrievers/chroma/langchain/Dockerfile .
```
### 4. Build Rerank Image
### 3. Build Rerank Image
```bash
docker build --no-cache -t opea/reranking-tei:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/reranks/tei/Dockerfile .
```
### 5. Build LLM Image
### 4. Build LLM Image
```bash
docker build --no-cache -t opea/llm-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/llms/text-generation/tgi/Dockerfile .
```
### 6. Build TEI Gaudi Image
### 5. Build TEI Gaudi Image
Since a TEI Gaudi Docker image hasn't been published, we'll need to build it from the [tei-guadi](https://github.com/huggingface/tei-gaudi) repository.
@@ -48,7 +43,7 @@ docker build --no-cache -f Dockerfile-hpu -t opea/tei-gaudi:latest .
cd ../..
```
### 7. Build MegaService Docker Image
### 6. Build MegaService Docker Image
To construct the Mega Service, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `searchqna.py` Python script. Build the MegaService Docker image using the command below:
@@ -56,7 +51,6 @@ To construct the Mega Service, we utilize the [GenAIComps](https://github.com/op
git clone https://github.com/opea-project/GenAIExamples.git
cd GenAIExamples/SearchQnA/docker
docker build --no-cache -t opea/searchqna:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
cd ../../..
```
Then you need to build the last Docker image `opea/searchqna:latest`, which represents the Mega service through following commands:

7
Translation/docker_compose/intel/cpu/xeon/README.md
View File

@@ -14,14 +14,11 @@ After launching your instance, you can connect to it using SSH (for Linux instan
First of all, you need to build Docker Images locally and install the python package of it.
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 1. Build LLM Image
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
docker build -t opea/llm-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/llms/text-generation/tgi/Dockerfile .
```

9
Translation/docker_compose/intel/hpu/gaudi/README.md
View File

@@ -6,14 +6,11 @@ This document outlines the deployment process for a Translation application util
First of all, you need to build Docker Images locally. This step can be ignored after the Docker images published to Docker hub.
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 1. Build LLM Image
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
docker build -t opea/llm-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/llms/text-generation/tgi/Dockerfile .
```
@@ -32,7 +29,7 @@ docker build -t opea/translation:latest --build-arg https_proxy=$https_proxy --b
Construct the frontend Docker image using the command below:
```bash
cd GenAIExamples/Translation//
cd GenAIExamples/Translation
docker build -t opea/translation-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f ./docker/Dockerfile .
```

10
VideoQnA/docker_compose/intel/cpu/xeon/README.md
View File

@@ -48,14 +48,11 @@ Port 5173 - Open to 0.0.0.0/0
First of all, you need to build Docker Images locally and install the python package of it.
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 1. Build Embedding Image
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
docker build -t opea/embedding-multimodal-clip:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/embeddings/multimodal_clip/Dockerfile .
```
@@ -84,7 +81,6 @@ docker build -t opea/lvm-video-llama:latest --build-arg https_proxy=$https_proxy
```bash
docker build -t opea/dataprep-multimodal-vdms:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/dataprep/vdms/multimodal_langchain/Dockerfile .
cd ..
```
### 6. Build MegaService Docker Image
@@ -104,7 +100,7 @@ docker build -t opea/videoqna:latest --build-arg https_proxy=$https_proxy --buil
Build frontend Docker image via below command:
```bash
cd ui
cd GenAIExamples/VideoQnA/ui/
docker build -t opea/videoqna-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f ./docker/Dockerfile .
```

10
VisualQnA/docker_compose/intel/cpu/xeon/README.md
View File

@@ -36,16 +36,12 @@ Port 5173 - Open to 0.0.0.0/0
First of all, you need to build Docker Images locally and install the python package of it.
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 1. Build LVM and NGINX Docker Images
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
docker build --no-cache -t opea/lvm-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/tgi-llava/Dockerfile .
docker build --no-cache -t opea/nginx:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/nginx/Dockerfile .
```
@@ -57,7 +53,6 @@ To construct the Mega Service, we utilize the [GenAIComps](https://github.com/op
git clone https://github.com/opea-project/GenAIExamples.git
cd GenAIExamples/VisualQnA
docker build --no-cache -t opea/visualqna:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
cd ../..
```
### 3. Build UI Docker Image
@@ -67,7 +62,6 @@ Build frontend Docker image via below command:
```bash
cd GenAIExamples/VisualQnA/ui
docker build --no-cache -t opea/visualqna-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f docker/Dockerfile .
cd ../../..
```
### 4. Pull TGI Xeon Image

19
VisualQnA/docker_compose/intel/hpu/gaudi/README.md
View File

@@ -6,28 +6,22 @@ This document outlines the deployment process for a VisualQnA application utiliz
First of all, you need to build Docker Images locally. This step can be ignored after the Docker images published to Docker hub.
### 1. Source Code install GenAIComps
### 1. Build LVM and NGINX Docker Images
```bash
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
```
### 2. Build LVM and NGINX Docker Images
```bash
docker build --no-cache -t opea/lvm-tgi:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/lvms/tgi-llava/Dockerfile .
docker build --no-cache -t opea/nginx:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/nginx/Dockerfile .
```
### 3. Pull TGI Gaudi Image
### 2. Pull TGI Gaudi Image
```bash
docker pull ghcr.io/huggingface/tgi-gaudi:2.0.4
docker pull ghcr.io/huggingface/tgi-gaudi:2.0.5
```
### 4. Build MegaService Docker Image
### 3. Build MegaService Docker Image
To construct the Mega Service, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `visuralqna.py` Python script. Build the MegaService Docker image using the command below:
@@ -38,19 +32,18 @@ docker build --no-cache -t opea/visualqna:latest --build-arg https_proxy=$https_
cd ../..
```
### 5. Build UI Docker Image
### 4. Build UI Docker Image
Build frontend Docker image via below command:
```bash
cd GenAIExamples/VisualQnA/ui
docker build --no-cache -t opea/visualqna-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f ./docker/Dockerfile .
cd ../../..
```
Then run the command `docker images`, you will have the following 5 Docker Images:
1. `ghcr.io/huggingface/tgi-gaudi:2.0.4`
1. `ghcr.io/huggingface/tgi-gaudi:2.0.5`
2. `opea/lvm-tgi:latest`
3. `opea/visualqna:latest`
4. `opea/visualqna-ui:latest`