[ChatQnA] Update README for without Rerank Pipeline (#740)

* update readme for chatqna w/o rerank Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: letonghan <letong.han@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-09-05 14:49:12 +08:00
parent 43b2ae59a1
commit 6b617d6743
3 changed files with 119 additions and 27 deletions
--- a/ChatQnA/README.md
+++ b/ChatQnA/README.md
@@ -101,9 +101,13 @@ This ChatQnA use case performs RAG using LangChain, Redis VectorDB and Text Gene

 The ChatQnA service can be effortlessly deployed on either Intel Gaudi2 or Intel XEON Scalable Processors.

+Two types of ChatQnA pipeline are supported now: `ChatQnA with/without Rerank`. And the `ChatQnA without Rerank` pipeline (including Embedding, Retrieval, and LLM) is offered for Xeon customers who can not run rerank service on HPU yet require high performance and accuracy.
+
+### Prepare Docker Image
+
 Currently we support two ways of deploying ChatQnA services with docker compose:

-1. Start services using the docker image on `docker hub`:
+1. Using the docker image on `docker hub`:

   ```bash
   docker pull opea/chatqna:latest
@@ -119,7 +123,9 @@ Currently we support two ways of deploying ChatQnA services with docker compose:
   docker pull opea/chatqna-conversation-ui:latest
   ```

-2. Start services using the docker images `built from source`: [Guide](./docker)
+2. Using the docker images `built from source`: [Guide](docker/xeon/README.md)
+
+   > Note: The **opea/chatqna-without-rerank:latest** docker image has not been published yet, users need to build this docker image from source.

 ### Required Models

--- a/ChatQnA/docker/gaudi/README.md
+++ b/ChatQnA/docker/gaudi/README.md
@@ -27,6 +27,8 @@ docker build --no-cache -t opea/retriever-redis:latest --build-arg https_proxy=$

 ### 4. Build Rerank Image

+> Skip for ChatQnA without Rerank pipeline
+
 ```bash
 docker build --no-cache -t opea/reranking-tei:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/reranks/tei/docker/Dockerfile .
 ```
@@ -88,23 +90,38 @@ cd ../..

 ### 8. Build MegaService Docker Image

-To construct the Mega Service, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `chatqna.py` Python script. Build the MegaService Docker image using the command below:
+1. MegaService with Rerank

-```bash
-git clone https://github.com/opea-project/GenAIExamples.git
-cd GenAIExamples/ChatQnA/docker
-docker build --no-cache -t opea/chatqna:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
-cd ../../..
-```
+   To construct the Mega Service with Rerank, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `chatqna.py` Python script. Build the MegaService Docker image using the command below:

-If you want to enable guardrails microservice in the pipeline, please use the below command instead:
+   ```bash
+   git clone https://github.com/opea-project/GenAIExamples.git
+   cd GenAIExamples/ChatQnA/docker
+   docker build --no-cache -t opea/chatqna:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
+   cd ../../..
+   ```

-```bash
-git clone https://github.com/opea-project/GenAIExamples.git
-cd GenAIExamples/ChatQnA/docker
-docker build --no-cache -t opea/chatqna-guardrails:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile_guardrails .
-cd ../../..
-```
+2. MegaService with Guardrails
+
+   If you want to enable guardrails microservice in the pipeline, please use the below command instead:
+
+   ```bash
+   git clone https://github.com/opea-project/GenAIExamples.git
+   cd GenAIExamples/ChatQnA/docker
+   docker build --no-cache -t opea/chatqna-guardrails:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile_guardrails .
+   cd ../../..
+   ```
+
+3. MegaService without Rerank
+
+   To construct the Mega Service without Rerank, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `chatqna_without_rerank.py` Python script. Build MegaService Docker image via below command:
+
+   ```bash
+   git clone https://github.com/opea-project/GenAIExamples.git
+   cd GenAIExamples/ChatQnA/docker
+   docker build --no-cache -t opea/chatqna-without-rerank:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile_without_rerank .
+   cd ../../..
+   ```

 ### 9. Build UI Docker Image

@@ -146,7 +163,7 @@ Then run the command `docker images`, you will have the following 8 Docker Image
 4. `opea/llm-tgi:latest` or `opea/llm-vllm:latest` or `opea/llm-vllm-ray:latest`
 5. `opea/tei-gaudi:latest`
 6. `opea/dataprep-redis:latest`
-7. `opea/chatqna:latest` or `opea/chatqna-guardrails:latest`
+7. `opea/chatqna:latest` or `opea/chatqna-guardrails:latest` or `opea/chatqna-without-rerank:latest`
 8. `opea/chatqna-ui:latest`

 If Conversation React UI is built, you will find one more image:
@@ -223,7 +240,10 @@ cd GenAIExamples/ChatQnA/docker/gaudi/
 If use tgi for llm backend.

 ```bash
+# Start ChatQnA with Rerank Pipeline
 docker compose -f compose.yaml up -d
+# Start ChatQnA without Rerank Pipeline
+docker compose -f compose_without_rerank.yaml up -d
 ```

 If use vllm for llm backend.
@@ -288,6 +308,8 @@ curl http://${host_ip}:7000/v1/retrieval \

 4. TEI Reranking Service

+> Skip for ChatQnA without Rerank pipeline
+
 ```bash
 curl http://${host_ip}:8808/rerank \
    -X POST \
@@ -297,6 +319,8 @@ curl http://${host_ip}:8808/rerank \

 5. Reranking Microservice

+> Skip for ChatQnA without Rerank pipeline
+
 ```bash
 curl http://${host_ip}:8000/v1/reranking \
  -X POST \
@@ -397,12 +421,31 @@ curl -X POST "http://${host_ip}:6007/v1/dataprep/get_file" \
     -H "Content-Type: application/json"
 ```

+Then you will get the response JSON like this. Notice that the returned `name`/`id` of the uploaded link is `https://xxx.txt`.
+
+```json
+[
+  {
+    "name": "nke-10k-2023.pdf",
+    "id": "nke-10k-2023.pdf",
+    "type": "File",
+    "parent": ""
+  },
+  {
+    "name": "https://opea.dev.txt",
+    "id": "https://opea.dev.txt",
+    "type": "File",
+    "parent": ""
+  }
+]
+```
+
 To delete the file/link you uploaded:

 ```bash
 # delete link
 curl -X POST "http://${host_ip}:6007/v1/dataprep/delete_file" \
-     -d '{"file_path": "https://opea.dev"}' \
+     -d '{"file_path": "https://opea.dev.txt"}' \
     -H "Content-Type: application/json"

 # delete file
--- a/ChatQnA/docker/xeon/README.md
+++ b/ChatQnA/docker/xeon/README.md
@@ -80,6 +80,8 @@ docker build --no-cache -t opea/retriever-redis:latest --build-arg https_proxy=$

 ### 3. Build Rerank Image

+> Skip for ChatQnA without Rerank pipeline
+
 ```bash
 docker build --no-cache -t opea/reranking-tei:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/reranks/tei/docker/Dockerfile .
 ```
@@ -118,14 +120,27 @@ cd ..

 ### 6. Build MegaService Docker Image

-To construct the Mega Service, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `chatqna.py` Python script. Build MegaService Docker image via below command:
+1. MegaService with Rerank

-```bash
-git clone https://github.com/opea-project/GenAIExamples.git
-cd GenAIExamples/ChatQnA/docker
-docker build --no-cache -t opea/chatqna:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
-cd ../../..
-```
+   To construct the Mega Service with Rerank, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `chatqna.py` Python script. Build MegaService Docker image via below command:
+
+   ```bash
+   git clone https://github.com/opea-project/GenAIExamples.git
+   cd GenAIExamples/ChatQnA/docker
+   docker build --no-cache -t opea/chatqna:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
+   cd ../../..
+   ```
+
+2. MegaService without Rerank
+
+   To construct the Mega Service without Rerank, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `chatqna_without_rerank.py` Python script. Build MegaService Docker image via below command:
+
+   ```bash
+   git clone https://github.com/opea-project/GenAIExamples.git
+   cd GenAIExamples/ChatQnA/docker
+   docker build --no-cache -t opea/chatqna-without-rerank:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile_without_rerank .
+   cd ../../..
+   ```

 ### 7. Build UI Docker Image

@@ -156,7 +171,7 @@ Then run the command `docker images`, you will have the following 7 Docker Image
 3. `opea/retriever-redis:latest`
 4. `opea/reranking-tei:latest`
 5. `opea/llm-tgi:latest` or `opea/llm-vllm:latest`
-6. `opea/chatqna:latest`
+6. `opea/chatqna:latest` or `opea/chatqna-without-rerank:latest`
 7. `opea/chatqna-ui:latest`

 ## 🚀 Start Microservices
@@ -238,7 +253,10 @@ cd GenAIExamples/ChatQnA/docker/xeon/
 If use TGI backend.

 ```bash
+# Start ChatQnA with Rerank Pipeline
 docker compose -f compose.yaml up -d
+# Start ChatQnA without Rerank Pipeline
+docker compose -f compose_without_rerank.yaml up -d
 ```

 If use vLLM backend.
@@ -285,6 +303,8 @@ curl http://${host_ip}:7000/v1/retrieval \

 4. TEI Reranking Service

+> Skip for ChatQnA without Rerank pipeline
+
 ```bash
 curl http://${host_ip}:8808/rerank \
    -X POST \
@@ -294,6 +314,8 @@ curl http://${host_ip}:8808/rerank \

 5. Reranking Microservice

+> Skip for ChatQnA without Rerank pipeline
+
 ```bash
 curl http://${host_ip}:8000/v1/reranking\
  -X POST \
@@ -384,12 +406,33 @@ curl -X POST "http://${host_ip}:6007/v1/dataprep/get_file" \
     -H "Content-Type: application/json"
 ```

+Then you will get the response JSON like this. Notice that the returned `name`/`id` of the uploaded link is `https://xxx.txt`.
+
+```json
+[
+  {
+    "name": "nke-10k-2023.pdf",
+    "id": "nke-10k-2023.pdf",
+    "type": "File",
+    "parent": ""
+  },
+  {
+    "name": "https://opea.dev.txt",
+    "id": "https://opea.dev.txt",
+    "type": "File",
+    "parent": ""
+  }
+]
+```
+
 To delete the file/link you uploaded:

+The `file_path` here should be the `id` get from `/v1/dataprep/get_file` API.
+
 ```bash
 # delete link
 curl -X POST "http://${host_ip}:6007/v1/dataprep/delete_file" \
-     -d '{"file_path": "https://opea.dev"}' \
+     -d '{"file_path": "https://opea.dev.txt"}' \
     -H "Content-Type: application/json"

 # delete file