From 2a2ff45e2b9dde9d986234784511a1de9d0e72cc Mon Sep 17 00:00:00 2001 From: Letong Han <106566639+letonghan@users.noreply.github.com> Date: Thu, 29 Aug 2024 21:22:59 +0800 Subject: [PATCH] Explain Default Model in ChatQnA and CodeTrans READMEs (#694) * explain default model in CodeTrans READMEs Signed-off-by: letonghan * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * explain default model in ChatQnA READMEs Signed-off-by: letonghan * add required models Signed-off-by: letonghan * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: letonghan Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> --- ChatQnA/README.md | 12 ++++++++++++ ChatQnA/docker/gaudi/README.md | 12 ++++++++++++ ChatQnA/docker/gpu/README.md | 12 ++++++++++++ ChatQnA/docker/xeon/README.md | 14 +++++++++++++- ChatQnA/docker/xeon/README_qdrant.md | 12 ++++++++++++ CodeTrans/README.md | 10 ++++++++++ CodeTrans/docker/gaudi/README.md | 12 ++++++++++-- CodeTrans/docker/xeon/README.md | 12 ++++++++++-- CodeTrans/kubernetes/README.md | 11 +++++++++++ CodeTrans/kubernetes/manifests/README.md | 10 ++++++++++ 10 files changed, 112 insertions(+), 5 deletions(-) diff --git a/ChatQnA/README.md b/ChatQnA/README.md index c1db8e15a..0697c8630 100644 --- a/ChatQnA/README.md +++ b/ChatQnA/README.md @@ -121,6 +121,18 @@ Currently we support two ways of deploying ChatQnA services with docker compose: 2. Start services using the docker images `built from source`: [Guide](./docker) +### Required Models + +By default, the embedding, reranking and LLM models are set to a default value as listed below: + +| Service | Model | +| --------- | ------------------------- | +| Embedding | BAAI/bge-base-en-v1.5 | +| Reranking | BAAI/bge-reranker-base | +| LLM | Intel/neural-chat-7b-v3-3 | + +Change the `xxx_MODEL_ID` in `docker/xxx/set_env.sh` for your needs. + ### Setup Environment Variable To set up environment variables for deploying ChatQnA services, follow these steps: diff --git a/ChatQnA/docker/gaudi/README.md b/ChatQnA/docker/gaudi/README.md index 75bd412d5..053484f77 100644 --- a/ChatQnA/docker/gaudi/README.md +++ b/ChatQnA/docker/gaudi/README.md @@ -159,6 +159,18 @@ If Guardrails docker image is built, you will find one more image: ## 🚀 Start MicroServices and MegaService +### Required Models + +By default, the embedding, reranking and LLM models are set to a default value as listed below: + +| Service | Model | +| --------- | ------------------------- | +| Embedding | BAAI/bge-base-en-v1.5 | +| Reranking | BAAI/bge-reranker-base | +| LLM | Intel/neural-chat-7b-v3-3 | + +Change the `xxx_MODEL_ID` below for your needs. + ### Setup Environment Variables Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below. diff --git a/ChatQnA/docker/gpu/README.md b/ChatQnA/docker/gpu/README.md index 41c2aef32..48c287fb5 100644 --- a/ChatQnA/docker/gpu/README.md +++ b/ChatQnA/docker/gpu/README.md @@ -87,6 +87,18 @@ Then run the command `docker images`, you will have the following 7 Docker Image ## 🚀 Start MicroServices and MegaService +### Required Models + +By default, the embedding, reranking and LLM models are set to a default value as listed below: + +| Service | Model | +| --------- | ------------------------- | +| Embedding | BAAI/bge-base-en-v1.5 | +| Reranking | BAAI/bge-reranker-base | +| LLM | Intel/neural-chat-7b-v3-3 | + +Change the `xxx_MODEL_ID` below for your needs. + ### Setup Environment Variables Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below. diff --git a/ChatQnA/docker/xeon/README.md b/ChatQnA/docker/xeon/README.md index b607e8532..dc8735928 100644 --- a/ChatQnA/docker/xeon/README.md +++ b/ChatQnA/docker/xeon/README.md @@ -161,6 +161,18 @@ Then run the command `docker images`, you will have the following 7 Docker Image ## 🚀 Start Microservices +### Required Models + +By default, the embedding, reranking and LLM models are set to a default value as listed below: + +| Service | Model | +| --------- | ------------------------- | +| Embedding | BAAI/bge-base-en-v1.5 | +| Reranking | BAAI/bge-reranker-base | +| LLM | Intel/neural-chat-7b-v3-3 | + +Change the `xxx_MODEL_ID` below for your needs. + ### Setup Environment Variables Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below. @@ -183,7 +195,7 @@ export your_hf_api_token="Your_Huggingface_API_Token" **Append the value of the public IP address to the no_proxy list** -``` +```bash export your_no_proxy=${your_no_proxy},"External_Public_IP" ``` diff --git a/ChatQnA/docker/xeon/README_qdrant.md b/ChatQnA/docker/xeon/README_qdrant.md index fd28d306a..f103d5a73 100644 --- a/ChatQnA/docker/xeon/README_qdrant.md +++ b/ChatQnA/docker/xeon/README_qdrant.md @@ -148,6 +148,18 @@ Then run the command `docker images`, you will have the following 7 Docker Image ## 🚀 Start Microservices +### Required Models + +By default, the embedding, reranking and LLM models are set to a default value as listed below: + +| Service | Model | +| --------- | ------------------------- | +| Embedding | BAAI/bge-base-en-v1.5 | +| Reranking | BAAI/bge-reranker-base | +| LLM | Intel/neural-chat-7b-v3-3 | + +Change the `xxx_MODEL_ID` below for your needs. + ### Setup Environment Variables Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below. diff --git a/CodeTrans/README.md b/CodeTrans/README.md index a56f02d64..b6a223576 100644 --- a/CodeTrans/README.md +++ b/CodeTrans/README.md @@ -22,6 +22,16 @@ Currently we support two ways of deploying Code Translation services on docker: 2. Start services using the docker images `built from source`: [Guide](./docker) +### Required Models + +By default, the LLM model is set to a default value as listed below: + +| Service | Model | +| ------- | ----------------------------- | +| LLM | HuggingFaceH4/mistral-7b-grok | + +Change the `LLM_MODEL_ID` in `docker/set_env.sh` for your needs. + ### Setup Environment Variable To set up environment variables for deploying Code Translation services, follow these steps: diff --git a/CodeTrans/docker/gaudi/README.md b/CodeTrans/docker/gaudi/README.md index ab9c661ae..6367e0c36 100755 --- a/CodeTrans/docker/gaudi/README.md +++ b/CodeTrans/docker/gaudi/README.md @@ -42,9 +42,17 @@ Then run the command `docker images`, you will have the following Docker Images: ## 🚀 Start Microservices -### Setup Environment Variables +### Required Models -Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below. Notice that the `LLM_MODEL_ID` indicates the LLM model used for TGI service. +By default, the LLM model is set to a default value as listed below: + +| Service | Model | +| ------- | ----------------------------- | +| LLM | HuggingFaceH4/mistral-7b-grok | + +Change the `LLM_MODEL_ID` below for your needs. + +### Setup Environment Variables ```bash export no_proxy=${your_no_proxy} diff --git a/CodeTrans/docker/xeon/README.md b/CodeTrans/docker/xeon/README.md index 270733f34..4d3fa6b54 100755 --- a/CodeTrans/docker/xeon/README.md +++ b/CodeTrans/docker/xeon/README.md @@ -50,9 +50,17 @@ Then run the command `docker images`, you will have the following Docker Images: ## 🚀 Start Microservices -### Setup Environment Variables +### Required Models -Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below. Notice that the `LLM_MODEL_ID` indicates the LLM model used for TGI service. +By default, the LLM model is set to a default value as listed below: + +| Service | Model | +| ------- | ----------------------------- | +| LLM | HuggingFaceH4/mistral-7b-grok | + +Change the `LLM_MODEL_ID` below for your needs. + +### Setup Environment Variables ```bash export no_proxy=${your_no_proxy} diff --git a/CodeTrans/kubernetes/README.md b/CodeTrans/kubernetes/README.md index f9b33fdc9..ed2b146ac 100644 --- a/CodeTrans/kubernetes/README.md +++ b/CodeTrans/kubernetes/README.md @@ -7,9 +7,20 @@ Please install GMC in your Kubernetes cluster, if you have not already done so, If you have only Intel Xeon machines you could use the codetrans_xeon.yaml file or if you have a Gaudi cluster you could use codetrans_gaudi.yaml In the below example we illustrate on Xeon. +## Required Models + +By default, the LLM model is set to a default value as listed below: + +|Service |Model | +|---------|-------------------------| +|LLM |HuggingFaceH4/mistral-7b-grok| + +Change the `MODEL_ID` in `codetrans_xeon.yaml` for your needs. + ## Deploy the RAG application 1. Create the desired namespace if it does not already exist and deploy the application + ```bash export APP_NAMESPACE=CT kubectl create ns $APP_NAMESPACE diff --git a/CodeTrans/kubernetes/manifests/README.md b/CodeTrans/kubernetes/manifests/README.md index 0ec842175..709d6ea0f 100644 --- a/CodeTrans/kubernetes/manifests/README.md +++ b/CodeTrans/kubernetes/manifests/README.md @@ -8,6 +8,16 @@ > You need to make sure you have created the directory `/mnt/opea-models` to save the cached model on the node where the CodeTrans workload is running. Otherwise, you need to modify the `codetrans.yaml` file to change the `model-volume` to a directory that exists on the node. +## Required Models + +By default, the LLM model is set to a default value as listed below: + +|Service |Model | +|---------|-------------------------| +|LLM |HuggingFaceH4/mistral-7b-grok| + +Change the `MODEL_ID` in `codetrans.yaml` for your needs. + ## Deploy On Xeon ```bash