diff --git a/ChatQnA/README.md b/ChatQnA/README.md index c1db8e15a..0697c8630 100644 --- a/ChatQnA/README.md +++ b/ChatQnA/README.md @@ -121,6 +121,18 @@ Currently we support two ways of deploying ChatQnA services with docker compose: 2. Start services using the docker images `built from source`: [Guide](./docker) +### Required Models + +By default, the embedding, reranking and LLM models are set to a default value as listed below: + +| Service | Model | +| --------- | ------------------------- | +| Embedding | BAAI/bge-base-en-v1.5 | +| Reranking | BAAI/bge-reranker-base | +| LLM | Intel/neural-chat-7b-v3-3 | + +Change the `xxx_MODEL_ID` in `docker/xxx/set_env.sh` for your needs. + ### Setup Environment Variable To set up environment variables for deploying ChatQnA services, follow these steps: diff --git a/ChatQnA/docker/gaudi/README.md b/ChatQnA/docker/gaudi/README.md index 75bd412d5..053484f77 100644 --- a/ChatQnA/docker/gaudi/README.md +++ b/ChatQnA/docker/gaudi/README.md @@ -159,6 +159,18 @@ If Guardrails docker image is built, you will find one more image: ## 🚀 Start MicroServices and MegaService +### Required Models + +By default, the embedding, reranking and LLM models are set to a default value as listed below: + +| Service | Model | +| --------- | ------------------------- | +| Embedding | BAAI/bge-base-en-v1.5 | +| Reranking | BAAI/bge-reranker-base | +| LLM | Intel/neural-chat-7b-v3-3 | + +Change the `xxx_MODEL_ID` below for your needs. + ### Setup Environment Variables Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below. diff --git a/ChatQnA/docker/gpu/README.md b/ChatQnA/docker/gpu/README.md index 41c2aef32..48c287fb5 100644 --- a/ChatQnA/docker/gpu/README.md +++ b/ChatQnA/docker/gpu/README.md @@ -87,6 +87,18 @@ Then run the command `docker images`, you will have the following 7 Docker Image ## 🚀 Start MicroServices and MegaService +### Required Models + +By default, the embedding, reranking and LLM models are set to a default value as listed below: + +| Service | Model | +| --------- | ------------------------- | +| Embedding | BAAI/bge-base-en-v1.5 | +| Reranking | BAAI/bge-reranker-base | +| LLM | Intel/neural-chat-7b-v3-3 | + +Change the `xxx_MODEL_ID` below for your needs. + ### Setup Environment Variables Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below. diff --git a/ChatQnA/docker/xeon/README.md b/ChatQnA/docker/xeon/README.md index b607e8532..dc8735928 100644 --- a/ChatQnA/docker/xeon/README.md +++ b/ChatQnA/docker/xeon/README.md @@ -161,6 +161,18 @@ Then run the command `docker images`, you will have the following 7 Docker Image ## 🚀 Start Microservices +### Required Models + +By default, the embedding, reranking and LLM models are set to a default value as listed below: + +| Service | Model | +| --------- | ------------------------- | +| Embedding | BAAI/bge-base-en-v1.5 | +| Reranking | BAAI/bge-reranker-base | +| LLM | Intel/neural-chat-7b-v3-3 | + +Change the `xxx_MODEL_ID` below for your needs. + ### Setup Environment Variables Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below. @@ -183,7 +195,7 @@ export your_hf_api_token="Your_Huggingface_API_Token" **Append the value of the public IP address to the no_proxy list** -``` +```bash export your_no_proxy=${your_no_proxy},"External_Public_IP" ``` diff --git a/ChatQnA/docker/xeon/README_qdrant.md b/ChatQnA/docker/xeon/README_qdrant.md index fd28d306a..f103d5a73 100644 --- a/ChatQnA/docker/xeon/README_qdrant.md +++ b/ChatQnA/docker/xeon/README_qdrant.md @@ -148,6 +148,18 @@ Then run the command `docker images`, you will have the following 7 Docker Image ## 🚀 Start Microservices +### Required Models + +By default, the embedding, reranking and LLM models are set to a default value as listed below: + +| Service | Model | +| --------- | ------------------------- | +| Embedding | BAAI/bge-base-en-v1.5 | +| Reranking | BAAI/bge-reranker-base | +| LLM | Intel/neural-chat-7b-v3-3 | + +Change the `xxx_MODEL_ID` below for your needs. + ### Setup Environment Variables Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below. diff --git a/CodeTrans/README.md b/CodeTrans/README.md index a56f02d64..b6a223576 100644 --- a/CodeTrans/README.md +++ b/CodeTrans/README.md @@ -22,6 +22,16 @@ Currently we support two ways of deploying Code Translation services on docker: 2. Start services using the docker images `built from source`: [Guide](./docker) +### Required Models + +By default, the LLM model is set to a default value as listed below: + +| Service | Model | +| ------- | ----------------------------- | +| LLM | HuggingFaceH4/mistral-7b-grok | + +Change the `LLM_MODEL_ID` in `docker/set_env.sh` for your needs. + ### Setup Environment Variable To set up environment variables for deploying Code Translation services, follow these steps: diff --git a/CodeTrans/docker/gaudi/README.md b/CodeTrans/docker/gaudi/README.md index ab9c661ae..6367e0c36 100755 --- a/CodeTrans/docker/gaudi/README.md +++ b/CodeTrans/docker/gaudi/README.md @@ -42,9 +42,17 @@ Then run the command `docker images`, you will have the following Docker Images: ## 🚀 Start Microservices -### Setup Environment Variables +### Required Models -Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below. Notice that the `LLM_MODEL_ID` indicates the LLM model used for TGI service. +By default, the LLM model is set to a default value as listed below: + +| Service | Model | +| ------- | ----------------------------- | +| LLM | HuggingFaceH4/mistral-7b-grok | + +Change the `LLM_MODEL_ID` below for your needs. + +### Setup Environment Variables ```bash export no_proxy=${your_no_proxy} diff --git a/CodeTrans/docker/xeon/README.md b/CodeTrans/docker/xeon/README.md index 270733f34..4d3fa6b54 100755 --- a/CodeTrans/docker/xeon/README.md +++ b/CodeTrans/docker/xeon/README.md @@ -50,9 +50,17 @@ Then run the command `docker images`, you will have the following Docker Images: ## 🚀 Start Microservices -### Setup Environment Variables +### Required Models -Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below. Notice that the `LLM_MODEL_ID` indicates the LLM model used for TGI service. +By default, the LLM model is set to a default value as listed below: + +| Service | Model | +| ------- | ----------------------------- | +| LLM | HuggingFaceH4/mistral-7b-grok | + +Change the `LLM_MODEL_ID` below for your needs. + +### Setup Environment Variables ```bash export no_proxy=${your_no_proxy} diff --git a/CodeTrans/kubernetes/README.md b/CodeTrans/kubernetes/README.md index f9b33fdc9..ed2b146ac 100644 --- a/CodeTrans/kubernetes/README.md +++ b/CodeTrans/kubernetes/README.md @@ -7,9 +7,20 @@ Please install GMC in your Kubernetes cluster, if you have not already done so, If you have only Intel Xeon machines you could use the codetrans_xeon.yaml file or if you have a Gaudi cluster you could use codetrans_gaudi.yaml In the below example we illustrate on Xeon. +## Required Models + +By default, the LLM model is set to a default value as listed below: + +|Service |Model | +|---------|-------------------------| +|LLM |HuggingFaceH4/mistral-7b-grok| + +Change the `MODEL_ID` in `codetrans_xeon.yaml` for your needs. + ## Deploy the RAG application 1. Create the desired namespace if it does not already exist and deploy the application + ```bash export APP_NAMESPACE=CT kubectl create ns $APP_NAMESPACE diff --git a/CodeTrans/kubernetes/manifests/README.md b/CodeTrans/kubernetes/manifests/README.md index 0ec842175..709d6ea0f 100644 --- a/CodeTrans/kubernetes/manifests/README.md +++ b/CodeTrans/kubernetes/manifests/README.md @@ -8,6 +8,16 @@ > You need to make sure you have created the directory `/mnt/opea-models` to save the cached model on the node where the CodeTrans workload is running. Otherwise, you need to modify the `codetrans.yaml` file to change the `model-volume` to a directory that exists on the node. +## Required Models + +By default, the LLM model is set to a default value as listed below: + +|Service |Model | +|---------|-------------------------| +|LLM |HuggingFaceH4/mistral-7b-grok| + +Change the `MODEL_ID` in `codetrans.yaml` for your needs. + ## Deploy On Xeon ```bash