diff --git a/CodeTrans/README.md b/CodeTrans/README.md index b70666273..5cfa9b27e 100644 --- a/CodeTrans/README.md +++ b/CodeTrans/README.md @@ -77,9 +77,9 @@ Currently we support two ways of deploying Code Translation services on docker: By default, the LLM model is set to a default value as listed below: -| Service | Model | -| ------- | ----------------------------- | -| LLM | HuggingFaceH4/mistral-7b-grok | +| Service | Model | +| ------- | ---------------------------------- | +| LLM | mistralai/Mistral-7B-Instruct-v0.3 | Change the `LLM_MODEL_ID` in `docker_compose/set_env.sh` for your needs. diff --git a/CodeTrans/docker_compose/intel/cpu/xeon/README.md b/CodeTrans/docker_compose/intel/cpu/xeon/README.md index fd29ce210..15f6414f0 100755 --- a/CodeTrans/docker_compose/intel/cpu/xeon/README.md +++ b/CodeTrans/docker_compose/intel/cpu/xeon/README.md @@ -57,9 +57,9 @@ Then run the command `docker images`, you will have the following Docker Images: By default, the LLM model is set to a default value as listed below: -| Service | Model | -| ------- | ----------------------------- | -| LLM | HuggingFaceH4/mistral-7b-grok | +| Service | Model | +| ------- | ---------------------------------- | +| LLM | mistralai/Mistral-7B-Instruct-v0.3 | Change the `LLM_MODEL_ID` below for your needs. diff --git a/CodeTrans/docker_compose/intel/hpu/gaudi/README.md b/CodeTrans/docker_compose/intel/hpu/gaudi/README.md index 1eb1812f2..04858bc23 100755 --- a/CodeTrans/docker_compose/intel/hpu/gaudi/README.md +++ b/CodeTrans/docker_compose/intel/hpu/gaudi/README.md @@ -49,9 +49,9 @@ Then run the command `docker images`, you will have the following Docker Images: By default, the LLM model is set to a default value as listed below: -| Service | Model | -| ------- | ----------------------------- | -| LLM | HuggingFaceH4/mistral-7b-grok | +| Service | Model | +| ------- | ---------------------------------- | +| LLM | mistralai/Mistral-7B-Instruct-v0.3 | Change the `LLM_MODEL_ID` below for your needs. diff --git a/CodeTrans/docker_compose/set_env.sh b/CodeTrans/docker_compose/set_env.sh index 5eae8f0cd..b4defd88c 100644 --- a/CodeTrans/docker_compose/set_env.sh +++ b/CodeTrans/docker_compose/set_env.sh @@ -4,7 +4,7 @@ # SPDX-License-Identifier: Apache-2.0 -export LLM_MODEL_ID="HuggingFaceH4/mistral-7b-grok" +export LLM_MODEL_ID="mistralai/Mistral-7B-Instruct-v0.3" export TGI_LLM_ENDPOINT="http://${host_ip}:8008" export MEGA_SERVICE_HOST_IP=${host_ip} export LLM_SERVICE_HOST_IP=${host_ip} diff --git a/CodeTrans/kubernetes/intel/README.md b/CodeTrans/kubernetes/intel/README.md index 9d6e63f8b..3799f9832 100644 --- a/CodeTrans/kubernetes/intel/README.md +++ b/CodeTrans/kubernetes/intel/README.md @@ -14,7 +14,7 @@ By default, the LLM model is set to a default value as listed below: |Service |Model | |---------|-------------------------| -|LLM |HuggingFaceH4/mistral-7b-grok| +|LLM |mistralai/Mistral-7B-Instruct-v0.3| Change the `MODEL_ID` in `codetrans.yaml` for your needs. diff --git a/CodeTrans/kubernetes/intel/README_gmc.md b/CodeTrans/kubernetes/intel/README_gmc.md index 1b932f4ea..0f66407d1 100644 --- a/CodeTrans/kubernetes/intel/README_gmc.md +++ b/CodeTrans/kubernetes/intel/README_gmc.md @@ -13,7 +13,7 @@ By default, the LLM model is set to a default value as listed below: |Service |Model | |---------|-------------------------| -|LLM |HuggingFaceH4/mistral-7b-grok| +|LLM |mistralai/Mistral-7B-Instruct-v0.3| Change the `MODEL_ID` in `codetrans_xeon.yaml` for your needs. diff --git a/CodeTrans/kubernetes/intel/cpu/xeon/gmc/codetrans_xeon.yaml b/CodeTrans/kubernetes/intel/cpu/xeon/gmc/codetrans_xeon.yaml index 889a1d21a..244e7eb54 100644 --- a/CodeTrans/kubernetes/intel/cpu/xeon/gmc/codetrans_xeon.yaml +++ b/CodeTrans/kubernetes/intel/cpu/xeon/gmc/codetrans_xeon.yaml @@ -29,6 +29,6 @@ spec: internalService: serviceName: tgi-service config: - MODEL_ID: HuggingFaceH4/mistral-7b-grok + MODEL_ID: mistralai/Mistral-7B-Instruct-v0.3 endpoint: /generate isDownstreamService: true diff --git a/CodeTrans/kubernetes/intel/cpu/xeon/manifest/codetrans.yaml b/CodeTrans/kubernetes/intel/cpu/xeon/manifest/codetrans.yaml index a68768e2f..442908343 100644 --- a/CodeTrans/kubernetes/intel/cpu/xeon/manifest/codetrans.yaml +++ b/CodeTrans/kubernetes/intel/cpu/xeon/manifest/codetrans.yaml @@ -64,7 +64,7 @@ metadata: app.kubernetes.io/version: "2.1.0" app.kubernetes.io/managed-by: Helm data: - MODEL_ID: "HuggingFaceH4/mistral-7b-grok" + MODEL_ID: "mistralai/Mistral-7B-Instruct-v0.3" PORT: "2080" HF_TOKEN: "insert-your-huggingface-token-here" http_proxy: "" diff --git a/CodeTrans/kubernetes/intel/hpu/gaudi/gmc/codetrans_gaudi.yaml b/CodeTrans/kubernetes/intel/hpu/gaudi/gmc/codetrans_gaudi.yaml index 5bc1bd5e2..b61ffef3e 100644 --- a/CodeTrans/kubernetes/intel/hpu/gaudi/gmc/codetrans_gaudi.yaml +++ b/CodeTrans/kubernetes/intel/hpu/gaudi/gmc/codetrans_gaudi.yaml @@ -29,6 +29,6 @@ spec: internalService: serviceName: tgi-gaudi-svc config: - MODEL_ID: HuggingFaceH4/mistral-7b-grok + MODEL_ID: mistralai/Mistral-7B-Instruct-v0.3 endpoint: /generate isDownstreamService: true diff --git a/CodeTrans/kubernetes/intel/hpu/gaudi/manifest/codetrans.yaml b/CodeTrans/kubernetes/intel/hpu/gaudi/manifest/codetrans.yaml index 541f31179..076104e77 100644 --- a/CodeTrans/kubernetes/intel/hpu/gaudi/manifest/codetrans.yaml +++ b/CodeTrans/kubernetes/intel/hpu/gaudi/manifest/codetrans.yaml @@ -64,7 +64,7 @@ metadata: app.kubernetes.io/version: "2.1.0" app.kubernetes.io/managed-by: Helm data: - MODEL_ID: "HuggingFaceH4/mistral-7b-grok" + MODEL_ID: "mistralai/Mistral-7B-Instruct-v0.3" PORT: "2080" HF_TOKEN: "insert-your-huggingface-token-here" http_proxy: "" diff --git a/CodeTrans/tests/test_compose_on_gaudi.sh b/CodeTrans/tests/test_compose_on_gaudi.sh index b246f4dc9..884f2dffa 100644 --- a/CodeTrans/tests/test_compose_on_gaudi.sh +++ b/CodeTrans/tests/test_compose_on_gaudi.sh @@ -31,7 +31,7 @@ function start_services() { export http_proxy=${http_proxy} export https_proxy=${http_proxy} - export LLM_MODEL_ID="HuggingFaceH4/mistral-7b-grok" + export LLM_MODEL_ID="mistralai/Mistral-7B-Instruct-v0.3" export TGI_LLM_ENDPOINT="http://${ip_address}:8008" export HUGGINGFACEHUB_API_TOKEN=${HUGGINGFACEHUB_API_TOKEN} export MEGA_SERVICE_HOST_IP=${ip_address} diff --git a/CodeTrans/tests/test_compose_on_xeon.sh b/CodeTrans/tests/test_compose_on_xeon.sh index 8cbcb2320..d1f55c9a3 100644 --- a/CodeTrans/tests/test_compose_on_xeon.sh +++ b/CodeTrans/tests/test_compose_on_xeon.sh @@ -30,7 +30,7 @@ function start_services() { cd $WORKPATH/docker_compose/intel/cpu/xeon/ export http_proxy=${http_proxy} export https_proxy=${http_proxy} - export LLM_MODEL_ID="HuggingFaceH4/mistral-7b-grok" + export LLM_MODEL_ID="mistralai/Mistral-7B-Instruct-v0.3" export TGI_LLM_ENDPOINT="http://${ip_address}:8008" export HUGGINGFACEHUB_API_TOKEN=${HUGGINGFACEHUB_API_TOKEN} export MEGA_SERVICE_HOST_IP=${ip_address} diff --git a/supported_examples.md b/supported_examples.md index e913fd22e..ec0624b1a 100644 --- a/supported_examples.md +++ b/supported_examples.md @@ -71,9 +71,9 @@ This document introduces the supported examples of GenAIExamples. The supported [CodeTrans](./CodeTrans/README.md) is an example of chatbot for converting code written in one programming language to another programming language while maintaining the same functionality. -| Framework | LLM | Serving | HW | Description | -| ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------- | --------------------------------------------------------------- | ----------- | ---------------- | -| [LangChain](https://www.langchain.com)/[LlamaIndex](https://www.llamaindex.ai) | [HuggingFaceH4/mistral-7b-grok](https://huggingface.co/HuggingFaceH4/mistral-7b-grok) | [TGI](https://github.com/huggingface/text-generation-inference) | Xeon/Gaudi2 | Code Translation | +| Framework | LLM | Serving | HW | Description | +| ------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------- | --------------------------------------------------------------- | ----------- | ---------------- | +| [LangChain](https://www.langchain.com)/[LlamaIndex](https://www.llamaindex.ai) | [mistralai/Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) | [TGI](https://github.com/huggingface/text-generation-inference) | Xeon/Gaudi2 | Code Translation | ### DocSum