jandres/GenAIExamples
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

Update TGI image versions (#1625)

Browse Source 
Signed-off-by: xiaotia3 <xiaotian.chen@intel.com>
This commit is contained in:
Xiaotian Chen
2025-04-01 11:27:51 +08:00
committed by GitHub
parent 583428c6a7
commit 1bd56af994
36 changed files with 54 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
DocSum/docker_compose/amd/gpu/rocm/README.md
View File

@@ -72,7 +72,7 @@ For gated models, you also need to provide [HuggingFace token](https://huggingfa
Since the `compose.yaml` will consume some environment variables, you need to setup them in advance as below.
```bash
export DOCSUM_TGI_IMAGE="ghcr.io/huggingface/text-generation-inference:2.3.1-rocm"
export DOCSUM_TGI_IMAGE="ghcr.io/huggingface/text-generation-inference:2.4.1-rocm"
export DOCSUM_LLM_MODEL_ID="Intel/neural-chat-7b-v3-3"
export HOST_IP=${host_ip}
export DOCSUM_TGI_SERVICE_PORT="18882"

2
DocSum/docker_compose/amd/gpu/rocm/compose.yaml
View File

@@ -3,7 +3,7 @@
services:
docsum-tgi-service:
image: ghcr.io/huggingface/text-generation-inference:2.3.1-rocm
image: ghcr.io/huggingface/text-generation-inference:2.4.1-rocm
container_name: docsum-tgi-service
ports:
- "${DOCSUM_TGI_SERVICE_PORT}:80"

2
DocSum/docker_compose/amd/gpu/rocm/set_env.sh
View File

@@ -5,7 +5,7 @@
export MAX_INPUT_TOKENS=2048
export MAX_TOTAL_TOKENS=4096
export DOCSUM_TGI_IMAGE="ghcr.io/huggingface/text-generation-inference:2.3.1-rocm"
export DOCSUM_TGI_IMAGE="ghcr.io/huggingface/text-generation-inference:2.4.1-rocm"
export DOCSUM_LLM_MODEL_ID="Intel/neural-chat-7b-v3-3"
export HOST_IP=${host_ip}
export DOCSUM_TGI_SERVICE_PORT="8008"