jandres/GenAIExamples
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

Organize set_env.sh paths and update README.md (#1920)

Browse Source 
Signed-off-by: ZePan110 <ze.pan@intel.com>
Co-authored-by: chen, suyue <suyue.chen@intel.com>
Co-authored-by: Ying Hu <ying.hu@intel.com>
This commit is contained in:
ZePan110
2025-05-20 10:05:00 +08:00
committed by GitHub
parent 09d93ecce6
commit 8a9f3f4351
20 changed files with 34 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
DocSum/docker_compose/intel/cpu/xeon/README.md
View File

@@ -27,9 +27,9 @@ Clone the GenAIExample repository and access the ChatQnA Intel Xeon platform Doc
```bash
git clone https://github.com/opea-project/GenAIExamples.git
cd GenAIExamples/DocSum/docker_compose
cd GenAIExamples/DocSum/docker_compose/intel
source set_env.sh
cd intel/cpu/xeon/
cd cpu/xeon/
```
NOTE: by default vLLM does "warmup" at start, to optimize its performance for the specified model and the underlying platform, which can take long time. For development (and e.g. autoscaling) it can be skipped with `export VLLM_SKIP_WARMUP=true`.
@@ -49,6 +49,7 @@ Some HuggingFace resources, such as some models, are only accessible if you have
To deploy the DocSum services, execute the `docker compose up` command with the appropriate arguments. For a default deployment, execute:
```bash
cd cpu/xeon/
docker compose up -d
```

5
DocSum/docker_compose/intel/hpu/gaudi/README.md
View File

@@ -29,9 +29,9 @@ Clone the GenAIExample repository and access the DocSum Intel® Gaudi® platform
```bash
git clone https://github.com/opea-project/GenAIExamples.git
cd GenAIExamples/DocSum/docker_compose
cd GenAIExamples/DocSum/docker_compose/intel
source set_env.sh
cd intel/hpu/gaudi/
cd hpu/gaudi/
```
NOTE: by default vLLM does "warmup" at start, to optimize its performance for the specified model and the underlying platform, which can take long time. For development (and e.g. autoscaling) it can be skipped with `export VLLM_SKIP_WARMUP=true`.
@@ -51,6 +51,7 @@ Some HuggingFace resources, such as some models, are only accessible if you have
To deploy the DocSum services, execute the `docker compose up` command with the appropriate arguments. For a default deployment, execute:
```bash
cd hpu/gaudi/
docker compose up -d
```

36
DocSum/docker_compose/intel/set_env.sh Normal file
View File

@@ -0,0 +1,36 @@
#!/usr/bin/env bash
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
pushd "../../" > /dev/null
source .set_env.sh
popd > /dev/null
export host_ip=$(hostname -I | awk '{print $1}') # Example: host_ip="192.168.1.1"
export no_proxy="${no_proxy},${host_ip}" # Example: no_proxy="localhost, 127.0.0.1, 192.168.1.1"
export http_proxy=$http_proxy
export https_proxy=$https_proxy
export HUGGINGFACEHUB_API_TOKEN=${HUGGINGFACEHUB_API_TOKEN}
export LLM_ENDPOINT_PORT=8008
export LLM_MODEL_ID="Intel/neural-chat-7b-v3-3"
export MAX_INPUT_TOKENS=1024
export MAX_TOTAL_TOKENS=2048
export LLM_PORT=9000
export LLM_ENDPOINT="http://${host_ip}:${LLM_ENDPOINT_PORT}"
export DocSum_COMPONENT_NAME="OpeaDocSumvLLM" # OpeaDocSumTgi
export FRONTEND_SERVICE_PORT=5173
export MEGA_SERVICE_HOST_IP=${host_ip}
export LLM_SERVICE_HOST_IP=${host_ip}
export ASR_SERVICE_HOST_IP=${host_ip}
export BACKEND_SERVICE_PORT=8888
export BACKEND_SERVICE_ENDPOINT="http://${host_ip}:${BACKEND_SERVICE_PORT}/v1/docsum"
export LOGFLAG=True
export NUM_CARDS=1
export BLOCK_SIZE=128
export MAX_NUM_SEQS=256
export MAX_SEQ_LEN_TO_CAPTURE=2048