vllm hpu fix version for bug fix (#903)

* vllm test

Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>

* fix vllm hpu version to fix bug

Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>

* refine readme

Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>

* fix vllm version

Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>

* update vllm ut model

Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>

* revert agent

Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>

* fix bug

Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>

---------

Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
This commit is contained in:
XinyaoWa
2024-11-15 15:10:27 +08:00
committed by GitHub
parent d211cb2dbd
commit e1475acb55
7 changed files with 9 additions and 17 deletions

View File

@@ -56,12 +56,6 @@ bash ./build_docker_vllm.sh hpu
Set `hw_mode` to `hpu`.
Note: If you want to enable tensor parallel, please set `setuptools==69.5.1` in Dockerfile.hpu before build docker with following command.
```
sed -i "s/RUN pip install setuptools/RUN pip install setuptools==69.5.1/g" docker/Dockerfile.hpu
```
#### Launch vLLM service on single node
For small model, we can just use single node.

View File

@@ -37,6 +37,7 @@ fi
if [ "$hw_mode" = "hpu" ]; then
git clone https://github.com/HabanaAI/vllm-fork.git
cd ./vllm-fork/
git checkout 3c39626
docker build -f Dockerfile.hpu -t opea/vllm-hpu:latest --shm-size=128g . --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy
cd ..
rm -rf vllm-fork

View File

@@ -56,12 +56,6 @@ bash ./build_docker_vllm.sh hpu
Set `hw_mode` to `hpu`.
Note: If you want to enable tensor parallel, please set `setuptools==69.5.1` in Dockerfile.hpu before build docker with following command.
```
sed -i "s/RUN pip install setuptools/RUN pip install setuptools==69.5.1/g" docker/Dockerfile.hpu
```
#### Launch vLLM service on single node
For small model, we can just use single node.

View File

@@ -32,6 +32,7 @@ fi
if [ "$hw_mode" = "hpu" ]; then
git clone https://github.com/HabanaAI/vllm-fork.git
cd ./vllm-fork/
git checkout 3c39626
docker build -f Dockerfile.hpu -t opea/vllm-hpu:latest --shm-size=128g . --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy
cd ..
rm -rf vllm-fork