vllm hpu fix version for bug fix (#903)
* vllm test Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * fix vllm hpu version to fix bug Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * refine readme Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * fix vllm version Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * update vllm ut model Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * revert agent Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> * fix bug Signed-off-by: Xinyao Wang <xinyao.wang@intel.com> --------- Signed-off-by: Xinyao Wang <xinyao.wang@intel.com>
This commit is contained in:
@@ -56,12 +56,6 @@ bash ./build_docker_vllm.sh hpu
|
||||
|
||||
Set `hw_mode` to `hpu`.
|
||||
|
||||
Note: If you want to enable tensor parallel, please set `setuptools==69.5.1` in Dockerfile.hpu before build docker with following command.
|
||||
|
||||
```
|
||||
sed -i "s/RUN pip install setuptools/RUN pip install setuptools==69.5.1/g" docker/Dockerfile.hpu
|
||||
```
|
||||
|
||||
#### Launch vLLM service on single node
|
||||
|
||||
For small model, we can just use single node.
|
||||
|
||||
@@ -37,6 +37,7 @@ fi
|
||||
if [ "$hw_mode" = "hpu" ]; then
|
||||
git clone https://github.com/HabanaAI/vllm-fork.git
|
||||
cd ./vllm-fork/
|
||||
git checkout 3c39626
|
||||
docker build -f Dockerfile.hpu -t opea/vllm-hpu:latest --shm-size=128g . --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy
|
||||
cd ..
|
||||
rm -rf vllm-fork
|
||||
|
||||
@@ -56,12 +56,6 @@ bash ./build_docker_vllm.sh hpu
|
||||
|
||||
Set `hw_mode` to `hpu`.
|
||||
|
||||
Note: If you want to enable tensor parallel, please set `setuptools==69.5.1` in Dockerfile.hpu before build docker with following command.
|
||||
|
||||
```
|
||||
sed -i "s/RUN pip install setuptools/RUN pip install setuptools==69.5.1/g" docker/Dockerfile.hpu
|
||||
```
|
||||
|
||||
#### Launch vLLM service on single node
|
||||
|
||||
For small model, we can just use single node.
|
||||
|
||||
@@ -32,6 +32,7 @@ fi
|
||||
if [ "$hw_mode" = "hpu" ]; then
|
||||
git clone https://github.com/HabanaAI/vllm-fork.git
|
||||
cd ./vllm-fork/
|
||||
git checkout 3c39626
|
||||
docker build -f Dockerfile.hpu -t opea/vllm-hpu:latest --shm-size=128g . --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy
|
||||
cd ..
|
||||
rm -rf vllm-fork
|
||||
|
||||
Reference in New Issue
Block a user