jandres/GenAIExamples
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

Merge branch 'main' of https://github.com/tylertitsworth/GenAIExamples into tylertitsworth-main

Browse Source 
Signed-off-by: lvliang-intel <liang1.lv@intel.com>
This commit is contained in:
lvliang-intel
2024-03-27 19:22:53 +08:00
parent 6c00ee5a0d 39348cb15b
commit 6b72d3bf13
14 changed files with 126 additions and 113 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
ChatQnA/README.md
View File

@@ -51,28 +51,22 @@ export TGI_LLM_ENDPOINT="http://xxx.xxx.xxx.xxx:8080"
## Launch Redis
```bash
docker pull redis/redis-stack:latest
docker compose -f langchain/docker/docker-compose-redis.yml up -d
```
## Launch LangChain Docker
### Build LangChain Docker Image
```bash
cd langchain/docker/
bash ./build_docker.sh
```
### Lanuch LangChain Docker
Update the `HUGGINGFACEHUB_API_TOKEN` environment variable with your huggingface token in the `docker-compose-langchain.yml`
```bash
cd langchain/docker
docker compose -f docker-compose-langchain.yml up -d
cd ../../
```
> [!NOTE]
> If you modified any files and want that change introduced in this step, add `--build` to the end of the command to build the container image instead of pulling it from dockerhub.
## Ingest data into redis
After every time of redis container is launched, data should be ingested in the container ingestion steps:

45
ChatQnA/langchain/docker/Dockerfile
View File

@@ -12,39 +12,26 @@
# See the License for the specific language governing permissions and
# limitations under the License.
FROM langchain/langchain
FROM langchain/langchain:latest
ARG http_proxy
ARG https_proxy
ENV http_proxy=$http_proxy
ENV https_proxy=$https_proxy
RUN apt-get update -y && apt-get install -y --no-install-recommends --fix-missing \
libgl1-mesa-glx \
libjemalloc-dev
RUN apt-get update && \
apt-get upgrade -y && \
apt-get install -y \
libgl1-mesa-glx \
libjemalloc-dev
RUN useradd -m -s /bin/bash user && \
mkdir -p /home/user && \
chown -R user /home/user/
RUN pip install --upgrade pip \
sentence-transformers \
redis \
unstructured \
unstructured[all-docs] \
langchain-cli \
pydantic==1.10.13 \
langchain==0.1.12 \
poetry \
pymupdf \
easyocr \
langchain_benchmarks \
pyarrow \
jupyter \
intel-extension-for-pytorch \
intel-openmp
USER user
ENV PYTHONPATH=/ws:/qna-app/app
COPY requirements.txt /tmp/requirements.txt
COPY qna-app /qna-app
WORKDIR /qna-app
RUN pip install --no-cache-dir --upgrade pip && \
pip install --no-cache-dir -r /tmp/requirements.txt
ENV PYTHONPATH=/home/user:/home/user/qna-app/app
WORKDIR /home/user/qna-app
COPY qna-app /home/user/qna-app
ENTRYPOINT ["/usr/bin/sleep", "infinity"]

17
ChatQnA/langchain/docker/build_docker.sh
View File

@@ -1,17 +0,0 @@
# Copyright (c) 2024 Intel Corporation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#!/bin/bash
docker build . -t qna-rag-redis:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy

9
ChatQnA/langchain/docker/docker-compose-langchain.yml
View File

@@ -15,9 +15,16 @@
version: '3'
services:
qna-rag-redis-server:
image: qna-rag-redis:latest
build:
args:
http_proxy: ${http_proxy}
https_proxy: ${https_proxy}
dockerfile: Dockerfile
image: intel/gen-ai-examples:qna-rag-redis-server
container_name: qna-rag-redis-server
environment:
- http_proxy=${http_proxy}
- https_proxy=${https_proxy}
- "REDIS_PORT=6379"
- "EMBED_MODEL=BAAI/bge-base-en-v1.5"
- "REDIS_SCHEMA=schema_dim_768.yml"

16
ChatQnA/langchain/docker/requirements.txt Normal file
View File

@@ -0,0 +1,16 @@
cryptography==42.0.4
easyocr
intel-extension-for-pytorch
intel-openmp
jupyter
langchain_benchmarks
langchain-cli
langchain==0.1.12
poetry
pyarrow
pydantic==1.10.13
pymupdf
redis
sentence-transformers
unstructured
unstructured[all-docs]

7
CodeGen/README.md
View File

@@ -14,7 +14,7 @@ docker pull ghcr.io/huggingface/tgi-gaudi:1.2.1
Alternatively, you can build the Docker image yourself with:
```bash
bash ./serving/tgi_gaudi/build_docker.sh
bash ./tgi_gaudi/build_docker.sh
```
## Launch TGI Gaudi Service
@@ -43,7 +43,7 @@ export TGI_ENDPOINT="xxx.xxx.xxx.xxx:8080"
## Launch Copilot Docker
### Build Copilot Docker Image
### Build Copilot Docker Image (Optional)
```bash
cd codegen
@@ -54,10 +54,9 @@ cd ..
### Lanuch Copilot Docker
```bash
docker run -it --net=host --ipc=host -v /var/run/docker.sock:/var/run/docker.sock copilot:latest
docker run -it -e http_proxy=${http_proxy} -e https_proxy=${https_proxy} --net=host --ipc=host -v /var/run/docker.sock:/var/run/docker.sock intel/gen-ai-examples:copilot bash
```
# Start Copilot Server
## Start the Backend Service

33
CodeGen/codegen/Dockerfile
View File

@@ -12,14 +12,25 @@
# See the License for the specific language governing permissions and
# limitations under the License.
FROM langchain/langchain
RUN apt-get update && apt-get -y install libgl1-mesa-glx
RUN pip install -U langchain-cli pydantic==1.10.13
RUN pip install langchain==0.1.11
RUN pip install shortuuid
RUN pip install huggingface_hub
RUN mkdir -p /ws
ENV PYTHONPATH=/ws
COPY codegen-app /codegen-app
WORKDIR /codegen-app
CMD ["/bin/bash"]
FROM langchain/langchain:latest
RUN apt-get update -y && apt-get install -y --no-install-recommends --fix-missing \
libgl1-mesa-glx \
libjemalloc-dev
RUN useradd -m -s /bin/bash user && \
mkdir -p /home/user && \
chown -R user /home/user/
USER user
COPY requirements.txt /tmp/requirements.txt
RUN pip install --no-cache-dir -U -r /tmp/requirements.txt
ENV PYTHONPATH=/home/user:/home/user/codegen-app
WORKDIR /home/user/codegen-app
COPY codegen-app /home/user/codegen-app
SHELL ["/bin/bash", "-c"]

5
CodeGen/codegen/requirements.txt Normal file
View File

@@ -0,0 +1,5 @@
huggingface_hub
langchain-cli
langchain==0.1.11
pydantic==1.10.13
shortuuid

4
DocSum/README.md
View File

@@ -42,7 +42,7 @@ export TGI_ENDPOINT="http://xxx.xxx.xxx.xxx:8080"
## Launch Document Summary Docker
### Build Document Summary Docker Image
### Build Document Summary Docker Image (Optional)
```bash
cd langchain/docker/
@@ -53,7 +53,7 @@ cd ../../
### Lanuch Document Summary Docker
```bash
docker run -it --net=host --ipc=host -v /var/run/docker.sock:/var/run/docker.sock document-summarize:latest
docker run -it --net=host --ipc=host -e http_proxy=${http_proxy} -e https_proxy=${https_proxy} -v /var/run/docker.sock:/var/run/docker.sock intel/gen-ai-examples:document-summarize bash
```

44
DocSum/langchain/docker/Dockerfile
View File

@@ -12,38 +12,24 @@
# See the License for the specific language governing permissions and
# limitations under the License.
FROM langchain/langchain
FROM langchain/langchain:latest
ARG http_proxy
ARG https_proxy
ENV http_proxy=$http_proxy
ENV https_proxy=$https_proxy
RUN apt-get update -y && apt-get install -y --no-install-recommends --fix-missing \
libgl1-mesa-glx \
libjemalloc-dev
RUN apt-get update && \
apt-get upgrade -y && \
apt-get install -y \
libgl1-mesa-glx \
libjemalloc-dev
RUN useradd -m -s /bin/bash user && \
mkdir -p /home/user && \
chown -R user /home/user/
RUN pip install --upgrade pip \
sentence-transformers \
langchain-cli \
pydantic==1.10.13 \
langchain==0.1.12 \
poetry \
langchain_benchmarks \
pyarrow \
jupyter \
docx2txt \
pypdf \
beautifulsoup4 \
python-multipart \
intel-extension-for-pytorch \
intel-openmp
USER user
ENV PYTHONPATH=/ws:/summarize-app/app
COPY requirements.txt /tmp/requirements.txt
COPY summarize-app /summarize-app
WORKDIR /summarize-app
RUN pip install --no-cache-dir --upgrade pip && \
pip install --no-cache-dir -r /tmp/requirements.txt
CMD ["/bin/bash"]
ENV PYTHONPATH=/home/user:/home/user/summarize-app/app
WORKDIR /home/user/summarize-app
COPY summarize-app /home/user/summarize-app

14
DocSum/langchain/docker/requirements.txt Normal file
View File

@@ -0,0 +1,14 @@
beautifulsoup4
docx2txt
intel-extension-for-pytorch
intel-openmp
jupyter
langchain_benchmarks
langchain-cli
langchain==0.1.12
poetry
pyarrow
pydantic==1.10.13
pypdf
python-multipart
sentence-transformers

4
VisualQnA/README.md
View File

@@ -12,13 +12,13 @@ This example guides you through how to deploy a [LLaVA](https://llava-vl.github.
```
cd serving/
docker build . --build-arg http_proxy=${http_proxy} --build-arg https_proxy=${http_proxy} -t llava_gaudi:latest
docker build . --build-arg http_proxy=${http_proxy} --build-arg https_proxy=${http_proxy} -t intel/gen-ai-examples:llava-gaudi
```
2. Start the LLaVA service on Intel Gaudi2
```
docker run -d -p 8084:80 -p 8085:8000 -v ./data:/root/.cache/huggingface/hub/ -e http_proxy=$http_proxy -e https_proxy=$http_proxy -v $PWD/llava_server:/llava_server --runtime=habana -e HABANA_VISIBLE_DEVICES=all -e OMPI_MCA_btl_vader_single_copy_mechanism=none --cap-add=sys_nice --ipc=host llava_gaudi
docker run -d -p 8084:80 -p 8085:8000 -v ./data:/root/.cache/huggingface/hub/ -e http_proxy=$http_proxy -e https_proxy=$http_proxy -v $PWD/llava_server:/llava_server --runtime=habana -e HABANA_VISIBLE_DEVICES=all -e OMPI_MCA_btl_vader_single_copy_mechanism=none --cap-add=sys_nice --ipc=host intel/gen-ai-examples:llava-gaudi
```
Here are some explanation about the above parameters:

23
VisualQnA/serving/Dockerfile
View File

@@ -13,22 +13,29 @@
# limitations under the License.
# HABANA environment
FROM vault.habana.ai/gaudi-docker/1.14.0/ubuntu22.04/habanalabs/pytorch-installer-2.1.1 as hpu
FROM vault.habana.ai/gaudi-docker/1.14.0/ubuntu22.04/habanalabs/pytorch-installer-2.1.1 AS hpu
RUN rm -rf /etc/ssh/ssh_host*
# Set environment variables
ENV LANG=en_US.UTF-8
ENV PYTHONPATH=/root:/usr/lib/habanalabs/:/optimum-habana
ENV PYTHONPATH=/home/user:/usr/lib/habanalabs/:/optimum-habana
# Install required branch
RUN git clone https://github.com/lkk12014402/optimum-habana.git && \
cd optimum-habana && \
git checkout enable_llava_generation
RUN git clone https://github.com/lkk12014402/optimum-habana.git /optimum-habana -b enable_llava_generation
RUN useradd -m -s /bin/bash user && \
mkdir -p /home/user && \
chown -R user /home/user/
USER user
COPY requirements.txt /tmp/requirements.txt
# Install dependency
RUN pip install --upgrade-strategy eager optimum[habana] && \
pip install fastapi uvicorn
RUN pip install --no-cache-dir -U -r /tmp/requirements.txt
# work dir should contains the server
WORKDIR /llava_server
COPY llava_server /llava_server
ENTRYPOINT ["python", "llava_server.py"]
ENTRYPOINT ["python", "llava_server.py"]

4
VisualQnA/serving/requirements.txt Normal file
View File

@@ -0,0 +1,4 @@
eager
fastapi
optimum[habana]
uvicorn