jandres/GenAIExamples
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

fix errors for running AgentQnA on xeon with openai and update readme (#1664)

Browse Source 
Signed-off-by: minmin-intel <minmin.hou@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
minmin-intel
2025-03-19 18:57:18 -07:00
committed by GitHub
parent 2204fe8e36
commit fecc22719a
7 changed files with 104 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
AgentQnA/README.md
View File

@@ -192,6 +192,7 @@ docker build -t opea/agent:latest --build-arg https_proxy=$https_proxy --build-a
```bash ```bash
vllm_port=8086 vllm_port=8086
model="meta-llama/Meta-Llama-3.1-70B-Instruct" model="meta-llama/Meta-Llama-3.1-70B-Instruct"
vllm_volume=$HF_CACHE_DIR # you should have set this env var in previous step
docker run -d --runtime=habana --rm --name "vllm-gaudi-server" -e HABANA_VISIBLE_DEVICES=0,1,2,3 -p $vllm_port:8000 -v $vllm_volume:/data -e HF_TOKEN=$HF_TOKEN -e HUGGING_FACE_HUB_TOKEN=$HF_TOKEN -e HF_HOME=/data -e OMPI_MCA_btl_vader_single_copy_mechanism=none -e PT_HPU_ENABLE_LAZY_COLLECTIVES=true -e http_proxy=$http_proxy -e https_proxy=$https_proxy -e no_proxy=$no_proxy -e VLLM_SKIP_WARMUP=true --cap-add=sys_nice --ipc=host opea/vllm-gaudi:latest --model ${model} --max-seq-len-to-capture 16384 --tensor-parallel-size 4 docker run -d --runtime=habana --rm --name "vllm-gaudi-server" -e HABANA_VISIBLE_DEVICES=0,1,2,3 -p $vllm_port:8000 -v $vllm_volume:/data -e HF_TOKEN=$HF_TOKEN -e HUGGING_FACE_HUB_TOKEN=$HF_TOKEN -e HF_HOME=/data -e OMPI_MCA_btl_vader_single_copy_mechanism=none -e PT_HPU_ENABLE_LAZY_COLLECTIVES=true -e http_proxy=$http_proxy -e https_proxy=$https_proxy -e no_proxy=$no_proxy -e VLLM_SKIP_WARMUP=true --cap-add=sys_nice --ipc=host opea/vllm-gaudi:latest --model ${model} --max-seq-len-to-capture 16384 --tensor-parallel-size 4
``` ```

8
AgentQnA/docker_compose/intel/cpu/xeon/README.md
View File

@@ -42,15 +42,13 @@ This example showcases a hierarchical multi-agent system for question-answering
``` ```
4. Prepare SQL database 4. Prepare SQL database
In this example, we will use the SQLite database provided in the [TAG-Bench](https://github.com/TAG-Research/TAG-Bench/tree/main). Run the commands below. In this example, we will use the Chinook SQLite database. Run the commands below.
``` ```
# Download data # Download data
cd $WORKDIR cd $WORKDIR
git clone https://github.com/TAG-Research/TAG-Bench.git git clone https://github.com/lerocha/chinook-database.git
cd TAG-Bench/setup cp chinook-database/ChinookDatabase/DataSources/Chinook_Sqlite.sqlite $WORKDIR/GenAIExamples/AgentQnA/tests/
chmod +x get_dbs.sh
./get_dbs.sh
``` ```
5. Launch Tool service 5. Launch Tool service

5
AgentQnA/docker_compose/intel/cpu/xeon/compose_openai.yaml
View File

@@ -92,4 +92,9 @@ services:
LANGCHAIN_PROJECT: "opea-supervisor-agent-service" LANGCHAIN_PROJECT: "opea-supervisor-agent-service"
CRAG_SERVER: $CRAG_SERVER CRAG_SERVER: $CRAG_SERVER
WORKER_AGENT_URL: $WORKER_AGENT_URL WORKER_AGENT_URL: $WORKER_AGENT_URL
SQL_AGENT_URL: $SQL_AGENT_URL
port: 9090 port: 9090
networks:
default:
driver: bridge

2
AgentQnA/docker_compose/intel/hpu/gaudi/README.md
View File

@@ -23,6 +23,7 @@ For more details, please refer to the deployment guide [here](../../../../README
export no_proxy="Your_No_Proxy" export no_proxy="Your_No_Proxy"
export TOOLSET_PATH=$WORKDIR/GenAIExamples/AgentQnA/tools/ export TOOLSET_PATH=$WORKDIR/GenAIExamples/AgentQnA/tools/
# for using open-source llms # for using open-source llms
export HUGGINGFACEHUB_API_TOKEN=<your-HF-token> export HUGGINGFACEHUB_API_TOKEN=<your-HF-token>
# Example export HF_CACHE_DIR=$WORKDIR so that no need to redownload every time # Example export HF_CACHE_DIR=$WORKDIR so that no need to redownload every time
@@ -78,6 +79,7 @@ For more details, please refer to the deployment guide [here](../../../../README
```bash ```bash
vllm_port=8086 vllm_port=8086
vllm_volume=$HF_CACHE_DIR # you should have set this env var in previous step
model="meta-llama/Meta-Llama-3.1-70B-Instruct" model="meta-llama/Meta-Llama-3.1-70B-Instruct"
docker run -d --runtime=habana --rm --name "vllm-gaudi-server" -e HABANA_VISIBLE_DEVICES=0,1,2,3 -p $vllm_port:8000 -v $vllm_volume:/data -e HF_TOKEN=$HF_TOKEN -e HUGGING_FACE_HUB_TOKEN=$HF_TOKEN -e HF_HOME=/data -e OMPI_MCA_btl_vader_single_copy_mechanism=none -e PT_HPU_ENABLE_LAZY_COLLECTIVES=true -e http_proxy=$http_proxy -e https_proxy=$https_proxy -e no_proxy=$no_proxy -e VLLM_SKIP_WARMUP=true --cap-add=sys_nice --ipc=host opea/vllm-gaudi:latest --model ${model} --max-seq-len-to-capture 16384 --tensor-parallel-size 4 docker run -d --runtime=habana --rm --name "vllm-gaudi-server" -e HABANA_VISIBLE_DEVICES=0,1,2,3 -p $vllm_port:8000 -v $vllm_volume:/data -e HF_TOKEN=$HF_TOKEN -e HUGGING_FACE_HUB_TOKEN=$HF_TOKEN -e HF_HOME=/data -e OMPI_MCA_btl_vader_single_copy_mechanism=none -e PT_HPU_ENABLE_LAZY_COLLECTIVES=true -e http_proxy=$http_proxy -e https_proxy=$https_proxy -e no_proxy=$no_proxy -e VLLM_SKIP_WARMUP=true --cap-add=sys_nice --ipc=host opea/vllm-gaudi:latest --model ${model} --max-seq-len-to-capture 16384 --tensor-parallel-size 4
``` ```

17
AgentQnA/tests/_test_compose_openai_on_xeon.sh
View File

@@ -20,23 +20,30 @@ function stop_agent_and_api_server() {
function stop_retrieval_tool() { function stop_retrieval_tool() {
echo "Stopping Retrieval tool" echo "Stopping Retrieval tool"
docker compose -f $WORKDIR/GenAIExamples/AgentQnA/retrieval_tool/docker/docker-compose-retrieval-tool.yaml down local RETRIEVAL_TOOL_PATH=$WORKPATH/../DocIndexRetriever
cd $RETRIEVAL_TOOL_PATH/docker_compose/intel/cpu/xeon/
container_list=$(cat compose.yaml | grep container_name | cut -d':' -f2)
for container_name in $container_list; do
cid=$(docker ps -aq --filter "name=$container_name")
echo "Stopping container $container_name"
if [[ ! -z "$cid" ]]; then docker rm $cid -f && sleep 1s; fi
done
} }
echo "=================== #1 Building docker images====================" echo "=================== #1 Building docker images===================="
bash 1_build_images.sh bash step1_build_images.sh
echo "=================== #1 Building docker images completed====================" echo "=================== #1 Building docker images completed===================="
echo "=================== #2 Start retrieval tool====================" echo "=================== #2 Start retrieval tool===================="
bash 2_start_retrieval_tool.sh bash step2_start_retrieval_tool.sh
echo "=================== #2 Retrieval tool started====================" echo "=================== #2 Retrieval tool started===================="
echo "=================== #3 Ingest data and validate retrieval====================" echo "=================== #3 Ingest data and validate retrieval===================="
bash 3_ingest_data_and_validate_retrieval.sh bash step3_ingest_data_and_validate_retrieval.sh
echo "=================== #3 Data ingestion and validation completed====================" echo "=================== #3 Data ingestion and validation completed===================="
echo "=================== #4 Start agent and API server====================" echo "=================== #4 Start agent and API server===================="
bash 4_launch_and_validate_agent_openai.sh bash step4_launch_and_validate_agent_openai.sh
echo "=================== #4 Agent test passed ====================" echo "=================== #4 Agent test passed ===================="
echo "=================== #5 Stop agent and API server====================" echo "=================== #5 Stop agent and API server===================="

73
AgentQnA/tests/step4_launch_and_validate_agent_openai.sh
View File

@@ -11,13 +11,22 @@ echo "WORKDIR=${WORKDIR}"
export ip_address=$(hostname -I | awk '{print $1}') export ip_address=$(hostname -I | awk '{print $1}')
export TOOLSET_PATH=$WORKDIR/GenAIExamples/AgentQnA/tools/ export TOOLSET_PATH=$WORKDIR/GenAIExamples/AgentQnA/tools/
function download_chinook_data(){
echo "Downloading chinook data..."
cd $WORKDIR
git clone https://github.com/lerocha/chinook-database.git
cp chinook-database/ChinookDatabase/DataSources/Chinook_Sqlite.sqlite $WORKDIR/GenAIExamples/AgentQnA/tests/
}
function start_agent_and_api_server() { function start_agent_and_api_server() {
echo "Starting CRAG server" echo "Starting CRAG server"
docker run -d --runtime=runc --name=kdd-cup-24-crag-service -p=8080:8000 docker.io/aicrowd/kdd-cup-24-crag-mock-api:v0 docker run -d --runtime=runc --name=kdd-cup-24-crag-service -p=8080:8000 docker.io/aicrowd/kdd-cup-24-crag-mock-api:v0
echo "Starting Agent services" echo "Starting Agent services"
cd $WORKDIR/GenAIExamples/AgentQnA/docker_compose/intel/cpu/xeon cd $WORKDIR/GenAIExamples/AgentQnA/docker_compose/intel/cpu/xeon/
bash launch_agent_service_openai.sh bash launch_agent_service_openai.sh
sleep 2m
} }
function validate() { function validate() {
@@ -35,19 +44,64 @@ function validate() {
} }
function validate_agent_service() { function validate_agent_service() {
echo "----------------Test agent ----------------" # # test worker rag agent
local CONTENT=$(http_proxy="" curl http://${ip_address}:9090/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{ echo "======================Testing worker rag agent======================"
"query": "Tell me about Michael Jackson song thriller" export agent_port="9095"
}') prompt="Tell me about Michael Jackson song Thriller"
local EXIT_CODE=$(validate "$CONTENT" "Thriller" "react-agent-endpoint") local CONTENT=$(python3 $WORKDIR/GenAIExamples/AgentQnA/tests/test.py --prompt "$prompt" --agent_role "worker" --ext_port $agent_port)
docker logs react-agent-endpoint # echo $CONTENT
local EXIT_CODE=$(validate "$CONTENT" "Thriller" "rag-agent-endpoint")
echo $EXIT_CODE
local EXIT_CODE="${EXIT_CODE:0-1}"
if [ "$EXIT_CODE" == "1" ]; then if [ "$EXIT_CODE" == "1" ]; then
docker logs rag-agent-endpoint
exit 1
fi
# # test worker sql agent
echo "======================Testing worker sql agent======================"
export agent_port="9096"
prompt="How many employees are there in the company?"
local CONTENT=$(python3 $WORKDIR/GenAIExamples/AgentQnA/tests/test.py --prompt "$prompt" --agent_role "worker" --ext_port $agent_port)
local EXIT_CODE=$(validate "$CONTENT" "8" "sql-agent-endpoint")
echo $CONTENT
# echo $EXIT_CODE
local EXIT_CODE="${EXIT_CODE:0-1}"
if [ "$EXIT_CODE" == "1" ]; then
docker logs sql-agent-endpoint
exit 1
fi
# test supervisor react agent
echo "======================Testing supervisor react agent======================"
export agent_port="9090"
local CONTENT=$(python3 $WORKDIR/GenAIExamples/AgentQnA/tests/test.py --agent_role "supervisor" --ext_port $agent_port --stream)
local EXIT_CODE=$(validate "$CONTENT" "Iron" "react-agent-endpoint")
# echo $CONTENT
echo $EXIT_CODE
local EXIT_CODE="${EXIT_CODE:0-1}"
if [ "$EXIT_CODE" == "1" ]; then
docker logs react-agent-endpoint
exit 1 exit 1
fi fi
} }
function remove_chinook_data(){
echo "Removing chinook data..."
cd $WORKDIR
if [ -d "chinook-database" ]; then
rm -rf chinook-database
fi
echo "Chinook data removed!"
}
function main() { function main() {
echo "==================== Prepare data ===================="
download_chinook_data
echo "==================== Data prepare done ===================="
echo "==================== Start agent ====================" echo "==================== Start agent ===================="
start_agent_and_api_server start_agent_and_api_server
echo "==================== Agent started ====================" echo "==================== Agent started ===================="
@@ -57,4 +111,9 @@ function main() {
echo "==================== Agent service validated ====================" echo "==================== Agent service validated ===================="
} }
remove_chinook_data
main main
remove_chinook_data

18
AgentQnA/ui/svelte/README.md
View File

@@ -21,10 +21,22 @@ Here're some of the project's features:
cd AgentQnA/ui/svelte cd AgentQnA/ui/svelte
``` ```
3. Modify the required .env variables. 3. Modify the required .env variables. The `AGENT_URL` should be in the form of the following:
``` ```
AGENT_URL = '' AGENT_URL = "http://${ip_address}:${agent_port}/v1/chat/completions"
```
For example: assume that the ip address of the host machine is 10.10.10.1, and the agent port is 9090,then
```
AGENT_URL = "http://10.10.10.1:9090/v1/chat/completions"
```
You can get the ip address of the host machine by running the command below:
```bash
export ip_address=$(hostname -I | awk '{print $1}')
``` ```
4. **For Local Development:** 4. **For Local Development:**
@@ -57,4 +69,4 @@ Here're some of the project's features:
docker run -d -p 5173:5173 --name agent-ui opea:agent-ui docker run -d -p 5173:5173 --name agent-ui opea:agent-ui
``` ```
- The application will be available at `http://localhost:5173`. - The application will be available at `http://${ip_address}:5173`. You can access it with a web browser on your laptop. Note the `ip_address` should be the ip address of the host machine where the UI container runs.