Add restful api access for ChatQnA (#128)

Signed-off-by: lvliang-intel <liang1.lv@intel.com>
Signed-off-by: chensuyue <suyue.chen@intel.com>

ChatQnA/microservice/gaudi/README.md

@@ -95,8 +95,6 @@ export INDEX_NAME="rag-redis"
 export HUGGINGFACEHUB_API_TOKEN=${your_hf_api_token}
 ```
 
-Note: Please replace with `your_ip` with you external IP address, do not use localhost.
-
 Note: Please replace with `host_ip` with you external IP address, do not use localhost.
 
 ### Start Microservice Docker Containers
@@ -185,10 +183,21 @@ Following the validation of all aforementioned microservices, we are now prepare
 
 ## 🚀 Construct Mega Service
 
-Modify the `initial_inputs` of line 34 in `chatqna.py`, then you will get the ChatQnA result of this mega service.
+To construct the Mega Service, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `chatqna.py` Python script. Upon executing the script, each microservice's intermediate results will be displayed, allowing users to verify the accuracy of the outcomes and make targeted modifications if necessary.
 
-All of the intermediate results will be printed for each microservice. Users can check the accuracy of the results to make targeted modifications.
+To launch the Mega Service, simply run the following command:
 
 ```bash
 python chatqna.py
 ```
+
+## 🚀 Access the Mega Service
+
+Once the script is executed, a FastAPI server will be initiated. Users can interact with the service through the `/v1/chatqna` endpoint. Here's an example using `curl`:
+
+```bash
+curl http://127.0.0.1:8888/v1/chatqna -H "Content-Type: application/json" -d '{
+    "model": "Intel/neural-chat-7b-v3-3",
+    "messages": "What is the revenue of Nike in 2023?"
+    }'
+```

ChatQnA/microservice/gaudi/chatqna.py

@@ -37,6 +37,7 @@ class ChatQnAService:
         self.service_builder.flow_to(embedding, retriever)
         self.service_builder.flow_to(retriever, rerank)
         self.service_builder.flow_to(rerank, llm)
+        self.service_builder.start_server()
 
     def schedule(self):
         self.service_builder.schedule(initial_inputs={"text": "What is the revenue of Nike in 2023?"})

ChatQnA/microservice/xeon/README.md

@@ -70,6 +70,8 @@ export INDEX_NAME="rag-redis"
 export HUGGINGFACEHUB_API_TOKEN=${your_hf_api_token}
 ```
 
+Note: Please replace with `host_ip` with you external IP address, do not use localhost.
+
 ### Start Microservice Docker Containers
 
 ```bash
@@ -154,9 +156,9 @@ Following the validation of all aforementioned microservices, we are now prepare
 
 ## 🚀 Construct Mega Service
 
-Modify the `initial_inputs` of line 34 in `chatqna.py`, then you will get the ChatQnA result of this mega service.
+To construct the Mega Service, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `chatqna.py` Python script. Upon executing the script, each microservice's intermediate results will be displayed, allowing users to verify the accuracy of the outcomes and make targeted modifications if necessary.
 
-All of the intermediate results will be printed for each microservices. Users can check the accuracy of the results to make targeted modifications.
+To launch the Mega Service, simply run the following command:
 
 ### Run Mega Service with Python
 
@@ -184,3 +186,14 @@ Then you can check the result of your chatqna service with the command below.
 ```bash
 docker logs chatqna-xeon-server
 ```
+
+## 🚀 Access the Mega Service
+
+Once the mega service docker is launched, a FastAPI server will be initiated. Users can interact with the service through the `/v1/chatqna` endpoint. Here's an example using `curl`:
+
+```bash
+curl http://127.0.0.1:8888/v1/chatqna -H "Content-Type: application/json" -d '{
+     "model": "Intel/neural-chat-7b-v3-3",
+     "messages": "What is the revenue of Nike in 2023?"
+     }
+```

ChatQnA/microservice/xeon/chatqna.py

@@ -41,6 +41,7 @@ class ChatQnAService:
         self.service_builder.flow_to(embedding, retriever)
         self.service_builder.flow_to(retriever, rerank)
         self.service_builder.flow_to(rerank, llm)
+        self.service_builder.start_server()
 
     def schedule(self):
         self.service_builder.schedule(initial_inputs={"text": "What is the revenue of Nike in 2023?"})

ChatQnA/tests/test_chatqna_on_gaudi.sh

@@ -115,7 +115,12 @@ function check_microservices() {
 }
 
 function run_megaservice() {
+    # Construct Mega Service
     python chatqna.py > ${LOG_PATH}/run_megaservice.log
+    # Access the Mega Service
+    curl http://127.0.0.1:8888/v1/chatqna -H "Content-Type: application/json" -d '{
+        "model": "Intel/neural-chat-7b-v3-3",
+        "messages": "What is the revenue of Nike in 2023?"}' > ${LOG_PATH}/curl_megaservice.log
 }
 
 function check_results() {
@@ -125,6 +130,10 @@ function check_results() {
         status=true
     fi
 
+    if [[ -f $LOG_PATH/curl_megaservice.log ]] && [[ $(grep -c "\$51.2 billion" $LOG_PATH/curl_megaservice.log) == 0 ]]; then
+        status=false
+    fi
+
     if [ $status == false ]; then
         echo "Response check failed, please check the logs in artifacts!"
         exit 1

ChatQnA/tests/test_chatqna_on_xeon.sh

@@ -108,16 +108,26 @@ function check_microservices() {
 }
 
 function run_megaservice() {
+    # Construct Mega Service
     python chatqna.py > ${LOG_PATH}/run_megaservice.log
+    # Access the Mega Service
+    curl http://127.0.0.1:8888/v1/chatqna -H "Content-Type: application/json" -d '{
+        "model": "Intel/neural-chat-7b-v3-3",
+        "messages": "What is the revenue of Nike in 2023?"}' > ${LOG_PATH}/curl_megaservice.log
 }
 
 function check_results() {
+
     echo "Checking response results, make sure the output is reasonable. "
     local status=false
     if [[ -f $LOG_PATH/run_megaservice.log ]] && [[ $(grep -c "\$51.2 billion" $LOG_PATH/run_megaservice.log) != 0 ]]; then
         status=true
     fi
 
+    if [[ -f $LOG_PATH/curl_megaservice.log ]] && [[ $(grep -c "\$51.2 billion" $LOG_PATH/curl_megaservice.log) == 0 ]]; then
+        status=false
+    fi
+
     if [ $status == false ]; then
         echo "Response check failed, please check the logs in artifacts!"
         exit 1