jandres/GenAIExamples
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

Adding files to deploy Translation application on ROCm vLLM (#1648)

Browse Source 
Signed-off-by: Chingis Yundunov <YundunovCN@sibedge.com>
Signed-off-by: Artem Astafev <a.astafev@datamonsters.com>
This commit is contained in:
chyundunovDatamonsters
2025-03-31 12:49:33 +07:00
committed by GitHub
parent b7f24762a3
commit 340fa075bd
8 changed files with 704 additions and 58 deletions
Download Patch File Download Diff File Expand all files Collapse all files

432
Translation/docker_compose/amd/gpu/rocm/README.md
View File

@@ -1,42 +1,117 @@
# Build and deploy Translation Application on AMD GPU (ROCm)
## Build images
## Build Docker Images
### Build the LLM Docker Image
### 1. Build Docker Image
```bash
### Cloning repo
git clone https://github.com/opea-project/GenAIComps.git
cd GenAIComps
- #### Create application install directory and go to it:
### Build Docker image
docker build -t opea/llm-textgen:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/llms/src/text-generation/Dockerfile .
```
```bash
mkdir ~/translation-install && cd translation-install
```
### Build the MegaService Docker Image
- #### Clone the repository GenAIExamples (the default repository branch "main" is used here):
```bash
### Cloning repo
git clone https://github.com/opea-project/GenAIExamples
cd GenAIExamples/Translation/
```bash
git clone https://github.com/opea-project/GenAIExamples.git
```
### Build Docker image
docker build -t opea/translation:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f Dockerfile .
```
If you need to use a specific branch/tag of the GenAIExamples repository, then (v1.3 replace with its own value):
### Build the UI Docker Image
```bash
git clone https://github.com/opea-project/GenAIExamples.git && cd GenAIExamples && git checkout v1.3
```
```bash
cd GenAIExamples/Translation/ui
### Build UI Docker image
docker build -t opea/translation-ui:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f docker/Dockerfile .
```
We remind you that when using a specific version of the code, you need to use the README from this version:
## Deploy Translation Application
- #### Go to build directory:
### Features of Docker compose for AMD GPUs
```bash
cd ~/translation-install/GenAIExamples/Translation/docker_image_build
```
1. Added forwarding of GPU devices to the container TGI service with instructions:
- Cleaning up the GenAIComps repository if it was previously cloned in this directory.
This is necessary if the build was performed earlier and the GenAIComps folder exists and is not empty:
```bash
echo Y | rm -R GenAIComps
```
- #### Clone the repository GenAIComps (the default repository branch "main" is used here):
```bash
git clone https://github.com/opea-project/GenAIComps.git
```
If you use a specific tag of the GenAIExamples repository,
then you should also use the corresponding tag for GenAIComps. (v1.3 replace with its own value):
```bash
git clone https://github.com/opea-project/GenAIComps.git && cd GenAIComps && git checkout v1.3
```
We remind you that when using a specific version of the code, you need to use the README from this version.
- #### Setting the list of images for the build (from the build file.yaml)
If you want to deploy a vLLM-based or TGI-based application, then the set of services is installed as follows:
#### vLLM-based application
```bash
service_list="vllm-rocm translation translation-ui llm-textgen nginx"
```
#### TGI-based application
```bash
service_list="translation translation-ui llm-textgen nginx"
```
- #### Optional. Pull TGI Docker Image (Do this if you want to use TGI)
```bash
docker pull ghcr.io/huggingface/text-generation-inference:2.3.1-rocm
```
- #### Build Docker Images
```bash
docker compose -f build.yaml build ${service_list} --no-cache
```
After the build, we check the list of images with the command:
```bash
docker image ls
```
The list of images should include:
##### vLLM-based application:
- opea/vllm-rocm:latest
- opea/llm-textgen:latest
- opea/nginx:latest
- opea/translation:latest
- opea/translation-ui:latest
##### TGI-based application:
- ghcr.io/huggingface/text-generation-inference:2.3.1-rocm
- opea/llm-textgen:latest
- opea/nginx:latest
- opea/translation:latest
- opea/translation-ui:latest
---
### Docker Compose Configuration for AMD GPUs
To enable GPU support for AMD GPUs, the following configuration is added to the Docker Compose file:
- compose_vllm.yaml - for vLLM-based application
- compose.yaml - for TGI-based
```yaml
shm_size: 1g
@@ -51,16 +126,14 @@ security_opt:
- seccomp:unconfined
```
In this case, all GPUs are thrown. To reset a specific GPU, you need to use specific device names cardN and renderN.
For example:
This configuration forwards all available GPUs to the container. To use a specific GPU, specify its `cardN` and `renderN` device IDs. For example:
```yaml
shm_size: 1g
devices:
- /dev/kfd:/dev/kfd
- /dev/dri/card0:/dev/dri/card0
- /dev/dri/render128:/dev/dri/render128
- /dev/dri/renderD128:/dev/dri/renderD128
cap_add:
- SYS_PTRACE
group_add:
@@ -69,60 +142,305 @@ security_opt:
- seccomp:unconfined
```
To find out which GPU device IDs cardN and renderN correspond to the same GPU, use the GPU driver utility
**How to Identify GPU Device IDs:**
Use AMD GPU driver utilities to determine the correct `cardN` and `renderN` IDs for your GPU.
### Go to the directory with the Docker compose file
### Set deploy environment variables
#### Setting variables in the operating system environment:
##### Set variable HUGGINGFACEHUB_API_TOKEN:
```bash
cd GenAIExamples/Translation/docker_compose/amd/gpu/rocm
### Replace the string 'your_huggingfacehub_token' with your HuggingFacehub repository access token.
export HUGGINGFACEHUB_API_TOKEN='your_huggingfacehub_token'
```
### Set environments
#### Set variables value in set_env\*\*\*\*.sh file:
In the file "GenAIExamples/Translation/docker_compose/amd/gpu/rocm/set_env.sh " it is necessary to set the required values. Parameter assignments are specified in the comments for each variable setting command
Go to Docker Compose directory:
```bash
cd ~/translation-install/GenAIExamples/Translation/docker_compose/amd/gpu/rocm
```
The example uses the Nano text editor. You can use any convenient text editor:
#### If you use vLLM
```bash
nano set_env_vllm.sh
```
#### If you use TGI
```bash
nano set_env.sh
```
If you are in a proxy environment, also set the proxy-related environment variables:
```bash
export http_proxy="Your_HTTP_Proxy"
export https_proxy="Your_HTTPs_Proxy"
```
Set the values of the variables:
- **HOST_IP, HOST_IP_EXTERNAL** - These variables are used to configure the name/address of the service in the operating system environment for the application services to interact with each other and with the outside world.
If your server uses only an internal address and is not accessible from the Internet, then the values for these two variables will be the same and the value will be equal to the server's internal name/address.
If your server uses only an external, Internet-accessible address, then the values for these two variables will be the same and the value will be equal to the server's external name/address.
If your server is located on an internal network, has an internal address, but is accessible from the Internet via a proxy/firewall/load balancer, then the HOST_IP variable will have a value equal to the internal name/address of the server, and the EXTERNAL_HOST_IP variable will have a value equal to the external name/address of the proxy/firewall/load balancer behind which the server is located.
We set these values in the file set_env\*\*\*\*.sh
- **Variables with names like "**\*\*\*\*\*\*\_PORT"\*\* - These variables set the IP port numbers for establishing network connections to the application services.
The values shown in the file set_env.sh or set_env_vllm they are the values used for the development and testing of the application, as well as configured for the environment in which the development is performed. These values must be configured in accordance with the rules of network access to your environment's server, and must not overlap with the IP ports of other applications that are already in use.
#### Set variables with script set_env\*\*\*\*.sh
#### If you use vLLM
```bash
. set_env_vllm.sh
```
#### If you use TGI
```bash
chmod +x set_env.sh
. set_env.sh
```
### Run services
### Start the services:
```
docker compose up -d
```
# Validate the MicroServices and MegaService
## Validate TGI service
#### If you use vLLM
```bash
curl http://${TRANSLATION_HOST_IP}:${TRANSLATIONS_TGI_SERVICE_PORT}/generate \
docker compose -f compose_vllm.yaml up -d
```
#### If you use TGI
```bash
docker compose -f compose.yaml up -d
```
All containers should be running and should not restart:
##### If you use vLLM:
- translationn-vllm-service
- translation-tgi-service
- translation-llm
- translation-backend-server
- translation-ui-server
- translation-nginx-server
##### If you use TGI:
- translation-tgi-service
- translation-llm
- translation-backend-server
- translation-ui-server
- translation-nginx-server
---
## Validate the Services
### 1. Validate the vLLM/TGI Service
#### If you use vLLM:
```bash
DATA='{"model": "haoranxu/ALMA-13B", "prompt": "What is Deep Learning?", "max_tokens": 100, "temperature": 0}'
curl http://${HOST_IP}:${TRANSLATION_VLLM_SERVICE_PORT}/v1/chat/completions \
-X POST \
-d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":17, "do_sample": true}}' \
-d "$DATA" \
-H 'Content-Type: application/json'
```
## Validate LLM service
Checking the response from the service. The response should be similar to JSON:
```json
{
"id": "cmpl-059dd7fb311a46c2b807e0b3315e730c",
"object": "text_completion",
"created": 1743063706,
"model": "haoranxu/ALMA-13B",
"choices": [
{
"index": 0,
"text": " Deep Learning is a subset of machine learning. It attempts to mimic the way the human brain learns. Deep Learning is a subset of machine learning. It attempts to mimic the way the human brain learns. Deep Learning is a subset of machine learning. It attempts to mimic the way the human brain learns. Deep Learning is a subset of machine learning. It attempts to mimic the way the human brain learns. Deep Learning is a subset of machine learning",
"logprobs": null,
"finish_reason": "length",
"stop_reason": null,
"prompt_logprobs": null
}
],
"usage": {
"prompt_tokens": 6,
"total_tokens": 106,
"completion_tokens": 100,
"prompt_tokens_details": null
}
}
```
If the service response has a meaningful response in the value of the "choices.message.content" key,
then we consider the vLLM service to be successfully launched
#### If you use TGI:
```bash
curl http://${TRANSLATION_HOST_IP}:9000/v1/chat/completions \
DATA='{"inputs":"What is Deep Learning?",'\
'"parameters":{"max_new_tokens":256,"do_sample": true}}'
curl http://${HOST_IP}:${TRANSLATION_TGI_SERVICE_PORT}/generate \
-X POST \
-d '{"query":"Translate this from Chinese to English:\nChinese: 我爱机器翻译。\nEnglish:"}' \
-d "$DATA" \
-H 'Content-Type: application/json'
```
## Validate MegaService
Checking the response from the service. The response should be similar to JSON:
```bash
curl http://${TRANSLATION_HOST_IP}:${TRANSLATION_BACKEND_SERVICE_PORT}/v1/translation -H "Content-Type: application/json" -d '{
"language_from": "Chinese","language_to": "English","source_language": "我爱机器翻译。"}'
```json
{
"generated_text": "\n\n What can it Do? What's the Hype? What Should You Do If"
}
```
## Validate Nginx service
If the service response has a meaningful response in the value of the "generated_text" key,
then we consider the TGI service to be successfully launched
### 2. Validate the LLM Service
```bash
curl http://${TRANSLATION_HOST_IP}:${TRANSLATION_NGINX_PORT}/v1/translation \
-H "Content-Type: application/json" \
-d '{"language_from": "Chinese","language_to": "English","source_language": "我爱机器翻译。"}'
DATA='{"query":"What is Deep Learning?",'\
'"max_tokens":32,"top_k":10,"top_p":0.95,"typical_p":0.95,"temperature":0.01,'\
'"repetition_penalty":1.03,"stream":false}'
curl http://${HOST_IP}:${TRANSLATION_LLM_SERVICE_PORT}/v1/chat/completions \
-X POST \
-d "$DATA" \
-H 'Content-Type: application/json'
```
Checking the response from the service. The response should be similar to JSON:
```json
{
"id": "",
"choices": [
{
"finish_reason": "length",
"index": 0,
"logprobs": null,
"text": " Deep Learning is a subset of machine learning. It attempts to mimic the way the human brain learns. Deep Learning is a subset of machine learning."
}
],
"created": 1742978568,
"model": "haoranxu/ALMA-13B",
"object": "text_completion",
"system_fingerprint": "2.3.1-sha-a094729-rocm",
"usage": {
"completion_tokens": 32,
"prompt_tokens": 6,
"total_tokens": 38,
"completion_tokens_details": null,
"prompt_tokens_details": null
}
}
```
### 3. Validate Nginx Service
```bash
DATA='{"language_from": "Chinese","language_to": "English","source_language": "我爱机器翻译。"}'
curl http://${HOST_IP}:${TRANSLATION_LLM_SERVICE_PORT}/v1/translation \
-d "$DATA" \
-H 'Content-Type: application/json'
```
Checking the response from the service. The response should be similar to JSON:
```textmate
data: {"id":"","choices":[{"finish_reason":"","index":0,"logprobs":null,"text":" I"}],"created":1743062099,"model":"haoranxu/ALMA-13B","object":"text_completion","system_fingerprint":"2.3.1-sha-a094729-rocm","usage":null}
data: {"id":"","choices":[{"finish_reason":"","index":0,"logprobs":null,"text":" love"}],"created":1743062099,"model":"haoranxu/ALMA-13B","object":"text_completion","system_fingerprint":"2.3.1-sha-a094729-rocm","usage":null}
data: {"id":"","choices":[{"finish_reason":"","index":0,"logprobs":null,"text":" machine"}],"created":1743062099,"model":"haoranxu/ALMA-13B","object":"text_completion","system_fingerprint":"2.3.1-sha-a094729-rocm","usage":null}
data: {"id":"","choices":[{"finish_reason":"","index":0,"logprobs":null,"text":" translation"}],"created":1743062099,"model":"haoranxu/ALMA-13B","object":"text_completion","system_fingerprint":"2.3.1-sha-a094729-rocm","usage":null}
data: {"id":"","choices":[{"finish_reason":"","index":0,"logprobs":null,"text":"."}],"created":1743062099,"model":"haoranxu/ALMA-13B","object":"text_completion","system_fingerprint":"2.3.1-sha-a094729-rocm","usage":null}
data: {"id":"","choices":[{"finish_reason":"eos_token","index":0,"logprobs":null,"text":"</s>"}],"created":1743062099,"model":"haoranxu/ALMA-13B","object":"text_completion","system_fingerprint":"2.3.1-sha-a094729-rocm","usage":{"completion_tokens":6,"prompt_tokens":3071,"total_tokens":3077,"completion_tokens_details":null,"prompt_tokens_details":null}}
data: [DONE]
```
### 4. Validate MegaService
```bash
DATA='{"language_from": "Chinese","language_to": "English","source_language": "我爱机器翻译。"}'
curl http://${HOST_IP}:${TRANSLATION_BACKEND_SERVICE_PORT}/v1/translation \
-H "Content-Type: application/json" \
-d "$DATA"
```
Checking the response from the service. The response should be similar to JSON:
```textmate
data: {"id":"","choices":[{"finish_reason":"","index":0,"logprobs":null,"text":" I"}],"created":1742978968,"model":"haoranxu/ALMA-13B","object":"text_completion","system_fingerprint":"2.3.1-sha-a094729-rocm","usage":null}
data: {"id":"","choices":[{"finish_reason":"","index":0,"logprobs":null,"text":" love"}],"created":1742978968,"model":"haoranxu/ALMA-13B","object":"text_completion","system_fingerprint":"2.3.1-sha-a094729-rocm","usage":null}
data: {"id":"","choices":[{"finish_reason":"","index":0,"logprobs":null,"text":" machine"}],"created":1742978968,"model":"haoranxu/ALMA-13B","object":"text_completion","system_fingerprint":"2.3.1-sha-a094729-rocm","usage":null}
data: {"id":"","choices":[{"finish_reason":"","index":0,"logprobs":null,"text":" translation"}],"created":1742978968,"model":"haoranxu/ALMA-13B","object":"text_completion","system_fingerprint":"2.3.1-sha-a094729-rocm","usage":null}
data: {"id":"","choices":[{"finish_reason":"","index":0,"logprobs":null,"text":"."}],"created":1742978968,"model":"haoranxu/ALMA-13B","object":"text_completion","system_fingerprint":"2.3.1-sha-a094729-rocm","usage":null}
data: {"id":"","choices":[{"finish_reason":"eos_token","index":0,"logprobs":null,"text":"</s>"}],"created":1742978968,"model":"haoranxu/ALMA-13B","object":"text_completion","system_fingerprint":"2.3.1-sha-a094729-rocm","usage":{"completion_tokens":6,"prompt_tokens":3071,"total_tokens":3077,"completion_tokens_details":null,"prompt_tokens_details":null}}
data: [DONE]
```
If the response text is similar to the one above, then we consider the service verification successful.
### 5. Validate Frontend
To access the UI, use the URL - http://${EXTERNAL_HOST_IP}:${TRANSLATION_FRONTEND_SERVICE_PORT} A page should open when you click through to this address:
![UI start page](../../../../assets/img/translation-ui-starting-page.png)
If a page of this type has opened, then we believe that the service is running and responding, and we can proceed to functional UI testing.
Let's enter the task for the service in the "Input" field. For example, "我爱机器翻译" with selected "German" as language source and press Enter. After that, a page with the result of the task should open:
![UI start page](../../../../assets/img/translation-ui-response-example.png)
If the result shown on the page is correct, then we consider the verification of the UI service to be successful.
### 6. Stop application
#### If you use vLLM
```bash
cd ~/translation-install/GenAIExamples/Translation/docker_compose/amd/gpu/rocm
docker compose -f compose_vllm.yaml down
```
#### If you use TGI
```bash
cd ~/translation-install/GenAIExamples/Translation/docker_compose/amd/gpu/rocm
docker compose -f compose.yaml down
```

2
Translation/docker_compose/amd/gpu/rocm/compose.yaml
View File

@@ -90,7 +90,7 @@ services:
- translation-backend-server
- translation-ui-server
ports:
- "${TRANSLATION_NGINX_PORT:-80}:80"
- "${TRANSLATION_NGINX_PORT:-80}:8080"
environment:
- no_proxy=${no_proxy}
- https_proxy=${https_proxy}

107
Translation/docker_compose/amd/gpu/rocm/compose_vllm.yaml Normal file
View File

@@ -0,0 +1,107 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
services:
translation-vllm-service:
image: ${REGISTRY:-opea}/vllm-rocm:${TAG:-latest}
container_name: translation-vllm-service
ports:
- "${TRANSLATION_VLLM_SERVICE_PORT:-8081}:8011"
environment:
no_proxy: ${no_proxy}
http_proxy: ${http_proxy}
https_proxy: ${https_proxy}
HUGGINGFACEHUB_API_TOKEN: ${TRANSLATION_HUGGINGFACEHUB_API_TOKEN}
HF_TOKEN: ${TRANSLATION_HUGGINGFACEHUB_API_TOKEN}
HF_HUB_DISABLE_PROGRESS_BARS: 1
HF_HUB_ENABLE_HF_TRANSFER: 0
WILM_USE_TRITON_FLASH_ATTENTION: 0
PYTORCH_JIT: 0
volumes:
- "./data:/data"
shm_size: 20G
devices:
- /dev/kfd:/dev/kfd
- /dev/dri/:/dev/dri/
cap_add:
- SYS_PTRACE
group_add:
- video
security_opt:
- seccomp:unconfined
- apparmor=unconfined
command: "--model ${TRANSLATION_LLM_MODEL_ID} --swap-space 16 --disable-log-requests --dtype float16 --tensor-parallel-size 1 --host 0.0.0.0 --port 8011 --num-scheduler-steps 1 --distributed-executor-backend \"mp\""
ipc: host
translation-llm:
image: ${REGISTRY:-opea}/llm-textgen:${TAG:-latest}
container_name: translation-llm-textgen-server
depends_on:
- translation-vllm-service
ports:
- "${TRANSLATION_LLM_PORT:-9000}:9000"
ipc: host
environment:
no_proxy: ${no_proxy}
http_proxy: ${http_proxy}
https_proxy: ${https_proxy}
LLM_ENDPOINT: ${TRANSLATION_LLM_ENDPOINT}
LLM_MODEL_ID: ${TRANSLATION_LLM_MODEL_ID}
HUGGINGFACEHUB_API_TOKEN: ${TRANSLATION_HUGGINGFACEHUB_API_TOKEN}
HF_TOKEN: ${TRANSLATION_HUGGINGFACEHUB_API_TOKEN}
LLM_COMPONENT_NAME: "OpeaTextGenService"
HF_HUB_DISABLE_PROGRESS_BARS: 1
HF_HUB_ENABLE_HF_TRANSFER: 0
restart: unless-stopped
translation-backend-server:
image: ${REGISTRY:-opea}/translation:${TAG:-latest}
container_name: translation-backend-server
depends_on:
- translation-vllm-service
- translation-llm
ports:
- "${TRANSLATION_BACKEND_SERVICE_PORT:-8888}:8888"
environment:
no_proxy: ${no_proxy}
https_proxy: ${https_proxy}
http_proxy: ${http_proxy}
MEGA_SERVICE_HOST_IP: ${TRANSLATION_MEGA_SERVICE_HOST_IP}
LLM_SERVICE_HOST_IP: ${TRANSLATION_LLM_SERVICE_HOST_IP}
LLM_SERVICE_PORT: ${TRANSLATION_LLM_PORT}
ipc: host
restart: always
translation-ui-server:
image: ${REGISTRY:-opea}/translation-ui:${TAG:-latest}
container_name: translation-ui-server
depends_on:
- translation-backend-server
ports:
- "${TRANSLATION_FRONTEND_SERVICE_PORT:-5173}:5173"
environment:
no_proxy: ${no_proxy}
https_proxy: ${https_proxy}
http_proxy: ${http_proxy}
BASE_URL: ${TRANSLATION_BACKEND_SERVICE_ENDPOINT}
ipc: host
restart: always
translation-nginx-server:
image: ${REGISTRY:-opea}/nginx:${TAG:-latest}
container_name: translation-nginx-server
depends_on:
- translation-backend-server
- translation-ui-server
ports:
- "${TRANSLATION_NGINX_PORT:-80}:8080"
environment:
no_proxy: ${no_proxy}
https_proxy: ${https_proxy}
http_proxy: ${http_proxy}
FRONTEND_SERVICE_IP: ${TRANSLATION_FRONTEND_SERVICE_IP}
FRONTEND_SERVICE_PORT: ${TRANSLATION_FRONTEND_SERVICE_PORT}
BACKEND_SERVICE_NAME: ${TRANSLATION_BACKEND_SERVICE_NAME}
BACKEND_SERVICE_IP: ${TRANSLATION_BACKEND_SERVICE_IP}
BACKEND_SERVICE_PORT: ${TRANSLATION_BACKEND_SERVICE_PORT}
ipc: host
restart: always
networks:
default:
driver: bridge

23
Translation/docker_compose/amd/gpu/rocm/set_env_vllm.sh Normal file
View File

@@ -0,0 +1,23 @@
#!/usr/bin/env bash
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
# SPDX-License-Identifier: Apache-2.0
export HOST_IP=''
export EXTERNAL_HOST_IP=''
export TRANSLATION_LLM_MODEL_ID="haoranxu/ALMA-13B"
export TRANSLATION_VLLM_SERVICE_PORT=8088
export TRANSLATION_LLM_ENDPOINT="http://${HOST_IP}:${TRANSLATION_VLLM_SERVICE_PORT}"
export TRANSLATION_LLM_PORT=9088
export TRANSLATION_HUGGINGFACEHUB_API_TOKEN=${HUGGINGFACEHUB_API_TOKEN}
export TRANSLATION_MEGA_SERVICE_HOST_IP=${HOST_IP}
export TRANSLATION_LLM_SERVICE_HOST_IP=${HOST_IP}
export TRANSLATION_FRONTEND_SERVICE_IP=${HOST_IP}
export TRANSLATION_FRONTEND_SERVICE_PORT=18122
export TRANSLATION_BACKEND_SERVICE_NAME=translation
export TRANSLATION_BACKEND_SERVICE_IP=${HOST_IP}
export TRANSLATION_BACKEND_SERVICE_PORT=18121
export TRANSLATION_BACKEND_SERVICE_ENDPOINT="http://${EXTERNAL_HOST_IP}:${TRANSLATION_BACKEND_SERVICE_PORT}/v1/translation"
export TRANSLATION_NGINX_PORT=18123