jandres/GenAIExamples
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

[Code Translation] Modify prompt and streaming format (#99)

Browse Source 
Signed-off-by: letonghan <letong.han@intel.com>
This commit is contained in:
Letong Han
2024-04-26 21:56:48 +08:00
committed by GitHub
parent cce82832cb
commit 3cc035634d
4 changed files with 10 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
CodeTrans/README.md
View File

@@ -23,7 +23,7 @@ bash launch_tgi_service.sh
```sh
cd langchain/docker
bash build_docker.sh
docker run -it --name code_trans_server --net=host --ipc=host -e TGI_ENDPOINT=${TGI ENDPOINT} -e HUGGINGFACEHUB_API_TOKEN=${HUGGINGFACE_API_TOKEN} -e SERVER_PORT=8000 -e http_proxy=${http_proxy} -e https_proxy=${https_proxy} code_tranlation:latest bash
docker run -it --name code_trans_server --net=host --ipc=host -e TGI_ENDPOINT=${TGI ENDPOINT} -e HUGGINGFACEHUB_API_TOKEN=${HUGGINGFACE_API_TOKEN} -e SERVER_PORT=8000 -e http_proxy=${http_proxy} -e https_proxy=${https_proxy} intel/gen-ai-examples:code-translation bash
```
Here is the explanation of some of the above parameters:

2
CodeTrans/langchain/docker/build_docker.sh
View File

@@ -14,4 +14,4 @@
# limitations under the License.
docker build . -t code_tranlation:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy
docker build . -t intel/gen-ai-examples:code-translation --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy

4
CodeTrans/langchain/docker/codetrans-app/prompts.py
View File

@@ -18,8 +18,12 @@ prompt_template = """
### System: Please translate the following {language_from} codes into {language_to} codes.
### Original codes:
'''{language_from}
{source_code}
'''
### Translated codes:
"""
codetrans_prompt_template = PromptTemplate.from_template(prompt_template)

7
CodeTrans/langchain/docker/codetrans-app/server.py
View File

@@ -78,9 +78,10 @@ class CodeTranslationAPIRouter(APIRouter):
print(f"[codetrans - stream] prompt:{prompt}")
async def stream_generator():
async for chunk in self.llm.astream_log(prompt):
print(f"[codetrans - stream] data: {chunk}")
yield f"data: {chunk}\n\n"
for chunk in self.llm.stream(prompt):
chunk_repr = repr(chunk.encode("utf-8"))
print(f"[codetrans - stream] data: {chunk_repr}")
yield f"data: {chunk_repr}\n\n"
yield "data: [DONE]\n\n"
return StreamingResponse(stream_generator(), media_type="text/event-stream")