jandres/GenAIExamples
1
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

Enable Benchmark with Langsmith (#96)

Browse Source 
* update benchmark with langsmith

Signed-off-by: letonghan <letong.han@intel.com>

---------

Signed-off-by: letonghan <letong.han@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
Letong Han
2024-05-24 19:53:50 +08:00
committed by GitHub
parent ecb8c7986a
commit dc5f486515
28 changed files with 66 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
comps/dataprep/redis/README.md
View File

@@ -17,6 +17,9 @@ Please refer to this [readme](../../../vectorstores/langchain/redis/README.md).
```bash
export REDIS_URL="redis://${your_ip}:6379"
export INDEX_NAME=${your_index_name}
export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=${your_langchain_api_key}
export LANGCHAIN_PROJECT="opea/gen-ai-comps:dataprep"
```
## Start Document Preparation Microservice for Redis with Python Script
@@ -41,6 +44,9 @@ docker build -t opea/gen-ai-comps:dataprep-redis-xeon-server --build-arg https_p
```bash
export REDIS_URL="redis://${your_ip}:6379"
export INDEX_NAME=${your_index_name}
export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=${your_langchain_api_key}
export LANGCHAIN_PROJECT="opea/gen-ai-comps:dataprep"
docker run -d --name="dataprep-redis-server" -p 6007:6007 --ipc=host -e http_proxy=$http_proxy -e https_proxy=$https_proxy -e REDIS_URL=$REDIS_URL -e INDEX_NAME=$INDEX_NAME opea/gen-ai-comps:dataprep-redis-xeon-server
```

1
comps/dataprep/redis/docker/docker-compose-dataprep-redis.yaml
View File

@@ -32,6 +32,7 @@ services:
REDIS_URL: ${REDIS_URL}
INDEX_NAME: ${INDEX_NAME}
TEI_ENDPOINT: ${TEI_ENDPOINT}
LANGCHAIN_API_KEY: ${LANGCHAIN_API_KEY}
restart: unless-stopped
networks:

2
comps/dataprep/redis/prepare_doc_redis.py
View File

@@ -23,6 +23,7 @@ from fastapi import File, Form, HTTPException, UploadFile
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.embeddings import HuggingFaceBgeEmbeddings, HuggingFaceHubEmbeddings
from langchain_community.vectorstores import Redis
from langsmith import traceable
from comps import DocPath, opea_microservices, register_microservice
from comps.dataprep.utils import docment_loader, parse_html
@@ -107,6 +108,7 @@ def ingest_link_to_redis(link_list: List[str]):
@register_microservice(name="opea_service@prepare_doc_redis", endpoint="/v1/dataprep", host="0.0.0.0", port=6007)
@traceable(run_type="tool")
async def ingest_documents(
files: Optional[Union[UploadFile, List[UploadFile]]] = File(None), link_list: Optional[str] = Form(None)
):

1
comps/dataprep/redis/requirements.txt
View File

@@ -5,6 +5,7 @@ fastapi
huggingface_hub
langchain
langchain-community
langsmith
numpy
opentelemetry-api
opentelemetry-exporter-otlp

3
comps/embeddings/README.md
View File

@@ -61,6 +61,9 @@ Start the embedding service with the TEI_EMBEDDING_ENDPOINT.
```bash
cd langchain
export TEI_EMBEDDING_ENDPOINT="http://localhost:$yourport"
export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=${your_langchain_api_key}
export LANGCHAIN_PROJECT="opea/gen-ai-comps:embeddings"
python embedding_tei_gaudi.py
```

1
comps/embeddings/langchain/docker/docker_compose_embedding.yaml
View File

@@ -25,6 +25,7 @@ services:
http_proxy: ${http_proxy}
https_proxy: ${https_proxy}
TEI_EMBEDDING_ENDPOINT: ${TEI_EMBEDDING_ENDPOINT}
LANGCHAIN_API_KEY: ${LANGCHAIN_API_KEY}
restart: unless-stopped
networks:

5
comps/embeddings/langchain/embedding_tei_gaudi.py
View File

@@ -15,8 +15,9 @@
import os
from langchain_community.embeddings import HuggingFaceHubEmbeddings
from langsmith import traceable
from comps import EmbedDoc768, ServiceType, TextDoc, opea_microservices, opea_telemetry, register_microservice
from comps import EmbedDoc768, ServiceType, TextDoc, opea_microservices, register_microservice
@register_microservice(
@@ -28,7 +29,7 @@ from comps import EmbedDoc768, ServiceType, TextDoc, opea_microservices, opea_te
input_datatype=TextDoc,
output_datatype=EmbedDoc768,
)
@opea_telemetry
@traceable(run_type="embedding")
def embedding(input: TextDoc) -> EmbedDoc768:
embed_vector = embeddings.embed_query(input.text)
embed_vector = embed_vector[:768] # Keep only the first 768 elements

1
comps/embeddings/langchain/requirements.txt
View File

@@ -2,6 +2,7 @@ docarray[full]
fastapi
huggingface_hub
langchain
langsmith
opentelemetry-api
opentelemetry-exporter-otlp
opentelemetry-sdk

6
comps/guardrails/README.md
View File

@@ -34,6 +34,9 @@ pip install -r requirements.txt
```bash
export HUGGINGFACEHUB_API_TOKEN=${your_hf_api_token}
export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=${your_langchain_api_key}
export LANGCHAIN_PROJECT="opea/gen-ai-comps:gaurdrails"
volume=$PWD/data
model_id="meta-llama/LlamaGuard-7b"
docker pull ghcr.io/huggingface/tgi-gaudi:1.2.1
@@ -68,6 +71,9 @@ In order to start TGI and LLM services, you need to setup the following environm
export HUGGINGFACEHUB_API_TOKEN=${your_hf_api_token}
export SAFETY_GUARD_ENDPOINT="http://${your_ip}:8088"
export LLM_MODEL_ID=${your_hf_llm_model}
export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=${your_langchain_api_key}
export LANGCHAIN_PROJECT="opea/gen-ai-comps:gaurdrails"
```
## Build Docker Image

1
comps/guardrails/langchain/docker/docker_compose_guardrails.yaml
View File

@@ -34,6 +34,7 @@ services:
http_proxy: ${http_proxy}
https_proxy: ${https_proxy}
SAFETY_GUARD_ENDPOINT: ${SAFETY_GUARD_ENDPOINT}
LANGCHAIN_API_KEY: ${LANGCHAIN_API_KEY}
restart: unless-stopped
networks:

6
comps/guardrails/langchain/guardrails_tgi_gaudi.py
View File

@@ -15,8 +15,9 @@
import os
from langchain_community.llms import HuggingFaceEndpoint
from langsmith import traceable
from comps import ServiceType, TextDoc, opea_microservices, opea_telemetry, register_microservice
from comps import ServiceType, TextDoc, opea_microservices, register_microservice
unsafe_categories = """O1: Violence and Hate.
Should not
@@ -75,7 +76,6 @@ unsafe_dict = {
}
@opea_telemetry
def moderation_prompt_for_chat(chat):
# For simplicity, we assume the chat dictionary correctly alternates "user" and "assistant" roles
# Feel free to include error checking if needed
@@ -113,7 +113,7 @@ Provide your safety assessment for {role} in the above conversation:
input_datatype=TextDoc,
output_datatype=TextDoc,
)
@opea_telemetry
@traceable(run_type="llm")
def safety_guard(input: TextDoc) -> TextDoc:
# prompt guardrails
response_input_guard = llm_guard(moderation_prompt_for_chat([{"role": "User", "content": input.text}]))

1
comps/guardrails/requirements.txt
View File

@@ -2,6 +2,7 @@ docarray[full]
fastapi
huggingface_hub
langchain_community
langsmith
opentelemetry-api
opentelemetry-exporter-otlp
opentelemetry-sdk

6
comps/llms/README.md
View File

@@ -20,6 +20,9 @@ pip install -r requirements.txt
```bash
export HUGGINGFACEHUB_API_TOKEN=${your_hf_api_token}
export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=${your_langchain_api_key}
export LANGCHAIN_PROJECT="opea/gen-ai-comps:llms"
docker run -p 8008:80 -v ./data:/data --name tgi_service --shm-size 1g ghcr.io/huggingface/text-generation-inference:1.4 --model-id ${your_hf_llm_model}
```
@@ -51,6 +54,9 @@ In order to start TGI and LLM services, you need to setup the following environm
export HUGGINGFACEHUB_API_TOKEN=${your_hf_api_token}
export TGI_LLM_ENDPOINT="http://${your_ip}:8008"
export LLM_MODEL_ID=${your_hf_llm_model}
export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=${your_langchain_api_key}
export LANGCHAIN_PROJECT="opea/gen-ai-comps:llms"
```
## Build Docker Image

1
comps/llms/summarization/tgi/docker_compose_llm.yaml
View File

@@ -35,6 +35,7 @@ services:
https_proxy: ${https_proxy}
TGI_LLM_ENDPOINT: ${TGI_LLM_ENDPOINT}
HUGGINGFACEHUB_API_TOKEN: ${HUGGINGFACEHUB_API_TOKEN}
LANGCHAIN_API_KEY: ${LANGCHAIN_API_KEY}
restart: unless-stopped
networks:

7
comps/llms/summarization/tgi/llm.py
View File

@@ -19,11 +19,12 @@ from langchain.chains.summarize import load_summarize_chain
from langchain.docstore.document import Document
from langchain.text_splitter import CharacterTextSplitter
from langchain_community.llms import HuggingFaceEndpoint
from langsmith import traceable
from comps import GeneratedDoc, LLMParamsDoc, ServiceType, opea_microservices, opea_telemetry, register_microservice
from comps import GeneratedDoc, LLMParamsDoc, ServiceType, opea_microservices, register_microservice
@opea_telemetry
@traceable(run_type="tool")
def post_process_text(text: str):
if text == " ":
return "data: @#$\n\n"
@@ -42,7 +43,7 @@ def post_process_text(text: str):
host="0.0.0.0",
port=9000,
)
@opea_telemetry
@traceable(run_type="llm")
def llm_generate(input: LLMParamsDoc):
llm_endpoint = os.getenv("TGI_LLM_ENDPOINT", "http://localhost:8080")
llm = HuggingFaceEndpoint(

1
comps/llms/summarization/tgi/requirements.txt
View File

@@ -3,6 +3,7 @@ fastapi
huggingface_hub
langchain==0.1.16
langserve
langsmith
opentelemetry-api
opentelemetry-exporter-otlp
opentelemetry-sdk

1
comps/llms/text-generation/tgi/docker_compose_llm.yaml
View File

@@ -35,6 +35,7 @@ services:
https_proxy: ${https_proxy}
TGI_LLM_ENDPOINT: ${TGI_LLM_ENDPOINT}
HUGGINGFACEHUB_API_TOKEN: ${HUGGINGFACEHUB_API_TOKEN}
LANGCHAIN_API_KEY: ${LANGCHAIN_API_KEY}
restart: unless-stopped
networks:

7
comps/llms/text-generation/tgi/llm.py
View File

@@ -16,11 +16,12 @@ import os
from fastapi.responses import StreamingResponse
from langchain_community.llms import HuggingFaceEndpoint
from langsmith import traceable
from comps import GeneratedDoc, LLMParamsDoc, ServiceType, opea_microservices, opea_telemetry, register_microservice
from comps import GeneratedDoc, LLMParamsDoc, ServiceType, opea_microservices, register_microservice
@opea_telemetry
@traceable(run_type="tool")
def post_process_text(text: str):
if text == " ":
return "data: @#$\n\n"
@@ -39,7 +40,7 @@ def post_process_text(text: str):
host="0.0.0.0",
port=9000,
)
@opea_telemetry
@traceable(run_type="llm")
def llm_generate(input: LLMParamsDoc):
llm_endpoint = os.getenv("TGI_LLM_ENDPOINT", "http://localhost:8080")
llm = HuggingFaceEndpoint(

1
comps/llms/text-generation/tgi/requirements.txt
View File

@@ -3,6 +3,7 @@ fastapi
huggingface_hub
langchain==0.1.16
langserve
langsmith
opentelemetry-api
opentelemetry-exporter-otlp
opentelemetry-sdk

3
comps/reranks/README.md
View File

@@ -16,6 +16,9 @@ pip install -r requirements.txt
```bash
export HUGGINGFACEHUB_API_TOKEN=${your_hf_api_token}
export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=${your_langchain_api_key}
export LANGCHAIN_PROJECT="opea/gen-ai-comps:reranks"
model=BAAI/bge-reranker-large
revision=refs/pr/4
volume=$PWD/data

1
comps/reranks/langchain/docker/docker_compose_reranking.yaml
View File

@@ -35,6 +35,7 @@ services:
https_proxy: ${https_proxy}
TEI_RERANKING_ENDPOINT: ${TEI_RERANKING_ENDPOINT}
HUGGINGFACEHUB_API_TOKEN: ${HUGGINGFACEHUB_API_TOKEN}
LANGCHAIN_API_KEY: ${LANGCHAIN_API_KEY}
restart: unless-stopped
networks:

5
comps/reranks/langchain/local_reranking.py
View File

@@ -12,9 +12,10 @@
# See the License for the specific language governing permissions and
# limitations under the License.
from langsmith import traceable
from sentence_transformers import CrossEncoder
from comps import RerankedDoc, SearchedDoc, ServiceType, opea_microservices, opea_telemetry, register_microservice
from comps import RerankedDoc, SearchedDoc, ServiceType, opea_microservices, register_microservice
@register_microservice(
@@ -26,7 +27,7 @@ from comps import RerankedDoc, SearchedDoc, ServiceType, opea_microservices, ope
input_datatype=SearchedDoc,
output_datatype=RerankedDoc,
)
@opea_telemetry
@traceable(run_type="llm")
def reranking(input: SearchedDoc) -> RerankedDoc:
query_and_docs = [(input.initial_query, doc.text) for doc in input.retrieved_docs]
scores = reranker_model.predict(query_and_docs)

5
comps/reranks/langchain/reranking_tei_xeon.py
View File

@@ -18,8 +18,9 @@ import os
import requests
from langchain_core.prompts import ChatPromptTemplate
from langsmith import traceable
from comps import LLMParamsDoc, SearchedDoc, ServiceType, opea_microservices, opea_telemetry, register_microservice
from comps import LLMParamsDoc, SearchedDoc, ServiceType, opea_microservices, register_microservice
@register_microservice(
@@ -31,7 +32,7 @@ from comps import LLMParamsDoc, SearchedDoc, ServiceType, opea_microservices, op
input_datatype=SearchedDoc,
output_datatype=LLMParamsDoc,
)
@opea_telemetry
@traceable(run_type="llm")
def reranking(input: SearchedDoc) -> LLMParamsDoc:
docs = [doc.text for doc in input.retrieved_docs]
url = tei_reranking_endpoint + "/rerank"

1
comps/reranks/requirements.txt
View File

@@ -1,6 +1,7 @@
docarray[full]
fastapi
langchain
langsmith
opentelemetry-api
opentelemetry-exporter-otlp
opentelemetry-sdk

3
comps/retrievers/README.md
View File

@@ -32,6 +32,9 @@ docker run -d --name="redis-vector-db" -p 6379:6379 -p 8001:8001 redis/redis-sta
export REDIS_URL="redis://${your_ip}:6379"
export INDEX_NAME=${your_index_name}
export TEI_EMBEDDING_ENDPOINT=${your_embedding_endpoint}
export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=${your_langchain_api_key}
export LANGCHAIN_PROJECT="opea/gen-ai-comps:retrievers"
```
## Start Retriever Service with Local Model

1
comps/retrievers/langchain/docker/docker_compose_retriever.yaml
View File

@@ -27,6 +27,7 @@ services:
REDIS_URL: ${REDIS_URL}
INDEX_NAME: ${INDEX_NAME}
TEI_EMBEDDING_ENDPOINT: ${TEI_EMBEDDING_ENDPOINT}
LANGCHAIN_API_KEY: ${LANGCHAIN_API_KEY}
restart: unless-stopped
networks:

13
comps/retrievers/langchain/retriever_redis.py
View File

@@ -16,17 +16,10 @@ import os
from langchain_community.embeddings import HuggingFaceBgeEmbeddings, HuggingFaceHubEmbeddings
from langchain_community.vectorstores import Redis
from langsmith import traceable
from redis_config import EMBED_MODEL, INDEX_NAME, INDEX_SCHEMA, REDIS_URL
from comps import (
EmbedDoc768,
SearchedDoc,
ServiceType,
TextDoc,
opea_microservices,
opea_telemetry,
register_microservice,
)
from comps import EmbedDoc768, SearchedDoc, ServiceType, TextDoc, opea_microservices, register_microservice
tei_embedding_endpoint = os.getenv("TEI_EMBEDDING_ENDPOINT")
@@ -38,7 +31,7 @@ tei_embedding_endpoint = os.getenv("TEI_EMBEDDING_ENDPOINT")
host="0.0.0.0",
port=7000,
)
@opea_telemetry
@traceable(run_type="retriever")
def retrieve(input: EmbedDoc768) -> SearchedDoc:
# Create vectorstore
if tei_embedding_endpoint:

1
comps/retrievers/requirements.txt
View File

@@ -2,6 +2,7 @@ docarray[full]
easyocr
fastapi
langchain_community
langsmith
opentelemetry-api
opentelemetry-exporter-otlp
opentelemetry-sdk