diff --git a/README.md b/README.md index c1ba2bd8d..1013c76bf 100644 --- a/README.md +++ b/README.md @@ -2,240 +2,108 @@ # Generative AI Examples -This project provides a collective list of Generative AI (GenAI) and Retrieval-Augmented Generation (RAG) examples such as chatbot with question and answering (ChatQnA), code generation (CodeGen), document summary (DocSum), etc. - -[![version](https://img.shields.io/badge/release-0.6-green)](https://github.com/opea-project/GenAIExamples/releases) +[![version](https://img.shields.io/badge/release-0.8-green)](https://github.com/opea-project/GenAIExamples/releases) [![license](https://img.shields.io/badge/license-Apache%202-blue)](https://github.com/intel/neural-compressor/blob/master/LICENSE) ---
-## GenAI Examples +## Introduction -All the examples are well-validated on Intel platforms. In addition, these examples are: +GenAIComps-based Generative AI examples offer streamlined deployment, testing, and scalability. All examples are fully compatible with Docker and Kubernetes, supporting a wide range of hardware platforms such as Gaudi, Xeon, and other hardwares. -- Easy to use. Use ecosystem-compliant APIs to build the end-to-end GenAI examples +## Architecture -- Easy to customize. Customize the example using different framework, LLM, embedding, serving etc. +GenAIComps is a service-based tool that includes microservice components such as llm, embedding, reranking, and so on. Using these components, various examples in GenAIExample can be constructed, including ChatQnA, DocSum, etc. -- Easy to deploy. Deploy the GenAI examples with performance on Intel platforms +GenAIInfra, part of the OPEA containerization and cloud-native suite, enables quick and efficient deployment of GenAIExamples in the cloud. -> **Note**: -> The below support matrix gives the validated configurations. Feel free to customize per your needs. +GenAIEvals measures service performance metrics such as throughput, latency, and accuracy for GenAIExamples. This feature helps users compare performance across various hardware configurations easily. -### ChatQnA +## Getting Started -[ChatQnA](./ChatQnA/README.md) is an example of chatbot for question and answering through retrieval argumented generation (RAG). +GenAIExamples offers flexible deployment options that cater to different user needs, enabling efficient use and deployment in various environments. Here’s a brief overview of the three primary methods: Python startup, Docker Compose, and Kubernetes. + +1. Docker Compose: Check the released docker images in [docker image list](./docker_images_list.md) for detailed information. +2. Kubernetes: Follow the steps at [K8s Install](https://github.com/opea-project/docs/tree/main/guide/installation/k8s_install) and [GMC Install](https://github.com/opea-project/docs/blob/main/guide/installation/gmc_install/gmc_install.md) to setup k8s and GenAI environment . + +Users can choose the most suitable approach based on ease of setup, scalability needs, and the environment in which they are operating. + +### Deployment - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FrameworkLLMEmbeddingVector DatabaseServingHWDescription
LangChain/LlamaIndexNeuralChat-7BBGE-BaseRedisTGI TEIXeon/Gaudi2/GPUChatbot
LangChain/LlamaIndexNeuralChat-7BBGE-BaseChromaTGI TEIXeon/Gaudi2Chatbot
LangChain/LlamaIndexMistral-7BBGE-BaseRedisTGI TEIXeon/Gaudi2Chatbot
LangChain/LlamaIndexMistral-7BBGE-BaseQdrantTGI TEIXeon/Gaudi2Chatbot
LangChain/LlamaIndexQwen2-7BBGE-BaseRedisTEIXeon/Gaudi2Chatbot
Use CasesDeployment
Docker ComposeKubernetes
XeonGaudi
ChatQnAXeon LinkGaudi LinkK8s Link
CodeGenXeon LinkGaudi LinkK8s Link
CodeTransXeon LinkGaudi LinkK8s Link
DocSumXeon LinkGaudi LinkK8s Link
SearchQnAXeon LinkGaudi LinkK8s Link
FaqGenXeon LinkGaudi LinkK8s Link
TranslationXeon LinkGaudi LinkK8s Link
AudioQnAXeon LinkGaudi LinkNot supported yet
VisualQnAXeon LinkGaudi LinkNot supported yet
-### CodeGen +## Support Examples -[CodeGen](./CodeGen/README.md) is an example of copilot designed for code generation in Visual Studio Code. - - - - - - - - - - - - - - - - - - -
FrameworkLLMServingHWDescription
LangChain/LlamaIndexmeta-llama/CodeLlama-7b-hfTGIXeon/Gaudi2Copilot
- -### CodeTrans - -[CodeTrans](./CodeTrans/README.md) is an example of chatbot for converting code written in one programming language to another programming language while maintaining the same functionality. - - - - - - - - - - - - - - - - - - -
FrameworkLLMServingHWDescription
LangChain/LlamaIndexHuggingFaceH4/mistral-7b-grokTGIXeon/Gaudi2Code Translation
- -### DocSum - -[DocSum](./DocSum/README.md) is an example of chatbot for summarizing the content of documents or reports. - - - - - - - - - - - - - - - - - - - - - - - - - -
FrameworkLLMServingHWDescription
LangChain/LlamaIndexNeuralChat-7BTGIXeon/Gaudi2Chatbot
LangChain/LlamaIndexMistral-7BTGIXeon/Gaudi2Chatbot
- -### Language Translation - -[Language Translation](./Translation/README.md) is an example of chatbot for converting a source-language text to an equivalent target-language text. - - - - - - - - - - - - - - - - - - -
FrameworkLLMServingHWDescription
LangChain/LlamaIndexhaoranxu/ALMA-13BTGIXeon/Gaudi2Language Translation
- -### SearchQnA - -[SearchQnA](./SearchQnA/README.md) is an example of chatbot for using search engine to enhance QA quality. - - - - - - - - - - - - - - - - - - - - - - - - - -
FrameworkLLMServingHWDescription
LangChain/LlamaIndexNeuralChat-7BTGIXeon/Gaudi2Chatbot
LangChain/LlamaIndexMistral-7BTGIXeon/Gaudi2Chatbot
- -### VisualQnA - -[VisualQnA](./VisualQnA/README.md) is an example of chatbot for question and answering based on the images. - - - - - - - - - - - - - - -
LLMHWDescription
LLaVA-1.5-7BGaudi2Chatbot
- -> **_NOTE:_** The `Language Translation`, `SearchQnA`, `VisualQnA` and other use cases not listing here are in active development. The code structure of these use cases are subject to change. +Check [here](./supported_examples.md) for detailed information of supported examples, models, hardwares, etc. ## Additional Content diff --git a/supported_examples.md b/supported_examples.md new file mode 100644 index 000000000..a04fb7208 --- /dev/null +++ b/supported_examples.md @@ -0,0 +1,220 @@ +
+ +# Supported Examples + +
+ +This document introduces the supported examples of GenAIExamples. The supported Vector Database, LLM models, serving frameworks and hardwares are listed as below. + +## ChatQnA + +[ChatQnA](./ChatQnA/README.md) is an example of chatbot for question and answering through retrieval argumented generation (RAG). + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FrameworkLLMEmbeddingVector DatabaseServingHWDescription
LangChain/LlamaIndexNeuralChat-7BBGE-BaseRedisTGI TEIXeon/Gaudi2/GPUChatbot
LangChain/LlamaIndexNeuralChat-7BBGE-BaseChromaTGI TEIXeon/Gaudi2Chatbot
LangChain/LlamaIndexMistral-7BBGE-BaseRedisTGI TEIXeon/Gaudi2Chatbot
LangChain/LlamaIndexMistral-7BBGE-BaseQdrantTGI TEIXeon/Gaudi2Chatbot
LangChain/LlamaIndexQwen2-7BBGE-BaseRedisTEIXeon/Gaudi2Chatbot
+ +### CodeGen + +[CodeGen](./CodeGen/README.md) is an example of copilot designed for code generation in Visual Studio Code. + + + + + + + + + + + + + + + + + + +
FrameworkLLMServingHWDescription
LangChain/LlamaIndexmeta-llama/CodeLlama-7b-hfTGIXeon/Gaudi2Copilot
+ +### CodeTrans + +[CodeTrans](./CodeTrans/README.md) is an example of chatbot for converting code written in one programming language to another programming language while maintaining the same functionality. + + + + + + + + + + + + + + + + + + +
FrameworkLLMServingHWDescription
LangChain/LlamaIndexHuggingFaceH4/mistral-7b-grokTGIXeon/Gaudi2Code Translation
+ +### DocSum + +[DocSum](./DocSum/README.md) is an example of chatbot for summarizing the content of documents or reports. + + + + + + + + + + + + + + + + + + + + + + + + + +
FrameworkLLMServingHWDescription
LangChain/LlamaIndexNeuralChat-7BTGIXeon/Gaudi2Chatbot
LangChain/LlamaIndexMistral-7BTGIXeon/Gaudi2Chatbot
+ +### Language Translation + +[Language Translation](./Translation/README.md) is an example of chatbot for converting a source-language text to an equivalent target-language text. + + + + + + + + + + + + + + + + + + +
FrameworkLLMServingHWDescription
LangChain/LlamaIndexhaoranxu/ALMA-13BTGIXeon/Gaudi2Language Translation
+ +### SearchQnA + +[SearchQnA](./SearchQnA/README.md) is an example of chatbot for using search engine to enhance QA quality. + + + + + + + + + + + + + + + + + + + + + + + + + +
FrameworkLLMServingHWDescription
LangChain/LlamaIndexNeuralChat-7BTGIXeon/Gaudi2Chatbot
LangChain/LlamaIndexMistral-7BTGIXeon/Gaudi2Chatbot
+ +### VisualQnA + +[VisualQnA](./VisualQnA/README.md) is an example of chatbot for question and answering based on the images. + + + + + + + + + + + + + + +
LLMHWDescription
LLaVA-1.5-7BGaudi2Chatbot
+ +> **_NOTE:_** The `Language Translation`, `SearchQnA`, `VisualQnA` and other use cases not listing here are in active development. The code structure of these use cases are subject to change.