3913c7bb36
* add TEI_ENDPOINT for dataprep Signed-off-by: letonghan <letong.han@intel.com> * add TEI_ENDPOINT for dataprep Signed-off-by: letonghan <letong.han@intel.com> * update ui test file Signed-off-by: letonghan <letong.han@intel.com> * add dataprep test into script Signed-off-by: letonghan <letong.han@intel.com> * refine e2e test of chatqna Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix missing dockerfile path Signed-off-by: letonghan <letong.han@intel.com> * check test issue Signed-off-by: letonghan <letong.han@intel.com> * check dataprep log Signed-off-by: letonghan <letong.han@intel.com> * reverse codetrans Signed-off-by: letonghan <letong.han@intel.com> * cd log path Signed-off-by: letonghan <letong.han@intel.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add hf token in compose yaml Signed-off-by: letonghan <letong.han@intel.com> * add redis_host Signed-off-by: letonghan <letong.han@intel.com> * add upload link test Signed-off-by: letonghan <letong.han@intel.com> * re-format validate dataprep Signed-off-by: letonghan <letong.han@intel.com> * fix typo Signed-off-by: letonghan <letong.han@intel.com> * fix frontend env name Signed-off-by: letonghan <letong.han@intel.com> * fix ci issue Signed-off-by: letonghan <letong.han@intel.com> --------- Signed-off-by: letonghan <letong.han@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Generative AI Examples
Introduction
GenAIComps-based Generative AI examples offer streamlined deployment, testing, and scalability. All examples are fully compatible with Docker and Kubernetes, supporting a wide range of hardware platforms such as Gaudi, Xeon, and other hardwares.
Architecture
GenAIComps is a service-based tool that includes microservice components such as llm, embedding, reranking, and so on. Using these components, various examples in GenAIExample can be constructed, including ChatQnA, DocSum, etc.
GenAIInfra, part of the OPEA containerization and cloud-native suite, enables quick and efficient deployment of GenAIExamples in the cloud.
GenAIEval measures service performance metrics such as throughput, latency, and accuracy for GenAIExamples. This feature helps users compare performance across various hardware configurations easily.
Getting Started
GenAIExamples offers flexible deployment options that cater to different user needs, enabling efficient use and deployment in various environments. Here’s a brief overview of the three primary methods: Python startup, Docker Compose, and Kubernetes.
- Docker Compose: Check the released docker images in docker image list for detailed information.
- Kubernetes: Follow the steps at K8s Install and GMC Install to setup k8s and GenAI environment .
Users can choose the most suitable approach based on ease of setup, scalability needs, and the environment in which they are operating.
Deployment
| Use Cases | Deployment | |||
|---|---|---|---|---|
| Docker Compose | Kubernetes | |||
| Xeon | Gaudi | |||
| ChatQnA | Xeon Link | Gaudi Link | K8s Link | |
| CodeGen | Xeon Link | Gaudi Link | K8s Link | |
| CodeTrans | Xeon Link | Gaudi Link | K8s Link | |
| DocSum | Xeon Link | Gaudi Link | K8s Link | |
| SearchQnA | Xeon Link | Gaudi Link | K8s Link | |
| FaqGen | Xeon Link | Gaudi Link | K8s Link | |
| Translation | Xeon Link | Gaudi Link | K8s Link | |
| AudioQnA | Xeon Link | Gaudi Link | Not supported yet | |
| VisualQnA | Xeon Link | Gaudi Link | Not supported yet | |
Support Examples
Check here for detailed information of supported examples, models, hardwares, etc.