add tool resource

Co-authored-by: Gimling <huangjl@ruyi.ai>
Co-authored-by: takatost <takatost@gmail.com>

.github/workflows/api-tests.yml

@@ -39,6 +39,14 @@ jobs:
       - name: Run Tool
         run: dev/pytest/pytest_tools.sh
 
+      - name: Set up Sandbox
+        uses: hoverkraft-tech/compose-action@v2.0.0
+        with:
+          compose-file: |
+            docker/docker-compose.middleware.yaml
+          services: |
+            sandbox
+
       - name: Run Workflow
         run: dev/pytest/pytest_workflow.sh
 

CONTRIBUTING.md

@@ -4,7 +4,7 @@ We need to be nimble and ship fast given where we are, but we also want to make
 
 This guide, like Dify itself, is a constant work in progress. We highly appreciate your understanding if at times it lags behind the actual project, and welcome any feedback for us to improve.
 
-In terms of licensing, please take a minute to read our short [License and Contributor Agreement](./license). The community also adheres to the [code of conduct](https://github.com/langgenius/.github/blob/main/CODE_OF_CONDUCT.md).
+In terms of licensing, please take a minute to read our short [License and Contributor Agreement](./LICENSE). The community also adheres to the [code of conduct](https://github.com/langgenius/.github/blob/main/CODE_OF_CONDUCT.md).
 
 ## Before you jump in
 

CONTRIBUTING_CN.md

@@ -4,7 +4,7 @@
 
 这份指南，就像 Dify 本身一样，是一个不断改进的工作。如果有时它落后于实际项目，我们非常感谢你的理解，并欢迎任何反馈以供我们改进。
 
-在许可方面，请花一分钟阅读我们简短的[许可证和贡献者协议](./license)。社区还遵守[行为准则](https://github.com/langgenius/.github/blob/main/CODE_OF_CONDUCT.md)。
+在许可方面，请花一分钟阅读我们简短的[许可证和贡献者协议](./LICENSE)。社区还遵守[行为准则](https://github.com/langgenius/.github/blob/main/CODE_OF_CONDUCT.md)。
 
 ## 在开始之前
 

README_JA.md

@@ -2,7 +2,7 @@
 
 <p align="center">
   <a href="https://cloud.dify.ai">Dify Cloud</a> ·
-  <a href="https://docs.dify.ai/getting-started/install-self-hosted">自己ホスティング</a> ·
+  <a href="https://docs.dify.ai/getting-started/install-self-hosted">セルフホスト</a> ·
   <a href="https://docs.dify.ai">ドキュメント</a> ·
   <a href="https://cal.com/guchenhe/dify-demo">デモのスケジュール</a>
 </p>
@@ -54,7 +54,7 @@ DifyはオープンソースのLLMアプリケーション開発プラットフ
 
 
 
-**2. 網羅的なモデルサポート**: 
+**2. 包括的なモデルサポート**: 
   数百のプロプライエタリ/オープンソースのLLMと、数十の推論プロバイダーおよびセルフホスティングソリューションとのシームレスな統合を提供します。GPT、Mistral、Llama3、およびOpenAI API互換のモデルをカバーします。サポートされているモデルプロバイダーの完全なリストは[こちら](https://docs.dify.ai/getting-started/readme/model-providers)をご覧ください。
 
 ![providers-v5](https://github.com/langgenius/dify/assets/13230914/5a17bdbe-097a-4100-8363-40255b70f6e3)
@@ -94,9 +94,9 @@ DifyはオープンソースのLLMアプリケーション開発プラットフ
   </tr>
   <tr>
     <td align="center">サポートされているLLM</td>
-    <td align="center">豊富なバリエーション</td>
-    <td align="center">豊富なバリエーション</td>
-    <td align="center">豊富なバリエーション</td>
+    <td align="center">バリエーション豊富</td>
+    <td align="center">バリエーション豊富</td>
+    <td align="center">バリエーション豊富</td>
     <td align="center">OpenAIのみ</td>
   </tr>
   <tr>
@@ -146,34 +146,34 @@ DifyはオープンソースのLLMアプリケーション開発プラットフ
 ## Difyの使用方法
 
 - **クラウド </br>**
-[こちら](https://dify.ai)のDify Cloudサービスを利用して、セットアップが不要で誰でも試すことができます。サンドボックスプランでは、200回の無料のGPT-4呼び出しが含まれています。
+[こちら](https://dify.ai)のDify Cloudサービスを利用して、セットアップ不要で試すことができます。サンドボックスプランには、200回の無料のGPT-4呼び出しが含まれています。
 
 - **Dify Community Editionのセルフホスティング</br>**
-この[スターターガイド](#quick-start)を使用して、環境でDifyをすばやく実行できます。
-さらなる参照や詳細な手順については、[ドキュメント](https://docs.dify.ai)をご覧ください。
+この[スターターガイド](#quick-start)を使用して、ローカル環境でDifyを簡単に実行できます。
+さらなる参考資料や詳細な手順については、[ドキュメント](https://docs.dify.ai)をご覧ください。
 
 - **エンタープライズ/組織向けのDify</br>**
 追加のエンタープライズ向け機能を提供しています。[こちらからミーティングを予約](https://cal.com/guchenhe/30min)したり、[メールを送信](mailto:business@dify.ai?subject=[GitHub]Business%20License%20Inquiry)してエンタープライズのニーズについて相談してください。 </br>
   > AWSを使用しているスタートアップや中小企業の場合は、[AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-t22mebxzwjhu6)のDify Premiumをチェックして、ワンクリックで独自のAWS VPCにデプロイできます。カスタムロゴとブランディングでアプリを作成するオプションを備えた手頃な価格のAMIオファリングです。
 
 
-## 先を見る
+## 最新の情報を入手
 
-GitHubでDifyにスターを付け、新しいリリースをすぐに通知されます。
+GitHub上でDifyにスターを付けることで、Difyに関する新しいニュースを受け取れます。
 
 ![star-us](https://github.com/langgenius/dify/assets/13230914/b823edc1-6388-4e25-ad45-2f6b187adbb4)
 
 
 
 ## クイックスタート
-> Difyをインストールする前に、マシンが以下の最小システム要件を満たしていることを確認してください：
+> Difyをインストールする前に、お使いのマシンが以下の最小システム要件を満たしていることを確認してください：
 > 
 >- CPU >= 2コア
 >- RAM >= 4GB
 
 </br>
 
-Difyサーバーを起動する最も簡単な方法は、当社の[docker-compose.yml](docker/docker-compose.yaml)ファイルを実行することです。インストールコマンドを実行する前に、マシンに[Docker](https://docs.docker.com/get-docker/)と[Docker Compose](https://docs.docker.com/compose/install/)がインストールされていることを確認してください。
+Difyサーバーを起動する最も簡単な方法は、[docker-compose.yml](docker/docker-compose.yaml)ファイルを実行することです。インストールコマンドを実行する前に、マシンに[Docker](https://docs.docker.com/get-docker/)と[Docker Compose](https://docs.docker.com/compose/install/)がインストールされていることを確認してください。
 
 ```bash
 cd docker
@@ -216,7 +216,7 @@ docker compose up -d
 * [Discord](https://discord.gg/FngNHpbcY7). 主に: アプリケーションの共有やコミュニティとの交流。
 * [Twitter](https://twitter.com/dify_ai). 主に: アプリケーションの共有やコミュニティとの交流。
 
-または、直接チームメンバーとミーティングをスケジュールします：
+または、直接チームメンバーとミーティングをスケジュール：
 
 <table>
   <tr>
@@ -227,7 +227,7 @@ docker compose up -d
     <td><a href='https://cal.com
 
 /guchenhe/30min'>ミーティング</a></td>
-    <td>無料の30分間のミーティングをスケジュールしてください。</td>
+    <td>無料の30分間のミーティングをスケジュール</td>
   </tr>
   <tr>
     <td><a href='mailto:support@dify.ai?subject=[GitHub]Technical%20Support'>技術サポート</a></td>
@@ -242,4 +242,4 @@ docker compose up -d
 
 ## ライセンス
 
-プロジェクトはMITライセンスの下で利用可能です。[LICENSE](LICENSE)をご参照ください。
+このリポジトリは、Dify Open Source License にいくつかの追加制限を加えた[Difyオープンソースライセンス](LICENSE)の下で利用可能です。

api/.env.example

@@ -54,6 +54,9 @@ ALIYUN_OSS_BUCKET_NAME=your-bucket-name
 ALIYUN_OSS_ACCESS_KEY=your-access-key
 ALIYUN_OSS_SECRET_KEY=your-secret-key
 ALIYUN_OSS_ENDPOINT=your-endpoint
+ALIYUN_OSS_AUTH_VERSION=v1
+ALIYUN_OSS_REGION=your-region
+
 # Google Storage configuration
 GOOGLE_STORAGE_BUCKET_NAME=yout-bucket-name
 GOOGLE_STORAGE_SERVICE_ACCOUNT_JSON=your-google-service-account-json-base64-string
@@ -160,6 +163,8 @@ API_TOOL_DEFAULT_READ_TIMEOUT=60
 HTTP_REQUEST_MAX_CONNECT_TIMEOUT=300
 HTTP_REQUEST_MAX_READ_TIMEOUT=600
 HTTP_REQUEST_MAX_WRITE_TIMEOUT=600
+HTTP_REQUEST_NODE_MAX_BINARY_SIZE=10485760 # 10MB
+HTTP_REQUEST_NODE_MAX_TEXT_SIZE=1048576 # 1MB
 
 # Log file path
 LOG_FILE=

api/config.py

@@ -215,6 +215,8 @@ class Config:
         self.ALIYUN_OSS_ACCESS_KEY=get_env('ALIYUN_OSS_ACCESS_KEY')
         self.ALIYUN_OSS_SECRET_KEY=get_env('ALIYUN_OSS_SECRET_KEY')
         self.ALIYUN_OSS_ENDPOINT=get_env('ALIYUN_OSS_ENDPOINT')
+        self.ALIYUN_OSS_REGION=get_env('ALIYUN_OSS_REGION')
+        self.ALIYUN_OSS_AUTH_VERSION=get_env('ALIYUN_OSS_AUTH_VERSION')
         self.GOOGLE_STORAGE_BUCKET_NAME = get_env('GOOGLE_STORAGE_BUCKET_NAME')
         self.GOOGLE_STORAGE_SERVICE_ACCOUNT_JSON_BASE64 = get_env('GOOGLE_STORAGE_SERVICE_ACCOUNT_JSON_BASE64')
 

api/constants/languages.py

@@ -1,6 +1,6 @@
 
 
-languages = ['en-US', 'zh-Hans', 'zh-Hant', 'pt-BR', 'es-ES', 'fr-FR', 'de-DE', 'ja-JP', 'ko-KR', 'ru-RU', 'it-IT', 'uk-UA', 'vi-VN']
+languages = ['en-US', 'zh-Hans', 'zh-Hant', 'pt-BR', 'es-ES', 'fr-FR', 'de-DE', 'ja-JP', 'ko-KR', 'ru-RU', 'it-IT', 'uk-UA', 'vi-VN', 'pl-PL']
 
 language_timezone_mapping = {
     'en-US': 'America/New_York',
@@ -16,6 +16,7 @@ language_timezone_mapping = {
     'it-IT': 'Europe/Rome',
     'uk-UA': 'Europe/Kyiv',
     'vi-VN': 'Asia/Ho_Chi_Minh',
+    'pl-PL': 'Europe/Warsaw',
 }
 
 

api/controllers/console/app/workflow_statistic.py

@@ -227,7 +227,7 @@ class WorkflowAverageAppInteractionStatistic(Resource):
                     {{start}}
                     {{end}}
                 GROUP BY date, c.created_by) sub
-            GROUP BY sub.created_by, sub.date
+            GROUP BY sub.date
             """
         arg_dict = {'tz': account.timezone, 'app_id': app_model.id, 'triggered_from': WorkflowRunTriggeredFrom.APP_RUN.value}
 

api/core/agent/cot_agent_runner.py

@@ -219,7 +219,7 @@ class CotAgentRunner(BaseAgentRunner, ABC):
                         tool_input={scratchpad.action.action_name: scratchpad.action.action_input},
                         thought=scratchpad.thought,
                         observation={scratchpad.action.action_name: tool_invoke_response},
-                        tool_invoke_meta=tool_invoke_meta.to_dict(),
+                        tool_invoke_meta={scratchpad.action.action_name: tool_invoke_meta.to_dict()},
                         answer=scratchpad.agent_response,
                         messages_ids=message_file_ids,
                         llm_usage=usage_dict['usage']

api/core/app/apps/base_app_generator.py

@@ -13,7 +13,9 @@ class BaseAppGenerator:
         for variable_config in variables:
             variable = variable_config.variable
 
-            if variable not in user_inputs or not user_inputs[variable]:
+            if (variable not in user_inputs
+                    or user_inputs[variable] is None
+                    or (isinstance(user_inputs[variable], str) and user_inputs[variable] == '')):
                 if variable_config.required:
                     raise ValueError(f"{variable} is required in input form")
                 else:
@@ -22,7 +24,7 @@ class BaseAppGenerator:
 
             value = user_inputs[variable]
 
-            if value:
+            if value is not None:
                 if variable_config.type != VariableEntity.Type.NUMBER and not isinstance(value, str):
                     raise ValueError(f"{variable} in input form must be a string")
                 elif variable_config.type == VariableEntity.Type.NUMBER and isinstance(value, str):
@@ -44,7 +46,7 @@ class BaseAppGenerator:
             if value and isinstance(value, str):
                 filtered_inputs[variable] = value.replace('\x00', '')
             else:
-                filtered_inputs[variable] = value if value else None
+                filtered_inputs[variable] = value if value is not None else None
 
         return filtered_inputs
 

api/core/helper/code_executor/code_executor.py

@@ -1,3 +1,4 @@
+from enum import Enum
 from typing import Literal, Optional
 
 from httpx import post
@@ -28,7 +29,25 @@ class CodeExecutionResponse(BaseModel):
     data: Data
 
 
+class CodeLanguage(str, Enum):
+    PYTHON3 = 'python3'
+    JINJA2 = 'jinja2'
+    JAVASCRIPT = 'javascript'
+
+
 class CodeExecutor:
+    code_template_transformers = {
+        CodeLanguage.PYTHON3: PythonTemplateTransformer,
+        CodeLanguage.JINJA2: Jinja2TemplateTransformer,
+        CodeLanguage.JAVASCRIPT: NodeJsTemplateTransformer,
+    }
+
+    code_language_to_running_language = {
+        CodeLanguage.JAVASCRIPT: 'nodejs',
+        CodeLanguage.JINJA2: CodeLanguage.PYTHON3,
+        CodeLanguage.PYTHON3: CodeLanguage.PYTHON3,
+    }
+
     @classmethod
     def execute_code(cls, language: Literal['python3', 'javascript', 'jinja2'], preload: str, code: str) -> str:
         """
@@ -44,9 +63,7 @@ class CodeExecutor:
         }
 
         data = {
-            'language': 'python3' if language == 'jinja2' else
-                        'nodejs' if language == 'javascript' else
-                        'python3' if language == 'python3' else None,
+            'language': cls.code_language_to_running_language.get(language),
             'code': code,
             'preload': preload
         }
@@ -86,15 +103,9 @@ class CodeExecutor:
         :param inputs: inputs
         :return:
         """
-        template_transformer = None
-        if language == 'python3':
-            template_transformer = PythonTemplateTransformer
-        elif language == 'jinja2':
-            template_transformer = Jinja2TemplateTransformer
-        elif language == 'javascript':
-            template_transformer = NodeJsTemplateTransformer
-        else:
-            raise CodeExecutionException('Unsupported language')
+        template_transformer = cls.code_template_transformers.get(language)
+        if not template_transformer:
+            raise CodeExecutionException(f'Unsupported language {language}')
 
         runner, preload = template_transformer.transform_caller(code, inputs)
 

api/core/model_runtime/model_providers/_position.yaml

@@ -26,4 +26,6 @@
 - yi
 - openllm
 - localai
+- volcengine_maas
 - openai_api_compatible
+- deepseek

api/core/model_runtime/model_providers/azure_openai/_constant.py

@@ -482,6 +482,82 @@ LLM_BASE_MODELS = [
             )
         )
     ),
+    AzureBaseModel(
+        base_model_name='gpt-4-turbo-2024-04-09',
+        entity=AIModelEntity(
+            model='fake-deployment-name',
+            label=I18nObject(
+                en_US='fake-deployment-name-label',
+            ),
+            model_type=ModelType.LLM,
+            features=[
+                ModelFeature.AGENT_THOUGHT,
+                ModelFeature.VISION,
+                ModelFeature.MULTI_TOOL_CALL,
+                ModelFeature.STREAM_TOOL_CALL,
+            ],
+            fetch_from=FetchFrom.CUSTOMIZABLE_MODEL,
+            model_properties={
+                ModelPropertyKey.MODE: LLMMode.CHAT.value,
+                ModelPropertyKey.CONTEXT_SIZE: 128000,
+            },
+            parameter_rules=[
+                ParameterRule(
+                    name='temperature',
+                    **PARAMETER_RULE_TEMPLATE[DefaultParameterName.TEMPERATURE],
+                ),
+                ParameterRule(
+                    name='top_p',
+                    **PARAMETER_RULE_TEMPLATE[DefaultParameterName.TOP_P],
+                ),
+                ParameterRule(
+                    name='presence_penalty',
+                    **PARAMETER_RULE_TEMPLATE[DefaultParameterName.PRESENCE_PENALTY],
+                ),
+                ParameterRule(
+                    name='frequency_penalty',
+                    **PARAMETER_RULE_TEMPLATE[DefaultParameterName.FREQUENCY_PENALTY],
+                ),
+                _get_max_tokens(default=512, min_val=1, max_val=4096),
+                ParameterRule(
+                    name='seed',
+                    label=I18nObject(
+                        zh_Hans='种子',
+                        en_US='Seed'
+                    ),
+                    type='int',
+                    help=I18nObject(
+                        zh_Hans='如果指定，模型将尽最大努力进行确定性采样，使得重复的具有相同种子和参数的请求应该返回相同的结果。不能保证确定性，您应该参考 system_fingerprint 响应参数来监视变化。',
+                        en_US='If specified, model will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. Determinism is not guaranteed, and you should refer to the system_fingerprint response parameter to monitor changes in the backend.'
+                    ),
+                    required=False,
+                    precision=2,
+                    min=0,
+                    max=1,
+                ),
+                ParameterRule(
+                    name='response_format',
+                    label=I18nObject(
+                        zh_Hans='回复格式',
+                        en_US='response_format'
+                    ),
+                    type='string',
+                    help=I18nObject(
+                        zh_Hans='指定模型必须输出的格式',
+                        en_US='specifying the format that the model must output'
+                    ),
+                    required=False,
+                    options=['text', 'json_object']
+                ),
+            ],
+            pricing=PriceConfig(
+                input=0.001,
+                output=0.003,
+                unit=0.001,
+                currency='USD',
+            )
+        )
+    ),
     AzureBaseModel(
         base_model_name='gpt-4-vision-preview',
         entity=AIModelEntity(

api/core/model_runtime/model_providers/bedrock/bedrock.yaml

@@ -70,6 +70,10 @@ provider_credential_schema:
           label:
             en_US: AWS GovCloud (US-West)
             zh_Hans: AWS GovCloud (US-West)
+        - value: ap-southeast-2
+          label:
+            en_US: Asia Pacific (Sydney)
+            zh_Hans: 亚太地区 (悉尼)
     - variable: model_for_validation
       required: false
       label:

api/core/model_runtime/model_providers/cohere/cohere.yaml

@@ -32,6 +32,15 @@ provider_credential_schema:
         zh_Hans: 在此输入您的 API Key
         en_US: Enter your API Key
       show_on: [ ]
+    - variable: base_url
+      label:
+        zh_Hans: API Base
+        en_US: API Base
+      type: text-input
+      required: false
+      placeholder:
+        zh_Hans: 在此输入您的 API Base，如 https://api.cohere.ai/v1
+        en_US: Enter your API Base, e.g. https://api.cohere.ai/v1
 model_credential_schema:
   model:
     label:
@@ -70,3 +79,12 @@ model_credential_schema:
       placeholder:
         zh_Hans: 在此输入您的 API Key
         en_US: Enter your API Key
+    - variable: base_url
+      label:
+        zh_Hans: API Base
+        en_US: API Base
+      type: text-input
+      required: false
+      placeholder:
+        zh_Hans: 在此输入您的 API Base，如 https://api.cohere.ai/v1
+        en_US: Enter your API Base, e.g. https://api.cohere.ai/v1

api/core/model_runtime/model_providers/cohere/llm/llm.py

@@ -173,7 +173,7 @@ class CohereLargeLanguageModel(LargeLanguageModel):
         :return: full response or stream response chunk generator result
         """
         # initialize client
-        client = cohere.Client(credentials.get('api_key'))
+        client = cohere.Client(credentials.get('api_key'), base_url=credentials.get('base_url'))
 
         if stop:
             model_parameters['end_sequences'] = stop
@@ -233,7 +233,8 @@ class CohereLargeLanguageModel(LargeLanguageModel):
 
         return response
 
-    def _handle_generate_stream_response(self, model: str, credentials: dict, response: Iterator[GenerateStreamedResponse],
+    def _handle_generate_stream_response(self, model: str, credentials: dict,
+                                         response: Iterator[GenerateStreamedResponse],
                                          prompt_messages: list[PromptMessage]) -> Generator:
         """
         Handle llm stream response
@@ -317,7 +318,7 @@ class CohereLargeLanguageModel(LargeLanguageModel):
         :return: full response or stream response chunk generator result
         """
         # initialize client
-        client = cohere.Client(credentials.get('api_key'))
+        client = cohere.Client(credentials.get('api_key'), base_url=credentials.get('base_url'))
 
         if stop:
             model_parameters['stop_sequences'] = stop
@@ -636,7 +637,7 @@ class CohereLargeLanguageModel(LargeLanguageModel):
         :return: number of tokens
         """
         # initialize client
-        client = cohere.Client(credentials.get('api_key'))
+        client = cohere.Client(credentials.get('api_key'), base_url=credentials.get('base_url'))
 
         response = client.tokenize(
             text=text,

api/core/model_runtime/model_providers/cohere/rerank/rerank.py

@@ -44,7 +44,7 @@ class CohereRerankModel(RerankModel):
             )
 
         # initialize client
-        client = cohere.Client(credentials.get('api_key'))
+        client = cohere.Client(credentials.get('api_key'), base_url=credentials.get('base_url'))
         response = client.rerank(
             query=query,
             documents=docs,

api/core/model_runtime/model_providers/cohere/text_embedding/text_embedding.py

@@ -141,7 +141,7 @@ class CohereTextEmbeddingModel(TextEmbeddingModel):
             return []
 
         # initialize client
-        client = cohere.Client(credentials.get('api_key'))
+        client = cohere.Client(credentials.get('api_key'), base_url=credentials.get('base_url'))
 
         response = client.tokenize(
             text=text,
@@ -180,7 +180,7 @@ class CohereTextEmbeddingModel(TextEmbeddingModel):
         :return: embeddings and used tokens
         """
         # initialize client
-        client = cohere.Client(credentials.get('api_key'))
+        client = cohere.Client(credentials.get('api_key'), base_url=credentials.get('base_url'))
 
         # call embedding model
         response = client.embed(

api/core/model_runtime/model_providers/deepseek/_assets/icon_l_en.svg

@@ -0,0 +1,22 @@
+<svg width="195.000000" height="41.359375" viewBox="0 0 195 41.3594" fill="none" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
+	<desc>
+			Created with Pixso.
+	</desc>
+	<defs>
+		<clipPath id="clip30_2029">
+			<rect id="_图层_1" width="134.577469" height="25.511124" transform="translate(60.422485 10.022217)" fill="white"/>
+		</clipPath>
+	</defs>
+	<g clip-path="url(#clip30_2029)">
+		<path id="path" d="M119.508 30.113L117.562 30.113L117.562 27.0967L119.508 27.0967C120.713 27.0967 121.931 26.7961 122.715 25.9614C123.5 25.1265 123.796 23.8464 123.796 22.5664C123.796 21.2864 123.512 20.0063 122.715 19.1716C121.919 18.3369 120.713 18.0364 119.508 18.0364C118.302 18.0364 117.085 18.3369 116.3 19.1716C115.515 20.0063 115.219 21.2864 115.219 22.5664L115.219 34.9551L111.806 34.9551L111.806 15.031L115.219 15.031L115.219 16.2998L115.845 16.2998C115.913 16.2219 115.981 16.1553 116.049 16.0884C116.903 15.3093 118.211 15.031 119.496 15.031C121.51 15.031 123.523 15.532 124.843 16.9233C126.162 18.3145 126.629 20.4517 126.629 22.5776C126.629 24.7036 126.151 26.8296 124.843 28.2319C123.535 29.6345 121.51 30.113 119.508 30.113Z" fill-rule="nonzero" fill="#4D6BFE"/>
+		<path id="path" d="M67.5664 15.5654L69.5117 15.5654L69.5117 18.5818L67.5664 18.5818C66.3606 18.5818 65.1434 18.8823 64.3585 19.717C63.5736 20.552 63.2778 21.832 63.2778 23.1121C63.2778 24.3921 63.5623 25.6721 64.3585 26.5068C65.1548 27.3418 66.3606 27.6423 67.5664 27.6423C68.7722 27.6423 69.9895 27.3418 70.7744 26.5068C71.5593 25.6721 71.8551 24.3921 71.8551 23.1121L71.8551 10.7124L75.2677 10.7124L75.2677 30.6475L71.8551 30.6475L71.8551 29.3787L71.2294 29.3787C71.1611 29.4565 71.0929 29.5234 71.0247 29.5901C70.1715 30.3691 68.8633 30.6475 67.5779 30.6475C65.5643 30.6475 63.5509 30.1467 62.2313 28.7554C60.9117 27.364 60.4453 25.2268 60.4453 23.1008C60.4453 20.9749 60.9231 18.8489 62.2313 17.4465C63.5509 16.0552 65.5643 15.5654 67.5664 15.5654Z" fill-rule="nonzero" fill="#4D6BFE"/>
+		<path id="path" d="M92.3881 22.845L92.3881 24.0581L83.299 24.0581L83.299 21.6428L89.328 21.6428C89.1914 20.7634 88.8729 19.9397 88.3042 19.3386C87.4851 18.4705 86.2224 18.1589 84.9711 18.1589C83.7198 18.1589 82.4572 18.4705 81.6381 19.3386C80.819 20.2068 80.5232 21.5315 80.5232 22.845C80.5232 24.1582 80.819 25.4939 81.6381 26.3511C82.4572 27.208 83.7198 27.531 84.9711 27.531C86.2224 27.531 87.4851 27.2192 88.3042 26.3511C88.418 26.2285 88.5203 26.095 88.6227 25.9614L91.9899 25.9614C91.6941 27.0078 91.2277 27.9539 90.5225 28.6885C89.1573 30.1243 87.0529 30.6475 84.9711 30.6475C82.8894 30.6475 80.7849 30.1355 79.4198 28.6885C78.0547 27.2415 77.5542 25.0376 77.5542 22.845C77.5542 20.6521 78.0433 18.437 79.4198 17.0012C80.7963 15.5654 82.8894 15.0422 84.9711 15.0422C87.0529 15.0422 89.1573 15.5542 90.5225 17.0012C91.8988 18.4482 92.3881 20.6521 92.3881 22.845Z" fill-rule="nonzero" fill="#4D6BFE"/>
+		<path id="path" d="M109.52 22.845L109.52 24.0581L100.431 24.0581L100.431 21.6428L106.46 21.6428C106.323 20.7634 106.005 19.9397 105.436 19.3386C104.617 18.4705 103.354 18.1589 102.103 18.1589C100.852 18.1589 99.5889 18.4705 98.7698 19.3386C97.9507 20.2068 97.6549 21.5315 97.6549 22.845C97.6549 24.1582 97.9507 25.4939 98.7698 26.3511C99.5889 27.208 100.852 27.531 102.103 27.531C103.354 27.531 104.617 27.2192 105.436 26.3511C105.55 26.2285 105.652 26.095 105.754 25.9614L109.122 25.9614C108.826 27.0078 108.359 27.9539 107.654 28.6885C106.289 30.1243 104.185 30.6475 102.103 30.6475C100.021 30.6475 97.9166 30.1355 96.5515 28.6885C95.1864 27.2415 94.6859 25.0376 94.6859 22.845C94.6859 20.6521 95.175 18.437 96.5515 17.0012C97.928 15.5654 100.021 15.0422 102.103 15.0422C104.185 15.0422 106.289 15.5542 107.654 17.0012C109.031 18.4482 109.52 20.6521 109.52 22.845Z" fill-rule="nonzero" fill="#4D6BFE"/>
+		<path id="path" d="M136.355 30.6475C138.437 30.6475 140.541 30.3469 141.906 29.49C143.271 28.6328 143.772 27.3306 143.772 26.0393C143.772 24.7483 143.282 23.4348 141.906 22.5889C140.541 21.7429 138.437 21.4312 136.355 21.4312C135.467 21.4312 134.648 21.3088 134.068 20.9861C133.488 20.6521 133.272 20.1511 133.272 19.6504C133.272 19.1494 133.477 18.6375 134.068 18.3147C134.648 17.9807 135.547 17.8694 136.434 17.8694C137.322 17.8694 138.22 17.9919 138.801 18.3147C139.381 18.6487 139.597 19.1494 139.597 19.6504L143.066 19.6504C143.066 18.3591 142.623 17.0457 141.383 16.2C140.143 15.354 138.243 15.0422 136.355 15.0422C134.466 15.0422 132.567 15.3428 131.327 16.2C130.087 17.0569 129.643 18.3591 129.643 19.6504C129.643 20.9414 130.087 22.2549 131.327 23.1008C132.567 23.9468 134.466 24.2585 136.355 24.2585C137.333 24.2585 138.414 24.3809 139.062 24.7036C139.711 25.0266 139.938 25.5386 139.938 26.0393C139.938 26.5403 139.711 27.0522 139.062 27.375C138.414 27.6978 137.424 27.8203 136.446 27.8203C135.467 27.8203 134.466 27.6978 133.829 27.375C133.192 27.0522 132.953 26.5403 132.953 26.0393L128.949 26.0393C128.949 27.3306 129.438 28.644 130.815 29.49C132.191 30.3359 134.273 30.6475 136.355 30.6475Z" fill-rule="nonzero" fill="#4D6BFE"/>
+		<path id="path" d="M160.903 22.845L160.903 24.0581L151.814 24.0581L151.814 21.6428L157.843 21.6428C157.707 20.7634 157.388 19.9397 156.82 19.3386C156 18.4705 154.738 18.1589 153.486 18.1589C152.235 18.1589 150.972 18.4705 150.153 19.3386C149.334 20.2068 149.039 21.5315 149.039 22.845C149.039 24.1582 149.334 25.4939 150.153 26.3511C150.972 27.208 152.235 27.531 153.486 27.531C154.738 27.531 156 27.2192 156.82 26.3511C156.933 26.2285 157.036 26.095 157.138 25.9614L160.505 25.9614C160.209 27.0078 159.743 27.9539 159.038 28.6885C157.673 30.1243 155.568 30.6475 153.486 30.6475C151.405 30.6475 149.3 30.1355 147.935 28.6885C146.57 27.2415 146.07 25.0376 146.07 22.845C146.07 20.6521 146.559 18.437 147.935 17.0012C149.312 15.5654 151.405 15.0422 153.486 15.0422C155.568 15.0422 157.673 15.5542 159.038 17.0012C160.414 18.4482 160.903 20.6521 160.903 22.845Z" fill-rule="nonzero" fill="#4D6BFE"/>
+		<path id="path" d="M178.035 22.845L178.035 24.0581L168.946 24.0581L168.946 21.6428L174.975 21.6428C174.839 20.7634 174.52 19.9397 173.951 19.3386C173.132 18.4705 171.87 18.1589 170.618 18.1589C169.367 18.1589 168.104 18.4705 167.285 19.3386C166.466 20.2068 166.17 21.5315 166.17 22.845C166.17 24.1582 166.466 25.4939 167.285 26.3511C168.104 27.208 169.367 27.531 170.618 27.531C171.87 27.531 173.132 27.2192 173.951 26.3511C174.065 26.2285 174.167 26.095 174.27 25.9614L177.637 25.9614C177.341 27.0078 176.875 27.9539 176.17 28.6885C174.804 30.1243 172.7 30.6475 170.618 30.6475C168.536 30.6475 166.432 30.1355 165.067 28.6885C163.702 27.2415 163.201 25.0376 163.201 22.845C163.201 20.6521 163.69 18.437 165.067 17.0012C166.443 15.5654 168.536 15.0422 170.618 15.0422C172.7 15.0422 174.804 15.5542 176.17 17.0012C177.546 18.4482 178.035 20.6521 178.035 22.845Z" fill-rule="nonzero" fill="#4D6BFE"/>
+		<rect id="rect" x="180.321533" y="10.022217" width="3.412687" height="20.625223" fill="#4D6BFE"/>
+		<path id="polygon" d="M189.559 22.3772L195.155 30.6475L190.935 30.6475L185.338 22.3772L190.935 15.7322L195.155 15.7322L189.559 22.3772Z" fill-rule="nonzero" fill="#4D6BFE"/>
+	</g>
+	<path id="path" d="M55.6128 3.47119C55.0175 3.17944 54.7611 3.73535 54.413 4.01782C54.2939 4.10889 54.1932 4.22729 54.0924 4.33667C53.2223 5.26587 52.2057 5.87646 50.8776 5.80347C48.9359 5.69409 47.2781 6.30469 45.8126 7.78979C45.5012 5.9585 44.4663 4.86499 42.8909 4.16357C42.0667 3.79907 41.2332 3.43457 40.6561 2.64185C40.2532 2.07715 40.1432 1.44849 39.9418 0.828857C39.8135 0.455322 39.6853 0.0725098 39.2548 0.00878906C38.7877 -0.0639648 38.6045 0.327637 38.4213 0.655762C37.6886 1.99512 37.4047 3.47119 37.4321 4.96533C37.4962 8.32739 38.9159 11.0059 41.7369 12.9102C42.0575 13.1289 42.1399 13.3474 42.0392 13.6665C41.8468 14.3225 41.6178 14.9602 41.4164 15.6162C41.2881 16.0354 41.0957 16.1265 40.647 15.9441C39.0991 15.2974 37.7618 14.3406 36.5803 13.1836C34.5745 11.2429 32.761 9.10181 30.4988 7.42529C29.9675 7.03345 29.4363 6.66919 28.8867 6.32275C26.5786 4.08154 29.189 2.24097 29.7935 2.02246C30.4254 1.79468 30.0133 1.01099 27.9708 1.02026C25.9283 1.0293 24.0599 1.71265 21.6786 2.62378C21.3306 2.7605 20.9641 2.8606 20.5886 2.94263C18.4271 2.53271 16.1831 2.44141 13.8384 2.70581C9.42371 3.19775 5.89758 5.28418 3.30554 8.84668C0.191406 13.1289 -0.54126 17.9941 0.356323 23.0691C1.29968 28.4172 4.02905 32.8452 8.22388 36.3076C12.5745 39.8972 17.5845 41.6558 23.2997 41.3186C26.771 41.1182 30.6361 40.6536 34.9958 36.9636C36.0948 37.5103 37.2489 37.7288 39.1632 37.8928C40.6378 38.0295 42.0575 37.8201 43.1565 37.5923C44.8784 37.2278 44.7594 35.6333 44.1366 35.3418C39.09 32.9912 40.1981 33.9478 39.1907 33.1733C41.7552 30.1394 45.6204 26.9868 47.1316 16.7732C47.2506 15.9624 47.1499 15.4521 47.1316 14.7961C47.1224 14.3953 47.214 14.2405 47.672 14.1948C48.9359 14.0491 50.1632 13.7029 51.2898 13.0833C54.5596 11.2976 55.8784 8.36377 56.1898 4.84692C56.2357 4.30933 56.1807 3.75342 55.6128 3.47119ZM27.119 35.123C22.2281 31.2783 19.856 30.0117 18.8759 30.0664C17.96 30.1211 18.1249 31.1689 18.3263 31.8523C18.537 32.5264 18.8118 32.9912 19.1964 33.5833C19.462 33.9751 19.6453 34.5581 18.9309 34.9956C17.3555 35.9705 14.6169 34.6675 14.4886 34.6038C11.3014 32.7268 8.63611 30.2485 6.75842 26.8594C4.94495 23.5974 3.89172 20.0989 3.71765 16.3633C3.67188 15.4614 3.9375 15.1423 4.83508 14.9785C6.0166 14.7598 7.23474 14.7141 8.41626 14.8872C13.408 15.6162 17.6577 17.8484 21.2206 21.3835C23.2539 23.397 24.7926 25.8025 26.3772 28.1531C28.0624 30.6494 29.8759 33.0276 32.184 34.9773C32.9991 35.6606 33.6494 36.1799 34.2722 36.5627C32.3947 36.7722 29.2622 36.8179 27.119 35.123ZM29.4637 20.0442C29.4637 19.6433 29.7843 19.3245 30.1874 19.3245C30.2789 19.3245 30.3613 19.3425 30.4346 19.3699C30.5354 19.4065 30.627 19.4612 30.7002 19.543C30.8285 19.6707 30.9017 19.8528 30.9017 20.0442C30.9017 20.4451 30.5812 20.7639 30.1782 20.7639C29.7751 20.7639 29.4637 20.4451 29.4637 20.0442ZM36.7452 23.7798C36.2781 23.9712 35.811 24.135 35.3622 24.1533C34.6661 24.1897 33.9059 23.9072 33.4938 23.561C32.8527 23.0234 32.3947 22.7229 32.2023 21.7844C32.1199 21.3835 32.1656 20.7639 32.239 20.4087C32.4038 19.6433 32.2206 19.1514 31.6803 18.7048C31.2406 18.3403 30.6819 18.2402 30.0682 18.2402C29.8392 18.2402 29.6287 18.1399 29.4729 18.0579C29.2164 17.9304 29.0059 17.6116 29.2073 17.2197C29.2714 17.0923 29.5829 16.7825 29.6561 16.7278C30.4896 16.2539 31.4513 16.4089 32.3397 16.7642C33.1641 17.1013 33.7869 17.7209 34.6844 18.5955C35.6003 19.6523 35.7651 19.9441 36.2872 20.7366C36.6995 21.3562 37.075 21.9939 37.3314 22.7229C37.4871 23.1785 37.2856 23.552 36.7452 23.7798Z" fill-rule="nonzero" fill="#4D6BFE"/>
+</svg>

api/core/model_runtime/model_providers/deepseek/_assets/icon_s_en.svg

@@ -0,0 +1,3 @@
+<svg width="60" height="50" viewBox="0 0 60 50" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xml:space="preserve" xmlns:serif="http://www.serif.com/" style="fill-rule:evenodd;clip-rule:evenodd;stroke-linejoin:round;stroke-miterlimit:2;">
+    <path id="path" d="M55.613,3.471C55.018,3.179 54.761,3.735 54.413,4.018C54.294,4.109 54.193,4.227 54.092,4.337C53.222,5.266 52.206,5.876 50.878,5.803C48.936,5.694 47.278,6.305 45.813,7.79C45.501,5.959 44.466,4.865 42.891,4.164C42.067,3.799 41.233,3.435 40.656,2.642C40.253,2.077 40.143,1.448 39.942,0.829C39.814,0.455 39.685,0.073 39.255,0.009C38.788,-0.064 38.605,0.328 38.421,0.656C37.689,1.995 37.405,3.471 37.432,4.965C37.496,8.327 38.916,11.006 41.737,12.91C42.058,13.129 42.14,13.347 42.039,13.666C41.847,14.323 41.618,14.96 41.416,15.616C41.288,16.035 41.096,16.127 40.647,15.944C39.099,15.297 37.762,14.341 36.58,13.184C34.575,11.243 32.761,9.102 30.499,7.425C29.968,7.033 29.436,6.669 28.887,6.323C26.579,4.082 29.189,2.241 29.794,2.022C30.425,1.795 30.013,1.011 27.971,1.02C25.928,1.029 24.06,1.713 21.679,2.624C21.331,2.761 20.964,2.861 20.589,2.943C18.427,2.533 16.183,2.441 13.838,2.706C9.424,3.198 5.898,5.284 3.306,8.847C0.191,13.129 -0.541,17.994 0.356,23.069C1.3,28.417 4.029,32.845 8.224,36.308C12.575,39.897 17.584,41.656 23.3,41.319C26.771,41.118 30.636,40.654 34.996,36.964C36.095,37.51 37.249,37.729 39.163,37.893C40.638,38.03 42.058,37.82 43.157,37.592C44.878,37.228 44.759,35.633 44.137,35.342C39.09,32.991 40.198,33.948 39.191,33.173C41.755,30.139 45.62,26.987 47.132,16.773C47.251,15.962 47.15,15.452 47.132,14.796C47.122,14.395 47.214,14.241 47.672,14.195C48.936,14.049 50.163,13.703 51.29,13.083C54.56,11.298 55.878,8.364 56.19,4.847C56.236,4.309 56.181,3.753 55.613,3.471ZM27.119,35.123C22.228,31.278 19.856,30.012 18.876,30.066C17.96,30.121 18.125,31.169 18.326,31.852C18.537,32.526 18.812,32.991 19.196,33.583C19.462,33.975 19.645,34.558 18.931,34.996C17.356,35.971 14.617,34.667 14.489,34.604C11.301,32.727 8.636,30.249 6.758,26.859C4.945,23.597 3.892,20.099 3.718,16.363C3.672,15.461 3.938,15.142 4.835,14.979C6.017,14.76 7.235,14.714 8.416,14.887C13.408,15.616 17.658,17.848 21.221,21.384C23.254,23.397 24.793,25.802 26.377,28.153C28.062,30.649 29.876,33.028 32.184,34.977C32.999,35.661 33.649,36.18 34.272,36.563C32.395,36.772 29.262,36.818 27.119,35.123ZM29.464,20.044C29.464,19.643 29.784,19.325 30.187,19.325C30.279,19.325 30.361,19.343 30.435,19.37C30.535,19.407 30.627,19.461 30.7,19.543C30.828,19.671 30.902,19.853 30.902,20.044C30.902,20.445 30.581,20.764 30.178,20.764C29.775,20.764 29.464,20.445 29.464,20.044ZM36.745,23.78C36.278,23.971 35.811,24.135 35.362,24.153C34.666,24.19 33.906,23.907 33.494,23.561C32.853,23.023 32.395,22.723 32.202,21.784C32.12,21.384 32.166,20.764 32.239,20.409C32.404,19.643 32.221,19.151 31.68,18.705C31.241,18.34 30.682,18.24 30.068,18.24C29.839,18.24 29.629,18.14 29.473,18.058C29.216,17.93 29.006,17.612 29.207,17.22C29.271,17.092 29.583,16.783 29.656,16.728C30.49,16.254 31.451,16.409 32.34,16.764C33.164,17.101 33.787,17.721 34.684,18.596C35.6,19.652 35.765,19.944 36.287,20.737C36.7,21.356 37.075,21.994 37.331,22.723C37.487,23.179 37.286,23.552 36.745,23.78Z" style="fill:rgb(77,107,254);fill-rule:nonzero;"/>
+</svg>

api/core/model_runtime/model_providers/deepseek/deepseek.py

@@ -0,0 +1,33 @@
+import logging
+
+from core.model_runtime.entities.model_entities import ModelType
+from core.model_runtime.errors.validate import CredentialsValidateFailedError
+from core.model_runtime.model_providers.__base.model_provider import ModelProvider
+
+logger = logging.getLogger(__name__)
+
+
+
+class DeepSeekProvider(ModelProvider):
+
+    def validate_provider_credentials(self, credentials: dict) -> None:
+        """
+        Validate provider credentials
+        if validate failed, raise exception
+
+        :param credentials: provider credentials, credentials form defined in `provider_credential_schema`.
+        """
+        try:
+            model_instance = self.get_model_instance(ModelType.LLM)
+
+            # Use `deepseek-chat` model for validate,
+            # no matter what model you pass in, text completion model or chat model
+            model_instance.validate_credentials(
+                model='deepseek-chat',
+                credentials=credentials
+            )
+        except CredentialsValidateFailedError as ex:
+            raise ex
+        except Exception as ex:
+            logger.exception(f'{self.get_provider_schema().provider} credentials validate failed')
+            raise ex

api/core/model_runtime/model_providers/deepseek/deepseek.yaml

@@ -0,0 +1,41 @@
+provider: deepseek
+label:
+  en_US: deepseek
+  zh_Hans: 深度求索
+description:
+  en_US: Models provided by deepseek, such as deepseek-chat、deepseek-coder.
+  zh_Hans: 深度求索提供的模型，例如 deepseek-chat、deepseek-coder 。
+icon_small:
+  en_US: icon_s_en.svg
+icon_large:
+  en_US: icon_l_en.svg
+background: "#c0cdff"
+help:
+  title:
+    en_US: Get your API Key from deepseek
+    zh_Hans: 从深度求索获取 API Key
+  url:
+    en_US: https://platform.deepseek.com/api_keys
+supported_model_types:
+  - llm
+configurate_methods:
+  - predefined-model
+provider_credential_schema:
+  credential_form_schemas:
+    - variable: api_key
+      label:
+        en_US: API Key
+      type: secret-input
+      required: true
+      placeholder:
+        zh_Hans: 在此输入您的 API Key
+        en_US: Enter your API Key
+    - variable: endpoint_url
+      label:
+        zh_Hans: 自定义 API endpoint 地址
+        en_US: Custom API endpoint URL
+      type: text-input
+      required: false
+      placeholder:
+        zh_Hans: Base URL, e.g. https://api.deepseek.com/v1 or https://api.deepseek.com
+        en_US: Base URL, e.g. https://api.deepseek.com/v1 or https://api.deepseek.com

api/core/model_runtime/model_providers/deepseek/llm/_position.yaml

@@ -0,0 +1,2 @@
+- deepseek-chat
+- deepseek-coder

api/core/model_runtime/model_providers/deepseek/llm/deepseek-chat.yaml

@@ -0,0 +1,64 @@
+model: deepseek-chat
+label:
+  zh_Hans: deepseek-chat
+  en_US: deepseek-chat
+model_type: llm
+features:
+  - agent-thought
+model_properties:
+  mode: chat
+  context_size: 32000
+parameter_rules:
+  - name: temperature
+    use_template: temperature
+    type: float
+    default: 1
+    min: 0.0
+    max: 2.0
+    help:
+      zh_Hans: 控制生成结果的多样性和随机性。数值越小，越严谨；数值越大，越发散。
+      en_US: Control the diversity and randomness of generated results. The smaller the value, the more rigorous it is; the larger the value, the more divergent it is.
+  - name: max_tokens
+    use_template: max_tokens
+    type: int
+    default: 4096
+    min: 1
+    max: 32000
+    help:
+      zh_Hans: 指定生成结果长度的上限。如果生成结果截断，可以调大该参数。
+      en_US: Specifies the upper limit on the length of generated results. If the generated results are truncated, you can increase this parameter.
+  - name: top_p
+    use_template: top_p
+    type: float
+    default: 1
+    min: 0.01
+    max: 1.00
+    help:
+      zh_Hans: 控制生成结果的随机性。数值越小，随机性越弱；数值越大，随机性越强。一般而言，top_p 和 temperature 两个参数选择一个进行调整即可。
+      en_US: Control the randomness of generated results. The smaller the value, the weaker the randomness; the larger the value, the stronger the randomness. Generally speaking, you can adjust one of the two parameters top_p and temperature.
+  - name: logprobs
+    help:
+      zh_Hans: 是否返回所输出 token 的对数概率。如果为 true，则在 message 的 content 中返回每个输出 token 的对数概率。
+      en_US: Whether to return the log probability of the output token. If true, returns the log probability of each output token in the content of message .
+    type: boolean
+  - name: top_logprobs
+    type: int
+    default: 0
+    min: 0
+    max: 20
+    help:
+      zh_Hans: 一个介于 0 到 20 之间的整数 N，指定每个输出位置返回输出概率 top N 的 token，且返回这些 token 的对数概率。指定此参数时，logprobs 必须为 true。
+      en_US: An integer N between 0 and 20, specifying that each output position returns the top N tokens with output probability, and returns the logarithmic probability of these tokens. When specifying this parameter, logprobs must be true.
+  - name: frequency_penalty
+    use_template: frequency_penalty
+    default: 0
+    min: -2.0
+    max: 2.0
+    help:
+      zh_Hans: 介于 -2.0 和 2.0 之间的数字。如果该值为正，那么新 token 会根据其在已有文本中的出现频率受到相应的惩罚，降低模型重复相同内容的可能性。
+      en_US: A number between -2.0 and 2.0. If the value is positive, new tokens are penalized based on their frequency of occurrence in existing text, reducing the likelihood that the model will repeat the same content.
+pricing:
+  input: '1'
+  output: '2'
+  unit: '0.000001'
+  currency: RMB

api/core/model_runtime/model_providers/deepseek/llm/deepseek-coder.yaml

@@ -0,0 +1,26 @@
+model: deepseek-coder
+label:
+  zh_Hans: deepseek-coder
+  en_US: deepseek-coder
+model_type: llm
+features:
+  - agent-thought
+model_properties:
+  mode: chat
+  context_size: 16000
+parameter_rules:
+  - name: temperature
+    use_template: temperature
+    min: 0
+    max: 1
+    default: 0.5
+  - name: top_p
+    use_template: top_p
+    min: 0
+    max: 1
+    default: 1
+  - name: max_tokens
+    use_template: max_tokens
+    min: 1
+    max: 32000
+    default: 1024

api/core/model_runtime/model_providers/deepseek/llm/llm.py

@@ -0,0 +1,113 @@
+from collections.abc import Generator
+from typing import Optional, Union
+from urllib.parse import urlparse
+
+import tiktoken
+
+from core.model_runtime.entities.llm_entities import LLMResult
+from core.model_runtime.entities.message_entities import (
+    PromptMessage,
+    PromptMessageTool,
+)
+from core.model_runtime.model_providers.openai.llm.llm import OpenAILargeLanguageModel
+
+
+class DeepSeekLargeLanguageModel(OpenAILargeLanguageModel):
+
+    def _invoke(self, model: str, credentials: dict,
+                prompt_messages: list[PromptMessage], model_parameters: dict,
+                tools: Optional[list[PromptMessageTool]] = None, stop: Optional[list[str]] = None,
+                stream: bool = True, user: Optional[str] = None) \
+            -> Union[LLMResult, Generator]:
+        self._add_custom_parameters(credentials)
+
+        return super()._invoke(model, credentials, prompt_messages, model_parameters, tools, stop, stream, user)
+
+    def validate_credentials(self, model: str, credentials: dict) -> None:
+        self._add_custom_parameters(credentials)
+        super().validate_credentials(model, credentials)
+
+
+    # refactored from openai model runtime, use cl100k_base for calculate token number
+    def _num_tokens_from_string(self, model: str, text: str,
+                                tools: Optional[list[PromptMessageTool]] = None) -> int:
+        """
+        Calculate num tokens for text completion model with tiktoken package.
+
+        :param model: model name
+        :param text: prompt text
+        :param tools: tools for tool calling
+        :return: number of tokens
+        """
+        encoding = tiktoken.get_encoding("cl100k_base")
+        num_tokens = len(encoding.encode(text))
+
+        if tools:
+            num_tokens += self._num_tokens_for_tools(encoding, tools)
+
+        return num_tokens
+
+    # refactored from openai model runtime, use cl100k_base for calculate token number
+    def _num_tokens_from_messages(self, model: str, messages: list[PromptMessage],
+                                  tools: Optional[list[PromptMessageTool]] = None) -> int:
+        """Calculate num tokens for gpt-3.5-turbo and gpt-4 with tiktoken package.
+
+        Official documentation: https://github.com/openai/openai-cookbook/blob/
+        main/examples/How_to_format_inputs_to_ChatGPT_models.ipynb"""
+        encoding = tiktoken.get_encoding("cl100k_base")
+        tokens_per_message = 3
+        tokens_per_name = 1
+
+        num_tokens = 0
+        messages_dict = [self._convert_prompt_message_to_dict(m) for m in messages]
+        for message in messages_dict:
+            num_tokens += tokens_per_message
+            for key, value in message.items():
+                # Cast str(value) in case the message value is not a string
+                # This occurs with function messages
+                # TODO: The current token calculation method for the image type is not implemented,
+                #  which need to download the image and then get the resolution for calculation,
+                #  and will increase the request delay
+                if isinstance(value, list):
+                    text = ''
+                    for item in value:
+                        if isinstance(item, dict) and item['type'] == 'text':
+                            text += item['text']
+
+                    value = text
+
+                if key == "tool_calls":
+                    for tool_call in value:
+                        for t_key, t_value in tool_call.items():
+                            num_tokens += len(encoding.encode(t_key))
+                            if t_key == "function":
+                                for f_key, f_value in t_value.items():
+                                    num_tokens += len(encoding.encode(f_key))
+                                    num_tokens += len(encoding.encode(f_value))
+                            else:
+                                num_tokens += len(encoding.encode(t_key))
+                                num_tokens += len(encoding.encode(t_value))
+                else:
+                    num_tokens += len(encoding.encode(str(value)))
+
+                if key == "name":
+                    num_tokens += tokens_per_name
+
+        # every reply is primed with <im_start>assistant
+        num_tokens += 3
+
+        if tools:
+            num_tokens += self._num_tokens_for_tools(encoding, tools)
+
+        return num_tokens
+
+    @staticmethod
+    def _add_custom_parameters(credentials: dict) -> None:
+        credentials['mode'] = 'chat'
+        credentials['openai_api_key']=credentials['api_key']
+        if 'endpoint_url' not in credentials or credentials['endpoint_url'] == "":
+            credentials['openai_api_base']='https://api.deepseek.com'
+        else:
+            parsed_url = urlparse(credentials['endpoint_url'])
+            credentials['openai_api_base']=f"{parsed_url.scheme}://{parsed_url.netloc}"
+

api/core/model_runtime/model_providers/jina/jina.yaml

@@ -19,6 +19,7 @@ supported_model_types:
   - rerank
 configurate_methods:
   - predefined-model
+  - customizable-model
 provider_credential_schema:
   credential_form_schemas:
     - variable: api_key
@@ -29,3 +30,40 @@ provider_credential_schema:
       placeholder:
         zh_Hans: 在此输入您的 API Key
         en_US: Enter your API Key
+model_credential_schema:
+  model:
+    label:
+      en_US: Model Name
+      zh_Hans: 模型名称
+    placeholder:
+      en_US: Enter your model name
+      zh_Hans: 输入模型名称
+  credential_form_schemas:
+    - variable: api_key
+      label:
+        en_US: API Key
+      type: secret-input
+      required: true
+      placeholder:
+        zh_Hans: 在此输入您的 API Key
+        en_US: Enter your API Key
+    - variable: base_url
+      label:
+        zh_Hans: 服务器 URL
+        en_US: Base URL
+      type: text-input
+      required: true
+      placeholder:
+        zh_Hans: Base URL, e.g. https://api.jina.ai/v1
+        en_US: Base URL, e.g. https://api.jina.ai/v1
+      default: 'https://api.jina.ai/v1'
+    - variable: context_size
+      label:
+        zh_Hans: 上下文大小
+        en_US: Context size
+      placeholder:
+        zh_Hans: 输入上下文大小
+        en_US: Enter context size
+      required: false
+      type: text-input
+      default: '8192'

api/core/model_runtime/model_providers/jina/rerank/rerank.py

@@ -2,6 +2,8 @@ from typing import Optional
 
 import httpx
 
+from core.model_runtime.entities.common_entities import I18nObject
+from core.model_runtime.entities.model_entities import AIModelEntity, FetchFrom, ModelPropertyKey, ModelType
 from core.model_runtime.entities.rerank_entities import RerankDocument, RerankResult
 from core.model_runtime.errors.invoke import (
     InvokeAuthorizationError,
@@ -38,9 +40,13 @@ class JinaRerankModel(RerankModel):
         if len(docs) == 0:
             return RerankResult(model=model, docs=[])
 
+        base_url = credentials.get('base_url', 'https://api.jina.ai/v1')
+        if base_url.endswith('/'):
+            base_url = base_url[:-1]
+
         try:
             response = httpx.post(
-                "https://api.jina.ai/v1/rerank",
+                base_url + '/rerank',
                 json={
                     "model": model,
                     "query": query,
@@ -103,3 +109,19 @@ class JinaRerankModel(RerankModel):
             InvokeAuthorizationError: [httpx.HTTPStatusError],  
             InvokeBadRequestError: [httpx.RequestError]
         }
+
+    def get_customizable_model_schema(self, model: str, credentials: dict) -> AIModelEntity:
+        """
+            generate custom model entities from credentials
+        """
+        entity = AIModelEntity(
+            model=model,
+            label=I18nObject(en_US=model),
+            model_type=ModelType.RERANK,
+            fetch_from=FetchFrom.CUSTOMIZABLE_MODEL,
+            model_properties={
+                ModelPropertyKey.CONTEXT_SIZE: int(credentials.get('context_size'))
+            }
+        )
+
+        return entity

api/core/model_runtime/model_providers/jina/text_embedding/text_embedding.py

@@ -4,7 +4,8 @@ from typing import Optional
 
 from requests import post
 
-from core.model_runtime.entities.model_entities import PriceType
+from core.model_runtime.entities.common_entities import I18nObject
+from core.model_runtime.entities.model_entities import AIModelEntity, FetchFrom, ModelPropertyKey, ModelType, PriceType
 from core.model_runtime.entities.text_embedding_entities import EmbeddingUsage, TextEmbeddingResult
 from core.model_runtime.errors.invoke import (
     InvokeAuthorizationError,
@@ -23,8 +24,7 @@ class JinaTextEmbeddingModel(TextEmbeddingModel):
     """
     Model class for Jina text embedding model.
     """
-    api_base: str = 'https://api.jina.ai/v1/embeddings'
-    models: list[str] = ['jina-embeddings-v2-base-en', 'jina-embeddings-v2-small-en', 'jina-embeddings-v2-base-zh', 'jina-embeddings-v2-base-de']
+    api_base: str = 'https://api.jina.ai/v1'
 
     def _invoke(self, model: str, credentials: dict,
                 texts: list[str], user: Optional[str] = None) \
@@ -39,11 +39,14 @@ class JinaTextEmbeddingModel(TextEmbeddingModel):
         :return: embeddings result
         """
         api_key = credentials['api_key']
-        if model not in self.models:
-            raise InvokeBadRequestError('Invalid model name')
         if not api_key:
             raise CredentialsValidateFailedError('api_key is required')
-        url = self.api_base
+
+        base_url = credentials.get('base_url', self.api_base)
+        if base_url.endswith('/'):
+            base_url = base_url[:-1]
+
+        url = base_url + '/embeddings'
         headers = {
             'Authorization': 'Bearer ' + api_key,
             'Content-Type': 'application/json'
@@ -70,7 +73,7 @@ class JinaTextEmbeddingModel(TextEmbeddingModel):
                 elif response.status_code == 500:
                     raise InvokeServerUnavailableError(msg)
                 else:
-                    raise InvokeError(msg)
+                    raise InvokeBadRequestError(msg)
             except JSONDecodeError as e:
                 raise InvokeServerUnavailableError(f"Failed to convert response to json: {e} with text: {response.text}")
 
@@ -118,8 +121,8 @@ class JinaTextEmbeddingModel(TextEmbeddingModel):
         """
         try:
             self._invoke(model=model, credentials=credentials, texts=['ping'])
-        except InvokeAuthorizationError:
-            raise CredentialsValidateFailedError('Invalid api key')
+        except Exception as e:
+            raise CredentialsValidateFailedError(f'Credentials validation failed: {e}')
 
     @property
     def _invoke_error_mapping(self) -> dict[type[InvokeError], list[type[Exception]]]:
@@ -137,7 +140,8 @@ class JinaTextEmbeddingModel(TextEmbeddingModel):
                 InvokeAuthorizationError
             ],
             InvokeBadRequestError: [
-                KeyError
+                KeyError,
+                InvokeBadRequestError
             ]
         }
     
@@ -170,3 +174,19 @@ class JinaTextEmbeddingModel(TextEmbeddingModel):
         )
 
         return usage
+
+    def get_customizable_model_schema(self, model: str, credentials: dict) -> AIModelEntity:
+        """
+            generate custom model entities from credentials
+        """
+        entity = AIModelEntity(
+            model=model,
+            label=I18nObject(en_US=model),
+            model_type=ModelType.TEXT_EMBEDDING,
+            fetch_from=FetchFrom.CUSTOMIZABLE_MODEL,
+            model_properties={
+                ModelPropertyKey.CONTEXT_SIZE: int(credentials.get('context_size'))
+            }
+        )
+
+        return entity

api/core/model_runtime/model_providers/leptonai/leptonai.py

@@ -0,0 +1,29 @@
+import logging
+
+from core.model_runtime.entities.model_entities import ModelType
+from core.model_runtime.errors.validate import CredentialsValidateFailedError
+from core.model_runtime.model_providers.__base.model_provider import ModelProvider
+
+logger = logging.getLogger(__name__)
+
+class LeptonAIProvider(ModelProvider):
+
+    def validate_provider_credentials(self, credentials: dict) -> None:
+        """
+        Validate provider credentials
+        if validate failed, raise exception
+
+        :param credentials: provider credentials, credentials form defined in `provider_credential_schema`.
+        """
+        try:
+            model_instance = self.get_model_instance(ModelType.LLM)
+
+            model_instance.validate_credentials(
+                model='llama2-7b',
+                credentials=credentials
+            )
+        except CredentialsValidateFailedError as ex:
+            raise ex
+        except Exception as ex:
+            logger.exception(f'{self.get_provider_schema().provider} credentials validate failed')
+            raise ex

api/core/model_runtime/model_providers/leptonai/leptonai.yaml

@@ -0,0 +1,29 @@
+provider: leptonai
+label:
+  zh_Hans: Lepton AI
+  en_US: Lepton AI
+icon_small:
+  en_US: icon_s_en.png
+icon_large:
+  en_US: icon_l_en.png
+background: "#F5F5F4"
+help:
+  title:
+    en_US: Get your API Key from Lepton AI
+    zh_Hans: 从 Lepton AI 获取 API Key
+  url:
+    en_US: https://dashboard.lepton.ai
+supported_model_types:
+  - llm
+configurate_methods:
+  - predefined-model
+provider_credential_schema:
+  credential_form_schemas:
+    - variable: api_key
+      label:
+        en_US: API Key
+      type: secret-input
+      required: true
+      placeholder:
+        zh_Hans: 在此输入您的 API Key
+        en_US: Enter your API Key

api/core/model_runtime/model_providers/leptonai/llm/_position.yaml

@@ -0,0 +1,6 @@
+- gemma-7b
+- mistral-7b
+- mixtral-8x7b
+- llama2-7b
+- llama2-13b
+- llama3-70b

api/core/model_runtime/model_providers/leptonai/llm/gemma-7b.yaml

@@ -0,0 +1,20 @@
+model: gemma-7b
+label:
+  zh_Hans: gemma-7b
+  en_US: gemma-7b
+model_type: llm
+features:
+  - agent-thought
+model_properties:
+  mode: chat
+  context_size: 8192
+parameter_rules:
+  - name: temperature
+    use_template: temperature
+  - name: top_p
+    use_template: top_p
+  - name: max_tokens
+    use_template: max_tokens
+    default: 1024
+    min: 1
+    max: 1024

api/core/model_runtime/model_providers/leptonai/llm/llama2-13b.yaml

@@ -0,0 +1,20 @@
+model: llama2-13b
+label:
+  zh_Hans: llama2-13b
+  en_US: llama2-13b
+model_type: llm
+features:
+  - agent-thought
+model_properties:
+  mode: chat
+  context_size: 4096
+parameter_rules:
+  - name: temperature
+    use_template: temperature
+  - name: top_p
+    use_template: top_p
+  - name: max_tokens
+    use_template: max_tokens
+    default: 1024
+    min: 1
+    max: 1024

api/core/model_runtime/model_providers/leptonai/llm/llama2-7b.yaml

@@ -0,0 +1,20 @@
+model: llama2-7b
+label:
+  zh_Hans: llama2-7b
+  en_US: llama2-7b
+model_type: llm
+features:
+  - agent-thought
+model_properties:
+  mode: chat
+  context_size: 4096
+parameter_rules:
+  - name: temperature
+    use_template: temperature
+  - name: top_p
+    use_template: top_p
+  - name: max_tokens
+    use_template: max_tokens
+    default: 1024
+    min: 1
+    max: 1024

api/core/model_runtime/model_providers/leptonai/llm/llama3-70b.yaml

@@ -0,0 +1,20 @@
+model: llama3-70b
+label:
+  zh_Hans: llama3-70b
+  en_US: llama3-70b
+model_type: llm
+features:
+  - agent-thought
+model_properties:
+  mode: chat
+  context_size: 8192
+parameter_rules:
+  - name: temperature
+    use_template: temperature
+  - name: top_p
+    use_template: top_p
+  - name: max_tokens
+    use_template: max_tokens
+    default: 1024
+    min: 1
+    max: 1024

api/core/model_runtime/model_providers/leptonai/llm/llm.py

@@ -0,0 +1,34 @@
+from collections.abc import Generator
+from typing import Optional, Union
+
+from core.model_runtime.entities.llm_entities import LLMResult
+from core.model_runtime.entities.message_entities import PromptMessage, PromptMessageTool
+from core.model_runtime.model_providers.openai_api_compatible.llm.llm import OAIAPICompatLargeLanguageModel
+
+
+class LeptonAILargeLanguageModel(OAIAPICompatLargeLanguageModel):
+    MODEL_PREFIX_MAP = {
+        'llama2-7b': 'llama2-7b',
+        'gemma-7b': 'gemma-7b',
+        'mistral-7b': 'mistral-7b',
+        'mixtral-8x7b': 'mixtral-8x7b',
+        'llama3-70b': 'llama3-70b',
+        'llama2-13b': 'llama2-13b',
+    } 
+    def _invoke(self, model: str, credentials: dict,
+                prompt_messages: list[PromptMessage], model_parameters: dict,
+                tools: Optional[list[PromptMessageTool]] = None, stop: Optional[list[str]] = None,
+                stream: bool = True, user: Optional[str] = None) \
+            -> Union[LLMResult, Generator]:
+        self._add_custom_parameters(credentials, model)
+        return super()._invoke(model, credentials, prompt_messages, model_parameters, tools, stop, stream)
+
+    def validate_credentials(self, model: str, credentials: dict) -> None:
+        self._add_custom_parameters(credentials, model)
+        super().validate_credentials(model, credentials)
+
+    @classmethod
+    def _add_custom_parameters(cls, credentials: dict, model: str) -> None:
+        credentials['mode'] = 'chat'
+        credentials['endpoint_url'] = f'https://{cls.MODEL_PREFIX_MAP[model]}.lepton.run/api/v1'
+        

api/core/model_runtime/model_providers/leptonai/llm/mistral-7b.yaml

@@ -0,0 +1,20 @@
+model: mistral-7b
+label:
+  zh_Hans: mistral-7b
+  en_US: mistral-7b
+model_type: llm
+features:
+  - agent-thought
+model_properties:
+  mode: chat
+  context_size: 8192
+parameter_rules:
+  - name: temperature
+    use_template: temperature
+  - name: top_p
+    use_template: top_p
+  - name: max_tokens
+    use_template: max_tokens
+    default: 1024
+    min: 1
+    max: 1024

api/core/model_runtime/model_providers/leptonai/llm/mixtral-8x7b.yaml

@@ -0,0 +1,20 @@
+model: mixtral-8x7b
+label:
+  zh_Hans: mixtral-8x7b
+  en_US: mixtral-8x7b
+model_type: llm
+features:
+  - agent-thought
+model_properties:
+  mode: chat
+  context_size: 32000
+parameter_rules:
+  - name: temperature
+    use_template: temperature
+  - name: top_p
+    use_template: top_p
+  - name: max_tokens
+    use_template: max_tokens
+    default: 1024
+    min: 1
+    max: 1024

api/core/model_runtime/model_providers/localai/localai.yaml

@@ -15,6 +15,7 @@ help:
 supported_model_types:
   - llm
   - text-embedding
+  - speech2text
 configurate_methods:
   - customizable-model
 model_credential_schema:
@@ -57,6 +58,9 @@ model_credential_schema:
         zh_Hans: 在此输入LocalAI的服务器地址，如 http://192.168.1.100:8080
         en_US: Enter the url of your LocalAI, e.g. http://192.168.1.100:8080
     - variable: context_size
+      show_on:
+        - variable: __model_type
+          value: llm
       label:
         zh_Hans: 上下文大小
         en_US: Context size

api/core/model_runtime/model_providers/localai/speech2text/speech2text.py

@@ -0,0 +1,101 @@
+from typing import IO, Optional
+
+from requests import Request, Session
+from yarl import URL
+
+from core.model_runtime.entities.common_entities import I18nObject
+from core.model_runtime.entities.model_entities import AIModelEntity, FetchFrom, ModelType
+from core.model_runtime.errors.invoke import (
+    InvokeAuthorizationError,
+    InvokeBadRequestError,
+    InvokeConnectionError,
+    InvokeError,
+    InvokeRateLimitError,
+    InvokeServerUnavailableError,
+)
+from core.model_runtime.errors.validate import CredentialsValidateFailedError
+from core.model_runtime.model_providers.__base.speech2text_model import Speech2TextModel
+
+
+class LocalAISpeech2text(Speech2TextModel):
+    """
+    Model class for Local AI Text to speech model.
+    """
+
+    def _invoke(self, model: str, credentials: dict, file: IO[bytes], user: Optional[str] = None) -> str:
+        """
+        Invoke large language model
+
+        :param model: model name
+        :param credentials: model credentials
+        :param file: audio file
+        :param user: unique user id
+        :return: text for given audio file
+        """
+        
+        url = str(URL(credentials['server_url']) / "v1/audio/transcriptions")
+        data = {"model": model}
+        files = {"file": file}
+
+        session = Session()
+        request = Request("POST", url, data=data, files=files)
+        prepared_request = session.prepare_request(request)
+        response = session.send(prepared_request)
+
+        if 'error' in response.json():
+            raise InvokeServerUnavailableError("Empty response")
+
+        return response.json()["text"]
+
+    def validate_credentials(self, model: str, credentials: dict) -> None:
+        """
+        Validate model credentials
+
+        :param model: model name
+        :param credentials: model credentials
+        :return:
+        """
+        try:
+            audio_file_path = self._get_demo_file_path()
+
+            with open(audio_file_path, 'rb') as audio_file:
+                self._invoke(model, credentials, audio_file)
+        except Exception as ex:
+            raise CredentialsValidateFailedError(str(ex))
+
+    @property
+    def _invoke_error_mapping(self) -> dict[type[InvokeError], list[type[Exception]]]:
+        return {
+            InvokeConnectionError: [
+                InvokeConnectionError
+            ],
+            InvokeServerUnavailableError: [
+                InvokeServerUnavailableError
+            ],
+            InvokeRateLimitError: [
+                InvokeRateLimitError
+            ],
+            InvokeAuthorizationError: [
+                InvokeAuthorizationError
+            ],
+            InvokeBadRequestError: [
+                InvokeBadRequestError
+            ],
+        }
+
+    def get_customizable_model_schema(self, model: str, credentials: dict) -> AIModelEntity | None:
+        """
+            used to define customizable model schema
+        """
+        entity = AIModelEntity(
+            model=model,
+            label=I18nObject(
+                en_US=model
+            ),
+            fetch_from=FetchFrom.CUSTOMIZABLE_MODEL,
+            model_type=ModelType.SPEECH2TEXT,
+            model_properties={},
+            parameter_rules=[]
+        )
+
+        return entity

api/core/model_runtime/model_providers/minimax/llm/llm.py

@@ -34,6 +34,8 @@ from core.model_runtime.model_providers.minimax.llm.types import MinimaxMessage
 
 class MinimaxLargeLanguageModel(LargeLanguageModel):
     model_apis = {
+        'abab6.5s-chat': MinimaxChatCompletionPro,
+        'abab6.5-chat': MinimaxChatCompletionPro,
         'abab6-chat': MinimaxChatCompletionPro,
         'abab5.5s-chat': MinimaxChatCompletionPro,
         'abab5.5-chat': MinimaxChatCompletionPro,

api/core/model_runtime/model_providers/openai/llm/llm.py

@@ -378,6 +378,11 @@ class OpenAILargeLanguageModel(_CommonOpenAI, LargeLanguageModel):
         if user:
             extra_model_kwargs['user'] = user
 
+        if stream:
+            extra_model_kwargs['stream_options'] = {
+                "include_usage": True
+            }
+        
         # text completion model
         response = client.completions.create(
             prompt=prompt_messages[0].content,
@@ -446,8 +451,24 @@ class OpenAILargeLanguageModel(_CommonOpenAI, LargeLanguageModel):
         :return: llm response chunk generator result
         """
         full_text = ''
+        prompt_tokens = 0
+        completion_tokens = 0
+
+        final_chunk = LLMResultChunk(
+            model=model,
+            prompt_messages=prompt_messages,
+            delta=LLMResultChunkDelta(
+                index=0,
+                message=AssistantPromptMessage(content=''),
+            )
+        )
+
         for chunk in response:
             if len(chunk.choices) == 0:
+                if chunk.usage:
+                    # calculate num tokens
+                    prompt_tokens = chunk.usage.prompt_tokens
+                    completion_tokens = chunk.usage.completion_tokens
                 continue
 
             delta = chunk.choices[0]
@@ -464,20 +485,7 @@ class OpenAILargeLanguageModel(_CommonOpenAI, LargeLanguageModel):
             full_text += text
 
             if delta.finish_reason is not None:
-                # calculate num tokens
-                if chunk.usage:
-                    # transform usage
-                    prompt_tokens = chunk.usage.prompt_tokens
-                    completion_tokens = chunk.usage.completion_tokens
-                else:
-                    # calculate num tokens
-                    prompt_tokens = self._num_tokens_from_string(model, prompt_messages[0].content)
-                    completion_tokens = self._num_tokens_from_string(model, full_text)
-
-                # transform usage
-                usage = self._calc_response_usage(model, credentials, prompt_tokens, completion_tokens)
-
-                yield LLMResultChunk(
+                final_chunk = LLMResultChunk(
                     model=chunk.model,
                     prompt_messages=prompt_messages,
                     system_fingerprint=chunk.system_fingerprint,
@@ -485,7 +493,6 @@ class OpenAILargeLanguageModel(_CommonOpenAI, LargeLanguageModel):
                         index=delta.index,
                         message=assistant_prompt_message,
                         finish_reason=delta.finish_reason,
-                        usage=usage
                     )
                 )
             else:
@@ -499,6 +506,19 @@ class OpenAILargeLanguageModel(_CommonOpenAI, LargeLanguageModel):
                     )
                 )
 
+        if not prompt_tokens:
+            prompt_tokens = self._num_tokens_from_string(model, prompt_messages[0].content)
+
+        if not completion_tokens:
+            completion_tokens = self._num_tokens_from_string(model, full_text)
+
+        # transform usage
+        usage = self._calc_response_usage(model, credentials, prompt_tokens, completion_tokens)
+
+        final_chunk.delta.usage = usage
+
+        yield final_chunk
+
     def _chat_generate(self, model: str, credentials: dict,
                        prompt_messages: list[PromptMessage], model_parameters: dict,
                        tools: Optional[list[PromptMessageTool]] = None, stop: Optional[list[str]] = None,
@@ -531,6 +551,7 @@ class OpenAILargeLanguageModel(_CommonOpenAI, LargeLanguageModel):
 
             model_parameters["response_format"] = response_format
 
+
         extra_model_kwargs = {}
 
         if tools:
@@ -547,6 +568,11 @@ class OpenAILargeLanguageModel(_CommonOpenAI, LargeLanguageModel):
         if user:
             extra_model_kwargs['user'] = user
 
+        if stream:
+            extra_model_kwargs['stream_options'] = {
+                'include_usage': True
+            }
+
         # clear illegal prompt messages
         prompt_messages = self._clear_illegal_prompt_messages(model, prompt_messages)
 
@@ -630,8 +656,24 @@ class OpenAILargeLanguageModel(_CommonOpenAI, LargeLanguageModel):
         """
         full_assistant_content = ''
         delta_assistant_message_function_call_storage: ChoiceDeltaFunctionCall = None
+        prompt_tokens = 0
+        completion_tokens = 0
+        final_tool_calls = []
+        final_chunk = LLMResultChunk(
+            model=model,
+            prompt_messages=prompt_messages,
+            delta=LLMResultChunkDelta(
+                index=0,
+                message=AssistantPromptMessage(content=''),
+            )
+        )
+
         for chunk in response:
             if len(chunk.choices) == 0:
+                if chunk.usage:
+                    # calculate num tokens
+                    prompt_tokens = chunk.usage.prompt_tokens
+                    completion_tokens = chunk.usage.completion_tokens
                 continue
 
             delta = chunk.choices[0]
@@ -667,6 +709,8 @@ class OpenAILargeLanguageModel(_CommonOpenAI, LargeLanguageModel):
             # tool_calls = self._extract_response_tool_calls(assistant_message_tool_calls)
             function_call = self._extract_response_function_call(assistant_message_function_call)
             tool_calls = [function_call] if function_call else []
+            if tool_calls:
+                final_tool_calls.extend(tool_calls)
 
             # transform assistant message to prompt message
             assistant_prompt_message = AssistantPromptMessage(
@@ -677,19 +721,7 @@ class OpenAILargeLanguageModel(_CommonOpenAI, LargeLanguageModel):
             full_assistant_content += delta.delta.content if delta.delta.content else ''
 
             if has_finish_reason:
-                # calculate num tokens
-                prompt_tokens = self._num_tokens_from_messages(model, prompt_messages, tools)
-
-                full_assistant_prompt_message = AssistantPromptMessage(
-                    content=full_assistant_content,
-                    tool_calls=tool_calls
-                )
-                completion_tokens = self._num_tokens_from_messages(model, [full_assistant_prompt_message])
-
-                # transform usage
-                usage = self._calc_response_usage(model, credentials, prompt_tokens, completion_tokens)
-
-                yield LLMResultChunk(
+                final_chunk = LLMResultChunk(
                     model=chunk.model,
                     prompt_messages=prompt_messages,
                     system_fingerprint=chunk.system_fingerprint,
@@ -697,7 +729,6 @@ class OpenAILargeLanguageModel(_CommonOpenAI, LargeLanguageModel):
                         index=delta.index,
                         message=assistant_prompt_message,
                         finish_reason=delta.finish_reason,
-                        usage=usage
                     )
                 )
             else:
@@ -711,6 +742,22 @@ class OpenAILargeLanguageModel(_CommonOpenAI, LargeLanguageModel):
                     )
                 )
 
+        if not prompt_tokens:
+            prompt_tokens = self._num_tokens_from_messages(model, prompt_messages, tools)
+
+        if not completion_tokens:
+            full_assistant_prompt_message = AssistantPromptMessage(
+                content=full_assistant_content,
+                tool_calls=final_tool_calls
+            )
+            completion_tokens = self._num_tokens_from_messages(model, [full_assistant_prompt_message])
+
+        # transform usage
+        usage = self._calc_response_usage(model, credentials, prompt_tokens, completion_tokens)
+        final_chunk.delta.usage = usage
+
+        yield final_chunk
+
     def _extract_response_tool_calls(self,
                                      response_tool_calls: list[ChatCompletionMessageToolCall | ChoiceDeltaToolCall]) \
             -> list[AssistantPromptMessage.ToolCall]:

api/core/model_runtime/model_providers/volcengine_maas/_assets/icon_l_en.svg

@@ -0,0 +1,23 @@
+<?xml version="1.0" encoding="utf-8"?>
+<svg viewBox="540.1546 563.7749 150.5263 25" xmlns="http://www.w3.org/2000/svg">
+  <g clip-path="url(#a)" transform="matrix(0.17650800943374634, 0, 0, -0.17650800943374634, 445.969970703125, 671.58935546875)">
+    <path fill="#00e5e5" d="M618.577 552.875l-13.795-59.901c-.218-.948.502-1.853 1.475-1.853h27.575c.973 0 1.693.905 1.474 1.853l-13.795 59.901c-.358 1.557-2.575 1.557-2.934 0"/>
+    <path fill="#00e5e5" d="M542.203 530.153l-8.564-37.188c-.218-.944.499-1.844 1.467-1.844h17.135c.968 0 1.684.9 1.467 1.844l-8.565 37.188c-.359 1.561-2.581 1.561-2.94 0"/>
+    <path fill="#006eff" d="M561.028 575.58l-19.024-82.607c-.219-.948.502-1.853 1.474-1.853h38.034c.973 0 1.693.905 1.474 1.853l-19.024 82.607c-.358 1.557-2.575 1.557-2.934 0"/>
+    <path fill="#006eff" d="M592.95 609.651l-26.871-116.678c-.218-.947.502-1.852 1.475-1.852h53.726c.973 0 1.693.905 1.475 1.852l-26.871 116.678c-.358 1.557-2.575 1.557-2.934 0"/>
+    <path fill="#00e5e5" d="M575.175 564.245l-16.414-71.271c-.218-.948.502-1.853 1.475-1.853h32.812c.973 0 1.693.905 1.475 1.853l-16.414 71.271c-.358 1.557-2.576 1.557-2.934 0"/>
+    <path fill="#4b4b4b" d="M677.358 492.868l-24.407 72.089h13.49l20.804-64.365 21.318 64.365h13.285l-24.614-72.089z"/>
+    <path fill="#4b4b4b" d="M744.98 502.857c3.09 0 5.922.788 8.496 2.368 2.575 1.579 4.617 3.691 6.128 6.333 1.509 2.643 2.266 5.544 2.266 8.703 0 3.09-.757 5.973-2.266 8.651-1.511 2.677-3.553 4.789-6.128 6.334-2.574 1.545-5.406 2.317-8.496 2.317-3.159 0-6.025-.79-8.6-2.369-2.574-1.58-4.618-3.674-6.127-6.282-1.511-2.611-2.266-5.493-2.266-8.651 0-3.159.755-6.06 2.266-8.703 1.509-2.642 3.553-4.754 6.127-6.333 2.575-1.58 5.441-2.368 8.6-2.368m0-10.813c-5.149 0-9.853 1.269-14.109 3.81-4.258 2.539-7.657 5.974-10.196 10.298-2.541 4.326-3.81 9.027-3.81 14.109 0 5.08 1.269 9.766 3.81 14.058 2.539 4.29 5.938 7.706 10.196 10.247 4.256 2.539 8.96 3.811 14.109 3.811 5.08 0 9.766-1.272 14.057-3.811 4.291-2.541 7.689-5.974 10.196-10.299 2.506-4.325 3.759-8.995 3.759-14.006 0-5.082-1.253-9.783-3.759-14.109-2.507-4.324-5.905-7.759-10.196-10.298-4.291-2.541-8.977-3.81-14.057-3.81"/>
+    <path fill="#4b4b4b" d="M782.261 564.957h11.844v-72.089h-11.844z"/>
+    <path fill="#4b4b4b" d="M831.693 492.044c-5.149 0-9.887 1.27-14.212 3.811-4.325 2.539-7.708 5.973-10.144 10.297-2.438 4.327-3.656 8.995-3.656 14.006 0 5.15 1.252 9.887 3.759 14.212 2.505 4.326 5.904 7.74 10.195 10.247 4.29 2.506 8.976 3.76 14.058 3.76 5.835 0 11.071-1.632 15.705-4.893 4.635-3.26 7.982-7.535 10.042-12.821l-10.917-2.986c-1.374 3.02-3.365 5.44-5.973 7.261-2.61 1.817-5.561 2.728-8.857 2.728-3.159 0-6.025-.79-8.599-2.368-2.575-1.58-4.619-3.708-6.128-6.386-1.511-2.678-2.266-5.597-2.266-8.754 0-3.158.755-6.076 2.266-8.753 1.509-2.677 3.569-4.789 6.179-6.333 2.609-1.546 5.458-2.317 8.548-2.317 3.363 0 6.333.908 8.908 2.728 2.575 1.819 4.548 4.205 5.922 7.158l10.917-2.987c-2.129-5.217-5.494-9.458-10.093-12.717-4.601-3.263-9.819-4.893-15.654-4.893"/>
+    <path fill="#4b4b4b" d="M892.043 502.755c3.09 0 5.955.788 8.599 2.368 2.642 1.578 4.754 3.708 6.334 6.386 1.578 2.677 2.369 5.559 2.369 8.649 0 3.09-.791 5.974-2.369 8.652-1.58 2.677-3.692 4.822-6.334 6.436-2.644 1.613-5.509 2.42-8.599 2.42-3.159 0-6.06-.79-8.703-2.368-2.643-1.58-4.721-3.708-6.23-6.386-1.511-2.678-2.266-5.596-2.266-8.754 0-3.158.755-6.059 2.266-8.701 1.509-2.644 3.569-4.756 6.179-6.334 2.608-1.58 5.526-2.368 8.754-2.368m-1.03-10.711c-4.943 0-9.492 1.27-13.646 3.811-4.155 2.539-7.45 5.973-9.886 10.297-2.438 4.327-3.656 8.995-3.656 14.006 0 5.15 1.218 9.887 3.656 14.212 2.436 4.326 5.731 7.74 9.886 10.247 4.154 2.506 8.703 3.76 13.646 3.76 3.638 0 6.968-.739 9.99-2.214 3.02-1.477 5.526-3.486 7.518-6.025v7.312h11.843v-54.582h-11.843v7.313c-1.992-2.541-4.498-4.533-7.518-5.974-3.022-1.442-6.352-2.163-9.99-2.163"/>
+    <path fill="#4b4b4b" d="M931.692 492.868v54.582h11.74v-7.209c1.854 2.402 4.239 4.359 7.158 5.87 2.917 1.51 6.23 2.266 9.938 2.266 3.981 0 7.724-.894 11.226-2.678 3.501-1.786 6.315-4.361 8.444-7.725 2.128-3.364 3.193-7.244 3.193-11.636v-33.47h-11.843v30.277c0 2.883-.602 5.389-1.803 7.518-1.202 2.128-2.816 3.774-4.84 4.943-2.026 1.167-4.274 1.751-6.746 1.751-2.816 0-5.355-.721-7.621-2.163-2.265-1.441-4.016-3.398-5.252-5.87-1.236-2.472-1.854-5.288-1.854-8.444v-28.012z"/>
+    <path fill="#4b4b4b" d="M1019.437 502.857c3.09 0 5.922.788 8.496 2.368 2.575 1.579 4.617 3.691 6.128 6.333 1.509 2.643 2.266 5.544 2.266 8.703 0 3.09-.757 5.973-2.266 8.651-1.511 2.677-3.553 4.789-6.128 6.334-2.574 1.545-5.406 2.317-8.496 2.317-3.159 0-6.025-.79-8.599-2.369-2.575-1.58-4.619-3.674-6.128-6.282-1.511-2.611-2.266-5.493-2.266-8.651 0-3.159.755-6.06 2.266-8.703 1.509-2.642 3.553-4.754 6.128-6.333 2.574-1.58 5.44-2.368 8.599-2.368m0-10.813c-5.149 0-9.853 1.269-14.109 3.81-4.258 2.539-7.657 5.974-10.196 10.298-2.541 4.326-3.81 9.027-3.81 14.109 0 5.08 1.269 9.766 3.81 14.058 2.539 4.29 5.938 7.706 10.196 10.247 4.256 2.539 8.96 3.811 14.109 3.811 5.08 0 9.766-1.272 14.058-3.811 4.29-2.541 7.688-5.974 10.195-10.299 2.506-4.325 3.759-8.995 3.759-14.006 0-5.082-1.253-9.783-3.759-14.109-2.507-4.324-5.905-7.759-10.195-10.298-4.292-2.541-8.978-3.81-14.058-3.81"/>
+    <path fill="#4b4b4b" d="M1057.026 492.868v72.089h51.287v-11.328h-39.135v-19.156h35.016v-11.122h-35.016v-19.155h39.135v-11.328z"/>
+    <path fill="#4b4b4b" d="M1118.92 492.868v54.582h11.74v-7.209c1.854 2.402 4.239 4.359 7.158 5.87 2.917 1.51 6.231 2.266 9.938 2.266 3.981 0 7.724-.894 11.226-2.678 3.501-1.786 6.316-4.361 8.444-7.725 2.128-3.364 3.193-7.244 3.193-11.636v-33.47h-11.843v30.277c0 2.883-.602 5.389-1.803 7.518-1.202 2.128-2.815 3.774-4.84 4.943-2.026 1.167-4.274 1.751-6.745 1.751-2.816 0-5.356-.721-7.621-2.163-2.266-1.441-4.017-3.398-5.253-5.87-1.236-2.472-1.854-5.288-1.854-8.444v-28.012z"/>
+    <path fill="#4b4b4b" d="M1207.077 504.094c3.09 0 5.955.754 8.599 2.266 2.642 1.508 4.754 3.551 6.334 6.127 1.578 2.574 2.369 5.406 2.369 8.496 0 3.089-.773 5.922-2.317 8.496-1.545 2.574-3.657 4.599-6.334 6.076-2.678 1.476-5.561 2.214-8.651 2.214-3.159 0-6.042-.738-8.651-2.214-2.61-1.477-4.686-3.502-6.231-6.076s-2.317-5.407-2.317-8.496c0-3.09.772-5.94 2.317-8.547 1.545-2.611 3.639-4.654 6.283-6.128 2.642-1.477 5.509-2.214 8.599-2.214m1.339-34.912c-4.052 0-7.948.756-11.689 2.266-3.743 1.51-6.969 3.809-9.681 6.899-2.713 3.09-4.618 6.899-5.716 11.431h11.844c.824-2.266 2.024-4.12 3.604-5.56 1.579-1.442 3.363-2.473 5.356-3.091 1.99-.617 4.05-.927 6.179-.927 2.883 0 5.474.584 7.775 1.752 2.3 1.168 4.12 2.986 5.458 5.458 1.339 2.471 2.009 5.595 2.009 9.372v4.427c-1.992-2.334-4.48-4.188-7.467-5.561-2.986-1.373-6.333-2.059-10.041-2.059-5.149 0-9.801 1.2-13.955 3.605-4.154 2.402-7.398 5.697-9.732 9.885-2.335 4.188-3.501 8.822-3.501 13.904 0 5.216 1.184 9.921 3.553 14.109 2.368 4.187 5.628 7.466 9.783 9.835 4.154 2.369 8.77 3.518 13.852 3.45 3.638-.069 6.968-.807 9.99-2.214 3.02-1.408 5.526-3.382 7.518-5.922v7.209h11.843v-50.463c0-5.767-1.22-10.762-3.656-14.985-2.438-4.221-5.699-7.414-9.784-9.577-4.085-2.163-8.599-3.243-13.542-3.243"/>
+    <path fill="#4b4b4b" d="M1247.035 547.45h11.844v-54.582h-11.844zm6.076 6.385c-2.129 0-3.949.703-5.458 2.111-1.511 1.406-2.266 3.175-2.266 5.304 0 2.059.755 3.81 2.266 5.252 1.509 1.442 3.329 2.163 5.458 2.163 2.128 0 3.93-.721 5.407-2.163 1.476-1.442 2.214-3.193 2.214-5.252 0-2.129-.738-3.898-2.214-5.304-1.477-1.408-3.279-2.111-5.407-2.111"/>
+    <path fill="#4b4b4b" d="M1270.515 492.868v54.582h11.74v-7.209c1.854 2.402 4.239 4.359 7.158 5.87 2.917 1.51 6.23 2.266 9.938 2.266 3.981 0 7.724-.894 11.226-2.678 3.501-1.786 6.315-4.361 8.444-7.725 2.128-3.364 3.193-7.244 3.193-11.636v-33.47h-11.843v30.277c0 2.883-.602 5.389-1.803 7.518-1.202 2.128-2.816 3.774-4.84 4.943-2.026 1.167-4.274 1.751-6.746 1.751-2.816 0-5.355-.721-7.621-2.163-2.265-1.441-4.016-3.398-5.252-5.87-1.236-2.472-1.854-5.288-1.854-8.444v-28.012z"/>
+    <path fill="#4b4b4b" d="M1374.428 524.895c-.481 2.334-1.459 4.496-2.935 6.488-1.477 1.991-3.331 3.57-5.562 4.738-2.232 1.167-4.754 1.751-7.569 1.751-2.816 0-5.373-.584-7.672-1.751-2.302-1.168-4.189-2.747-5.665-4.738-1.477-1.992-2.455-4.154-2.935-6.488zm-15.963-32.852c-5.149 0-9.87 1.27-14.16 3.811-4.292 2.539-7.673 5.974-10.145 10.298-2.471 4.326-3.707 8.994-3.707 14.006 0 5.149 1.252 9.886 3.759 14.212 2.506 4.326 5.904 7.74 10.196 10.247 4.29 2.506 8.976 3.759 14.057 3.759 4.531 0 8.754-1.03 12.668-3.089 3.913-2.06 7.157-4.859 9.732-8.394 2.574-3.537 4.238-7.502 4.995-11.894.685-3.09.72-6.146.102-9.167h-43.872c.48-2.746 1.494-5.115 3.038-7.105 1.545-1.992 3.467-3.519 5.768-4.583 2.299-1.065 4.822-1.597 7.569-1.597 2.884 0 5.681.602 8.394 1.803 2.711 1.2 4.959 2.832 6.745 4.892l10.402-2.678c-2.129-4.394-5.597-7.913-10.402-10.556-4.806-2.643-9.853-3.965-15.139-3.965"/>
+  </g>
+</svg>

api/core/model_runtime/model_providers/volcengine_maas/_assets/icon_l_zh.svg

@@ -0,0 +1,39 @@
+<?xml version="1.0" encoding="utf-8"?>
+<svg viewBox="0 0 101.7291 25" xmlns="http://www.w3.org/2000/svg">
+  <defs>
+    <clipPath id="clip0_4967_21175">
+      <rect width="99.9412" height="24.5665" fill="white" transform="translate(0 4)"/>
+    </clipPath>
+  </defs>
+  <g clip-path="url(#clip0_4967_21175)" transform="matrix(1.017889022827, 0, 0, 1.017889022827, 0.893925011158, -4.07626192043)" style="transform-origin: 49.9706px 0.0704009px;">
+    <path d="M57.8088 12.1255H56.1122C56.0096 12.1255 55.9264 12.2087 55.9264 12.3113V27.7947C55.9264 27.8973 56.0096 27.9806 56.1122 27.9806H57.8088C57.9114 27.9806 57.9946 27.8973 57.9946 27.7947V12.3113C57.9946 12.2087 57.9114 12.1255 57.8088 12.1255Z" fill="#4B4B4B"/>
+    <path d="M80.4334 12.1895H78.6511C78.5471 12.1895 78.4628 12.2737 78.4628 12.3777V28.3781C78.4628 28.4821 78.5471 28.5663 78.6511 28.5663H80.4334C80.5374 28.5663 80.6216 28.4821 80.6216 28.3781V12.3777C80.6216 12.2737 80.5374 12.1895 80.4334 12.1895Z" fill="#4B4B4B"/>
+    <path d="M51.7994 14.0625H50.1076C50.0036 14.0625 49.9194 14.1468 49.9194 14.2507V28.2639C49.9194 28.3679 50.0036 28.4522 50.1076 28.4522H51.7994C51.9034 28.4522 51.9876 28.3679 51.9876 28.2639V14.2507C51.9876 14.1468 51.9034 14.0625 51.7994 14.0625Z" fill="#4B4B4B"/>
+    <path d="M63.823 14.0625H62.1312C62.0272 14.0625 61.9429 14.1468 61.9429 14.2507V28.2639C61.9429 28.3679 62.0272 28.4522 62.1312 28.4522H63.823C63.9269 28.4522 64.0112 28.3679 64.0112 28.2639V14.2507C64.0112 14.1468 63.9269 14.0625 63.823 14.0625Z" fill="#4B4B4B"/>
+    <path d="M63.9135 26.3862V28.1066C63.9135 28.1989 63.8769 28.2874 63.8116 28.3526C63.7464 28.4178 63.6579 28.4545 63.5656 28.4545H50.3578C50.2656 28.4545 50.1771 28.4178 50.1118 28.3526C50.0466 28.2874 50.01 28.1989 50.01 28.1066V26.3862H63.9135Z" fill="#4B4B4B"/>
+    <path d="M92.3091 13.9837V12.6398C92.3091 12.5885 92.2675 12.5469 92.2162 12.5469H84.1409C84.0896 12.5469 84.048 12.5885 84.048 12.6398V13.9837C84.048 14.035 84.0896 14.0766 84.1409 14.0766H92.2162C92.2675 14.0766 92.3091 14.035 92.3091 13.9837Z" fill="#4B4B4B"/>
+    <path d="M99.6315 13.0328V14.4911C99.6315 14.5157 99.6217 14.5394 99.6043 14.5568C99.5868 14.5742 99.5632 14.584 99.5385 14.584H94.3822C94.3316 14.584 94.2831 14.5639 94.2474 14.5282C94.2116 14.4924 94.1916 14.4439 94.1916 14.3934V13.1281C94.1916 13.0776 94.2116 13.0291 94.2474 12.9933C94.2831 12.9576 94.3316 12.9375 94.3822 12.9375H99.5385C99.5632 12.9375 99.5868 12.9473 99.6043 12.9647C99.6217 12.9821 99.6315 13.0058 99.6315 13.0304V13.0328Z" fill="#4B4B4B"/>
+    <path d="M43.605 14.437C43.574 15.2495 43.462 17.1128 43.0784 20.046C43.075 20.0729 43.0774 20.1002 43.0853 20.1261C43.0932 20.152 43.1064 20.176 43.1242 20.1964C43.1419 20.2169 43.1638 20.2334 43.1883 20.2448C43.2129 20.2563 43.2395 20.2624 43.2666 20.2629H45.2205C45.267 20.2621 45.3116 20.2445 45.3462 20.2135C45.3808 20.1824 45.403 20.1398 45.4088 20.0937C45.7876 17.139 45.8853 15.2733 45.9234 14.4513C45.9244 14.4257 45.9202 14.4003 45.911 14.3765C45.9018 14.3526 45.8879 14.3309 45.8701 14.3126C45.8522 14.2944 45.8309 14.2799 45.8073 14.2702C45.7837 14.2604 45.7583 14.2555 45.7328 14.2559H43.7956C43.7465 14.2552 43.6991 14.2738 43.6634 14.3076C43.6278 14.3415 43.6068 14.3879 43.605 14.437V14.437Z" fill="#4B4B4B"/>
+    <path d="M34.2359 14.4394C34.2693 15.2566 34.3789 17.1343 34.7649 20.0889C34.769 20.1161 34.7672 20.1437 34.7596 20.1701C34.7521 20.1965 34.7389 20.2209 34.721 20.2417C34.7032 20.2625 34.681 20.2792 34.6561 20.2907C34.6312 20.3022 34.6041 20.3081 34.5766 20.3082H32.6085C32.5617 20.3074 32.5167 20.29 32.4818 20.2589C32.4468 20.2279 32.4241 20.1853 32.4178 20.139C32.0366 17.1605 31.9294 15.2805 31.8984 14.4536C31.8974 14.4278 31.9017 14.4021 31.911 14.3779C31.9203 14.3538 31.9344 14.3318 31.9525 14.3133C31.9705 14.2948 31.9921 14.2802 32.016 14.2703C32.0399 14.2605 32.0655 14.2555 32.0914 14.2559H34.0453C34.0946 14.2558 34.1421 14.2749 34.1776 14.3092C34.2132 14.3434 34.2341 14.39 34.2359 14.4394V14.4394Z" fill="#4B4B4B"/>
+    <path d="M83.8741 15.4494C83.8503 15.4518 83.8281 15.4628 83.8119 15.4803C83.7956 15.4979 83.7864 15.5208 83.7859 15.5448V17.1055C83.7855 17.1187 83.7879 17.1319 83.7931 17.1441C83.7982 17.1564 83.8058 17.1674 83.8155 17.1764C83.8252 17.1854 83.8368 17.1923 83.8493 17.1965C83.8619 17.2007 83.8752 17.2022 83.8884 17.2008C85.2871 17.0697 86.3188 16.0666 86.457 14.5488C86.458 14.5356 86.4563 14.5225 86.4519 14.5101C86.4476 14.4977 86.4407 14.4863 86.4318 14.4767C86.4228 14.4671 86.412 14.4594 86.3999 14.4542C86.3879 14.4489 86.3749 14.4463 86.3617 14.4463H84.7938C84.7708 14.4462 84.7485 14.4546 84.7313 14.4701C84.7141 14.4855 84.7033 14.5067 84.7009 14.5297C84.5889 15.3351 84.1386 15.4375 83.8741 15.4494Z" fill="#4B4B4B"/>
+    <path d="M94.9875 15.5781H93.3743C93.3602 15.5783 93.3462 15.5815 93.3334 15.5875C93.3206 15.5936 93.3093 15.6024 93.3002 15.6133C93.2912 15.6242 93.2846 15.6369 93.2809 15.6506C93.2773 15.6642 93.2766 15.6785 93.279 15.6925C93.8032 18.6448 96.3457 20.5057 99.8436 20.4986C99.8693 20.498 99.8938 20.4875 99.912 20.4693C99.9302 20.4511 99.9407 20.4266 99.9413 20.4009V18.8211C99.9413 18.7958 99.9313 18.7716 99.9134 18.7537C99.8955 18.7358 99.8713 18.7258 99.846 18.7258C97.8301 18.7258 95.6308 17.9156 95.0804 15.652C95.0752 15.6312 95.0632 15.6127 95.0464 15.5993C95.0297 15.586 95.0089 15.5785 94.9875 15.5781V15.5781Z" fill="#4B4B4B"/>
+    <path d="M87.1408 12.1895H85.3394C85.2881 12.1895 85.2465 12.2311 85.2465 12.2824V14.1624C85.2465 14.2137 85.2881 14.2553 85.3394 14.2553H87.1408C87.1922 14.2553 87.2338 14.2137 87.2338 14.1624V12.2824C87.2338 12.2311 87.1922 12.1895 87.1408 12.1895Z" fill="#4B4B4B"/>
+    <path d="M91.0177 12.1895H89.2163C89.165 12.1895 89.1234 12.2311 89.1234 12.2824V14.1624C89.1234 14.2137 89.165 14.2553 89.2163 14.2553H91.0177C91.0691 14.2553 91.1107 14.2137 91.1107 14.1624V12.2824C91.1107 12.2311 91.0691 12.1895 91.0177 12.1895Z" fill="#4B4B4B"/>
+    <path d="M98.9262 14.3032C98.9262 14.2484 98.9262 14.2008 98.9262 14.165H97.1105C97.1105 14.2151 97.1105 14.277 97.1105 14.3533C97.1272 14.9657 97.1653 16.3977 96.1907 17.4104C95.4497 18.18 94.2345 18.5875 92.5761 18.6232C92.5633 18.6232 92.5507 18.6258 92.539 18.6307C92.5273 18.6356 92.5167 18.6429 92.5078 18.652C92.4989 18.6611 92.4919 18.6719 92.4873 18.6837C92.4827 18.6956 92.4804 18.7082 92.4807 18.7209L92.4998 20.3388C92.4998 20.352 92.5024 20.365 92.5074 20.3771C92.5125 20.3893 92.5198 20.4003 92.5291 20.4096C92.5384 20.4189 92.5494 20.4263 92.5616 20.4313C92.5737 20.4363 92.5867 20.4389 92.5999 20.4389C94.7635 20.396 96.4123 19.8003 97.4989 18.6709C99 17.1054 98.9452 15.0705 98.9262 14.3032Z" fill="#4B4B4B"/>
+    <path d="M95.6117 12.168H93.9938C93.9697 12.1685 93.9468 12.1782 93.9295 12.195C93.9123 12.2118 93.902 12.2345 93.9009 12.2585C93.89 12.6263 93.7498 12.9785 93.5049 13.2531C93.2599 13.5277 92.9259 13.7071 92.5617 13.7597C92.5394 13.7631 92.519 13.7743 92.5042 13.7913C92.4893 13.8083 92.481 13.83 92.4807 13.8526V15.4729C92.4807 15.4858 92.4833 15.4986 92.4884 15.5105C92.4935 15.5224 92.5009 15.5331 92.5103 15.542C92.5197 15.5509 92.5307 15.5579 92.5428 15.5624C92.555 15.5669 92.5679 15.5689 92.5808 15.5682C93.4229 15.5146 94.2135 15.1438 94.7931 14.5305C95.3728 13.9173 95.6985 13.1071 95.7046 12.2633C95.7049 12.2509 95.7028 12.2385 95.6982 12.227C95.6937 12.2154 95.6869 12.2049 95.6783 12.196C95.6696 12.1871 95.6593 12.1801 95.6478 12.1753C95.6364 12.1704 95.6241 12.168 95.6117 12.168V12.168Z" fill="#4B4B4B"/>
+    <path d="M67.2708 12.7498V14.4439C67.2708 14.4938 67.2906 14.5417 67.3259 14.577C67.3612 14.6123 67.4091 14.6322 67.459 14.6322H73.2826C73.3066 14.6322 73.3296 14.6417 73.3466 14.6587C73.3636 14.6757 73.3731 14.6987 73.3731 14.7227V16.6289C73.3734 16.641 73.3713 16.6531 73.3669 16.6643C73.3625 16.6756 73.3559 16.6859 73.3475 16.6945C73.339 16.7032 73.3289 16.7101 73.3178 16.7148C73.3066 16.7195 73.2947 16.7219 73.2826 16.7219H67.459C67.4091 16.7219 67.3612 16.7417 67.3259 16.777C67.2906 16.8123 67.2708 16.8602 67.2708 16.9101V22.5859C67.2708 22.6106 67.2757 22.6351 67.2851 22.6579C67.2946 22.6808 67.3085 22.7015 67.3259 22.719C67.3434 22.7365 67.3642 22.7504 67.387 22.7598C67.4099 22.7693 67.4343 22.7741 67.459 22.7741H73.2826C73.3068 22.7748 73.3298 22.7848 73.3468 22.8022C73.3637 22.8196 73.3731 22.8428 73.3731 22.8671V26.0314C73.3734 26.056 73.3689 26.0805 73.3597 26.1033C73.3505 26.1261 73.3368 26.1469 73.3195 26.1644C73.3023 26.1819 73.2817 26.1958 73.259 26.2053C73.2362 26.2148 73.2119 26.2197 73.1873 26.2197H68.8983C68.8561 26.2195 68.8151 26.2335 68.7818 26.2594C68.7485 26.2853 68.7249 26.3217 68.7148 26.3626L68.2978 28.1735C68.2946 28.1872 68.2944 28.2014 68.2975 28.2151C68.3005 28.2288 68.3066 28.2417 68.3154 28.2527C68.3241 28.2637 68.3352 28.2726 68.3478 28.2787C68.3604 28.2848 68.3743 28.2879 68.3883 28.2879H74.8767C75.0258 28.2879 75.1688 28.2287 75.2743 28.1232C75.3798 28.0178 75.439 27.8747 75.439 27.7256V20.887C75.439 20.8371 75.4192 20.7892 75.3839 20.7539C75.3486 20.7186 75.3007 20.6987 75.2508 20.6987H69.432C69.4197 20.6991 69.4075 20.6969 69.3961 20.6923C69.3846 20.6878 69.3743 20.6809 69.3656 20.6723C69.3569 20.6636 69.35 20.6532 69.3455 20.6418C69.3409 20.6303 69.3387 20.6181 69.3391 20.6058V18.8759C69.3391 18.8513 69.3489 18.8276 69.3663 18.8102C69.3837 18.7928 69.4074 18.783 69.432 18.783H75.2555C75.3055 18.783 75.3533 18.7631 75.3886 18.7278C75.4239 18.6925 75.4438 18.6447 75.4438 18.5947V12.7498C75.4432 12.7 75.4231 12.6525 75.388 12.6173C75.3528 12.5822 75.3053 12.5621 75.2555 12.5615H67.459C67.4091 12.5615 67.3612 12.5814 67.3259 12.6167C67.2906 12.652 67.2708 12.6998 67.2708 12.7498V12.7498Z" fill="#4B4B4B"/>
+    <path d="M91.8088 14.9033H85.3562C85.3316 14.9033 85.3079 14.9131 85.2905 14.9305C85.2731 14.948 85.2633 14.9716 85.2633 14.9962V16.1328C85.2629 16.1451 85.2651 16.1574 85.2697 16.1688C85.2742 16.1802 85.2811 16.1906 85.2898 16.1993C85.2985 16.208 85.3088 16.2148 85.3203 16.2194C85.3317 16.2239 85.3439 16.2261 85.3562 16.2258H90.148C90.1603 16.2254 90.1725 16.2276 90.1839 16.2322C90.1954 16.2367 90.2057 16.2436 90.2144 16.2523C90.2231 16.261 90.23 16.2713 90.2345 16.2827C90.2391 16.2942 90.2413 16.3064 90.2409 16.3187V18.8707C90.2409 18.8953 90.2311 18.9189 90.2137 18.9364C90.1963 18.9538 90.1726 18.9636 90.148 18.9636H89.5928C89.5719 18.9636 89.5515 18.9706 89.5351 18.9836C89.5187 18.9966 89.5071 19.0147 89.5023 19.0351L89.1901 20.3837C89.1869 20.3974 89.1868 20.4116 89.1898 20.4253C89.1929 20.439 89.199 20.4519 89.2077 20.4629C89.2164 20.4739 89.2275 20.4828 89.2401 20.4889C89.2528 20.495 89.2666 20.4981 89.2807 20.4981H91.2322C91.4344 20.4981 91.6283 20.4178 91.7713 20.2748C91.9143 20.1318 91.9947 19.9378 91.9947 19.7356V15.0916C91.995 15.067 91.9904 15.0425 91.9812 15.0197C91.972 14.9969 91.9584 14.9761 91.9411 14.9586C91.9238 14.9411 91.9032 14.9272 91.8805 14.9177C91.8578 14.9082 91.8334 14.9033 91.8088 14.9033V14.9033Z" fill="#4B4B4B"/>
+    <path d="M90.7961 21.5732V26.9321C90.7961 26.9567 90.7912 26.9811 90.7817 27.0038C90.7722 27.0265 90.7583 27.0471 90.7408 27.0644C90.7233 27.0817 90.7025 27.0953 90.6797 27.1045C90.6569 27.1137 90.6325 27.1183 90.6079 27.118H89.3092C89.2674 27.1177 89.2266 27.1313 89.1934 27.1568C89.1602 27.1823 89.1364 27.2181 89.1258 27.2586L88.8017 28.45C88.7976 28.4643 88.7969 28.4793 88.7997 28.4939C88.8025 28.5085 88.8086 28.5223 88.8176 28.5341C88.8267 28.5459 88.8383 28.5554 88.8517 28.5619C88.8651 28.5684 88.8798 28.5717 88.8946 28.5715H92.2591C92.4572 28.5709 92.6468 28.4918 92.7866 28.3515C92.9264 28.2113 93.005 28.0213 93.005 27.8233V21.5732H90.7961Z" fill="#4B4B4B"/>
+    <path d="M84.4936 16.867V19.5023C84.4936 19.5269 84.4985 19.5513 84.508 19.574C84.5174 19.5967 84.5314 19.6173 84.5489 19.6346C84.5664 19.6519 84.5871 19.6655 84.61 19.6747C84.6328 19.6839 84.6572 19.6885 84.6818 19.6882H88.8779C88.9272 19.6882 88.9745 19.6686 89.0093 19.6337C89.0442 19.5989 89.0638 19.5516 89.0638 19.5023V16.867C89.0641 16.8423 89.0595 16.8179 89.0503 16.7951C89.0411 16.7723 89.0275 16.7515 89.0102 16.734C88.9929 16.7165 88.9723 16.7026 88.9496 16.6931C88.9269 16.6836 88.9025 16.6787 88.8779 16.6787H84.6818C84.6319 16.6787 84.584 16.6985 84.5487 16.7338C84.5134 16.7691 84.4936 16.817 84.4936 16.867V16.867ZM87.4888 18.6302H86.0591C86.0346 18.6296 86.0113 18.6196 85.994 18.6023C85.9767 18.585 85.9668 18.5617 85.9662 18.5373V17.8201C85.9668 17.7956 85.9767 17.7723 85.994 17.755C86.0113 17.7377 86.0346 17.7277 86.0591 17.7271H87.4888C87.5134 17.7271 87.537 17.7369 87.5545 17.7544C87.5719 17.7718 87.5817 17.7954 87.5817 17.8201V18.5349C87.582 18.5473 87.5798 18.5596 87.5753 18.5712C87.5708 18.5827 87.564 18.5933 87.5553 18.6022C87.5467 18.611 87.5363 18.6181 87.5249 18.6229C87.5134 18.6277 87.5012 18.6302 87.4888 18.6302V18.6302Z" fill="#4B4B4B"/>
+    <path d="M99.3193 22.1593V20.987C99.3193 20.9357 99.2777 20.894 99.2264 20.894H84.5746C84.5233 20.894 84.4817 20.9357 84.4817 20.987V22.1593C84.4817 22.2106 84.5233 22.2522 84.5746 22.2522H99.2264C99.2777 22.2522 99.3193 22.2106 99.3193 22.1593Z" fill="#4B4B4B"/>
+    <path d="M99.0906 24.1823V23.0099C99.0906 22.9586 99.049 22.917 98.9976 22.917H84.8058C84.7544 22.917 84.7128 22.9586 84.7128 23.0099V24.1823C84.7128 24.2336 84.7544 24.2752 84.8058 24.2752H98.9976C99.049 24.2752 99.0906 24.2336 99.0906 24.1823Z" fill="#4B4B4B"/>
+    <path d="M99.8602 26.191V25.0187C99.8602 24.9674 99.8186 24.9258 99.7673 24.9258H84.0361C83.9848 24.9258 83.9432 24.9674 83.9432 25.0187V26.191C83.9432 26.2424 83.9848 26.284 84.0361 26.284H99.7673C99.8186 26.284 99.8602 26.2424 99.8602 26.191Z" fill="#4B4B4B"/>
+    <path d="M37.8625 18.3203C37.9459 23.0621 33.8356 26.2336 31.386 26.398C31.338 26.3998 31.2925 26.4202 31.2592 26.4549C31.2259 26.4895 31.2073 26.5357 31.2073 26.5838V28.3042C31.2073 28.3292 31.2124 28.354 31.2221 28.377C31.2319 28.4 31.2463 28.4209 31.2643 28.4382C31.2823 28.4556 31.3036 28.4691 31.327 28.478C31.3504 28.4869 31.3753 28.491 31.4003 28.4901C33.2279 28.3804 35.301 27.394 36.8688 25.869C38.2794 24.5037 39.9665 22.0851 39.9665 18.3179V12.056C39.9665 12.0067 39.9469 11.9594 39.912 11.9246C39.8772 11.8897 39.8299 11.8701 39.7806 11.8701H38.0555C38.0062 11.8701 37.9589 11.8897 37.924 11.9246C37.8892 11.9594 37.8696 12.0067 37.8696 12.056L37.8625 18.3203Z" fill="#4B4B4B"/>
+    <path d="M39.9546 18.3203C39.8688 23.0621 43.9886 26.2336 46.4286 26.398C46.4769 26.3998 46.5226 26.4201 46.5563 26.4547C46.59 26.4893 46.6091 26.5355 46.6097 26.5838V28.3042C46.6097 28.3294 46.6046 28.3544 46.5947 28.3776C46.5848 28.4008 46.5702 28.4217 46.5519 28.4391C46.5337 28.4565 46.512 28.47 46.4884 28.4788C46.4647 28.4875 46.4395 28.4914 46.4143 28.4901C44.5891 28.3805 42.5137 27.394 40.9482 25.869C39.5376 24.5037 37.8506 22.0851 37.8506 18.318V12.056C37.8506 12.0067 37.8702 11.9594 37.905 11.9246C37.9399 11.8897 37.9872 11.8701 38.0364 11.8701H39.7616C39.7862 11.8698 39.8106 11.8744 39.8334 11.8836C39.8563 11.8928 39.877 11.9064 39.8945 11.9237C39.9121 11.941 39.926 11.9616 39.9354 11.9843C39.9449 12.007 39.9498 12.0314 39.9498 12.056L39.9546 18.3203Z" fill="#4B4B4B"/>
+    <path d="M20.511 15.3019L17.2442 28.1928C17.2362 28.2282 17.2364 28.2649 17.2447 28.3001C17.2531 28.3354 17.2694 28.3683 17.2923 28.3964C17.3153 28.4244 17.3444 28.4468 17.3773 28.4619C17.4103 28.477 17.4462 28.4844 17.4825 28.4835H24.0137C24.0499 28.4844 24.0859 28.477 24.1188 28.4619C24.1518 28.4468 24.1809 28.4244 24.2038 28.3964C24.2268 28.3683 24.2431 28.3354 24.2514 28.3001C24.2598 28.2649 24.26 28.2282 24.252 28.1928L20.9685 15.3019C20.9541 15.2524 20.924 15.209 20.8827 15.178C20.8415 15.1471 20.7913 15.1304 20.7397 15.1304C20.6882 15.1304 20.638 15.1471 20.5968 15.178C20.5555 15.209 20.5254 15.2524 20.511 15.3019V15.3019Z" fill="#00E5E5"/>
+    <path d="M2.53051 18.2228L-5.28338e-06 28.1924C-0.00799016 28.2277 -0.00780431 28.2644 0.000538111 28.2997C0.00888053 28.335 0.0251596 28.3679 0.0481365 28.3959C0.0711133 28.4239 0.100182 28.4464 0.133131 28.4615C0.166079 28.4766 0.202039 28.484 0.238273 28.4831H5.28025C5.31649 28.484 5.35245 28.4766 5.38539 28.4615C5.41834 28.4464 5.44741 28.4239 5.47039 28.3959C5.49336 28.3679 5.50964 28.335 5.51799 28.2997C5.52633 28.2644 5.52651 28.2277 5.51853 28.1924L2.98563 18.2228C2.97054 18.1742 2.94032 18.1318 2.89938 18.1016C2.85844 18.0714 2.80892 18.0552 2.75807 18.0552C2.70722 18.0552 2.6577 18.0714 2.61676 18.1016C2.57582 18.1318 2.5456 18.1742 2.53051 18.2228V18.2228Z" fill="#00E5E5"/>
+    <path d="M6.99344 9.96839L2.38275 28.1919C2.37498 28.2263 2.37494 28.262 2.38262 28.2964C2.3903 28.3308 2.40552 28.363 2.42717 28.3908C2.44882 28.4186 2.47637 28.4413 2.50783 28.4572C2.53929 28.473 2.57388 28.4817 2.60911 28.4826H11.8329C11.8691 28.4835 11.9051 28.4761 11.938 28.461C11.971 28.4459 12 28.4235 12.023 28.3955C12.046 28.3675 12.0623 28.3345 12.0706 28.2993C12.079 28.264 12.0791 28.2273 12.0712 28.1919L7.44855 9.96839C7.43347 9.91982 7.40325 9.87736 7.36231 9.8472C7.32136 9.81705 7.27185 9.80078 7.221 9.80078C7.17015 9.80078 7.12063 9.81705 7.07969 9.8472C7.03874 9.87736 7.00852 9.91982 6.99344 9.96839Z" fill="#006EFF"/>
+    <path d="M14.9472 4.17346C14.9321 4.1249 14.9019 4.08244 14.861 4.05228C14.82 4.02213 14.7705 4.00586 14.7197 4.00586C14.6688 4.00586 14.6193 4.02213 14.5784 4.05228C14.5374 4.08244 14.5072 4.1249 14.4921 4.17346L8.18963 28.192C8.18165 28.2273 8.18183 28.264 8.19017 28.2993C8.19852 28.3346 8.2148 28.3675 8.23777 28.3955C8.26075 28.4235 8.28982 28.446 8.32277 28.4611C8.35572 28.4762 8.39168 28.4835 8.42791 28.4827H21.0233C21.0596 28.4835 21.0955 28.4762 21.1285 28.4611C21.1614 28.446 21.1905 28.4235 21.2135 28.3955C21.2364 28.3675 21.2527 28.3346 21.2611 28.2993C21.2694 28.264 21.2696 28.2273 21.2616 28.192L14.9472 4.17346Z" fill="#006EFF"/>
+    <path d="M10.3175 12.6188L6.31915 28.1903C6.31074 28.2258 6.31061 28.2628 6.31875 28.2984C6.3269 28.3339 6.34311 28.3672 6.36614 28.3955C6.38916 28.4238 6.41839 28.4465 6.45155 28.4617C6.48472 28.4769 6.52094 28.4844 6.55743 28.4834H14.535C14.5715 28.4844 14.6077 28.4769 14.6409 28.4617C14.674 28.4465 14.7033 28.4238 14.7263 28.3955C14.7493 28.3672 14.7655 28.3339 14.7737 28.2984C14.7818 28.2628 14.7817 28.2258 14.7733 28.1903L10.7726 12.6188C10.7575 12.5702 10.7273 12.5278 10.6863 12.4976C10.6454 12.4674 10.5959 12.4512 10.545 12.4512C10.4942 12.4512 10.4447 12.4674 10.4037 12.4976C10.3628 12.5278 10.3326 12.5702 10.3175 12.6188Z" fill="#00E5E5"/>
+  </g>
+</svg>

api/core/model_runtime/model_providers/volcengine_maas/_assets/icon_s_en.svg

@@ -0,0 +1,8 @@
+<?xml version="1.0" encoding="utf-8"?>
+<svg viewBox="-0.006 0 24.6978 24.9156" xmlns="http://www.w3.org/2000/svg">
+  <path d="M20.511 15.3019L17.2442 28.1928C17.2362 28.2282 17.2364 28.2649 17.2447 28.3001C17.2531 28.3354 17.2694 28.3683 17.2923 28.3964C17.3153 28.4244 17.3444 28.4468 17.3773 28.4619C17.4103 28.477 17.4462 28.4844 17.4825 28.4835H24.0137C24.0499 28.4844 24.0859 28.477 24.1188 28.4619C24.1518 28.4468 24.1809 28.4244 24.2038 28.3964C24.2268 28.3683 24.2431 28.3354 24.2514 28.3001C24.2598 28.2649 24.26 28.2282 24.252 28.1928L20.9685 15.3019C20.9541 15.2524 20.924 15.209 20.8827 15.178C20.8415 15.1471 20.7913 15.1304 20.7397 15.1304C20.6882 15.1304 20.638 15.1471 20.5968 15.178C20.5555 15.209 20.5254 15.2524 20.511 15.3019V15.3019Z" fill="#00E5E5" transform="matrix(1.0178890228271484, 0, 0, 1.0178890228271484, -1.952212187461555e-7, -4.077521402283104)"/>
+  <path d="M2.53051 18.2228L-5.28338e-06 28.1924C-0.00799016 28.2277 -0.00780431 28.2644 0.000538111 28.2997C0.00888053 28.335 0.0251596 28.3679 0.0481365 28.3959C0.0711133 28.4239 0.100182 28.4464 0.133131 28.4615C0.166079 28.4766 0.202039 28.484 0.238273 28.4831H5.28025C5.31649 28.484 5.35245 28.4766 5.38539 28.4615C5.41834 28.4464 5.44741 28.4239 5.47039 28.3959C5.49336 28.3679 5.50964 28.335 5.51799 28.2997C5.52633 28.2644 5.52651 28.2277 5.51853 28.1924L2.98563 18.2228C2.97054 18.1742 2.94032 18.1318 2.89938 18.1016C2.85844 18.0714 2.80892 18.0552 2.75807 18.0552C2.70722 18.0552 2.6577 18.0714 2.61676 18.1016C2.57582 18.1318 2.5456 18.1742 2.53051 18.2228V18.2228Z" fill="#00E5E5" transform="matrix(1.0178890228271484, 0, 0, 1.0178890228271484, -1.952212187461555e-7, -4.077521402283104)"/>
+  <path d="M6.99344 9.96839L2.38275 28.1919C2.37498 28.2263 2.37494 28.262 2.38262 28.2964C2.3903 28.3308 2.40552 28.363 2.42717 28.3908C2.44882 28.4186 2.47637 28.4413 2.50783 28.4572C2.53929 28.473 2.57388 28.4817 2.60911 28.4826H11.8329C11.8691 28.4835 11.9051 28.4761 11.938 28.461C11.971 28.4459 12 28.4235 12.023 28.3955C12.046 28.3675 12.0623 28.3345 12.0706 28.2993C12.079 28.264 12.0791 28.2273 12.0712 28.1919L7.44855 9.96839C7.43347 9.91982 7.40325 9.87736 7.36231 9.8472C7.32136 9.81705 7.27185 9.80078 7.221 9.80078C7.17015 9.80078 7.12063 9.81705 7.07969 9.8472C7.03874 9.87736 7.00852 9.91982 6.99344 9.96839Z" fill="#006EFF" transform="matrix(1.0178890228271484, 0, 0, 1.0178890228271484, -1.952212187461555e-7, -4.077521402283104)"/>
+  <path d="M14.9472 4.17346C14.9321 4.1249 14.9019 4.08244 14.861 4.05228C14.82 4.02213 14.7705 4.00586 14.7197 4.00586C14.6688 4.00586 14.6193 4.02213 14.5784 4.05228C14.5374 4.08244 14.5072 4.1249 14.4921 4.17346L8.18963 28.192C8.18165 28.2273 8.18183 28.264 8.19017 28.2993C8.19852 28.3346 8.2148 28.3675 8.23777 28.3955C8.26075 28.4235 8.28982 28.446 8.32277 28.4611C8.35572 28.4762 8.39168 28.4835 8.42791 28.4827H21.0233C21.0596 28.4835 21.0955 28.4762 21.1285 28.4611C21.1614 28.446 21.1905 28.4235 21.2135 28.3955C21.2364 28.3675 21.2527 28.3346 21.2611 28.2993C21.2694 28.264 21.2696 28.2273 21.2616 28.192L14.9472 4.17346Z" fill="#006EFF" transform="matrix(1.0178890228271484, 0, 0, 1.0178890228271484, -1.952212187461555e-7, -4.077521402283104)"/>
+  <path d="M10.3175 12.6188L6.31915 28.1903C6.31074 28.2258 6.31061 28.2628 6.31875 28.2984C6.3269 28.3339 6.34311 28.3672 6.36614 28.3955C6.38916 28.4238 6.41839 28.4465 6.45155 28.4617C6.48472 28.4769 6.52094 28.4844 6.55743 28.4834H14.535C14.5715 28.4844 14.6077 28.4769 14.6409 28.4617C14.674 28.4465 14.7033 28.4238 14.7263 28.3955C14.7493 28.3672 14.7655 28.3339 14.7737 28.2984C14.7818 28.2628 14.7817 28.2258 14.7733 28.1903L10.7726 12.6188C10.7575 12.5702 10.7273 12.5278 10.6863 12.4976C10.6454 12.4674 10.5959 12.4512 10.545 12.4512C10.4942 12.4512 10.4447 12.4674 10.4037 12.4976C10.3628 12.5278 10.3326 12.5702 10.3175 12.6188Z" fill="#00E5E5" transform="matrix(1.0178890228271484, 0, 0, 1.0178890228271484, -1.952212187461555e-7, -4.077521402283104)"/>
+</svg>

api/core/model_runtime/model_providers/volcengine_maas/client.py

@@ -0,0 +1,108 @@
+import re
+from collections.abc import Callable, Generator
+from typing import cast
+
+from core.model_runtime.entities.message_entities import (
+    AssistantPromptMessage,
+    ImagePromptMessageContent,
+    PromptMessage,
+    PromptMessageContentType,
+    SystemPromptMessage,
+    UserPromptMessage,
+)
+from core.model_runtime.model_providers.volcengine_maas.errors import wrap_error
+from core.model_runtime.model_providers.volcengine_maas.volc_sdk import ChatRole, MaasException, MaasService
+
+
+class MaaSClient(MaasService):
+    def __init__(self, host: str, region: str):
+        self.endpoint_id = None
+        super().__init__(host, region)
+
+    def set_endpoint_id(self, endpoint_id: str):
+        self.endpoint_id = endpoint_id
+
+    @classmethod
+    def from_credential(cls, credentials: dict) -> 'MaaSClient':
+        host = credentials['api_endpoint_host']
+        region = credentials['volc_region']
+        ak = credentials['volc_access_key_id']
+        sk = credentials['volc_secret_access_key']
+        endpoint_id = credentials['endpoint_id']
+
+        client = cls(host, region)
+        client.set_endpoint_id(endpoint_id)
+        client.set_ak(ak)
+        client.set_sk(sk)
+        return client
+
+    def chat(self, params: dict, messages: list[PromptMessage], stream=False) -> Generator | dict:
+        req = {
+            'parameters': params,
+            'messages': [self.convert_prompt_message_to_maas_message(prompt) for prompt in messages]
+        }
+        if not stream:
+            return super().chat(
+                self.endpoint_id,
+                req,
+            )
+        return super().stream_chat(
+            self.endpoint_id,
+            req,
+        )
+
+    def embeddings(self, texts: list[str]) -> dict:
+        req = {
+            'input': texts
+        }
+        return super().embeddings(self.endpoint_id, req)
+
+    @staticmethod
+    def convert_prompt_message_to_maas_message(message: PromptMessage) -> dict:
+        if isinstance(message, UserPromptMessage):
+            message = cast(UserPromptMessage, message)
+            if isinstance(message.content, str):
+                message_dict = {"role": ChatRole.USER,
+                                "content": message.content}
+            else:
+                content = []
+                for message_content in message.content:
+                    if message_content.type == PromptMessageContentType.TEXT:
+                        raise ValueError(
+                            'Content object type only support image_url')
+                    elif message_content.type == PromptMessageContentType.IMAGE:
+                        message_content = cast(
+                            ImagePromptMessageContent, message_content)
+                        image_data = re.sub(
+                            r'^data:image\/[a-zA-Z]+;base64,', '', message_content.data)
+                        content.append({
+                            'type': 'image_url',
+                            'image_url': {
+                                'url': '',
+                                'image_bytes': image_data,
+                                'detail': message_content.detail,
+                            }
+                        })
+
+                message_dict = {'role': ChatRole.USER, 'content': content}
+        elif isinstance(message, AssistantPromptMessage):
+            message = cast(AssistantPromptMessage, message)
+            message_dict = {'role': ChatRole.ASSISTANT,
+                            'content': message.content}
+        elif isinstance(message, SystemPromptMessage):
+            message = cast(SystemPromptMessage, message)
+            message_dict = {'role': ChatRole.SYSTEM,
+                            'content': message.content}
+        else:
+            raise ValueError(f"Got unknown PromptMessage type {message}")
+
+        return message_dict
+
+    @staticmethod
+    def wrap_exception(fn: Callable[[], dict | Generator]) -> dict | Generator:
+        try:
+            resp = fn()
+        except MaasException as e:
+            raise wrap_error(e)
+
+        return resp

api/core/model_runtime/model_providers/volcengine_maas/errors.py

@@ -0,0 +1,156 @@
+from core.model_runtime.model_providers.volcengine_maas.volc_sdk import MaasException
+
+
+class ClientSDKRequestError(MaasException):
+    pass
+
+
+class SignatureDoesNotMatch(MaasException):
+    pass
+
+
+class RequestTimeout(MaasException):
+    pass
+
+
+class ServiceConnectionTimeout(MaasException):
+    pass
+
+
+class MissingAuthenticationHeader(MaasException):
+    pass
+
+
+class AuthenticationHeaderIsInvalid(MaasException):
+    pass
+
+
+class InternalServiceError(MaasException):
+    pass
+
+
+class MissingParameter(MaasException):
+    pass
+
+
+class InvalidParameter(MaasException):
+    pass
+
+
+class AuthenticationExpire(MaasException):
+    pass
+
+
+class EndpointIsInvalid(MaasException):
+    pass
+
+
+class EndpointIsNotEnable(MaasException):
+    pass
+
+
+class ModelNotSupportStreamMode(MaasException):
+    pass
+
+
+class ReqTextExistRisk(MaasException):
+    pass
+
+
+class RespTextExistRisk(MaasException):
+    pass
+
+
+class EndpointRateLimitExceeded(MaasException):
+    pass
+
+
+class ServiceConnectionRefused(MaasException):
+    pass
+
+
+class ServiceConnectionClosed(MaasException):
+    pass
+
+
+class UnauthorizedUserForEndpoint(MaasException):
+    pass
+
+
+class InvalidEndpointWithNoURL(MaasException):
+    pass
+
+
+class EndpointAccountRpmRateLimitExceeded(MaasException):
+    pass
+
+
+class EndpointAccountTpmRateLimitExceeded(MaasException):
+    pass
+
+
+class ServiceResourceWaitQueueFull(MaasException):
+    pass
+
+
+class EndpointIsPending(MaasException):
+    pass
+
+
+class ServiceNotOpen(MaasException):
+    pass
+
+
+AuthErrors = {
+    'SignatureDoesNotMatch': SignatureDoesNotMatch,
+    'MissingAuthenticationHeader': MissingAuthenticationHeader,
+    'AuthenticationHeaderIsInvalid': AuthenticationHeaderIsInvalid,
+    'AuthenticationExpire': AuthenticationExpire,
+    'UnauthorizedUserForEndpoint': UnauthorizedUserForEndpoint,
+}
+
+BadRequestErrors = {
+    'MissingParameter': MissingParameter,
+    'InvalidParameter': InvalidParameter,
+    'EndpointIsInvalid': EndpointIsInvalid,
+    'EndpointIsNotEnable': EndpointIsNotEnable,
+    'ModelNotSupportStreamMode': ModelNotSupportStreamMode,
+    'ReqTextExistRisk': ReqTextExistRisk,
+    'RespTextExistRisk': RespTextExistRisk,
+    'InvalidEndpointWithNoURL': InvalidEndpointWithNoURL,
+    'ServiceNotOpen': ServiceNotOpen,
+}
+
+RateLimitErrors = {
+    'EndpointRateLimitExceeded': EndpointRateLimitExceeded,
+    'EndpointAccountRpmRateLimitExceeded': EndpointAccountRpmRateLimitExceeded,
+    'EndpointAccountTpmRateLimitExceeded': EndpointAccountTpmRateLimitExceeded,
+}
+
+ServerUnavailableErrors = {
+    'InternalServiceError': InternalServiceError,
+    'EndpointIsPending': EndpointIsPending,
+    'ServiceResourceWaitQueueFull': ServiceResourceWaitQueueFull,
+}
+
+ConnectionErrors = {
+    'ClientSDKRequestError': ClientSDKRequestError,
+    'RequestTimeout': RequestTimeout,
+    'ServiceConnectionTimeout': ServiceConnectionTimeout,
+    'ServiceConnectionRefused': ServiceConnectionRefused,
+    'ServiceConnectionClosed': ServiceConnectionClosed,
+}
+
+ErrorCodeMap = {
+    **AuthErrors,
+    **BadRequestErrors,
+    **RateLimitErrors,
+    **ServerUnavailableErrors,
+    **ConnectionErrors,
+}
+
+
+def wrap_error(e: MaasException) -> Exception:
+    if ErrorCodeMap.get(e.code):
+        return ErrorCodeMap.get(e.code)(e.code_n, e.code, e.message, e.req_id)
+    return e

api/core/model_runtime/model_providers/volcengine_maas/llm/llm.py

@@ -0,0 +1,284 @@
+import logging
+from collections.abc import Generator
+
+from core.model_runtime.entities.common_entities import I18nObject
+from core.model_runtime.entities.llm_entities import LLMResult, LLMResultChunk, LLMResultChunkDelta, LLMUsage
+from core.model_runtime.entities.message_entities import (
+    AssistantPromptMessage,
+    PromptMessage,
+    PromptMessageTool,
+    UserPromptMessage,
+)
+from core.model_runtime.entities.model_entities import (
+    AIModelEntity,
+    FetchFrom,
+    ModelPropertyKey,
+    ModelType,
+    ParameterRule,
+    ParameterType,
+)
+from core.model_runtime.errors.invoke import (
+    InvokeAuthorizationError,
+    InvokeBadRequestError,
+    InvokeConnectionError,
+    InvokeError,
+    InvokeRateLimitError,
+    InvokeServerUnavailableError,
+)
+from core.model_runtime.errors.validate import CredentialsValidateFailedError
+from core.model_runtime.model_providers.__base.large_language_model import LargeLanguageModel
+from core.model_runtime.model_providers.volcengine_maas.client import MaaSClient
+from core.model_runtime.model_providers.volcengine_maas.errors import (
+    AuthErrors,
+    BadRequestErrors,
+    ConnectionErrors,
+    RateLimitErrors,
+    ServerUnavailableErrors,
+)
+from core.model_runtime.model_providers.volcengine_maas.llm.models import ModelConfigs
+from core.model_runtime.model_providers.volcengine_maas.volc_sdk import MaasException
+
+logger = logging.getLogger(__name__)
+
+
+class VolcengineMaaSLargeLanguageModel(LargeLanguageModel):
+    def _invoke(self, model: str, credentials: dict, prompt_messages: list[PromptMessage],
+                model_parameters: dict, tools: list[PromptMessageTool] | None = None,
+                stop: list[str] | None = None, stream: bool = True, user: str | None = None) \
+            -> LLMResult | Generator:
+        return self._generate(model, credentials, prompt_messages, model_parameters, tools, stop, stream, user)
+
+    def validate_credentials(self, model: str, credentials: dict) -> None:
+        """
+        Validate credentials
+        """
+        # ping
+        client = MaaSClient.from_credential(credentials)
+        try:
+            client.chat(
+                {
+                    'max_new_tokens': 16,
+                    'temperature': 0.7,
+                    'top_p': 0.9,
+                    'top_k': 15,
+                },
+                [UserPromptMessage(content='ping\nAnswer: ')],
+            )
+        except MaasException as e:
+            raise CredentialsValidateFailedError(e.message)
+
+    def get_num_tokens(self, model: str, credentials: dict, prompt_messages: list[PromptMessage],
+                       tools: list[PromptMessageTool] | None = None) -> int:
+        if len(prompt_messages) == 0:
+            return 0
+        return self._num_tokens_from_messages(prompt_messages)
+
+    def _num_tokens_from_messages(self, messages: list[PromptMessage]) -> int:
+        """
+        Calculate num tokens.
+
+        :param messages: messages
+        """
+        num_tokens = 0
+        messages_dict = [
+            MaaSClient.convert_prompt_message_to_maas_message(m) for m in messages]
+        for message in messages_dict:
+            for key, value in message.items():
+                num_tokens += self._get_num_tokens_by_gpt2(str(key))
+                num_tokens += self._get_num_tokens_by_gpt2(str(value))
+
+        return num_tokens
+
+    def _generate(self, model: str, credentials: dict, prompt_messages: list[PromptMessage],
+                  model_parameters: dict, tools: list[PromptMessageTool] | None = None,
+                  stop: list[str] | None = None, stream: bool = True, user: str | None = None) \
+            -> LLMResult | Generator:
+
+        client = MaaSClient.from_credential(credentials)
+
+        req_params = ModelConfigs.get(
+            credentials['base_model_name'], {}).get('req_params', {}).copy()
+        if credentials.get('context_size'):
+            req_params['max_prompt_tokens'] = credentials.get('context_size')
+        if credentials.get('max_tokens'):
+            req_params['max_new_tokens'] = credentials.get('max_tokens')
+        if model_parameters.get('max_tokens'):
+            req_params['max_new_tokens'] = model_parameters.get('max_tokens')
+        if model_parameters.get('temperature'):
+            req_params['temperature'] = model_parameters.get('temperature')
+        if model_parameters.get('top_p'):
+            req_params['top_p'] = model_parameters.get('top_p')
+        if model_parameters.get('top_k'):
+            req_params['top_k'] = model_parameters.get('top_k')
+        if model_parameters.get('presence_penalty'):
+            req_params['presence_penalty'] = model_parameters.get(
+                'presence_penalty')
+        if model_parameters.get('frequency_penalty'):
+            req_params['frequency_penalty'] = model_parameters.get(
+                'frequency_penalty')
+        if stop:
+            req_params['stop'] = stop
+
+        resp = MaaSClient.wrap_exception(
+            lambda: client.chat(req_params, prompt_messages, stream))
+        if not stream:
+            return self._handle_chat_response(model, credentials, prompt_messages, resp)
+        return self._handle_stream_chat_response(model, credentials, prompt_messages, resp)
+
+    def _handle_stream_chat_response(self, model: str, credentials: dict, prompt_messages: list[PromptMessage], resp: Generator) -> Generator:
+        for index, r in enumerate(resp):
+            choices = r['choices']
+            if not choices:
+                continue
+            choice = choices[0]
+            message = choice['message']
+            usage = None
+            if r.get('usage'):
+                usage = self._calc_usage(model, credentials, r['usage'])
+            yield LLMResultChunk(
+                model=model,
+                prompt_messages=prompt_messages,
+                delta=LLMResultChunkDelta(
+                    index=index,
+                    message=AssistantPromptMessage(
+                        content=message['content'] if message['content'] else '',
+                        tool_calls=[]
+                    ),
+                    usage=usage,
+                    finish_reason=choice.get('finish_reason'),
+                ),
+            )
+
+    def _handle_chat_response(self,  model: str, credentials: dict, prompt_messages: list[PromptMessage], resp: dict) -> LLMResult:
+        choices = resp['choices']
+        if not choices:
+            return
+        choice = choices[0]
+        message = choice['message']
+
+        return LLMResult(
+            model=model,
+            prompt_messages=prompt_messages,
+            message=AssistantPromptMessage(
+                content=message['content'] if message['content'] else '',
+                tool_calls=[],
+            ),
+            usage=self._calc_usage(model, credentials, resp['usage']),
+        )
+
+    def _calc_usage(self,  model: str, credentials: dict, usage: dict) -> LLMUsage:
+        return self._calc_response_usage(model=model, credentials=credentials,
+                                         prompt_tokens=usage['prompt_tokens'],
+                                         completion_tokens=usage['completion_tokens']
+                                         )
+
+    def get_customizable_model_schema(self, model: str, credentials: dict) -> AIModelEntity | None:
+        """
+            used to define customizable model schema
+        """
+        max_tokens = ModelConfigs.get(
+            credentials['base_model_name'], {}).get('req_params', {}).get('max_new_tokens')
+        if credentials.get('max_tokens'):
+            max_tokens = int(credentials.get('max_tokens'))
+        rules = [
+            ParameterRule(
+                name='temperature',
+                type=ParameterType.FLOAT,
+                use_template='temperature',
+                label=I18nObject(
+                    zh_Hans='温度',
+                    en_US='Temperature'
+                )
+            ),
+            ParameterRule(
+                name='top_p',
+                type=ParameterType.FLOAT,
+                use_template='top_p',
+                label=I18nObject(
+                    zh_Hans='Top P',
+                    en_US='Top P'
+                )
+            ),
+            ParameterRule(
+                name='top_k',
+                type=ParameterType.INT,
+                min=1,
+                default=1,
+                label=I18nObject(
+                    zh_Hans='Top K',
+                    en_US='Top K'
+                )
+            ),
+            ParameterRule(
+                name='presence_penalty',
+                type=ParameterType.FLOAT,
+                use_template='presence_penalty',
+                label={
+                    'en_US': 'Presence Penalty',
+                    'zh_Hans': '存在惩罚',
+                },
+                min=-2.0,
+                max=2.0,
+            ),
+            ParameterRule(
+                name='frequency_penalty',
+                type=ParameterType.FLOAT,
+                use_template='frequency_penalty',
+                label={
+                    'en_US': 'Frequency Penalty',
+                    'zh_Hans': '频率惩罚',
+                },
+                min=-2.0,
+                max=2.0,
+            ),
+            ParameterRule(
+                name='max_tokens',
+                type=ParameterType.INT,
+                use_template='max_tokens',
+                min=1,
+                max=max_tokens,
+                default=512,
+                label=I18nObject(
+                    zh_Hans='最大生成长度',
+                    en_US='Max Tokens'
+                )
+            ),
+        ]
+
+        model_properties = ModelConfigs.get(
+            credentials['base_model_name'], {}).get('model_properties', {}).copy()
+        if credentials.get('mode'):
+            model_properties[ModelPropertyKey.MODE] = credentials.get('mode')
+        if credentials.get('context_size'):
+            model_properties[ModelPropertyKey.CONTEXT_SIZE] = int(
+                credentials.get('context_size', 4096))
+        entity = AIModelEntity(
+            model=model,
+            label=I18nObject(
+                en_US=model
+            ),
+            fetch_from=FetchFrom.CUSTOMIZABLE_MODEL,
+            model_type=ModelType.LLM,
+            model_properties=model_properties,
+            parameter_rules=rules
+        )
+
+        return entity
+
+    @property
+    def _invoke_error_mapping(self) -> dict[type[InvokeError], list[type[Exception]]]:
+        """
+        Map model invoke error to unified error
+        The key is the error type thrown to the caller
+        The value is the error type thrown by the model,
+        which needs to be converted into a unified error type for the caller.
+
+        :return: Invoke error mapping
+        """
+        return {
+            InvokeConnectionError: ConnectionErrors.values(),
+            InvokeServerUnavailableError: ServerUnavailableErrors.values(),
+            InvokeRateLimitError: RateLimitErrors.values(),
+            InvokeAuthorizationError: AuthErrors.values(),
+            InvokeBadRequestError: BadRequestErrors.values(),
+        }

api/core/model_runtime/model_providers/volcengine_maas/llm/models.py

@@ -0,0 +1,12 @@
+ModelConfigs = {
+    'Skylark2-pro-4k': {
+        'req_params': {
+            'max_prompt_tokens': 4096,
+            'max_new_tokens': 4000,
+        },
+        'model_properties': {
+            'context_size': 4096,
+            'mode': 'chat',
+        }
+    }
+}

api/core/model_runtime/model_providers/volcengine_maas/text_embedding/text_embedding.py

@@ -0,0 +1,132 @@
+import time
+from typing import Optional
+
+from core.model_runtime.entities.model_entities import PriceType
+from core.model_runtime.entities.text_embedding_entities import EmbeddingUsage, TextEmbeddingResult
+from core.model_runtime.errors.invoke import (
+    InvokeAuthorizationError,
+    InvokeBadRequestError,
+    InvokeConnectionError,
+    InvokeError,
+    InvokeRateLimitError,
+    InvokeServerUnavailableError,
+)
+from core.model_runtime.errors.validate import CredentialsValidateFailedError
+from core.model_runtime.model_providers.__base.text_embedding_model import TextEmbeddingModel
+from core.model_runtime.model_providers.volcengine_maas.client import MaaSClient
+from core.model_runtime.model_providers.volcengine_maas.errors import (
+    AuthErrors,
+    BadRequestErrors,
+    ConnectionErrors,
+    RateLimitErrors,
+    ServerUnavailableErrors,
+)
+from core.model_runtime.model_providers.volcengine_maas.volc_sdk import MaasException
+
+
+class VolcengineMaaSTextEmbeddingModel(TextEmbeddingModel):
+    """
+    Model class for VolcengineMaaS text embedding model.
+    """
+
+    def _invoke(self, model: str, credentials: dict,
+                texts: list[str], user: Optional[str] = None) \
+            -> TextEmbeddingResult:
+        """
+        Invoke text embedding model
+
+        :param model: model name
+        :param credentials: model credentials
+        :param texts: texts to embed
+        :param user: unique user id
+        :return: embeddings result
+        """
+        client = MaaSClient.from_credential(credentials)
+        resp = MaaSClient.wrap_exception(lambda: client.embeddings(texts))
+
+        usage = self._calc_response_usage(
+            model=model, credentials=credentials, tokens=resp['total_tokens'])
+
+        result = TextEmbeddingResult(
+            model=model,
+            embeddings=[v['embedding'] for v in resp['data']],
+            usage=usage
+        )
+
+        return result
+
+    def get_num_tokens(self, model: str, credentials: dict, texts: list[str]) -> int:
+        """
+        Get number of tokens for given prompt messages
+
+        :param model: model name
+        :param credentials: model credentials
+        :param texts: texts to embed
+        :return:
+        """
+        num_tokens = 0
+        for text in texts:
+            # use GPT2Tokenizer to get num tokens
+            num_tokens += self._get_num_tokens_by_gpt2(text)
+        return num_tokens
+
+    def validate_credentials(self, model: str, credentials: dict) -> None:
+        """
+        Validate model credentials
+
+        :param model: model name
+        :param credentials: model credentials
+        :return:
+        """
+        try:
+            self._invoke(model=model, credentials=credentials, texts=['ping'])
+        except MaasException as e:
+            raise CredentialsValidateFailedError(e.message)
+
+    @property
+    def _invoke_error_mapping(self) -> dict[type[InvokeError], list[type[Exception]]]:
+        """
+        Map model invoke error to unified error
+        The key is the error type thrown to the caller
+        The value is the error type thrown by the model,
+        which needs to be converted into a unified error type for the caller.
+
+        :return: Invoke error mapping
+        """
+        return {
+            InvokeConnectionError: ConnectionErrors.values(),
+            InvokeServerUnavailableError: ServerUnavailableErrors.values(),
+            InvokeRateLimitError: RateLimitErrors.values(),
+            InvokeAuthorizationError: AuthErrors.values(),
+            InvokeBadRequestError: BadRequestErrors.values(),
+        }
+
+    def _calc_response_usage(self, model: str, credentials: dict, tokens: int) -> EmbeddingUsage:
+        """
+        Calculate response usage
+
+        :param model: model name
+        :param credentials: model credentials
+        :param tokens: input tokens
+        :return: usage
+        """
+        # get input price info
+        input_price_info = self.get_price(
+            model=model,
+            credentials=credentials,
+            price_type=PriceType.INPUT,
+            tokens=tokens
+        )
+
+        # transform usage
+        usage = EmbeddingUsage(
+            tokens=tokens,
+            total_tokens=tokens,
+            unit_price=input_price_info.unit_price,
+            price_unit=input_price_info.unit,
+            total_price=input_price_info.total_amount,
+            currency=input_price_info.currency,
+            latency=time.perf_counter() - self.started_at
+        )
+
+        return usage

api/core/model_runtime/model_providers/volcengine_maas/volc_sdk/__init__.py

@@ -0,0 +1,4 @@
+from .common import ChatRole
+from .maas import MaasException, MaasService
+
+__all__ = ['MaasService', 'ChatRole', 'MaasException']

api/core/model_runtime/model_providers/volcengine_maas/volc_sdk/base/__init__.py

@@ -0,0 +1 @@
+

api/core/model_runtime/model_providers/volcengine_maas/volc_sdk/base/auth.py

@@ -0,0 +1,144 @@
+# coding : utf-8
+import datetime
+
+import pytz
+
+from .util import Util
+
+
+class MetaData:
+    def __init__(self):
+        self.algorithm = ''
+        self.credential_scope = ''
+        self.signed_headers = ''
+        self.date = ''
+        self.region = ''
+        self.service = ''
+
+    def set_date(self, date):
+        self.date = date
+
+    def set_service(self, service):
+        self.service = service
+
+    def set_region(self, region):
+        self.region = region
+
+    def set_algorithm(self, algorithm):
+        self.algorithm = algorithm
+
+    def set_credential_scope(self, credential_scope):
+        self.credential_scope = credential_scope
+
+    def set_signed_headers(self, signed_headers):
+        self.signed_headers = signed_headers
+
+
+class SignResult:
+    def __init__(self):
+        self.xdate = ''
+        self.xCredential = ''
+        self.xAlgorithm = ''
+        self.xSignedHeaders = ''
+        self.xSignedQueries = ''
+        self.xSignature = ''
+        self.xContextSha256 = ''
+        self.xSecurityToken = ''
+
+        self.authorization = ''
+
+    def __str__(self):
+        return '\n'.join(['{}:{}'.format(*item) for item in self.__dict__.items()])
+
+
+class Credentials:
+    def __init__(self, ak, sk, service, region, session_token=''):
+        self.ak = ak
+        self.sk = sk
+        self.service = service
+        self.region = region
+        self.session_token = session_token
+
+    def set_ak(self, ak):
+        self.ak = ak
+
+    def set_sk(self, sk):
+        self.sk = sk
+
+    def set_session_token(self, session_token):
+        self.session_token = session_token
+
+
+class Signer:
+    @staticmethod
+    def sign(request, credentials):
+        if request.path == '':
+            request.path = '/'
+        if request.method != 'GET' and not ('Content-Type' in request.headers):
+            request.headers['Content-Type'] = 'application/x-www-form-urlencoded; charset=utf-8'
+
+        format_date = Signer.get_current_format_date()
+        request.headers['X-Date'] = format_date
+        if credentials.session_token != '':
+            request.headers['X-Security-Token'] = credentials.session_token
+
+        md = MetaData()
+        md.set_algorithm('HMAC-SHA256')
+        md.set_service(credentials.service)
+        md.set_region(credentials.region)
+        md.set_date(format_date[:8])
+
+        hashed_canon_req = Signer.hashed_canonical_request_v4(request, md)
+        md.set_credential_scope('/'.join([md.date, md.region, md.service, 'request']))
+
+        signing_str = '\n'.join([md.algorithm, format_date, md.credential_scope, hashed_canon_req])
+        signing_key = Signer.get_signing_secret_key_v4(credentials.sk, md.date, md.region, md.service)
+        sign = Util.to_hex(Util.hmac_sha256(signing_key, signing_str))
+        request.headers['Authorization'] = Signer.build_auth_header_v4(sign, md, credentials)
+        return
+
+    @staticmethod
+    def hashed_canonical_request_v4(request, meta):
+        body_hash = Util.sha256(request.body)
+        request.headers['X-Content-Sha256'] = body_hash
+
+        signed_headers = dict()
+        for key in request.headers:
+            if key in ['Content-Type', 'Content-Md5', 'Host'] or key.startswith('X-'):
+                signed_headers[key.lower()] = request.headers[key]
+
+        if 'host' in signed_headers:
+            v = signed_headers['host']
+            if v.find(':') != -1:
+                split = v.split(':')
+                port = split[1]
+                if str(port) == '80' or str(port) == '443':
+                    signed_headers['host'] = split[0]
+
+        signed_str = ''
+        for key in sorted(signed_headers.keys()):
+            signed_str += key + ':' + signed_headers[key] + '\n'
+
+        meta.set_signed_headers(';'.join(sorted(signed_headers.keys())))
+
+        canonical_request = '\n'.join(
+            [request.method, Util.norm_uri(request.path), Util.norm_query(request.query), signed_str,
+             meta.signed_headers, body_hash])
+
+        return Util.sha256(canonical_request)
+
+    @staticmethod
+    def get_signing_secret_key_v4(sk, date, region, service):
+        date = Util.hmac_sha256(bytes(sk, encoding='utf-8'), date)
+        region = Util.hmac_sha256(date, region)
+        service = Util.hmac_sha256(region, service)
+        return Util.hmac_sha256(service, 'request')
+
+    @staticmethod
+    def build_auth_header_v4(signature, meta, credentials):
+        credential = credentials.ak + '/' + meta.credential_scope
+        return meta.algorithm + ' Credential=' + credential + ', SignedHeaders=' + meta.signed_headers + ', Signature=' + signature
+
+    @staticmethod
+    def get_current_format_date():
+        return datetime.datetime.now(tz=pytz.timezone('UTC')).strftime("%Y%m%dT%H%M%SZ")

api/core/model_runtime/model_providers/volcengine_maas/volc_sdk/base/service.py

@@ -0,0 +1,207 @@
+import json
+from collections import OrderedDict
+from urllib.parse import urlencode
+
+import requests
+
+from .auth import Signer
+
+VERSION = 'v1.0.137'
+
+
+class Service:
+    def __init__(self, service_info, api_info):
+        self.service_info = service_info
+        self.api_info = api_info
+        self.session = requests.session()
+
+    def set_ak(self, ak):
+        self.service_info.credentials.set_ak(ak)
+
+    def set_sk(self, sk):
+        self.service_info.credentials.set_sk(sk)
+
+    def set_session_token(self, session_token):
+        self.service_info.credentials.set_session_token(session_token)
+
+    def set_host(self, host):
+        self.service_info.host = host
+
+    def set_scheme(self, scheme):
+        self.service_info.scheme = scheme
+
+    def get(self, api, params, doseq=0):
+        if not (api in self.api_info):
+            raise Exception("no such api")
+        api_info = self.api_info[api]
+
+        r = self.prepare_request(api_info, params, doseq)
+
+        Signer.sign(r, self.service_info.credentials)
+
+        url = r.build(doseq)
+        resp = self.session.get(url, headers=r.headers,
+                                timeout=(self.service_info.connection_timeout, self.service_info.socket_timeout))
+        if resp.status_code == 200:
+            return resp.text
+        else:
+            raise Exception(resp.text)
+
+    def post(self, api, params, form):
+        if not (api in self.api_info):
+            raise Exception("no such api")
+        api_info = self.api_info[api]
+        r = self.prepare_request(api_info, params)
+        r.headers['Content-Type'] = 'application/x-www-form-urlencoded'
+        r.form = self.merge(api_info.form, form)
+        r.body = urlencode(r.form, True)
+        Signer.sign(r, self.service_info.credentials)
+
+        url = r.build()
+
+        resp = self.session.post(url, headers=r.headers, data=r.form,
+                                 timeout=(self.service_info.connection_timeout, self.service_info.socket_timeout))
+        if resp.status_code == 200:
+            return resp.text
+        else:
+            raise Exception(resp.text)
+
+    def json(self, api, params, body):
+        if not (api in self.api_info):
+            raise Exception("no such api")
+        api_info = self.api_info[api]
+        r = self.prepare_request(api_info, params)
+        r.headers['Content-Type'] = 'application/json'
+        r.body = body
+
+        Signer.sign(r, self.service_info.credentials)
+
+        url = r.build()
+        resp = self.session.post(url, headers=r.headers, data=r.body,
+                                 timeout=(self.service_info.connection_timeout, self.service_info.socket_timeout))
+        if resp.status_code == 200:
+            return json.dumps(resp.json())
+        else:
+            raise Exception(resp.text.encode("utf-8"))
+
+    def put(self, url, file_path, headers):
+        with open(file_path, 'rb') as f:
+            resp = self.session.put(url, headers=headers, data=f)
+            if resp.status_code == 200:
+                return True, resp.text.encode("utf-8")
+            else:
+                return False, resp.text.encode("utf-8")
+
+    def put_data(self, url, data, headers):
+        resp = self.session.put(url, headers=headers, data=data)
+        if resp.status_code == 200:
+            return True, resp.text.encode("utf-8")
+        else:
+            return False, resp.text.encode("utf-8")
+
+    def prepare_request(self, api_info, params, doseq=0):
+        for key in params:
+            if type(params[key]) == int or type(params[key]) == float or type(params[key]) == bool:
+                params[key] = str(params[key])
+            elif type(params[key]) == list:
+                if not doseq:
+                    params[key] = ','.join(params[key])
+
+        connection_timeout = self.service_info.connection_timeout
+        socket_timeout = self.service_info.socket_timeout
+
+        r = Request()
+        r.set_schema(self.service_info.scheme)
+        r.set_method(api_info.method)
+        r.set_connection_timeout(connection_timeout)
+        r.set_socket_timeout(socket_timeout)
+
+        headers = self.merge(api_info.header, self.service_info.header)
+        headers['Host'] = self.service_info.host
+        headers['User-Agent'] = 'volc-sdk-python/' + VERSION
+        r.set_headers(headers)
+
+        query = self.merge(api_info.query, params)
+        r.set_query(query)
+
+        r.set_host(self.service_info.host)
+        r.set_path(api_info.path)
+
+        return r
+
+    @staticmethod
+    def merge(param1, param2):
+        od = OrderedDict()
+        for key in param1:
+            od[key] = param1[key]
+
+        for key in param2:
+            od[key] = param2[key]
+
+        return od
+
+
+class Request:
+    def __init__(self):
+        self.schema = ''
+        self.method = ''
+        self.host = ''
+        self.path = ''
+        self.headers = OrderedDict()
+        self.query = OrderedDict()
+        self.body = ''
+        self.form = dict()
+        self.connection_timeout = 0
+        self.socket_timeout = 0
+
+    def set_schema(self, schema):
+        self.schema = schema
+
+    def set_method(self, method):
+        self.method = method
+
+    def set_host(self, host):
+        self.host = host
+
+    def set_path(self, path):
+        self.path = path
+
+    def set_headers(self, headers):
+        self.headers = headers
+
+    def set_query(self, query):
+        self.query = query
+
+    def set_body(self, body):
+        self.body = body
+
+    def set_connection_timeout(self, connection_timeout):
+        self.connection_timeout = connection_timeout
+
+    def set_socket_timeout(self, socket_timeout):
+        self.socket_timeout = socket_timeout
+
+    def build(self, doseq=0):
+        return self.schema + '://' + self.host + self.path + '?' + urlencode(self.query, doseq)
+
+
+class ServiceInfo:
+    def __init__(self, host, header, credentials, connection_timeout, socket_timeout, scheme='http'):
+        self.host = host
+        self.header = header
+        self.credentials = credentials
+        self.connection_timeout = connection_timeout
+        self.socket_timeout = socket_timeout
+        self.scheme = scheme
+
+
+class ApiInfo:
+    def __init__(self, method, path, query, form, header):
+        self.method = method
+        self.path = path
+        self.query = query
+        self.form = form
+        self.header = header
+
+    def __str__(self):
+        return 'method: ' + self.method + ', path: ' + self.path

api/core/model_runtime/model_providers/volcengine_maas/volc_sdk/base/util.py

@@ -0,0 +1,43 @@
+import hashlib
+import hmac
+from functools import reduce
+from urllib.parse import quote
+
+
+class Util:
+    @staticmethod
+    def norm_uri(path):
+        return quote(path).replace('%2F', '/').replace('+', '%20')
+
+    @staticmethod
+    def norm_query(params):
+        query = ''
+        for key in sorted(params.keys()):
+            if type(params[key]) == list:
+                for k in params[key]:
+                    query = query + quote(key, safe='-_.~') + '=' + quote(k, safe='-_.~') + '&'
+            else:
+                query = query + quote(key, safe='-_.~') + '=' + quote(params[key], safe='-_.~') + '&'
+        query = query[:-1]
+        return query.replace('+', '%20')
+
+    @staticmethod
+    def hmac_sha256(key, content):
+        return hmac.new(key, bytes(content, encoding='utf-8'), hashlib.sha256).digest()
+
+    @staticmethod
+    def sha256(content):
+        if isinstance(content, str) is True:
+            return hashlib.sha256(content.encode('utf-8')).hexdigest()
+        else:
+            return hashlib.sha256(content).hexdigest()
+
+    @staticmethod
+    def to_hex(content):
+        lst = []
+        for ch in content:
+            hv = hex(ch).replace('0x', '')
+            if len(hv) == 1:
+                hv = '0' + hv
+            lst.append(hv)
+        return reduce(lambda x, y: x + y, lst)

api/core/model_runtime/model_providers/volcengine_maas/volc_sdk/common.py

@@ -0,0 +1,79 @@
+import json
+import random
+from datetime import datetime
+
+
+class ChatRole:
+    USER = "user"
+    ASSISTANT = "assistant"
+    SYSTEM = "system"
+    FUNCTION = "function"
+
+
+class _Dict(dict):
+    __setattr__ = dict.__setitem__
+    __getattr__ = dict.__getitem__
+
+    def __missing__(self, key):
+        return None
+
+
+def dict_to_object(dict_obj):
+    # 支持嵌套类型
+    if isinstance(dict_obj, list):
+        insts = []
+        for i in dict_obj:
+            insts.append(dict_to_object(i))
+        return insts
+
+    if isinstance(dict_obj, dict):
+        inst = _Dict()
+        for k, v in dict_obj.items():
+            inst[k] = dict_to_object(v)
+        return inst
+
+    return dict_obj
+
+
+def json_to_object(json_str, req_id=None):
+    obj = dict_to_object(json.loads(json_str))
+    if obj and isinstance(obj, dict) and req_id:
+        obj["req_id"] = req_id
+    return obj
+
+
+def gen_req_id():
+    return datetime.now().strftime("%Y%m%d%H%M%S") + format(
+        random.randint(0, 2 ** 64 - 1), "020X"
+    )
+
+
+class SSEDecoder:
+    def __init__(self, source):
+        self.source = source
+
+    def _read(self):
+        data = b''
+        for chunk in self.source:
+            for line in chunk.splitlines(True):
+                data += line
+                if data.endswith((b'\r\r', b'\n\n', b'\r\n\r\n')):
+                    yield data
+                    data = b''
+        if data:
+            yield data
+
+    def next(self):
+        for chunk in self._read():
+            for line in chunk.splitlines():
+                # skip comment
+                if line.startswith(b':'):
+                    continue
+
+                if b':' in line:
+                    field, value = line.split(b':', 1)
+                else:
+                    field, value = line, b''
+
+                if field == b'data' and len(value) > 0:
+                    yield value

api/core/model_runtime/model_providers/volcengine_maas/volc_sdk/maas.py

@@ -0,0 +1,213 @@
+import copy
+import json
+from collections.abc import Iterator
+
+from .base.auth import Credentials, Signer
+from .base.service import ApiInfo, Service, ServiceInfo
+from .common import SSEDecoder, dict_to_object, gen_req_id, json_to_object
+
+
+class MaasService(Service):
+    def __init__(self, host, region, connection_timeout=60, socket_timeout=60):
+        service_info = self.get_service_info(
+            host, region, connection_timeout, socket_timeout
+        )
+        self._apikey = None
+        api_info = self.get_api_info()
+        super().__init__(service_info, api_info)
+
+    def set_apikey(self, apikey):
+        self._apikey = apikey
+
+    @staticmethod
+    def get_service_info(host, region, connection_timeout, socket_timeout):
+        service_info = ServiceInfo(
+            host,
+            {"Accept": "application/json"},
+            Credentials("", "", "ml_maas", region),
+            connection_timeout,
+            socket_timeout,
+            "https",
+        )
+        return service_info
+
+    @staticmethod
+    def get_api_info():
+        api_info = {
+            "chat": ApiInfo("POST", "/api/v2/endpoint/{endpoint_id}/chat", {}, {}, {}),
+            "embeddings": ApiInfo(
+                "POST", "/api/v2/endpoint/{endpoint_id}/embeddings", {}, {}, {}
+            ),
+        }
+        return api_info
+
+    def chat(self, endpoint_id, req):
+        req["stream"] = False
+        return self._request(endpoint_id, "chat", req)
+
+    def stream_chat(self, endpoint_id, req):
+        req_id = gen_req_id()
+        self._validate("chat", req_id)
+        apikey = self._apikey
+
+        try:
+            req["stream"] = True
+            res = self._call(
+                endpoint_id, "chat", req_id, {}, json.dumps(req).encode("utf-8"), apikey, stream=True
+            )
+
+            decoder = SSEDecoder(res)
+
+            def iter_fn():
+                for data in decoder.next():
+                    if data == b"[DONE]":
+                        return
+
+                    try:
+                        res = json_to_object(
+                            str(data, encoding="utf-8"), req_id=req_id)
+                    except Exception:
+                        raise
+
+                    if res.error is not None and res.error.code_n != 0:
+                        raise MaasException(
+                            res.error.code_n,
+                            res.error.code,
+                            res.error.message,
+                            req_id,
+                        )
+                    yield res
+
+            return iter_fn()
+        except MaasException:
+            raise
+        except Exception as e:
+            raise new_client_sdk_request_error(str(e))
+
+    def embeddings(self, endpoint_id, req):
+        return self._request(endpoint_id, "embeddings", req)
+
+    def _request(self, endpoint_id, api, req, params={}):
+        req_id = gen_req_id()
+
+        self._validate(api, req_id)
+
+        apikey = self._apikey
+
+        try:
+            res = self._call(endpoint_id, api, req_id, params,
+                             json.dumps(req).encode("utf-8"), apikey)
+            resp = dict_to_object(res.json())
+            if resp and isinstance(resp, dict):
+                resp["req_id"] = req_id
+            return resp
+
+        except MaasException as e:
+            raise e
+        except Exception as e:
+            raise new_client_sdk_request_error(str(e), req_id)
+
+    def _validate(self, api, req_id):
+        credentials_exist = (
+            self.service_info.credentials is not None and
+            self.service_info.credentials.sk is not None and
+            self.service_info.credentials.ak is not None
+        )
+
+        if not self._apikey and not credentials_exist:
+            raise new_client_sdk_request_error("no valid credential", req_id)
+
+        if not (api in self.api_info):
+            raise new_client_sdk_request_error("no such api", req_id)
+
+    def _call(self, endpoint_id, api, req_id, params, body, apikey=None, stream=False):
+        api_info = copy.deepcopy(self.api_info[api])
+        api_info.path = api_info.path.format(endpoint_id=endpoint_id)
+
+        r = self.prepare_request(api_info, params)
+        r.headers["x-tt-logid"] = req_id
+        r.headers["Content-Type"] = "application/json"
+        r.body = body
+
+        if apikey is None:
+            Signer.sign(r, self.service_info.credentials)
+        elif apikey is not None:
+            r.headers["Authorization"] = "Bearer " + apikey
+
+        url = r.build()
+        res = self.session.post(
+            url,
+            headers=r.headers,
+            data=r.body,
+            timeout=(
+                self.service_info.connection_timeout,
+                self.service_info.socket_timeout,
+            ),
+            stream=stream,
+        )
+
+        if res.status_code != 200:
+            raw = res.text.encode()
+            res.close()
+            try:
+                resp = json_to_object(
+                    str(raw, encoding="utf-8"), req_id=req_id)
+            except Exception:
+                raise new_client_sdk_request_error(raw, req_id)
+
+            if resp.error:
+                raise MaasException(
+                    resp.error.code_n, resp.error.code, resp.error.message, req_id
+                )
+            else:
+                raise new_client_sdk_request_error(resp, req_id)
+
+        return res
+
+
+class MaasException(Exception):
+    def __init__(self, code_n, code, message, req_id):
+        self.code_n = code_n
+        self.code = code
+        self.message = message
+        self.req_id = req_id
+
+    def __str__(self):
+        return ("Detailed exception information is listed below.\n" +
+                "req_id: {}\n" +
+                "code_n: {}\n" +
+                "code: {}\n" +
+                "message: {}").format(self.req_id, self.code_n, self.code, self.message)
+
+
+def new_client_sdk_request_error(raw, req_id=""):
+    return MaasException(1709701, "ClientSDKRequestError", "MaaS SDK request error: {}".format(raw), req_id)
+
+
+class BinaryResponseContent:
+    def __init__(self, response, request_id) -> None:
+        self.response = response
+        self.request_id = request_id
+
+    def stream_to_file(
+            self,
+            file: str
+    ) -> None:
+        is_first = True
+        error_bytes = b''
+        with open(file, mode="wb") as f:
+            for data in self.response:
+                if len(error_bytes) > 0 or (is_first and "\"error\":" in str(data)):
+                    error_bytes += data
+                else:
+                    f.write(data)
+
+        if len(error_bytes) > 0:
+            resp = json_to_object(
+                str(error_bytes, encoding="utf-8"), req_id=self.request_id)
+            raise MaasException(
+                resp.error.code_n, resp.error.code, resp.error.message, self.request_id
+            )
+
+    def iter_bytes(self) -> Iterator[bytes]:
+        yield from self.response

api/core/model_runtime/model_providers/volcengine_maas/volcengine_maas.py

@@ -0,0 +1,10 @@
+import logging
+
+from core.model_runtime.model_providers.__base.model_provider import ModelProvider
+
+logger = logging.getLogger(__name__)
+
+
+class VolcengineMaaSProvider(ModelProvider):
+    def validate_provider_credentials(self, credentials: dict) -> None:
+        pass

api/core/model_runtime/model_providers/volcengine_maas/volcengine_maas.yaml

@@ -0,0 +1,151 @@
+provider: volcengine_maas
+label:
+  en_US: Volcengine
+description:
+  en_US: Volcengine MaaS models.
+icon_small:
+  en_US: icon_s_en.svg
+icon_large:
+  en_US: icon_l_en.svg
+  zh_Hans: icon_l_zh.svg
+background: "#F9FAFB"
+help:
+  title:
+    en_US: Get your Access Key and Secret Access Key from Volcengine Console
+  url:
+    en_US: https://console.volcengine.com/iam/keymanage/
+supported_model_types:
+  - llm
+  - text-embedding
+configurate_methods:
+  - customizable-model
+model_credential_schema:
+  model:
+    label:
+      en_US: Model Name
+      zh_Hans: 模型名称
+    placeholder:
+      en_US: Enter your Model Name
+      zh_Hans: 输入模型名称
+  credential_form_schemas:
+    - variable: volc_access_key_id
+      required: true
+      label:
+        en_US: Access Key
+        zh_Hans: Access Key
+      type: secret-input
+      placeholder:
+        en_US: Enter your Access Key
+        zh_Hans: 输入您的 Access Key
+    - variable: volc_secret_access_key
+      required: true
+      label:
+        en_US: Secret Access Key
+        zh_Hans: Secret Access Key
+      type: secret-input
+      placeholder:
+        en_US: Enter your Secret Access Key
+        zh_Hans: 输入您的 Secret Access Key
+    - variable: volc_region
+      required: true
+      label:
+        en_US: Volcengine Region
+        zh_Hans: 火山引擎地区
+      type: text-input
+      default: cn-beijing
+      placeholder:
+        en_US: Enter Volcengine Region
+        zh_Hans: 输入火山引擎地域
+    - variable: api_endpoint_host
+      required: true
+      label:
+        en_US: API Endpoint Host
+        zh_Hans: API Endpoint Host
+      type: text-input
+      default: maas-api.ml-platform-cn-beijing.volces.com
+      placeholder:
+        en_US: Enter your API Endpoint Host
+        zh_Hans: 输入 API Endpoint Host
+    - variable: endpoint_id
+      required: true
+      label:
+        en_US: Endpoint ID
+        zh_Hans: Endpoint ID
+      type: text-input
+      placeholder:
+        en_US: Enter your Endpoint ID
+        zh_Hans: 输入您的 Endpoint ID
+    - variable: base_model_name
+      show_on:
+        - variable: __model_type
+          value: llm
+      label:
+        en_US: Base Model
+        zh_Hans: 基础模型
+      type: select
+      required: true
+      options:
+        - label:
+            en_US: Skylark2-pro-4k
+          value: Skylark2-pro-4k
+          show_on:
+            - variable: __model_type
+              value: llm
+        - label:
+            en_US: Custom
+            zh_Hans: 自定义
+          value: Custom
+    - variable: mode
+      required: true
+      show_on:
+        - variable: __model_type
+          value: llm
+        - variable: base_model_name
+          value: Custom
+      label:
+        zh_Hans: 模型类型
+        en_US: Completion Mode
+      type: select
+      default: chat
+      placeholder:
+        zh_Hans: 选择对话类型
+        en_US: Select Completion Mode
+      options:
+        - value: completion
+          label:
+            en_US: Completion
+            zh_Hans: 补全
+        - value: chat
+          label:
+            en_US: Chat
+            zh_Hans: 对话
+    - variable: context_size
+      required: true
+      show_on:
+        - variable: __model_type
+          value: llm
+        - variable: base_model_name
+          value: Custom
+      label:
+        zh_Hans: 模型上下文长度
+        en_US: Model Context Size
+      type: text-input
+      default: '4096'
+      placeholder:
+        zh_Hans: 输入您的模型上下文长度
+        en_US: Enter your Model Context Size
+    - variable: max_tokens
+      required: true
+      show_on:
+        - variable: __model_type
+          value: llm
+        - variable: base_model_name
+          value: Custom
+      label:
+        zh_Hans: 最大 token 上限
+        en_US: Upper Bound for Max Tokens
+      default: '4096'
+      type: text-input
+      placeholder:
+        zh_Hans: 输入您的模型最大 token 上限
+        en_US: Enter your model Upper Bound for Max Tokens

api/core/model_runtime/model_providers/xinference/llm/llm.py

@@ -28,7 +28,10 @@ from core.model_runtime.entities.common_entities import I18nObject
 from core.model_runtime.entities.llm_entities import LLMMode, LLMResult, LLMResultChunk, LLMResultChunkDelta
 from core.model_runtime.entities.message_entities import (
     AssistantPromptMessage,
+    ImagePromptMessageContent,
     PromptMessage,
+    PromptMessageContent,
+    PromptMessageContentType,
     PromptMessageTool,
     SystemPromptMessage,
     ToolPromptMessage,
@@ -61,8 +64,8 @@ from core.model_runtime.utils import helper
 
 
 class XinferenceAILargeLanguageModel(LargeLanguageModel):
-    def _invoke(self, model: str, credentials: dict, prompt_messages: list[PromptMessage], 
-                model_parameters: dict, tools: list[PromptMessageTool] | None = None, 
+    def _invoke(self, model: str, credentials: dict, prompt_messages: list[PromptMessage],
+                model_parameters: dict, tools: list[PromptMessageTool] | None = None,
                 stop: list[str] | None = None, stream: bool = True, user: str | None = None) \
         -> LLMResult | Generator:
         """
@@ -99,7 +102,7 @@ class XinferenceAILargeLanguageModel(LargeLanguageModel):
         try:
             if "/" in credentials['model_uid'] or "?" in credentials['model_uid'] or "#" in credentials['model_uid']:
                 raise CredentialsValidateFailedError("model_uid should not contain /, ?, or #")
-            
+
             extra_param = XinferenceHelper.get_xinference_extra_parameter(
                 server_url=credentials['server_url'],
                 model_uid=credentials['model_uid']
@@ -111,10 +114,13 @@ class XinferenceAILargeLanguageModel(LargeLanguageModel):
                     credentials['completion_type'] = 'completion'
                 else:
                     raise ValueError(f'xinference model ability {extra_param.model_ability} is not supported, check if you have the right model type')
-                
+
             if extra_param.support_function_call:
                 credentials['support_function_call'] = True
 
+            if extra_param.support_vision:
+                credentials['support_vision'] = True
+
             if extra_param.context_length:
                 credentials['context_length'] = extra_param.context_length
 
@@ -135,7 +141,7 @@ class XinferenceAILargeLanguageModel(LargeLanguageModel):
         """
         return self._num_tokens_from_messages(prompt_messages, tools)
 
-    def _num_tokens_from_messages(self, messages: list[PromptMessage], tools: list[PromptMessageTool], 
+    def _num_tokens_from_messages(self, messages: list[PromptMessage], tools: list[PromptMessageTool],
                                   is_completion_model: bool = False) -> int:
         def tokens(text: str):
             return self._get_num_tokens_by_gpt2(text)
@@ -155,7 +161,7 @@ class XinferenceAILargeLanguageModel(LargeLanguageModel):
                     text = ''
                     for item in value:
                         if isinstance(item, dict) and item['type'] == 'text':
-                            text += item.text
+                            text += item['text']
 
                     value = text
 
@@ -191,7 +197,7 @@ class XinferenceAILargeLanguageModel(LargeLanguageModel):
             num_tokens += self._num_tokens_for_tools(tools)
 
         return num_tokens
-    
+
     def _num_tokens_for_tools(self, tools: list[PromptMessageTool]) -> int:
         """
         Calculate num tokens for tool calling
@@ -234,7 +240,7 @@ class XinferenceAILargeLanguageModel(LargeLanguageModel):
                     num_tokens += tokens(required_field)
 
         return num_tokens
-    
+
     def _convert_prompt_message_to_text(self, message: list[PromptMessage]) -> str:
         """
             convert prompt message to text
@@ -260,7 +266,26 @@ class XinferenceAILargeLanguageModel(LargeLanguageModel):
             if isinstance(message.content, str):
                 message_dict = {"role": "user", "content": message.content}
             else:
-                raise ValueError("User message content must be str")
+                sub_messages = []
+                for message_content in message.content:
+                    if message_content.type == PromptMessageContentType.TEXT:
+                        message_content = cast(PromptMessageContent, message_content)
+                        sub_message_dict = {
+                            "type": "text",
+                            "text": message_content.data
+                        }
+                        sub_messages.append(sub_message_dict)
+                    elif message_content.type == PromptMessageContentType.IMAGE:
+                        message_content = cast(ImagePromptMessageContent, message_content)
+                        sub_message_dict = {
+                            "type": "image_url",
+                            "image_url": {
+                                "url": message_content.data,
+                                "detail": message_content.detail.value
+                            }
+                        }
+                        sub_messages.append(sub_message_dict)
+                message_dict = {"role": "user", "content": sub_messages}
         elif isinstance(message, AssistantPromptMessage):
             message = cast(AssistantPromptMessage, message)
             message_dict = {"role": "assistant", "content": message.content}
@@ -277,7 +302,7 @@ class XinferenceAILargeLanguageModel(LargeLanguageModel):
             message_dict = {"tool_call_id": message.tool_call_id, "role": "tool", "content": message.content}
         else:
             raise ValueError(f"Unknown message type {type(message)}")
-        
+
         return message_dict
 
     def get_customizable_model_schema(self, model: str, credentials: dict) -> AIModelEntity | None:
@@ -338,8 +363,18 @@ class XinferenceAILargeLanguageModel(LargeLanguageModel):
                 completion_type = LLMMode.COMPLETION.value
             else:
                 raise ValueError(f'xinference model ability {extra_args.model_ability} is not supported')
-            
+
+
+        features = []
+
         support_function_call = credentials.get('support_function_call', False)
+        if support_function_call:
+            features.append(ModelFeature.TOOL_CALL)
+
+        support_vision = credentials.get('support_vision', False)
+        if support_vision:
+            features.append(ModelFeature.VISION)
+
         context_length = credentials.get('context_length', 2048)
 
         entity = AIModelEntity(
@@ -349,10 +384,8 @@ class XinferenceAILargeLanguageModel(LargeLanguageModel):
             ),
             fetch_from=FetchFrom.CUSTOMIZABLE_MODEL,
             model_type=ModelType.LLM,
-            features=[
-                ModelFeature.TOOL_CALL
-            ] if support_function_call else [],
-            model_properties={ 
+            features=features,
+            model_properties={
                 ModelPropertyKey.MODE: completion_type,
                 ModelPropertyKey.CONTEXT_SIZE: context_length
             },
@@ -360,22 +393,22 @@ class XinferenceAILargeLanguageModel(LargeLanguageModel):
         )
 
         return entity
-    
-    def _generate(self, model: str, credentials: dict, prompt_messages: list[PromptMessage], 
+
+    def _generate(self, model: str, credentials: dict, prompt_messages: list[PromptMessage],
                  model_parameters: dict, extra_model_kwargs: XinferenceModelExtraParameter,
-                 tools: list[PromptMessageTool] | None = None, 
+                 tools: list[PromptMessageTool] | None = None,
                  stop: list[str] | None = None, stream: bool = True, user: str | None = None) \
             -> LLMResult | Generator:
         """
             generate text from LLM
 
             see `core.model_runtime.model_providers.__base.large_language_model.LargeLanguageModel._generate`
-            
+
             extra_model_kwargs can be got by `XinferenceHelper.get_xinference_extra_parameter`
         """
         if 'server_url' not in credentials:
             raise CredentialsValidateFailedError('server_url is required in credentials')
-        
+
         if credentials['server_url'].endswith('/'):
             credentials['server_url'] = credentials['server_url'][:-1]
 
@@ -408,11 +441,11 @@ class XinferenceAILargeLanguageModel(LargeLanguageModel):
                     'function': helper.dump_model(tool)
                 } for tool in tools
             ]
-
+        vision = credentials.get('support_vision', False)
         if isinstance(xinference_model, RESTfulChatModelHandle | RESTfulChatglmCppChatModelHandle):
             resp = client.chat.completions.create(
                 model=credentials['model_uid'],
-                messages=[self._convert_prompt_message_to_dict(message) for message in prompt_messages], 
+                messages=[self._convert_prompt_message_to_dict(message) for message in prompt_messages],
                 stream=stream,
                 user=user,
                 **generate_config,
@@ -497,7 +530,7 @@ class XinferenceAILargeLanguageModel(LargeLanguageModel):
         """
         if len(resp.choices) == 0:
             raise InvokeServerUnavailableError("Empty response")
-        
+
         assistant_message = resp.choices[0].message
 
         # convert tool call to assistant message tool call
@@ -527,7 +560,7 @@ class XinferenceAILargeLanguageModel(LargeLanguageModel):
         )
 
         return response
-
+    
     def _handle_chat_stream_response(self, model: str, credentials: dict, prompt_messages: list[PromptMessage],
                                         tools: list[PromptMessageTool],
                                         resp: Iterator[ChatCompletionChunk]) -> Generator:
@@ -544,7 +577,7 @@ class XinferenceAILargeLanguageModel(LargeLanguageModel):
 
             if delta.finish_reason is None and (delta.delta.content is None or delta.delta.content == ''):
                 continue
-            
+
             # check if there is a tool call in the response
             function_call = None
             tool_calls = []
@@ -573,9 +606,9 @@ class XinferenceAILargeLanguageModel(LargeLanguageModel):
                 prompt_tokens = self._num_tokens_from_messages(messages=prompt_messages, tools=tools)
                 completion_tokens = self._num_tokens_from_messages(messages=[temp_assistant_prompt_message], tools=[])
 
-                usage = self._calc_response_usage(model=model, credentials=credentials, 
+                usage = self._calc_response_usage(model=model, credentials=credentials,
                                                   prompt_tokens=prompt_tokens, completion_tokens=completion_tokens)
-                
+
                 yield LLMResultChunk(
                     model=model,
                     prompt_messages=prompt_messages,
@@ -608,7 +641,7 @@ class XinferenceAILargeLanguageModel(LargeLanguageModel):
         """
         if len(resp.choices) == 0:
             raise InvokeServerUnavailableError("Empty response")
-        
+
         assistant_message = resp.choices[0].text
 
         # transform assistant message to prompt message
@@ -670,9 +703,9 @@ class XinferenceAILargeLanguageModel(LargeLanguageModel):
                 completion_tokens = self._num_tokens_from_messages(
                     messages=[temp_assistant_prompt_message], tools=[], is_completion_model=True
                 )
-                usage = self._calc_response_usage(model=model, credentials=credentials, 
+                usage = self._calc_response_usage(model=model, credentials=credentials,
                                                   prompt_tokens=prompt_tokens, completion_tokens=completion_tokens)
-                
+
                 yield LLMResultChunk(
                     model=model,
                     prompt_messages=prompt_messages,

api/core/model_runtime/model_providers/xinference/xinference_helper.py

@@ -14,13 +14,15 @@ class XinferenceModelExtraParameter:
     max_tokens: int = 512
     context_length: int = 2048
     support_function_call: bool = False
+    support_vision: bool = False
 
-    def __init__(self, model_format: str, model_handle_type: str, model_ability: list[str], 
-                 support_function_call: bool, max_tokens: int, context_length: int) -> None:
+    def __init__(self, model_format: str, model_handle_type: str, model_ability: list[str],
+                 support_function_call: bool, support_vision: bool, max_tokens: int, context_length: int) -> None:
         self.model_format = model_format
         self.model_handle_type = model_handle_type
         self.model_ability = model_ability
         self.support_function_call = support_function_call
+        self.support_vision = support_vision
         self.max_tokens = max_tokens
         self.context_length = context_length
 
@@ -71,7 +73,7 @@ class XinferenceHelper:
             raise RuntimeError(f'get xinference model extra parameter failed, url: {url}, error: {e}')
         if response.status_code != 200:
             raise RuntimeError(f'get xinference model extra parameter failed, status code: {response.status_code}, response: {response.text}')
-        
+
         response_json = response.json()
 
         model_format = response_json.get('model_format', 'ggmlv3')
@@ -87,17 +89,19 @@ class XinferenceHelper:
             model_handle_type = 'chat'
         else:
             raise NotImplementedError(f'xinference model handle type {model_handle_type} is not supported')
-        
+
         support_function_call = 'tools' in model_ability
+        support_vision = 'vision' in model_ability
         max_tokens = response_json.get('max_tokens', 512)
 
         context_length = response_json.get('context_length', 2048)
-        
+
         return XinferenceModelExtraParameter(
             model_format=model_format,
             model_handle_type=model_handle_type,
             model_ability=model_ability,
             support_function_call=support_function_call,
+            support_vision=support_vision,
             max_tokens=max_tokens,
             context_length=context_length
         )

api/core/model_runtime/model_providers/yi/yi.yaml

@@ -33,7 +33,7 @@ provider_credential_schema:
     - variable: endpoint_url
       label:
         zh_Hans: 自定义 API endpoint 地址
-        en_US: CUstom API endpoint URL
+        en_US: Custom API endpoint URL
       type: text-input
       required: false
       placeholder:

api/core/tools/entities/tool_entities.py

@@ -77,12 +77,13 @@ class ToolInvokeMessage(BaseModel):
         LINK = "link"
         BLOB = "blob"
         IMAGE_LINK = "image_link"
+        CHUNK = "chunk"
 
     type: MessageType = MessageType.TEXT
     """
         plain text, image url or link url
     """
-    message: Union[str, bytes] = None
+    message: Union[str, bytes, list] = None
     meta: dict[str, Any] = None
     save_as: str = ''
 

api/core/tools/provider/_position.yaml

@@ -30,3 +30,4 @@
 - qrcode
 - dingtalk
 - feishu
+- slack

api/core/tools/provider/builtin/bing/tools/bing_web_search.py

@@ -40,7 +40,7 @@ class BingSearchTool(BuiltinTool):
         news = response['news']['value'] if 'news' in response else []
         computation = response['computation']['value'] if 'computation' in response else None
 
-        if result_type == 'link':
+        if result_type == 'link' or result_type == 'chunk':
             results = []
             if search_results:
                 for result in search_results:
@@ -72,7 +72,7 @@ class BingSearchTool(BuiltinTool):
                     ))
                     
             return results
-        else:
+        if result_type == 'text' or result_type == 'chunk':
             # construct text
             text = ''
             if search_results:

api/core/tools/provider/builtin/code/tools/simple_code.py

@@ -1,6 +1,6 @@
 from typing import Any
 
-from core.helper.code_executor.code_executor import CodeExecutor
+from core.helper.code_executor.code_executor import CodeExecutor, CodeLanguage
 from core.tools.entities.tool_entities import ToolInvokeMessage
 from core.tools.tool.builtin_tool import BuiltinTool
 
@@ -11,10 +11,10 @@ class SimpleCode(BuiltinTool):
             invoke simple code
         """
 
-        language = tool_parameters.get('language', 'python3')
+        language = tool_parameters.get('language', CodeLanguage.PYTHON3)
         code = tool_parameters.get('code', '')
 
-        if language not in ['python3', 'javascript']:
+        if language not in [CodeLanguage.PYTHON3, CodeLanguage.JAVASCRIPT]:
             raise ValueError(f'Only python3 and javascript are supported, not {language}')
         
         result = CodeExecutor.execute_code(language, '', code)

api/core/tools/provider/builtin/google/tools/google_search.py

@@ -6,6 +6,7 @@ from serpapi import GoogleSearch
 
 from core.tools.entities.tool_entities import ToolInvokeMessage
 from core.tools.tool.builtin_tool import BuiltinTool
+from core.workflow.nodes.llm.knowledge_resource import KnowledgeResource
 
 
 class HiddenPrints:
@@ -35,7 +36,7 @@ class SerpAPI:
         self.serpapi_api_key = api_key
         self.search_engine = GoogleSearch
 
-    def run(self, query: str, **kwargs: Any) -> str:
+    def run(self, query: str, **kwargs: Any) -> str | list[KnowledgeResource]:
         """Run query through SerpAPI and parse result."""
         typ = kwargs.get("result_type", "text")
         return self._process_response(self.results(query), typ=typ)
@@ -64,63 +65,79 @@ class SerpAPI:
         return params
 
     @staticmethod
-    def _process_response(res: dict, typ: str) -> str:
+    def _process_response(res: dict, typ: str) -> str | list[KnowledgeResource]:
         """Process response from SerpAPI."""
         if "error" in res.keys():
             raise ValueError(f"Got error from SerpAPI: {res['error']}")
-        
-        if typ == "text":
-            toret = ""
+        chunks = []
+        toret = ""
+        if typ == "text" or typ == "chunk":
             if "answer_box" in res.keys() and type(res["answer_box"]) == list:
                 res["answer_box"] = res["answer_box"][0] + "\n"
             if "answer_box" in res.keys() and "answer" in res["answer_box"].keys():
                 toret += res["answer_box"]["answer"] + "\n"
+                chunks.append(KnowledgeResource(content=res["answer_box"]["answer"], title=res["answer_box"]["answer"]))
             if "answer_box" in res.keys() and "snippet" in res["answer_box"].keys():
                 toret += res["answer_box"]["snippet"] + "\n"
+                chunks.append(
+                    KnowledgeResource(content=res["answer_box"]["snippet"], title=res["answer_box"]["snippet"]))
             if (
-                "answer_box" in res.keys()
-                and "snippet_highlighted_words" in res["answer_box"].keys()
+                    "answer_box" in res.keys()
+                    and "snippet_highlighted_words" in res["answer_box"].keys()
             ):
                 for item in res["answer_box"]["snippet_highlighted_words"]:
                     toret += item + "\n"
+                    chunks.append(KnowledgeResource(content=item, title=item))
             if (
-                "sports_results" in res.keys()
-                and "game_spotlight" in res["sports_results"].keys()
+                    "sports_results" in res.keys()
+                    and "game_spotlight" in res["sports_results"].keys()
             ):
                 toret += res["sports_results"]["game_spotlight"] + "\n"
+                chunks.append(KnowledgeResource(content=res["sports_results"]["game_spotlight"],
+                                                title=res["sports_results"]["game_spotlight"]))
             if (
-                "shopping_results" in res.keys()
-                and "title" in res["shopping_results"][0].keys()
+                    "shopping_results" in res.keys()
+                    and "title" in res["shopping_results"][0].keys()
             ):
                 toret += res["shopping_results"][:3] + "\n"
+                chunks.append(KnowledgeResource(content=res["shopping_results"][:3], title=res["shopping_results"][:3]))
             if (
-                "knowledge_graph" in res.keys()
-                and "description" in res["knowledge_graph"].keys()
+                    "knowledge_graph" in res.keys()
+                    and "description" in res["knowledge_graph"].keys()
             ):
                 toret = res["knowledge_graph"]["description"] + "\n"
+                chunks.append(KnowledgeResource(content=res["knowledge_graph"]["description"],
+                                                title=res["knowledge_graph"]["description"]))
             if "snippet" in res["organic_results"][0].keys():
                 for item in res["organic_results"]:
                     toret += "content: " + item["snippet"] + "\n" + "link: " + item["link"] + "\n"
+                    chunks.append(KnowledgeResource(content=item["snippet"], title=item["title"], url=item["link"]))
             if (
-                "images_results" in res.keys()
-                and "thumbnail" in res["images_results"][0].keys()
+                    "images_results" in res.keys()
+                    and "thumbnail" in res["images_results"][0].keys()
             ):
                 thumbnails = [item["thumbnail"] for item in res["images_results"][:10]]
                 toret = thumbnails
+                chunks.append(KnowledgeResource(content=thumbnails, title=thumbnails))
             if toret == "":
                 toret = "No good search result found"
-        elif typ == "link":
+        if typ == "link" or typ == "chunk":
             if "knowledge_graph" in res.keys() and "title" in res["knowledge_graph"].keys() \
                     and "description_link" in res["knowledge_graph"].keys():
                 toret = res["knowledge_graph"]["description_link"]
+                chunks.append(KnowledgeResource(content=res["knowledge_graph"]["description"],
+                                                title=res["knowledge_graph"]["title"],
+                                                url=res["knowledge_graph"]["knowledge_graph_search_link"])
+                              )
             elif "knowledge_graph" in res.keys() and "see_results_about" in res["knowledge_graph"].keys() \
-                and len(res["knowledge_graph"]["see_results_about"]) > 0:
+                    and len(res["knowledge_graph"]["see_results_about"]) > 0:
                 see_result_about = res["knowledge_graph"]["see_results_about"]
                 toret = ""
                 for item in see_result_about:
                     if "name" not in item.keys() or "link" not in item.keys():
                         continue
                     toret += f"[{item['name']}]({item['link']})\n"
+                    chunks.append(KnowledgeResource(content=f"[{item['name']}]({item['link']})\n", title=item['name'], url=item['link']))
             elif "organic_results" in res.keys() and len(res["organic_results"]) > 0:
                 organic_results = res["organic_results"]
                 toret = ""
@@ -128,6 +145,7 @@ class SerpAPI:
                     if "title" not in item.keys() or "link" not in item.keys():
                         continue
                     toret += f"[{item['title']}]({item['link']})\n"
+                    chunks.append(KnowledgeResource(content=f"[{item['title']}]({item['link']})\n", title=item['title'], url=item['link']))
             elif "related_questions" in res.keys() and len(res["related_questions"]) > 0:
                 related_questions = res["related_questions"]
                 toret = ""
@@ -135,6 +153,7 @@ class SerpAPI:
                     if "question" not in item.keys() or "link" not in item.keys():
                         continue
                     toret += f"[{item['question']}]({item['link']})\n"
+                    chunks.append(KnowledgeResource(content=f"[{item['question']}]({item['link']})\n", title=item['title'], url=item['link']))
             elif "related_searches" in res.keys() and len(res["related_searches"]) > 0:
                 related_searches = res["related_searches"]
                 toret = ""
@@ -142,15 +161,19 @@ class SerpAPI:
                     if "query" not in item.keys() or "link" not in item.keys():
                         continue
                     toret += f"[{item['query']}]({item['link']})\n"
+                    chunks.append(KnowledgeResource(content=f"[{item['query']}]({item['link']})\n", title=item['query'], url=item['link']))
             else:
                 toret = "No good search result found"
+        if typ == "chunk":
+            return chunks
         return toret
 
+
 class GoogleSearchTool(BuiltinTool):
-    def _invoke(self, 
+    def _invoke(self,
                 user_id: str,
-               tool_parameters: dict[str, Any], 
-        ) -> Union[ToolInvokeMessage, list[ToolInvokeMessage]]:
+                tool_parameters: dict[str, Any],
+                ) -> Union[ToolInvokeMessage, list[ToolInvokeMessage]]:
         """
             invoke tools
         """
@@ -160,5 +183,9 @@ class GoogleSearchTool(BuiltinTool):
         result = SerpAPI(api_key).run(query, result_type=result_type)
         if result_type == 'text':
             return self.create_text_message(text=result)
-        return self.create_link_message(link=result)
-    
+        elif result_type == 'link':
+            return self.create_link_message(link=result)
+        elif result_type == 'chunk':
+            return self.create_chunk_message(chunks=result)
+        else:
+            raise ValueError(f"Invalid result type: {result_type}")

api/core/tools/provider/builtin/google/tools/google_search.yaml

@@ -39,6 +39,11 @@ parameters:
           en_US: link
           zh_Hans: 链接
           pt_BR: link
+      - value: chunk
+        label:
+          en_US: chunk
+          zh_Hans: 分段
+          pt_BR: chunk
     default: link
     label:
       en_US: Result type

api/core/tools/provider/builtin/slack/_assets/icon.svg

@@ -0,0 +1,22 @@
+<?xml version="1.0" ?>
+<svg width="54" height="54" viewBox="0 0 54 54" xmlns="http://www.w3.org/2000/svg" role="presentation">
+  <title>Slack</title>
+  <g fill="none" fill-rule="evenodd">
+    <path
+      d="M19.712.133a5.381 5.381 0 0 0-5.376 5.387 5.381 5.381 0 0 0 5.376 5.386h5.376V5.52A5.381 5.381 0 0 0 19.712.133m0 14.365H5.376A5.381 5.381 0 0 0 0 19.884a5.381 5.381 0 0 0 5.376 5.387h14.336a5.381 5.381 0 0 0 5.376-5.387 5.381 5.381 0 0 0-5.376-5.386"
+      fill="#44BEDF"
+    ></path>
+    <path
+      d="M53.76 19.884a5.381 5.381 0 0 0-5.376-5.386 5.381 5.381 0 0 0-5.376 5.386v5.387h5.376a5.381 5.381 0 0 0 5.376-5.387m-14.336 0V5.52A5.381 5.381 0 0 0 34.048.133a5.381 5.381 0 0 0-5.376 5.387v14.364a5.381 5.381 0 0 0 5.376 5.387 5.381 5.381 0 0 0 5.376-5.387"
+      fill="#2EB67D"
+    ></path>
+    <path
+      d="M34.048 54a5.381 5.381 0 0 0 5.376-5.387 5.381 5.381 0 0 0-5.376-5.386h-5.376v5.386A5.381 5.381 0 0 0 34.048 54m0-14.365h14.336a5.381 5.381 0 0 0 5.376-5.386 5.381 5.381 0 0 0-5.376-5.387H34.048a5.381 5.381 0 0 0-5.376 5.387 5.381 5.381 0 0 0 5.376 5.386"
+      fill="#ECB22E"
+    ></path>
+    <path
+      d="M0 34.249a5.381 5.381 0 0 0 5.376 5.386 5.381 5.381 0 0 0 5.376-5.386v-5.387H5.376A5.381 5.381 0 0 0 0 34.25m14.336-.001v14.364A5.381 5.381 0 0 0 19.712 54a5.381 5.381 0 0 0 5.376-5.387V34.25a5.381 5.381 0 0 0-5.376-5.387 5.381 5.381 0 0 0-5.376 5.387"
+      fill="#E01E5A"
+    ></path>
+  </g>
+</svg>

api/core/tools/provider/builtin/slack/slack.py

@@ -0,0 +1,8 @@
+from core.tools.provider.builtin.slack.tools.slack_webhook import SlackWebhookTool
+from core.tools.provider.builtin_tool_provider import BuiltinToolProviderController
+
+
+class SlackProvider(BuiltinToolProviderController):
+    def _validate_credentials(self, credentials: dict) -> None:
+        SlackWebhookTool()
+        pass

api/core/tools/provider/builtin/slack/slack.yaml

@@ -0,0 +1,13 @@
+identity:
+  author: Pan YANG
+  name: slack
+  label:
+    en_US: Slack
+    zh_Hans: Slack
+    pt_BR: Slack
+  description:
+    en_US: Slack Webhook
+    zh_Hans: Slack Webhook
+    pt_BR: Slack Webhook
+  icon: icon.svg
+credentials_for_provider:

api/core/tools/provider/builtin/slack/tools/slack_webhook.py

@@ -0,0 +1,43 @@
+from typing import Any, Union
+
+import httpx
+
+from core.tools.entities.tool_entities import ToolInvokeMessage
+from core.tools.tool.builtin_tool import BuiltinTool
+
+
+class SlackWebhookTool(BuiltinTool):
+    def _invoke(self, user_id: str, tool_parameters: dict[str, Any]
+                ) -> Union[ToolInvokeMessage, list[ToolInvokeMessage]]:
+        """
+            Incoming Webhooks
+            API Document: https://api.slack.com/messaging/webhooks
+        """
+
+        content = tool_parameters.get('content', '')
+        if not content:
+            return self.create_text_message('Invalid parameter content')
+
+        webhook_url = tool_parameters.get('webhook_url', '')
+
+        if not webhook_url.startswith('https://hooks.slack.com/services/'):
+            return self.create_text_message(
+                f'Invalid parameter webhook_url ${webhook_url}, not a valid Slack webhook URL')
+
+        headers = {
+            'Content-Type': 'application/json',
+        }
+        params = {}
+        payload = {
+            "text": content,
+        }
+
+        try:
+            res = httpx.post(webhook_url, headers=headers, params=params, json=payload)
+            if res.is_success:
+                return self.create_text_message("Text message was sent successfully")
+            else:
+                return self.create_text_message(
+                    f"Failed to send the text message, status code: {res.status_code}, response: {res.text}")
+        except Exception as e:
+            return self.create_text_message("Failed to send message through webhook. {}".format(e))

api/core/tools/provider/builtin/slack/tools/slack_webhook.yaml

@@ -0,0 +1,40 @@
+identity:
+  name: slack_webhook
+  author: Pan YANG
+  label:
+    en_US: Incoming Webhook to send message
+    zh_Hans: 通过入站 Webhook 发送消息
+    pt_BR: Incoming Webhook to send message
+  icon: icon.svg
+description:
+  human:
+    en_US: Sending a message on Slack via the Incoming Webhook
+    zh_Hans: 通过入站 Webhook 在 Slack 上发送消息
+    pt_BR: Sending a message on Slack via the Incoming Webhook
+  llm: A tool for sending messages to a chat on Slack.
+parameters:
+  - name: webhook_url
+    type: string
+    required: true
+    label:
+      en_US: Slack Incoming Webhook url
+      zh_Hans: Slack 入站 Webhook 的 url
+      pt_BR: Slack Incoming Webhook url
+    human_description:
+      en_US: Slack Incoming Webhook url
+      zh_Hans: Slack 入站 Webhook 的 url
+      pt_BR: Slack Incoming Webhook url
+    form: form
+  - name: content
+    type: string
+    required: true
+    label:
+      en_US: content
+      zh_Hans: 消息内容
+      pt_BR: content
+    human_description:
+      en_US: Content to sent to the channel or person.
+      zh_Hans: 消息内容文本
+      pt_BR: Content to sent to the channel or person.
+    llm_description: Content of the message
+    form: llm

api/core/tools/provider/builtin/time/tools/current_time.py

@@ -17,11 +17,12 @@ class CurrentTimeTool(BuiltinTool):
         """
         # get timezone
         tz = tool_parameters.get('timezone', 'UTC')
+        fm = tool_parameters.get('format') or '%Y-%m-%d %H:%M:%S %Z'
         if tz == 'UTC':
-            return self.create_text_message(f'{datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S %Z")}')
-
+            return self.create_text_message(f'{datetime.now(timezone.utc).strftime(fm)}')
+        
         try:
             tz = pytz_timezone(tz)
         except:
             return self.create_text_message(f'Invalid timezone: {tz}')
-        return self.create_text_message(f'{datetime.now(tz).strftime("%Y-%m-%d %H:%M:%S %Z")}')
+        return self.create_text_message(f'{datetime.now(tz).strftime(fm)}')

api/core/tools/provider/builtin/time/tools/current_time.yaml

@@ -12,6 +12,19 @@ description:
     pt_BR: A tool for getting the current time.
   llm: A tool for getting the current time.
 parameters:
+  - name: format
+    type: string
+    required: false
+    label:
+      en_US: Format
+      zh_Hans: 格式
+      pt_BR: Format
+    human_description:
+      en_US: Time format in strftime standard.
+      zh_Hans: strftime 标准的时间格式。
+      pt_BR: Time format in strftime standard.
+    form: form
+    default: "%Y-%m-%d %H:%M:%S"
   - name: timezone
     type: select
     required: false

api/core/tools/tool/tool.py

@@ -1,6 +1,6 @@
 from abc import ABC, abstractmethod
 from enum import Enum
-from typing import Any, Optional, Union
+from typing import Any, Optional, Union, List
 
 from pydantic import BaseModel, validator
 
@@ -15,6 +15,7 @@ from core.tools.entities.tool_entities import (
     ToolRuntimeVariablePool,
 )
 from core.tools.tool_file_manager import ToolFileManager
+from core.workflow.nodes.llm.knowledge_resource import KnowledgeResource
 
 
 class Tool(BaseModel, ABC):
@@ -233,7 +234,10 @@ class Tool(BaseModel, ABC):
                     ToolParameter.ToolParameterType.STRING, 
                     ToolParameter.ToolParameterType.SELECT,
                 ] and not isinstance(tool_parameters[parameter.name], str):
-                    tool_parameters[parameter.name] = str(tool_parameters[parameter.name])
+                    if tool_parameters[parameter.name] is None:
+                        tool_parameters[parameter.name] = ''
+                    else:
+                        tool_parameters[parameter.name] = str(tool_parameters[parameter.name])
                 elif parameter.type == ToolParameter.ToolParameterType.NUMBER \
                     and not isinstance(tool_parameters[parameter.name], int | float):
                     if isinstance(tool_parameters[parameter.name], str):
@@ -241,6 +245,10 @@ class Tool(BaseModel, ABC):
                             tool_parameters[parameter.name] = int(tool_parameters[parameter.name])
                         except ValueError:
                             tool_parameters[parameter.name] = float(tool_parameters[parameter.name])
+                    elif isinstance(tool_parameters[parameter.name], bool):
+                        tool_parameters[parameter.name] = int(tool_parameters[parameter.name])
+                    elif tool_parameters[parameter.name] is None:
+                        tool_parameters[parameter.name] = 0
                 elif parameter.type == ToolParameter.ToolParameterType.BOOLEAN:
                     if not isinstance(tool_parameters[parameter.name], bool):
                         # check if it is a string
@@ -330,6 +338,8 @@ class Tool(BaseModel, ABC):
             create an image message
 
             :param image: the url of the image
+            :param save_as: the save_as
+
             :return: the image message
         """
         return ToolInvokeMessage(type=ToolInvokeMessage.MessageType.IMAGE, 
@@ -341,6 +351,7 @@ class Tool(BaseModel, ABC):
             create a link message
 
             :param link: the url of the link
+            :param save_as: the save_as
             :return: the link message
         """
         return ToolInvokeMessage(type=ToolInvokeMessage.MessageType.LINK, 
@@ -352,21 +363,37 @@ class Tool(BaseModel, ABC):
             create a text message
 
             :param text: the text
+            :param save_as: the save_as
             :return: the text message
         """
         return ToolInvokeMessage(type=ToolInvokeMessage.MessageType.TEXT, 
                                  message=text,
                                  save_as=save_as
                                  )
+
+    def create_chunk_message(self, chunks: List[KnowledgeResource], save_as: str = '') -> ToolInvokeMessage:
+        """
+            create a chunk message
+
+            :param chunks: the chunks
+            :param save_as: the save_as
+            :return: the text message
+        """
+        return ToolInvokeMessage(type=ToolInvokeMessage.MessageType.CHUNK,
+                                 message=chunks,
+                                 save_as=save_as
+                                 )
     
     def create_blob_message(self, blob: bytes, meta: dict = None, save_as: str = '') -> ToolInvokeMessage:
         """
             create a blob message
 
             :param blob: the blob
+            :param meta: the meta
+            :param save_as: the save_as
             :return: the blob message
         """
-        return ToolInvokeMessage(type=ToolInvokeMessage.MessageType.BLOB, 
+        return ToolInvokeMessage(type=ToolInvokeMessage.MessageType.BLOB,
                                  message=blob, meta=meta,
                                  save_as=save_as
                                  )

api/core/tools/tool_engine.py

@@ -131,7 +131,7 @@ class ToolEngine:
 
             # hit the callback handler
             workflow_tool_callback.on_tool_end(
-                tool_name=tool.identity.name, 
+                tool_name=tool.identity.name,
                 tool_inputs=tool_parameters, 
                 tool_outputs=response
             )

api/core/workflow/nodes/code/code_node.py

@@ -1,7 +1,7 @@
 import os
 from typing import Optional, Union, cast
 
-from core.helper.code_executor.code_executor import CodeExecutionException, CodeExecutor
+from core.helper.code_executor.code_executor import CodeExecutionException, CodeExecutor, CodeLanguage
 from core.workflow.entities.node_entities import NodeRunResult, NodeType
 from core.workflow.entities.variable_pool import VariablePool
 from core.workflow.nodes.base_node import BaseNode
@@ -39,7 +39,7 @@ class CodeNode(BaseNode):
         :param filters: filter by node config parameters.
         :return:
         """
-        if filters and filters.get("code_language") == "javascript":
+        if filters and filters.get("code_language") == CodeLanguage.JAVASCRIPT:
             return {
                 "type": "code",
                 "config": {
@@ -53,7 +53,7 @@ class CodeNode(BaseNode):
                             "value_selector": []
                         }
                     ],
-                    "code_language": "javascript",
+                    "code_language": CodeLanguage.JAVASCRIPT,
                     "code": JAVASCRIPT_DEFAULT_CODE,
                     "outputs": {
                         "result": {
@@ -77,7 +77,7 @@ class CodeNode(BaseNode):
                         "value_selector": []
                     }
                 ],
-                "code_language": "python3",
+                "code_language": CodeLanguage.PYTHON3,
                 "code": PYTHON_DEFAULT_CODE,
                 "outputs": {
                     "result": {

api/core/workflow/nodes/http_request/entities.py

@@ -1,9 +1,13 @@
+import os
 from typing import Literal, Optional, Union
 
 from pydantic import BaseModel, validator
 
 from core.workflow.entities.base_node_data_entities import BaseNodeData
 
+MAX_CONNECT_TIMEOUT = int(os.environ.get('HTTP_REQUEST_MAX_CONNECT_TIMEOUT', '300'))
+MAX_READ_TIMEOUT = int(os.environ.get('HTTP_REQUEST_MAX_READ_TIMEOUT', '600'))
+MAX_WRITE_TIMEOUT = int(os.environ.get('HTTP_REQUEST_MAX_WRITE_TIMEOUT', '600'))
 
 class HttpRequestNodeData(BaseNodeData):
     """
@@ -36,9 +40,9 @@ class HttpRequestNodeData(BaseNodeData):
         data: Union[None, str]
 
     class Timeout(BaseModel):
-        connect: int
-        read:  int
-        write:  int
+        connect: int = MAX_CONNECT_TIMEOUT
+        read:  int = MAX_READ_TIMEOUT
+        write:  int = MAX_WRITE_TIMEOUT
 
     method: Literal['get', 'post', 'put', 'patch', 'delete', 'head']
     url: str
@@ -46,4 +50,5 @@ class HttpRequestNodeData(BaseNodeData):
     headers: str
     params: str
     body: Optional[Body]
-    timeout: Optional[Timeout]
+    timeout: Optional[Timeout]
+    mask_authorization_header: Optional[bool] = True

api/core/workflow/nodes/http_request/http_executor.py

@@ -1,4 +1,5 @@
 import json
+import os
 from copy import deepcopy
 from random import randint
 from typing import Any, Optional, Union
@@ -13,11 +14,10 @@ from core.workflow.entities.variable_pool import ValueType, VariablePool
 from core.workflow.nodes.http_request.entities import HttpRequestNodeData
 from core.workflow.utils.variable_template_parser import VariableTemplateParser
 
-MAX_BINARY_SIZE = 1024 * 1024 * 10  # 10MB
-READABLE_MAX_BINARY_SIZE = '10MB'
-MAX_TEXT_SIZE = 1024 * 1024 // 10  # 0.1MB
-READABLE_MAX_TEXT_SIZE = '0.1MB'
-
+MAX_BINARY_SIZE = int(os.environ.get('HTTP_REQUEST_NODE_MAX_BINARY_SIZE', str(1024 * 1024 * 10))) # 10MB
+READABLE_MAX_BINARY_SIZE = f'{MAX_BINARY_SIZE / 1024 / 1024:.2f}MB'
+MAX_TEXT_SIZE = int(os.environ.get('HTTP_REQUEST_NODE_MAX_TEXT_SIZE', str(1024 * 1024))) # 10MB # 1MB
+READABLE_MAX_TEXT_SIZE = f'{MAX_TEXT_SIZE / 1024 / 1024:.2f}MB'
 
 class HttpExecutorResponse:
     headers: dict[str, str]
@@ -344,10 +344,13 @@ class HttpExecutor:
         # validate response
         return self._validate_and_parse_response(response)
     
-    def to_raw_request(self) -> str:
+    def to_raw_request(self, mask_authorization_header: Optional[bool] = True) -> str:
         """
         convert to raw request
         """
+        if mask_authorization_header == None:
+            mask_authorization_header = True
+            
         server_url = self.server_url
         if self.params:
             server_url += f'?{urlencode(self.params)}'
@@ -356,6 +359,17 @@ class HttpExecutor:
 
         headers = self._assembling_headers()
         for k, v in headers.items():
+            if mask_authorization_header:
+                # get authorization header
+                if self.authorization.type == 'api-key':
+                    authorization_header = 'Authorization'
+                    if self.authorization.config and self.authorization.config.header:
+                        authorization_header = self.authorization.config.header
+                    
+                    if k.lower() == authorization_header.lower():
+                        raw_request += f'{k}: {"*" * len(v)}\n'
+                        continue
+            
             raw_request += f'{k}: {v}\n'
 
         raw_request += '\n'

api/core/workflow/nodes/http_request/http_request_node.py

@@ -1,5 +1,4 @@
 import logging
-import os
 from mimetypes import guess_extension
 from os import path
 from typing import cast
@@ -9,14 +8,15 @@ from core.tools.tool_file_manager import ToolFileManager
 from core.workflow.entities.node_entities import NodeRunResult, NodeType
 from core.workflow.entities.variable_pool import VariablePool
 from core.workflow.nodes.base_node import BaseNode
-from core.workflow.nodes.http_request.entities import HttpRequestNodeData
+from core.workflow.nodes.http_request.entities import (
+    MAX_CONNECT_TIMEOUT,
+    MAX_READ_TIMEOUT,
+    MAX_WRITE_TIMEOUT,
+    HttpRequestNodeData,
+)
 from core.workflow.nodes.http_request.http_executor import HttpExecutor, HttpExecutorResponse
 from models.workflow import WorkflowNodeExecutionStatus
 
-MAX_CONNECT_TIMEOUT = int(os.environ.get('HTTP_REQUEST_MAX_CONNECT_TIMEOUT', '300'))
-MAX_READ_TIMEOUT = int(os.environ.get('HTTP_REQUEST_MAX_READ_TIMEOUT', '600'))
-MAX_WRITE_TIMEOUT = int(os.environ.get('HTTP_REQUEST_MAX_WRITE_TIMEOUT', '600'))
-
 HTTP_REQUEST_DEFAULT_TIMEOUT = HttpRequestNodeData.Timeout(connect=min(10, MAX_CONNECT_TIMEOUT),
                                                            read=min(60, MAX_READ_TIMEOUT),
                                                            write=min(20, MAX_WRITE_TIMEOUT))
@@ -63,7 +63,9 @@ class HttpRequestNode(BaseNode):
             process_data = {}
             if http_executor:
                 process_data = {
-                    'request': http_executor.to_raw_request(),
+                    'request': http_executor.to_raw_request(
+                        mask_authorization_header=node_data.mask_authorization_header
+                    ),
                 }
             return NodeRunResult(
                 status=WorkflowNodeExecutionStatus.FAILED,
@@ -82,7 +84,9 @@ class HttpRequestNode(BaseNode):
                 'files': files,
             },
             process_data={
-                'request': http_executor.to_raw_request(),
+                'request': http_executor.to_raw_request(
+                    mask_authorization_header=node_data.mask_authorization_header
+                ),
             }
         )
 

api/core/workflow/nodes/llm/knowledge_resource.py

@@ -0,0 +1,27 @@
+from typing import Any, Optional
+
+from pydantic import BaseModel
+
+
+class KnowledgeResource(BaseModel):
+    """
+    Knowledge Resource.
+    """
+    content: str
+    title: str
+    url: Optional[str] = None
+    icon: Optional[str] = None
+    resource_from: Optional[str] = None
+    score: Optional[float] = None
+    metadata: Optional[dict[str, Any]] = None
+
+    def to_dict(self):
+        return {
+            'content': self.content,
+            'title': self.title,
+            'url': self.url,
+            'icon': self.icon,
+            'resource_from': self.resource_from,
+            'score': self.score,
+            'metadata': self.metadata
+        }

api/core/workflow/nodes/llm/llm_node.py

@@ -22,6 +22,7 @@ from core.workflow.entities.node_entities import NodeRunMetadataKey, NodeRunResu
 from core.workflow.entities.variable_pool import VariablePool
 from core.workflow.nodes.base_node import BaseNode
 from core.workflow.nodes.llm.entities import LLMNodeData, ModelConfig
+from core.workflow.nodes.llm.knowledge_resource import KnowledgeResource
 from core.workflow.utils.variable_template_parser import VariableTemplateParser
 from extensions.ext_database import db
 from models.model import Conversation
@@ -262,7 +263,7 @@ class LLMNode(BaseNode):
                 for item in context_value:
                     if isinstance(item, str):
                         context_str += item + '\n'
-                    else:
+                    elif isinstance(item, dict):
                         if 'content' not in item:
                             raise ValueError(f'Invalid context structure: {item}')
 
@@ -271,6 +272,12 @@ class LLMNode(BaseNode):
                         retriever_resource = self._convert_to_original_retriever_resource(item)
                         if retriever_resource:
                             original_retriever_resource.append(retriever_resource)
+                    elif isinstance(item, KnowledgeResource):
+                        context_str += item.content + '\n'
+
+                        retriever_resource = self._convert_to_original_retriever_resource(item.to_dict())
+                        if retriever_resource:
+                            original_retriever_resource.append(retriever_resource)
 
                 if self.callbacks and original_retriever_resource:
                     for callback in self.callbacks:
@@ -311,6 +318,9 @@ class LLMNode(BaseNode):
             }
 
             return source
+        if ('metadata' in context_dict and '_source' in context_dict['metadata']
+                and context_dict['metadata']['_source'] == 'tool'):
+            return context_dict
 
         return None
 

api/core/workflow/nodes/question_classifier/question_classifier_node.py

@@ -1,3 +1,4 @@
+import json
 import logging
 from typing import Optional, Union, cast
 
@@ -62,13 +63,20 @@ class QuestionClassifierNode(LLMNode):
             prompt_messages=prompt_messages,
             stop=stop
         )
-        categories = [_class.name for _class in node_data.classes]
+        category_name = node_data.classes[0].name
+        category_id = node_data.classes[0].id
         try:
             result_text_json = parse_and_check_json_markdown(result_text, [])
-            #result_text_json = json.loads(result_text.strip('```JSON\n'))
-            categories_result = result_text_json.get('categories', [])
-            if categories_result:
-                categories = categories_result
+            # result_text_json = json.loads(result_text.strip('```JSON\n'))
+            if 'category_name' in result_text_json and 'category_id' in result_text_json:
+                category_id_result = result_text_json['category_id']
+                classes = node_data.classes
+                classes_map = {class_.id: class_.name for class_ in classes}
+                category_ids = [_class.id for _class in classes]
+                if category_id_result in category_ids:
+                    category_name = classes_map[category_id_result]
+                    category_id = category_id_result
+
         except Exception:
             logging.error(f"Failed to parse result text: {result_text}")
         try:
@@ -81,17 +89,15 @@ class QuestionClassifierNode(LLMNode):
                 'usage': jsonable_encoder(usage),
             }
             outputs = {
-                'class_name': categories[0] if categories else ''
+                'class_name': category_name
             }
-            classes = node_data.classes
-            classes_map = {class_.name: class_.id for class_ in classes}
 
             return NodeRunResult(
                 status=WorkflowNodeExecutionStatus.SUCCEEDED,
                 inputs=variables,
                 process_data=process_data,
                 outputs=outputs,
-                edge_source_handle=classes_map.get(categories[0], None),
+                edge_source_handle=category_id,
                 metadata={
                     NodeRunMetadataKey.TOTAL_TOKENS: usage.total_tokens,
                     NodeRunMetadataKey.TOTAL_PRICE: usage.total_price,
@@ -210,8 +216,13 @@ class QuestionClassifierNode(LLMNode):
             -> Union[list[ChatModelMessage], CompletionModelPromptTemplate]:
         model_mode = ModelMode.value_of(node_data.model.mode)
         classes = node_data.classes
-        class_names = [class_.name for class_ in classes]
-        class_names_str = ','.join(f'"{name}"' for name in class_names)
+        categories = []
+        for class_ in classes:
+            category = {
+                'category_id': class_.id,
+                'category_name': class_.name
+            }
+            categories.append(category)
         instruction = node_data.instruction if node_data.instruction else ''
         input_text = query
         memory_str = ''
@@ -248,7 +259,7 @@ class QuestionClassifierNode(LLMNode):
             user_prompt_message_3 = ChatModelMessage(
                 role=PromptMessageRole.USER,
                 text=QUESTION_CLASSIFIER_USER_PROMPT_3.format(input_text=input_text,
-                                                              categories=class_names_str,
+                                                              categories=json.dumps(categories),
                                                               classification_instructions=instruction)
             )
             prompt_messages.append(user_prompt_message_3)
@@ -257,7 +268,7 @@ class QuestionClassifierNode(LLMNode):
             return CompletionModelPromptTemplate(
                 text=QUESTION_CLASSIFIER_COMPLETION_PROMPT.format(histories=memory_str,
                                                                   input_text=input_text,
-                                                                  categories=class_names_str,
+                                                                  categories=json.dumps(categories),
                                                                   classification_instructions=instruction)
             )
 

api/core/workflow/nodes/question_classifier/template_prompts.py

@@ -6,7 +6,7 @@ QUESTION_CLASSIFIER_SYSTEM_PROMPT = """
     ### Task
     Your task is to assign one categories ONLY to the input text and only one category may be assigned returned in the output.Additionally, you need to extract the key words from the text that are related to the classification.
     ### Format
-    The input text is in the variable text_field.Categories are specified as a comma-separated list in the variable categories or left empty for automatic determination.Classification instructions may be included to improve the classification accuracy.
+    The input text is in the variable text_field.Categories are specified as a category list with two filed category_id and category_name in the variable categories .Classification instructions may be included to improve the classification accuracy.
     ### Constraint
     DO NOT include anything other than the JSON array in your response.
     ### Memory
@@ -18,33 +18,35 @@ QUESTION_CLASSIFIER_SYSTEM_PROMPT = """
 
 QUESTION_CLASSIFIER_USER_PROMPT_1 = """
     { "input_text": ["I recently had a great experience with your company. The service was prompt and the staff was very friendly."],
-    "categories": ["Customer Service", "Satisfaction", "Sales", "Product"],
+    "categories": [{"category_id":"f5660049-284f-41a7-b301-fd24176a711c","category_name":"Customer Service"},{"category_id":"8d007d06-f2c9-4be5-8ff6-cd4381c13c60","category_name":"Satisfaction"},{"category_id":"5fbbbb18-9843-466d-9b8e-b9bfbb9482c8","category_name":"Sales"},{"category_id":"23623c75-7184-4a2e-8226-466c2e4631e4","category_name":"Product"}],
     "classification_instructions": ["classify the text based on the feedback provided by customer"]}
 """
 
 QUESTION_CLASSIFIER_ASSISTANT_PROMPT_1 = """
 ```json
     {"keywords": ["recently", "great experience", "company", "service", "prompt", "staff", "friendly"],
-    "categories": ["Customer Service"]}
+    "category_id": "f5660049-284f-41a7-b301-fd24176a711c",
+    "category_name": "Customer Service"}
 ```
 """
 
 QUESTION_CLASSIFIER_USER_PROMPT_2 = """
     {"input_text": ["bad service, slow to bring the food"],
-    "categories": ["Food Quality", "Experience", "Price" ], 
+    "categories": [{"category_id":"80fb86a0-4454-4bf5-924c-f253fdd83c02","category_name":"Food Quality"},{"category_id":"f6ff5bc3-aca0-4e4a-8627-e760d0aca78f","category_name":"Experience"},{"category_id":"cc771f63-74e7-4c61-882e-3eda9d8ba5d7","category_name":"Price"}],
     "classification_instructions": []}
 """
 
 QUESTION_CLASSIFIER_ASSISTANT_PROMPT_2 = """
 ```json
     {"keywords": ["bad service", "slow", "food", "tip", "terrible", "waitresses"],
-    "categories": ["Experience"]}
+    "category_id": "f6ff5bc3-aca0-4e4a-8627-e760d0aca78f",
+    "category_name": "Experience"}
 ```
 """
 
 QUESTION_CLASSIFIER_USER_PROMPT_3 = """
     '{{"input_text": ["{input_text}"],',
-    '"categories": ["{categories}" ], ',
+    '"categories": {categories}, ',
     '"classification_instructions": ["{classification_instructions}"]}}'
 """
 
@@ -54,16 +56,16 @@ You are a text classification engine that analyzes text data and assigns categor
 ### Task
 Your task is to assign one categories ONLY to the input text and only one category may be assigned returned in the output.  Additionally, you need to extract the key words from the text that are related to the classification.
 ### Format
-The input text is in the variable text_field. Categories are specified as a comma-separated list in the variable categories or left empty for automatic determination. Classification instructions may be included to improve the classification accuracy. 
+The input text is in the variable text_field. Categories are specified as a category list in the variable categories or left empty for automatic determination. Classification instructions may be included to improve the classification accuracy. 
 ### Constraint 
 DO NOT include anything other than the JSON array in your response.
 ### Example
 Here is the chat example between human and assistant, inside <example></example> XML tags.
 <example>
-User:{{"input_text": ["I recently had a great experience with your company. The service was prompt and the staff was very friendly."],"categories": ["Customer Service, Satisfaction, Sales, Product"], "classification_instructions": ["classify the text based on the feedback provided by customer"]}}
-Assistant:{{"keywords": ["recently", "great experience", "company", "service", "prompt", "staff", "friendly"],"categories": ["Customer Service"]}}
-User:{{"input_text": ["bad service, slow to bring the food"],"categories": ["Food Quality, Experience, Price" ], "classification_instructions": []}}
-Assistant:{{"keywords": ["recently", "great experience", "company", "service", "prompt", "staff", "friendly"],"categories": ["Customer Service"]}}{{"keywords": ["bad service", "slow", "food", "tip", "terrible", "waitresses"],"categories": ["Experience""]}}
+User:{{"input_text": ["I recently had a great experience with your company. The service was prompt and the staff was very friendly."], "categories": [{{"category_id":"f5660049-284f-41a7-b301-fd24176a711c","category_name":"Customer Service"}},{{"category_id":"8d007d06-f2c9-4be5-8ff6-cd4381c13c60","category_name":"Satisfaction"}},{{"category_id":"5fbbbb18-9843-466d-9b8e-b9bfbb9482c8","category_name":"Sales"}},{{"category_id":"23623c75-7184-4a2e-8226-466c2e4631e4","category_name":"Product"}}], "classification_instructions": ["classify the text based on the feedback provided by customer"]}}
+Assistant:{{"keywords": ["recently", "great experience", "company", "service", "prompt", "staff", "friendly"],"category_id": "f5660049-284f-41a7-b301-fd24176a711c","category_name": "Customer Service"}}
+User:{{"input_text": ["bad service, slow to bring the food"], "categories": [{{"category_id":"80fb86a0-4454-4bf5-924c-f253fdd83c02","category_name":"Food Quality"}},{{"category_id":"f6ff5bc3-aca0-4e4a-8627-e760d0aca78f","category_name":"Experience"}},{{"category_id":"cc771f63-74e7-4c61-882e-3eda9d8ba5d7","category_name":"Price"}}], "classification_instructions": []}}
+Assistant:{{"keywords": ["bad service", "slow", "food", "tip", "terrible", "waitresses"],"category_id": "f6ff5bc3-aca0-4e4a-8627-e760d0aca78f","category_name": "Customer Service"}}
 </example> 
 ### Memory
 Here is the chat histories between human and assistant, inside <histories></histories> XML tags.
@@ -71,6 +73,6 @@ Here is the chat histories between human and assistant, inside <histories></hist
 {histories}
 </histories>
 ### User Input
-{{"input_text" : ["{input_text}"], "categories" : ["{categories}"],"classification_instruction" : ["{classification_instructions}"]}}
+{{"input_text" : ["{input_text}"], "categories" : {categories},"classification_instruction" : ["{classification_instructions}"]}}
 ### Assistant Output
 """

api/core/workflow/nodes/tool/tool_node.py

@@ -70,13 +70,14 @@ class ToolNode(BaseNode):
             )
 
         # convert tool messages
-        plain_text, files = self._convert_tool_messages(messages)
+        plain_text, files, chunks = self._convert_tool_messages(messages)
 
         return NodeRunResult(
             status=WorkflowNodeExecutionStatus.SUCCEEDED,
             outputs={
                 'text': plain_text,
-                'files': files
+                'files': files,
+                'chunks': chunks
             },
             metadata={
                 NodeRunMetadataKey.TOOL_INFO: tool_info
@@ -111,7 +112,7 @@ class ToolNode(BaseNode):
         
         return template_parser.format(inputs)
 
-    def _convert_tool_messages(self, messages: list[ToolInvokeMessage]) -> tuple[str, list[FileVar]]:
+    def _convert_tool_messages(self, messages: list[ToolInvokeMessage]) -> tuple[str, list[FileVar], list]:
         """
         Convert ToolInvokeMessages into tuple[plain_text, files]
         """
@@ -125,8 +126,9 @@ class ToolNode(BaseNode):
         # extract plain text and files
         files = self._extract_tool_response_binary(messages)
         plain_text = self._extract_tool_response_text(messages)
+        chunks = self._extract_tool_response_chunk(messages)
 
-        return plain_text, files
+        return plain_text, files, chunks
 
     def _extract_tool_response_binary(self, tool_response: list[ToolInvokeMessage]) -> list[FileVar]:
         """
@@ -180,6 +182,30 @@ class ToolNode(BaseNode):
             for message in tool_response
         ])
 
+    def _extract_tool_response_chunk(self, tool_response: list[ToolInvokeMessage]) -> list:
+        """
+        Extract tool response text
+        """
+        all_chunks = []
+        node_data = cast(ToolNodeData, self.node_data)
+        icon = ToolManager.get_tool_icon(
+            tenant_id=self.tenant_id,
+            provider_type=node_data.provider_type,
+            provider_id=node_data.provider_id
+        )
+        for message in tool_response:
+            if message.type == ToolInvokeMessage.MessageType.CHUNK:
+                for chunk in message.message:
+                    chunk.icon = icon
+                    chunk.resource_from = node_data.title
+                    chunk.metadata = {
+                        '_source': 'tool'
+                    }
+                    all_chunks.append(chunk.to_dict())
+        return all_chunks
+
+
+
     @classmethod
     def _extract_variable_selector_to_variable_mapping(cls, node_data: ToolNodeData) -> dict[str, list[str]]:
         """

api/extensions/storage/aliyun_storage.py

@@ -13,13 +13,21 @@ class AliyunStorage(BaseStorage):
 
     def __init__(self, app: Flask):
         super().__init__(app)
+
         app_config = self.app.config
         self.bucket_name = app_config.get('ALIYUN_OSS_BUCKET_NAME')
+        oss_auth_method = aliyun_s3.Auth
+        region = None
+        if app_config.get('ALIYUN_OSS_AUTH_VERSION') == 'v4':
+            oss_auth_method = aliyun_s3.AuthV4
+            region = app_config.get('ALIYUN_OSS_REGION')
+        oss_auth = oss_auth_method(app_config.get('ALIYUN_OSS_ACCESS_KEY'), app_config.get('ALIYUN_OSS_SECRET_KEY'))
         self.client = aliyun_s3.Bucket(
-            aliyun_s3.Auth(app_config.get('ALIYUN_OSS_ACCESS_KEY'), app_config.get('ALIYUN_OSS_SECRET_KEY')),
+            oss_auth,
             app_config.get('ALIYUN_OSS_ENDPOINT'),
             self.bucket_name,
-            connect_timeout=30
+            connect_timeout=30,
+            region=region,
         )
 
     def save(self, filename, data):

api/pyproject.toml

@@ -55,4 +55,6 @@ HUGGINGFACE_TEXT_GEN_ENDPOINT_URL = "a"
 HUGGINGFACE_TEXT2TEXT_GEN_ENDPOINT_URL = "b"
 HUGGINGFACE_EMBEDDINGS_ENDPOINT_URL = "c"
 MOCK_SWITCH = "true"
-CODE_MAX_STRING_LENGTH = "80000"
+CODE_MAX_STRING_LENGTH = "80000"
+CODE_EXECUTION_ENDPOINT="http://127.0.0.1:8194"
+CODE_EXECUTION_API_KEY="dify-sandbox"

api/requirements.txt

@@ -9,7 +9,7 @@ flask-restful~=0.3.10
 flask-cors~=4.0.0
 gunicorn~=22.0.0
 gevent~=23.9.1
-openai~=1.13.3
+openai~=1.26.0
 tiktoken~=0.6.0
 psycopg2-binary~=2.9.6
 pycryptodome==3.19.1

api/services/agent_service.py

@@ -1,3 +1,6 @@
+import pytz
+from flask_login import current_user
+
 from core.app.app_config.easy_ui_based_app.agent.manager import AgentConfigManager
 from core.tools.tool_manager import ToolManager
 from extensions.ext_database import db
@@ -46,11 +49,13 @@ class AgentService:
         else:
             executor = 'Unknown'
 
+        timezone = pytz.timezone(current_user.timezone)
+
         result = {
             'meta': {
                 'status': 'success',
                 'executor': executor,
-                'start_time': message.created_at.isoformat(),
+                'start_time': message.created_at.astimezone(timezone).isoformat(),
                 'elapsed_time': message.provider_response_latency,
                 'total_tokens': message.answer_tokens + message.message_tokens,
                 'agent_mode': app_model.app_model_config.agent_mode_dict.get('strategy', 'react'),

api/services/dataset_service.py

@@ -43,6 +43,7 @@ from services.vector_service import VectorService
 from tasks.clean_notion_document_task import clean_notion_document_task
 from tasks.deal_dataset_vector_index_task import deal_dataset_vector_index_task
 from tasks.delete_segment_from_index_task import delete_segment_from_index_task
+from tasks.disable_segment_from_index_task import disable_segment_from_index_task
 from tasks.document_indexing_task import document_indexing_task
 from tasks.document_indexing_update_task import document_indexing_update_task
 from tasks.duplicate_document_indexing_task import duplicate_document_indexing_task
@@ -131,13 +132,9 @@ class DatasetService:
 
     @staticmethod
     def get_dataset(dataset_id):
-        dataset = Dataset.query.filter_by(
+        return Dataset.query.filter_by(
             id=dataset_id
         ).first()
-        if dataset is None:
-            return None
-        else:
-            return dataset
 
     @staticmethod
     def check_dataset_model_setting(dataset):
@@ -1245,6 +1242,25 @@ class SegmentService:
         cache_result = redis_client.get(indexing_cache_key)
         if cache_result is not None:
             raise ValueError("Segment is indexing, please try again later")
+        if 'enabled' in args and args['enabled'] is not None:
+            action = args['enabled']
+            if segment.enabled != action:
+                if not action:
+                    segment.enabled = action
+                    segment.disabled_at = datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None)
+                    segment.disabled_by = current_user.id
+                    db.session.add(segment)
+                    db.session.commit()
+                    # Set cache to prevent indexing the same segment multiple times
+                    redis_client.setex(indexing_cache_key, 600, 1)
+                    disable_segment_from_index_task.delay(segment.id)
+                    return segment
+        if not segment.enabled:
+            if 'enabled' in args and args['enabled'] is not None:
+                if not args['enabled']:
+                    raise ValueError("Can't update disabled segment")
+            else:
+                raise ValueError("Can't update disabled segment")
         try:
             content = args['content']
             if segment.content == content:
@@ -1252,8 +1268,9 @@ class SegmentService:
                     segment.answer = args['answer']
                 if 'keywords' in args and args['keywords']:
                     segment.keywords = args['keywords']
-                if 'enabled' in args and args['enabled'] is not None:
-                    segment.enabled = args['enabled']
+                segment.enabled = True
+                segment.disabled_at = None
+                segment.disabled_by = None
                 db.session.add(segment)
                 db.session.commit()
                 # update segment index task
@@ -1298,12 +1315,16 @@ class SegmentService:
                 segment.completed_at = datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None)
                 segment.updated_by = current_user.id
                 segment.updated_at = datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None)
+                segment.enabled = True
+                segment.disabled_at = None
+                segment.disabled_by = None
                 if document.doc_form == 'qa_model':
                     segment.answer = args['answer']
                 db.session.add(segment)
                 db.session.commit()
                 # update segment vector index
                 VectorService.update_segment_vector(args['keywords'], segment, dataset)
+
         except Exception as e:
             logging.exception("update segment index failed")
             segment.enabled = False