Wang, Kai Lawrence 3d3ac59bfb [ChatQnA] Update the default LLM to llama3-8B on cpu/gpu/hpu (#1430) ...

Update the default LLM to llama3-8B on cpu/nvgpu/amdgpu/gaudi for docker-compose deployment to avoid the potential model serving issue or the missing chat-template issue using neural-chat-7b.

Slow serving issue of neural-chat-7b on ICX: #1420
Signed-off-by: Wang, Kai Lawrence <kai.lawrence.wang@intel.com>

2025-01-20 22:47:56 +08:00

..

rocm

[ChatQnA] Update the default LLM to llama3-8B on cpu/gpu/hpu (#1430)

2025-01-20 22:47:56 +08:00