- Align benchmark default config:
1. Update default helm charts version.
2. Add `# mandatory` comment.
3. Update default model ID for LLM.
- Fix deploy issue:
1. Support different `replicaCount` for w/ w/o rerank test.
2. Add `max_num_seqs` for vllm.
3. Add resource setting for tune mode.
- Fix Benchmark issue:
1. Update `user_queries` and `concurrency` setting.
2. Remove invalid parameters.
3. Fix `dataset` and `prompt` setting. And dataset ingest into db.
5. Fix the benchmark hang issue with large user queries. Update `"processes": 16` will fix this issue.
6. Update the eval_path setting logical.
- Optimize benchmark readme.
- Optimize the log path to make the logs more readable.
Signed-off-by: chensuyue <suyue.chen@intel.com>
Signed-off-by: Cathy Zhang <cathy.zhang@intel.com>
Signed-off-by: letonghan <letong.han@intel.com>
