【昇腾】MindIE - Benchmark

1. MindIE 服务化

1.1 环境准备

参数说明：

device用于挂载卡，下面的例子是挂载了8张卡

倒数第二行的镜像名称记得修改

docker run -itd --privileged  --name=mindie --net=host \
   --shm-size 500g \
   --device=/dev/davinci0 \
   --device=/dev/davinci1 \
   --device=/dev/davinci2 \
   --device=/dev/davinci3 \
   --device=/dev/davinci4 \
   --device=/dev/davinci5 \
   --device=/dev/davinci6 \
   --device=/dev/davinci7 \
   --device=/dev/davinci_manager \
   --device=/dev/hisi_hdc \
   --device /dev/devmm_svm \
   -v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
   -v /usr/local/Ascend/firmware:/usr/local/Ascend/firmware \
   -v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi \
   -v /usr/local/sbin:/usr/local/sbin \
   -v /etc/hccn.conf:/etc/hccn.conf \
   swr.cn-south-1.myhuaweicloud.com/ascendhub/mindie:1.0.0-800I-A2-py311-openeuler24.03-lts \
   bash

# 进入容器
 docker exec -it mindie bash

1.2 MindIE基础配置

cd /usr/local/Ascend/mindie/latest/mindie-service/
vim conf/config.json

# 几个参数需要修改：
httpsEnabled: false
modelName: 类似vllm的modelName，用于标识模型，可以随意配置
modelWeightPath: 本地模型路径
npuDeviceIds: 使用的NPU卡，默认[[0,1,2,3]]，暂时可不修改
worldSize: 使用的卡数，和npuDeviceIds对应，默认4

1
2
3

# 启动MindIE
cd /usr/local/Ascend/mindie/latest/mindie-service
./bin/mindieservice_daemon

测试服务化

curl -H "Accept: application/json" -H "Content-type: application/json"  -X POST -d '{
    "prompt": "你是谁",
    "max_tokens": 200,
    "repetition_penalty": 1.03,
    "presence_penalty": 1.2,
    "frequency_penalty": 1.2,
    "temperature": 0.5,
    "top_k": 10,
    "top_p": 0.95,
    "stream": false,
    "ignore_eos": false
}' http://127.0.0.1:1025/generate

‍

2. Benchmark

2.1 配置

# 配置文件权限问题
chmod 640  /usr/local/lib/python3.11/site-packages/mindieclient/python/config/config.json
# 日志打屏
export MINDIE_LOG_TO_STDOUT="benchmark:1; client:1"

2.2 数据集

2.1 合成数据（性能测试）

可以指定输入输出的token数量
生成的数据为A[空格]，如输入token数为5: “A A A A A”
输出token数不会因为结束符而停止。可以根据真实场景的输出更好的测试性能。

在任意位置创建：vim synthetic_config.json

样例

{
    "Input":{
        "Method": "uniform",
        "Params": {"MinValue": 1, "MaxValue": 200}
    },
    "Output": {
        "Method": "gaussian",
        "Params": {"Mean": 100, "Var": 200, "MinValue": 1, "MaxValue": 100}
    },
    "RequestCount": 100 
}

参数	含义	取值范围
Input	输入配置	-
Output	输出配置	-
RequestCount	请求次数，即样本数量	[1,1048576]
Method	采样方法	取 “uniform”、”gaussian”或”zipf”。
Params	采样方法中对应的采样参数	取值详情请参见表2。
“Input” 中的 “MinValue”	token 数量最小值	[1,1048576]
“Input” 中的 “MaxValue”	token 数量最大值	[1,1048576]
“Output” 中的 “MinValue”	token 数量最小值	[1,1048576]
“Output” 中的 “MaxValue”	token 数量最大值	[1,1048576]
“gaussian” 中的 “Mean”	高斯分布均值	[-3.0 x 10^38, 3.0 x 10^38]
“gaussian” 中的 “Var”	高斯分布方差	[0, 3.0 x 10^38]
“zipf” 中的 “Alpha”	zipf分布Alpha系数	(1.0,10.0]
注：1048576 = 2^20 = 1 M。

Benchmark命令： 合成数据配置路径使用SyntheticConfigPath指定

benchmark \
--DatasetType "synthetic" \
--ModelName llama_7b \
--ModelPath "/{模型权重路径}/llama_7b" \
--TestType vllm_client \
--Http https://{ipAddress}:{port} \
--ManagementHttp https://{managementIpAddress}:{managementPort} \
--Concurrency 128 \
--MaxOutputLen 20 \
--TaskKind stream \
--Tokenizer True \
--SyntheticConfigPath /{配置文件路径}/synthetic_config.json

测试环境：910B、四卡、DeepSeek-R1-Qwen-32B

# 配置：
{
    "Input":{
        "Method": "uniform",
        "Params": {"MinValue": 500, "MaxValue": 1000}
    },
    "Output": {
        "Method": "gaussian",
        "Params": {"Mean": 100, "Var": 200, "MinValue": 1, "MaxValue": 100}
    },
    "RequestCount": 2000
}

# 输出
+---------------------+----------------+-----------------+----------------+---------------+----------------+----------------+-----------------+------+
|              Metric |        average |             max |            min |           P75 |            P90 |        SLO_P90 |             P99 |    N |
+---------------------+----------------+-----------------+----------------+---------------+----------------+----------------+-----------------+------+
|      FirstTokenTime |    762.4963 ms |    9074.5516 ms |    118.6991 ms |   567.9139 ms |   1415.2258 ms |   1415.2258 ms |    8181.3668 ms | 2000 |
|          DecodeTime |    170.4084 ms |     8435.523 ms |      0.0157 ms |   246.6154 ms |     444.639 ms |    185.2339 ms |     767.7023 ms | 2000 |
|      LastDecodeTime |    262.8068 ms |     928.5161 ms |     20.0195 ms |   423.1779 ms |    617.8357 ms |    617.8357 ms |     911.0484 ms | 2000 |
|       MaxDecodeTime |   1279.4515 ms |     8435.523 ms |     75.1517 ms |   907.9784 ms |   2699.2586 ms |   2699.2586 ms |    7724.6093 ms | 2000 |
|        GenerateTime |  16700.8135 ms |   20314.2266 ms |   4299.7291 ms | 18216.5194 ms |  18719.7726 ms |  18719.7726 ms |   20156.1953 ms | 2000 |
|         InputTokens |         753.29 |             999 |            500 |         877.0 |          951.0 |          951.0 |          995.01 | 2000 |
|     GeneratedTokens |        94.5235 |             100 |             55 |         100.0 |          100.0 |          100.0 |           100.0 | 2000 |
| GeneratedTokenSpeed | 5.8279 token/s | 21.2943 token/s | 4.2824 token/s | 5.677 token/s | 6.0752 token/s | 6.0752 token/s | 14.8526 token/s | 2000 |
| GeneratedCharacters |        189.047 |             200 |            110 |         200.0 |          200.0 |          200.0 |           200.0 | 2000 |
|           Tokenizer |      4.7329 ms |      33.6573 ms |      1.3688 ms |     5.6392 ms |      6.5679 ms |      6.5679 ms |       8.5305 ms | 2000 |
|         Detokenizer |      1.0233 ms |       1.2207 ms |      0.6008 ms |      1.081 ms |       1.086 ms |       1.086 ms |       1.0986 ms | 2000 |
|  CharactersPerToken |            2.0 |               / |              / |             / |              / |              / |               / | 2000 |
|  PostProcessingTime |           0 ms |            0 ms |           0 ms |          0 ms |           0 ms |           0 ms |            0 ms | 2000 |
|         ForwardTime |           0 ms |            0 ms |           0 ms |          0 ms |           0 ms |           0 ms |            0 ms | 2000 |
+---------------------+----------------+-----------------+----------------+---------------+----------------+----------------+-----------------+------+
[2025-04-29 19:12:10.619+08:00] [13134] [281473497909536] [benchmark] [INFO] [output.py:121] 
The BenchMark test common metric result is:
+------------------------+---------------------+
|          Common Metric |               Value |
+------------------------+---------------------+
|            CurrentTime | 2025-04-29 19:12:10 |
|            TimeElapsed |          263.6115 s |
|             DataSource |                None |
|                 Failed |           0( 0.0% ) |
|               Returned |      2000( 100.0% ) |
|                  Total |      2000[ 100.0% ] |
|            Concurrency |                 128 |
|              ModelName |                 llm |
|                   lpct |           1.0122 ms |
|             Throughput |        7.5869 req/s |
|          GenerateSpeed |    717.1425 token/s |
| GenerateSpeedPerClient |      5.6027 token/s |
|               accuracy |                   / |
+------------------------+---------------------+

‍

本文采用署名-非商业性使用-相同方式共享 4.0 国际许可协议，转载请注明出处。