测试使用Kafka自带的测试脚本，通过命令对Kafka发起写入MQ消息和Kafka消费MQ消息的请求。模拟10w、级别的消息处理结果
./bin/kafka-producer-perf-test.sh --topic test_perf --num-records 100000 --record-size 1000 --throughput 2000 --producer-props bootstrap.servers=localhost:9092
本地集群测试10000 records sent, 2000.0 records/sec (1.91 MB/sec), 37.9 ms avg latency, 981.0 ms max latency.10006 records sent, 2001.2 records/sec (1.91 MB/sec), 0.8 ms avg latency, 11.0 ms max latency.10004 records sent, 2000.4 records/sec (1.91 MB/sec), 0.7 ms avg latency, 13.0 ms max latency.10005 records sent, 2000.2 records/sec (1.91 MB/sec), 0.6 ms avg latency, 8.0 ms max latency.10001 records sent, 2000.2 records/sec (1.91 MB/sec), 0.6 ms avg latency, 4.0 ms max latency.10005 records sent, 2000.6 records/sec (1.91 MB/sec), 0.6 ms avg latency, 12.0 ms max latency.10002 records sent, 2000.4 records/sec (1.91 MB/sec), 0.6 ms avg latency, 4.0 ms max latency.10001 records sent, 2000.2 records/sec (1.91 MB/sec), 0.7 ms avg latency, 5.0 ms max latency.10001 records sent, 2000.2 records/sec (1.91 MB/sec), 0.7 ms avg latency, 8.0 ms max latency.100000 records sent, 1999.320231 records/sec (1.91 MB/sec), 4.39 ms avg latency, 981.00 ms max latency, 1 ms 50th, 2 ms 95th, 171 ms 99th, 224 ms 99.9th.10000 records sent, 2000.0 records/sec (1.91 MB/sec), 17.9 ms avg latency, 562.0 ms max latency.10004 records sent, 2000.4 records/sec (1.91 MB/sec), 0.8 ms avg latency, 14.0 ms max latency.10004 records sent, 2000.8 records/sec (1.91 MB/sec), 0.7 ms avg latency, 10.0 ms max latency.10002 records sent, 2000.0 records/sec (1.91 MB/sec), 0.7 ms avg latency, 11.0 ms max latency.10004 records sent, 2000.4 records/sec (1.91 MB/sec), 0.6 ms avg latency, 4.0 ms max latency.10004 records sent, 2000.4 records/sec (1.91 MB/sec), 0.7 ms avg latency, 5.0 ms max latency.10004 records sent, 2000.4 records/sec (1.91 MB/sec), 0.7 ms avg latency, 4.0 ms max latency.10002 records sent, 2000.4 records/sec (1.91 MB/sec), 0.7 ms avg latency, 12.0 ms max latency.10004 records sent, 2000.8 records/sec (1.91 MB/sec), 0.6 ms avg latency, 7.0 ms max latency.100000 records sent, 1999.360205 records/sec (1.91 MB/sec), 2.40 ms avg latency, 562.00 ms max latency, 1 ms 50th, 2 ms 95th, 97 ms 99th, 161 ms 99.9th.10002 records sent, 2000.0 records/sec (1.91 MB/sec), 4.1 ms avg latency, 466.0 ms max latency.10004 records sent, 2000.4 records/sec (1.91 MB/sec), 0.7 ms avg latency, 17.0 ms max latency.10004 records sent, 2000.4 records/sec (1.91 MB/sec), 0.6 ms avg latency, 4.0 ms max latency.10002 records sent, 2000.0 records/sec (1.91 MB/sec), 0.6 ms avg latency, 4.0 ms max latency.10005 records sent, 2001.0 records/sec (1.91 MB/sec), 0.6 ms avg latency, 7.0 ms max latency.9997 records sent, 1999.4 records/sec (1.91 MB/sec), 0.7 ms avg latency, 14.0 ms max latency.10002 records sent, 2000.0 records/sec (1.91 MB/sec), 0.6 ms avg latency, 4.0 ms max latency.10002 records sent, 2000.4 records/sec (1.91 MB/sec), 0.6 ms avg latency, 5.0 ms max latency.10003 records sent, 2000.6 records/sec (1.91 MB/sec), 0.6 ms avg latency, 4.0 ms max latency.100000 records sent, 1999.480135 records/sec (1.91 MB/sec), 0.98 ms avg latency, 466.00 ms max latency, 1 ms 50th, 1 ms 95th, 22 ms 99th, 39 ms 99.9th.
挂载nfs 后测试
10002 records sent, 1999.6 records/sec (1.91 MB/sec), 18.9 ms avg latency, 497.0 ms max latency.10007 records sent, 2001.4 records/sec (1.91 MB/sec), 0.8 ms avg latency, 11.0 ms max latency.10001 records sent, 1999.8 records/sec (1.91 MB/sec), 0.7 ms avg latency, 15.0 ms max latency.10004 records sent, 2000.4 records/sec (1.91 MB/sec), 0.6 ms avg latency, 3.0 ms max latency.9998 records sent, 1999.6 records/sec (1.91 MB/sec), 0.7 ms avg latency, 13.0 ms max latency.10004 records sent, 2000.0 records/sec (1.91 MB/sec), 0.7 ms avg latency, 5.0 ms max latency.10006 records sent, 2001.2 records/sec (1.91 MB/sec), 0.6 ms avg latency, 10.0 ms max latency.10002 records sent, 2000.0 records/sec (1.91 MB/sec), 0.6 ms avg latency, 5.0 ms max latency.10002 records sent, 2000.4 records/sec (1.91 MB/sec), 0.6 ms avg latency, 12.0 ms max latency.100000 records sent, 1999.560097 records/sec (1.91 MB/sec), 2.50 ms avg latency, 497.00 ms max latency, 1 ms 50th, 2 ms 95th, 117 ms 99th, 180 ms 99.9th.10002 records sent, 2000.0 records/sec (1.91 MB/sec), 4.6 ms avg latency, 459.0 ms max latency.10010 records sent, 2001.2 records/sec (1.91 MB/sec), 0.7 ms avg latency, 11.0 ms max latency.10003 records sent, 2000.6 records/sec (1.91 MB/sec), 0.7 ms avg latency, 5.0 ms max latency.9999 records sent, 1999.8 records/sec (1.91 MB/sec), 0.7 ms avg latency, 18.0 ms max latency.10002 records sent, 2000.4 records/sec (1.91 MB/sec), 0.7 ms avg latency, 6.0 ms max latency.10002 records sent, 2000.4 records/sec (1.91 MB/sec), 0.7 ms avg latency, 10.0 ms max latency.10004 records sent, 2000.4 records/sec (1.91 MB/sec), 0.6 ms avg latency, 3.0 ms max latency.10003 records sent, 2000.6 records/sec (1.91 MB/sec), 0.7 ms avg latency, 9.0 ms max latency.10003 records sent, 2000.2 records/sec (1.91 MB/sec), 0.6 ms avg latency, 3.0 ms max latency.100000 records sent, 1999.280259 records/sec (1.91 MB/sec), 1.06 ms avg latency, 459.00 ms max latency, 1 ms 50th, 1 ms 95th, 23 ms 99th, 45 ms 99.9th.10002 records sent, 2000.0 records/sec (1.91 MB/sec), 19.8 ms avg latency, 561.0 ms max latency.10008 records sent, 2001.2 records/sec (1.91 MB/sec), 0.8 ms avg latency, 12.0 ms max latency.10004 records sent, 2000.4 records/sec (1.91 MB/sec), 0.7 ms avg latency, 15.0 ms max latency.10002 records sent, 2000.0 records/sec (1.91 MB/sec), 0.6 ms avg latency, 12.0 ms max latency.10005 records sent, 2001.0 records/sec (1.91 MB/sec), 0.6 ms avg latency, 4.0 ms max latency.10003 records sent, 2000.2 records/sec (1.91 MB/sec), 0.6 ms avg latency, 5.0 ms max latency.10002 records sent, 2000.4 records/sec (1.91 MB/sec), 0.6 ms avg latency, 6.0 ms max latency.10002 records sent, 2000.4 records/sec (1.91 MB/sec), 0.6 ms avg latency, 13.0 ms max latency.10003 records sent, 2000.6 records/sec (1.91 MB/sec), 0.6 ms avg latency, 2.0 ms max latency.100000 records sent, 1999.240289 records/sec (1.91 MB/sec), 2.58 ms avg latency, 561.00 ms max latency, 1 ms 50th, 2 ms 95th, 105 ms 99th, 182 ms 99.9th.
10w 写入测试结果分析