如何删除重复的行并聚合相应的值

时间:2017-02-03 12:02:20

标签: bash shell

由于使用JMeter的CMDRunner.jar进行了一段时间内提供延迟/请求的性能测试,因此我使用了逗号分隔值的CSV。

2017/02/03 11:15:41.593,15,End-to-end request-response,15
2017/02/03 11:15:41.609,6,Request body is proxied successfully to the backend,6
2017/02/03 11:15:41.616,5,x-junction-path validator,5
2017/02/03 11:15:41.622,9,Invalid content type sent by client ,9
2017/02/03 11:15:41.634,3,Invalid content type requested by client ,3
2017/02/03 11:15:42.595,10,End-to-end request-response,10
2017/02/03 11:15:42.606,10,Request body is proxied successfully to the backend,10
2017/02/03 11:15:42.616,9,x-junction-path validator,9
2017/02/03 11:15:42.625,8,Invalid content type sent by client ,8
2017/02/03 11:15:42.635,5,Invalid content type requested by client ,4
2017/02/03 11:15:43.599,3,End-to-end request-response,3
2017/02/03 11:15:43.603,6,Request body is proxied successfully to the backend,6
2017/02/03 11:15:43.609,7,x-junction-path validator,7
2017/02/03 11:15:43.617,4,Invalid content type sent by client ,4
2017/02/03 11:15:43.622,7,Invalid content type requested by client ,7

我想聚合每个请求的延迟,这意味着每个请求只有一个条目和相应的总延迟,即。分别为第3和第4列。是否有jmeter插件来获取此结果,或者我如何在BASH中执行此操作?

预期输出(例如):

注意:由于这是一个聚合,因此时间戳和经过时间(第二列)无关。

End-to-end request-response,12.31
equest body is proxied successfully to the backend,6.1
x-junction-path validator,5.0
Invalid content type sent by client, 3.12
Invalid content type requested by client ,3.01

1 个答案:

答案 0 :(得分:2)

Awk可以帮助您解决此问题,

awk 'BEGIN{FS=OFS=","}{unique[$3]+=$4; count[$3]++;}END{for (i in unique) print i, unique[i]/count[i]}' file
x-junction-path validator,7
Invalid content type requested by client ,4.66667
Invalid content type sent by client ,7
Request body is proxied successfully to the backend,7.33333
End-to-end request-response,9.33333

也可以printf.2度精度浮点运算

awk 'BEGIN{FS=","}{unique[$3]+=$4; count[$3]++;}END{for (i in unique) printf "%s,%0.2f\n", i, unique[i]/count[i]}' file
x-junction-path validator,7.00
Invalid content type requested by client ,4.67
Invalid content type sent by client ,7.00
Request body is proxied successfully to the backend,7.33
End-to-end request-response,9.33