响应时间相对于测试持续时间的变化

Question

我只想了解以下情况：

场景01：当我用50个用户运行测试10分钟时，交易的第90百分位响应时间为10秒。但是，当我运行相同的测试20分钟时，这个时间响应时间是7秒。我已经多次观察到响应时间的这种变化。为什么响应时间（第90次）随着运行持续时间的变化而不断变化

场景02：当用户加载为50时，一个特定事务的响应时间表示登录为10秒。但是，当我用100个用户运行测试时，登录事务的响应时间为6秒。请解释一下

Answer 1

你有一个独立的统计数据，响应时间。但是你没有考虑使用能够推动响应时间的资源。您也在寻找非常短的窗口，因此它们的样本集较小，它们在统计上是否有效？您是否还有其他问题，例如您的测试设计缺乏控制因素，缺乏对预期结果的检查或零思考时间或起搏模型 - 即使是不受控制的初始和测试条件也可能会影响这导致了不可重复的结果。

考虑附加的小部件以及这对您的数据收集意味着什么？分析

Answer 2

There could be multiple explanations, unfortunately your numbers are not enough to tell for sure what is the reason behind. The possible causes could be in:

1. Ramp-up time. If users are arriving gradually application can adopt itself for increasing load due to JIT compilation or warming up caches, for instance first X transactions take longer as application is not ready for them but as you continue the application performs internal optimisations therefore subsequent transactions take less time
2. The inconsistency in results might be caused by JMeter itself as if JMeter is not capable of sending requests fast enough - it will result in lower throughput and increased response times so make sure to follow JMeter Best Practices

Get used to monitor baseline OS health metrics, to wit CPU, RAM, Disk, Network, Swap usage, etc. on both application under test and JMeter side as it might be the case that there is a banal lack of resources which impacting the performance or there is another activity which consumes resources so results are not consistent. You can use JMeter PerfMon Plugin for this.