具有MapR存储池的TestDFSIO

时间:2018-08-16 13:43:11

标签: hadoop benchmarking hadoop2 mapr

我想计算有无MapR storage pools的MapR-FS的性能。 使用TestDFSIO我得到“奇怪”的结果:

  • 没有存储池:

    18/08/16 11:44:43 INFO fs.TestDFSIO: ----- TestDFSIO ----- : write
    18/08/16 11:44:43 INFO fs.TestDFSIO: Date & time: Thu Aug 16 11:44:43 UTC 2018
    18/08/16 11:44:43 INFO fs.TestDFSIO: Number of files: 100
    18/08/16 11:44:43 INFO fs.TestDFSIO: Total MBytes processed: 10000.0
    18/08/16 11:44:43 INFO fs.TestDFSIO: Throughput mb/sec: 871.1560240439063
    18/08/16 11:44:43 INFO fs.TestDFSIO: Average IO rate mb/sec: 963.9425048828125
    18/08/16 11:44:43 INFO fs.TestDFSIO: IO rate std deviation: 246.8920002354248
    18/08/16 11:44:43 INFO fs.TestDFSIO: Test exec time sec: 157.62
    
  • 具有存储池(默认3个磁盘):

    18/08/16 11:42:54 INFO fs.TestDFSIO: ----- TestDFSIO ----- : write 
    18/08/16 11:42:54 INFO fs.TestDFSIO: Date & time: Thu Aug 16 11:42:54 UTC 2018
    18/08/16 11:42:54 INFO fs.TestDFSIO: Number of files: 100
    18/08/16 11:42:54 INFO fs.TestDFSIO: Total MBytes processed: 10000.0
    18/08/16 11:42:54 INFO fs.TestDFSIO: Throughput mb/sec: 325.2561392096276
    18/08/16 11:42:54 INFO fs.TestDFSIO: Average IO rate mb/sec: 436.1356506347656
    18/08/16 11:42:54 INFO fs.TestDFSIO: IO rate std deviation: 197.1355193652064
    18/08/16 11:42:54 INFO fs.TestDFSIO: Test exec time sec: 46.616
    

使用存储池时,测试速度要快得多(47 s和158 s),但吞吐量和平均IO速率更差。是否应将它们乘以存储池中的磁盘数量(在我的情况下为3)?

0 个答案:

没有答案