PHP + PThreads + Redis / Predis = zend_mm_heap已损坏?

时间:2017-03-27 18:15:21

标签: php multithreading redis pthreads predis

我几天来一直在为一个神秘的错误而苦苦挣扎。我正在使用PHP 7.1.0RC3(我使用 ZTS / pthreads启用重新编译了自己)。最近,我一直在研究一个用Redis取代MySQL的重构器,以优化我的应用程序中的非磁盘数据I / O.

我有一个脚本为每个加密货币市场创建一个线程(A)。线程(A)为每个交易策略创建另一个线程(B)。 B线程始终在A线程之前同步。

我一直收到此错误:zend_mm_heap corrupted。每次运行脚本时,它都会在执行的不同点发生。

我已经尝试了所有建议的修补程序,100个Google网页。垃圾收集,PHP配置/编译,所有内容都已经过详细审核。我还没找到任何与" zend_mm_heap损坏",PThreads和Redis / Predis,相关的内容,而其他"修复"没有效果......

从Redis读取所有数据并保存到Redis(通过Predis)。仅在线程A或线程B中使用Predis时出现错误消息(每次更改)。当两个Predis方法调用快速连续发生时(例如,设置为get),它会更频繁地发生。

如果我使用shell_exec()直接通过CLI运行Redis命令,我就不会看到此错误。但是,它明显变慢了,所以可能只是缺少这种边缘情况?

如果有人能对以下任何一方提出任何意见/想法,我将不胜感激:

  1. 如何在我的应用程序中找到它的来源?
  2. 专用的Redis服务器是否可以修复此问题?
  3. 我应该向https://github.com/krakjoe/pthreads提交错误报告吗?
  4. 我应该向https://github.com/nrk/predis提交错误报告吗?
  5. 有没有人知道为什么zend_mm_heap corrupted可能会发生,一般来说,在一个线程" safe"环境(又名:ZTS)?或者我该如何防范呢?
  6. 我应该尝试编译甚至更多不同版本的PHP 7+吗? https://github.com/php/php-src
  7. 我没有选择,请怜悯。我可以共享代码示例,但在这里粘贴它们不会非常有用。

    修改 我用./configure --enable-maintainer-zts --enable-pthreads --with-pthreads-sanitize --with-mysqli --enable-embedded-mysqli重新编译了PHP,我看到了一个我以前没见过的新错误:

    ==2716== ERROR: AddressSanitizer: heap-use-after-free on address 0x600800001c15 at pc 0xe25a87 bp 0x7f7a049feae0 sp 0x7f7a049fead0
    READ of size 1 at 0x600800001c15 thread T1632
        #0 0xe25a86 (/usr/local/bin/php+0xe25a86)
        #1 0xe953c8 (/usr/local/bin/php+0xe953c8)
        #2 0xe4935b (/usr/local/bin/php+0xe4935b)
        #3 0xcbede3 (/usr/local/bin/php+0xcbede3)
        #4 0x98cfa9 (/usr/local/bin/php+0x98cfa9)
        #5 0x99dc3c (/usr/local/bin/php+0x99dc3c)
        #6 0x7f7b57e74a97 (/usr/lib64/libasan.so.0.0.0+0x19a97)
        #7 0x7f7b5694edc4 (/usr/lib64/libpthread-2.17.so+0x7dc4)
        #8 0x7f7b5667d73c (/usr/lib64/libc-2.17.so+0xf773c)
    0x600800001c15 is located 5 bytes inside of 48-byte region [0x600800001c10,0x600800001c40)
    freed by thread T1623 here:
        #0 0x7f7b57e71009 (/usr/lib64/libasan.so.0.0.0+0x16009)
        #1 0xe259fb (/usr/local/bin/php+0xe259fb)
        #2 0xe953c8 (/usr/local/bin/php+0xe953c8)
        #3 0xe4935b (/usr/local/bin/php+0xe4935b)
        #4 0xcbede3 (/usr/local/bin/php+0xcbede3)
        #5 0x98cfa9 (/usr/local/bin/php+0x98cfa9)
        #6 0x99dc3c (/usr/local/bin/php+0x99dc3c)
        #7 0x7f7b57e74a97 (/usr/lib64/libasan.so.0.0.0+0x19a97)
    previously allocated by thread T0 here:
        #0 0x7f7b57e71129 (/usr/lib64/libasan.so.0.0.0+0x16129)
        #1 0xdb0118 (/usr/local/bin/php+0xdb0118)
        #2 0xe64f6c (/usr/local/bin/php+0xe64f6c)
        #3 0xe68c5d (/usr/local/bin/php+0xe68c5d)
        #4 0xcc4dcc (/usr/local/bin/php+0xcc4dcc)
        #5 0xcc5972 (/usr/local/bin/php+0xcc5972)
        #6 0x111f82c (/usr/local/bin/php+0x111f82c)
        #7 0x44c781 (/usr/local/bin/php+0x44c781)
        #8 0x7f7b565a7b34 (/usr/lib64/libc-2.17.so+0x21b34)
    Thread T1632 created by T0 here:
        #0 0x7f7b57e65c3a (/usr/lib64/libasan.so.0.0.0+0xac3a)
        #1 0x99e4cb (/usr/local/bin/php+0x99e4cb)
        #2 0x975d2a (/usr/local/bin/php+0x975d2a)
        #3 0x11105c1 (/usr/local/bin/php+0x11105c1)
        #4 0xf71de2 (/usr/local/bin/php+0xf71de2)
        #5 0x97b385 (/usr/local/bin/php+0x97b385)
        #6 0x11126bc (/usr/local/bin/php+0x11126bc)
        #7 0xf71de2 (/usr/local/bin/php+0xf71de2)
        #8 0x97b385 (/usr/local/bin/php+0x97b385)
        #9 0x11126bc (/usr/local/bin/php+0x11126bc)
        #10 0xf71de2 (/usr/local/bin/php+0xf71de2)
        #11 0x97b385 (/usr/local/bin/php+0x97b385)
        #12 0x11126bc (/usr/local/bin/php+0x11126bc)
        #13 0xf71de2 (/usr/local/bin/php+0xf71de2)
        #14 0x97b385 (/usr/local/bin/php+0x97b385)
        #15 0x11126bc (/usr/local/bin/php+0x11126bc)
        #16 0xf71de2 (/usr/local/bin/php+0xf71de2)
        #17 0x97b385 (/usr/local/bin/php+0x97b385)
        #18 0x1110b28 (/usr/local/bin/php+0x1110b28)
        #19 0xf71de2 (/usr/local/bin/php+0xf71de2)
        #20 0x97b385 (/usr/local/bin/php+0x97b385)
        #21 0x11126bc (/usr/local/bin/php+0x11126bc)
        #22 0xf71de2 (/usr/local/bin/php+0xf71de2)
        #23 0x97b385 (/usr/local/bin/php+0x97b385)
        #24 0x111c28a (/usr/local/bin/php+0x111c28a)
        #25 0xe52281 (/usr/local/bin/php+0xe52281)
        #26 0xcc766f (/usr/local/bin/php+0xcc766f)
        #27 0x1121d36 (/usr/local/bin/php+0x1121d36)
        #28 0x44d202 (/usr/local/bin/php+0x44d202)
        #29 0x7f7b565a7b34 (/usr/lib64/libc-2.17.so+0x21b34)
    Thread T1623 created by T0 here:
        #0 0x7f7b57e65c3a (/usr/lib64/libasan.so.0.0.0+0xac3a)
        #1 0x99e4cb (/usr/local/bin/php+0x99e4cb)
        #2 0x975d2a (/usr/local/bin/php+0x975d2a)
        #3 0x11105c1 (/usr/local/bin/php+0x11105c1)
        #4 0xf71de2 (/usr/local/bin/php+0xf71de2)
        #5 0x97b385 (/usr/local/bin/php+0x97b385)
        #6 0x11126bc (/usr/local/bin/php+0x11126bc)
        #7 0xf71de2 (/usr/local/bin/php+0xf71de2)
        #8 0x97b385 (/usr/local/bin/php+0x97b385)
        #9 0x11126bc (/usr/local/bin/php+0x11126bc)
        #10 0xf71de2 (/usr/local/bin/php+0xf71de2)
        #11 0x97b385 (/usr/local/bin/php+0x97b385)
        #12 0x11126bc (/usr/local/bin/php+0x11126bc)
        #13 0xf71de2 (/usr/local/bin/php+0xf71de2)
        #14 0x97b385 (/usr/local/bin/php+0x97b385)
        #15 0x11126bc (/usr/local/bin/php+0x11126bc)
        #16 0xf71de2 (/usr/local/bin/php+0xf71de2)
        #17 0x97b385 (/usr/local/bin/php+0x97b385)
        #18 0x1110b28 (/usr/local/bin/php+0x1110b28)
        #19 0xf71de2 (/usr/local/bin/php+0xf71de2)
        #20 0x97b385 (/usr/local/bin/php+0x97b385)
        #21 0x11126bc (/usr/local/bin/php+0x11126bc)
        #22 0xf71de2 (/usr/local/bin/php+0xf71de2)
        #23 0x97b385 (/usr/local/bin/php+0x97b385)
        #24 0x111c28a (/usr/local/bin/php+0x111c28a)
        #25 0xe52281 (/usr/local/bin/php+0xe52281)
        #26 0xcc766f (/usr/local/bin/php+0xcc766f)
        #27 0x1121d36 (/usr/local/bin/php+0x1121d36)
        #28 0x44d202 (/usr/local/bin/php+0x44d202)
        #29 0x7f7b565a7b34 (/usr/lib64/libc-2.17.so+0x21b34)
    Shadow bytes around the buggy address:
      0x0c017fff8330: fa fa 00 00 00 00 00 00 fa fa 00 00 00 00 00 fa
      0x0c017fff8340: fa fa 00 00 00 00 00 fa fa fa 00 00 00 00 00 fa
      0x0c017fff8350: fa fa 00 00 00 00 00 fa fa fa 00 00 00 00 00 00
      0x0c017fff8360: fa fa 00 00 00 00 00 00 fa fa 00 00 00 00 00 fa
      0x0c017fff8370: fa fa 00 00 00 00 00 fa fa fa 00 00 00 00 00 00
    =>0x0c017fff8380: fa fa[fd]fd fd fd fd fd fa fa 00 00 00 00 00 00
      0x0c017fff8390: fa fa 00 00 00 00 00 00 fa fa 00 00 00 00 00 00
      0x0c017fff83a0: fa fa 00 00 00 00 00 00 fa fa 00 00 00 00 00 00
      0x0c017fff83b0: fa fa 00 00 00 00 00 00 fa fa 00 00 00 00 00 00
      0x0c017fff83c0: fa fa 00 00 00 00 00 00 fa fa 00 00 00 00 00 fa
      0x0c017fff83d0: fa fa 00 00 00 00 00 fa fa fa 00 00 00 00 00 fa
    Shadow byte legend (one shadow byte represents 8 application bytes):
      Addressable:           00
      Partially addressable: 01 02 03 04 05 06 07
      Heap left redzone:     fa
      Heap righ redzone:     fb
      Freed Heap region:     fd
      Stack left redzone:    f1
      Stack mid redzone:     f2
      Stack right redzone:   f3
      Stack partial redzone: f4
      Stack after return:    f5
      Stack use after scope: f8
      Global redzone:        f9
      Global init order:     f6
      Poisoned by user:      f7
      ASan internal:         fe
    ==2716== ABORTING
    

1 个答案:

答案 0 :(得分:0)

我发现内存损坏的来源,以及"修复"非常简单。我发现唯一的"非致命的"在pthreads中使用phpiredis $ client的方法是在当前本地范围内显式连接/执行命令/ disconnect。

总结一下,以下结果会导致高负载下的内存损坏:

  1. 尝试在线程上下文中通过引用将phpiredis $客户端传递给另一个对象
  2. 尝试按照线程上下文中的值将phpiredis $客户端传递给另一个对象
  3. 尝试在线程上下文中除了本地范围之外的任何地方实例化/使用phpiredis $客户端(例如:在另一个类中,_Redis,当我们当前在Investor :: save时)
  4. 尝试在线程上下文中静态重用phpiredis连接
  5. 但是,当以相同的方式/上下文将MySQLi $客户端传递给另一个对象时,不会发生内存损坏。

    我希望这可以拯救别人在我见证的绝望的一周。这就是必须在pthreads中使用phpiredis的方法:

    // Instantiate a local raw phpiredis $client and execute get/set commands using $client directly
    // @Result: This WORKS :D
    $client = phpiredis_connect('127.0.0.1', 6379);
    $key = "table_name-_-$this->id";
    foreach($args as $prop=>$val)
        if(isset($prop) && isset($val))
            phpiredis_command($client, "hset $key $prop $val");
    $obj = phpiredis_command($client, "hgetall $key");
    echo json_encode($obj);
    phpiredis_disconnect($client);