Question

我有一个非常奇怪的情况，我设置了一个在我的Lumen数据库队列中运行的作业，除了处理第一个作业之外的所有作业。我确实不断收到这个错误：

[2017-12-12 22:07:10] lumen.ERROR: Symfony\Component\Debug\Exception\FatalErrorException: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 702558208 bytes) in /var/www/vhosts/XXXXXXXXX$
Stack trace:
#0 /var/www/vhosts/XXXXXXXX/vendor/laravel/lumen-framework/src/Concerns/RegistersExceptionHandlers.php(54): Laravel\Lumen\Application->handleShutdown()
#1 [internal function]: Laravel\Lumen\Application->Laravel\Lumen\Concerns\{closure}()
#2 {main}

我已经尝试过允许内存限制上升，但是我在使用不同的内存值时仍然会遇到相同的错误。

我觉得很奇怪，它始终是第一份工作，所有其他工作都完美无缺。我应该在第一份工作中寻找不良数据吗？

我的代码基本上是这样的：

这是我的命令文件

namespace App\Console\Commands;

use App\Jobs\UpdateNNNAppListJob;
use Illuminate\Console\Command;
use App\Services\MiddlewareApi;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Log;
use Mockery\Exception;
use Illuminate\Support\Facades\Queue;

class AddEmailsToAppList extends Command
{
    /**
     * The name and signature of the console command.
     *
     * @var string
     */
    protected $signature = 'addemails:nnnmobileapp';

    /**
     * The console command description.
     *
     * @var string
     */
    protected $description = 'This will add all mobile app users in the database to the nnn mobile app list.';

    /**
     * Create a new command instance.
     *
     * @return void
     */
    public function __construct()
    {
        parent::__construct();
    }

    public function handle()
    {
        $chunkSize = 500; //this is the most middleware can handle with its bulk signup call
        $emailChunks = $this->getEmailsToAdd($chunkSize);
        $jobDelay = 120; //time between queued jobs
        $jobDelayTimeKeeper = 60; //This will be the actual time delay that will be put into the later method

        foreach ($emailChunks as $emailChunk) {
            Queue::later($jobDelayTimeKeeper, new UpdateMmpAppListJob($emailChunk));
            $jobDelayTimeKeeper = $jobDelayTimeKeeper + $jobDelay;
        }
    }

    public function getEmailsToAdd($chunkSize)
    {
        $emails = DB::table('app_users')
            ->join('app_datas', 'app_datas.customer_number', '=', 'app_users.customer_number')
            ->select('app_users.email')
            ->get()
            ->chunk($chunkSize);

        return $emails;
    }
}

这是我的工作档案

<?php
namespace App\Jobs;

use App\Services\MiddlewareApi;
use Illuminate\Support\Facades\Log;
use Mockery\Exception;

class UpdateMmpAppListJob extends Job
{

    /**
     * Array of emails to update list with
     * @var array
     */
    protected $emailArray;

    /**
     * The number of times the job may be attempted.
     *
     * @var int
     */
    public $tries = 2;

    public function __construct($emailArray)
    {
        $this->emailArray = $emailArray;
    }

    public function handle()
    {
        $listCodeToAddTo = 'NNNAPP';
        $sourceId = 'NNNNNNN';

        $middlewareApi = new MiddlewareApi();

        try {
            $middlewareApi->post_add_customer_signup_bulk($listCodeToAddTo, $this->emailArray, $sourceId);
        } catch (\Exception $e) {
            Log::error('An error occurred with theUpdateMmpAppListJob: ' . $e);
            mail('djarrin@NNN.com', 'UpdateNnnAppListJob Failure', 'A failure in the UpdateNnnAppListJob, here is the exception: ' . $e);
        }

    }

    public function failed(\Exception $exception)
    {
        mail('djarrin@moneymappress.com', 'Push Processor Que Failure', 'A failure in the UpdateMmpAppListJob, here is the exception: ' . $exception);
    }
}

对此问题的任何帮助/建议都将不胜感激。

Answer 1

您的代码调用->get()，它会将整个结果加载到内存中。这会导致您看到的巨大内存分配。删除它，让->chunk(...)使用数据库构建器，而不是get（）返回的内存中的Collection。您还必须提供对处理每个块的块的回调。

public function handle() {
    $chunkSize = 500; //this is the most middleware can handle with its bulk signup call
    $jobDelay = 120; //time between queued jobs
    $jobDelayTimeKeeper = 60; //This will be the actual time delay that will be put into the later method

    DB::table('app_users')
        ->join('app_datas', 'app_datas.customer_number', '=', 'app_users.customer_number')
        ->select('app_users.email')
        ->chunk($chunkSize, function($emailChunk) use (&$jobDelayTimeKeeper, $jobDelay) {
            Queue::later($jobDelayTimeKeeper, new UpdateMmpAppListJob($emailChunk));
            $jobDelayTimeKeeper = $jobDelayTimeKeeper + $jobDelay;
        });
}

Answer 2

上述概念是正确的，但需要这种语法才能超越

[2017-12-14 22:08:26] lumen.ERROR: RuntimeException: You must specify an orderBy clause when using this function. in /home/vagrant/sites/nnn/vendor/illuminate/database/Query/Builder.php:1877

这是针对Lumen 5.5：

public function handle()
    {
        $chunkSize = 500; //this is the most middleware can handle with its bulk signup call
        $jobDelay = 120; //time between queued jobs
        $jobDelayTimeKeeper = 60; //This will be the actual time delay that will be put into the later method

        $emails = DB::table('app_users')
            ->join('app_datas', 'app_datas.customer_number', '=', 'app_users.customer_number')
            ->select('app_users.email')->orderBy('app_users.id', 'desc');

        $emails->chunk($chunkSize, function($emailChunk) use (&$jobDelayTimeKeeper, $jobDelay) {
            Queue::later($jobDelayTimeKeeper, new UpdateMmpAppListJob($emailChunk));
            $jobDelayTimeKeeper = $jobDelayTimeKeeper + $jobDelay;
        });
    }

流明数据库队列第一个作业总是失败允许的内存耗尽

2 个答案: