Laravel一些作业超出了最大运行尝试次数/超时

时间:2019-05-12 14:10:13

标签: laravel queue jobs

我正在使用PubMed中的Eutilz为我的输出模型(https://www.ncbi.nlm.nih.gov/books/NBK25500/)保存摘要。输出模型包含有关单个文章/期刊的信息。

我已根据Eutilz准则使用https://github.com/hamburgscleanest/guzzle-advanced-throttle

对请求的速率进行了限制。

我已经与模型观察员建立了一个在创建Output模型时将触发的工作。它从Eutilz中提取抽象文本。

将此任务投入工作时,某些超出了最大尝试次数/运行时间过长。我没有任何错误消息可以确定出什么问题了。在失败的情况下,所有分派到ProcessAbstract方法的个人都将正确提取Abstract。

如果我用空的摘要提取所有输出模型并解雇了一项工作,则它运行良好。它不适用于模型观察者。

  **class OutputObserver**
    {
    /**
     * Handle the output "created" event.
     *
     * @param  \App\Output  $output
     * @return void
     */
    public function created(Output $output)
    {
      ProcessAbstract::dispatch($output);
    }



    **class ProcessAbstract implements ShouldQueue**
    {
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $tries = 1;
    public $timeout = 120;
    protected $output;


    /**
     * Create a new job instance.
     *
     * @return void
     */
    public function __construct(Output $output)
    {
        $this->onQueue('abstracts');
        $this->onConnection('redis');
        $this->output= $output;
    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {

        Redis::funnel('key')->limit(1)->then(function () {
            // Job logic...
            $abstract = new AbstractFetcher($this->output);
            $abstract->fetch();
        }, function () {
            // Could not obtain lock...
            return $this->release(10);
        });
    }


    **class AbstractFetcher**
    {
    protected $output;
    public $apikey ='*******************';


    public function __construct(Output $output)
    {
        $this->doi = $output->doi;
        $this->output = $output;
    }

    public function fetch(){

        //TODO check the response codes
    try{
        $client = LaravelGuzzleThrottle::client(['base_uri' => 
    'https://eutils.ncbi.nlm.nih.gov/']);
        $res = $client->get($this->getAPIUrl());
        $xmlid = simplexml_load_string($res->getBody());
        $xmlid = $xmlid->IdList->Id->__toString();
        $client = LaravelGuzzleThrottle::client(['base_uri' => 
    'https://eutils.ncbi.nlm.nih.gov/']);
        $res = $client->get($this->getOutputUrl($xmlid));
        $xmlid = simplexml_load_string($res->getBody());

        if(isset($xmlid->PubmedArticle->MedlineCitation->Article->Abstract-  >AbstractText)){
            $stringpieces=[];
            foreach($xmlid->PubmedArticle->MedlineCitation->Article- 
   >Abstract->AbstractText as $text) {
                $stringpieces[]= (string) $text;
            }
            $abstract= implode(' ', $stringpieces);

            Output::where('doi', $this->doi)->update(['abstract' => 
    $abstract]);
        } else {
            die();
        }

    } catch(Exception $e){
        Log::alert('Error with processing abstract');
        Log::alert('Output ID ' . $this->output->id);
        Log::alert('Error Message' . $e->getMessage());
        Log::alert('API Url ' . $this->getAPIUrl());
        Log::alert('Output Url ' . $this->getOutputUrl('test'));
    };


    }

    public function getAPIUrl (){
        return  'https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&WebEnv=1&usehistory=y&term='.$this->doi . '$api_key=' . $this->apikey;
    }

    public function getOutputUrl($xmid){
        $field = array('db' => 'pubmed', 'retmode' => 'xml', 'rettype' => 'abstract', 'id' => $xmid);
        return "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?".http_build_query($field);
    }

}

求职成功完成

0 个答案:

没有答案