我们有多个任务要在同一个api上运行，因为api的速率受到限制，因此我们需要限制进行多少次调用（100 x 120秒），以免被锁定。每次锁定持续30分钟，将冻结整个团队，并可能造成可怕的问题。
系统基于Laravel构建，我们正在使用队列进行此api调用。我们需要做的第一件事是，每晚系统都会对api进行大约950次调用，以将api上的内容与我们的数据库同步。第二和第三项工作将按需提供，每当用户需要时，每个工作可能需要运行约100至200个呼叫。 （一天中可能同时有多个用户需要此服务，因此排队队列应防止我们超过速率限制）。
我们只用一名工人就可以在主管中进行所有设置，而且看起来工作正常。但后来我们意识到，并不是按需调用所有作业（第二和第三种），如果用户需要对8件事采取行动，它们将出现在队列行上，但只运行2或3。其余的将得到永远不要捡起，不跑，甚至没有失败。通过将工人数量增加到8，可以解决此问题。完成此操作后，如果用户需要对8项可行的事情采取行动，但我们的夜间900项任务开始失败。
我们还注意到它可以处理8个工作，但是如果我们让用户最多执行50个任务，它也会失败。
我们在/etc/supervisor/conf.d上的主管配置：
[unix_http_server]
file = /tmp/supervisor.sock
chmod = 0777
chown= ubuntu:ubuntu
[supervisord]
logfile = /home/ubuntu/otmas/supervisord.log
logfile_maxbytes = 50MB
logfile_backups=10
loglevel = info
pidfile = /tmp/supervisord.pid
nodaemon = false
minfds = 1024
minprocs = 200
umask = 022
user = ubuntu
identifier = supervisor
directory = /tmp
nocleanup = true
childlogdir = /tmp
strip_ansi = false
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl = unix:///tmp/supervisor.sock
[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /home/ubuntu/www/artisan horizon
autostart=true
autorestart=true
user=ubuntu
numprocs=8
redirect_stderr=true
stdout_logfile=/home/ubuntu/otmas/worker.log
我们的队列配置：
'connections' => [
'redis' => [
'driver' => 'redis',
'connection' => 'default',
'queue' => 'default',
'retry_after' => 460,
'block_for' => 140,
],
],
我们的视野配置：
'environments' => [
'production' => [
'supervisor-1' => [
'connection' => 'redis',
'queue' => ['default'],
'balance' => 'simple',
'processes' => 10,
'tries' => 3,
],
],
'local' => [
'supervisor-1' => [
'connection' => 'redis',
'queue' => ['default'],
'balance' => 'simple',
'processes' => 3,
'tries' => 3,
],
],
'develop' => [
'supervisor-1' => [
'connection' => 'redis',
'queue' => ['default','email'],
'balance' => 'auto',
'processes' => 3,
'tries' => 3,
'timeout' => 3,
'delay' => 3,
],
],
],
我们在database.php上的redis配置：
'redis' => [
'client' => 'predis',
'default' => [
'host' => env('REDIS_HOST', '127.0.0.1'),
'password' => env('REDIS_PASSWORD', null),
'port' => env('REDIS_PORT', 6379),
'database' => env('REDIS_DB', 0),
'read_write_timeout' => -1,
],
'cache' => [
'host' => env('REDIS_HOST', '127.0.0.1'),
'password' => env('REDIS_PASSWORD', null),
'port' => env('REDIS_PORT', 6379),
'database' => env('REDIS_CACHE_DB', 1),
'read_write_timeout' => -1,
],
],
每晚的工作：
class GetProjectTasks implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
protected $project;
public $tries = 540;
public $timeout = 15;
public function __construct(ZohoProject $project)
{
$this->project = $project;
}
public function handle()
{
\Log::info("before redis get tasks for project: ".$this->project->id);
try {
$client = new GuzzleClient();
$accessToken = Setting::where('name','zoho_access_token')->get()->first();
$requestHeaders = [
'cache-control' => 'no-cache',
'Authorization' => 'Bearer '.$accessToken->value
];
$requestOptions = [
'headers' => $requestHeaders
];
$zohoTasksRestApi = Setting::where('name','zoho_projectapi_restapi')->get()->first()->value;
$project = $this->project->id;
} catch(Exception $e) {
Log::notice('Exception caught');
throw $e;
}
Redis::throttle('zohoprojecttasksget')->allow(85)->every(120)->then(function () use ($client, $zohoTasksRestApi, $portalId, $project, $requestOptions) {
try {
$zohoTasksResponse = $client->request('GET', $zohoTasksRestApi.'portal/'.$portalId.'/projects/'.$project.'/tasks/', $requestOptions);
$zohoTasksResult = json_decode((string) $zohoTasksResponse->getBody(), true);
if($zohoTasksResult) {
foreach ($zohoTasksResult['tasks'] as $key => $task) {
if (array_has($task, 'start_date') && array_has($task, 'end_date')) {
$taskStartDate = \Carbon\Carbon::createFromTimestamp((int)$task['start_date_long']/1000, 'America/Santiago');
$taskEndDate = \Carbon\Carbon::createFromTimestamp((int)$task['end_date_long']/1000, 'America/Santiago');
$zohoTask = ZohoTask::updateOrCreate(['id' => $task['id']], [
'name' => $task['name'],
'zoho_project_id' => $project,
'registry' => json_encode($task),
'start_date' => $taskStartDate,
'end_date' => $taskEndDate
]);
if(array_has($task, ['details.owners'])) {
if($task['details']['owners'][0]['name'] != 'Unassigned') {
foreach ($task['details']['owners'] as $key => $owner) {
$user = ZohoUser::find($owner['id']);
$zohoTask->assignees()->save($user);
}
}
}
} else {
$zohoTask = ZohoTask::updateOrCreate(['id' => $task['id']], [
'name' => $task['name'],
'zoho_project_id' => $project,
'registry' => json_encode($task),
]);
}
}
}
} catch(Exception $e) {
Log::notice('Exception caught');
throw $e;
}
}, function () use ($project) {
return $this->release(85);
});
}
}
按需工作示例：
class ReplaceTask implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
protected $task, $taskToReplace;
public $tries = 140;
public $timeout = 15;
public function __construct(ZohoTask $task, ZohoTask $taskToReplace)
{
$this->task = $task;
$this->taskToReplace = $taskToReplace;
}
public function handle()
{
\Log::info("before redis replace task: ".$this->taskToReplace.' with task:: '.$this->task);
try {
$client = new GuzzleClient();
$accessToken = Setting::where('name','zoho_access_token')->get()->first();
$requestHeaders = [
'cache-control' => 'no-cache',
'Authorization' => 'Bearer '.$accessToken->value
];
$ZohoProjectsRestApi = Setting::where('name','zoho_projectapi_restapi')->get()->first()->value;
$task = $this->task;
$taskToReplace = $this->taskToReplace;
$project = $task->project;
} catch(Exception $e) {
Log::notice('Exception caught');
throw $e;
}
\Log::info("before redis task: ".$task->id.' to replace '.$taskToReplace->id.' Project: '.$project->id);
Redis::throttle('zohoupdatetask')->allow(50)->every(120)->then(function () use ($client, $ZohoProjectsRestApi, $portalId, $project, $requestHeaders, $task, $taskToReplace) {
try {
$assignee = $taskToReplace->assignees->last();
$assignResponse = $client->request('POST', $ZohoProjectsRestApi.'portal/'.$portalId.'/projects/'.$project->id.'/tasks/'.$task->id.'/', [
'headers' => $requestHeaders,
'form_params' => [
'start_date' => $taskToReplace->start_date->format('m-d-Y'),
'start_time' => $taskToReplace->start_date->format('h:i A'),
'person_responsible' => $assignee->id
]
]);
\Log::info($assignResponse->getBody());
if($assignResponse->getStatusCode() == 200) {
$task->assignees()->detach();
$assignResult = json_decode((string) $assignResponse->getBody(), true);
$taskRegistry = $assignResult['tasks'][0];
$taskStartDate = \Carbon\Carbon::createFromTimestamp((int)$taskRegistry['start_date_long']/1000, 'America/Santiago');
$taskEndDate = \Carbon\Carbon::createFromTimestamp((int)$taskRegistry['end_date_long']/1000, 'America/Santiago');
$task->start_date = $taskStartDate;
$task->end_date = $taskEndDate;
$task->registry = json_encode($taskRegistry);
$task->assignees()->save($assignee);
$task->save();
\Log::info("Task Ok move");
}
} catch(Exception $e) {
Log::notice('Exception caught');
throw $e;
}
}, function () use ($project, $task, $taskToReplace) {
\Log::info("Task hit throttle");
return $this->release(120);
});
\Log::info("after redis");
}
}
Horizon任务：（被搁置的任务永远不会被奔跑或拾取） Tasks appear on the queue, but never run or fail, it seems
