Question

我发现了很多信息和脚本示例，展示了如何对API的用户进行速率限制，但是我无法找到任何关于在施加这些限制时如何限制自己的API请求的示例

我总是使用sleep或usleep命令等代码限制我的脚本，但感觉这是一种低效的处理方式，尤其是当API端点具有相当高的速率限制和锤击时API达到极限之前也是低效的。

例如，Google的API限制因您使用的API而异，并且可以增加/减少，在这种情况下，硬编码到代码中的固定速率限制似乎只是原始猜测工作！

我错过了一些非常明显的东西吗？或者这不像我预期的那样普遍吗？

Answer 1

嗯，首先要做的事情 - 你应该只在你真正需要的时候调用任何外部API - 提供商会非常感谢你。

我通常会通过两种方式“限制”我自己的API使用限制 - 如果可能的话，将结果缓存N个时间，通常比API本身的硬限制少很多。但是，这仅适用于非常具体的情况。

第二个是持久性/半持久性计数器，您可以将计数器存储在某种内存后端以及限制周期开始的时间。每次调用API之前，请检查存储并查看当前时间减去间隔是否开始以及您已经发出的请求数量是否小于API。如果是，您可以发出请求 - 如果间隔较大，您可以重置限制，如果您的下一个请求超出限制，并且您仍处于上一个间隔，则可以显示相当错误。在每个外部请求上，如果超过间隔时间，则更新间隔时间并增加计数器。

Answer 2

好吧，对于咯咯笑声，我把一个限制器类放在一起，它允许你指定每秒，每分钟和每小时的限制。我无法抗拒使用循环队列的充分理由！

如果您有多个进程正在进行消费，无论是否同时进行，您都必须设计一种方法来自行存储和/或共享使用历史记录。

// LIMITER.PHP
class Limiter
{
  private $queue = array();
  private $size;
  private $next;

  private $perSecond;
  private $perMinute;
  private $perHour;

  // Set any constructor parameter to non-zero to allow adherence to the
  // limit represented. The largest value present will be the size of a
  // circular queue used to track usage.
  // -------------------------------------------------------------------
  function __construct($perSecond=0,$perMinute=0,$perHour=0)
  {
    $this->size = max($perSecond,$perMinute,$perHour);
    $this->next = 0;

    $this->perSecond = $perSecond;
    $this->perMinute = $perMinute;
    $this->perHour   = $perHour;

    for($i=0; $i < $this->size; $i++)
      $this->queue[$i] = 0;
  }

  // See if a use would violate any of the limits specified. We return true
  // if a limit has been hit.
  // ----------------------------------------------------------------------
  public function limitHit($verbose=0)
  {    
    $inSecond = 0;
    $inMinute = 0;
    $inHour   = 0;

    $doneSecond = 0;
    $doneMinute = 0;
    $doneHour   = 0;

    $now = microtime(true);

    if ( $verbose )
      echo "Checking if limitHit at $now<br>\n";

    for ($offset=1; $offset <= $this->size; $offset++)
    {
      $spot = $this->next - $offset;
      if ( $spot < 0 )
        $spot = $this->size - $offset + $this->next;

      if ( $verbose )
        echo "... next $this->next size $this->size offset $offset spot $spot utime " . $this->queue[$spot] . "<br>\n";

      // Count and track within second
      // -----------------------------
      if ( $this->perSecond && !$doneSecond && $this->queue[$spot] >= microtime(true) - 1.0 )
        $inSecond++;
      else
        $doneSecond = 1;

      // Count and track within minute
      // -----------------------------
      if ( $this->perMinute && !$doneMinute && $this->queue[$spot] >= microtime(true) - 60.0 )
        $inMinute++;
      else
        $doneMinute = 1;

      // Count and track within hour
      // ---------------------------
      if ( $this->perHour && !$doneHour && $this->queue[$spot] >= microtime(true) - 3600.0 )
        $inHour++;
      else
        $doneHour = 1;

      if ( $doneSecond && $doneMinute && $doneHour )
        break;
    }

    if ( $verbose )
      echo "... inSecond $inSecond inMinute $inMinute inHour $inHour<br>\n";

    if ( $inSecond && $inSecond >= $this->perSecond )
    {
      if ( $verbose )
        echo "... limit perSecond hit<br>\n";
      return TRUE;
    }
    if ( $inMinute && $inMinute >= $this->perMinute )
    {
      if ( $verbose )
        echo "... limit perMinute hit<br>\n";
      return TRUE;
    }
    if ( $inHour   && $inHour   >= $this->perHour   )
    {
      if ( $verbose )
        echo "... limit perHour hit<br>\n";
      return TRUE;
    }

    return FALSE;
  }

  // When an API is called the using program should voluntarily track usage
  // via the use function.
  // ----------------------------------------------------------------------
  public function usage()
  {
    $this->queue[$this->next++] = microtime(true);
    if ( $this->next >= $this->size )
      $this->next = 0;
  }
}

// ##############################
// ### Test the limiter class ###
// ##############################

$psec = 2;
$pmin = 4;
$phr  = 0;

echo "Creating limiter with limits of $psec/sec and $pmin/min and $phr/hr<br><br>\n";
$monitorA = new Limiter($psec,$pmin,$phr);

for ($i=0; $i<15; $i++)
{
  if ( !$monitorA->limitHit(1) )
  {
    echo "<br>\n";
    echo "API call A here (utime " . microtime(true) . ")<br>\n";
    echo "Voluntarily registering usage<br>\n";
    $monitorA->usage();
    usleep(250000);
  }
  else
  {
    echo "<br>\n";
    usleep(500000);
  }
}

为了演示它的实际应用，我在限制检查功能中加入了一些“详细模式”语句。这是一些示例输出。

Creating limiter with limits of 2/sec and 4/min and 0/hr

Checking if limitHit at 1436267440.9957
... next 0 size 4 offset 1 spot 3 utime 0
... inSecond 0 inMinute 0 inHour 0

API call A here (utime 1436267440.9957)
Voluntarily registering usage
Checking if limitHit at 1436267441.2497
... next 1 size 4 offset 1 spot 0 utime 1436267440.9957
... next 1 size 4 offset 2 spot 3 utime 0
... inSecond 1 inMinute 1 inHour 0

API call A here (utime 1436267441.2497)
Voluntarily registering usage
Checking if limitHit at 1436267441.5007
... next 2 size 4 offset 1 spot 1 utime 1436267441.2497
... next 2 size 4 offset 2 spot 0 utime 1436267440.9957
... next 2 size 4 offset 3 spot 3 utime 0
... inSecond 2 inMinute 2 inHour 0
... limit perSecond hit

Checking if limitHit at 1436267442.0007
... next 2 size 4 offset 1 spot 1 utime 1436267441.2497
... next 2 size 4 offset 2 spot 0 utime 1436267440.9957
... next 2 size 4 offset 3 spot 3 utime 0
... inSecond 1 inMinute 2 inHour 0

API call A here (utime 1436267442.0007)
Voluntarily registering usage
Checking if limitHit at 1436267442.2507
... next 3 size 4 offset 1 spot 2 utime 1436267442.0007
... next 3 size 4 offset 2 spot 1 utime 1436267441.2497
... next 3 size 4 offset 3 spot 0 utime 1436267440.9957
... next 3 size 4 offset 4 spot 3 utime 0
... inSecond 1 inMinute 3 inHour 0

API call A here (utime 1436267442.2507)
Voluntarily registering usage
Checking if limitHit at 1436267442.5007
... next 0 size 4 offset 1 spot 3 utime 1436267442.2507
... next 0 size 4 offset 2 spot 2 utime 1436267442.0007
... next 0 size 4 offset 3 spot 1 utime 1436267441.2497
... next 0 size 4 offset 4 spot 0 utime 1436267440.9957
... inSecond 2 inMinute 4 inHour 0
... limit perSecond hit

Checking if limitHit at 1436267443.0007
... next 0 size 4 offset 1 spot 3 utime 1436267442.2507
... next 0 size 4 offset 2 spot 2 utime 1436267442.0007
... next 0 size 4 offset 3 spot 1 utime 1436267441.2497
... next 0 size 4 offset 4 spot 0 utime 1436267440.9957
... inSecond 2 inMinute 4 inHour 0
... limit perSecond hit

Checking if limitHit at 1436267443.5027
... next 0 size 4 offset 1 spot 3 utime 1436267442.2507
... next 0 size 4 offset 2 spot 2 utime 1436267442.0007
... next 0 size 4 offset 3 spot 1 utime 1436267441.2497
... next 0 size 4 offset 4 spot 0 utime 1436267440.9957
... inSecond 0 inMinute 4 inHour 0
... limit perMinute hit

Checking if limitHit at 1436267444.0027
... next 0 size 4 offset 1 spot 3 utime 1436267442.2507
... next 0 size 4 offset 2 spot 2 utime 1436267442.0007
... next 0 size 4 offset 3 spot 1 utime 1436267441.2497
... next 0 size 4 offset 4 spot 0 utime 1436267440.9957
... inSecond 0 inMinute 4 inHour 0
... limit perMinute hit

Answer 3

使用Jobs来包装API调用，并将其推送到单独的队列：
```
ApiJob::dispatch()->onQueue('api');
```
使用Redis或mxl/laravel-queue-rate-limit软件包的队列速率限制（我是作者）。另请参见SO answer about its usage。
如果使用mxl/laravel-queue-rate-limit，则在设置之后运行队列工作器：
```
$ php artisan queue:work --queue api
```

Answer 4

我想我们不能用几句话回答你的问题。它真实地反映了与您的应用程序相关的架构。为了让我对重复进行API速率限制，我使用缓存来存储我的API的值和利用率。我必须在约会时找不到准备好的代码。

限制自己超载外部API的速度

4 个答案: