Firestore批量插入速度太慢

时间:2020-06-21 21:43:36

标签: php mysql multithreading asynchronous google-cloud-firestore

我有一个用PHP开发的Web API,可与MySQL通信。我希望将数据库更改为Firestore,但是在MySQL中,当我批量插入4500条记录时,它几乎是即时的,但是在使用Firestore时,大约需要15分钟。

关于这是为什么的任何想法?

我已经尝试了这两个代码:

use Google\Cloud\Firestore\DocumentReference;
use Google\Cloud\Firestore\FirestoreClient;
use Spatie\Async\Pool;

/* @var FirestoreClient */
private $database;
/* @var DocumentReference */
private $collectionReference;

代码1:

$pool = Pool::create();
$graduatesFailed = [];
/** @var Graduate $graduate */
foreach ($graduates as $graduate) {
    $pool[] = async(function () use ($graduate) {
        return $this->collectionReference->add($graduate->jsonSerialize());
    })->catch(function ($e) use ($graduatesFailed, $graduate) {
        array_push($graduatesFailed, $graduate->getUser());
    });
}

await($pool);

代码2:

$writeBatch = $this->database->batch();
$count = 0;
$graduatesFraction = array_slice($graduates, 0, 500);
$graduatesFailed = [];
while ($count < count($graduates)) {
    /** @var Graduate $graduate */
    foreach ($graduatesFraction as $graduate) {
        try {
            $count++;
            $writeBatch->create(
                $this->collectionReference->document($graduate->getUser()),
                $graduate->jsonSerialize()
            );
        } catch (Exception $e) {
            array_push($graduatesFailed, $graduate->getUser());
        }
    }
    $writeBatch->commit();
}

1 个答案:

答案 0 :(得分:0)

Firestore并未针对繁重的写入负载进行优化。实际上,它具有写入速率限制,建议您在documentation中进行检查。用这种方式将Firestore与MySQL进行比较是不公平的。 Firestore针对大规模读取负载进行了优化(每个集合考虑数十亿个文档),而无需扩大工作量,并且当集合变得庞大时也不会降低性能。