我有一个自定义查询,它从旧系统中获取数据并将其映射到新系统中的模型。查询如下所示:
$companies = DB::connection('legacy')->select("...");
由于这是很多数据,我想使用Eloquent的块功能(只是从他们的文档复制的示例代码):
User::chunk(200, function($users)
{
foreach ($users as $user)
{
//
}
});
我该如何实现?
编辑:我的代码现在看起来像这样,导致没有响应:
DB::connection('legacy')->select("SELECT * FROM companies")->chunk(200, function($companies) {
foreach ($companies as $company) {
// dd($company);
$entity = Entity::firstOrNew(['external_id' => $company->companyKey]);
$entity->name = $company->companyName;
$entity->save();
}
});
答案 0 :(得分:7)
The chunk
feature is only available for Eloquent models and QueryBuilder requests, e.g.
DB::table('tbl')->where('num', '>', 3)->chunk(500, function($rows) {
// process $rows
});
But it won't work for DB::select('...')
request.
You need to either use a QueryBuilder request, or use an underlying PDO object to query the database, e.g:
$pdo = DB::getPdo();
$sth = $pdo->prepare("SELECT ....");
$sth->execute();
while ($row = $sth->fetch(PDO::FETCH_ASSOC))
{
// ...
}
答案 1 :(得分:4)
AS-IS
如果'遗产'表格有' id'列和那里的1,2,3,4,5 ... 100编号数据。
<?php
$max = 10;
$total = DB::connection('legacy')->select("...")->count();
$pages = ceil($total / $max);
for ($i = 1; $i < ($pages + 1); $i++) {
$offset = (($i - 1) * $max);
$start = ($offset == 0 ? 0 : ($offset + 1));
$legacy = DB::connection('legacy')->select("...")->skip($start)->take($max)->get();
/* Do stuff. */
$legacyIds = $legacy->lists("id");
echo "i = " . $i . ": \n";
print_r($legacyIds);
}
//Result
i = 1:
Array
(
[0] => 1
[1] => 2
[2] => 3
[3] => 4
[4] => 5
[5] => 6
[6] => 7
[7] => 8
[8] => 9
[9] => 10
)
i = 2:
Array
(
[0] => 12
[1] => 13
[2] => 14
[3] => 15
[4] => 16
[5] => 17
[6] => 18
[7] => 19
[8] => 20
[9] => 21
) ...
TO-DO
$max = 10;
$total = DB::connection('legacy')->select("...")->count();
$pages = ceil($total / $max);
for ($i = 1; $i < ($pages + 1); $i++) {
$offset = (($i - 1) * $max);
$legacy = DB::connection('legacy')->select("...")->skip($offset)->take($max)->get();
/* Do stuff. */
}
答案 2 :(得分:3)
尝试这样的事情:
<?php
$max = 100;
$total = DB::connection('legacy')->select("...")->count();
$pages = ceil($total / $max);
for ($i = 1; $i < ($pages + 1); $i++) {
$offset = (($i - 1) * $max);
$start = ($offset == 0 ? 0 : ($offset + 1));
$legacy = DB::connection('legacy')->select("...")->skip($start)->take($max)->get();
/* Do stuff. */
}
基本上重复了Laravel的Paginator所做的事情而没有额外的开销。
答案 3 :(得分:2)
此页面上的很多答案都会查找所有记录,以便查看有多少“页面”。这可能很慢而且不需要,因为我们没有分页我们正在分块。我们只需知道分页以显示用户时的总页数。
因此,在开始时获取所有记录的计数的替代方法是执行以下操作:
$recordsRemaining = true;
$lookupsCompleted = 0;
$chunkSize = 200;
while($recordsRemaining){
$companies = DB::connection('legacy')->select("...")->skip($chunkSize*$lookupsCompleted)->take($chunkSize)->get();
if($legacy->count() < $chunkSize){
$recordsRemaining = false;
}
foreach($companies as $company){
//Do something
}
$lookupsCompleted++;
}
这与接受的答案完全相同,但效率更高。
答案 4 :(得分:1)
这些答案都不适合我。我根据@deyes的答案创建了自己的函数。
private static function chunk($query, $max, $function) {
$counter = preg_replace('/SELECT (.*?) FROM/', 'SELECT COUNT(*) FROM', $query);
$total = DB::connection('legacy')->select($counter)[0];
$total = (array)$total;
$total = $total['COUNT(*)'];
$pages = ceil($total / $max);
for ($i = 1; $i < ($pages + 1); $i++) {
$offset = (($i - 1) * $max);
$start = ($offset == 0 ? 0 : ($offset + 1));
$items = DB::connection('legacy')->select($query . ' LIMIT ' . $offset . ', ' . $max);
$function($items);
unset($items);
}
}
<强>用法强>
YourClass::chunk('SELECT * FROM tablename', 50, function($items) {
//Work with $items.
});
请注意,这是一个简单的快速修复,您的查询可能必须相当简单,因为我使用搜索替换来构建计数查询,而我只是在LIMIT X上添加,Y到查询结束但它适用于我。
答案 5 :(得分:1)
偶然发现了这个问题,但在某些情况下可能会派上用场的小技巧。
$query = 'SELECT * FROM ... JOIN ... UNION ... WHATEVER ... GROUP BY';
// This is the important part:
$query = '(' . $query . ') somealias';
\DB::table(\DB::raw($query))->chunk(1000, function($rows){
// Do something
});
然后执行查询laravel:
select * from (...) somealias LIMIT ... OFFSET ...
这应该至少在Laravel 5.1中起作用。但我不明白为什么它不应该在4 +中工作。
编辑:我也不确定您应该如何设置驱动程序,因为我还没有这个需要,但我&#39 ;我相信它是可能的。
答案 6 :(得分:-1)
我相信您可以在查询构建器上使用chunk
。 E.g。
DB::connection('legacy')->select("...")->chunk(200, function($companies){
//do something with $companies
});
答案 7 :(得分:-2)
尝试使用 orderBy 子句:
DB::table('tbl')->where('num', '>', 3)->orderBy('id')->chunk(500, function($rows) {
// process $rows
});