使用Laravel 5.2对DB进行批量插入时存在性能问题。 我从server1获取数据，格式化数据并批量插入server2。 最初，性能似乎很好，但随着记录的增加，插入需要时间。
如何提高性能？
我的控制器是：
<?php
namespace App\Models\Soa\Ocean;
use Illuminate\Database\Eloquent\Model;
use DB;
class RejectedR11 extends Model{
protected $table= "R11RejectedData";
public $primaryKey = "R11RejectedDataID";
public $connection = "ocean";
public static function deleteRejectedR11($data){
for($i=0;$i<count($data);$i++){
DB::connection('ocean')->statement('DELETE FROM R11RejectedData WHERE R11RejectedDataID = '.$data[$i]['R11RejectedDataID']);
}
}
}
?>
RejectedR11型号：
<?php
namespace App\Models\SOA;
use Illuminate\Database\Eloquent\Model;
use Session;
use DB;
class OceanUpload extends Model{
public $table = 'fanda_soa_ocean';
public $primaryid = 'ocean_id';
public $timestamps = false;
protected $connection = 'main_db';
public static function insertRejected($data,$code,$type){
$res = array();
if(!empty($data)){
for($i=0;$i<count($data);$i++){
$invodate = explode(" ",str_replace('?','',trim(preg_replace("/[^a-zA-Z0-9_.\s()+-@:\"\'\\\\\&-]+/", "",preg_replace('/_x000D_/', '', $data[$i]['InvoiceDate']) ))));
$duedate = isset($data[$i]['DueDate'])?explode(" ",str_replace('?','',trim(preg_replace("/[^a-zA-Z0-9_.\s()+-@:\"\'\\\\\&-]+/", "",preg_replace('/_x000D_/', '', $data[$i]['DueDate']) )))):"-";
$res[] = array(
'ocean_countrycode'=>$code,
'ocean_countrystruct'=>$type,
'ocean_status'=>"Rejected",
'ocean_vendorname'=>isset($data[$i]['VendorName'])?str_replace('?','',trim(preg_replace("/[^a-zA-Z0-9_.\s()+-@:\"\'\\\\\&-]+/", "",preg_replace('/_x000D_/', '', $data[$i]['VendorName']) ))):"-",
'ocean_vendornum'=>isset($data[$i]['VendorNum'])?str_replace('?','',trim(preg_replace("/[^a-zA-Z0-9_.\s()+-@:\"\'\\\\\&-]+/", "",preg_replace('/_x000D_/', '', $data[$i]['VendorNum']) ))):"-",
'ocean_vendorsite'=>isset($data[$i]['VendorSiteCode'])?str_replace('?','',trim(preg_replace("/[^a-zA-Z0-9_.\s()+-@:\"\'\\\\\&-]+/", "",preg_replace('/_x000D_/', '', $data[$i]['VendorSiteCode']) ))):"-",
'ocean_currency'=>isset($data[$i]['InvoiceCurrencyCode'])?str_replace('?','',trim(preg_replace("/[^a-zA-Z0-9_.\s()+-@:\"\'\\\\\&-]+/", "",preg_replace('/_x000D_/', '', $data[$i]['InvoiceCurrencyCode']) ))):"-",
'ocean_invoicenum'=>isset($data[$i]['InvoiceNum'])?str_replace('?','',trim(preg_replace("/[^a-zA-Z0-9_.\s()+-@:\"\'\\\\\&-]+/", "",preg_replace('/_x000D_/', '', $data[$i]['InvoiceNum']) ))):"-",
'ocean_invoiceamt'=>isset($data[$i]['InvoiceAmountSUM'])?str_replace('?','',trim(preg_replace("/[^a-zA-Z0-9_.\s()+-@:\"\'\\\\\&-]+/", "",preg_replace('/_x000D_/', '', $data[$i]['InvoiceAmountSUM']) ))):"-",
'ocean_invodate'=>date('Y-m-d',strtotime($invodate[0])),
'ocean_termsdate'=>isset($data[$i]['TermsDate'])?str_replace('?','',trim(preg_replace("/[^a-zA-Z0-9_.\s()+-@:\"\'\\\\\&-]+/", "",preg_replace('/_x000D_/', '', $data[$i]['TermsDate']) ))):"-",
'ocean_approvstatus'=>isset($data[$i]['ApprovalStatus'])?str_replace('?','',trim(preg_replace("/[^a-zA-Z0-9_.\s()+-@:\"\'\\\\\&-]+/", "",preg_replace('/_x000D_/', '', $data[$i]['ApprovalStatus']) ))):"-",
'ocean_approvname'=>isset($data[$i]['ApproverName'])?str_replace('?','',trim(preg_replace("/[^a-zA-Z0-9_.\s()+-@:\"\'\\\\\&-]+/", "",preg_replace('/_x000D_/', '', $data[$i]['ApproverName']) ))):"-",
'ocean_approvdate'=>isset($data[$i]['ApprovalActionDate'])?str_replace('?','',trim(preg_replace("/[^a-zA-Z0-9_.\s()+-@:\"\'\\\\\&-]+/", "",preg_replace('/_x000D_/', '', $data[$i]['ApprovalActionDate']) ))):"-",
'ocean_duedate'=>($duedate <> "-")?date('Y-m-d',strtotime($duedate[0])):"-",
'ocean_invocurrcode'=>isset($data[$i]['InvoiceCurrencyCode'])?str_replace('?','',trim(preg_replace("/[^a-zA-Z0-9_.\s()+-@:\"\'\\\\\&-]+/", "",preg_replace('/_x000D_/', '', $data[$i]['InvoiceCurrencyCode']) ))):"-",
'ocean_imagerefnum'=>isset($data[$i]['ImagingRefNumber'])?str_replace('?','',trim(preg_replace("/[^a-zA-Z0-9_.\s()+-@:\"\'\\\\\&-]+/", "",preg_replace('/_x000D_/', '', $data[$i]['ImagingRefNumber']) ))):"-",
'ocean_reasonlabel'=>isset($data[$i]['ReasonLabel'])?str_replace('?','',trim(preg_replace("/[^a-zA-Z0-9_.\s()+-@:\"\'\\\\\&-]+/", "",preg_replace('/_x000D_/', '', $data[$i]['ReasonLabel']) ))):"-",
'ocean_reasoncode'=>isset($data[$i]['ReasonCode'])?str_replace('?','',trim(preg_replace("/[^a-zA-Z0-9_.\s()+-@:\"\'\\\\\&-]+/", "",preg_replace('/_x000D_/', '', $data[$i]['ReasonCode']) ))):"-",
'ocean_recent'=>1,
'ocean_uploadby'=>Session::get('login_id'),
'ocean_uploadon'=>date('Y-m-d'),
);
foreach($res as $val):
DB::statement('UPDATE '.'fanda.fanda_soa_ocean'.' SET ocean_recent = 0 where ocean_countrycode = "'.$val['ocean_countrycode'].'" and ocean_vendorname = "'.$val['ocean_vendorname'].'" and ocean_invoicenum = "'.$val['ocean_invoicenum'].'"');
DB::statement('INSERT IGNORE INTO '.'fanda.fanda_soa_ocean'.' ('.implode(',',array_keys($val)).') values (?'.str_repeat(',?',count($val) - 1).') ON DUPLICATE KEY UPDATE ocean_recent = 1',array_values($val));
endforeach;
}
}
}
}
?>
OceanUpload模型：
long startTime = System.currentTimeMillis();
byte[] plain = plaintext.getText().getBytes();
byte[] K = key.getText().getBytes();
byte[] encrypted = encrypt(plain, K);
String a = bytesToHex(encrypted);
encryptedtext.setText(a);
long stopTime = System.currentTimeMillis();
long elapsedTime = stopTime - startTime;
time.setText(elapsedTime);
答案 0 :(得分：1)
您的代码变得越来越慢，因为每次添加新项目时，UPDATE中的所有项目都会运行INSERT和$res个查询。所以你一遍又一遍地运行相同的查询。
您可以通过收集ID并在最后运行一个查询来改进DELETE查询：
DELETE FROM R11RejectedData WHERE IN ($ids)
您可以使用预准备语句改进INSERT和UPDATE查询：
$sql = 'INSERT INTO table1 (column1, ...) VALUES (?, ...)';
$statement = DB::getPdo()->prepare($sql);
foreach(...) {
$statement->execute([$value1, ...]);
}
$statement->closeCursor();
答案 1 :(得分：1)
我离Laravel真的很远，但是最近不得不在OctoberCMS（基于Laravel）中向MySQL DB发出潜在的巨大更新请求。因此，这是我使用“插入重复键更新”的快速功能的拙劣示例，可以一次插入多个条目：
class MyModel extends Model
{
...
/**
* @param array $columns Columns definition (map), e.g.: ['id', 'date', 'col2', [...]]
* @param array $sets Actual data for insert [[1, '2018-03-01', 'someVal'], [...]]
* @return bool
* @throws \Exception
*/
public static function onDuplicateKeyUpdate(array $columns, array $sets = [])
{
$colCount = count($columns);
if(!$sets || $colCount < 2){
return false;
}
$model = new static();
$keyName = $model->getKeyName();
if(array_key_exists($keyName, $columns)){
throw new \Exception('Columns should contain key `' . $model->getKeyName() . '`');
}
$model->addFillable([$keyName]);
$sqlParams = $sqlVals = $sqlCols = [];
foreach($sets as $k => $attributes){
if(array_diff_key($columns, $attributes)){
throw new \Exception('Invalid attributes provided. Should be: ' . print_r($columns, true) . ' Provided: ' . print_r($attributes, true));
}
$model->fill(array_combine($columns, $attributes));
if(count($attributes) != count($model->getAttributes())){
throw new \Exception('Impossible to process some columns: ' . print_r($columns) . '`');
}
$model->validate();
$sqlVals[] = '(?' . str_repeat(',?', $colCount - 1) . ')';
array_push($sqlParams, ...$attributes);
}
foreach($columns as $column){
if($column !== $keyName){
$sqlCols[] = '`' . $column . '`=VALUES(`' . $column . '`)';
}
}
$query = $model->newBaseQueryBuilder();
$grammar = $query->getGrammar();
$connection = $query->getConnection();
$table = $grammar->wrapTable($model->getTable());
$columns = ' (' . $grammar->columnize($columns) . ')';
return $connection->insert('INSERT INTO ' . $table . $columns . ' VALUES ' . implode(',', $sqlVals) . ' ON DUPLICATE KEY UPDATE ' . implode(', ', $sqlCols), $sqlParams);
}
}
这里的想法是最大限度地利用模型功能，例如类型转换，检查是否允许更新列等。还使用了一些laravel技术，例如包装表和列。