Question

我在导入超过1,00,000行的CSV文件时遇到问题。我正在使用fgetcsv从CSV文件中获取数据。

我的要求是首先截断表格，从CSV文件中提取数据并将其存储在数据库中。但每次它只获取40000到50000行。任何人都可以建议我是什么问题？我的代码如下，

$destination    = realpath('../../admin/webroot/');
$originalPath   = $destination."/".$this->request->data['Importmodel']['importfile'];

# Get Table Fields
$tableFields = ClassRegistry::init('Systemoption')->getTableDetails($this->request->data['Importmodel']['modeloptions']);

ClassRegistry::init($this->request->data['Importmodel']['modeloptions'])->query("TRUNCATE TABLE " . $this->request->data['Importmodel']['modeloptions']);
$row = 1;
if(($handle = fopen($originalPath, "r")) !== false) {
    try {
        while (($data = fgetcsv($handle, 0, "|")) !== false) {
            $num = count($data);

            if($num != (count($tableFields) - 1)) { 
                unlink($originalPath);
                $this->Session->setFlash("Table \"".$this->request->data['Importmodel']['modeloptions']."\" fields and \"".$fileName."\" fields does not match", 'default', array('class'=>'flash_bad'));
                $this->redirect(array('action' => 'index'));
                exit;
            }

            $queryData = array();
            for ($c=0; $c < $num; $c++) {
                $queryData[$this->request->data['Importmodel']['modeloptions']][$tableFields[$c+1]] = $data[$c];
            }

            ClassRegistry::init($this->request->data['Importmodel']['modeloptions'])->create();
            ClassRegistry::init($this->request->data['Importmodel']['modeloptions'])->save($queryData);
            $row++;
        }
        $result = 1;
    } catch (Exception $e) {
        $result = 0;    
    }
}

if($result == 1) {
    unlink($originalPath);
    $this->Session->setFlash("Table \"" . $this->request->data['Importmodel']['modeloptions'] . "\" data populated successfully", 'default', array('class'=>'flash_good'));
    $this->redirect(array('action' => 'index'));
    exit;
} else {
    $this->Session->setFlash("Error : Table \"" . $this->request->data['Importmodel']['modeloptions'] . "\" data not fully populated", 'default', array('class'=>'flash_bad'));
    $this->redirect(array('action' => 'index'));
    exit;
}

Answer 1

你是从控制台做的吗？如果没有，请尝试增加php.ini中的内存和时间限制

在php或Cakephp中使用fgetcsv导入大型CSV文件

1 个答案: