MySQL：内存分配问题

Question

我在计算机上使用WAMP，当我运行从Excel文件导入数据的算法时，它只需要太长时间。首先，它会抛出一个没有足够内存的错误。然后，我在memory_limit上扩展了php.ini参数。但是，这一次，它运行了很长时间，然后它只显示着名的“页面无法显示”（连接中止）。

此外，该算法只需导入5行。但它有点长。它根据从Excel文件中获取的值和stuf ..

创建对象

那么，您怎么看？

N.B：我正在使用Symfony2，Doctrine2和PHPExcel库来帮助操作类似Excel的文件。

编辑（这是代码）：

<?php

public function bachelierAction(){

        $inputFileType = 'Excel2007';
        $inputFileName = 'Excel_Files/Bacheliers/Bacheliers12.xlsx';


        /**  Create a new Reader of the type defined in $inputFileType  **/
        $objReader = \PHPExcel_IOFactory::createReader($inputFileType);
        $objReader->setReadDataOnly(true);
        /**  Define how many rows we want to read for each "chunk"  **/
        $chunkSize = 1;
        /**  Create a new Instance of our Read Filter  **/
        $chunkFilter = new chunkReadFilter();

        /**  Tell the Reader that we want to use the Read Filter  **/
        $objReader->setReadFilter($chunkFilter);
        $objPHPExcel = $objReader->load($inputFileName);

        // The Entity Manager :
        $em = $this->getDoctrine()->getManager();

        $anneObj = new Annee();
        $anneObj->setAnnee(2012);


        for ($startRow = 2; $startRow <= 6 ; $startRow += $chunkSize) {
            /*  Tell the Read Filter which rows we want this iteration  */
            $chunkFilter->setRows($startRow,$chunkSize);
            /**  Load only the rows that match our filter  **/
            $objPHPExcel = $objReader->load($inputFileName);
            $sheet = $objPHPExcel->getSheet(0);

            // Extraire les données du bachelier :
            $cne = $sheet->getCellByColumnAndRow(1,$startRow)->getValue();
            $nom = $sheet->getCellByColumnAndRow(2,$startRow)->getValue();
            $prenom = $sheet->getCellByColumnAndRow(3,$startRow)->getValue();
            $sexe = $sheet->getCellByColumnAndRow(5,$startRow)->getValue();;
            $dateNaissance = $sheet->getCellByColumnAndRow(6,$startRow)->getValue();

            // Création du bachelier
            $bachelier = new Bachelier();
            $bachelier->setCne($cne)
                      ->setDateNaissance($dateNaissance)
                      ->setNom($nom)
                      ->setPrenom($prenom)
                      ->setSexe($sexe);

            $em->persist($bachelier);


            // Extraire les données du bac :
            $moyenneBac = $sheet->getCellByColumnAndRow(10,$startRow)->getValue();
            $typeBac = $sheet->getCellByColumnAndRow(8,$startRow)->getValue();
            $sessionBac = $sheet->getCellByColumnAndRow(13,$startRow)->getValue();
            $mentionBac = $sheet->getCellByColumnAndRow(11,$startRow)->getValue();
            $etabBac = $sheet->getCellByColumnAndRow(14,$startRow)->getValue();


            $typeBacObj = $em->getRepository('PFASIGBundle:TypeBac')->find($typeBac);
            $sessionBacObj = $em->getRepository('PFASIGBundle:SessionBac')->find($sessionBac);
            $mentionBacObj = $em->getRepository('PFASIGBundle:MentionBac')->find($mentionBac);
            $etabBacObj = $em->getRepository('PFASIGBundle:EtablissementBac')->find($etabBac);

            // Création de l'établissement du Bac
            if($etabBacObj == null ){
                // Type de l'établissement du Bac
                $typeEtabBac = $sheet->getCellByColumnAndRow(16,$startRow)->getValue();
                $typeEtabBacObj = $em->getRepository('PFASIGBundle:TypeEtablissementBac')->find($typeEtabBac);

                // La délégation de l'établissement du Bac
                $delegBac = $sheet->getCellByColumnAndRow(19,$startRow)->getValue();
                $delegBacObj = $em->getRepository('PFASIGBundle:Delegation')->find($delegBac);

                $etabBacObj = new EtablissementBac();
                $etabBacLibelle = $sheet->getCellByColumnAndRow(15,$startRow)->getValue();

                $etabBacObj->setCodeLieu($etabBac)
                           ->setNomLieu($etabBacLibelle)
                           ->setDelegation($delegBac)
                           ->setTypeEtabBac($typeEtabBac);

                $em->persist($etabBacObj);

            }

            // Création du bac
            $bac = new Bac();
            $bac->setAnneeBac($anneObj)
                ->setBachelier($bachelier)
                ->setEtabBac($etabBacObj)
                ->setMentionBac($mentionBacObj)
                ->setMoyenne($moyenneBac)
                ->setSessionBac($sessionBac)
                ->setTypeBac($typeBacObj);

            $em->persist($bac);

        }


        $msg = "Les " + $startRow + " bacheliers ont été importés avec succès.";

        try{
            $em->flush();

        }catch(\Doctrine\DBAL\DBALException $e){
            $msg = "Les données n'ont pas pu être importées ! Une erreur est survenue !";

        }

        return $this->render('PFASIGBundle:Carte:bachelier.html.twig',array('msg' => $msg));

    }

编辑2： 在应用Mark的推荐之后，这里的代码只是为了测试：

public function bachelierAction(){

        $inputFileType = 'Excel2007';
        $inputFileName = 'Excel_Files/Bacheliers/Bacheliers12.xlsx';

        // Cell caching ::
        $cacheMethod = \PHPExcel_CachedObjectStorageFactory::cache_to_apc;
        $cacheSettings = array( 'cacheTime' => 600);
        \PHPExcel_Settings::setCacheStorageMethod($cacheMethod, $cacheSettings);


        /**  Create a new Reader of the type defined in $inputFileType  **/
        $objReader = \PHPExcel_IOFactory::createReader($inputFileType);
        $objReader->setReadDataOnly(true);
        /**  Define how many rows we want to read for each "chunk"  **/
        $chunkSize = 1;
        /**  Create a new Instance of our Read Filter  **/
        $chunkFilter = new chunkReadFilter();

        /**  Tell the Reader that we want to use the Read Filter  **/
        $objReader->setReadFilter($chunkFilter);
        $objPHPExcel = $objReader->load($inputFileName);

        // The Entity Manager :
        $em = $this->getDoctrine()->getManager();

        $anneObj = new Annee();
        $anneObj->setAnnee(2012);


        for ($startRow = 2; $startRow <= 6 ; $startRow += $chunkSize) {
            /*  Tell the Read Filter which rows we want this iteration  */
            $chunkFilter->setRows($startRow,$chunkSize);
            /**  Load only the rows that match our filter  **/
            $objPHPExcel = $objReader->load($inputFileName);
            $sheet = $objPHPExcel->getSheet(0);

            // Extraire les données du bachelier :
            list( ,$cne,$nom,$prenom, ,$sexe,$dateNaissance) = $sheet->rangeToArray('A'.$startRow.':F'.$startRow);
        }

        return $this->render('PFASIGBundle:Carte:bachelier.html.twig',array('msg' => $cne));

    }

不幸的是，我仍然收到错误： FatalErrorException: Error: Allowed memory size of 157286400 bytes exhausted (tried to allocate 111967106 bytes) in C:\wamp\www\symf_PFA_v1\vendor\codeplex\phpexcel\PHPExcel\Reader\Excel2007.php line 428

为了记录，在我的php.ini中，我得到了：

memory_limit=150M apc.shm_size=180M

编辑3：：这是当前代码（增强型）

public function bachelierAction(){

        $inputFileType = 'Excel2007';
        $inputFileName = 'myfile.xlsx';

        // Cell caching ::
        $cacheMethod = \PHPExcel_CachedObjectStorageFactory::cache_to_apc;
        $cacheSettings = array( 'cacheTime' => 600);
        \PHPExcel_Settings::setCacheStorageMethod($cacheMethod, $cacheSettings);

        /**  Create a new Reader of the type defined in $inputFileType  **/
        $objReader = \PHPExcel_IOFactory::createReader($inputFileType);
        $objReader->setReadDataOnly(true);
        /**  Define how many rows we want to read for each "chunk"  **/
        $chunkSize = 10;
        /**  Create a new Instance of our Read Filter  **/
        $chunkFilter = new chunkReadFilter();

        /**  Tell the Reader that we want to use the Read Filter  **/
        $objReader->setReadFilter($chunkFilter);
        $objPHPExcel = $objReader->load($inputFileName);


        $sheet = $objPHPExcel->setActiveSheetIndex(0);

for ($startRow = 2; $startRow <= 6 ; $startRow += $chunkSize) {
            /*  Tell the Read Filter which rows we want this iteration  */
            $chunkFilter->setRows($startRow,$chunkSize);
            /**  Load only the rows that match our filter  **/
            $objPHPExcel = $objReader->load($inputFileName);
            $sheet = $objPHPExcel->setActiveSheetIndex(0);

            $result = $sheet->rangeToArray('A'.$startRow.':G'.$startRow);
            list( ,$cne,$nom,$prenom, ,$sexe,$dateNaissance) = $result[0];

            // some process
            $result = $sheet->rangeToArray('I'.$startRow.':O'.$startRow);
            list($typeBac,,$moyenneBac,$mentionBac,,$sessionBac,$etabBac) = $result[0]; 

            // some process

        }
                // render the Twig template
}

班级 chunkReadFilter ：

namespace Me\MyBundle\Entity;

/**  Define a Read Filter class 
 * implementing PHPExcel_Reader_IReadFilter  */


class chunkReadFilter implements \PHPExcel_Reader_IReadFilter 
{ 
    private $_startRow = 0; 
    private $_endRow   = 0; 

    /**  Set the list of rows that we want to read  */ 
    public function setRows($startRow, $chunkSize) { 
        $this->_startRow = $startRow; 
        $this->_endRow   = $startRow + $chunkSize; 
    } 

    public function readCell($column, $row, $worksheetName = '') { 
        //  Only read the heading row, and the configured rows 
        if (($row == 1) ||
            ($row >= $this->_startRow && $row < $this->_endRow)) { 
            return true; 
        } 
        return false; 
    } 
} 

Answer 1

观察＃1

利用PHPExcel的rangeToArray（）方法，而不是使用getCellByColumnAndRow（）单独读取每个单元格。

e.g。

list ($cne, $nom, $prenom, , $sexe, $dateNaissance) = 
    $sheet->rangeToArray('A'.$startRow.':F'.$startRow);

观察＃2

使用单元格缓存优先于通过读取过滤器进行分块，这样您就不会经常为每个块重新读取整个文件，但仍然保存内存

观察＃3

或者，一次使用略高于1行的块大小。如果你有100行，那么你需要重读整个文件100次;将块大小设置为10，你只需要重读10次;将其设置为内存允许的最高值。

观察＃4

将观察＃1，＃2和＃3与一些实验结合起来，看看哪种块大小和缓存方法可以为您提供最佳的内存和速度平衡。