如何在php中使用大内容优化文件创建

时间:2017-03-14 06:34:43

标签: php sql-server magento optimization query-optimization

我编写了一个脚本来将Mssql表复制到mysql表。表包含超过200000行。当Iam尝试创建一个200000行的文件时,面临执行超时。甚至我已经将批量处理用于循环。

请使用优化的方法帮助我更快地创建大文件内的文件。

请参阅代码并帮助我优化将大表从mssql复制到mysql

的方法
                $resource = Mage::getSingleton('core/resource');
                $writeConnection = $resource->getConnection('core_write');
                $totalcount = 200000;

                $offset  = 0;
                $limit  = 900;
                $counter = 0;
                $cmpval = 0;


                while ($offset<$totalcount) {                   
                    $content = "";
                    $table="SELECT  * FROM emp ORDER BY id DESC OFFSET ".$offset." ROWS FETCH NEXT ".$limit." ROWS ONLY";
                    $tableResult = mssql_query($table,$link) or die(mssql_get_last_message());

                    $sql_rows = array();
                    while(list($name ,$desig, $dep) = mssql_fetch_array($tableResult)){                 



                                $sql_rows[] = ",$name,$desig, $dep,";
                                $counter++;

                    }

                    $content = implode('\n', $sql_rows);    
                    $aaaa = 'echo "'.$content.'" >> '.$_SERVER['DOCUMENT_ROOT'].'/tmp/myText.txt';      
                    $out = shell_exec($aaaa);

                    // $offset  = $limit;
                    $offset  = $offset+$limit;
                    $cmpval = $totalcount-$offset;              
                    if($cmpval<$limit){                 
                        $limit = $cmpval;
                    }else{
                        $limit = $limit;
                    }                               
                }                                



                $qry1="LOAD DATA INFILE '".$_SERVER['DOCUMENT_ROOT'] . "/tmp/myText.txt"."' INTO TABLE emp FIELDS TERMINATED BY ',';";

                $writeConnection->query($qry1);

                echo $counter." Total Records Inserted.";

0 个答案:

没有答案