解析一个非常大的JSON文件

时间:2016-12-29 00:19:12

标签: php json parsing memory-management memory-limit

我正在尝试导入JSON文件。我的系统Ram是8GB,memory_limit是-1。我仍然得到错误

  

致命错误:允许的内存大小耗尽385875968字节(尝试过   在中分配190844952个字节)   d:\ XAMPP \ htdocs中\ Builder_Ux \的wp-content \上传\ builderux \ 2ae8cd02-24d6-4979-af0b-fddd322ffef9-1 \ text.php   第20行

我还尝试使用streamer以块的形式读取数据。但由于数据是json所以我只有在读完整个文件后才能解码json。这是我的代码

$files=array('/phaseplanoption.json');
foreach($files as $filename){
    $filepath=__DIR__ .$filename;
    $parser=new Bx_JSON_Parser;
    $jsondata=$parser->parse($filepath);
    $arraydata=json_decode($jsondata,true);
    //echo $content=file_get_contents($filename);
    //$content=json_decode($content,true);
    echo count($arraydata['d']); echo "<br>";
}



class Bx_JSON_Parser
{
    public $jsoncontent='';
    public function parse($filename){
        $success = $this->file_get_contents_chunked($filename,4096,function($chunk,&$handle,$iteration){
            $this->jsoncontent=$this->jsoncontent.$chunk;   // this is the line no 20
            /*
                * Do what you will with the {&chunk} here
                * {$handle} is passed in case you want to seek
                ** to different parts of the file
                * {$iteration} is the section fo the file that has been read so
                * ($i * 4096) is your current offset within the file.
            */

        });

        return $this->jsoncontent;
    }
    public function file_get_contents_chunked($file,$chunk_size,$callback)
    {
        try
        {
            $handle = fopen($file, "r");
            $i = 0;
            while (!feof($handle))
            {
                call_user_func_array($callback,array(fread($handle,$chunk_size),&$handle,$i));
                $i++;
            }

            fclose($handle);

        }
        catch(Exception $e)
        {
             trigger_error("file_get_contents_chunked::" . $e->getMessage(),E_USER_NOTICE);
             return false;
        }

        return true;
    }

}

错误在第20行,第20行是$this->jsoncontent=$this->jsoncontent.$chunk;我知道它,因为我正在为变量分配所有内容。但有人可以告诉我这是做什么更好的方法吗?

文件大小为623 MB

0 个答案:

没有答案