拆分不是一项微不足道的任务

Question

我有一个超过16 MB的CSV文件。

当我用它阅读时：

$exportString = @file_get_contents($url, false, stream_context_create($contextOptions)

我只是想回声一下：

$data=explode(';', $exportString);
echo $data[0];

然后此消息显示在我的浏览器中：

Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 71 bytes) in....

在此之后，我想在MySQL DB中导入它。

任何帮助？

Answer 1

在 php.ini 文件中，您可以增加允许的内存大小

memory_limit = 512M

或者在脚本顶部放置：

ini_set("memory_limit","512M");

或者，如果您无权访问 php.ini ，请在根目录中创建 .htaccess 文件并将其放入

php_value memory_limit = "512M"

编辑：268435456字节= 256MB，所以要大一点！

请记住，拥有巨大的内存限制并不能代替编写好的代码。最好使用file_get_contents个附加参数offset和length将其拆分为多个块。

拆分不是一项微不足道的任务

但是，这是一个关于如何做到这一点的简单算法！

1. Initialize an empty string
  (begin a loop)
2. Grab a chunk from your file and append that to the string
3. Search for the last \n character in that string (MAKE SURE IT ISN'T PART OF DATA)
  a. If \n doesn't exist, continue
  b. If it does, grab the first substring up to that point and process that.
     Once finished grab the rest of substring assign it to your initial string.
  (loop until finished)
4. If there is data in the string left, do processing on that as well.

现在，用于查找字符串中最后一个“\ n”的算法

1. Initialize a variable called $inString = false and 
2. Initialize a variable $newLinePos = -1
3. Loop through each character of the string
  (begin loop)
  a. If the current charater is a double quote (")
     AND the character before IS NOT a backslash (\)
     Then set $inString = !$inString;
  b. If $inString Then continue;
  c. If the current character is the newline (\n)
     Then set $newLinePos to the index of the current character
  (end loop)
4. If $newLinePos == -1 then we have not found any valid \n and we need to grab more
   Otherwise, go on with the next part as perscribed above

如何在没有内存耗尽的情况下读取大型CSV文件？

1 个答案:

拆分不是一项微不足道的任务