如何一次将831 MB文本文件读入数组?

时间:2017-03-19 17:32:26

标签: arrays excel vba excel-vba large-files

我想从831 MB文本文件中读取数据。不幸的是我得到运行时错误14.超出字符串空间。我该如何更改代码?我认为应该可以将整个数据读入我的笔记本上的数组(W10 64位,24 GB RAM,CPU 7500U,Excel 2016)。

您可以从The Global Drifter Program(buoydata_15001_jun16.dat)获取数据。我需要处理前7列的数据,这些列用空格分隔,而不是用分号。

Sub DelimitedTextFileToArray()
'PURPOSE: Load an Array variable with data from a delimited text file
'SOURCE: www.TheSpreadsheetGuru.com

Dim Delimiter As String
Dim TextFile As Integer
Dim FilePath As String
Dim FileContent As String
Dim LineArray() As String
Dim DataArray() As String
Dim TempArray() As String
Dim rw As Long, col As Long

'Inputs
  Delimiter = ";"
  FilePath = "C:\buoydata_15001_jun16.txt"
  rw = 0

'Open the text file in a Read State
  TextFile = FreeFile
  Open FilePath For Input As TextFile

'Store file content inside a variable
  FileContent = Input(LOF(TextFile), TextFile)

'Close Text File
  Close TextFile

'Separate Out lines of data
  LineArray() = Split(FileContent, vbCrLf)

'Read Data into an Array Variable
  For x = LBound(LineArray) To UBound(LineArray)
    If Len(Trim(LineArray(x))) <> 0 Then
      'Split up line of text by delimiter
        TempArray = Split(LineArray(x), Delimiter)

      'Determine how many columns are needed
        col = UBound(TempArray)

      'Re-Adjust Array boundaries
        ReDim Preserve DataArray(col, rw)

      'Load line of data into Array variable
        For y = LBound(TempArray) To UBound(TempArray)
          DataArray(y, rw) = TempArray(y)
        Next y
    End If

    'Next line
      rw = rw + 1

  Next x

End Sub 

1 个答案:

答案 0 :(得分:0)

我不认为Excel会处理一个大小近1GB的文件。为什么不使用为R,Python,SQL Server甚至MS Access等大型数据集设计的工具。虽然Access甚至可能会因为在导入更大的数据集时很快碎片化而窒息。我想你可以尝试看看会发生什么!!