Question

我需要获取文件的前10mb并从中计算md5，我该如何实现？我找不到任何文件读取样本。我有这样的事情：

FileStream file = new FileStream(fileName, FileMode.Open);
MD5 md5 = new MD5CryptoServiceProvider();
byte[] retVal = md5.ComputeHash(file);
file.Close();

StringBuilder sb = new StringBuilder();
for (int i = 0; i < retVal.Length; i++)
{
     sb.Append(retVal[i].ToString("x2"));
}
var md5Final = sb.ToString();

但它读取整个文件。

Answer 1

您可以使用TransformBlock以块的形式读取文件并将其以块的形式提供给MD5CryptoServiceProvider。这样，您不必为缓冲区消耗10 MB内存。示例：

long read = 0;
int r = -1; 
const long bytesToRead = 10 * 1024 * 1024;
const int bufferSize = 10*1024;
byte[] buffer = new byte[bufferSize];
MD5 md5 = new MD5CryptoServiceProvider();
using(var stream = new FileStream(fileName, FileMode.Open, FileAccess.Read) )
{
    while(read <= bytesToRead && r != 0) 
    {
        read += (r = stream.Read(buffer, 0, bufferSize));
        md5.TransformBlock(buffer, 0, r, null, 0);
    }
}
md5.TransformFinalBlock(buffer, 0,0);
string md5Final = String.Join("", md5.Hash.Select(x => x.ToString("x2")));

Answer 2

要使用小块读取文件的一部分，您可以尝试：

public byte[] ReadPart(Stream stream)
{
    byte[] buffer = new byte[1024];
    int read=0;

    int chunk;
    while ( (chunk = stream.Read(buffer, read, buffer.Length-read)) > 0)
    {
        read += chunk;
        if (read == buffer.Length && read < 10*1024*1024) // 10MB
        {
            int nextByte = stream.ReadByte();
            if (nextByte==-1)
            {
                return buffer;
            }

            byte[] newBuffer = new byte[buffer.Length*2];
            Array.Copy(buffer, newBuffer, buffer.Length);
            newBuffer[read] = (byte)nextByte;
            buffer = newBuffer;
            read++;
        }
    }
    // Buffer is now too big. Shrink it.
    byte[] ret = new byte[read];
    Array.Copy(buffer, ret, read);
    return ret;
}

然后在此阵列上计算MD5 ......

Answer 3

  var bytes = new byte[10000000]; // Or 10*1024*1024 according to your def of a MB
  int realLength;

  using (var file = new FileStream(filename, FileMode.Open))
  {
    realLength = file.Read(bytes, 0, bytes.Length);
    // The file may be shorter than expected.
  }

  var md5 = new MD5CryptoServiceProvider();
  byte[] hash = md5.ComputeHash(bytes, 0, realLength);

我不得不承认，分配10MB可能不是最优雅的解决方案。我用10KB而不是10MB写它。 YMMV。

Answer 4

你可以将第一个10Mb读入一个byte []，然后给ComputeHash一个MemoryStream的实例，由byte []支持。

Answer 5

您可以实施一个Stream类来限制基础Stream对象的长度（在您的情况下为FileStream）：

public class LengthLimitedStream : Stream
{
    private Stream baseStream;
    private long maxLength;

    public LengthLimitedStream(Stream stream, long maxLength)
    {
        if (stream.Position > maxLength) { throw new IOException(); }

        this.baseStream = stream;
        this.maxLength = maxLength;
    }

    public override bool CanRead
    {
        get { return this.baseStream.CanRead; }
    }

    public override long Length
    {
        get { return Math.Min(this.maxLength, this.baseStream.Length); }
    }

    public override long Position
    {
        get
        {
            return this.baseStream.Position;
        }
        set
        {
            if (value > maxLength)
            {
                throw new IOException();
            }

            this.baseStream.Position = value;
        }
    }

    public override int Read(byte[] buffer, int offset, int count)
    {
        if (this.Position + offset + count > this.maxLength)
        {
            count = (int)(this.maxLength - (this.Position + offset));
        }

        return this.baseStream.Read(buffer, offset, count);
    }

    // Lots of stuff omitted you may want to implement but not important in this case ...
}

（添加参数验证等等，但你会得到这个想法）

这样您仍然可以使用ComputeHash(Stream)方法，它看起来简洁明了（在我看来）。

Answer 6

如何将文件转换为字节数组，这样就可以遍历前10MB。获取字节数组=＆gt;

private byte [] StreamFile(string filename)
{ 
FileStream fs = new FileStream(filename, FileMode.Open,FileAccess.Read); 

// Create a byte array of file stream length 
 byte[] fileData = new byte[fs.Length]; 

//Read block of bytes from stream into the byte array 
fs.Read(fileData,0,System.Convert.ToInt32(fs.Length)); 

//Close the File Stream 
fs.Close(); 

return fileData; //return the byte data
}

在此之后，我将循环遍历fileData，并取10个第一个MB并执行MD5

使用c＃从文件中读取前10MB

6 个答案: