Question

我正在开发Objective C Cocoa应用程序。我测试CC_MD5 in CommonCrypto，它运作得很好;然而，当我给它5个gygabyte文件时，我的整个计算机都冻结了并且崩溃了。 MD5算法将输入处理为512字节的块，并不一定需要所有输入。在Objective C或C中是否有一个库要求下一个512字节的块，而不是一次性获取所有输入？

Answer 1

在obj-C中计算大文件的MD5有一个很棒的主题： http://www.iphonedevsdk.com/forum/iphone-sdk-development/17659-calculating-md5-hash-large-file.html

以下是有人提出的解决方案：

+(NSString*)fileMD5:(NSString*)path
{
    NSFileHandle *handle = [NSFileHandle fileHandleForReadingAtPath:path];
    if( handle== nil ) return @"ERROR GETTING FILE MD5"; // file didnt exist

    CC_MD5_CTX md5;

    CC_MD5_Init(&md5);

    BOOL done = NO;
    while(!done)
    {
        NSAutoreleasePool * pool = [NSAutoreleasePool new];
        NSData* fileData = [handle readDataOfLength: CHUNK_SIZE ];
        CC_MD5_Update(&md5, [fileData bytes], [fileData length]);
        if( [fileData length] == 0 ) done = YES;
                [pool drain];
    }
    unsigned char digest[CC_MD5_DIGEST_LENGTH];
    CC_MD5_Final(digest, &md5);
    NSString* s = [NSString stringWithFormat: @"%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x%02x",
                   digest[0], digest[1], 
                   digest[2], digest[3],
                   digest[4], digest[5],
                   digest[6], digest[7],
                   digest[8], digest[9],
                   digest[10], digest[11],
                   digest[12], digest[13],
                   digest[14], digest[15]];
    return s;
}

Answer 2

CC_MD5()旨在立即处理其所有输入。 5GB可能比它实际存储的任何地方都多。对于较大的数据，如果您使用CC_MD5_CTX，CC_MD5_Init()，CC_MD5_Update()和CC_MD5_Final()，CommonCrypto可以一次对其进行操作。查看CommonCrypto文档或Google以获取更多信息和示例代码。

Answer 3

这是使用dispatch apis更好的方法，以提高效率。我在生产中使用它并且工作正常！

    #import "CalculateMD5.h"

// Cryptography
#include <CommonCrypto/CommonDigest.h>

@implementation CalculateMD5

- (id)init
{
    self = [super init];
    if (self)
    {
        MD5ChecksumOperationQueue = dispatch_queue_create("com.test.calculateMD5Checksum", DISPATCH_QUEUE_SERIAL);
    }
    return self;
}

- (void)closeReadChannel
{
    dispatch_async(MD5ChecksumOperationQueue, ^{
        dispatch_io_close(readChannel, DISPATCH_IO_STOP);
    });
}

- (void)MD5Checksum:(NSString *)pathToFile TCB:(void(^)(NSString *md5, NSError *error))tcb
{
    // Initialize the hash object
    __block CC_MD5_CTX hashObject;
    CC_MD5_Init(&hashObject);

    readChannel = dispatch_io_create_with_path(DISPATCH_IO_STREAM,
                                               pathToFile.UTF8String,
                                               O_RDONLY, 0,
                                               MD5ChecksumOperationQueue,
                                               ^(int error) {
                                                   [self closeReadChannel];
                                               });

    if (readChannel == nil)
    {
        NSError* e = [NSError errorWithDomain:@"MD5Error"
                                         code:-999 userInfo:@{
                   NSLocalizedDescriptionKey : @"failed to open file for calculating MD5."
                      }];
        tcb(nil, e);
        return;
    }

    dispatch_io_set_high_water(readChannel, 512*1024);

    dispatch_io_read(readChannel, 0, SIZE_MAX, MD5ChecksumOperationQueue, ^(bool done, dispatch_data_t data, int error) {
        if (error != 0)
        {
            NSError* e = [NSError errorWithDomain:@"ExamSoftMD5"
                                             code:error userInfo:@{
                       NSLocalizedDescriptionKey : @"failed to read from file for calculating MD5."
                          }];
            tcb(nil, e);
            [self closeReadChannel];
            return;
        }

        if (dispatch_data_get_size(data) > 0)
        {
            const void *buffer = NULL;
            size_t size = 0;
            data = dispatch_data_create_map(data, &buffer, &size);

            CC_MD5_Update(&hashObject, (const void *)buffer, (CC_LONG)size);
        }

        if (done == YES)
        {
            // Compute the hash digest
            unsigned char digest[CC_MD5_DIGEST_LENGTH];
            CC_MD5_Final(digest, &hashObject);

            // Compute the string result
            char *hash = calloc((2 * sizeof(digest) + 1), sizeof(char));
            for (size_t i = 0; i < sizeof(digest); ++i)
            {
                snprintf(hash + (2 * i), 3, "%02x", (int)(digest[i]));
            }

            tcb(@(hash), nil);

            [self closeReadChannel];
        }
    });
}


@end

是否有一个MD5库不同时需要整个输入？

3 个答案: