在C中读取大文件时出现内存错误

时间:2014-07-08 19:28:30

标签: c memory valgrind

我非常喜欢在C中使用malloc / realloc,因此需要一些帮助。以下是大型程序的简短片段,我打算读取一个大(fasta)文件,差不多80000个行号,并存储标有>的第一行。 (名称)和后续行(序列)在两个单独的数组中 - fasta_name和fasta_seq。我使用了一个标准库,它将输出作为一个结构,但我需要将它更改为数组,因为稍后在程序中我需要检查用户提供的输入文件中的每个fasta条目。

输入文件格式为:

  

P21306 \ n   MSAWRKAGISYAAYLNVAAQAIRSSLKTELQTASVLNRSQTDAFYTQYKNGTAASEPTPITK \ n   P38077 \ n   MLSRIVSNNATRSVMCHQAQVGILYKTN​​PVRTYATLKEVEMRLKSIKNIEKITKTMKIVASTRLSKAEKA \ n

=======================

代码是:

KSEQ_INIT(gzFile,gzread)///生物学家的外部图书馆阅读FASTA格式文件///

  int main(int argc,char *argv[])
  {

char **fasta_name=(char **)malloc(sizeof(char *)*80000);
for(i=0;i<size;i++)
{
    fasta_name[i]=(char*)malloc(sizeof(char)*50);
}
char **fasta_seq=(char**)malloc(sizeof(char *)*80000);
for(i=0;i<size;i++)
{
    fasta_seq[i]=(char*)malloc(sizeof(char)*5000);
}

fpf = gzopen("fasta_seq_nr_uniprot.txt", "r"); 
seq = kseq_init(fpf); 

while((l = kseq_read(seq)) >= 0) 
{ 
    strcpy(fasta_name[index1],seq->name.s);
    strcpy(fasta_seq[index1],seq->seq.s);
    index1++;
}

kseq_destroy(seq); 
    gzclose(fpf); 

for(i=0;i<size;i++)
{
    free(fasta_name[i]);
}
for(i=0;i<size;i++)
{
    free(fasta_seq[i]);
}

free(fasta_name);
free(fasta_seq);

程序显示没有编译错误,但使用Valgrind显示以下内存错误和分段错误。

$ valgrind --track-origins=yes --leak-check=full ./Gwidd_uniprot_map2 xaa==3511== Memcheck, a memory error detector
==3511== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.
==3511== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info
==3511== Command: ./map2 xaa
==3511== 
--3511-- ./map2:
--3511-- dSYM directory has wrong UUID; consider using --dsymutil=yes
Opening file xaa
==3511== Use of uninitialised value of size 8
==3511==    at 0x100012C43: __strcpy_chk (mc_replace_strmem.c:893)
==3511==    by 0x100001A78: __inline_strcpy_chk (in ./map2)
==3511==    by 0x10000183E: main (in ./map2)
==3511==  Uninitialised value was created by a heap allocation
==3511==    at 0x100011345: malloc (vg_replace_malloc.c:236)
==3511==    by 0x10000170C: main (in ./map2)
==3511== 
==3511== Invalid write of size 1
==3511==    at 0x100012C43: __strcpy_chk (mc_replace_strmem.c:893)
==3511==    by 0x100001A78: __inline_strcpy_chk (in ./map2)
==3511==    by 0x10000183E: main (in ./map2)
==3511==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==3511== 
==3511== 
==3511== Process terminating with default action of signal 11 (SIGSEGV)
==3511==  Access not within mapped region at address 0x0
==3511==    at 0x100012C43: __strcpy_chk (mc_replace_strmem.c:893)
==3511==    by 0x100001A78: __inline_strcpy_chk (in ./map2)
==3511==    by 0x10000183E: main (in ./map2)
==3511==  If you believe this happened as a result of a stack
==3511==  overflow in your program's main thread (unlikely but
==3511==  possible), you can try to increase the size of the
==3511==  main thread stack using the --main-stacksize= flag.
==3511==  The main thread stack size used in this run was 8388608.
==3511== 
==3511== HEAP SUMMARY:
==3511==     in use at exit: 6,674,813 bytes in 3,664 blocks
==3511==   total heap usage: 3,807 allocs, 143 frees, 6,698,108 bytes allocated
==3511== 
==3511== LEAK SUMMARY:
==3511==    definitely lost: 0 bytes in 0 blocks
==3511==    indirectly lost: 0 bytes in 0 blocks
==3511==      possibly lost: 0 bytes in 0 blocks
==3511==    still reachable: 6,674,813 bytes in 3,664 blocks
==3511==         suppressed: 0 bytes in 0 blocks
==3511== Reachable blocks (those to which a pointer was found) are not shown.
==3511== To see them, rerun with: --leak-check=full --show-reachable=yes
==3511== 
==3511== For counts of detected and suppressed errors, rerun with: -v
==3511== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
Segmentation fault

编辑编辑:

char **fasta_name=(char **)malloc(sizeof(char *)*80000);
if(fasta_name == NULL)
{
    printf("Unable to allocate memory.\n");
    exit(EXIT_FAILURE);
}
for(i=0;i<size;i++)
{
    fasta_name[i]=(char*)malloc(sizeof(char)*50);
    if(fasta_name[i] == NULL)
    {
        printf("Unable to allocate memory.\n");
        exit(EXIT_FAILURE);
    }
}
char **fasta_seq=(char**)malloc(sizeof(char *)*80000);
if(fasta_seq == NULL)
{
    printf("Unable to allocate memory.\n");
    exit(EXIT_FAILURE);
}
for(i=0;i<size;i++)
{
    fasta_seq[i]=(char*)malloc(sizeof(char)*5000);
    if(fasta_seq[i] == NULL)
    {
        printf("Unable to allocate memory.\n");
        exit(EXIT_FAILURE);
    }
}

fpf = gzopen("fasta_seq_nr_uniprot.txt", "r"); 
seq = kseq_init(fpf); 
while((l = kseq_read(seq)) >= 0) 
{ 
    strcpy(fasta_name[index1],seq->name.s);
    strcpy(fasta_seq[index1],seq->seq.s);
    index1++;
}
kseq_destroy(seq); 
gzclose(fpf); 

1 个答案:

答案 0 :(得分:2)

基于错误Access not within mapped region at address,我会说您正在访问超出界限的内存,导致您的分段错误。在调用malloc()之后检查malloc()是否没有返回NULL,否则你可能会冒险访问你实际上并不拥有的内存。

每次int *i = (int *) malloc(sizeof(int)); if(!i){ fprintf(stderr, "Something went wrong with malloc(), exiting...\n"); exit(1); } 次呼叫后执行此操作。例如:

{{1}}

要检查的另一件事是查看index1是否超出了您的分配范围。我没有看到它被声明或初始化的位置。你是否有可能为fasta_name和其他人分配太小,因为你使用一个不断增加索引的while循环读取信息?