使用多线程的字数统计程序:大文件大小

时间:2014-01-25 19:30:42

标签: c multithreading parallel-processing pthreads

我正在尝试编写一个程序来计算大文件中的单词。我正在做多线程。但我的程序给出了分段错误,我只是被困在这里。我正在寻找导师的任何建议:代码如下:

INPUT:file name 输出:分段错误

代码为:

   #include <stdio.h>
#include <pthread.h>
#include <stdlib.h>


struct thread_data{
    FILE *fp;
    long int offset;
    int start;
    int blockSize;
};

int words=0;  

void *countFrequency(void* data){

    struct thread_data* td=data;
    char *buffer = malloc(td->blockSize);

    int i,c;
    i=0;c=0;
    enum states { WHITESPACE, WORD };
    int state = WHITESPACE;

    fseek(td->fp, td->offset, td->start);

        char last = ' '; 
        while ((fread(buffer, td->blockSize, 1, td->fp))==1){

            if ( buffer[0]== ' ' || buffer[0] == '\t'  ){
            state = WHITESPACE;
            }
            else if (buffer[0]=='\n'){
            //newLine++;
                state = WHITESPACE;
            }
            else {
                if ( state == WHITESPACE ){
                    words++;
                }
                state = WORD;
            }
            last = buffer[0];
    }
    free(buffer);

    pthread_exit(NULL);

    return NULL;
}

int main(int argc, char **argv){

    int nthreads, x, id, blockSize,len;
    //void *state;
    FILE *fp;
    pthread_t *threads;

    struct thread_data data[nthreads];

    if (argc < 2){
        fprintf(stderr, "Usage: ./a.out <file_path>");
        exit(-1);
    }

    if((fp=fopen(argv[1],"r"))==NULL){
        printf("Error opening file");
        exit(-1);
    }  

    printf("Enter the number of threads: ");
    scanf("%d",&nthreads);
    threads = malloc(nthreads*sizeof(pthread_t));

    fseek(fp, 0, SEEK_END);
    len = ftell(fp);  
    printf("len= %d\n",len);

    blockSize=(len+nthreads-1)/nthreads;
    printf("size= %d\n",blockSize);

    for(id = 0; id < nthreads; id++){

        data[id].fp=fp;
        data[id].offset = blockSize;
        data[id].start = id*blockSize+1;

        }
        //LAST THREAD
        data[nthreads-1].start=(nthreads-1)*blockSize+1;

        for(id = 0; id < nthreads; id++)
            pthread_create(&threads[id], NULL, &countFrequency,&data[id]);

    for(id = 0; id < nthreads; id++)
        pthread_join(threads[id],NULL);

    fclose(fp);
    //free(threads);

    //pthread_exit(NULL);

    printf("%d\n",words); 
    return 0;  
}

1 个答案:

答案 0 :(得分:3)

类型转换无法修复错误的代码 - 它只会伪装它或使其更加错误。让我们来看看这些错误:

struct thread_data* td=(struct thread_data)data; /* wrong */

您无法将struct thread_data *投射到struct thread_data,也无法将struct thread_data分配给struct thread_data *。错误的不必要的强制转换是导致错误的唯一原因。

x = pthread_create(&threads[id], NULL, &countFrequency, (void *)data); /* wrong */

其次,您也不能将struct thread_data投射到void * - 您需要一个实际指针,例如data地址

x = pthread_create(&threads[id], NULL, &countFrequency, &data);

也没有强制转换,因为指向数据类型的指针自然会转换为void *。当然,因为data只有一个副本,所有线程都要共享它,并且所有线程都写入它的最后一个值。这不会很顺利 - 你需要每个线程一个struct thread_data

第三,那些警告告诉你你的线程函数有错误的签名:

void *countFrequency(struct thread_data *data) /* wrong */

结合第一点,让所有类型都正确无误,再次不需要强制转换。

void *countFrequency(void *data) {
    struct thread_data* td = data;