C ++ - cURL cmd,stderr管道编程,非常慢

时间:2015-12-18 22:00:50

标签: c++ unix curl pipe openframeworks

注意:我正在使用openFrameworks,但我不认为这是理解我的问题所需的知识。

概述

OSX 10.8.5

XCode 5.1.1

我有一个fork()的程序,子进程运行此cURL cmd:

curl -c cookies.txt -b cookies.txt -k https://www.space-track.org/ajaxauth/login -d 'identity=email@example.com&password=p4$$w0rd&query=https://www.space-track.org/basicspacedata/query/class/tle_latest/ORDINAL/1/EPOCH/%3Enow-30/orderby/OBJECT_ID/format/json' > ~/../myApp/bin/data/myData.json -#

此:

  • 管理Space Track API
  • 的Cookie文件
  • 使用我的Space Track帐户登录
  • 将数据集(约15,000个json对象)下载到“myData.json”
  • 显示进度条######################################################################## 100.0%

子级将进度条管道传输到父级,一次1个字符。父同时将进度条打印到控制台。以下是我的代码的整个fork()部分:

.h (相关的全球变量)

{
    bool forkOnce, isBufferFull;
    int childToParentPipe[2]; // Child to parent pipe. [0] = read (output), [1] = write (input)
    char ch;
    int pid, outCount;
    string buffer; 
}

的.cpp

if(!forkOnce){ //we're in an ∞ loop, so we need to control the flow

    //make pipe
    if( pipe(childToParentPipe) < 0){
        perror("Can't make pipe");
        _exit(1);
    }

    //vars for dealing w piped chars
    buffer = ""; // string that holds 1 line of progress meter
    isBufferFull = false; // is ^ buffer string full?
    outCount = 0; //counts chars coming from pipe (for carriage returns)

    cout << "Forking..." << endl;
    pid = fork();
}
switch(pid){
    case -1:{
        perror("Can't fork");
        _exit(EXIT_FAILURE);
        break;
    }
    case 0:{ // child

        if(!forkOnce){ // do child stuff once

            close(2); // Close current stderr
            dup( childToParentPipe[1]); //Make stderr go to write end of pipe
            close( childToParentPipe[0]); //Close read end of pipe because it's not needed in child process

            //make output file for stdout
            int out = creat("/.../myApp/bin/data/myData.json", S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH);

            if (out == -1){
                perror("Can't make file");
                _exit(EXIT_FAILURE);
            } else { //if file successfully created
                dup2(out,STDOUT_FILENO);// redirect stdout to the file                
                execlp( [the curl command goes here] , NULL); // exec the command
                close(out); // close file
                _exit(0);

            }
        }
        break;
    }
    default:{ // parent

        close(childToParentPipe[1]);

        //while num bytes piped out is 1 (meaning we haven't reached EOF yet)
        if( read(childToParentPipe[0], &ch, 1) == 1) {

            if(outCount == 80){
                isBufferFull = true;
                outCount = 0;
            }

            if(!isBufferFull){
                buffer += ch;
            } else {
                char* bufferChars = strdup(buffer.c_str());
                bufferChars[outCount] = ch;
                buffer = bufferChars;
            }

            cout << buffer << endl;

            outCount++;
        }

        // wait for child process
        int status = 0;
        waitpid(-1, &status, WNOHANG);
        //printf("child status:%d\n",status); // "0"

        signal(SIGCHLD,signalHandler); //signalHandler = func that finalizes this process

        break;
    }
}

forkOnce = true;

问题

与在终端中运行cURL下载相比,我的应用程序中的cURL下载非常慢:

enter image description here

...有时候,在我的应用中,这个错误会在下载过程中打印到控制台几次:

2015-12-18 15:44:16.595 forkTest2Debug[22858:5f1f] -[NSPersistentUIManager writeWindowSnapshot:length:width:height:bytesPerRow:toFile:inDirectory:encryptingWithKey:uuid:checksum:isUserWaitingImpatientlyForThisThingToFinish:]: 0 == ftruncate(fd, finalFileSize) failed on line 2770: Bad file descriptor

无法弄清楚为什么我会收到Bad file descriptor错误。我似乎无法找到关于isUserWaitingImpatientlyForThisThingToFinishftruncate的任何官方文档,在此上下文中告知我。

我的应用程序以60fps的速度运行,因此管道的输出以此速率输出字符。这可能会填满我的管道,因为字符比它们被送出的速度快得多吗?

造成这种缓慢的原因是什么?如何解决?

1 个答案:

答案 0 :(得分:1)

以下是popen()的简单示例,这可能会帮助您按照自己的方式进行操作

#include <stdio.h>

int main(int argc, char *argv[])
{
    char output[100];
    char command[100];
    ssize_t length;
    FILE *pipe;
    if (argc < 2)
        return -1; // Missing command line parameters
    if (snprintf(command, sizeof(command), "curl %s", argv[1]) >= sizeof(command))
        return -1; // URL too long
    pipe = popen(command, "r");
    if (pipe == NULL)
        return -1; // Execution failure
    while ((length = fread(output, 1, sizeof(output), pipe)) > 0)
        fwrite(output, 1, length, stdout);
    pclose(pipe);
    return 0;
}

像这样编译和测试

$ gcc -Wall -Werror -g3 -O0 SOURCE.c -o _curl_test
$ ./_curl_test http://stackoverflow.com