Boost Asio和Beast mulitpart / form-data从streambuf保存二进制文件

时间:2018-11-24 07:29:53

标签: c++ boost-asio boost-beast

我的结果看起来像...

POST /post HTTP/1.1
Host: localhost:3003
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0)    
Gecko/20100101 Firefox/62.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://localhost:3003/profile
Content-type: multipart/form-data
Content-Length: 14708
Cookie: mycookie=7bdbed41954cd4133a172acb92988e58
Connection: keep-alive

-----------------------------4636945214860352321751082034
...
binary characters...
...
-----------------------------4636945214860352321751082034

购自

boost::asio::async_read(
    socket_,
    strmbuffer_,
    boost::asio::transfer_exactly(bytes_to_transfer),
    strand_.wrap(
    [this, self](boost::system::error_code ec, std::size_t bytes_transferred)
    {
        std::stringstream ss;
        ss << buffer_data;  // from socket_.async_read_some()
        ss << &strmbuffer_; // now stringstream contains everything

        // the character routine which writes the above
        // and which i use for output...

        std::string output_file = "../../upload/test.png";
        std::ofstream outfile(output_file);
        char c;
        unsigned bl = boundary.length();
        bool endfile = false;
        unsigned bufsize = 512;
        if(outfile){
            char buffer[bufsize];
            while(!endfile){
                // here looks like below
                // to find and pass the first boundary
            }   // then stream continues...
            while(!endfile){
                unsigned j = 0;
                unsigned k;
                memset(buffer, 0, bufsize); // redundant
                while(j < bufsize && ss.get(c) && !endfile){
                    buffer[j] = c;
                    k = 0;
                    while(boundary[bl - 1 - k] == buffer[j - k]){
                        if(k >= bl - 1){
                            endfile = true;
                            break;
                        }
                        k++;
                     }
                     j++;
                 }
                 outfile.write(buffer, j);
                 j = 0;
             }
         }
    }
);

...基本上。因此,接收

socket_.async_read_some()

给我一​​个

boost::array<char, 8192> buffer_;

这给了我http请求信息。但是在multipart / form-data的情况下,它读取通过了第一个边界,这意味着下一个read()看不到它。啊! (async_read_until()也是如此。)因此,在

boost::asio::async_read()

我转换了

boost::asio::streambuf strmbuffer_;

到字符串流,并将它们添加到上面的std :: cout结果中。

我完全不相信我应该使用stringstream。但是上面的例程(使用stringstream)在Boost :: Beast中可以正常工作。它不在Asio中。不幸的是,我相信在Beast中收到http请求的string_body类型的大小限制为1兆。不知道该如何更改。

我在任何地方都找不到关于这个主题的东西。也许,信息太危险了。如果他们告诉我,他们将不得不杀死我。我应该在Asio中使用什么将二进制数据写入磁盘?

2 个答案:

答案 0 :(得分:1)

默认情况下,HTTP请求解析器以1兆字节的限制开始。这是为了防止客户端发送大量或无休止的正文数据时发生资源耗尽攻击。您可以通过以所需的最大值调用parser::body_limit来轻松更改此限制。在文档中对此进行了描述:

https://www.boost.org/doc/libs/1_68_0/libs/beast/doc/html/beast/ref/boost__beast__http__parser/body_limit.html https://www.boost.org/doc/libs/1_68_0/libs/beast/doc/html/beast/using_http/buffer_oriented_parsing.html

为了调整解析器参数,例如正文限制(或标头限制),您将需要使用“解析器流操作”界面。解释如下:

https://www.boost.org/doc/libs/1_68_0/libs/beast/doc/html/beast/using_http/parser_stream_operations.html

答案 1 :(得分:0)

我将发布自己的部分解决方案。它可在14.2kb png上正常工作。否则,除非调整如下所示的“魔术”数字,否则可能会导致段错误。

我正在使用Boost Asio HTTP Server示例C ++ 11。

在connection.hpp中,更改...

//boost::array<char, 8192> buffer_;
boost::array<char, 512> buffer_;

还添加...

boost::asio::streambuf strmbuffer_;

在connection.cpp中,这是connection :: handle_read()...

void connection::handle_read(const boost::system::error_code& e, std::size_t bytes_transferred)
{
    if (!e)
    {
        //*** buffer_.data() for this file ***
        //
        //POST /post HTTP/1.1
        //Host: localhost:3003
        //User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0
        //Accept: */*
        //Accept-Language: en-US,en;q=0.5
        //Accept-Encoding: gzip, deflate
        //Referer: http://localhost:3003/profile
        //Content-type: multipart/form-data
        //Content-Length: 14710
        //Cookie: knowstoryadmin=7bdbed41954cd4133a172acb92988e58
        //Connection: keep-alive
        //
        //-----------------------------14071968205478138611648202646
        //Content-Disposition: form-data; name="admin_profile_image_load"; filename="tlc-logo.png"
        //Content-Type: image/png
        //
        //�PNG
        //▒

        std::stringstream strm1;
        std::string buffer_data = buffer_.data();
        strm1 << buffer_data;

        std::string method;
        std::smatch match_method;
        std::regex regex_method ("\\b([^ ]*)( )([^ ]*)( HTTP/1.1)([^ ]*)");

        std::string content_type;
        std::smatch match_content_type;
        std::regex regex_content_type ("\\b(Content-type: )([^ ]*)");

        std::string line;

        while (std::getline(strm1, line)) {
            if (std::regex_search(line, match_method, regex_method)) {
                method = match_method[0];
                method = method.substr(0, method.find(' '));
                boost::trim(method);
                //std::cout << method << std::endl;
            }
            if (std::regex_search(line, match_content_type, regex_content_type))     {
                content_type = match_content_type[0];
                boost::erase_all(content_type, "Content-type:");
                boost::trim(content_type);
                //std::cout << content_type << std::endl;
            }
        }

        if (method == "POST") {
            if (content_type == "multipart/form-data") {

                std::string content_length;
                std::smatch match_content_length;
                std::regex regex_content_length ("\\b(Content-Length: )([^ ]*)");

                std::string filename;
                std::smatch match_filename;
                std::regex regex_filename ("\\b(filename)([^ ]*)");

                std::string action;
                std::smatch match_action;
                std::regex regex_action ("\\b(name)([^ ]*)");

                std::string boundary;
                std::smatch match_boundary;
                std::regex regex_boundary ("([-]{10,}[0-9]{10,})");

                std::string line;

                strm1.clear();
                strm1 << buffer_data;
                while (std::getline(strm1, line)) {
                    if (std::regex_search(line, match_content_length, regex_content_length)) {
                        //Content-Length: 14710
                        content_length = match_content_length[0];
                        boost::erase_all(content_length, "Content-Length:");
                        boost::trim(content_length);
                        //std::cout << content_length << std::endl;
                    }
                    if (std::regex_search(line, match_filename, regex_filename)) {
                        filename = match_filename[0];
                        boost::erase_all(filename, "\"");
                        boost::erase_all(action, ";");
                        boost::erase_all(filename, "filename=");
                        std::size_t found = filename.find_last_of(".");
                        std::size_t len = filename.length();
                        std::string mime = filename.substr(found, len);
                        boost::trim(filename);
                        //std::cout << filename << std::endl;
                        //std::cout << mime << std::endl;
                    }
                    if (std::regex_search(line, match_action, regex_action)) {
                        action = match_action[0];
                        boost::erase_all(action, "\"");
                        boost::erase_all(action, ";");
                        boost::erase_all(action, "name=");
                        boost::trim(action);
                        //std::cout << action << std::endl;
                    }
                    if (std::regex_search(line, match_boundary, regex_boundary)) {
                        boundary = match_boundary[0];
                        boost::trim(boundary);
                        //std::cout << boundary << std::endl;
                    }
                }

                //pubseekpos works as expected, but useless here
                //strmbuffer_.pubseekpos(bytes_transferred);

                //content length minus bytes_transfered does NOT yield
                //the right result. The number, 392, is the 'magic' number
                //adjustment for this file size, approx 14.2kb, that i found
                //by trial and error.
                //Adjusting the magic number is necessary for every image size
                //in order to avoid a segfault.
                //bytes_transferred, for each read(), is the only 'reliable'
                //number with which to work, as far as i know.
                //If there is a brainier way of calculating this,
                //i don't care, anymore.
                int n_content_length = std::stoi(content_length);
                int transfer = n_content_length - bytes_transferred + 392;
                auto self(shared_from_this());
                boost::asio::async_read(
                    socket_,
                    strmbuffer_,
                    boost::asio::transfer_exactly(transfer),
                    strand_.wrap(
                        [this, self, boundary](boost::system::error_code ec, std::size_t bytes_transferred)
                        {
                            std::stringstream strm2;
                            strm2 << &strmbuffer_;
                            std::string line;
                            unsigned bufsize = 512;
                            while (std::getline(strm2, line))
                            {
                                if(line.length() == 1){
                                    std::string output_file = "../../upload/test.png";
                                    std::ofstream outfile(output_file);
                                    char c;
                                    unsigned bl = boundary.length();
                                    bool endfile = false;
                                    if(outfile){
                                        char buffer[bufsize];
                                        while(!endfile){
                                            unsigned j = 0;
                                            unsigned k;
                                            while(j < bufsize && strm2.get(c) && !endfile){
                                                buffer[j] = c;
                                                k = 0;
                                                while(boundary[bl - 1 - k] == buffer[j - k]){
                                                    if(k >= bl - 1){
                                                        endfile = true;
                                                        break;
                                                    }
                                                    k++;
                                                }
                                                j++;
                                            }
                                            outfile.write(buffer, j);
                                            j = 0;
                                        };
                                        outfile.close();
                                        std::cout << "outfile close" << std::endl;
                                        break;
                                    }
                                }
                            }
                        }
                    )
                );
            }
            else {
                // POST AJAX
                std::cout << "connection " << method << std::endl;
            }
        }
        else {
            boost::tribool result;
            boost::tie(result, boost::tuples::ignore) = request_parser_.parse(
                request_, buffer_.data(), buffer_.data() + bytes_transferred);

            if (result)
            {
                request_handler_.handle_request(
                    request_,
                    reply_);

                boost::asio::async_write(
                    socket_,
                    reply_.to_buffers(),
                    strand_.wrap(
                        boost::bind(
                            &connection::handle_write,
                            shared_from_this(),
                            boost::asio::placeholders::error)
                    ));
            }
            else if (!result)
            {
                reply_ = reply::stock_reply(reply::bad_request);
                boost::asio::async_write(
                    socket_,
                    reply_.to_buffers(),
                    strand_.wrap(
                        boost::bind(&connection::handle_write, shared_from_this(),
                        boost::asio::placeholders::error)));
            }
            else
            {
                socket_.async_read_some(
                    boost::asio::buffer(buffer_),
                    strand_.wrap(
                        boost::bind(
                            &connection::handle_read,
                            shared_from_this(),
                            boost::asio::placeholders::error,
                            boost::asio::placeholders::bytes_transferred)));
            }
        }
    }
}

对于此Asio示例,此部分解决方案相当“友好”。 request_handler类保持不变并提供GET。

在connection :: start()中,我尝试了async_read。但是结果甚至更难以预测。这意味着我必须附加两个字符串,一个由strmbuffer1_构成,另一个由strmbuffer2_构成,以便在文件输出循环中一起构造(破解)流。无论我的准备工作在cout <<中多么精确,都会丢失大约500或更多的字节,从而写入不完整的图像文件。

在内存中的某个地方存在并且必须存在我上传的二进制文件及其句柄所需的完整信息。但是从Asio类中提取它似乎很棘手。

这就是我与Boost Asio的距离。就我所知。对于我想要的,我的答案不在这里。