C ++ boost asio Windows文件句柄async_read_until无限循环 - 没有eof

时间：2013-07-18 14:34:15

标签： c++ boost file-io asynchronous boost-asio

我在VS2010上使用boost 1.50，使用 Windows文件HANDLE 进行读取（与使用套接字的asio相比，这似乎相对不常见）。

问题

  

handle_read回调到达第8行并返回第一位，并附加第1行;进一步的回调再次从第2行循环，令人作呕：

  • 打开一个简短的文本文件（如下）
  • 获得第1行到第7行的正确内容的预期handle_read回调
  • 下一个回调有一个长于预期的字节读取 length参数
  • 虽然未使用length，但 getline从asio流缓冲区中提取相应较长的行
  • 从输入文件中提取内容切换到中间行以重复第一行
  • 进一步handle_read回调回收第2行到第7行，然后发生“长混合”行问题
  • ad nauseum

输入

LINE 1 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789
LINE 2 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789
...3--E similarly...
LINE F abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789

输出

这是前15行输出（它会永远持续）：

line #1, length 70, getline() [69] 'LINE 1 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #2, length 70, getline() [69] 'LINE 2 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
...line #3 through #6 are fine too...
line #7, length 70, getline() [69] 'LINE 7 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #8, length 92, getline() [91] 'LINE 8 abcdefghijklmnoLINE 1 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #9, length 70, getline() [69] 'LINE 2 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
...line #10 through #13 are fine...
line #14, length 70, getline() [69] 'LINE 7 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #15, length 92, getline() [91] 'LINE 8 abcdefghijklmnoLINE 1 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
...

请注意输出行＃8和＃15是输入LINE 8和LINE 1的混合。

代码

#include "stdafx.h"

#include <cassert>
#include <iostream>
#include <string>

#include <boost/asio.hpp>
#include <boost/bind.hpp>

#include <Windows.h>
#include <WinBase.h>

class AsyncReader
{
  public:
    AsyncReader(boost::asio::io_service& io_service, HANDLE handle)
      : io_service_(io_service),
        input_buffer(/*size*/ 8192),
        input_handle(io_service, handle)
    {
        start_read();
    }

    void start_read()
    {
        boost::asio::async_read_until(input_handle, input_buffer, '\n',
            boost::bind(&AsyncReader::handle_read, this,
                boost::asio::placeholders::error,
                boost::asio::placeholders::bytes_transferred));
    }

    void handle_read(const boost::system::error_code& error, std::size_t length);
    // void handle_write(const boost::system::error_code& error);

  private:
    boost::asio::io_service& io_service_;
    boost::asio::streambuf input_buffer;
    boost::asio::windows::stream_handle input_handle;
};

void AsyncReader::handle_read(const boost::system::error_code& error, std::size_t length)
{
    if (!error)
    {
        static int count = 0;
        ++count;

        // method 1: (same problem)
        // const char* pStart = boost::asio::buffer_cast<const char*>(input_buffer.data());
        // std::string s(pStart, length);
        // input_buffer.consume(length);

        // method 2:
        std::istream is(&input_buffer);
        std::string s;
        assert(std::getline(is, s));

        std::cout << "line #" << count << ", length " << length << ", getline() [" << s.size() << "] '" << s << "'\n";

        start_read();
    }
    else if (error == boost::asio::error::not_found)
        std::cerr << "Did not receive ending character!\n";
    else
        std::cerr << "Misc error during read!\n";
}
int _tmain(int argc, _TCHAR* argv[])
{
    boost::asio::io_service io_service;

    HANDLE handle = ::CreateFile(TEXT("c:/temp/input.txt"),
                                 GENERIC_READ,
                                 0, // share mode
                                 NULL, // security attribute: NULL = default
                                 OPEN_EXISTING, // creation disposition
                                 FILE_FLAG_OVERLAPPED,
                                 NULL // template file
                                );

    AsyncReader obj(io_service, handle);

    io_service.run();

    std::cout << "Normal termination\n";
    getchar();
    return 0;
}

我的想法

  • 可能是CreateFile选项中的某些内容 - 在我切换到FILE_FLAG_OVERLAPPED之前它根本不起作用 - 不确定是否有其他要求甚至不显示为错误。 <？/ LI>
  • 我已经尝试input_buffer.commit甚至.consume - 不确定是否有类似我应该做的事情，即使我能找到的所有示例代码（对于套接字）都建议{{ 1}}照顾...
  • 恼怒/我想念Linux ....

3 个答案:

答案 0 :(得分：5)

stream_handle将始终在偏移零处读取。我认为它适用于套接字句柄，对常规文件没用。

如果streambuf尚未包含换行符，则调用async_read_until（）将获得512个字节。第一个调用读取超过7行。当提取七行时，剩余字符（“LINE 8 abcdefghijklmno”）没有换行符，并且（相同）附加512字节。

要解决这个问题，我建议使用random_access_handle。您必须手动跟踪文件位置，并将async_read_until替换为async_read_at。

class AsyncReader
{
  ...
  void start_read()
  {
    async_read_at(input_handle, input_offset, input_buffer, ...);
  }
private:
  boost::asio::windows::random_access_handle input_handle;
  boost::uint64_t input_offset;
};

void AsyncReader::handle_read(const boost::system::error_code& error,
                              std::size_t length)
{
  input_offset += length;
  if (!error || error == boost::asio::error::eof)
  {
    ...

答案 1 :(得分：5)

This邮件列表帖子描述了同样的问题。虽然带有CreateFile的{​​{1}}允许异步I / O，但它不会将其建立为Boost.Asio上下文中的流。对于流，Boost.Asio将FILE_FLAG_OVERLAPPED实现为read_some，偏移量始终为read_some_at。这是问题的根源，正如ReadFile()文档所述：

  

对于支持字节偏移的文件，必须指定一个字节偏移量，以便从文件开始读取。

适应类型要求

Boost.Asio是非常通用的，通常需要参数来满足某种类型的要求，而不是特定的类型。因此，通常可以调整I / O对象或其服务以获得所需的行为。首先，必须确定适应的接口需要支持的内容。在这种情况下，async_read_until接受满足AsyncReadStream类型要求的任何类型。 0的要求非常基本，需要AsyncReadStream成员函数。

由于需要在整个组合void async_read_some(MutableBufferSequence, ReadHandler)操作中跟踪偏移值，因此可以引入满足ReadHandler要求的简单类型，它将包装应用程序的ReadHandler，并相应地更新偏移量。

async_read_until

将通过ADL找到namespace detail { /// @brief Handler to wrap asynchronous read_some_at operations. template <typename Handler> class read_some_offset_handler { public: read_some_offset_handler(Handler handler, boost::uint64_t& offset) : handler_(handler), offset_(offset) {} void operator()( const boost::system::error_code& error, std::size_t bytes_transferred) { offset_ += bytes_transferred; // If bytes were transferred, then set the error code as success. // EOF will be detected on next read. This is to account for // the read_until algorithm behavior. const boost::system::error_code result_ec = (error && bytes_transferred) ? make_error_code(boost::system::errc::success) : error; handler_(result_ec, bytes_transferred); } //private: Handler handler_; boost::uint64_t& offset_; }; /// @brief Hook that allows the wrapped handler to be invoked /// within specific context. This is critical to support /// composed operations being invoked within a strand. template <typename Function, typename Handler> void asio_handler_invoke( Function function, detail::read_some_offset_handler<Handler>* handler) { boost_asio_handler_invoke_helpers::invoke( function, handler->handler_); } } // namespace detail 挂钩，以支持在适当的上下文中调用用户处理程序。当在strand内调用组合操作时，这对于踏板安全至关重要。有关组合操作和线索的更多详细信息，请参阅this answer。

以下课程将调整boost::asio::windows::random_access_handle以满足asio_handler_invoke的类型要求。

AsyncReadStream

或者，可以为boost::asio::windows::basic_stream_handle提供符合​​StreamHandleService类型要求的自定义类型，并根据/// @brief Adapts AsyncRandomAccessReadDevice to support AsyncReadStream. template <typename AsyncRandomAccessReadDevice> class basic_adapted_stream : public AsyncRandomAccessReadDevice { public: basic_adapted_stream( boost::asio::io_service& io_service, HANDLE handle ) : AsyncRandomAccessReadDevice(io_service, handle), offset_(0) {} template<typename MutableBufferSequence, typename ReadHandler> void async_read_some( const MutableBufferSequence& buffers, ReadHandler handler) { async_read_at(*this, offset_, buffers, detail::read_some_offset_handler<ReadHandler>(handler, offset_)); } private: boost::uint64_t offset_; }; 实施async_read_some。

async_read_some_at

我在示例代码中选择了简单性，但多个I / O对象将使用相同的服务。因此，/// @brief Service that implements async_read_some with async_read_some_at. class offset_stream_handle_service : public boost::asio::windows::stream_handle_service { private: // The type of the platform-specific implementation. typedef boost::asio::detail::win_iocp_handle_service service_impl_type; public: /// The unique service identifier. static boost::asio::io_service::id id; /// Construct a new stream handle service for the specified io_service. explicit offset_stream_handle_service(boost::asio::io_service& io_service) : boost::asio::windows::stream_handle_service(io_service), service_impl_(io_service), offset_(0) {} /// Start an asynchronous read. template <typename MutableBufferSequence, typename ReadHandler> void async_read_some( implementation_type& impl, const MutableBufferSequence& buffers, ReadHandler handler) { // Implement async_read_some in terms of async_read_some_at. The provided // ReadHandler will be hoisted in an internal handler so that offset_ can // be properly updated. service_impl_.async_read_some_at(impl, offset_, buffers, detail::read_some_offset_handler<ReadHandler>(handler, offset_)); } private: // The platform-specific implementation. service_impl_type service_impl_; boost::uint64_t offset_; }; boost::asio::io_service::id offset_stream_handle_service::id; 需要管理每个处理程序的偏移量，以便在多个I / O对象使用该服务时正常运行。

要使用适应的类型，请将offset_stream_handle_service成员变量修改为AsyncReader::input_handle（适应的I / O对象）或basic_adapted_stream<boost::asio::windows::random_access_handle>（适应的服务）。

实施例

以下是基于原始代码的完整示例，仅修改boost::asio::windows::basic_stream_handle<offset_stream_handle_service>的类型：

AsyncReader::input_handler

使用原始问题的输入时会产生以下输出：

line #1, length 70, getline() [69] 'LINE 1 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #2, length 70, getline() [69] 'LINE 2 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #3, length 70, getline() [69] 'LINE 3 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #4, length 70, getline() [69] 'LINE 4 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #5, length 70, getline() [69] 'LINE 5 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #6, length 70, getline() [69] 'LINE 6 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #7, length 70, getline() [69] 'LINE 7 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #8, length 70, getline() [69] 'LINE 8 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #9, length 70, getline() [69] 'LINE 9 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #10, length 70, getline() [69] 'LINE 0 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #11, length 70, getline() [69] 'LINE A abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #12, length 70, getline() [69] 'LINE B abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #13, length 70, getline() [69] 'LINE C abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #14, length 70, getline() [69] 'LINE D abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #15, length 70, getline() [69] 'LINE E abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
Misc error during read!
Normal termination

我的输入文件在LINE F的末尾没有#include "stdafx.h" #include <cassert> #include <iostream> #include <string> #include <boost/asio.hpp> #include <boost/bind.hpp> #include <Windows.h> #include <WinBase.h> namespace detail { /// @brief Handler to wrap asynchronous read_some_at operations. template <typename Handler> class read_some_offset_handler { public: read_some_offset_handler(Handler handler, boost::uint64_t& offset) : handler_(handler), offset_(offset) {} void operator()( const boost::system::error_code& error, std::size_t bytes_transferred) { offset_ += bytes_transferred; // If bytes were transferred, then set the error code as success. // EOF will be detected on next read. This is to account for // the read_until algorithm behavior. const boost::system::error_code result_ec = (error && bytes_transferred) ? make_error_code(boost::system::errc::success) : error; handler_(result_ec, bytes_transferred); } //private: Handler handler_; boost::uint64_t& offset_; }; /// @brief Hook that allows the wrapped handler to be invoked /// within specific context. This is critical to support /// composed operations being invoked within a strand. template <typename Function, typename Handler> void asio_handler_invoke( Function function, detail::read_some_offset_handler<Handler>* handler) { boost_asio_handler_invoke_helpers::invoke( function, handler->handler_); } } // namespace detail /// @brief Adapts AsyncRandomAccessReadDevice to support AsyncReadStream. template <typename AsyncRandomAccessReadDevice> class basic_adapted_stream : public AsyncRandomAccessReadDevice { public: basic_adapted_stream( boost::asio::io_service& io_service, HANDLE handle ) : AsyncRandomAccessReadDevice(io_service, handle), offset_(0) {} template<typename MutableBufferSequence, typename ReadHandler> void async_read_some( const MutableBufferSequence& buffers, ReadHandler handler) { async_read_at(*this, offset_, buffers, detail::read_some_offset_handler<ReadHandler>(handler, offset_)); } private: boost::uint64_t offset_; }; /// @brief Service that implements async_read_some with async_read_some_at. class offset_stream_handle_service : public boost::asio::windows::stream_handle_service { private: // The type of the platform-specific implementation. typedef boost::asio::detail::win_iocp_handle_service service_impl_type; public: /// The unique service identifier. static boost::asio::io_service::id id; /// Construct a new stream handle service for the specified io_service. explicit offset_stream_handle_service(boost::asio::io_service& io_service) : boost::asio::windows::stream_handle_service(io_service), service_impl_(io_service), offset_(0) {} /// Start an asynchronous read. template <typename MutableBufferSequence, typename ReadHandler> void async_read_some( implementation_type& impl, const MutableBufferSequence& buffers, ReadHandler handler) { // Implement async_read_some in terms of async_read_some_at. The provided // ReadHandler will be hoisted in an internal handler so that offset_ can // be properly updated. service_impl_.async_read_some_at(impl, offset_, buffers, detail::read_some_offset_handler<ReadHandler>(handler, offset_)); } private: // The platform-specific implementation. service_impl_type service_impl_; boost::uint64_t offset_; }; boost::asio::io_service::id offset_stream_handle_service::id; #ifndef ADAPT_IO_SERVICE typedef basic_adapted_stream< boost::asio::windows::random_access_handle> adapted_stream; #else typedef boost::asio::windows::basic_stream_handle< offset_stream_handle_service> adapted_stream; #endif class AsyncReader { public: AsyncReader(boost::asio::io_service& io_service, HANDLE handle) : io_service_(io_service), input_buffer(/*size*/ 8192), input_handle(io_service, handle) { start_read(); } void start_read() { boost::asio::async_read_until(input_handle, input_buffer, '\n', boost::bind(&AsyncReader::handle_read, this, boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred)); } void handle_read(const boost::system::error_code& error, std::size_t length); // void handle_write(const boost::system::error_code& error); private: boost::asio::io_service& io_service_; boost::asio::streambuf input_buffer; adapted_stream input_handle; }; void AsyncReader::handle_read(const boost::system::error_code& error, std::size_t length) { if (!error) { static int count = 0; ++count; // method 1: (same problem) // const char* pStart = boost::asio::buffer_cast<const char*>(input_buffer.data()); // std::string s(pStart, length); // input_buffer.consume(length); // method 2: std::istream is(&input_buffer); std::string s; assert(std::getline(is, s)); std::cout << "line #" << count << ", length " << length << ", getline() [" << s.size() << "] '" << s << "'\n"; start_read(); } else if (error == boost::asio::error::not_found) std::cerr << "Did not receive ending character!\n"; else std::cerr << "Misc error during read!\n"; } int _tmain(int argc, _TCHAR* argv[]) { boost::asio::io_service io_service; HANDLE handle = ::CreateFile(TEXT("c:/temp/input.txt"), GENERIC_READ, 0, // share mode NULL, // security attribute: NULL = default OPEN_EXISTING, // creation disposition FILE_FLAG_OVERLAPPED, NULL // template file ); AsyncReader obj(io_service, handle); io_service.run(); std::cout << "Normal termination\n"; getchar(); return 0; } 个字符。因此，\n被调用，错误为AsyncReader::handle_read()和boost::asio::error::eof的内容包含LINE F.修改最终的else案例后打印更多信息：

input_buffer

我得到以下输出：

line #1, length 70, getline() [69] 'LINE 1 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #2, length 70, getline() [69] 'LINE 2 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #3, length 70, getline() [69] 'LINE 3 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #4, length 70, getline() [69] 'LINE 4 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #5, length 70, getline() [69] 'LINE 5 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #6, length 70, getline() [69] 'LINE 6 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #7, length 70, getline() [69] 'LINE 7 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #8, length 70, getline() [69] 'LINE 8 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #9, length 70, getline() [69] 'LINE 9 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #10, length 70, getline() [69] 'LINE 0 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #11, length 70, getline() [69] 'LINE A abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #12, length 70, getline() [69] 'LINE B abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #13, length 70, getline() [69] 'LINE C abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #14, length 70, getline() [69] 'LINE D abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
line #15, length 70, getline() [69] 'LINE E abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
Error: End of file
stream contents: 'LINE F abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
Normal termination

答案 2 :(得分：2)

一个选项是在调用用户的ReadHandler之前将fseek()文件移到下一个位置。然后，async_read_some()可以实现为async_read_at(ftell())。

AsyncReader可以使用ReadUntilHandle而不是stream_handle：

class ReadUntilHandle : public boost::asio::windows::random_access_handle
{
  public:
    ReadUntilHandle(boost::asio::io_service& ios, HANDLE handle)
      : boost::asio::windows::random_access_handle(ios, handle)
    {}

    template <typename MutableBufferSequence, typename Handler>
    void async_read_some(const MutableBufferSequence& buffers, Handler& handler)
    {
      LARGE_INTEGER offset;
      offset.QuadPart = 0;
      if (::SetFilePointerEx(native_handle(), offset, &offset, FILE_CURRENT)) {
        async_read_some_at(offset.QuadPart, buffers,
                           std::bind(&on_read_complete<Handler>, handler,
                                     native_handle(), std::ref(get_io_service()),
                                     std::placeholders::_1, std::placeholders::_2));
      } else {
        boost::system::error_code error(::GetLastError(), boost::asio::error::get_system_category());
        get_io_service().post(boost::asio::detail::bind_handler(handler, error, 0));
      }
    }
  private:
    template <typename Handler> static void
    on_read_complete(Handler& handler, HANDLE native_handle, boost::asio::io_service& ios,
                   boost::system::error_code error, std::size_t length)
    {
      if (0 != length) { // update file position
        LARGE_INTEGER offset;
        offset.QuadPart = length;
        if (!::SetFilePointerEx(native_handle, offset, NULL, FILE_CURRENT) && !error) {
          error.assign(::GetLastError(),  boost::asio::error::get_system_category());
        }
      }
      ios.dispatch(boost::asio::detail::bind_handler(handler, error, length));
    }
};
相关问题
最新问题