如何通过使用boost日志来防止内存增长?

时间:2018-05-18 10:53:14

标签: c++ memory boost boost-log boost-logging

在此link中运行boost log的示例代码将继续增加内存使用量。我认为这是一次内存泄漏。

boost lib版本:1.64.0

This is the piece of code I'm checking more carefully
/*
 *          Copyright Andrey Semashev 2007 - 2013.
 * Distributed under the Boost Software License, Version 1.0.
 *    (See accompanying file LICENSE_1_0.txt or copy at
 *          http://www.boost.org/LICENSE_1_0.txt)
 */
#include <stdexcept>
#include <stdio.h>
#include <string>
#include <iostream>


#include <fstream>
#include <boost/ref.hpp>
#include <boost/bind.hpp>
#include <boost/thread/thread.hpp>
#include <boost/thread/barrier.hpp>

#include <boost/smart_ptr/shared_ptr.hpp>
#include <boost/date_time/posix_time/posix_time.hpp>

#include <boost/log/common.hpp>
#include <boost/log/expressions.hpp>
#include <boost/log/attributes.hpp>
#include <boost/log/sources/logger.hpp>
#include <boost/log/sinks.hpp>
//#include <boost/log/sinks/sync_frontend.hpp>
//#include <boost/log/sinks/text_file_backend.hpp>

namespace logging = boost::log;
namespace attrs = boost::log::attributes;
namespace src = boost::log::sources;
namespace sinks = boost::log::sinks;
namespace expr = boost::log::expressions;
namespace keywords = boost::log::keywords;

using boost::shared_ptr;

enum
{
    LOG_RECORDS_TO_WRITE = 10000,
    THREAD_COUNT = 19
};


//BOOST_LOG_INLINE_GLOBAL_LOGGER_DEFAULT(tmp_logger, src::logger_mt)
BOOST_LOG_INLINE_GLOBAL_LOGGER_DEFAULT(test_lg, src::logger_mt)

//! This function is executed in multiple threads
void thread_fun(boost::barrier& bar)
{
    // Wait until all threads are created
    bar.wait();

    // Now, do some logging
    // modify code start
    while(1) {
        BOOST_LOG(test_lg::get()) << "Log record Log record Log record Log record Log record Log record Log record Log record ";
    }
    // modify code end
}

int main(int, char*[])
{
    try
    {
        // Open a rotating text file
        shared_ptr< std::ostream > strm(new std::ofstream("test.log"));

        if (!strm->good()) {
            throw std::runtime_error("Failed to open a text log file");
        }

        // Create a text file sink
        shared_ptr< sinks::synchronous_sink< sinks::text_ostream_backend > > sink(
                new sinks::synchronous_sink< sinks::text_ostream_backend >
                );

        sink->locked_backend()->add_stream(strm);

        sink->set_formatter(
             expr::format("%1%: [%2%] [%3%] - %4%")
             % expr::attr< unsigned int >("RecordID")
             % expr::attr< boost::posix_time::ptime >("TimeStamp")
             % expr::attr< attrs::current_thread_id::value_type >("ThreadID")
             % expr::smessage
            );

        // Add it to the core
        logging::core::get()->add_sink(sink);

        // Add some attributes too
        logging::core::get()->add_global_attribute("TimeStamp", attrs::local_clock());
        logging::core::get()->add_global_attribute("RecordID", attrs::counter< unsigned int >());
        logging::core::get()->add_global_attribute("ThreadID", attrs::current_thread_id());

        // Create logging threads
        boost::barrier bar(THREAD_COUNT);
        boost::thread_group threads;

        for (unsigned int i = 0; i < THREAD_COUNT; ++i)
            threads.create_thread(boost::bind(&thread_fun, boost::ref(bar)));

        // Wait until all action ends
        threads.join_all();
    }
    catch (std::exception& e)
    {
        std::cout << "FAILURE: " << e.what() << std::endl;
        return 1;
    }
    return 0;
}

随着while循环的继续,可用大小的数量继续减少。 开始时(18:50:56)的免费容量为28,331KB,55分钟后(19:45:14)的空闲容量为13,449KB,差异约为15MB

编辑:/ proc / meminfo“MemFree”行的结果

它继续缩小,导致oom,最终导致内核恐慌并重新启动。

如何解决此问题?我期待听到您的意见和建议

1 个答案:

答案 0 :(得分:1)

  

使用“cat / proc / meminfo”

中的“MemFree”值完成内存验证

这不是衡量流程内存使用情况的可靠方法。

相反,在内存分析器下运行代码,如valgrind --tool = massif。

我运行了你的修改过的例子,但有20个线程,在Massif下10分钟,它清楚地表明没有增长:

enter image description here

注意:

  1. 您可能有另一个进程泄漏内存
  2. 你可能在tmpfs中拥有(隐式)日志文件,使可用内存缩小
  3. 旧版本的boost可能存在错误(我使用boost 1.67测试)