Question

我想在硬盘上存储一个巨大的std::list<my_big_struct>（但在这个例子中很少）。

此外，我还希望将许多std::list<std::list<my_big_struct>::const_iterator>存储到硬盘上。

如何在没有深层复制的情况下存档（空间不足）？

以my_big_struct为简单int。

的简短示例

#include <list>
#include <fstream>
#include <boost/serialization/serialization.hpp>
#include <boost/serialization/list.hpp>
#include <boost/archive/binary_oarchive.hpp>

typedef int my_big_struct;

int main()
{
    std::list<my_big_struct> a = {3,2,7,4,9};
    std::list<std::list<my_big_struct>::const_iterator> aa;
    std::list<std::list<my_big_struct>::const_iterator> ab;

    std::list<my_big_struct>::const_iterator find = a.begin();
    aa.emplace(aa.begin(), find++);
    ab.emplace(ab.begin(), find);

    {
        std::ofstream f("test_a.bin", std::ios::binary);
        boost::archive::binary_oarchive o(f);
        o << a; // OK
    }

    // how to store aa and ab onto harddisk without hardcopy ???
}

Answer 1

我不确定设计（这听起来像保留易失性记忆关系的设计气味）。

如果您确信您的设计合理，我建议使用managed_shared_memory（这是Boost Interprocess）。

然后，您可以依赖std::list¹。

的迭代器稳定性

演示

这说明了它的工作原理：

<强> Live On Coliru

#include <boost/interprocess/managed_mapped_file.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <boost/interprocess/containers/list.hpp>
#include <iostream>
namespace bip = boost::interprocess;

typedef int my_big_struct;

namespace shared {
    using segment = bip::managed_mapped_file;
    using manager = segment::segment_manager;

    template <typename T> using alloc = bip::allocator<T, manager>;
    template <typename T> using list  = boost::interprocess::list<T, alloc<T> >;
}

int main()
{
    using List   = shared::list<my_big_struct>;
    using ItList = shared::list<List::const_iterator>;

    std::remove("test.dat");
    {
        shared::segment segment(bip::open_or_create, "test.dat", 10u<<20); // 10 megabyte

        auto& a  = *segment.find_or_construct<List>("a")(segment.get_segment_manager());
        auto& aa = *segment.find_or_construct<ItList>("aa")(segment.get_segment_manager());
        auto& ab = *segment.find_or_construct<ItList>("ab")(segment.get_segment_manager());

        a.assign({3,2,7,4,9});
        List::const_iterator find = a.begin();
        aa.emplace(aa.begin(), find++);

        ab.emplace(ab.begin(), find);
        ab.emplace(ab.begin(), std::next(find,3));
    }

    // the file is persisted
    {
        shared::segment segment(bip::open_only, "test.dat"); // read it back

        auto& a  = *segment.find_or_construct<List>("a")(segment.get_segment_manager());
        auto& ab = *segment.find_or_construct<ItList>("ab")(segment.get_segment_manager());

        for (auto iterator : ab) {
            std::cout << "Iterator in ab points to " << *iterator << " in list a\n";

            my_big_struct const* pointer = &*iterator;
            for (auto& element : a) {
                if (&element == pointer) {
                    std::cout << "Matching element is found in a (by physical address)\n";
                }
            }
        }

    }
}

打印

Iterator in ab points to 9 in list a
Matching element is found in a (by physical address)
Iterator in ab points to 2 in list a
Matching element is found in a (by physical address)

备注

为一些碎片开销做好准备，因此请注意如何构建数据结构（如果可能，请事先管理分配）。

您可以在Boost Interprocess

上查看我的一些旧答案

boost::interprocess::managed_shared_memory
boost::interprocess::managed_mapped_file
scoped_allocator_adaptor为了更方便

¹BoostInterprocess使用offset_ptr在内部管理抽象（您不必了解此实现细节）

如何将迭代器（或类似的东西）存储到硬盘上？

1 个答案:

演示

备注

更多样本