我使用protobuf-net将方向图序列化为文件。
我的(简化)课程如下:
[ProtoContract]
public class Network
{
// All of the devices on the network
[ProtoMember(1)]
public readonly Dictionary<int, Device> Vertex;
// The list of connections
[ProtoMember(2)]
public readonly Dictionary<int, List<Device>> Nodes;
}
[ProtoContract(AsReferenceDefault = true)]
public class Device
{
[ProtoMember(1)]
public readonly int Id;
// All the devices with a direct path to this node
[ProtoMember(2)]
public readonly Dictionary<int, Device> PathTo;
// All the devices directly reachable from this node
[ProtoMember(3)]
public readonly Dictionary<int, Device> PathFrom;
// the nodes that this is connected to
[ProtoMember(4)]
public readonly int[] Nodes = new int[2];
}
如果我尝试使用protobuf-net序列化Network
类的实例,它只有在设备的PathTo
和PathFrom
字典为空时才有效。
一旦我开始为每个Device
填充这些词典(注意图表的方向),尝试使用protobuf-net进行序列化会导致堆栈溢出。
有谁知道为什么会堆栈溢出?
我已经阅读了这个问题:Protobuf-net object reference deserialization using Dictionary: A reference-tracked object changed reference during deserialization,根据Mark对其答案的编辑,他通过添加我在Device
类上使用的AsReferenceDefault属性来修复此图引用。 / p>
对我来说,看起来它正在将字典中的所有元素视为独特的个体,同时它遍历列表。
考虑到网络中有大约300万台设备,这很快就会导致堆栈溢出。
调试窗口堆栈的屏幕截图:
答案 0 :(得分:0)
作为优化的一部分,我调整了我存储边缘的方式,因此我也最终修改了我的数据结构,以便我可以从对象引用中删除循环。
我创建了Edge
类来跟踪对象之间的所有边缘和边缘的方向,然后我向OnDeserialized
添加了Graph
方法,以便它运行通过所有边缘并重新添加对每个链接类的所有引用。
作为一个额外的好处,这个数据结构可以快速保存/加载(大约3,000,000个顶点,大约5,000,000个边缘需要大约18秒才能加载,而旧结构大约需要30秒),并且遍历速度更快(完整遍历为3s并修改链接,而旧结构为~50s)。
[ProtoContract]
public class Graph
{
// All of the devices on the graph
[ProtoMember(1)]
public readonly Dictionary<int, Vertex> Vertex;
// All of the links in the graph
[ProtoMember(2)]
public readonly List<Edge> Links;
[OnDeserialized]
public void OnDeserialized() {
// Add references to all of the links
foreach (Edge l in Links) {
l.Vertices[0] = this.Vertex[l.VertexIds[0]];
l.Vertices[1] = this.Vertex[l.VertexIds[1]];
}
}
}
// A vertex of the graph
[ProtoContract]
public class Vertex
{
[ProtoMember(1)]
public readonly int Id;
/*
we index the edges by the ID of the other device.
It makes it easy to find specific edges and makes it trivial to only track 1 copy of an edge
*/
[ProtoMember(2)]
public readonly Dictionary<int, Edge> Edges;
}
// An edge
[ProtoContract(AsReferenceDefault = true)]
public class Edge
{
public readonly Vertex[] Vertices;
[ProtoMember(1)]
public readonly int[] VertexIds;
[ProtoMember(2)]
public readonly Direction Direction;
public Edge() {
this.Vertices = new Vertex[2];
this.Direction = Direction.None;
/*
important to note that we don't set the VertexIds array in this constructor.
protobuf-net will create it for us, and if we create it here, we'll end up with a 4 length array.
*/
}
}
/*
denotes to which index in a edge's devices array the direction the edge goes.
[Flags] so we can do checks easier:
Edge.Direction == Direction.Both implies Edge.Direction & Direction.ZeroToOne != 0
*/
[Flags]
public enum Direction {
None = 0,
ZeroToOne = 1,
OneToZero = 2,
Both = 3
}