我想使用尽可能少的空间在我的磁盘上存储sbyte [,](不能花费几秒钟来保存或加载)并在以后将其恢复。
我无法将其序列化为xml:无法序列化System.SByte [,]类型的对象。不支持多维数组。
我无法将其转换为MemoryStream:无法转换为' sbyte [,]'到' int'
除了创建一个文本文件并逐个循环之外..还有哪些选项。
如果它有任何不同,阵列的大小可以超过100 000 x 100 000。该文件还需要可供不同的操作系统和计算机使用。
更新。
我将数组展平为1D sbyte[]
,然后将Stream stream = new MemoryStream(byteArray);
转换为流并将其保存到磁盘以及包含尺寸的单独文件。
[TestMethod]
public void sbyteTo1dThenBack()
{
sbyte[,] start = new sbyte[,]
{
{1, 2},
{3, 4},
{5, 6},
{7, 8},
{9, 10}
};
sbyte[] flattened = new sbyte[start.Length];
System.Buffer.BlockCopy(start, 0, flattened, 0, start.Length * sizeof(sbyte));
sbyte[,] andBackAgain = new sbyte[5, 2];
Buffer.BlockCopy(flattened, 0, andBackAgain, 0, flattened.Length * sizeof(sbyte));
var equal =
start.Rank == andBackAgain.Rank &&
Enumerable.Range(0, start.Rank).All(dimension => start.GetLength(dimension) == andBackAgain.GetLength(dimension)) &&
andBackAgain.Cast<sbyte>().SequenceEqual(andBackAgain.Cast<sbyte>());
Assert.IsTrue(equal);
}
将此作为将流保存到磁盘的基础。 https://stackoverflow.com/a/5515894/937131
如果其他人发现它有用的话,这是我为扁平化和不平整而写的测试用例。
active
答案 0 :(得分:4)
根据我的评论,我觉得写出所有内容的字节数组等同于此。这可能不是最有效的方法,并且缺少您需要提供的大量错误处理代码,但是,它在我的测试中有效。
编辑:此外,BitConverter.ToInt32()可能取决于处理器的“Endianness”。如果您打算在ARM或其他非x86系统上使用此代码,请参阅Scott Chamberlain关于如何解决此问题的评论。
public static class ArraySerializer
{
public static void SaveToDisk(string path, SByte[,] input)
{
var length = input.GetLength(1);
var height = input.GetLength(0);
using (var fileStream = File.OpenWrite(path))
{
fileStream.Write(BitConverter.GetBytes(length), 0, 4);//Store the length
fileStream.Write(BitConverter.GetBytes(height), 0, 4);//Store the height
var lineBuffer = new byte[length];
for (int h = 0; h < height; h++)
{
for (int l = 0; l < length; l++)
{
unchecked //Preserve sign bit
{
lineBuffer[l] = (byte)input[h,l];
}
}
fileStream.Write(lineBuffer,0,length);
}
}
}
public static SByte[,] ReadFromDisk(string path)
{
using (var fileStream = File.OpenRead(path))
{
int length;
int height;
var intBuffer = new byte[4];
fileStream.Read(intBuffer, 0, 4);
length = BitConverter.ToInt32(intBuffer, 0);
fileStream.Read(intBuffer, 0, 4);
height = BitConverter.ToInt32(intBuffer, 0);
var output = new SByte[height, length]; //Note, for large allocations, this can fail... Would fail regardless of how you read it back
var lineBuffer = new byte[length];
for (int h = 0; h < height; h++)
{
fileStream.Read(lineBuffer, 0, length);
for (int l = 0; l < length; l++)
unchecked //Preserve sign bit
{
output[h,l] = (SByte)lineBuffer[l];
}
}
return output;
}
}
}
以下是我测试它的方式:
void Main()
{
var test = new SByte[20000, 25000];
var length = test.GetLength(1);
var height = test.GetLength(0);
var lineBuffer = new byte[length];
var random = new Random();
//Populate with random data
for (int h = 0; h < height; h++)
{
random.NextBytes(lineBuffer);
for (int l = 0; l < length; l++)
{
unchecked //Let's use first bit as a sign bit for SByte
{
test[h,l] = (SByte)lineBuffer[l];
}
}
}
var sw = Stopwatch.StartNew();
ArraySerializer.SaveToDisk(@"c:\users\ed\desktop\test.bin", test);
Console.WriteLine(sw.Elapsed);
sw.Restart();
var test2 = ArraySerializer.ReadFromDisk(@"c:\users\ed\desktop\test.bin");
Console.WriteLine(sw.Elapsed);
Console.WriteLine(test.GetLength(0) == test2.GetLength(0));
Console.WriteLine(test.GetLength(1) == test2.GetLength(1));
Console.WriteLine(Enumerable.Cast<SByte>(test).SequenceEqual(Enumerable.Cast<SByte>(test2))); //Dirty hack to compare contents... takes a very long time
}
在我的系统上(使用SSD),该测试需要大约2.7秒才能写入或读取20kx25k阵列的内容。要添加压缩,您只需将FileStream
打包在GZipStream
。