Question

我正在测试BigMemory Go的表现，我注意到数据丢失了。

这是单元测试：

public class BigMemoryGoPerformanceTest {

    public static void main(String[] args) throws IOException {
        BigMemoryGoPerformanceTest test = new BigMemoryGoPerformanceTest();
        test.testBigMemoryGoPerformance(args[0], Long.valueOf(args[1]), Long.valueOf(args[2]));
    }

    @DataProvider(name = "bigMemoryGoPerformance")
    public Object[][] bigMemoryGoPerformance() {
        return new Object[][] {
                   { "very small", 250000, 1 }
                  ,{ "small", 1000000, 2 }
                  ,{ "large", 10000000, 2 }
                  ,{ "very large", 50000000, 4 }
        };
    }

    @Test(dataProvider="bigMemoryGoPerformance")
    public void testBigMemoryGoPerformance(String testName, long maxRegisters, long externalMemory) throws IOException {

        Configuration managerConfiguration = new Configuration();
        managerConfiguration.updateCheck(true)
            .monitoring(Configuration.Monitoring.AUTODETECT)
            .name("config")
            .cache(new CacheConfiguration()
                .name("testperformance")
                .maxBytesLocalHeap(128, MemoryUnit.MEGABYTES)
                .maxBytesLocalOffHeap(externalMemory, MemoryUnit.GIGABYTES)
                .eternal(true)
            );

        CacheManager manager = CacheManager.create(managerConfiguration);
        Cache cache = manager.getCache("testperformance");

        try {
            System.out.println("First pass: insert " + maxRegisters + " nodes.");
            long mms = System.currentTimeMillis();
            for (long i = 0; i < maxRegisters; i++) {
                ItemForTesting item = new ItemForTesting();
                item.id = i;
                item.lastWay = i;
                item.latitude = new Double(i);
                item.longitude = new Double(i);
                cache.put( new Element(i, item) );
            }
            long timeInMMS = System.currentTimeMillis() - mms;
            System.out.println(testName + " --> Inserted " + maxRegisters + " registers in " + timeInMMS + " mms. Performance: " + ((long)(maxRegisters * 1000d / timeInMMS)) + " regs per seconds writing." );


            System.out.println("Second pass: reading " + maxRegisters + " nodes.");
            mms = System.currentTimeMillis();
            for (long i = 0; i < maxRegisters; i++) {
                Element element = cache.get(i);
                ItemForTesting item = (ItemForTesting)element.getObjectValue(); // <--- Null point exception !!!!!
            }
            timeInMMS = System.currentTimeMillis() - mms;
            System.out.println(testName + " --> Read " + maxRegisters + " registers in " + timeInMMS + " mms. Performance: " + ((long)(maxRegisters * 1000d / timeInMMS)) + " regs per seconds reading." );

            System.out.println("Third pass: updating " + maxRegisters + " nodes.");
            mms = System.currentTimeMillis();
            for (long i = 0; i < maxRegisters; i++) {
                Element element = cache.get(i);
                ItemForTesting item = (ItemForTesting)element.getObjectValue();
                item.latitude = item.latitude +1;
                cache.put( new Element(i, item) );
            }
            timeInMMS = System.currentTimeMillis() - mms;
            System.out.println(testName + " --> Updated " + maxRegisters + " registers in " + timeInMMS + " mms. Performance: " + ((long)(maxRegisters * 1000d / timeInMMS)) + " regs per seconds reading." );

            System.out.println("Fourth pass: deleting " + maxRegisters + " nodes.");
            mms = System.currentTimeMillis();
            for (long i = 0; i < maxRegisters; i++) {
                Element element = cache.get(i);
                ItemForTesting item = (ItemForTesting)element.getObjectValue();
                item.latitude = item.latitude +1;
                cache.remove( new Element(i, item) );
            }
            timeInMMS = System.currentTimeMillis() - mms;
            System.out.println(testName + " --> Removed " + maxRegisters + " registers in " + timeInMMS + " mms. Performance: " + ((long)(maxRegisters * 1000d / timeInMMS)) + " regs per seconds reading." );


        } finally {
            if (manager != null) manager.shutdown();
        }

    }

}

在“非常大”中，使用50.000.000次迭代，当在第二次传递中读取数据时，返回null。所以数据丢失!! 配置设置为eternal = true。如何避免丢失数据？

有什么不对？

谢谢！

Answer 1

我怀疑你需要固定元素以避免它们过期。尝试看起来像这样的配置：

<cache name="myBigMemoryGoStore"
    maxBytesLocalHeap="1M"
    eternal="true"
    maxBytesLocalOffHeap="2G">
    <persistence strategy="none"/>
    <pinning store="inCache" />
    <sizeOfPolicy maxDepth="100000" maxDepthExceededBehavior="continue"/>

http://terracotta.org/documentation/4.0/bigmemorygo/configuration/data-life

Answer 2

由于堆/副堆压力，数据可能会被逐出。如上所述，您可以使用固定来确保永不丢失数据。但是只有在确定缓存大小时才应该使用固定，因为固定缓存可能会增长到无限大小，或者根据您可能使用的版本导致OOME。

另一方面，为了测试BigMemory Go的性能，我建议使用多线程测试来获得更多的吞吐量而不是仅仅使用大量的迭代。

干杯， Ridhav

如何避免Terracotta BigMemory Go中的数据丢失

2 个答案: