卡桑德拉(Cassandra)几乎不受压缩OOPS问题的影响吗?

时间:2019-01-04 04:38:11

标签: java pointers cassandra

我正在阅读Java Compressed OOPS。我看到Java做了很多改变,将 compressed 指针用于其所有对象引用。

我能理解的是,从-XX:+UseCompressedOops-XX:-UseCompressedOops的大量对象中,您可能最终会使用更多的 内存。

假设您有100万个对象,在第一种情况下,您需要4Mb的数据作为引用,在第二种情况下,您需要8Mb。

现在,我有一台具有64Gb RAM的计算机,并希望为Cassandra使用50Gb。但是现在我想知道额外的18Gb是否真正有用,还是会被许多无用指针数据填充?

所以我的问题是:

Cassandra是否管理大量小对象,还是只管理几个大对象? (即是否为每个单元格,每一行,每个sstable分配一个缓冲区?)

在第一种情况下,超过32Gb的Cassandra高速缓存与50Gb的带宽相比,我们可能收获不了多少。

1 个答案:

答案 0 :(得分:2)

从堆的角度来看,这是我的一个集群节点的样子:

# /usr/lib/jvm/jdk1.8.0_121/bin/jmap -histo -F 2480
Attaching to process ID 2480, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.121-b13
Iterating over heap. This may take a while...
Object Histogram:

num       #instances    #bytes  Class description
--------------------------------------------------------------------------
1:              1300610 1008416856      byte[]
2:              8451955 405693840       java.nio.HeapByteBuffer
3:              2349359 75179488        org.apache.cassandra.db.BufferCell
4:              2723073 73455464        java.nio.ByteBuffer[]
5:              1804559 57745888        org.apache.cassandra.db.composites.CompoundSparseCellName
6:              510867  46169960        java.lang.Object[]
7:              989235  23741640        java.util.concurrent.ConcurrentSkipListMap$Node
8:              25068   20455288        double[]
9:              851046  20425104        org.apache.cassandra.db.composites.CompoundDenseCellName
10:             580063  18562016        com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node
11:             580056  18561792        com.googlecode.concurrentlinkedhashmap.ConcurrentHashMapV8$Node
12:             42317   15771032        long[]
13:             600999  14423976        com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$WeightedValue
14:             590310  14167440        org.apache.cassandra.cache.KeyCacheKey
15:             554649  13311576        org.apache.cassandra.db.RowIndexEntry
16:             35239   12545512        int[]
17:             434971  10439304        java.lang.Long
18:             143327  9172928 java.nio.DirectByteBufferR
19:             354204  8500896 java.lang.String
20:             98616   8241104 char[]
21:             37512   8102592 com.codahale.metrics.WeightedSnapshot$WeightedSample[]
22:             181096  7243840 org.apache.cassandra.db.BufferExpiringCell
23:             284486  6827664 org.apache.cassandra.db.AtomicBTreeColumns$Holder
24:             264728  6353472 org.apache.cassandra.cql3.ColumnIdentifier
25:             264405  6345720 java.util.concurrent.ConcurrentSkipListMap$Index
26:             122067  4882680 org.apache.cassandra.io.sstable.IndexHelper$IndexInfo
27:             1       4194320 com.googlecode.concurrentlinkedhashmap.ConcurrentHashMapV8$Node[]
28:             99838   2396112 org.apache.cassandra.dht.Murmur3Partitioner$LongToken
29:             89034   2136816 org.apache.cassandra.db.BufferDecoratedKey
30:             88888   2133312 org.apache.cassandra.db.AtomicBTreeColumns
31:             64001   2048032 org.apache.cassandra.db.composites.CompoundSparseCellName$WithCollection
32:             58297   1399128 java.util.ArrayList
33:             56381   1353144 java.util.EnumMap$EntryIterator$Entry
34:             26794   1286112 java.util.HashMap
35:             37450   1198400 java.util.HashMap$Node
36:             47639   1143336 java.lang.Double
37:             35632   1140224 com.codahale.metrics.WeightedSnapshot$WeightedSample
38:             24311   972440  java.util.TreeMap$Entry
39:             18219   874512  org.apache.cassandra.io.sstable.format.big.SSTableNamesIterator
40:             35651   855624  org.apache.cassandra.db.composites.BoundedComposite
41:             35610   854640  org.apache.cassandra.db.composites.CompoundComposite
42:             26165   837280  java.util.TreeMap$KeyIterator
43:             1248    818688  io.netty.util.internal.shaded.org.jctools.queues.MpscArrayQueue
44:             25419   813408  org.apache.cassandra.db.RowIndexEntry$IndexedEntry
45:             50021   800336  java.lang.Integer
46:             6585    736576  java.lang.Class
47:             4182    736032  org.apache.cassandra.net.MessagingService$Verb[]
48:             7718    652400  java.util.HashMap$Node[]
49:             19391   620512  java.util.concurrent.ConcurrentHashMap$Node
50:             9349    600624  java.util.Hashtable$Entry[]
51:             12504   600192  java.util.TimSort
52:             18661   597152  java.util.Vector
53:             4100    557600  com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$PaddedAtomicReference
54:             4953    554736  java.net.SocksSocketImpl
55:             13215   528656  java.util.concurrent.ConcurrentSkipListMap$Index[]
56:             15888   508416  java.util.concurrent.ConcurrentSkipListMap$HeadIndex
57:             19379   465096  org.apache.cassandra.db.ArrayBackedSortedColumns$4
58:             9339    448272  java.util.Hashtable
59:             4520    419824  short[]
60:             25970   415520  java.lang.Object
61:             16148   387552  javax.management.ObjectName$Property
62:             9358    374320  java.security.ProtectionDomain
63:             12504   300096  com.codahale.metrics.WeightedSnapshot
64:             9356    299392  java.security.CodeSource
65:             12367   296808  java.util.concurrent.ConcurrentLinkedDeque$Node
66:             5891    282768  org.apache.cassandra.net.MessageOut
67:             5676    272448  java.util.TreeMap
68:             8542    271256  javax.management.ObjectName$Property[]
69:             8034    257088  java.util.EnumMap$EntryIterator
70:             7077    226464  java.net.InetAddress$InetAddressHolder
71:             7072    226304  java.util.ArrayList$Itr
72:             5644    225760  org.apache.cassandra.db.PreHashedDecoratedKey
73:             9288    222912  java.util.concurrent.ConcurrentLinkedQueue$Node
74:             3085    222120  com.sun.jmx.remote.util.OrderClassLoaders
75:             3085    222120  javax.management.remote.rmi.RMIConnectionImpl$CombinedClassLoader
76:             3085    222120  javax.management.remote.rmi.RMIConnectionImpl$CombinedClassLoader$ClassLoaderWrapper
77:             6730    215360  java.util.concurrent.locks.AbstractQueuedSynchronizer$Node
78:             8413    201912  org.apache.cassandra.db.DeletionInfo
79:             12504   200064  com.codahale.metrics.WeightedSnapshot$1
80:             4096    196608  java.util.concurrent.ConcurrentHashMap$TreeNode
81:             250     196544  java.util.concurrent.ConcurrentHashMap$Node[]
82:             4881    195240  org.apache.cassandra.db.ArrayBackedSortedColumns
83:             4800    192000  java.util.ArrayList$SubList
84:             150     191840  io.netty.buffer.PoolSubpage[]
85:             3952    189696  org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask
86:             7635    183240  org.apache.thrift.protocol.TField
87:             3716    178368  org.apache.cassandra.db.SliceByNamesReadCommand
88:             2778    177792  io.netty.buffer.PoolSubpage
89:             1979    174152  java.lang.reflect.Method
90:             4271    170840  javax.management.ObjectName
91:             3522    169056  org.apache.cassandra.io.sstable.Descriptor
92:             2340    168480  org.apache.thrift.protocol.TBinaryProtocol
93:             2990    167240  org.apache.cassandra.db.Cell[]
94:             5076    162432  org.apache.cassandra.db.ArrayBackedSortedColumns$ForwardsCellIterator
95:             425     159800  java.lang.Thread
96:             3988    159520  com.google.common.collect.SingletonImmutableBiMap
97:             4949    158368  java.net.Socket
98:             6548    157152  java.net.Inet4Address
99:             4870    155840  sun.misc.FDBigInteger
100:            9730    155680  java.util.HashSet
[...]
2865:           1       16      org.apache.cassandra.service.PendingRangeCalculatorService$1
2866:           1       16      sun.reflect.generics.tree.VoidDescriptor
2867:           1       16      org.apache.cassandra.thrift.Cassandra$Processor$insert
Total :         25759325        -Xms7967M -Xmx7967M -Xmn800M
Heap traversal took 1130.769 seconds.

堆上有25.759.325个实例,并且该节点以-Xms7967M -Xmx7967M -Xmn800M开始-因此这里有很多实例。

有关调整资源的信息,另请参见:https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/operations/opsTuneJVM.html

在我的一项测试中,我最终在单个主机上运行了4个Cassandra实例,每个主机具有8GB的堆(在具有64GB RAM的机器上,留出32GB的OS页面缓存空间,以确保每个主机没有机架感知主机崩溃时丢失多个副本)。