我正在阅读Java Compressed OOPS。我看到Java做了很多改变,将 compressed 指针用于其所有对象引用。
我能理解的是,从-XX:+UseCompressedOops
到-XX:-UseCompressedOops
的大量对象中,您可能最终会使用更多的 内存。
假设您有100万个对象,在第一种情况下,您需要4Mb的数据作为引用,在第二种情况下,您需要8Mb。
现在,我有一台具有64Gb RAM的计算机,并希望为Cassandra使用50Gb。但是现在我想知道额外的18Gb是否真正有用,还是会被许多无用指针数据填充?
所以我的问题是:
Cassandra是否管理大量小对象,还是只管理几个大对象? (即是否为每个单元格,每一行,每个sstable分配一个缓冲区?)
在第一种情况下,超过32Gb的Cassandra高速缓存与50Gb的带宽相比,我们可能收获不了多少。
答案 0 :(得分:2)
从堆的角度来看,这是我的一个集群节点的样子:
# /usr/lib/jvm/jdk1.8.0_121/bin/jmap -histo -F 2480
Attaching to process ID 2480, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.121-b13
Iterating over heap. This may take a while...
Object Histogram:
num #instances #bytes Class description
--------------------------------------------------------------------------
1: 1300610 1008416856 byte[]
2: 8451955 405693840 java.nio.HeapByteBuffer
3: 2349359 75179488 org.apache.cassandra.db.BufferCell
4: 2723073 73455464 java.nio.ByteBuffer[]
5: 1804559 57745888 org.apache.cassandra.db.composites.CompoundSparseCellName
6: 510867 46169960 java.lang.Object[]
7: 989235 23741640 java.util.concurrent.ConcurrentSkipListMap$Node
8: 25068 20455288 double[]
9: 851046 20425104 org.apache.cassandra.db.composites.CompoundDenseCellName
10: 580063 18562016 com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node
11: 580056 18561792 com.googlecode.concurrentlinkedhashmap.ConcurrentHashMapV8$Node
12: 42317 15771032 long[]
13: 600999 14423976 com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$WeightedValue
14: 590310 14167440 org.apache.cassandra.cache.KeyCacheKey
15: 554649 13311576 org.apache.cassandra.db.RowIndexEntry
16: 35239 12545512 int[]
17: 434971 10439304 java.lang.Long
18: 143327 9172928 java.nio.DirectByteBufferR
19: 354204 8500896 java.lang.String
20: 98616 8241104 char[]
21: 37512 8102592 com.codahale.metrics.WeightedSnapshot$WeightedSample[]
22: 181096 7243840 org.apache.cassandra.db.BufferExpiringCell
23: 284486 6827664 org.apache.cassandra.db.AtomicBTreeColumns$Holder
24: 264728 6353472 org.apache.cassandra.cql3.ColumnIdentifier
25: 264405 6345720 java.util.concurrent.ConcurrentSkipListMap$Index
26: 122067 4882680 org.apache.cassandra.io.sstable.IndexHelper$IndexInfo
27: 1 4194320 com.googlecode.concurrentlinkedhashmap.ConcurrentHashMapV8$Node[]
28: 99838 2396112 org.apache.cassandra.dht.Murmur3Partitioner$LongToken
29: 89034 2136816 org.apache.cassandra.db.BufferDecoratedKey
30: 88888 2133312 org.apache.cassandra.db.AtomicBTreeColumns
31: 64001 2048032 org.apache.cassandra.db.composites.CompoundSparseCellName$WithCollection
32: 58297 1399128 java.util.ArrayList
33: 56381 1353144 java.util.EnumMap$EntryIterator$Entry
34: 26794 1286112 java.util.HashMap
35: 37450 1198400 java.util.HashMap$Node
36: 47639 1143336 java.lang.Double
37: 35632 1140224 com.codahale.metrics.WeightedSnapshot$WeightedSample
38: 24311 972440 java.util.TreeMap$Entry
39: 18219 874512 org.apache.cassandra.io.sstable.format.big.SSTableNamesIterator
40: 35651 855624 org.apache.cassandra.db.composites.BoundedComposite
41: 35610 854640 org.apache.cassandra.db.composites.CompoundComposite
42: 26165 837280 java.util.TreeMap$KeyIterator
43: 1248 818688 io.netty.util.internal.shaded.org.jctools.queues.MpscArrayQueue
44: 25419 813408 org.apache.cassandra.db.RowIndexEntry$IndexedEntry
45: 50021 800336 java.lang.Integer
46: 6585 736576 java.lang.Class
47: 4182 736032 org.apache.cassandra.net.MessagingService$Verb[]
48: 7718 652400 java.util.HashMap$Node[]
49: 19391 620512 java.util.concurrent.ConcurrentHashMap$Node
50: 9349 600624 java.util.Hashtable$Entry[]
51: 12504 600192 java.util.TimSort
52: 18661 597152 java.util.Vector
53: 4100 557600 com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$PaddedAtomicReference
54: 4953 554736 java.net.SocksSocketImpl
55: 13215 528656 java.util.concurrent.ConcurrentSkipListMap$Index[]
56: 15888 508416 java.util.concurrent.ConcurrentSkipListMap$HeadIndex
57: 19379 465096 org.apache.cassandra.db.ArrayBackedSortedColumns$4
58: 9339 448272 java.util.Hashtable
59: 4520 419824 short[]
60: 25970 415520 java.lang.Object
61: 16148 387552 javax.management.ObjectName$Property
62: 9358 374320 java.security.ProtectionDomain
63: 12504 300096 com.codahale.metrics.WeightedSnapshot
64: 9356 299392 java.security.CodeSource
65: 12367 296808 java.util.concurrent.ConcurrentLinkedDeque$Node
66: 5891 282768 org.apache.cassandra.net.MessageOut
67: 5676 272448 java.util.TreeMap
68: 8542 271256 javax.management.ObjectName$Property[]
69: 8034 257088 java.util.EnumMap$EntryIterator
70: 7077 226464 java.net.InetAddress$InetAddressHolder
71: 7072 226304 java.util.ArrayList$Itr
72: 5644 225760 org.apache.cassandra.db.PreHashedDecoratedKey
73: 9288 222912 java.util.concurrent.ConcurrentLinkedQueue$Node
74: 3085 222120 com.sun.jmx.remote.util.OrderClassLoaders
75: 3085 222120 javax.management.remote.rmi.RMIConnectionImpl$CombinedClassLoader
76: 3085 222120 javax.management.remote.rmi.RMIConnectionImpl$CombinedClassLoader$ClassLoaderWrapper
77: 6730 215360 java.util.concurrent.locks.AbstractQueuedSynchronizer$Node
78: 8413 201912 org.apache.cassandra.db.DeletionInfo
79: 12504 200064 com.codahale.metrics.WeightedSnapshot$1
80: 4096 196608 java.util.concurrent.ConcurrentHashMap$TreeNode
81: 250 196544 java.util.concurrent.ConcurrentHashMap$Node[]
82: 4881 195240 org.apache.cassandra.db.ArrayBackedSortedColumns
83: 4800 192000 java.util.ArrayList$SubList
84: 150 191840 io.netty.buffer.PoolSubpage[]
85: 3952 189696 org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask
86: 7635 183240 org.apache.thrift.protocol.TField
87: 3716 178368 org.apache.cassandra.db.SliceByNamesReadCommand
88: 2778 177792 io.netty.buffer.PoolSubpage
89: 1979 174152 java.lang.reflect.Method
90: 4271 170840 javax.management.ObjectName
91: 3522 169056 org.apache.cassandra.io.sstable.Descriptor
92: 2340 168480 org.apache.thrift.protocol.TBinaryProtocol
93: 2990 167240 org.apache.cassandra.db.Cell[]
94: 5076 162432 org.apache.cassandra.db.ArrayBackedSortedColumns$ForwardsCellIterator
95: 425 159800 java.lang.Thread
96: 3988 159520 com.google.common.collect.SingletonImmutableBiMap
97: 4949 158368 java.net.Socket
98: 6548 157152 java.net.Inet4Address
99: 4870 155840 sun.misc.FDBigInteger
100: 9730 155680 java.util.HashSet
[...]
2865: 1 16 org.apache.cassandra.service.PendingRangeCalculatorService$1
2866: 1 16 sun.reflect.generics.tree.VoidDescriptor
2867: 1 16 org.apache.cassandra.thrift.Cassandra$Processor$insert
Total : 25759325 -Xms7967M -Xmx7967M -Xmn800M
Heap traversal took 1130.769 seconds.
堆上有25.759.325个实例,并且该节点以-Xms7967M -Xmx7967M -Xmn800M
开始-因此这里有很多实例。
有关调整资源的信息,另请参见:https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/operations/opsTuneJVM.html
在我的一项测试中,我最终在单个主机上运行了4个Cassandra实例,每个主机具有8GB的堆(在具有64GB RAM的机器上,留出32GB的OS页面缓存空间,以确保每个主机没有机架感知主机崩溃时丢失多个副本)。