Cassandra CPU负载过高问题(3.11.1)

时间:2019-01-12 03:19:18

标签: java cassandra cassandra-3.0 cpu-load

需要一些帮助来解决我对cassandra的高CPU使用率问题。

我们有一个12节点的Cassandra集群,其规格如下。

  • 8核
  • 带有G1GC的16GB HEAP / 32GB RAM

突然我开始看到一些较高的cpu负载(在8个核心节点上大约为18-24)

cassandra堆栈跟踪显示了很多可运行的线程,如下所示。

sun.nio.ch.FileDispatcherImpl.read0(Native Method)
 MessagingService-Incoming-/10.xx.xx.xx
 MessagingService-Incoming-/10.xx.xx.xx
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
at org.apache.cassandra.io.util.NIODataInputStream.reBuffer(NIODataInputStream.java:66)
at org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:144)
at org.apache.cassandra.io.util.RebufferingInputStream.readPrimitiveSlowly(RebufferingInputStream.java:108)
at org.apache.cassandra.io.util.RebufferingInputStream.readInt(RebufferingInputStream.java:188)
at org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:179)
at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:94)

"epollEventLoopGroup-2-9": running
at io.netty.channel.epoll.Native.epollWait0(Native Method)
at io.netty.channel.epoll.Native.epollWait(Native.java:117)
at io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:226)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:250)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
at java.lang.Thread.run(Thread.java:748)

上述第一个线程有35个出现,第二个线程有24个出现。

任何人都可以找出这里有什么问题吗?

在集群方面,

  • 没有任何待处理的压缩/任务。
  • &GC暂停时间低于100ms

谢谢

0 个答案:

没有答案