我有一个netty实现,客户端/服务器,其中许多线程与SimpleChannelInboundHandler共享一个通道。
我们实施的每秒消息数(TPS)将非常高1000+。每条消息都发送一个java对象(TransactionBundle)并使用messageReceived()从服务器读回响应)
send()方法发送的已发送消息存储在Handler中由唯一键访问的ConcurrentHashMap中,来自messageReceived()的响应使用此键查找原始发送消息以删除对象来自 ConcurrentHashMap并完成该过程。
在send()方法中,我们使用FutureTask等待关联的响应通过messageReceived从服务器返回。返回时,响应会通知此FutureTask完成。
问题是随着消息数量的增加,响应时间不断增加,直到我们开始在SimpleChannelInboundHandler.send()中看到FutureTask超时(60秒)。
我之前在writeAndFlush中添加了一个ChannelFutureListener,并且发现写入的发送时间越来越长,除非它处于低负载状态,否则它不是瞬时的。基于时间我已经采取了感觉就像服务器不能 处理写入负载。
好像频道无法处理我们发送给它的负载。有趣的是,来自服务器的响应非常快,只有发送受到我能说的影响。
我正在考虑增加缓冲区大小等。我已在下面提供了相关的客户端代码。
频道创建为:
public ClientConnection(String host, String port, EventLoopGroup eventLoopGroup) throws SSLException, InterruptedException {
HOST = System.getProperty("host", host);
PORT = Integer.parseInt(System.getProperty("port", port));
SslContext sslCtx;
if (SSL) {
sslCtx = SslContext.newClientContext(InsecureTrustManagerFactory.INSTANCE);
} else {
sslCtx = null;
}
this.eventLoopGroup = eventLoopGroup;
// Make the connection attempt.
future = getBootStrap(sslCtx).connect(HOST, PORT).sync();
InetSocketAddress socket = (InetSocketAddress) future.channel().remoteAddress();
verifiedRemoteHost = socket.getHostString();
verfiedRemotePort = socket.getPort();
handler = (ClientConnectionHandler) future.channel().pipeline().last();
}
private Bootstrap getBootStrap(final SslContext sslCtx) {
Bootstrap b = new Bootstrap();
b.group(eventLoopGroup)
.channel(NioSocketChannel.class)
.handler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel ch) throws Exception {
ChannelPipeline p = ch.pipeline();
if (sslCtx != null) {
p.addLast(sslCtx.newHandler(ch.alloc(), HOST, PORT));
}
p.addLast(new ObjectEncoder());
p.addLast(new ObjectDecoder(ClassResolvers.cacheDisabled(null)));
p.addLast(new ClientConnectionHandler());
}
});
return b;
}
SimpleChannelInboundHandler.send()方法是:
public TransactionBundle send(Hashtable bundleIn) {
//TransactionBundle is a HashMap
TransactionBundle bundleOut;
String registryId = getUniqueKey();
bundleIn.put(CLIENT_REGISTRY_ID, registryId);
registry.put(registryId, getRegistryItem());
ctx.writeAndFlush(bundleIn); //Adding a listener here indicated that the write was progressively getting slower
logger.trace("Sent transaction for registryId [{}]", registryId);
try {
registry.get(registryId).getFutureTask().get(FUTURE_TIMEOUT, TimeUnit.SECONDS); //time-outs occur here
} catch (InterruptedException | ExecutionException | TimeoutException ex) {
logger.error("FutureTask Failed [{}] for registryId [{}]", ex.getMessage(), registryId, ex);
}
logger.trace("FutureTask completed for registryId [{}]", registryId);
....other tasks
return bundleOut;
}
SimpleChannelInboundHandler.messageReceived()方法是:
@Override
protected void messageReceived(ChannelHandlerContext chc, Hashtable bundle) throws Exception {
String registryId = (String) bundle.get(CLIENT_REGISTRY_ID);
logger.trace("Received message for registry ID [{}]", registryId);
ClientConnectionRegistryItem registryItem = registry.get(registryId);
if (registryItem != null) {
//FutureTask is still waiting for a response
FutureTask futureTask = registryItem.getFutureTask();
if (futureTask != null) {
if (!futureTask.isDone()) {
//FutureTask is not completed so is ready to receive the response. Add the bundle to the ClientConnectionRegistryItem
registryItem.setBundle(bundle);
logger.trace("Registry ID [{}] bundle set", registryId);
} else {
//FutureTask has completed already. Log the details
logger.trace("Registry ID [{}] timeout.", registryId);
}
futureTaskExecutor.execute(futureTask);
}
} else {
//Message response with no matching registryId
//Orphaned transaction should be cleaned up with the timeout of the FutureTask
logger.error("Registry ID [{}] not found in registry.", registryId);
}
}