Question

我正在使用flink弹性搜索连接器将数据发送到弹性搜索索引。

但我无法运行该作业并获得此异常。如何解决这个问题？。我缺少任何设置或配置吗？有谁能指导我？

io.netty.channel.ChannelException: The operating system does not offer enough file handles to open the network connection. Please increase the number of of available file handles.
    at org.apache.flink.runtime.io.network.netty.NettyClient.connect(NettyClient.java:207)
    at org.apache.flink.runtime.io.network.netty.PartitionRequestClientFactory.createPartitionRequestClient(PartitionRequestClientFactory.java:81)
    at org.apache.flink.runtime.io.network.netty.NettyConnectionManager.createPartitionRequestClient(NettyConnectionManager.java:60)
    at org.apache.flink.runtime.io.network.partition.consumer.RemoteInputChannel.requestSubpartition(RemoteInputChannel.java:112)
    at org.apache.flink.runtime.io.network.partition.consumer.SingleInputGate.requestPartitions(SingleInputGate.java:419)
    at org.apache.flink.runtime.io.network.partition.consumer.SingleInputGate.getNextBufferOrEvent(SingleInputGate.java:441)
    at org.apache.flink.streaming.runtime.io.BarrierTracker.getNextNonBlocked(BarrierTracker.java:85)
    at org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:192)
    at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:63)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:261)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:665)
    at java.lang.Thread.run(Thread.java:748)
Caused by: io.netty.channel.ChannelException: Failed to open a socket.
    at io.netty.channel.socket.nio.NioSocketChannel.newSocket(NioSocketChannel.java:62)
    at io.netty.channel.socket.nio.NioSocketChannel.<init>(NioSocketChannel.java:72)
    at sun.reflect.GeneratedConstructorAccessor8.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at java.lang.Class.newInstance(Class.java:442)
    at io.netty.bootstrap.AbstractBootstrap$BootstrapChannelFactory.newChannel(AbstractBootstrap.java:453)
    at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:306)
    at io.netty.bootstrap.Bootstrap.doConnect(Bootstrap.java:133)
    at io.netty.bootstrap.Bootstrap.connect(Bootstrap.java:115)
    at org.apache.flink.runtime.io.network.netty.NettyClient.connect(NettyClient.java:198)
    ... 11 more
Caused by: java.net.SocketException: Too many open files
    at sun.nio.ch.Net.socket0(Native Method)
    at sun.nio.ch.Net.socket(Net.java:411)
    at sun.nio.ch.Net.socket(Net.java:404)
    at sun.nio.ch.SocketChannelImpl.<init>(SocketChannelImpl.java:105)
    at sun.nio.ch.SelectorProviderImpl.openSocketChannel(SelectorProviderImpl.java:60)
    at io.netty.channel.socket.nio.NioSocketChannel.newSocket(NioSocketChannel.java:60)
    ... 21 more

Answer 1

为了增加系统中文件句柄的数量，如果您具有root权限，则可以通过ulimit -n为进程设置它们。如果要更改系统范围内的文件句柄数，请更改/proc/sys/fs/file-max中的值（在Linux上）。

如何解决使用弹性搜索连接器将流数据发送到弹性搜索索引时flink中有太多打开文件异常？

1 个答案: