Hyperledger锯齿占用Docker中的所有RAM（MemoryError / LMDB / RAM）

时间：2019-12-22 04:22:19

标签： docker out-of-memory blockchain lmdb hyperledger-sawtooth

我创建了一个存储5MB有效负载的事务处理器。这样做（500个事务存储循环）会增加内存消耗，直到我的32GB RAM服务器上没有更多的RAM空间（提交约220个事务）为止。此时，验证器将停止处理以下错误消息：

influxdb_1                      | [httpd] 172.22.0.5 - - [21/Dec/2019:19:02:52 +0000] "POST /write?db=metrics&precision=s HTTP/1.1" 204 0 "-" "Python-urllib/3.6" 7d4dc556-2424-11ea-8377-0242ac16000e 112925
sawtooth-validator-default-1    | [2019-12-21 19:02:54.397 ERROR    threadpool] (Consensus) Unhandled exception during execution of task _HandlerManager.execute.<locals>.wrapped
sawtooth-validator-default-1    | Traceback (most recent call last):
sawtooth-validator-default-1    |   File "/usr/lib/python3/dist-packages/sawtooth_validator/concurrent/threadpool.py", line 83, in wrapper
sawtooth-validator-default-1    |     return_value = fn(*args, **kwargs)
sawtooth-validator-default-1    |   File "/usr/lib/python3/dist-packages/sawtooth_validator/networking/dispatch.py", line 444, in wrapped
sawtooth-validator-default-1    |     return callback(self._handler.handle(connection_id, message))
sawtooth-validator-default-1    |   File "/usr/lib/python3/dist-packages/sawtooth_validator/consensus/handlers.py", line 378, in handle
sawtooth-validator-default-1    |     self._proxy.validate_block(block_id)
sawtooth-validator-default-1    |   File "/usr/lib/python3/dist-packages/sawtooth_validator/consensus/proxy.py", line 225, in validate_block
sawtooth-validator-default-1    |     block = next(self._block_manager.get([block_id]))
sawtooth-validator-default-1    |   File "/usr/lib/python3/dist-packages/sawtooth_validator/ffi.py", line 203, in __next__
sawtooth-validator-default-1    |     payload = from_rust_vec(vec_ptr, vec_len, vec_cap)
sawtooth-validator-default-1    |   File "/usr/lib/python3/dist-packages/sawtooth_validator/ffi.py", line 106, in from_rust_vec
sawtooth-validator-default-1    |     py_bytes = bytes(vec_ptr[:vec_len.value])
sawtooth-validator-default-1    | MemoryError
influxdb_1                      | [httpd] 172.22.0.18 - - [21/Dec/2019:19:02:54 +0000] "POST /write?db=metrics&precision=s HTTP/1.1" 204 0 "-" "Python-urllib/3.6" 7e939083-2424-11ea-8379-0242ac16000e 25652
sawtooth-validator-default-1    | [2019-12-21 19:02:56.425 INFO     (unknown file)] [src/journal/block_validator.rs: 265] Block 4a6a1342c829d81752127f5b95deaff1b9cac847ec3675714a2b1b086d281e6f44979beef8506041a89d841e089d8ae36e2513e3f65a91c2e634fa23a4052e06 passed validation
influxdb_1                      | [httpd] 172.22.0.4 - - [21/Dec/2019:19:02:58 +0000] "POST /write?db=metrics&precision=s HTTP/1.1" 204 0 "-" "Python-urllib/3.6" 80da4488-2424-11ea-837b-0242ac16000e 177245
influxdb_1                      | [httpd] 172.22.0.3 - - [21/Dec/2019:19:02:58 +0000] "POST /write?db=metrics&precision=s HTTP/1.1" 204 0 "-" "Python-urllib/3.6" 80d75795-2424-11ea-837a-0242ac16000e 374255
sawtooth-validator-default-1    | [2019-12-21 19:03:01.169 ERROR    threadpool] (Consensus) Unhandled exception during execution of task _HandlerManager.execute.<locals>.wrapped
sawtooth-validator-default-1    | Traceback (most recent call last):
sawtooth-validator-default-1    |   File "/usr/lib/python3/dist-packages/sawtooth_validator/concurrent/threadpool.py", line 83, in wrapper
sawtooth-validator-default-1    |     return_value = fn(*args, **kwargs)
sawtooth-validator-default-1    |   File "/usr/lib/python3/dist-packages/sawtooth_validator/networking/dispatch.py", line 444, in wrapped
sawtooth-validator-default-1    |     return callback(self._handler.handle(connection_id, message))
sawtooth-validator-default-1    |   File "/usr/lib/python3/dist-packages/sawtooth_validator/consensus/handlers.py", line 378, in handle
sawtooth-validator-default-1    |     self._proxy.validate_block(block_id)
sawtooth-validator-default-1    |   File "/usr/lib/python3/dist-packages/sawtooth_validator/consensus/proxy.py", line 225, in validate_block
sawtooth-validator-default-1    |     block = next(self._block_manager.get([block_id]))
sawtooth-validator-default-1    |   File "/usr/lib/python3/dist-packages/sawtooth_validator/ffi.py", line 203, in __next__
sawtooth-validator-default-1    |     payload = from_rust_vec(vec_ptr, vec_len, vec_cap)
sawtooth-validator-default-1    |   File "/usr/lib/python3/dist-packages/sawtooth_validator/ffi.py", line 106, in from_rust_vec
sawtooth-validator-default-1    |     py_bytes = bytes(vec_ptr[:vec_len.value])
sawtooth-validator-default-1    | MemoryError
influxdb_1                      | [httpd] 172.22.0.2 - - [21/Dec/2019:19:03:01 +0000] "POST /write?db=metrics&precision=s HTTP/1.1" 204 0 "-" "Python-urllib/3.6" 82953819-2424-11ea-837c-0242ac16000e 23421
influxdb_1                      | [httpd] 172.22.0.9 - - [21/Dec/2019:19:03:03 +0000] "POST /write?db=metrics&precision=s HTTP/1.1" 204 0 "-" "Python-urllib/3.6" 8383f49a-2424-11ea-837d-0242ac16000e 35783
sawtooth-settings-tp-default-0  | INFO  | sawtooth_sdk::messag | Received Disconnect
sawtooth-settings-tp-default-0  | INFO  | sawtooth_sdk::proces | Trying to Reconnect
sawtooth-settings-tp-default-0  | INFO  | sawtooth_sdk::proces | connecting to endpoint: tcp://validator-0:4004
sawtooth-settings-tp-default-0  | INFO  | sawtooth_sdk::proces | sending TpRegisterRequest: sawtooth_settings 1.0
sawtooth-poet-engine-0          | [2019-12-21 19:03:06.079 ERROR    base_events] Task exception was never retrieved
sawtooth-poet-engine-0          | future: <Task finished coro=<_SendReceiveThread._monitor_disconnects() done, defined at /usr/lib/python3/dist-packages/sawtooth_sdk/messaging/stream.py:135> exception=CancelledError()>
sawtooth-poet-engine-0          | concurrent.futures._base.CancelledError
sawtooth-poet-engine-0          | [2019-12-21 19:03:06.127 ERROR    base_events] Task exception was never retrieved
sawtooth-poet-engine-0          | future: <Task finished coro=<_SendReceiveThread._monitor_disconnects() done, defined at /usr/lib/python3/dist-packages/sawtooth_sdk/messaging/stream.py:135> exception=CancelledError()>
sawtooth-poet-engine-0          | concurrent.futures._base.CancelledError
sawtooth-poet-engine-0          | [2019-12-21 19:03:06.286 ERROR    engine] Unhandled exception in message loop
sawtooth-poet-engine-0          | Traceback (most recent call last):
sawtooth-poet-engine-0          |   File "/usr/lib/python3/dist-packages/sawtooth_poet_engine/engine.py", line 192, in start
sawtooth-poet-engine-0          |     self._try_to_publish()
sawtooth-poet-engine-0          |   File "/usr/lib/python3/dist-packages/sawtooth_poet_engine/engine.py", line 201, in _try_to_publish
sawtooth-poet-engine-0          |     if self._initialize_block():
sawtooth-poet-engine-0          |   File "/usr/lib/python3/dist-packages/sawtooth_poet_engine/engine.py", line 63, in _initialize_block
sawtooth-poet-engine-0          |     initialize = self._oracle.initialize_block(chain_head)
sawtooth-poet-engine-0          |   File "/usr/lib/python3/dist-packages/sawtooth_poet_engine/oracle.py", line 80, in initialize_block
sawtooth-poet-engine-0          |     return self._publisher.initialize_block(block_header)
sawtooth-poet-engine-0          |   File "/usr/lib/python3/dist-packages/sawtooth_poet/poet_consensus/poet_block_publisher.py", line 448, in initialize_block
sawtooth-poet-engine-0          |     poet_settings_view=poet_settings_view):
sawtooth-poet-engine-0          |   File "/usr/lib/python3/dist-packages/sawtooth_poet/poet_consensus/consensus_state.py", line 701, in validator_has_claimed_block_limit
sawtooth-poet-engine-0          |     key_block_claim_limit = poet_settings_view.key_block_claim_limit
sawtooth-poet-engine-0          |   File "/usr/lib/python3/dist-packages/sawtooth_poet/poet_consensus/poet_settings_view.py", line 182, in key_block_claim_limit
sawtooth-poet-engine-0          |     validate_function=lambda value: value > 0)
sawtooth-poet-engine-0          |   File "/usr/lib/python3/dist-packages/sawtooth_poet/poet_consensus/poet_settings_view.py", line 95, in _get_config_setting
sawtooth-poet-engine-0          |     value_type=value_type)
sawtooth-poet-engine-0          |   File "/usr/lib/python3/dist-packages/sawtooth_poet/state/settings_view.py", line 75, in _get_setting
sawtooth-poet-engine-0          |     SettingsView.setting_address(key))
sawtooth-poet-engine-0          |   File "/usr/lib/python3/dist-packages/sawtooth_poet_engine/oracle.py", line 267, in get
sawtooth-poet-engine-0          |     addresses=[address])
sawtooth-poet-engine-0          |   File "/usr/lib/python3/dist-packages/sawtooth_sdk/consensus/zmq_service.py", line 333, in get_state
sawtooth-poet-engine-0          |     response_type=response_type)
sawtooth-poet-engine-0          |   File "/usr/lib/python3/dist-packages/sawtooth_sdk/consensus/zmq_service.py", line 32, in _send
sawtooth-poet-engine-0          |     ).result(self._timeout).content
sawtooth-poet-engine-0          |   File "/usr/lib/python3/dist-packages/sawtooth_sdk/messaging/future.py", line 38, in content
sawtooth-poet-engine-0          |     raise ValidatorConnectionError()
sawtooth-poet-engine-0          | sawtooth_sdk.messaging.exceptions.ValidatorConnectionError: the connection to the validator was lost
sawtooth-poet-engine-0          | [2019-12-21 19:03:06.540 ERROR    zmq_driver] Uncaught driver exception
sawtooth-poet-engine-0          | Traceback (most recent call last):
sawtooth-poet-engine-0          |   File "/usr/lib/python3/dist-packages/sawtooth_sdk/consensus/zmq_driver.py", line 83, in _driver_loop
sawtooth-poet-engine-0          |     message = future.result(1)
sawtooth-poet-engine-0          |   File "/usr/lib/python3.6/concurrent/futures/_base.py", line 430, in result
sawtooth-poet-engine-0          |     raise CancelledError()
sawtooth-poet-engine-0          | concurrent.futures._base.CancelledError
sawtooth-poet-validator-registry-tp-0 | Task exception was never retrieved
sawtooth-poet-validator-registry-tp-0 | future: <Task finished coro=<_SendReceiveThread._monitor_disconnects() done, defined at /usr/lib/python3/dist-packages/sawtooth_sdk/messaging/stream.py:135> exception=CancelledError()>
sawtooth-poet-validator-registry-tp-0 | concurrent.futures._base.CancelledError
sawtooth-validator-default-1    | [2019-12-21 19:03:07.304 INFO     (unknown file)] [src/journal/chain.rs: 206] Building fork resolution for chain head 'Block(id: b4d266cb53f8760dbcc18fca0615b234ceb7dc944cd435722f7363af118bdf1455a01f41e103a16dc3983b0bbc2960f251fbf3102b977f216cf4758d79fdba48, block_num: 21, state_root_hash: 0b9c7c83b2f7534ba723c16f9e71d0a35f271f485a06c87480f5a4ec46963144, previous_block_id: e5e53884f13e15fe7474ec450150093b3f5976974c4a2529a25e8150d1519a921ae0606de70a63273680500954ace604665e8faaad701bad12caa84b8934aa8b)' against new block 'Block(id: 4a6a1342c829d81752127f5b95deaff1b9cac847ec3675714a2b1b086d281e6f44979beef8506041a89d841e089d8ae36e2513e3f65a91c2e634fa23a4052e06, block_num: 22, state_root_hash: 0b9c7c83b2f7534ba723c16f9e71d0a35f271f485a06c87480f5a4ec46963144, previous_block_id: b4d266cb53f8760dbcc18fca0615b234ceb7dc944cd435722f7363af118bdf1455a01f41e103a16dc3983b0bbc2960f251fbf3102b977f216cf4758d79fdba48)'
influxdb_1                      | [httpd] 172.22.0.3 - - [21/Dec/2019:19:03:09 +0000] "POST /write?db=metrics&precision=s HTTP/1.1" 204 0 "-" "Python-urllib/3.6" 872e224f-2424-11ea-837e-0242ac16000e 68105
influxdb_1                      | [httpd] 172.22.0.4 - - [21/Dec/2019:19:03:09 +0000] "POST /write?db=metrics&precision=s HTTP/1.1" 204 0 "-" "Python-urllib/3.6" 87491b1d-2424-11ea-837f-0242ac16000e 26546
sawtooth-validator-default-0 exited with code 137
sawtooth-rest-api-default-0     | [2019-12-21 19:03:09.607 WARNING  route_handlers] Validator disconnected while waiting for response
influxdb_1                      | [httpd] 172.22.0.5 - - [21/Dec/2019:19:03:09 +0000] "POST /write?db=metrics&precision=s HTTP/1.1" 204 0 "-" "Python-urllib/3.6" 876cfac6-2424-11ea-8380-0242ac16000e 95041
sawtooth-validator-default-2    | [2019-12-21 19:03:00.566 ERROR    threadpool] (Consensus) Unhandled exception during execution of task _HandlerManager.execute.<locals>.wrapped
sawtooth-validator-default-2    | Traceback (most recent call last):
sawtooth-validator-default-2    |   File "/usr/lib/python3/dist-packages/sawtooth_validator/concurrent/threadpool.py", line 83, in wrapper
sawtooth-validator-default-2    |     return_value = fn(*args, **kwargs)
sawtooth-validator-default-2    |   File "/usr/lib/python3/dist-packages/sawtooth_validator/networking/dispatch.py", line 444, in wrapped
sawtooth-validator-default-2    |     return callback(self._handler.handle(connection_id, message))
sawtooth-validator-default-2    |   File "/usr/lib/python3/dist-packages/sawtooth_validator/consensus/handlers.py", line 378, in handle
sawtooth-validator-default-2    |     self._proxy.validate_block(block_id)
sawtooth-validator-default-2    |   File "/usr/lib/python3/dist-packages/sawtooth_validator/consensus/proxy.py", line 225, in validate_block
sawtooth-validator-default-2    |     block = next(self._block_manager.get([block_id]))
sawtooth-validator-default-2    |   File "/usr/lib/python3/dist-packages/sawtooth_validator/ffi.py", line 203, in __next__
sawtooth-validator-default-2    |     payload = from_rust_vec(vec_ptr, vec_len, vec_cap)
sawtooth-validator-default-2    |   File "/usr/lib/python3/dist-packages/sawtooth_validator/ffi.py", line 106, in from_rust_vec
sawtooth-validator-default-2    |     py_bytes = bytes(vec_ptr[:vec_len.value])
sawtooth-validator-default-2    | MemoryError

我使用带有诗人算法的Hyperledger Sawtooth 1.2（风铃），influxdb和graphana。我创建了一个脚本来评估服务器上的CPU和RAM消耗，并在图像CPU and RAM consumption中观察结果。 CPU and RAM

此外，我使用HTOP工具对系统进行了分析，甚至SWAP也被完全消耗，如this figure所示，其中显示了发生错误之前的消耗时刻。 HTOP

在常见问题解答（https://sawtooth.hyperledger.org/faq/validator/）中，我发现以下说法：

  

锯齿将区块链存储在LMDB数据库中   /var/lib/Sawtooth/block-00.lmdb。 LMDB数据库是一个“稀疏”文件   表示在使用之前，不会为文件分配存储空间（已写入   至）。只要文件系统数据库不应该用完内存   有可用的存储空间。内存错误可能发生在Kubernetes或   没有Docker或其他虚拟机环境   存储卷已映射到VM。

由于我在同一台服务器上的Docker容器中运行三个验证器，因此我想问题是由于LMDB，事务未存储在磁盘上，而是保留在RAM中。

如何解决这个问题？

0 个答案:

没有答案
相关问题
最新问题