我正在尝试在网页中实施反馈，以使用户可以从Excel工作表开始漫长的过程（看，是的...）。对于每行数据，处理时间约为1秒，并且公共数据长度在40到100个项目之间，因此总处理时间可以大于一分钟。
我正在显示页面中数据的预览，并通过websocket启动该过程，并希望显示同一websocket的进度。
处理本身是由外部程序包进行的，页面的复杂性极低，因此我将其包装在一个Lite单个文件中。
我的问题是，在websocket路由中开始的长时间处理阻碍了反馈，直到反馈完成为止，并且所有进行事件都在结束时同时发送。据我了解，这与Mojolicious的事件循环有关，我应该分开开始处理，以避免冻结websocket的处理。
请注意，我已经尝试使用EventSource的单独反馈渠道在处理过程中向客户推送一些进度，但是最后一次显示相同的完成情况。
这是我的代码简化了，我正在使用sleep()模拟长时间的过程。我从
perl mojo_notify_ws.pl daemon
您能建议如何修改网络套接字路由以允许实时反馈吗？
use Mojolicious::Lite;
use Mojo::JSON qw(encode_json decode_json j);
use Data::Dumper;
$|++;
any '/' => sub {
my $c = shift;
$c->render('index');
};
my $peer;
websocket '/go' => sub {
use Carp::Always;
my $ws = shift;
$peer = $ws->tx;
app->log->debug(sprintf 'Client connected: %s', Dumper $peer->remote_address);
# do not subscribe to 'text' else 'json' won't work
#$ws->on(text => sub {
# my ($ws, $msg) = @_;
# app->log->debug("Received text from websocket: `$msg`");
# });
# $peer->send('{"type": "test"}');
# say 'default inactivity timeout='. (p $ws->inactivity_timeout());
$ws->inactivity_timeout(120);
$ws->on(json => sub {
my ($ws, $msg) = @_;
app->log->debug('Received from websocket:', Dumper(\$msg));
unless($msg){
app->log->debug('Received empty message? WTF?!');
return;
}
my $prompt = $msg->{cmd};
return unless $prompt;
app->log->debug(sprintf 'Received: `%s`', $prompt // 'empty??');
# simulate
my $loop = Mojo::IOLoop->singleton;
# $loop->subprocess( sub {
# my $sp = shift;
for my $cell (1..3) {
# $loop->delay( sub {
app->log->debug("sending cell $cell");
my $payload = {
type => 'ticket',
cell => $cell,
result => $cell % 2 ? 'OK' : 'NOK'
};
$ws->send( { json => $payload } );
sleep(2);
# $loop->timer(2, sub {say 'we have waited 2 secs!';})->wait;
# });
};
# }, sub {} );#subprocess
app->log->debug('sending end of process ->websocket');
$ws->send({json => { type => 'end' } });
});
$ws->on(finish => sub {
my ($ws, $code, $reason) = @_;
$reason = '' unless defined $reason;
app->log->debug("Client disconnected: $code ($reason)");
});
app->log->debug('Reached end of ws route definition');
};
app->start;
__DATA__
@@ index.html.ep
<html>
<head>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.js"></script>
<script>
var timerID = 0;
function keepAlive(ws) {
var timeout = 20000;
if (ws.readyState == ws.OPEN) {
ws.send('ping');
}
timerId = setTimeout(function(){keepAlive(ws);}, timeout);
}
function cancelKeepAlive() {
if (timerId) {
clearTimeout(timerId);
}
}
function flagCell(cell, result){
var id='#CELL_' + cell;
var cell = $(id);
if(cell) {
if (result=='OK') {
cell.css('color', 'green');
cell.text('⯲');
} else {
cell.css('color','red');
cell.text('✘');
}
}
}
function process(){
//debugger;
console.log('Opening WebSocket');
var ws = new WebSocket('<%= url_for('go')->to_abs %>');
ws.onopen = function (){
console.log('Websocket Open');
//keepAlive(ws);
ws.send(JSON.stringify({cmd: "let's go Perl"}));
};
//incoming
ws.onmessage = function(evt){
var data = JSON.parse(evt.data);
console.log('WS received '+JSON.stringify(data));
if (data.type == 'ticket') {
console.log('Server has send a status');
console.log('Cell:'+data.cell + ' res:' + data.result);
flagCell(data.cell, data.result);
} else if (data.type == 'end') {
console.log('Server has finished.');
//cancelKeepAlive();
ws.close();
} else {
console.log('Unknown message:' + evt.data);
}
};
ws.onerror = function (evt) {
console.log('ws error:', evt.data);
}
ws.onclose = function (evt) {
if(evt.wasClean) {
console.log('Connection closed cleanly');
} else {
console.log('Connection reseted');
}
console.log('Code:'+ evt.code + ' Reason:' + evt.reason);
}
}
</script>
</head>
<body>
<button type=button id='upload' onclick="process();">Go</button><br>
<div style='font-family:sans;'>
<table border="1px">
<tr><td id="CELL_1"> </td><td>Foo</td></tr>
<tr><td id="CELL_2"> </td><td>Bar</td></tr>
<tr><td id="CELL_3"> </td><td>Baz</td></tr>
</table>
</div>
</body>
</html>
编辑：
Grinnz提供了一个合适的解决方案，但为记录起见，这是我尝试使用Mojo::IOLoop::Subprocess回调，但随后我没有任何反馈。我在Linux上运行，Subprocess似乎分叉，并且父进程似乎立即终止了websocket 编辑：否：我最终发现{ {1}}放置在错误的位置，因为它应该放置在在父端运行的第二个$ws->send()中，而不是在子进程中运行的第一个sub{}中。应当将此代码重构为每个循环迭代有一个subprocess，并具有结束通知的最后一步。
这是经过修改的on(json)
$ws->on(json => sub {
my ($ws, $msg) = @_;
app->log->debug('Received from websocket:', Dumper(\$msg));
unless($msg){
app->log->debug('Received empty message? WTF?!');
return;
}
my $prompt = $msg->{cmd};
return unless $prompt;
app->log->debug(sprintf 'Received: `%s`', $prompt // '<empty??>');
# my $loop = Mojo::IOLoop->singleton;
my $subprocess = Mojo::IOLoop::Subprocess->new;
app->log->debug("we are pid $$");
$subprocess->run(
sub {
my $sp = shift;
for my $cell (1..3) {
app->log->debug("starting process for cell $cell in pid $$");
sleep(2);
app->log->debug("sending cell $cell to ws");
my $payload = {
type => 'ticket',
cell => $cell,
result => $cell % 2 ? 'OK' : 'NOK'
};
$ws->send( { json => $payload } ); # FIXME: actually this line is in the wrong place
# and should be in the second sub{}
};
},
sub {
my ($sp, $err, @results) = @_;
$ws->reply->exception($err) and return if $err;
app->log->debug('sending end of process ->websocket');
$ws->send({json => { type => 'end' } });
});
# Start event loop if necessary
$subprocess->ioloop->start unless $subprocess->ioloop->is_running;
});
以及相应的日志：
[Wed Oct 3 19:51:58 2018] [debug] Received: `let's go Perl`
[Wed Oct 3 19:51:58 2018] [debug] we are pid 8898
[Wed Oct 3 19:51:58 2018] [debug] Client disconnected: 1006 ()
[Wed Oct 3 19:51:58 2018] [debug] starting process for cell 1 in pid 8915
[Wed Oct 3 19:52:00 2018] [debug] sending cell 1 to ws
[Wed Oct 3 19:52:00 2018] [debug] starting process for cell 2 in pid 8915
[Wed Oct 3 19:52:02 2018] [debug] sending cell 2 to ws
[Wed Oct 3 19:52:02 2018] [debug] starting process for cell 3 in pid 8915
[Wed Oct 3 19:52:04 2018] [debug] sending cell 3 to ws
[Wed Oct 3 19:52:04 2018] [debug] sending end of process ->websocket
[Wed Oct 3 19:52:04 2018] [debug] Client disconnected: 1005 ()
我还尝试了Mojo::IOLoop->delay，以类似于Promise解决方案的方式生成了一系列复杂的步骤，但是该方法最后还要一次发送所有通知：
$ws->on(json => sub {
my ($ws, $msg) = @_;
app->log->debug('Received from websocket:', Dumper(\$msg));
unless($msg){
app->log->debug('Received empty message? WTF?!');
return;
}
my $prompt = $msg->{cmd};
return unless $prompt;
app->log->debug(sprintf 'Received: `%s`', $prompt // '<empty??>');
app->log->debug("we are pid $$");
my @steps;
for my $cell (1..3) {
push @steps,
sub {
app->log->debug("subprocess for cell pid $cell");
# my $sp = shift;
my $delay = shift;
sleep(2);
app->log->debug("end of sleep for cell $cell");
$delay->pass($cell % 2 ? 'OK' : 'NOK');
},
sub {
my $delay = shift;
my $result = shift;
app->log->debug("sending cell $cell from pid $$ - result was $result");
my $payload = {
type => 'ticket',
cell => $cell,
result => $result
};
$ws->send( { json => $payload } );
$delay->pass;
};
}
# add final step to notify end of processing
push @steps, sub {
my $delay = shift;
app->log->debug('sending end of process ->websocket');
$ws->send({json => { type => 'end' } });
$delay->pass;
};
my $delay = Mojo::IOLoop::Delay->new;
app->log->debug("Starting delay...");
$delay->steps( @steps );
app->log->debug("After the delay");
});
答案 0 :(得分：3)
您可以使用线程而不是子进程来完成工作。创建线程后，您需要一个循环，该循环通过websocket更新进度。
如果要处理在所有情况下都必须完成的关键工作负载（Websocket消失，网络关闭等），则应将其委派给另一个保留的守护程序，并通过文件或套接字传达其状态。 / p>
如果这不是很重要的工作负载，并且您可以轻松地重新启动它，那么这可能就是您的模板。
# Insert this at module header
# use threads;
# use Thread::Queue;
my $queue = Thread::Queue->new();
my $worker = threads->create(sub {
# dummy workload. do your work here
my $count = 60;
for (1..$count) {
sleep 1;
$queue->enqueue($_/$count);
}
# undef to signal end of work
$queue->enqueue(undef);
return;
});
# blocking dequeuing ends when retrieving an undef'd value
while(defined(my $item = $queue->dequeue)) {
# update progress via websocket
printf("%f %\n", $item);
}
# join thread
$worker->join;
答案 1 :(得分：3)
It is not possible to magically make Perl code non-blocking.这就是为什么您的阻止操作阻止了websocket响应和事件循环。
单个子进程将无法工作，因为只有处理请求的原始工作进程才能响应websocket，而子进程只能返回一次。但是，您可以使用子流程来准备要发送的每个响应。但是，您对子流程的使用不太正确。
传递给子流程的第一个子例程在fork中执行，因此不会阻塞主流程。一旦子进程完成，第二个子例程将在父级中执行，并接收第一个子例程的返回值。这是您需要发送回复的地方。
除此之外的任何代码都将在子进程启动之前执行，因为这是异步代码，因此您需要通过回调对逻辑进行排序。您可以使用promises简化复杂的排序过程。
use Mojo::Promise;
$ws->on(json => sub {
my ($ws, $msg) = @_;
app->log->debug('Received from websocket:', Dumper(\$msg));
unless($msg){
app->log->debug('Received empty message? WTF?!');
return;
}
my $prompt = $msg->{cmd};
return unless $prompt;
app->log->debug(sprintf 'Received: `%s`', $prompt // 'empty??');
my $promise = Mojo::Promise->new->resolve; # starting point
# attach follow-up code for each cell, returning a new promise representing the whole chain so far
for my $cell (1..3) {
$promise = $promise->then(sub {
my $promise = Mojo::Promise->new;
Mojo::IOLoop->subprocess(sub {
app->log->debug("sending cell $cell");
sleep(2);
my $payload = {
type => 'ticket',
cell => $cell,
result => $cell % 2 ? 'OK' : 'NOK'
};
return $payload;
}, sub {
my ($sp, $err, $payload) = @_;
return $promise->reject($err) if $err; # indicates subprocess died
$ws->send( { json => $payload }, sub { $promise->resolve } );
});
# here, the subprocess has not been started yet
# it will be started when this handler returns to the event loop
# then the second callback will run once the subprocess exits
return $promise;
};
}
# chain from last promise
$promise->then(sub {
app->log->debug('sending end of process ->websocket');
$ws->send({json => { type => 'end' } });
})->catch(sub {
my $err = shift;
# you can send or log something here to indicate an error occurred in one of the subprocesses
});
});
如果合适，我可以进一步介绍一些其他选项：Mojo::IOLoop::ReadWriteFork，它将使您仅启动一个子进程并不断从中接收STDOUT（您需要自己对有效负载进行序列化以将其发送给STDOUT，就像Mojo :: JSON一样；或通过两个过程都可以连接的外部发布/订阅代理将状态信息发送回父级的常规子过程，例如Postgres，Redis或Mercury（也需要序列化）。
答案 2 :(得分：2)
我对您的更新示例进行了小幅更改，以使其按预期工作。您可以使用progress模块的Subprocess功能来确保从长子进程异步地通过websocket发送正确的数据。
代码现在按我的预期工作，每当子进程进行一次迭代时，表状态就会在客户端进行更新。
源代码的相关部分如下所示：
$ws->on(
json => sub {
my ( $ws, $msg ) = @_;
app->log->debug( 'Received from websocket:', Dumper( \$msg ) );
unless ($msg) {
app->log->debug('Received empty message? WTF?!');
return;
}
my $prompt = $msg->{cmd};
return unless $prompt;
app->log->debug( sprintf 'Received: `%s`', $prompt // '<empty??>' );
# my $loop = Mojo::IOLoop->singleton;
my $subprocess = Mojo::IOLoop::Subprocess->new;
app->log->debug("we are pid $$");
$subprocess->run(
sub {
my $sp = shift;
for my $cell ( 1 .. 3 ) {
app->log->debug(
"starting process for cell $cell in pid $$");
sleep(2);
app->log->debug("sending cell $cell to ws");
my $payload = {
type => 'ticket',
cell => $cell,
result => $cell % 2 ? 'OK' : 'NOK'
};
$sp->progress($payload);
}
},
sub {
my ( $sp, $err, @results ) = @_;
#$ws->send( { json => $payload } );
$ws->reply->exception($err) and return if $err;
app->log->debug('sending end of process ->websocket');
$ws->send( { json => { type => 'end' } } );
}
);
# Start event loop if necessary
$subprocess->on(
progress => sub {
my ( $subprocess, $payload ) = @_;
$ws->send( { json => $payload } );
}
);
$subprocess->ioloop->start unless $subprocess->ioloop->is_running;
}
);