我的任务是使用外部实用程序(addr2line)过滤perl脚本中的一些数据。数据量非常大。我需要将大量数据打印到程序的stdin
并读回大量数据(从程序的stdout
到我的脚本)。
现在我用IPC::Open2
执行此操作,但我不会混合阅读和写作。这合法吗?请Open2
缓冲管道中的任何大小的数据吗?
我的代码:
my $cmd="addr2line -e $prog_name ";
use IPC::Open2;
local (*Reader, *Writer);
my $pid = open2(\*Reader, \*Writer, $cmd);
for(@requests) { # this array is HUGE, 100s of thousands of entries
print Writer "$_\n";
}
close Writer;
for(@requests) {
$function_name = <Reader>;
$filesource = <Reader>;
#... store ..
}
close Reader;
waitpid($pid,0);
答案 0 :(得分:3)
是的,您将以编写程序的方式遇到缓冲区容量限制。您的输入缓冲区(Reader
)将填满并阻止外部程序的执行。
混合读取和写入会有所帮助,因为您将以与外部程序填充它相同的速率清空输入缓冲区。
另一件有用的事情是使用文件进行进程间通信而不是管道或套接字(如IPC::Open2
那样)。然后,您将受限于可用磁盘空间量。你可以自己做,虽然Forks::Super
默认使用IPC文件。
use Forks::Super 'open2';
...
my ($Reader,$Writer,$pid) = open2(@command);
for (@requests) { print $Writer "$_\n" }
close $Writer;
for (@requests) { ... read ... }
close $Reader;
waitpid $pid,0;
答案 1 :(得分:3)
管道尺寸有限。你的方法会陷入僵局
Parent Child
------ -----
... ...
Wait for data in Writer
Put data in Writer
Read data from Writer
Put data in Reader
Wait for data in Writer
Put data in Writer
Read data from Writer
Put data in Reader
=> Blocks cause Reader is full
Put data in Writer
Put data in Writer
...
Put data in Writer
Put data in Writer
=> Blocks cause Writer is full
一种可能的解决方案:
use strict;
use warnings;
use threads;
use IPC::Open2 qw( open2 );
my @cmd = ("addr2line", "-e", $prog_name);
local (*Reader, *Writer);
my $pid = open2(\*Reader, \*Writer, @cmd);
my $thread = async {
for (;;) {
$function_name = <Reader>;
last if !defined($function_name);
$filesource = <Reader>;
#... store ..
}
close Reader;
};
{
my @requests = ...;
for(@requests) { # this array is HUGE, 100s of thousands of entries
print Writer "$_\n";
}
close Writer;
}
$thread->join();
waitpid($pid, 0);
或者,IPC::Run还有一些工具可以使这一切变得简单。
unixy方式是使用IO::Select,但这是一个真正的痛苦。