是否有shell命令来延迟缓冲区?

时间:2014-01-07 19:04:39

标签: bash shell

我正在寻找一个shell命令X,例如,当我执行:

command_a | X 5000 | command_b

stdout的{​​{1}}写在command_a stdin(至少)5秒后。

一种延迟缓冲。

据我所知,command_b / buffer可以以恒定的速率(每秒固定的字节数)写入。相反,我想要一个恒定的延迟及时(t = 0是mbuffer读取X输出块时,在t = 5000时它必须将此块写入{{ 1}})。

[edit] 我已经实现了它:https://github.com/rom1v/delay

5 个答案:

答案 0 :(得分:7)

我知道你说你正在寻找一个shell命令,但是如何使用子shell来获得优势呢?类似的东西:

command_a | (sleep 5; command_b)

所以grep文件cat - 通过(我知道,我知道,使用cat,但只是一个例子):

cat filename | (sleep 5; grep pattern)

更完整的例子:

$ cat testfile
The
quick
brown
fox
$ cat testfile | (sleep 5; grep brown)
# A 5-second sleep occurs here
brown

或者,正如Michale Kropat所建议的那样,使用sleep的群组命令也可以工作(并且可以说更正确)。像这样:

$ cat testfile | { sleep 5; grep brown; }

注意:不要忘记命令后面的分号(这里是grep brown),因为它是必要的!

答案 1 :(得分:2)

因为看起来这样的命令不存在,我在C中实现了它: https://github.com/rom1v/delay

delay [-b <dtbufsize>] <delay>

答案 2 :(得分:1)

你的问题引起了我的兴趣,我决定回来玩它。这是Perl中的基本实现。它可能不是可移植的(ioctl),仅在Linux上测试过。

基本理念是:

  • 每X微秒读取可用输入
  • 将每个输入块存储在哈希中,并将当前时间戳作为键
  • 还将当前时间戳推送到队列(数组)
  • 在队列中查找最旧的时间戳,如果延迟足够长,则从散列中写入+丢弃数据
  • 重复

最大缓冲区大小

存储数据的最大大小。如果达到,则在写入后空间可用时,将不会读取其他数据。

<强>性能

对于您的要求(几Mb / s)可能不够快。我的最大吞吐量是639 Kb / s,见下文。

<强>测试

# Measure max throughput:
$ pv < /dev/zero | ./buffer_delay.pl > /dev/null

# Interactive manual test, use two terminal windows:
$ mkfifo data_fifo
terminal-one $ cat > data_fifo
terminal-two $ ./buffer_delay.pl < data_fifo

# now type in terminal-one and see it appear delayed in terminal-two.
# It will be line-buffered because of the terminals, not a limitation 
# of buffer_delay.pl

<强> buffer_delay.pl

#!/usr/bin/perl
use strict;
use warnings;
use IO::Select;
use Time::HiRes qw(gettimeofday usleep);
require 'sys/ioctl.ph';

$|++;

my $delay_usec = 3 * 1000000; # (3s) delay in microseconds
my $buffer_size_max = 10 * 1024 * 1024 ; # (10 Mb) max bytes our buffer is allowed to contain.
                              # When buffer is full, incoming data will not be read
                              # until space becomes available after writing
my $read_frequency = 10;      # Approximate read frequency in Hz (will not be exact)

my %buffer;                   # the data we are delaying, saved in chunks by timestamp
my @timestamps;               # keys to %buffer, used as a queue
my $buffer_size = 0;          # num bytes currently in %buffer, compare to $buffer_size_max

my $time_slice = 1000000 / $read_frequency; # microseconds, min time for each discrete read-step

my $sel = IO::Select->new([\*STDIN]);
my $overflow_unread = 0;      # Num bytes waiting when $buffer_size_max is reached

while (1) {
    my $now = sprintf "%d%06d", gettimeofday;  # timestamp, used to label incoming chunks

    # input available?
    if ($overflow_unread || $sel->can_read($time_slice / 1000000)) {

        # how much?
        my $available_bytes;
        if ($overflow_unread) {
            $available_bytes = $overflow_unread;
        }
        else {
            $available_bytes = pack("L", 0);
            ioctl (STDIN, FIONREAD(), $available_bytes);
            $available_bytes = unpack("L", $available_bytes);
        }

        # will it fit?
        my $remaining_space = $buffer_size_max - $buffer_size;
        my $try_to_read_bytes = $available_bytes;
        if ($try_to_read_bytes > $remaining_space) {
            $try_to_read_bytes = $remaining_space;
        }

        # read input
        if ($try_to_read_bytes > 0) {
            my $input_data;
            my $num_read = read (STDIN, $input_data, $try_to_read_bytes);
            die "read error: $!" unless defined $num_read;
            exit if $num_read == 0;       # EOF
            $buffer{$now} = $input_data;  # save input
            push @timestamps, $now;       # save the timestamp
            $buffer_size += length $input_data;
            if ($overflow_unread) {
                $overflow_unread -= length $input_data;
            }
            elsif (length $input_data < $available_bytes) {
                $overflow_unread = $available_bytes - length $input_data;
            }
        }
    }

    # write + delete any data old enough
    my $then = $now - $delay_usec; # when data is old enough
    while (scalar @timestamps && $timestamps[0] < $then) {
        my $ts = shift @timestamps;
        print $buffer{$ts} if defined $buffer{$ts};
        $buffer_size -= length $buffer{$ts};
        die "Serious problem\n" unless $buffer_size >= 0;
        delete $buffer{$ts};
    }

    # usleep any remaining time up to $time_slice
    my $time_left = (sprintf "%d%06d", gettimeofday) - $now;
    usleep ($time_slice - $time_left) if $time_slice > $time_left;
}

欢迎在下面发表评论和建议!

答案 3 :(得分:0)

这样的东西?

#!/bin/bash
while :
do
   read line 
   sleep 5
   echo $line
done

将文件另存为“slowboy”,然后执行

chmod +x slowboy

并以

运行
command_a | ./slowboy | command_b

答案 4 :(得分:0)

这可能有效

time_buffered () {
   delay=$1
   while read line; do
       printf "%d %s\n" "$(date +%s)" "$line"
   done | while read ts line; do
       now=$(date +%s)
       if (( now - ts < delay)); then
           sleep $(( now - ts ))
       fi
       printf "%s\n" "$line"
   done
}

commandA | time_buffered 5 | commandB

第一个循环用时间戳标记其输入的每一行,并立即将其提供给第二个循环。第二个循环检查每一行的时间戳,并在必要时休眠,直到第一次读取之后$delay秒,然后输出该行。