Question

我正在尝试同时从多个docker容器中获取日志（顺序无关紧要）。如果types.ContainerLogsOption.Follow设置为false，则按预期工作。

如果types.ContainerLogsOption.Follow设置为true，有时日志输出会在几个日志后卡住，并且没有后续日志打印到stdout。

如果输出没有卡住，它会按预期工作。

此外，如果我重新启动一个或所有容器，命令不会像docker logs -f containerName那样退出。

func (w *Whatever) Logs(options LogOptions) {
    readers := []io.Reader{}

    for _, container := range options.Containers {
        responseBody, err := w.Docker.Client.ContainerLogs(context.Background(), container, types.ContainerLogsOptions{
            ShowStdout: true,
            ShowStderr: true,
            Follow:     options.Follow,
        })
        defer responseBody.Close()

        if err != nil {
            log.Fatal(err)
        }
        readers = append(readers, responseBody)
    }

    // concatenate all readers to one
    multiReader := io.MultiReader(readers...)

    _, err := stdcopy.StdCopy(os.Stdout, os.Stderr, multiReader)
    if err != nil && err != io.EOF {
        log.Fatal(err)
    }
}

基本上我的实现与docker logs https://github.com/docker/docker/blob/master/cli/command/container/logs.go没有太大区别，因此我想知道导致这个问题的原因。

Answer 1

正如JimB评论的那样，由于io.MultiReader的操作，该方法不起作用。您需要做的是从每个响应中分别读取每个响应并组合输出。由于您正在处理日志，因此分解换行符是有意义的。 bufio.Scanner为单个io.Reader执行此操作。因此，一种选择是创建一种同时扫描多个读取器的新类型。

您可以像这样使用它：

scanner := NewConcurrentScanner(readers...)
for scanner.Scan() {
    fmt.Println(scanner.Text())
}
if err := scanner.Err(); err != nil {
    log.Fatalln(err)
}

并发扫描程序的示例实现：

// ConcurrentScanner works like io.Scanner, but with multiple io.Readers
type ConcurrentScanner struct {
    scans  chan []byte   // Scanned data from readers
    errors chan error    // Errors from readers
    done   chan struct{} // Signal that all readers have completed
    cancel func()        // Cancel all readers (stop on first error)

    data []byte // Last scanned value
    err  error
}

// NewConcurrentScanner starts scanning each reader in a separate goroutine
// and returns a *ConcurrentScanner.
func NewConcurrentScanner(readers ...io.Reader) *ConcurrentScanner {
    ctx, cancel := context.WithCancel(context.Background())
    s := &ConcurrentScanner{
        scans:  make(chan []byte),
        errors: make(chan error),
        done:   make(chan struct{}),
        cancel: cancel,
    }

    var wg sync.WaitGroup
    wg.Add(len(readers))

    for _, reader := range readers {
        // Start a scanner for each reader in it's own goroutine.
        go func(reader io.Reader) {
            defer wg.Done()
            scanner := bufio.NewScanner(reader)

            for scanner.Scan() {
                select {
                case s.scans <- scanner.Bytes():
                    // While there is data, send it to s.scans,
                    // this will block until Scan() is called.
                case <-ctx.Done():
                    // This fires when context is cancelled,
                    // indicating that we should exit now.
                    return
                }
            }
            if err := scanner.Err(); err != nil {
                select {
                case s.errors <- err:
                    // Reprort we got an error
                case <-ctx.Done():
                    // Exit now if context was cancelled, otherwise sending
                    // the error and this goroutine will never exit.
                    return
                }
            }
        }(reader)
    }

    go func() {
        // Signal that all scanners have completed
        wg.Wait()
        close(s.done)
    }()

    return s
}

func (s *ConcurrentScanner) Scan() bool {
    select {
    case s.data = <-s.scans:
        // Got data from a scanner
        return true
    case <-s.done:
        // All scanners are done, nothing to do.
    case s.err = <-s.errors:
        // One of the scanners error'd, were done.
    }
    s.cancel() // Cancel context regardless of how we exited.
    return false
}

func (s *ConcurrentScanner) Bytes() []byte {
    return s.data
}

func (s *ConcurrentScanner) Text() string {
    return string(s.data)
}

func (s *ConcurrentScanner) Err() error {
    return s.err
}

以下是Go Playground中的一个示例：https://play.golang.org/p/EUB0K2V7iT

您可以看到并发扫描程序输出是交错的。而不是阅读所有一个读者，然后转到下一个，如io.MultiReader所见。

多个泊坞窗容器日志

1 个答案: