Golang同步Goroutines

时间:2018-07-28 22:39:32

标签: go ssh concurrency parallel-processing goroutine

我正在开发用于同时配置网络设备的SSH客户端应用程序,并且在实现并发方面遇到问题。我的程序吸收了一部分主机和一条配置命令以发送到每个主机。我正在使用sync.WaitGroup等待所有主机完成配置。这对于少量主机来说可以正常工作,但是很快我的配置goroutine中的功能便开始随机出现故障。如果我在出现故障的主机上重新运行该程序,则某些将成功,某些将再次失败。我必须重复此过程,直到仅保留具有实际错误的主机为止。它总是失败,或者说authentication failed: auth methods tried [none password]...的认证,或者sysDescr中的值未添加到某些Device字段中。好像有许多主机和goroutine正在运行时,它们开始提前返回或返回某种东西。我真的不确定发生了什么。

这是我的代码示例:

package main

import (
    "fmt"
    "net"
    "os"
    "sync"
    "time"

    "golang.org/x/crypto/ssh"
)

func main() {
    // When there are many hosts, many calls to Dial and
    // sysDescr fail. If I rerun the program on the unsuccessful
    // hosts, nothing fails and the expected output is produced.
    var hosts []string
    cfg := &ssh.ClientConfig{
        User:            "user",
        Auth:            []ssh.AuthMethod{ssh.Password("pass")},
        HostKeyCallback: ssh.InsecureIgnoreHostKey(),
        Timeout:         10 * time.Second,
    }
    results := make(chan *result, len(hosts))

    var wg sync.WaitGroup
    wg.Add(len(hosts))
    for _, host := range hosts {
        go connect(host, cfg, results, &wg)
    }
    wg.Wait()
    close(results)

    for res := range results {
        if res.err != nil {
            fmt.Fprintln(os.Stderr, res.Err)
            continue
        }
        d := res.device
        fmt.Println(d.addr, d.hostname, d.vendor, d.os, d.model, d.version)
        d.Client.Close()
    }
}

// Device represents a network device.
type Device struct {
    *ssh.Client
    addr     string
    hostname string
    vendor   string
    os       string
    model    string
    version  string
}

// Dial establishes an ssh client connection to a remote host.
func Dial(host, port string, cfg *ssh.ClientConfig) (*Device, error) {
    // get host info in background, may take a second
    info := make(chan map[string]string)
    go func(c *Client) {
        info <- sysDescr(host)
        close(info)
    }(c)

    // establish ssh client connection to host
    client, err := ssh.Dial("tcp", net.JoinHostPort(host, addr), cfg)
    if err != nil {
        return nil, err
    }

    m := <-info
    d := &Device{
        Client: client,
        addr: m["addr"],
        hostname: m["hostname"],
        vendor: m["vendor"],
        os: m["os"],
        model: m["model"],
        version: m["version"],
    }
    return d, nil
}

// sysDescr attempts to gather information about a remote host.
func sysDescr(host string) map[string]string {
    // get host info
}

// result is the result of connecting to a device.
type result struct {
    device *Device
    err    error
}

// connect establishes an ssh client connection to a host.
func connect(host string, cfg *ssh.ClientConfig, results chan<- *result, wg *sync.WaitGroup) {
    defer wg.Done()
    device, err := Dial(host, "22", cfg)
    results <- &result{device, err}
}

我做错什么了吗?我是否可以限制产生的goroutine的数量,而不是为每个主机都产生goroutine?

1 个答案:

答案 0 :(得分:1)

是的!要回答您的第二个问题,可以使用许多模式来限制并发。最大的两个是:

我个人更喜欢工作池类型,因为IMO可以更好地帮助使业务逻辑与计划分开,但两者都是完全有效的,并且在许多项目中普遍存在。