如何正确限制goroutines的数量

时间:2017-04-27 12:08:15

标签: go goroutine limiting

我进入了'stdin'的URL行: $ echo -e'https://golang.org \ nhttps://godoc.org \ nhttps://golang.org'|去运行1.go。 任务是从每个WEB页面的单词“Go”获取。但我不允许启动超过5个goroutines并且只能使用标准库 这是我的代码:

    package main

    import (
      "fmt"
      "net/http"
      "bufio"
      "os"
      "regexp"
      "io/ioutil"
      "time"
    )

func worker(id int, jobs<-chan string, results chan<-int) {
  t0 := time.Now()
  for url := range jobs {
    resp, err := http.Get(url)
    if err != nil {
      fmt.Println("problem while opening url", url)
      results<-0
      //continue
    }
    defer resp.Body.Close()
    html, err := ioutil.ReadAll(resp.Body)
    if err != nil {
      continue
    }
    regExp:= regexp.MustCompile("Go")
    matches := regExp.FindAllStringIndex(string(html), -1)
    t1 := time.Now()
    fmt.Println("Count for", url, ":", len(matches), "Elapsed time:", 
t1.Sub(t0),  "works id", id)
    results<-len(matches)
  }
}

func main(){
  scanner := bufio.NewScanner(os.Stdin)
  jobs := make(chan string, 100)
  results := make(chan int, 100)
  t0 := time.Now()
  for w:= 0; w<5; w++{
    go worker(w, jobs, results)
  }
  var tasks int = 0
  res := 0
  for scanner.Scan() {
      jobs <- scanner.Text()
      tasks ++
  }
  close(jobs)
  for a := 1; a <= tasks; a++ {
    res+=<-results
  }
  close(results)
  t2 := time.Now()
  fmt.Println("Total:",res, "Elapsed total time:", t2.Sub(t0) );
}

我认为直到我将超过5个URL(其中一个不正确)传递给stdin才有效。输出是:

 goroutine 9 [running]:
 panic ...

显然,已经开始了额外的goroutnes。怎么解决?可能有更方便的方法来限制goroutines的数量?

1 个答案:

答案 0 :(得分:1)

  

goroutine 9 [跑步]:

一些goroutine由运行时启动,并通过Web提取启动。

查看您的代码,您只启动了5个goroutines。

如果您真的想知道正在运行的例程,请使用runtime.Numgoroutine