在Go中实现作业工作者池

时间:2016-02-21 23:35:38

标签: go queue threadpool

由于Go没有泛型,所有预制解决方案都使用我不太喜欢的类型转换。我也希望自己实现它并尝试以下代码。但是,有时它不会等待所有goroutines,我是否过早关闭了工作渠道?我没有任何东西要从他们那里取。我可能也使用了伪输出通道并等待从它们获取确切的数量但是我相信以下代码也应该工作。我错过了什么?

THREADS_PER_INCH = 4
STEPS_PER_REV = 400

Current_steps = 0

def move(n):
    """
    Given an input, n, measured in inches of position from zero, 
    compute the number of steps required to move the {WHAT IS IT?}
    from its current position to the target position. Print the 
    number of clockwise/counterclockwise steps required.
    """
    global Current_steps
    target_inches = float(n)
    target_steps = int(target_inches * THREADS_PER_INCH * STEPS_PER_REV)

    if target_steps == Current_steps:
        print("Already here! Nothing to do...")
    elif target_steps > Current_steps:
        print("Turn CCW, {} steps".format(target_steps - Current_steps))
    else: # target_steps < Current_steps
        print("Turn CW, {} steps".format(Current_steps - target_steps))

    Current_steps = target_steps

move(.5)    # move up 1/2 of inch
move(.125)  

2 个答案:

答案 0 :(得分:1)

在goroutine外部调用wg.Add并将指针传递给等待组。

如果从goroutine内部调用了Add,那么主goroutine可以在goroutines有机会运行之前调用Wait。如果尚未调用Add,则Wait将立即返回。

将指针传递给goroutine。否则,goroutines会使用自己的等待组副本。

func jobWorker(id int, jobs <-chan string, wg *sync.WaitGroup) {

    defer wg.Done()

    for job := range jobs {
        item := ParseItem(job)
        item.SaveItem()
        MarkJobCompleted(item.ID)
        log.Println("Saved", item.Title)
    }
}

// ProcessJobs processes the jobs from the list and deletes them
func ProcessJobs() {

    jobs := make(chan string)

    list := GetJobs()
    // Start workers
    var wg sync.WaitGroup
    for w := 0; w < 10; w++ {
        wg.Add(1)
        go jobWorker(w, jobs, &wg)
    }

    for _, url := range list {
        jobs <- url
    }

    close(jobs)
    wg.Wait()
}

答案 1 :(得分:1)

您需要将指针传递给waitgroup,否则每个作业都会收到它自己的副本。

func jobWorker(id int, jobs <-chan string, wg *sync.WaitGroup) {
    wg.Add(1)
    defer wg.Done()

    for job := range jobs {
        item := ParseItem(job)
        item.SaveItem()
        MarkJobCompleted(item.ID)
        log.Println("Saved", item.Title)
    }
}

// ProcessJobs processes the jobs from the list and deletes them
func ProcessJobs() {

    jobs := make(chan string)

    list := GetJobs()
    // Start workers
    var wg sync.WaitGroup
    for w := 0; w < 10; w++ {
        go jobWorker(w, jobs, &wg)
    }

    for _, url := range list {
        jobs <- url
    }

    close(jobs)
    wg.Wait()
}

在此处查看差异:without pointerwith pointer