Golang map len()报告>地图为空时为

时间:2015-11-23 13:21:40

标签: go

短篇小说:

我遇到的问题是以前有数据但现在应该为空的地图报告的len()为>即使它似乎是空的,我也不知道为什么。

更长的故事:

我需要一次处理一些设备。每个设备都可以包含许多消息。 Go的并发似乎是一个明显的开始,所以我写了一些代码来处理它,它似乎很好地 。然而...

我为每个设备启动了一个goroutine 。在main()函数中,我有map,其中包含每个设备。当收到消息时,我会检查设备是否已经存在,如果没有,我会创建它,将其存储在地图中,然后将消息传递给设备接收缓冲的通道。

这很好用,每个设备都处理得很好。但是,我需要设备(及其goroutine)终止,因为它在预设的时间内没有收到任何消息。我通过检查goroutine本身自收到最后一条消息后经过了多少时间来完成此操作,如果goroutine被视为陈旧,则接收通道将关闭。但是如何从地图中删除?

所以我传入了一个指向地图的指针,我让goroutine从地图中删除设备并在返回之前关闭接收通道。但问题是,最后我发现len()函数返回值> 0,但是当我输出地图本身时,我发现它是空的。

我已经写了一个玩具示例来尝试复制错误,而且我确实看到len()正在报告>当地图显然为空时为0。我最后一次尝试时看到了10.在那之前的时间14.在那之前的时间,53。

所以我可以复制这个错误,但是我不确定这个错误是在我身上还是在Go上。 len()如何报告>当显然没有物品时0?

以下是我如何复制的一个例子。我使用Go v1.5.1 windows / amd64

就我而言,这里有两件事:

  1. 我是否正确管理goroutines(可能不是)和
  2. 为什么len(m)报告>当没有物品时0?
  3. 全部谢谢

    示例代码:

    package main
    
    import (
        "log"
        "os"
        "time"
    )
    
    const (
        chBuffSize        = 100             // How large the thing's channel buffer should be
        thingIdleLifetime = time.Second * 5 // How long things can live for when idle
        thingsToMake      = 1000            // How many things and associated goroutines to make
        thingMessageCount = 10              // How many messages to send to the thing
    )
    
    // The thing that we'll be passing into a goroutine to process -----------------
    type thing struct {
        id string
        ch chan bool
    }
    
    // Go go gadget map test -------------------------------------------------------
    func main() {
        // Make all of the things!
        things := make(map[string]thing)
        for i := 0; i < thingsToMake; i++ {
            t := thing{
                id: string(i),
                ch: make(chan bool, chBuffSize),
            }
            things[t.id] = t
    
            // Pass the thing into it's own goroutine
            go doSomething(t, &things)
    
            // Send (thingMessageCount) messages to the thing
            go func(t thing) {
                for x := 0; x < thingMessageCount; x++ {
                    t.ch <- true
                }
            }(t)
        }
    
        // Check the map of things to see whether we're empty or not
        size := 0
        for {
            if size == len(things) && size != thingsToMake {
                log.Println("Same number of items in map as last time")
                log.Println(things)
                os.Exit(1)
            }
            size = len(things)
            log.Printf("Map size: %d\n", size)
            time.Sleep(time.Second)
        }
    }
    
    // Func for each goroutine to run ----------------------------------------------
    //
    // Takes two arguments:
    // 1) the thing that it is working with
    // 2) a pointer to the map of things
    //
    // When this goroutine is ready to terminate, it should remove the associated
    // thing from the map of things to clean up after itself
    func doSomething(t thing, things *map[string]thing) {
        lastAccessed := time.Now()
        for {
            select {
            case <-t.ch:
                // We received a message, so extend the lastAccessed time
                lastAccessed = time.Now()
            default:
                // We haven't received a message, so check if we're allowed to continue
                n := time.Now()
                d := n.Sub(lastAccessed)
                if d > thingIdleLifetime {
                    // We've run for >thingIdleLifetime, so close the channel, delete the
                    // associated thing from the map and return, terminating the goroutine
                    close(t.ch)
                    delete(*things, string(t.id))
                    return
                }
            }
    
            // Just sleep for a second in each loop to prevent the CPU being eaten up
            time.Sleep(time.Second)
        }
    }
    

    只是添加;在我的原始代码中,这是永远循环。该程序旨在监听TCP连接并接收和处理数据,因此检查映射计数的功能正在其自身的goroutine中运行。但是,此示例具有完全相同的症状,即使映射len()检查在main()函数中,并且它旨在处理初始数据突发然后突破循环。

    更新2015/11/23 15:56 UTC

    我在下面重构了我的例子。我不确定我是否误解了@RobNapier,但是效果要好得多。但是,如果我将thingsToMake更改为更大的数字,例如100000,那么我会收到很多这样的错误:

    goroutine 199734 [select]:
    main.doSomething(0xc0d62e7680, 0x4, 0xc0d64efba0, 0xc082016240)
            C:/Users/anttheknee/go/src/maptest/maptest.go:83 +0x144
    created by main.main
            C:/Users/anttheknee/go/src/maptest/maptest.go:46 +0x463
    

    我不确定问题是我是否要求Go做太多,或者我是否已经理解了解决方案。有什么想法吗?

    package main
    
    import (
        "log"
        "os"
        "time"
    )
    
    const (
        chBuffSize        = 100             // How large the thing's channel buffer should be
        thingIdleLifetime = time.Second * 5 // How long things can live for when idle
        thingsToMake      = 10000           // How many things and associated goroutines to make
        thingMessageCount = 10              // How many messages to send to the thing
    )
    
    // The thing that we'll be passing into a goroutine to process -----------------
    type thing struct {
        id   string
        ch   chan bool
        done chan string
    }
    
    // Go go gadget map test -------------------------------------------------------
    func main() {
        // Make all of the things!
        things := make(map[string]thing)
    
        // Make a channel to receive completion notification on
        doneCh := make(chan string, chBuffSize)
    
        log.Printf("Making %d things\n", thingsToMake)
        for i := 0; i < thingsToMake; i++ {
            t := thing{
                id:   string(i),
                ch:   make(chan bool, chBuffSize),
                done: doneCh,
            }
            things[t.id] = t
    
            // Pass the thing into it's own goroutine
            go doSomething(t)
    
            // Send (thingMessageCount) messages to the thing
            go func(t thing) {
                for x := 0; x < thingMessageCount; x++ {
                    t.ch <- true
                    time.Sleep(time.Millisecond * 10)
                }
            }(t)
        }
        log.Printf("All %d things made\n", thingsToMake)
    
        // Receive on doneCh when the goroutine is complete and clean the map up
        for {
            id := <-doneCh
            close(things[id].ch)
            delete(things, id)
            if len(things) == 0 {
                log.Printf("Map: %v", things)
                log.Println("All done. Exiting")
                os.Exit(0)
            }
        }
    }
    
    // Func for each goroutine to run ----------------------------------------------
    //
    // Takes two arguments:
    // 1) the thing that it is working with
    // 2) the channel to report that we're done through
    //
    // When this goroutine is ready to terminate, it should remove the associated
    // thing from the map of things to clean up after itself
    func doSomething(t thing) {
        timer := time.NewTimer(thingIdleLifetime)
        for {
            select {
            case <-t.ch:
                // We received a message, so extend the timer
                timer.Reset(thingIdleLifetime)
            case <-timer.C:
                // Timer returned so we need to exit now
                t.done <- t.id
                return
            }
        }
    }
    

    更新2015/11/23 16:41 UTC

    看似正常工作的已完成代码。如果有任何可以改进的地方,请随时告诉我,但这是有效的(睡觉是故意看到进展,因为它太快了!)

    package main
    
    import (
        "log"
        "os"
        "strconv"
        "time"
    )
    
    const (
        chBuffSize        = 100             // How large the thing's channel buffer should be
        thingIdleLifetime = time.Second * 5 // How long things can live for when idle
        thingsToMake      = 100000          // How many things and associated goroutines to make
        thingMessageCount = 10              // How many messages to send to the thing
    )
    
    // The thing that we'll be passing into a goroutine to process -----------------
    type thing struct {
        id       string
        receiver chan bool
        done     chan string
    }
    
    // Go go gadget map test -------------------------------------------------------
    func main() {
        // Make all of the things!
        things := make(map[string]thing)
    
        // Make a channel to receive completion notification on
        doneCh := make(chan string, chBuffSize)
    
        log.Printf("Making %d things\n", thingsToMake)
    
        for i := 0; i < thingsToMake; i++ {
            t := thing{
                id:       strconv.Itoa(i),
                receiver: make(chan bool, chBuffSize),
                done:     doneCh,
            }
            things[t.id] = t
    
            // Pass the thing into it's own goroutine
            go doSomething(t)
    
            // Send (thingMessageCount) messages to the thing
            go func(t thing) {
                for x := 0; x < thingMessageCount; x++ {
                    t.receiver <- true
                    time.Sleep(time.Millisecond * 100)
                }
            }(t)
        }
        log.Printf("All %d things made\n", thingsToMake)
    
        // Check the `len()` of things every second and exit when empty
        go func() {
            for {
                time.Sleep(time.Second)
                m := things
                log.Printf("Map length: %v", len(m))
                if len(m) == 0 {
                    log.Printf("Confirming empty map: %v", things)
                    log.Println("All done. Exiting")
                    os.Exit(0)
                }
            }
        }()
    
        // Receive on doneCh when the goroutine is complete and clean the map up
        for {
            id := <-doneCh
            close(things[id].receiver)
            delete(things, id)
        }
    }
    
    // Func for each goroutine to run ----------------------------------------------
    //
    // When this goroutine is ready to terminate it should respond through t.done to
    // notify the caller that it has finished and can be cleaned up. It will wait
    // for `thingIdleLifetime` until it times out and terminates on it's own
    func doSomething(t thing) {
        timer := time.NewTimer(thingIdleLifetime)
        for {
            select {
            case <-t.receiver:
                // We received a message, so extend the timer
                timer.Reset(thingIdleLifetime)
            case <-timer.C:
                // Timer expired so we need to exit now
                t.done <- t.id
                return
            }
        }
    }
    

1 个答案:

答案 0 :(得分:6)

map不是线程安全的。您无法安全地访问多个goroutine上的map。你可以破坏地图,正如你在这种情况下看到的那样。

goroutine不应允许goroutine修改地图,而是应该在返回之前将其标识符写入通道。主循环应该观察该通道,当标识符返回时,应该从地图中删除该元素。

您可能希望阅读Go并发模式。特别是,您可能需要查看Fan-out/Fan-in。看看底部的链接。 Go博客有很多关于并发的信息。

请注意,您的goroutine正忙着等待检查超时。没有理由这样做。你“睡觉(1秒)”的事实应该是一个错误的线索。相反,看看time.Timer会给你一个在一段时间后会收到值的陈,你可以重置。

您的问题是如何将数字转换为字符串:

        id:   string(i),

使用i作为符文(int32)创建一个字符串。例如,string(65)A。一些不相​​等的符文决定等于字符串。你发生碰撞并关闭同一个频道两次。见http://play.golang.org/p/__KpnfQc1V

你的意思是:

        id:   strconv.Itoa(i),