短篇小说:
我遇到的问题是以前有数据但现在应该为空的地图报告的len()
为>即使它似乎是空的,我也不知道为什么。
更长的故事:
我需要一次处理一些设备。每个设备都可以包含许多消息。 Go的并发似乎是一个明显的开始,所以我写了一些代码来处理它,它似乎很好地 。然而...
我为每个设备启动了一个goroutine 。在main()
函数中,我有map
,其中包含每个设备。当收到消息时,我会检查设备是否已经存在,如果没有,我会创建它,将其存储在地图中,然后将消息传递给设备接收缓冲的通道。
这很好用,每个设备都处理得很好。但是,我需要设备(及其goroutine)终止,因为它在预设的时间内没有收到任何消息。我通过检查goroutine本身自收到最后一条消息后经过了多少时间来完成此操作,如果goroutine被视为陈旧,则接收通道将关闭。但是如何从地图中删除?
所以我传入了一个指向地图的指针,我让goroutine从地图中删除设备并在返回之前关闭接收通道。但问题是,最后我发现len()
函数返回值> 0,但是当我输出地图本身时,我发现它是空的。
我已经写了一个玩具示例来尝试复制错误,而且我确实看到len()
正在报告>当地图显然为空时为0。我最后一次尝试时看到了10.在那之前的时间14.在那之前的时间,53。
所以我可以复制这个错误,但是我不确定这个错误是在我身上还是在Go上。 len()
如何报告>当显然没有物品时0?
以下是我如何复制的一个例子。我使用Go v1.5.1 windows / amd64
就我而言,这里有两件事:
len(m)
报告>当没有物品时0?全部谢谢
示例代码:
package main
import (
"log"
"os"
"time"
)
const (
chBuffSize = 100 // How large the thing's channel buffer should be
thingIdleLifetime = time.Second * 5 // How long things can live for when idle
thingsToMake = 1000 // How many things and associated goroutines to make
thingMessageCount = 10 // How many messages to send to the thing
)
// The thing that we'll be passing into a goroutine to process -----------------
type thing struct {
id string
ch chan bool
}
// Go go gadget map test -------------------------------------------------------
func main() {
// Make all of the things!
things := make(map[string]thing)
for i := 0; i < thingsToMake; i++ {
t := thing{
id: string(i),
ch: make(chan bool, chBuffSize),
}
things[t.id] = t
// Pass the thing into it's own goroutine
go doSomething(t, &things)
// Send (thingMessageCount) messages to the thing
go func(t thing) {
for x := 0; x < thingMessageCount; x++ {
t.ch <- true
}
}(t)
}
// Check the map of things to see whether we're empty or not
size := 0
for {
if size == len(things) && size != thingsToMake {
log.Println("Same number of items in map as last time")
log.Println(things)
os.Exit(1)
}
size = len(things)
log.Printf("Map size: %d\n", size)
time.Sleep(time.Second)
}
}
// Func for each goroutine to run ----------------------------------------------
//
// Takes two arguments:
// 1) the thing that it is working with
// 2) a pointer to the map of things
//
// When this goroutine is ready to terminate, it should remove the associated
// thing from the map of things to clean up after itself
func doSomething(t thing, things *map[string]thing) {
lastAccessed := time.Now()
for {
select {
case <-t.ch:
// We received a message, so extend the lastAccessed time
lastAccessed = time.Now()
default:
// We haven't received a message, so check if we're allowed to continue
n := time.Now()
d := n.Sub(lastAccessed)
if d > thingIdleLifetime {
// We've run for >thingIdleLifetime, so close the channel, delete the
// associated thing from the map and return, terminating the goroutine
close(t.ch)
delete(*things, string(t.id))
return
}
}
// Just sleep for a second in each loop to prevent the CPU being eaten up
time.Sleep(time.Second)
}
}
只是添加;在我的原始代码中,这是永远循环。该程序旨在监听TCP连接并接收和处理数据,因此检查映射计数的功能正在其自身的goroutine中运行。但是,此示例具有完全相同的症状,即使映射len()
检查在main()
函数中,并且它旨在处理初始数据突发然后突破循环。
更新2015/11/23 15:56 UTC
我在下面重构了我的例子。我不确定我是否误解了@RobNapier,但是效果要好得多。但是,如果我将thingsToMake
更改为更大的数字,例如100000,那么我会收到很多这样的错误:
goroutine 199734 [select]:
main.doSomething(0xc0d62e7680, 0x4, 0xc0d64efba0, 0xc082016240)
C:/Users/anttheknee/go/src/maptest/maptest.go:83 +0x144
created by main.main
C:/Users/anttheknee/go/src/maptest/maptest.go:46 +0x463
我不确定问题是我是否要求Go做太多,或者我是否已经理解了解决方案。有什么想法吗?
package main
import (
"log"
"os"
"time"
)
const (
chBuffSize = 100 // How large the thing's channel buffer should be
thingIdleLifetime = time.Second * 5 // How long things can live for when idle
thingsToMake = 10000 // How many things and associated goroutines to make
thingMessageCount = 10 // How many messages to send to the thing
)
// The thing that we'll be passing into a goroutine to process -----------------
type thing struct {
id string
ch chan bool
done chan string
}
// Go go gadget map test -------------------------------------------------------
func main() {
// Make all of the things!
things := make(map[string]thing)
// Make a channel to receive completion notification on
doneCh := make(chan string, chBuffSize)
log.Printf("Making %d things\n", thingsToMake)
for i := 0; i < thingsToMake; i++ {
t := thing{
id: string(i),
ch: make(chan bool, chBuffSize),
done: doneCh,
}
things[t.id] = t
// Pass the thing into it's own goroutine
go doSomething(t)
// Send (thingMessageCount) messages to the thing
go func(t thing) {
for x := 0; x < thingMessageCount; x++ {
t.ch <- true
time.Sleep(time.Millisecond * 10)
}
}(t)
}
log.Printf("All %d things made\n", thingsToMake)
// Receive on doneCh when the goroutine is complete and clean the map up
for {
id := <-doneCh
close(things[id].ch)
delete(things, id)
if len(things) == 0 {
log.Printf("Map: %v", things)
log.Println("All done. Exiting")
os.Exit(0)
}
}
}
// Func for each goroutine to run ----------------------------------------------
//
// Takes two arguments:
// 1) the thing that it is working with
// 2) the channel to report that we're done through
//
// When this goroutine is ready to terminate, it should remove the associated
// thing from the map of things to clean up after itself
func doSomething(t thing) {
timer := time.NewTimer(thingIdleLifetime)
for {
select {
case <-t.ch:
// We received a message, so extend the timer
timer.Reset(thingIdleLifetime)
case <-timer.C:
// Timer returned so we need to exit now
t.done <- t.id
return
}
}
}
更新2015/11/23 16:41 UTC
看似正常工作的已完成代码。如果有任何可以改进的地方,请随时告诉我,但这是有效的(睡觉是故意看到进展,因为它太快了!)
package main
import (
"log"
"os"
"strconv"
"time"
)
const (
chBuffSize = 100 // How large the thing's channel buffer should be
thingIdleLifetime = time.Second * 5 // How long things can live for when idle
thingsToMake = 100000 // How many things and associated goroutines to make
thingMessageCount = 10 // How many messages to send to the thing
)
// The thing that we'll be passing into a goroutine to process -----------------
type thing struct {
id string
receiver chan bool
done chan string
}
// Go go gadget map test -------------------------------------------------------
func main() {
// Make all of the things!
things := make(map[string]thing)
// Make a channel to receive completion notification on
doneCh := make(chan string, chBuffSize)
log.Printf("Making %d things\n", thingsToMake)
for i := 0; i < thingsToMake; i++ {
t := thing{
id: strconv.Itoa(i),
receiver: make(chan bool, chBuffSize),
done: doneCh,
}
things[t.id] = t
// Pass the thing into it's own goroutine
go doSomething(t)
// Send (thingMessageCount) messages to the thing
go func(t thing) {
for x := 0; x < thingMessageCount; x++ {
t.receiver <- true
time.Sleep(time.Millisecond * 100)
}
}(t)
}
log.Printf("All %d things made\n", thingsToMake)
// Check the `len()` of things every second and exit when empty
go func() {
for {
time.Sleep(time.Second)
m := things
log.Printf("Map length: %v", len(m))
if len(m) == 0 {
log.Printf("Confirming empty map: %v", things)
log.Println("All done. Exiting")
os.Exit(0)
}
}
}()
// Receive on doneCh when the goroutine is complete and clean the map up
for {
id := <-doneCh
close(things[id].receiver)
delete(things, id)
}
}
// Func for each goroutine to run ----------------------------------------------
//
// When this goroutine is ready to terminate it should respond through t.done to
// notify the caller that it has finished and can be cleaned up. It will wait
// for `thingIdleLifetime` until it times out and terminates on it's own
func doSomething(t thing) {
timer := time.NewTimer(thingIdleLifetime)
for {
select {
case <-t.receiver:
// We received a message, so extend the timer
timer.Reset(thingIdleLifetime)
case <-timer.C:
// Timer expired so we need to exit now
t.done <- t.id
return
}
}
}
答案 0 :(得分:6)
map
不是线程安全的。您无法安全地访问多个goroutine上的map
。你可以破坏地图,正如你在这种情况下看到的那样。
goroutine不应允许goroutine修改地图,而是应该在返回之前将其标识符写入通道。主循环应该观察该通道,当标识符返回时,应该从地图中删除该元素。
您可能希望阅读Go并发模式。特别是,您可能需要查看Fan-out/Fan-in。看看底部的链接。 Go博客有很多关于并发的信息。
请注意,您的goroutine正忙着等待检查超时。没有理由这样做。你“睡觉(1秒)”的事实应该是一个错误的线索。相反,看看time.Timer
会给你一个在一段时间后会收到值的陈,你可以重置。
您的问题是如何将数字转换为字符串:
id: string(i),
使用i
作为符文(int32
)创建一个字符串。例如,string(65)
为A
。一些不相等的符文决定等于字符串。你发生碰撞并关闭同一个频道两次。见http://play.golang.org/p/__KpnfQc1V
你的意思是:
id: strconv.Itoa(i),