我需要读取从另一个goroutine设置的结构字段,afaik直接这样做,即使确定会有没有并发访问权限(写入在读取之前完成,通过{{1发出信号) }})可能导致陈旧数据
将指针发送到结构(在第一个goroutine中创建,在第二个中修改,由第三个读取)将解决可能的过期问题,考虑到我可以保证没有并发访问?
我想避免复制,因为结构很大并且包含巨大的Bytes.Buffer填充在第二个goroutine中,我需要从第3个读取
有一个锁定选项,但考虑到我知道不会有并发访问,这似乎是一种矫枉过正
答案 0 :(得分:4)
为了防止在保留阅读能力的同时修改结构,您通常会嵌入sync.RWMutex。这不是免税。您可以在传输过程中简单lock your struct for writes并在方便的时候解锁它。
package main
import (
"fmt"
"sync"
"time"
)
// Big simulates your big struct
type Big struct {
sync.RWMutex
value string
}
// pump uses a groutine to take the slice of pointers to Big,
// locks the underlying structs and sends the pointers to
// the locked instances of Big downstream
func pump(bigs []*Big) chan *Big {
// We make the channel buffered for this example
// for illustration purposes
c := make(chan *Big, 3)
go func() {
for _, big := range bigs {
// We lock the struct before sending it to the channel
// so it can not be changed via pointer while in transit
big.Lock()
c <- big
}
close(c)
}()
return c
}
// sink reads pointers to the locked instances of Big
// reads them and unlocks them
func sink(c chan *Big) {
for big := range c {
fmt.Println(big.value)
time.Sleep(1 * time.Second)
big.Unlock()
}
}
// modify tries to achieve locks to the instances and modify them
func modify(bigs []*Big) {
for _, big := range bigs {
big.Lock()
big.value = "modified"
big.Unlock()
}
}
func main() {
bigs := []*Big{&Big{value: "Foo"}, &Big{value: "Bar"}, &Big{value: "Baz"}}
c := pump(bigs)
// For the sake of this example, we wait until all entries are
// send into the channel and hence are locked
time.Sleep(1 * time.Second)
// Now we try to modify concurrently before we even start to read
// the struct of which the pointers were sent into the channel
go modify(bigs)
sink(c)
// We use sleep here to keep waiting for modify() to finish simple.
// Usually, you'd use a sync.waitGroup
time.Sleep(1 * time.Second)
for _, big := range bigs {
fmt.Println(big.value)
}
}
答案 1 :(得分:3)
这有很多答案,这取决于你的数据结构和程序逻辑。
见:How to lock/synchronize access to a variable in Go during concurrent goroutines?
并且:How to use RWMutex in Golang?
1-使用Stateful Goroutines和频道
2-使用sync.Mutex
3-使用同步/原子
4-使用WaitGroup
5-使用程序逻辑(Semaphore)
...
1:Stateful Goroutines和频道:
我模拟了非常相似的样本(想象你想从一个SSD读取并写入另一个速度不同的SSD):
在这个示例代码中,一个goroutine(名为write)做一些工作准备数据并填充大结构,另一个goroutine(名为read)从big struct读取数据然后做一些工作,而manger goroutine,保证不会同时访问相同的数据。
与渠道完成的三个goroutines之间的沟通。在您的情况下,您可以使用指针来显示频道数据,或者像这个样本一样使用全局结构
输出将是这样的:
mean = 36.6920166015625 stdev = 6.068973186592054
我希望这可以帮助你理解。
工作示例代码:
package main
import (
"fmt"
"math"
"math/rand"
"runtime"
"sync"
"time"
)
type BigStruct struct {
big []uint16
rpos int
wpos int
full bool
empty bool
stopped bool
}
func main() {
wg.Add(1)
go write()
go read()
go manage()
runtime.Gosched()
stopCh <- <-time.After(5 * time.Second)
wg.Wait()
mean := Mean(hist)
stdev := stdDev(hist, mean)
fmt.Println("mean=", mean, "stdev=", stdev)
}
const N = 1024 * 1024 * 1024
var wg sync.WaitGroup
var stopCh chan time.Time = make(chan time.Time)
var hist []int = make([]int, 65536)
var s *BigStruct = &BigStruct{empty: true,
big: make([]uint16, N), //2GB
}
var rc chan uint16 = make(chan uint16)
var wc chan uint16 = make(chan uint16)
func next(pos int) int {
pos++
if pos >= N {
pos = 0
}
return pos
}
func manage() {
dataReady := false
var data uint16
for {
if !dataReady && !s.empty {
dataReady = true
data = s.big[s.rpos]
s.rpos++
if s.rpos >= N {
s.rpos = 0
}
s.empty = s.rpos == s.wpos
s.full = next(s.wpos) == s.rpos
}
if dataReady {
select {
case rc <- data:
dataReady = false
default:
runtime.Gosched()
}
}
if !s.full {
select {
case d := <-wc:
s.big[s.wpos] = d
s.wpos++
if s.wpos >= N {
s.wpos = 0
}
s.empty = s.rpos == s.wpos
s.full = next(s.wpos) == s.rpos
default:
runtime.Gosched()
}
}
if s.stopped {
if s.empty {
wg.Done()
return
}
}
}
}
func read() {
for {
d := <-rc
hist[d]++
}
}
func write() {
for {
wc <- uint16(rand.Intn(65536))
select {
case <-stopCh:
s.stopped = true
return
default:
runtime.Gosched()
}
}
}
func stdDev(data []int, mean float64) float64 {
sum := 0.0
for _, d := range data {
sum += math.Pow(float64(d)-mean, 2)
}
variance := sum / float64(len(data)-1)
return math.Sqrt(variance)
}
func Mean(data []int) float64 {
sum := 0.0
for _, d := range data {
sum += float64(d)
}
return sum / float64(len(data))
}
5:某些用例的另一种方式(更快):
这里是另一种使用共享数据结构进行读取作业/写入作业/处理作业的方法,它在第一篇文章中被分开,现在这里做同样的3个作业没有通道且没有互斥 。
工作样本:
package main
import (
"fmt"
"math"
"math/rand"
"time"
)
type BigStruct struct {
big []uint16
rpos int
wpos int
full bool
empty bool
stopped bool
}
func manage() {
for {
if !s.empty {
hist[s.big[s.rpos]]++ //sample read job with any time len
nextPtr(&s.rpos)
}
if !s.full && !s.stopped {
s.big[s.wpos] = uint16(rand.Intn(65536)) //sample wrire job with any time len
nextPtr(&s.wpos)
}
if s.stopped {
if s.empty {
return
}
} else {
s.stopped = time.Since(t0) >= 5*time.Second
}
}
}
func main() {
t0 = time.Now()
manage()
mean := Mean(hist)
stdev := StdDev(hist, mean)
fmt.Println("mean=", mean, "stdev=", stdev)
d0 := time.Since(t0)
fmt.Println(d0) //5.8523347s
}
var t0 time.Time
const N = 100 * 1024 * 1024
var hist []int = make([]int, 65536)
var s *BigStruct = &BigStruct{empty: true,
big: make([]uint16, N), //2GB
}
func next(pos int) int {
pos++
if pos >= N {
pos = 0
}
return pos
}
func nextPtr(pos *int) {
*pos++
if *pos >= N {
*pos = 0
}
s.empty = s.rpos == s.wpos
s.full = next(s.wpos) == s.rpos
}
func StdDev(data []int, mean float64) float64 {
sum := 0.0
for _, d := range data {
sum += math.Pow(float64(d)-mean, 2)
}
variance := sum / float64(len(data)-1)
return math.Sqrt(variance)
}
func Mean(data []int) float64 {
sum := 0.0
for _, d := range data {
sum += float64(d)
}
return sum / float64(len(data))
}