Question

我有一个用Go编写的新TCP服务器，它连接了100多个客户端。每个客户端都会在需要集中查看的数据中进行流式传输，因为他们正在查看来自不同位置的空中波的无线电数据包，然后进行分析。代码工作但我看到很多争用和增加锁定周围的CPU，并考虑如何避免锁定（如果可能）或围绕它进行优化。

当TCP服务器为收到的每个数据包旋转GoRoutine时，addMessage功能需要一定程度的同步。这些数据包也将在稍后的另一个函数中进行分析，该函数在地图上执行RLock()。

cullMessages()函数每秒被调用一次，它本身就会被捕获并且可能真的变慢，有时需要2-3秒才能运行，这会将问题复杂化为接下来的2-3次操作排队等待解锁并立即运行！

任何想法/想法都将不胜感激！

var dataMessagesMutex sync.RWMutex
var dataMessages map[string][]*trackingPacket_v1

// Function is called from each TCP client who need to share this data
func addMessage(trackingPacket *trackingPacket_v1) {
    dataMessagesMutex.Lock()
    dataMessages[trackingPacket.packetID] = append(dataMessages[trackingPacket.packetID], trackingPacket)
    dataMessagesMutex.Unlock()
}

// Function called on a loop, need to delete based on age here
func cullMessages() {
    cullTS := time.Now().Add(-time.Second * MODES_MAX_MESSAGE_AGE)

    dataMessagesMutex.Lock()
    defer dataMessagesMutex.Unlock()

    for avr, data := range dataMessages {
        sort.Sort(PacketSorter(data))
        highestIndex := 0

        for i, messages := range data {
            if cullTS.Sub(messages.ProcessedTime) > 0 {
                // Need to delete the message here
                messages = nil
                highestIndex = i
            }
        }
        // Copy the new slice into the data variable
        data = data[highestIndex+1:]

        if len(data) == 0 {
            // Empty Messages, delete
            delete(dataMessages, avr)
        }
    }
}

更新：添加了分析功能

func processCandidates() {
    mlatMessagesMutex.RLock()
    defer dataMessagesMutex.RUnlock()

    for _, data := range dataMessages {
        numberOfMessages := len(data)
        for a := 0; a < numberOfMessages; a++ {
            packetA := data[a]
            applicablePackets := []*trackingPacket_v1{packetA}
            for b := 0; b < numberOfMessages; b++ {
                // Don't compare identical packets
                if b == a {
                    continue
                }

                packetB := data[b]

                // Only consider this packet if it's within an acceptable
                // timestamp threshold
                tsDelta := math.Abs(packetA.NormalisedTS - packetB.NormalisedTS)

                if tsDelta < MAX_MESSAGE_TS_DIFF {
                    // Finally, we need to make sure that only one message per
                    // station is included in our batch
                    stationAlreadyRepresented := false
                    for i := 0; i < len(applicablePackets); i++ {
                        if applicablePackets[i].Sharecode == packetB.Sharecode {
                            stationAlreadyRepresented = true
                        }
                    }

                    if stationAlreadyRepresented == false {

                        applicablePackets = append(applicablePackets, packetB)
                    }
                }
            }

            // Remove any stations which are deemed too close to one another
            if len(applicablePackets) >= MIN_STATIONS_NEEDED {
                applicablePackets = cullPackets(applicablePackets)
            }

            // Provided we still have enough packets....
            if len(applicablePackets) >= MIN_STATIONS_NEEDED {
                // Generate a hash for this batch...
                hash := generateHashForPackets(applicablePackets)
                batchIsUnique := true

                for _, packet := range applicablePackets {
                    if packet.containsHash(hash) {
                        batchIsUnique = false
                        break
                    }
                }

                if batchIsUnique == true {
                    for _, packet := range applicablePackets {
                        packet.addHash(hash)
                    }

                    go sendOfDataForWork(applicablePackets)
                }
            }

        }
    }
}

Answer 1

不是拥有一张大地图，而是为每个packetID设置一个goroutine。调度程序goroutine可以有map[string]chan *trackingPacket_v1，并在适当的通道上发送传入的数据包。然后，该packetID的goroutine将数据包收集到本地切片中，并剔除它们并定期分析它们。

不知何故，您需要终止未在MODES_MAX_MESSAGE_AGE中收到数据包的goroutine。调度员goroutine可能会跟踪最近看到每个packetID的时间，并定期检查并检查那些太旧的数据包ID。然后它将关闭这些通道并将其从地图中删除。当分析goroutine发现其通道已关闭时，它将退出。

大量瞬态对象 - 避免争用

1 个答案: