Question

我正在将一个旧的小C项目重写为Go（学习Go）。

该项目基本上从文件中读取一些二进制数据，对所述数据进行一些过滤，然后将其打印到stdout。

代码的主要部分如下所示（省略错误处理）：

type netFlowRow struct {
    Timestamp uint32
    Srcip     [4]byte
    Dstip     [4]byte
    Proto     uint16
    Srcport   uint16
    Dstport   uint16
    Pkt       uint32
    Size      uint64
}

func main() {
    // ...
    file, _ := os.Open(path)
    for j := 0; j < infoRow.Count; j++ {
        netRow := netFlowRow{}
        binary.Read(file, binary.BigEndian, &netRow)

        // ...
        fmt.Printf("%v", netRow)
    }
}

做了天真的重写之后，go版本的运行速度比C版慢10倍（~40s vs 2-3s）。我用pprof进行了分析，它向我展示了这个：

(pprof) top10
39.96s of 40.39s total (98.94%)
Dropped 71 nodes (cum <= 0.20s)
Showing top 10 nodes out of 11 (cum >= 39.87s)
      flat  flat%   sum%        cum   cum%
    39.87s 98.71% 98.71%     39.87s 98.71%  syscall.Syscall
     0.09s  0.22% 98.94%     40.03s 99.11%  encoding/binary.Read
         0     0% 98.94%     39.87s 98.71%  io.ReadAtLeast
         0     0% 98.94%     39.87s 98.71%  io.ReadFull
         0     0% 98.94%     40.03s 99.11%  main.main
         0     0% 98.94%     39.87s 98.71%  os.(*File).Read
         0     0% 98.94%     39.87s 98.71%  os.(*File).read
         0     0% 98.94%     40.21s 99.55%  runtime.goexit
         0     0% 98.94%     40.03s 99.11%  runtime.main
         0     0% 98.94%     39.87s 98.71%  syscall.Read

我读到这个吗？ syscall.Syscall基本上是消费者的主要时间吗？它是从文件读取的地方吗？

UPD。我使用了bufio.Reader并获得了这个配置文件：

(pprof) top10
34.16s of 36s total (94.89%)
Dropped 99 nodes (cum <= 0.18s)
Showing top 10 nodes out of 33 (cum >= 0.56s)
      flat  flat%   sum%        cum   cum%
    31.99s 88.86% 88.86%        32s 88.89%  syscall.Syscall
     0.43s  1.19% 90.06%      0.64s  1.78%  runtime.mallocgc
     0.39s  1.08% 91.14%      1.06s  2.94%  encoding/binary.(*decoder).value
     0.28s  0.78% 91.92%      0.99s  2.75%  reflect.(*structType).Field
     0.28s  0.78% 92.69%      0.28s  0.78%  runtime.duffcopy
     0.24s  0.67% 93.36%      1.64s  4.56%  encoding/binary.sizeof
     0.22s  0.61% 93.97%     34.51s 95.86%  encoding/binary.Read
     0.22s  0.61% 94.58%      0.22s  0.61%  runtime.mach_semaphore_signal
     0.07s  0.19% 94.78%      1.28s  3.56%  reflect.(*rtype).Field
     0.04s  0.11% 94.89%      0.56s  1.56%  runtime.newobject

Answer 1

binary.Read会慢一些，因为它使用reflection。我建议使用bufio.Reader进行基准测试并手动调用binary.BigEndian方法来读取结构：

type netFlowRow struct {
    Timestamp uint32   // 0
    Srcip     [4]byte  // 4
    Dstip     [4]byte  // 8
    Proto     uint16   // 12
    Srcport   uint16   // 14
    Dstport   uint16   // 16
    Pkt       uint32   // 18
    Size      uint64   // 22
}

func main() {
    // ...
    file, _ := os.Open(path)
    r := bufio.NewReader(file)
    for j := 0; j < infoRow.Count; j++ {
        var buff [4 + 4 + 4 + 2 + 2 + 2 + 4 + 8]byte
        if _, err := io.ReadFull(r, buff[:]); err != nil {
            panic(err)
        }
        netRow := netFlowRow{
            Timestamp: binary.BigEndian.Uint32(buff[:4]),
            // Srcip
            // Dstip
            Proto: binary.BigEndian.Uint16(buff[12:14]),
            Srcport: binary.BigEndian.Uint16(buff[14:16]),
            Dstport: binary.BigEndian.Uint16(buff[16:18]),
            Pkt: binary.BigEndian.Uint32(buff[18:22]),
            Size: binary.BigEndian.Uint64(buff[22:30]),
        }
        copy(netRow.Srcip[:], buff[4:8])
        copy(netRow.Dstip[:], buff[8:12])

        // ...
        fmt.Printf("%v", netRow)
    }
}

二进制。慢吗？

1 个答案: