在尝试向influxdb中插入数据以从grafana提取数据时,我们面临着问题。截至目前,正在进行一项测试,我们将从主机收集的数据直接插入到influxdb db中,事实是,这变得非常巨大,并且这仅是来自单个主机的信息,我们的PROD中有数百个主机环境是我们的主要目标。
我们试图做的是仅在需要调查中断时才插入数据,为此,我们计划从当前正在创建的RAW文件中进行插入。我们遇到了一个解决方案,但NMON可以满足我们的需要,但NMON文件却可以满足我们的需要,但老实说,我对Golang并不了解,这是他们用于该语言的语言,或者至少我是这么认为的。
我在这个社区中发现有一个PARSER插件是telegraf的一部分,但我找不到如何使用它来从collects提供的RAW文件中获取信息并将其插入到influxdb中
如果有人能就此事给我提建议,我将不胜感激。
谢谢。
输出示例:
################################################################################
# Collectl: V3.6.5-2 HiRes: 1 Options: -D
# Host: localhost DaemonOpts: -f /var/log/collectl -r00:00,7,60 -m -F60 -s+YZ -i60
# Booted: 1548722801.34 [20190128-18:46:41]
# Distro: Red Hat Enterprise Linux Server release 6.9 (Santiago) Platform: VMware Virtual Platform
# Date: 20190130-230000 Secs: 1548910800 TZ: -0600
# SubSys: bcdfijmnstYZ Options: Interval: 60:60 NumCPUs: 10 NumBud: 3 Flags: ix
# Filters: NfsFilt: EnvFilt: TcpFilt: ituc
# HZ: 100 Arch: x86_64-linux-thread-multi PageSize: 4096
# Cpu: GenuineIntel Speed(MHz): 2992.968 Cores: 1 Siblings: 1 Nodes: 1
# Kernel: 2.6.32-696.20.1.el6.x86_64 Memory: 198339428 kB Swap: 17825788 kB
# NumDisks: 22 DiskNames: sda sdb sdc sde sdd dm-0 dm-1 dm-2 dm-3 dm-4 dm-5 dm-6 dm-7 dm-8 dm-9 dm-10 dm-11 dm-12 dm-13 dm-14 dm-15 dm-16
# NumNets: 3 NetNames: lo:?? eth0:10000 eth1:10000
# NumSlabs: 201 Version: 2.1
# SCSI: CD:1:00:00:00 DA:2:00:00:00 DA:2:00:01:00 DA:2:00:02:00 DA:2:00:03:00 DA:2:00:04:00
################################################################################
>>> 1548910800.001 <<<
buddy Node 0, zone DMA 1 1 1 3 3 2 1 1 0 0 2
buddy Node 0, zone DMA32 14 18 15 17 14 7 7 11 8 4 106
buddy Node 0, zone Normal 403 386 5583 3572 13604 10598 7080 3627 2055 2 27
cpu 22087348 256 7084173 153333789 15626 4857 2989099 0 0
cpu0 2343703 23 756440 15044112 1503 394 389489 0 0
cpu1 1424427 40 403584 16804532 1978 2 13682 0 0
cpu2 1400317 30 399191 16835804 1824 1 12608 0 0
cpu3 2373492 20 777245 15024216 1405 441 357515 0 0
cpu4 2305879 21 754504 15129573 1544 395 334811 0 0
cpu5 2406073 20 826459 14953667 1377 1854 337925 0 0
cpu6 2524873 19 813584 14794741 1330 456 388679 0 0
cpu7 2377199 26 774277 14997185 1586 440 369269 0 0
cpu8 2431979 19 818481 14856251 1454 501 415840 0 0
cpu9 2499402 33 760403 14893703 1620 370 369277 0 0
intr 5904658682
ctxt 4292174746
processes 1824586
procs_running 3
procs_blocked 0
int 0: 844 0 0 0 0 0 0 0 0 0 IO-APIC-edge timer
int 1: 10 1146 0 0 0 0 0 0 0 0 IO-APIC-edge i8042
int 8: 1 0 0 0 0 0 0 0 0 0 IO-APIC-edge rtc0
int 9: 0 0 0 0 0 0 0 0 0 0 IO-APIC-fasteoi acpi
int 12: 110 0 0 1080 0 0 0 0 0 0 IO-APIC-edge i8042
int 14: 0 0 0 0 0 0 0 0 0 0 IO-APIC-edge ata_piix
int 15: 95 0 0 0 0 639862 0 0 0 0 IO-APIC-edge ata_piix
int 16: 4 0 0 0 0 0 0 0 0 2 IO-APIC-fasteoi vmwgfx
int 24: 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge pciehp
int 25: 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge pciehp
int 26: 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge pciehp
int 27: 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge pciehp
int 28: 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge pciehp
int 29: 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge pciehp
int 30: 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge pciehp
int 31: 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge pciehp
int 32: 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge pciehp
int 33: 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge pciehp
int 34: 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge pciehp
int 35: 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge pciehp
int 36: 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge pciehp
int 37: 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge pciehp
int 38: 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge pciehp
int 39: 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge pciehp
int 40: 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge pciehp
disk 253 3 dm-3 817 0 6530 1019 1 0 8 6 0 187 1025
disk 253 4 dm-4 261 0 3298 154 78648 0 629184 94613 0 12780 94788
disk 253 5 dm-5 122 0 970 30 1 0 8 0 0 30 30
disk 253 6 dm-6 128 0 1018 49 1 0 8 6 0 55 55
disk 253 7 dm-7 122 0 970 43 1 0 8 0 0 43 43
disk 253 8 dm-8 14705 0 173314 18512 30827989 0 246623912 309099632 0 580567 309838167
disk 253 9 dm-9 257 0 2050 167 1 0 8 0 0 80 167
disk 253 10 dm-10 142 0 1130 58 112 0 896 81 0 107 139
disk 253 11 dm-11 26758 0 1183658 26638 9156 0 73248 27014 0 15012 53652
disk 253 12 dm-12 133 0 1058 59 6 0 48 2 0 49 61
disk 253 13 dm-13 133 0 1058 56 6 0 48 2 0 46 58
disk 253 14 dm-14 5088 0 615178 3932 107319 0 858552 1285013 0 3111 1288946
disk 253 15 dm-15 15240 0 410298 10749 1538387 0 12307096 7551698 0 121712 7567479
disk 253 16 dm-16 544 0 20778 753 989864 0 7918912 2153755 0 479168 2155269
load 0.21 0.16 0.10 1/2757 24089
tcp-Ip: Forwarding DefaultTTL InReceives InHdrErrors InAddrErrors ForwDatagrams InUnknownProtos InDiscards InDelivers OutRequests OutDiscards OutNoRoutes ReasmTimeout ReasmReqds ReasmOKs ReasmFails FragOKs FragFails FragCreates
tcp-Ip: 2 64 2345808100 0 65987 0 0 0 2345023989 2542097859 0 88 0 1374787 657103 0 658849 0 1378045
tcp-Icmp: InMsgs InErrors InDestUnreachs InTimeExcds InParmProbs InSrcQuenchs InRedirects InEchos InEchoReps InTimestamps InTimestampReps InAddrMasks InAddrMaskReps OutMsgs OutErrors OutDestUnreachs OutTimeExcds OutParmProbs OutSrcQuenchs OutRedirects OutEchos OutEchoReps OutTimestamps OutTimestampReps OutAddrMasks OutAddrMaskReps
tcp-Icmp: 9644 24 1453 0 0 0 0 4223 3966 2 0 0 0 10125 0 1934 0 0 0 0 3966 4223 0 2 0 0
tcp-Tcp: RtoAlgorithm RtoMin RtoMax MaxConn ActiveOpens PassiveOpens AttemptFails EstabResets CurrEstab InSegs OutSegs RetransSegs InErrs OutRsts
tcp-Tcp: 1 200 120000 -1 189518 203747 116 1705 1538 2337716464 2532808789 6750 1 1108
tcp-Udp: InDatagrams NoPorts InErrors OutDatagrams RcvbufErrors SndbufErrors
tcp-Udp: 7155937 2287 12361 9272237 1 0
fs-ds 279615 266755 45 0 0 0
fs-is 121177 118
fs-fnr 9728 0 6815744
nfsc-net 0 0 0 0
nfsc-rpc 499842 1085 499886
nfsc-proc2 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
nfsc-proc3 22 0 105381 0 2333 383140 6 1704 0 0 0 0 0 0 0 0 0 0 44 7201 20 10 0
答案 0 :(得分:0)
Telegraf's docs
显示如何读取收集的文件的示例,
您使用文件输入,并使用data_format = "collectd"
来使用收集的解析器。
旁注,直接InfluxDB can accept CollectD。 (这可能需要此file,但似乎未包含在我的安装中)
要进行速率限制,请查看收集的interval,或者如果您更喜欢使用telegraf,它也有such an option。
我不确定influxdb的选项,但是您可能会对它们的downsampling and retention功能感兴趣。