如何修改下面的代码来处理时区,请注意没有小数。
2015-12-22T11:57:11-08:00
,-8:00
导致我出现问题,纪元时间会考虑时区吗?
timegm(datetime.strptime(datestring, "%Y-%m-%dT%H:%M:%S.%f").timetuple())
答案 0 :(得分:0)
这里有一些问题。
回答你的问题,是的,时区很重要。 UNIX纪元是自1970-01-01T00:00:00 + 00:00以来的秒数。更重要的是,即使您在解析字符串时正确分配#Agent components
a1.sources = src1
a1.channels = chan1
a1.sinks = sink1
#configuring the souce
a1.sources.src1.type = spooldir
a1.sources.src1.spoolDir = /home/hdeveloper/Logs
a1.sources.src1.deserializer=org.apache.flume.sink.solr.morphline.BlobDeserializer$Builder
#configuring the channel
a1.channels.chan1.type = file
#Checkpoint directory
a1.channels.chan1.checkpointDir = /home/hdeveloper/flume-metadata/.flume/file-channel/checkpoint
#log files created in Data directory while running flume-agent
a1.channels.chan1.dataDirs = /home/hdeveloper/flume-metadata/.flume/file-channel/data
#configuring the Sink
a1.sinks.sink1.type = hdfs
a1.sinks.sink1.hdfs.path = hdfs://localhost:9000/twitterdata
a1.sinks.sink1.hdfs.useLocalTimeStamp = true
a1.sinks.sink1.hdfs.fileType = CompressedStream
a1.sinks.sink1.hdfs.codeC = GzipCodec
a1.sources.src1.channels = chan1
a1.sinks.sink1.channel = chan1
------------------------------------------------
command to execute :
[hdeveloper@localhost apache-flume-1.6.0-bin]$ bin/flume-ng agent --conf ./conf --conf-file ./conf/spool.conf --name a1 -Dflume.root.logger=INFO,console
对象的datetime
,tzinfo
也不会考虑timetuple
。您需要使用tzinfo
所以现在你只需要正确解析日期时间。有些解决方案不使用外部库,但我发现解析ISO8601日期字符串的最简单方法是使用pip提供的python-dateutil包:
utctimetuple