Question

我有一个机器数据进入hdfs，如下所示，第8个字段是UTC时间（060037），我需要将其转换为IST并将时间格式设为hh：mm：ss using pig

VTS,01,0097,9739965515,NM,GP,20,060037,V,0000.0000,N,00000.0000,E,0.0,0.0,061114,0068,00,4000,00,999,149,9594
VTS,01,0097,9739965515,SP,GP,33,060113,V,0000.0000,N,00000.0000,E,0.0,0.0,061114,0068,00,4000,00,999,152,B927

使用字符串函数我试图将其转换为unix日期格式现在我得到时间像2014-11-06 06:01:13它的UTC格式如何将其转换为IST是否有任何内置函数可用于执行此操作？

A = LOAD '/user/hue/Anas' AS (line:chararray);
B = FOREACH A {
             splitRow = TOKENIZE(line,'+++');
             GENERATE FLATTEN(splitRow) AS newList;
          }
C = FOREACH B GENERATE FLATTEN(STRSPLIT(newList,',',23));
D = FILTER C BY $1==01;
E = foreach D generate $7 as time,$15 as date;
F = foreach E generate SUBSTRING(time,0,2) as hh,SUBSTRING(time,2,4) as mm,SUBSTRING(time,4,6) as ss,SUBSTRING(date,0,2) as date,SUBSTRING(date,2,4) as month,SUBSTRING(date,4,6) as year;
G = foreach F generate CONCAT('20',CONCAT(year,CONCAT('-',CONCAT(month,CONCAT('-',date))))) as date,CONCAT(hh,CONCAT(':',CONCAT(mm,CONCAT(':',ss)))) as time;
H = FOREACH G GENERATE CONCAT(date,CONCAT(' ',time)) AS UTC;
DUMP H;

Answer 1

请将以下3行添加到现有代码中，它将起作用

I = FOREACH H GENERATE ToDate(UTC,'yyyy-MM-dd HH:mm:ss','UTC') AS UTCTime;
J = FOREACH I GENERATE ToDate(ToString(UTCTime,'yyyy-MM-dd HH:mm:ss.SSSZ'),'yyyy-MM-dd HH:mm:ss.SSSZ','Asia/Kolkata') AS ISTTime;
DUMP J

UTC时间的输出：

(2014-11-06 06:00:37)
(2014-11-06 06:01:13)

IST时间的输出：

(2014-11-06T11:30:37.000+05:30)
(2014-11-06T11:31:13.000+05:30)

此 ISTTime 位于datetime对象中，现在您可以使用所有内置函数（GetDay（），GetTime（）等）。

Answer 2

你尝试过使用皮球吗？

我认为格式化功能可以满足您的需求。

https://gist.github.com/griggheo/1780912

您可能必须为IST转换编写自己的UDF，但可以使用python或ruby或....进行内联。

如何使用pig将UTC时间转换为IST

2 个答案: