Question

我必须将JSON加载到Bigquery中。根据Bigquery文档，我以正确的格式制作了JSON，即换行符，每行用一个JSON对象分隔。

现在，JSON文件大约有1000万行，并且在加载时出现以下错误：

读取数据时出错，错误消息：JSON表遇到太多错误，放弃了。行数：2165;错误：1.请查看错误流以获取更多详细信息。

当我找到2165行时，它看起来如下：

{"deviceId":"13231fd01222a28e","dow":"Wednesday","downloadFlag":"N","email":"clstone898@gmail.com","emailSha256":"1bdf11821f867799bde022ccb57a2e899f827c988b4275571ffd60279c863272","event":"streamStop","firebaseUID":"UDVC3hyQpBWLCnlhXhjAQBeI95Q2","halfHourFull":"08h1","liveFlag":"Y","localDate":"2018-02-07","localHalfHour":1,"login":"google","minutesSinceMidnight":497,"quarterHourFull":"08q2","stationName":"Fox hit 101.9","streamListenMethod":"BluetoothA2DPOutput","timestampLocal":"2018-02-07T08:017:04.679+11:00","timestampUTC":"2018-02-06T21:17:04.679Z"}

当我加载这一行时，它将成功加载。请指导/建议此处不正确。

我正在使用架构自动检测选项从Bigquery UI加载此json。

样本记录如下：

{"deviceId":"3c7a345dafcff93f","dow":"Tuesday","downloadFlag":"N","email":"psloper.ps@gmail.com","emailSha256":"1cebae8c35db32edcd35e746863fc65a04ac68f2f5b3350f2df477a86bfaa07d","event":"streamStop","firebaseUID":"AMFYYjsvZjauhCktJ5lUzZj0d3D2","halfHourFull":"21h2","liveFlag":"Y","localDate":"2018-02-06","localHalfHour":2,"login":"google","minutesSinceMidnight":1311,"quarterHourFull":"21q4","stationName":"hit 105","streamListenMethod":"Headphones","timestampLocal":"2018-02-06T21:51:40.216+10:00","timestampUTC":"2018-02-06T11:51:40.216Z"}
{"deviceId":"2f1a8c84c738b752","dow":"Wednesday","downloadFlag":"N","email":"kory.maxwell@icloud.com","emailSha256":"13348786c15bff95e4afb4968a9bdbe883b70206a737c02c89fc8215f2a4e101","event":"streamStop","facebookId":"1784054201892593","firebaseUID":"Tx1bHjP6dhaDB2nl2c7yi2KZHsq2","halfHourFull":"06h1","liveFlag":"Y","localDate":"2018-02-07","localHalfHour":1,"login":"facebook","minutesSinceMidnight":384,"quarterHourFull":"06q2","stationName":"hit 105","streamListenMethod":"BluetoothA2DPOutput","timestampLocal":"2018-02-07T06:24:44.533+10:00","timestampUTC":"2018-02-06T20:24:44.533Z"}
{"deviceId":"AA1D685F-6BF6-B0DC-0000-000000000000","dow":"Wednesday","email":"lozza073@bigpond.com","emailSha256":"525db286e9a35c9f9f55db0ce338762eee02c51955ede6b35afb7e808581664f","event":"streamStart","facebookId":"10215879897177171","firebaseUID":"f2efT61sW5gHTfgEbtNfyaUKWaF3","halfHourFull":"7h2","liveFlag":"Y","localDate":"2018-02-07","localHalfHour":2,"login":"facebook","minutesSinceMidnight":463,"quarterHourFull":"7q3","stationName":"Fox hit 101.9","streamListenMethod":"Speaker","timestampLocal":"2018-02-07T07:43:00.39+11:00","timestampUTC":"2018-02-06T20:43:00.39Z"}
{"deviceId":"AEFD39FC-B116-4063-0000-000000000000","dow":"Wednesday","event":"catchUpPause","facebookId":"379907925802180","firebaseUID":"vQPh9tbO3Yge88fpMyNUFzJO7dl1","halfHourFull":"7h2","liveFlag":"N","localDate":"2018-02-07","localHalfHour":2,"login":"facebook","minutesSinceMidnight":465,"quarterHourFull":"7q4","stationName":"Fox hit 101.9","streamListenMethod":"USBAudio","timestampLocal":"2018-02-07T07:45:08.524+11:00","timestampUTC":"2018-02-06T20:45:08.524Z"}
{"deviceId":"AA1D685F-6BF6-B0DC-0000-000000000000","dow":"Wednesday","email":"lozza073@bigpond.com","emailSha256":"525db286e9a35c9f9f55db0ce338762eee02c51955ede6b35afb7e808581664f","event":"streamStop","facebookId":"10215879897177171","firebaseUID":"f2efT61sW5gHTfgEbtNfyaUKWaF3","halfHourFull":"7h2","liveFlag":"Y","localDate":"2018-02-07","localHalfHour":2,"login":"facebook","minutesSinceMidnight":475,"quarterHourFull":"7q4","stationName":"Fox hit 101.9","streamListenMethod":"Speaker","timestampLocal":"2018-02-07T07:55:35.788+11:00","timestampUTC":"2018-02-06T20:55:35.788Z"}
{"deviceId":"AA1D685F-6BF6-B0DC-0000-000000000000","dow":"Wednesday","email":"lozza073@bigpond.com","emailSha256":"525db286e9a35c9f9f55db0ce338762eee02c51955ede6b35afb7e808581664f","event":"streamStart","facebookId":"10215879897177171","firebaseUID":"f2efT61sW5gHTfgEbtNfyaUKWaF3","halfHourFull":"7h2","liveFlag":"Y","localDate":"2018-02-07","localHalfHour":2,"login":"facebook","minutesSinceMidnight":477,"quarterHourFull":"7q4","stationName":"Fox hit 101.9","streamListenMethod":"Speaker","timestampLocal":"2018-02-07T07:57:42.343+11:00","timestampUTC":"2018-02-06T20:57:42.343Z"}
{"deviceId":"13231fd01222a28e","dow":"Wednesday","downloadFlag":"N","email":"clstone898@gmail.com","emailSha256":"1bdf11821f867799bde022ccb57a2e899f827c988b4275571ffd60279c863272","event":"streamStop","firebaseUID":"UDVC3hyQpBWLCnlhXhjAQBeI95Q2","halfHourFull":"08h1","liveFlag":"Y","localDate":"2018-02-07","localHalfHour":1,"login":"google","minutesSinceMidnight":497,"quarterHourFull":"08q2","stationName":"Fox hit 101.9","streamListenMethod":"BluetoothA2DPOutput","timestampLocal":"2018-02-07T08:017:04.679+11:00","timestampUTC":"2018-02-06T21:17:04.679Z"}

非常感谢您的帮助。

Answer 1

好吧，请看一下特定的2165行：

{"deviceId":"13231fd01222a28e","dow":"Wednesday","downloadFlag":"N","email":"clstone898@gmail.com","emailSha256":"1bdf11821f867799bde022ccb57a2e899f827c988b4275571ffd60279c863272","event":"streamStop","firebaseUID":"UDVC3hyQpBWLCnlhXhjAQBeI95Q2","halfHourFull":"08h1","liveFlag":"Y","localDate":"2018-02-07","localHalfHour":1,"login":"google","minutesSinceMidnight":497,"quarterHourFull":"08q2","stationName":"Fox hit 101.9","streamListenMethod":"BluetoothA2DPOutput","timestampLocal":"2018-02-07T08:017:04.679+11:00","timestampUTC":"2018-02-06T21:17:04.679Z"}

特别是：

“ timestampLocal”：“ 2018-02-07T08：017：04.679 + 11：00”

错误消息：

无法将值转换为时间戳：无法解析 '2018-02-07T08：017：04.679 + 11：00'作为时间戳记。所需格式为 YYYY-MM-DD HH：MM [：SS [.SSSSSS]]

因此，如果将“ T08：017：04.679”更改为“ T08：17：04.679”（17分钟而不是017），则它可以工作。：）

使用UI将JSON加载到Bigquery时出错

1 个答案: