Question

我见过一些类似的问题，但由于问题不完全相同，或者解决方案不适用于我的情况，我在这里发布我的问题。

我正在解析包含csv_line列中的csv行的表。问题是某些列的逗号,也是字段分隔符。这些列嵌入引号中。

我正在做的解析是：

with  
sample as (
select 'field1,field3,"http://another.domain/abc/...eIds=111,222,333,444,...,",CustomerX,end' as csv_line)

select 

 regexp_extract(csv_line,'(,?(".*?"|[^,]*)){1}') as f1
 regexp_extract(csv_line,'(,?(".*?"|[^,]*)){n}') as fn

from raw_sample

我试图替换字符/逗号。

我知道 OpenCSVSerde 允许在Create表中定义分隔符和转义双引号，但我可能会查找可以设置的属性，或者可能是正则表达式，可以用正确的方式进行拆分。

提前致谢

Answer 1

with raw_sample as (
select 'field1,field2,fiend3,123,456,"http://some.domain/abc/Player.aspx?playerID=111&BrowseIds=2221,423062611,423870887,424044345,...,",THIS_IS_MY,en,20 294 998 1001,end' as raw_line
)

select  regexp_extract(raw_line,'(,?(".*?"|[^,]*)){01}',2) as c01
       ,regexp_extract(raw_line,'(,?(".*?"|[^,]*)){02}',2) as c02
       ,regexp_extract(raw_line,'(,?(".*?"|[^,]*)){03}',2) as c03
       ,regexp_extract(raw_line,'(,?(".*?"|[^,]*)){04}',2) as c04
       ,regexp_extract(raw_line,'(,?(".*?"|[^,]*)){05}',2) as c05
       ,regexp_extract(raw_line,'(,?(".*?"|[^,]*)){06}',2) as c06
       ,regexp_extract(raw_line,'(,?(".*?"|[^,]*)){07}',2) as c07
       ,regexp_extract(raw_line,'(,?(".*?"|[^,]*)){08}',2) as c08
       ,regexp_extract(raw_line,'(,?(".*?"|[^,]*)){09}',2) as c09
       ,regexp_extract(raw_line,'(,?(".*?"|[^,]*)){10}',2) as c10

from    raw_sample
;

+--------+--------+--------+-----+-----+-----------------------------------------------------------------------------------------------------+------------+-----+-----------------+-----+
|  c01   |  c02   |  c03   | c04 | c05 |                                                 c06                                                 |    c07     | c08 |       c09       | c10 |
+--------+--------+--------+-----+-----+-----------------------------------------------------------------------------------------------------+------------+-----+-----------------+-----+
| field1 | field2 | fiend3 | 123 | 456 | "http://some.domain/abc/Player.aspx?playerID=111&BrowseIds=2221,423062611,423870887,424044345,...," | THIS_IS_MY | en  | 20 294 998 1001 | end |
+--------+--------+--------+-----+-----+-----------------------------------------------------------------------------------------------------+------------+-----+-----------------+-----+

HIVE - 手动解析数据用双引号括起并用逗号分隔

1 个答案: