在Java中使用Regex解析csv文件' |'作为分隔符

时间:2014-06-24 13:55:29

标签: java regex parsing csv

任何人都可以帮我使用正则表达式吗? 我有一个Java程序,它读入.csv文件以将其加载到数据库中。

目前使用Pattern csvPattern = Pattern.compile("\\s*(\"[^\"]*\"|[^|]*)\\s*,?");

但是我知道matcher = csvPattern.matcher(line);逐行读取文件我只得到空值。 这些文件具有以下格式(许多morre行,其中一些使用逗号,' |'作为分隔符,并在每行的末尾):

第一个文件的抽象:

0|ALGERIA|0| haggle. carefully final deposits detect slyly agai|
1|ARGENTINA|1|al foxes promise slyly according to the regular accounts. bold requests alon|
2|BRAZIL|1|y alongside of the pending deposits. carefully special packages are about the ironic forges. slyly special |

第二

|Customer#000000001|IVhzIApeRb ot,c,E|15|25-989-741-2988|711.56|BUILDING|to the even, regular platelets. regular, ironic epitaphs nag e|
2|Customer#000000002|XSTf4,NCwDVaWNe6tEgvwfmRchLXak|13|23-768-687-3665|121.65|AUTOMOBILE|l accounts. blithely ironic theodolites integrate boldly: caref|
3|Customer#000000003|MG9kdTD2WBHm|1|11-719-748-3364|7498.12|AUTOMOBILE| deposits eat slyly ironic, even instructions. express foxes detect slyly. blithely even accounts abov|
4|Customer#000000004|XxVSJsLAGtn|4|14-128-190-5944|2866.83|MACHINERY| requests. final, regular ideas sleep final accou|

第三

5|Supplier#000000005|Gcdm2rJRzl5qlTVzc|11|21-151-690-3663|-283.84|. slyly regular pinto bea|
6|Supplier#000000006|tQxuVm7s7CnK|14|24-696-997-4969|1365.79|final accounts. regular dolphins use against the furiously ironic decoys. |
7|Supplier#000000007|s,4TicNGB4uO6PaSqNBUq|23|33-990-965-2201|6820.35|s unwind silently furiously regular courts. final requests are deposits. requests wake quietly blit|
8|Supplier#000000008|9Sq4bBH2FQEmaFOocY45sRTxo6yuoG|17|27-498-742-3860|7627.85|al pinto beans. asymptotes haggl|
9|Supplier#000000009|1KhUgZegwM3ua7dsYmekYBsK|10|20-403-398-8662|5302.37|s. unusual, even requests along the furiously regular pac|

第四:

1|2|3325|771.64|, even theodolites. regular, final theodolites eat after the carefully pending foxes. furiously regular deposits sleep slyly. carefully bold realms above the ironic dependencies haggle careful|
1|2502|8076|993.49|ven ideas. quickly even packages print. pending multipliers must have to are fluff|
1|5002|3956|337.09|after the fluffily ironic deposits? blithely special dependencies integrate furiously even excuses. blithely silent theodolites could have to haggle pending, express requests; fu|
1|7502|4069|357.84|al, regular dependencies serve carefully after the quickly final pinto beans. furiously even deposits sleep quickly final, silent pinto beans. fluffily reg|

第五:

1|155190|7706|1|17|21168.23|0.04|0.02|N|O|1996-03-13|1996-02-12|1996-03-22|DELIVER IN PERSON|TRUCK|egular courts above the|
1|67310|7311|2|36|45983.16|0.09|0.06|N|O|1996-04-12|1996-02-28|1996-04-20|TAKE BACK RETURN|MAIL|ly final dependencies: slyly bold |

第六:

134823|saddle midnight thistle honeydew lime|Manufacturer#4|Brand#43|STANDARD BURNISHED BRASS|44|WRAP CAN|1857.82|ges. furiously ir|
134824|coral red indian thistle sandy|Manufacturer#5|Brand#55|PROMO BURNISHED COPPER|29|LG JAR|1858.82|final p|
134825|saddle purple orchid cornsilk medium|Manufacturer#4|Brand#44|PROMO POLISHED NICKEL|21|LG CASE|1859.82|nal accounts us|
134826|turquoise sky lime cornsilk peach|Manufacturer#1|Brand#11|SMALL BURNISHED TIN|25|SM CAN|1860.82| haggle|

第七:

0|AFRICA|lar deposits. blithely final packages cajole. regular waters are final requests. regular accounts are according to |
1|AMERICA|hs use ironic, even requests. s|

第八:

4|136777|O|32151.78|1995-10-11|5-LOW|Clerk#000000124|0|sits. slyly regular warthogs cajole. regular, regular theodolites acro|
5|44485|F|144659.20|1994-07-30|5-LOW|Clerk#000000925|0|quickly. bold deposits sleep slyly. packages use slyly|

(csv是使用TPC fpr tpc-h中的DBGen工具创建的,如果你想知道的话)

我希望你明白我需要什么,并可以帮助我解决这个问题。非常感谢你!

编辑:使用String.split(" |");'确实很明显,但事实是,我正在使用的程序非常复杂,并且在各个部分都依赖于regex.pattern和regex.matcher。因为我对程序和java本身不是很熟悉,对我来说唯一的解决方案是使用给定的代码,只需用正常的表达式替换正则表达式。

EDIT2:我试图在OLTP-Bench中使用这个TPC-H实现:https://github.com/ben-reilly/oltpbench/blob/master/src/com/oltpbenchmark/benchmarks/tpch/TPCHLoader.java#L347

其中有问题的行是347.它是TPC-H数据库基准测试的完整实现,但没有数据生成器。所以我使用TPC提供的dbgen工具来生成csv文件。我无法与开发人员保持联系。

2 个答案:

答案 0 :(得分:0)

我建议使用String.split("|");。这将为您提供一系列字符串,表示您的行周围的文本。

但是,如果你真的想使用正则表达式:

Pattern csvPattern = Pattern.compile("\\s*(\\d*)\\|(([^|]+)\\|)+");

这个应该匹配一个数字(或没有),一个管道,然后重复模式Something|

答案 1 :(得分:0)

根据你的来源,你可能只是用管道替换逗号,因为从注释中,所有模式都是将字符串拆分为分隔符(双引号中的字符串除外)

例如:来自

\\s*(\"[^\"]*\"|[^,]*)\\s*,?

\\s*(\"[^\"]*\"|[^|]*)\\s*\\|?

至于你的号码例外,你需要调试你调用CSV加载器的方式。

我之前从未使用过该工具,但是如果你看看第352行

for (int i = 0; i < types.length; ++i) {

现在看一下从第362行开始的开关块:它定义了每个字段应该转换成的类型。

switch(types[i]) {
    case DOUBLE:
        prepStmt.setDouble(i+1, Double.parseDouble(field));
        break;
...

如果您没有正确指定类型,这种类型的转换可能会导致问题。