将大型json文件导入MongoDB

时间:2013-03-06 23:20:34

标签: json mongodb mongoimport database

所以我有一个有多条记录的json文件,看起来像这样......

{"2597401":[{
"jobID":"2597401",
"account":"TG-CCR120014",
"user":"c",
"pkgT":{"pgi/7.2-5":{"libA":["libpgc.so"],"flavor":["default"]}},
"startEpoch":"1338497979",
"runTime":"1022",
"execType":"user:binary",
"exec":"/share/home/01482/c/appker/ranger/NPB3.3.1/NPB3.3-MPI/bin/ft.D.64",
"numNodes":"4",
"sha1":"5a79879235aa31b6a46e73b43879428e2a175db5",
"execEpoch":1336766742,
"execModify":"Fri May 11 15:05:42 2012",
"startTime":"Thu May 31 15:59:39 2012",
"numCores":"64",
"sizeT":{"bss":"1881400168","text":"239574","data":"22504"}},

{"jobID":"2597401",
"account":"TG-CCR120014",
"user":"c",
"pkgT":{"pgi/7.2-5":{"libA":["libpgc.so"],"flavor":["default"]}},
"startEpoch":"1338497946",
"runTime":"33",
"execType":"user:binary",
"exec":"/share/home/01482/c/appker/ranger/NPB3.3.1/NPB3.3-MPI/bin/cg.C.64",
"numNodes":"4",
"sha1":"caf415e011e28b7e4e5b050fb61cbf71a62a9789",
"execEpoch":1336766735,
"execModify":"Fri May 11 15:05:35 2012",
"startTime":"Thu May 31 15:59:06 2012",
"numCores":"64",
"sizeT":{"bss":"29630984","text":"225749","data":"20360"}},

{"jobID":"2597401",
"account":"TG-CCR120014",
"user":"c",
"pkgT":{"pgi/7.2-5":{"libA":["libpgc.so"],"flavor":["default"]}},
"startEpoch":"1338500447",
"runTime":"145",
"execType":"user:binary",
"exec":"/share/home/01482/appker/ranger/NPB3.3.1/NPB3.3-MPI/bin/mg.D.64",
"numNodes":"4",
"sha1":"173de32e1514ad097b1c051ec49c4eb240f2001f",
"execEpoch":1336766756,
"execModify":"Fri May 11 15:05:56 2012",
"startTime":"Thu May 31 16:40:47 2012",
"numCores":"64",
"sizeT":{"bss":"456954120","text":"426186","data":"22184"}},

{"2597401":[{....

每条记录都在一行上。所以json文件的一般格式如下:

{"1111111: [{"jobID":value,"account":value,"user":value,"pkgT":{value:{"libA":[VALUES],"flavor":[value]}},"startEpoch":value,"runTime":value,"execType":value,"exec":value,"numNodes":value,"sha1":value,"execEpoch":value,"execModify":value,"startTime":value,"numCores":value,"sizeT":{"bss":value,"text":value,"data":value}}  

我正在尝试使用mongoimport使用以下命令将它们导入数据库:

mongoimport --db dbName --collection collectionName fileName --jsonArray

我读到每条记录都应该在它自己的行上,就是这样。但是,当我导入文件时,它将其作为单个文档(也称为记录)导入。如何让mongo将此文件解释为包含多个记录而不是单个巨大的文档?谢谢你的帮助!!!

1 个答案:

答案 0 :(得分:1)

我能够解决它。我有大括号围绕文件中的所有文本。当我删除它们时,它成功地单独导入了记录。