由于Geojson是实际的json我以为我可以使用mongoimport从.geojson文件将数据加载到我的MongoDB数据库中。
但是我收到以下错误:
exception:BSON representation of supplied JSON is too large: code FailedToParse: FailedToParse: Expecting '{': offset:0
该文件为25MB,这是它的一个片段:
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"id": "node/2661561690",
"properties": {
"timestamp": "2014-02-08T17:58:24Z",
"version": "1",
"changeset": "20451306",
"user": "Schandlers",
"uid": "51690",
"natural": "tree",
"id": "node/2661561690"
},
"geometry": {
"type": "Point",
"coordinates": [
-66.9162255,
10.5056439
]
}
},
// ... Omitted data
{
"type": "Feature",
"id": "node/2664472516",
"properties": {
"timestamp": "2014-02-10T04:27:30Z",
"version": "2",
"changeset": "20477473",
"user": "albertoq",
"uid": "527105",
"name": "Distribuidora Brithijos (Aceites)",
"shop": "car_parts",
"id": "node/2664472516"
},
"geometry": {
"type": "Point",
"coordinates": [
-66.9388903,
10.4833647
]
}
}
]
}
答案 0 :(得分:14)
下载jq(这是类似sed的程序,但是对于JSON)
然后运行:
jq --compact-output ".features" input.geojson > output.geojson
然后
mongoimport --db dbname -c collectionname --file "output.geojson" --jsonArray
答案 1 :(得分:8)
现在你有一系列功能。 MongoDB会将此视为一个文档。尝试从geojson的开头删除以下内容:
{
"type": "FeatureCollection",
"features": [
此外,从geojson的末尾删除以下内容:
]
}
编辑 - 此外,mongo预计每行一个文档。因此,请确保您的文件之间只有\ n! e.g。
...
},\n
{
"type": "Feature",
"id": "node/2664472516",
...
答案 2 :(得分:1)
首先,为了验证您的GeoJSON文件是否准确,您可以使用Geojsonlint,QGIS等等。
然后,要将数据导入您的收藏集,请使用Mongoimport:
mongoimport --db MY_DATABASE_NAME -c MY_COLLECTION_NAME --type json --file "MY_GEOJSON_FILENAME"
用您的有效名称替换上面的3个变量。显然,请确保您当前的目录包含该文件。
答案 3 :(得分:1)
ParoX的想法效果很好,但是有16MB的限制。
- jsonArray接受在单个JSON数组中使用多个MongoDB文档表示的数据导入。仅限进口16 MB或 小。
如果文件大小超过16MB,则可以执行此操作
jq --compact-output“.features []”input.geojson> output.geojson
这将为您提供一个对象的一行,最后没有逗号。
{.....}
{.......}
{...}
{"type":"Feature","geometry":{"type":"Point","coordinates":[-80.87088507656375,35.21515162500578]},"properties":{"name":"ABBOTT NEIGHBORHOOD PARK","address":"1300 SPRUCE ST"}}
{"type":"Feature","geometry":{"type":"Point","coordinates":[-80.83775386582222,35.24980190252168]},"properties":{"name":"DOUBLE OAKS CENTER","address":"1326 WOODWARD AV"}}
{"type":"Feature","geometry":{"type":"Point","coordinates":[-80.83827000459532,35.25674709224663]},"properties":{"name":"DOUBLE OAKS NEIGHBORHOOD PARK","address":"2605 DOUBLE OAKS RD"}}
{"type":"Feature","geometry":{"type":"Point","coordinates":[-80.83697759172735,35.25751734669229]},"properties":{"name":"DOUBLE OAKS POOL","address":"1200 NEWLAND RD"}}
{"type":"Feature","geometry":{"type":"Point","coordinates":[-80.81647652154736,35.40148708491418]},"properties":{"name":"DAVID B. WAYMER FLYING REGIONAL PARK","address":"15401 HOLBROOKS RD"}}
{"type":"Feature","geometry":{"type":"Point","coordinates":[-80.83556459443902,35.39917224760999]},"properties":{"name":"DAVID B. WAYMER COMMUNITY PARK","address":"302 HOLBROOKS RD"}}
{"type":"Feature","geometry":{"type":"Polygon","coordinates":[[[-80.72487831115721,35.26545403190955],[-80.72135925292969,35.26727607954368],[-80.71517944335938,35.26769654625573],[-80.7125186920166,35.27035945142482],[-80.70857048034668,35.268257165144064],[-80.70479393005371,35.268397319259996],[-80.70324897766113,35.26503355355979],[-80.71088790893555,35.2553619492954],[-80.71681022644043,35.2553619492954],[-80.7150936126709,35.26054831539319],[-80.71869850158691,35.26026797976481],[-80.72032928466797,35.26061839914875],[-80.72264671325684,35.26033806376283],[-80.72487831115721,35.26545403190955]]]},"properties":{"name":"Plaza Road Park"}}
mongoimport --db dbname -c collectionname --file“output.geojson”--jsonArray
答案 4 :(得分:1)
此Python脚本旨在通过一步将GeoJSON文件导入MongoDB:https://github.com/rtbigdata/geojson-mongo-import.py
答案 5 :(得分:0)
如果问题是您的文档集大小超过16Mb,则可以使用batchSize
选项,该选项设置批处理中的文档数。例如:
mongoimport -d mydb -c mycol data.json -j 4 --batchSize=100
请注意-j
选项,它有助于通过使用多个工作程序来增加数据库的输出。
使用'mongoimport'的' - help'选项奇怪地没有记录batchSize
选项,去图!