进口期间停止运营

Question

我正在尝试开发一个我想要包含openstreetmaps的应用程序。我决定尝试使用mapbox。 我正在尝试从osm.pbf文件导入数据。我有一个多处理器系统，可以做很多工作，但在导入过程中它似乎停止工作。我只是要等待还是有错误？

ubuntu@ip-xx-xx-xx-xx:~/Documents/mapbox-osm-bright-5f237ac/shp$ sudo imposm -U postgres -d osm -m /home/ubuntu/Documents/mapbox-osm-bright-5f237ac/imposm-mapping.py --read --write --optimize --concurrency 10 --cache-dir /mnt/cache --overwrite-cache --deploy-production-tables /mnt/cc/sweden-latest.osm.pbf
loading /home/ubuntu/Documents/mapbox-osm-bright-5f237ac/imposm-mapping.py as mapping
password for postgres at localhost:
[11:30:54] ## reading /mnt/cc/sweden-latest.osm.pbf
[11:30:54] coords: 17545k nodes: 350k ways: 2506k relations: 17k (estimated)
[11:32:02] coords: 26272k nodes: 101k ways: 28k relations: 0k

然后我在很长一段时间后按Ctrl-C

^CProcess CacheWriterProcess-3:: 101k ways: 28k relations: 0k
Process CacheWriterProcess-5:
Traceback (most recent call last):
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process ParserProgress-1:
Traceback (most recent call last):
    self.run()
    self.run()
  File "/usr/local/lib/python2.7/dist-packages/imposm/reader.py", line 123, in run
  File "/usr/local/lib/python2.7/dist-packages/imposm/reader.py", line 123, in run
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    data = self.queue.get()
    data = self.queue.get()
  File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get
  File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get
    res = self._recv()
    res = self._recv()
KeyboardInterrupt
KeyboardInterrupt
    self.run()
  File "/usr/local/lib/python2.7/dist-packages/imposm/util/__init__.py", line 54, in run
    log_statement = self.queue.get()
  File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get
Process CacheWriterProcess-2:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    res = self._recv()
KeyboardInterrupt
    self.run()
  File "/usr/local/lib/python2.7/dist-packages/imposm/reader.py", line 123, in run
    data = self.queue.get()
  File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get
    res = self._recv()
KeyboardInterrupt
Process CacheWriterProcess-4:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/local/lib/python2.7/dist-packages/imposm/reader.py", line 123, in run
Traceback (most recent call last):
  File "/usr/local/bin/imposm", line 9, in <module>
    data = self.queue.get()
  File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get
    load_entry_point('imposm==2.5.0', 'console_scripts', 'imposm')()
  File "/usr/local/lib/python2.7/dist-packages/imposm/app.py", line 253, in main
    res = self._recv()
KeyboardInterrupt
    reader.read(arg)
  File "/usr/local/lib/python2.7/dist-packages/imposm/reader.py", line 88, in read
    parser.parse(filename)
  File "/usr/local/lib/python2.7/dist-packages/imposm/parser/simple.py", line 62, in parse
    return self.parse_pbf_file(filename)
  File "/usr/local/lib/python2.7/dist-packages/imposm/parser/simple.py", line 73, in parse_pbf_file
    return self._parse(filename, PBFMultiProcParser)
  File "/usr/local/lib/python2.7/dist-packages/imposm/parser/simple.py", line 140, in _parse
    time.sleep(0.001)
KeyboardInterrupt
Process Process-6:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python2.7/dist-packages/imposm/parser/simple.py", line 113, in parse_it
    parser.parse(input)
  File "/usr/local/lib/python2.7/dist-packages/imposm/parser/pbf/multiproc.py", line 70, in parse
    pos_queue.put(pos)
  File "/usr/lib/python2.7/multiprocessing/queues.py", line 311, in put
    if not self._sem.acquire(block, timeout):
KeyboardInterrupt

现在刚刚工作2-3％而且似乎永远不会芬兰，任何想法？

Answer 1

这是因为imposm中存在错误。要安装补丁来修复它：

pip install https://github.com/omniscale/imposm-parser/tarball/master

Answer 2

可以通过下载未打包的OSM-XML文件来规避它。

在某些情况下，Imposm在阅读PBF格式时遇到问题。但是解压缩的OSM-XML文件可以解决问题。

在丹麦，它是pbf-file（166 MB）与osm-xml-file（261 MB，4.4 GB，解压缩）

root@drake:/mnt/disk/maps# imposm -U postgres -d osm -m /usr/local/share/maps/style/osm-bright-master/imposm-mapping.py --cache-dir cache  --read --write --optimize --deploy-production-tables denmark-latest.osm --overwrite-cache
Enabling Shapely speedups.
loading /usr/local/share/maps/style/osm-bright-master/imposm-mapping.py as mapping

[14:53:01] ## reading denmark-latest.osm
[14:53:01] coords: 22391k nodes: 447k ways: 3198k relations: 22k (estimated)
[14:58:38] coords: 15171k nodes: 33k ways: 1587k relations: 6k
[14:58:39] reading took 5 m 37s
[14:58:39] ## dropping/creating tables
[14:58:39] ## writing data
[14:58:47] relations: 6k/6k
[14:58:51] relations: total time 12s for 6783 (562/s)
[15:08:18] ways: 1587k/1587k
[15:08:19] ways: total time 9 m 28s for 1587006 (2793/s)
[15:08:27] nodes: 32k/33k
[15:08:28] nodes: total time 8 s for 33279 (3805/s)
[15:08:28] ## creating generalized tables
[15:08:57] generalizing tables took 28s
[15:08:57] ## creating union views
[15:08:57] creating views took 0 s
[15:08:57] ## creating geometry indexes
[15:09:18] creating indexes took 21s
[15:09:18] writing took 10m 39s
[15:09:18] ## optimizing tables
Clustering table osm_new_mainroads
Clustering table osm_new_buildings
Clustering table osm_new_mainroads_gen1
Clustering table osm_new_mainroads_gen0
Clustering table osm_new_amenities
Clustering table osm_new_waterareas_gen1
Clustering table osm_new_waterareas_gen0
Clustering table osm_new_motorways_gen0
Clustering table osm_new_aeroways
Clustering table osm_new_motorways
Clustering table osm_new_transport_points
Clustering table osm_new_railways_gen0
Clustering table osm_new_railways_gen1
Clustering table osm_new_landusages
Clustering table osm_new_waterways
Clustering table osm_new_railways
Clustering table osm_new_motorways_gen1
Clustering table osm_new_waterareas
Clustering table osm_new_places
Clustering table osm_new_admin
Clustering table osm_new_barrierways
Clustering table osm_new_minorroads
Clustering table osm_new_barrierpoints
Clustering table osm_new_landusages_gen1
Clustering table osm_new_landusages_gen0
Vacuum analyze
[15:09:53] optimizing took 35s
[15:09:53] imposm took 16m 52s