穿越大阵列的小阵列

时间:2016-02-23 17:42:31

标签: python numpy

我有一个服务器列表,例如

servrs = ['A','B','C']

然后是大约100,000英尺长的坐标列表( route_csv

例如

['query_id', 'from_lat', 'from_lon', 'to_lat', 'to_lon']
[1, 51.2, -0.5, 51.3, -0.6]

我想做的事情(出于多线程的目的)是将坐标列表分成与我有服务器一样多的块,然后将每个块与服务器相关联。订单并不重要

我的方式很糟糕,因为我正在循环一个像这样的numpy数组:

def CreateUrls(route_csv, servrs):
    out_urls = []
    i = 0
    while True:
        for x in servrs:
            out_urls.append(["%s/viaroute?loc=%.5f,%.5f&loc=%.5f,%.5f&alt=false&geometry=false" %
                            (x,
                             route_csv[i, ]['from_lat'],
                             route_csv[i, ]['from_lon'],
                             route_csv[i, ]['to_lat'],
                             route_csv[i, ]['to_lon']),
                             route_csv[i, ]['query_id']])
            i += 1
            if i == len(route_csv):
                return out_urls

编辑:为了提供更多的上下文,然后像这样输入脚本;

def RegOsrm(url_input):
    url_to_geocode, query_id = url_input
    try_c = 0
    while try_c < 3:
        try:
            response = requests.get(url_to_geocode)
            json_geocode = response.json()
            status = int(json_geocode['status'])
            # Found route between points
            if status == 200:
                tot_time_s = json_geocode['route_summary']['total_time']
                tot_dist_m = json_geocode['route_summary']['total_distance']
                used_from = json_geocode['via_points'][0]
                used_to = json_geocode['via_points'][1]
                out = [query_id,
                       status,
                       tot_time_s,
                       tot_dist_m,
                       used_from[0],
                       used_from[1],
                       used_to[0],
                       used_to[1]]
                print(out)
                return out
            # Cannot find route between points (code errors as 999)
            elif status == 207:
              return [query_id, 999, 0, 0, 0, 0, 0, 0]
        except Exception:
            time.sleep(2)
            try_c += 1
    print("Failed: %d %s" % (query_id, url_to_geocode))
    return [query_id, 999, 0, 0, 0, 0, 0, 0]

url_routes = CreateUrls(route_csv=routes, servrs=servrs)
del routes, servrs
print("Created %d urls" % len(url_routes))

# Calculate routes using all threads
pool = Pool()
calc_routes = pool.map(RegOsrm, url_routes)
pool.close()
pool.join()

0 个答案:

没有答案