使用python在Google Earth Engine中查询大型FeatureCollection

时间:2017-07-13 11:58:49

标签: python shapefile google-earth-engine

我正在尝试从here下载的坦桑尼亚形状文件。

    # im -> {Image} ee.Image({...})
    # self.geom_coll -> {FeatureCollection} ee.FeatureCollection({...}). containing 
    # 3000 features.
    # spacereducer() -> ee.Reducer.mean
    # self.scale -> 10 #Changing this value to small number gives error

    feats = im.reduceRegions(self.geom_coll, spacereducer(), self.scale)
    flist = getInfo_werrorcontrol(feats,
                          self.errorcheck)['features']

def getInfo_werrorcontrol(featureCollection, errorcontrolon=True):
    """
    Wrapper to add error control to GEE evaluations.

    For large computations GEE sometimes times out and needs to be
    restarted. This does so in a controlled manner with out 
    interrrupting the program flow.
    """
    if errorcontrolon:
        i=0
        while True:
            try:
                with timeout.timeout(10*60):
                    return featureCollection.getInfo() # In this line I am getting exception.
            except NameError:
                exc_type, exc_value, exc_traceback = sys.exc_info()
                lines = traceback.format_exception(exc_type, exc_value, exc_traceback)
                print ''.join('!! ' + line for line in lines)
                i+=1
                print 'attempts: '+str(i)
                if i > 20:
                    raise ValueError('to many attempts') 
                elif i > 10:
                    print 'waiting 2 minutes'
                    time.sleep(60*2)
    else:
        return featureCollection.getInfo()

self.scale更改为10会引发以下错误:

featureCollection.getInfo()

  

ee.ee_exception.EEException:服务器返回HTTP代码:413

self.scale更改为1000会引发此错误:

  

ee.ee_exception.EEException:计算超时

使用较大区域处理形状文件的正确方法是什么?

1 个答案:

答案 0 :(得分:1)

请注意,对Earth Engine API的请求为already wrapped with exponential backoff。因此,您的代码无法解决问题。这些错误是如何从您发布的代码中产生的,但在任何一种情况下,答案可能都是相同的:导出结果。例如:

printf "%s\n" "${GenderArray[@]}" |
awk 'NR==FNR{g[NR+1]="\t" $0; next} {print $0 g[FNR]}' - file

导出的输出将在您的Google云端硬盘文件夹中显示。要详细了解规模,请参阅this doc