Question

所以我有一个程序，我将一些数据预加载到lru缓存中，并且运行多个并行处理。有两个主要的预加载缓存调用：openExcelWithCache和returnReferenceDataWithCache。这两个函数的实现如下：

from functools import lru_cache

@lru_cache(maxsize=32)
def openExcelWithCache(pathFile, blDataOnly):
    return(openpyxl.load_workbook(file,data_only = blDataOnly)

@lru_cache(maxsize=32)
def returnReferenceDataWithCache(tableName, id):
    logAtLevel("INFO", "Retrieving uncached reference for Id: " + str(id) + "(" + str(type(id)) + ") table: " + tableName + ".")
    return self.interface.referencequery(tbl, id)

我的并行类设置如下，并调用上述函数进行预加载：

from multiprocessing.dummy import Pool

#Pre-loading
openExcelWithCache('File1',True)
openExcelWithCache('File1',False)
openExcelWithCache('File2',True)

returnReferenceDataWithCache('Tbl1', 1)


lsFolders = ['Folder1', 'Folder2']

pool = Pool( processes = 6 )

 # instantiated by call to 
pool.map(worker, lsFolders)

def worker(clientFolder):
    iterateThroughFiles = IterateThroughFiles()
    iterateThroughFiles.runProcess(clientFolder)

IterateThroughFiles类是：

class IterateThroughFiles( object ):

  def runProcess( self, folder ):


    openExcelWithCache('File1',True)
    print(openExcelWithCache.cache_info()) #outputs: CacheInfo(hits=1,misses=3,maxsize=32,currsize=3)
    openExcelWithCache('File1',False)
    print(openExcelWithCache.cache_info()) #outputs: CacheInfo(hits=2,misses=3,maxsize=32,currsize=3)
    openExcelWithCache('File2',True)
    print(openExcelWithCache.cache_info()) #outputs: CacheInfo(hits=3,misses=3,maxsize=32,currsize=3)


    returnReferenceDataWithCache('Tbl1', 1)
    print(retrieveReferenceDataWithCache.cache_info()) #outputs: CacheInfo(hits=0,misses=2,maxsize=32,currsize=2)

因此，出于某种原因，openExcelWithCache缓存正常工作，但不是returnReferenceDataWithCache函数的缓存。同一个呼叫仍然会导致“未命中”并生成新的密钥对。我检查了'1'的类型对于两者都是相同的，所以我不确定这里发生了什么。

我为写这个有点伪代码而道歉，如果还有其他需要澄清的话，请告诉我。

缓存在并行进程中未按预期运行

0 个答案: