加速Pandas申请带有请求的API调用

时间:2018-01-18 16:24:24

标签: python multithreading python-3.x pandas python-requests

我有一个df中的IP地址列表。这些IP地址使用@ApplicationPath("/") public class NfvDeployerApp extends Application{ private Set<Object> resources = new HashSet<Object>(); public NfvDeployerApp() { resources.add(new ServiceMain()); } } 在GET请求中发送到ARIN数据库,我有兴趣获取该IP地址的组织或客户。我在public static boolean initDB() { if(initialized) { return initialized; } else { deployedNffgSet = new ConcurrentHashMap<>(); NfvReaderFactory factory = NfvReaderFactory.newInstance(); try { monitor = factory.newNfvReader(); } catch (NfvReaderException e) { e.printStackTrace(); } activeHosts = readHosts(); vnfCatalog = readVnfTypes(); monitorNffgs = readNffgs(); nffgNodesMap = new ConcurrentHashMap<>(); nffgHostsMap = new ConcurrentHashMap<>(); nffgNodeRelationships = new ConcurrentHashMap<>(); nffgHostsRelationships = new ConcurrentHashMap<>(); return true; } } requests内使用了requests Session(),希望能够加快API调用速度。这是代码:

requests-futures

添加常规FuturesSession() s = requests.Session() session = FuturesSession(session=s, max_workers=10) def getIPAddressOrganization(IP_Address): url = 'https://whois.arin.net/rest/ip/' + IP_Address + '.json' request = session.get(url) response = request.result().json() try: organization = response['net']['orgRef']['@name'] except KeyError: organization = response['net']['customerRef']['@name'] return organization df['organization'] = df['IP'].apply(getIPAddressOrganization) 可以帮助提高性能,但requests Session()没有帮助(可能是由于我缺乏知识)。

requests-futures FuturesSession()应该如何与pandas一起使用,和/或是否有其他方法可以加快API调用的效果?

0 个答案:

没有答案