在后台获取数据 - 可能使用线程

时间:2014-11-17 19:27:36

标签: python multithreading python-3.x urllib

我想使用Python 3定期从网站上获取一些数据。同时我想处理上一次请求的数据。

虽然没有响应(或者如果发生错误),我想继续使用之前的(有效)值。

以下是一个例子:

import threading
import urllib.request
import urllib.error

import time

start_time = time.time()

req = urllib.request.Request("http://fake-response.appspot.com/")

text = "Before response"


def fetch():
    try:
        response = urllib.request.urlopen(req)
        return response.read().decode('latin1')
    except (urllib.error.HTTPError, urllib.error.URLError) as e:
        print(e)

print("Initial text is \"%s\"" % text)

text = threading.Thread(target=fetch)

while time.time() - start_time < 15:
    print("%s - text: \"%s\"" % (time.strftime("%H:%M:%S"), text))
    # This should print '<TIME> - text: "Before response" for ten seconds
    # After that, '<TIME> - text: {"response":"This request has finished sleeping for 10 seconds"}' should be displayed

    time.sleep(0.5)

我找到了how to get the return value from a thread in python?,但我不知道这是否适用。

我更喜欢一个简单但不涉及额外库的解决方案(我尽量保持内存占用率低,因为这只是我项目的副业)。

1 个答案:

答案 0 :(得分:0)

不知道这是不是很好的做法,但我找到了一个更精细的基于类的设计的解决方案:

import threading
import urllib.request
import urllib.error

import time

start_time = time.time()

req = urllib.request.Request("http://fake-response.appspot.com/")


class Fetcher(threading.Thread):
    def __init__(self):
        self.text = "No response yet!"
        super().__init__()

    def run(self):
        try:
            response = urllib.request.urlopen(req)
            self.text = response.read().decode('latin1')
        except (urllib.error.HTTPError, urllib.error.URLError) as e:
            print(e)

fetcher = Fetcher()

print("Initial text is \"%s\"" % fetcher.text)

fetcher.start()

text = fetcher.text

while time.time() - start_time < 15:
    print("%s - text: \"%s\"" % (time.strftime("%H:%M:%S"), fetcher.text))
    # This should print '<TIME> - text: "No response yet!"' for ten seconds
    # After that, '<TIME> - text: {"response":"This request has finished sleeping for 10 seconds"}' should be displayed

    time.sleep(0.5)