threading.Thread卡在调用httplib2.Http.request上

时间:2014-01-12 13:12:26

标签: python python-multithreading httplib2

剧本的前几行解释了结构和机制。

我面临的问题是执行工作卡在第53行。一旦下载程序对第一个请求执行操作,它就会正确生成api,但在到达http_object.request(audioscrobbler_api)时它会卡住。

脚本在另一个系统上编码和测试,并产生了正确的结果。

我可以确认httplib2包没有被破坏,因为它正常运行,而该库的方法(包括request)是从其他脚本调用的。

导致脚本卡住的原因是什么?

脚本:

#
# Album artwork downloading module for Encore Music Player application.
# Loosely based on the producer-consumer model devised by E W Djikstra.
#
# The Downloader class (implemented as a daemon thread) acts as the consumer
# in the system where it reads requests from the buffer and tries to fetch the
# artwork from ws.audioscrobbler.com (LastFM's web service portal).
#
# Requester class, the producer, is a standard thread class that places the request
# in the buffer when started.
#
# DBusRequester class provides an interface to the script and is made available on
# the session bus of the DBus daemon under the name of 'com.encore.AlbumArtDownloader'
# which enables the core music player to request downloads.
#

import threading, urllib, httplib2, md5, libxml2, os, dbus, dbus.service, signal
from collections import deque
from gi.repository import GObject
from dbus.mainloop.glib import DBusGMainLoop

requests = deque()
mutex    = threading.Lock()
count    = threading.Semaphore(0)

DBusGMainLoop(set_as_default = True)

class Downloader(threading.Thread):

        def __init__(self):
                threading.Thread.__init__(self)

        def run(self):

                while True:
                        print "=> Downloader waiting for requests"
                        count.acquire()  # wait for new request if buffer is empty

                        mutex.acquire()  # enter critical section
                        request = requests.popleft()
                        mutex.release()  # leave critical section

                        (p, q) = request

                        try:
                                print "=> Generating api for %s by %s" % (p,q) 
                                params             = urllib.urlencode({'method': 'album.getinfo', 'api_key': 'XXX', 'artist': p, 'album': q})
                                audioscrobbler_api = "http://ws.audioscrobbler.com/2.0/?%s" % params
                                print "=> Generated URL %s" % (audioscrobbler_api)

                                http_object   = httplib2.Http()
                                print "=> Requesting response"
                                resp, content = http_object.request(audioscrobbler_api)
                                print "=> Received response"

                                if not resp.status == 200:
                                        print "Unable to fetch artwork for %s by %s" % (q, p)
                                        continue                 # proceed to the next item in queue if request fails

                                doc  = libxml2.parseDoc(content)
                                ctxt = doc.xpathNewContext()
                                res  = ctxt.xpathEval("//image[@size='medium']") # grab the element containing the link to a medium sized artwork

                                if len(res) < 1:
                                        continue                 # proceed to the next item in queue if the required image node is not found

                                image_uri = res[0].content           # extract uri from node

                                wget_status = os.system("wget %s -q --tries 3 -O temp" % (image_uri))

                                if not wget_status == 0:
                                        continue                 # proceed to the next item in queue if download fails

                                artwork_name = "%s.png" % (md5.md5("%s + %s" % (p, q)).hexdigest())

                                os.system("convert temp -resize 64x64 %s" % artwork_name)
                        except:
                                pass                         # handle http request error 

class Requester(threading.Thread):

        def __init__(self, request):
                self.request = request
                threading.Thread.__init__(self)

        def run(self):
                mutex.acquire() # enter critical section
                if not self.request in requests:
                        requests.append(self.request)
                        count.release() # signal downloader

                        mutex.release() # leave critical section

class DBusRequester(dbus.service.Object):

        def __init__(self):
                bus_name = dbus.service.BusName('com.encore.AlbumArtDownloader', bus=dbus.SessionBus())
                dbus.service.Object.__init__(self, bus_name, '/com/encore/AlbumArtDownloader')

        @dbus.service.method('com.encore.AlbumArtDownloader')
        def queue_request(self, artist_name, album_name):

                request   = (artist_name, album_name)
                requester = Requester(request)
                requester.start()

def sigint_handler(signum, frame):
        """Exit gracefully on receiving SIGINT."""

        loop.quit()

signal.signal(signal.SIGINT, sigint_handler)

downloader_daemon = Downloader()
downloader_daemon.daemon = True
downloader_daemon.start()

requester_service = DBusRequester()

loop = GObject.MainLoop()
loop.run()

执行Ctrl-C

=> Downloader waiting for requests
=> Generating api for paul van dyk by evolution
=> Generated URL http://ws.audioscrobbler.com/2.0/?album=evolution&api_key=XXXXXXXXXXXXXXXXXXXX&method=album.getinfo&artist=paul+van+dyk
=> Requesting response
^C

谢谢!

2 个答案:

答案 0 :(得分:0)

当您的脚本停留在第53行时,您是否可以使用Ctrl + C中断执行并向我们显示traceback python给出的内容?

答案 1 :(得分:0)

问题是由Python的全局解释器锁(GIL)引起的。

GObject.threads_init()

解决了这个问题。