Question

我正在尝试测试一些使用urllib2和lxml的python代码。

我已经看过几篇博客帖子和堆栈溢出帖子，人们想用urllib2来测试抛出的异常。我没有看到测试成功通话的例子。

我走的是正确的道路吗？

有没有人建议让它发挥作用？

这是我到目前为止所做的：

import mox
import urllib
import urllib2
import socket
from lxml import etree

# set up the test
m = mox.Mox()
response = m.CreateMock(urllib.addinfourl)
response.fp = m.CreateMock(socket._fileobject)
response.name = None # Needed because the file name is checked.
response.fp.read().AndReturn("""<?xml version="1.0" encoding="utf-8"?>
<foo>bar</foo>""")
response.geturl().AndReturn("http://rss.slashdot.org/Slashdot/slashdot")
response.read = response.fp.read # Needed since __init__ is not called on addinfourl.
m.StubOutWithMock(urllib2, 'urlopen')
urllib2.urlopen(mox.IgnoreArg(), timeout=10).AndReturn(response)
m.ReplayAll()

# code under test
response2 = urllib2.urlopen("http://rss.slashdot.org/Slashdot/slashdot", timeout=10)
# Note: response2.fp.read() and response2.read() do not behave the same, as defined above.
# In [21]: response2.fp.read()
# Out[21]: '<?xml version="1.0" encoding="utf-8"?>\n<foo>bar</foo>'
# In [22]: response2.read()
# Out[22]: <mox.MockMethod object at 0x97f326c>
xcontent = etree.parse(response2)

# verify test
m.VerifyAll()

失败了：

Traceback (most recent call last):
  File "/home/jon/mox_question.py", line 22, in <module>
    xcontent = etree.parse(response2)
  File "lxml.etree.pyx", line 2583, in lxml.etree.parse (src/lxml/lxml.etree.c:25057)
  File "parser.pxi", line 1487, in lxml.etree._parseDocument (src/lxml/lxml.etree.c:63708)
  File "parser.pxi", line 1517, in lxml.etree._parseFilelikeDocument (src/lxml/lxml.etree.c:63999)
  File "parser.pxi", line 1400, in lxml.etree._parseDocFromFilelike (src/lxml/lxml.etree.c:62985)
  File "parser.pxi", line 990, in lxml.etree._BaseParser._parseDocFromFilelike (src/lxml/lxml.etree.c:60508)
  File "parser.pxi", line 542, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/lxml.etree.c:56659)
  File "parser.pxi", line 624, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:57472)
  File "lxml.etree.pyx", line 235, in lxml.etree._ExceptionContext._raise_if_stored (src/lxml/lxml.etree.c:6222)
  File "parser.pxi", line 371, in lxml.etree.copyToBuffer (src/lxml/lxml.etree.c:55252)
TypeError: reading from file-like objects must return byte strings or unicode strings

这是因为response.read（）没有返回我期望它返回的内容。

Answer 1

我根本不会钻研urllib2内部。这超出了我所关心的范围。这是使用StringIO执行此操作的简单方法。这里的关键是你打算解析为XML只需要在鸭子类型方面就像文件一样，它不需要是一个真正的addinfourl实例。

import StringIO
import mox
import urllib2
from lxml import etree

# set up the test
m = mox.Mox()
response = StringIO.StringIO("""<?xml version="1.0" encoding="utf-8"?>
<foo>bar</foo>""")
m.StubOutWithMock(urllib2, 'urlopen')
urllib2.urlopen(mox.IgnoreArg(), timeout=10).AndReturn(response)
m.ReplayAll()

# code under test
response2 = urllib2.urlopen("http://rss.slashdot.org/Slashdot/slashdot", timeout=10)
xcontent = etree.parse(response2)

# verify test
m.VerifyAll()

Answer 2

回应彼得所说的，我只想补充一点，你可能不需要关心lxml内部，而不是urllib2。通过模拟lxml.etree，您可以完全隔离您真正需要测试的代码，您自己的代码。这是一个这样做的例子，还展示了如何使用模拟对象来测试response.getcode（）调用。

import mox
from lxml import etree
import urllib2

class TestRssDownload(mox.MoxTestBase):

    def test_rss_download(self):
        expected_response = self.mox.CreateMockAnything()
        self.mox.StubOutWithMock(urllib2, 'urlopen')
        self.mox.StubOutWithMock(etree, 'parse')
        self.mox.StubOutWithMock(etree, 'iterwalk')
        title_elem = self.mox.CreateMock(etree._Element)
        title_elem.text = 'some title'

        # Set expectations 
        urllib2.urlopen("http://rss.slashdot.org/Slashdot/slashdot", timeout=10).AndReturn(expected_response)
        expected_response.getcode().AndReturn(200)
        etree.parse(expected_response).AndReturn('some parsed content')
        etree.iterwalk('some parsed content', tag='{http://purl.org/rss/1.0/}title').AndReturn([('end', title_elem),])

        # Code under test
        self.mox.ReplayAll()
        self.production_code()

    def production_code(self):
        response = urllib2.urlopen("http://rss.slashdot.org/Slashdot/slashdot", timeout=10)
        response_code = response.getcode()
        if 200 != response_code:
            raise Exception('Houston, we have a problem ({0})'.format(response_code))
        tree = etree.parse(response)
        for ev, elem in etree.iterwalk(tree, tag='{http://purl.org/rss/1.0/}title'):
            # Do something with elem.text
            print('{0}: {1}'.format(ev, elem.text))

Answer 3

看起来你的失败根本与mox无关 - 导致错误的行是从response2读取，这是对slashdot的直接调用。也许检查那个对象，看看它的内容是什么？

编辑：我没有看到上面的m.StubOutWithMock(urllib2, 'urlopen')行，所以我认为你正在比较两个电话;一个嘲笑（响应）而另一个不响应（响应2）。更新后的答案如下。

使用pymox模拟urllib2.urlopen和lxml.etree.parse

3 个答案: