Python客户端

时间:2018-04-07 13:00:49

标签: python-requests apache-tika tika-server

Apache Tika应该可以通过HTTP从Python程序访问,但我无法让它工作。

我正在使用此命令运行the server(最后有两个选项):

java -jar tika-server-1.17.jar --port 5677 -enableUnsecureFeatures -enableFileUrl

它适用于卷曲:

curl -v -T /tmp/tmpsojwBN  http://localhost:5677/tika
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 5677 (#0)
> PUT /tika HTTP/1.1
> Host: localhost:5677
> User-Agent: curl/7.47.0
> Accept: */*
> Accept-Encoding: gzip, deflate
> Content-Length: 418074
> Expect: 100-continue
> 
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Date: Sat, 07 Apr 2018 12:28:41 GMT
< Transfer-Encoding: chunked
< Server: Jetty(8.y.z-SNAPSHOT)

但是当我尝试类似的东西时(尝试了不同的标题组合,这里我重新创建了与python-tika客户端使用的相同的标题):

with tempfile.NamedTemporaryFile() as tmp_file:
    download_file(url, tmp_file)
    payload = open(tmp_file.name, 'rb')
    headers = {
        'Accept': 'application/json',
        'Content-Disposition': 'attachment; filename={}'.format(
            os.path.basename(tmp_file.name))}
    response = requests.put(TIKA_ENDPOINT_URL + '/tika', payload,
                            headers=headers,
                            verify=False)

我尝试使用有效负载以及fileUrl - 在服务器上使用WARN javax.ws.rs.ClientErrorException: HTTP 406 Not Acceptable和java堆栈跟踪的相同结果。完整跟踪:

WARN  javax.ws.rs.ClientErrorException: HTTP 406 Not Acceptable
    at org.apache.cxf.jaxrs.utils.SpecExceptions.toHttpException(SpecExceptions.java:117)
    at org.apache.cxf.jaxrs.utils.ExceptionUtils.toHttpException(ExceptionUtils.java:173)
    at org.apache.cxf.jaxrs.utils.JAXRSUtils.findTargetMethod(JAXRSUtils.java:542)
    at org.apache.cxf.jaxrs.interceptor.JAXRSInInterceptor.processRequest(JAXRSInInterceptor.java:177)
    at org.apache.cxf.jaxrs.interceptor.JAXRSInInterceptor.handleMessage(JAXRSInInterceptor.java:77)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307)
    at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
    at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:274)
    at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:261)
    at org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:76)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1088)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1024)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.Server.handle(Server.java:370)
    at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
    at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:973)
    at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1035)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:641)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:231)
    at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Thread.java:748)

我还尝试比较(与nc -l localhost 5677 | less)两个请求(有效缩写)的不同之处:

来自卷曲:

PUT /tika HTTP/1.1
Host: localhost:5677
User-Agent: curl/7.47.0
Accept: */*
Content-Length: 418074
Expect: 100-continue

%PDF-1.4
%<D3><EB><E9><E1>
1 0 obj
<</Creator (Chromium)

来自Python请求库:

PUT /tika HTTP/1.1
Host: localhost:5677
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: application/json
User-Agent: python-requests/2.13.0
Content-type: application/pdf
Content-Length: 246176

%PDF-1.4
%<D3><EB><E9><E1>
1 0 obj
<</Creator (Chromium)

问题是,从Python调用Tika服务器的正确方法是什么?

我还在仅客户端模式下尝试了python tika库,并通过jnius使用了tika-app。使用tika客户端,以及使用带有pyjnius的tika-app.jar,当我在芹菜工作者中使用它时,我只会冻结(调用永不返回)。同时,pyjnius / tika-app和tika-python脚本在脚本中都很好用:我还没弄清楚芹菜工作者里面出了什么问题。我想,与错误的地方的线程和/或初始化有关。但这是另一个问题的主题。

以下是tika-python的要求:

PUT /tika HTTP/1.1
Host: localhost:5677
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: application/json
User-Agent: python-requests/2.13.0
Content-Disposition: attachment; filename=tmpb3YkTq
Content-Length: 183234

%PDF-1.4
%<D3><EB><E9><E1>
1 0 obj
<</Creator (Chromium)

现在看来这对tika服务器来说是个问题:

$ tika-python --verbose --server 'localhost' --port 5677 parse all /tmp/tmpb3YkTq
2018-04-08 09:44:11,555 [MainThread  ] [INFO ]  Writing ./tmpb3YkTq_meta.json
(<open file '<stderr>', mode 'w' at 0x7f0b688eb1e0>, 'Request headers: ', {'Accept': 'application/json', 'Content-Disposition': 'attachment; filename=tmpb3YkTq'})
(<open file '<stderr>', mode 'w' at 0x7f0b688eb1e0>, 'Response headers: ', {'Date': 'Sun, 08 Apr 2018 06:44:13 GMT', 'Transfer-Encoding': 'chunked', 'Content-Type': 'application/json', 'Server': 'Jetty(8.y.z-SNAPSHOT)'})
['./tmpb3YkTq_meta.json']

比照:

$ tika-python --verbose --server 'localhost' --port 5677 parse text /tmp/tmpb3YkTq
2018-04-08 09:43:38,326 [MainThread  ] [INFO ]  Writing ./tmpb3YkTq_meta.json
(<open file '<stderr>', mode 'w' at 0x7fc3eee4a1e0>, 'Request headers: ', {'Accept': 'application/json', 'Content-Disposition': 'attachment; filename=tmpb3YkTq'})
(<open file '<stderr>', mode 'w' at 0x7fc3eee4a1e0>, 'Response headers: ', {'Date': 'Sun, 08 Apr 2018 06:43:38 GMT', 'Content-Length': '0', 'Server': 'Jetty(8.y.z-SNAPSHOT)'})
2018-04-08 09:43:38,409 [MainThread  ] [WARNI]  Tika server returned status: 406
['./tmpb3YkTq_meta.json']

0 个答案:

没有答案