tokio-curl:将输出捕获到本地`Vec` - 可能比借来的值更长

时间:2016-09-13 15:04:01

标签: rust rust-tokio

我不太清楚Rust是否能够理解生命和闭包......

尝试使用tokio-curl将下载的数据收集到矢量中:

extern crate curl;
extern crate futures;
extern crate tokio_core;
extern crate tokio_curl;

use std::io::{self, Write};
use std::str;

use curl::easy::Easy;
use tokio_core::reactor::Core;
use tokio_curl::Session;

fn main() {
    // Create an event loop that we'll run on, as well as an HTTP `Session`
    // which we'll be routing all requests through.
    let mut lp = Core::new().unwrap();
    let mut out = Vec::new();
    let session = Session::new(lp.handle());

    // Prepare the HTTP request to be sent.
    let mut req = Easy::new();
    req.get(true).unwrap();
    req.url("https://www.rust-lang.org").unwrap();
    req.write_function(|data| {
            out.extend_from_slice(data);
            io::stdout().write_all(data).unwrap();
            Ok(data.len())
        })
        .unwrap();

    // Once we've got our session, issue an HTTP request to download the
    // rust-lang home page
    let request = session.perform(req);

    // Execute the request, and print the response code as well as the error
    // that happened (if any).
    let mut req = lp.run(request).unwrap();
    println!("{:?}", req.response_code());
    println!("out: {}", str::from_utf8(&out).unwrap());
} 

产生错误:

error[E0373]: closure may outlive the current function, but it borrows `out`, which is owned by the current function
  --> src/main.rs:25:24
   |
25 |     req.write_function(|data| {
   |                        ^^^^^^ may outlive borrowed value `out`
26 |             out.extend_from_slice(data);
   |             --- `out` is borrowed here
   |
help: to force the closure to take ownership of `out` (and any other referenced variables), use the `move` keyword, as shown:
   |     req.write_function(move |data| {

进一步调查,我发现Easy::write_function需要'static生命周期,但是如何从curl-rust文档收集输出的示例使用Transfer::write_function代替:

use curl::easy::Easy;

let mut data = Vec::new();
let mut handle = Easy::new();
handle.url("https://www.rust-lang.org/").unwrap();
{
    let mut transfer = handle.transfer();
    transfer.write_function(|new_data| {
        data.extend_from_slice(new_data);
        Ok(new_data.len())
    }).unwrap();
    transfer.perform().unwrap();
}
println!("{:?}", data);

Transfer::write_function does not require the 'static lifetime

impl<'easy, 'data> Transfer<'easy, 'data> {
    /// Same as `Easy::write_function`, just takes a non `'static` lifetime
    /// corresponding to the lifetime of this transfer.
    pub fn write_function<F>(&mut self, f: F) -> Result<(), Error>
        where F: FnMut(&[u8]) -> Result<usize, WriteError> + 'data
    {
...

但我无法在tokio-curl的Transfer上使用Session::perform个实例,因为它requires the Easy type

pub fn perform(&self, handle: Easy) -> Perform {

transfer.easy是一个私有字段,可直接传递给session.perform

这是tokio-curl的问题吗?也许它应该将transfer.easy字段标记为公开或实现perform_transfer之类的新功能?是否有其他方法可以使用tokio-curl每次传输收集输出?

1 个答案:

答案 0 :(得分:2)

使用期货库时,首先要了解的是,您无法控制代码将在哪个线程上运行。

此外,curl&#39;} Easy::write_function的文档说:

  

请注意,此函数的生命周期限制为'static,但这通常限制太多。要使用堆栈数据,请考虑调用transfer方法,然后使用write_function配置可引用堆栈本地数据的回调。

最直接的解决方案是使用某种类型的锁定原语来确保一次只有一个线程可以访问该向量。您还必须在主线程和闭包之间共享向量的所有权:

use std::sync::Mutex;
use std::sync::Arc;

let out = Arc::new(Mutex::new(Vec::new()));
let out_closure = out.clone();

// ...

req.write_function(move |data| {
    let mut out = out_closure.lock().expect("Unable to lock output");
    // ...
}).expect("Cannot set writing function");

// ...

let out = out.lock().expect("Unable to lock output");
println!("out: {}", str::from_utf8(&out).expect("Data was not UTF-8"));

不幸的是,tokio-curl库目前不支持使用允许基于堆栈的数据的Transfer类型。