如何创建和写入内存映射文件?

时间:2015-02-14 15:14:30

标签: rust mmap

  

编者注:此代码示例来自1.0之前的Rust版本,它使用的代码在Rust 1.0中不存在。一些答案已经更新,以回答更新版本的Rust的核心问题。

我正在尝试使用std::os::MemoryMap创建内存映射文件。目前的方法如下:

use std::os;
use std::ptr;
use std::old_io as io;
use std::os::unix::prelude::AsRawFd;
use std::os::MapOption;

let path = Path::new("test.mmap");

let f = match io::File::open_mode(&path, io::Open, io::ReadWrite) {
    Ok(f) => f,
    Err(err) => panic!("Could not open file: {}", err),
};

let mmap_opts = &[
    MapOption::MapReadable,
    MapOption::MapWritable,
    MapOption::MapFd(f.as_raw_fd())
];

let mmap = match os::MemoryMap::new(1024*1024, mmap_opts) {
    Ok(mmap) => {
        println!("Successfully created the mmap: {}", mmap.len());
        mmap
    }
    Err(err) => panic!("Could not read the mmap: {}", err),
};

unsafe {
   let data = mmap.data();

    if data.is_null() {
        panic!("Could not access data from memory mapped file")
    }

    let src = "Hello!";
    ptr::copy_memory(data, src.as_ptr(), src.as_bytes().len());
}

此程序失败

Process didn't exit successfully: `target/mmap` (status=4)

在调用ptr::copy_memory或对数据进行任何其他操作时。

  • 我无法写入(或读取)MemoryMap
  • 中的数据的原因是什么
  • 在Rust中使用MemoryMap的正确方法是什么?

2 个答案:

答案 0 :(得分:14)

真正的答案是使用提供此功能的箱子,理想情况是以跨平台的方式。

extern crate memmap;

use std::{
    fs::OpenOptions,
    io::{Seek, SeekFrom, Write},
};

const SIZE: u64 = 1024 * 1024;

fn main() {
    let src = "Hello!";

    let mut f = OpenOptions::new()
        .read(true)
        .write(true)
        .create(true)
        .open("test.mmap")
        .expect("Unable to open file");

    // Allocate space in the file first
    f.seek(SeekFrom::Start(SIZE)).unwrap();
    f.write_all(&[0]).unwrap();
    f.seek(SeekFrom::Start(0)).unwrap();

    let mut data = unsafe {
        memmap::MmapOptions::new()
            .map_mut(&f)
            .expect("Could not access data from memory mapped file")
    };

    data[..src.len()].copy_from_slice(src.as_bytes());
}

请注意,此代码仍可能导致未定义的行为。由于切片由文件支持,因此文件的内容(以及切片)可能会从Rust程序之外的更改,从而打破unsafe块应该的不变量保持。程序员需要确保文件在地图生命周期内不会发生变化。不幸的是,箱子本身并没有提供太多帮助来防止这种情况发生,甚至没有任何文件警告用户。


如果您希望使用较低级别的系统调用,则缺少两个主要部分:

  1. mmap doesn't allocate any space on its own,因此您需要在文件中设置一些空间。如果没有这个,我在macOS上运行时会得到Illegal instruction: 4

  2. MemoryMap(是)private by default因此您需要将映射标记为 public ,以便将更改写回文件(我假设您希望保存写入)。如果没有这个,代码就会运行,但文件永远不会改变。

  3. 这是一个适合我的版本:

    extern crate libc;
    
    use std::{
        fs::OpenOptions,
        io::{Seek, SeekFrom, Write},
        os::unix::prelude::AsRawFd,
        ptr,
    };
    
    fn main() {
        let src = "Hello!";
    
        let size = 1024 * 1024;
    
        let mut f = OpenOptions::new()
            .read(true)
            .write(true)
            .create(true)
            .open("test.mmap")
            .expect("Unable to open file");
    
        // Allocate space in the file first
        f.seek(SeekFrom::Start(size as u64)).unwrap();
        f.write_all(&[0]).unwrap();
        f.seek(SeekFrom::Start(0)).unwrap();
    
        // This refers to the `File` but doesn't use lifetimes to indicate
        // that. This is very dangerous, and you need to be careful.
        unsafe {
            let data = libc::mmap(
                /* addr: */ ptr::null_mut(),
                /* len: */ size,
                /* prot: */ libc::PROT_READ | libc::PROT_WRITE,
                // Then make the mapping *public* so it is written back to the file
                /* flags: */ libc::MAP_SHARED,
                /* fd: */ f.as_raw_fd(),
                /* offset: */ 0,
            ) as *mut u8;
    
            if data.is_null() {
                panic!("Could not access data from memory mapped file")
            }
    
            ptr::copy_nonoverlapping(src.as_ptr(), data, src.len());
        }
    }
    

答案 1 :(得分:8)

最新版本:

use std::ptr;
use std::fs;
use std::io::{Write, SeekFrom, Seek};
use std::os::unix::prelude::AsRawFd;
use mmap::{MemoryMap, MapOption};

// from crates.io
extern crate mmap;
extern crate libc;

fn main() {
    let size: usize = 1024*1024;

    let mut f = fs::OpenOptions::new().read(true)
                                      .write(true)
                                      .create(true)
                                      .open("test.mmap")
                                      .unwrap();

    // Allocate space in the file first
    f.seek(SeekFrom::Start(size as u64)).unwrap();
    f.write_all(&[0]).unwrap();
    f.seek(SeekFrom::Start(0)).unwrap();

    let mmap_opts = &[
        // Then make the mapping *public* so it is written back to the file
        MapOption::MapNonStandardFlags(libc::consts::os::posix88::MAP_SHARED),
        MapOption::MapReadable,
        MapOption::MapWritable,
        MapOption::MapFd(f.as_raw_fd()),
    ];

    let mmap = MemoryMap::new(size, mmap_opts).unwrap();

    let data = mmap.data();

    if data.is_null() {
        panic!("Could not access data from memory mapped file")
    }

    let src = "Hello!";
    let src_data = src.as_bytes();

    unsafe {
        ptr::copy(src_data.as_ptr(), data, src_data.len());
    }
}