Question

我正在尝试将页面移至NUMA 0：

    if (0 != move_pages(getpid(), nPages, &ptrToSHM, nodes, status, MPOL_MF_MOVE_ALL)) {
        std::cout << "failed to move pages for /dev/shm/" << name << " to NUMA " << numaNodeID << " because " << strerror(errno) << std::endl;
    }

它返回零，然后在调用move_pages()之后检查这些页面的位置：

    if (0 != move_pages(0, nPages, &ptrToSHM, nullptr, status, 0)) {
        std::cout << "failed to inquiry pages for /dev/shm/" << name << " because " << strerror(errno) << std::endl;
    }
    else {
        for (uint32_t i = 0; i < nPages; i++) {
            std::cout << "/dev/shm/" << name << "'s page # " << i << " locate at numa node " << status[i] << std::endl;
        }
    }

它会打印：

/dev/shm/test's page # 0 locate at numa node -2
/dev/shm/test's page # 1 locate at numa node -14

根据manpage，它指出： nodes is an array of integers that specify the desired location for each page. Each element in the array is a node number. nodes can also be NULL, in which case move_pages() does not move any pages but instead will return the node where each page currently resides, in the status array. Obtaining the status of each page may be necessary to determine pages that need to be moved.

我想知道为什么尽管移动页面和查询都返回成功，但是为什么打印负值呢？

谢谢！

P.S。我还尝试了numa_move_pages()，其效果与仅调用numa_move_pages()的{{1}}相同。

Answer 1

您对shm_open和mmap的使用可能不会获得所需的大页面。

move_pages syscall（和libnuma包装器）适用于x86_64的4096字节标准页面。

然后您以错误的方式使用了move_pages并使用了错误的第三个参数“ pages”。它不应该是指向内存的指针。但是指向本身将包含nPages指针的数组的指针：

http://man7.org/linux/man-pages/man2/move_pages.2.html

  long move_pages(int pid, unsigned long count, void **pages,
                   const int *nodes, int *status, int flags);

   pages is an array of pointers to the pages that should be moved.
   These are pointers that should be aligned to page boundaries.
   Addresses are specified as seen by the process specified by pid.

如果在“页面”中没有正确的指针，则根据errno 14（从moreutils包中得出），您将得到-14，即EFAULT。

//https://stackoverflow.com/questions/54546367/fail-to-query-via-move-pages
//g++ 54546367.move_pages.cc -o 54546367.move_pages -lnuma -lrt
#include <cstdint>
#include <iostream>
#include <numaif.h>
#include <sys/mman.h>
#include <fcntl.h>
#include <errno.h>
#include <unistd.h>
#include <string.h>
#include <limits>

int main(int argc, char** argv) {
    const constexpr uint64_t size = 256lu * 1024;// * 1024;
    const constexpr uint32_t nPages = size / (4lu * 1024);
    void * pages[nPages];
    int32_t status[nPages];
    std::fill_n(status, nPages, std::numeric_limits<int32_t>::min());

//  auto fd = shm_open("test_shm", O_RDWR|O_CREAT, 0666);
//  void* ptr = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
    void* ptr = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, 0, 0);
    std::cout << "Ptr is " << ptr << std::endl;
    if (ptr == MAP_FAILED) {
//      if (fd > 0) close(fd);
        throw "failed to map hugepages";
    }
    memset(ptr, 0x41, nPages*4096);
    for(uint32_t i = 0; i<nPages; i++) {
        pages[i] = &((char*)ptr)[i*4096];
    }

    if (0 != move_pages(0, nPages, pages, nullptr, status, 0)) {
        std::cout << "failed to inquiry pages because " << strerror(errno) << std::endl;
    }
    else {
        for (uint32_t i = 0; i < nPages; i++) {
            std::cout << "page # " << i << " locates at numa node " << status[i] << std::endl;
        }
    }
    munmap(ptr, size);
//  close(fd);
}

在NUMA机器上，它从taskset -c 7 ./54546367.move_pages开始输出相同的节点，并在numactl -i all ./54546367.move_pages时插入（0 1 0 1）。

无法通过move_pages（）查询

1 个答案: