PACKET_MMAP TX_RING后续send()调用出现问题

时间:2018-11-29 21:53:42

标签: c sockets linux-kernel mmap raw-sockets

感谢您抽出宝贵的时间对此进行研究。我已经处理了一个星期左右的问题,我很傻。任何输入都会很棒。

我目前正在使用PACKET_MMAP TX_RING V3(我尝试过V2和V1,但是我也遇到了同样的问题。我决定使用V3,因为我能够在一个块上保留多个eth帧,而不是在V1和V2中,每个tp_block只能包含1帧),以避免在我的端口扫描程序的发送过程中过多的系统调用。我没有设置RX_RING,所以这不是图片的一部分。

现在,我仅在一台主机上进行测试,并且只击中了我所知道的5个端口。

所以...情况如下:

我有一个TX_R​​ING:[1个区块|块大小4096 | 32帧|框架尺寸128]

TX_RING设置的代码:

int                     prepare_packetmmap_tx_ring(t_thread *thread)
{
    int                 tpacket_v;
    struct sockaddr_ll  sll_loc;

    /* Step 1 Create PF_PACKET socket.*/
    thread->sock = socket(PF_PACKET, SOCK_RAW, htons(ETH_P_ALL));
    if (thread->sock == -1)
    {
        toggle_thread_alive(thread);
        return (hermes_error(FAILURE, "socket() %s", strerror(errno)));
    }

    memset(&sll_loc, 0, sizeof(struct sockaddr_ll));
    memcpy(&sll_loc.sll_addr, thread->pool->iface.if_hwaddr, ETH_ALEN);
    sll_loc.sll_ifindex = thread->pool->iface.inx;
    sll_loc.sll_family = AF_PACKET;
    sll_loc.sll_protocol = htons(ETH_P_ALL);
    /* Bind our socket to the interface */
    if (bind(thread->sock, (sockaddr *)&sll_loc, sizeof(struct sockaddr_ll)) == -1)
    {
        toggle_thread_alive(thread);
        return (hermes_error(FAILURE, "bind() %s", strerror(errno)));
    }

    tpacket_v = TPACKET_V3;
    if (setsockopt(thread->sock, SOL_PACKET, PACKET_VERSION, &tpacket_v, sizeof(tpacket_v)) < 0)
    {
        toggle_thread_alive(thread);
        return (hermes_error(FAILURE, "setsockopt() PACKET_VERSION %s", strerror(errno)));
    }

    /* Step 3 determine sizes for PACKET_TX_RING and allocate txring via setsockopt() */
    thread->txring.tpr.tp_block_size = (uint)getpagesize();
    thread->txring.tpr.tp_frame_size = TPACKET3_HDRLEN + sizeof(struct ethhdr) + sizeof(struct iphdr) + DEF_TRAN_HDRLEN + thread->pool->env->cpayload_len;
    thread->txring.tpr.tp_frame_size = pow2_round(thread->txring.tpr.tp_frame_size);
    thread->txring.tpr.tp_block_nr = (THRD_HSTGRP_MAX / (thread->txring.tpr.tp_block_size / thread->txring.tpr.tp_frame_size));
    thread->txring.tpr.tp_frame_nr = thread->txring.tpr.tp_block_nr * (thread->txring.tpr.tp_block_size / thread->txring.tpr.tp_frame_size);
    thread->txring.size = thread->txring.tpr.tp_block_size * thread->txring.tpr.tp_block_nr;
    thread->txring.doffset = sizeof(struct tpacket3_hdr);

    if (setsockopt(thread->sock, SOL_PACKET, PACKET_TX_RING, (void *)&thread->txring.tpr, sizeof(thread->txring.tpr)) < 0)
    {
        toggle_thread_alive(thread);
        return (hermes_error(FAILURE, "setsockopt() PACKET_TX_RING %s", strerror(errno)));
    }

    printf("frame size: %d | blocksize: %d | block count %d | frame count %d\n", thread->txring.tpr.tp_frame_size, thread->txring.tpr.tp_block_size, thread->txring.tpr.tp_block_nr, thread->txring.tpr.tp_frame_nr);
    extra_sock_opts(thread->sock, thread->txring.size);
    /* Step 4 actually map ring to user space */
    if (!(thread->txring.ring = mmap(0, thread->txring.size,
            PROT_READ | PROT_WRITE,
            MAP_SHARED,
            thread->sock, 0)))
    {
        toggle_thread_alive(thread);
        return (hermes_error(FAILURE, "mmap() TX_RING %s", strerror(errno)));
    }
    return (SUCCESS);
}

所以……一切都很好,对吗?让我们继续我的戒指填充过程:

        nt                      fill_tx_ring(t_thread *thread, t_frame *ethframe)
{
    struct tpacket3_hdr *frame;
    void                *data;
    uint16_t            *srcports;
    uint16_t            *dstports;
    uint                ring_i;
    uint                hst_i;
    int                 ret;

    ring_i = 0;
    hst_i = 0;
    ret = 0;
    srcports = thread->pool->env->ports.flat;
    dstports = thread->pool->env->dstports;
    while (hst_i < thread->hstgrpsz && ring_i < thread->txring.tpr.tp_frame_nr)
    {
        if (thread->hstgrp[hst_i].health.done == true)
        {
            hst_i++;
            continue;
        }
        frame = (struct tpacket3_hdr *)(thread->txring.ring + (thread->txring.tpr.tp_frame_size * ring_i));
        switch((volatile uint32_t)frame->tp_status)
        {
            case TP_STATUS_WRONG_FORMAT:
                return(hermes_error(FAILURE, "TX_RING wrong format in frame %i of thread %d", ring_i, thread->id));
            case TP_STATUS_AVAILABLE:
                data = (uint8_t *)frame + thread->txring.doffset;
                ethframe->ip->daddr = thread->hstgrp[hst_i].result->ip.s_addr;
                ip_checksum(ethframe->ip);
                ethframe->tcp->source = htons(dstports[thread->hstgrp[hst_i].health.portinx]);
                ethframe->tcp->dest = htons(srcports[thread->hstgrp[hst_i].health.portinx]);
                tcp_checksum(ethframe->ip, (uint16_t *)ethframe->tcp);
                memcpy(data, ethframe->buffer, ethframe->size);
                frame->tp_next_offset = 0;
                frame->tp_len = ethframe->size;
                frame->tp_status = TP_STATUS_SEND_REQUEST;
                printf("setup a frame\n");
                ret++; hst_i++; ring_i++;
                break;
            default:
                printf("skipped ring with status %d\n", frame->tp_status);
                ring_i++;
                break;
        }
    }
    return (ret);
}

(编辑/注释:我也尝试过在我的套接字上轮询POLL_OUT,直到帧上的tp_status更改为止,这将获得相同的结果。)

因此,此填充过程将填充一帧,因为我仅扫描一台主机,并且一次只向一台主机发送一条消息。 第一次发送效果很好!它发送帧,我在Wireshark上看到它,得到响应,处理它,然后返回到fill_tx_ring()例程。以下是我的扫描功能,因此您可以看到发生的顺序:

void                    syn_scan(t_thread *thread)
{
    struct timeval      sent = {0};
    struct timeval      now = {0};
    long ms;
    int                 ret;
    t_frame             frame;

    memset(&frame, 0, sizeof(t_frame));
    init_ethframe(thread, &frame);
    thread->scancnt = thread->hstgrpsz;
    while (thread->scancnt > 0)
    {
        gettimeofday(&now, NULL);
        if ((ms = timediff_ms(&now, &sent)) >= DEF_INIT_RTT_TIMEOUT)
        {
            fill_tx_ring(thread, &frame);
            send_task(thread);
            gettimeofday(&sent, NULL);
            thread->rxfilter.fd.events = POLL_IN;
            if ((ret = poll(&thread->rxfilter.fd, 1, 
DEF_INIT_RTT_TIMEOUT)) < 0)
                hermes_error(FAILURE, "poll() %s", strerror(errno));
            if (ret > 0)
                pcap_dispatch(thread->rxfilter.handle,
                              thread->hstgrpsz, handle_packet, 
(u_char*)thread);
            else
                printf("didn't get anything\n");
        }
        else
        {
            usleep((useconds_t) ms * 1000);
        }

    }
    free(frame.buffer);
    frame.buffer = NULL;
}

现在是真正的问题:对send()的所有其他调用都返回0。在随后对fill_tx_ring()的调用中,我发现我先前填充的所有帧都没有被内核更新(由于发送失败) (),似乎内核甚至从未接收到请求或查看环)...它们的tp_status字段仍设置为TP_STATUS_SEND_REQUEST。

即使成功发送了send()(假设第三个成功),您也会注意到已经用于传输帧的帧仍都设置为1(TP_STATUS_SEND_REQUEST)。

程序记录的输出如下:

index 2 | mac: a8:7d:12:1:b6:46:| ip: 10.43.1.162 | gwip: 10.43.1.254 
| gwmac: 0:35:1a:54:ca:17:
frame size: 128 | blocksize: 4096 | block count 1 | frame count 32
ports total 5

setup a frame
sent 1 packets
packet processed
a8  7d 12 01 b6 46 00 35 
1a 54 ca 17 08 00 45  00 00 2c 00 00 40 00 
3f 06 1f 7c 0a 72 06  12 0a 2b 01 a2 16 00 
b1 f8 52 03 f6 a0 00  00 00 02 60 12 ff ff 
81 11 00 00 02 04 05  b4 00 00 

setup a frame
didn't send anything.
didn't get anything

skipped ring with status 1
setup a frame
sent 1 packets
packet processed
a8  7d 12 01 b6 46 00 35 
1a 54 ca 17 08 00 45  00 00 2c 00 00 40 00 
3f 06 1f 7c 0a 72 06  12 0a 2b 01 a2 58 00 
ac ac 06 32 5f 44 00  00 00 02 60 12 ff ff 
69 49 00 00 02 04 05  b4 00 00 

skipped ring with status 1
setup a frame
didn't send anything.
didn't get anything

skipped ring with status 1
skipped ring with status 1
setup a frame
sent 1 packets
packet processed
a8  7d 12 01 b6 46 00 35 
1a 54 ca 17 08 00 45  00 00 2c 00 00 40 00 
3f 06 1f 7c 0a 72 06  12 0a 2b 01 a2 d3 0c 
e7 f1 14 3c 13 c4 00  00 00 02 60 12 ff ff 
5e ff 00 00 02 04 05  b4 00 00 

skipped ring with status 1
skipped ring with status 1
setup a frame
didn't send anything.
didn't get anything

skipped ring with status 1
skipped ring with status 1
skipped ring with status 1
setup a frame
sent 1 packets
packet processed
a8  7d 12 01 b6 46 00 35 
1a 54 ca 17 08 00 45  00 00 2c 00 00 40 00 
3f 06 1f 7c 0a 72 06  12 0a 2b 01 a2 ea 0c 
8a 10 f0 d0 aa 9b 00  00 00 02 60 12 ff ff 
49 5d 00 00 02 04 05  b4 00 00 

skipped ring with status 1
skipped ring with status 1
skipped ring with status 1
setup a frame
didn't send anything.
didn't get anything
ms 1000
skipped ring with status 1
skipped ring with status 1
skipped ring with status 1
skipped ring with status 1
setup a frame
sent 1 packets
packet processed
a8  7d 12 01 b6 46 00 35 
1a 54 ca 17 08 00 45  00 00 2c 00 00 40 00 
3f 06 1f 7c 0a 72 06  12 0a 2b 01 a2 0c 17 
ee 5f a7 98 89 b2 00  00 00 02 60 12 ff ff 
45 0d 00 00 02 04 05  b4 00 00 
thread 1 closing 0 still alive

那么内核为什么不把我的帧还给我,为什么对send()的所有其他调用都无法发送我在环中设置的帧?

提前感谢!

您可以看到完整的代码:

项目: https://github.com/dauie/hermes.git

tx设置: https://github.com/Dauie/hermes/blob/master/src/thread.c

tx_fill / send: https://github.com/Dauie/hermes/blob/master/src/scan.c

0 个答案:

没有答案