5.1. spp_nfv

spp_nfv is a DPDK secondary process and communicates with primary and other peer processes via TCP sockets or shared memory. spp_nfv consists of several threads, main thread for maanging behavior of spp_nfv and worker threads for packet forwarding.

As initialization of the process, it calls rte_eal_init(), then specific initialization functions for resources of spp_nfv itself.

After initialization, main thread launches worker threads on each of given slave lcores with rte_eal_remote_launch(). It means that spp_nfv requires two lcores at least. Main thread starts to accept user command after all of worker threads are launched.

5.1.1. Initialization

In main funciton, spp_nfv calls rte_eal_init() first as other DPDK applications, forward_array_init() and port_map_init() for initializing port forward array which is a kind of forwarding table.

int
main(int argc, char *argv[])
{
        ....

        ret = rte_eal_init(argc, argv);
        if (ret < 0)
                return -1;
        ....

        /* initialize port forward array*/
        forward_array_init();
        port_map_init();
        ....

Port forward array is implemented as an array of port structure. It consists of RX, TX ports and its forwarding functions, rte_rx_burst() and rte_tx_burst() actually. Each of ports are identified with unique port ID. Worker thread iterates this array and forward packets from RX port to TX port.

/* src/shared/common.h */

struct port {
    uint16_t in_port_id;
    uint16_t out_port_id;
    uint16_t (*rx_func)(uint16_t, uint16_t, struct rte_mbuf **, uint16_t);
    uint16_t (*tx_func)(uint16_t, uint16_t, struct rte_mbuf **, uint16_t);
};

Port map is another kind of structure for managing its type and statistics. Port type for indicating PMD type, for example, ring, vhost or so. Statistics is used as a counter of packet forwarding.

/* src/shared/common.h */

struct port_map {
        int id;
        enum port_type port_type;
        struct stats *stats;
        struct stats default_stats;
};

Final step of initialization is setting up memzone. In this step, spp_nfv just looks up memzone of primary process as a secondary.

/* set up array for port data */
if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
        mz = rte_memzone_lookup(MZ_PORT_INFO);
        if (mz == NULL)
                rte_exit(EXIT_FAILURE,
                        "Cannot get port info structure\n");
        ports = mz->addr;

5.1.2. Launch Worker Threads

Worker threads are launched with rte_eal_remote_launch() from main thread. RTE_LCORE_FOREACH_SLAVE is a macro for traversing slave lcores while incrementing lcore_id and rte_eal_remote_launch() is a function for running a function on worker thread.

lcore_id = 0;
RTE_LCORE_FOREACH_SLAVE(lcore_id) {
        rte_eal_remote_launch(main_loop, NULL, lcore_id);
}

In this case, main_loop is a starting point for calling task of worker thread nfv_loop().

static int
main_loop(void *dummy __rte_unused)
{
        nfv_loop();
        return 0;
}

5.1.3. Parsing User Command

After all of worker threads are launched, main threads goes into while loop for waiting user command from SPP controller via TCP connection. If receiving a user command, it simply parses the command and make a response. It terminates the while loop if it receives exit command.

while (on) {
        ret = do_connection(&connected, &sock);
        ....
        ret = do_receive(&connected, &sock, str);
        ....
        flg_exit = parse_command(str);
        ....
        ret = do_send(&connected, &sock, str);
        ....
}

parse_command() is a function for parsing user command as named. There are several commnads for spp_nfv as described in Secondary Commands. Command from controller is a simple plain text and action for the command is decided with the first token of the command.

static int
parse_command(char *str)
{
        ....

        if (!strcmp(token_list[0], "status")) {
                RTE_LOG(DEBUG, SPP_NFV, "status\n");
                memset(str, '\0', MSG_SIZE);
        ....

                } else if (!strcmp(token_list[0], "add")) {
                RTE_LOG(DEBUG, SPP_NFV, "Received add command\n");
                if (do_add(token_list[1]) < 0)
                        RTE_LOG(ERR, SPP_NFV, "Failed to do_add()\n");

        } else if (!strcmp(token_list[0], "patch")) {
                RTE_LOG(DEBUG, SPP_NFV, "patch\n");
        ....
}

For instance, if the first token is add, it calls do_add() with given tokens and adds port to the process.