2. Install DPDK and SPP¶

Before setting up SPP, you need to install DPDK. In this document, briefly described how to install and setup DPDK. Refer to DPDK documentation for more details. For Linux, see Getting Started Guide for Linux .

2.1. Required Packages¶

Installing packages for DPDK and SPP is almost the on Ubunu and CentOS, but names are different for some packages.

For CentOS, you can install packages with RPM instead of using yum command as described in Build RPM Packages.

2.1.1. Ubuntu¶

To compile DPDK, it is required to install following packages.

$ sudo apt install libnuma-dev \
  libarchive-dev \
  build-essential

You also need to install linux-headers of your kernel version.

$ sudo apt install linux-headers-$(uname -r)

Some of SPP secondary processes depend on other libraries and you fail to compile SPP without installing them.

SPP provides libpcap-based PMD for dumping packet to a file or retrieve it from the file. spp_nfv and spp_pcap use libpcap-dev for packet capture. spp_pcap uses liblz4-dev and liblz4-tool to compress PCAP file.

$ sudo apt install libpcap-dev \
  liblz4-dev \
  liblz4-tool

text2pcap is also required for creating pcap file which is included in wireshark.

$ sudo apt install wireshark

2.1.2. CentOS¶

Before installing packages for DPDK, you should add IUS Community repositories with yum command.

$ sudo yum install https://centos7.iuscommunity.org/ius-release.rpm

To compile DPDK, required to install following packages.

$ sudo yum install numactl-devel \
  libarchive-devel \
  kernel-headers \
  kernel-devel

As same as Ubuntu, you should install additional packages because SPP provides libpcap-based PMD for dumping packet to a file or retrieve it from the file. spp_nfv and spp_pcap use libpcap-dev for packet capture. spp_pcap uses liblz4-dev and liblz4-tool to compress PCAP file. text2pcap is also required for creating pcap file which is included in wireshark.

$ sudo apt install libpcap-dev \
  libpcap \
  libpcap-devel \
  lz4 \
  lz4-devel \
  wireshark \
  wireshark-devel \
  libX11-devel

2.2. DPDK¶

Clone repository and compile DPDK in any directory.

$ cd /path/to/any
$ git clone http://dpdk.org/git/dpdk

Installing on Ubuntu and CentOS are almost the same, but required packages are just bit different.

PCAP is disabled by default in DPDK configuration. CONFIG_RTE_LIBRTE_PMD_PCAP and CONFIG_RTE_PORT_PCAP defined in config file common_base should be changed to y to enable PCAP.

# dpdk/config/common_base
CONFIG_RTE_LIBRTE_PMD_PCAP=y
...
CONFIG_RTE_PORT_PCAP=y

Compile DPDK with target environment.

$ cd dpdk
$ export RTE_SDK=$(pwd)
$ export RTE_TARGET=x86_64-native-linuxapp-gcc  # depends on your env
$ make install T=$RTE_TARGET

PCAP is disabled by default in DPDK configuration, so should be changed if you use this feature. CONFIG_RTE_LIBRTE_PMD_PCAP and CONFIG_RTE_PORT_PCAP defined in config file common_base should be changed to y to enable PCAP.

# dpdk/config/common_base
CONFIG_RTE_LIBRTE_PMD_PCAP=y
...
CONFIG_RTE_PORT_PCAP=y

Compile DPDK with options for target environment.

$ cd dpdk
$ export RTE_SDK=$(pwd)
$ export RTE_TARGET=x86_64-native-linuxapp-gcc  # depends on your env
$ make install T=$RTE_TARGET

2.3. Pyhton¶

Python3 and pip3 are also required because SPP controller is implemented in Pyhton3. Required packages can be installed from requirements.txt.

# Ubuntu
$ sudo apt install python3 \
  python3-pip

For CentOS, you need to specify minor version of python3. Here is an example of installing python3.6.

# CentOS
$ sudo yum install python36 \
  python36-pip

SPP provides requirements.txt for installing required packages of Python3. You might fail to run pip3 without sudo on some environments.

$ pip3 install -r requirements.txt

For some environments, pip3 might install packages under your home directory $HOME/.local/bin and you should add it to $PATH environment variable.

2.4. SPP¶

Clone SPP repository and compile it in any directory.

$ cd /path/to/any
$ git clone http://dpdk.org/git/apps/spp
$ cd spp
$ make  # Confirm that $RTE_SDK and $RTE_TARGET are set

If you use spp_mirror in deep copy mode, which is used for cloning whole of packet data for modification, you should change configuration of copy mode in Makefile of spp_mirror before. It is for copying full payload into a new mbuf. Default mode is shallow copy.

# src/mirror/Makefile
#CFLAGS += -Dspp_mirror_SHALLOWCOPY

Note

Before run make command, you might need to consider if using deep copy for cloning packets in spp_mirror. Comparing with shallow copy, it clones entire packet payload into a new mbuf and it is modifiable, but lower performance. Which of copy mode should be chosen depends on your usage.

2.5. Binding Network Ports to DPDK¶

Network ports must be bound to DPDK with a UIO (Userspace IO) driver. UIO driver is for mapping device memory to userspace and registering interrupts.

2.5.1. UIO Drivers¶

You usually use the standard uio_pci_generic for many use cases or vfio-pci for more robust and secure cases. Both of drivers are included by default in modern Linux kernel.

# Activate uio_pci_generic
$ sudo modprobe uio_pci_generic

# or vfio-pci
$ sudo modprobe vfio-pci

You can also use kmod included in DPDK instead of uio_pci_generic or vfio-pci.

$ sudo modprobe uio
$ sudo insmod kmod/igb_uio.ko

2.5.2. Binding Network Ports¶

Once UIO driver is activated, bind network ports with the driver. DPDK provides usertools/dpdk-devbind.py for managing devices.

Find ports for binding to DPDK by running the tool with -s option.

$ $RTE_SDK/usertools/dpdk-devbind.py --status

Network devices using DPDK-compatible driver
============================================
<none>

Network devices using kernel driver
===================================
0000:29:00.0 '82571EB ... 10bc' if=enp41s0f0 drv=e1000e unused=
0000:29:00.1 '82571EB ... 10bc' if=enp41s0f1 drv=e1000e unused=
0000:2a:00.0 '82571EB ... 10bc' if=enp42s0f0 drv=e1000e unused=
0000:2a:00.1 '82571EB ... 10bc' if=enp42s0f1 drv=e1000e unused=

Other Network devices
=====================
<none>
....

You can find network ports are bound to kernel driver and not to DPDK. To bind a port to DPDK, run dpdk-devbind.py with specifying a driver and a device ID. Device ID is a PCI address of the device or more friendly style like eth0 found by ifconfig or ip command..

# Bind a port with 2a:00.0 (PCI address)
./usertools/dpdk-devbind.py --bind=uio_pci_generic 2a:00.0

# or eth0
./usertools/dpdk-devbind.py --bind=uio_pci_generic eth0

After binding two ports, you can find it is under the DPDK driver and cannot find it by using ifconfig or ip.

$ $RTE_SDK/usertools/dpdk-devbind.py -s

Network devices using DPDK-compatible driver
============================================
0000:2a:00.0 '82571EB ... 10bc' drv=uio_pci_generic unused=vfio-pci
0000:2a:00.1 '82571EB ... 10bc' drv=uio_pci_generic unused=vfio-pci

Network devices using kernel driver
===================================
0000:29:00.0 '...' if=enp41s0f0 drv=e1000e unused=vfio-pci,uio_pci_generic
0000:29:00.1 '...' if=enp41s0f1 drv=e1000e unused=vfio-pci,uio_pci_generic

Other Network devices
=====================
<none>
....

2.6. Confirm DPDK is setup properly¶

For testing, you can confirm if you are ready to use DPDK by running DPDK’s sample application. l2fwd is good example to confirm it before SPP because it is very similar to SPP’s worker process for forwarding.

$ cd $RTE_SDK/examples/l2fwd
$ make
  CC main.o
  LD l2fwd
  INSTALL-APP l2fwd
  INSTALL-MAP l2fwd.map

In this case, run this application simply with just two options while DPDK has many kinds of options.

-l: core list

-p: port mask

$ sudo ./build/app/l2fwd \
  -l 1-2 \
  -- -p 0x3

It must be separated with -- to specify which option is for EAL or application. Refer to L2 Forwarding Sample Application for more details.

2.7. Build Documentation¶

This documentation is able to be built as HTML and PDF formats from make command. Before compiling the documentation, you need to install some of packages required to compile.

For HTML documentation, install sphinx and additional theme.

$ pip3 install sphinx \
  sphinx-rtd-theme

For PDF, install inkscape and latex packages. Some of package names are a little bit different between Ubuntu and CentOS. Here is an example on Ubuntu.

# Ubuntu
$ sudo apt install inkscape \
  texlive-latex-extra \
  texlive-latex-recommended

You might also need to install latexmk in addition to if you use Ubuntu 18.04 LTS.

$ sudo apt install latexmk

This is an example of package installation on CentOS.

# CentOS
$ sudo yum install inkscape \
  latexmk  \
  texlive-latex \
  texlive-*.noarch

On CentOS, you are required to install additional style files, tabulary, capt-of and needspace, without using yum command. It should be added to texlive’s directory by yourself.

First, download style file packages from CTAN and unzip the files in any directory.

$ cd /path/to/any
$ CTAN_PKG_URL=http://mirrors.ctan.org/macros/latex/contrib
$ wget ${CTAN_PKG_URL}/{tabulary,capt-of,needspace}.zip
$ unzip '*.zip'

Then, generate sty files by running installer scripts in each of directories.

$ for pkg in tabulary capt-of needspace
  do cd ${pkg}; latex ${pkg}.ins; cd ..
  done

You can find sty files if installer is finished successfully.

$ ls ./*/*.sty
  ./capt-of/capt-of.sty  ./needspace/needspace.sty  ./tabulary/tabulary.sty

Finally, move them to texlive’s directory and run texhash.

$ sudo mv tabulary capt-of needspace \
  /usr/share/texlive/texmf-dist/tex/latex/
$ sudo texhash

After package installation, HTML documentation is compiled by running make with doc-html. This command launch sphinx for compiling HTML documents. Compiled HTML files are created in docs/guides/_build/html/ and You can find the top page index.html in the directory.

$ make doc-html

PDF documentation is compiled with doc-pdf which runs latex for. Compiled PDF file is created as docs/guides/_build/html/SoftPatchPanel.pdf.

$ make doc-pdf

You can also compile both of HTML and PDF documentations with doc or doc-all.

$ make doc
# or
$ make doc-all

2.8. Build RPM Packages¶

This section describes how to build RPMs of DPDK and SPP from spec files.

2.8.1. Required Packages¶

First of all, install dev tools.

$ sudo yum groupinstall "Development Tools"
$ sudo yum install rpm-build rpmdevtools

Install EPEL (Extra Packages for Enterprise Linux) and required packages.

$ sudo rpm -ivh \
  https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

$ sudo yum install python-pip \
  kernel-devel \
  kernel-headers \
  libpcap-devel \
  doxygen \
  python-sphinx \
  inkscape \
  numactl-devel \
  kernel-devel-$(uname -r) \
  wireshark

$ sudo yum install texlive \
  texlive-latex \
  texlive-xetex \
  texlive-collection-xetex \
  texlive-*.noarch \
  latexmk

Then, install sphinx and its theme with pip.

$ sudo pip install sphinx==1.3.1 sphinx_rtd_theme

2.8.2. DPDK¶

Instead of creating spec file from scratch, you get it provided as a part of dpdk-stable release. However, the version of DPDK in this spec file is just bit old from our targeting version v18.08.1. So we need to make some updates for the spec file before running build task.

Considering working directory for building RPMs, although you can build it in any directory, it is already provided as spp/rpmbuild for conciseness. Download spec file and source in the directory as following.

$ wget -P rpmbuild/SPECS \
  http://git.dpdk.org/dpdk-stable/plain/pkg/dpdk.spec
$ wget -P rpmbuild/SOURCES \
  https://fast.dpdk.org/rel/dpdk-18.08.1.tar.xz

Edit this spec file by following example of diff styled (diff -u exactly) below. + means a newly added line, and - means a deleted line.

Name: dpdk
-Version: 18.11.5
+Version: 18.08.1
 Release: 1
 Packager: packaging@6wind.com
 URL: http://dpdk.org
-Source: http://dpdk.org/browse/dpdk/snapshot/dpdk-%{version}.tar.gz
+Source: https://fast.dpdk.org/rel/dpdk-%{version}.tar.xz
...

Remove entries of no needed architectures, in this case, other than x86_64.

...
-ExclusiveArch: i686 x86_64 aarch64
-%ifarch aarch64
-%global machine armv8a
-%global target arm64-%{machine}-linuxapp-gcc
-%global config arm64-%{machine}-linuxapp-gcc
-%else
-%global machine default
+ExclusiveArch: x86_64
+%global machine native
 %global target %{_arch}-%{machine}-linuxapp-gcc
 %global config %{_arch}-native-linuxapp-gcc
-%endif
...

Update BuildRequires for additional required packages.

...
 BuildRequires: doxygen, python-sphinx, inkscape
-BuildRequires: texlive-collection-latexextra
+BuildRequires: numactl-devel wireshark texlive texlive-latex
+BuildRequires: texlive-xetex texlive-collection-xetex latexmk
...

Add -n option in %prep section.

...
 %prep
-%setup -q
+%setup -q -n %{name}-stable-%{version}
...

Add modifications of config params in %build section.

...
 %build
...
 sed -ri 's,(LIBRTE_PMD_PCAP=).*,\1y,'      %{target}/.config
+sed -ri 's,(CONFIG_RTE_LIBRTE_PMD_PCAP=).*,\1y,' %{target}/.config
+sed -ri 's,(CONFIG_RTE_PORT_PCAP=).*,\1y,'       %{target}/.config
+sed -ri 's,(CONFIG_RTE_EAL_PMD_PATH=).*,\1\"%{_libdir}/dpdk\",' %{target}/.config
 make O=%{target} %{?_smp_mflags}
...

The rest of things is to change to move some libraries in %install section.

...
 %install
 rm -rf %{buildroot}
 make install O=%{target} DESTDIR=%{buildroot} \
       prefix=%{_prefix} bindir=%{_bindir} sbindir=%{_sbindir} \
       includedir=%{_includedir}/dpdk libdir=%{_libdir} \
       datadir=%{_datadir}/dpdk docdir=%{_docdir}/dpdk
+mkdir %{buildroot}%{_libdir}/dpdk
+mv %{buildroot}%{_libdir}/librte_pmd_* %{buildroot}%{_libdir}/dpdk
+mv %{buildroot}%{_libdir}/dpdk/librte_pmd_ring* %{buildroot}%{_libdir}/
+mv %{buildroot}%{_libdir}/dpdk/librte_pmd_vhost* %{buildroot}%{_libdir}/
...

Finally, build RPM with updated spec file.

$ rpmbuild -ba --define="_topdir /path/to/spp/rpmbuild" SPECS/dpdk.spec

You can find three RPMs after build tasks are done in succeed.

$ ls rpmbuild/RPMS/x86_64/
dpdk-18.08.1-1.x86_64.rpm
dpdk-debuginfo-18.08.1-1.x86_64.rpm
dpdk-devel-18.08.1-1.x86_64.rpm

Now, you can install DPDK with yum localinstall, or remove with yum erase.

# Install DPDK
$ sudo yum localinstall rpmbuild/RPMS/x86_64/dpdk-18.08.1-1.x86_64.rpm

# remove
$ sudo yum erase dpdk

2.8.3. SPP¶

For SPP, spec file is provided as spp/rpmbuild/SPECS/spp.spec. You download SPP v18.08.4, or update Version entry in the spec file if you use other version.

$ wget -P rpmbuild/SOURCES \
  http://git.dpdk.org/apps/spp/snapshot/spp-18.08.4.tar.gz

Installing DPDK with RPMs created in the previous section is required for building SPP’s RPM pakcage.

$ cd rpmbuild/RPMS/x86_64/
$ sudo yum localinstall dpdk-18.08.1-1.x86_64.rpm \
  dpdk-devel-18.08.1-1.x86_64.rpm \

Packages for building SPP should also be installed as described in Required Packages and Build Documentation.

Return to the project root and start building SPP RPM.

$ cd /path/to/spp
$ rpmbuild -ba --define="_topdir /path/to/spp/rpmbuild" SPECS/spp.spec

Confirm that two RPMs are generated in RPM/x86_64 directory.

$ ls rpmbuild/RPMS/x86_64/
spp-18.08.3-1.x86_64.rpm
spp-debuginfo-18.08.3-1.x86_64.rpm

You can install SPP with yum localinstall, or remove with yum erase.

# install SPP
$ sudo yum localinstall rpmbuild/RPMS/x86_64/spp-18.08.3-1.x86_64.rpm

# remove SPP
$ sudo yum erase spp