Packet capturing from the command line

Goals

Create a software component implementing capturing network traffic with the following requirements.

• For SR140 installation, this component will be installed optional. 
• Only the traffic from and to the machine running SR140 will be captured.
• Only IP traffic will always be captured.
• The basic configurable filtering is available:
  • Capture only UDP packets or only TCP packets, or both TCP and UDP (default).
  • Capture only packets coming from and to specified list of IP addresses (the other end is always local SR140 host machine)
  • Capture only packets coming to or from specific ports on local SR140 machine.  
• The resulting output files will be in .pcapng format files. They will be open for view/analysis in Wireshark UI.
• The resulting files will be of reasonable size. There will be rotation when max size is reached, also when day changes.

Additionally to helping resolve customer issues, the component will be useful internally for debugging new features, perhaps for automated tests etc.

Background and strategic fit

For debugging capturing, it has been requested to have the ability to capture network traces from the command line.  Customers are not comfortable running Wireshark and they also do not like capturing all network traffic. The request is to be able to capture network traffic from the command line without running the Wireshark GUI. It would be useful to only capture a certain call for debugging call issues but the initial request is to just capture from the command line. Some other customers may want to avoid having network sniffing capacity installed on their network. So the solution should be implemented as an optional component. The user will decide whether to install it or not in the course of the SR140 installation process. 

Assumptions

The executable that loads and uses btcap dynamic library has to be run with elevated permissions. It should be run as local Administrator in Windows. In Linux, it has to be started as root (or sudo). If btcap is loaded into the context of a non-elevated process then Start() entry point will return error.  

Requirements

API Design

The capture traffic component is implemented as a dynamic library. Implementing it this way works well for making it optionally installed. The library is named btcap.dll on Windows and btcap.so on Linux. That naming schema appears to be in line with other library names in the problem area (pcap, wincap and so on). The code utilizing btcap library may be placed in a variety of executables including the SR140 server process itself. The executables using btcap library must be running as Administrator (in Window) or root (in Linux).  

Important definitions:

• Capture Point. This is defined as one of IP network interfaces available on the host machine PLUS the IP version of packets that we want to capture (IPv4 or IPv6). So, for every NIC installed there can be 2 capture points configured - one for IPv4, and one for IPv6. Capture Point concept is defined simply because the underlying capture technologies require to choose one adapter AND one IP version to setup the packet capture. There will be various ways to identify a Capture Point. The simplest way to point to a particular capture point is to provide a IP address string. Btcap will then understand whether it is IPv4 or IPv6, and then identify the network interface that has the specified IP address. If none of the local IP interfaces have the specified address, then an attempt to start capture will fail. 
• Capture Session. This describes the process of capturing network traffic from a single Capture Point. An executable using btcap library may start multiple capture sessions if necessary. With appropriate configuration, some or all of them can capture from the same Capture Point. 

Basic API

With the terms defined above, the exposed API is very simple, and is the same for Windows and Linux. There are only 2 C functions exported from the library. One of them starts new capture session, the other eventually stops it. Capture session is configured as a sections in btcap config file. The config parameters include capture point, output directory, capture filters, and more. Btcap config file is described in details later in this document.

With this basic API, it is user's responsibility to dynamically load the btcap library and locate exported functions. Later in this document, a simplified level C++ API is described which incapsulates loading library bookkeeping. 

// Start one Capture Session configured in the config file section named sectionName. Session ID is returned in *sessionID.

int btcap_start(const char* configFilePath, const char* sectionName, int* sessionID);

// Stop Capture Session identified with sessionID.

int btcap_stop(int sessionID);

Both methods return 0 on success or a non-zero error code. 

To start a capture session the user calls btcap_start() providing (a) a valid path to a btcap config file and (b) a valid section name within that file.

First, btcap_start() attempts to (a) open the config file, (b) read the section in the file, (c) read session parameters from that section, and (d) validate session parameters. If any of that fails with appropriate error code. 

If all required parameters are read from the config file, btcap_start() attempts to start the capture session in a separate "capture thread". btcap_start() itself returns as soon as the capture thread successfully starts. If the capture cannot start for whatever reason, then the btcap_start() will return an error code, and the capture thread will not exist upon return. So btcap_start() is not blocking.

On success, btcap_start() fills the sessionID with the new session's ID. That is an opaque identifier of the session, similar to Unix file descriptor. It is later used in btcap_stop().

To stop a capture session the user calls btcap_stop() providing the ID of the previously saved session. btcap_stop() signals the capture thread to stop and waits until it is done before returning.

Basic Usage Scenario

Here is the basic sequence of operations utilizing the btcap library (in pseudo-code). 

// The code loading btcap library and obtaining exported entry point pointers by names is omitted. If loading failed, the program exits.  

const char* configFilePath = ".\btcap.cfg";   // Config file path for this session

const char* configFileSection = "wlan0-ipv4"; // Config file section name

// Start capture session

int sessionID = 0;

int rc = btcap_start(configFilePath, configFileSection, &sessionID);

if(rc)

    return 1;

// Session started in separate thread. Do other stuff...

// ... 

// Later stop the capture session

btcap_stop(sessionID);

// Unload btcap library

Btcap Config File Overview

Btcap config file has the same format as SR140 call control file. It consists of sections. Section names can be anything in square brackets. Each section configures one capture session. btcap_start() API (described earlier) takes both config file path and section name as arguments. 

One section defines one capture session which maps to one capture point There can be more than one sections configured for the same capture point.

The following parameters can be configured in a section.    

capture_ip: IPv4 or IPv6 address of a network interface, defining the capture point for this session. Example: capture_ip = 192.168.5.111. This parameter cannot be missing or empty, there is not default value. 

peer_ip: Capture traffic between these IP addresses and the address specified in capture_ip. The value is a comma-separated list. Each item in the list is an IP address or a subnet. The asterisk symbol '*' is used to depict subnets. Example: peer_ip = 192.168.5.*, 192.168.7.118 . There is also a special value, * , depicting ANY IP address. Default value is *. This parameter cannot be empty. If it is missing, the default value will be assigned.

tcp_packets: Boolean parameter instructing the tool whether to capture TCP packets (value of 1) or not (value of 0). Default value is 0. NOTE: UDP packets are always captured. Example: tcp_packets = 1

local_port_include: Capture packets on these ports on capture_ip interface (local ports). Default value is *, instructing the system to capture from any local port. This parameter cannot be empty. The value is a comma-separated list. Items in the list are either individual port numbers or intervals.  Example: local_port_include = 1983, 1984, 2010-2025

local_port_exclude: Exclude specified local ports even if they are to be included in local_port_include. Default value is empty. The value is a comma-separated list. Items in the list are either individual port numbers or intervals. Example: local_port_exclude = 1980,11908,9898

remote_port_include: Capture packets to/from these remote ports. Default value is *, instructing the system to capture any remote port. This parameter cannot be empty. The value is a comma-separated list. Items in the list are either individual port numbers or intervals. Example: remote_port_include = 1983, 1984, 2010-2025

remote_port_exclude: Exclude specified remote ports even if they are to be included in remote_port_include. Default value is empty. The value is a comma-separated list. Items in the list are either individual port numbers or intervals. Example: remote_port_exclude = 1980,11908,9898

results_dir: The path to a directory where the pcapng files are saved by this session. Example: results_dir = ../bin/pcapng-output

max_result_file_size: Max pcapng file size before the new file is started. Example: max_result_file_size = 55555

max_number_of_result_files: Max number of pcapng files to be created before the system starts to delete oldest files. Example: max_number_of_result_files = 100

More parameters will be added as necessary. 

Higher level API - BTCapHelper

A higher level C++ API in form of class BTCapHelper was created mainly to hide the technical details of loading library and also hide the differences in that area between OS platforms. Here is the class definition.

class BTCapHelper
{
public:
virtual ~BTCapHelper() {}
virtual int Start(const char* configFilePath, const char* configFileSection) = 0;
virtual int Stop() = 0;
};

bool BTCap_isAvailable(const char* pathToLibrary);
BTCapHelper* BTCap_GetHelper(const char* pathToLibrary);

Here, BTCap_GetHelper() is the factory function that will create an instance of BTCapHelper-derived platform-dependent implementation.

The BTCapHelper::Start() will (a) load the library, (b) find and save function pointers for btcap_start and btcap_stop, and (c) call btcap_start(). BTCapHelper will remember the sessionID. 

The BTCapHelper::Stop() will (a) call btcap_stop() then (b) unload the library.

Usage scenario with BTCapHelper

Here is the complete command line program for Windows that utilizes BTCapHelper API.

static bool g_exit = false;

// Console CTRL handler function will be called on separate thread
static BOOL WINAPI console_ctrl_handler(DWORD dwCtrlType)
{
    switch (dwCtrlType)
    {
        case CTRL_C_EVENT: // Ctrl+C
        case CTRL_BREAK_EVENT: // Ctrl+Break
        case CTRL_CLOSE_EVENT: // Closing the console window
        g_exit = true;
        return TRUE;
    }
    return FALSE;
}

#include "../include/btcap-helper.h"

const char* libraryPath = "./btcap.dll";

int main(int argc, char* argv[])
{
    // TODO: pass these as command line arguments.
    const char* btcapConfigPath = "./btcap.cfg";
    const char* btcapConfigSection = "wlan0-ipv4";

    printf("Checking if BTCap is available...\n");
    bool ok = BTCap_isAvailable(libraryPath);
    if (!ok)
    {
        printf(" BTCap is NOT available. Exiting.\n");
        return 10;
    }

    printf(" BTCap is available.\n");

    int rc;

    BTCapHelper* btcap = BTCap_GetHelper(libraryPath);

    printf("Starting btcap session...\n");

    rc = btcap->Start(btcapConfigPath, btcapConfigSection);
    if(rc)
    {
        printf("btcap_start FAILED. Exiting.\n");
        return 11;
    }

    SetConsoleCtrlHandler(console_ctrl_handler, TRUE);

    printf("Running btcap session...\n");

    while (!g_exit)
    {
        // Do other stuff
        Sleep(1000);
    }

    printf("Stopping btcap session...\n");

    btcap->Stop();

    delete btcap;

    printf("Exiting...\n");

    return 0;
}

Implementation Overview

Questions

Below is a list of questions to be addressed as a result of this requirements document:

Not Doing