Rolling PCAP (Packet Capture) for a production network

It is common to see many SOCs (Security Operations Centers) wanting a packet capture of a recent event to trace down some network activity either part of an attack or an investigation.  A file format called PCAP is a good way to store network data on the disk.  However no one can afford to store PCAP forever, so a rolling packet capture depending on your network bandwidth is a very viable way to collect and store PCAP.



I started working with this idea after talking part in the DNS-OARC DITL (Day In The Life) of the internet exercise.  Geoffrey Sisson wrote an early tcpdump-split tool which is a very handy way to split PCAP files into time segments and store them on disk using a simple Unix FIFO (First In First Out)  or named pipes as much smaller PCAP files.  I just added a few features and scripts to make a rolling PCAP possible for any type of packet capture.

Here are the details of using this type of PCAP solution for a network sensor:

1. Download  the tcpdump-split files and the related scripts
2. Compile and install the files in this package
[Defautl installs to /opt/sbin/]
2. Modify the script to suite your settings
[Change files in /opt/sbin/*.sh to suit your needs]
3. Run the script to start capturing
[Default script captures data on eth0 interface and stores in /var/spool/dumps/%Y/%m/%d/%s.pcap format like /var/spool/dumps/2013/05/19/13..pcap for May 19, 2013 data files]

[user@unix4]$ wget -O tcpdump-split-1.1.tar.gz  "https://drive.google.com/uc?export=download&id=0B8n-PrcbTItkcDVGMDE0V0duWUk" [user@unix4]$ tar zxvf  tcpdump-split-1.1.tar.gz [user@unix4]$ cd  tcpdump-split-1.1 [user@unix4]$ make [user@unix4]$ sudo make install [user@unix4]$ mkdir -m0777  /var/spool/dumps/ #Or you could chown nobody /var/spool/dumps/ [user@unix4]$ sudo /opt/sbin/rolling-capture.sh & [user@unix4]$ ls -Rt /var/spool/dumps/ /var/spool/dumps/: 2013 /var/spool/dumps/2013: 03 /var/spool/dumps/2013/03: 24 /var/spool/dumps/2013/03/24: 1395690600.pcap.gz  1395692400.pcap.gz  1395694200.pcap.gz  1395696000.pcap.gz 1395691200.pcap.gz  1395693000.pcap.gz  1395694800.pcap.gz  1395696600.pcap.gz 1395691800.pcap.gz  1395693600.pcap.gz  1395695400.pcap.gz

There are a few commercial solutions that also try to address this problem of storing a rolling PCAP and provide PCAP metadata that is searchable for locating packets of interest.  You could also store the files even smaller window time such as "30 seconds" in hourly folders, just change the format of the folder to be /var/spool/dumps/%Y/%m/%d/%H/%s.pcap