Browse Recent:

Posted on

Bash Script – Move Files into Subdirectories Based on Modified Date – OR – Moving Ntop Netflow Dumps into Subdirectories by Date

In a busy network environment, it is critical to have some form of network monitoring on all your servers and equipment. Network monitoring comes in many different forms and flavors, whether it be to monitor critical system services and applications via SNMP, WMI, or some proprietary third party software, or just generically pinging some devices to make sure they are up.

Nowadays, it is getting more and more necessary to dig deeper to be able to track what end users are doing and what websites they are visiting and this is where the Cisco Netflow comes in handy. Essentially, netflow allows us to peer down into the network traffic streams and give us vital source, destination, and protocol information coming to and from our network hosts but isn’t quite as storage intensive as doing a full fledged pcap dump, which makes historical accounting of this data a whole lot nicer.
Continue reading “Bash Script – Move Files into Subdirectories Based on Modified Date – OR – Moving Ntop Netflow Dumps into Subdirectories by Date”

Posted on

Bash Script – Pcapsipdump Spool Directory File Rotation – OR – Rotate Directories Based on YYYYMMDD Format

In the config file for the pcapsipdump program, there is a retention option where you are supposed to be able to enter the number of days to keep the directories in your spool folder and it should auto purge out the old directories/files, however, this option either doesn’t appear to be implemented yet or doesn’t work if you run the app as a different user (could be a permissions issue maybe?).

Anyhow, this script is my solution to the problem. It could also very easily be modified and used where someone needs to search and delete directories that are based on YYYYMMDD format with the ability to whitelist or ignore certain directories, hence the alternate title. Yet another alternate title could even be “Bash Script Functions – Convert YYYYMMDD to Unix Time and Vice Versa” but obviously the script would need modified a little because it has been tailored to fit my needs.

When run as a cron job, the script will grab both the spool directory and the retention period from the pcapsipdump config file and will purge out any old folders based on YYYYMMDD format so they aren’t eating up all your valuable disk space.

I also added a couple of noteworthy user-configurable options. The first being a directory whitelist feature (IGNORE_DIRS array), this option can be used in situations where there are directories that you don’t want purged or would like to keep the data indefinitely and that can also be used in conjunction with the PURGE_ALL feature to delete out any subdirectories that aren’t specifically listed in the whitelist if you are a clean freak.

Download it now – pcaprotate.sh

#!/bin/bash
#
# pcapsipdump file rotation
# By Nathan Thomas
# 12/05/2014
#

### VARS ###
# Location of the pcapsipdump config file
CONF_FILE='/etc/default/pcapsipdump';

# Keep the spool folder clean - Delete all folders (except those in the ignored directories array)
# yes - Keep the spool folder free from any directories not in the ignore list
# no - Don't worry about any other random folders in the spool directory
 Continue reading "Bash Script – Pcapsipdump Spool Directory File Rotation – OR – Rotate Directories Based on YYYYMMDD Format"
Posted on

pcapsipdump – Dump SIP Signalling and RTP Streams Per Call Leg into Separate Pcap Files

For a VoIP administrator, in order to properly diagnose and troubleshoot issues sometimes, you need the ability to go to be able to go back in time and dig deep within packet traces to get to the heart of the problem. Unfortunately, this means following the NSA’s policy and doing blanket data collection if you can handle the disk storage requirements.

Sure if the problem is easily reproducible, then you can either do a SIP trace in your VoIP software to get just the signalling data or if you need the RTP streams too, you can use “tcpdump” to capture the packets and avoid this scenario altogether but that isn’t always the case and using tcpdump certainly does have it’s limitations.

Capture (pcap) files get huge and harder to work with, you have to create filters to find the data you need, the list goes on and on. Of course, there are some helpful switches you can use to try to get around this when trying to capture data for long periods of time but then you still run into the problem of needing to run the application as a daemon, rotating log files, and then you still have to dig extensively for the data you need.
Continue reading “pcapsipdump – Dump SIP Signalling and RTP Streams Per Call Leg into Separate Pcap Files”