Annotated libpcap code - ScaleBox project

The following code is from the ScaleBox project of Dr. Striegel at the University of Notre Dame.

Callback function

The callback function for pcap is called each time there is a packet. You can be certain that there was in fact a packet read in by the system if you were called. Think of the callback much like an AFS callback or an ISR (Interrupt Service Routine). There is no return value from this function which means you will need some way to give the packet up (global memory array, pointer to an array) or do your processing directly in the callback.

Initial Setup / Typecasting

/** The libpcap callback function that is provided by the input thread
 * for grabbing packets from libpcap. 
 * @param args   A typecast AdapterPCap object for returning the packet
 * @param header The pcap structure read in by libpcap denoting size and time
 * @param packet The actual data in a byte array
 */
void pkt_callback (u_char *args, const struct pcap_pkthdr *header, const u_char *packet) 
{
   AdapterPCap      * pAdapter;
   Packet         * pPacket;
   
   pAdapter = (AdapterPCap *) args;

The start of the function is your basic setup and typecasts (bleh) for working around the generic type of the callback function. In this case, we are passing in an AdapterPCap? object that will allow us to have a place to put the packet once we get it.

Find a place to put the packet when we get it

   pPacket = (Packet *) g_MemPool.getObject(MEMPOOL_OBJ_PACKET);
   
   if(pPacket == NULL) {
      cerr << "Packet callback failed, no valid packet in the memory pool" << endl;
      return;
   }

Here, we are allocating the packets from a central global memory pool. Doing a malloc or new on each new incoming packet via libpcap is a VERY BAD thing if you want to have any sort of performance. The global memory pool object will allocate as necessary and will pool objects when they are not being used to avoid the malloc/new hit.

If you are writing a simple block of code and processing packets sequentially (i.e. read one, process it, read one, process it), you can simply allocate a single unsigned char or char array of bytes with the size being tied to the maximum size of packet you plan on reading in. In general, I usually allocate 2k or so but that is just personal preference. To be extra efficient, the maximum Ethernet packet size is sufficient (1514 bytes) with checks to avoid buffer overflows.

Did we get the packet?

   if(header->caplen != header->len) {
      cerr << "Warning: libpcap burped and didn't get us the whole packet, bailing on this packet" << endl;
      pAdapter->setReadPacket(NULL);   
      pPacket->release();   
//      g_theMonitor.addStat(MONITOR_STAT_PCAP_ERR,1);
      return;
   }

The caplen field from pcap tells you the actual captured length of the packet in terms of bytes. The len field tells how long the actual packet was. Most likely, this operation will always succeed provided that you have set your snapshot length to a sufficient value. If not, modify how you opened the adapter in your pcap open device function call.

We recently ran into this issue when porting the code to Mac OS X. While Linux happily worked with the tutorial code on-line from other sources, the Mac code kept giving the above warning. Lo and behold, Linux ignored the snaplen argument and went on its merry way to put the entire packet out there. Mac OS X of course did not which made life lots of fun.

Move the data from pcap to a more portable form

   // Set the arrival time of the packet to denote when libpcap got it   
   pPacket->setArrivalTime((timeval *) &(header->ts));      
   pPacket->setData(header->caplen, (char *) packet);      
   pAdapter->addInputQueue(pPacket);
}

Since the ScaleBox code is based on Packet objects being passed along chains to different modules, we need to move the data from the raw byte array offered by pcap into something a bit more portable. First things, first we grab the time from the header itself. The arrival timestamp is when pcap grabbed the packet which is a high resolution timer containing both the time in seconds and partial seconds in the form of microseconds. The code also passes the data to what is essence a copy operator (setData) that _memcpy_'s the packet byte array into the Packet object. The last line places the packet in a queue for later processing by an external thread.

Open the device (initial open)

int AdapterPCap::openDevice () {
   char    errbuf[PCAP_ERRBUF_SIZE];
   int   nResult;

   if(strcmp(m_szDevName, "UNKNOWN") == 0) {
      cerr << "No name specified for adapter, exiting...." << endl;
      exit(-1);
   }
   
   cout << "Device Name: " << m_szDevName << endl;

   m_pDevice = pcap_open_live(m_szDevName,MAX_PKT_SIZE,1,1,errbuf);

The MAX_PKT_SIZE is defined in Packet.h to be 2000. The openlive operation for pcap takes in the device name (nominally the same as what you get from ifconfig -a), the maximum size of the packet to capture, whether or not it should be in promiscuous mode, the maximum time to wait, and a buffer for error information.

   if(m_pDevice == NULL) {
      cerr << "Libpcap open live device failed for " << m_szDevName << endl;
      cerr << "  " << errbuf << endl;
      return -1;
   }

The most important return condition for libpcap is the pointer to the pcap device itself. If this fails, well, you are kind of done to say the least. Failures to open the device usually fall into the following categories:

• Not the super-user: Yup, you have to login as root, su to become the super-user, or do a sudo to execute the code in order to have permission to run in promiscuous mode. Promiscuous mode isn't that exciting any more as most environments are switched but in a wifi context (802.11), there is is still quite a bit of benefit.
• Too many opens for the device: While I'm not sure this is as much of a problem today, there is an issue with how many processes can have the device itself open, i.e. each process does its own openlive operation.
• Adapter does not exist: If you do not see the adapter in ifconfig -a, you cannot open it in libpcap. While libpcap does not require that the adapter has a valid IP address, it does at least need to exist in the first place.

   // Force the adapter to record only traffic inbound to the adapter (Rx)
   //  and not to monitor any outbound traffic
   if(pcap_setdirection(m_pDevice, PCAP_D_IN)) {
      cerr << "Warning: Issue setting direction for pcap device" << endl;
     }

   // For Mac OS X < 10.5 (Leopard), you will need to comment out the
   //  setdirection line

Most of the time, you will only be wanting to get a copy of inbound packets. Keep in mind that libpcap is not on the critical path, i.e. think of pcap more like a tap or mirror of the data rather than having the ability to direct or shape packets. Hence, if you are trying to do a firewall or traffic engineering for apps on the same device, pcap is not the way to go unless you place it on a different box.

  
   cout << "Opened up device " << m_szDevName << " successfully..." << endl;
   setName(m_szDevName);

   startThreads();   
      
   return 0;
}   

void AdapterPCap::readPacket () {
   // Need to supplement this with a sleep that at least defers control
   // to someone else
   
   if(m_bBatchRead) {
      pcap_dispatch(m_pDevice,-1,pkt_callback,(u_char *) this);
   } else {
      pcap_dispatch(m_pDevice,-1,pkt_callback,(u_char *) this);
   }      
}

void    AdapterPCap::setReadPacket (Packet * pPacket) {
   m_pReadPacket = pPacket;   
}

char * AdapterPCap::getDevName () {
   return m_szDevName;
}


void AdapterPCap::setDevName (char * pDevName) {
   cout << "Setting device name to " << pDevName << endl;
   strncpy(m_szDevName, pDevName,PCAP_DEVICE_NAME_LEN);
}

void AdapterPCap::setDevName(string sDevName) {
   cout << "Setting device name to " << sDevName << endl;
   strncpy(m_szDevName, sDevName.c_str(), PCAP_DEVICE_NAME_LEN);
}

void   AdapterPCap::startDevice () {
   openDevice();
}

void   AdapterPCap::dumpBasicInfo  () {
   
}

Writing a packet out

void AdapterPCap::writePacket (Packet * pPacket) {
   int      nBytes;

   // All variants of Linux, Winpcap, and Mac OS X >= 10.5
   nBytes = pcap_inject(m_pDevice, pPacket->getData(), pPacket->getLength());   
   
   // Uncomment and comment the above line for Mac OS X 10.4
   //nBytes = write(pcap_fileno(m_pDevice), pPacket->getData(), pPacket->getLength());
   
   
   if(nBytes < 0) {
      cerr << "Error: Could not write the packet on the device " << m_szDevName << " of " << pPacket->getLength() << " bytes long." << endl;
   } 
}