httpry is a specialized packet sniffer designed for displaying and logging HTTP traffic. It is not intended to perform analysis itself, but to capture, parse, and log the traffic for later analysis. It can be run in real-time displaying the traffic as it is parsed, or as a daemon process that logs to an output file. It is written to be as lightweight and flexible as possible, so that it can be easily adaptable to different applications.
What can you do with it? Here's a few ideas:
Here's an example of the log file output using the default output format string:
# httpry version 0.1.3
# Fields: timestamp,source-ip,dest-ip,direction,method,host,request-uri,http-version,status-code,reason-phrase
01/14/2007 22:39:41 192.168.0.166 64.81.127.136 > GET www.promode.org / HTTP/1.1 - -
01/14/2007 22:39:43 64.81.127.136 192.168.0.166 < - - - HTTP/1.1 200 OK
01/14/2007 22:39:43 192.168.0.166 64.81.127.136 > GET www.promode.org /style.css HTTP/1.1 - -
01/14/2007 22:39:43 64.81.127.136 192.168.0.166 < - - - HTTP/1.1 404 Not Found
Of course, the fun of collecting data is finding ways to analyze it. The httpry log files are designed to be easily parsed by command line utilities, but sometimes you need to dig a little deeper. Complementing the core httpry program is a set of parsing scripts for mining information out of generated log files. Most of these scripts are written as plugins for a core parsing script and include functionality for extracting search terms, searching for specified terms within client flows, and outputting the logs in XML among other things. It is relatively straightforward to write custom plugins for additional parsing tasks.
This release of httpry brings some worthy fixes and improvements. It now compiles correctly on newer versions of FreeBSD, and has been successfully tested on other platforms as well. A new "quiet" flag (-q) makes it easier to integrate the program's output directly into other scripts and programs. In addition, the parsing of HTTP headers is now more resiliant to non-standard formatting and several minor bugs and issues are resolved.
Some notable changes have been made to the scripts as well. The content analysis log parsing plugin has a new "streak" scoring rule. Also, the rotate log script has been significantly rewritten for readability and to (hopefully) make it a little more OS independent.
If you are using FreeBSD, you can also get httpry as a FreeBSD port.