httpry

Current version: 0.1.7

dumpster / jason / httpry


core program

httpry is a specialized packet sniffer designed for displaying and logging HTTP traffic. It is not intended to perform analysis itself, but to capture, parse, and log the traffic for later analysis. It can be run in real-time displaying the traffic as it is parsed, or as a daemon process that logs to an output file. It is written to be as lightweight and flexible as possible, so that it can be easily adaptable to different applications.

What can you do with it? Here's a few ideas:

Here's an example of the log file output using the default output format string:

# httpry version 0.1.7
# Fields: timestamp,source-ip,dest-ip,direction,method,host,request-uri,http-version,status-code,reason-phrase
2009-01-12 15:02:31 192.168.0.16 209.85.171.103 > GET www.google.com / HTTP/1.1 - -
2009-01-12 15:02:31 192.168.0.16 209.85.171.103 > GET www.google.com / HTTP/1.1 - -
2009-01-12 15:02:32 192.168.0.16 209.85.171.103 > GET www.google.com / HTTP/1.1 - -
2009-01-12 15:02:33 192.168.0.16 209.85.171.103 > GET www.google.com / HTTP/1.1 - -
2009-01-12 15:02:33 209.85.171.103 192.168.0.16 < - - - HTTP/1.1 200 OK
2009-01-12 15:02:33 192.168.0.16 209.85.171.103 > GET www.google.com /intl/en_ALL/images/logo.gif HTTP/1.1 - -
2009-01-12 15:02:33 209.85.171.103 192.168.0.16 < - - - HTTP/1.1 200 OK
2009-01-12 15:02:33 192.168.0.16 209.85.171.103 > GET www.google.com /extern_js/f/CgJlbhICdXMrMAo4DSwrMA44AywrMBg4Ayw/AQ-hC7_2R8g.js HTTP/1.1 - -
2009-01-12 15:02:33 209.85.171.103 192.168.0.16 < - - - HTTP/1.1 200 OK
2009-01-12 15:02:33 192.168.0.16 209.85.173.101 > GET clients1.google.com /generate_204 HTTP/1.1 - -
2009-01-12 15:02:33 209.85.173.101 192.168.0.16 < - - - HTTP/1.1 204 No Content

parsing scripts

Of course, the fun of collecting data is finding ways to analyze it. The log files are designed to be easily parsed by command line utilities, but sometimes you need to dig a little deeper. Complementing the core httpry program is a set of parsing scripts for mining information out of generated log files. Most of these scripts are written as plugins for a core parsing script and include functionality for extracting search terms, searching for specified terms within client flows, and outputting the logs in XML among other things. It is relatively straightforward to write custom plugins for additional parsing tasks.

latest news

This release brings substantial improvements to some existing features. IPv6 parsing can now follow extension headers that are present in the captured packets. Also, the rate statistics code has been substantially overhauled to handle an arbitrary number of hosts, along with a couple of additional switches for controlling behavior. Additionally, this release adds an optional switch to specify the PID filename, which is helpful when running multiple instances of httpry on the same box. For specifics of the changes in this release, check out the changelog

As with many previous releases, most of the major features and improvements in this version are a direct result of contributions of code or ideas, which are always appreciated. The doc/AUTHORS file specifically lists those individuals as their contributions are greatly appreciated! The httpry codebase is hosted on GitHub if you would like to file a bug or contribute back to the project.

Download httpry 0.1.7