Deep packet inspection (DPI) is a controversial technology and has become a red flag for privacy protection activists. Particularly the debate between US cable operator Comcast and the government regulator FCC has attracted much media attention and contributed to the discredit of DPI. The FCC says in its Memorandum Opinion and Order against Comcast:

“[…] Comcast opens its customers’ mail because it wants to deliver mail not based on the address or type of stamp on the envelope but on the type of letter contained therein.”

This indicates a basic misunderstanding of DPI technology. The correct analogy would be the automated scan of a postcard for certain patterns (e.g. correct stamp, address and content keywords). Opening an  envelope would instead compare with breaking the encryption of an encrypted message.

So much for analogies. More important, however, is to understand the technical details of DPI – and these depend very much on the application that uses DPI techniques. If the application is traffic or bandwidth management, there is no interest in any personal user data. The only goal is to find out the application of the network flows generated by a user. After this classification, rules based on the ISP’s policy can be applied. Here is what happens in the DPI engine in the case of BitTorrent, for instance:

1. Scan the beginning of the first packet’s payload of each new flow for the string “0×13BitTorrent protocol”. Reading the full conversation is neither necessary nor affordable for multi-gigabit traffic management.
2. If a hit occurs, mark the flow as BitTorrent. A classified flow is never again scanned for any patterns.

Another example for a DPI application is virus scanning. Virus scanners not only examine messages more comprehensively for known virus patterns, but they also unpack compressed files to look inside archives – something that goes far beyond what is happening in bandwidth management systems. And nobody seems to mind, most likely because everybody agrees with the usefulness of this application.

But what about DPI-based bandwidth management – do we really need it? Well, to stay with analogies, all of  the world’s postal services have long ago started to offer different types of expedited services, usually indicated with priority stamps or airmail stickers. Interestingly, the Internet has failed to come up with similar differentiated services, although researchers have tried hard. There is no generally available method to mark an IP phone conversation as particularly important. And DPI can fix this, at least partially. It can detect important – and less important – application flows and treat them respectively. The two main limitations are that it requires explicit support for each application, and it is only effective in the network where the bandwidth management system is installed.

And one last point: recently, a technique called “behavioral analysis” (BA) has emerged in the DPI arena. BA uses statistical analysis and heuristics to detect idiosyncratic patterns in the packet and flow level behavior of an application. It is mainly used to classify encrypted communication where classic DPI with pattern matching does not work. There have been recent claims that BA works just as good as DPI, but without any privacy concerns. This is plain wrong. BA either generates more false matches, or it is computationally more expensive. So the best solution really is to combine the two approaches: using DPI where possible and amend it with BA where necessary. From the privacy perspective, it is just the same as DPI. It detects the application of a network flow. Period.

This leads back to the beginning of this post and to my conclusion: as so often with technology, the concern – in our case for privacy – does not come from the technology itself. It is only a tool, and the important thing is how we use it. Yes, it is possible to spy on our communication with DPI methods, but this is not happening in the case of bandwidth management. DPI and its derivative BA are great tools for this application, and thus I had rather called this post

The Future of DPI!

Tags: deep packet inspection, dpi

This entry was posted on Monday, November 17th, 2008 at 11:13:41 by Klaus Mochalski. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.