RFC 4475 is not enough

When beSTORM is used to test VoIP products, it’s usually the standard SIP, SDP and RTP fuzzing. But we were recently asked about opinion on RFC 4475, which was an interesting case study. RFC 4475 for those who do not know is an IETF standard whose goal is to give[s] examples of Session Initiation Protocol (SIP) test messages designed to exercise and “torture” a SIP implementation. This is great but as the RFC states, these are just a few examples – to be more specific 49 discrete examples.

These 49 examples claim to check a broad range of problems that a SIP parser may come across, and that it should either ignore, reject it or handle it correctly.These examples try to test more than one malformed, incorrect or problematic field at a time – opening the possibility that one problematic field is preventing others from being processed.

My problem with these 49 cases is that they seem to be very tailored, testing for specific stuff, without testing all the possible variables of that same example. Lets take the Content-Length header. One example checks the resilience to a negative value, another to a large positive, another yet to the value of zero (0).Did you notice what is missing, for example where is the off-by one underflows/overflows?

Another example is the use of IP addresses inside the sample data, a carelessness or a small oversight by the tester might make the whole example invalid and not parser-able by the test subject. It might be discarded by the product making the entire test worthless, but the tester happy for ‘passing’ the test. It’s like passing a final exam by not showing up!
In conclusion, running those 49 examples is not straight forward, in addition once you ran them and passed, can you say you are ok? From experience I can tell you that in many cases, both our customers and open source products we have tested with beSTORM failed the complete fuzzing test while they passed the RFC 4475 – beSTORM simply discovered one or more vulnerabilities in them that simply didn’t fit any of the 49 examples provided inside the RFC 4475 torture examples.

My recommendation? Testing for those 49 examples only tells you that you are compliant with RFC 4475. Only a serious fuzzer will tell you if your product is secure against SIP, SDP or RTP based attacks.