Freeswitch SIP Capture over TLS with SNGREP and HEP

February 22, 2019

Encrypt all the things! This has been a popular sentiment in the open source community for the last few years. And with the rise of Lets Encrypt this has never been easier and cheaper to do.

One industry that hasn’t been so quick to adopt these new secure mentalities are wholesale VoIP providers. But this is changing and providers like Simwood, Voxbone, DIDLogic and Twilio now offer encrypted voice calls using SIP over TLS with SRTP media.

SIP over TLS - An operations nightmare

I have been using encrypted calls with Freeswitch for a number of years. However one aspect that has caused frustration is diagnosing SIP problems when using SIP over an encrypted connection (TLS).

Normally when you are diagnosing a SIP issue, one would use a SIP capture tool like SNGREP or Homer.

These tools work by hooking themselves into the raw network packets that are flowing between your Freeswitch server and the remote user’s handset.

However encrypted TLS connections are designed to stop exactly this sort of interception, and so these tools are not able to understand the raw packets they capture. This renders SIP diagnosis using raw packet captures useless!

HEP protocol to the rescue!

One approach to solve this problem is to restrict the type of encrypted connection that Freeswitch can establish (specifically forward secrecy) and then give the private key that your server is using to the SIP capture tool.

Tools like SNGREP support this feature using the --keyfile flag.

However I’ve never managed to get that to work, and besides, limiting the strength of the encryption you’re using so that it can be intercepted seems to defeat the purpose.

This is where a protocol called HEP (Homer Encapsulation Protocol) comes to the rescue. As the name suggests, it originally came out of the Homer SIP capture project. But since then has been integrated into many SIP tools, including Freeswitch and SNGREP.

The basic idea behind HEP is to provide a way for the voice server itself to send the raw SIP packets to a dedicated capture process after the packets have been decrypted.

This way, the voice server and handset can use any form of encryption they both support, and the decrypted packets can be sent to a local capture agent.

HEP and Freeswitch

Enabling HEP in Freeswitch is very simple.

In /etc/freeswitch/autoload_configs/sofia.conf.xml add a local SIP cature destination as follows:

<global_settings>
  <param name="capture-server" value="udp:127.0.0.1:9060"/>
</global_settings>

Then restart Freeswitch and run:

fs_cli -x 'sofia global capture on'

Note: You will need to run this each time you want to start a capture, and when you’re done run:

fs_cli -x 'sofia global capture off'

HEP and SNGREP

To start SNGREP and tell it to listen for decrypted packets from from Freeswitch, start it as so:

sngrep -L udp:127.0.0.1:9060 -d lo

The -d lo part is to instruct sngrep to switch its raw packet capture to the local loopback interface. This is a bit of a kludge, as really I would prefer to disable its raw packet capture entirely, which is not currently possible. But by only listening on the local loop back interface it should avoid you getting any packets captured twice.

A security limitation in Freeswitch

But there is a problem with this. If you’ve tried up until this point you will have found that you are seeing some packets in SNGREP, but they are only the responses that Freeswitch is sending to remote end points.

All incoming packets are missing. I wasn’t the first person to experience this.

Freeswitch has a bug (FS-9657) from October 2016 that means that incoming SIP packets are not delivered to SNGREP via HEP. Only outgoing packets are recorded.

This is a major problem, and effectively makes the HEP feature useless when using TLS. Which in turn effectively makes using TLS SIP in production (where you need to support your customer’s issues with SIP traces) unrealistic.

A 3 line patch solution

This was a source of frustration to me, and in true open source style, I “scratched my own itch” and in August 2018 spent some time trying to figure out what was causing this bug.

I am not a C programmer by trade, but the Freeswitch code is well structured and easy to follow.

Eventually I stumbled across the libsofia-sip-ua/tport directory.

In this directory are several files implementing the different transports that the SOFIA SIP library supports; UDP, TCP and TLS.

After comparing the code in tport_type_tcp.c with the code in tport_type_tls.c I discovered that 3 critical lines were missing from the TLS transport file. The lines that sent the received decrypted packets to the capture server.

So I added these lines in and made a pull-request.

Sadly after over 5 months later, the Freeswitch team haven’t given any feedback on my patch, despite several requests in the issue tracker and in their Slack channel. I was hoping that such a small patch, that would allow more people to use Freeswitch with TLS, would be accepted quickly (or at least be responded to with feedback). I guess they are too busy.

Anyway, never mind, if you are experiencing this issue too, I have created a patch file you can apply directly to a Freeswitch 1.8 tree.

Happy secure VoIPing!

Update 2019-02-26

Anthony Minessale, the Freeswitch founder, e-mailed me today and apologised for missing my pull-request and advised that this code will be merged shortly and be available in the next 1.8 point release. He explained they get thousands of bug requests and that sometimes they can miss some.

The code has now been merged! :)