Remote debugging in IDA Pro by http tunnelling

IDA Pro provides remote debugging capability that allows us to debug a target binary residing on a different machine over the network. This feature is very useful in situations such as when we want to debug an executable for an arm device as installing IDA on it is not possible. IDA can remotely debug another binary in two ways - through a gdbserver or by the provided debugger servers (located in dbgsrv directory).

These debugging servers transport the debugger commands, messages and relevant data over a TCP/IP network through BSD sockets. So far so good, but what if the debugging server resided on a virtual host hosting multiple domain names? We cannot use sockets anymore.

A socket connection between two endpoints is characterized by a pair of socket addresses, one for each node. The socket address, in turn, comprises of the IP address and a port number. For an incoming socket connection, a server hosting multiple domains on the same IP address cannot decide which domain to actually forward the request based on socket address alone. Thus remote debugging using sockets is not possible. However, this is not entirely true as there are techniques such as port forwarding (aka virtual server) that can be used to reroute the incoming traffic to various private IPs based on a pre-decided table. Port forwarding capability is not available everywhere so we can ignore it for now. Instead, it would be much better if sockets supported connections based on domain names as described in this paper Name-based Virtual Hosting in TCP.

The Application Layer Protocol HTTP solves the virtual host problem by including the Host header in HTTP messages. It seems that if we can wrap the transport layer socket traffic in plain old HTTP messages our problem would be solved. The rest of the blog post describes this process in detail.

The problem

A few days ago, I was trying some CTF challenge involving an arm binary. The binary was loaded in IDA within a Windows XP VM. Debugging the binary would require a Linux box at the minimum with qemu-arm installed. Rather than powering up my ubuntu VM, I decided to debug it remotely on cloud9. Cloud9 is a sort of VPS that provide Docker Ubuntu containers called as workspaces where we can run whatever we want. The arm binary can be debugged using qemu as follows:

$ qemu-arm-static -g 8081 ./challenge

We are using the user mode emulation capability of qemu to run non-native elf binaries. The port on which qemu listens for incoming gdb connections is specified by the -g flag and is 8081 in this case. We have specified port 8081 as it is one of the few ports cloud9 allows incoming connections. Now, if we try to attach to the process in IDA using remote gdb debugger as the debugger type configured as shown in Figure 1, IDA fails.

Figure 1: Remote debugger configuration (ignore the paths)

This is expected as the container on which the debuggee is running is on a virtual host where multiple containers have same IP addresses with different domain names. A socket connects by IP addresses and not by domain names thus it is not possible to connect to our container using sockets. We can get a clearer picture using netcat.

Let us create a netcat server listening on port 8081 as shown in Figure 2.

Figure 2: Netcat server listening on port 8081

We can try to connect to this server from our Windows XP VM as shown in Figure 3.

Figure 3: Trying to connect to our netcat server

Unsurprisingly, this fails too for the same reason.

The workaround

We have seen that socket connection is be made using IP addresses. However, if we connect using HTTP we can use domain names. This is possible because of the Host header as mentioned earlier, Let's test this concept.

We create a netcat server listening on port 8081 which replies with a HTTP "HELLO WORLD" message. This is done as shown in Figure 4

Figure 4: Netcat server replying with HTTP message

For the client part in the Windows XP box, we use curl instead of netcat as shown in Figure 5. We choose curl over netcat as we are performing an HTTP transaction and not a socket connection.

Figure 5: Using curl to connect to the netcat server

The connection succeeds and we get the HELLO WORLD response. The netcat server running on cloud9 also displays the success status as in Figure 6.

Figure 6: Netcat server replied to the request

From the above experiments, it is clear that we must use HTTP in order to establish a connection to the remote container running on a virtual host. Similarly, if we intend to debug remotely an app using IDA we must also use HTTP instead of sockets.

Using HTTP Tunnelling

We have seen that connection using HTTP is only possible. If we want to use sockets, it must be wrapped in HTTP. This technique of encapsulating one protocol over HTTP is called HTTP tunnelling. Wikipedia explains this best. Primarily, HTTP tunnels are used to bypass restrictive network environments like firewalls where only connections to well-known ports are permitted. We can reuse the tunnelling technique for debugging in IDA as well.

A Http tunnel application consists of two parts - server and client both communicating over HTTP. Before using a Http tunnel the situation was like Figure 7.

Figure 7: Socket connection

After using Http tunnel, the situation would look like Figure 8.

Figure 8: Http tunnelling

The debugger and tunnel client reside on the same machine though they are depicted as separate computers. Similarly, the tunnel client and the debuggee reside on the same cloud9 container. The tunnel client-server pair encapsulates the socket in an Http connection. Using this mechanism we can remotely debug using IDA.

Searching for a Http tunnelling application, I came across Chisel. It is open-source and written in Go. Compiling this from source is simple:

$ git clone https://github.com/jpillora/chisel.git
$ cd chisel
$ go build -o chisel # for compiling native linux binaries
$ GOOS=windows GOARCH=386 go build -o chisel.exe # cross compiling for windows x86

Remote configuration

We run the chisel server on cloud9 listening on port 8081 on all network interfaces:

$ ./chisel server --port=8081
2017/03/31 19:57:03 server: Fingerprint 07:4e:00:e4:82:9b:76:3a:3a:70:55:30:2e:1d:c2:82
2017/03/31 19:57:03 server: Listening on 8081...

qemu runs with the gdbserver listening on port 23946 for incoming gdb connections from IDA.

$ qemu-arm-static -g 23946 ./challenge

The connection between chisel server and qemu is through sockets. The debugger traffic wrapped in Http will be passed to chisel server at port 8081, chisel will extract the payload of the message and pass it to qemu at port 23946 over a socket.

Local configuration

In our Windows XP box we run chisel in client mode with the following command line:

C:\>chisel client qemu-extremecoders-re.c9users.io:8081 1234:23946
2017/04/01 01:28:36 client: Connecting to ws://qemu-extremecoders-re.c9users.io:8081
2017/04/01 01:28:38 client: Fingerprint 07:4e:00:e4:82:9b:76:3a:3a:70:55:30:2e:1d:c2:82
2017/04/01 01:28:39 client: Connected (Latency 203.125ms)

The remote Url on which the chisel server listens (qemu-extremecoders-re.c9users.io:8081) is specified along with the port.

The second set of port (1234:23946) separated by a colon specifies the port mapping from local to remote. It means incoming traffic to chisel client at local port 1234 will be forwarded to the chisel server which will, in turn, relay the traffic over a socket to port 23946 where qemu is listening.

Finally, we need to configure IDA to use the local chisel client as the remote host. This is done as per Figure 9.

Figure 9: IDA remote debugger configuration

The hostname is specified as 127.0.0.1 and the port as 1234. This is the address where the chisel client is accepting socket connections.

At this point, if we try to attach to the remote process, it succeeds with the following message as in Figure 10.

Figure 10: Attach successful

Mission accomplished!

Final words

Http tunnelling is a very effective technique in scenarios where only Http connections are allowed or possible. In this case of remote debugging, we used http tunnelling since normal socket connections cannot be established. With this we come to the end of this post. Hope you find this useful. Ciao!