Debugging Network Calls with Webhook Testers

Debugging network calls is hard. The moment you leave your own computer and have to interact with another service somewhere over a remote connection, you sacrifice visibility.

I’ve unfortunately spent a significant amount of time trying to get a connection to work, only to realize the code wasn’t doing what I expected.

As a short example, look at this Python snippet with the requests library.

import requests
url = "https://example.com/"
data = {'hello': 1}
response = requests.post(url, data=data)

I spent a lot of time trying to figure out why this didn’t work until I eventually discovered that data won’t work if you put it in the data= argument for requests, it must be json=. However, it won’t fail. Instead, it will send it as form data.

hello=1

Compare this output:

response = requests.post(url, json=data)

{ "hello": 1 }

Now that I know this, I can make sure to do it right the next time. The problem is, there are a lot of libraries that do network calls. Sometimes they’ll convert an object to JSON, sometimes it won’t. Sometimes it’ll error, sometimes it will just send nothing.

Webhook Testers

This is why, if you’re about to do any code that does a network call, do it to a webhook tester first. Read through the headers, URL and the body (if included). Things to check include:

Was there some kind of implicit conversion (ex. numbers to strings)
Did it drop the body?
Are all headers being provided correctly?

You can use ones provided as a service, but be wary of doing network requests that provide authorization headers. You may be leaking credentials!

You can also use local mocking servers, which I’ll cover in a moment, but there is a big advantage to using a public facing one: you can call it from anywhere.

Services like val.town won’t have access to your local network, so the only place you can send a network call to debug it is to a public facing one.

To deal with this, I self-host a service called Request Baskets. Make sure to lock it down.

Mock It Instead

You can alternatively use mock servers to do this. A popular one I like is Mockoon. You don’t need to use all the advanced features to get what you really need, logging. This will allow you to make requests and view those requests. It does allow you to easily mock that endpoint, if you want that as well (which you should).

Conclusion

In any case, don’t waste your time trying to figure out why a network call doesn’t work. First, make sure that what you think is happening actually is happening, and you’ll save yourself a lot of time.