How To Effectively Use Webhooks For Email Delivery

Thank you to everyone who attended this week’s Mailgun Google Hangout on How to Effectively Use Webhooks for Email Delivery featuring Mailgun Email Expert, Chris Hammer. Below is the video along with a timestamp of the sections and a transcript of the Q&A. For your reference, you can access the deck here and we also recommend reading our Guide to Webhooks.

2:00 What is a webhook and how do they work? 2:30 Events and Parameters Defined 7:30 Steps to set up and secure a webhook 10:00 Use Cases

Q&A

1. What are your best practices are for getting delivery notifications in real time without DDOSing the receiving server when sending at scale (i.e. XXX,XXX’s of emails send all at once)?

A. Our API would be a much better fit whenever you are sending such a large amount of emails. When looking to monitor delivery notifications you would want to move away from webhooks after reaching a certain threshold of emails sent per day. This threshold can change depending on the environment where you are hosting your webhook endpoint.

2. Is it possible to use webhooks to create opt-in forms on sites without using 3rd party plugins/services? If so, can you collect submitter first name?

A. Unfortunately no. Webhooks work off of events that occur on emails you have already sent and ideally, you are sending to users who have already opted in to receiving your emails.

3. Would you comment on how best to test a webhook call thru various use cases? For example, using the API to send an email to a known bouncing email and see and process the webhook postback? And where I can find a table of expected responses to various use cases?

A. We have a “Test Webhook” button for every Webhook event type in the Mailgun control panel. I’d recommend using this to test out each webhook when configuring that endpoint. You can find the data sent in each webhook in our documentation. We also have a tool that you can use, bin.mailgun.net, where you can send these requests and view the data sent. I’d also suggest checking out runscope for testing webhooks as well. It’s similar to bin.mailgun.net, however it provides a bit more information about the requests.

4. Can I set up a website like WordPress to send all emails from the website through Mailgun, on the same domain name I might be using for email at another provider like Rackspace email or Hosted Exchange?

A. Yes for sending. No for receiving. Only one email server can receive messages for a given domain name. It could be either Mailgun or Google servers, but not both. However, you can use the same domain for sending at multiple servers. If you’d like to register your domain at multiple servers for sending but you don’t want to receive email at Mailgun, don’t configure your MX records to point to Mailgun.

If you’re receiving emails elsewhere with your domain, we recommend using a subdomain at Mailgun so you can also receive emails at Mailgun. This helps improve deliverability and allows us to more easily deal with any issues that arise with recipient email servers.

5. You mentioned polling the API when sending at scale. Is polling the only option or is there a pub sub service (which webhooks is an example of, probably)?

A. Polling the API would be the only option at the moment. Our Events API is great for this, as you can poll for specific time ranges, events, and other user data.

6. Can I use webhooks to have people directly respond to my forum by replying to the email sent to them that comes from the forum? (People get an email when the forum is updated and I would rather them just reply to the email then click a link to go to the forum and respond there.)

A. Webhooks wouldn’t be the right solution here, however, our Routes feature (which is similar) would work great. Essentially, what you would want to do is have forum thread have an ID number associated with it and send an email to these recipients with these ID as the sender (Ex. 1234@forum.domain.com as the sender). You can then configure this domain to receive emails with Mailgun and use our Routes feature to POST data associated with these replies to your forum. Our Routes feature works similarly to Webhooks in that we will POST information about these messages to your endpoint. The difference being that Webhooks works off of different Events that occur for these messages (links clicked, unsubscribes, etc) and Routes obtain data from incoming emails (replier, subject, message content, etc)

7. After configuring some Mailgun Webhooks in our API I would like to comment about the testing process. At the Mailgun website, there is no log of what callbacks were made to our API.

A. This is true. We currently do not have much logging for our Webhooks feature. Right now we only log data about the Webhook whenever it fails to POST to the endpoint (exception to this is the delivered endpoint, which isn’t logged at all). The Event data associated with that Webhook event should be in the logs, however, it will not contain all the information that the Webhook might.

8. There is only a ‘Test Webhook’ link on the page where we wire up each webhook. This posts a set of test parameters to the configured link, but I have found nowhere where I can see a log of it, when and the contents of an actual webhook POST made in repo.

A. See answer to 7.

9. Is there a way to get the response as JSON or XML Format for a delivered Webhook?

A. Unfortunately not, our Webhook will post the data url-encoded or in multipart (depending on the content of the Webhook)

10. At what level of emails would you say you should move to a polled model rather than a webhook model?

A. This is a hard question to answer as this can change based on how much you want to put into your infrastructure and what webhooks you’re interested in. Consider this when deciding whether to implement a polled or webhook model: Each email you send can have several events associated with it (delivered, clicked, opened, etc). If there are at least 3 events associated with each email you send, would your infrastructure be able to handle X emails times 3 per hour/day? If not, you would want to look into a polled model (API) instead. When looking to determine this you may also want to see how many requests/minute your infrastructure network can handle.

If you’re sending a large amount of messages, all within a short time, keep in mind Mailgun will rapidly try and deliver the delivery Webhook events as quickly as they occur. If your infrastructure is not capable of handling thousands of requests per minute, the requests may time out and drop. When you receive the Webhook POST, are you storing that data into a database? Sometimes the influx of traffic will cause the database to run slower, causing the requests to slow or time out. Just like a flood of visitors to your website.