Handle Incoming Emails Like A Pro [Mailgun API 2.0]

It's like an old family photo album this far back on our blog! This post first came out way back in 2011.

Sending emails used to be hard but with the Mailgun Email Sending API we moved beyond SMTP and MIME and made it a trivial, dare I say, pleasant experience.

But if your application needs to engage in email conversations, as is often the case with enterprise apps, sending is less than half the battle. Let’s say you want to build an email-based bot which plugs into the workflow inside of your enterprise app. Consider the obstacles you’d need to overcome:

Transport issues. The incoming messages need to reach your app somehow. This means there needs to be an MX record somewhere which points to a mail server, capable of forwarding incoming mail into your code. There are many gotchas here. One of them is that you probably want to generate a proper “invalid mailbox” bounces to notify senders if they made a typo in the address.

Spam handling issues. Once you start accepting mail for a given MX domain, you must realize that eventually most of the incoming email will be spam. And even if your mail acceptance policy is whitelist-based, having your app deal with spam attacks is not always desirable.

MIME parsing issues. We have talked about this before, but parsing MIME is a painful exercise in most programming languages. MIME parsing libraries are not numerous and many of them suffer from poor tolerance to real world traffic.

Message Content. Your problems with parsing go well beyond MIME. Most incoming emails are usually replies to messages sent earlier and you usually want to remove the giant quoted parts. Oftentimes, extracting a person’s signature from a message body is desired, as well.

Looks quite daunting for something as simple as getting a snippet of text into an application. Wouldn’t it be nice to have something similar to Sinatra or Flask and be able to simply say in Python:

Ruby folks love blocks. Ruby folks would love to be able to say:

The idea behind these code examples is similar to the routing mechanism featured in modern MVC frameworks but instead of matching URLs to controller actions, we want to define routes that mach a recipient address pattern to a function in your code.

But how do you go about implementing email_in() given the difficulties listed above?

Mailgun Routes to the rescue!

We designed them specifically with this use case in mind and they are, by far, the most pleasant way to build a two-way email messaging app. Trust us, we are email doctors!  But enough talking, lets do something.

First, lets define a new route in the Mailgun control panel. This route will match the recipient address to “.*@myapp.com” regular expression, and if there is a match, it will do two things:

• It will parse the message and POST it into the URL “/emailin” of your app.

• It will also forward the copy of the message to a developer’s mailbox, say developer@myapp.com

Now we can handle messages that go into @myapp.com by adding the following code into the web app:

Lets send a test message using Gmail to our web app and see what is posted. Here’s the screenshot of the message as it appears in GMail. Notice that it has an actual body of what’s been written (stripped-text), the signature and the quoted part which in most cases is irrelevant:

The POSTed data is shown below as dumped Python’s MultiDict request. Note that in addition to synthetic parameters like “recipient” or “stripped-text”, Mailgun also posts all MIME headers into the app, so you have a full access to everything:

POST parameters (dumped in the log as Python’s MultiDict)

('From', u'Ev Kontsevoy '), ('sender', u'ev@mailgunhq.com'), ('To', u'Awesome Bot '), ('attachment-count', u'1'), ('Subject', u'Re: Your application')]) ('stripped-text', u'My application is attached.nThanks.'), ('stripped-html', u'HTML version of stripped-text'), ('body-html', u'[full html version of the message]'), ('body-plain', u'[full text version of the message]'), ('stripped-signature', u'-- nEv Kontsevoy,nCo-founder and CEO of Mailgun.net - the emailnplatform for developers.'), ('recipient', u'bot@hello.mailgun.org'), ('subject', u'Re: Your application'), ('timestamp', u'1320695889'), ('signature', u'b8869291bd72f1ad38238429c370cb13a109eab01681a31b1f4a2751df1e3379'), ('token', u'9ysf1gfmskxxsp1zqwpwrqf2qd4ctdmi5e$k-ajx$x0h846u88'), ('In-Reply-To', u'Message-Id-of-original-message'), ('Date', u'Mon, 7 Nov 2011 11:58:06 -0800'), ('Message-Id', u'message-id-goes-here'), ('X-Originating-Ip', u'[216.156.80.78]'), # NOTE: ALL message fields are parsed and pasted. If some fields (like "Cc") are # missing here it only means they were absent from the message.

POSTed files:

('attachment-1', FileStorage: 'application.pdg')

Lets review. What do we have here?

• Incoming email traffic is matched against a regular expression applied to a message recipient.

• Matching messages are parsed, checked for spam and HTTP POSTed into the URL of your application.

• Quoted parts of the message are separated (stripped), signature is detected.

• If your application is down and not responding, messages will be queued and subsequent delivery attempts will be made for up to 3 days.

• Additionally, matched messages can be optionally stored in a mailbox for archival, backup or debugging purposes.

• None of the complexities matter to you anymore. You’re busy writing sweet profit-making code.

And by the way, manipulating email routes can be done programmatically via an API. This allows you to build a magical Flask/Sinatra-like @email_in() decorator which would bind everything together.

Mailgun routes can do more than a simple recipient address matching. They support regular expression captures, match-if-nothing-else matched behavior, they have priority of execution, they’re basically a simple mail routing programming language.

There you have it. Pure awesomeness. Now lets get busy and rid the world of dumb “no-reply@” emails. It’s about time.

Peace! — Mailgunners

Edit: feel free to participate in discussion on HN