mutt's secret sauce

by on
8 minute read

Apple Mail, Outlook, Geary, Thunderbird, N1... I've gone through them all and more. Emails have always been an issue for me because I've moved between different kinds of operating systems and couldn't settle on one email client that was good on all of them. Mostly, they all suck, and I had never got a good integration with Vim/NeoVim. For this I even tried Vmail and didn't like it too!

I've been a terminal user for good time and have been moving more and more stuff to the terminal, as much as possible! This trend peaked after I started using tmuxp, an awesome tmux session manager. I started to plan sessions for kinds of tasks I'd be doing, I have session recipes for project X, project Y, instant messaging, writing on this blog and, off course, email. Each of them get started by a single command usually issued from my main session (and usually command completed by prezto/zsh). Yes, my full flagged email client is a tmux session!

But the tmux session (started by tmuxp load -d mail) is just the skin, the core resides on NeoMutt, the only email client that doesn't suck that much.

NeoMutt

NeoMutt, the Mutt variant that cares to embed a lot of historically useful patches and other improvements, is a great email client, but it has its limitations and quirks. The code base of this beast is quite antiquated, and people that come in search of its rescue quite often don't care, or even abominates, HTML emails, for example. NeoMutt itself have no support for such thing, or even multipart/alternative emails, while at the same time it shines for plain text and PGP. This deficiency is what I've set up to address.

HTML Emails from Markdown/AsciiDoc/Whatever

Not supporting multipart/alternative is a show stopper for HTML emails, without it you're left to solutions that replace a non multipart text email for a non multipart HTML one, I guess you won't want your underworld alt-text email friends furious right? But at least Mutt follows the well-known Unix adage: "do one thing, do it well", so this mantra save us. Mutt allows you to leave the task of sending the email for another program, the email sender in email parlance is known as MTA (Mail Transfer Agent) and my choice for it is msmtp.

The email sender

Configuration of msmtp is really simple, so I leave that for the docs. This is how I set NeoMutt to use an external MTA:

set sendmail = "/bin/sh $HOME/.mutt/scripts/sendmail"

Where $HOME/.mutt/scripts/sendmail is:

#!/bin/sh

ACCOUNT=`lpass show --username $LPASS_USER`
PASSWORDEVAL="lpass show --password $LPASS_USER"
MIMEmbellish | msmtp -a "$ACCOUNT" --passwordeval "$PASSWORDEVAL" "$@"

I start NeoMutt with $LPASS_USER=MYUSER mutt, which is enough information for it and the sendmail script to ask for per account credentials from my password manager.

The email data coming from NeoMutt goes to the sendmails stdin, and as can be checked from the sendmail script, data is piped to MIMEmbellish before reaching msmtp.

MIMEmbellish: The email embellisher

MIMEmbellish is the power horse of the transition from text-only to text+HTML. This script simply takes the email from stdin and transforms it into a multipart email with HTML using pandoc.

It can easily be tried from terminal, for example:

curl --silent https://raw.githubusercontent.com/mikel/mail/master/spec/fixtures/emails/plain_emails/raw_email_simple.eml \
    | MIMEmbellish

The repository for that URL contains several file email samples.

MIMEmbellish is quite smart though, on seeking the email target for conversion it will traverse multipart/mixed if necessary (for the case you added some attachments from NeoMutt) and also pick the text part from signed emails. It will do this at the top of the MIME three solely, it doesn't try to recurse indefinitely, and it will grab solely the first target part it seeks to do the conversion.

MIMEmbellish doesn't accept arguments yet but I've strived to make its source simple to modify, there are several customization points in it. By default, it seeks for text/plain and text/markdown for conversion to HTML. It not only can produce an alternative HTML part, it can also replace the target textual part (if it's not signed) with other conversion (like Markdown to plain text), but this is not enabled by default.

Automatic Attachments

MIMEmbellish will search for local machine file:/// links in the Markdown contents and automatically add them as attachments, so you may not even bother manually adding attachments in the Mutt sending screen. Also, a nice side effect is that file:/// image links in Markdown get shown in preview 🙂

Automatic Image References

Attachments added in the NeoMutt interface will get cid:filename references automatically, which can be referred to from inside Markdown contents. If you add /foo/bar/my_img.png you can use ![](cid:my_img) from Markdown for example. You also can just use automatic attachments like previously explained for the same task like ![](file:///foo/bar/my_img.png) and not add it from NeoMutt interface.

cid:filename references are created from Content-Disposition's filename parameter, without the extension. This parameter is automatically added by NeoMutt while adding an attachment.

Email Style Sheets

It's not a must to have a full flagged email style sheet, though, I've grabbed the single one I could find online that I felt it could fit for the task and forked to apply my tweaks:

After applying my modifications I've minified the style sheet with cleancss and added it to my pandoc HTML template for emails.

The preview link above shows how emails actually may look.

Fixes for Gmail

Even though a style sheet isn't needed, to have functional inline discussion and quotations, it's necessary to have some style fixes. For Gmail, the blockquote style must be inline and have class="gmail_quote", I do this hard coded from MIMEmbellish:

def gmailfy(payload):
    return payload.replace('<blockquote>',
                           '<blockquote class="gmail_quote" style="'
                           'padding: 0 7px 0 7px;'
                           'border-left: 2px solid #cccccc;'
                           'font-style: italic;'
                           'margin: 0 0 7px 3px;'
                           '">')

The style hard coded here is an excerpt from the original typographic-email style sheet. Without this, discussion quotations are lost during conversation.

Note that not all of HTML and CSS is supported by HTML emails, see Gmail restrictions for example.

Instant Preview

I use a modd recipe for monitoring NeoMutt's temporary directory for mail file changes for rendering. I set tmpdir = '/tmp/mutt' in my .muttrc since NeoMutt uses /tmp/ by default and I can't monitor the entire /tmp directory:

@shell = bash
@pandoc = pandoc -f markdown-blank_before_blockquote -t HTML5 --standalone --highlight-style=tango --variable=pagetitle:Email --katex --template=$HOME/.pandoc/templates/email.html

neomutt-* {
    prep +onchange: "
        # mutt/pandoc email preview
        shopt -s nullglob
        umask 077
        [[ -z '@mods' ]] || for i in @mods; do [[ -w $i ]] && @pandoc $i -o email.html; done
        [[ -z '@mods' ]] && rm -f email.html
        exit 0
    "
}

With this recipe, anytime I write to disk the changes of the email being authored, the HTML version from pandoc is updated on /tmp/mutt/email.html, which I leave open on Firefox for automatic reloading through Auto Reload, but that could also be served by something like devd. I execute modd from /tmp/mutt/.

For previewing I use slightly different pandoc output settings, for example, for LaTeX math I use \(\KaTeX\) instead of image generation.

tmux tips

At the top of the command chain is tmux with the big help of tmuxp. I start all my mail accounts on individual NeoMutt instances, one per tmux window ( I do this because NeoMutt multi account support isn't that great):

session_name: mail
windows:
- window_name: oblita
  panes:
    - LPASS_USER=OBLITA_USER mutt
- window_name: francisco
  shell_command_before:
    - sleep 1
  panes:
    - LPASS_USER=FRANCISCO_USER mutt
- window_name: modd
  shell_command_before:
    - mkdir /tmp/mutt
    - cd -P /tmp/mutt
  panes:
    - modd -f ~/.mutt/modd.conf

With this tmuxp recipe, I just do tmuxp -d load mail and both NeoMutt instances and the modd instance for email previewing are loaded, which allows me to write Markdown emails with preview anytime.

I also have this script for when new emails arrive on NeoMutt:

#!/bin/sh

tput bel
paplay /usr/share/sounds/freedesktop/stereo/message.oga

The NeoMutt setting is set new_mail_command = "/bin/sh $HOME/.mutt/scripts/new_mail"

It plays a sound, and echoes a bell, but given bell in my system doesn't make any sound, this is just so tmux highlights the window name for which there's a new message, because it can monitor for bells (monitor-bell is a tmux 2.6 setting though):

# alerts
set -g monitor-bell on
set -g bell-action any

Conclusion

There was some work to reach this state but it has payed off, it's all saved in dotfiles now anyway and I don't need to fight with the email client anymore, I use this both on ArchLinux as well as on Windows (through WSL), it's just Markdown on Vim/NeoVim now, to easily write emails with complex content I'd never figure out how elsewhere 🎉, and without making plain text a second class citizen!

This was my second stage on NeoMutt setup, on my first one I've setup .mailcap, etc, and fought a bit for Firefox to assume HTML email fragments as UTF-8. I read most emails from NeoMutt's terminal interface most of the time, but sometimes when I need to see HTML parts, it's quite easy to open them in a new Firefox tab due to proper mailcap configuration. I was enjoying NeoMutt a lot but didn't bother that my plain text emails, written with glorious 80 column limit, would be viewed with random line breaks on mobile clients for example. Being unable to produce short links for long URLs was also annoying. So, this is now all fixed. I'm happy enough with my current setup and NeoMutt's native IMAP, but maybe on a third and final stage in a remote future I try notmuch and offlineimap.

mutt, neomutt, email, gmail, markdown, html, multipart, pandoc, modd, tmux, tmuxp, vim, python
Spotted a mistake in this article? Why not suggest an edit!