In mutt, my UrlViewWithPython can't deal when it gets handed base64, so instead of detecting base64 in my urlview I did it mutt with the following:

set my_pager = "| filter htmldecode | $HOME/projects/python/urlview.py\n" message-hook '~h base64' 'macro pager u "| filter base64 $my_pager "' message-hook '!~h base64' 'macro pager u $my_pager'