Heroku Scheduler and the UnicodeEncodeError

Ruminate runs a scheduled task on Heroku every hour to grab the new entries from the configured news feeds.

I was noticing that some feeds were not updating, checking the logs I saw that I was getting a:

'ascii' codec can't encode character u'\u2022' in position 49: ordinal not in range(128)

This was odd because if I ran it from the command line on my machine it worked, and even stranger when I ran it with the heroku run it also worked.

Turning to Google I found this answer on StackOverflow - http://stackoverflow.com/a/11762169/27907

The long and short of it is that the print statement will happily default to ascii in python, if not told otherwise.  I am guessing that when run from my console it picks up that it is utf8, but when running detached it defaults back to ascii.  As the answer states this is an easy fix by just setting enviroment variable PYTHONIOENCODING to the codec that you want.  In this case utf8.

Running the following command sets the environment variable on the heroku app to the correct value:

heroku config:add PYTHONIOENCODING=utf8

This fixed the issue, and got my missing entries into the app. 

Hope this helps.