The Invent with Python Blog

Sun 25 March 2012

How to Code a Twitter Bot in Python on Dreamhost

Posted by Al Sweigart in python   

I made a twitter bot that checks every hour for someone who has asked the question, “Why do homeless people have dogs?” and automatically replies, “Because a dog will love you even though you are homeless.” It’s running right now at @YHobosHaveDogs.

Figuring out how to code this took a couple evenings and a little hair pulling, so I decided to document the process in this blog article to make it easier for the next programmer. This will be making a Twitter bot in Python using the python-twitter module (which runs on Python 2, not Python 3), and then running the bot from my Dreamhost server (but most likely any web host will work just fine. Or if you have a computer that is always online, you can run the bot from that). First we will run the bot from our machine to test it out, and then load it onto the Dreamhost web host. (I’m running a Windows box, but the steps should work on any OS.)

  1. Download the python-twitter module (I tried some of the other modules but didn’t like them as much.)
  2. Unzip this file and from the command line in the unzipped directory, run “python setup.py install” to install the twitter module.
  3. Run Python and then type, “import twitter”. If you don’t get any error messages, the module has installed correctly.
  4. Go to http://twitter.com and create a new Twitter account for this bot. (You might need to get a throwaway email address to sign up with.)
  5. Now you need to set up this account so your script can make calls to Twitter’s API. Go to https://dev.twitter.com/ and sign in.
  6. Click on “Create an App”, or just go directly to https://dev.twitter.com/apps/new and enter in the details. I don’t think any of this info actually matters. You don’t need a callback URL. Click “Create your Twitter Application”.
  7. On the new app’s web page, and under “Application Type” click on the “Read, Write, and Access direct messages” radio button. This is so that the bot can make posts and send DMs. Click “Update this Twitter Application’s settings”.
  8. Go back to the Details tab, and at the bottom of the screen click “Create my access token”.
  9. When the page reloads, you will need to copy the following four pieces of information from this page: “Consumer Key”, “Consumer Secret”, “Access Token”, and “Access Token Secret”

You’re now ready to start coding your Twitter bot.

Logging In Your Bot

The reason you had to jump through all those hoops is because a Twitter application uses the OAuth system so that it can post tweets and DMs (Direct Messages) on behalf of other people’s accounts. This is a more than we need to do (our bot will only post from its own account), but OAuth is the only way programs can log in to Twitter. (This system is why you can give third parties’ software permission to post things from your twitter account without giving them your password, such as having Facebook tweet your Facebook status updates.)

If you are reading any other tutorials on the web or see twitter Python modules where you can log in with just the account’s username and password (or mention “Basic authentication”), these tutorials/modules are out of date. Twitter no longer lets you log in with just the username and password as of August of 2010.

Open a text editor or IDE to write Python code, or just test this code from the interactive shell (replacing the keys and secrets with the values you got for your app):

>>> import twitter
>>> api = twitter.Api(consumer_key='og82uZE9GOLMDsQxileh65', consumer_secret='me7hXfqDOv0KZuHICWnpdxTNykQBoUrS1zVgb4ARFt', access_token_key='iINPSMrwktD4ZLn01WhfoRb7a2OJ6F38zKlYBTQcm5UvjgHGAe', access_token_secret='5Opjh1qMaHAyz2Bw9i4bxPKNLE0RFeJuYsn8TdtUGWv')

This will now log you in to your bot’s account. Note that if you ever revoke the access tokens (which you are unlikely to do by accident), you will have to change the value in the access_token_key and access_token_secret parameters.

At this point, your code can make any of the function calls that the twitter module provides. Here’s a complete API reference: http://inventwithpython.com/twitter.html.

Here’s the source code to my @YHobosHaveDogs bot. It basically searches for the text '"homeless people" dogs' and then replies to them (if they posted the message in the last 24 hours, the bot hasn’t replied to them before, and if this isn’t a reply to someone the bot has replied to.) My initial testing shows that the bot’s reply to these search results is relevant most of the time.

Download Twitter bot's code: yhobos_script.py

It’s very important that you test your bot before putting it out in the world, especially if it messages strangers. You don’t want to accidentally spam people. The PostUpdate() method will give you an error if you try to post the exact same message as one you’ve posted before, but this won’t prevent all problems.

What I do to test is comment out the PostUpdate() calls and slip in a print statement instead, so I can see what the program would post without it actually posting anything. Twitter has rate-limiting for a max of 350 calls per hour.

Once you have your bot working, you'll want to have the bot's script regularly called. If you have a desktop that is always connected to the Internet, you can use Windows' Scheduled Tasks or on Mac you can use iCal, cron, or launchd to have the bot's script run every hour or so.

If you have a webhost like Dreamhost, you can use cron to schedule repeated calls to the bot script. But first you will have to set up the Python twitter module on Dreamhost. This requires a few extra steps because you cannot install modules to the system's Python binary. Instead we will use virtualenv to create our own Python to install the twitter module to.

Installing Modules to Python on Dreamhost

SSH into your Dreamhost account (I use PuTTY on Windows. Linux and Mac OS X have the command line "ssh" program that you can run from a terminal screen.)

run the following commands to install VirtualEnv: (you might want to read the Dreamhost wiki's Python page for details. I also found this page to be helpful.)

wget http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.4.3.tar.gz
tar xzvf virtualenv-1.4.3.tar.gz
cd virtualenv-1.4.3/
python virtualenv.py $HOME/local
mv ~/local/lib/python2.5 ~/local/lib/python

(The last command is to rename the python2.5 directory to python, which the twitter module's installer seems to demand.)

Then install the python-twitter module (which will also install the oauth module it needs):

wget http://python-twitter.googlecode.com/files/python-twitter-0.8.2.tar.gz
tar xzvf python-twitter-0.8.2.tar.gz
cd python-twitter-0.8.2/
/home/yourusername/local/python setup.py install --home=~/local

Setting Up Cron on Dreamhost for your Bot

You will have to set the bot's script file as executable, which you can do by entering this command:

chmod 774 mytwitterbot.py

And add this code to the very top of the bot script so that Dreamhost will use your virtualenv Python to run the script in /home/yourusername/local/bin instead of the system's Python in /usr/bin:

#!/home/yourusername/local/bin/python

Also, I found that I had to change the Python path at the beginning of the script since Dreamhost's cron daemon didn't have it set up. There's probably a more graceful way to handle this, but I simply added this code to the beginning of my twitter bot code:

sys.path = ['/home/yourusername/yhoboshavedogs', '/home/yourusername/local/lib/python/python_twitter-0.8.2-py2.5.egg', '/home/yourusername/local/lib/python/oauth2-1.5.211-py2.5.egg', '/home/yourusername/local/lib/python/httplib2-0.7.4-py2.5.egg', '/home/yourusername/local/lib/python/simplejson-1.9.2-py2.5-linux-x86_64.egg', '/home/yourusername/local/lib/python/python_twitter-0.8.2-py2.5.egg', '/home/yourusername/local/lib/python/oauth2-1.5.211-py2.5.egg', '/home/yourusername/local/lib/python/simplejson-1.9.2-py2.5-linux-x86_64.egg', '/home/yourusername/local/lib/python2.5', '/home/yourusername/local/lib/python', '/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2', '/usr/lib/python2.5/lib-tk', '/usr/lib/python2.5/lib-dynload', '/usr/local/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages/Numeric', '/usr/lib/python2.5/site-packages/PIL', '/var/lib/python-support/python2.5', '/usr/lib/site-python', '/home/yourusername/local/lib/python2.5']

From the Dreamhost Panel's cron jobs section, click on "Add a new cron job". In the form that appears, add "/home/yourusername/mytwitterbot.py" as the command to run. You can also add your email address so that you are emailed any errors or output that appear from the cron job.

I have my bot set up to not print out anything if it doesn't update anything. This way my email address doesn't get flooded with emails when the bot doesn't do anything.

At this point, the bot should be set up. Leave a comment on this blog post if you have questions about anything.

API Reference Summary

The API Reference Documentation has some good pointers for the functions in the module. But here's my own remarks and summary of it. “Status” is the term that the module uses to mean a Twitter post or update. To post a twitter status message (after being authenticated):

>>> status = api.PostUpdate('I love python-twitter!')
>>> print status.text
I love python-twitter!

To fetch your friends (after being authenticated):

>>> users = api.GetFriends()
>>> print [u.name for u in users]

To post a direct message to a user:

>>> directMessage = api.PostDirectMessage([email protected]', 'This is my message.')

For the recipient, you can also use that user’s id.

Comments