How to Code a Twitter Bot in Python on Dreamhost

I made a twitter bot that checks every hour for someone who has asked the question, “Why do homeless people have dogs?” and automatically replies, “Because a dog will love you even though you are homeless.” It’s running right now at @YHobosHaveDogs.

Figuring out how to code this took a couple evenings and a little hair pulling, so I decided to document the process in this blog article to make it easier for the next programmer. This will be making a Twitter bot in Python using the python-twitter module (which runs on Python 2, not Python 3), and then running the bot from my Dreamhost server (but most likely any web host will work just fine. Or if you have a computer that is always online, you can run the bot from that). First we will run the bot from our machine to test it out, and then load it onto the Dreamhost web host. (I’m running a Windows box, but the steps should work on any OS.)

  1. Download the python-twitter module (I tried some of the other modules but didn’t like them as much.)
  2. Unzip this file and from the command line in the unzipped directory, run “python setup.py install” to install the twitter module.
  3. Run Python and then type, “import twitter”. If you don’t get any error messages, the module has installed correctly.
  4. Go to http://twitter.com and create a new Twitter account for this bot. (You might need to get a throwaway email address to sign up with.)
  5. Now you need to set up this account so your script can make calls to Twitter’s API. Go to https://dev.twitter.com/ and sign in.
  6. Click on “Create an App”, or just go directly to https://dev.twitter.com/apps/new and enter in the details. I don’t think any of this info actually matters. You don’t need a callback URL. Click “Create your Twitter Application”.
  7. On the new app’s web page, and under “Application Type” click on the “Read, Write, and Access direct messages” radio button. This is so that the bot can make posts and send DMs. Click “Update this Twitter Application’s settings”.
  8. Go back to the Details tab, and at the bottom of the screen click “Create my access token”.
  9. When the page reloads, you will need to copy the following four pieces of information from this page: “Consumer Key”, “Consumer Secret”, “Access Token”, and “Access Token Secret”

You’re now ready to start coding your Twitter bot.

Logging In Your Bot

The reason you had to jump through all those hoops is because a Twitter application uses the OAuth system so that it can post tweets and DMs (Direct Messages) on behalf of other people’s accounts. This is a more than we need to do (our bot will only post from its own account), but OAuth is the only way programs can log in to Twitter. (This system is why you can give third parties’ software permission to post things from your twitter account without giving them your password, such as having Facebook tweet your Facebook status updates.)

If you are reading any other tutorials on the web or see twitter Python modules where you can log in with just the account’s username and password (or mention “Basic authentication”), these tutorials/modules are out of date. Twitter no longer lets you log in with just the username and password as of August of 2010.

Open a text editor or IDE to write Python code, or just test this code from the interactive shell (replacing the keys and secrets with the values you got for your app):

>>> import twitter
>>> api = twitter.Api(consumer_key='og82uZE9GOLMDsQxileh65', consumer_secret='me7hXfqDOv0KZuHICWnpdxTNykQBoUrS1zVgb4ARFt', access_token_key='iINPSMrwktD4ZLn01WhfoRb7a2OJ6F38zKlYBTQcm5UvjgHGAe', access_token_secret='5Opjh1qMaHAyz2Bw9i4bxPKNLE0RFeJuYsn8TdtUGWv')

This will now log you in to your bot’s account. Note that if you ever revoke the access tokens (which you are unlikely to do by accident), you will have to change the value in the access_token_key and access_token_secret parameters.

At this point, your code can make any of the function calls that the twitter module provides. Here’s a complete API reference: http://inventwithpython.com/twitter.html.

Here’s the source code to my @YHobosHaveDogs bot. It basically searches for the text '"homeless people" dogs' and then replies to them (if they posted the message in the last 24 hours, the bot hasn’t replied to them before, and if this isn’t a reply to someone the bot has replied to.) My initial testing shows that the bot’s reply to these search results is relevant most of the time.

Download Twitter bot's code: yhobos_script.py

It’s very important that you test your bot before putting it out in the world, especially if it messages strangers. You don’t want to accidentally spam people. The PostUpdate() method will give you an error if you try to post the exact same message as one you’ve posted before, but this won’t prevent all problems.

What I do to test is comment out the PostUpdate() calls and slip in a print statement instead, so I can see what the program would post without it actually posting anything. Twitter has rate-limiting for a max of 350 calls per hour.

Once you have your bot working, you'll want to have the bot's script regularly called. If you have a desktop that is always connected to the Internet, you can use Windows' Scheduled Tasks or on Mac you can use iCal, cron, or launchd to have the bot's script run every hour or so.

If you have a webhost like Dreamhost, you can use cron to schedule repeated calls to the bot script. But first you will have to set up the Python twitter module on Dreamhost. This requires a few extra steps because you cannot install modules to the system's Python binary. Instead we will use virtualenv to create our own Python to install the twitter module to.

Installing Modules to Python on Dreamhost

SSH into your Dreamhost account (I use PuTTY on Windows. Linux and Mac OS X have the command line "ssh" program that you can run from a terminal screen.)

run the following commands to install VirtualEnv: (you might want to read the Dreamhost wiki's Python page for details. I also found this page to be helpful.)

wget http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.4.3.tar.gz
tar xzvf virtualenv-1.4.3.tar.gz
cd virtualenv-1.4.3/
python virtualenv.py $HOME/local
mv ~/local/lib/python2.5 ~/local/lib/python

(The last command is to rename the python2.5 directory to python, which the twitter module's installer seems to demand.)

Then install the python-twitter module (which will also install the oauth module it needs):

wget http://python-twitter.googlecode.com/files/python-twitter-0.8.2.tar.gz
tar xzvf python-twitter-0.8.2.tar.gz
cd python-twitter-0.8.2/
/home/yourusername/local/python setup.py install --home=~/local

Setting Up Cron on Dreamhost for your Bot

You will have to set the bot's script file as executable, which you can do by entering this command:

chmod 774 mytwitterbot.py

And add this code to the very top of the bot script so that Dreamhost will use your virtualenv Python to run the script in /home/yourusername/local/bin instead of the system's Python in /usr/bin:

#!/home/yourusername/local/bin/python

Also, I found that I had to change the Python path at the beginning of the script since Dreamhost's cron daemon didn't have it set up. There's probably a more graceful way to handle this, but I simply added this code to the beginning of my twitter bot code:

sys.path = ['/home/yourusername/yhoboshavedogs', '/home/yourusername/local/lib/python/python_twitter-0.8.2-py2.5.egg', '/home/yourusername/local/lib/python/oauth2-1.5.211-py2.5.egg', '/home/yourusername/local/lib/python/httplib2-0.7.4-py2.5.egg', '/home/yourusername/local/lib/python/simplejson-1.9.2-py2.5-linux-x86_64.egg', '/home/yourusername/local/lib/python/python_twitter-0.8.2-py2.5.egg', '/home/yourusername/local/lib/python/oauth2-1.5.211-py2.5.egg', '/home/yourusername/local/lib/python/simplejson-1.9.2-py2.5-linux-x86_64.egg', '/home/yourusername/local/lib/python2.5', '/home/yourusername/local/lib/python', '/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2', '/usr/lib/python2.5/lib-tk', '/usr/lib/python2.5/lib-dynload', '/usr/local/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages/Numeric', '/usr/lib/python2.5/site-packages/PIL', '/var/lib/python-support/python2.5', '/usr/lib/site-python', '/home/yourusername/local/lib/python2.5']

From the Dreamhost Panel's cron jobs section, click on "Add a new cron job". In the form that appears, add "/home/yourusername/mytwitterbot.py" as the command to run. You can also add your email address so that you are emailed any errors or output that appear from the cron job.

I have my bot set up to not print out anything if it doesn't update anything. This way my email address doesn't get flooded with emails when the bot doesn't do anything.

At this point, the bot should be set up. Leave a comment on this blog post if you have questions about anything.

API Reference Summary

The API Reference Documentation has some good pointers for the functions in the module. But here's my own remarks and summary of it. “Status” is the term that the module uses to mean a Twitter post or update. To post a twitter status message (after being authenticated):

>>> status = api.PostUpdate('I love python-twitter!')
>>> print status.text
I love python-twitter!

To fetch your friends (after being authenticated):

>>> users = api.GetFriends()
>>> print [u.name for u in users]

To post a direct message to a user:

>>> directMessage = api.PostDirectMessage('@myfriend', 'This is my message.')

For the recipient, you can also use that user’s id.

27 comments.

  1. Does this work for running on a random PC rather than on website with a static hostname? I tried to see if there was a way to use Twitter’s newer OAuth system to send tweets via a simple commandline app but it seemed like the new system is only designed for websites rather than clients.

  2. @Chris – It seems to be working on my remote Ubuntu box. I haven’t actually started coding the bot yet, but playing around with API defintely results in manipulations of the Twitter account. This is a good sign. Just make sure that you have all the most up-to-date modules installed and you should be golden. When I had the previous version of python-twitter-module installed, there were errors–probably because of the old OAuth system.

    Thank you Al for the enlightening tutorial. This is exactly the quick and dirty way of reintroducing myself to Python that I was looking for. I’m glad I caught it while it was still fresh.

  3. I was unable to unzip the python-twitter file. Is it still working accurately? Thanks

  4. I’ve fixed the link. Thanks for pointing it out.

  5. Hey, thanks for the article it inspired me to try an idea I had long ago for a twitterbot and the first steps are already done. Thanks!

    P.S.: in case anyone is interested I uploaded it to github: https://github.com/Willyfrog/tuitorrent

  6. Hi Al, just tried following your example here, but ran into ImportError: No module named oauth2. How do I fix this error? Your tutorial looks cool so I would like to try it out.

    Thanks
    Helen

  7. That’s an interesting post. I read through some of the tweets that your bot responded to and it’s sad how many people seem like they couldn’t care less for the homeless people. You might not have intended it, but it’s an interesting social experiment.

  8. I am having the same problem as Helen above.
    ImportError: No module named oauth2

  9. To those having issues with the oauth2 (and will have issues with the httplib module as well):

    Google oauth2 and httplib seperately. You’ll want to put those unzipped folders in your python\lib\ directory. In the case of httplib you’ll have to drill down a few folders and grab the “httplib” folder from the Python2 folder. Once you get those two modules in place everything in this post works flawlessly.

    Thanks for this by the way!

  10. Hi Al

    Excellent tutorial. I have adapted it for my twitter bot and got it working. However, after a few posts it stopped with the error: IndexError: list index out of range

    I fould this error came up when a post in the log yhoboshavedogs_log.txt had a carrage return in it, when I removed the CR it worked again.

    Can you add some code to cope with this?

    Thanks

    Keith

  11. opps (!) missed a bit, error was after:
    alreadyMessaged[i] = alreadyMessaged[i].split(‘|’)[1]

  12. I used the script but it suspended my account a few times. What was I doing wrong to get my account suspended. I turn the tweeting to be every 30 seconds and it still suspended it

    Thanks

  13. @FacelessLoser: Minimum rate limit to avoid Twitterjail is ~2min/Tweet

  14. Hi, can you SSH if you have shared hosting with dreamhost?

    REPLY: Yes, you can SSH into your box with Dreamhost’s shared hosting plan. -Al

  15. Thanks very much, I based my python script on your example which was really well written for someone that had never used the Twitter API before. Much appreciated.

  16. i want to learn how to make bot?
    can someone suggest me any book or website?

  17. Thanks very much for this. It was exactly what I needed.

  18. I notice your bot hasn’t tweeted since Nov. 27. Did something change or did you pull the plug? Just asking because I was about to tackle this. Looks like a great tutorial.

  19. Twitter suspended the account, since it comments at people which makes it fit a spam bot profile. I got the suspension lifted, but I’m holding off on having it update anymore.

    -Al

  20. Thanks Al. That’s a shame. It seemed harmless enough and was an interesting experiment. I wonder what triggered the suspension, i.e. a bot of their own, or a complaint from a person. But good to know. Is the tutorial and code as is, still valid?

  21. Yes, the code in this tutorial still works.

  22. , TwitterError(u’Could not authenticate with OAuth.

    this is working fine, taking all the twitt as per my requirements, but the above error is coming, I guess for what it’s not posting anything. please help

  23. I just wanted to point out the the same twitter name could be for y hobo shave dogs. Just sayin’. Thanks for the tutorial!

  24. [...] found a very helpful step by step guide that served as a great starting point, but I couldn’t use my website to host the script. So [...]

  25. Hi, I just downloaded and tried using the module – but it won’t post to Twitter. Looks like the authentication was recently changed. Is there a chance of seeing if the code could be updated as well?

    Regards,
    Helen

  26. I think this script no longer works since twitter updates its API to version 1.1 (CMIIW).

  27. Hi I am having this error
    C:\Python27\yhoboshavedogs>python yhobos_script.py
    Traceback (most recent call last):
    File “yhobos_script.py”, line 47, in
    File “C:\Python27\Lib\site-packages\twitter.py”, line 2401, in GetSearch
    for x in data ['results']:
    KeyError: ‘results’

    Please help

Post a comment.