serverless twitter bot

Create a Serverless Twitter Bot with Airbrake and AWS Lambda – Part 1

Over the next week we’ll be learning how to create a fully-automated, NodeJS, serverless Twitter bot with real-time error monitoring capabilities via Airbrake. Our Twitter bot will be capable of performing just about any task you can imagine, and it will do so serverlessly, by making use of the power of AWS Lambda functions. Let’s get right into it!

Creating a Project

The first thing to do is decide where you want to create your project. I’ll be keeping things simple by naming my project twitter-bot. As is commonly the case, we also want to use source control management, so I’ll be using Git locally and uploading to GitHub as a remote repo:

We’ll be using the twitter NPM package to simplify communicating with the Twitter API, so let’s install that (along with its own dependents):

Working with Secret API Credentials

As with most Git repositories, there are some files or filetypes we don’t want to expose to the public, so we’ll add a .gitignore file to the twitter-bot project directory. Note: I am using VS Code and WebStorm to develop this project, so the code command can be replaced with whatever command you use to open your favorite text editor.

We’ll need to get some authentication API keys from Twitter, which provide authorization to perform automated tweets and other functionality, and some of this information should be kept private. We’ll be storing these Twitter API keys in the twitter-api-credentials.js file, so let’s add that explicit file to the .gitignore immediately, before we accidentally add some secret keys and commit them to the Git repository:

Now, let’s double-check that we’ve setup everything correctly and that our secrets won’t be unintentionally committed. We’ll start by creating the twitter-api-credentials.js file and adding some irrelevant text to it. Here we’re using the >> Unix command to append the text 12345 to the twitter-api-credentials.js file, then outputting the contents of the file using the cat command:

Our credentials file is created, so now we should double-check that .gitignore is working correctly and won’t add twitter-api-credentials.js to the repository. A simple git status command should do the trick and show what untracked files are waiting to be added:

That looks great, since everything except the ignored twitter-api-credentials.js is listed there. Let’s get into the smart habit of committing changes to Git by making our initial commit right now:

We’ll also be storing this project on a remote, public GitHub repository, so we need to set the origin and use the git push command to ensure our changes are uploaded:

Now we need to actually obtain some Twitter API credentials, so we start by logging into our Twitter account and then visiting https://apps.twitter.com/. Click Create New App then fill out the form as you see fit. For this example we’ll use the following:

  • Name: AirbrakeArticles
  • Description: Tweeting random Airbrake.io articles!
  • Website: https://github.com/GabeStah/twitter-bot

Agree to the terms of use, then create your app!

Next, we need to create and copy the API keys into the twitter-api-credentials.js file for use in our project. Click on the Keys and Access Tokens tab at the top, then click Create my access token at the bottom. Per the Twitter NPM documentation we need the consumer_key, consumer_secret, access_token_key, and access_token_secret values from Twitter. Open the twitter-api-credentials.js and paste the following template into it:

Now, copy and paste each value from the Twitter App page into the appropriate field and save the file, which now has all the secret keys necessary to connect to and use Twitter programmatically.

At this point we can actually begin creating our application logic by first testing that our Twitter API connection works. Start by creating the base application file for your Node project. The default is usually index.js:

We’ll start by requiring the twitter-api-credentials.js file that exports our credentials, along with the twitter NPM module. We’ll then instantiate a new Twitter object and pass the credentials to it, which will perform authentication and authorization for us while working with the Twitter API:

Everything is ready to go, but how do we actually use the twitter module? It primarily provides convenience methods for sending GET and POST HTTP method requests to the Twitter API. For example, if we want to post a tweet we need to send a POST request to the statuses/update API endpoint, as shown in the official documentation. The API accepts a number of required (and optional) parameters to be included with the request.

To illustrate, let’s perform a simple test tweet of "Am I a robot?" from our bot account. Add the following to index.js:

Now, to test that everything works just run your Node application. If all was setup correctly you’ll see the output in the console log to confirm it works. You can also refresh your Twitter account page to see the new "Am I a robot?" tweet:

Customizing Our Bot

Alright, we’ve got a working bot, but right now it only does exactly what we tell it. Let’s improve things by customizing it a bit and getting it to do something more interesting. For this example, we’ll use our AirbrakeArticle Twitter bot account to tweet out random Airbrake.io articles that have been published in the past. There are many ways to accomplish this, but we’ll start with the easiest technique. Since https://airbrake.io/blog uses WordPress, we can access the RSS or Atom feed by appending /feed or /feed/atom to the base URL, respectively. For example, opening https://airbrake.io/blog/feed/atom provides the Atom feed of the most recent articles. Unfortunately, this doesn’t give us programmatic access to historical data, since RSS feeds are limited to only the most recent information, but it’s a good starting point.

There’s little reason to reinvent the wheel, so we’ll be using the feedparser NPM module to simplify the process of retrieving the latest Airbrake article feed. The use of the --save flag within the npm install command forces the package being installed to be automatically added to the package.json dependencies field, so we don’t have to manually add it ourselves:

We then need to require('feedparser') at the top of index.js, along with the request built-in module for handling our request to the feed URL (https://airbrake.io/blog/feed/atom). The top of index.js should look something like this now:

We need to respond to a few different events to process the incoming feed request, since it happens asynchronously:

We start with feed.on('response'), which fires when the request(...) made to the feed URL receives a response. If we get a 200 error code back, we know it was successful so we pipe the response to the feedparser object. From there, feedparser.on('readable') fires when there’s a readable stream available, which means we have some feed data to parse. Within this function we’re outputting each article to the log and then adding them to the articles collection array. Finally, feedparser.on('end') is invoked when feedparser completes the task of reading the stream, so we want to actually produce a tweet of a random Article when that occurs.

We’ve also added the tweetArticle(article) and tweetRandomArticle() helper functions as simple wrappers for using the twitter module to POST to the statuses/update API endpoint:

Alright, everything is setup, so let’s try running our application again and see what happens:

Awesome! Everything works as intended. We parsed the Atom feed from WordPress to capture the latest articles, then selected a random article and tweeted the title and the URL on our Twitter Bot account, AirbrakeArticle.

We now have the basic structure of our application up and running, so next week we’ll refine it and make it better suited to the real-world by implementing real-time error monitoring via Airbrake’s NodeJS package. We’ll also ensure our Twitter bot application can run automatically and serverlessly by using AWS Lambda. Stay tuned!