EXCLUSIVE: What the Hell Happened to BadBlue News?

Update 8/4: BadBlue News has been restored.

Caution: this is a really techno-geeky article describing what in the Sam Hill happened to BadBlue and the progress made thus far in restoring it

I've gotten a fair number of emails over the last few days expressing concern that the news aggregator BadBlue.com is not updating news stories.

Now: a little bit of background on the site. It is a full automated news-gathering site that leverages social media (e.g., Twitter) to determine which stories are trending. Its database consists of over 750 news sites that are scanned pretty much continuously for "hot" stories.

I created BadBlue in 2011, initially working with the great John Hawkins (then of Right Wing News and now of Brass Pills), but eventually going out on my own. I used Amazon Web Services (AWS), the powerful cloud hosting platform, to spin up several Linux machines to host the content. If memory serves, the original launch of BadBlue was in January of 2012, more than seven years ago.

Twitter Makes a Change

When I noticed, with the help of several readers, that BadBlue wasn't updating I pulled some log files and determined my programmatic connection to Twitter (also called an API) was failing to connect. Mystified, I checked the @BadBlueNews Twitter account; it was fine, not suspended or impeded in any way. So that was a relief.

In researching the Twitter API, it hit me: Twitter had a new requirement, which they turned on in late July. This requirement was to use a new encryption standard called TLS 1.2, versus the original version called TLS 1.0.

So, my challenge was to upgrade the encryption library, called OpenSSL, to a more recent version. Not a problem, right?

First attempt: figure out a way just to update OpenSSL to a more modern version. Bam! No dice, there didn't appear to be any way to upgrade OpenSSL in place since it was such an old version.

Second attempt: upgrade the entire AMP stack (Apache-MySQL-PHP) on the Linux box, which would include OpenSSL. Bammity Bam! Sorry, Luther, the version of Linux was so old, it was incompatible with an upgrade of the AMP stack.

Third attempt: okay, I'll just upgrade the entire LAMP stack (Linux operating system plus AMP)! I use a service called Bitnami (quite good, actually) to manage my Linux servers. The one drawback with Bitnami is that they tightly control the images they distribute. There are no upgrades permitted in the Bitnami ecosystem. I can kind of understand that, but it's very painful in situations like these when I need just one library updated. So, Bammity Bam Again!

Ding-Ding: It's Round Two!

Alright, f-it, I thought. Time to just provision a brand new server. Since I use Bitnami to provision my cloud instances, I went straight there and launched a new LAMP machine. Bammity-Freaking-Bam! The error message simply stated that my AWS machines were so old (2011), Bitnami couldn't launch a new instance.

Their advice on fixing this? Create a brand new AWS account, so I would get the latest VPC (Virtual Private Cloud) features, required to provision new machines.

Okay, done. Now, Bitnami, let's build a new LAMP box from scratch!

INTERMISSION WHILE AWS BUILDS THE NEW MACHINE (I WENT TO WORK OUT)

I went back to the Bitnami control panel and punched out a new LAMP box. Success! Now it was time to begin porting over the BadBlue database. Bamapalooza! The scripts I had used to create tables and indexes in MySQL had syntax errors and wouldn't run? What in the hell?

Agh!

After reviewing the DDL, the field name RANK was no longer legal (I guess it's become a keyword, or perhaps it's not gender-friendly). All of the references to RANK in the database description language and the code itself had to be changed to RANKING.

Best of luck, dumbass

Of course, I've gone on far too long screeching this digital diatribe.

Suffice it to say that I've overcome a few challenges and have yet to solve the latest: PHP no longer supports the MySQL programming library, and has moved instead to its own version called MySQLi (improved). My next task is to convert over and validate all of the code that references mysql_ to mysqli_. Good times, good times.

If you've read this far, congratulations. I suspect some are using this tale as a drug-free cure for insomnia.

* * * * * * * * *

By Crom's word, I hope to complete the upgrade process in the next several days, so please bear with me.