Robots Are Doing Better Than Humans on Twitter

This is all about an experiment I started back in July 2009, called TwiBots. Initially it was supposed to be a simple 24/7 online tweep (Michael Davis) saying a bunch of stuff randomly. But then we (Michael and I hehe) started picking out certain topics, feeding content from certain RSS feeds, filtering all content by keywords and regular expressions.

Davis Got 4500 Followers in 6 Months

When the new Retweet API came along, I wrote an RT module which uses the Twitter Search API to find relevant tweets and users, then retweets those messages or just somehow interacts with a user. As soon as the Twitter Lists API was announced, I started working on the Lists module, which eventually became a simple “sorting-the-tweeps” based on their keywords – web design, design, wordpress, etc. Soon enough, I found out that Lists couldn’t contain more than 500 members (although some glitch made 501 possible). Web design 2, wordpress 2, etc wouldn’t be as fancy. I also tried building a conversation list of tweeps that by any means talked to the robot, but then again, the 500 limit broke all my hopes ;)

There were other modules which I worked on really hard, such as RThx module or Random Buzz, DM Control. Some of them worked, some of them were turned off after a few days (yes, you guessed it – Random Buzz, that really made some noize ;)

So, what did I achieve? Me – nothing. Michael did though, in 6 months he went up to 4500+ followers, while following a little more than 200 people himself, has been featured in ~ 250 lists, sent ~ 55,000 tweets and retweets all based on four keywords (or hashtags) – design, web design, wordpress and jquery. Built 4 lists based on these keywords, 500 members in each. Total list followers is a little less than 150 (which is quite good actually).

For comparison take a look at my account – (@kovshenin), in a little bit more than a year I got ~ 1700 followers. It took Michael a couple of months to reach that. The chart below illustrates the followers growth during the last three months. Human (me – blue) vs Robot (Michael – red). Yeah, I added a new module in mid December ;)

I manually logged into Michael’s account recently to check out how he’s doing, and I was kind of surprised to see that people really are talking to the guy, thanking him for retweets, asking him for further reads, wishing him a great day and handing over some coffee. Michael doesn’t usually reply to these and he’s a little bit shy sometimes, besides, he never drinks coffee ;)

A few days ago I decided to give Michael a rest, so tuned his backend to a new Twitter account with a few different settings, especially in timing. I’ll be switching to other keywords and feeds in the next few weeks. So let’s see if he’s as good as Michael, or perhaps better? ;)

I’m not sure what I’ll do with Michael’s account. TweetValue said it’s worth over $5k … anyone? ;) Or should I just throw it away.. Or run a contest for his password? …

Twitter Robot in PHP: Twibots Draft

As I promised quite some time ago, I’m putting out a draft of the Twitter Robot I wrote. Make sure you read Create Your Own Automated Twitter Robot in PHP before going on. The current functionality is as follows:

• Tweets around the clock
• Tweets from RSS feeds, supporting prefix and postfix text (for adding hashtags)
• Retweet via the Twitter Search API and build conversation lists
• Shoot random sentences at users who mention you, thank them for retweets
• Control your robot via your own Twitter account by sending him direct messages
• All this is Twitter OAuth powered, no password required
• Such robots are called Twibots

Create Your Own Twitter Robot in PHP

Now, before downloading the code, I have to warn you that it’s completely unorganized. The code is horrible, comments are awkward, the database being used is SQLite (just for the fun of it) and it’s very very glitchy. Be prepared for Twitter suspending your account for ’strange activity’ and use this at your own risk, don’t run here blaming me for that ;) I also suggest you’d contact Twitter to get your IP addresses and Twitter account white-listed before you start, especially if you plan to tweet very often (which I wouldn’t recommend). Use this at your own risk, and please keep my copyrights and preferably the OAuth application IDs.

Download: here (version 0.1)

Operation Instructions.. To say the truth it’s pretty tough, no web interface, not buttons, no config files. There are a bunch of files there, some of them useless. There’s the Snoopy class for reading and parsing RSS, there’s the Twitter OAuth class, and two core php files – cron.php and oauth.php. Open up cron.php, there are some comments and examples there. Make sure you get your own bit.ly API key and secret. Also make sure you get a connection with the twibots.sqlite database which has a couple of empty tables. Those will be used for tokens and dump data for unrepeated tweets.

Once you’re done configuring, use the command-line php in order to make it work. It goes something like this:

# php cron.php oauth register
# Please browse to https://twitter.com/…
# php cron.php oauth validate 123465
# Authentication successful, greetings @ev ;)

# php cron.php random
# tweeting a random RSS feed …

# php cron.php reply
# sending replies…

# php cron.php dm
# reading direct messages

# php cron.php retweet
# retweeting…

You’ll have to put that in your crontab file and launch by schedule. Don’t run them too often though, as Twitter doesn’t like flooding, especially from newly created accounts. Any questions or suggestions are welcome in the comments below, but please, don’t tell me the code is horrible, I know it is, and I wouldn’t have posted it if you didn’t ask ;) Cheers!

The Twitter API v2 Transition

It’s a mess around the current working copy of the Twitter API, there are more issues than functionality and the whole naming and renaming is a total disaster. Today for instance I tried a simple search query to the API and kept receiving “400 Bad Request” errors without any further explenation. As soon as I changed the address from search.twitter.com to api.twitter.com/1 (I got this from Abraham Williams’ php code), which is not clearly mentioned anywhere in the Twitter docs, everything started working fine.

Twitter API: Solving Compatibility Issues

But then I realised that the from_user_id field that’s being returned is far from the correct user ID. People in the Twitter Development Talk Google group stated this problem a few times (since March 2009 I believe). It seems that the “wrong” user IDs are meant for the second version of the Twitter API, thus cannot be used before it’s released. But wait! What the hack should I do with my app now? It’s not working y’know! Here you go:

1
2
3
4
5
6
7
8
9
10

$response = $oauth->get('search', array('q' => $search_query));
foreach($response->results as $result)
{
	$id = $result->id;
	$text = $result->text; 
	$user_name = $result->from_user;
 
	$user = $oauth->get("users/show", array("screen_name" => $user_name));
	$user_id = $user->id; // Get the old-style user ID
}

Yeah, that’s one extra API call, but it solves things temporarily ;)

I guess there’s nothing that we could really do right now, and it’s probably true that we’ll have to rewrite some parts of our code as soon as Twitter API v2 is released, but then again, what about the apps which stopped development? Will they stop working? Tonnes of Twitter clients and web apps still use basic authentication. Twitter mentioned that everybody must use OAuth these days, and that basic auth will be closed sooner or later.

Oh well, software comes, software goes. The best thing to do right now would be sign up to Twitter API Announce Google group and follow @twitterapi. By the way, @web2feed can now use the new features of the API to retweet messages based on hashtags and build user lists ;)

The Twitter OAuth PHP Class Gets Even Better

Or perhaps simpler?.. Together with the Twitter API itself, the TwitterOAuth PHP class (the one by Abraham Williams) is being updated too! According to GitHub the latest changeset was commited on December 3rd so yeah, I tried to take a look at what’s going on there a few days ago and was quite disapointed. Disappointed with the fact that all my previous code was broken without giving any reason.

Just like everybody else, I never read the readme or other documentation files so I dug straight into the class code and examples. Soon after I realized that the new changes were not that bad, so instead of the usual 5 lines of code, I shortened it up to only one. I stopped worrying about parsing XML or JSON, converting them to objects, and I stopped typing in the full address for Twitter API calls. Abraham did all that for us, so all we have left is:

1
2
3

$credentials = $oauth->get("account/verify_credentials");
if ($oauth->http_code == 200)
	echo "Hey there, {$credentials->screen_name}!";

I’m not going to publish all the new features and stuff (read about them at GitHub), but hey, this is quite sweet isn’t it? The only drawback was having to rewrite some parts of the code I wrote for the past few months (the Twitter Robots stuff), but I guess that’s partly my bad as it’s not as organized as it should be. That’s the main reason why I’m not publishing the whole code here yet, have a lot of cleaning up to do ;)

Meanwhile you may take a look at this buddy: @web2feed. I turned off the auto-replies because they were getting quite annoying, and I’ve added a couple of feeds to the big list, oh and it’s DM controlled too!

Facebook API Experiments: Twitter Tags

For quite some time now I’ve been dreaming about getting into the Facebook Platform (API, Markup Language, etc) but haven’t had good enough reasons to do so. One good friend of mine Timothy gave me those reasons not so long ago and guess what! I came up with my very first Facebook App! It’s still in the sandbox, don’t rush searching for it in the applications database. I’m not going to share any code as it’s all clumsy but I do want to share some screenshots and my first experience with the Facebook API.

My Very First Facebook App

One very interesting thing about the API is the testing console, where you could run any Facebook API methods, very useful for debugging. Interface design is very catchy too, with the new Facebook Markup Language and the FBML testing console, so quick and robust! Pepole are still asking for form elements extensions though, such as the checkbox or option – it seems that they come out buggy as fb:editor-custom.

Anyways, the first thing that came into my mind is adding a Twitter Tags tab to a profile or fan page, so I used the Foller.me API and within a few hours I managed to get a fully working (or not) canvas application tab. I’m surprised by how well-written the Facebook PHP Client Library is. All the methods are explained in the code comments and do exactly what they’re expected to. The Facebook Markup Language takes care of application settings, private areas (app configuration, etc) and everything else. The overall Facebook Developers Documentation is okay, I’ll give that an 8 out of 10 and I like them running the MediaWiki software ;) Here are some screenshots of my application:

 

Still not sure where this is going but Timothy had some great ideas about custom Facebook Apps. I guess a release of the Twitter Tags application would be handy for some profiles (or at least for some more experience), so I might push that forward in the near future and file an “add to facebook database” request to go public. After that we’ll aim for something bigger.