Tag Archives: foller.me

 

Foller.me: MySQL Tweaking & Optimization

As I mentioned in the interview with @enked on his website Chidimar.com, I had serious problems with MySQL database optimization on the Foller.me project. The current public stable version (beta-1) is using the MyISAM engine and it’s not holding much data – profiles, locations, geo points for the followers geography.

In the new version (currently dev-1 and hopefully beta-2 in a few days) I changed most of the old tables and added new ones, and using InnoDB this time. You see, it’s not very easy to scan through ~1,000,000 relations for the @mashable account ;) and I bumped into a ~10 second delay before the @mashable profile showed up at Foller.me. Slow query showed that one of the simplest queries caused that slow-mo – it took 6 seconds to execute! The guys at Stack Overflow helped me optimize the query and the two tables I was having problems with, so I came down to ~2 seconds for that query, neat!

Digging further I managed to tune the MySQL server up a little bit (caching, all sorts of buffers, etc – you should definitely take a look at MySQLTuner, it’s a perl script that helps you tune pretty much all the MySQL config) which decreased the query execution time to 1 s. The peeps at Stack Overflow said it’s pretty okay for that kind of query to execute for 3 seconds on over 2 million rows, so I thought that 1 second is final. Phew! :)

Now, think about the MySQL query cache. It doesn’t work in my situation, simply because I shoot UPDATEs and INSERTs at the relations table every five minutes or so (via a cron job), so there actually is a way to perform even higher. Thought of temporary tables, views and triggers (and even stored procedures). Nah.. Simply caching that query would be good, right? I mean if I cache the whole profile for an hour, why wouldn’t I cache the relations result set? Cache the query.. Aha, but I thought slightly further. Why not cache the whole page with memcached? I’ll keep you updated with the results.

Published 3 years ago with no comments tagged , , , , , , ,


 

Have You Tried the Amazon Web Services?

Amazon EC2, EBS, S3.. I’ve been looking for the perfect web hosting for over two years now. Is this it?

A few months ago I really liked MediaTemple cause they offered pretty good US hosting starting from $20/mo, which was quite good for the Foller.me project, so at the starting point I chose them. Their service is cool, definitely worth the money, but. A few weeks have passed, along with some major development on the service update and I got stuck with MySQL and overall server performance. It’s pretty tough to scan through 2,000,000 relations from @cnnbrk and then geocode their locations so I thought that I need to fine-tune MySQL and work out a more powerful caching system.

Yes, MediaTemple do offer dedicated MySQL grids for $50/mo, so that’s $70/mo overall. Not that bad, but thinking ahead, I’d also like to tweak up my http server, so that’d be a virtual dedicated plan for $50/mo, which makes $100/mo in total. Woah! And that’s just the start (around 500 megs RAM, 20 GB disk space and 1 TB bandwidth).

Now the Amazon Web Services offers a 2 GB RAM, 1.6 GHz virtual machine for only $0.10/hr, that makes ~$70/mo. Put up an Elastic Block Store (EBS) up to 1 TB and attach it to the instance around $20/mo. and perhaps an Amazon S3 bucket $10/mo. That makes about $100/mo in total. It’s not just the price though, I loved the way you’re in total control of whatever is happening on your server. You tune it however you like, whenever you like. Save bundled volumes and start over at any time. One-click EBS volume backups, elastic IP address and up to 20 instanced running simultaneously (you can increase this number by contacting Amazon). You also get to pick whatever OS you’d like to run (they’re called AMIs). You can build your own bundled OSs and make them available public.

Oh, and one of the best things about Amazon EC2 (Elastic Cloud) is that it’s so flexible! Switching servers has never been so easy. Start a new instance, attach an EBS, tune it up. Associate your old Elastic IP address to the new instance and voila! Go ahead and terminate your old instance, cause you’re riding your new mustang now!

I’m also sure that you can setup multiple servers and network balancers.. Like clustered computing y’know, the possibilities are endless! But I’m too far away from that at the moment, though I’m sure that whenever I have some free time, I will throw some experiments in that field ;) I’ve already setup Trac and SVN server a few days ago, works great!

Virtual Private Servers, Dedicated Servers, blah blah blah. Those are from the past. It’s Amazon Web Services. Go get your account right now ;)

Published 3 years ago with 4 comments tagged , , , , , ,


 

Foller.me Gets Featured on Startups Live TV!

StartupsLive.tv is definitely one of the best shows I’ve seen around these days and guess what! I got there!

Startups Live is the non-biased platform for any startup to get live exposure, a captive audience, and valuable feedback from potential business partners, customers, members, and investors.

I sent a request to Startups Live asking them to feature the Foller.me Twitter service I’ve created and bang! I got on the show! It was really amazing, I’ve never seen anything like it!

The girls are so wonderful! Charissa was the one interviewing me and Dannie took care of the promo and blog post. We had so much fun during the live recording, that we couldn’t stop reading the Foller.me profiles after the show, so if you missed the live version, you’ll never get to see what happened beyond the recorded version at Ustream.tv, sorry!

I was very nervous on the show, but Charissa and Dannie said that I never showed that (hope that this is true), although when I played back the recorded version and looked at myself, I kinda laughed. Do I really look that funny on TV? And the panda, oh that was weird, wasn’t it ;) but I couldn’t help it! Then, when Charissa asked me about the business model of Foller.me, wow! I didn’t know what to say, cause I haven’t really thought about that (much).

The guests in the chatroom during the show were amazing too, especially when we started reading their Foller.me profiles out loud, heh that was fun! And my brand-new-to-twitter brother @SoulSeekah got his one-minute celebrity spotlight on the show too, thanks for that girls!

Anyway, if you’ve missed the show, I feel terribly sorry for you! So go ahead and watch (at least) the recorded version on Startups Live. You won’t regret it.

Don’t forget to follow the girls on Twitter: Charissa Cowart and Dannie McClain. And definitely follow Startups Live on Twitter for the hottest news. Oh, I almost forgot, they have a Startups Live Facebook group. I suggest you join that too ;)

Published 3 years ago with 3 comments tagged , , , , , , ,


 

Foller.me: An Insight on Followers Geography

Hello everyone. Very busy week here and more good news! I have been asked about the future of Foller.me by many friends from Twitter and e-mail and IM, so I thought I’d share my thoughts here in public.

Foller.me is going quite good although there are a couple of bugs and issues that haven’t been solved yet, but we’re working on it very hard. The future? Well, that’s actually unpredictable, really. I’ll explain. We work out the areas that you (the end-users) want us to, and that is why your feedback is very important and it basically defines what we will be doing next. Yes, there are loads of comments and suggestions from inside the developers team, but we’re really looking forward to making a Twitter service that you people would use, and that’s our main goal.

Now today I’d like to give you a tiny little insight on what’s coming up next on Foller.me. We call it Followers Geography. Ever wondered where your followers are from? Or perhaps you wonder where your followers’ followers are from? Yes, there’s a location field in Twitter profiles which most people fill out quite good. Combining Google Geocoding and Google Maps, we’ve managed to scan through your followers’ locations and point them out on a map. Oh, and it also shows the current profile’s location (we totally forgot about that at first, hehe).

followers_geography

There are a few restrictions here, and the main one is definitely Google Maps API. Google allows around 15,000 API queries every day (while Twitter provides us with 20,000 every hour!) AND restrict their bulkness – there should be a 200-300 ms interval between each API call. So, generally you would have to wait for around 5 minutes to digest a profile with a thousand followers, and that’s where we came up with our great caching mechanism. The API calls are not made from within your browser. They run on a time-based (cron) job from our server to geocode new locations. Yes, it still takes a while to digest everyone, but you don’t get to see the delay and that’s the bright part of it!

The second restriction is your followers count. Your browser can’t handle 1,000,000 markers on a Google Map and that’s why we’re down to a maximum number of 500 followers (thus, locations) which works fast and doesn’t overload the browser with javascript. We’re working on a pagination system so don’t worry, you’ll get to see the rest in no time ;)

The public beta of Followers Geography is planned to be released by the end of next week. It’s going to be a new section below the three tag clouds and will be a small part of the huge “Followers” section in the future. If anybody would like to take a look at the living version post your requests into the comments, and I’ll provide you with an address, username and password by e-mail in exchange for good feedback ;)

That’s it for now. So how d’u guys like the idea? Anything else you’d like to see on that map?

Published 3 years ago with 6 comments tagged , , , , ,


 

Foller.me has Launched!

I’m glad to announce that Foller.me has launched today!! Hurray! We’ve already got some cool articles, shoutouts and tweets about the start, and more to (hopefully) come this night. Actually, some of them were written yesterday – timezones issue I guess.

follerme_screen

Anyway, we’ve reached our first goal yesterday night – 1000 unique profile views and most of our first few fans are very excited about The Foller.me Rundown which has been launched a couple of days ago.

The public version is available right over here: foller.me and you might also be interested in the blog: blog.foller.me. Thanks for all your support guys, we wouldn’t have done this without you, and YES, keep it up, because we’ve got plenty of surprizes coming to Foller.me this month, including a Greasemonkey script to inject Foller.me directly into Twitter, a WordPress widget and..

Oh yeah.. Totally forgot.. YES. A new section called Followers Geography! Yes, we are going to digest all your followers and spread them out on a Google Map! We’re running a few more tests on the developers version of Foller.me to make sure we have no problems with the Twitter and Google API limits. I will definitely announce once it’s public, so stay tuned!

For the latest Foller.me information you might consider following @follerme on Twitter.

Published 3 years ago with 7 comments tagged , , , , ,


 

Foller.me is Ready to Launch. We're going LIVE!

Good news everyone! We’re totally ready to launch! Everything is going as planned right now and we’re having a very busy weekend, but the launch date was announced and there’s nothing we can do to stop ;)

Might I start with a little press release? You can post this wherever you want, blogs, forums, chats, … newspapers, TV, radio ;) but please, schedule that for Monday (1st of June), okay?

Foller.me is a new third-party Twitter service that renders instant detailed information about a particular Twitter user’s latest 200 posts. It would generally take hours of scouting and pulling together information from hundreds of posts to scrape up some sort of insight into the user’s topic interests on Twitter. Foller.me does this in a flash, providing 3 tag clouds which represent the user’s key topics on Twitter.

How does it work? Foller.me gets access to the profile of a user via the Twitter API, scanning all the public info and his/her latest 200 tweets. It builds up three tag clouds: topics, #hashtags and @mentions, based on the user’s recent activity, excluding all stop words.

Let’s say you’ve received 20 new followers today and you cannot decide whether to follow back or not. You do NOT have the time to read 200 tweets on each of them, right? Nevertheless, you can’t just follow everyone, since you’ll get unwanted tweets in your friends timeline. Here’s the key: take a peek at ther Foller.me profiles and you’re done!

More information at foller.me
Blog at blog.foller.me

The beta version is still alive, but I’ll shoot that one down on Sunday. Everything’s set and almost working on Foller.me and take a look at your searchbox. It should indicate (lights up blue in FF3) that there’s a new search plugin available. Click on the drop-down list and select Foller.me. Anyway, I’m not sure that this would be enough, I mean wouldn’t it be nicer if there was a little button in your Firefox statusbar that you could click and input a name? Your thoughts on this one.

Published 3 years ago with 5 comments tagged , , , , , ,


 

New Twitter Startup! Beta Testers Wanted!

Hello everyone. You might be wondering why I haven’t updated my WordPress plugins for the last two weeks. I was very busy with this new Twitter service I got in mind and I’m proud to announce that a semi-public beta is already up and running. You might want to read the blog before you go test: blog.foller.me. The beta’s located here: beta.foller.me. I’m really looking forward to your comments, bug reports, suggestions and any other kind of feedback.

Planning to launch officially (still with a beta logo though) next week or perhaps the week after if everything goes wrong. Still looking for suitable hosting services in the US (currently hosted in Moscow, Russia).

Thanks.

Published 3 years ago with 3 comments tagged , , , ,



newer posts → back to top