So after a good long while in development and being kept under wraps I’m finally able to write about the newest big project I’ve been a part of, Kiobo. Kiobo’s goal is:
Kiobo helps users enrich their browsing experience by making browsing a more social activity. We enable users to explore and leverage their social graph’s browsing behavior. Users can effortlessly find not only new sites relevant to their interest but also people with similar browsing behavior.
Kiobo, as a development job, was actually a challenge, as there were a couple of big questions. “How do you hold every website that a user goes to in a day?”, “How do you keep the database moving day after day, with one person able to submit thousands of records a day?”, and “How do you integrate with Facebook, the social networker’s tool of choice?” Luckily I had help on this project, in the form of Bruce and Nate who are probably the two most influential developers in my life (both have mentored me at some point or another). We also received help on the database side of things from Will who hails from the land of MS SQL, but has since started getting down and dirty with our database of choice MySQL.
This project started out because of the semi sucess that iStalkr received, because of iStalkr and the fairly recent bout with maintaining a server that was storing TONs of information I was a little wary to start on a project that had an even broader scope of what it would be tracking. We spent a good long time designing visualizations, planning out the database schema that would provide the best performance versus ease of use, and working out how all these things were going to fit together since this was by far the most ambititous project we had worked on. Bruce and I worked primarily on the Rails side while Nate worked on the firefox plugin and visualizations. The project also spawned an oAuth plugin that Bruce wrote that is actually pretty nice, and that Nate integrated into the toolbar.
This project taught me that with the right database design that databases go from fragile bottlenecks that slow everything down to super speedy efficient ways of storing vast amounts of information. One of the things that I learned from the problems we had with iStalkr was that storing text fields on large tables is a nightmare waiting to happen. Luckily with this project we weren’t archiving the internet but were instead just logging where you went which was a lot less data we had to store. We haven’t quite gotten to the point where we’ve started implementing caching but it is coming up soon. Luckily I’ve gained quite a bit of experience during this time with working with memcached, page caching and slaving databases, so I feel confident that I’ll be able to improve performance in future releases.
We were able to solve the problem of performance by avoiding just one linked table, and instead going with multiple tables funnelling the information so that what ever the purpose of the query was you could work with the minimum number of records.
Privacy was a huge concern with this project. Let’s face it, everyone goes to an internet site once in a while that they aren’t proud of it’s the nature of the internet now-a-days, one transposed letter is taking you to who knows where, to see who knows what. With Kiobo people needed to be able to access and see EVERYTHING that we had on them, and be able to do with that what they like. This instituted the need for multiple levels of privacy, as well as an easy way for someone to delete records.
Also I got my first crack at the rfacebook gem (which ran into serious problems when we upgraded to Rails2.0) and then the facebookr gem which since then I’ve had a lot more experience with on other projects, such as rorbook. The facebook implementation started out as only a “link your account to facebook then login with facebook” but quickly grew to a facebook app, along with befriending people who are your friends on facebook. The facebook implementation isn’t all quite there yet, but we’re working towards getting it there.
As for the business side, well I try to stay out of that as much as possible and luckily Lynn translates it and just tells me the highlights.
Kiobo was a really fun project to get started, and I look forward to hopefully getting a chance to create an HTML based version of the site to go along with the flash based version that is up now (what can I say, I’m prejudiced against flash).