whuffie project projects, whuffie
The latest Powered by Geek project isn’t a web application. It is a look into the process of turning a knowledgeable developer with no social footprint into a well connected social dynamo. The first developer enjoying the luxury of this transformation is me, Dan Ahern.
This project is called http://thewhuffieproject.com and it is architected and executed by a group of the PBG team : Michael, Lynn, and myself.
If you are not familiar with the term Whuffie, don’t be ashamed, I
wasn’t either. In Cory Doctorow’s book Down and Out in the Magic Kingdom, he defines Whuffie as social capital, or reputation. The Whuffie Project’s goal is to help me build up my reputation to the point at which the people who matter in my world know who I am as much as I know who they are.
I’ve been chosen for our maiden voyage for a few reasons :
1. I am engrossed in the social media landscape. I specialize in digesting content from social media sites and processing it new and useful ways.
2. I want this. I want to be more involved with the community that I love. I want to be recognized for the work I’m doing in the community, and I want to be involved in the direction of the web. I have talent, I have good ideas, and I believe that I can make the
world a better place through my work with social media.
3. I take direction well. I have a sincere willingness to do whatever Michael, Lynn or the Internet at large have in mind to build social capital.
We recently did a preliminary interview between Lynn and myself regarding our expectations for The Whuffie Project, and discussed paths that I may follow in the beginning to achieve my goal.
Our first step is to introduce Dan Ahern to the web, officially.
I have been a Ruby on Rails developer for 3 years. I originally began my programming career working on a compiled (c++) cgi binary at the company where I met Lynn Wallenstein. I had a natural affinity for procedural thought and problem solving, and in the years following I experimented with other programming languages like PHP and Python. These are powerful languages, but they lacked the elegance and ease of use that I was searching for. After a long conversation with Lynn, we decided on a new landscape: Ruby on Rails. Since falling in love with the language, and falling back in love with programming, I’ve tamed Ruby to build social media sites LongTimeLost, iStalkr, Playericious and Kiobo. I have contracted for major firms in the Ruby community like FiveRuns, and I’ve been involved in cutting edge work with social media APIs and other creative web services.
projects, ruby facebook, projects, ruby
Another project Powered By Geek recently did was an entirely Facebook application called Environs. This was our first real foray into the world of Facebook applications. For the Rails side we used a gem called facebooker this application, we also used RoRBook (http://github.com/lwallenstein/rorbook/tree/master) as a base. Here is the about page written by Lynn about Environs.
You have friends
You are connected to those friends on social services
Because you’re all web 2.0 tastic, you and your friends use, like, a kerbillion social services
We quickly show you what you use, what they use, and where you haven’t connected yet.
Ah, haven’t I already connected with these people already?
Yep… at least once. But while you’re connected to some of your friends through most social services, you’re probably not connected to all your friends on all social services. And maybe that’s the way you like it, which is totally cool.
Also, there might be some apps you’ve thought sound interesting, but you didn’t know if anyone you knew was using them. Which makes sense, because using a social service all by your lonesome is kind of… well, not the point. But we can tell you, hey, 4 people you know are using Brightkite right now. Come on over! You know people here!
Why on earth would I want to do this?
Social services are fundamentally about information. What people like (for example, del.icio.us), what they watch (YouTube) and what they’re doing (twitter). You’ve already decided you want some information on some of your friends. What we do is give you the option of easily expanding the amount of information you can get about those friends. Like, say you follow your friend Mikey on BrightKite, and you know he likes dive bars. Awesome. But what if we told you hes also on flickr, and takes crazy-good photos of taco trucks. You might not have known that, but by expanding the number of services you use to connect to Mikey, you get to learn more about him. Basically, were a match-maker… but for people you’ve already met. Its your choice, though — we give you the option to connect with people on additional services, but you don’t have to connect to anyone on anything.
Screenshot of Environs
The other tricky aspect of this project was the main aspect of it, the parsing of other services. Each of the other web services (ex. Twitter, Last.Fm, Digg, etc.) have their own format for doing everything. We had to consolidate a parser to read all those varying formats and return the same result set so that it could be displayed. I had done stuff similar to this with iStalkr so I was prepared for gathering and reading all sorts of standard formats (RSS, XML, JSON). How we solved the problem was to create a library module, I initially designed it, but then submitted it to Bruce to help me improve the readability. This is still an ongoing problem, as more and more services are added to Environs I need to investigate their API’s and get them working.
Screenshot of Environs Service Matching
Some services such as Flickr or MySpace are extremely tricky (in fact I’m still working on a way to parse MySpace). This might turn into somewhat of a rant, but services that allow you to set a URL to one thing, but then have a completely different value for tracking users, make it next to impossible for application developers to work with them. Flickr has their login, user id, and url, some queries to their API will accept any of the 3, however others want only the user id. Luckily, Flickr has a way around this because they have an API call that you could provide the piece of the information on the user and have it return the user id. Other services don’t offer this and it makes it a lot harder to get the data we need.
Overall, I think the project turned out pretty well. It’s a work in progress, there are still some services that we need to get working. Also, we’ve got some plans for recommendations that are in the pipelines. It’s a fun project that will only get better in time (at least until someone finally agrees on how to standardize these separate friend’s lists on each site).
projects, ruby projects, rails
So after a good long while in development and being kept under wraps I’m finally able to write about the newest big project I’ve been a part of, Kiobo. Kiobo’s goal is:
Kiobo helps users enrich their browsing experience by making browsing a more social activity. We enable users to explore and leverage their social graph’s browsing behavior. Users can effortlessly find not only new sites relevant to their interest but also people with similar browsing behavior.
Kiobo, as a development job, was actually a challenge, as there were a couple of big questions. “How do you hold every website that a user goes to in a day?”, “How do you keep the database moving day after day, with one person able to submit thousands of records a day?”, and “How do you integrate with Facebook, the social networker’s tool of choice?” Luckily I had help on this project, in the form of Bruce and Nate who are probably the two most influential developers in my life (both have mentored me at some point or another). We also received help on the database side of things from Will who hails from the land of MS SQL, but has since started getting down and dirty with our database of choice MySQL.
This project started out because of the semi sucess that iStalkr received, because of iStalkr and the fairly recent bout with maintaining a server that was storing TONs of information I was a little wary to start on a project that had an even broader scope of what it would be tracking. We spent a good long time designing visualizations, planning out the database schema that would provide the best performance versus ease of use, and working out how all these things were going to fit together since this was by far the most ambititous project we had worked on. Bruce and I worked primarily on the Rails side while Nate worked on the firefox plugin and visualizations. The project also spawned an oAuth plugin that Bruce wrote that is actually pretty nice, and that Nate integrated into the toolbar.
This project taught me that with the right database design that databases go from fragile bottlenecks that slow everything down to super speedy efficient ways of storing vast amounts of information. One of the things that I learned from the problems we had with iStalkr was that storing text fields on large tables is a nightmare waiting to happen. Luckily with this project we weren’t archiving the internet but were instead just logging where you went which was a lot less data we had to store. We haven’t quite gotten to the point where we’ve started implementing caching but it is coming up soon. Luckily I’ve gained quite a bit of experience during this time with working with memcached, page caching and slaving databases, so I feel confident that I’ll be able to improve performance in future releases.
We were able to solve the problem of performance by avoiding just one linked table, and instead going with multiple tables funnelling the information so that what ever the purpose of the query was you could work with the minimum number of records.
Privacy was a huge concern with this project. Let’s face it, everyone goes to an internet site once in a while that they aren’t proud of it’s the nature of the internet now-a-days, one transposed letter is taking you to who knows where, to see who knows what. With Kiobo people needed to be able to access and see EVERYTHING that we had on them, and be able to do with that what they like. This instituted the need for multiple levels of privacy, as well as an easy way for someone to delete records.
Also I got my first crack at the rfacebook gem (which ran into serious problems when we upgraded to Rails2.0) and then the facebookr gem which since then I’ve had a lot more experience with on other projects, such as rorbook. The facebook implementation started out as only a “link your account to facebook then login with facebook” but quickly grew to a facebook app, along with befriending people who are your friends on facebook. The facebook implementation isn’t all quite there yet, but we’re working towards getting it there.
As for the business side, well I try to stay out of that as much as possible and luckily Lynn translates it and just tells me the highlights.
Kiobo was a really fun project to get started, and I look forward to hopefully getting a chance to create an HTML based version of the site to go along with the flash based version that is up now (what can I say, I’m prejudiced against flash).