Working, Working, Working

No Comments

Starting with this post I’m going to try a new format for my writing style as recommended by Lynn.  Focus more on bullet points and logical breaks, less on writing what comes to mind.

  • Why I do it?
  • What I do?
  • How do I do it?
  • What I’ve learned?

Why I do it?

Lately, due to an abundance of free time, I’ve driven myself into working almost constantly (I live in a town whose night life ends around 9pm, and isn’t much of a life to begin with).  Over a year ago I moved to St. George, Utah to work for a small internet startup.  You won’t find me talking about my day job much here though because while it’s nice for a day job it’s not where my passion lies.  However, far before I moved up here I fell in love with Ruby on Rails and was working with Powered By Geek and Lynn.  Since my day job can only bring in so much revenue and frankly isn’t as creatively satisfying as my PBG projects, I not only spend 8 hours at work, but come home and spend some time almost nightly writing code.

What do I do?

Both jobs I am the Lead (sometimes the only) Developer.  I specialize in Ruby on Rails (although I’ve been known to dabble in JavaScript and CSS).  Most of my time is spent pondering how to solve problems that push the capabilities of Rails and Web Development in general.  Sometimes I do a great job (Kiobo and Environs) and sometimes I run into major problems (iStalkr) and try to learn what I did wrong and how to improve.  Because Lynn’s somewhat of a leader and thinker in the Web 2.0 revolution (iStalkr was one of the first site that focused on lifestreaming), I tend to specialize in Web 2.0 apps, not to mention social networking and data portability.

How do I do it?

First off, if you’re going to spend 10-16 hours a day working you need a stead supply of caffeine.  It might not be the most health conscious thing, but to maintain higher brain functions, I feel like I need to keep a flow of caffeine.

Second you need to love what you do.  I had been playing with programming since I was 12 years old, however until I ran into Ruby on Rails I was content to specialize in IT Administration.  Once I started learning Rails though I was hooked.  I started to love web development, it became fun and the framework gave me a structured device to create something unique.  Not to mention the gems plugin manager, where I don’t feel like I’m constantly reinventing the wheel.

What I’ve learned?

I’ve learned Ruby on Rails far better than if it was just a hobby or a full time job.  It’s both my hobby and full time job.  I’ve also had the opportunity to see other developer’s code, sit down and ask questions about why they do things a certain way.

Databases and optimization have also been a huge issue to learn.  There is only so far you can take an idea with an unoptimized database (which iStalkr taught me).  Once again I got a chance to work with people who specialize in databases and could give me pointers on how to improve.  Databases, unfortunately, seem like something that you can only learn optimization for after you run into the issue.

Other items I’ve learned are:

  • Site APIs (digg, delicious, flickr, etc.)
  • RSS (both creating them and parsing them)
  • XML (parsing and creating)
  • ATOM
  • oAuth
  • OpenID
  • Facebook (app creation)
  • Lots more!

On Environs

No Comments

Environs Logo

Another project Powered By Geek recently did was an entirely Facebook application called Environs. This was our first real foray into the world of Facebook applications. For the Rails side we used a gem called facebooker this application, we also used RoRBook (http://github.com/lwallenstein/rorbook/tree/master) as a base.  Here is the about page written by Lynn about Environs.

You have friends
You are connected to those friends on social services

Because you’re all web 2.0 tastic, you and your friends use, like, a kerbillion social services

We quickly show you what you use, what they use, and where you haven’t connected yet.


Ah, haven’t I already connected with these people already?
Yep… at least once. But while you’re connected to some of your friends through most social services, you’re probably not connected to all your friends on all social services. And maybe that’s the way you like it, which is totally cool.

Also, there might be some apps you’ve thought sound interesting, but you didn’t know if anyone you knew was using them. Which makes sense, because using a social service all by your lonesome is kind of… well, not the point. But we can tell you, hey, 4 people you know are using Brightkite right now. Come on over! You know people here!


Why on earth would I want to do this?
Social services are fundamentally about information. What people like (for example, del.icio.us), what they watch (YouTube) and what they’re doing (twitter). You’ve already decided you want some information on some of your friends. What we do is give you the option of easily expanding the amount of information you can get about those friends. Like, say you follow your friend Mikey on BrightKite, and you know he likes dive bars. Awesome. But what if we told you hes also on flickr, and takes crazy-good photos of taco trucks. You might not have known that, but by expanding the number of services you use to connect to Mikey, you get to learn more about him. Basically, were a match-maker… but for people you’ve already met. Its your choice, though — we give you the option to connect with people on additional services, but you don’t have to connect to anyone on anything.
Screenshot of Environs

Screenshot of Environs

This app was tricky in two ways. First it was the first real Facebook app I have done, which meant there was a large learning curve as things that work just fine with a regular Rails site some times just don’t work with Facebook. Ajax and JavaScript in particular are completely different, and require a “hand roll” approach. This necessitated a lot of trial and error, not to mention the learning curve of what Facebook’s changes to JavaScript effect.

The other tricky aspect of this project was the main aspect of it, the parsing of other services. Each of the other web services (ex. Twitter, Last.Fm, Digg, etc.) have their own format for doing everything. We had to consolidate a parser to read all those varying formats and return the same result set so that it could be displayed. I had done stuff similar to this with iStalkr so I was prepared for gathering and reading all sorts of standard formats (RSS, XML, JSON). How we solved the problem was to create a library module, I initially designed it, but then submitted it to Bruce to help me improve the readability. This is still an ongoing problem, as more and more services are added to Environs I need to investigate their API’s and get them working.

Screenshot of Environs Service Matching

Screenshot of Environs Service Matching

Some services such as Flickr or MySpace are extremely tricky (in fact I’m still working on a way to parse MySpace). This might turn into somewhat of a rant, but services that allow you to set a URL to one thing, but then have a completely different value for tracking users, make it next to impossible for application developers to work with them. Flickr has their login, user id, and url, some queries to their API will accept any of the 3, however others want only the user id. Luckily, Flickr has a way around this because they have an API call that you could provide the piece of the information on the user and have it return the user id. Other services don’t offer this and it makes it a lot harder to get the data we need.

Overall, I think the project turned out pretty well. It’s a work in progress, there are still some services that we need to get working. Also, we’ve got some plans for recommendations that are in the pipelines. It’s a fun project that will only get better in time (at least until someone finally agrees on how to standardize these separate friend’s lists on each site).

On Kiobo

No Comments

So after a good long while in development and being kept under wraps I’m finally able to write about the newest big project I’ve been a part of, Kiobo.  Kiobo’s goal is:

Kiobo helps users enrich their browsing experience by making browsing a more social activity. We enable users to explore and leverage their social graph’s browsing behavior. Users can effortlessly find not only new sites relevant to their interest but also people with similar browsing behavior.

Kiobo, as a development job, was actually a challenge,  as there were a couple of big questions.  “How do you hold every website that a user goes to in a day?”, “How do you keep the database moving day after day, with one person able to submit thousands of records a day?”, and “How do you integrate with Facebook, the social networker’s tool of choice?”  Luckily I had help on this project, in the form of Bruce and Nate who are probably the two most influential developers in my life (both have mentored me at some point or another).  We also received help on the database side of things from Will who hails from the land of MS SQL, but has since started getting down and dirty with our database of choice MySQL.

This project started out because of the semi sucess that iStalkr received, because of iStalkr and the fairly recent bout with maintaining a server that was storing TONs of information I was a little wary to start on a project that had an even broader scope of what it would be tracking.  We spent a good long time designing visualizations, planning out the database schema that would provide the best performance versus ease of use, and working out how all these things were going to fit together since this was by far the most ambititous project we had worked on.  Bruce and I worked primarily on the Rails side while Nate worked on the firefox plugin and visualizations.  The project also spawned an oAuth plugin that Bruce wrote that is actually pretty nice, and that Nate integrated into the toolbar.

This project taught me that with the right database design that databases go from fragile bottlenecks that slow everything down to super speedy efficient ways of storing vast amounts of information.  One of the things that I learned from the problems we had with iStalkr was that storing text fields on large tables is a nightmare waiting to happen.  Luckily with this project we weren’t archiving the internet but were instead just logging where you went which was a lot less data we had to store.  We haven’t quite gotten to the point where we’ve started implementing caching but it is coming up soon.  Luckily I’ve gained quite a bit of experience during this time with working with memcached, page caching and slaving databases, so I feel confident that I’ll be able to improve performance in future releases.

We were able to solve the problem of performance by avoiding just one linked table, and instead going with multiple tables funnelling the information so that what ever the purpose of the query was you could work with the minimum number of records.

Privacy was a huge concern with this project.  Let’s face it, everyone goes to an internet site once in a while that they aren’t proud of it’s the nature of the internet now-a-days, one transposed letter is taking you to who knows where, to see who knows what.  With Kiobo people needed to be able to access and see EVERYTHING that we had on them, and be able to do with that what they like.  This instituted the need for multiple levels of privacy, as well as an easy way for someone to delete records.

Also I got my first crack at the rfacebook gem (which ran into serious problems when we upgraded to Rails2.0) and then the facebookr gem which since then I’ve had a lot more experience with on other projects, such as rorbook.  The facebook implementation started out as only a “link your account to facebook then login with facebook” but quickly grew to a facebook app, along with befriending people who are your friends on facebook.  The facebook implementation isn’t all quite there yet, but we’re working towards getting it there.

As for the business side, well I try to stay out of that as much as possible and luckily Lynn translates it and just tells me the highlights.

Kiobo was a really fun project to get started, and I look forward to hopefully getting a chance to create an HTML based version of the site to go along with the flash based version that is up now (what can I say, I’m prejudiced against flash).

On Conversion

No Comments

This week, after running into one RubyGem too many that was required to be compiled, I’ve switched over to the Mac for development. For the last 4 months (since I bought the 17inch MBP) I’ve been flirting with development on it. Until now it’s been the trophy wife that has been trotted out when I needed to do something “unixy”.  The transition hasn’t been easy, years of UltraEdit and Windows have engrained a muscle memory of hot keys deep into my subconscious.  I’ll go over some of the pros and cons I’ve noticed in my first 3 days as a hard core mac only user.

Pros:

  • It’s pretty – I am still blown away by how even little things are pretty in MacOS.  Things like the dock, the icons, even the transition between desktop spaces, are fun to just look at.  None of these things are extremely new to me (Windows Vista has a very pretty task switcher and Ubuntu’s Compiz has a neat desktop transition). But the amazing thing is that as pretty as it looks all those things don’t seem to slow it down.
  • Hot keys – It seems like there is a hot key for everything. As I start to learn more and more of them I’m starting to need the mouse less and less.
  • It’s a Unix environment – Things like the terminal, compiling, SSH are all built in. You don’t have to use DOS which is like a crippled kid, compared to a Unix terminal.  You don’t have to deal with running one “administrator” console and one “normal” console.
  • Unix Programs – You can run things like memcached (this is important because some of our sites have gotten to the point where we need it, and this way I don’t have to setup a special environment where memcached isn’t running.

Cons:

  • The trackpad sucks – I don’t know what it is, the giant track pad just doesn’t seem to work as well as any of my other laptop’s track pads.  I could hook up a wireless mouse, but when I’m sitting outside or in my recliner working a mouse isn’t feasible.  It also lacks a right click, why I just need one giant button is beyond me.
  • The keyboard layout – I’m almost certain the mac keyboard is a slightly different size than a standard pc keyboard.  Not to mention the lack of a 10-key pad even on the large 17 inch laptop.
  • The Apple, Option and Ctrl keys still trick me – muscle memory works against me ctrl+s doesn’t save my document.  Ctrl+arrow doesn’t jump around words.  I’m still trying to teach my fingers where they need to be to do things that I don’t even think about in windows/unix machines.
  • Task switching – I love alt+tabbing (apple+tabbing) and when the task switcher comes up I don’t see 20 things, however you pay for that because firefox browser windows are all open under the same task and to switch between windows in a task you need to push F10.  And I have to admit that besides F1-F5 I don’t tend to use the function keys in Windows/Unix.

So far I love TextMate, as limited as my understanding of the hotkeys are it seems like anything I want to do can be done by typing a little then hitting a button and it will do the rest for me.  I love going into a terminal and typing “mate project” and it opens my entire project up.

I also think that the 1920×1200 resolution is a bit too high for a 17 inch monitor (yes I’ve got oodles of desktop space but the font is a little impractical).  Watching movies on it looks amazing (if only it came with a bluray player).

Working with Rails on the Mac is a dream.  You have the ultra powerful text editor with TextMate, and you also have the ease of working in a Unix environment.  Something needs to be compiled?  No problem, you don’t need to install mingw or visual studio, spend hours setting it up, send the right flags.  This also applies to things like SSH.  Working with a private/public based ssh keys in windows with putty, was alright, but had serious limitations.  I installed OpenSSH on windows then spent hours upon hours configuring and getting it to finally work with key (which after about 5 hours I did get it working exactly like a Unix ssh environement).  With my Mac I do the same thing I have done a half million times in Linux, took me 30 seconds.

I can see now why so many Rails developers sound like Apple fanboys.  After working with Windows so long I had learned to work around strange errors and compile problems and turn a blind eye to all the issues I had, because I knew the environment I could whip out code from memory, hot keys were all based on muscle memory, but I seemed to be living in a very primitive age, one where all the time I saved not learning a new environment was lost fixing random errors and problems working in that environment.

The one thing I’m still wary about is using both a Windows/Linux OS at the same time as the Mac OS.  It’s easy to fall into old habits and just go back to what you know and feel safe with (it’s what happened about 2 weeks after I got my Mac).  I’m in the process of getting my boss to get a Mac or Mac Clone for me so I can go into total Mac Goodness immersion.

Initiation by fire – part 3

1 Comment

Beginner mistake number 3:

Associations.  If you’ve worked with relational databases at all you probably know the concepts one-to-one, one-to-many, and many-to-many, and you’ve probably cried your self to sleep on a numerous occasions writing join statements that probably are longer than some children’s books.  Every Rails developers learns the basics of Rail’s model associations.

class User < ActiveRecord::Base
has_many :items
end

That little bit of code will let you automatically find all the items that have the same user_id as the users id.  Powerful stuff!  However, what most beginning developers don’t learn is the additional things that you can do with the model associations.  Find you’re doing a query to get all the user’s active items all the time?  Well why keep writing the query when you can make it an association and then call it the same way as user.items?

has_many :active_items, :class_name => 'Item', :conditions => ["status = 'Active'"]

Now you can find the user and do  user.items AND user.active_items.  This saves a ton of typing and testing, since the query is always going to be the same, you can test once and then not worry if you misspelled ‘Active’ the 30th time you wrote the query.  Also in case you’re crying foul because :class_name came out of nowhere, all it does is tell what model to use to relate it to.  Rails normally does this for you because you’re creating a relationship with the same name or pluralization of the name of the model.

Many-to-many… alright you can stop wincing now.  Yes, many to many relationships are powerful, and useful for certain situations however they make for nasty SQL statements and even nastier code implementation.  However with the has_many :through they can act just like a one-to-many relationship.

class User < ActiveRecord::Base
has_many :roles, :through => :users_roles
end

class UsersRole < ActiveRecord::Base
belongs_to :role
belongs_to :user
end

class Role < ActiveRecord::Base
has_many :users, :through => :users_roles
end

Now you can retrieve the user and get the user’s roles, or retrieve the role and get the users.  Or you can even retrieve the user and then get the user’s roles, then take one of the roles and get all of the users with that role.  Pretty neat!

The fun doesn’t stop there.  If you have a group of items linked to the user, and you don’t want orphaned items when the user decides to close up shop and delete his user account then you can even add a line to the model telling it to get rid of all the users items when the user is deleted.

has_many :items, :dependent => :destroy

If you are feeling really crazy you can even take associations one step further and write an association that uses another association.  For example you have standard association then tell your next association to go through your standard association

has_many :admin_memberships, :class_name => 'Membership', :dependent => :destroy, :conditions => ['admin = 1']
has_many :administered_groups, :through => :admin_memberships, :source => :group

Now I know I threw another curve ball in there with :source.  That, like :class_name, tells it what model to use except you use it with :through associations, once again the reason you hadn’t seen it before is because Rails is normally pretty smart about these kind of things and will attempt to use the :through value as the model unless you specify something else using :source.

Finally lets move on to what, in my opinion, is the coolest thing you can do in a model.  Polymorphic relationships, which by the way as soon as you say that to a nontechnical boss, they’ll most likely leave you alone because the word alone is enough to scare off the un-technical.  What does this association with the power to scare off bosses actually do?  Well lets say a user has multiple things associated with them that you want to dynamically load.  A picture can either be of an item or a knickknack.  Sure you could define them all as associations, but why limit yourself?  You want to find them without having to think about what you’re findign.

class Picture < ActiveRecord::Base
belongs_to :resource, :polymorphic => true, :dependent => :destroy
end

class Item < ActiveRecord::Base
has_one :picture, :as => :resource
end

class Knickknacks < ActiveRecord::Base
has_one :picture, :as => :resource
end

Now on your picture table you’d add resource_type and resource_id so they can be associated.  Now by finding a picture and calling picture.resource will return the associated item or knickknack.

However by doing all these neat associations you run into what web developers call the n+1 query problem.  You have all this power at your fingertips and you’re generating a sql query every time you try to call an associated items.  Fear not, for Ruby on Rails has already foreseen your dilemma.  Ruby on Rails has something called eager loading, which is where you can banish those nightmares you’re still having about the 5 page join statement you wrote last year.  By adding :include into your user find Rails will magically generate that giant join statement for you and load up your associations via that statement.  No fuss, no muss, you get easy to read code and you can even eager load records from your eager loading.  First lets look at the normal code.

@user = User.find(params[:id], :include => :items)

Now to find a user, load all the items, and then load the item’s picture

@user = User.find(params[:id], :include => {:items => :picture})

That will generate a giant SQL join for you and will let you call something like user.item.first.picture without even having to think about all the magical SQL that needed to be generated.

Initiation by fire – part 2

2 Comments

Beginner mistake number 2:

Names and idioms. One thing I learned about the Rails community is that all the good developers secretly wish that they were English majors and Math majors at the same time. I know it sounds silly, but Ruby devs will call you down for using a crazy model name while at the same time praise you for using a complex idiom that nobody can read unless they understand the idiom. One of my first forays into RoR development was a site called LongTimeLost (http://longtimelost.com). This is also the first site where I started working with other Rails developers and learning what is acceptable and what isn’t when building a Rails site. After a developer took one look at my code he immediately noted that ‘looking4people’ is a horrible name for a table, and at the time I couldn’t understand what he was talking about, no one but us was going to see the code, what did it matter what I named things (I admit now looking back on my choice for a table name I’m quite embarrassed). It turns out in Rails naming is important. Rails does so much magical box stuff based on what you name things, have a table called users in your database? Well the way to search it with active record is User.find. The naming also affects how easy it is to read parts of your code. Looking4Person.find really isn’t good reading and Rails above most else should be readable.

The dichotomy of Rails though, is idioms. Idioms, which can cut multiple lines of code into one line. For example, probably the most popular idiom takes this code:

if foo=’bar’
puts ‘a’
else
puts ‘b’
end

And turns it into:

puts (foo==’bar’ ? ‘a’ : ‘b’ )

Clean, elegant, and if you don’t know the idiom, you have no idea what it’s doing. Thus is the way of idioms. The best place to keep up to date on idiomatic code is http://therailsway.com/. There is no hard and fast way to learn them, mostly they come by seeing them, saying “Wow, that is cool, what does it do exactly?” and then remembering it so you can impress others later.

Naming on the other hand does have a hard and fast way to do it. For the most part imagine what you are going to name your objects in sentences and ask your self if they make sense the best ways are the associations (which I’ll write about later) and the find method.
Looking4Person.find (a good example of a bad name)
User.find (a good name)
Now with the associations:
A User has_many Relationships
A Group belongs_to Admin
A User has_many Friends through Relationships (the has_many :through will be talked about later)

Variable names also should also avoid camel case like the plague. Camel case, while the standard for languages like Java is something that will make an experienced RoR developer’s blood boil. And I’ll give you an example why, which is easier to read:
a) authenticationMethodGoesHere
b) authentication_method_goes_here
The answer is obviously b, it looks more like English and is far easier to read at a glance, which is what Rails developers love.

Initiation by fire – part 1

1 Comment

Starting off as a Rails developer I ran into some problems coming from a Java/C++ background. Until Rails I more learned the languages for fun and something to play with in my free time, however by doing this I picked up some bad habits. I’ve worked with one of the top Ruby developers out there and learned to mimic his style and question his choices in code. I’ve learned a lot in the last year and a half I’ve been coding and now am even leading up a development team and teaching them the ins and the outs of Ruby.

Beginner mistake number 1:
For loops. I’m not so sure how the books teach it now but when I was initially learning Ruby all the books showed loops as

for item in collection

This is really easy to read, and will make developers from other languages feel a little more at ease with the language (it’s got the word for in there, and everyone knows what a for loop is). However you’ll quickly learn that using those for loops is really only for beginners. The real power of the for loop in rails doesn’t even have the word for in it. The real power comes from each.

collection.each do |item|

Not only is this also easy to read but it gives you so much more control over what you are doing. Want to pass in a counter variable? Simple

collection.each_with_index do |item, counter|

Or maybe you just want to get the keys from a group of hashes.

collection.each_key do |item_key|

There are many more possibilities, and by using each you are learning the style of what is probably the most powerful module in the Ruby language the enumerable module.

Because I can't stand to be upstaged.

No Comments

I barely have time to do anything, however I can’t be upstaged by either of my colleagues, Lynn or Will (both of whom write about interesting things), I’ve decided to actually post some useful comments. As the main Ruby on Rails programmer for Powered By Geek I’ve had some interesting experiences trying to stretch the language around whatever Lynn can imagine up for me. So I’ll be talking about rails and writing tips and tricks I’ve learned similar to Lynn with web-design and Will with database administration.

Newer Entries