HyperTree – Converting JIT Examples to JQuery

2 Comments

Recently, while working on the Kiobo project with Lynn we stumbled across the JIT (JavaScript Information Visualization Toolkit) library. It is an amazing toolkit that has helped us see the future of JavaScript visualizations, which has previously been an area where only Flash/ActionScript has had the power to display highly animated and appealing client side visualization of data. On the current Kiobo site, you can see it is almost entirely written in Flash/ActionScript, but the future leads us in another direction.

The JIT is quite easy to integrate into your own site. Simply taking the examples and filling in your own JSON object with data can make quite an impression on people visiting your project. One issue that I ran into during implementation was that all the examples were written with the MooTools framework in mind. MooTools is a great framework, but Kiobo was already using the JQuery library for our JavaScript needs. I took on the task of converting the examples into JQuery to use them with Kiobo.

To implement JQuery with Kiobo I used the JRails plugin to replace all the Prototype methods. What follows won’t be a recreation of the entire tutorial, only the situations where you will need to change things.

First, rather than calling the MooTools library, we’ll call the JQuery library (and while we’re at it we’ll convert all the standard HTML to Rails).

So in the example you have this code:

<html>
<head>
<link type="text/css" rel="stylesheet" href="/static/css/example-hypertree.css" />

<!--[if IE]>
<script type="text/javascript" src="/static/js/excanvas.js"></script>
<![endif]-->
<script type="text/javascript" src="/static/js/mootools-1.2.js" ></script>
<script type="text/javascript" src="/static/js/hypertree/Hypertree.js" ></script>
<script type="text/javascript" src="/static/js/example/example-hypertree.js" ></script>

</head>
<body onload="init();">
<div id="infovis"></div>
</body>
</html>

Once you have converted everything and called your JQuery library it should look like this:

<html>
<head>

<!--[if IE]>
<%= javascript_include_tag 'excanvas' %>
<![endif]-->

<%= javascript_include_tag :defaults %>
<%= javascript_include_tag 'example/example-hypertree' %>
<%= javascript_include_tag 'Hypertree' %>
<%= stylesheet_link_tag 'example-hypertree' %>
</head>
<body onload="init();">
<div id="infovis"></div>
</body>
</html>

This will load the JQuery Library, as well as the Hypertree library, the example-hypertree library and the stylesheet example-hypertree.

At this point you will be able to continue the tutorial and see your first HyperTree. However, when you get to adding labels things once again diverge.

The example uses this code:

function init() {
var json = //json data mentioned above...
//Create a new canvas instance.
var canvas = new Canvas('mycanvas', {
//Where to inject the canvas. Any HTML container will do.
'injectInto':'infovis',
//Width and height of canvas, default's to 200.
'width': 900,
'height':500,
//Canvas styles.
'styles': {
'fillStyle': '#ddd',
'strokeStyle': '#dd00bb'
}
});
var ht= new Hypertree(canvas, {

onCreateLabel: function(domElement, node) {
$(domElement).set('html', node.name + " and text");
}

});

ht.loadTreeFromJSON(json);
ht.compute();
ht.plot();
}

This will not work the same with JQuery as it has some MooTools specific calls, particularly:
$(domElement).set(‘html’, node.name + ” and text”)
However this is an easy fix with JQuery
$(domElement).html(node.name + ” and text”)
So your entire file should look like

function init() {
var json = //json data mentioned above...
//Create a new canvas instance.
var canvas = new Canvas('mycanvas', {
//Where to inject the canvas. Any HTML container will do.
'injectInto':'infovis',
//Width and height of canvas, default's to 200.
'width': 900,
'height':500,
//Canvas styles.
'styles': {
'fillStyle': '#ddd',
'strokeStyle': '#dd00bb'
}
});
var ht= new Hypertree(canvas, {

onCreateLabel: function(domElement, node) {
$(domElement).html(node.name + " and text");
}

});

ht.loadTreeFromJSON(json);
ht.compute();
ht.plot();
}

When placing the labels you will also need to change the code a bit. I couldn’t find a similar method inside the JQuery core for the .toInt() method so we’ll fall back to parseInt.

function init() {
var json = //json data...
//Create a new canvas instance.
var canvas = new Canvas('mycanvas', {
//Where to inject the canvas. Any HTML container will do.
'injectInto':'infovis',
//Width and height of canvas, default's to 200.
'width': 900,
'height':500,
//Canvas styles.
'styles': {
'fillStyle': '#ddd',
'strokeStyle': '#dd00bb'
}
});
var ht= new Hypertree(canvas, {
onCreateLabel: function(domElement, node) {
$(domElement).html(node.name + " and text");
},

//Take the left style property and
// substract half of the label actual width.
onPlaceLabel: function(tag, node) {
var width = tag.offsetWidth;
var intX = tag.style.left.toInt();
intX -= width/2;
tag.style.left = intX + 'px';
}

});

ht.loadTreeFromJSON(json);
ht.compute();
ht.plot();
}

Use this example instead:

function init() {
var json = //json data...
//Create a new canvas instance.
var canvas = new Canvas('mycanvas', {
//Where to inject the canvas. Any HTML container will do.
'injectInto':'infovis',
//Width and height of canvas, default's to 200.
'width': 900,
'height':500,
//Canvas styles.
'styles': {
'fillStyle': '#ddd',
'strokeStyle': '#dd00bb'
}
});
var ht= new Hypertree(canvas, {
onCreateLabel: function(domElement, node) {
$(domElement).html(node.name + " and text");
},

//Take the left style property and
// substract half of the label actual width.
onPlaceLabel: function(tag, node) {
var width = tag.offsetWidth;
var intX = parseInt(tag.style.left);
intX = intX-(width/2);
tag.style.left = intX + 'px';
},

});

ht.loadTreeFromJSON(json);
ht.compute();
ht.plot();
}

Finally, adding events to the HyperTree. This is actually completely different than the MooTools example
.addEvents({‘click’: function(e) {ht.onClick(node.id)}});
It will instead be replaced with click method
.click( function () {ht.onClick(node.id)} );

So your final code should look like this:

function init() {
var json = //some json data
//Create a new canvas instance.
var canvas = new Canvas('mycanvas', {
//Where to inject the canvas. Any HTML container will do.
'injectInto':'infovis',
//Width and height of canvas, default's to 200.
'width': 900,
'height':500,
//Canvas styles.
'styles': {
'fillStyle': '#ddd',
'strokeStyle': '#dd00bb'
}
});
var ht= new Hypertree(canvas, {
onCreateLabel: function(domElement, node) {
$(domElement).set('html', node.name + " and text").addEvents({
//Call the "onclick" method from
//the hypertree to move the hypertree
//correspondingly.
//This method takes the clicked node's id.
'click': function(e) {
ht.onClick(node.id);
}
});

},

//Take the left style property and
//substract half of the label actual width.
onPlaceLabel: function(tag, node) {
var width = tag.offsetWidth;
var intX = tag.style.left.toInt();
intX -= width/2;
tag.style.left = intX + 'px';
}

});

ht.loadTreeFromJSON(json);
ht.compute();
ht.plot();
}

Your code:

function init() {
var json = //some json data
//Create a new canvas instance.
var canvas = new Canvas('mycanvas', {
//Where to inject the canvas. Any HTML container will do.
'injectInto':'infovis',
//Width and height of canvas, default's to 200.
'width': 900,
'height':500,
//Canvas styles.
'styles': {
'fillStyle': '#ddd',
'strokeStyle': '#dd00bb'
}
});
var ht= new Hypertree(canvas, {
onCreateLabel: function(domElement, node) {
$(domElement).html(node.name + " and text").click( function () {ht.onClick(node.id)} );
},

//Take the left style property and
//substract half of the label actual width.
onPlaceLabel: function(tag, node) {
var width = tag.offsetWidth;
var intX = tag.style.left.toInt();
intX -= width/2;
tag.style.left = intX + 'px';
}
});

ht.loadTreeFromJSON(json);
ht.compute();
ht.plot();
}

The Whuffie Project

2 Comments

The latest Powered by Geek project isn’t a web application. It is a look into the process of turning a knowledgeable developer with no social footprint into a well connected social dynamo.  The first developer enjoying the luxury of this transformation is me, Dan Ahern.

This project is called http://thewhuffieproject.com and it is architected and executed by a group of the PBG team : Michael, Lynn, and myself.

If you are not familiar with the term Whuffie, don’t be ashamed, I
wasn’t either. In Cory Doctorow’s book Down and Out in the Magic Kingdom, he defines Whuffie as social capital, or reputation.  The Whuffie Project’s goal is to help me build up my reputation to the point at which the people who matter in my world know who I am as much as I know who they are.

I’ve been chosen for our maiden voyage for a few reasons :

1. I am engrossed in the social media landscape.  I specialize in digesting content from social media sites and processing it new and useful ways.
2. I want this.  I want to be more involved with the community that I love.  I want to be recognized for the work I’m doing in the community, and I want to be involved in the direction of the web.  I have talent, I have good ideas, and I believe that I can make the
world a better place through my work with social media.
3. I take direction well.  I have a sincere willingness to do whatever Michael, Lynn or the Internet at large have in mind to build social capital.

We recently did a preliminary interview between Lynn and myself regarding our expectations for The Whuffie Project, and discussed paths that I may follow in the beginning to achieve my goal.

Our first step is to introduce Dan Ahern to the web, officially.

I have been a Ruby on Rails developer for 3 years. I originally began my programming career working on a compiled (c++) cgi binary at the company where I met Lynn Wallenstein.  I had a natural affinity for procedural thought and problem solving, and in the years following I experimented with other programming languages like PHP and Python.  These are powerful languages, but they lacked the elegance and ease of use that I was searching for.  After a long conversation with Lynn, we decided on a new landscape: Ruby on Rails.  Since falling in love with the language, and falling back in love with programming, I’ve tamed Ruby to build social media sites LongTimeLost, iStalkr, Playericious and Kiobo.  I have contracted for major firms in the Ruby community like FiveRuns, and I’ve been involved in cutting edge work with social media APIs and other creative web services.

Rails Rumble '08

1 Comment

Rails Rumble was an amazing experience.  I think I slept about 6 hours over a 56 hour period and while I became grouchy I don’t think I ever stopped having fun.  The fun stopped after the first day of voting.  Starting during the second day of voting I <3 Games plummeted in the rankings.  I am not really sure what happened to cause this fall.  I am not the most objective person but in testing some of these other apps I’ve run into some real problems that I don’t understand how something that doesn’t quite work could be ranked higher than our app which I think is about 90% done.

Problems with voting

  • With any voting system there are always ways to game the system.  The OpenID requirement for Rails Rumble voting would make gaming rather easy I would think.  Without a captcha or email validation it would be fairly easy for a machine to sign up for voting as many times as it wanted.
  • Voting with most people for Rails Rumble is only based off of a glance, there is very little actual signing up to use the app going on.
  • No way to explain what your feelings about the app were.

I think if some of these issues were addressed I would feel a little better about the voting system.  I think the 8 random apps a day idea was a good one that helps prevent the event from becoming an popularity contest.

Gaming the system is always going to be a problem.  Bots can be created that get around almost every system to stop them, but when the systems are few and far between to stop them they can have a field day and can dramatically skewer results.  I do know a few people who aren’t bots that just said “Oh, well I’ll just give every other app 1 star across the board” and it shows.  Rather than 3 stars being the “average” value for an app it is the high value.  I know there won’t ever be a 5.0 over all app, but we’re talking about 2.9 being the high value and 1.6 being the low value.  That is a lot of 1 and 2 stars over the 4 or 5 stars.  On a scale of 1-5 then 3 should be the average, that should be “this app is unremarkable, there is nothing wrong, but nothing right either” instead if we keep going along the way we are currently going then 2.9 is going to be the winning score, which using the normal averages would mean that it is slightly below average, and while I’ll be the first to admit none of these apps are above average against an application where you have an unlimited time frame and tons of QA testers.  If you keep in mind that every single one of the apps was designed in 48 hours by 1-4 people I would have to say a great many of them are above average.

One of the problems I have with the voting system is that the results tend to be skewed towards pretty on the front page apps.  Powered By Geek didn’t exactly cater to this, we hid a lot of functionality after sign up, but we did put a little video up explaining about the app.  The real problem I have is the number of people who aren’t even signing up who are voting on the app.  I know for a fact that our login/signup process works because we had a guy dedicated to testing that procedure during the actual competition and I’ve created about 10 accounts as well with my numerous Open IDs and as regular users.  Looking at our numbers we’ve had 79 users sign up, and when you take into account all my test accounts that is about 70 users total who have signed up and yet we have 240 votes on our site.  That is a 1:4 ratio of people who actually looked at our site vs people who just voted either after looking at our home page or without looking at the site at all.

I’m wary to point my finger at people and point out flaws as to why they shouldn’t be at their position, however some of the apps I tested had main functionality broken and yet they are sitting in the top 30 spots.  Testing is almost nonexistent, a friend was testing other sites and uploaded a picture that he figured would disappear after a while but instead it was on there for over 24 hours.  The site only had slots to show 5 pictures that means that it took over a day to upload 5 pictures on the site.

My final real problem is that there is no way to tell why people voted one way or another.  I think a short description on why they are voting one way or another would help everyone involved to understand what people liked and what people disliked about the app, not to mention it would make it easier to tell when someone was gaming the system.

Overall I think the Rails Rumble brings an awesome service to the community.  48 hours with the right motivation (competition) is really useful as it can really get an interesting idea or product built.  Not to mention it helps bring you together with friends towards a common goal.  I really just wish everyone could come away from it feeling like the voting wasn’t just a game.

Tech Demo

1 Comment

So in preparation for Rails Rumble I’ve been researching a lot of sexy little plugins.  We have also had a need to test this plugins to make sure they’ll work not just locally but in a shared hosting environment (the first time you get burned and spend an entire weekend, 16 hours, trying to figure out why something that works perfectly on your local box dies horribly on shared hosts you learn to test everything).  To accomplish this testing we’ve needed a small little app so I can get the bare minimum for the plugins in there.  I’d like to post everything I did while I wait for Lynn to push everything up to test. Mostly this is just a quick run down because someone else has already done a far better job than I could explaining in depth how to install and configure all this stuff. This is also by no means how you should do any of this, this is solely for testing out various plugins and packages.


The plugins, gems, and kitchen sink

What I’ve got up and working on my local box so far is:

  1. Simple Email Notification with ActionMailer
  2. SMS Fu
  3. Twitter4r
  4. Gravatar Profile Image
  5. Juggernaut
  6. BackgrounDRB
  7. GoogleCalendar/ICal

Simple Email Notification

Won’t really go into this, it’s obvious and everyone has done it a million times

SMS Fu

  • Clone it
  • Include it
  • Put this code in there:

Controller:

def index
@carriers = %w(alltell ameritech at&t bellsouthmobility blueskyfrog boost cellularsouth helio kajeet metropcs powertel pscwireless qwest southernlink suncom t-mobile virgin verizon)
end

def text_message
deliver_sms(params[:pages][:phone_number],params[:pages][:carrier],"Test Text Message")
redirect_to :action => 'index'
end


View:

<% form_for :pages, :url => {:action => :text_message} do |f| -%>
Phone Number: <%= f.text_field :phone_number %>
<br/>
Carrier: <%= f.select :carrier, @carriers %>
<br/>
<%= submit_tag "Send Text" %>
<% end -%>

Twitter4r

  • Install the gem
  • require the gem
  • Use the following code

Controller

def twitter_message
client = Twitter::Client.new(:login => params[:pages][:user_name], :password => params[:pages][:password])
if client.authenticate?(params[:pages][:user_name], params[:pages][:password])
new_message = client.status(:post, params[:pages][:text_message])
return redirect_to :action => 'index'
else
flash[:notice] = "Failed to authenticate"
return redirect_to :action => 'index'
end
end


View

<% form_for :pages, :url => {:action => :twitter_message} do |f| -%>
Twitter User Name: <%= f.text_field :user_name %>
<br/>
Twitter Password: <%= f.password_field :password %>
<br/>
Text To Post: <%= f.text_field :text_message, :limit => 140 %>
<br/>
<%= submit_tag "Send Twitter" %>
<% end -%>

Gravatar

Just use the code in your application helper

Juggernaut

  • Make sure you have json and eventmachine installed
  • install the gem
  • install the plugin
  • juggernaut -g juggernaut.yml (to make the config file)
  • Use the following code
  • juggernaut -c juggernaut.yml (to start the push server)

Layout (or anywhere you want to use Juggernaut)

<%= javascript_include_tag 'juggernaut/swfobject' %>
<%= javascript_include_tag 'juggernaut/juggernaut' %>
<%= juggernaut(:channels => ['chat', 'notifications']) %>

View

<fieldset><legend>Chat</lengend>
<ul id="chat_data" style="list-style:none">
</ul>
</fieldset>
<%= form_remote_tag(
:url => { :action => :send_data },
:complete => "$('chat_input').value = '#{session.id}'" ) %>
<%= text_field_tag( 'chat_input', session.id, { :size => 20, :id => 'chat_input'} ) %>
<%= submit_tag "Chat" %>
</form>
<br/><br/>
<fieldset><legend>Notifications</legend>
<ul id="notification_data" style="list-style:none">
</ul>
</fieldset>

juggernaut_hosts.yml

:hosts:
- :port: 5001
:host: 127.0.0.1
:public_host: how_the_public_accesses_your_site.com
:public_port: 5001

BackgrounDRB

  • install chronic and packet requirements
  • Clone the code
  • rake backgroundrb:setup
  • ruby script/generate worker notifications
  • use the following code
  • ruby script/background start (to start the backgrounDRB)

backgroundrb.yml

:backgroundrb:
:ip: 0.0.0.0
:port: 11006
:schedules:
:notifications_worker:
:check_notifications:
:trigger_args:
:start: <%= Time.now + 5.seconds %>
:end: <%= Time.now + 1.year %>
:repeat_interval: <%= 5.minutes %>

notifications_worker.rb

class NotificationsWorker < BackgrounDRb::MetaWorker
set_worker_name :notifications_worker
def create(args = nil)
logger.info 'created worker'
end

def check_notifications
if true
logger.info 'sending notification'
Juggernaut.send_to_channel("new Insertion.Top(\"notification_data\", \"<li>Sending Notification at #{Time.now}<\/li>\");", "notifications")
end
end
end

View

<fieldset><legend>Notifications</legend>
<ul id="notification_data" style="list-style:none">
</ul>
</fieldset>

ICalendar

  • Install the gem
  • require the gem
  • Use the following code
  • Add to Google Calendar to check your work

View

def calendar
@cal = Icalendar::Calendar.new

[{:name => 'Meeting', :start => Time.now.beginning_of_day, :end => Time.now},
{:name => 'Greeting', :start => Time.now.beginning_of_day-1.day, :end => Time.now-1.day}].each do |comp|
event = Icalendar::Event.new
event.start = comp[:start].strftime("%Y%m%dT%H%M%S")
event.end = comp[:end].strftime("%Y%m%dT%H%M%S")
event.summary = comp[:name]
@cal.add event
end
@cal.publish
headers['Content-Type'] = "text/calendar; charset=UTF-8"
render :text => @cal.to_ical, :layout => false
end





What you will end up with is a very ugly little app that will look like this:

Rails Rumble

1 Comment

Powered By Geek will be participating in Rails Rumble 08.

I’ll be working along side Lynn Wallenstein, Nathan Ostgard and William Harris to create a Rails application in 48 hours.  We’re super excited, and I’m going to try and keep a log of what is going on to post it (we’ll see how well that goes).

Database Explanation

2 Comments

What is a database?

You’ve probably used or seen an Excel spreadsheet.  Well, a database looks similar, but it’s a lot more powerful, a database can do things that Excel only dreamed of doing.  A “database” is made up of “tables” which are made up of a bunch of “columns” and “rows”.

Excel Terminology | Database Terminology
-----------------------------------------
Excel File        |  Database
Sheets            |  Tables
Letter Columns    |  Columns
Number Rows       |  Rows

Each table holds a specific type of records or rows.  So the tables of a simple database “schema” (or design) would look like this:

+-----------------------------+
| Tables_in_truth_development |
+-----------------------------+
| answers                     |
| choices                     |
| polls                       |
| users                       |
+-----------------------------+

Notice how it is broken up, there is a table for users in there.  That will hold the information for all the users.  We make tables logically for the different items we would like to keep track of.

Databases that are used most commonly are called “Relational Databases”.  What that means is that each of the tables can link to another table.  The most common way to imagine databases is to draw webs how they relate.  Truth Database
As you can see in the picture: A User is related to polls and answers.  A Poll is related to answers, choices and users.  A Choice is related to polls and answers.  Finally, an Answer is related to choices, polls, and users.

This lets us do stuff like to find a User record and then see all his answers and polls.

Now that we know a little bit more about tables, lets look and what a table consists of.  It is made up of columns (or fields) that we want to track. If we were to look at it it would look like this:

+-------------+
| Field       |
+-------------+
| id          |
| facebook_id |
| name        |
| created_at  |
| updated_at  |
+-------------+

Now if you want to picture it in your mind each field would be a letter in Excel.

A table is also made up of rows (or records) which is one full set of filled out columns.  If you were to look at it it would look like this:

+----+-------------+-----------+---------------------+---------------------+
| id | facebook_id | name      | created_at          | updated_at          |
+----+-------------+-----------+---------------------+---------------------+
|  2 |   678674096 | Dan Ahern | 2008-08-08 23:47:36 | 2008-08-08 23:47:36 |
+----+-------------+-----------+---------------------+---------------------+

Rows are the numbers in excel.

The way we link the tables together is to put information from one table into another.  Lets look at a Poll:

+----+---------+---------------------+---------------------+--------------------+
| id | user_id | question            | created_at          | updated_at         |
+----+---------+---------------------+---------------------+--------------------+
| 11 |       2 | What should I wear? | 2008-08-08 23:38:43 | 2008-08-08 23:38:43|
+----+---------+---------------------+---------------------+--------------------+

Notice how the poll has a column called “user_id” and that column has the same number as the “id” field in user?  This is how we know who the poll belongs to.

And then if we wanted to see the choices for the poll we can use the “id” column from poll and find those numbers in choices in the column called “poll_id”, and we get back this:

+----+---------+-----------------+---------------------+---------------------+
| id | poll_id | name            | created_at          | updated_at          |
+----+---------+-----------------+---------------------+---------------------+
| 33 |      11 | Nothing         | 2008-08-08 23:38:43 | 2008-08-08 23:38:43 |
| 34 |      11 | A Monkey        | 2008-08-08 23:38:43 | 2008-08-08 23:38:43 |
| 35 |      11 | T-Shirt & Jeans | 2008-08-08 23:38:43 | 2008-08-08 23:38:43 |
+----+---------+-----------------+---------------------+---------------------+

Now to actually look through a database ourselves would be time consuming and wouldn’t be very helpful (we would have to look at all the users, get the id of the user that we want, then look the polls and find the user_id that matches our user’s id, then look at all the choices and find the ones that match our poll’s id) so there is a way to interact with databases called SQL (Structured Query Language).  So we can basically ask the database questions and it will return answers.  To get the first result above (the tables) I typed:

show tables;

Notice how my SQL statement or query ends in a semicolon “;”?  From the command line this is usually how you tell your database that you are done typing the commands and it should go out and get you the answers.  The answer that query returns are the list of all the tables in the database.  In order the first three queries I performed were:

(Get the list of tables)
show tables;

(Get the columns in users)
describe users;

(Get all the user rows)
select * from users;

Now that last one was a little different.  In the last one I’m saying
“select” (go get)
“*” (all values of all the columns)
“from users” (from the users table)
So it reads “go get all the values of all the columns from the users table” and the database goes out and fetches it.

The next query I did after I saw the user was to do:

select * from polls where user_id = 2;

The “select * from polls” is the same as what we did for the users but we added a “condition” (something that the database should look for)
“where user_id = 2″ (find only records where user_id is equal to 2)
So it reads “go get all the values of all the columns from the polls table and find only records where the user_id is equal to 2″ and the database went out and fetched those results for me.

The final query I performed was:

select * from choices where poll_id = 11;
This does the exact same thing as the query above only it looks in the choices table and finds only records where the poll_id is equal to 11.

Multiple Databases

All the examples up until now are assuming you only have one database and you have already selected it.  However that isn’t normally the case. Lets move back into our Excel analogy and apply it to the database and look at some of the SQL used as well.

Starting from the beginning

GOAL: Find the database we want.
EXCEL EQUIVALENT:  Finding the Excel file we want in a folder full of excel files we have to look at the names for all the excel files.
SQL: show databases;
RESULT:

+------------------------------+
| Database                     |
+------------------------------+
| information_schema           |
| classifieds_development      |
| dotcom_development           |
| environs_development         |
| fiveruns_blog                |
| hcl                          |
| inever_development           |
| kiobo_development            |
| mysql                        |
| playericious                 |
| test                         |
| truth_development            |
| upillar                      |
| upillar_database_development |
+------------------------------+

NEXT STEP: We want to get into the database.

GOAL: Get into the database we want to use.
EXCEL EQUIVALENT: Double click on the file
SQL: use truth_development;
RESULT:
Database changed
NEXT STEP: We want to look at all the tables.

GOAL: Look at all the tables
EXCEL EQUIVALENT: Look through the sheets tabs.
SQL: show tables;
RESULT:

+-----------------------------+
| Tables_in_truth_development |
+-----------------------------+
| answers                     |
| choices                     |

| polls                       |
| users                       |
+-----------------------------+

NEXT STEP: We want to look at all the users.

GOAL: Find all the users.
EXCEL EQUIVALENT: Click on the sheet labeled “Users”
SQL: select * from users;
RESULTS:

+----+-------------+------------------+---------------------+---------------------+
| id | facebook_id | name             | created_at          | updated_at          |
+----+-------------+------------------+---------------------+---------------------+
|  2 |   678674096 | Dan Ahern        | 2008-08-08 23:47:36 | 2008-08-08 23:47:36 |
|  4 |   569397655 | William Harris   | 2008-08-08 23:52:50 | 2008-08-08 23:52:50 |
|  5 |   500092293 | Lynn Wallenstein | 2008-08-24 01:16:52 | 2008-08-24 01:16:52 |
|  8 |  1424543768 | Benjamin Leiter  | 2008-09-11 08:58:08 | 2008-09-11 08:58:08 |
+----+-------------+------------------+---------------------+---------------------+

Hopefully this helps someone somewhere. I wrote it as a beginning tutorial for a friend on database design and administration.

On Multiple Projects

2 Comments

If you’ve ever been a contractor you too have faced the juggling act that is having many different projects in the air.  There is quite a bit of work that goes in to just managing those projects, much less actually doing the work on them.  I’m going to write a bit about how I handle them and manage to keep a semblance of sanity.  Right now I’m looking at my “in progress” list and I’ve got 10 projects open at various stages of completion.


Organization

One of the big problems with having so many projects going at once is that there is just no way that you can remember where you were with each project.  There are a ton of great online tools that I would recommend for keeping everything organized.

Before you start signing up for every organizational tool out there though, I would impart a tale of woe on you.  Starting a large project Powered By Geek wanted to make sure we crossed all the T’s and dotted all the I’s so we set out to use just about every organizational tool out there.  We had Basecamp, Campfire, Lighthouse, a Wiki, a large chunk of API documentation, Google Docs, and many more.  We actually had everything in a proper and logical place, however the organizational scheme that we used backfired.  Trying to get into the swing of a project and having to open up 10 different tabs and login to 10 different tools soon ended up with people getting lazy (I happened to be the laziest of them all) and just throwing documentation together into other tools, trying to consolidate it so I didn’t have to track it across other tools.  Things quickly got out of sync, and confusion starting happening between developers.  However once we had pared down the list of apps to just a few major ones and kept information in less places things started to move a lot smoother.  The moral of the story is to try to organize stuff as well as you can within a few tools that you are comfortable with, and then slowly integrate other tools if there is a need for them.

My MUST HAVE tools are really rather simple.  You need ticket tracking, milestone tracking, repository tracking, a place to write messages and warnings, and finally if you have more than two people a place where you can all get together and talk about the project(s).


Ticket Tracking, Milestone Tracking, and Notice/Messages

Lighthouse is a great tool and manages to fairly simply cover most of the needs of a small contracting house.  The ticket tracking is fairly easy to use (simple enough that you could give a client access and have them report the errors/changes themselves) and is also pretty customizable.  It also allows for milestones to be created and tickets assigned to them which makes it somewhat easy to see a break down of milestones and how far along you are on them.  Its message system is decent (I’ve been using it to write standards and code guidelines), I wish it stood out more but as long as people know to check it then it is useful.  Lighthouse only really does the ticket system well, the rest of the items it does just good enough that it makes sense to use the features rather than bring another tool into the mix, but it is fairly inexpensive.  The ticketing system does lack some somewhat standard features though such as parent/child ticketing and time tracking, but you learn to live without them (or in the case of time tracking you learn to just keep track of it somewhere else).  Since most of PBG’s projects are done on a task/goal basis the time tracking wasn’t that big of a deal for me.


Code Repository

As of right now, there is no better choice for a code repository than Github.  If you are managing multiple projects then I don’t think it is possible to live without Github.  If you aren’t using it, then I would seriously look at switching over.  The only drawback to it is their strange payment levels, because of the limit on private repositories you’ll often have to pay for a much larger package than you would need (it basically amounts to a dollar per private repository).  Other than the payment schema (which still isn’t very costly) it is probably the best investment in easily adding and managing your projects.


Chat System

Most of PBG’s projects are usually done with Lynn and I, so we just use Gtalk for our messaging system.  As soon as you add a third person however, you need a place to go to type questions and ideas where everyone can read them.  Preferably a place that is logged and allows you to upload attachments.  For this I would recommend Campfire the multi-line paste tool isn’t as nice as Pastie or Gist but it does allow you to fairly simply paste a chunk of code and have it easily readable by everyone.  It allows you to upload attachments or paste picture urls and have them show up in the chat.  It’s a great tool for talking to everyone (even if they aren’t on the same work schedule as you, since the chat log will always display the entire conversations).  It’s not all that pricey either, which means it’s great to try and see how everyone gets along using it.


Thought Process

With multiple projects it’s very easy to get overwhelmed.  If you have 5 different projects open in TextMate, 5 different terminal tabs running servers on 5 different ports and you jump between the projects fixing bugs and working with different people on the different projects then you’re likely to lose your mind.  Development requires your mind to warm up to your project.  The flow of one project is often vastly different than another project (an online classified site has a different mindset than a life streaming site), so set a side time where you focus on one project (a couple of hours, a day, a week, whatever) so you can get into the mindset of the one project.

Line your ducks up before shooting.  I’ve found that using the ticketing system to dictate where I need to go and what needs to be done works the best.  I make myself tickets for everything, from features that need to be completed, to a link that needs to be added on to the page.  Then once I’m in the flow of that one project I can burn through all the tickets quickly and efficiently.  The best part about using a ticket system is that you can think about the project any time you want, come up with a good feature idea and submit your self a ticket explaining it, then forget about it.  Trying to juggle ideas around your head will only lead to you missing something that you thought was wonderful and having a coworker ask you why the feature you thought was such a great idea was left out.

Write notes to your self in a centralized place.  Commenting your code and documenting what you’ve done is probably the one most noticeable failing in every developer I’ve met, and this is usually because the developer has all the commentary for the code written up in his head.  I know when someone asks anything about any of the projects I’m working on, I can usually tell them what file they need to edit, where they are looking, what the variables are named, what method is doing what, etc.  The problem with this is that the more projects that you are involved with the more likely it is that you’re going to get some things confused.  When I was doing 2-3 projects at once this wasn’t a problem, but when you start to get into the double digits this problem starts to show up.  My solution to this is to write little notes to myself and others explaining what is going on, similar to actually documenting my program.  This helps because now I don’t have to keep everything that is going on in memory and people don’t have to ask me questions that would be trivial to answer themselves if it was written down.  It is more work, and it’s dull work that seems to have no purpose, until you go back into a project after a month break and can just quickly read over your notes and get back into the swing of what you were doing and how you were thinking.

Try not to leave things hanging.  Unless you are working on something daily, you’ll want to try to avoid leaving things half completed.  Finish out tickets completely and get to a good stopping point before you close everything out.  That way you don’t need to remember exactly where you were at the next time you start up.  If it’s unavoidable and you absolutely have to stop half way in then write your self a detailed note on where you were at and your current thoughts on how to proceed.  This will make sure that you can pick up where you left off and hopefully put the finish on it before you have to stop again.



I hope these thoughts help.  With Rails development being cyclical in nature and companies managing multiple projects at the same time this issue is going to be cropping up more and more.

Development Communications Standard

No Comments

In the recent past there have been a ton of applications that I’ve followed almost religiously.  I’d read forum posts about how their software or service was coming along great, then they disappear off the face of the planet for months of a time, with no updates.  Lynn Wallenstein and I were talking last night and we decided that there should be a standard for development communications.  Right now it’s still in the early idea phase, but any input would be appreciated.

As a developer, especially one that works with insane deadlines, I know that you can’t always hit the mark.  As a corporate developer, I’ve also learned that if you aren’t going to hit a deadline it’s good to to just put it out there and let everyone know what is going on.  In the corporate world it is a necessity, plans need to be changed, milestones need to be rearranged.  With the rise of web based services, and small application development, it’s easy to say “It free or almost free, be happy that we’re doing it at all”, this is an understandable attitude, and one I’ve often shared, but it doesn’t help the people who actually use your service, and people who want to rely on your software.

Lately I’ve noticed development teams (especially Rails) have a very hot/cold development cycle.  Powered By Geek does it, as well as many others.  We have so many great ideas that we want to get done, and the minute we hit a good new idea, development on our other, on going, projects slows down.  Or we’ll get a client job that takes priority over the rest of our side projects, which moves all the different items we’re working on to the back burner.  Fairly often this is accompanied with little to no updates on progress, or even a mention that we’re going to be putting it on hold.

A development communications standard would lay the ground work for development teams to do public communications with progress.  This wouldn’t leave customers so frustrated when their favorite software mentions a linux version coming soon then disappears for 6 months, or a service that you’re really interested in stops almost all communications about the project.

I personally love the idea, both as a developer and as a consumer.  As a developer, more communication with your core audience, letting them know when things are going to be done and what is going to be done, is a good practice.  As a consumer, knowing when things are going to be happening and what to look for would resolve the disappointment I feel when I look at a blog entry about a feature or product I really want, only to see that it was dated 6 months ago and there hasn’t been an update since.

SVN & GIT

2 Comments

With the recent change over to my Mac, I’ve started to really take a look at the differences between SVN and GIT.  I’m still in the process of learning how to use both bundles for TextMate so when I’m actually in “work mode” I’ll usually forego playing around and just use the command line and I’ve noticed some definite differences.

SVN

SVN is actually what I’ve been using the longest, and am much more familiar with.  However using it from the command line is something I’m not very familiar with because up until my switch to my Mac I have always used TortoiseSVN which is probably the best implementation of an SVN client out there because of it’s integration right into the Explorer shell.


Some of the pros of SVN:

  • Output is much more descriptive
  • Without branching it is completely straightforward
  • A lot of documentation out there to help you get started
  • Easier to move between old revisions

And, of course, some of the cons:

  • Slow and clunky feeling
  • Hard to setup initially
  • Pain to fix merge errors
  • Branching is a chore
  • Require an “always on” server/connection


Pros In Depth


Descriptive output

By default, committing, updating, checking out in SVN yeilds very clear and easy to read output.

Macintosh:upillar_motors danahern$ svn up
U    app/models/notifier.rb
U    app/views/notifier/invite_users_private_en.rhtml
U    app/views/notifier/question_for_seller_en.rhtml
U    app/views/notifier/reply_to_buyer_en.rhtml
Updated to revision 5776.

You get a clear understanding of what it is doing every step of the way.  With GIT the updating is a little harder to understand

Macintosh:truth danahern$ git pull
remote: Counting objects: 17, done.
remote: Compressing objects: 100% (10/10), done.
Unpacking objects: 100% (14/14), done.
remote: Total 14 (delta 4), reused 14 (delta 4)
From git@github.com:lwallenstein/truth
ecd1c5c..73c70a0  master     -> origin/master
Updating ecd1c5c..73c70a0
Fast forward
README.txt |   62 +++++++++++++++++++++++++++——————————–
1 files changed, 28 insertions(+), 34 deletions(-)


Straight forward workflow

As long as you aren’t branching SVN’s workflow is a lot more straight forward than GIT.  You have your server repository which is the definitive copy, then you have your local copy that you work on and then check files into the server repository.

Tons of documentation

Both have been around for a long time, however GIT has only recently really hit the limelight as the new choice for code repositories.  SVN has oodles of documentation out there, and while this won’t be a factor for much longer, GIT is still catching up with regards to documentation explaining the how’s and the why’s.

Moving between revisions

This might be a short coming on my part, but SVN feels a lot easier to move between revisions.  You do a little research to see what revision you want to grab and then just do an update with a “-r <revision number>”.  I haven’t yet gotten the hang of reverting to previous revisions with GIT. Thanks to Pieter, I’ve found out that reverting in GIT is just as easy as SVN (In fact far easier when using Github). git checkout


Cons In Depth


Slow and Clunky

SVN really doesn’t excel at giving me the feeling that it’s actually doing something.  It feels slow, committing a large file or groups of large files leave you sitting at “Transmitting Data.” forever.  No status indication, which means you don’t know if you should sit there and watch, or maybe go grab a bite to eat, enjoy some TV then come back and hope it is done.

Hard to setup initially

I haven’t really ever had to do this by my self, but from the experiences I’ve had with others as the developer waiting patiently to start a project or drop an already alpha product into subversion this seems to be a time consuming and difficult process.  I can’t go into the details on what went wrong each time, but I do know that by the end of the process everyone involved was frustrated and praying that there would be a better solution one day.  It should be noted though that once it’s up, as long as you don’t change anything, it stays up pretty solid.

Pain to fix merge errors

Merge errors are a way of life when working with others on a code repository.  The merge errors in SVN are a pain to fix.  Not only does SVN leave the file name with extra characters with the files diff’d into the file name, but it generates 3 different versions of the file and sticks those in there as well.  In reality, one of those solutions or the other would make more sense as a standalone.

Branching in SVN

Branching in SVN is actually so much of a pain, that rather than creating a branch we’ll either create a new repository or just say “screw it” and make a backup copy of the repo and just start committing.  Once you use git’s branching there is no way you could actually stand to go back to SVN.

Require an always on connection

This hasn’t been a real problem for a while, however, on the occasions where our SVN server goes down or for some reason or another I can’t connect to it all development stops.


GIT

Git is the relatively new to me versioning software.  I haven’t checked out any of the GUI based interfaces for it yet (and at a glance it would appear there aren’t many), but the command line version has been treating me right for a while now.

Some of the pros of GIT:

  • Everything is faster than SVN
  • Branching is a breeze
  • Github
  • Merge is amazing

Some of the cons:

  • Hard to understand what is going on
  • Github’s pricing system
  • No really good UI
  • Few web based tools

Pros In Depth


Everything is faster

This might be just a perception, but everything on git seems faster than SVN.  From checkout, to merge, to commit, everything seems to speed along on git.  It could be just that it’s more verbose, you can see it actually working instead of sitting there pondering

Branching

My heart actually expands three sizes at the thought of how easy the branching is to handle on git.  So many projects I’ve worked on would benefit greatly from being able to branch off and work on a big revision.  The problem that’s held me back so far is that SVN is such a pain to branch, I look forward to getting to branch a lot more.

Github

Github is a great tool.  I would almost wholeheartedly recommend it, and if you only have a few things you’re working on I do in fact recommend it from the bottom of my heart.  It is amazingly simple to setup, and if you see a public project you’d like to patch it’s actually impossible to mess up checking out, and commit a patch. There are some problems with it, which I’ll expound upon in the con section.

Merging

Git is infinitely better at handling merging when compared against SVN.  I’ve actually never had a bad merge issue with git.  It’s also far faster than the SVN merge.  It’s a win all around.


Cons In Depth


Hard to understand

Git’s pro is also a con.  It’s very verbose, it tells you exactly what is going on.  It’s hard to understand what it is doing at every step.  Especially when compared to the simple output of SVN.  (To see an example scroll back up to the SVN Pro)

Github’s pricing schema

No one knows better than I do the value of making a buck off your hard work, and I would be more than happy to pay for such a wonderful service as Github. The problem I have with Github’s pricing schema is the limit on space or projects.  However, the way it is setup is based on public/private repos as well as space, this means that because of the limit on private repositories you can’t post all your private client work.  Powered By Geek contracts out to a LOT of companies, and if we want to host all our repos with them then it would cost us a small fortune.

No good UI tool

As of right now there isn’t a good graphical user interface for interacting with git repositories.  SVN has been main stream for a while longer and has had more time for tools to develop, and between Versions on the Mac and Tortiose on Windows. Right now the Git GUI tools are very basic, and not really useable.

Few web based tools

There aren’t that many web based tools out there for viewing repositories.  The SVN tools are pretty mature and refined, however, the git tools aren’t there yet. From the little I’ve read, the git API tools are still developing.  So I don’t think this is going to be a con for very long.


Final Thoughts

I think for our new projects GIT makes a lot more sense than SVN. Since I’ve been working with git with the command line and have gotten used to it I can’t really imagine moving back to SVN. Git really seems like the better tool, but it needs to mature a little more, I think shortly there will be no real con to using git as opposed to SVN.

Phone Conversions

2 Comments

I’ve become a fanboy.  I keep telling myself no, but slowly I’m coming to terms.  Last week I picked up an iPhone (bought off ebay so I wouldn’t have to extend my ATT contract).  So I’m going to list some thoughts, and I’m going to compare it to my steallar little Nokia E70 that I had before.  This should hopefully be both similar and contrasting to something that a good friend Michael wrote here on his phone changing experiences.

Let’s start with the Pro’s of the iPhone:

  • Beautiful large display screen
  • Novel multitouch navigation
  • Accurate GPS
  • Almost Fully Functioning Web Browser
  • Nice Looking Applications

The Screen

The screen is on par, if not better, than the screen on iPod 5th Gen that I have, and at over double the size it’s amazing.  I’ve been watching Dr. Horrible on it and just amazed at the clarity (this also really gets me interested in buying movies and shows off iTunes, except for their crappy interface for letting me download my content onto multiple computers). Applications look great and you can see details almost on par with my 1920×1200 Mac Book Pro.

The Multitouch

The multitouch is a neat thing to play with for a while.  Similar to DS stylus, this can add a lot of functionality for some stuff and can make for frustrating user experience for other stuff.  Moving around Vay (http://phobos.apple.com/WebObjects/MZStore.woa/wa/viewSoftware?id=284940607&mt=8) a remake of a Sega CD RPG is horrible, while moving around Google Maps is great.

GPS Functionality

The GPS functionality almost makes me want to drop the money for an iPhone SDK and learn Objective C, because it’s accurate enough that you could make a lot of neat apps for it (and looking at the app store people are already starting).  This is my first time having built in GPS in my phone (my car has an awesome GPS system in it) and just imagining the possibilities of things you could do with it has me excited to see what other developers can come up with.

Safari in your pocket

This would be the one thing I think that makes it worth while to get an iPhone.  Although the Nokia’s browser is pretty good, some of the stuff it does can really ruin a web experience (not rendering CSS right, sometimes barely rendering anything at all).  As someone who does web development for a living, and whose secondary job is just keeping up with all the new developments and sites coming out on the internet (Lynn keeps me well informed), this is almost a necessity.  I don’t think I could ever go back to an experience less than the standard the iPhone has set, the large screen, speedy connection and near full featured experience has really made me question how I could live with out it.

Look at the pretty little app

The application designs for most things, looks great.  I’ll talk about the cons of the UI a bit later.  The applications with reflective surfaces and easy to understand buttons.  Every app has a very iPhone feel to it, and you are fairly comfortable moving between apps without the bone jarring feel of a completely different layout, and completely different icons.

Now on to the cons:

  • Text entry sucks
  • The UI layout is pretty bad
  • Working with iTunes and multiple computers is impossible
  • Just using touch controls is irritating

Text Entry or how the helk van u tuoe on thus thomg

Coming from the Nokia E70 with it’s flip out real QWERTY keyboard to the iPhone’s virtual keyboard was and still is a real challenge.  Typing on the Nokia had the feel of a real key being depressed and the keys were big enough to type with my thumbs, typing on the iPhone has no tactile feedback and the touch screen makes for some pretty inaccurate key presses when trying to use it to type in the vertical position.  I’ve read enough about it to know to just type the words you’re trying to get and the iPhone will hopefully figure out the word you’re typing and replace it when you hit space.  That works somewhat, however, as someone who prides himself on not using all the acronyms that have sprung up, and who tries to spell everything right, this is annoying when you type in a full word, only to realize that the iPhone has no idea what you’re typing.  Even when everything is working perfectly and it’s correcting my typing mistakes I still type slower than on the real Nokia keyboard.

The UI

The UI on the iPhone really annoys me, menu’s are the worst offender, where you have to hit the upper left corner to go back to the previous level.  The problem with that layout is normally I’m using my right hand to hold and navigate my phone, so I have to move my thumb up to the top of the phone to press the button or I’ll have to bring a second hand into the action, with the phone laying down this isn’t a bad thing, but I hardly ever use the iPhone on a flat surface.  Other things about the UI bother me, how it takes more than one key press to bring up certain things like voice mail.  The whole SMS process, while neat that it displays it in a chat like format, is extremely hard to use and figure out.

iTunes and multiple computers

While I generally like iTunes for managing content on my iPod the iPhone is a different story.  Unbeknownst to me, who figured it would work just like an iPod, I setup my iPhone on my MBP laptop, intending to also set it up on my home desktop since I tend to keep only one copy of my downloads and then just connect the device to the computer that has what I want.  Once I got home with the iPhone and attempted to set it up to my desktop I found out that I couldn’t actually access my items on my desktop without first syncing it to my desktop and losing everything, which would mean that I would no longer be able to sync it to my laptop.  This is annoying because this system has worked for almost a year with my iPod with no problems.  Now I have to make sure that my PC and Mac sync up and share their iTunes libraries so that I can access all my content, not a major problem, but a giant inconvenience.

Only Touching

This is probably a giant limitation to the iPhone.  I understand they were going for the sleek and elegant look and didn’t want a bunch of buttons and knobs and whatnot on their phone, however this limits the iPhone to just the touch based approach, and honestly somethings would be a lot better with a different mode of input.  What I’m imagining is actually similar to the DS, where you would have a little directional pad and maybe a button or two.  This would allow you to navigate certain programs (like the RPG mentioned above) that aren’t conducive to the touch only mentality.

Overall

The iPhone is a neat gadget and some things like the web browser really make an experience for me.  However I think that I might be happier with my Nokia as a phone and an iPod Touch as a web browser (if I could get one that would connect to the internet via my Nokia’s bluetooth connection and use the GPS functionality of the bluetooth GPS receiver).  It’s a nice phone and well designed, but I feel like there are some fundamental flaws in the way it does things and the things that it does really well have nothing to do with actual phone functionality.  I’m going to stick with the phone for a while to see how it fares after more than a week’s use, I’m hoping I’ll grow to understand the reasoning behind some of the things I think of as flaws, and if not, then at least there are some Android phones down the pipeline, not to mention some really cool stuff coming out of Nokia.

Older Entries