Thoughts, guides, explorations.

Archives

This Blog is the work of Michael Latta (technomage.com, @Techno_Mage).

The content is as much to get thoughts on paper, as to be read, and a little in hopes of helping someone walking a path similar to mine.

A New Blog

Years ago, I thought having my own site implemented by myself would show sills and value to someone reading it. The problem is that it required more work than I had time to maintain, and more effort to publish posts than with a pre-built system like Ghost.

So after years I have abandoned the custom site and moved the posts to this blog on Gost(pro). In a few months/years I may move it again. Ghost(pro) made it very easy to get this setup in a few hours, including migrating posts, and customizing the theme.

Only time will tell if this gets me to write more, once project schules again impact my time.

Adding an automatic build to a Frank test setup

Frank is an automated test framework for iOS applications. It integrates with cucumber to allow acceptance testing of iOS apps. Mostly it is used with the simulator as reliable launching of apps on a device has not yet been achieved outside XCode.

The following adds an automated build prior to running the cucumber tests.

%x{xcodebuild -workspace #{APPNAME.gsub(/Frank$/,'')}.xcodeproj/project.xcworkspace -scheme #{APPNAME} -configuration Debug -sdk iphonesimulator build}

The xcodebuild command is part of Apple's developer tools, and provides command line builds. The %x{} syntax in Ruby executes a system (shell) command.

The -workspace flag indicates what workspace is used for the build. This flag is needed to place output in the same location as XCode 4.x uses for its builds. The #{APPNAME.gsub(/Frank$/,'')} computes the name of the project from the target name which is set in APPNAME in default Frank setups, within the env.rb file. Thus if the project is Foo and the target is FooFrank this computes Foo for the project name. The workspace being used is the default project specific workspace within the project file itself. This should match most single project cases. If you have multiple projects in one workspace you may need to change this to match your created workspace file.

The -scheme flag tells xcodebuild which scheme to use for the build which also selects the target being built.

The -configuration Debug selects the Debug build configuration, and -sdk iphonesimulator uses the latest sdk version in the simulator.

The final build on the command indicates that the build action for xcodebuild to perform is 'build'.

Crawlable Javascript Pages with KnockoutJS

For sites or applications that wish to have their pages visible in search engines, or accessible by browsers not using javascript (screen readers for example), it is important to be crawlable. This means the server can produce pure html output that represents the content of the page or application. In modern client (javascript) focused projects this takes a bit of effort.

Google has defined a standard for using HTML4 which allows the server to render page content specific to hash values in links (#anchor). Normally these hash values are not sent to a server as they are supposed to be internal to the page, for navigation purposes. But, in HTML4 browsers these are also used to initiate javascript updates to a page, resulting in unique content for each hash tag. In HTML5 there is a new history API that allows javascript to alter page contents based on normal urls, rather than needing hash tags to indicate new content. In this case the browser simply needs to be able to render all urls the application or page uses in all its links, even if they would be generated in javascript when used within an HTML5 browser.

In this post I will cover the use of HTML5 history API to present a dynamic site, while still supporting server side rendering of all content for crawling, and while defining the content only once. KnockoutJS is used as the client-side javascript library for this post, as it is low overhead, and approachable, while keeping most of the focus on the content. For the server side I will be using Rails 3.1, but the technique applies equally to any server or client frameworks.

The Components

For this post I will be presenting one page in detail and describing how the system works for multiple pages. This site uses this technique and can be viewed as a larger example.

  • Server Side Controller
  • Server Side Layout
  • Server Side View
  • Client Side Logic
  • Server Side Routes

The general technique is to render the content on the server into a string. This content is then either rendered either into the overall HTML page when served by the server, or inserted into the page when navigating using javascript. The History API is used to provide smooth navigation on the client side. A variation on this technique is to render all pages into javascript templates on initial page load allowing fully autonomous navigation between pages. For relatively small static sites this would work, or sites that are largely data driven where the templates would be small, and per-page data fetched as part of the navigation.

For initial page load you can either render the content on the server then re-render on the client for consistency or only render pages on navigation in the client. In the case of Knockout bindings are established on page load and so this example re-renders the content after page load to allow the bindings to operate automatically after each navigation.

The Server Side Controller

The server side rendering of content relies on the ability to render content as HTML to a string, then use that string as either JSON for an AJAX request, or directly into an HTML response to a direct request.

class HomeController < ApplicationController
  ...
  def index
    @page_title = 'TechnoMage'
    @page_tag = 'home'
    html = render_to_string(:format => :html,
      :file => 'home/index.html.erb', :layout => false)
    respond_to do |format|
      format.html { render :html => html }
      format.json { render :json => json_for_content(html)}
    end
  end
  def about
    @page_title = 'TechnoMage - about'
    @page_tag = 'about'
    html = render_to_string(:format => :html,
      :file => 'home/about.html.erb', :layout => false)
    respond_to do |format|
      format.html { render :html => html }
      format.json { render :json => json_for_content(html)}
    end
  end
end

When rendering content to JSON some additional information is passed from the server to update the page. The page title is updated on each navigation, and the current page tag is passed so that any navigation menu can be updated. When the content is rendered to HTML these values are still used when re-rendering using javascript, but not directly used when javascript is disabled or not processed when viewed by a crawler such as a search engine.

class ApplicationController < ActionController::Base
  ...
  def json_for_content html
    {:cur=>@page_tag,
      :page_title => @page_title,
      :content => html}.to_json.html_safe
  end
  helper_method :json_for_content
end

The Server Side Layout

In Rails when a page is rendered to HTML a layout is used to render common elements of all or related pages. In this example the layout provides the navigation menu and handles creating the Knockout bindings for content and setting up the History API to provide javascript based navigation.

You will notice in several places in this layout that the server side rendering is substituting in data for things like the page title, paths for link's href, and the classes to apply to navigation elements so the current page is marked as current. These all support access by crawlers and javascript disabled browsers or screen readers to present complete content. In addition to these static elements there are KnockoutJS bindings to allow dynamic update of these elements to support javascript based navigation updates. In the case of page title since that is in the header, we alter the page title explicitly upon navigation.

<!DOCTYPE html>
<html>
<head>
  <title><%= @page_title %></title>
  <%= stylesheet_link_tag    "application" %>
  <%= javascript_include_tag "application" %>
  <%= csrf_meta_tags %>
</head>

<body class="blog" data-bind="attr: {class: cat}">
  <div id="topbar">
    <a href="<%= root_path %>"
      onclick="return show_home();"><div id="logo"></div></a>
    <div id="nav">
      <%= link_to "Home", root_path,
                  :class=>"first #{'current' if @page_tag=='home'}",
                  :onclick=>"return show_home();",
                  :"data-bind"=>"css: {current: isCurrent('home')}" %>
      <%= link_to "About", about_path, :onclick=>"return show_about()",
                  :class=>"middle #{'current' if @page_tag=='about'}",
                  :"data-bind"=>"css: {current: isCurrent('about')}" %>
      ...
    </div>
  </div>

  <div id="content" data-bind="dom: rendered_content()">
    <% @content = yield %>
    <%= @content %>
  </div>
</body>
</html>

<script>
  jQuery(function() {
    // Copy the html into KO data and re-render to allow KO to modify
    // it in the future, while still falling back on plain html if
    // js is not enabled
    js = <%= json_for_content @content %>;
    ko.mapping.fromJS(js, {}, model);
    ko.applyBindings(model);
    //
    // Handle history changes
    //
    var History = window.History;
    if (History.enabled) {
      History.Adapter.bind(window,'statechange',function() {
        var state = History.getState();
        load(state.url);
      });
    }
  });
</script>

The Server Side View

The server side views are just normal HTML views in Rails. They contain no special content for this technique.

<h1>Home</h1>
<p>This is a sample page view containing just HTML.</p>

<h1>About</h1>
<p>This is a sample page view containing just HTML.</p>

The Client Side Logic

The client side logic to navigate to a page is presented below. It uses the history.js library which provides a cross-browser interface to the HTML5 history API while accounting for various browser specific differences.

The main function is the goto function that navigates to a new URL using the history API. There are helper functions for various page URLs based on the rails named route helpers which are statically generated into the applicaiton.js.erb file as part of the asset pipeline support in Rails 3.1.

function goto(path) {
  var History = window.History;
  if (History.enabled) {
    History.pushState(null, null, path);
    return false;
  }
  return true;
}

function show_home() {
  return goto('/');
}

function show_about() {
  return goto('/about');
}
  ...

The client side logic to respond to a url and present the contents is presented below. It uses the history.js library which provides a cross-browser interface to the HTML5 history API while accounting for various browser specific differences.

The History API calls the callback in the layout upon receiving a request to navigate from the above goto function. This ensures that the URL bar of the browser, the presented content, and any bookmarks are all in sync. That callback calls this load function to actually load the content for the requested path. This function then uses an AJAX request to get the JSON version of the content and supporting data from the server, then uses KnockoutJS to update the page's DOM, and then the page title. Note that in this case we replace the root path '/' with '/home' so the .json extension is properly formated. This requires that the server be configured to reply with the same results for both '/' and '/home'.

function load(path) {
  if (path.match(/\/$/)) {
      path = path + 'home';
  }
  jQuery.get(path+'.json', function(data) {
      ko.mapping.fromJS(data, {}, model);
      document.title = model.page_title();
  });
}

The Server Side Routes

The final component is the server side routes that map URLs to controllers and actions. The below supports the sample pages used above for home and about actions. These are standard Rails 3.1 and only included for completeness.

get "about", :to => "home#about", :as => :about
get 'home', :to => "home#index", :as => :home
root :to => 'home#index', :as => :root

Recap

This post provides a complete set of Rails 3.1 server side code, and required client side code to use the KnockoutJS and history.js libraries on the client to create a site that is both crawlable by search engines, supports raw HTML as a fall back or to support applications like screen readers, and when javascript is available presents a seamless navigation model between pages for HTML5 browsers.

While history.js does provide support for hash tag based javascript navigation on the client, it greatly complicates the life on the client and server to support this style of URL and navigation. Google has defined a means for sites to use hash tag navigation and still provide search engines content to index. It is overly complex but should be considered if you need to support HTML4 and search engine crawling. History.js also provides a means to support HTML4 without the google protocol which is cleaner than the google option, but still more complex than the HTML5 version presented here.

The Value Of Experience

I had a conversation for a potential engagement, and was asked "What leadership do you have to offer to someone with 10 years experience?" This is a very good question. As a senior developer with over 30 years of experience I need to be able to articulate the value I bring to an engagement, and many business or even technical people may believe that with 10 years of experience developers know what they need to know.

I do not expect to need to tightly direct a group of developers with 10 years of experience each. They should be quite capable of working independently. There are two areas where I have seen consistent value contributed, above technical abilities, that may not be as present in any given team, and where I contribute value: culture and judgement.

Culture

In any development group there is a culture. Some group or company cultures follow traditional cult approaches: 1) we vs. they, 2) leaving is failure, 3) we know better. While these create a sense of closeness, they have many drawbacks.

The cult like cultures result in a stagnant culture which, once established, never changes. While such practices can lead to team cohesion at the beginning of a project, in the long run it has several negative effects:

  1. it tends to exclude new members, by its nature those that do not conform or agree with the cult prior to joining are rejected,
  2. it tends to reject new inputs due to the input being foreign to the existing culture,
  3. it does not recognize mistakes, or values contrary to the doctrine of the cult.

A recent example of this in the Ruby community is the prevalence of developers to look down on other developers that use a different editor than they have found productive. In particular the VIM sub-culture is very vocal, and often indicated by a quick query in an interview about which editor does an applicant use. There is nothing wrong with enthusiasm for a tool that is useful, but when it becomes a gating condition on evaluating a candidate's value, a cult mentality has set in.

What senior developers like myself contribute in this area is having worked in many different groups. We have seen enough different developer cultures to have an idea about what culture is forming in a group. The better senior developers contribute to the building of a culture which supports an openness to ideas, to being adaptable, and to embracing change.

Agile/XP type processes strive to build software that is resilient and open to change, the group process and culture needs the same focus. Just as software needs to be refactored to incorporate new inputs and requirements, the team dynamics and culture needs to be reexamined periodically and updated to meet the new and changing organizational requirements and to incorporate new experience and industry inputs.

Judgement

While I would hold it true that technical judgement improves with experience, I would assert this is more true if that experience is varied and shows diverse domains. Twenty years in the same industry or the same job clearly is not as valuable as twenty years in different industries, and in different roles. A developer that knows many languages is well accepted to be more valuable than a developer that has only worked in one language, because their thinking is more varied and their approach to any problem can be richer. The same is true of developers that have worked on many types of projects in many types of organizations, and in many different roles. They bring a richer set of experiences.

In addition to technical judgement, there is a critical aspect to judgement that in my experience forms fairly late in many developer's careers. This is the weighing of business value vs. technical value. This must include consideration for the actual individuals doing the work, and the time required to deliver the business value. This is a complex decision making process.

  1. Technical: risk, effort required, and team skills.
  2. Business: value, revenue, expense, and impact on customers.
  3. Opportunity cost: what could the same effort produce in other areas of the project?

Recent processes attempt to deal with these factors in prescriptive ways. They establish rules to deal with these issues. This has drawbacks as covered in the culture section, when applied too dogmatically. These rules are most definitely the attempt by senior developers to codify their judgement, but such codification is never as accurate as a person, or persons, with good judgement.

It is good to have language around this area such as "technical debt", or "refactoring", or to include customers in the decision making process. Having language for a practice increases the cultural awareness of the practice. Ultimately a person with good judgement and a good understanding of both technical and business aspects is the best option.