I have a terrifying confession to make: Until very recently, I didn’t play video games.

That may not be particularly appalling to many of you, but for someone in my industry, that’s pretty embarrassing. Game designers are an integral part of the interactive design community, and I’ve always felt an affinity with them, despite not really knowing their territory. In fact, one of my favourite blogs is written by Clint Hocking, a former creative director at Ubisoft (and high school friend of mine). But I have long been more than a little sheepish about publicly admitting to not having a clue about gaming, aside from what I’ve read on Clint’s blog.

That said, it’s always educational to be late on the uptake with a particular technology; as a web designer I feel like it can help me empathize with the people using my designs who might not spend even a fraction of the time I spend online. I’ve learned a few things from my experience that feel particularly relevant to the work we do with our clients.

First, I had valid reasons for not playing video games until now. I didn’t have access to a console; I didn’t feel like spending money on one until its value had been shown to me; I had seen a lot of games that didn’t appeal to my interests and sensibilities; and perhaps most fundamentally, I was so green that even working the controller was daunting to me — that is, the basics of navigation and control felt uncomfortable and clunky. All of these practical resistance points are also true for many website users; even now when those of us in the industry refer to ourselves as “veterans” and to the up-and-coming generation as “digital natives,” there are still huge swaths of people on the planet who simply don’t need to use the internet to get through their days — as my uncle Ken, a farmer & logger in Northern Ontario, reminds me regularly. For those people, the only thing that’s going to make them interested in the technology is for it to become relevant to their lives.

It wasn’t until I discovered a game that I truly fell in love with (more on that in a moment) that I became invested enough to want to work through my painful awkwardness with the controller and my hesitations about the financial investment. (I haven’t touched on the time investment required, which is significant — this was another fear I had about games, and one that proved well-founded as I have found myself losing all track of time while immersed in the gaming experience.) But once I found a game I connected with, the obstacles fell away and my desire to interact with the game overcame all resistance. Us web designers are fond of parroting the maxim that “Content is king,” but we haven’t always been good at practicing what we preach — this has been a good reminder to me that what’s inside really is what counts. And just as all websites aren’t created with equal amounts of attention to content, neither are all video games.

The game that stole my heart is Assassin’s Creed II, an action-adventure game set in Renaissance Italy. Your character is a young assassin from a noble family who is avenging a betrayal of his family — but it’s actually far more complicated than that. You’re actually a modern-day guy who’s been projected back into the body of this 15th-century assassin, and you’re trying to solve a mystery involving the Knights Templar that traverses centuries. In the process, you travel through the gorgeously-rendered streets of Florence, Venice, and points in between, running over rooftops, climbing church towers, dodging guards, exploring secret tombs, hiding in crowds, stealing treasure, and oh yes — hanging out with Leonardo da Vinci. There are puzzles to solve, swordfights, thieving missions, and even some economic development to manage in a small town where your uncle’s villa is located.

The premise itself appeals to me more than some other games, but the real genius of Assassin’s Creed II lies in its execution. Game designers clearly put an enormous amount of thought into human motivation, and there’s something in this game for just about anyone. The rewards that you earn over the course of the game are a mix of financial (good old-fashioned cash), status (your armour and weapons get progressively better — and pricier — as you progress), relational (you meet new characters & gain access to allies) and intellectual (solving puzzles). It’s been interesting to observe my responses to the various types of rewards and see how they correspond with my real-life predilections. I think the real hook for me has been the mystery-solving part, because it lends an over-arching mission and urgency to the game that I have only ever felt when reading a complex and layered page-turner of a novel.

I honestly never imagined that a video game could make me feel so immersed, and provide such a strong emotional connection. Now that I’ve had this experience I’m far more likely to explore the rest of the gaming world and see what else is out there. But it’s also got me thinking about the factors that shift behaviour and that motivate us — which makes me further resolved to focus my summer reading on those topics. I’ve picked up Switch and Made to Stick (both by Chip & Dan Heath) but I welcome other suggestions.

• What is CKAN?
• CKAN’s API
• The Foundation
• The Build
• Theming
• Homepage Chart

Caching

API calls are expensive. There’s no doubt about that. Particularly when you’re returning large amounts of data. To avoid any issues of the CKAN API being exhausted from requests and to ensure that the site remained responsive, I decided to leverage Drupals caching mechanisms and pretty much cached everything I could, within reason. The Chart, Tag Cloud, Tag lists, Ministry lists, All Packages list and all individual packages are cached. The issue with caching on this site is that if a package gets updated on the CKAN instance, we need to know about that on our Drupal site immediately and then clear the appropriate caches so that the most recent data can be retrieved.

For caching I created a table called ‘cache_ckan’, that stores everything I need. To create this table I used the schema of the existing cache table and put that in my .install file in my module directory.

/**
 * Implementation of hook_install().
 */
function ckan_install() {
  drupal_install_schema('ckan');
}
 
/**
 * Implementation of hook_uninstall().
 */
function ckan_uninstall() {
  drupal_uninstall_schema('ckan');
}
 
/**
 * Implementation of hook_schema().
 */
function ckan_schema() {
  $schema = array();
  $schema['cache_ckan'] = drupal_get_schema_unprocessed('system', 'cache');
  return $schema;
}

Whenever this module is enabled this schema will be run and the table will be created.

What is stored in the ckan_cache table?

There are various items stored in the cache table.

1. The Homepage chart data
2. Tag lists
3. Ministry lists
4. List of all datasets

Let’s take the list of all packages as an example. I covered how I implemented the paging in my previous post. As this list is paginated it’s important that every page be cached to improve the speed of the site. As the paging mechanism is already implemented it’s just a case of creating a cache table entry (ckan:all{page-number}) for each page, and then checking for it’s existence when loading the page.

if(($cache = cache_get('ckan:all'.$page, 'cache_ckan')) && !empty($cache->data)) { // If cached data exists for this page...
	$results = $cache->data;
} else {
	$ckan = ckan_ckan();
 
	$start = 0;
	$items_per_page = variable_get('ckan_items_per_page', 4);
	if($page) {
		// If we're in a page, we need to set where to start the list
		$start = $page * $items_per_page;
	}
 
	// Set the offset to the number of records in
	$offset = $start;
	// Limit to the number of items per page 
	$limit = $items_per_page;
 
	try {
		$results = $ckan->advancedSearch(array('groups' => 'canadagov', 'all_fields' => '1', 'offset' => $offset, 'limit' => $limit));
	} catch (Exception $e){
		return $e->getMessage();
	}
 
	// If the API call worked
	watchdog('ckan', 'Called CKAN API for list of all packages');
    	cache_set('ckan:all'.$page, $results, 'cache_ckan');
}

This method is very simple and very effective. It means the pages load lightning fast and only one page of data at a time is retrieved.

How does the cache get cleared/updated

Datasets/Packages change all the time on the CKAN instance, so how do you make sure that the Drupal site has the most current data. This module has two ways of managing that.

1. Using hook_form to redirect to CKAN

As the CKAN nodes on Drupal are created on the fly and hold very little information, there is really no need to access the EDIT form for these nodes. Whenever an admin user clicks the edit tab on the node, they are automatically redirected to the appropriate CKAN package editing screen. hook_form is called to retrieve the form that is displayed when one attempts to “create/edit” an item. For CKAN content types, the user is redirect to the CKAN instance.

/**
 * Implementation of hook_form
 *
 * Redirect the user to ca.ckan.net package edit screen on edit
 */
function ckan_form(&$node, $form_state) {
  if($node->type == 'ckan') {
  	drupal_goto('http://ca.ckan.net/package/edit/'.$node->body);
  }
}

When the CKAN form is submitted, CKAN then redirects back to the Drupal site and calls a specific URL that tells Drupal to call CKAN again to get the package information and populate the node. To clarify, the process is

1. Redirect http://www.datadotgc.ca/node/X/edit to http://ca.ckan.net/package/edit/{name of X}
2. On save of CKAN Package, redirect to http://www.datadotgc.ca/{special_url}/{name_of_X}
3. Load the node with {name_of_X}
4. Call CKAN to get the (updated) data for Package {name_of_X}
5. Save the node with updated data

Using Cron and an Atom Feed

CKAN provides an Atom feed of recent updates to the Packages. Cron checks this feed every time it runs. If the feed has changed since the last cron run, then we know there have been updates and we clear all of the caches.

/**
 * Implementation of hook_cron()
 *
 **/
function ckan_cron() {
	// Get the md5sum of the current atom feed
	$current_feed = trim(md5_file('http://ca.ckan.net/revision/list?format=atom'));
	watchdog('ckan', 'Current feed md5: '. $current_feed);
	// Retrieve the previously stored md5sum
	$previous_feed = variable_get('ckan_atom_feed_md5', $current_feed);
	watchdog('ckan', 'Previous feed md5: '.$previous_feed);
 
	// If there have been changes
	if($current_feed != $previous_feed) {
		watchdog('ckan', 'ATOM feed has updated, clearing caches and deleting nodes');
		// Flush all the caches
		cache_clear_all('*', 'cache_ckan', TRUE);
  	        // Set the previous feed md5
		variable_set('ckan_atom_feed_md5', $current_feed);
	}
}

Tag cloud creation

I borrowed some code from the Tagadelic module to achieve the tag cloud

/**
 * Build a tag cloud based on the settings provided
 *
 * @return	String	A themed list of weighted tags
 */
function ckan_tag_cloud() {
	// If there is cached data
	if(($cache = cache_get('ckan:tags', 'cache_ckan')) && !empty($cache->data)) {
		$results = unserialize($cache->data);	
	} else {
		$ckan = ckan_ckan();
		$results = $ckan->getTagCount();
		watchdog('ckan', 'Called CKAN API for tag cloud');
		cache_set('ckan:tags', serialize($results), 'cache_ckan');
	}
 
	// Let's sort them by weight first off
	foreach ($results as $key => $row) {
    $tag[$key]  = $row[0];
    $weight[$key] = $row[1];
	}
	array_multisort($weight, SORT_DESC, $results);
 
	// Now let's get the top X number of tags
	$results = array_slice($results, 0, variable_get('ckan_tagcloud_total', 40));
 
	// Now build the tags
	$tags = ckan_tag_build_weighted($results);
	// Sort them
	$tags = ckan_tag_sort($tags);
	// Theme them
	$output = theme('ckan_weighted_tags', $tags);
	return $output;
}
 
/**
 * Theme function that renders the HTML for the tags
 * @ingroup themable
 */
function theme_ckan_weighted_tags($tags) {
  $output = '';
  foreach ($tags as $tag) {
    $output .= l($tag['name'], 'data/tag/'.$tag['name'], array('attributes' => array('class' => "tagcloud level".$tag['weight'], 'rel' => 'tag'))) ." \n";
  }
  return $output;
}

Using the CKAN Search API for all lists

Ok, so what’s this all about? CKAN has some nice API calls like /api/rest/package/PACKAGE-REF that return a list of Packages. However these return the name/id of the Package ONLY. In our case, for our listings, we wanted other data, such as the tags attached to the Package as well as a brief description.

The only way to get this data was to do a search API call /api/search/package and pass some extra parameters, in this case all_fields=1 and department={name of Ministry}.

all_fields=1 tells the search to return all Package fields, not just the name/id; just as is if you called /api/rest/package/PACKAGE-REF.

department={name of Ministry} tells the search to return all packages that have a department of {name of Ministry}. The lovely folks at CKAN added this functionality for us on request.

What does this look like, well it’s pretty simple really. Call the advancedSearch() function. Pass it a few parameters and it returns you all the data you need. Here’s the function itself:

public function advancedSearch($parameters){
	foreach($parameters as $key => $value) {
		$querystring .= $key .'='. urlencode($value) .'&';
	}
	$results = $this->transfer('api/search/package?'. $querystring);
	if (!$results->count){
		throw new CkanException("Search Error");
	}
	return $results;	
}

And here is that function being called for the list of Ministry Packages. The offset and limit are for the paging mechanism:

// Call the function
$results = $ckan->advancedSearch(array('department' => $ministry, 'all_fields' => '1', 'offset' => $offset, 'limit' => $limit));

There’s a lot more functionality in this module, more than I can go through in a blog post, even 5 posts. If you’re trying to integrate Drupal with a CKAN instance and are not sure where to start then please leave a comment and I’ll get back in touch.

When I moved to Mac, I missed the little green guy.  Every time I had to copy a url, username and password from a spreadsheet and paste them manually into an internet browser, the heart ache increased.  Plus, there was all kinds of silliness with the spreadsheet: who had it open, who deleted that field, what is the most current version of the FTP info, etc.

When I migrated to Mac I had accumulated 249 passwords in my Roboform! The amount of information that people are beginning to need to be able to track in their personal and professional lives is outrageous.

At Raised Eyebrow we have thousands of passwords and they needed to be stored uber securely.  Moreover, the data is a more complex set of information, Client Names, Contact Info, FTP Info, Database Info, CMS Info, Host Info, Stats Info, etc.  We combed through the market of Mac-based password keepers. 1Password, KeePassX seemed to be the best options that emerged at the time.  1Password, which is proprietary, seemed more feature rich, while KeePass, which is open source, seemed easier to install as a shared network installation.  Nonetheless, the amount of customization we required in terms of fields made neither solution appealing. So we kept on using a spreadsheet.

One day in utter frustration we had a brainstorming session.  We build websites, so maybe we could build an internal site that would keep our data?  From that session was born the idea for our super secure, one stop shop, password vault website.  That was 6 months ago.

Today, I just finished entering the last password from our main password list! What we developed in many respects is akin to a custom contact database.  We have different content types: Client, Website, Newsletter, Domain, etc.  Seeing the product, in its current state makes me wonder whether we should have invested the time we took to develop it into implementing and customizing a CRM (customer relationship manager)?  Could the data be accommodated in an off the shelf solution, like Salesforce or Daylight with customization?

Now, we are on the eve of embarking on a CRM implementation process. Starting such a process makes we wonder about all our systems: which ones work? Which ones should we keep? Which ones can be folded into the CRM? How can we streamline our workflow, while ensuring the integrity and security of our data?  How can specialized industries be best served by CRM software?

One major benefit from developing our password keeper is that we engaged in the exact process that would be required to clean up our data if we were going to import it into a CRM.  Instead of having a laundry list of every type of password in one long list, we have nice clean data parsed into types.  Also, it was like putting out a fire so that we wouldn’t be smoked out while we began to think about a CRM project. So, I’m happy with the path we’ve taken and looking forward to seeing what we do next.

If you have password overload, there are lots of solutions out there to make your life easier. And if you’ve recently done a CRM project in a all Mac environment, let me know how it went.

This is a position for someone with solid technical skills — someone who can rock HTML and CSS, who prides themselves on attention to detail and clean, elegant code. You’ll be turning design comps into themes for Drupal and WordPress sites, so experience with one (or both) of those CMS’s is preferred, but if you’re confident in your coding skills and are new to Drupal & WordPress, don’t let that stop you.

Because we’re a small shop, everyone here tends to wear a few hats, so we’re looking for someone with smarts, adaptability and a serious appetite for learning. We pride ourselves on the quality of our work, our efficiency, and our passion for our clients — and we’re looking for someone whose shares those values.

In our beautiful, heritage Gastown office, you’ll find a friendly team of experts, a ridiculously well-stocked tea cabinet, and abundant opportunities to work on projects that allow you to flex your technical muscles and bring your heart and ideals to work.

Sound like a fit for you (or someone you know)? The job description, with details on how to apply, is right here.

So, where do you turn if your needs are relatively straightforward and your budget is limited? I would recommend one of the following tools:

Picnik

Picnik is a powerful online application that allows you to upload & edit images on the fly. There is a free version but the options are rather limited; however, it might do the trick for what your client needs. The premium version starts at $2.08/month. Picnik even integrates with Flickr, Picasa & Facebook, so if your client uses those tools that might come in handy. I find it a little slow, compared to Photoshop (mostly because of the time required to upload & download files), but it’s a very good tool for people with no/little budget.

Photoshop Elements

This is a stripped-down version of Photoshop that’s perfect for what most of our clients need, which is basic photo & image editing tools for web use. It’s under $100 and provides a good chunk of the core Photoshop functionality — in some ways I would recommend it over Photoshop for basic-level users, because with fewer options it’s less confusing.

Photoshop.com

Adobe’s answer to Picnik is an online editing toolset that you can use for free; the only limitation is file storage. The free accounts comes with 2 GB of storage space for your files, and beyond that you pay an annual subscription fee that varies (anywhere from $20-$500 USD) based on the amount of storage you need.

What is CKAN?

CKAN is a registry or catalogue system for datasets or other “knowledge” resources. CKAN aims to make it easy to find, share and reuse open content and data, especially in ways that are machine automatable.

CKAN is a nice big database that is built to accept user input of the type of data we’re trying to collect for datadotgc.ca. It has a slick front and back end that allows administrative access to the collected data.
You can find out more on their website.

CKAN’s API

In order to utilize the power of CKAN I needed to link it up to Drupal. CKAN has a powerful and flexible API that I used extensively in the module.

The Foundation

Early on in the project I got in touch with the wonderful team at CKAN and they then put me in touch with Sean Burlington from the data.gov.uk development team. They had also built their site in Drupal and Seán had lots of information on how they tweaked their CKAN site to work with Drupal. He worked hard to open source some of the work that they had done, and released it just in time for us to get started. Seáns module provided the basic API connectivity we needed to get started and was the foundation for our module.

The Build

How do you integrate Drupal with the CKAN API? Let’s start with the basics:

CKAN stores the individual datasets that you see on Datadotgc.ca as ‘Packages‘. It became clear that these ‘Packages’ could be directly mapped to the standard node architecture in Drupal. To achieve this I created a content type in the module that stored all the data I needed.

/**
 * Define module-provided node types.
 */
function ckan_node_info() {
  return array(
  'ckan' => array(
    'name'           => t('CKAN Package'),
    'module'         => 'ckan',
    'description'    => t('A package of Open Data.'),
    'has_title'      => TRUE,
    'title_label'    => t('Title'),
    'has_body'       => TRUE,
    'body_label'     => t('Package Description'),
    'min_word_count' => 0,
    'locked'         => TRUE
    )
  );
}
 
function ckan_create_node($ckan_data) {
  $node = array(
    'title'   => $ckan_data->title,
    'uid'     => 1,
    'body'    => $ckan_data->name,
    'promote' => 1,
    'path'    => 'dataset/' . $ckan_data->name,
    'type'    => 'ckan',
    'comment' => 2,
  );
}

As you can see from the code, the only data elements to be set when a node is created are Title, Body and Path. The body of the node is set to be the name of the CKAN package, which is in fact a simple string: geogratisnat_hydrography_v100.

The more complex CKAN data was not mapped to any CCK fields as you might think, but instead it is pulled from CKAN when the node is loaded. This simplifies the Drupal side of things by ensuring that we don’t have to keep track of any changes to the structure or contents of the dataset that may happen on the CKAN side.

Here is an example of some package data:

[maintainer] => Government of Canada, Natural Resources Canada, Centre for Topographic Information (Sherbrooke)
[name] => 1996_population_census_data_canada
[author] => Government of Canada, Natural Resources Canada, Canada Centre for Remote Sensing, GeoAccess Division, The Atlas of Canada
[url] => ftp://ftp.geogratis.gc.ca/atlas/Population_Ecumene_Census/1996/
[notes] => The parts of Canada making up the 1996 Settled Area, (or Population Ecumene), represents a selection of the 5984 Census Subdivisions (CSD) as defined by Statistics Canada for the 1996 Census. The selection process essentially removes those CSDs with very large areas and/or very low populations. Some of British Columbia's CSD boundaries have been further modified to better conform to the distinctive settlement patterns in the Cordilleran regions. The 1996 Settled Area is an attempt to balance the needs of national scale choropleth mapping with the spatial reality that the majority of Canada's land area contains very few people. The Settled Area represents more than 98% of the Canadian population captured in the 1996 Census of Canada.
[title] => 1996 Population (Ecumene) Census Data, Canada
[download_url] => ftp://ftp.geogratis.gc.ca/atlas/Population_Ecumene_Census/1996/1996.zip

When a node is loaded, the package data is pulled from CKAN and then cached for later use.

/**
 * Load node-type-specific information
 */
function ckan_load($node){
  $ckan = ckan_ckan();
  if(($cache = cache_get('ckan:'. $node->body, 'cache_ckan')) && !empty($cache->data)) {
    // Get the cached data
    $node->ckan = $cache->data;
  } else {
    try {
      // Call the API to get the package data
      $node->ckan = $ckan->getPackage($node->body);
    } catch (Exception $e){
      drupal_set_message($e->getMessage(), 'error');
    }
    // Cache this package data for later use
    cache_set('ckan:'. $node->body, $node->ckan, 'cache_ckan');	
    watchdog('ckan', 'Called CKAN API for '.$node->body.' package - ckan_load()');
  }
  return $node;
}

Once the CKAN data has been added to the node object it’s relatively easy to output this data in a node template. I created a template file in my theme called node_ckan.tpl.php and here’s an example of how I displayed some of the CKAN package data in there:

< ?php if ($title): ?>
 <h1 id="page-title" class="title tk-museo-slab">< ?php print $title; ?></h1>
< ?php endif; ?>
 
< ?php if ($ckan->name): ?>
  <div class="package-name">(< ?php print $ckan->name; ?>)</div>
< ?php endif; ?>
 
< ?php if ($ckan->url): ?>
 <div class="package-link">< ?php print l($ckan->url, $ckan->url, $options = array('attributes' => array('class' => 'link'))); ?></div>
< ?php endif; ?>

You can see from all of the above examples I’m using a class object called “ckan” to store our data. This came from Sean’s module and is a simple class that provides the connectivity to the CKAN API. Here’s a brief synopsis of how it works:

1. First I need a way to connect to the API. That’s relatively straightforward using the curl libraries in php.
   ^?View Code PHP

   $ch = curl_init($this->url . $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_HEADER, 0); curl_setopt($ch, CURLOPT_TIMEOUT, 30); $result = curl_exec($ch); $info = curl_getinfo($ch); curl_close($ch);

2. Now create some functions that allow various parts of the API to be called. Below is an example of two class functions which call the API.
   ^?View Code PHP

   // Get an individual package public function getPackage($package) { $package = $this->transfer('api/rest/package/' . urlencode($package)); if (!$package->name){ throw new CkanException("Package Load Error"); } return $package; }   // Get a list of all packages public function getPackageList(){ $list = $this->transfer('api/rest/package/'); if (!is_array($list)){ throw new CkanException("Package List Error"); } return $list; }

All of these API calls return a JSON object which is then decoded into an object using the PHP function:

json_decode($results);

Theming

Once that data has been successfully retrieved and decoded from JSON into something that’s easier to handle, it needs to be themed. For this I created two theming functions; one for creating an individual list item, the other to create a formatted list of these individual items. I call the theme function whenever I get results from the API, and pass it the results, along with the title I want to appear on the listings page.

// Theme the results retrieved from API call
theme('ckan_results', $results, 'All Packages');
 
/**
 * Theme search results
 */
function theme_ckan_results($results, $title = '') {
  // Two global variables needed by the pager.
  // Taken from pager_query() in pager.inc
  global $pager_page_array, $pager_total;
 
  $output = '';
 
  // Grab the 'page' query parameter.
  // Taken from pager_query() in pager.inc
  $page = isset($_GET['page']) ? $_GET['page'] : '';
 
  // Convert comma-separated $page to an array, used by other functions.
  // Taken from pager_query() in pager.inc
  $pager_page_array = explode(',', $page);
 
  // Generate the data for page the requested and add it to the output.
  $items_per_page = variable_get('ckan_items_per_page', 4);
  // If there are less results than the specified number of items per page, reset the number of items per page
  if($results->count < $items_per_page) { $items_per_page = $results->count; }
 
  // Initialize pager
  $start = 0;
  // If it's not the first page
  if($page) {
    // Set the data to start displaying on the correct page
    $start = $page * $items_per_page;
  }
 
   if($title) {
     $output = '<h1 id="page-title">'.$title.'</h1>';
   }
 
  $output .= '<h3 class="resultscount">Your search returned '.$results->count .' records</h3>';
 
  // Theme the individual results
  for ($i = 0; $i < $items_per_page; $i++) {
      $output .=  theme('ckan_item', $results->results[$i]);
  }
 
  // Put some magic in the two global variables
  // Based on code in pager_query() in pager.inc
  $total_results = $results->count;
  $pager_total[0] = ceil($total_results / $items_per_page); //ckan_number_of_pages();
  $pager_page_array[0] =
    max(0, min(
      (int)$pager_page_array[0],
      ((int)$pager_total[0]) - 1)
    );
 
  // Add the pager to the output.
  $output .= theme('pager', NULL, $items_per_page, 0);
 
  return $output;
}
 
/**
 * Theme individual search items
 */
function theme_ckan_item($item) {
  // Link the title to the dataset
  $output .= '<h2>' . l($item->title, 'dataset/' . urlencode(check_plain($item->name))) . '</h2>';
  // Truncate the notes field
  if($item->notes) {
    $output .= '<p>' . truncate_utf8($item->notes, 250, $wordsafe = FALSE, $dots = TRUE) . '</p>';
  }
  // Output any tags	
  if(count($item->tags) > 0) {
    foreach($item->tags as $key => $value) {
      $items[] = l($value, 'data/tag/'.$value);
    }
  $seperated = implode(', ', $items);
  $output .= '<p><strong>Tags:</strong> ' .$seperated. '</p>';
  }	
return $output;
}

One thing you need to consider when displaying a list of results is having a pager built in so that you can break the list into bite-sized chunks. This took quite a while to figure out how to do. The problem was that the API call resulted in a lot of data and that resulted in a significant delay loading the page, due to the amount of time to complete the round trip to the API, along with the time taken to render all of that in a pager. CKAN however is very clever. When you call the API and ask for a list of packages, it returns the packages, but it also returns, as a variable, the count of the records your query generated. As well as that you can use the parameters ‘offset‘ and ‘limit‘ just like in SQL. What’s even more clever here is that it still returns the variable that holds the count of the records the query generated, but it also only returns the number of packages determined by the ‘offset‘ and ‘limit‘ parameters.

So if an API call to list all packages for a certain tag would normally return 200 records, and you specify a limit of 10 and an offset of 10, the data returned will contain a count of the number of records normally generated by that call, 200, but will only return 10 packages in the data, as specified by the offset and limit. This came in extremely useful for the pager as I just passed an offset and limit each time a page was loaded and then cached the returned data.

$ckan = ckan_ckan();
$start = 0;
$items_per_page = variable_get('ckan_items_per_page', 4);
if($page) {
  // If we're in a page, we need to set where to start the list
  $start = $page * $items_per_page;
}
 
// Set the offset
$offset = $start;
// Limit to the number of items per page 
$limit = $items_per_page;
 
// Get the list of tags with their count
try {
  $results = $ckan->advancedSearch(array('department' => $ministry, 'all_fields' => '1', 'offset' => $offset, 'limit' => $limit));
} catch (Exception $e){
  drupal_goto(variable_get('ckan_no_results_page', 'sorry'));
}

Homepage Chart

There was a requirement for a graph on the homepage the displayed the number of packages attributed to each Government Ministry. The quickest way to do this was using Google Chart Tools. It was relatively straightforward to get the data we needed. I did however have to do some funky sorting to get the data in the correct order. I also found a wonderful tutorial that really helped to clear up some of the label/legend issues I was having.

/**
 * Function to build a Google Chart
 *
 * @return	  string	HTML code with img tag
 *
 **/
function ckan_chart() {
  // If there is a cached version of the chart
  if(($cache = cache_get('ckan:chart', 'cache_ckan')) && !empty($cache->data)) {
    $image = $cache->data;
  } else {
    watchdog('ckan', 'Called Google API to build chart');
    // Get the list of ministries
    $ministries = explode("\r\n", filter_xss(variable_get('ckan_ministry_list', '')));
    // Set up our data array
    $data = array();
    foreach($ministries as $ministry) {
      $ckan = ckan_ckan();
      // Get the list of tags with their count
      try {
        $results = $ckan->advancedSearch(array('department' => $ministry, 'all_fields' => '0', 'offset' => '0', 'limit' => '1'));
        $count = $results->count;
      } catch (Exception $e){
        $count = 0;
      }
      // Cache the count to use on the Ministry list page '/ministry'
      cache_set('ckan:ministry_'. $ministry .'_count', $count, 'cache_ckan');
      $chart->data[$ministry. ' ('. $count . ')'] = $count;
    }
    // Sort the array in reverse order - most packages first and maintain index association
    arsort($chart->data);
    // Return all the keys of the data array - the names of the ministries
    $chart->legend = array_keys($chart->data);
    // Get the range of the chart - highest + a quarter
    $range = round(current($chart->data) * 1.25, -1);
    // Grid spacing  100/MaxRange*IntervalAmount
    $grid = 100/$range * 50;
    // Chart size, must be less than 30k pixels
    $chart->size = array(
      '590',
      '380'
    );
 
    // Create query
    $chart->query =
      'cht=bhg&'.	// Type
      'chd=t:'.implode(',', $chart->data).'&'.	// Data
      'chs='.$chart->size[0].'x'. $chart->size[1].'&'.	// Size
      'chco=cc0000&'.	// Color ( Remove # from string )
      'chxt=x,y&'.	// X,Y axis labels
      'chxr=0,0,'.$range.'&'. // Range
      'chxs=1,000000,13|0,000000,13&'. 	// Axis colors and font size
      'chg='.$grid.',0,5,5&'. // Grid verticalgridlines, horizontalgridlines, linesize, gapsize
      'chds=0,'.$range.'&'.	// Scale
      'chma=0,0,0,0&'. //left_margin, right_margin, top_margin, bottom_margin| legend_width, legend_height
      'chbh=13,0,2&'.	// bar_width_or_scale, space_between_bars, space_between_groups
      'chxl=1:|'.implode('|', array_reverse($chart->legend, TRUE)).'&'; //|Jan|Feb|Mar|Apr|May'
 
    $api_path = 'http://chart.apis.google.com/chart?';
    $url = $chart->query;
    $image = sprintf('<img src="%s" alt="%s" style="width:%spx;height:%spx;" />', $api_path.$url, 'Who\'s Sharing', $chart->size[0], $chart->size[1]);
      cache_set('ckan:chart', $image, 'cache_ckan');
  }	
  return $image;
}

So that’s a brief(not-so) overview of some of the fundamentals of how I integrated Drupal with CKAN and was able to create nodes and listings directly from API calls.

In my next post I’ll cover some very important areas of the module development such as:

• Caching
• Tag cloud creation
• Using the CKAN Search API for all lists

It seems like there are two ways to do this. You can either rewrite the SQL directly, or try to hack the query object that views provides.

A quick Google search and I ended up on Drupal.org

Which then led me to the Views documentation

From here I was able to deduce that the

hook_views_pre_execute()

hook was what I needed.

I created my module and added the hook:

/**
 * Implementation of hook_views_pre_execute()
 *
 **/
function other_views_pre_execute(&$view) {
  switch($view->name) {
    // If it's my view
    case 'my_view':
      switch($view->current_display) {
        // If it's the block on the author page
	case 'block_5':
	  // Set the extra argument
	  $view->build_info['query_args'][2] = $view->build_info['query_args'][1];
	  // Rewrite the query
	  $view->build_info['query'] = "SELECT ... ";
	  break;
       }
   }
}

Lets step through this:

function other_views_pre_execute(&$view) {

We need to make sure the word hook is replaced with the name of our module, in this case it’s called ‘other’

switch($view->name) {
  // If it's my view
  case 'my_view':

Each view is represented by a unique name. We need to modify the query for a certain view, not all of them, in this case the view is called ‘my_view’.

switch($view->current_display) {
  // If it's the block on the author page
  case 'block_5':

Views can have multiple displays, such as blocks, pages and attachments. In this case we want to target the block display with the name/id ‘block_5′

// Set the extra argument
$view->build_info['query_args'][2] = $view->build_info['query_args'][1];
// Rewrite the query
$view->build_info['query'] = "SELECT ... ";
break;

In this case our query is going to have an extra argument added to the query. These arguments are an array $view->build_info['query_args']. They appear in the form ‘%s’ and ‘%d’ in the query text, like this:

WHERE (node.status <> 0) AND (node.type in ('%s'))
AND ((node_node_data_field_publication_centre_authors.title = '%s') OR (node_node_data_field_publication_first_author.title ='%s'))

In this case the first ‘%s’ = $view->build_info['query_args'][0], the second one = $view->build_info['query_args'][1] and so on.

And that’s it. Your query is rewritten. One thing to watch our for, do not add any extra fields in the SELECT statement, this can cause all sorts of crazy problems.

The alternative to using the pre_execute hook is to use the query_alter hook.

This seems a lot trickier and even Earl Miles himself hates this hook, so I suggest staying away from it, but if you’re interested in what it looks like here’s a quick Krumo screenshot of what it looks like. This was achieved using the devel module along with the kpr() command. So

kpr($view->query); 

Anatomy of a View Query

Sometimes you might want your form to redirect to somewhere and pass a few bits of information in the querystring, so that you can do something when you get there.

For example, I needed a recent form to redirect depending on which checkboxes were checked to build a certain URL:

if($form_state['values']['filters']['open']) {	$query[] = 'o=1'; } 
if($form_state['values']['filters']['down']) {	$query[] = 'd=1'; }
$form_state['redirect'] = array('data/search/'.$keyword, implode('&', $query));

If “open” checkbox is checked, it will redirect to “data/search/keyword?o=1″. I then get the value of this using $_GET['o']. Simple eh?

When we were planning the session, Leif suggested we use a format I wasn’t familiar with — the Samoan Circle, a modified fishbowl process that encourages listening and made room for many of the attendees to come forward and speak. The chairs in the room were arranged in the round, and we pulled four chairs into the middle of the room, facing each other, so that everyone could see and hear the folks who were talking.

The theme of the session was “Socially Conscious Geek: making money while doing good,” and we provided some framing questions, but we knew that whoever showed up would have their own questions and goals for the discussion, so we tried to move quickly beyond our introductions and dive into the conversation proper.

I’m so glad we did, because as soon as we opened it up to the room, the conversation went in very interesting directions. Some of the themes that emerged were:

• Recruiting challenges for nonprofits: Someone asked how nonprofits can attract staff when the salaries are generally lower than they are in the corporate sector. Some great responses to this question emerged, including:
  • Explore what gets geeks interested, and offer up incentives that matter to them
  • Sell the lifestyle & “sleep better at night” benefits
  • Offer other benefits that provide competitive advantage with corporate positions, e.g. keep work hours reasonable, paid volunteer time, etc.
  • Someone mentioned that in the US, if you work for nonprofit organizations for 10 years after college, your outstanding student loans are forgiven. Given the size of the average student loan these days, that could provide a tangible financial benefit in the tens, or even hundreds of thousands of dollars.
  • Recognize that nonprofits can offer more opportunities for leadership & effecting change within the organization, because nonprofits often have leaner & more democratic structures. (I want to point out that this is definitely not always the case, though I do think it’s a valid point. However, I’ve seen some highly inefficient nonprofit organizations in my time.)
• There was some great discussion of how our definitions of sustainability must include financial sustainability; this is true for both for-profit and not-for-profit organizations. We need to both make money and do good; neither can be sacrificed. Some models put forward for this included:
  • Businesses can build charitable giving into their modus operandi, i.e. give x% of pre-tax profits to a particular cause or organization.
  • Design & tech agencies can offer discounts to nonprofit clients & stay profitable by engaging high-profit corporate clients.
  • Social enterprise models, e.g. nonprofits creating products & services people are willing to pay for.
  • Seed funding from progressive coalitions
• We also heard a number of people ask how we can collaborate better across movements and sectors – there is a real hunger for cross-pollination of ideas and learning from one another’s successes and failures.
• One women shared her story of working at a large, multinational corporation, feeling somewhat discouraged by said corporation’s lack of a formal social responsibility mandate, and responding by starting interest groups within her company. This has had the effect of both raising the profile of various causes, but also concrete changes in the business such as influencing purchasing decisions.
• Someone said they would love to see a crowdsourcing app that could leverage companies’ internal technical skills & resources and connect & share them with nonprofits who need them.

Those were my notes, but there’s plenty more available on Twitter — attendees tweeted their session notes using the hashtag #scgeek.

But this year we’re back, this time with a panel that we hope will prove helpful to geeks of both the design and techie varieties – both camps are in full force at SXSW’s Interactive conference, which is a pure, 24/7, all-out geekfest – who are looking to bring their values into their paid work, whether it’s by learning to work with clients in the nonprofit sector, creating apps that help mission-driven organizations work more effectively, or infusing some environmental or social-justice savvy into their workplaces. The panel’s called “Socially Conscious Geek: Makin’ Money While Doin’ Good”, and it takes place Saturday, March 13^th.

The topic was inspired by a topic our friend Noah Scalin (of Skull-A-Day and Another Limited Rebellion fame) spoke about at the 2009 HOW Conference. His focus was more on graphic design, since that’s his area of expertise, but when I asked him about riffing on his ideas for an interactive design audience, he shared his materials with me with typical generosity. (This is a guy who lives and breathes a DIY, share-and-share-alike mentality that a lot of designers could learn from. But that’s a topic for another day.)

This time I’ll be attending the conference solo, while Emira spends some time with her new baby; my co-facilitator for the session is Leif Utne, a friend and colleague who works for Zanby, a software company that specializes in supporting online social communities for change. Leif’s background includes a stint as the publisher of Worldchanging.org, and lots of experience building online communities; he’s also a journalist, activist and social entrepreneur.  Zanby is used to support The UpTake, a web video community that among other things was heavily involved in documenting the RNC and DNC prior to the 2008 election, and was also engaged in citizen vote monitoring during the election. He brings both amazing facilitation skills as well as a solid understanding of how businesses and nonprofits can work together in mutually beneficial ways.

We’ll be conducting our session in a highly interactive, inclusive format called a Core Conversation: Leif and I will be kick-starting the discussion and planting some seeds, but the idea is that whoever shows up will have a chance to step forward and contribute questions and ideas – so I’m really excited to see how it develops.

We’re hoping to address questions like:

• What is a socially conscious geek?
• How can I specialize in socially conscious work and earn a living wage?
• What are some of the unique opportunities and challenges of specializing in socially conscious work?
• What is the difference between non-profit and no-profit clients?
• How do I make the transition from corporate work to working with non-profit and/or mission-driven clients?
• What is the “triple bottom line” approach to business?
• What are some examples of alternative business models that are emerging among socially conscious geeks?
• How is socially conscious work different than “green” work?
• What are some examples of clients I might be able to work with as a socially conscious geek?
• How should I market myself to values-driven clients? Is it different from marketing to corporate clients?

Ultimately, though, the decisions about what we cover will rest with whoever shows up. So please help us spread the word – and if you’ll be at SXSW, we’d love to see you there!