Tag Archives: tutorial

Create jQuery Plugins: The Easy Way

If you’re a solid user of jQuery or just a tinkerer you may have thought to yourself “I have a great idea for a jQuery plugin”.  Your next steps of course would probably be to look at the jQuery documentation and eventually to have a look at someone else’s jQuery plugin source code.  In doing so you may have said to yourself or aloud “From what depths of perdition hath this code been spawned!”  I’ve seen quite a few jQuery plugin source files and they are, how do I put this delicately…written like crap a lot of the time.  Truth is, there’s an easy and elegant way to build jQuery plugins.

Let’s start with a simple example.  Let’s say you want to build a plugin that changes the color of all the nodes referenced by a selector.  In other words $(“.make-me-red”).red(); a very simple approach with the plugin actually called “red”.  Here’s the code:

Some simple HTML

<div class="make-me-red">One</div>
<div class="make-me-red">Two</div>
<div class="make-me-red">Three</div>


The jQuery plugin code

$.fn.red = function(){
    $(this).css("color", "red");


Invoke the plugin



Yes, yes, magical…I know.  That’s all there really is to it, let’s walk through the plugin code.  The “$” or “jQuery” reference has an object called “fn”, functions bound to this object are accessible from the returned selector array.  The “this” variable referenced in the function body contains all the elements matching the selector at the time of the call.  The “css()” method call applies the style attribute with color=”red” to all matching selector elements.  Easy peasy, now let’s try something a bit more useful.

Let’s create a jQuery plugin that makes use of options being passed in and performs something semi-useful, not too useful, just semi-useful.  Check it out:

Some simple HTML

<div class="make-me-move">Hi There!</div>


The jQuery Plugin Code

$.fn.jiggle = function(options){
    //setup default options
    var defaultOptions = {
        duration: 500,
        amount: 10
    //apply passed in options to default options
    options = $.extend(defaultOptions, options);
    //divide the time into fourths
    var d = options.duration/4;
    //animate the element
        .css("position", "relative")
        .animate({left: -options.amount}, d)
        .animate({left: -options.amount}, d)
        .animate({left: -options.amount}, d)
        .animate({left: -options.amount}, d)
        .animate({left: 0}, d);


Invoke the plugin



Now that was a bit more complex, let’s walk through it.  This plugin is called “jiggle”, wonder why.  The first change here is the options variable being passed in.  This variable allows the plugin to be configurable from the calling code.  In this instance we have exposed the duration (in milliseconds) and the amount (movement).  The first thing we do is set default values to handle the case that the calling code does not pass anything, this is the purpose of the “defaultOptions”.  Next we make a call to $.extend to override any default values passed into the plugin from the calling code by way of “options”.  We then set options to the merged result of “defaultOptions” and “options” (where “options” overrides “defaultOptions”), that way we have a single reference to all the values we will use.  Finally, we animate the elements by changing the css position and making subsequent calls to the animate method.  Now let’s invoke the same call again with custom values passed in by way of “options”

        duration: 1000


Implementing options the way we have in our plugin code allows us to optionally (pun intended) pass some, all or none of the values into the calling code.  With this example we have set the duration to 1 second (1000 milliseconds).  Simple.

Hopefully this quick start will help propel you into the halls of fame in the jQuery plugin community, and if not, that’s cool too 🙂  Thanks for reading.

Understanding JavaScript Promises In jQuery

You may or may not have heard the new buzz about promises being introduced into JavaScript and if you don’t know what they are, you’re not alone.  I’ll attempt to explain what they are and why you should or maybe shouldn’t care about them.  First let’s understand what promises are.

Promise defined: “The Promise interface represents a proxy for a value not necessarily known at its creation time. It allows you to associate handlers to an asynchronous action’s eventual success or failure. This let asynchronous methods to return values like synchronous methods: instead of the final value, the asynchronous method returns a promise of having a value at some point in the future.”

Basically, it allows you to setup chaining for a value or values and determine a functional response.  For example, let’s say you need to get two values from two separate AJAX calls, but you don’t want to proceed unless both have loaded correctly and if either fails, you want to bail out of the entire thing.  Although not an optimum approach, this will serve as a more or less real world problem.  Consider the following code:


var first_call = false;
var second_call = false;
//make the first call
    url: "/some/page",
    success: function(data){
        //set a success variable
        first_call = true;
//make the second call
    url: "/some/other-page",
    success: function(data){
        //set a success variable
        second_call = true;
if(first_call && second_call){
    //we're good to go - do something
    //ruh roh - bail out!


Now obviously, the above code isn’t fantastic but it will suit our needs for now.  Let’s walk through it.  The first ajax call if successful sets the first_call variable to true and the same goes for the second.  If all is good, we can proceed and do what we need to, otherwise we execute our bail out code.  So now let’s check out the same functional code executed with promises:

$.when( $.ajax("/some/page"), $.ajax("/some/other-page") )
    .done(function(first_call, second_call){
        //we're good to go - do something
        //ruh roh - bail out!

So, looking at this latest example we see not only is the code much more compact, but now we can chain even more calls together, no use of outside variables for state checking and still we end up in the same place.  This is the power that promises provide you.  In the above example, we have implemented jQuery’s $.when call which will take a list of operations as the arguments.  For each operation, it will return the value into the done() function as the ordered arguments (ie. done(first_call_data, second_call_data, …)).  If a failure occurs for any of the calls, the fail() method is invoked where we can make use of our bail code and either try again, display an error message, etc.  If we wanted to add more calls to this promise we would do so by adding them directly to the when() function and creating the corresponding argument in the done() method.

    .done(function(first_call, second_call, third_call, fourth_call){
        //we're good to go - do something
        //ruh roh - bail out!

We can continue to chain $.when calls together to simply an otherwise complex system of dependencies.  Promises also provide us with a way of implementing our own custom promise behavior by taking advantage of methods such as .reject() .resolve() and .state().  However, this is a bit beyond the scope of this article.  For more information regarding jQuery promises have a look at:

Promise – http://api.jquery.com/promise/

Deferred Object – http://api.jquery.com/category/deferred-object/

D3 Drag And Drop

Implementing drag and drop in D3 (http://d3js.org/) is a pretty simple task when you know how to do it, the following is a walkthrough from setup to execution and the events along the way.

First thing we are going to want to do is setup our drag behavior.  This is the object that will be responsible for handling the actual drag and drop, events and will be added to the call chain of the objects of which we want to allow the drag and drop behavior.

var drag = d3.behavior.drag();

The above statement initializes a new instance of the drag behavior.  We can use it to bind this behavior to the call chain of other objects.  Let’s assume we want to allow all nodes having class “draggable” to have this drag and drop behavior.  To do this we simply add the drag behavior we just instantiated to the call chain of the objects:

var drag = d3.behavior.drag();

It’s that simple, now all objects containing the “draggable” class will now have this behavior.  But what if we need to do something a bit more complex?  What if we need to have other things happen along the way, update other areas during a drag, initialize or change variables on start and end drag.

var drag = d3.behavior.drag()
    .on("dragstart", function(){
        //do some drag start stuff...
    .on("drag", function(){
        //hey we're dragging, let's update some stuff
    .on("dragend", function(){
        //we're done, end some stuff

As you would expect, each on(…) call defines a hook into the event chain of the drag behavior allowing you to customize the behavior as you see fit.

Relatively Positioned Absolute Elements


Absolutely positioned elements can be quite useful, but what if you have something centered or off to one side and you need to just nudge an absolutely positioned element.  The following will show you a handy technique for moving absolute elements relative to the elements around it using nothing but CSS.

Getting Started

First let’s put together a scenario.  Let’s say you have an XHTML document like the following:


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<div class="article">
<img src="http://www.gravatar.com/avatar/205e460b479e2e5b48aec07710c08d50" />
<label class="title">A Test Article</label>
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. In nec libero ac sapien dapibus dapibus. Sed a enim risus, non hendrerit nunc.
Sed non scelerisque ipsum. Nulla congue accumsan quam id pharetra. Nunc dolor purus, aliquam eu sodales id, pulvinar a sem. Nunc vel lorem
ut ligula ultricies sollicitudin sit amet quis justo. Morbi tincidunt porta ipsum, at viverra tellus vehicula nec. Ut arcu nunc, ultrices
nec pellentesque dignissim, molestie vitae lorem. Sed id ultricies leo. Nam porttitor justo non arcu eleifend imperdiet.</p>


To make things a bit more complex for this example, let’s say the article was centered and you wanted to have the avatar image left positioned outside of the article container.  Using relative positioning this is quite easy, but we don’t want a blank area where the avatar image.  So to solve this we need to position the element absolutely.  Let’s add the following CSS declarations:


.article {
font: 13px sans-serif, tahoma, helvetica, arial;
width: 400px;
margin: 0px auto;
border: solid 1px #efefef;
padding: 10px;
.article label.title {
font-weight: bold;
.article cite.author {
font-style: italic;
margin-left: 20px;
.article img.avatar {
position: absolute;
padding: 3px;
border: solid 1px #efefef;


Now when we view the page, the problem becomes very self evident.



Scenario Problem

Scenario Problem


To solve the problem, we need to use both absolute positioning so the element does not take up visual space in the DOM, and we need to somehow position the element using relative coordinates.  We can do this by using positive and negative margins very much the same way you would use top or left when the element is positioned relatively.  Consider the following:

margin-left: -120px;

Let’s add this declaration to our avatar image in CSS and see what happens.


.article {
font: 13px sans-serif, tahoma, helvetica, arial;
width: 400px;
margin: 0px auto;
border: solid 1px #efefef;
padding: 10px;
.article label.title {
font-weight: bold;
.article cite.author {
font-style: italic;
margin-left: 20px;
.article img.avatar {
position: absolute;
padding: 3px;
border: solid 1px #efefef;
margin-left: 120px;



Scenario Solution

Scenario Solution

Perfect! The avatar image is outside the visible article container which is bordered, the article is centered using margin auto.  We’ve in essence, relatively positioned an absolute element.


Download Project Files

Quick Start Solr


The following is a quick start guide for gettting Solr configured, up and running in a few minutes.  All of my examples will be performed on CentOS Linux.

Getting Started

First things first, make sure you have Java installed and ready.  Next download a Solr release:


Then extract it:

user@computer:$ tar -xpf apache-solr-1.4.1.tgz

This is just preference, but I like to work off of a short name by creating a symbolic link to the full version directory like so:

user@computer:$ ln -s apache-solr-1.4.1 apache-solr

Now we’re going to need a solid web server, I suggest using something like Tomcat or other comparable Java server.  Download tomcat at:


Then extract it:

user@computer:$ tar xpf apache-tomcat-6.0.29.tar.gz

Once again, not a requirement but for personal ease of use, I create a symbolic link.

user@computer:$ ln -s apache-tomcat-6.0.29 apache-tomcat

Next let’s copy out the WAR file to tomcat:

user@computer:$ cp apache-solr/example/webapps/solr.war apache-tomcat/webapps/

Now we need to copy out an example Solr configuration directory:

user@computer:$ cp -R apache-solr/example/solr .

Good, now that’s it for the prep work. Now all we have to do is configure Solr for our specific needs.

Configuring Solr

First let’s setup or configuration file that tells Solr where to store it’s data files.  Open solr/conf/solrconfig.xml in an editor:

user@computer:$ vi solr/conf/solrconfig.xml

Find the section that looks like this:

<!-- Used to specify an alternate directory to hold all index data
other than the default ./data under the Solr home.
If replication is in use, this should match the replication configuration. -->

Change the <dataDir> contents to point to where the data will be stored.  In our case use the location of the solr directory.  Something like:


Otherwise when you start Tomcat, it will just put the data directory in the directory in your current directory, wherever that may be.  Not a good idea.

Now it’s on to the fun part.  We need to tell Solr how to index our data.  That is, what form to store the data in.  Open solr/conf/schema.xml in an editor.

vi solr/conf/schema.xml

Now look for this section:

<!-- Valid attributes for fields:
name: mandatory - the name for the field
type: mandatory - the name of a previously defined type from the
<types> section
indexed: true if this field should be indexed (searchable or sortable)
stored: true if this field should be retrievable
compressed: [false] if this field should be stored using gzip compression
(this will only apply if the field type is compressable; among
the standard field types, only TextField and StrField are)
multiValued: true if this field may contain multiple values per document
omitNorms: (expert) set to true to omit the norms associated with
this field (this disables length normalization and index-time
boosting for the field, and saves some memory).  Only full-text
fields or fields that need an index-time boost need norms.
termVectors: [false] set to true to store the term vector for a
given field.
When using MoreLikeThis, fields used for similarity should be
stored for best performance.
termPositions: Store position information with the term vector.
This will increase storage costs.
termOffsets: Store offset information with the term vector. This
will increase storage costs.
default: a value that should be used if no value is specified
when adding a document.

This is going to be a bit confusing if this is your first time dealing with Solr, but just hang in there, it’s not really as bad as it seems.

Inside the <fields> node is where we will define how and what data will be stored.  Let’s say for instance that all you wanted to store was:


Say, fields from a database user table for instance.  In that case we could remove all of the current field definitions, that is, all of the <field name=”… nodes and replace them with our own.  In this case what we’d want is:

<field name="id" type="string" indexed="true" stored="true" required="true" />
<field name="first_name" type="string" indexed="true" stored="true" required="true" />
<field name="last_name" type="string" indexed="true" stored="true" required="true" />

Here’s what the above defines:

Each node defines a field to be used by Solr (obviously).
The “name” attribute tells us, well…the name.
The “type” attribute specifies the type of the data.
The “indexed” attribute determines whether or not the field is searchable.  We can actually create fields that get kept by Solr, but are not searchable.
The “stored” attribute tells Solr to keep the data when it’s received.
The “required” attribute tells Solr whether it needs to explicitly refuse data that does not contain all the required fields.  In other words, anything we mark as required, better be there.

So now we’ve defined what our data will look like, we need to create a simple search field that will aggregate all of the values together for searching purposes.  In other words, what we have will allow us to specifically search for a “first_name”, “last_name” or “id”.  But what if we don’t know what we’re searching for just yet.  Let’s create a field that we can put the first and last name into to make it more searchable.  To do this create the following node:

<field name="text" type="text" indexed="true" stored="false" multiValued="true"/>

We will store all field concatenated search values in the text field.  Now if you scroll down in your editor a bit, you’ll see this:

<!-- Field to use to determine and enforce document uniqueness.
Unless this field is marked with required="false", it will be a required field
<!-- field for the QueryParser to use when an explicit fieldname is absent -->

This was already setup for is.  It tells Solr to use the “id” field as an index for all the records we are going to send to it.  It also tells Solr that if we don’t specify a specific field to search, to just use the “text” field we defined.

Alright, now let’s tell Solr to copy the “first_name” and “last_name” fields into our searchable “text” field.  Scroll down a bit in your editor and find the <copyField> nodes similar to this:

<copyField source="cat" dest="text"/>
<copyField source="name" dest="text"/>
<copyField source="manu" dest="text"/>
<copyField source="features" dest="text"/>
<copyField source="includes" dest="text"/>
<copyField source="manu" dest="manu_exact"/>

Go ahead and remove all of those, we’re going to define our own.  We only want our “first_name” and “last_name” fields to get copied into our default searchable “text” field:

<copyField source="first_name" dest="text"/>
<copyField source="last_name" dest="text"/>

Now we’re good to go, let’s spin up the server.

user@computer:$ ./tomcat/bin/catalina.sh run

You should see a bunch of output for Tomcat.  The “run” argument tells Tomcat to launch in a single process debug mode.  If you have made any errors up to this point.  Tomcat will show them to you.  Hopefully you haven’t, but if you have just read through the output carefully and correct any mistakes as you find them.

Assuming all went well, you can open up a browser and go to the admin page for Solr at: http://localhost:8080/solr

Solr Admin Welcome Screen

Solr Admin Welcome Screen

Once you see the “Welcome to Solr!” screen, you know you’ve arrived. 🙂

The only thing left to do is to start populating your Solr instance with data and try searching it.  Shall we?

Data Importing

There are a number of ways to import data into Solr, one of the easiest is to send and post directly to Solr.  For this example, let’s build a simple PHP XML command line script called “update-solr.php”:

 * Simple general purpose function for updating Solr
 * @param string $url Solr URL for updating data
 * @param string $postXml XML containing update information
 * @return string Solr response **/
function updateSolr($url, $postXml){
    $ch = curl_init($url);
    curl_setopt($ch, CURLOPT_POSTFIELDS, $postXml);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_POST, 1);
    curl_setopt($ch, CURLOPT_FRESH_CONNECT, 1);
    $header[] = "SOAPAction: $url";
    $header[] = "MIME-Version: 1.0";
    $header[] = "Content-type: text/xml; charset=utf-8";
    curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
    $r = curl_exec($ch);
        echo "Error: " . curl_error($ch) . "n";
    return $r;
}//end updateSolr()
// Build a simple XML string to add a new record.
// PHP command line parameters are received in $argv array.$xml = "
<field name="id">$argv[1]</field>
<field name="first_name">$argv[2]</field>
<field name="last_name">$argv[3]</field>
updateSolr($argv[0], $xml);
// Lastly let's commit our changes
updateSolr($url, "<commit />");

The above command line PHP script should be pretty self explanatory, but just in case it is not, here’s the explanation.  The “updateSolr” function makes to call to Solr and posts the XML to it.  The information is set via the command line upon invocation.  So let’s try making an update:

First let’s make the script executable:

user@computer:$ chmod a+x update-solr.php

Next let’s update Solr with a few test records:

user@computer:$ ./update-solr.php http://localhost:8080/solr/update 1 John Doe
user@computer:$ ./update-solr.php http://localhost:8080/solr/update 2 Jane Doe

Next let’s view what’s been imported into Solr so far.  Open a browser and go to http://localhost:8080/solr, next click the “Solr Admin” link.

Solr Admin Page

Solr Admin Page

Next replace the Query String field (currently displaying “solr”) with a search for value.  Something like “John” or “Jane”.  Next click the “Search” button and the results are displayed in XML.  That’s all there is to it.  If you’d like to do a get request to try searching from your application, simply copy the URL currently displayed in the address bar for the search results and modify as you see fit.

How To Convert M2TS To WMV For Free


The following is a tutorial for how to convert M2TS files to WWMV files for free.  All the examples given are for use with Windows but should work with any operating system other than the binaries.


So after purchasing an extremely cool Panasonic Blueray/HD video camcorder, I realized that the format it stores directly from the import software is in M2TS format.  Which on its own is fine, but good luck trying to use that format with any other software, especially After Effects.  There are tons of software applications out there to do the conversion for you but they all cost, suck terribly or some combination of both.  I’ve found a way to do it for free on any platform.

Getting Started

First thing you’re going to need is the latest version of FFMPEG.  For windows get the unofficial night build from here:


Just click on the static “w32” link, download and unzip any location of your choice.


To convert the M2TS file to WMV, I created a simple batch file to use:

ffmpeg -i %1 -vcodec wmv2 -b 16000k -acodec wmav2 -ac 2 -ab 128k -ar 44100 -s 1920x1080 %1.avi

Just for reference if you’re unfamiliar with Windows batch, the “%1” is the first argument passed to the batch command.  You may need to change the following arguments to match your camera’s capture settings:

The “-s” argument is the source dimension of the video.
The “-ab” argument is the audio bit rate.
The “-b” argument is the video bit rate.

Save this file and call it whatever you want (ie. convert.bat).  Once you’ve adjusted the arguments to fit your camera settings execute the command using the following:

user@computer:$ convert.bat myvideo.m2ts

That’s all there is to it. Let me know if you have any problems. Enjoy!

jQuery Quick Start Tutorial


The following is a quick crash course in using jQuery.  If you’ve never used jQuery before and want to know to how without reading a ton, this is the tutorial for you.

Getting Started

First download the latest version and include it in a page.  Grab the minified version for speed or if you want to read through the code yourself, grab the uncompressed version.


Next simply include the library in your HTML page just like any other JavaScript file.

<script type="text/javascript" src="script/jquery-1.4.4.min.js"></script>

Now we’re ready to start using it.

Dom Traversal

So how do we select a node or nodes.  We have a few different choices, we can use CSS selector syntax, or we can use xpath syntax.  Let’s try an example on the following markup.

<div class="container">
<span  id="title">Hello World</span>

To apply a standard CSS selector we could use something like “.container #title{ style }” or just “#title{ style }”.  Same with jQuery, here’s an example:

<script type="text/javascript">
    var titleSpan = $(".container #title");
    //title span is now set to the span title node
    titleSpan = $("#title");
    //this does the same as well. 

Now if you’re unfamiliar with jQuery, the above syntax may look a bit odd but here’s the breakdown.  “$” is an alias of the jQuery object, which is what you’ll use to interact with all jQuery APIs.  The long hand for of the above would look like this:

<script type="text/javascript">

Both do the same thing, wait until the document has fully loaded, then process the function passed into the ready event.  There are several methods for traversing the DOM.  To view the entire list checkout:


DOM Manipulation

So how do we work with the DOM, change it’s contents, etc.  jQuery makes this really easy, and here’s how.  Let’s say we wanted to get the text content of the title node using the following HTML.

<div class="container">
<span id="title">Hello World</span>

Here’s all we need to do.

var titleText = $("#title").text();

Boom! Done!  Very easy.  But what about changing the text contents of the node. Again:

$("#title").text("Hello Dolly");

Pie!  So if you pass an argument to the text method, it sets the content, without an argument, it retrieves it.  What if we needed to get the innerHTML value of a node.  Here’s what we do:

var html = $(".container").html();

That’s all there is to it, this example grabs the “container” class and retrieves the html content of the node.  What about setting it you ask? Check it out:

$(".container").html('<span id="footer">Goodbye</span>');

That’s all there is to it.  For more information on DOM manipulation, have a look at:


Working With Events

So now it’s time to be productive.  Let’s do something semi-based in the real world.  Let’s say we have a list of anchors that we want to attach a click event to.  Now the old school approach would be to do something like this:

var els = document.getElementsByTagName("a");
int len = els.length
for(var i=0; i<len; i++){
    if(els[i].className == ".showImage"){
        els[i].onclick = function(event){

This approach would allow us to get all the anchor elements off the page, then iterate each to find a match for class “showImage”, then bind a click event to that node and continue.  Using jQuery the same thing can be done if far less code:


Quite a difference, but both perform the same functionality.  In jQuery a selector can return a single or an array of elements depending on the markup and selector syntax used.  If the above looks a little confusing, consider this example:

    //this will return all a.showImage and call this each function on a per returned node basis
    //basically, this is a for each
    var anchor = $(this); //returns the current iteration object of each

The above does the same as the example before it.  Difference is the amount of code written and for demonstration purposes you can hopefully better understand what’s going on behind the scenes.

So now we now to to bind events, but what about unbinding events.  This example should help:

    var anchor = $(this);
    //bind the click event to the anchor
    anchor.bind("click", function(){
    //unbind the click event from the anchor

When you use “object.click”, it’s the same as calling the “bind” method of the object.  Therefore to remove the “bind”, simply “unbind”.  For more information on handling jQuery events see:



That’s about the gist of it.  The rest you should be able to discover on your own.  If you have any questions or find anything I’ve mentioned here to be confusing, please feel free to ask in the comments.

Creating RightScale Attachment Scripts


The following is a primer on creating RightScale scripts that execute with attachments.


Creating RightScale scripts can be a little daunting at first, but with a little time and effort one can quickly ramp up from beginner to expert.  The examples show here will be executed on a Linux CentOS instance.

Basic Script

So let’s go over some RightScale script basics.  First let’s start by creating a new script.  Click the Design->RightScripts menu option.

Design -> RightScripts

Design -> RightScripts

Next click the “New” button.

New Button

New Button

Now you should be looking at the “New RightScript” screen.  Here we will give the script a name, enter a description, identify input values and write the actual script to be executed on the client machine.

New RightScript Screen

New RightScript Screen

Let’s begin by entering a name and description for the RightScript.  Enter any name and description you want.

It’s important to note here, that changing the RightScript name, will not affect any instance you have the scripts attached to.  In other words, once you attach a script to an instance, if you decide to rename that script later, feel free to do so with no consequences.  The next time you see the script on the instance, it will have the updated name, there’s no need to go back and re-add it.

Inputs will be used to dynamically get values entered from the user or from a predefined value.  I’ll explain this more in detail in a little while.  First let’s create a simple script.  Let’s create a simple script that copies a file from one location to the other.  Enter the following into the script field:

cp /tmp/file1 /tmp/file2

Now I know that this is extremely simple, but we’ll build from this.  So the script when executed get’s run against the default interpreter, in this instance BASH, and it copies a file that does not exist called “file1” to the same directory and names it “file2”.  Pretty useless, but now let’s make it a bit more useful.  Let’s say we wanted to have a utility to copy any file from one location to another from within the RightScale interface.  Let’s modify the script to do so.  Change the script to the following:


Now click the “Identify” button located just above the script field.  Did you see what happened?  RightScale realizes that the variables “$FROM_LOCATION” and “$TO_LOCATION” are not defined and offers them as input values to be input by the user and the time of execution.  In other words, when you decide to execute this script, it will prompt the user for the value of the “$FROM_LOCATION” variable and the “$TO_LOCATION” variable.

By default, the input types are “Any”, meaning any free text value.  You can if you wish change the type to be a dropdown and set a distinct set of values from the user to choose from.  But for this example let’s leave it as any.

Now this example is a bit more useful.  We can now pretty much move any file from one location to another through the RightScale interface.  But what about if we wanted to have a file uploaded and then copied out to a directory.  Let’s try that shall we?  First let’s modify the script a bit to use an uploaded attachment:

Attachment Script


Next let’s hit the “Identify” button again.  Notice, the “$FROM_LOCATION” input field disappears because it is no longer needed, but where is the “$ATTACH_DIR” input variable.  The “$ATTACH_DIR” input variable is defined by RightScale to allow you to access any attachments you decide to upload to the script.  Let’s try this and see how it works.

Let’s first enter a description for the “$TO_LOCATION” field we’ve added.  This will allow users to understand what the input field is to be used for.  For now enter this in the “TO_LOCATION” description field:

“Enter the destination for the attached file. (ie. /tmp)”

Then click the “Save” button.  You should now be presented with the tabbed interface to your script.



Next click the “Attachments” tab.



Here we will click the “Choose File” button and upload a file attachment to our script.  Create a simple text file called “test.txt”, click “Choose File”, select the newly created file “test.txt” and finally click “Upload”.  You should then see the file listed about the “Upload file” area.

Attachment Uploaded

Attachment Uploaded

Finally let’s try running the script to see how it works.

Executing The Script

To execute the script, we’ll need to select a running instance to have the script act upon.  First let’s get our deployments and select and instance.  Do this by selecting Manage->Deployments.

Manage -> Deployments

Manage -> Deployments

Then click on the server template you want to execute the script on from the list of servers.  Now click on the “Scripts” tab for your selected server template.

Template Scripts

Template Scripts

Next click the “Add Script” button under the “Operational Scripts” area.

Select A Script Dialog

Select A Script Dialog

You should now be presented with the “Select a script” dialog.  Click “Unpublished” and click on your newly created script, then click the “Select” button.  This action will attach the script to the server template and make it executable on running instances built off this template.  To execute the script, Go back to your Manage->Deployments screen and select the actual running server, click the “Scripts” tab, then click the blue “Run” button located to the right of the script.

Once the “Run” button has been clicked, the “Queued”, “Status” and “Completed” status will appear in the top left status pane.  If all goes well, you’ll have a successfully created and run script.  You can verify this by clicking on the “Completed” status when it appears.  Or by opening a terminal into to instance and verifying the appearance of the test.txt file.

How To Cluster Rabbit-MQ


This explanation of clustering Rabbit-MQ assumes that you’ve had some experience with Rabbit-MQ.  At least to the point of being able to get Rabbit-MQ up and running and processing messages.  For this explanation I will be using CentOS linux, other linux distributions may or may not require slight modifications to the setup process.  You will need at least 2 machines or virtual instances up and running and install Rabbit-MQ on both.


Clustering Rabbit-MQ is actually very simple once you understand what’s going on and how it actually works.  There is no need for a load balancer or any other hardware/software component and the idea is simple.  Send all messages to the master queue and let the master distribute the messages down to the slaves.

Create Short Names

First, we need to change the host name and host entries of our machines to something short.  Rabbit-MQ has trouble clustering queues will fully qualified DNS names.  We’ll need a single short word host and route.  For now, let’s use the names “master” for the master head, then “slave1”, “slave2” … “slaveN” respectively for the rest.

Set the master host name to “master”

user@computer:$ echo "master" > /proc/sys/kernel/hostname

Next we need to set the entries in the /etc/hosts file to allow the short names to be aliased to machine or instance IPs.  Open the /etc/hosts file in your favorite editor and add the following lines:

user@computer:$ cat /etc/hosts   localhost   localhost.localdomain master master.localdomain slave1 slave1.localdomain slave2 slave2.localdomain

Please note: Your particular /etc/hosts file will look different that the above. You’ll need to substitute your actual ip and domain suffix for each entry.

Make sure each slave you plan to add has an entry in the /etc/hosts file of the master. To verify your settings for each of the entries you provide, try pinging them by their short name.

user@computer:$ ping master
PING master ( 56(84) bytes of data.
64 bytes from master ( icmp_seq=1 ttl=61 time=0.499 ms
64 bytes from master ( icmp_seq=2 ttl=61 time=0.620 ms
64 bytes from master ( icmp_seq=3 ttl=61 time=0.590 ms
64 bytes from master ( icmp_seq=4 ttl=61 time=0.494 ms

If you get something like the above, you’re good to go. If not, take a good look at your settings and adjust them until you do.

Once your short names are setup in the master /etc/hosts file, copy the /etc/hosts file to every slave so that all machines have the same hosts file entries, or to be more specific, that each machine has the master and slave routes. If you’re familiar with routing, feel free to just add the missing routes.

Then for each slave update the host name.

user@computer:$ echo "slave1" > /proc/sys/kernel/hostname
user@computer:$ echo "slave2" > /proc/sys/kernel/hostname
user@computer:$ echo "slave3" > /proc/sys/kernel/hostname

Synchronize ERLang Cookie

Next we need to synchronize our ERlang cookie. Rabbit-MQ needs this to be the same on all machines for them to communicate properly. The file we need is located on the master at /var/lib/rabbitmq/.erlang.cookie, we’ll cat this value then update all the cookies on the slave.

user@computer:$ cat /var/lib/rabbitmq/.erlang.cookie

Copy the value displayed by the cat.

Please notice that the file itself is storing the value without a carriage return nor a line feed. This value needs to go into the slaves the same way. Do so be executing the following command on each slave. Make sure you use the “-n” flag.

First let’s make sure we stop the rabbitmq-server on the slaves before updating the ERlang cookie.

user@computer:$ service rabbitmq-server stop
user@computer:$ service rabbitmq-server stop
user@computer:$ service rabbitmq-server stop

Next let’s update the cookie and start the service back up.

user@computer:$ echo -n "DQRRLCTUGOBCRFNPIABC" > /var/lib/rabbitmq/.erlang.cookie
user@computer:$ service rabbitmq-server start
user@computer:$ echo -n "DQRRLCTUGOBCRFNPIABC" > /var/lib/rabbitmq/.erlang.cookie
user@computer:$ service rabbitmq-server start
user@computer:$ echo -n "DQRRLCTUGOBCRFNPIABC" > /var/lib/rabbitmq/.erlang.cookie
user@computer:$ service rabbitmq-server start

Once again substitute the “DQRRLCTUGOBCRFNPIABC” value with your actual ERlang cookie value.

Create The Cluster

Now we cluster the queues together. Starting with the master, issue the following commands:

user@computer:$ rabbitmqctl stop_app
user@computer:$ rabbitmqctl reset
user@computer:$ rabbitmqctl start_app

Next we cluster the slaves to the master. For each slave execute the following commands:

user@computer:$ rabbitmqctl stop_app
user@computer:$ rabbitmqctl reset
user@computer:$ rabbitmqctl cluster rabbit@master
user@computer:$ rabbitmqctl start_app

These commands actually do the clustering of the slaves to the master. To verify that everything is in working order issue the following command on any master or slave instance:

user@computer:$ rabbitmqctl status
Status of node rabbit@master ...
{mnesia,"MNESIA CXC 138 12","4.4.3"},
{os_mon,"CPO CXC 138 46","2.1.6"},
{sasl,"SASL CXC 138 11",""},
{stdlib,"ERTS CXC 138 10","1.15.3"},
{kernel,"ERTS CXC 138 10","2.12.3"}]},

Notice the lines containing nodes and running_nodes. They should list out all of the mater and slave entries. If they do not, you may have done something wrong. Go back and try executing all the steps again. Otherwise, you’re good to go. Start sending messages to the master and watch as they are distributed to each of the slave nodes.

You can always dynamically add more slave nodes. To do this, updated the /etc/hosts file of all the machines with the new entry. Copy the master ERlang cookie value to the new slave. Execute the commands to cluster the slave to the master and verify.


If you accidentally update the cookie value before you’ve stopped the service, you could get strange errors the next time you start the rabbitmq-server. If this happens just issue the following command:

user@computer:$ rm /var/lib/rabbitmq/mnesia/rabbit/* -f

This removes the mnesia storage and allows you to restart the rabbitmq-server without errors.

Using Hadoop And PHP

Getting Started

So first things first.  If you haven’t used Hadoop before you’ll first need to download a Hadoop release and make sure you have Java and PHP installed.  To download Hadoop head over to:


Click on download a release and choose a mirror.  I suggest choosing the most recent stable release.  Once you’ve downloaded Hadoop, unzip it.

user@computer:$ tar xpf hadoop-0.20.2.tar.gz

I like to create a symlink to the hadoop-<release> directory to make things easier to manage.

user@computer:$ link -s hadoop-0.20.2 hadoop

Now you should have everything you need to start creating a Hadoop PHP job.

Creating The Job

For this example I’m going to create a simple Map/Reduce job for Hadoop.  Let’s start by understanding what we want to happen.

  1. We want to read from an input system – this is our mapper
  2. We want to do something with what we’ve mapped – this is our reducer

At the root of your development directory, let’s create another directory called script.  This is where we’ll store our PHP mapper and reducer files.

user@computer:$ ls

user@computer:$ mkdir script

Now let’s being creating our mapper script in PHP.  Go ahead and create a PHP file called mapper.php under the script directory.

user@computer:$ touch script/mapper.php

Now let’s look at the basic structure of a PHP mapper.

//this can be anything from reading input from files, to retrieving database content, soap calls, etc.
//for this example I'm going to create a simple php associative array.
$a = array(
'first_name' => 'Hello',
'last_name' => 'World'
//it's important to note that anything you send to STDOUT will be written to the output specified by the mapper.
//it's also important to note, do not forget to end all output to STDOUT with a PHP_EOL, this will save you a lot of pain.
echo serialize($a), PHP_EOL;

So this example is extremely simple.  Create a simple associative array and serialize it.  Now onto the reducer.  Create a PHP file in the script directory called reducer.php.

user@computer:$ touch script/reducer.php

Now let’s take a look at the layout of a reducer.

//Remember when I said anything put out through STDOUT in our mapper would go to the reducer.
//Well, now we read from the STDIN to get the result of our mapper.
//iterate all lines of output from our mapper
while (($line = fgets(STDIN)) !== false) {
    //remove leading and trailing whitespace, just in case 🙂
    $line = trim($line);
    //now recreate the array we serialized in our mapper
    $a = unserialize($line);
    //Now, we do whatever we need to with the data.  Write it out again so another process can pick it up,
    //send it to the database, soap call, whatever.  In this example, just change it a little and
    //write it back out.
    $a['middle_name'] = 'Jason';
    //do not forget the PHP_EOL
    echo serialize($a), PHP_EOL;
}//end while

So now we have a very simple mapper and reducer ready to go.


So now let’s run it and see what happens.  But first, a little prep work.  We need to specify the input directory that will be used when the job runs.

user@computer:$ mkdir input
user@computer:$ touch input/conf

Ok, that was difficult.  We have an input directory and we’ve created an empty conf file.  The empty conf file is just something that the mapper will use to get started.  For now, don’t worry about it.  Now let’s run this bad boy.  Make sure you have your JAVA_HOME set, this is usually in the /usr directory.  You can set this by running #export JAVA_HOME=/usr.

user@computer:$ hadoop/bin/hadoop jar hadoop/contrib/streaming/hadoop-0.20.2-streaming.jar -mapper script/mapper.php -reducer script/reducer.php -input input/* -output output

So here’s what the command does.  The first part executes the hadoop execute script.  The “jar” argument tells hadoop to use a jar, in this case it tells it to use “hadoop/contrib/streaming/hadoop-0.20.2-streaming.jar”.  Next we pass the mapper and reducer arguments to the job and specify input and output directories.  If we wanted to, we could pass configuration information to the mapper, or files, etc.  We would just use the same line read structure that we used in the reducer to get the information.  That’s what would go in the input directory if we needed it to.  But for this example, we’ll just pass nothing.  Next the output directory will contain the output of our reducer.  In this case if everything works out correct, it will contain the PHP serialized form of our modified $a array.  If all goes well you should see something like this:

user@computer:$ hadoop/bin/hadoop jar hadoop/contrib/streaming/hadoop-0.20.2-streaming.jar -mapper script/mapper.php -reducer script/reducer.php -input input/* -output output

10/12/10 12:53:56 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
10/12/10 12:53:56 WARN mapred.JobClient: No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
10/12/10 12:53:56 INFO mapred.FileInputFormat: Total input paths to process : 1
10/12/10 12:53:56 INFO streaming.StreamJob: getLocalDirs(): [/tmp/hadoop-root/mapred/local]
10/12/10 12:53:56 INFO streaming.StreamJob: Running job: job_local_0001
10/12/10 12:53:56 INFO streaming.StreamJob: Job running in-process (local Hadoop)
10/12/10 12:53:56 INFO mapred.FileInputFormat: Total input paths to process : 1
10/12/10 12:53:56 INFO mapred.MapTask: numReduceTasks: 1
10/12/10 12:53:56 INFO mapred.MapTask: io.sort.mb = 100
10/12/10 12:53:57 INFO mapred.MapTask: data buffer = 79691776/99614720
10/12/10 12:53:57 INFO mapred.MapTask: record buffer = 262144/327680
10/12/10 12:53:57 INFO streaming.PipeMapRed: PipeMapRed exec [/root/./script/mapper.php]
10/12/10 12:53:57 INFO streaming.PipeMapRed: MRErrorThread done
10/12/10 12:53:57 INFO streaming.PipeMapRed: Records R/W=0/1
10/12/10 12:53:57 INFO streaming.PipeMapRed: MROutputThread done
10/12/10 12:53:57 INFO streaming.PipeMapRed: mapRedFinished
10/12/10 12:53:57 INFO mapred.MapTask: Starting flush of map output
10/12/10 12:53:57 INFO mapred.MapTask: Finished spill 0
10/12/10 12:53:57 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000000_0 is done. And is in the process of commiting
10/12/10 12:53:57 INFO mapred.LocalJobRunner: Records R/W=0/1
10/12/10 12:53:57 INFO mapred.TaskRunner: Task 'attempt_local_0001_m_000000_0' done.
10/12/10 12:53:57 INFO mapred.LocalJobRunner:
10/12/10 12:53:57 INFO mapred.Merger: Merging 1 sorted segments
10/12/10 12:53:57 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 70 bytes
10/12/10 12:53:57 INFO mapred.LocalJobRunner:
10/12/10 12:53:57 INFO streaming.PipeMapRed: PipeMapRed exec [/root/./script/reducer.php]
10/12/10 12:53:57 INFO streaming.PipeMapRed: R/W/S=1/0/0 in:NA [rec/s] out:NA [rec/s]
10/12/10 12:53:57 INFO streaming.PipeMapRed: Records R/W=1/1
10/12/10 12:53:57 INFO streaming.PipeMapRed: MROutputThread done
10/12/10 12:53:57 INFO streaming.PipeMapRed: MRErrorThread done
10/12/10 12:53:57 INFO streaming.PipeMapRed: mapRedFinished
10/12/10 12:53:57 INFO mapred.TaskRunner: Task:attempt_local_0001_r_000000_0 is done. And is in the process of commiting
10/12/10 12:53:57 INFO mapred.LocalJobRunner:
10/12/10 12:53:57 INFO mapred.TaskRunner: Task attempt_local_0001_r_000000_0 is allowed to commit now
10/12/10 12:53:57 INFO mapred.FileOutputCommitter: Saved output of task 'attempt_local_0001_r_000000_0' to file:/root/output
10/12/10 12:53:57 INFO mapred.LocalJobRunner: Records R/W=1/1 > reduce
10/12/10 12:53:57 INFO mapred.TaskRunner: Task 'attempt_local_0001_r_000000_0' done.
10/12/10 12:53:57 INFO streaming.StreamJob:  map 100%  reduce 100%
10/12/10 12:53:57 INFO streaming.StreamJob: Job complete: job_local_0001
10/12/10 12:53:57 INFO streaming.StreamJob: Output: output

If you get errors where it’s complaining about the output directory, just remove the output directory and try again.


Once you’ve got something similar to the above and no errors, we can check out the result.

user@computer:$ cat output/*


There we go, a serialized form of our modified PHP array $a.  That’s all there is to it.  Now, go forth and Hadoop.