Tower Of Power

(Originally posted on my company's blog)

I've been talking a lot about automation as of late and posted a generic, overload broad view of the tools and utilities I use here at Net Perspective. However, I wanted to go into detail about my real world uses of a particular tool, Phing.

In my time with Net Perspective (and our previous existence), I've completely dreaded live deployment days, and yes they were full days. An entire day dedicated to waiting for FTP to finish uploads, chmod'ing directories (because Plesk hates me), setting up databases, and testing. All of this despite the fact that we did periodic staging deployments on the same server!

I guess a lot of thanks goes out to Pro PHP: Patterns, Frameworks, Testing, and More by Kevin McArthur. While I found this book to feel more like a Zend Framework tutorial, it did provide a brief introduction to 2 utilities I use here at work, only one of which we care about at the moment: Phing.

Phing is a build environment based on Ant (familiar to those in Java world). For those not familiar, Phing uses XML based build files with tags representing tasks and task groupings. At Net Perspective, I use Phing build scripts for everything from packaging up tarballs of utilities we've published (simple jQuery plug-ins, etc.) to automating the live deployment of client sites. The latter I'm particularly happy with as my deployment time has been reduced from a few hours between 10 minutes max (for a brand new, fresh upload) and about 30 seconds (for an update).

Basic Phing usage is pretty self explanatory, however you may wish to search Google for Phing tutorials, or read my post I linked above.

Further more, my build scripts make use of custom command executions requiring both the execution environment to be *NIX based and for you to read this article on using rsync over ssh without a password (using pre-shared keys).

Read the rest of my post here...

I'm fairly excited because as of only a week or so ago I found the closures rfc, and not only did I find that the rfc has a working patch, apparently it's already in PHP 5.3.

You see, apparently it's not good enough that we get name spaces and late static binding, oh no, we get closures too!

I downloaded the Win32 snapshot from PHP snapshots page and indeed, closure support was included.

 
<?php
$closure = function ($args) use ($global) { /*Body*/ };
?>
 

According to the rfc, one must manually define external variables used within a function, however in my own tests you can still use the global keyword. The difference between the two is the use statement preserves the value of $global at creation and the global keyword would work as you would expect it to with a normal function. For example:

 
<?php
$global = "Global Variable";
 
$closureUse = function ($arg) use ($global) { echo $global . " - " . $arg; };
$closureGlobal = function ($arg) { global $global; echo $global . " - " . $arg; };
 
echo "Basic Closure Tests\n";
echo "-------------------\n\n";
 
echo "\$closureUse('test'): ";
$closureUse('test');
echo "\n";
 
echo "\$closureGlobal('test'): ";
$closureGlobal('test');
echo "\n";
 
echo "\n";
echo "Global Closure Tests\n";
echo "--------------------\n\n";
 
$global = "Global Variable (Changed)";
 
echo "\$closureUse('test') (changed \$global): ";
$closureUse('test');
echo "\n";
 
echo "\$closureGlobal('test') (changed \$global): ";
$closureGlobal('test');
echo "\n";
?>
 

Would output:

 
Basic Closure Tests
-------------------
 
$closureUse('test'): Global Variable - test
$closureGlobal('test'): Global Variable - test
 
Global Closure Tests
--------------------
 
$closureUse('test') (changed $global): Global Variable - test
$closureGlobal('test') (changed $global): Global Variable (Changed) - test
 

You can, as well, return references with a closure by putting the & between the function keyword and the parenthesis.

 
<?php
$closureReturnsReference = function & ($args) use ($global) { /*Body*/ };
?>
 

Just one of many shiny fancy things to look forward to in PHP 5.3.

Edit 7/23/2008:

I should mention to the people that trashed my examples on reddit, I'm sorry for assuming you knew what closures were and the the real world uses of them. I will try to refrain from giving a concise example of how a new language feature interacts with existing features and conventions, especially in PHP where things are a bit disorganized.

I apologize about the month without updates. Our business manager and lead designer ("Director of Operations" and "Creative Director") both had to go to the hospital at about the same time and the shortest stay was almost 2 weeks. They both pulled through just fine and everything is back to normal.

I hope to be back sometime later this week with some visible signs of awesomeness on a side project of mine.

In the mean time I've been playing with Flex Builder 3 (hooray for free Educational licenses) and while it's cool, so far the documentation is hindering my development efforts. I have a project planned where I intend to have an AIR client and am evaluating using Flex instead of HTML+JavaScript. We'll see how far that goes.

And finally I've been playing around with FriendFeed and have an account setup. I, for one, welcome our new stalker overlords.

One of the guiding principles I have lived by the came from my middle school swimming coach. He told us:

Don't do more work than you have to.

On first glance, it doesn't seem to be very good advice. However when you realize that what he was getting at was we shouldn't waste energy on a poor stroke, the quote takes on a much deeper and more profound meaning. It is this principle that takes a good developer and makes him great.

Profile of a Good Developer

Good developers are almost a dime a dozen. Walk in to any computer science course or web development shop and you'll find that the majority, if not all, are good developers.

These "good" developers can sit down and work through an object oriented architecture, they can construct their SQL queries, and can write an application that just works.

A "good" developer checks in his source code at the proper times, he comments his code as well as could be expected, and you can usually be guaranteed his code will compile and execute with very little fuss.

Profile of a GREAT Developer

What separates the great developers from the good developers is LAZINESS! You will find that the good developers, while producing clean, working code, operate much slower than great developers, and for the reason that separates the good from the great.

Good developers are set in their ways of just manually working with everything. A good developer handles deployment with FTP, and updates database structure by hand. In the end, the good developer ends up wasting time doing menial tasks. FTP alone can kill up to 30 minutes of your day. Good developers are very set in their ways and do not like new things introduced.

Meanwhile GREAT developers HATE menial tasks. They abhor having to drop into the console every time they wish to test or deploy, and loath dropping into the database to manually update structure. So these GREAT developers begin to write scripts and integrate all their tools to reduce their menial overhead as much as possible.

The GREAT developer is lazy because they do not want to do things themselves, so they spend 20 minutes automating their tasks so they'll never deal with it again. And when you think about it, when you could spend 20 minutes once writing automated deploy scripts but instead spend 20 minutes every deploy in FTP, you are doing more work than you have to.

What Does This Mean For Me?

All of us developers should strive to be as lazy as possible. Do you have a task you have to do more than 3 times total? Write a script or a build task for it. Do you spend a lot of time testing features? Automate your testing. And if possible, integrate the systems so you never even have to run those scripts manually, reducing user intervention to one step.

I know I try to be as lazy as possible, do you?

Edit 6/10/2008:

I feel I should make sure everyone knows that I am to an extent being facetious with the word "laziness" but I'm being completely serious on the "don't do more work than you have to." While a good developer can handle long, arduous tasks that are prone to human error, a great developer avoids them in the first place.

I recently had fun with an all-to-common issue with SQL driven websites: hierarchical data. For those who don't like big words, think trees. Other people have already discussed storage methods, and I would actually highly suggest you read the writeup if you haven't already.

While it is fairly straightforward to deal with, in our case we use HTML_QuickForm to handle our forms and are using QuickForm's hierselect to select a category.

The issue starts showing its face in 2 distinct areas: (1) the client is not yet sure how deep they need their categories to go, and (2) the hierselect requires a very specific format of data to be passed in.

Continue reading ‘Hierarchical Data With PHP and MySQL’ »

Peter Zaitsev posted a very interesting test on how PHP and Apache handle runaway PHP scripts.

I'm sure all of us have had a long executing SQL script or at least a runaway script, and he points out even with ignore_user_abort set to FALSE and max execution times set both in PHP and Apache, a runaway SQL query can execute beyond the timeouts, even when aborted by the user (stop button on the browser).

Even using the function connection_aborted() to check for a user abort fails.

However, the script will cease on a user abort if it performs regular ob_flush(); flush(); commands. For example:

<?php
echo("Hello");
for($i=0;$i<10000;$i++)
{
	sleep(1);
	echo('.');
	ob_flush();
	flush();
}
?>

Just one more reason why I love the MySql Performance Blog.

I just rolled out some new automation tools for a few projects here at work and so far I've been extremely happy.

Much to my embarrassment, development has previously been outside of source control due to the fact that we develop sites, we don't deploy packaged applications, and we don't have a cohesive IT setup (everyone sets up their desktop to their liking so maintaining consistent development environments across all computers is difficult).

However, thanks to SVN, Xinc and Phing (and DBDeploy), this has changed! Now everything is in source control and automatically deployed to our dev server upon commit. I am currently talking with Arno about perfecting svn tag monitoring to automate staging and (possibly) live deployments, so I'll post about what I did when that's finished.

The great thing about this setup is all pieces are technically interchangeable. If you don't like Xinc you can use CruiseControl. If you don't like Phing you can use Pake, or a shell script even. If you don't like DBDeploy you can roll your own setup or swap it out for your database versioning system of your choice!

However this post will cover Xinc, Phing, and DBDeploy as (a) I have experience with them and none of the others, and (b) they integrate extremely well (Xinc and Phing's primary distribution method of choice are PEAR channels).

Continue reading ‘Automating the Development Workflow’ »

 
from indiana import jones
import whip
import ruins
 
cave = new Ruins()
 
trap = indiana.steal( cave.treasure )
indiana.escape( trap )
 

ImportError: Snakes? I hate snakes!

One of my favorite features of C# is Partial Classes. For the uninitiated, it is a way of defining a class in two separate locations. Very useful when you have code generation utilities such as LINQ.

Unfortunately, PHP has no such feature (though if anyone's listening it would be a great feature to add to the PHP6 feature list), however thanks to the magic of __call($method, $args), __get($key), and __set($key, $value) overload functions as well as passing by reference (aah the good ol' &) we can imitate partial classes.

The idea behind this partial class hack is to instance a copy of the partial class in question and have our magic functions forward any undefined requests to the partial class.

The partial class (probably with the naming convention Partial_CLASSNAME) will contain a reference to the main class and also have magic functions forwarding undefined requests to the main class. The reason why we have our magic functions in the partial class is so that any internal references can still be made (methods in Partial_CLASSNAME must have access to the methods and members in CLASSNAME).

The constructor in the main class will automatically seek out the partial class (and additional coding can be done to seek out more than 1 partial class as well as do some integrity checking) so that the programmer does not have to intervene to form the 'full class'.

Continue reading ‘Partial Classes in PHP’ »

My copy of Pro PHP: Patterns, Frameworks, Testing and More by Kevin McArthur has already been worth the money I spent on it.

In our, albeit slow, upgrade to more modern development practices (code version and unit testing being the last elements on my list), I was particularly fearful of code versioning namely concerning issues of automating a build process to export a copy of the repository to our shared development server, or just have a local server installed for everyone one.

However Chapter 8 has pointed me to two programs that I sincerely regret not having heard of until now: Phing and Xinc.

Phing is a build system written in PHP that works extremely well with PHP and PHPUnit. You can configure your own build targets, for example like "get" (SVN update essentially), "test" (run PHPUnit or any other unit testing), "try" (copy to a development server), and "deploy" (copy to your live server).

Xinc is a continuous integration server. Essentially, it monitors an SVN repo and performs a user defined action (usually running a phing buildscript) on any changes to the repo. Which is awesome because its solves the issue of updating the dev server. Having Xinc run a > phing try on SVN updates is exactly what we need! I'm even having delusions of grandeur of having it monitor tags for doing live deployments as well.

Don't take my word as gospel, I haven't actually installed and tested these systems out, but when I do you can rest assured I will be writing about it.