PHP File Uploads And Background Conversion: Oopsie

So I spent an inordinate amount of time tracking down a bug recently for a former employer in a system that accepts media file uploads and converts them on the fly.

The system spawns off a small script that manages mplayer/ffmpeg/lame into the background and holds on to the PID to track the conversion process. HOWEVER, there was a problem in the system where large files were being ignored and no error messages were popping up.

What ended up happening was I forgot to take into account PHP’s file upload behavior in the conversion process.

When PHP accepts a file upload it creates a temporary file (usually in /tmp/). The caveat is PHP frees (read: deletes) the temporary uploaded file at the end of script execution.

So, while the converters could convert a small file in the same encoding as the desired output, a large file (say, a 10mb MPEG) that needed to be converted to a different format (say, a FLV), the converter would begin working in the background but when the PHP script finished executing, PHP would delete the file right out from under the converter’s feet.

So, a tip to anyone doing anything similar, go ahead and at least rename/move the file (I created a unique MD5 hash from the current time plus a few other things and use that as the new file name) and pass that to any background processing functions so PHP won’t delete the file at the end of processing.

It will save a WHOLE lot of headaches for you in the future.

Also, if anyone is interested, I will probably write up a tutorial on how to write a PHP script that will convert your MP3’s in the background.

PHP, Mumbles (Growl), and DBus: Sweeet

So, after reading Mark Shuttlework’s blog on ideas for notifications in Ubuntu (basically mimicking Growl notifications for the Mac), I decided I wanted that kind of functionality, but… you know, NOW!

There are a multitude of options available but currently I’m liking Mumbles. Unlike, say, Specto, which does the monitoring itself, Mumbles provides a DBus interface, a command-line app named mumbles-send, and (I’m not sure if it’s implemented in the current stable download) libnotify support. Woutc, from the Specto project, commented below explaining that Specto is not intended to be a Mumbles competitor, but a package to easily monitor system internals (I quote “…the purposes from mumbles and specto are different…specto is monitoring what happens outside your desktop, mumbles monitors what happens on your desktop (or in your network)…”) Apparently he has plans to build a Mumbles plugin so that one can optionally have Specto send its messages to Mumbles for display.

I decided the best easiest route is to access the internal DBus API, however the forums and other resources on the Mumbles site… well… just plain suck. And by suck I mean tell you that something exists and… thaaats about it.

D-Feet browsing Mumble's Growl interface

D-Feet browsing Mumble's Growl interface

Well in my Google quest I discovered the existence of D-Feet, a DBus debugging tool (on Ars of all places). Thanks to a quick sudo apt-get install d-feet I found the existence of a public interface in info.growl.Growl that allows for a Notify(title, message) signal to be passed.

Well, with a place to access all I needed was DBus integration with PHP (because I want to start sending debug notification via Growl/Mumbles like this idea on the Zend Framework incubator).

Luckily, GREE Labs provides a DBus C extension for PHP that was easily downloaded and installed. Once installed I read the documentation to get an idea on how to use the API (which is a 1:1 mapping to the DBus API).

After a bit of hacking I finally came up with an alpha product:

<?php
function null_callback()
{
        var_dump(func_get_args());
}

$dbus = dbus_bus_get(DBUS_BUS_SESSION);

$m = new DBusMessage(DBUS_MESSAGE_TYPE_SIGNAL);

$m->setPath('/info/growl/Growl');
$m->setInterface('info.growl.Growl');
$m->setMember('Notify');
$m->appendArgs('Hello World!');
$m->appendArgs('Lorem Ipsum Dolor Sit Amet. (' . time() . ')');

$r= $dbus->send($m);

After running the script in the console, did I see success?

Mumbles and PHP Test: Success!

Mumbles and PHP Test: Success!

Damn right I did!

Though with a caveat:

Warning: dbusconnection::sendwithreplyandblock(): dbus_connection_send_with_reply_and_block() failed (Traceback (most recent call last):
File "/var/lib/python-support/python2.5/dbus/service.py", line 643, in _message_cb
(candidate_method, parent_method) = _method_lookup(self, method_name, interface_name)
File "/var/lib/python-support/python2.5/dbus/service.py", line 244, in _method_lookup
raise UnknownMethodException('%s is not a valid method of interface %s' % (method_name, dbus_interface))
UnknownMethodException: org.freedesktop.DBus.Error.UnknownMethod: Unknown method: Notify is not a valid method of interface info.growl.Growl
) in /home/daniel/test.php on line 21

Call Stack:
0.0002 64496 1. {main}() /home/daniel/test.php:0
0.0024 66372 2. dbusconnection->sendwithreplyandblock() /home/daniel/test.php:21

Apparently Mumbles seems to be throwing an Exception that it’s failing to to find the Notify signal (though everything works correctly). I guess I could ignore this, use the error suppressor (@), or even yell at GREE Labs to have the objects throw exceptions so I can catch them… I guess I’ll hack on it some more and report back. I should probably spend more time learning the DBus specs since this is my first project playing with DBus…

Update: Though it’s not listed on the API page for PHP DBus’s API, there is a method send() that takes only a single argument (the message object).

Edit:

I saw a question on Reddit asking what is the use of this technique if it’s limited to the desktop it was called on and PHP is primarily a server side language. Why not do this in Python or Perl?

Well, Mumbles was written in Python so there’s no point in me doing this in Python: it’s already been done.

However, the primary use case of a technique like this is having a web app post notifications and errors for the developer. When I work on a site I have a local copy running on my desktop and/or laptop, so when it posts Growl/Mumbles notifications, I get them on my desktop. It’s great for situations where, say, I have a page in my PHP app that never displays its contents because it processes data then redirects to another page. If a warning or other non-fatal error that I should really fix occurs, then I would normally have to dig through the system wide PHP error log (if you even have error logging enabled). However, if I wrote a custom error handler that posts errors to Growl/Mumbles as they happen, when I visit a transitory page like I described I get little Growl/Mumbles notifications showing that I screwed up!

On Looking For A Job And Online Reputation

As many of you know I am finishing up my last semester at school and at the same time on the lookout for jobs (For those interested, you can look at my résumé).

As a part of the last few classes I have to finish, I am taking a CPSC Seminar course which so far has consisted of talks from the Texas A&M Career Center. A lot of talk has been on managing your personal on line reputation. Besides the obvious “don’t post pictures of you on Facebook doing keg stands” or “don’t post embarrassing, topless photos of yourself”.

Me (without my shirt) at Fish Camp

A lot of these tips are based on the idea that you don’t want potential employers that read your blog (Hi prospective employers!) and research your internet famousness to screen you based on what they gather from their research. From not hiring you because all of your Facebook pictures involve you being trashed at a party to an extreme scenario of a Baptist corporate recruiter seeing that post you did on how much you hate Baptists. The theme is the same: don’t let the company screen you unnecessarily. Usually the safe bet is to TIGHTLY titrate the amount of information you post on the internet.

However if one does a cursory Google search of my name, one will find that I have not shied away from associating my content with myself. Granted the first page of Google results is mayhaps even sexy to recruiters (hey look, a couple of my technical blog posts linked on PHPDeveloper.org, my LinkedIn account), if one can see or gets access to my Facebook profile or read the rest of my blog, one will notice that I do not hesitate in expressing myself.

Why would I put myself out there like that? Because while companies are out shopping for employees, I am out shopping for companies. And such shopping extends beyond basic research on a company and visiting their work environment.

How many of you found the above photo offensive? uncouth? unprofessional? ridiculous? How many people would put that person lower on a hiring list because of the information you’ve gathered so far in seeing that photo? If you were, I would bid you adieu (yes, that picture is of me) because a company that would penalize me without investigating as to why a picture is ridiculous. Would the backstory that this picture was taken of me performing a skit as a Fish Camp counselor in front of 150 freshmen, trying to provide a little humor as we spent a little over 3 days preparing them for their time at Texas A&M, change your opinions?

The way I see it is the company that would weigh so heavily on the superficial and not on the content (leeway given to the fact that not every company will have the time to do full background research on me) is not a company I wish to work for. If they do this while they’re in the process of hiring me, what do I have to look forward to in my future with that company? Apparently not much.

Maybe this is just an excuse to myself for having a lot of information about me out in the wild, however I feel like I have carefully filtered information about me onto the internet to aid in my job search, and to help potential recruiters understand my personality a bit better. Because as cantankerous as I sound above, the “not being hired by X firm” works both ways. I don’t want to work at a firm where I will not be a good fit. I’ll cause friction, drop morale, and generally be ineffective.

Plus this post has been a great chance to explain that picture.

My brother’s actually blogging!

Hey everyone, my brother is actually blogging! Check out Eric Cousineau’s blog. Right now it’s looking like the focus of his posting will be about Digital Mars D programming language. Everyone go give him some traffic and encourage him to keep posting!

Blog Updates

Just made a few changes to the blog. I added a recaptcha to the comment system as well as subscribing to the comments via email. I also updated the code highlighter so we get the pretty boxes and line numbering (and the code will look much nicer on skinnier browsers).

I also added my résumé for those interested in me. If you have any suggestions for improvements on the résumé feel free to let me know.

The code highlighter was a big enough change that I think sometime in the future I’ll be updating the blog design in the future. However for now I’m just going to try to come up with stuff to post!

OSS Bugs Are Features

I’ve been thinking a lot lately about a presentation given by whurley at a refreshBCS meeting a long time ago. He gave his presentation on Open Source Software, only SLIGHTLY an era of his expertise (just in case you can’t tell, that was sarcasm) and raise an interesting point.

He talked about how when BMC Software picked him up to head up their open source efforts, he battled a little bit the developers. They wanted to make sure the code was perfected, polished, and at its best whenever they started dumping it on the public. Of course they’d want to do this, releasing code as OSS is almost equivalent to deciding to present yourself to the work naked and without makeup. They wanted their friends who could see the code to be in awe, and their code base to be a shining example of a project done right.

Whurley wanted them just to release it. Not because he wanted it now, not because of impatience or deadlines or anything else. He wanted them to just release it because, and I’m paraphrasing him, releasing open source thats a little buggy will actually help grow and foster the project. Granted, you do not introduce bugs, you just open and display the code and start making releases even though there are still tickets in your bug tracking system.

He gave the hypothetical example of the stereotypical arrogant OSS developer finding a bug in BMC’s FooBar software, calling the BMC devs dumbasses, fixing it (because it was a small and easy bug), and now being invested in the software. You don’t really lose much reputation, but you gain an invested and loyal following.

While this sounds good and true, and it makes a lot of sense (if there’s nothing to do, OSS developers will never get involved), it never really clicked for me until I associated another story with whurley’s principle.

I meet Elizabeth Marie Smith (aka auroraeosrose) at ZendCon08 and had good times talking with her and many of the other #phpc members. At one point in the conference, at an opening for one of her talks (a brief overview of PECL extensions), she told the audience how she got into programming PHP and PHP C extensions. She wasn’t a computer science major, she only started in the early 2000’s (prior to that not much computer science experience).

What had happened was, to the best of my recollection and her seven things post, in the early days of 2001-ish she was really big into Anime and wanted to start a fan site. Her dad helped her install a forum (written in PHP) on a server he was running out of his house on a cable connection.

Well, as with many of the forum projects, there were bugs to be had in the software. Elizabeth was dedicated to the idea of her site and decided to break open the forum and dig around to fix the bugs. Fix them she did but everything kinda snowballed from there. In fact it was bugs in the PHP-GTK extension that caused her to learn C (not an easy feat) and become the maintainer of PHP-GTK as well as the Windows port of PHP maintainer.

It was bugs and problems in the software that caused her to be where she is now, it was bugs in the OSS world that caused her to latch on to PHP-GTK and the Windows PHP port and get them to where they are today. Though she told this story a long time ago, and whurley’s presentation was even further back yet, it only just now clicked that I had seen a real life example of whurley’s principle of “just publish it, flaws and all.”

The lesson from all this can even be applied outside of OSS development. It’s a lesson on perfectionism and can be applied towards my personal development practices. I know that agile teaches to publish early and publish often, fixing bugs and adding features on the way, but perfectionism still plagues me. I have maybe 10 drafts on this blog on small little PHP and Python topics I could publish that I never do because I feel it’s not good enough, or too simple. I have had code at work that I could have released a week ago but instead I obsess over making sure its “perfect” when a simple staging deployment will keep everyone happy, keep my moral up, and show me where the REAL bugs are.

New extended life battery for my T61; or, holy crap I’m mobile again!

So my dinky default (6-cell I believe, maybe 4-cell) battery that came with my Lenovo Thinkpad T61 has been giving out on me. I believe my problem was I never flexed the battery when I first got it and now I was only getting about an hours worth of work out of my laptop.

While I didn’t have super high expectations of battery life out of my 15″ laptop, I did specifically get the Intel graphics adapter to shave off my power draw and did everything available to reduce power (Intel speed step turned to power saving, turning down the backlight, increasing writeback time, turning on laptop_mode, disabling the bluetooth radio, disabling compiz, etc.), I was still getting an abysmal battery life out of my laptop (it wouldn’t even last through a single class).

Well i finally got tired of it, walked down the street to the batteries plus, and got myself the ‘off brand’ Rayovac Extended Capacity battery for the T60 (which is almost exactly the same as the T61).

All I can say is I’m so happy I’m getting 4+ hours out of my laptop now and I don’t even have to disable compiz or do excessive tuning! The battery jutting out of the back of the laptop (as you can see in this image, with thanks to notebookcheck.net and Google image search for the image) doesn’t bother me at all, and in fact it provides a nice grip for when I’m holding the laptop up and walking around.

Hopefully now I’ll be able to get more work done on my laptop and not be tied to a wall outlet (which defeats the purpose of a laptop in the first place)…

Zend Framework Module Init Script (Controller Plugin)

Well, it’s been a while since I’ve done a PHP update hasn’t it? Well alls well here in CStat, I have a quickie for you folks.

Recently at work I had the need to run a script before every single controller (namely to add a plugin folder to Dwoo) for a specific module that I did not desire for any other modules.

I could have subclassed all my controllers to extend a custom action controller that handled this in the init() method, however I’m lazy so I wrote a quick Zend Controller Plugin to handle this for me.

What it does is grabs the module name from the request object on routeShutdown (the routeShutdown happens after the route has been parsed and the request object has been generated, but before the action controller is instantiated and executed). From there it uses Zend_Font_Controller::getInstance()->getModuleDirectory(...) to grab the literal path to the module directory, tacks on our init file name (in our case “init.php”), and if it exists execute it in its own clean environment.

The init.php file has access to $this, where $this is an instance of the controller plugin (in particular we can do $this->getRequest() inside the init script).

The code is as follows:

<?php
/**
 * This checks the current request module's directory for an initFile (defaults
 * to init.php) and runs it before the controller is loaded.
 *
 * @copyright 2009 Daniel Cousineau
 * @license http://opensource.org/licenses/mit-license.php MIT License
 * @version 0.1.0
 */
class My_Controller_Plugin_ModuleInit extends Zend_Controller_Plugin_Abstract
{
	public static $initFileName = "init.php";

	/**
	 * @param Zend_Controller_Request_Abstract $request
	 * @return null
	 */
	public function routeShutdown(Zend_Controller_Request_Abstract $request)
	{
		$moduleName = $request->getModuleName();

		$moduleDirectory = Zend_Controller_Front::getInstance()->getModuleDirectory($moduleName);

		//Trim the paths and filenames to prevent any problems
		$initFile = rtrim($moduleDirectory,'/\') . '/' . ltrim(self::$initFileName,'/\');

		$this->runInitFile($initFile);
	}

	/**
	 * Run the file in its own cleaned scope
	 *
	 * @param string $_initFile location of the input file
	 * @return null
	 */
	protected function runInitFile($_initFile)
	{
		if( file_exists($_initFile) )
			include_once $_initFile;
	}
}

As always all code I post on this blog is covered under the MIT license.

My Seven Things (Or: Hey look, the cool kids picked me to play on the team!)

I just got tagged by the BAMF Chris Cornutt, creator of the (in)famous site PHP Developer.org to participate in the “Seven Things” blog tagfest.

  • I’m barely 21 years old. For some reason people are always surprised by this. I’m convinced it’s my beard. That and how many 21 year olds ask for money to attend ZendCon ‘08 for their birthday?
  • I was a state science fair finalist in 8th grade I did my science fair project on Artificial Intelligence and refuse to release any more details or information on my project. Let’s just say it was an early point in my programming and computer career.
  • I am an Eagle Scout as well as hold my Vigil Honor in the OA. I was very involved with Boy Scouts in my youth (and plan to continue my involvement in the future when time, money, and internal BSA politics allow), grabbing my Eagle Scout, hiking Philmont twice (65 miles the first trip, 110 the second trip), and getting my Vigil Honor in the OA posthumously.
  • I originally came to Texas A&M to be a CG artist (focusing on SFX). I picked A&M because it had a great graduate CG program (called the Viz Lab) and figured grabbing a CPSC with focus on Art at A&M would help out. I still am fascinated by computer graphics, but realized I was more interested in software architecture when I had more fun planning the OOP design of my ray tracer than actually filling in the guts of the ray tracer.
  • I used to play “teacher” with my younger brother. I would come home from school excited about the math and science I learned (everyone yell NERD) and teach my younger brother (3 years younger) what I learned on this badass little chalkboard in my room. I probably should regret this because my brother just showed up to college taking Calculus 3 his first semester (and acing it).
  • I love Psychology. It was my favorite class in High School (aced the AP test with a 5) and my only regret about college is I never had the time to take any psych classes (though Human-Computer Interaction was an awesome class). If I didn’t do my degree in computer science, I would have picked Psychology as my major.
  • I was an avid trumpet player in Highschool. We’re talking top 2-3 chairs, section leader, All State Solo and Ensemble, $2,000 hand made horn, and 7 years playing experience under my belt before I retired.

Now I kinda got tagged late in the game, but I still need to tag some people:

  • Jeff Carouth, a former co worker, recent php|architect contributor, and the first proof that there were developers that were actually smarter than me in College Station, Texas (but certainly not the last :P).
  • Chris Weldon, the man who started/ran the local PHPUG and the man who showed me that Jeff and I were not the only PHP users in College Station.
  • Ben Floyd, a really great developer I’ve had the pleasure of meeting through refreshBCS. Though he’s primarily a .NET developer (with some serious skills, the Texas A&M Campus Maps site is basically all him), he has done some PHP projects (and even though he gritches about it, I know he’s enjoyed working in the language, even just a little bit).
  • Ben Burket, I would like to see him blog again (he’s another friend I meet through refreshBCS), he’s “my ruby friend” (but a merb developer, noone throw lettuce). I poke fun at his language, he pokes fun at mine, and we both meet at the middle with python.
  • Andrew Ellis, a former coworker who is a security guru. No doubt he will exploit some obscure vulnerability in wordpress to take down my site for tagging him.
  • whurley, the “hey look I know someone famous” friend who’s been exceptionally nice to me. The guy’s an evil genius and he scares the crap out of me, but hey, you only get better playing a smarter opponent. I don’t know if he’ll ever see this or have time to post, regardless you should follow this link and read his stuff.
  • “Henrick”, I read his blog and he posts regularly with some pretty interesting stuff. I have no idea who he is and now officially call him out!

And, of course, the rules to pass on:

  • Link your original tagger(s), and list these rules on your blog.
  • Share seven facts about yourself in the post – some random, some wierd.
  • Tag seven people at the end of your post by leaving their names and the links to their blogs.
  • Let them know they’ve been tagged by leaving a comment on their blogs and/or Twitter.

MiniBarBCS in Bryan, Texas THIS FRIDAY (November 14th, 2008)!

This is a post to make sure anyone interested in development, design, or new media entrepreneurship shows up to MiniBarBCS this Friday starting at 5pm. The event is being held on the 5th floor of the Varisco Building in Bryan, Texas (219 N. Main St., Bryan, TX 77803).

So far the proposed topics cover everything from iPhone development to “Design for Developers” to .NET to PHP, Ruby, Coworking, and anything else under the sun.

So if you’re around the area this Friday, show up and join in the fun! Just make sure to RSVP in the wiki linked above (the password/invite key is “c4mp”) or the Facebook event.