Security::cipher() problems and Suhosin

This problem can now be solved by setting the encryption type in the CookieComponent to rijndael.

Over a week ago I migrated one of my sites to a MediaTemple (ve) Server. I installed nginx, PHP 5.3.2 (with Suhosin), MySQL, APC and Memcache. After a little configuration and debugging, everything was working smoothly except encrypted cookies. I was using my AutoLogin component and login never persisted, but regular cookies worked correctly. I thought it would be an easy fix but I was wrong (I will post another blog on this topic).

While trying to figure out the encryption problem, I debugged the CookieComponent. I tested each function individually around a fake array. I tested __implode(), __encrypt(), __explode() and Security::cipher(). My results were all over the place. When I tested the script locally, everything worked correctly and the ciphering process worked as it should (below). But on the live server, all the encryption strings would change each refresh and the un-cipher would never work.

Implode:
username|gearvOsh,password|dsadnas79dn7n1279dnadnadb1p2sdasd,hash|761c1168c645784b78f5f967160ab390,time|1286152916
Ciphered:
z~¡|ßÛí�"� =�'¥¾E¨ÿÏy� íåÇv¸”^ëS ¦ì•ˆxuŽ?��ãbm}ÜfÇ“„bËV´w±(ÐU�"éQ’Qñn²^6¿2r‰½ùé‹ÄáÄ,‡ ›d¤<ó`'ëÆìÏg$V")ËdÁ&µ‰
Base64 encode:
Q2FrZQ==.en6hfN/b7dMfij3Spb4ERaj/z3ng7eXHdriUXutTIKbslYh4dR6OP8bjYm193GbHk4Riy1a0d7Eowx+QVZbpUZJR8W6yXja/MnKJvfnpixsIxOHELIcgm2QHpDzzYCfrxhrsz2ckVggijwcpy2TBJrWJ
Using Cookie::__encrypt():
Q2FrZQ==.en6hfN/b7dMfij3Spb4ERaj/z3ng7eXHdriUXutTIKbslYh4dR6OP8bjYm193GbHk4Riy1a0d7Eowx+QVZbpUZJR8W6yXja/MnKJvfnpixsIxOHELIcgm2QHpDzzYCfrxhrsz2ckVggijwcpy2TBJrWJ
Base64 decode:
z~¡|ßÛí�"� =�'¥¾E¨ÿÏy� íåÇv¸”^ëS ¦ì•ˆxuŽ?Æãbm}ÜfÇ“„bËV´w±(ÐU�"éQ’Qñn²^6¿2r‰½ùé‹ÄáÄ,‡ ›d¤<ó`'ëÆìÏg$V")ËdÁ&µ‰
De-ciphered:
username|gearvOsh,password|dsadnas79dn7n1279dnadnadb1p2sdasd,hash|761c1168c645784b78f5f967160ab390,time|1286152916

As you can see from the debug (on my local machine), the beginning and resulting strings are the same. I then found out the problem was with Security::cipher() and srand(). I later found out that Suhosin was overriding the srand() and mt_srand() functions, which was causing my seed to always change. I tried disabling Suhosins overwrites, but to no avail.

// Did not work
ini_set('suhosin.mt_srand.ignore', 0);
ini_set('suhosin.srand.ignore', 0);

After some googling and a lot of frustration, I finally found a solution. Thanks to Edgar Valarezo, I was able to get my encrypted cookies working properly. However, I had to overwrite CakePHP's core Security class and will need to maintain it every time I upgrade Cake. Here is the new cipher() snippet for your convenience.

function cipher($text, $key = '') {
    $key .= Configure::read('Security.cipherSeed');
    $out = '';
    $textLength = strlen($text);
    $keyLength = strlen($key);
    $k = 0;
    for ($i = 0; $i < $textLength; $i++) {
        $seed = md5($key . $key[($k++) % $keyLength]);
        $mask = hexdec($seed[6] . $seed[9]); // :)
        $out .= chr(ord($text[$i]) ^ $mask);
    }
    return $out;
}

I see this as a temporary fix and would love any information on this topic. If you ran into this same problem, how did you correct? Is it possible to seed correctly with Suhosin? Thanks again!

Single or multiple feed aggregation?

I recently published a beta version of my feeds plugin for CakePHP. This plugin was previously the FeedAggregator component, but it made more sense to break it up into a datasource and model, and finally package as a plugin. The datasource is pretty straight forward as it accepts an array of URLs (RSS feeds), fetches each one through an HTTP request, parses the XML into an array, and then returns the result. The model is simply there for your convenience.

Now the dilemma I am running into is whether or not the datasource should only parse one feed at a time or multiple feeds (currently this). It can go either way: the datasource parses multiple feeds and uses the model to return them, or the datasource parses one feed and the model manages multiple connections and merging. Now the big question for you guys... Should the datasource parse one feed at a time or multiple feeds?

Currently you use the model to pass an array of URLs (through the conditions option), the limit, which fields (elements in the XML) you want returned, and some cache/feed settings. Here is a quick example:

// Multiple feed parsing
$feeds = $this->Aggregator->find('all', array(
	'conditions' => array(
		'Starcraft 2 Armory' => 'http://feeds.feedburner.com/starcraft',
		'Miles Johnson' => 'http://feeds.feedburner.com/milesj'
	),
	'feed' => array(
		'cache' => 'feedCacheKey',
		'expires' => '+24 hours',
		'explicit' => true
	)
));

And I am assuming single feed parsing would look something like this:

$feed = $this->Aggregator->find('first', array(
    'conditions' => array('http://feeds.feedburner.com/milesj'),
    'feed' => array(
        'cache' => 'feedCacheKey',
        'expires' => '+24 hours',
        'explicit' => true
    )
));

I am kind of split on how I should go about this and would really love your opinion. I am currently leaning towards multiple feed parsing (current implementation), but if someone has a good argument in not doing so, I will change it.

Making sure debug is off in production

Over a year ago I wrote about turning debug off automatically in production. That post I wrote is completely wrong (to an extent). The theory is correct but the execution was incorrect. Even one of the comments pointed out the problem, but I haven't had time to blog about it till now.

About a month ago I realized my implementation was wrong when one of my live sites was outputting MySQL errors and database information (including passwords) to all my users. Since debug in core.php was set to 2, and then disabled to 0 in bootstrap.php, the errors were being triggered before bootstrap was loaded. This was a huge problem as it printed out vital DB information.

It is an easy fix however, simply switch around the values from my previous entry. Debug in core.php should be set to 0 and in bootstrap.php it should be set to 2! That fixes the startup errors that appear before the bootstrap process.

if (env('REMOTE_ADDR') == '127.0.0.1') {
	Configure::write('debug', 2);
}

Service Operational

A little over a week ago I finally got my cast removed from my right hand. It felt so good to get the cast off after 3-4 weeks of wearing one. I was unable to move my pinky finger nor wiggle my wrist for the first hour, but my hand is good as new now. However, the knuckle does not heal back in place so I no longer have a visible pinky knuckle (so weird looking). I still have some pain and discomfort when putting pressure on it, but as long as I can use a computer I don't care.

Anyways, if you have been following me on GitHub or Twitter you may have noticed me updating my scripts a bit. My goal is to do a polish pass for all my scripts and get them to a "final" version. Hopefully this final version will be the last for all my scripts; excluding any critical bug fixes. I'm hoping to be able to finalize these so that I may work on other projects like the Armory websites and my hidden PHP 5.3 framework.

So if you have any feature requests for my scripts before they get polished, be sure to let me know! I will however be updating the Forum and Uploader plugins for a while since those are extremely popular.

I am planning to start blogging again but I am still on mandatory overtime at work, so my free time is basically non-existent. Regardless, I am just glad to be back, so expect more to come in the future!

Out of Service

Time for a short story on my hiatus and current situation. For the past few weeks I have been working mandatory overtime at my job. It basically takes up all my weekdays so now I have no time to update my scripts except on the weekends; but then I usually want to rest and not be on the computer. But last Friday I suffered a Boxers fracture on my right hand (dominant hand) and fractured my pinky metacarpal bone. By doing so, I have a splint and bandages on my right hand rendering it useless for the time being. I now have to code and use the mouse solely with my left hand; which I just did for this whole entry... very annoying.

So during all this, I have been unable to post lately nor have time to update my scripts. I am hoping by May I can start getting back on my regular routine. I can't even play games, and can barely shower (it's quite hard with one hand, especially since I can't get the splint wet). I will see you all in a few weeks!

P.S. Do not punch hard surfaces, the outcome sucks.

The widely unused and powerful setAction()

It has been quite a while since I last developed in Cake, but earlier today I spent a good 5+ hours working on some new controller internals for one of my apps. While working on this app, I thought I would give the not so used method, setAction() a spin. If you are unfamiliar with setAction(), it's a method that will internally forward one action to another. This is incredibly useful as it allows you to switch actions without having to do another HTTP request, nor having to reload all the controller and related objects; could be a huge saver on overhead.

Here's a quick example of a regular action:

public function profile($id) {
	$user = $this->User->findById($id);
	if (empty($user)) {
		$this->redirect(array('action' => 'listing'));
	}
	$this->set('user', $user);
}

What the action is doing is relatively simple. First it grabs a user based on an ID, checks to see if the result is empty, if so redirects to the users listing page, else sets the data. You could either redirect the action like I have, or you can simply throw an error message in the view saying something like "No user was found with that ID". I personally prefer redirecting in most cases, but it's a matter of preference. Lets move forward and try setAction() now.

public function profile($id) {
	$user = $this->User->findById($id);
	if (empty($user)) {
		return $this->setAction('listing');
	}
	$this->set('user', $user);
}

As you can see, I simply switched the redirect() with setAction(). Now if we visit a profile with an non-existent user, the listing action will be rendered without having to do another HTTP request. Also, do notice the use of return; this is extremely important because if you do not use return, all the logic after the setAction() will be ran and processed, causing unneeded overhead and unexpected results.

One downside that I have ran into with this approach is that the URL in the address bar does not change. But 99% of the time it won't matter as the page you render in its place will contain links to the correct pages or the user won't be copy and pasting the URL anyways.

Upgrading the forum plugin to 1.8

I have been working on the new 1.8 version of the forum for almost 2 months, simply because version 1.7 was completely broken. It used PHP 5.3 constants which didn't work on 99% of the users machines. This problem has now been corrected in the new 1.8 version, but that's not all that changed. All topics, forums and forum categories now use the sluggable behavior for pretty URLs. Furthermore, the Installer has been rewritten completely for faster and easier usage. The main purpose of this post is to direct you in the 1.8 upgrade process as there are many things you need to do.

Overwriting an older version

You will need to overwrite ALL files within the plugin. Basically every view, every model, every controller, etc has been modified in some way; primarily to support the new slugs. Sorry to all those who have changed the HTML.

Patching your plugin

The new installer system creates a legacy config file to be used for future upgrades and to maintain the current installation. If you are doing a fresh install, you can skip this step. Once you have updated all the files, you will need to patch your installation to build that config file. You can do this from within the install controller.

yourdomain.com/forum/install/patch

The patch process will require 3 things: the database config you want to use, the table prefix value (if you have one) and a checkbox that determines if you used the plugins native user table or a pre-existing one.

Upgrading the database

With the new slug changes, it requires you to upgrade your database tables with the new column and all rows to be populated with their slug equivalent. This process takes 2 steps to achieve, the first having you run an SQL command from within the installer. You can access this process at the URL below.

yourdomain.com/forum/install/upgrade_1_8

The second step requires you to use the Cake console and to run the slugify shell. This shell will populate all the empty slug rows (it can be used any time, not just the upgrade process). Your shell command may look like the following.

cake -app /path/to/app slugify
Fresh install

If you are running a fresh install, the process is extremely simply. Just go to the install controller, hit the "Begin Installation" button and follow the steps. The process will check your database for conflicts, create the tables, patch the install and create the admin user. I tried to make it as simply and straight forward as possible.

See if it's working!

Whether you did a fresh install, or patched an old one, the last thing to do is actually use the forum. You should be checking to make sure all the slugs were generated, the pages work, the links link correctly, so on and so forth. If you find a bug or problem, PLEASE report it to me so I can fix it.

Enjoy! (Sorry for the long wait time)

Setting up cron jobs with Cake shells

This post will primarily be about the process I had to go through to get cron jobs and shells working on Dreamhost. The main problem I kept running into was that the shells would simply not work at all through a cron job; yet they worked when I manually did it through SSH. The weird thing was the source of the console/cake file was being printed in my logs, instead of executed. Below is a quick checklist of things to look out for:

TERM environment variable not set

This isn't really an error, but the TERM variable is used by the server environment and does not affect the shell, even if the error is thrown. Now I may not know much about server maintenance and configuration, but adding the following code to console/cake seemed to do the trick (Hats off to Matt Curry for the tip). And of course, you should change linux to whatever your server is.

TERM=linux
export TERM
Cron environment not PHP 5

This was a weird problem I ran into. It seemed the environment in which my crons ran was PHP 4, so I was receiving unexpected results and would receive this error (it mainly depends where the php folder is located and if its loaded correctly).

console/cake: line 30: exec: php: not found

To fix this, I had to rewrite the cake console file and update it with the path to the PHP folder (path should represent your servers structure). You should only need to do this if you can't upgrade PHP to 5 for some reason.

exec php -q ${LIB}cake.php -working "${APP}" "$@"
# change to
exec /usr/local/php5/bin/php -q ${LIB}cake.php -working "${APP}" "$@"
Making sure your cron has ownership

A simple problem with a simple fix. Since the user that uploaded the console files was different than the user running the cron, I would receive this error:

sh: cake/console/cake: Permission denied

All I had to do was switch the cron user to the user that owns the file, and it fixed the permissions problem.

Making sure the cake file uses unix line endings

This was another pain in the ass that took me forever to get working. My cron environment would throw this error once I fixed the things above (this applies to my Dreamhost server and may not apply to all).

sh: cake/console/cake: /bin/bash^M: bad interpreter: No such file or directory

It required me contacting my host and asking for help on this one. I tried saving all my files as unix and made sure my FTP was not converting them in any way. Still nothing. So my host told me about this SSH command called dos2unix which would convert the file to unix line endings, and wallah, magic! Everything seemed to be working now.

dos2unix cake/console/cake

These are just a few of the minor setbacks I had to deal with when setting up my cron jobs. Most of this really applies to my Dreamhost server, as my Media Temple server worked right away with no configuration. Furthermore, here are some more Dreamhost articles that I used for this problem, that might be of aid:

Editing Your Environment Profile
Make PHP5 the Default in the Shell

Converting a SimpleXML object to an array

This functionality can now be found within the Titon Utility library.

If you have been following my Twitter, you would of heard me complaining about converting a SimpleXML object into an array. I am still having that problem, so if you can get it working correctly (my test so far below), I would be greatly appreciative. If you have never used the SimpleXML object, it can be quite awesome when actually reading an XML document - but once it comes to converting it to something else, it comes straight from the darkest depths of hell. Every property of the object, is also a SimpleXML object, so on and so forth. Each property/object has a method children(), which returns more properties, or attributes() which returns attributes; weirdly enough, children() also return attributes. Furthermore, you can't just echo the object out to get a value, you have to turn it into a string. You can see where this can get quite difficult and confusing, as it always spits out data your not expecting.

After countless hours, I was able to get it to properly convert to an array... about 95% of the time... while keeping attributes and parent/children hierarchy. The only scenario where it doesn't convert properly, is when you have nodes within a node that has attributes (which is kind of rare in my opinion). Here's a small little example:

// Works just fine
<root>
	<node foo="bar">I'm a node!</node>
</root>
// Does not work
<root>
	<node foo="bar">
		<childNode>I'm here to make your life miserable!</childNode>
		<childNode>Me too!</childNode>
	</node>
</root>

Besides that little instance, I am able to properly turn an XML document with attributes, and multiple nodes with the same name, all into a perfectly replicated array. Here is the code I wrote to achieve such an amazing task (sarcasm).

/**
 * Convert a SimpleXML object into an array (last resort).
 *
 * @access public
 * @param object $xml
 * @param boolean $root - Should we append the root node into the array
 * @return array
 */
public function xmlToArray($xml, $root = true) {
	if (!$xml->children()) {
		return (string)$xml;
	}
	$array = array();
	foreach ($xml->children() as $element => $node) {
		$totalElement = count($xml->{$element});
		if (!isset($array[$element])) {
			$array[$element] = "";
		}
		// Has attributes
		if ($attributes = $node->attributes()) {
			$data = array(
				'attributes' => array(),
				'value' => (count($node) > 0) ? xmlToArray($node, false) : (string)$node
				// 'value' => (string)$node (old code)
			);
			foreach ($attributes as $attr => $value) {
				$data['attributes'][$attr] = (string)$value;
			}
			if ($totalElement > 1) {
				$array[$element][] = $data;
			} else {
				$array[$element] = $data;
			}
		// Just a value
		} else {
			if ($totalElement > 1) {
				$array[$element][] = xmlToArray($node, false);
			} else {
				$array[$element] = xmlToArray($node, false);
			}
		}
	}
	if ($root) {
		return array($xml->getName() => $array);
	} else {
		return $array;
	}
}

I know exactly where the problem resides also. Its the value index of the $data array. (The little bastard below).

$data = array('attributes' => array(), 'value' => (string)$node);
// Should be
$data = array('attributes' => array(), 'value' => xmlToArray($node, false));

A simple fix right? Nope! When you do that, it totally breaks... for some reason. The first line of the function (!$xml->children()) gets passed since the element passed does have children, since it has attributes; now I can never understand why attributes count as children when you have attributes(). I tried many different conditionals to get it working, I tried unsetting the attributes (but can't determine the property), and all these other routes... but to no avail. But I digress, seeing as how it works 95% of the time, and the case it doesn't work isn't used that much. However, if you can figure it out, I will be in your debt forever.

The fate of class suffixes in 5.3

Before PHP 5.3 was released, many programmers would suffix their classnames with the package and or module that it pertains to. For example, SessionHelper, UsersController, CommentModel, so on and so forth. But with the recent upgrade to namespaces, are these types of suffixes still required?

// Current Practice
SessionHelper();
// Namespace Practice
\app\libs\helpers\Session();
// Redundant Practice
\app\libs\helpers\SessionHelper();

That's one of many dilemma's I am running into right now when dealing with past practices and theories, and applying them to newer code. Namespaces make the suffix redundant as the package is now part of the class name (unless you are aliasing the namespace). The only deal breaking reason I can think of right now as to keep suffixes; is when you are editing multiple files (or browsing a folder), as it helps identify files from each other. But on the other hand, how many times are you going to have multiple files opened name User? Maybe twice for the model and controller.

I'm mainly posting this entry to get input from everyone else and what you think the approach should be. Right now I'm going down the path of not suffixing my classes, as the similar file name is not that much of a problem or hindrance to me. If you have any convincing arguments, let me hear them (especially since this will apply to my newer scripts)!