Uploader

Plugin: File Uploader and Image Transformer

Warning! This codebase is deprecated and will no longer receive support; excluding critical issues.

A CakePHP plugin that will validate and upload files through the model layer. Provides support for image transformation and remote storage transportation.

  • Upload files automatically through Model::save() by using the AttachmentBehavior
  • Validate files automatically through Model::save() by using the FileValidationBehavior
  • Extensive list of validation rules: size, ext, type, height, width, etc
  • Transform the uploaded image or create new images: crop, scale, rotate, flip, etc
  • Automatic file deletion when a database record is deleted or updated
  • Supports transporting files to remote storage systems like AWS
  • Exif reading and processing support

1. Installation

The current documentation only refers to v4.x of the plugin. If you are installing v3.x, most of the installation and configuration processes are similar, the major differences are the setting names. I suggest combing over the source code to determine what the differences are.

Installing v4.x

The plugin must use Composer for installation so that all dependencies are also installed, there is no alternative (use v3.x if you cannot use Composer). Learn more about using Composer in CakePHP. The plugin uses Transit and AWS SDK internally for all file uploading functionality.

{
	"config": {
		"vendor-dir": "Vendor"
	},
	"require": {
		"mjohnson/uploader": "4.*"
	}
}

Be sure to enable Composer at the top of Config/core.php.

require_once dirname(__DIR__) . '/Vendor/autoload.php';
Installing v3.x

Since this version has no dependencies, it can be installed without Composer (but still supports it). Being an older version, many of its features are not up to date and most support is deprecated.

To install the plugin, simply download/clone the project and place the contents into app/Plugin/Uploader.

2. Configuration

All file uploading is done through the model layer and is configured with the AttachmentBehavior. For every database column that contains a file path (either relative or absolute), an attachment configuration should be defined. Once a configuration exists, calling a simple Model::save() will upload the file, trigger any image transformations and finally save the file path in the database. Below are the default and all available configuration settings. For a more detailed explanation on what each setting does, jump to the next chapters.

The key for each configuration is the database column. In the following example, the file path will be uploaded to the "image" column.
public $actsAs = array(
	'Uploader.Attachment' => array(
		// Do not copy all these settings, it's merely an example
		'image' => array(
			'nameCallback' => '',
			'append' => '',
			'prepend' => '',
			'tempDir' => TMP,
			'uploadDir' => '',
			'transportDir' => '',
			'finalPath' => '',
			'dbColumn' => '',
			'metaColumns' => array(),
			'defaultPath' => '',
			'overwrite' => false,
			'stopSave' => true,
			'allowEmpty' => true,
			'transforms' => array(),
			'transformers' => array(),
			'transport' => array(),
			'transporters' => array(),
			'curl' => array()
		)
	)
);

Once you have your attachment defined, you will need to add the input field in the form. Both the form and input will need the file type applied.

echo $this->Form->create('Model', array('type' => 'file'));
// Other inputs
echo $this->Form->input('image', array('type' => 'file'));
echo $this->Form->end('Submit');

And finally, just call Model::save() and your file should upload! It's as easy as that. Be sure to add validation to the file using the FileValidationBehavior.

if ($this->Model->save($this->request->data, true)) {
	// Do something
}

3. Changing Upload Directories

There are 4 settings that deal with determining the destination folder for uploaded files, they are tempDir, uploadDir, transportDir, and finalPath.

The tempDir (string) setting determines where the files should be uploaded temporarily. Files are uploaded to a temporary directory so that any image transformations can be executed and moved in a staging like environment. The default value is set to CakePHP's application tmp directory and usually does not need to change.

The uploadDir (string) setting determines where the files should be moved to permanently after being uploaded to the temporary directory. This value should be an absolute path to another location on the file system. The default value is set to the files/uploads folder within CakePHP's webroot folder.

The finalPath (string) setting is a path that is prepended onto the file name before being saved into the database. This value can be an absolute path (like a domain name), or a relative path (like the files/uploads path). This setting is best used to prepend a relative path that is publicly accessible via an HTTP URL, else only the file name would be saved into the database.

public $actsAs = array(
	'Uploader.Attachment' => array(
		'image' => array(
			'tempDir' => TMP,
			'uploadDir' => '/var/www/app/webroot/img/uploads/',
			'finalPath' => '/img/uploads/'
		)
	)
);

The transportDir (string) setting allows uploads to be placed in a custom folder when being transported. It works exactly like finalPath, but only applies to transports.

Based on the configuration above, if a file with the name test.jpg was uploaded, the value in the database would be saved as img/uploads/test.jpg and the file would reside at /var/www/app/webroot/img/uploads/test.jpg. Furthermore, if you only want to save the file name in the database, you could set finalPath to an empty string. There are multiple ways to configure both these settings but they are always used in conjunction.

Only uploadDir, transportDir and finalPath can be used in transformation configurations.

To make configuration easier, the finalPath will automatically be appended to the default uploadDir. This permits the uploadDir setting to be omitted.

4. Modifying File Names

There are 3 settings that deal with modifying the file name, they are nameCallback, append and prepend.

The nameCallback (string) setting will accept the name of a method found with the current model. This method will be triggered and the return value will be used as the new file name. The first argument of the method will be the current file name, and the second argument will be a Transit\File object. This object can be used to grab information like the extension, mime type, file size, etc.

public $actsAs = array(
	'Uploader.Attachment' => array(
		'image' => array(
			'nameCallback' => 'formatName'
		)
	)
);

public function formatName($name, $file) {
	return sprintf('%s-%s', $name, $file->size());
}

The prepend and append (string) settings can be used to append and prepend static text to the file name.

public $actsAs = array(
	'Uploader.Attachment' => array(
		'image' => array(
			'append' => '-original',
			'prepend' => 'hd-'
		)
	)
);

All 3 of these settings can be used within each transformation configuration.

5. Process Flow Handling

There are 3 settings that deal with process flow handling, they are overwrite, stopSave and allowEmpty. Process flow refers to the flow of processing a file, executing all its configuration, and returning a response. The flow can be interrupted at any time, especially when an upload or transform error occurs.

The overwrite (boolean:false) setting determines whether existent files should be overwritten by the new file. If this setting is turned off, the new file will have an incremental number appended to its name.

The stopSave (boolean:true) setting will completely stop the Model::save() query from executing if some sort of error occurs. If this setting is turned off, the query will execute without an image being uploaded. It's usually best to leave this value at true.

The allowEmpty (boolean:true) setting will allow the Model::save() query to continue if the input file field is empty. This setting is useful on edit pages where uploading a new image is not required.

The defaultPath (string) setting allows for a path to be used when an empty file upload happens. This allows for default or fallback images to be used more easily. This setting will only trigger if allowEmpty is true. This also applies to transforms.

6. Transforming Images (Resize, Crop, etc)

The greatest aspect of the plugin is the image transformation system. Transformations can be applied to the uploaded image, or can be used to generate additional images (like thumbnails). There are no limits to how many transforms can be defined, but the more you add, the longer the processing time. Like other settings, transforms can be defined within the transforms setting.

Like before, the file path will be set to the "image" column and the transformed paths will be set to "imageSmall" and "imageMedium" respectively.
App::uses('AttachmentBehavior', 'Uploader.Model/Behavior');

// In the model
public $actsAs = array(
	'Uploader.Attachment' => array(
		'image' => array(
			'overwrite' => true,
			'transforms' => array(
				'imageSmall' => array(
					'class' => 'crop',
					'append' => '-small',
					'overwrite' => true
					'self' => false,
					'width' => 100,
					'height' => 100
				),
				'imageMedium' => array(
					'class' => 'resize',
					'append' => '-medium',
					'width' => 800,
					'height' => 600,
					'aspect' => false
				)
			)
		)
	)
);

Like the parent setting, each transform accepts the following settings: nameCallback, append, prepend, uploadDir, transportDir, finalPath, defaultPath, and overwrite. There are 2 settings which are specific to transforms, they are class and self. The class setting defines which type of transformation to use. The self setting determines whether or not the image transformations should be applied to the uploaded file, or whether to create additional files (defaults to false).

There are 7 types of image transformations that are currently available, they are resize, crop, flip, scale, rotate, fit, and exif. Each method has additional settings that can also be applied.

Resize
Allows one to change the width or height of an image programmatically.

  • width (int) - The width to resize to
  • height (int) - The height to resize to
  • quality (int:100) - The quality of the image (jpg only)
  • expand (boolean:false) - If false, will not allow the image to be resized larger than its original dimensions
  • aspect (boolean:true) - If true, will maintain aspect ratio when scaling up or down
  • mode (string:width) - Use the width or height for aspect ratio calculations (accepts width or height)

Crop
Allows one to crop out an area of the image.

  • width (int) - The width to crop out
  • height (int) - The height to crop out
  • quality (int:100) - The quality of the image (jpg only)
  • location (string:center) - Which area to crop (accepts center, top, right, bottom, left)

Flip
Allows one to flip the image horizontally or vertically.

  • quality (int:100) - The quality of the image (jpg only)
  • direction (string:vertical) - The direction to flip (accepts vertical, horizontal, both)

Scale
Allows one to scale an image up or down programmatically.

  • quality (int:100) - The quality of the image (jpg only)
  • percent (float:0.5) - The percentage to scale with

Rotate
Allows one to rotate an image to a certain degree.

  • quality (int:100) - The quality of the image (jpg only)
  • degrees (int:180) - The degrees to rotate with

Fit
Allows one to fit an image within a certain dimension. Any gap in the background will be filled with a color. If no background fill is provided, the image will simply be resized.

  • width (int) - The width to fit to
  • height (int) - The height to fit to
  • quality (int:100) - The quality of the image (jpg only)
  • fill (array) - An array of RGB values
  • vertical (string:center) - Direction to align image vertically
  • horizontal (string:center) - Direction to align image horizontally

Exif
The Exif transformation will rotate, flip, and fix orientation issues for Exif images. It will also strip Exif data so that it isn't publicly available. Exif transformation should be applied as the first self-transformation.

public $actsAs = array(
	'Uploader.Attachment' => array(
		'image' => array(
			'transforms' => array(
				array(
					'class' => 'exif',
					'self' => true
				),
				// Other transformations
			)
		)
	)
);

7. Transporting To The Cloud

Now a days it's best to store your images in the cloud instead of your local server. The Uploader supports the transportation of files to remote storage systems like AWS. A transport can be defined using the transport setting.

To use AWS functionality, add the SDK as a composer dependency: aws/aws-sdk-php
App::uses('AttachmentBehavior', 'Uploader.Model/Behavior');

// In the model
public $actsAs = array(
	'Uploader.Attachment' => array(
		'image' => array(
			'overwrite' => true,
			'transport' => array(
				'class' => AttachmentBehavior::S3,
				'accessKey' => 'ACCESS',
				'secretKey' => 'SECRET',
				'bucket' => 'bucket',
				'region' => Aws\Common\Enum\Region::US_EAST_1,
				'folder' => 'sub/folder/'
			)
		)
	)
);

Only one transport can be defined per configuration; it also applies to any child transformations. Currently only Amazon S3 and Glacier are supported. The Uploader uses the AWS PHP SDK internally to transport files — it's best to read the documentation and source code for the SDK to determine which constants and settings to use.

Amazon Simple Storage Service
The full AWS S3 URL will be saved as the path in the database.

  • accessKey (string) - The access key given to you by AWS
  • secretKey (string) - The secret key given to you by AWS
  • bucket (string) - The bucket to move files to
  • folder (string) - The folder to place the file in within the bucket
  • scheme (string:https) - The HTTP protocol scheme to use (accepts http or https)
  • region (string) - The region the bucket is located in (should use Aws\Common\Enum\Region constants)
  • storage (string:standard) - The storage setting for each file (should use Aws\S3\Enum\Storage constants, defaults to Storage::STANDARD)
  • acl (string:public-read) - The access permissions for each file (should use Aws\S3\Enum\CannedAcl constants, defaults to CannedAcl::PUBLIC_READ)
  • encryption (string) - Server side encryption algorithm to use (accepts AES256 or an empty string)
  • meta (array) - A mapping of meta data for each file
  • returnUrl (bool:true) - Return the full S3 URL or the S3 key

Amazon Glacier
The archive ID will be saved in place of the path in the database.

  • accessKey (string) - The access key given to you by AWS
  • secretKey (string) - The secret key given to you by AWS
  • vault (string) - The vault to move files to
  • accountId (string) - An AWS IAM account ID for authorization (can usually be blank)
  • region (string) - The region the vault is located in (should use Aws\Common\Enum\Region constants)

8. Custom Transformers and Transporters

If there's ever a situation where the current list of transformers or transporters do not suffice, a custom class can be written. The custom transformer must extend the Transit\Transformer interface, while the custom transporter must extend the Transit\Transporter interface. Take a look at the Transit repository for examples.

Once the custom class is created, the class name will need to be defined within the attachment settings. This can be done within the transformers and transporters arrays.

public $actsAs = array(
	'Uploader.Attachment' => array(
		'image' => array(
			'transformers' => array(
				'grayscale' => 'Namespace\Class\GrayscaleTransformer'
			),
			'transporters' => array(
				'dropbox' => 'Namespace\Class\DropboxTransporter'
			)
		)
	)
);

Now set the class setting within each transform or transport setting to the key for the custom class. This will now permit custom classes to be used when saving records by autoloading and instantiating the defined classes.

public $actsAs = array(
	'Uploader.Attachment' => array(
		'image' => array(
			'transforms' => array(
				'image_gray' => array('class' => 'grayscale')
			)
		)
	)
);

9. Importing Remote Files

By default, the AttachmentBehavior uploads files through HTTP post and grabs its data from $_FILES (while using a file input field). There are times where you want to import a file from another location, instead of uploading a file from the client. Currently there are 3 methods of importing.

In the examples below, the "image" attachment will be used.
Copying remote files

This allows the user to paste an HTTP URL to a file (most likely an image) and have the Uploader copy it. To set this up, all you need to do is create an input field that is not a file. Remote file importing will only work if the URL begins with http (so add a validation rule!).

// Change this
echo $this->Form->input('image', array('label' => 'Upload', 'type' => 'file'));
// To this
echo $this->Form->input('image', array('label' => 'Remote URL'));

Remote imports support customizeable cURL options through the curl setting. The array should map cURL constants to option values.

public $actsAs = array(
	'Uploader.Attachment' => array(
		'image' => array(
			'curl' => array(CURLOPT_SSL_VERIFYPEER => true)
		)
	)
);
Copying local files

This works in a similar fashion to remote files, but the primary difference is that it copies a file from the local file system. This should really only be used by administrators and developers, or some sort of system that has a pre-defined set of file system paths.

echo $this->Form->input('image', array('label' => 'Local Path'));
Copying from PHPs input stream

The last line of defense for importing is copying a file from PHPs input stream. This functionality is primarily used for AJAX file uploading purposes and will rarely be used outside of that. Since this method is rather complex and differs per implementation, I will try and create a broad example.

// 1) AJAX pushes a file upload to /files/upload?name=file.jpg

// 2) FilesController::upload() handles the import by setting the file name for the image
public function upload() {
	$this->request->data['Upload']['image'] = $this->request->query['name'];
	
	if ($this->Upload->save($this->request->data)) {
		// File uploaded and record saved
	}
}

The primary difference between regular uploading and importing is that stream importing requires the destination file name to be passed as the value. Once the value is set, it will attempt to copy the file from the stream and process the file with the attachment settings as if it was a regular upload.

10. Modifying Settings With Callbacks

There are times when you need to modify the attachment settings dynamically before the upload occurs, for example changing the destination folder. You can achieve this by defining callbacks within the model. These callbacks are beforeUpload(), beforeTransform() and beforeTransport() — they are pretty self explanatory. You only need to define a callback when you want to modify something.

 // Lets change some settings
public function beforeUpload($options) {
	$options['append'] = '-original';
	
	return $options;
}

// Or maybe place the files in files/uploads/resize/
public function beforeTransform($options) {
	$options['finalPath'] = '/files/uploads/' . $options['method'] . '/' 
	$options['uploadDir'] = WWW_ROOT . $options['finalPath'];
	
	return $options;
}

// And even change the S3 folder
public function beforeTransport($options) {
	$options['folder'] = 'img/' . $this->data[$this->alias]['slug'] . '/';
	
	return $options;
}

There are no restrictions on what you can modify, so have fun with it.

11. Deleting Files

By default, files are automatically deleted any time a database record is deleted, or anytime a record update occurs and the previous file will be overwritten. However, there are times when you want to delete the files manually but not delete the associated record. You can achieve this by calling deleteFiles() through the respective model.

For example, say the Image model has 3 file fields: small, medium and large.

// Delete all
$this->Image->deleteFiles($id);

// Delete only medium
$this->Image->deleteFiles($id, array('medium'));

12. Saving Meta Data

Attachments also support the saving of meta data in the database. If you want to save the extension, or mime type or file size alongside the path, you can do so. The following meta fields (including Exif data) are available: basename (name with ext), ext, name (without ext), size, type (mime type), width, height, exif.make, exif.model, exif.exposure, exif.orientation, exif.fnumber, exif.date, exif.iso, and exif.focal. You can save these fields by defining the metaColumns settings.

public $actsAs = array(
	'Uploader.Attachment' => array(
		'image' => array(
			'metaColumns' => array(
				'ext' => 'extension',
				'type' => 'mimeType',
				'size' => 'fileSize',
				'exif.model' => 'camera'
			)
		)
	)
);

Every key in the metaColumns setting should be one of the available meta fields, and the value should be the database column to save it to.

Meta data is only derived from the original file, not the transformations. Transformed files do not have metaColumns support.

13. Validating An Upload

Like any upload form, you want to validate the file before it is actually uploaded. We can do this by using the FileValidationBehavior and our models built in validation system. The Uploader validation can the following: width, height, minWidth, minHeight, maxWidth, maxHeight, filesize, extension, type, mimeType and required. All we need to do is set the validation rules by applying an $actsAs behavior, like so.

Here are a few examples of how to use the behavior to add form validation.

public $actsAs = array(
	'Uploader.FileValidation' => array(
		'image' => array(
			'maxWidth' => 100,
			'minHeight' => 100,
			'extension' => array('gif', 'jpg', 'png', 'jpeg'),
			'type' => 'image',
			'mimeType' => array('image/gif'),
			'filesize' => 5242880,
			'required' => true
		)
	)
);

The code above would allow basic customization for validation. If you want to extend this validation even further and have custom error messages, you can do it like so. Additionally, you can validate more than one field.

public $actsAs = array(
	'Uploader.FileValidation' => array(
		'image' => array(
			'extension' => array('gif', 'jpg', 'png', 'jpeg'),
			'required' => array(
				'value' => true,
				'error' => 'File required'
			)
		),
		'thumbnail' => array(
			'required' => false
		)
	)
);

The validation rules also accept the same options as CakePHP's validation system. This allows us to use options like on and allowEmpty.

public $actsAs = array(
	'Uploader.FileValidation' => array(
		'image' => array(
			'required' => array(
				'value' => false,
				'on' => 'update',
				'allowEmpty' => true
			)
		)
	)
);

Frequent Questions

  • 1. How do I use the database ID as the file name?

    Since the record hasn't been saved yet, you will need to query the database for the last ID and increment it. You can accomplish this using the nameCallback setting.

    public function nameCallback($name, $file) {
    	$data = $this->find('first', array(
    		'order' => array($this->primaryKey => 'DESC'),
    		'limit' => 1
    	));
    
    	if ($data) {
    		return $data[$this->alias][$this->primaryKey]++;
    	}
    
    	return $name;
    }
    If any records get deleted, this example will fail as the next ID will not always be the next incremented ID. To get the actual increment ID, you would need to query the table status for the value.
  • 2. How do I remove the "resize-100x100" from transformations file names?

    As a side-effect of the image transformation, all transformed images will have their file names appended with strings like "-resize-100x100" and "-crop-250x100". This happens so that files are not overwritten on accident while the image is being created. This string can not be removed with an append or prepend setting, you will simply end up with a file name like "prepend_name-resize-100x100_append". To completely remove it, you will need to apply a nameCallback to each transform. This callback will return the original files name — but be sure that it will not overwrite the original!

    public function transformNameCallback($name, $file) {
    	return $this->getUploadedFile()->name();
    }
  • 3. How do I define an absolute path for tempDir and uploadDir?

    These two settings require an absolute path or else you will end up some weirdness when the relative path resolves to the wrong folder. Since you can't append strings and constants within class property definitions, you can define a constant outside of the model.

    define('UPLOAD_DIR', WWW_ROOT . '/img/uploads/');
    
    class Upload {
    	public $actsAs = array(
    		'Uploader.Attachment' => array(
    			'image' => array(
    				'uploadDir' => UPLOAD_DIR,
    				'finalPath' => '/img/uploads/'
    			)
    		)
    	);
    }

    Or you can modify the settings in a callback.

    public function beforeUpload($options) {
    	$options['finalPath'] = '/img/uploads/' 
    	$options['uploadDir'] = WWW_ROOT . $options['finalPath'];
    	 
    	return $options;
    }
  • 4. How to upload through a has many?

    The easiest way to upload multiple files is to upload through a has many association. As an example, I'll use an Image model that belongs to Product and has the AttachmentBehavior defined for the path field.

    • Product has many Image
    • Image belongs to Product

    The first step is to create the view (there's no limit to the amount of images that can be listed).

    echo $this->Form->create('Product', array('type' => 'file'));
    echo $this->Form->input('title');
    echo $this->Form->input('Image.0.path', array('type' => 'file'));
    echo $this->Form->input('Image.1.path', array('type' => 'file'));
    echo $this->Form->end('Create');

    The only other step is to change save() to saveAssociated() in the controller when saving the Product record.

    if ($this->Product->saveAssociated($this->request->data)) {
    	// Do something
    }

    Be sure to set dependent to true in the settings or the associated records (and files) will not be deleted when the primary record is deleted.

  • 5. How to upload multiple files?

    Uploading multiple files not through an association can be quite tricky, but is possible. As an example, I'll use an Image model that has the AttachmentBehavior defined for the path field. To begin, we must convert our single file view to support multiple files. We can achieve this by numerically indexing the path fields.

    echo $this->Form->create('Image', array('type' => 'file'));
    echo $this->Form->input('Image.0.path', array('type' => 'file'));
    echo $this->Form->input('Image.0.caption');
    echo $this->Form->input('Image.1.path', array('type' => 'file'));
    echo $this->Form->input('Image.1.caption');
    echo $this->Form->end('Upload');

    Then to upload the files, call saveMany() in the controller. The only gotcha is that the Image array index needs to be passed; this is simply a problem with CakePHP's data structure expectancy.

    if ($this->Image->saveMany($this->request->data['Image'])) {
    	// Do something
    }
  • 6. How do I upload files larger than 2MB?

    When uploading really large files, the dreaded white screen can appear, or the memory exhausted error, both of which cause the request to break with no errors in the logs. This white screen is caused by PHP's inability to handle large file uploads with basic settings.

    The first change that needs to be made is upping the max upload size in php.ini. Change these values to whatever you please. Be sure to read the notes on each setting also.

    ; Maximum allowed size for uploaded files.
    upload_max_filesize = 40M
    ; Must be greater than or equal to upload_max_filesize
    post_max_size = 40M

    The second change is to increase the memory limit from the default 16MB in php.ini.

    memory_limit = 100M

    If neither of these changes solve the white screen, you can disable the memory and time limit so the script can continue without exiting early. Place the following code at the top of the controller (or in bootstrap) that handles uploading.

    ini_set('memory_limit', '-1');
    set_time_limit(0);
  • 7. Will the uploader work on Heroku?

    Yes it will... with a few changes. Since Heroku restricts file system writing outside of the /tmp folder, files must be transported to S3 (or another storage system), as well as having all file uploads point to the tmp folder. The Heroku build pack must also have the required PHP extensions enabled: gd, fileinfo, curl and exif (if you need it).

    The following options need to be applied for all uploads and transforms.

    'tempDir' => '/tmp',
    'uploadDir' => '/tmp',
    'finalPath' => ''