ZF2 subforms usage and validation

zf2Differently from Zend Framework 1, I couldn’t find any good documentation about the usage of subforms in Zend Framework 2.
After a bit of investigation, I understood how to use them.
When the subforms extend the `Fieldset` ZF2 class, they can be added to the form with a name, and that name (e.g. subform1 and subform2) will be used with getData() and setData() as a key of the subform data array.
The validation can stay in each related subform, the isValid() ZF2 Form will recursively validate all the subforms.
Example:
A `MainForm` contains two subforms (named `subform1` and `subform2`) of the same class `SubForm`.
Once filled in, getData() will return the following. Equally, the same array structure will fill in the related fields.
    $data = array(
      ‘formElement’ => ‘…’,
      ‘subform1’ => array(
         ‘first’ => ‘…’,
         ‘second’ => ‘…’,
      ),
      ‘subform2’ => array(
         ‘first’ => ‘…’,
         ‘second’ => ‘…’,
      )
    );
Here the code:

Doctrine 2 + zf2 working with both SQL server and MySQL

The web application I’m working on (PHP 5.3, zf2, Doctrine2, on MySQL/Apache) has recently raised interest from the clients and it now needs to be installed on their premises, often a Windows server( arghhh!). That meant we had to make the application compatible with both MySQL and SQL-Server.
Here I summarize the changes and the solutions we adopted.
Continue reading

Raw query on Zend framework

Zend_Db_Table_Abstract::getDefaultAdapter()->query($sql);
Zend_Db_Table_Abstract::getDefaultAdapter()->query($sql)->fetch(); //if you need to fetch results

if it does not work, use the db handler Pdo), and call – for example – exec()

Zend_Db_Table_Abstract::getDefaultAdapter()->getConnection()->exec($sql);

How to use Zend Framework hostname and chain routes to map a module (e.g. admin area) to a different subdomain.

In addition to the standard Static, Route and Regex routes, Zend Framework implements two more less famous – but useful – ones: hostname and chain routes. The first one works with the subdomain (instead of the path) and it can be instantiated with some default parameters (e.g.  “module”=>”admin”). It’s possible to specify a variable in the route, but in my project I’ve specified the few subdomains manually as there were some exceptions to consider.

Each route that needs to be called on the subdomain must be chained separately to its hostname route using the chain route.  The name of the route remains unique in the global space, not in the hostname. That allows to refer a route by name from any part of the application.

Example with module “default” (frontend) and module “admin” (admin area)

How the dispatcher works

  • www.site.com/, the dispatcher will call indexAction of the IndexController of the default module.
  • www.site.com/login, the dispatcher will find the route /login and will load AuthController::logicAction
  • admin.site.com/ dispatches indexAction of IndexController of admin module
  • admin.site.com/login does NOT consider the route “login” as it’s chained to the other hostname, it will dispatch indexAction of LoginController (if exists) of admin module
  • admin.site.com/settings will dispatch indexAction of SettingsController of admin module

How to generate links to routes chained to hostnames (from view scripts, any module):

  • $this->view->url(array(‘controller’=>’aaa’, ‘action’ => ‘bbb’), ‘default’); // will print http://www.site.com/aaa/bbb, even if called from admin area view script
  • $this->view->url(array(), ‘login’); // will print http://www.site.com/login
  • $this->view->url(array(‘controller’=>’aaa’, ‘action’ => ‘bbb’), ‘admin‘); // will print http://admin.site.com/aaa/bbb
  • $this->view->url(array(, ‘adminSettings‘); // will print http://admin.site.com/settings, even if called from a frontend view script

Note: if the name of the route (default, admin, adminSettings, login…) is not specified, the current one will be used. To make a link the the other action of the same controller, you can just use $this->view->url(array(‘ ‘action’ => ‘ccc’));

 

My considerations about Db models and RESTful application

I’ve worked on more than one project using complicate DB layers and HTTP client/server systems (e.g. RESTful apps) to interface with the DB server.

My opinion: when working on a project (especially a team project), easy solutions are better then clever ones. OK, OK, a design pattern is an elegant solution to a common problem, but before doing that, keep in mind two basic design principles: KISS (Keep It Simple, Stupid) and YAGNI (You Ain’t Gonna Need It).

A DB layer is the ground of an application, and it must be simple, solid and extensible.

My idea: Simple, maintainable and extensible Model classes for DB access

Make a class for each DB table, all of them extending a common abstract one (for future extensions). Each class has its own methods to insert/retrieve/delete/update the data. Of course do not make a method for each operations, group them using multi-purpose methods and class constants in arrays. Methods accepts params as array and return data using arrays (so easy to deal with). Common insert/delete/update return data in the same format (e.g. numbers of records updated, or exception, no different values for each method). All the input and return params must be documented with PHPDoc comments including all the possible values of an input array key . You can write a constant for each column in the db, so you do not have to look at the DB to know what’s the column name, just use the IDE autocompletition for everything.
e.g.

$newsModel = News::getInstance();
/* get news of the category id=2, limited by 5 and ordered by date.
Join with parent table and select with columns are needed. */
$rows = $newModel->getRecords(array(
  News::FILTER_CATEGORY = 2,
  News::ORDER_CREATED_DESC,
  News::LIMIT => 5,
  News::JOIN => News::JOIN_AUTHORS,
  News::COLUMNS => array(News::FIELD_TITLE, Authors::FIELD_NAME)
));
foreach ($rows as $row) {
   //echo $row[News::FIELD_TITLE]; //piece of news title
   //echo $row[Authors::FIELD_NAME]; //author name (joined)
];

}

Semplicity and Maintainability

One db table => only one class. All the logic inside the models. If I alter the db table, easy to change the centralized logic and costants.
From the controller, I write the code to access the database using IDE autocompletition in a few seconds.
If I want to add another condition to the query, just add to the array. To add a support to something new to the class (new filter, new join, new column), just add the constants, the logic and update the PHPDoc comments.

Caching advantage

This approach make very easy to implement a cache system with automatic tagging and cache invalidation by using wrappers and magic methods [see post about it ].

Extensibility, e.g. move to a client/server via HTTP (web service)

– You want to switch from Zend_Db to Doctrine / NoSQL or optimize the queries ? no problems, just replace the inside logic. the methods still have same input/output. Much easier if you have written unit testing. You sure don’t have to write another layer to dynamically switch between database (that requires a huge effort, in contrast with YAGNI ).

– You want to move the database to a different server and use HTTP calls to get the data (like RESTful apps) ? I have a elegant, don’t-repeat-yourself and easy solution with PHP 5.3. Make the getInstance() of the super class returning a wrapper (you can do that, or alternatively, use a external factory class) that through magic methods, makes the HTTP calls to the server passing [class name, method name, array of params] and decode the response into an array (just one class for all models !). The server will use the same db models classes (shared libs) and implement the other side of the logic with a correspondent one class ($model = new $modelName(); return $model->$methodName($params);). So, basically you code as you are using a local db, but the request is actually going through HTTP to another server. You can switch back to the local db approeach by just changing the abstract DB model class to return the db instance instead of the wrapper.

Drawbacks ?

Most of the application can use this approach and do not need over-engineering of the DB layer. Please follow with your comments with any problems you see in this approach.

ZFDebug panel quick tutorial

In this post I’ll just write some notes to install the ZFDebug panel for Zend Framework. That’s basically a front controller plugin that displays at the bottom of the layout (slides from the bottom), contains “debug panels” (cache, variables, zend db, time info etc…) and it’s easy to extend.

Copy libraries and images

Grab the last stable zip from the Downloads page, or just export the SVN (just my preference as it’s a debug panel, unstable features are OK).

#install library
cd <zendProjectRoot>/library
svn export http://zfdebug.googlecode.com/svn/trunk/library/ZFDebug/
# [not necessary] copy images (if images dir already exists, go inside and export a level less)
cd <zendProjectRoot>/public
svn export http://zfdebug.googlecode.com/svn/trunk/web/images/

Activate using a boostrap _init() method
Now, the plugin can be activated using a Bootstrap _init method (some options cna be moved to application.ini, but some others don’t, so better to keep everything together in a _init method IMO).

copy the whole method from here
http://zfdebug.googlecode.com/svn/trunk/demos/Zend_Application_Bootstrap.php
to your Boostrap. To add plugins, read the comments inside that file. Better to create plugin classes to make easy updating ZFDebug from SVN.

The minimum code to load should be the following (not tested). Of course add the logic to show only on local environment or under some conditions (auth / cookie / env)

protected function _initZFDebug()
{
    // register namespace
    $autoloader = Zend_Loader_Autoloader::getInstance()
                  ->registerNamespace('ZFDebug');

    // Create ZFDebug instance
    $zfdebug = new ZFDebug_Controller_Plugin_Debug(array(
        'plugins' => array(
            'Variables',
            'Html',
            #'Database' => array('adapter' => array('standard' => Zend_Db_Table_Abstract::getDefaultAdapter())),
            #'File' => array('basePath' => 'path/to/application/root'),
            #'Memory',
            #'Time',
            #'Registry',
            #'Cache' => array('backend' => Zend_Registry::get('cache')->getBackend()),
            'Exception'
        )
    ));

    // Register ZFDebug with the front controller
    $front = $this->getResource('FrontController');
    $front->registerPlugin($zfdebug);
}

BSD licence

Display full query in the Exception of Zend_Db_Statement_Pdo

When working with Zend Framework 1.x and Zend_Db in particular, you might bump into an SQL problem like this

Syntax error or access violation: 
1064 You have an error in your SQL
syntax; check the manual that corresponds 
to your MySQL server version for 
the right syntax to use near ') 
ORDER BY `order` ASC' at line 1

OK, thanks a lot Zend, but what’s the query ?

the statement is kept inside $_stmt property of Zend_Db_Statement, and to display it, you need to acess the queryString property

$_stmt->queryString

Depending on the Zend version, the Zend_Db_Statement class might have a getter for it, so you can debug with

var_dump($zfDbStmt->getDriverStatement()->queryString);

You can also – of course – enable the MySQL general log and monitor the log file

[mysqld]
general_log_file = /path/to/logfile
general_log = 1

Optimising Zend Framework applications (2) – cache pages and PHP accelerator [updated]

[continue from previous post]

4. Use an op-code cache/accelerator (Apc, XCache)

PHP is very fast but it’s not compiled. A op-code cache helps. See this comparison as an example of what could be the performance increase. That does not mean we can skip the other optimisation, code bottlenecks should be removed anyway.
Optimising code is also helpful to understand what are the common code “slowness” and avoid writing them again in the future.

5. Cache pages (before zend framework bootstrap)

Even though you have optimised the code, you still have to bootstrap and run the zend application, and the whole process takes time (dispatching, controller logic, scripts  render). An solution I’ve recently used is caching the whole HTML as a result of the processing (see another post about caching HTML pages of a generic website).

There are many solutions ways to cache the output (apache modules, reverse proxies, zend server page cache). The best depends on the needs. Moving the logic to the application level usually allows more customisations.

Page caching can be done by using Zend_Cache_Frontend_Page. It basically uses ob_start() with a callback that saves the result into cache. I haven’t found any interesting article about its best use, except the fact that it should be used in a controller plugin. I’d say it’s better to instantiate a separate cache object and activate it directly into the index.php and caching before the zend application actually start (boostrap). In my local enviroment, when the page cache is valid, the response time is 2 ms, against 800 ms required to bootstrap and load the application.
See the following code to see where to place it. Note: I instantiate Zend_Application with no options just to have the autoloader available to load the needed classes.

# index.php

// create APPLICATION_* vars and set_include_path [...]

require_once 'Zend/Application.php';
$application = new Zend_Application(APPLICATION_SERVERNAME);
// Zend_Cache_Frontend_Page
$pageCache = Zend_Cache::factory(
    'Page', 'File',
    array(
        'caching' => true,
        'lifetime'=>300, //5 minutes
        'ignore_user_abort' => true,
        'ignore_user_abort' => 100,
        'memorize_headers' => true,
        // enable debug only on localhost
        'debug_header' => APPLICATION_ENV == 'localhost',
        'default_options' => array(
            'cache' => true,
            // test the following, depends on how you use sessions and browser plugins
            'cache_with_cookie_variables' => true,
            'cache_with_post_variables' => false
        ),
        // whitelist approach
        'regexps' => array(
           '^.*$' => array('cache'=>false),
            /* homepage */
            '^/$' => array('cache'=>true, 'specific_lifetime'=>650),
            /* example of other pages*/
            '^/(pagex|pagey)/' => array('cache'=>true,'specific_lifetime'=>700),
            // []...
        )
    ),
    array(
        'cache_dir' => APPLICATION_PATH . '/data/pagecache',
        'hashed_directory_level'=>2,
        'file_name_prefix'=>'zendpagecache'
    )
);
// start page cache, except for cron job and when user is logged
// note: I haven't tested yet if using Zend_Auth here is a good solution
if (PHP_SAPI !=='cli' && !Zend_Auth::getInstance()->hasIdentity()) {
  $pageCache->start();
}
// the following code is not executed when page cache is valid
$appSettings = new Zend_Config_Ini(APPLICATION_PATH . '/configs/sites.ini', APPLICATION_SERVERNAME);
$application->setOptions($appSettings->toArray());

$application->bootstrap();

if (PHP_SAPI !=='cli') {
    $application->run();
}

Of course page caching must be set carefully, depending on the traffic of the application. Ideally, a customised cron job should fetch the pages, invalidate the cache for each one and rebuild them before they normally expire, so that the user will always find the fast cached page. On my project a similar system has improved the average loading time by 8 times (so an external spider will consider the site in a different way). Links to keep cached should be at least the ones on the sitemap.

Set carefully all the settings of zend page cache frontend: cookie, get and post data, regexpr of URLs to cache. Note that no logic is executed when the cache is valid, so an eventual visitors counter or any other database writing query will not work.

Optimising Zend Framework applications (1) – cache db objects, PHP code profiling and optimisation

I’m optimizing some zend framework applications these days and I’ve been reading, researching, optimising and measuring results.

Some links to read before:
Optimising a Zend Framework application – by Rob Allen – PHPUK February 2011
Profile your PHP application and make it fly – by Lorenzo Alberton – PHPNW 9 Oct 2010

The optimisation process is iterative: measure the performances, improve the worst problem and starto over. Go on until performances are not satisfactory.
That does not mean that we can start writing our application and optimising everything later. An eye on performances is needed since the beginning, in order to select an architecture that will support the traffic.

In this post I’ll explain some common techniques I’ve used to optimise a high traffic application. Optimisation is done at the end, but we should be aware of what can be done, in order to design the application in a way that is easier to optimise if needed.

1. Cache database calls with Zend_Cache

Database is often a relevant bottleneck, especially when shared with other applications and we cannot expect it responding immediately. A database server often make the script hanging and increase the average page loading time. Note that in some scenarios (e.g: local server, simple queries operating on table with small indexes and db server caching in memory) the db could be faster than the a disk access to get the cached object. Measure if possible.

For a recent application, I’ve written a wrapper/system to automatically cache all the database models methods to get data using a default lifetime. Depending on the architecture of the site, as possible the cache should not expire and be invalidaded when the data is manually flagged as invalid (from admin area). If that is not possible, set a reasoable lifetime.

The normal cache logic is : if valid, load from disk/memory cache, if not, load fresh data (db). In case the loading of fresh data is not possible (e.g.: db not answering), I suggest a change: if the fresh data not available, use old cache instead of throwing exception or interrupt the application (and log the error). If you use Zend cache, look at the second param of  Zend_Cache_Core::load().

2. Profile and optimise queries

Even though the data is retrieved most of the time from the cache, queries might be need to be optimised: explain queries and check if indexes are added, select which index to use, change field type if needed (an interesting book about it). Remember that MySQL (and in general any db server) works very fast with simple queries (no joins) and small indexes. Consider denormalising the db structure and avoid some frequent joins. You can use MySQL triggers to automatically update the columns when changed on the parent tables.

3. Profile and optimise PHP code

To profile the PHP scripts use Xdebug  profiler + firefox extension (to enable when needed via cookie)  + K/WinCacheGrind. Another tool is xhprof (web based) by facebook, that shows the functions most frequently called.

Command line tool for CRUD (Zend Framework scaffolding libraries)

I’ve created a command line tool for my scaffolding zend framework libraries.

Those libraries are basically abstract related classes (Controller, Form, DbTable, Filter and Order forms) that contains all the logic to make CRUD (Create, Read, Update, Delete) forms for admin area. Thanks to those libraries, a complete CRUD logic requires only a few lines of codes. Everyting is completely customisable by extending featuers. See the page for demo, svn, other info

That tool reads automatically table structure and automatically creates controller with CRUD actions, forms, filters forms, order forms, templates.  Of course everyting is completely customizable (in zend framework style by using inheritance).

example

// create CRUD action to manage table categories.
// The classes named using "Categories" as prefix or suffix
php -f zfcrud.php create crud Categories -t categories -d mydb

see source file zfcrud.php