The Zend Server Job Queue component provides offline asynchronous processing of tasks and activities that can be run independently of the end user experience.
The jobs approach is used for streamlining offline processing of PHP scripts. It provides the ability to delay execution of “heavy” parts of web applications that originate from direct user interaction with a Web Server. As result it may significantly improve response time and reduce Web Server load.
Any PHP script can create a job which will be executed later, separately from a current HTTP request. The job itself is just an additional HTTP request to another PHP script that can be done on the same or different Web Server.
As opposed to “fire and forget” systems (like cron), Job Queue is a job management system that provides advanced capabilities such as:
Keeping track of batch jobs, including their status, execution time and output
Different schedule strategies based on time, job priorities and dependencies
Run-time statistics
Web-based management
Looking for other ways to improve performance:
Job Queue for a single server and for a cluster are based on different architectural layouts.
The Zend Server Job Queue is comprised of three main parts, the Job Queue Daemon, the Job Queue Client and one or more servers that execute the jobs.
The heart of Zend Job Queue is the Job Queue Daemon – this external service takes care of managing scheduling, queuing and executing tasks. It also communicates with the Zend Server User Interface to report any errors and failures.
Jobs are successfully queued once they are written to persistent storage. This makes sure that even if the daemon has to be restarted, or even in case of complete system failure, jobs are not lost.
Jobs, be they scheduled, recurring or one-time queued jobs are all passed to the Daemon by the Job Queue Client extension. This simple PHP extension provides an easy to use API for creating and managing jobs from within PHP. It communicates with the Job Queue Daemon through a TCP socket or a UNIX domain socket.
The Job Queue Client is also a part of the Zend Server Administration Interface – which, through the same extension, allows the end user to create and manage jobs from the convenience of a web GUI.
As far as execution is concerned, running a job is no more than sending a simple HTTP request. When creating a new job through API or GUI, the user specifies the URL for this HTTP request, along with some possible optional parameters. When it's time to run the job, the Job Queue Daemon sends an HTTP request to the server specified by the URL along with the provided headers and parameters, and waits for a response.
The executing HTTP server can be any HTTP server – it might even be the same server that queued the job. This design makes it very easy to manage the system. You don't have to (but can, if needed) set up dedicated servers for handling off-line jobs. You don't have to (but can, if needed) create and manage a special PHP configuration for this environment. You can reuse existing code to a level where sometimes the difference between off-line and on-line execution is decided based on a simple request parameter.
The Zend Server Cluster Job Queue is comprised of four main components:
Web Servers- The servers in which the PHP and Job Queue Extensions
reside.
In some cases, these servers execute jobs as well. In other case, jobs
are executed on Backend Web servers.
Job Queue Extensions- A PHP extension that passes the jobs to the Job Queue Daemons.
Job Queue Daemons- The component that is in charge of managing scheduling, queing and executing tasks.
MySQL Data Base- The component that stores the jobs created throughout the cluster.
The Web servers hold the PHP and the Job Queue Extensions, and receive HTTP requests from the Job Queue Daemons.
Jobs, be they scheduled, recurring or one-time queued jobs are all passed to the Daemon by the Job Queue Client extension. This simple PHP extension provides an easy to use API for creating and managing jobs from within PHP. It communicates with the Job Queue Daemon through a TCP socket or a UNIX domain socket.
The Job Queue Client is also a part of the Zend Server Administration Interface – which, through the same extension, allows the end user to create and manage jobs from the convenience of a web GUI.
The heart of Zend Job Queue is the Job Queue Daemon – this external service takes care of managing scheduling, queuing and executing tasks. It also communicates with the Zend Server User Interface to report any errors and failures. The Job Queue Daemons receive the job details from the Job Queue Extensions over a local connection (a TCP or UNIX domain socket). They then store them in the MySQL DB, while checking for additional jobs to be executed in the DB. Jobs are successfully queued once they are written to persistent storage. This makes sure that even if the daemon has to be restarted, or even in case of complete system failure, jobs are not lost.
A Zend Server cluster uses a MySQL DB server that serves its various components and holds the Job Queue tables. When a node is attached to the cluster, its Job Queue Daemon is directed to use this server and begin submitting and executing jobs through it. The default setup uses a MySQL DB server utilizing InnoDB, the leading transactional storage engine for the MySQL open source database. Because this storage engine is ACID compliant and transactional the default setup of MySQL in the clustered configuration is already highly available and fault tolerant. Other MySQL configurations are possible, please consult MySQL experts for other configuration that are compatible with Zend Server’s usage patterns.
Note:
Job Queue in a Zend Server cluster uses the same scheme that Zend Monitor uses. For more information, see Zend Monitor.
In clustered setups, the Job Queue component uses a MySQL DB as a storage backend. Consequently, performance depends on MySQL's configuration settings.
For a deployment that requires high-performance job queuing, you should consider modifying MySQL’s strategy for flushing data to disk. This can be done by changing the 'innodb_flush_log_at_trx_commit' directive value from 1 to 2, when:
1= Flush data to disk every time a transaction is committed (default).
2= Flush data to disk every second.
For more information, please refer to Oracle / MySQL documentation.
Version
Zend Server 6.3
Zend Server Online Help 6.3v02
© 1999-2014 Zend Technologies, Ltd. All rights reserved.