marsbomber

Ruby (1.9.3) Psych YAML parser issue

2012-01-29T00:00:00-08:00

Ruby 1.9.3 uses a more restrict YAML parser, Psych, instead of the old Syck parser. This caused me issues when using Settingslogic. Even doing what suggested by Settingslogic to set the YAML parser back to Syck didn't help.

A simple test script

Running it under Ruby 1.9.3 outputs this

Using: psych
{"defaults"=>{"cool"=>{"bang"=>"wow", "fruit"=>"apple"}}, "development"=>{"cool"=>{"fruit"=>"banana"}}}
Using: syck
{"defaults"=>{"cool"=>{"bang"=>"wow", "fruit"=>"apple"}}, "development"=>{"cool"=>{"fruit"=>"banana"}}}

You could see "development" lost its "bang" hash key.

There are ways to get around the issue by re-building Ruby with libyaml flag. I consider this as harmful than helpful. So I reverted my application.yml back to its very dumb form, no default options, no merging, just duplicate all settings for all environments. It's definitely stupid to do so, but for now, it's considered as a temporary workaround.

Ruote workflow engine with RabbitMQ (Part 2)

2012-01-20T00:00:00-08:00

I explained how I setup Ruote and RabbitMQ in last post. From there on, you basically already have a working system. All there's left is to have external services subscribe to the RabbitMQ job queues, parse the workitem JSON, do whatever and post the modified/amended JSON back to RabbitMQ, default queue is named "ruote_workitems", which is what the RuoteAMQP receiver listens on.

In this post, I'll demonstrate how a DaemonKit generated daemon script consumes the RabbitMQ job messages and how we construct a DaemonKit compatible Ruote process definition.

Generate DaemonKit project

rvm 1.9.3@daemonkit_ruote
gem install daemon-kit
daemon-kit daemon -i ruote
cd ruote
bundle

Make daemon listen to correct AMQP queues by editing ruote.yml config

defaults: &defaults
  amqp:
    queues:
      - ldap_job
      - email_job

Edit participant class by adding the following 2 methods

def ldap
  workitem["ldap"] = "done"
  # just to demonstrate the AMQP message carries the ID fields and it can be consumed
  workitem["original_cust_id"] = workitem["cust_id"]
end

def email
  workitem["email"] = "done"
end

Let's start everything up

rabbitmq-server
cd /my-ruotekit-enabled-rails-project && rails s
RAKE_TASK=true rake ruote:run_worker
cd /my-daemonkit-ruote-project && bin/daemon

Now let's work out how we wire everything up by carefully crafting a process definition.

Go to http://localhost:3000/_ruote/processes/new in your browser and enter the following as process definition

Ruote.process_definition :name => 'suspend_account', :revision => '0.1' do
  sequence do
    ldap :command => "/sample/ldap"
    # slot in editor to check ldap process result status
    email :command => "/sample/email"
    # slot in editor to check email process result status
    notifier :forget => true # this could be email or final results back in notification queue
  end
end

Enter the following into the workitem fields box

{"cust_id": 111}

Then launch it! If everything goes well(it should!), you'll see no processes listed under the ruotekit app interface. And you'll have 4 queues listed under RabbitMQ's management interface.

email_job, 0 job
ldap_job, 0 job
notify_job, 1 job
ruote_workitems, 0 job

By popping the message out of the notify_job queue, you'll see the serialised JSON string contains ldap done and email done key-value pairs. It shows me the job is done, and all participants are exercised correctly.

Now, we go back a step and explain a little on the process definition we crafted.

If a remote participant will be processed by a DaemonKit daemon script, we need to give each of those participants a hash, which contains a :command key. The value for :command key needs to follow a certain convention too. The convention is "/participant_class_name/method_name". DaemonKit will classify and constantize "participant_class_name" in to ParticipantClassName object, and "method_name" will be sent to it. So in our example, when the daemon script sees command "/sample/ldap", it'll invoke a call to "Sample#ldap".

The value we entered into the workitem fields textbox is simply be merged into the initial workitem hash, and it'll be sent along to all participants.

That is it. Hope it all makes sense, and please tweet me if you found I've got any bits laid out wrongly or explained wrongly.

Ruote workflow engine with RabbitMQ (Part 1)

2012-01-20T00:00:00-08:00

At MYOB, I am assigned the task to build a component based workflow application. There are several requirements this application has to address. Here are a few

Decoupled components
Resilient workflow
Configurable workflow
Every single piece of the application needs to be scalable

A messaging based system becomes somewhat a natural choice to address the decoupling, distributable and scalable issue. While in the hunt of a good workflow system, I was pointed to a Ruby gem called Ruote by @jmettraux. Ruote has also got several companion gems. One of them is ruote-amqp.

Rails Sample Ruote AMQP is the most complete example I could find on Github. This excellent sample project almost covers all aspect. However its Rails default README file doesn't do its justice. I had to dig hard to figure out how all the pieces are put together.

Enough mumbling from me. The following are the steps I implemented to get an end-to-end Ruote + RabbitMQ going on my MBP. As prerequisites, I have Homebrew and RVM installed.

Install RabbitMQ

This is rather simple with the help from Homebrew.

brew install rabbitmq
rabbitmq-plugins enable rabbitmq_management
rabbitmq-server

Now you can monitor your running RabbitMQ instance from the browser at http://127.0.0.1:55672/mgmt/ (login using guest - guest)

Setup RuoteKit enabled Rails project

Create a new rails app called lcp

rails new lcp

Step in the project root, create .rvmrc

rvm 1.9.3@lcp --create

Run bundle install after amending Gemfile with

gem "ruote", "~> 2.2.0"
gem "ruote-kit", "~> 2.2.0.3"
gem "ruote-amqp", "~> 2.2.0"

Add ruote kit initializer. As you could see, the file shown is a work in progress.

# config/initializers/ruote-kit.rb

# TODO: move setting values into a config file
AMQP.settings[:host] = '127.0.0.1'
#AMQP.settings[:vhost] = '/'
#AMQP.settings[:user] = 'guest'
#AMQP.settings[:pass] = 'guest'

# run ruote engine without worker
# run ruote rake task for workers, rake ruote:run_worker
RuoteKit.engine = Ruote::Engine.new(RUOTE_STORAGE)

if ENV["RAKE_TASK"]
  RuoteAMQP::Receiver.new(RuoteKit.engine)
else # don't register participants in rake tasks
  RuoteKit.engine.register do
    participant :ldap, RuoteAMQP::ParticipantProxy, :queue => "ldap_job"
    participant :email, RuoteAMQP::ParticipantProxy, :queue => "email_job"
    participant :notifier, RuoteAMQP::ParticipantProxy, :queue => "notify_job", :forget => true
    #participant :editor

    # register the catchall storage participant named '.+'
    catchall
  end
end

# when true, the engine will be very noisy (stdout)
RuoteKit.engine.context.logger.noisy = false

Add ruote worker rake task

# lib/tasks/ruote.rake

namespace :ruote do
  desc "Run a worker thread for ruote"
  task :run_worker => :environment do
    RuoteKit.run_worker(RUOTE_STORAGE)
  end
end

Last not least, mount RuoteKit to our Rails app by adding the following to the routes file

# ruote-kit
match '/_ruote' => RuoteKit::Application
match '/_ruote/*path' => RuoteKit::Application

This completes first half of the story. We now have a running RabbitMQ instance with admin monitoring, and a RuoteKit enabled Rails app. The Rails app has several RuoteAMQP participants registered, a ruote rake task ready to run Ruote workers, and of course the RuoteKit rack app that allows us to manually launch Ruote jobs.

In the next post, I'll complete the full picture by implementing a DaemonKit generated Ruote daemon script, which processes the AMQP jobs published by the ruote worker running on the Rails app side.

But for now, PEACE!

Ruby basics - Array

2012-01-02T00:00:00-08:00

Here are some basic Ruby array methods that I should learn to use.

& and | Array intersection and union

a = [1, 2, 3]
b = [2, 3, 4]
a & b               #=> [2, 3]
a | b               #=> [1, 2, 3, 4]

collect and select

a = [1, 2, 3, 4]
a.collect(&:odd?)   #=> [true, false, true, false]
a.select(&:odd?)    #=> [1, 3]

delete_if and reject

a = [1, 2, 3, 4]
a.reject(&:odd?)    #=> [2, 4]
a                   #=> [1, 2, 3, 4]
a.delete_if(&:odd?) #=> [2, 4]
a                   #=> [2, 4]

push, pop, shift and unshift

a = [1, 2, 3, 4]
a.push 5            #=> [1, 2, 3, 4, 5]
a                   #=> [1, 2, 3, 4, 5]
a.pop               #=> 5
a                   #=> [1, 2, 3, 4]
a.unshift 0         #=> [0, 1, 2, 3, 4]
a                   #=> [0, 1, 2, 3, 4]
a.shift             #=> 0
a                   #=> [1, 2, 3, 4]

Ruby basics - String

2011-12-23T00:00:00-08:00

One night, I was studying some code written by Aleksey. I spotted his usage of Ruby string's "%" notation. It shows me how elegant Ruby can be, given knowing all those Ruby basics.

To confess, I learned (and still learning) Ruby from learning Rails. At this moment, I defintely think it's a bad idea! Basics should always be taught/learned before the magical frameworks. So, I'll start a series of posts to go through some of these basics.

Ruby string % notation

%Q - Interpolated string. Character used after %Q will be used as the new string delimeter, hence it should be balanced. This makes normal string delimeters such as " and ' become auto-escaped. If the new delimeter also appears in the string, it only needs to be manually escaped if it's unbalanced.

%Q[Don't need to escape " and '. Need to escape \[ manually, but not [ and ]]

%r - Similar to %Q, but constructs a Regexp object. No need to escape those slashes!

%W - Feed it with a string of words separated by spaces, get back an array of words.

%x - Interpolated shell commands. Same as using backticks.

New years resolutions

2011-12-22T00:00:00-08:00

Another year almost goes passed by. Time for doing this new years resolutions thing again.

Let's look back first, and see what I have accomplished in 2011.

~~Work hard on my MYOB works. Make the project we're working on a big success!~~
Work smart on my freelance works. Try getting recurring income.
~~Rails 3 is out for a while now. I WILL make a pet RoR app this year!~~
~~Get myself into Test Driven Development and study Behaviour Driven Development.~~
~~Keep spending quality time with family. At least 2 road trips.~~
~~Take over the duty of sending Oscar to his childcare (that is 3 mornings a week).~~

Okay! 5 out of 6 items are stiked out. Not too bad!

Now, look forward. In 2012, here are the things I want to make happen.

I'll be re-joining MYOB and work on a new project. I want a big success on that!
Keep working on freelance recurring income. I believe things will work out this year.
Finish off my Rails 3 side project.
Keep doing morning and afternoon childcare runs (3 days a week).
Family time! Again, at least 2 family trips.
EDITED: Do exercise to lose weight.

This is it! I'll see how things are tracking by the end of 2012!

Install RVM Ruby on Ubuntu 11.10

2011-12-16T00:00:00-08:00

Because OS X crashes every 2nd day, I thought it's time to give Ubuntu another good spin. First task is to setup my Ruby working environment.

On a fresh Ubuntu 11.10 VM, I firstly installed curl and git via apt-get

sudo apt-get install curl git-core

I then installed RVM using the normal RVM installation command

bash < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer)

Then I tried to install Ruby 1.9.3, I ran into all sorts of issues, readline not available, yaml failed to build, etc. After some fooling around, I found that all I needed to do was to RTFM! Running the following gives me all required packages needed to install an MRI (and others) ruby...

rvm requirements

Believe or not, after doing the sudo apt-get install line from rvm requirements, RVM is happy as!

Lesson learnt: before start blaming a tool (especially a free one!), let's check what the README says first!

Ruby block and Regex

2011-12-06T00:00:00-08:00

Open source is fantastic! I learnt something new from reading Rails source code today again.

Today's surprise was brough to me by the power of regular expression and Ruby blocks.

Rails 3 Net::HTTPHeader module has the following method

def urlencode(str)
  str.dup.force_encoding('ASCII-8BIT').gsub(/[^a-zA-Z0-9_\.\-]/){'%%%02x' % $&.ord}
end

1st, I learnt the gsub method could take a block as its second argument.

2nd, inside the block, all regular expression back references could be used, e.g. $& for referencing all matching elements.

Android screen grab

2011-11-29T00:00:00-08:00

I needed to publish an Android app for one of our clients today. Screenshots of the app is required for publishing.

To do it, extremely simple.

Get application onto testing Android device
Connect Android device to computer, make sure device's debug mode is enabled.
From the computer, start the DDMS program that's shipped with the Android SDK
From DDMS, highlight the connected Android device, and go to menu "Device" - "Screen capture"

Not too hard. Have to say Apple's native screen capture feature on iOS devices is a lot simpler.

RVM update

2011-11-05T00:00:00-07:00

RVM is super easy to use, and it manages Ruby Gems well. Most of the Ruby/Rails projects I work on are using its Gemset feature.

I was running RVM version 1.6 and decided to perfom and upgrade.

rvm get latest
rvm upgrade 1.9.2-p180 1.9.2-p290

Second command from above is extremely useful. As per example above, it updates my local Ruby 1.9.2-p180 to 1.9.2-p290. All gemsets associated with the 1.9.2-p180 are also relinked to the new 1.9.2-p290. Handy!

Create SQLite DB from CSV

2011-10-22T04:42:02-07:00

Recently a project I'm working on requires me to populate an SQLite database with CSV contents. A very straight forward task that I have done so many times before in PHP.

This round Ruby is the pick. Everything just feels a bit nicer.

Ruby string interpolation

2011-08-20T03:26:23-07:00

We store configuration strings in YAML files. It's common that we need to substitute part of the string with runtime values. Hence we introduce some kind of placeholders in the config strings. Given an example that our application may need to hit an external URL to retrieve some remote resources at runtime. To do this, we could make a config string in our application's config YAML file like this

remote_url_template: "http://remote.domain.com/give/me/RESOURCE1/and/RESOURCE2"

Then in our code, we substitute "RESOURCE1" and "RESOURCE2" with the real resource names we require. In my early days of programming in Ruby, I used some very ordinary string substitution method like this

# remote_url_template is read from the config yaml
resource_one = "coke"
resource_two = "chips"
remote_url = remote_url_template.gsub("RESOURCE1", resource_one).gsub("RESOURCE2", resource_two)

It works, but I don't really like it... So I found something better, that is to use Ruby's string interpolation methods

# in yaml file
remote_url_template: "http://remote.domain.com/give/me/%s/and/%s"

# in ruby code
# remote_url_template is read from the config yaml
resource_one = "coke"
resource_two = "chips"
remote_url = sprintf(remote_url_template, resource_one, resource_two)

A lot nicer, but still can be better. Thanks to a colleague of mine, Aleksey Gureiev, another more elegant way can be used

# in yaml file
remote_url_template: "http://remote.domain.com/give/me/%s/and/%s"

# in ruby code
# remote_url_template is read from the config yaml
resource_one = "coke"
resource_two = "chips"
remote_url = remote_url_template % [resource_one, resource_two]

That is satisfactory!

Manage Rails app configurations

2011-07-23T06:06:12-07:00

Most of applications require application configuration files. The Rails applications I worked on pretty much all require an initialiser (or should it be the Rails convention initialiZer) to load a configuration YAML file. I'm aware that there are gems out there providing this kind of configuration solutions, such as SettingsLogic. For a simple problem like this one, I think we could roll our own too (oh, the not invented here syndrome).

The codebase I worked on typically has an initialiser called load_config.rb. It's got something like this

# config/initializers/load_config.rb

AppConfig = YAML.load_file(File.join(RAILS_ROOT, 'config', 'config.yml'))

# Override config options by correct environment
env_options = AppConfig.delete(RAILS_ENV)
AppConfig.merge!(env_options) unless env_options.nil?

Easy and good. There's one more thing I'd love to add to this. I'd love to have a local config file, config.local.yml, that can overwrite config values defined in the config.yml. And this config.local.yml is not source controlled, means every developer can make tweak the configs to tailor to their own development environment without needing to touch the source controlled configuration file, config.yml.

To do this, just add the following block of code to the load_config.rb

local_config_filepath = File.join(RAILS_ROOT, 'config', 'config.local.yml')
if File.exist?(local_config_filepath)
  AppConfig.merge!(YAML.load_file(local_config_filepath))
end

CoffeeScript Basics

2011-07-19T21:35:35-07:00

Rails 3.1 will support CoffeeScript out of the box. I gave the new boy in town a little test. I think it'll be a great tool for web developers writing javascript code. CoffeeScript is short and precise, generated JavaScript variables are properly scoped, class inheritance is made super easy, love the syntax sugar. Oh and yes, mixing in jQuery is easy too.

Here's a little play script I wrote to illustrate how a JavaScript class can be written in CoffeeScript

class Person
  ### private var ###
  my_name = null

  ### constructor ###
  constructor: (name) ->;
    my_name = name

  ### public instance method ###
  speak: (saying)->;
    alert "#{saying} #{fancy_name()}!"
 
  ### private instance method ###
  fancy_name = ->;
    if my_name == "Jim"
      "Jimbob"
    else if my_name == "Dan"
      "DanDaMan"
    else
      my_name

  ### class method ###
  @feature: ->;
    alert "1 Person has only 1 head"

Person.feature()
jim = new Person("Jim")
jim.speak("Hi")

The above CoffeeScript will be translated into the following JavaScript

var Person, jim;
Person = (function() {
  /* private var */  var fancy_name, my_name;
  my_name = null;
  /* constructor */
  function Person(name) {
    my_name = name;
  }
  /* public instance method */
  Person.prototype.speak = function(saying) {
    return alert("" + saying + " " + (fancy_name()) + "!");
  };
  /* private instance method */
  fancy_name = function() {
    if (my_name === "Jim") {
      return "Jimbob";
    } else if (my_name === "Dan") {
      return "DanDaMan";
    } else {
      return my_name;
    }
  };
  /* class method */
  Person.feature = function() {
    return alert("1 Person has only 1 head");
  };
  return Person;
})();
Person.feature();
jim = new Person("Jim");
jim.speak("Hi");

Background jobs on Heroku

2011-06-30T02:43:19-07:00

I'm recently asked what my recent technology excitement is, of course in the context of programming. The name of Heroku came into my mind instantly. I've been using Heroku for all my Rails based applications recently. I'm amazed how easy it is to deploy and scale (to a degree) applications on to Heroku.

Heroku's default way of running background jobs is using the delayed_job Ruby gem, which essentially records background jobs into a database table. And on Heroku, you can start a worker to monitor and process the job queue. This involves a lot of database polling as you could imagine.

So, in one of the applications I'm working on, I tried something more elegant (in my opinion). Github uses a Ruby gem called Resque for their background jobs.

Resque uses a Redis based backend to store queued jobs, so it's fast in job logging and job retrieving.
Resque allows you to separate jobs into different queues, so everything becomes better categorised.
Resque also provides a Sinatra based web interface to monitor all jobs including failures, so better monitoring and better debugging.

All good. I also opted to use a Resque plugin called resque-heroku-autoscaler. This allows me to dynamically scale the number of Heroku workers up and down base on the jobs recorded in the queue. Since Heroku charges on worker usage, so with this auto scaler, I only pay for what's used.

As for the implementation, it's scattered across many files. Here's my implementation, which handles the delivery of welcome emails upon membership creation.

Gemfile

gem 'resque', :require => "resque/server"
gem 'resque-heroku-autoscaler'

config/resque.yml

development: 127.0.0.1:6379
test: 127.0.0.1:6379

config/initializers/resque.rb

require 'resque'

heroku_environments = ["staging", "production"]
rails_root = ENV['RAILS_ROOT'] || File.dirname(__FILE__) + '/../..'
rails_env = ENV['RAILS_ENV'] || 'development'

unless heroku_environments.include?(rails_env)
  resque_config = YAML.load_file(rails_root + '/config/resque.yml')
  Resque.redis = resque_config[rails_env]
else
  uri = URI.parse(ENV["REDISTOGO_URL"])
  Resque.redis = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
end

config/initializers/resque_heroku_autoscaler_setup.rb

require 'resque/plugins/resque_heroku_autoscaler'

Resque::Plugins::HerokuAutoscaler.config do |c|
  c.heroku_user = 'heroku_user_name'
  c.heroku_pass = ENV['HEROKU_PASSWORD']
  c.heroku_app  = ENV['HEROKU_APP']

  if Rails.env.production?
    c.new_worker_count do |pending|
      (pending/5).ceil.to_i
    end
  end

  if Rails.env.development?
    c.scaling_disabled = true
  end
end

app/models/membership_observer.rb

class MembershipObserver < ActiveRecord::Observer
  def after_create(membership)
    Resque.enqueue(MailerCallback, "MyMailer", :welcome_email, membership.id)
  end
end

app/models/mailer_callback.rb

require 'resque/plugins/resque_heroku_autoscaler'

class MailerCallback
  extend Resque::Plugins::HerokuAutoscaler

  # resque queue name
  def self.queue
    :email_queue
  end

  # resque callback method
  def self.perform(mailer, email_type, *args)
    mailer.constantize.send(email_type, args).deliver
  end
end

Reference

Ruby on Rails, 2 months in

2011-04-24T05:41:35-07:00

I've started my "senior" developer role in doing Ruby on Rails projects for a bit over 2 months. Reason I quoted the "senior" bit is that I'm really a newbie to it. I guess the seniority is more about general web based application development rather than RoR. Having said that, RoR isn't that foreign to me at all. I started studying it a year ago. However my self-study doesn't compensate my lack of real world working experience. In the past 2 months, lots of ideas and knowledges are exchanged between me and my peers. I learnt a lot of Rails of course. And I believe I have also contributed some non-rails thinking into the Rails project.

2 months in, I learnt that Rails is moden and rapid in doing web development (I absolutely love it). However it's convention over configuration nature sometimes dictates the way developers think too much. Somehow, developers' creativities become canvased by Rails. You find yourself battling with routes, ActiveRecord models, DRY controllers a lot. The application architectural deliberating stage is somewhat overlooked, if not bypassed. Most of the time, this will lead to future potential problems, which could be security issues, scaling issues, performance issues or just tedious labour work in doing repetitive tasks.

As I'm still learning, what I believe now is that don't let Rails control how you author software,

write non AR models to describe your domain if required
use service layer classes to encapsulate business logic if it makes sense
off load long running, non critical tasks to a job queue (loving the resque gem now) when it's appropriate
the list goes on

Let Rails help you instead,

use its powerful command line tool
utilise its ecosystem, all those wonderful gems
make a use of its RESTful controllers, exposing API is never as easy as before
look into it's respond_to feature, making your RESTful API respond to XML, json, or whatever is made super easy
the list goes on

To summarise, the basic software engineering practices apply to Rails or non Rails applications the same way.

Bend the rails to let your train reach it's destination. Expect more Rails related posts from now on.

Install and setup Gitosis

2011-04-01T23:14:00-07:00

Github is great for social coding. To get private repositories, it's starting price is cheap and fair. However for someone who already owns a Linux VPS, as well as an adequate backup solution, you may want to do it the hard way: setup your own remote central Git server using Gitosis. This is how I did it on my Ubuntu VPS, which is hosted on MediaTemple. My setup steps are pretty much copied from 2 great Gitosis setup tutorials, which I will reference at the foot of this post. This reference post probably benefits myself more than anyone else.

SSH into the remote VPS machine,

Step 1: Setup Git

sudo apt-get install git-core

Step 2: Setup Gitosis

sudo apt-get install python-setuptools
cd ~/downloads
git clone git://eagain.net/gitosis.git
cd gitosis
sudo python setup.py install

Step 3: Setup git user

sudo adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git

Step 4: Set up Gitosis admin user access and init Gitosis admin repository

From your local computer, generate (if not done before) and copy the public SSH key to the VPS server. Place and name it as "/tmp/id_rsa.pub", then

sudo -H -u git gitosis-init < /tmp/id_rsa.pub
sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update

Up to this point, your have done all the works required on the VPS server. You will next jump back to your local machine and admin Gitosis from there.

First thing first, let's clone the Gitosis admin repository, so that you could tweak any admin configuration settings locally and update your Gitosis server with a simple git push

cd ~/VpsAdmin
git clone git@YOUR_SERVER:gitosis-admin.git

Next you'll create a new project locally, push it to the Git server hosted on the VPS, and make the new project's remoto repository accessible by other team members.

Step 1: Add new project to Gitosis admin config

vim ~/VpsAdmin/gitosis-admin/gitosis.conf

Add a new config block to the bottom of the file, like this

[group new_shared_project]
members = jim bob
writable = new_shared_project

Note that "new_shared_project" will be your new project's remote Git repository name. "jim" in this case is yourself (Gitosis admin) and "bob" is your team member, who will also have access to this project. "jim" matches the name your used in your public key, which was used when setting up Gitosis from previous steps. "bob" matches the name, which your team member used in his public key, which will be used in the next step.

Step 2: add team member's public SSH key to Gitosis

Obtain Bob's public SSH key and put it under ~/VpsAdmin/gitosis-admin/keydir. Give it a sensible file name so that you can easily relate it back Bob in the future.

Step 3: Commit changes and push it up to Gitosis server

cd ~/VpsAdmin/gitosis-admin
git commit -a -m "create new repository and setup access for Bob"
git push

Step 4: Create new project and push it to remote Gitosis server

cd ~/Workspace
rails new new_shared_project
cd new_shared_project
git init
git remote add origin git@YOUR_SERVER:new_shared_project.git
git add .
git commit -a -m "initial import"
git push origin master:refs/heads/master

Step 5: Share the remote repository with your team member

Team member Bob can access the remote repository from any computers that have his public SSH key installed

git clone git@YOUR_SERVER:new_shared_project.git

Reference

Setup Synergy

2011-03-02T02:26:50-08:00

I spent some time this morning setting up Synergy. Synergy allows me to use one set of mouse and keyboard to control multiple computers. The setup is is fairly straight forward. I just downloaded the latest stable release and followed the instruction from the Synergy site. My Synergy server is my MBP, and my Synergy client is an OS X 10.6 server. After the initial setup, my mouse is successfully shared. However my keyboard sharing is not working. A friend of mine suggested me to try an old release from the Synergy2 project, so I did. I downloaded Synergy2 1.3.1 from here, repeat the same setup procedure. Both keyboard and mouse sharing are working nicely.

I then created a couple of automator applications, one on my MBP, the other on the OS X server. This allows me to start the Synergy server and client automatically on system startups.

On the Synergy server (my MBP). I created an automator application. Drag "Run Shell Script" into the workflow. The script I'm using is

#!/bin/sh
prog=(/PATH-TO-SYNERGY/synergys --no-daemon --config /PATH-TO-SYNERGY-CONF/synergy.conf)

### Stop any currently running Synergy client
killall ${prog[0]##*/}

### Start the new client
exec "${prog[@]}" -f > /tmp/synergy.log 2>&1 &

Save the automator application into /Applications. Add the new application to account login items from System Preferences.

Similarly, on the Synergy client (OS X Server) machine, repeat the above steps. Substitute the script content with

/PATH-TO-SYNERGY/synergyc -f SERVER-MACHINE-HOSTNAME > /tmp/synergy.log 2>&1 &

This is it. See if this helps you in any ways and let me know if the automation bit can be improved (I'm sure there are better ways to register startup processes on os x).

Reference

New job starting tomorrow

2011-02-16T00:00:00-08:00

I finished my last day’s work at MYOB and will be starting a new job tomorrow straight away.

It’s been a very interesting journey at MYOB for the last 7 months or so. The colleagues are not just very nice, but also very knowledgable in what they do. The product we’re building is exciting and challenging (and BIG, hopefully, it’ll be launched soon, and I can start talking about it!!!). My Zend Framework and Doctrine skills are exercised on a daily basis. Even better, MYOB’s Agile methodology adoption progress is applaudable. Having the whole business behind such a methodology is easy to say, but very hard to implement. Thumbs up!!!

Moving on to the new job at ProjectProject. Technically, it’s a whole new chapter for me. Ruby will be used over PHP. To me, this change is like having dinner with knives and forks instead of chopsticks. It’ll be a new set of tools to learn and master. Luckily at the new company, the business guys are adventurous and smart, the dev guys are nice and sharp. I’m sure I’ll learn heaps and contribute heaps from and to my new role.

Open folder in Textmate

2011-01-08T03:51:41-08:00

This is a note to myself. Create an automator service with

Saving the script as "Open in Textmate". From now on, I can right click on any file or folder from Finder to open it in Textmate.

Reference: http://justaddwater.dk/2009/12/08/how-to-add-open-folder-in-textmate-to-finder-context-menu/

New years resolutions - 2011

2010-12-28T02:02:56-08:00

It's been another year. 2010 is at its end. Now it's time to review my 2010 resolutions and set goals for 2011.

Goals I set for 2010. Half are achieved.

I ended up having no time to make a Titanium mobile app nor a RoR pet project. This was partly caused by me starting a new job in July. As for the house, I did get a mower for the lawn. However nothing's done to the house itself. We might just leave it as it, and do something major to it later on (maybe 2012).

~~Take a get-away holiday. It’s been almost 4 years since my last proper holiday. My wife deserves better …~~
~~Cutback on freelancing and spend more time with family, especially my son, Oscar.~~
Do something with the house please … paint the doors, replace the windows, fix up the deck and more importantly let’s mow the bloody lawn.
Make one hobby mobile application using “Titanium”.
Learn Ruby on Rails by writing one side project.
~~Add a portfolio page to the ElinkMedia site.~~

Looking forward, following are the things I want to achieve in 2011.

Work hard on my MYOB works. Make the project we're working on a big success!
Work smart on my freelance works. Try getting recurring income.
Rails 3 is out for a while now. I WILL make a pet RoR app this year!
Get myself into Test Driven Development and study Behaviour Driven Development.
Keep spending quality time with family. At least 2 road trips.
Take over the duty of sending Oscar to his childcare (that is 3 mornings a week).

This is it. I'll see how I'm tracking throughout 2011.

NHibernate - update tables with triggers

2010-11-17T22:06:05-08:00

I was finishing off the windows service project I was working on. After installing the windows service on the production server, I got exceptions thrown by Castle ActiveRecord. After looking into the stack trace, I found that NHibernate was throwing exceptions like "Unexpected row count: 2; expected: 1".

After some investigation, I found this exception was caused by the trigger implemented on a database table that NHibernates tries to update.

Not sure if this is a database specific error. My application interacts with a MS SQL Server 2005 instance. The workaround is kinda simple.

In the trigger, find the statement that does the updates. Wrap it with "SET NOCOUNT ON" and "SET NOCOUNT OFF". Issue fixed.

Enable Memcache on MAMP

2010-11-02T01:59:40-07:00

I switched to MAMP from XAMPP for a while. A few days back I needed to make my localdev PHP script use Memcached servers. After some digging around, I found this nice post http://www.lullabot.com/articles/setup-memcached-mamp-sandbox-environment

I simply followed the steps to setup my local memcached service. However when I try to enable my local MAMP with the memcache module, I ran into some issues. Since I use PHP 5.3.2 on my localdev, I couldn't find a suitable memcache.so to plugin to my MAMP installation. So, I did some research, and compiled the it myself from source. Here's how I did it

Download PHP 5.3.2 source
Untar the source and place everything into /Applications/MAMP/bin/php5.3/include/php
Open terminal and go to the /Applications/MAMP/bin/php5.3/include/php directory
Run ./configure to generate all necessary header files, which will be used to compile memcache.so
Download the stable version of memcache from PECL
Unpack the source and go into that directory from terminal
Run the following

jimli@Jim-Lis-MBP: phpize
jimli@Jim-Lis-MBP: CFLAGS='-O3 -fno-common -arch i386 -arch x86_64 -g -Os' LDFLAGS='-O3 -arch i386 -arch x86_64' CXXFLAGS='-O3 -fno-common -arch i386 -arch x86_64 -g -Os' ./configure --disable-dependancy-tracking --disable-shared -enable-static
jimli@Jim-Lis-MBP: make
jimli@Jim-Lis-MBP: sudo cp modules/memcache.so /Applications/MAMP/bin/php5.3/lib/php/extensions/no-debug-non-zts-20090626/.

Enable memcache from php.ini, restart MAMP

That's it. Hope you find it's useful.

Reference

http://www.lullabot.com/articles/setup-memcached-mamp-sandbox-environment

Castle AR and NHibernate

2010-08-01T05:23:08-07:00

It's been a while since my last post. So a bit update first.

Lots of things actually happened or happening in July. I started a new job from the start of the July, hence very busy! I also needed to help out my former employer for the hiring of the new developer. The interview process was quite tiring, but very different and very challenging. I actually enjoyed it and I believe I learnt a lot from it.

As for the technical development side, with my new job, I'm doing a new project in Zend Framework of course. I'm getting my grasp of it and hopefully I can make some valuable contributions along the way. I'm also finishing off a C#.Net project for my former employer as well. This blog post will be about the 2 .Net libraries that I used for this project.

For this .Net project, I started with using the NHibernate ORM library. I found my hate of XML configuration pretty soon after I kick started the project. Consider the complexity of this application is quite low, I jumped on the Castle ActiveRecord library, which uses NHibernate as the underlying ORM.

Don't want to offend anyone, but the documentation of both libraries are quite average. So here are some STUFF I had to trial and error to find out myself.

1st, for a ASP.Net project, here's part of the Web.config to get Castle going and also logging the raw SQLs that are generated by NHibernate.

<configSections>
    <!-- your normal stuff ... -->
    <section name="activerecord" type="Castle.ActiveRecord.Framework.Config.ActiveRecordSectionHandler, Castle.ActiveRecord" />
    <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net" />
</configSections>

<activerecord isWeb="true" isDebug="true" >
    <config>
        <add key="connection.provider" value="NHibernate.Connection.DriverConnectionProvider" />
        <add key="dialect" value="NHibernate.Dialect.MsSql2000Dialect" />
        <add key="connection.driver_class" value="NHibernate.Driver.SqlClientDriver" />
        <add key="connection.connection_string" value="Data Source=JIMLIWIN7\SQLEXPRESS;Initial Catalog=DBNAME;Persist Security Info=True;User ID=USER;Password=PASSWORD" />
        <add key="proxyfactory.factory_class" value="NHibernate.ByteCode.Castle.ProxyFactoryFactory, NHibernate.ByteCode.Castle" />
        <add key="show_sql" value="true" />
    </config>
</activerecord>

<log4net>
    <appender name="NHibernateRollingFileAppender" type="log4net.Appender.RollingFileAppender">
        <file value="LogNHibernate.txt"/>
        <appendToFile value="true"/>
        <rollingStyle value="Size"/>
        <datePattern value="yyyyMMdd"/>
        <maxSizeRollBackups value="10"/>
        <maximumFileSize value="10MB"/>
        <layout type="log4net.Layout.PatternLayout">
            <conversionPattern value="%date - %message%newline"/>
        </layout>
    </appender>

    <logger name="NHibernate.SQL" additivity="false">
        <level value="DEBUG"/>
        <!-- ALL, DEBUG, INFO, WARN, ERROR, FATAL or OFF -->
        <appender-ref ref="NHibernateRollingFileAppender"/>
    </logger>
</log4net>

2nd, in my entity classes, I have properties that are linked to nullable fields. To get this whole AR thing happy, I do the following. The trick is the "?".

private System.DateTime? _dateDeleted;

[Property("DateDeleted", ColumnType = "Timestamp")]
public virtual System.DateTime? DateDeleted
{
    get
    {
        return this._dateDeleted;
    }
    set
    {
        this._dateDeleted = value;
    }
}

There are other things that have killed me hours. But I think those are just because of my own stupidity and incompetency in .NET ... So I'll keep those to myself. Bye for now.

Zend Framework 1 and Doctrine 2 integration - modular setup

2010-06-19T19:11:12-07:00

This is a follow up post to my previous "Zend Framework 1 and Doctrine 2 integration".

I've created a new branch on my Github project "zf1-doctrine2". The "modular_setup" branch shows how we could setup Zend Framework with the modular approach, while still be able to use Doctrine 2 as the ORM for each module.

Oh well, the idea is to make each module decoupled with the rest of the application, so really, you could use whatever database handling mechanism in each module.

Check it out from my Github project here.

Reference

http://github.com/marsbomber/zf1-doctrine2/tree/modular_setup

Zend Framework 1 and Doctrine 2 integration

2010-06-18T21:18:37-07:00

Doctrine 2 is in beta. It is for sure one of the most exciting things that caught my attention recently.

As a Zend Framework user, I started looking into ways to integrate Doctrine 2 with Zend Framework (currently in version 1).

The same as any other times, the solutions are already out there. The integration's already been done by Giorgio Sironi's nakedphp project.

So ... I borrowed code from nakedphp and put up a skeleton ZF projects myself.

My sample project is available on Github. You can find it here.

Have a play, let me know if anything. Hope it'll help people like myself, who cannot wait to try out new and cool things.

Reference

http://github.com/marsbomber/zf1-doctrine2

Compile APC for XAMPP on Snow Leopard

2010-05-03T22:34:14-07:00

I'm using XAMPP as my local development environment on Snow Leopard. The XAMPP package does not include APC (Alternative PHP Cache). If we simply do a PECL install, you'll find that PHP complains about the APC architecture. This happens because the XAMPP for OS X is 32 bit, and Snow Leopard is 64 bit, hence PECL builds a 64 bit apc.so.

To overcome this issue, we need to manually compile APC from source and here is how I did it.

First, download APC source from http://pecl.php.net/package/APC and double click on the tar file within finder will unzip it.

Then

cd ~/Downloads/APC-3.1.3p1/APC-3.1.3p1/
phpize
MACOSX_DEPLOYMENT_TARGET=10.6 CFLAGS="-arch i386 -g -Os  -pipe -no-cpp-precomp" CCFLAGS="-arch i386 -g -Os  -pipe" CXXFLAGS="-arch i386 -g -Os  -pipe" LDFLAGS="-arch i386 -bind_at_load" ./configure --enable-apc --with-apxs --with-php-config=/Applications/XAMPP/xamppfiles/bin/php-config-5.3.1
make
sudo cp modules/apc.so /Applications/XAMPP/xamppfiles/lib/php/php-5.3.1/extensions/no-debug-non-zts-20090626/.

We should see no errors and we should have the new APC so file copied into our PHP extension directory.

Next, enable APC from php.ini by adding the following 2 lines

extension=apc.so
apc.shm_size = 32

Now if you do php -m from command line, you'll see APC listed as a loaded module.

Hope it helps and save you from pulling your hair out.

Reference

Integrate Zend_Tool with Netbeans 6.9 beta

2010-04-26T21:27:50-07:00

I've recently read a lot about Zend Framework (Zend_Tool) integration with Netbeans. Today I decided to download the Netbeans 6.9 beta and setup everything on my local development machine (OS X).

Here is how I did it on OS X with XAMPP.

First, I included Zend Framework library via a PEAR repository hosted on Google Code.

sudo pear channel-discover zend.googlecode.com/svn
sudo pear install zend/zend

Completing the above will make Zend Framework available through PEAR, which should be in your PHP include_path.

Next, I downloaded and extracted the minimal Zend Framework package. Then I did

cd ~/Downloads/ZendFramework-1.10.3-minimal/bin
sudo cp zf.php /Applications/XAMPP/xamppfiles/bin/zf.php
sudo cp zf.sh /Applications/XAMPP/xamppfiles/bin/zf

After this, I can run zf from command line without having any errors. So ... I did this

zf --setup storage-directory
zf --setup config-file

I can now download the latest Netbeans 6.9 beta and do the normal install. After installation, start up Netbeans, go to "Netbeans" -> "Preferences" from the menu bar. Navigate to the "PHP" -> "Zend" section. Enter "/Applications/XAMPP/xamppfiles/bin/zf" into the "Zend script" input box. Then click on "Register Provider" button.

Now Netbeans should have updated the zf config file, which I created by running the "zf --setup config-file" command. And at this stage, I can create Zend Framework based projects and run "zf" commands to manipulate my projects within Netbeans.

However, if I jump back to the terminal shell, I would notice the good old "zf" command is broken and complaining it cannot find a file called "NetBeansCommandsProvider.php". Since I still want my terminal command line zf script working, I did this

sudo ln -s /Applications/NetBeans/NetBeans\ 6.9\ Beta.app/Contents/Resources/NetBeans/php/zend/NetBeansCommandsProvider.php /Applications/XAMPP/xamppfiles/lib/php/pear/NetBeansCommandsProvider.php

Now I am happy. Hope someone find it's useful

Config phpUnderControl for ZF based applications

2010-03-21T17:35:49-07:00

I recently reinstalled my MacBook Pro, and it proves that my earlier post on how to setup phpUnderCotnrol on Snow Leopard work well. I just followed the steps and phpUnderControl's up and running again.

Now, I want to go into the details a bit to show the configurations I have done to use phpUnderControl to do continuos integration for my Zend Framework based applications. The testing application I used here is the ZFDoctrine12 project I've created. You could pull down the source code of it from my github project.

First, I create the necessary folder structure for a new phpUnderControl project, as well as the ant build file.

sudo mkdir /opt/cruisecontrol/projects/zfdoctrine12
sudo mkdir /opt/cruisecontrol/projects/zfdoctrine12/source
sudo mkdir /opt/cruisecontrol/projects/zfdoctrine12/build
sudo mkdir /opt/cruisecontrol/projects/zfdoctrine12/build/api
sudo mkdir /opt/cruisecontrol/projects/zfdoctrine12/build/coverage
sudo mkdir /opt/cruisecontrol/projects/zfdoctrine12/build/logs
sudo mkdir /opt/cruisecontrol/projects/zfdoctrine12/build/php-code-browser
sudo touch /opt/cruisecontrol/projects/zfdoctrine12/build.xml

Next, the ant build file content. You can see it's fairly minimal, but you should get the idea and know how to beef it up

<?xml version="1.0" encoding="UTF-8"?>
<project name="zfdoctrine12" default="build" basedir=".">
  <target name="build" depends="php-codesniffer,phpunit"/>
  <target name="php-codesniffer">
    <exec executable="phpcs" dir="${basedir}/source" output="${basedir}/build/logs/checkstyle.xml" error="/var/tmp//checkstyle.error.log">
      <arg line="-n --report=checkstyle --standard=ZEND application"/>
    </exec>
  </target>
  <target name="phpunit">
    <exec executable="phpunit" dir="${basedir}/source/tests" failonerror="on">
      <arg line=" --log-junit ${basedir}/build/logs/phpunit.xml --coverage-clover ${basedir}/build/logs/phpunit.coverage.xml --coverage-html ${basedir}/build/coverage"/>
    </exec>
  </target>
</project>

Next, I want to clone the project from my local git repository into my phpUnderControl monitored project directory. To do this, we do

cd /opt/cruisecontrol/projects/zfdoctrine12/source/
sudo git clone ~/Sites/zfdoctrine12/ .

You can see that I'm only cloning the project from my local project repository, however this can be easily changed to utilise a remote git repository. e.g. repositories hosted on github.

I admit that after doing the git clone, I had to manually create and copy a couple of files. This was because of my .gitignore file from my local repository. So, go to the .gitignore file from your project repo, and do whatever changes that make sense to you.

Now, let's setup the phpUnderControl config.xml file, to make phpUnderControl become aware of our newly created project. Add the following XML snippet to your /opt/cruisecontrol/config.xml.

<project name="zfdoctrine12">
  <listeners>
    <currentbuildstatuslistener file="logs/${project.name}/status.txt"/>
  </listeners>
  <modificationset quietperiod="60">
    <git localWorkingCopy="projects/${project.name}/source/" />
  </modificationset>
  <bootstrappers>
    <gitbootstrapper localWorkingCopy="projects/${project.name}/source/" />
  </bootstrappers>
  <schedule interval="300">
    <ant anthome="apache-ant-1.7.0" buildfile="projects/${project.name}/build.xml"/>
  </schedule>
  <log dir="logs/${project.name}">
    <merge dir="projects/${project.name}/build/logs/"/>
  </log>
  <publishers>
    <artifactspublisher
      dir="projects/${project.name}/build/api"
      dest="artifacts/${project.name}"
      subdirectory="api"/>
    <artifactspublisher
      dir="projects/${project.name}/build/coverage"
      dest="artifacts/${project.name}"
      subdirectory="coverage"/>
    <execute
      command="phpcb
              --log projects/${project.name}/build/logs
              --source projects/${project.name}/source
              --output projects/${project.name}/build/php-code-browser"/>
    <artifactspublisher
      dir="projects/${project.name}/build/php-code-browser"
      dest="artifacts/${project.name}"
      subdirectory="php-code-browser"/>
    <execute command="/Applications/XAMPP/xamppfiles/bin/phpuc graph logs/${project.name} artifacts/${project.name}"/>
    <email
      buildresultsurl="http://localhost:8080/cruisecontrol/buildresults/zfdoctrine12"
      mailhost="smtp.gmail.com"
      mailport="465"
      usessl="true"
      username="USERNAME"
      password="PASSWORD"
      returnaddress="SENDER_EMAIL">
      <always address="RECIPIENT_EMAIL"/>
    </email>
  </publishers>
</project>

The above project config tells CruiseControl to look for VCS (git in this case) changes, if any changes are detected, a build will be triggered every 5 min. The result is then emailed out to a nominated email address.

This is it. It should give you a start point, and then it's up to you to tune it yourself. Hope it helps for people like myself, who is a beginner in Continuous Integration.

Reference

Day 1 of learning Symfony

2010-03-06T20:51:37-08:00

Long weekend. I'll try to keep it short.

I'm a Zend Framework devotee. However Symfony is always on my to-learn list. I'm finally motivated today to give it a crack. Not to deny, a potential job lead also plays an important part in this.

I just spent the last 2 hours to download the library via SVN, follow the "Practical symfony" guide to setup a project and everything. All pretty straight forward. I'm on "Day 5" of the guide now. I'll stop here and finish the "tech" part of the day with this post (before my wife goes mad at me ...).

So, what do I think about Symfony?

People talk about the steep learning curve of studying Symfony, but I didn't find it's too hard to follow. I think this is because of 2 reasons. 1, I understand the MVC pattern pretty well from my Zend Framework projects. 2, Symfony has got heaps of similarity to Ruby on Rails and I have finished reading my RoR book at the start of the year.

I quite like the Symfony view helpers, the YAML style configurations, the out of the box modular approach and the command line generators.

If time allows, I'll blog more about Symfony in the future. For the next a couple of weeks or so, I might have to do some C#.Net for my next project.

Reference

http://www.symfony-project.org/jobeet/1_4/Doctrine/en/

Modal window over flash objects

2010-02-19T00:44:53-08:00

I've been working on a WordPress site for a client. All pretty straight forward stuff. Only special requirement from the client is to have a categorised file repository as a WordPress page, which allows users to download files. One of the categories needs to be protected. So users click on a file download link and see a modal window with a form. Fill in the details, download the file.

The issue surfaced when we replaced the site banner with a flash (this is of course IE specific ...). The popup modal window slides behind the flash object. This turned out to be a common issue and the fix is simple.

<object classid="clsid:D27CDB6E...." width="1151" height="209" ... >
    .... you other flash params stuff ...
    <param name="wmode" value="transparent">
</object>

The "wmode" transparent is the key!

I'm using SimpleModal, a jQuery plugin modal library.

Reference

http://www.ericmmartin.com/projects/simplemodal/

Zend Framework on shared host

2010-02-09T03:09:14-08:00

I came across an article made by Rob Allen (author of Zend Framework in Action) today. It gives us a very nice solution on how to host a Zend Framework project on a shared host.

The issue with running Zend Framework based application on a shared host is that sometimes you simply do not have the luxury to have "public" as your document root. When I say "public", I refer to the web root created in Zend Framework (the directory that contains index.php). Sometimes, your FTP root is your document root. This means you are forced to dump all other Zend Framework folders inside the document root.

2 issues with that,

Security, everything becomes exposed to the web.
Ugly URL, you'll end up with something like http://mydomain.com/public/

Rob's post addressed this problem really well.

Reference

http://akrabat.com/zend-framework/zend-framework-on-a-shared-host/

Netbeans debugger with Zend Framework

2010-02-08T01:28:06-08:00

Are you sick of doing var dumps on browser screens or error logs when you need to debug some errors? Have you seen the .Net guys step through their code line by line to figure out what went wrong? With PHP, you can do the same. Here's how I do it.

The IDE I use is Netbeans (6.8 on Mac). It's shipped with a built-in debugger, which works with Xdebug.

So ... we first need to make sure our PHP is running Xdebug instead of Zend. I have explained how I have Xdebug setup on my XAMPP stack on Mac from one of my previous posts, Setup phpUnderControl on Snow Leopard. We can verify whether Xdebug is setup properly by loading up our phpinfo page. We should see something like this (notice the last line prints Xdebug)

Now, let's go to Netbeans. Netbeans out of the box listens on port 9000, which is the exact port number we set our Xdebug remote_port to in our previous step. This means if we are not running our application with mod_rewrite, we now should be able to set a breakpoint in our code, hit the debug icon, and have everything going.

But ... there's always a but. When we develop Zend Framework based applications, in fact any applications that utilising mod_rewrite. We will run into trouble. We'll find the debugger always hits the index controller, index action. To overcome this, we need to do the following.

Go to the Zend Framework application's project property in Netbeans. Go to the "Run Configuration" option. Click on Advanced. For "Debug URL", choose "Ask Every Time".

Now, if we want to debug the email action from the index controller, when we hit the debug icon in Netbeans, it'll pop up a dialog box asking us to type in the debug URL.

Once we Ok the dialog, Firefox will fire up (after you install the Netbeans debugger), and off you go!

The screenshots above are grabbed from my ZFWithDoctrine project on github.

That's it, and hope it helps.

File include cache in Zend Framework

2010-02-06T18:12:18-08:00

Another box of tissue gone ... runny nose syndrome continues ... still cannot type or think ...

Let me now blog something simple, but handy.

Problem ... Zend Framework uses many types of plugins, how do we efficiently tell the application where to find the plugin files? For example, when we use a view helper, should our application find it in the Zend library directory? Should our application find it in the application view helpers directory? Or should the application find it at our own custom library directory?

Fix ... it's easy.

Make the following extra line of config in your application config file

cache.classFileIncludeCache = APPLICATION_PATH "/../data/pluginLoaderCache.php"

Add the following code to your Bootstrap.php

/**
 * Setup include file cache to increase performance
 *
 * @return void
 * @author Jim Li
 */
protected function _initFileInlcudeCache()
{
    $classFileIncCacheOptions = $this->getOption('cache');
    $classFileIncCache = $classFileIncCacheOptions['classFileIncludeCache'];

    if(file_exists($classFileIncCache)) {
        include_once $classFileIncCache;
    }
    Zend_Loader_PluginLoader::setIncludeFileCache($classFileIncCache);
}

Attention: make sure the "/data/" directory is writable by your web server.

Reference

http://framework.zend.com/manual/en/zend.loader.pluginloader.html#zend.loader.pluginloader.performance.example

Zend Framework integrate with Symfony DI Container

2010-02-06T03:57:25-08:00

I want to integrate Symfony DI Container and Zend Framework together. It's been put on the back burner for a while. After trashed a box of tissue because of my runny nose today, I finally got something up. And let me share it here.

I'm not going to explain what dependency injection means, and how the Symfony DI Contain works here ... I'm actually not going to explain anything here today(I feel really crap at the moment) ... I'll find another time to get into the details.

For now, you can head to github, download the source code and have a play. Just make sure that the "/data/" directory is writable by your web server, hence it is used to store the generated DI service class file.

Download

http://github.com/marsbomber/zf-with-doctrine

Reference

http://blog.starreveld.com/2009/11/using-symfony-di-container-with.html

Beautify Zend Framework project error pages

2010-02-04T04:20:57-08:00

Thanks to Udo Schochtert (http://www.psiop.ch/), I become aware of this really nice Agavi style error handler. I basically followed http://www.iezzi.ch/archives/397 to get most of it going. However there are 2 little glitches I want to point out.

If you follow http://www.iezzi.ch/archives/397, most likely you'll forget, or not even aware of this (Thanks to Udo again, he pointed this out in the original blog's comments section). In the application.ini file, we need to add

resources.frontController.errorview = shiny

Since I'm beautifying the ErrorController and its pages in my zf-with-doctrine project (http://github.com/marsbomber/zf-with-doctrine). I ran into issues with displaying the ZFDebug menu on the error pages. What I had to do was simple. Go to the ErrorController.php, errorAction method, and add 2 lines to disable the ZFDebug plugin

// Disable ZFDebug plugin
$frontController = Zend_Controller_Front::getInstance();
$frontController-&gt;unregisterPlugin('ZFDebug_Controller_Plugin_Debug');

If interested, you can check my zf-with-doctrine project out from github, http://github.com/marsbomber/zf-with-doctrine

Hope you like it and find it's useful.

Download

http://github.com/marsbomber/zf-with-doctrine

Reference

http://www.iezzi.ch/archives/397

Setup phpUnderControl on Snow Leopard

2010-02-03T19:28:18-08:00

Once in a while, people do pointless things. So ... here I am. I setup phpUnderControl on Snow Leopard last night ... that's right. I know you probably will not ever run your central continuous integration service on a Mac ... but you know ... just in case. Having said this, after I installed phpUnderControl on my "LOCAL" machine, I think it does give me a good view of what's happening with me project, which is quite cool.

First of all, I use XAMPP stack on OS X for my development. XAMPP provides us some important modules to complete the whole puzzle. Here's what I did. I'm not sure if they are all necessary, but at least I know it worked for me.

With XAMPP, there are 2 important bits, the PHP and PEAR executable. I have the following

jimli@Jim-Lis-MacBook-Pro~: ll /usr/bin/pear
lrwxr-xr-x  1 root  wheel  39 29 Nov 08:45 /usr/bin/pear -> /Applications/XAMPP/xamppfiles/bin/pear
jimli@Jim-Lis-MacBook-Pro~: ll /usr/bin/php
lrwxr-xr-x  1 root  wheel  38 28 Nov 18:50 /usr/bin/php -> /Applications/XAMPP/xamppfiles/bin/php

First of all, let's install xdebug. Download the precompiled xdebug.so file from http://aspn.activestate.com/ASPN/Downloads/Komodo/RemoteDebugging

Open /Applications/XAMPP/xamppfiles/etc/php.ini in an editor, append the file with

[xdebug]
zend_extension=/Applications/XAMPP/xamppfiles/lib/php/php-5.3.0/extensions/no-debug-non-zts-20090626/xdebug.so
xdebug.remote_enable=on
xdebug.remote_handler=dbgp
xdebug.remote_host=localhost
xdebug.remote_port=9000
xdebug.file_link_format = "txmt://open?url=file://%f&line=%l"

Make sure the extension path is where you put the downloaded xdebug.so file to.

Now, let's install a few things

sudo pear channel-discover components.ez.no
sudo pear install -a ezc/Graph

Need to set preferred_state to alpha to get phpDocumentor for some reason.

sudo pear config-set preferred_state alpha
sudo pear channel-discover pear.phpunit.de
sudo pear install --alldeps phpunit/phpUnderControl
sudo pear config-set preferred_state stable

You can see, we did not install phpUnit, I believe phpUnit should be shipped with the latest XAMPP by default. Now, Download cruisecontrol here. Locate your download from a shell and do

sudo unzip cruisecontrol-bin-2.8.2.zip -d /opt
sudo ln -s /opt/cruisecontrol-bin-2.8.2 /opt/cruisecontrol
sudo phpuc install /opt/cruisecontrol
sudo phpuc example /opt/cruisecontrol

Ok. I also assume you have subversion and Java installed. Make sure you set the JAVA_HOME path in your environment. I did this by editing ~/.bashrc

JAVA_HOME="/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home";export JAVA_HOME

You may change the path accordingly base on your Java setup.

Now, edit /opt/cruisecontrol/config.xml, it should have something like

<?xml version="1.0"?>
<cruisecontrol>
  <project name="connectfour">
    <listeners>
      <currentbuildstatuslistener file="logs/${project.name}/status.txt"/>
    </listeners>
    <bootstrappers>
      <antbootstrapper anthome="apache-ant-1.7.0" buildfile="projects/${project.name}/build.xml" target="clean"/>
    </bootstrappers>
    <modificationset quietperiod="30">
      <!-- touch any file in connectfour project to trigger a build -->
      <filesystem folder="projects/${project.name}"/>
    </modificationset>
    <schedule interval="300">
      <ant anthome="apache-ant-1.7.0" buildfile="projects/${project.name}/build.xml"/>
    </schedule>
    <log>
      <merge dir="projects/${project.name}/target/test-results"/>
    </log>
    <publishers>
      <onsuccess>
        <artifactspublisher dest="artifacts/${project.name}" file="projects/${project.name}/target/${project.name}.jar"/>
      </onsuccess>
    </publishers>
  </project>
  <project name="php-under-control">
    <listeners>
      <currentbuildstatuslistener file="logs/${project.name}/status.txt"/>
    </listeners>
    <modificationset>
      <alwaysbuild/>
    </modificationset>
    <bootstrappers/>
    <schedule interval="300">
      <ant anthome="apache-ant-1.7.0" buildfile="projects/${project.name}/build.xml"/>
    </schedule>
    <log dir="logs/${project.name}">
      <merge dir="projects/${project.name}/build/logs/"/>
    </log>
    <publishers>
      <artifactspublisher dir="projects/${project.name}/build/api" dest="artifacts/${project.name}" subdirectory="api"/>
      <artifactspublisher dir="projects/${project.name}/build/coverage" dest="artifacts/${project.name}" subdirectory="coverage"/>      <execute command="phpcb --log projects/${project.name}/build/logs --source projects/${project.name}/source/src --output projects/${pro
ject.name}/build/php-code-browser"/>      <artifactspublisher dir="projects/${project.name}/build/php-code-browser" dest="artifacts/${project.name}" subdirectory="php-code-brow
ser"/>
      <execute command="/Applications/XAMPP/xamppfiles/bin/phpuc graph logs/${project.name} artifacts/${project.name}"/>
    </publishers>
  </project>
</cruisecontrol>

Now you can start and stop cruisecontrol by doing

cd /opt/cruisecontrol
sudo -E ./cruisecontrol.sh

kill `cat /opt/cruisecontrol/cc.pid`

After cruisecontrol is started, you can go to http://localhost:8080/cruisecontrol/ in your browser. Now everything should be good, except the metrics page. After some googling, I found this http://criticallog.thornet.net/?p=39 Basically you need to do

cd /Applications/XAMPP/xamppfiles/lib/php/pear/phpUnderControl/Graph/Input/
sudo mv ClassComplexityInput.php ClassComplexityInput.php.bak

Everything should now work!

Reference:

Doctrine tricks

2010-01-29T18:45:47-08:00

I have been using Doctrine ORM for a while now. Here I want to discuss 2 tricky issues I have encountered and resolved.

First, I'm paranoid of deleting records from the database. Things like registered users, I would rather have them marked as "DELETED", but still keep the record in the database, just in case ...

With Doctrine, the SoftDelete behaviour is perfect for this purpose. When a model is enabled with the SoftDelete behaviour, an extra field "deleted_at" is automatically added to the corresponding database table by default. So, when you call the delete() function on a model object, the corresponding data row in the database receives a timestamp on the "deleted_at" field. Data having values on the "deleted_at" field will not be returned by any find methods in the future.

It all sounds good, right? Not until you start trying to cascade "delete" on associations. Since we are not really deleting the record, DBMS level delete cascading will simply not work (In Doctrine YAML schema term, the "onDelete: CASCADE" will not work). Now, I'll illustrate the problem and the solution with a typical blog example.

In a blog application, we have 2 entities, article and comment. 1 article has many comments. When an article is "soft deleted", all the associated comments should go to. The schema YAML is something like this

Article:
  actAs: [Timestampable, SoftDelete]
  columns:
    article_content:
    type: string()
  relations:
    Comments:
      foreign: article_id
      type: many
      cascade: [delete]

Comment:
  actAs:[Timestampable, SoftDelete]
  columns:
    article_id:
      type: integer
    comment_content:
      type: string()
  relations:
    Article:
      foreign: id
      type: one

So, you can see, the trick is done on line 10 of the yaml file. With "cascade: [delete]" specified. You now can soft delete an article and all associated comments by doing something like this

$article = Doctrine_Core::getTable('Article')->findOneById(1);
$article->delete();

Now it works as expected.

One thing I need to point out is performance. Doing SoftDelete means Doctrine will do a lot separate update queries behind the scene (to set the deleted_at field). For models with a big association list, performance could degrade dramatically. I normally enable SoftDelete on all necessary models to start with. As the project continues, my understanding of the problem domain improves too, then I look at taking the SoftDelete off some unnecessary models. If some models do need to keep historical data, and performance is becoming a problem because of SoftDelete, I then look at other data archiving strategies so that we could fix up the bottle neck again by removing the SoftDelete behaviour on the offending models.

Let's move onto the next issue.

There are times that your application will need to obtain a model record, hold it memory, massage some attribute values, then save it back to the database. The issue raised is that after a record (we call this record A) is retrieved and before it's saved back, if another part of your application code retrieves the same record again. By default, Doctrine will flush record A and set all attributes back to its original values. This means any changes you make in memory on record A are lost. Let's use the article model again here to illustrate the issue in code.

$articleOne = Doctrine_Core::getTable('Article')->findOneById(1);
$articleOne->article_contents = "replace old with new";

$articleOneDup = Doctrine_Core::getTable('Article')->findOneById(1);

// you will see the article_contents will be reset back to its original value, changes are lost
var_dump($articleOne);

This is the default behaviour of Doctrine. If you ever want to change this behaviour, you could use something like this

Doctrine_Manager::getInstance()->setAttribute(Doctrine::ATTR_HYDRATE_OVERWRITE, false);

That's it. Hope someone find this useful.

Authentication service

2010-01-26T01:22:47-08:00

This is a followup to my previous post "ZF Authentication using Doctrine". After posting my implementation of using a ZendX doctrine auth adapter. I got some feedback from Jon Lebensold from ZendCasts. The issue raised is that my implementation makes the controller fat, which is considered as a bad practice. I cannot agree more on that. So ... I tried to refactor the code and come up with the introduction of a service layer class named "AuthenticationService".

Now the LoginController class is made thinner with the following code.

public function loginAction()
{
    $authService = new Model_AuthenticationService();
    if($authService->isAuthenticated() == true) {
        $this->_redirect('/index');
    }

    if($this->getRequest()->isPost()) {
        $this->view->headTitle('Login');
        // collect the data from the user
        $loginUsername = $this->getRequest()->getParam('username', '');
        $loginPassword = $this->getRequest()->getParam('password', '');

        $authResult = $authService->authenticate(
            $loginUsername,
            $loginPassword
        );

        if($authResult == true) {
            // auth good, do your logic
        }
        else {
            // auth bad, do your logic
        }
    } else {
        $this->_redirect('/login');
    }
}

Code for AuthenticationService.php

class Model_AuthenticationService
{
    private $_authenticationMessage = '';

    public function getAuthenticationMessage()
    {
        return $this->_authenticationMessage;
    }

    public function isAuthenticated()
    {
        return Zend_Auth::getInstance()->hasIdentity();
    }

    public function authenticate($username, $password)
    {
        $doctrineAuthAdapter = new ZendX_Doctrine_Auth_Adapter(
            Doctrine_core::getConnectionByTableName('Model_User')
        );
        $doctrineAuthAdapter->setTableName('Model_User u')
            ->setIdentityColumn('u.username')
            ->setCredentialColumn('u.password')
            ->setIdentity($username)
            ->setCredential(Model_Utility::encryptPassword($password));

        $myAuth = Zend_Auth::getInstance();
        $authResult = $myAuth->authenticate($doctrineAuthAdapter);
        if(!$authResult->isValid()) {
            $this->_authenticationMessage = 'You have entered invalid username and password.';
            return false;
        } else {
            $identity = $doctrineAuthAdapter->getResultRowObject(null, 'password');
            $myAuth->getStorage()->write($identity);
            return true;
        }
    }
}

As we can see, the AuthenticationService class is now doing most of the heavy lifting stuff. The controller is left with the duty of checking user login state and dispatching appropriate actions when necessary.

Hope this makes sense, and again more than happy to discuss.

Reference

ZF authentication using Doctrine

2010-01-23T14:27:21-08:00

Edit: There's a follow up article http://blog.elinkmedia.net.au/2010/01/26/auth-service/

As one of my everyday morning tasks, I just checked the rss feeds from my news reader. Interesting enough, one of my favourite blog authors, Jon Lebensold from ZendCasts, just published a new post on "Writing a Zend_Auth_Adapter with Doctrine".

The issue that we are trying to solve is to handle user login authentication from an application, which uses Zend Framework and Doctrine ORM. Jon took the approach of writing a custom zend auth adaptor to accomplish the task. I watched the video from ZendCasts, and clearly this is another quality video and it will just work for you.

However here, I want to show you an alternative approach to solve the same problem. I basically used an already written authentication doctrine adapter from the ZendX incubator. The class file can be found here.

To autoload the ZendX_Doctrine_Auth_Adapter class file, obviously you need to include the file in the correct directory inside the "library" directory. So that is "library/ZendX/Doctrine/Auth/Adapter.php".

Now let's look at some code. Please note that I am only showing topic-related code here, all other security checking, error checking code have been taken out.

class LoginController extends Zend_Controller_Action
{
    public function loginAction()
    {
        // collect the data from the user
        $loginUsername = $this->getRequest()->getParam('username', '');
        $loginPassword = $this->getRequest()->getParam('password', '');

        $myAuth = Zend_Auth::getInstance();

        // do the authentication
        $authAdapter = $this->_getAuthAdapter($loginUsername, $loginPassword);
        $result = $myAuth->authenticate($authAdapter);
        if (!$result->isValid()) {
            // take care of invalid login
        } else {
            // empty password
            $identity = $authAdapter->getResultRowObject(null, 'password');
            $myAuth->getStorage()->write($identity);

            // all good, do you redirect or whatever
        }
    }

    protected function _getAuthAdapter($username, $password)
    {
        $authAdapter = new ZendX_Doctrine_Auth_Adapter(Doctrine::getConnectionByTableName('Model_User'));

        $encryptedPassword = Model_Utility::encryptPassword($password);

        $authAdapter->setTableName('Model_User u')
            ->setIdentityColumn('u.username')
            ->setCredentialColumn('u.password')
            ->setIdentity($username)
            ->setCredential($encryptedPassword);

        return $authAdapter;
    }

Ok. Most of the code should be pretty self explanatory. The authentication is done with the following assumptions.

Your login users are stored in a table called "user", with at least 2 columns, "username" and "password".
You have your Doctrine "Model_User" class setup.
If you store encrypted or hashed passwords in your database (I do), the encryption logic is handled by a static method call from a utility class, "Model_Utility".

This is it. In our case, we do not need to create an authentication method in "Model_User" as Jon showed us in his video tutorial. The authentication magic is all done by the "ZendX_Doctrine_Auth_Adapter" class.

Hope you find this useful and let me know if you have issues or suggestions.

Reference

http://www.zendcasts.com/writing-a-zend_auth_adapter-with-doctrine/2010/01/

New year's resolution

2010-01-06T02:20:01-08:00

It's 2010 and it's again time to set goals. It's important that the goals are realistic and achievable. So ... here they are

Take a get-away holiday. It's been almost 4 years since my last proper holiday. My wife deserves better ...
Cutback on freelancing and spend more time with family, especially my son, Oscar.
Do something with the house please ... paint the doors, replace the windows, fix up the deck and more importantly let's mow the bloody lawn.
Make one hobby mobile application using "Titanium".
Learn Ruby on Rails by writing one side project.
Add a portfolio page to the ElinkMedia site.

It really sounds like a relaxing year from the programming side. That's right! Family is this year's focus.

I'll keep updating this post and hopefully I can tick off all 6 items at the end of year 2010.

Happy new year!

ZF, Doctrine and Unit Tests

2009-12-03T02:44:13-08:00

Recently there are a lot of very interesting blog posts about using Zend Framework with the Doctrine ORM. I am really glad to see more and more people start talking about application design patterns, test driven development, etc. This trend means the PHP community is preparing to take PHP into the next level (enterprise? no?) . As a PHP developer, I see a lot of things to learn.

Back to the topic. I've been testing to use both Zend Framework and Doctrine for a while now. As a matter of fact, I never really have them hook up perfectly. The issue I faced was about the Doctrine command line tool. I just cannot manage to bootstrap the Doctrine command line script correctly. I can get Doctrine command line script to drop database, create database, generate models in PEAR style (which is used by the Zend Framework) from the yaml schema file. But I cannot get it to create tables, generate SQL, etc.

This did not seem to be a real issue until I started wondering about unit test my Doctrine model classes. Ideally, when I do my unit tests, I can re-create my testing database, reload my data fixture file. So ... I started digging.

Eric Leclerc made a really helpful blog post recently about setting up Zend Framework and Doctrine. Even better, Eric kindly created a sample application on github. I downloaded it, had a play, worked really well!

Next is unit testing. Lucky we still have Jon Lebensold, the creator of ZendCasts. Jon just made another great podcast on "Writing Doctrine Unit Tests with Zend_Test" yesterday. I downloaded the source code from Jon's episode. I started merging it with Eric's sample application.

After some tweaks and changes. I believe I finally have a working copy. That is Zend Framework + Doctrine + ZFDebug + Zend_Test. All in one project (no need to create models in a model generator project with Doctrine, copy them across to my ZF application).

I have pushed my sample application to github to share with others who are interested in using Doctrine with Zend Framework. You can find the application at http://github.com/marsbomber/zf-with-doctrine.

Hope it helps. And also, my sincere thanks to Eric Leclerc and Jon Lebensold.

Reference

Titanium development platform

2009-11-14T22:58:56-08:00

There are occasions that desktop application become required as part of a project. To me, as a PHP application developer, this is almost impossible. PHP is a scripting language used by the web. Desktop application and PHP almost do not link at all.

Until recently I came across the Titanium solution. It bridges the gap between PHP and the native OS platform. With Titanium, I can enjoy building desktop, even mobile, applications using the technology that I am comfortable with. That is build desktop and mobile applications with PHP, HTML, CSS and Javascript. Sounds familiar? That's right! We can now utilise the whole web development technology to build desktop apps. Just like Adobe Air, minus the learning curve of flash, actionscripts, etc. Having said this, PHP just became supported by Titanium. So things are just shaping out, but I believe we should have a very workable platform once Titanium comes out of its beta.

The lack of PHP support might not be an issue if we think about application design. The desktop application we build might just be a thin client, which acts as a browser replacement. It might be a good practice to actually keep all the complex domain logic on the web/application server. We could then use the SOA approach to make thin client functional by calling exposed web service methods by the server.

Anyway, we should definitely keeping an eye on this one! http://www.appcelerator.com/

What's next? I'm having a break from my freelancing works at the moment. I'm actually waiting for my Ruby on Rails book, Agile Web Development with Rails, to arrive (Right! I'm self-learning RoR). Amazon seems to be running out of stock on this particular one ... Luck that I borrowed a book from a good friend of mine last night. It'll keep me busy for a while, and my friend has got another book that I'll borrow and read (Currently have "Domain-Driven Design" by Eric Evans, and "Patterns of Enterprise Application Architecture" by Martin Fowler is on the list).

Hopefully I should be able to talk something about the "Domain-Driven Design" book in my next post. Keep tuned!!!

Reference

http://www.appcelerator.com/

Application logging with Zend_Log

2009-10-22T18:20:34-07:00

It's been a while since my last post here... I've been pretty busy with projects and other things. However I did manage to finish the book I recently bought, Zend Enterprise PHP Patterns. It's an interesting book. Even the book title mentions PHP Patterns, it's actually more about good practice with Zend related technologies. It's a shame that not every PHP developer will have the chance to access those Zend products (me included ...), however I did manage to get some interesting ideas out of the book. To name one, application logging. I have to say, I have over looked the aspect of application level logging for a long time. I so rely on the server access and error log, and there are instances that I fail to find necessary information from those big ... massive server log files.

So here's how I did my application logging with Zend_Log.

In my Bootstrap.php, I added the following init method. You can see the logger object is then stored in the registry. You can see I'm not doing anything fancy here, just logging to a file, which is specified in my ini file.

protected  function _initApplicationLogger(){
  // init firebug log writer
  $writerFirebug = new Zend_Log_Writer_Firebug();
  $filterFirebug = new Zend_Log_Filter_Priority(Zend_Log::DEBUG, '==');
  $writerFirebug->addFilter($filterFirebug);

  // log file
  $logFile = Zend_Registry::get('config')->logFiles;

  // application log writer
  $writerFileApplication = new Zend_Log_Writer_Stream($logFile->application);
  $filterFileApplication = new Zend_Log_Filter_Priority(Zend_Log::INFO, '&lt;=');
  $writerFileApplication->addFilter($filterFileApplication);

  // error log writer
  $writerFileApplicationError = new Zend_Log_Writer_Stream($logFile->error);
  $filterFileApplicationError = new Zend_Log_Filter_Priority(Zend_Log::ERR, '==');
  $writerFileApplicationError->addFilter($filterFileApplicationError);

  // security log writer
  $writerFileApplicationSecurity = new Zend_Log_Writer_Stream($logFile->security);
  $filterFileApplicationSecurity = new Zend_Log_Filter_Priority(Zend_Log::ALERT, '==');
  $writerFileApplicationSecurity->addFilter($filterFileApplicationSecurity);

  // add writers to logger
  $logger = new Zend_log($writerFirebug);
  $logger->addWriter($writerFileApplication);
  $logger->addWriter($writerFileApplicationError);
  $logger->addWriter($writerFileApplicationSecurity);

  Zend_Registry::set("logger" , $logger);
}

Next, I want to log my application errors, so the natural place to do this is the ErrorController.php file. I just added the following lines to the bottom of the errorAction() method

$this->logger->log($errors->exception->getMessage() . PHP_EOL . $errors->exception->getTraceAsString(), Zend_Log::ERR);

Since the application I'm writing is accessible by authenticated users only, it is important to log security breaches. How I did this is a bit interesting. I think there must be better ways to do this ... but I don't know how ... so here it is.

I have implemented my own controller action class, MyApp_Controller_Action. It extends the standard Zend_Controller_Action. So all my protected controllers extend MyApp_Controller_Action instead of Zend_Controller_Action.

class MyApp_Controller_Action extends Zend_Controller_Action {
  ...
  protected $logger;
  public function init() {
    ...
    $this->logger = Zend_Registry::get("logger");
  }

  ...

  public function checkAcl() {
    ...
    if($noGood) {
      $this->logSecurityBreach();
    }
  }

  protected function logSecurityBreach() {
    // log some basic info here. extend it to include extra info that you need
    $errorLog = 'Invalid Access Attempt' . PHP_EOL .
      'Resource: ' . $this->getRequest()->getRequestUri() . PHP_EOL .
      'Offending IP: ' . $this->getRequest()->getClientIp() . PHP_EOL .
      'Offending user: ' . $this->identity->Username;
      $this->logger->log($errorLog, Zend_Log::ALERT);
  }
}

Since the logSecurityBreach function is made "protected", so that in the controllers, who extend MyApp_Controller_Action, can call the logSecurityBreach method whenever there are any other access control violations. For example, a specific security error can be logged if a logged in user tweaked the URL (like an ID param/token in the URL) to access some other resources that should not be visible to his login.

So here it is. A basic use of Zend_Log. Hope it makes sense.

Reference

http://www.apress.com/book/view/9781430219743

Best practices of PHP development

2009-09-19T06:52:40-07:00

I just finished listening to this ZendCon podcast. It's about some good practice tips in doing PHP development. They are quite basic, but I'm sure that not all PHP developers follow their suggestions ... At least I know I'm still some way apart from the suggested "Best Practices".

People, who are interested, can find it here. To save you some time, the companion PDF slides can also be downloaded here.

Reference

http://www.phppodcasts.com/2009/09/18/the-zendcon-sessions-episode-26-best-practices-of-php-development/

GIT on Windows with Netbeans

2009-09-16T23:58:21-07:00

I've been using SVN (via Tortoise SVN) on my windows development machine for a while. The experience is quite average. It does what I need, but I still had to play with my local Apache configuration and the speed is not that impressive. GIT is apparently becoming the industry version control program these days, so I decided to give it a try!

To set it up on Windows, it's actually quite simple. You first need to download "msysgit", available from Google Code. Follow the instruction to install it. Since we are using Windows, to play safe, I chose the installation location to be at "C://GIT", so that we will not potentially run into the infamous "C://Programs Files" issue.

So ... after installation, we are pretty much good to go. Fire up the newly installed Git Bash, you then pretty much work in a "linux-like" shell, with only git of course :)

I'm using Netbeans for my development and am naturally lazy, so that I thought it'll be better if I can do my version control stuff inside the IDE. I got lucky again. Download "NBGit" from Google Code (wonder why the downloads are not from GitHub ...), and load it up with Netbeans. All good.

I then ask, how do I push the source code to a remote location/computer, so that if my local computer crashes, I can still have my version controlled copies of everything?

If you don't mind others to see and grab your code, go to GitHub. If you don't mind spending some dollars every month to be able to host private projects and you want to access the files from everywhere , go to GitHub. Otherwise read on ...

If you have a spare machine sitting at home and not doing much, you then might want to load it up with your favorite Linux OS. Install git-core, and SSH server. All you need to do now is to setup a few empty GIT repositories on the Linux machine, note down the Linux machine's IP and your empty GIT repository locations. You have a remote GIT server. I'm not going to describe how to do each of these tasks in details, since there are a lot good tutorials on the net already.

Anyway, that's the setup I use, and I hope you may find it's useful.

Common library files organization in ZF

2009-09-05T01:07:29-07:00

Once you have used ZF for a couple of projects, you will find that there are library files actually live independent to whichever application you are implementing. These common library files should be reusable by all you ZF projects. Now the question is where you should place these files so that a typical ZF project can make use of them without a glitch?

I have described my ZF project structure in an earlier post HERE, and now I want to elaborate a bit more on folder "library/MyApp", because that's where I keep my common library files.

I have a rather simple version shown here. Basically only 3 things to look at.

Let's start with "MyApp/View/Helper".

This is the place where I keep all my project independent View Helper files. Things like

baseUrl, as the name says, return the base URL for your application
loginUserInfo, assemble things like login username, last login time, and even your application logout text/link
flashMessenger, I use a customized view helper in conjunction with ZF's flash messenger action helper to make different markups for different type of error messages. i.e. good message in green, bad message in red.
etc

I assume you already know that you need to follow ZF's naming conversion for these helpers files to make autoloading painless. For example, the baseUrl view helper, you will end up with something like this

class MyApp_View_Helper_BaseUrl {
  function baseUrl() {
    $fc = Zend_Controller_Front::getInstance();
    return $this-&gt;_baseUrl =  $fc-&gt;getBaseUrl();
  }
}

Next, let's look at ViewSetup.php file under "MyApp/Controller/Plugin"

I basically have this file built based on Rob Allen's book "Zend Framework in Action". This is the file that sets up your view using various view related hooks. Particularly I have 3 hooks looked after.

dispatchLoopStartup Use this hook to setup things like doctype, application common JS and CSS file. And also register our MyAPP view helpers
preDispatch I personally use this hook to inject more CSS and/or JS into the script header. Because at this point, we know the view is dispatched by which controller. We could then load in any controller specific resources.
postDispatch I use this hook to set my application HTML head title. ZF allows you to set a separator for your header title tokens. And I always add 1 more token, which is the application name to the title. So ... your paper header title can look something like this "ActionName - ControllerName - ApplicationName". It somewhat makes sense to me, but if you do not like the idea, just do not implement this hook.

To make a use of this ViewSetup file, I have this too init function in my bookstrap

protected function _initCustomLibraryNamespaces() {
  $loader = Zend_Loader_Autoloader::getInstance();
  $loader->registerNamespace('MyApp_');
}

protected function _initViewHelpers() {
  $frontController = Zend_Controller_Front::getInstance();

  $frontController-&gt;registerPlugin(new MyApp_Controller_Plugin_ViewSetup(), 98);
  $this-&gt;bootstrap('layout');
}

OK, last bit, let's talk about "MyApp/Controller/Action.php"

A bit background first, I recently just started looking into Ruby on Rails as a self learning course. One RoR programming principle is DRY (Don't Repeat Yourself). The same motivation made me implement this "MyApp/ControllerAction.php" file. There are a lot of common things you do with pretty much all your application controller classes. Things like authentication, maybe ACL (depends on your application specific requirement), flash messenger, logger, etc. I hate repeat myself, that's why I have these common tasks or methods implemented here. So ... the Action.php class declaration goes like this

class MyApp_Controller_Action extends Zend_Controller_Action

It actually extends the Zend_Controller_Action, so that it should not work any unexpected at all. and a typical application controller will have something like this

class IndexController extends MyApp_Controller_Action

and of course if a certain controller does NOT require any extras implemented in the customized Action.php file, you can make it easily fall back to something like this

class ErrorController extends Zend_Controller_Action

That's it! Hope it helps and let me know if you have better ways.

jQuery library and plugins

2009-09-03T06:51:47-07:00

No need to mention, most of the web developers have long suffered from the chaotic but must-have JavaScript hand coding days ... I was "lucky" enough to learn JavaScript the hard way from my first job. Traversing the DOM was like a nightmare, repeating the same form validation 20 times, running into browser compatibility issues (tell me about it!!!), etc.

Tasks are made easy now with quite a number of JavaScript frameworks. I started with YUI, and now try to do more with jQuery. What I found with YUI is that I still need to write quite some code to do a task. On the other hand, with jQuery, I tend to write very little. I think this is because of the number of quality jQuery plug-ins. That's the same story with Firefox and its Addons.

Anyway, in most of my recent applications, I pretty much have the following plugins loaded in one minified .js file

cmxform
ajaxupload
jquery.autocomplete
jquery.metadata
jquery-validate
jquery-tablesorter

and of course the jQuery UI js combined with any necessary widgets.

About robots.txt

2009-08-31T17:52:21-07:00

A while ago, I was asked the question of how we can tell search engine spiders how and where to search and index our site. We know the answer is robots.txt. Placing this special file under your site root will do the trick. I've been handcrafting the robots.txt file since I know its existence until I recently found there are generators out there can help you with the job ... silly me!

To save you some time, http://www.mcanerin.com/en/search-engine/robots-txt.asp is what I use now.

Connect to MS SQL Server using PDO

2009-08-29T23:06:55-07:00

I am not sure if having PHP talking to SQL Server is a common practice, but I have to do it recently with a new Zend Framework based project. Using the Zend Framework means I will not use my good old pear MDB2 database abstraction mechanism to make my PHP application talk to SQL Server. PDO is used to handle this task. And of course, as soon as you move out of your comfort zone, you have problems.

My local server, hosted on XAMPP, cannot establish connection to my local 2005 SQL Server. After spending hours on this problem, I found it is caused by a DLL file shipped with the pre-compiled PHP was out dated. The updated ntwdblib.dll file can be downloaded from http://www.dlldll.com/ntwdblib.dll_download.html or http://www.userscape.com/ntwdblib.dll

For a XAMPP installation, you will need to paste the DLL file in "C:\xampp\php" and "C:\xampp\apache\bin". After this, everything should be good to go!

Setup local PHP development environment on Windows

2009-08-29T01:00:40-07:00

Here I want to show you how I have my local php development environment setup with XAMPP, Xdebug and PHPUnit. With this setup, you will be able to develop and run your PHP applications on your windows localhost, be able to unit test your PHP application and have a code coverage report generated by PHPUnit.

Let's download and install XAMPP first.

This is very straight forward. We just need to download the latest XAMPP zip file from http://www.apachefriends.org/en/xampp-windows.html and extract all contents to "C:\xampp". I use "C:\xampp" to avoid windows permission issues. To verify the XAMPP installation, find and double click on the "C:\xampp</em>xampp-control.exe" file to bring up the control panel, click on the "Start" button next to "Apache" to start the Apache web server. Now, start your browser and go to http://localhost, you should see the XAMPP default welcome page. Your localhost document root is at "C:\xampp\htdocs\".

Next, let's install and configure Xdebug.

Download the latest precompiled Xdebug DLL from http://www.xdebug.org/download.php. Put the DLL inside "C:\xampp\php\ext\".

We need to tell our PHP installation to use Xdebug. There are 2 php.ini files come with XAMPP, "C:\xampp\php\php.ini" and "C:\xampp\apache\bin\php.ini". The file under "C:\xampp\" controls the command line PHP. Obviously the php.ini under "C:\xampp\apache\bin\" controls the Apache PHP module. For some reason, enabling Xdebug for the Apache PHP module will lead to unexpected Apache server crashes. So I only have Xdebug enabled for my command line PHP. 2 reasons for this. One is that I really do not have time to figure out why it's not working for my Apache PHP module. Second is my motivation of installing Xdebug at the first place. I'm using Zend Framework for my PHP project development, I get a comprehensive error stack trace from the framework already. I want to install Xdebug, so that when I run my PHP unit tests, I can let PHPUnit generate a nice looking code coverage report (Xdebug is required for this job).

So ... to configure Xdebug for my command line PHP, I need to modify "C:\xampp\php\php.ini". In the ini file, find sections for "[Zend]", comment out all lines under it. Find section "[XDebug]", comment out all lines, and put in this

[XDebug] xdebug.remote_enable=1 xdebug.remote_host="localhost" xdebug.remote_port=9000 xdebug.remote_handler="dbgp" zend_extension_ts="C:\xampp\php\ext\php_xdebug-2.0.5-5.2.dll"

Okay. we now can move onto PHPUnit

We can download and install PHPUnit with XAMPP's pear support. Open windows command prompt as administrator and do the following

cd c:\xampp\php go-pear.bat PEAR_ENV.reg pear channel-discover pear.phpunit.de pear install phpunit/PHPUnit

We should now have PHPUnit download and configured at "C:\xampp\php\PEAR\PHPUnit\".

Conclusion

We now have a working local copy of PHP, MySQL, Xdebug and PHPUnit. With this setup, we are ready to develop and test our applications on the localhost.

My Zend Framework project setup

2009-08-27T02:57:40-07:00

When starting implementing a new project, I want to start working on a basic code base, so that I can jump in the project specific coding. I believe I'm not alone, and that's why people fall in love with the rapid development scaffolding idea from Symfony, RoR, etc. I personally prefer a different path. I try to make a clean, generic project skeleton to cover the common ground. Building this skeleton project itself is a very valuable learning process. It makes you learn every single step you do, as opposed to the scaffolding approach, which does things for you without needing you to understand it.

Here is what I have done with learning and working on Zend Framework.

When I first started using Zend Framework, I was simply following the quick start guide and making it into my own project skeleton. I then start building my project from the skeleton project.

I then bought and read the book, Zend Framework in Action, by Rob Allen (It's a great book on Zend Framework even the sample code throughout the book have errors and mistakes and sometimes the topics from chapter to chapter do not follow very smoothly. It is still a very good book for developers just got across the ZF newbie mark). From the book, I adapted Rob's project structure and a lot of good ideas. I, of course, updated my home made ZF skeleton project.

From there on, I constantly learn new ideas from the ZF blogs and posts, hence keep enhancing my skeleton project. I want to share my skeleton project structure here, and hope it might help ZF adopters have a shortened learning curve.

It's very basic and very similar to the file structure generated by Zend_Tool.

I have 4 top level folders

admin. The "admin" folder is my project web root. Instead of "admin", it can be rename to anything.

application. The "application" folder holds the project core.

library. The library folder contains a local copy of Zend Framework, and also my own library files under sub folder "MyApp".

tests. Folder "tests" holds my unit testing files and the PHPUnit generated code coverage report

I deliberately expanded the "application" - "models" folder. Under "models", I have a sub folder "Db" and other PHP model files. Reason is I am using a hybrid approach here to handle database interactions.

Sub folder "Db" contains domain objects implementing "Table Data Gateway" and "Row Data Gateway" design patterns using the Zend Framework shipped Zend_Db support.

Files on the same level as folder "Db" are domain classes that typically require resources from many table. These "shortcut" classes can be used under the situation where using the domain classes under the "Db" will impose too much of a performance hit. Using these "shortcut" classes means we are sacrificing abstraction to gain performance. These classes hence should be implemented and used carefully.

I am also looking at implementing Doctrine together with Zend Framework in my skeleton project. I'll post an updated diagram in a future post.

MVC & Zend Framework

2009-08-26T18:26:24-07:00

As high speed internet becomes available to more and more people, the computer world gets its focus back to the centralised computing style. This opens up opportunities to web development companies. They are no longer just building "another website". More complicated web applications get in the radar. The increased complexity in developing web applications requires web development methodology change, as well as web developers' programming mindset change.

Programmers can no longer hack their way through, for a decent sized project anyway. A typical web application project will take longer time to develop and require a lot more resources to complete the work. And thanks to our high speed internet again, these people resources can be everywhere around the globe. A clean and structured way of developing projects becomes required. For the sake of not reinventing the wheels, having a jump start and working on a robust code base, a web development framework(s) is normally used.

The MVC pattern becomes a de facto to web applications these days. I'm not saying MVC is the only pick. We can still implement our project in other styles just as efficient and effective. MVC is however commonly adapted and used in the community (the PHP community at least). If you are not sure about what MVC is, you can have a look on Wikipedia or go HERE for a PHP specific explanation.

The guidance of MVC should lead us to have a clean and purposely build project structure, which should be self-explanatory in terms of what class does what.

To take the MVC approach further, we will then naturally start hunting for a well developed, enhanced and maintained framework. There are quite a number of PHP framework libraries out there. They all have pros and cons and I'll leave that to your own research.

I have used the Zend Framework for over a year and a half now. The reason I chose Zend over the others is because of its "use at will" characteristics. This means for those adhoc and small through-away jobs, I can still benefit from using Zend Framework's robust base classes as my external libraries, without the constraints of having to follow the MVC pattern. This means you are never forced to over engineer projects. Let's face it, there are always jobs that you never need a full blown MVC design. Determine a project implementation strategy is a separate issue to this post. Hopefully we can do another post on that later.

Apart from the "use at will" flexibility, we have to mention the robust code base that Zend and the PHP community provides. Zend Framework gives us a jump start on any MVC styled projects. We can use Zend_Tool to generate a basic Zend Framework recommended project file structure. However we still have the flexibility to alter and change the structure to make it fit in our company's convention, as long as it makes sense to the developers. From there on, any developers who are familiar with Zend Framework should be able to jump in and start working.

Enterprise PHP

2009-08-24T06:55:49-07:00

As my first blog post at the elink media blog, I want to start with a discussion on this "Enterprise PHP" topic.

Doing a search on Google, the most interesting posts are posted back in 2004 and 2005. You will find people taking about the strength and weaknesses of PHP, comparing with JAVA of course (enterprise de facto? pretty much).

This post is particularly interesting, because of the writing itself and one of the user comments. Martin Fowler's enterprise application definition was quoted and used as a measurement to check if PHP is enterprise ready. According to Martin, PHP should be nowhere near his enterprise grade. However I found one of the replies (replied in 2004) actually matches what I always believe.

PHP will not fit in Martin Fowler's enterprise definition back in 2004, the statement is made and based on the enterprise definition defined long time ago. We know that IT moves world faster than ever. And I would say, web moves IT faster than ever too. This makes me question whether Martin Fowler's definition is still correct. Or has the definition of enterprise itself evolved over the past decade?

Having said that, I am a fan of Martin Fowler's enterprise design patterns. It helped me a lot with my jobs. If you've never come across him, I strongly recommend you do a Google search and read about his design patterns. Not saying you need to follow it, but at least you should be able to get some good ideas out of it.

Let's leave this as an open discussion. I'll discuss this further after I finish reading the "Zend Enterprise PHP Patterns" book I just bought.

Happy blogging

2009-08-23T10:58:29-07:00

Welcome to the elink media blog. This is a technical blog to highlight and discuss any technical aspects that elink media is focusing on. Some tips and tricks or simply some useful links will also be shared to all audiences.