Ruby 1.9.3 uses a more restrict YAML parser, Psych, instead of the old Syck parser. This caused me issues when using Settingslogic. Even doing what suggested by Settingslogic to set the YAML parser back to Syck didn't help.
A simple test script
Running it under Ruby 1.9.3 outputs this
Using: psych
{"defaults"=>{"cool"=>{"bang"=>"wow", "fruit"=>"apple"}}, "development"=>{"cool"=>{"fruit"=>"banana"}}}
Using: syck
{"defaults"=>{"cool"=>{"bang"=>"wow", "fruit"=>"apple"}}, "development"=>{"cool"=>{"fruit"=>"banana"}}}
You could see "development" lost its "bang" hash key.
There are ways to get around the issue by re-building Ruby with libyaml flag. I consider this as harmful than helpful. So I reverted my application.yml back to its very dumb form, no default options, no merging, just duplicate all settings for all environments. It's definitely stupid to do so, but for now, it's considered as a temporary workaround.
I explained how I setup Ruote and RabbitMQ in last post. From there on, you basically already have a working system. All there's left is to have external services subscribe to the RabbitMQ job queues, parse the workitem JSON, do whatever and post the modified/amended JSON back to RabbitMQ, default queue is named "ruote_workitems", which is what the RuoteAMQP receiver listens on.
In this post, I'll demonstrate how a DaemonKit generated daemon script consumes the RabbitMQ job messages and how we construct a DaemonKit compatible Ruote process definition.
Generate DaemonKit project
rvm 1.9.3@daemonkit_ruote
gem install daemon-kit
daemon-kit daemon -i ruote
cd ruote
bundle
Make daemon listen to correct AMQP queues by editing ruote.yml config
defaults: &defaults
amqp:
queues:
- ldap_job
- email_job
Edit participant class by adding the following 2 methods
def ldap
workitem["ldap"] = "done"
# just to demonstrate the AMQP message carries the ID fields and it can be consumed
workitem["original_cust_id"] = workitem["cust_id"]
end
def email
workitem["email"] = "done"
end
Let's start everything up
rabbitmq-server
cd /my-ruotekit-enabled-rails-project && rails s
RAKE_TASK=true rake ruote:run_worker
cd /my-daemonkit-ruote-project && bin/daemon
Now let's work out how we wire everything up by carefully crafting a process definition.
Go to http://localhost:3000/_ruote/processes/new in your browser and enter the following as process definition
Ruote.process_definition :name => 'suspend_account', :revision => '0.1' do
sequence do
ldap :command => "/sample/ldap"
# slot in editor to check ldap process result status
email :command => "/sample/email"
# slot in editor to check email process result status
notifier :forget => true # this could be email or final results back in notification queue
end
end
Enter the following into the workitem fields box
Then launch it! If everything goes well(it should!), you'll see no processes listed under the ruotekit app interface. And you'll have 4 queues listed under RabbitMQ's management interface.
- email_job, 0 job
- ldap_job, 0 job
- notify_job, 1 job
- ruote_workitems, 0 job
By popping the message out of the notify_job queue, you'll see the serialised JSON string contains ldap done and email done key-value pairs. It shows me the job is done, and all participants are exercised correctly.
Now, we go back a step and explain a little on the process definition we crafted.
If a remote participant will be processed by a DaemonKit daemon script, we need to give each of those participants a hash, which contains a :command key. The value for :command key needs to follow a certain convention too. The convention is "/participant_class_name/method_name". DaemonKit will classify and constantize "participant_class_name" in to ParticipantClassName object, and "method_name" will be sent to it. So in our example, when the daemon script sees command "/sample/ldap", it'll invoke a call to "Sample#ldap".
The value we entered into the workitem fields textbox is simply be merged into the initial workitem hash, and it'll be sent along to all participants.
That is it. Hope it all makes sense, and please tweet me if you found I've got any bits laid out wrongly or explained wrongly.
At MYOB, I am assigned the task to build a component based workflow application. There are several requirements this application has to address. Here are a few
- Decoupled components
- Resilient workflow
- Configurable workflow
- Every single piece of the application needs to be scalable
A messaging based system becomes somewhat a natural choice to address the decoupling, distributable and scalable issue. While in the hunt of a good workflow system, I was pointed to a Ruby gem called Ruote by @jmettraux. Ruote has also got several companion gems. One of them is ruote-amqp.
Rails Sample Ruote AMQP is the most complete example I could find on Github. This excellent sample project almost covers all aspect. However its Rails default README file doesn't do its justice. I had to dig hard to figure out how all the pieces are put together.
Enough mumbling from me. The following are the steps I implemented to get an end-to-end Ruote + RabbitMQ going on my MBP. As prerequisites, I have Homebrew and RVM installed.
Install RabbitMQ
This is rather simple with the help from Homebrew.
brew install rabbitmq
rabbitmq-plugins enable rabbitmq_management
rabbitmq-server
Now you can monitor your running RabbitMQ instance from the browser at http://127.0.0.1:55672/mgmt/ (login using guest - guest)
Setup RuoteKit enabled Rails project
Create a new rails app called lcp
Step in the project root, create .rvmrc
Run bundle install after amending Gemfile with
gem "ruote", "~> 2.2.0"
gem "ruote-kit", "~> 2.2.0.3"
gem "ruote-amqp", "~> 2.2.0"
Add ruote kit initializer. As you could see, the file shown is a work in progress.
# config/initializers/ruote-kit.rb
# TODO: move setting values into a config file
AMQP.settings[:host] = '127.0.0.1'
#AMQP.settings[:vhost] = '/'
#AMQP.settings[:user] = 'guest'
#AMQP.settings[:pass] = 'guest'
# run ruote engine without worker
# run ruote rake task for workers, rake ruote:run_worker
RuoteKit.engine = Ruote::Engine.new(RUOTE_STORAGE)
if ENV["RAKE_TASK"]
RuoteAMQP::Receiver.new(RuoteKit.engine)
else # don't register participants in rake tasks
RuoteKit.engine.register do
participant :ldap, RuoteAMQP::ParticipantProxy, :queue => "ldap_job"
participant :email, RuoteAMQP::ParticipantProxy, :queue => "email_job"
participant :notifier, RuoteAMQP::ParticipantProxy, :queue => "notify_job", :forget => true
#participant :editor
# register the catchall storage participant named '.+'
catchall
end
end
# when true, the engine will be very noisy (stdout)
RuoteKit.engine.context.logger.noisy = false
Add ruote worker rake task
# lib/tasks/ruote.rake
namespace :ruote do
desc "Run a worker thread for ruote"
task :run_worker => :environment do
RuoteKit.run_worker(RUOTE_STORAGE)
end
end
Last not least, mount RuoteKit to our Rails app by adding the following to the routes file
# ruote-kit
match '/_ruote' => RuoteKit::Application
match '/_ruote/*path' => RuoteKit::Application
This completes first half of the story. We now have a running RabbitMQ instance with admin monitoring, and a RuoteKit enabled Rails app. The Rails app has several RuoteAMQP participants registered, a ruote rake task ready to run Ruote workers, and of course the RuoteKit rack app that allows us to manually launch Ruote jobs.
In the next post, I'll complete the full picture by implementing a DaemonKit generated Ruote daemon script, which processes the AMQP jobs published by the ruote worker running on the Rails app side.
But for now, PEACE!
Here are some basic Ruby array methods that I should learn to use.
& and | Array intersection and union
a = [1, 2, 3]
b = [2, 3, 4]
a & b #=> [2, 3]
a | b #=> [1, 2, 3, 4]
collect and select
a = [1, 2, 3, 4]
a.collect(&:odd?) #=> [true, false, true, false]
a.select(&:odd?) #=> [1, 3]
delete_if and reject
a = [1, 2, 3, 4]
a.reject(&:odd?) #=> [2, 4]
a #=> [1, 2, 3, 4]
a.delete_if(&:odd?) #=> [2, 4]
a #=> [2, 4]
push, pop, shift and unshift
a = [1, 2, 3, 4]
a.push 5 #=> [1, 2, 3, 4, 5]
a #=> [1, 2, 3, 4, 5]
a.pop #=> 5
a #=> [1, 2, 3, 4]
a.unshift 0 #=> [0, 1, 2, 3, 4]
a #=> [0, 1, 2, 3, 4]
a.shift #=> 0
a #=> [1, 2, 3, 4]
One night, I was studying some code written by Aleksey. I spotted his usage of Ruby string's "%" notation. It shows me how elegant Ruby can be, given knowing all those Ruby basics.
To confess, I learned (and still learning) Ruby from learning Rails. At this moment, I defintely think it's a bad idea! Basics should always be taught/learned before the magical frameworks. So, I'll start a series of posts to go through some of these basics.
Ruby string % notation
%Q - Interpolated string. Character used after %Q will be used as the new string delimeter, hence it should be balanced. This makes normal string delimeters such as " and ' become auto-escaped. If the new delimeter also appears in the string, it only needs to be manually escaped if it's unbalanced.
%Q[Don't need to escape " and '. Need to escape \[ manually, but not [ and ]]
%r - Similar to %Q, but constructs a Regexp object. No need to escape those slashes!
%W - Feed it with a string of words separated by spaces, get back an array of words.
%x - Interpolated shell commands. Same as using backticks.
Another year almost goes passed by. Time for doing this new years resolutions thing again.
Let's look back first, and see what I have accomplished in 2011.
Work hard on my MYOB works. Make the project we're working on a big success!- Work smart on my freelance works. Try getting recurring income.
Rails 3 is out for a while now. I WILL make a pet RoR app this year!Get myself into Test Driven Development and study Behaviour Driven Development.Keep spending quality time with family. At least 2 road trips.Take over the duty of sending Oscar to his childcare (that is 3 mornings a week).
Okay! 5 out of 6 items are stiked out. Not too bad!
Now, look forward. In 2012, here are the things I want to make happen.
- I'll be re-joining MYOB and work on a new project. I want a big success on that!
- Keep working on freelance recurring income. I believe things will work out this year.
- Finish off my Rails 3 side project.
- Keep doing morning and afternoon childcare runs (3 days a week).
- Family time! Again, at least 2 family trips.
- EDITED: Do exercise to lose weight.
This is it! I'll see how things are tracking by the end of 2012!
Because OS X crashes every 2nd day, I thought it's time to give Ubuntu another good spin. First task is to setup my Ruby working environment.
On a fresh Ubuntu 11.10 VM, I firstly installed curl and git via apt-get
sudo apt-get install curl git-core
I then installed RVM using the normal RVM installation command
bash < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer)
Then I tried to install Ruby 1.9.3, I ran into all sorts of issues, readline not available, yaml failed to build, etc. After some fooling around, I found that all I needed to do was to RTFM! Running the following gives me all required packages needed to install an MRI (and others) ruby...
Believe or not, after doing the sudo apt-get install line from rvm requirements, RVM is happy as!
Lesson learnt: before start blaming a tool (especially a free one!), let's check what the README says first!
Open source is fantastic! I learnt something new from reading Rails source code today again.
Today's surprise was brough to me by the power of regular expression and Ruby blocks.
Rails 3 Net::HTTPHeader module has the following method
def urlencode(str)
str.dup.force_encoding('ASCII-8BIT').gsub(/[^a-zA-Z0-9_\.\-]/){'%%%02x' % $&.ord}
end
1st, I learnt the gsub method could take a block as its second argument.
2nd, inside the block, all regular expression back references could be used, e.g. $& for referencing all matching elements.
I needed to publish an Android app for one of our clients today. Screenshots of the app is required for publishing.
To do it, extremely simple.
- Get application onto testing Android device
- Connect Android device to computer, make sure device's debug mode is enabled.
- From the computer, start the DDMS program that's shipped with the Android SDK
- From DDMS, highlight the connected Android device, and go to menu "Device" - "Screen capture"
Not too hard. Have to say Apple's native screen capture feature on iOS devices is a lot simpler.
RVM is super easy to use, and it manages Ruby Gems well. Most of the Ruby/Rails projects I work on are using its Gemset feature.
I was running RVM version 1.6 and decided to perfom and upgrade.
rvm get latest
rvm upgrade 1.9.2-p180 1.9.2-p290
Second command from above is extremely useful. As per example above, it updates my local Ruby 1.9.2-p180 to 1.9.2-p290. All gemsets associated with the 1.9.2-p180 are also relinked to the new 1.9.2-p290. Handy!