SOA In the City

How fast you can send messages through ServiceMix is highly dependent on how the embedded ActiveMQ broker within ServiceMix is configured. This is because the ServiceMix NMR uses ActiveMQ as its messaging engine, so choosing the correct connection factories and persistenceAdapter can give you easy gains in performance.

In ActiveMQ 5.0, a new high performance journaling persistence adapter was introduced called the AMQ Message Store.

1
2
3
4
5
6
7
8

 <amq:persistenceAdapter>
         <!-- Goodbye journaledJDBC --> 
         <!--amq:journaledJDBC journalLogFiles="5" dataDirectory="./data/amq"/ -->

         <!-- Hello AMQ Message Store -->
         <amq:amqPersistenceAdapter directory="file://./data/amq"/>

 </amq:persistenceAdapter>

   With JournaledJDBC:

   [java] Overall time: 137701 ms
   [java] Messages per sec: 7.262111386264443
   [java] 
   [java] Time: 139.603
   [java]
   [java] OK (1 test)

   With AMQ Message Store:

   [java] Overall time: 20511 ms
   [java] Messages per sec: 48.75432694651651
   [java]
   [java] Time: 23.556
   [java]
   [java] OK (1 test)

1
2
3

/trunk/pom.xml
-        <activemq-version>4.1.1</activemq-version>
+        <activemq-version>5.1.0</activemq-version>

/distributions/apache-servicemix/src/main/release/conf/jndi.xml
-       xmlns:amqra="http://activemq.org/ra/1.0"
+       xmlns:amqra="http://activemq.apache.org/schema/ra"

This Artix/E demo is pretty cool.

An iPhone is shown using Artix/E to talk to a Chumby. The accelerometer data from the iPhone is streamed over the wire to the chumby's chumball widget.

[INFO] The service unit my-jms-consumer-su does not have a dependency which is packaged as a jbi-component or a project property 'componentName'

If you see the above error when trying to build your servicemix service assembly, the trick is to look at the pom.xml for the component with the problem. In this case the ServiceMix SU is'my-jms-consumer-su'.

1
2
3

        <properties>
                <servicemix-version>3.3.1.0-fuse</servicemix-version>
        </properties>

1
2
3
4

        <properties>
                <servicemix-version>3.3.1.0-fuse</servicemix-version>
                <componentName>servicemix-jms</componentName>
        </properties>

This problem only tends to occur when you have multiple su's within the assembly that depend on the same servicemix component (e.g. servicemix-jms). Make sure you add the componentName to all of the Service Unit poms.xml's that use that component.

I've been running performance tests with Fuse ESB 3.3.1 and the spring based JMS endpoints over the last day or two.

One gotcha I've seen mentioned before with ActiveMQ is on the producer side you need to use a PooledConnectionFactory when using springs jmsTemplate. The same applies when using the Servicemix jms:provider endpoint (as its using jmsTemplate under the covers).

1
2
3
4
5
6
7
8
9
10
11

 <jms:provider
      service="test:MySpringProviderService"
      endpoint="mySpringProvider"
      destinationName="queue/OUT"
      connectionFactory="#connectionFactory"
      pubSubDomain="false"
      stateless="false" />

 <bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
    <property name="brokerURL" value="tcp://localhost:61616" />
 </bean>

change the connectionFactory to something like this:

1
2
3
4
5
6
7
8
9

 <bean id="connectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory">
    <property name="connectionFactory">
      <bean class="org.apache.activemq.ActiveMQConnectionFactory">
        <property name="brokerURL" value="tcp://localhost:61616" />
      </bean>
    </property>
    <property name="maxConnections" value="1" />
    <property name="maximumActive" value="1" />
  </bean>

Then by the numbers for my test flow (Jms Consumer->EIP->Bean->JMS Provider & processing/10K worth of messages) ...

     [java] Overall time: 40360 ms
     [java] Messages per sec: 247.77006937561944
     [java] 
     [java] Time: 41.064
     [java] 
     [java] OK (1 test)
     [java]    

     [java] Overall time: 908363 ms
     [java] Messages per sec: 11.008814757976712
     [java] 
     [java] Time: 909.34
     [java] OK (1 test)
     [java] 
 

Those couple of lines of XML make a big difference. Without the pool, the JMS connection is torn down each time and that accounts for the disparity.

A second area to look out for is the jms:consumer endpoint configuration. Make sure you set the cacheLevel to CACHE_CONNECTION (1) or CACHE_CONSUMER (3). The default is CACHE_NONE. Below we set the cacheLevel for the underlying DefaultMessageListenerContainer to CACHE_CONSUMER

1
2
3
4
5
6
7

  <jms:consumer service="test:MySpringConsumerService"
        endpoint="mySpringConsumer"
        targetService="test:MySpringProviderService"
        destinationName="queue/SPRING_IN"
        connectionFactory="#connectionFactory"
        concurrentConsumers="1"
        cacheLevel="3"/>

So I got my Chumby a couple of days ago and its very solid so far. The chumby arrived promptly within a couple of days of ordering it - in a nicely packaged canvas bag with no molded plastic packaging or wire ties to cut through.

As far as setup goes, it was very straightforward. It connected to my wireless network first time, and after configuring some flash widgets to play using my account on the chumby website I was up and running. The chumby started to cycle through the set of selected widgets (in my case - current weather for my zipcode, blog rss feeds (particularly good for blog comments feeds), google news, stock quotes, flickr, utube most popular etc). As far as the widgets go, the user experience is different to browsing the web. Your fed small digestible chunks of the web at a time and after a couple of days of usage its quite addictive.

With the basics over with (and as the chumby runs an embedded Linux 2.6 kernel) the next step was to see if I could connect to the chumby. Thanks to the linux librarian, I discovered it was possible to ssh. I just needed to enable sshd using the "pi" icon in the chumby options panel. After folowing these steps, I fired up putty, entered the default password of "root" and I had a shell to the chumby. Very cool!

The Linux librarian also mentioned streaming audio from a SlimServer so I figured I'd give that a try also. This took a bit of fiddling around but I eventually got it working.

Here's the steps to get basic mp3 streaming to the chumby using SlimServer (and things that screwed me up):

1) Install SlimServer 6.5 from here

2) Start the slimserver. It will listen on http://localhost:9000 and you can connect to this url using a web browser to bring up the slimserver admin interface. When you first connect you can give it the location of your music folder and it will find all the music on your computer

3) ssh to the chumby. Start the 'btPlay' mp3 streaming utility to connect back to your slimserver.

 
>btPlay http://:9000/stream.mp3

4) In the slimserver admin configure the content you want to stream to the chumby client. I made the mistake of adding a file with a .m4a extension first. After some time I resorted to running the slimserver with debug output on the command line and the mistake was obvious enough .. not obvious without doing that. Switching to an mp3 and voila! I had mp3 audio streaming working. The Chumby is not a squeezebox but the sound is fine for background listening.

5) Next step was to try and get radio streaming working. I figured I'd have a stab at getting AlienBBC working. I installed the AlienBBC install pack and tried to stream the Chris Moyles show .. silence...(followed by some swearing ....more silence ...)* and back to the slimserver debug output ..

 
>slim --d_source --d_plugins -d_remotestream

- Build and install lame.exe into the "\server\Bin\MSWin32-x86-multi-thread" directory

 

        # Check client - only stream to known slim clients
	#if (!defined($client) || !$client->isPlayer()) {
	#	$::d_remotestream && msg "Alien only streams to Slim players\n";
	#	return undef;
	#}

(If your in an RSS Reader and swf doesn't embed click here)

Some videos from one of the creators of the upcoming hackable/open source Chumby.

A chip-on-board (CoB) wire bonder:

Amazingly, the PCB's of modern electronic toys don't seem to have a single visible chip (although they have all kinds of smart logic and sounds samples). The chips have been replaced with little blobs of epoxy - i.e COB (silicon without the casing). I'd wondered about this when I was pulling apart one of Ellie's VTech toys a few months back.. mystery now revealed as the Chumby has the same thing.

Injection Molding:

Also pretty interesting on how the plastic casings are made.

Quality control on the modern electronics assembly line:

Its easy to take for granted all the work that must go on behind the scenes to produce something like an iPod or any other electronic gizmo. Interesting perspective on how the Chumby folks managed their quality control remotely, especially with all the recent news headlines on this topic.

So who knew? My electricity meter has an "interface" (infrared port) that allows you to read power usage statistics realtime.

A company called Blueline innovations has developed a wireless powercost monitor that transmits the power usage stats back to a display device so you can see how much electricity your using realtime. Even better, if your an NSTAR customer in MA you can get one for 29.95$ before July 31st.

I got my Powercost monitor installed today. The install should have been easy but it took a good hour to get the reader aligned with the sensor on the meter, but now that its working I can start to see the worst offenders ..

Top of the list is the clothes drier at about 3Kwh .. next looks to be the water heater at 2kwh. Our refrigerator is ancient and that looks to pull about 800w much more frequently than it should need to.

Realtime feedback in terms of dollars and cents really makes you think about how much electricity your using.

You can also draw some parallels to the world of SOA. The monitor, while far removed from the world of IT, does give you insight into the operation of the business - in this case my house.

Now I can see if I run my drier for an hour its going to cost me approximately 0.54cents (assuming an effective NSTAR rate of 0.18cents per kw/hour with taxes etc..). The technology is aligned to the house's capability to consume power. The impact of using a given appliance is immediately visible and traceable. There is also opportunity for improved efficiency as your have deeper insight and control over whats costing you money. All in all - its an incremental step towards a poor mans SOA!

JRuby 1.0 was released last Saturday so I had a chance to take it for a quick testdrive today.

In the example below I used Orbix java 6.3, so I added Orbix's corba-asp.jar and a jar containing the helper classes for my client to the java classpath. Then it's just a matter of calling 'jruby corba_client.rb'. It all pretty much worked out of the box.

require 'java'

include_class 'java.lang.System'
include_class 'java.util.Properties'
include_class 'org.omg.CORBA.ORB'
include_class 'simple_persistent.Simple.SimpleObject'
include_class 'simple_persistent.Simple.SimpleObjectHelper'

props = System.getProperties
props.put("org.omg.CORBA.ORBClass", "com.iona.corba.art.artimpl.ORBImpl")
props.put("org.omg.CORBA.ORBSingletonClass", "com.iona.corba.art.artimpl.ORBSingleton")


def import_object(orb, filename)
  
  ior = ''
  f = File.open(filename, "r") 
  f.each_line do |line|
    ior += line
  end

  puts "Have IOR: " + ior
  
  orb.string_to_object(ior);
end

puts "Initializing ORB"

# Create a non null args array
args = java.lang.String[1].new 
args[0]=""

orb = ORB.init(args, props)

objref = import_object(orb, "server.ior");

simple = SimpleObjectHelper.narrow(objref)

puts "Invoking call_me"
simple.call_me
puts "Done!"

It took a bit of digging to figure out how to create a java array in Ruby. The JRuby Cookbook was helpful there.

JRuby opens up a whole new set of possibilities in terms of using Ruby scripting (or Ruby on Rails for that matter) to interface with legacy backend enterprise systems.

Arggh. Here's a cryptic error from Visual Studio 2003 to watch out for when trying to add a SOAP/HTTP Web Reference.

Custom tool warning: DiscoCodeGenerator unable to initialize code generator.  No code generated.

In this case the WSDL was generated by Artix, and it only contained a SOAP 1.2 binding. This wsdl definitons looked like this:

<definitions name="my_server" 
    targetNamespace="http://schemas.iona.com/idl/my_server.idl" 
    xmlns="http://schemas.xmlsoap.org/wsdl/" 
    xmlns:corba="http://schemas.iona.com/bindings/corba" 
    xmlns:corbatm="http://schemas.iona.com/typemap/corba/my_server.idl" 
    xmlns:soap12="http://schemas.xmlsoap.org/wsdl/soap12/" 
    xmlns:tns="http://schemas.iona.com/idl/my_server.idl" 
    xmlns:wsa="http://www.w3.org/2005/08/addressing" 
    xmlns:xsd="http://www.w3.org/2001/XMLSchema" 
    xmlns:xsd1="http://schemas.iona.com/idltypes/my_server.idl">

The error is caused because there is no SOAP 1.1 binding/namespace in the WSDL. In order to make it work you have to use(or add) a SOAP 1.1 binding - or switch to VS2005 which supports SOAP 1.2.