Monday, March 25, 2013

Compare Tibco EMS Performance Under Load with Clojure

We have migrated our Tibco EMS infrastructure from the older Sun V490 servers to the newer HP ProLiant DL380 G7 servers. One of the concerns was how does the new servers perform under load compared with the older servers. I needed to create a load test that compared the performance of the two different types of servers. I turned to Clojure to help me put together this load test and here is the load test code:


Load Test Script in Clojure

; Leveraging Apache Commons to generate the random message
(def msg64 (RandomStringUtils/randomAlphanumeric 64))
(def msg128 (RandomStringUtils/randomAlphanumeric 128))
(def msg256 (RandomStringUtils/randomAlphanumeric 256))
(def msg64k (RandomStringUtils/randomAlphanumeric (* 64 1024)))
(def msg128k (RandomStringUtils/randomAlphanumeric (* 128 1024)))
(def msg256k (RandomStringUtils/randomAlphanumeric (* 256 1024)))
(def msg512k (RandomStringUtils/randomAlphanumeric (* 512 1024)))
(def msg1mb (RandomStringUtils/randomAlphanumeric (* 1024 1024)))
(def persistent-delivery (.intValue (javax.jms.DeliveryMode/PERSISTENT)))


; Send messages
(defn send-messages [server-url user password queue-name data iterations]
  (with-open [connection (-> (TibjmsQueueConnectionFactory. server-url)
            (.createQueueConnection user password))]
    (let [session (.createQueueSession connection false Session/AUTO_ACKNOWLEDGE) 
         queue   (.createQueue session  queue-name)]
      (with-open [sender (.createSender session queue)]
  (dotimes [_ iterations]
          (let [message (.createTextMessage session)]
            (.setJMSDeliveryMode message persistent-delivery)
            (.setText message data)
            (.send sender message)))))))
   

(def test-scenarios (list
 {:label "64 bytes" :msg msg64 :n 5000 }
 {:label "128 bytes" :msg msg128 :n 5000 }
 {:label "256 bytes" :msg msg256 :n 5000 }
 {:label "64KB bytes" :msg msg64k :n 2000 }
 {:label "128KB bytes" :msg msg128k :n 1000 }
 {:label "256KB bytes" :msg msg256k :n 1000 }
 {:label "512KB bytes" :msg msg512k :n 1000 }
 {:label "1MB bytes" :msg msg1mb :n 1000 }))

(doseq [my-test test-scenarios]
  (let [label (:label my-test)
       message (:msg my-test)
       n (:n my-test)]
    (println "\n\n")
 (println (str "Testing for " label " sized messages with n = " n))
 (time (send-messages server-url username password test-queue message n))))

Here's the message consumer part of the load test:


Message Consumer in Clojure

                
; Consume Queue Text messages asynchronously
(defn get-queue-text-messages [server-url user password queue-name process-message]
    (future
        (with-open [connection (-> (TibjmsQueueConnectionFactory. server-url)
                                   (.createQueueConnection user password))]
            (let [session (.createQueueSession connection false Session/AUTO_ACKNOWLEDGE)
                  queue (.createQueue session  queue-name)]
                (with-open [receiver (.createReceiver session queue)]             
                    (.start connection)
                    (loop [acc 0]                      
                        (process-message (.receive receiver) acc)
                        (recur (+ acc 1))))))))

; Dump just the message id      
(defn dump-message-id [message n]
    (println (str n)))

     
; Create function aliases with connection information embedded                   
(defn consume-messages [queue-name message-processor]
    (get-queue-text-messages  server-url username password queue-name message-processor))
 
; Start consuming messages asynchronously
(consume-messages queueName dump-message-id)     

Here are the test results:


Load Test Result on Sun V490

Message SizeNumber of MessagesProcess Time (ms)Throughput (msg/s)Throughput (MB/s)
64 5,000 2,772.84 1,803.21 0.11
128 5,000 1,943.23 2,573.04 0.31
256 5,000 1,899.24 2,632.63 0.64
64K 1,000 3,020.98 331.02 20.69
128K 1,000 4,414.00 226.55 28.32
256K 1,000 20,911.61 47.82 11.96
512K 1,000 39,213.23 25.50 12.75
1MB 1,000 80,337.55 12.45 12.45

Load Test Result on HP ProLiant DL380 G7

Message SizeNumber of MessagesProcess Time (ms)Throughput (msg/s)Throughput (MB/s)
64 5,000 1,340.22 3,730.74 0.23
128 5,000 1,345.18 3,716.99 0.45
256 5,000 1,040.59 4,804.95 1.17
64K 1,000 2,178.69 458.99 28.69
128K 1,000 3,553.02 281.45 35.18
256K 1,000 7,490.54 133.50 33.38
512K 1,000 35,078.26 28.51 14.25
1MB 1,000 77,076.51 12.97 12.97

From this test result, we can conclude that HP ProLiant DL380 G7 performed better than Sun V490 under load.

10 comments:

Anonymous said...

Nice approach, and great timing - I need to establish the capability of our Tibco infrastructure very quickly, and am just coming up to speed with Clojure.

21cssIndia said...


Tibco Active Spaces Online Training
Tibco Administration Online Training
Tibco BPM Online Training
Tibco Business Studio, AMX Online Training
Tibco BW/EMS Online Training
Tibco Developer Online Training
Tibco Hawk Online Training
Tibco Spotfire Online Training
Tibco Developer Online Training - EAI concepts - Introduction to Integration and EAI tools - EAI tools in the market - What is TIBCO? - TIBCO - TIBCO Runtime Agent (TRA) - EAI tools in the market - How TIBCO supports EAI - TIBCO Business Works - Introduction to Business works - Installation of TIBCO - Overview of TIBCO designer and Tester Utility - Working with palettes with complex Transformations. - FILE - XML - PARSE - HTTP - - JDBC - SOAP - WSDL - XML tools - XML activities - GENERAL Configuration palette - JMS palette - Java palette - Transaction palette - Service palette - Creating Alias Library - Overview of Grouping and its usage - Overview and usage of various variables in TIBCO BW - Xpath formula builder - WEB Services - Introduction to Web services and SOAP - WebServies implementation in Tibco BW using - SOAP Palette / Service Palette - TIBCO EMS - Employees to learn at their own pace and maintain control of learning “where, when and how” with boundless access 24/7by 21st Century Software Solutions. contact@21cssindia.com ---- Call Us +919000444287

Unknown said...

Thanks for Information SAP Online Training

Kannan said...

Please provide me detailed instructions as how to configure this scripts and run as I see lots of error when I run this code.
No such namespace RandomStringUtils on the first def command

Please advice

Tibco Online Training said...

It was very nice article and it is very useful to Tibco learners.We also provide Tibco online training software online training.

Tibco Online Training said...

It was very nice article and it is very useful to Tibco servicegrid learners.We also provide Tibco online training software online training.

Unknown said...

Thank You for sharing your article. I like it. We provides TIBCO Online Training in Hyderabad at affordable prices.

Krishna kumar said...

Hi There,

Love it absolutely! So crystalline. No mumbo jumbo. No non-sense. Straight and simple. You guys need a standing ovation for your good work.

How does one read a passed value within an Iron Python script. I am attempting to use a Link in a visualization that has a parameter from which I will set a Document Property to control the context of the target Page? I found "How to Execute an IronPython Script with Multiple Input Parameters from within Another IronPython Script in TIBCO Spotfire® Using IronPython Scripting" that reads the parameter name and description but stops short of showing how to read the value passed to the script.

By the way do you have any YouTube videos, would love to watch it. I would like to connect you on LinkedIn, great to have experts like you in my connection (In case, if you don’t have any issues).
Please keep providing such
valuable information.



Many Thanks,
Krishna kumar

Unknown said...

Hello Buddy,


Gasping at your brilliance! Thanks a tonne for sharing all that content. Can’t stop reading. Honestly!


I have a visualization that plots from a data table that is being replaced by a TERR function every time a marking is changed. The problem is that my custom formatting for the table columns is lost every time the table is replaced. Basically I want a few of my tooltips based on my columns to have a custom number format that I've defined. Every time I remark, the table is replaced, and the formatting is lost. I've thought about a few different solutions:





It was cool to see your article pop up in my google search for the process yesterday. Great Guide.
Keep up the good work!


Many Thanks,
Ajeeth

Unknown said...

Hi There,


Nice to be visiting your blog again, it has been months for me. Well this article that I've been waited for so long.

Currently, we are manually starting up our Interior and Exterior servers and I have the task of scripting that start-up in Red Hat Linux 5/6/7 and then being able to validate that start-up.
Is there documentation available for this, or has someone created a method and could direct me to a way to handle this task?I have basic knowledge of TIBCO, including some installation, but my primary job is supporting the entire infrastructure TIBCO lives in not the day to day operations. So I may have missed some areas in the documentation that are pertinent to my question.
I have to assume that whatever tasks happen in the GUI have underlying code or scripts that could be deployed in an automated fashion.
THANK YOU!! This saved my butt today, I’m immensely grateful.


Obrigado,
Abhiram