Dodgy benchmarks
For anyone following JavaBlogs, you’ve probably skimmed over a bunch of Sri Lankan shirtlifters cooing recently about a highly dodgy benchmark that somehow proves that Axis2 is faster than XFire. These turds are of course the same bunch of clowns who are very busy plopping their tagnuts all over the WS-* racket at Apache. The benchmark is by none other than WSO2; the company that effectively owns Axis2.
What’s interesting about this benchmark is that it manages to compare apples with oranges, and yet has its proponents drawing some truly perplexing conclusions.
There are some, ahem, issues with this benchmark. Like any vendor driven benchmark desperate to compete, It manages to compare wildly different things, just to find a case whereby they ‘win’. Instead of comparing the stacks fairly, they focussed on comparing the data binding parts. What they proved in fact is that ADB (a proprietary Axis2 data binding framework) is faster than JAXB (data binding specification). OK, not so surprising. If you wanted to be fair and picked the fastest data binding for XFire too, you could test Axis2/ADB against XFire/JiBX (JiBX being a frightfully fast OSS data binding framework). You’d see that XFire shoves a sharp pointy plunger in the general (or maybe even not general) vicinity of the WSO2′s development team’s collective asshole.
In fact, buried in the so called benchmark, you’d find the results themselves prove that XFire is faster than Axis2 when the comparison is fair and uses the same binding framework. Somehow though, this rather crucial fact is lost in the conclusions.
What’s more impressive is that no sooner than the company issues this highly dodgy report, its employees all start blogging about it and bragging that they finally managed to concoct a benchmark whereby they beat their competition.
Eran Chinthaka’s (whoever the fuck he is) blog has a particularly hilarious entry, with such gems as ‘We never did marketing for Axis2 telling bad things about other stacks’. Well my friend, I certainly am going to tell bad things about your stack, given your smug, snide, and ignorant post. The real icing though has to be ‘…then I feel Axis2 is even better than the numbers mentioned in the paper’. Huh? What, you, someone with a vested interest in your product, feel that the benchmark that you published and skewed to highlight your product, is not as biased as it should be? That reality, despite not even conforming to the distortion field imposed by your assumptions, somehow goes above and beyond it? I had no idea that the drugs available in Sri Lanka are so superior to those available to the rest of us.
This is of course, another ‘open source’ company. I thought that with JBoss being bought out and the old cabal being effectively neutered, we’d seen the end of the snide slimey comments by OSS against OSS, but our Sri Lankan friends seem keen to assume the throne of corporate fuckwittery in the name of OSS.
I hope this benchmark is taken as what it is, a company defending its own product against what is obviously a superior OSS solution. Friends don’t let friends use Axis2. Do you want to base your webservices infrastructure on a product from a company desperate to take your money, written as a student project, with a bunch of incompetent weasels at the helm trying very hard not to look like used car salesmen? Do you want something that integrates and embeds well withint your existing infrastructure, or do you want something that hijacks it instead?
Still, good thing we have the goodship Apache; that thriving haven of open communities, that final refuge for corporate flotsam and jetsam, that last bastion against whored solutions foisted on an unwilling and uninterested public, eh?
January 31st, 2007 at 2:36 am
First Post!
?????? ???!
January 31st, 2007 at 3:58 am
Xfire is great to get a beginner started with Web-Services. As an absolute beginner you can get your first WS up and running in one or two hours. As such, I use it extensively in prototyping and testing.
Having said that, I can’t help but observe that Xfire implies a large number of dependencies. In a recent test, my HelloWorld web-service had a footprint of 13MB. I don’t mind that much the size if it were not for the time wasted while launching my tests.
As for Axis2, it’s a disaster.
My colleagues have had good results using Tango a.k.a. WSIT which interoperates well with .Net’s WCF. Given that much of the boost in interoperability is due to JAXB2 and given that Xfire can use JAXB as an XML encoder, Xfire remains as a reasonably good candidate for WCF interoperability.
January 31st, 2007 at 10:24 am
You are looking for – Axis2/ADB vs XFire/JiBX. Anything else?
January 31st, 2007 at 4:48 pm
both axis2 and xfire sucks. use Java6.
February 1st, 2007 at 3:28 am
Hani you bigot, it’s not the Sri Lankan’s fault that Turks can’t cut code.
February 2nd, 2007 at 10:14 am
Come on Hani, everyone knows that you are a XFire contributor. So, your comments are obviously biased.
February 2nd, 2007 at 9:56 pm
The JAX-WS 2.1 RI was released today and the announcements include some performance numbers. The spotlight at TheAquarium is at http://blogs.sun.com/page/theaquarium?anchor=jax_ws_2_1_fcs.
February 2nd, 2007 at 11:08 pm
actually it is fun to watch Sun’s reference implemetation kicking ass of both axis and xfire.
February 3rd, 2007 at 1:58 am
I was an Xfire user.. nothing special really. With JAX-WS, Glassfish and Netbeans5.5 there has been no reason to look anywhere else. Might give Axis2 a go – must be something good for Hani to go on about it.
February 4th, 2007 at 2:44 am
So Hani, you complain about people not spelling your name right, and then we look on the XFire “Team” page and you have some “Hani Suileman”. Even your own team cant spell it right. Man thats funny. And you’re developing projects in the “Codehaus of Crack” now … my thats a turn around.
February 4th, 2007 at 12:34 pm
Isn’t XFire being merged with something else and incibated as
CXF at Apache – or have I got the wrong end of the stick?
February 4th, 2007 at 9:55 pm
I just saw Sanjiva’s response to this bile:
http://www.bloglines.com/blog/sanjiva?id=176
Hey Hani is it true you’re just an Apache reject?
February 6th, 2007 at 2:01 pm
Ok. Here’ you go. You can see Axis2/JiBX vs XFire/JiBX vs Axis2/ADB here:
http://www.bloglines.com/blog/paulfremantle?id=68
February 6th, 2007 at 2:02 pm
Hani… I ran the XFire/JIBX vs Axis2/ADB test. I’m afraid it didn’t work out as you hoped. Take a look: http://www.bloglines.com/blog/paulfremantle?id=68
February 8th, 2007 at 11:12 am
Hey H.Suleiman – TIME to BILE yourself!!! Don’t you think?
After looking on
http://www.bloglines.com/blog/paulfremantle?id=68
looks like you were tottaly wrong with your attacks on the benchmarks
February 10th, 2007 at 2:48 am
Hey Hani you should be kicked on your ass
February 10th, 2007 at 2:55 am
Haha.. the title should be ‘Dodgy Blogs’.
I’d stay the hell away from Xfire if Hani was contributing to it!
February 11th, 2007 at 3:07 am
“The bottom line? Axis2 with JIBX is 2.5% faster than XFire with JIBX, and Axis2/ADB is 5% faster. These are averages – on some tests XFire was a little faster. I don?t want to make out that these are huge performance leads. They aren?t. Most user?s wouldn?t see any real difference between the stacks in terms of performance. In the article we were trying to point out that both stacks are fast and that SOAP is not slow. However, these numbers clearly show that there is no ?sharp pointy plunger? in the vicinity of Axis2.”
2.5% to 5% seems to be a lot about nothing….
XFire seems to have the industry momentum, the better documentation and a more active community….
The last time I used Axis, it was a really bad experience filled with poor documentation and strange stack traces. XFire is a joy to work with….
-1 Axis
+1 XFire
February 11th, 2007 at 7:05 am
Handbags down, ladies!!!!!
February 23rd, 2007 at 1:23 am
Friends definitely don’t let friends use Axis2. See my blog (URL below) for related observations…