HTMLUnit's village idiots
It’s been a while since a village idiot hat has been awarded. While this is by no means a promise of regular hat givings, in this particular instance the recipient is gagging for it.
The winner is HTMLUnit. HTMLUnit, for those who are unaware, is basically a piddly little framework that lets you create web requests, and parses back the resultant html/javascript and lets you click on things programatically to simulate actual users. It’s great (in theory) for end to end acceptance testing.
Thankfully, the name is a total misnomer. For one thing, it’s not about unit testing at all. Therefore, has nothing to do with JUnit. It wisely avoids the spastic route taken by things like xmlunit, which provides classes tightly coupled to JUnit.
The premise is a pretty decent one, it’s the implementation, as usual, which makes the baby jesus cry hot tears of rage and anger while biting off the nearest nipple.
It all started innocently enough. I have a rather complex page that the current version of htmlunit fails on. Knowing how the oss thing works, I knew I’d just be told to try the cvs version if I reported the problem. Thusly I went off to cvs (sourceforge, the first sin), and grabbed the festering pile of goatpoo.
First problem, I see maven crud splattered all over. My heart sinks as I see the 3 tongued kisses of death; maven.xml, project.xml, and project.properties. This cannot possibly go well. Amazingly, there’s a build.xml there too, so I might be spared. I’m not hopeful mind you.
My dismay was in fact well placed. The ant file is nothing more than the year old solidified wankstain that maven squirts out. It manages to die before compiling a single line (it tries to specify an output dir for javac that doesn’t exist).
As an aside, I’d like to know who wrote the maven to ant plugin. I have never, in all these years, seen such badly written buildfiles. Now I know maven guys are fuckwitted rumpranging ADD-ridden gizmowhores, but surely they can’t know so little of how ant works or of how it should be used. Even their callous disregard for users and the real world doesn’t go the whole way towards explaining the mindboggling incompetence of this build.xml.
To cut a long story short, the file is a herring sporting a horrific red hue. Making this thing work would require an impressive collection of miracles from all major religions.
So we’re onto maven. Running maven, of course, is always a great excuse for a day out, maybe catch a movie or two, and a nice dinner, while it figures out what day it is, what it should do about it, and how to calculate the worst and slowest way of running a few thousand tests.
Eventually, a feeble jar is farted out. Plopping said jar in my tests reveals a rather dismal result. Cookies no longer work across invocations.
Ohdear. Nevermind, I’m a java guy, I can debug this sort of thing. Armed with nothing more than an abiding hatred for java developers and a trusty debugger, I sally forth into the brackish waters that are the htmlunit source code.
Hrm, here we go. At some point, one of these gurus of incompetence decides that we should internally store a map of HttpClients, keyed by host url. Thusly can clients be reused and state preserved across invocations, for a given site.
Said guru however didn’t actually bother fix the thing that gets stuff out of said map. So while the key is now host + port, the blocks that attempts to retrieve things from the map does not include the port. These days, every url has a port, thusly, much wailing and gnashing of teeth ensues.
Apparently, this problem has trickled down to other lost souls who use htmlunit cvs, such as the canoo people, who also have the odd user bleating about cookies being ignored.
So the fix is simple. A few lines of code later and I’m ready to attempt the Herculean task of convincing maven to give me a jar. An outing, couple of movies, and dinner later, maven proclaims that some tests have failed. Hilariously enough, the ‘helpful information’ it gives me about this build failure is line and column information of the jelly script junit plugin that failed. Highly relevant I’m sure, just not to anyone except the halfwit author of said plugin.
So after a few days of pointlessly scrolling around, I find the culprit. It’s at this point that I completely lose it and start laughing hysterically. I can’t quite believe the test I’m looking at.
The test for http state works thusly: It gets at the class that holds the map. It uses reflection to access the (private) map, makes it accessible, and manually
puts a value in it. Having done so, it then calls a method which basically just reads from the map, and then compares the two.
It’s mind boggling to me that someone would write such a complex and awkward test to basically test that HashMap functions as a….map. Of course, this test and a dozen other just like it now fail, since the keys they use to do the manual insertion cannot exists anymore
. The change to the internals of the class being tested generate keys of a different format, so the test is testing a state that cannot ever, ever happen.
It’s sad and pitiful that things like this happen. It’s even sadder and even more traumatising that it’s not that rare, and is even more common than not. The collaborative effort of so much incompetence should not happen, it should be a freak accident that only happens very rarely, instead of daily, over and over again.
August 15th, 2005 at 9:45 am
Try httpunit; somewhat more robust.
August 15th, 2005 at 10:32 am
I wonder how much “affordable open source” is actually costing the industry.
August 15th, 2005 at 10:35 am
Well, “affordable open source” is a great way to make a job for yourself. Write some really bad oss pos and then hire yourself out as a consultant to fix and maintain it. I guess Scott Adams had it right in the Dilbert where Wally said “I’m gonna write me a minivan!”
August 15th, 2005 at 11:27 am
Is it just me or is the BileBlog so fucking BOOOOORING these days?
August 15th, 2005 at 11:55 am
i’ve had very good experiences with httpunit (http://www.httpunit.org), though not for discrete unit tests. sounds like it is in the same space as htmlunit. try it for this thing you’re testing, especially if htmlunit is giving you fits. would like to know what you think.
August 15th, 2005 at 1:20 pm
Try grinder. Works like a charm for testing web apps. Use the latest 3.0 beta release though…it has some nice features and is quite stable.
http://grinder.sourceforge.net/
August 15th, 2005 at 2:01 pm
Is it just me or are the comments to BileBlog so fucking BOOOOOOOOOORING these days?
August 15th, 2005 at 2:30 pm
Dare I recommend JMeter. The GUI might not be the prettiest chick on the block, but this old Hen should get the job done for you. You can even record your test plan using JMeter’s Proxy.
http://cvs.apache.org/viewcvs.cgi/*checkout*/jakarta-jmeter/xdocs/usermanual/jmeter_proxy_step_by_step.pdf
August 15th, 2005 at 4:18 pm
Hani, it’s more fun when you write “baby jeebus”.
Got that?
August 15th, 2005 at 5:52 pm
You should put little clever icons on your posts indicating what you are biling about. Like the geese in your TestNG article. Those geese were so clever, I knew exactly what the article was about without reading it. Saved me some time. Btw, there are some great goose recipes out there. A good way to start is to enter “goose” on http://www.google.com. Once you found some recipes they usually hyperlink to other great recipes. What I did was, I sat down on a rainy weekend with a block and pencil and like the way I described above, I found about 20 goose recipes I never heard about before. I didn’t have the time though to test them all out though because I’m reading all this crap on the internet which I would not need to read if they only had clever icons. At this point I hope you figured out that geese and time and the length of this specific comment are not totally randomly connected.
August 15th, 2005 at 9:09 pm
Hani, you are trying hard to make your biles useful rather than entertaining. I am so displeased that I am stamping on my headgear.
P.S. I am writing from Heaven surrounded by naked women.
KF.
August 15th, 2005 at 10:43 pm
This blog sounds like you jogged your disgusing sun.java ass to Mcdonalds, soiled your pants, did 3 sets 300lbs leg squats, undressed, and transcribed your underpants onto jroller.
Clearly your ant skillz are uber elite, hence your unhealthy build.xml fetish.
Ok, I tried, but I can’t duplicate the bile master.
I need more bile Hani! Daily bile!
August 16th, 2005 at 3:40 am
Guards! Arrest this impostor who posts in My holy name!
JBoss rules.
August 16th, 2005 at 5:17 am
Hey Hany – Try JWebUnit jwebunit.sf.net
August 16th, 2005 at 11:38 am
Bring it home to daddy!
August 16th, 2005 at 1:30 pm
High on detail, Hani. Time it was too.
August 16th, 2005 at 2:34 pm
You might want to check out JHttpJXmlJPerl.unit.unit.unit. Use the 0.19 alpha, it’s really quite stable.
jhttp://unit.forgesource.org/wtfiiwtgfsa.net
August 17th, 2005 at 12:32 am
Hey Hani:
You might want to try JHttpUnitPuncher I’ve had really good luck with that!
When I am unable to punch my unit manually, I can invoke some JHttpUnitPuncher methods to punch my unit automatically. It’s really nice to be able to do that because punching my unit can be tedious due to the size and complexity.
August 17th, 2005 at 2:17 am
Gads, could you be any more of an idiot? Look at the wankers that you be a-ttractin’!
August 17th, 2005 at 2:50 pm
Hani, you useless pile of beetle processed elephant dung – get your f***** right column of nogoodfornuthin collection of links to deprecated postings to NOT float all over the rest of the text JEEZUS! – go read a CSS standard ..
August 17th, 2005 at 3:22 pm
I hate mankind because of projects like htmlunit. I’ll tell dad and he’ll beat the shit out of you losers. Sit on my right, pledge allegiance and ye shall be saved.
August 18th, 2005 at 4:04 pm
Did you see me fall off that horse? His dick reminds me of my movie career. Buy my records. NOW!
August 18th, 2005 at 5:43 pm
Weak bile. But true, so go get eclipse TPTP, oh, yeah, I forgot you don’t like eclipse, tough shit then.
August 19th, 2005 at 6:22 pm
You guys complain SOOOOOO much. And I have yet to see a good bile about Lotus Notes. Granted, in that case it’s more of an editorial function, or, if you prefer – and I think you do – a matter of digging out from under the steaming pile rather than finding the nuggets. Perhaps a monthly Notes bile … would be … would be too easy … ah, now I understand.
Thanks anyway.
Yeah, the Puncher thing would be good, too.
August 19th, 2005 at 10:25 pm
Outsource your unit testing. I’m sure you could hire a dozen offshore dudes for a month for the time you spent trying to get that crap to work.
August 20th, 2005 at 10:54 am
test
August 20th, 2005 at 10:55 am
another test
August 20th, 2005 at 12:40 pm
> P.S. I am writing from Heaven surrounded by naked women.
That’s OK … I’m banging your wives.
- Abdullah
August 28th, 2005 at 2:54 pm
Long time no bile!
September 2nd, 2005 at 9:27 am
I heard Hani got arrested after trying to pilot a Lufthansa 747 after having raided the drink cart of all the the Absolut.
The copilot noticed that he started punching his clown in the cockpit which aroused suspicions. “That’s not the flight control I was referring to”, the copilot exclaimed as he left to summons the authorities. When he returned with police, Hani was making monkey noises and smearing his feces all over the navigational computer’s monitor ranting about not being able to read the documentation for said system.
Now he’s sitting in a Jordanian prison waiting for trial–hence no bile.
September 4th, 2005 at 1:55 pm
Nice story, but they don’t have trials in Jordania!