FpML - Yet Another Clever Idea
I’ve had the displeasure of working with the fpML spec (financial products markup language) for the last few days, and the one glaring fact that seems to present itself every minute is what a terrible terrible idea it is to use XML (with a bucketload of schemas or dtd’s in tow) for everything. This spec (to save you the bother of looking it up) is supposed to make life much easier for sharing and dealing wih financial derivatives. The main argument for this ’standard’ is that currently, the counterparties involved have to spend a while getting to know each other and agreeing on a data format (and more importantly, the data itself) to use. There’s a bewildering range of technology used. Some financial institutes are J2EE’ed to the hilt, and can easily offer you an interface in the J2EE API of your choice. Some like receiving very many faxes. Most are somewhere in between. So the effort to have these two firms talking to one another varies somewhat.
FpML cleverly states that by adhering to this spec, you’re removing all this effort. It’s certainly an interesting point to make. Of course, if you were the read the spec you’d realise that for any given firm, it’ll define a dazzling array of completely useless and unrequired elements. It also allows for an equally dazzling array of validation schemes. They provide defaults, but realistically, one often ends up having to define yet another damn schema to define one’s validation scheme.
Of course, this is without even considering the dubious value of having XML schemas in the first place. They’re hardly ideal for any sort of load as one could have written the trade in blood and hand delivered it in the time it takes to validate a reasonable sized xml document against its schema. So, does anyone have any suggestions for a lightweight runtime schema validator? Something that is under 2mb? Something that isn’t relax-ng (jarfest) or xerces (bloatware)?
This is not even considering the fairly rapidly changing spec. It went from DTD’s, to schemas, 4 major version numbers, erratas, modifications, clarifications, blahblahblah.
So at the end of the day what do you have? Hours spent deliberating over a spec playing hunt the field. You know you want to transmit an optional early termination date, for example, but now that you’ve become a slave to the big fancy spec, instead of sticking it into your well formed yet surprisingly bloat free xml document, you have to trawl through some spec to find the correct way of expressing it. Needless to say, the counterparty might or might not support it, the hard work of having to talk to them and decide on exactly what data they expect and what they’ll flatly refuse still has to be done. If you’re lucky, it’ll have some passing semblace to the schema, most days one isn’t very lucky at all I’m afraid.
June 24th, 2003 at 10:58 am
A boring comment: I agree with you.
Not-so-long ago one of my friends stated that as programming moved from structural to OO, the data structures had to move along from tabular (=delimited) to tree-like. Which makes sense, but there is no magic to it.
This sheds some light onto the DTD definitition struggles: would it be realistic to try do define an universal (and fixed) table structure for the problem the FpML is trying to address? And why should DTD do any better?
Nice watching you walking the tightrope ;-)
June 25th, 2003 at 7:38 am
Okay, its got problems, just like almost every technology.
So, what pray tell is the solution? Afterall, this seems to be a general enough problem that everyone seems to face.
Please, I would for once like to hear one creative idea from you. Does’t have to be a viable idea, any idea will do.
June 25th, 2003 at 8:13 am
I’ve had to work with FXML and FSML (e.g. related to e-check processing), which I assume are similar in their level of disgustification.
Most of these are “good intentions” and occasionally “good ideas” which result in terrible implementations. Unfortunately, it seems to just be the way it works.
June 25th, 2003 at 9:45 am
Hey if someone could give a good idea of just how to improve the current situation then I’ll be extremely happy (and possibly extremely wealthy).
I did research on “spontaneous business exchange” at IBM TJ Watson and I really don’t have an answer. Some clues but no solution.
June 25th, 2003 at 2:18 pm
The problem from what I can tell is that everyone involved is trying to come up with an all-encompassing uber-spec that will solve every problem in the entire universe, resulting in something so complex that it misses its intention and solves no problems.
Here’s my solution: Create the spec the old-fashioned way. Yes, you can use XML (for once, this is what it’s for), but write a schema for two entities to communicate NOW. Don’t include everything you think may or may not happen.
Wanna share with someone else? Change it. You can even call it ‘refactoring the spec’ if you wanna feel good about it.
The point is that it’s almost impossible to sit down and come up with some file format that’s all-encompassing and will work for everything. And a spec is NEVER going to be a replacement for communication (by which I mean human communication).
So, the solution is to change the problem being solved to one that is worthy of solving.
August 3rd, 2004 at 11:55 am
As every technology it has its limitations. You are not talking about problems of FpML, which I am sure it has them. You are talking about limitations of XML and XML Schema in general (enumerations, schemes, validation,…).
I think you don’t see the big picture here. I think you are very focused on the programming issues but the reality here is that banks are spending millions (and maybe bilions) of dollars in data interfaces because each one uses its own data format. One bank trades with several banks at the same time so the problem is not 1 party to 1 party (with a simple xml format defined by the two parts), the problem is n parties to n parties.
FpML is developed by volunteers so, you are more than welcome to participate :)