I had forgotten to re-enable the comments posting script when I started posting again regularly. If anyone has tried to comment and not been able to, I do apologize, it should be working now.

Popularity: 14%

I have been getting questions concerning the performance of Tango in the XML benchmarks I have been running, with people wondering how something that is not C/C++ could be so fast. “They must be cheating!”

This post intends to explain how D, and subsequently Tango, can perform so well, even against C/C++. To read more about D, please visit the home page for D - D Programming Language. Tango is an alternate ’standard’ library for the D programming language, with a design philosophy of building a great library, with extensive documentation, and providing the greatest functionality in the most efficient manner possible. How do they do that you ask?

• Array slicing - Using D’s array slicing feature, you don’t need to create a copy of a string in parsing. Tango maps the DOM nodes directly onto the input buffer, so for any part of the XML, only one text version exists, that being the input. VTD-XML does something similar, and RapidXml does similar, but actually mutates the input buffer while parsing, so creates one copy before starting the parse. Java6 SAX, on the other hand, turns character arrays into strings for element and attribute names, etc, and that means more allocation.
• Native code - Yes, yes, I know. C/C++ are native as well. I write this because Java is not, and while todays’ JIT can compete and in cases outrun native code, if you are allocating hundreds of megabytes during processing, you are going to slow down in garbage collection. D is native code, to be fast right out of the gate, but also uses garbage collection. This just shows that the approach to design is MORE important than the choice of language. You will note that RapidXml is competing on speed, but when it comes to competing on efficency (speed/ram used), it loses, because it copies the input buffer to modify it during parse. Both RapidXml and Tango use internally managed freelists for nodes in the tree. One tweak to the RapidXml design could improve its efficiency an order of magnitude.
• Avoid heap allocation as much as possible - More allocation means more CPU time. Tango’s internal philosophy is one in which you allocate as little as possible, and allocate on the stack if possible. This makes the code much faster.
• Design choices - Tango XML has made specific design decisions which affect the speed, but can impact the ability of the parser compared to the parser that you are used to using today. For example, Tango XML, in its fastest incarnation, does NOT do entity decoding. If you are down at a system level, and you are building some xml proxy, you don’t need to do this. These sorts of design decisions are starting to show in the fastest/newest xml parsers. RapidXml fastest configuration does the same. Java StaX parsers have an option for the same. These sorts of omissions by design can be layered on top of the existing parser, so that YOU can make the tradeoff between speed and a particular feature. It seems as though good library design is a lost art since some point in the late 70’s/early 80’s. Features over efficiency have prevailed in the last 2 decades because ‘CPU is less expensive than programmer time’. I am not saying this is bad, just showing that good design, both in language and standard library, can create some impressive numbers.

Comments are open if other D people would like to add their $.02.

Popularity: 27%

I created a benchmark similar to the one that VTD-XML uses. Basically, since most xml processing is mutation, this benchmark parses an input xml file, executes various xpaths on the file, modifying the document in 2 instances, and then serializes the new document. The steps are listed below:

1. Parse blog.xml, preparing to query the resulting document
2. Perform the following xpath queries, or their equivalents, once each:
   • count(//*) (10390 for this document)
   • //item (a list of those 10390 items)
   • /blog/item (similar to the previous, except you know the path)
   • //text() (all text nodes)
   • count(//item)
   • count(/blog/item)
   • /blog/item[@num=’a781′]
   • /blog/item/body/p/a
3. Mutate the document by removing the resulting nodes from the last 2 queries (performed inline with the queries)
4. serialize the modified document back out

I created this benchmark for 4 products (the ones that have xpath or xpath-like support, if you know of another one, please submit me some code, and I will be happy to run and aggregate the results):

• DOM4J - TransformDOM4J.java
• Java6 DOM - TransformDOM.java
• VTD-XML - TransformVTD.java
• Tango Document - transformtango.d

After the run, I take the average cycle time, and turn that into the followin graph showing cycles per second. blog.xml is 1.3MB, so you can multiply these numbers by 1.3 to get the Megabytes per second number for each tool.

Some notes of the implementations:

• Tango, while not actually having an actual xpath parser, has the requisite power in its query language to be able to pull this off with aplomb
• You will note that the VTD code does NOT delete the /blog/item[@num=’a781′] node, because its XMLModifier is unable to perform deletes inside a delete. If someone knows how to fix this, please let me know

Would also note that these benchmarks were run on an Intel Q6700 quad core machine at 2.66 GHz, with 4GB of RAM, running Ubunu Linux.

Popularity: 17%

I have added the recent RapidXml to the graphs. Note that the RAM usage for RapidXml skyrockets, cost it efficiency. Noted on their homepage, they make a copy of the input buffer, because the input is ‘destroyed’ while parsing. I would assume that this memory usage would fit the machine it is running on, but that is a HUGE amount of allocation.

Popularity: 17%

I have started writing this post as a sidebar in comparing the parsers in my benchmarks. I will post what I know, and add more to it as I am informed by the community. Consider this a living post. Where something is just a fact, I list it as a Pro, such as language developed.

Popularity: 17%

Aaron was kind enough to help me out with the RapidXml test. RapidXml is written in highly-tuned C++, and does give Tango a run for the money. I am really glad we are starting to add some non-Java alternatives, so we can see what native code can do. Without further ado, the code is bench_rapidxml.cpp, which was compiled via:

g++ bench_rapidxml.cpp -O2 -o bencn

Results for hamlet.xml:

stonecobra@jeff-home:~/xmlbench$ vi bench_rapidxml.cpp
stonecobra@jeff-home:~/xmlbench$ g++ bench_rapidxml.cpp -O2 -o bench
stonecobra@jeff-home:~/xmlbench$ ./bench
Document Length: 279628 bytes
Data Length: 279629 bytes
Fastest:313.362203 MB/s
Fastest:312.956579 MB/s
Fastest:313.055406 MB/s
Fastest:301.303166 MB/s
Fastest:310.668081 MB/s
Fastest:310.523743 MB/s
Fastest:310.924893 MB/s
Fastest:310.434819 MB/s
Fastest:310.868351 MB/s
Fastest:310.745189 MB/s
Default:172.539398 MB/s
Default:172.309405 MB/s
Default:172.501116 MB/s
Default:172.385035 MB/s
Default:172.386038 MB/s
Default:172.455936 MB/s
Default:172.498550 MB/s
Default:172.357293 MB/s
Default:172.331007 MB/s
Default:172.326775 MB/s
strlen:3543.806666 MB/s
strlen:3589.165483 MB/s
strlen:3590.035209 MB/s
strlen:3560.508898 MB/s
strlen:3587.427295 MB/s
strlen:3590.035209 MB/s
strlen:3573.965308 MB/s
strlen:3589.551976 MB/s
strlen:3590.276875 MB/s
strlen:3565.793459 MB/s

Average parsing speed: 310.48 MB/sec in fastest mode, 172.41 MB/sec in default mode.

Results for soap_mid.xml:

stonecobra@jeff-home:~/xmlbench$ vi bench_rapidxml.cpp
stonecobra@jeff-home:~/xmlbench$ g++ bench_rapidxml.cpp -O2 -o bench
stonecobra@jeff-home:~/xmlbench$ ./bench
Document Length: 134334 bytes
Data Length: 134335 bytes
Fastest:197.352607 MB/s
Fastest:197.097866 MB/s
Fastest:196.779684 MB/s
Fastest:197.276936 MB/s
Fastest:197.096047 MB/s
Fastest:188.870551 MB/s
Fastest:197.026330 MB/s
Fastest:197.164297 MB/s
Fastest:197.156408 MB/s
Fastest:196.966655 MB/s
Default:121.320212 MB/s
Default:121.256024 MB/s
Default:121.385734 MB/s
Default:121.286215 MB/s
Default:121.236746 MB/s
Default:121.340896 MB/s
Default:121.295172 MB/s
Default:121.264861 MB/s
Default:121.311711 MB/s
Default:121.360322 MB/s
strlen:3608.479264 MB/s
strlen:3586.658061 MB/s
strlen:3619.080745 MB/s
strlen:3613.568366 MB/s
strlen:3619.694270 MB/s
strlen:3615.812122 MB/s
strlen:3615.403959 MB/s
strlen:3609.495937 MB/s
strlen:3615.914177 MB/s
strlen:3612.651269 MB/s

Average parsing speed: 196.28 MB/sec in fastest mode, 121.31 MB/sec in default mode.

Popularity: 15%

All I can say at this moment, is “finally”. I was about 2 weeks away from tossing the iPhone and scoring a Crackberry.

iPhone - Enterprise

Push email is what I need the MOST in a phone, and the iPhone wasn’t cutting the mustard, until maybe sometime in the really near future that we aren’t allowed to know at this point, but the teaser seems to be enough.

Popularity: 17%

I was helping someone on IRC in #d.tango try to use tango.text.xml to parse and display data from an xml document. We ended up building a simple example using HttpGet to get the document, Document to parse it, and Document’s xpath-like querying functionality to extract the useful bits.

import tango.io.File;
import tango.io.Stdout;
import tango.text.xml.Document;
import tango.net.http.HttpGet;
void main ()
{
        auto doc = new Document!(char);
        auto page = new HttpGet (\"http://www.google.com/ig/api?weather=London\");
        auto content = cast (char[]) page.read;
	doc.parse (content);
	foreach( node; doc.query.descendant[\"forecast_conditions\"])
	{
	  Stdout.formatln(\"forecast for {} is {} with a high of {}\",
			  node.query[\"day_of_week\"].attribute.nodes[0].value,
			  node.query[\"condition\"].nodes[0].getAttribute(\"data\").value,
			  node.query[\"high\"].nodes[0].getAttribute(\"data\").value);
	}
}

The D programming language coupled with Tango as a standard library allows you to become a productive programmer.

Update: Please ignore the backslashes in the code if you are trying to run this example. For some reason, Wordpress is mucking around with the output.

Popularity: 17%

From my mistaken typing in the aalto benchmark, I accidentally benchmarked the default Java6 StaX parser, so this graph changes the axis to allow more players, and adds the real Aalto numbers. Click to view the graphs in full size.

Popularity: 17%

Thanks to Paul Findlay, we finally have a possible contender in the Java camp with Aalto.

This goes to show you how good library design and the D Programming Language come together to kick serious butt.

PS: I am looking for anyone to do comparisons with MSXML, RapidXML, etc. More native code help is needed. Send me email at scott aht dotnot daht org.

Popularity: 17%

Next up from Paul Findlay: Aalto. Aalto.java:

How it was run:

Results:

Average for hamlet.xml: 147.22 MB/sec
Average for soap_mid.xml: 43.80 MB/sec

As noted on the website, Aalto does seem to be quite fast on the “fast path”. Impressive for a Java solution at this point.

Update: 2008-03-03 13:15 PST: Thanks to Paul Findlay for catching my misspelling of the aalto.jar in the java run command. These numbers posted are actually for the default Java6 StaX parser, and not Aalto. Re-running, I get:

Average for hamlet.xml: 147.85 MB/sec
Average for soap_mid.xml: 85.95 MB/sec
Much more impressive numbers from the Java camp. Graphs will be updated later today.

Popularity: 16%

Another benchmark from Paul Findlay, using Javolution. Here is Javolution.java:

Average for hamlet.xml: 51.16 MB/sec
Average for soap_mid.xml: 45.93 MB/sec

Most of the Java camp is starting to look the same.

Popularity: 16%

Thanks to Paul Findlay for submitting 3 new Java benchmarks, the first of which is for Woodstox. The file is Woodstox.java, listed here:

I built it and ran it with the following commands:

And the results:

Average for hamlet.xml: 79.76 MB/sec
Average for soap_mid.xml: 49.53 MB/sec

Popularity: 16%

I added Java DOM to the graphs. Building a tree in memory is not the fastest way to parse a doc, but it is the easiest way to modify the doc after parsing. Java 6 DOM shows off not too terribly bad in the parsing speed, but with all the allocation going on, RAM usage skyrockets, and the efficiency graph shows the pain.

This goes to show you how good library design and the D Programming Language come together to kick serious butt.

Popularity: 17%

I have added a Java 6 DOM to the benchmark, so I could compare the Tango DOM. I used Dom.java, listed here:

Results on the quad core machine:

Hamlet average: 49.63
Soap_mid average: 29.22