I had forgotten to re-enable the comments posting script when I started posting again regularly. If anyone has tried to comment and not been able to, I do apologize, it should be working now.
Popularity: 5%
I have been getting questions concerning the performance of Tango in the XML benchmarks I have been running, with people wondering how something that is not C/C++ could be so fast. “They must be cheating!”
This post intends to explain how D, and subsequently Tango, can perform so well, even against C/C++. To read more about D, please visit the home page for D - D Programming Language. Tango is an alternate ’standard’ library for the D programming language, with a design philosophy of building a great library, with extensive documentation, and providing the greatest functionality in the most efficient manner possible. How do they do that you ask?
- Array slicing - Using D’s array slicing feature, you don’t need to create a copy of a string in parsing. Tango maps the DOM nodes directly onto the input buffer, so for any part of the XML, only one text version exists, that being the input. VTD-XML does something similar, and RapidXml does similar, but actually mutates the input buffer while parsing, so creates one copy before starting the parse. Java6 SAX, on the other hand, turns character arrays into strings for element and attribute names, etc, and that means more allocation.
- Native code - Yes, yes, I know. C/C++ are native as well. I write this because Java is not, and while todays’ JIT can compete and in cases outrun native code, if you are allocating hundreds of megabytes during processing, you are going to slow down in garbage collection. D is native code, to be fast right out of the gate, but also uses garbage collection. This just shows that the approach to design is MORE important than the choice of language. You will note that RapidXml is competing on speed, but when it comes to competing on efficency (speed/ram used), it loses, because it copies the input buffer to modify it during parse. Both RapidXml and Tango use internally managed freelists for nodes in the tree. One tweak to the RapidXml design could improve its efficiency an order of magnitude.
- Avoid heap allocation as much as possible - More allocation means more CPU time. Tango’s internal philosophy is one in which you allocate as little as possible, and allocate on the stack if possible. This makes the code much faster.
- Design choices - Tango XML has made specific design decisions which affect the speed, but can impact the ability of the parser compared to the parser that you are used to using today. For example, Tango XML, in its fastest incarnation, does NOT do entity decoding. If you are down at a system level, and you are building some xml proxy, you don’t need to do this. These sorts of design decisions are starting to show in the fastest/newest xml parsers. RapidXml fastest configuration does the same. Java StaX parsers have an option for the same. These sorts of omissions by design can be layered on top of the existing parser, so that YOU can make the tradeoff between speed and a particular feature. It seems as though good library design is a lost art since some point in the late 70’s/early 80’s. Features over efficiency have prevailed in the last 2 decades because ‘CPU is less expensive than programmer time’. I am not saying this is bad, just showing that good design, both in language and standard library, can create some impressive numbers.
Comments are open if other D people would like to add their $.02.
Popularity: 16%
I was helping someone on IRC in #d.tango try to use tango.text.xml to parse and display data from an xml document. We ended up building a simple example using HttpGet to get the document, Document to parse it, and Document’s xpath-like querying functionality to extract the useful bits.
import tango.io.File;
import tango.io.Stdout;
import tango.text.xml.Document;
import tango.net.http.HttpGet;
void main ()
{
auto doc = new Document!(char);
auto page = new HttpGet (\"http://www.google.com/ig/api?weather=London\");
auto content = cast (char[]) page.read;
doc.parse (content);
foreach( node; doc.query.descendant[\"forecast_conditions\"])
{
Stdout.formatln(\"forecast for {} is {} with a high of {}\",
node.query[\"day_of_week\"].attribute.nodes[0].value,
node.query[\"condition\"].nodes[0].getAttribute(\"data\").value,
node.query[\"high\"].nodes[0].getAttribute(\"data\").value);
}
}
The D programming language coupled with Tango as a standard library allows you to become a productive programmer.
Update: Please ignore the backslashes in the code if you are trying to run this example. For some reason, Wordpress is mucking around with the output.
Popularity: 7%
From my mistaken typing in the aalto benchmark, I accidentally benchmarked the default Java6 StaX parser, so this graph changes the axis to allow more players, and adds the real Aalto numbers. Click to view the graphs in full size.
Popularity: 8%
Next is Java 6’s default SAX implementation, Xerces. The code used was Sax.java, listed here:
import org.xml.sax.helpers.DefaultHandler;
import org.xml.sax.XMLReader;
import org.xml.sax.InputSource;
import org.xml.sax.helpers.XMLReaderFactory;
import java.io.*;
public class Sax extends DefaultHandler
{
public Sax ()
{
super();
}
public static byte[] getBytesFromFile(File file) throws IOException {
InputStream is = new FileInputStream(file);
long length = file.length();
byte[] bytes = new byte[(int)length];
int offset = 0;
int numRead = 0;
while (offset < bytes.length
&& (numRead=is.read(bytes, offset, bytes.length-offset)) >= 0) {
offset += numRead;
}
if (offset < bytes.length) {
throw new IOException("Could not completely read file "+file.getName());
}
is.close();
return bytes;
}
public static void main (String args[]) throws Exception
{
int iterations = 2000;
XMLReader xr = XMLReaderFactory.createXMLReader();
System.out.println(xr.getClass().getName());
Sax handler = new Sax();
xr.setContentHandler(handler);
xr.setErrorHandler(handler);
byte[] content = Sax.getBytesFromFile(new File("soap_mid.xml"));
ByteArrayInputStream bais = new ByteArrayInputStream(content);
for (int i = 0; i < 10; i++) {
long start = System.currentTimeMillis();
for (int j = 0; j < iterations; j++) {
xr.parse(new InputSource(bais));
bais.reset();
}
long stop = System.currentTimeMillis();
double timer = (stop - start) / 1000.0;
double total = (content.length * iterations) / (timer * (1024 * 1024));
System.out.print(total);
System.out.println(" MB/s");
}
}
}
Results for hamlet.xml and soap_mid.xml, respectively:
stonecobra@jeff-home:~/xmlbench$ javac Sax.java
stonecobra@jeff-home:~/xmlbench$ java Sax
com.sun.org.apache.xerces.internal.parsers.SAXParser
76.29067136262248 MB/s
78.73458569472892 MB/s
79.06138207768956 MB/s
79.43820129521801 MB/s
79.35546548074598 MB/s
79.01453088831019 MB/s
79.17875348813743 MB/s
79.23756997416339 MB/s
79.98621528135779 MB/s
79.86643957713294 MB/s
stonecobra@jeff-home:~/xmlbench$ vi Sax.java
stonecobra@jeff-home:~/xmlbench$ javac Sax.java
stonecobra@jeff-home:~/xmlbench$ java Sax
com.sun.org.apache.xerces.internal.parsers.SAXParser
37.13359003481657 MB/s
39.63213785618475 MB/s
39.705837787112095 MB/s
39.75512354386879 MB/s
39.77363726175634 MB/s
40.43266076064926 MB/s
40.59280279471393 MB/s
40.567094876541226 MB/s
40.34988523468258 MB/s
40.38804716901551 MB/s
Average parsing speed: 79.02 and 39.83 MB/sec, respectively. Note that I did remove the DTD declaration from hamlet.xml for this benchmark, since it was erroring out trying to find play.dtd.
Ouput from java -version:
stonecobra@jeff-home:~/xmlbench$ java -version
java version “1.6.0_03″
Java(TM) SE Runtime Environment (build 1.6.0_03-b05)
Java HotSpot(TM) Server VM (build 1.6.0_03-b05, mixed mode)
Popularity: 8%
Definetly not Mary Zubiate quality, but decent to eat, and 2x better than store bought crap… Just slightly better than the uncooked tortillas from Costco…
Borrowed from Homesick Texan, who also borrowed it:
Texas Flour Tortillas (adapted from The Border Cookbook by Cheryl Alters Jamison and Bill Jamison)
Ingredients:
Two cups of all-purpose flour (can make them whole wheat by substituting one cup of whole-wheat flour for white flour)
1 1/2 teaspoons of baking powder
1 teaspoon of salt
2 teaspoons of vegetable oil
3/4 cups of warm milk
Method:
Mix together the flour, baking powder, salt and oil.
Slowly add the warm milk.
Stir until a loose, sticky ball is formed.
Knead for two minutes on a floured surface. Dough should be firm and soft.
Place dough in a bowl and cover with a damp cloth or plastic wrap for 20 minutes.
After the dough has rested, break off eight sections, roll them into balls in your hands, place on a plate (make sure they aren’t touching) and then cover balls with damp cloth or plastic wrap for 10 minutes. (It’s very important to let the dough rest, otherwise it will be like elastic and won’t roll out to a proper thickness and shape.)
After dough has rested, one at a time place a dough ball on a floured surface, pat it out into a four-inch circle, and then roll with a rolling pin from the center until it’s thin and about eight inches in diameter. (If you roll out pie crusts you’ll have no problem with this.) Don’t over work the dough, or it’ll be stiff. Keep rolled-out tortillas covered until ready to cook.
In a dry iron skillet or comal heated on high, cook the tortilla about thirty seconds on each side. It should start to puff a bit when it’s done.
Keep cooked tortillas covered wrapped in a napkin until ready to eat.
Can be reheated in a dry iron skillet, over your gas-burner flame or in the oven wrapped in foil.
While you probably won’t have any leftovers, you can store in the fridge tightly wrapped in foil or plastic for a day or so.
Makes eight tortillas.
As a small postscript, since I was on the subject of Zubiates, I wanted to let you all know that it IS a WELL-KNOWN FACT that Orlando Zubiate can kick the ass of his older brother Fernando Zubiate, any day of the week, and sometimes while napping. Orlando just didn’t want to say anything in order to keep Fernando’s ego in check, as the truth surely would have crushed him the same as Orlando’s right hand. Comments are open for anyone with a first hand account of Orlando smashing Fernando’s will to live.
Popularity: 5%
Got myself an iPhone, along with both of my brothers. Ending up picking it up for them, since every place in the Bay Area was out of stock.
Good things:
- Packaging, as always is truly top notch. What you would expect from Apple.
- Activation SHOULD be this easy on any carrier. Why should it be more than a few edit boxes and a submit?
- The screen is beautiful.
- The touchscreen interface is nice
- FeedLounge loaded directly, with no modifications. Shows just a bit small, but a double tap nicely zooms into one of the 3 columns
- The auto-rotating apps thing is very nice
- The little magnifying glass for moving the cursor within a text box is a nice trick. Very usable.
- Google Earth style zooming from point to point driving directions is really sweet.
Not so good things:
- Not all the apps are rotation aware, and the YouTube app actually forces you to rotate based on searching or viewing video
- Why should it be locked from WiFi access before you activate cell service? If you have WiFi access, why can’t it activate itself via iTunes?
- Why doesn’t it allow you to purchase music directly to the device from iTunes? I smell something here
- No keyboard, so keyboard navigation in FeedLounge is useless…
- Not all apps rotate, which is very inconsistent, I didn’t expect that from Apple
- No copy and paste
- Where is iChat????
- Safari seems to go to sleep and not keep the app active in the background, like Windows Mobile can. You see it reloading the page when you re-activate Safari. Saves power, I am sure, but makes building apps just a bit harder.
- Why in blazes are there no cool GPS apps on this thing???
- Why can’t I build my own apps for this thing? This device is one of the best pieces of hardware I have laid my hands on, and now I can’t develop for it? I understand the want/weak need to keep it tied up, but this thing would fly with an ISV community like the Palm has/had
In short, this is the prettiest, most usable piece of hardware I have seen, while simultaneously being the most unusable device I have used in a long time (not counting my thermostat). A few software revs, and this thing will fly.
While most have picked on the EDGE connection, the WiFi connection makes Safari on the iPhone the MOST CAPABLE mobile browser I have ever seen. And the fastest to boot. I am tempted to cancel the phone features and use it as an office remote command/control device via the WiFi connection.
Popularity: 20%
Every once in a while, I get to write Java code. Having always used an external library (jakarta-regexp) to do regular expressions in my Java code, this constitutes my first try with the JDK 1.4 RegEx classes:
Pattern pattern = Pattern.compile(”^([0-9]*):.*”);
Matcher matcher = pattern.matcher(title);
if (matcher.matches()) {
result = matcher.group(1) + “: “;
}
Popularity: 77%
DB9
- DCD
- RxD
- TxD
- DTR
- GND
- DSR
- RTS
- CTS
- RI
RJ45
- RTS
- DTR
- TxD
- GND
- GND
- RxD
- DSR/DCD
- CTS
Hook DB9 2, 3, and 5 to RJ45 3, 6, and 4/5. Done.
Popularity: 71%
I’m heading to the hospital now, hope to be online again soon.
Popularity: 76%
Free and Open Source software has allowed me personally to build some wonderful things in my life, and that is the main reason for my contributions back.
Off the top of my head, in developing FeedLounge, we have used:
- Firefox
- Apache 2
- Subversion
- WordPress
- Python
- MySQL
- PostgreSQL
- CentOS
- Emacs
- Just about everything GNU, make, awk, grep, gcc, etc
- Twisted
- SQLObject
- FormEncode
- memcached
- MochiKit
- vi
- OpenSSH
- PHP
- lighttpd
- daedalus
- feedparser
- feedvalidator
- Berkeley DB
I want to thank all of these projects, and also all of the ones that I have used and not mentioned. May 2006 bring even more great software to the development desktops and laptops of the world.
Popularity: 69%
Having used Skype very little in the past, mostly because I have a lot of cell phone minutes, I had fogotten it was there, until a few folks at Apache set up a Skype conference call to talk about some infrastructure related items.
Since I do not own a headset for the computer, and I lost the charger to the abysmal Bluetooth headset that I have, I was looking for other options. I found that Skype has the ability to forward to your cell phone number when you are not online. After trying to find it in the options, I Googled a bit more and found out it only worked on the Windows client.
Although that sucks, I do still have a Windows machine “just lying around”, and I installed the latest Skype client for Windows. It seems that the call forwarding option works when you make yourself look Offline, but the Windows client is the one doing the work.
I tested it out with Leo, and it seemed to work just fine, but I have to ask myself, if this uses SkypeOut minutes (I am paying for it), shouldn’t it also work when I am not even net-connected? That would seem like the most appropriate time to use such a feature, no?
Popularity: 28%
I have been in the (un)fortunate position of negiotation several contracts lately, and I am amazed at the response I get during the negotiation phase of the contract phase.
It seems that a large portion of people refuse to change the text of a contract in negotiation, in favor of trying to convince you that the intent of the contract is NOT what the contract says. They seem to think that is good enough, and the intent will cover us both. The only problem I have with that is two-fold:
- Why in the name of all that is good and holy do you have a f**king contract in the first place, if your intention is not to hold to every word. Why can’t we just handshake and call it good, like my dad and his dad used to do. Oh yeah, because people no longer hold to their word.
- I am signing something that I do not agree to, that can be enforced by law.
When I ask about the first reason, I just get stupid looks, but when I approach the second issue, the answer is ALWAYS that the contract text only matters if things get bad. Guess what, that is the part of the contract that I care the most about!!! When things go south, the contract is the only thing I have to protect me and you, and you are unwilling to allow the protection in a mutual manner when you refuse to change the contract. Give me a freakin’ break.
I have had no less than 5 of such incidents in the last 4 months. I am soo lucky.
I think a good bit of the ACTUAL reason for these refusals to negotiate is the fear of having to go talk to the lawyer again. That is the only reasoning I can come up with that is anywhere near sanity.
Popularity: 37%
Alex is ruminating on choices for a new development machine, and since he mentioned me in the comments, I thought I might be able to add some fuel to the woodpile, so to speak.
First, the comment about VMWare. It is true that the fastest Windows machine I have ever owned was a VMWare instance of Windows 2000 running on a Linux box. It was hands down the fastest machine I have ever used. It was only a P4 2.8GHz with 2GB of RAM and dual 60GB drives (no RAID). I don’t know all the factors that led to that being the case, as I didn’t do any special configuration fo the Linux box or the VMWare instance. Everything just worked, kind of like a car built on Wednesday.
The second fastest Windows machine I ever owned was a 486DX-50 box with 64MB of RAM running Windows for Workgroups 3.11. Not only did the main memory run at CPU speed (the last intel CPU to do that, IIRC), on bootup I created a 32MB RAM drive, copied the windows directory to it, and then booted Windows out of RAM into… RAM. That machine just screamed. Nevermind that the 64MB of RAM at the time set me back something like $3500 alone. I guess I am just a hardware whore.
Enough nostalgia.
The choice is Linux, Windows or Mac. For Alex, as I said in my comment, I just don’t think that the polish is there on the Linux Desktop for you to be happy. Nothing against the Linux Desktop, it’s just that I think the savior of the Linux desktop is the command line, and Alex is not a HUGE command line guy like I am. If you were to try Linux, maybe you should build one of these: Ultimate Linux Box 2005. 4 CPUs (could be dual core now, so 8 CPUs), 32GB of RAM, O decibels of noise, I would love one, if any sponsor would like to ship me one. 
So, that leaves Windows or Mac.
I can say I am comfortable with either, they both have strengths and weaknesses. Since Apple is in the middle of ‘the big switch’, I would go with a nice beefy desktop, using the new dual core chips, with about 4 GB of RAM, and 4 hard drives in a RAID 0/1 configuration, for safety and speed. With 2 24″ LCDs in a vertical configuration, and a dual DVI video card, you should be well under the price of the Apple Dual G5 with 30″ display, and have more of everything on a technical level, just some loss of good looks. 2005/6 is the year of dual everything, it seems.
And if you don’t install VirusMaker, I mean Outlook, you should be able to only do scheduled virus scans, instead of active scanning. I would also recommend a nice little hardware firewall, because $80 is cheap insurance.
Popularity: 30%
David Hornik has a great post on the power of blogging:
That is the power of blogging and it is why I spend as many hours as I do wandering around the blog “library.” Moreover, it is precisely why so many smart people are focused upon the hard question of how to abstract that temporal proximity from the blogs themselves. I am convinced startups will solve this problem, not the Googles and Yahoos of this world, and look forward to seeing (and perhaps funding) the different approaches that emerge.
“Temporal proximity” is the technical term for “Tom likes what Steve likes”. This hidden power of hyperlinking is set to explode in the coming months and years, as many people are hard at work on this “problem”.
This is also part of the reason that I believe many ’subscriptions’ in the future will be based on aggregations of items in temporal proximity to one another. My 1500 subscriptions give me a lot of great information, but with over 15 million feeds out there, I am outnumbered by 3orders of magnitude.
I could conceive of the ability of aggregators to make up one order of magnitude in the near future, possibly 2 within a few years, but by that time, we will still be behind 3 orders, as the blogosphere continues to grow. The only way to make that up is to help the user find what they want using some sort of temporal proximity technique, be that cross-linking, keyword search, folksonomy, you-name-it.
It is going to take some serious outside the box thinking to satisfy this informavore (sounds a lot like information whore).
Popularity: 29%
I have been serving ads to IE readers for about 6 months now. I chose IE readers because:
- I am not one, so I don’t have to see my ads (hypocritical?)
- I wanted to test the performance of this ‘ad’ thing, without violating the first choice.
So, after 6 months, what have I learned? Basically, the ads are not contextual enough for my blog. I am ‘all over the map’ in blogging topics, so I don’t make the big bucks like Russell Beattie.
I am leaving them for now, but they may be gone soon, as I don’t see this curing world hunger, or even covering my hosting costs in the next few years.
Maybe I need to talk about stuff that is more interesting, or more single subject?
Popularity: 15%
Took the red-eye out to St. Louis this morning, because I figured not getting good sleep in a hotel room is the same as not getting good sleep on a plane. The jury is still out on which is worse…
Popularity: 30%
In his article Web 2.0: It’s a great time to be an investor, Clarence Wooten talks about the ease with which one can start a company in today’s web economy.
What’s different about today’s Web is that new technologies and behaviors that have popularized the blogging phenomenon, are also transforming the Web from a medium where information is simply published and remains static, into a platform where applications reside and services are distributed.
Indeed, web-based applications are sprouting like weeds everywhere on the Web, and in a few years you will be able to find an application for practically anything that you want to accomplish. If one is not available, it might be a simple matter of binding a few applications together to complete the type of task that you need to do.
I use these services to create RSS feeds from tags so that new postings are delivered daily to my news aggregator . I now categorize the most relevant information into new RSS feeds of blogs that I then subscribe to. In some cases, I use a collection of these feeds to create TagClouds that generate automatic folksonomies that allow me to analyze the popularity of related information (tags) over time. Does this make me a geek?
Today, that does make you a geek. In about a year, maybe two, you will be mainstream with this idea. Allow everyone reading feeds on the ‘net to help you point yourself to the information that is relevant/important to you. You don’t have to trust people implicitly, just trust that people as a group will be drawn to the same information on a topic by topic basis.
…cheaper hardware, free software infrastructure and search engine marketing have made it less expensive to start a company. Additionally, I’d add that Web 2.0 has changed the rules for entrepreneurship by lowering barriers to entry enabling bootstrapped startups to gain visibility not through advertising, but primarily through social networks and blog fueled promotion. These changes are impacting venture investing as well – consumer applications are now back in vogue…
Word of mouth advertising in the web age? Say it isn’t so! I have to thank everyone out there that has made an impact to the success of FeedLounge by blogging about it, begging to use it (who ever heard of an RSS hunger strike?), even offering to help with the development. Having talked to some people over the past week, they tend to not see the consumer en-vogue theme. I can only hope that everyone eventually sees that both spaces, consumer and enterprise, are valid spaces, given a sane approach to either.
Not to repeat everyone else in this space, but yes Virginia, now is a great time to be a startup type in this old-is-new web world. Clarence, since blogs are the new watercooler, would you like to take FeedLounge for a spin? We are here, now, helping to change the landscape of what is Web 2.0.
Popularity: 17%
While on a customer site, I came across the following on my Windows laptop:
That is just funny on so many levels…
Popularity: 20%
Arrived back in St. Louis this afternoon. Will be here for 2 weeks, with a small break in between to go to my father-in-law’s wedding in Gatlinburg, TN this next Saturday. Given the choices of 7-8 hours total travel time with a plane ticket, and 8 hours of driving time, I am a sucker and chose the driving time. It has been a LONG while since I did a road trip, and that is where I have historically been able to do a lot of deep thinking on subjects, technical and otherwise.
Spending tonight working the the FeedLounge database, hoping to bring it back to a level of sanity before we install the ‘real’ database server in the colo.
Popularity: 17%
