Fat doesn't matter

In a previous post, Big Fat Catalog Indexes, I was really concerned about the amount of data that is indexed and how this results in big transaction sizes. Recently I went back to that test and decided to pack the Data.fs. After packing, the 2 Gbyte Data.fs shrunk to 95 MB. Duh! So although this test seemed like a real bummer for cataloging in the ZODB, it turned out to be a non-issue. The size of the transaction doesn't really impact on the index speed as more benchmarks proved. The schema lookup in Archetypes is the major bottleneck when indexing documents in Plone. See the thread on plone-dev for more detail if you missed it.

I'm always suprised when I do these benchmarks, and they often make me feel really stupid. And since I'm stupid I keep doing benchmarks, even though the conclusions in this month's stats totally contradict last month's. Maybe I'm not so stupid and we're all learning what the ZODB can do as we grow with it year after year. If this is not the case, just remember ignorance is bliss ;-) You gotta admit though, it is one awesome piece of technology! Thanks Jim.