The Upfront Diet for Plone

Over the years I have tried very hard to unpack the truth about Plone's performance problems and many other developers have done so too. A few products have seen the light to address the speed of Plone of which CacheFu, experimental.catalogqueryplan, archetypes.schematuning and SimpleReferenceField are the ones that I use often. Besides these products, the plone core has seen many optimisations over the years but naturally, and unfortunately, some new performance bugs are introduced a long the way. Generally though we have enough tools to save the day but I don't think that we have done enough to solve Plone's performance problems and with that the perception that Plone is slow.

There is something that complicates how this perception is formed, specifically the fact that Plone is not only a CMS, it is a development platform as well. If it wasn't, I think it wouldn't be any good in it's primary role as a CMS. With tools like ArchgenXML and paster, developers can churn out new content types and Plone products faster than Mcdonalds slides hamburgers across the counter. Yes, they are equally unhealthy! These content types inherit too much fat from the existing Plone environment and add even more fat by design. A lot of the fat can be attributed to Plone's obsession with the catalog and its irrational fear of waking up objects. The truth is, you can't use Plone if you empty your portal catalog: folder contents won't work, navigation will break, portlets will be empty and there won't be any breadcrumbs to help you run from the Grimm brothers' evil old witch. We really don't need all content types to be indexed, to have metadata, workflow, history and complicated references. In short, you don't need all content types to have all the features. This is the biggest problem with developing for Plone. Unless you know your way around 10 million lines of code and 10 years of history really well, you will find it surprisingly hard to make simple content types with less features. In other development frameworks you typically start with nothing and add the features as you need them. For example, nothing is searchable by default, you add the index when required. I believe that the same is possible with Plone, and that such an approach will serve it much better in the long run. This brings me to upfront.diet.

upfront.diet is an experimental product that will explore the ways that Plone can be turned in a development platform that lets you add features as needed, rather than taking them away. By default no content type should require being indexed or being part of a workflow. To make this work, we will rewrite the existing content listings to not use the catalog but rather list the contents of the folder using the traditional ObjectManager API using objectIds or objectValues. To filter the content for a specific role or user, we will store allowedRolesAndUsers as a local keyword index on the folder itself. From this point on we will carefully consider any additional index requirements imposed by searching and sorting and rather than installing all these indexes by default, we will allow adding them through configuration.

upfront.diet will patch existing content types to use SimpleReferenceField instead of the default Archetypes ReferenceField to remove the dependency on the reference catalog. In our experience the reference catalog is a major performance bottleneck on content types with as little as two or three references.

Both the portal_catalog and reference_catalog are often responsible for conflict errors in a Plone site so avoiding them for simpler operations that don't require them must be a step in the right direction. I hope that one can end up with an extremely light weight portal_catalog that only contains indexes that are required for site-wide content searches. This should greatly reduce conflict errors and catalog bloat. Additionally moving to an implementation where modifying content don't involve a single tool but modify attributes in a local context instead is a much better fit when using the ZODB.

Let's hope this diet works and we can make a toast on Plone's health. Prost!