Home Company Services Portfolio Contact us nav spacer

Chapter 2. Templating

Chapter 2. Templating

Templating with DTML

Jean Jordaan

Introduction: what is templating?

In this part of the course, we're having a first look at using Zope to serve HTML pages to browsers. Zope provides many ways to do this, and the first one we'll look at is the Dynamic Templating Markup Language. This isn't because it's the best. Actually, it's because we want to get it over and done with. It also happens to illustrate many of the issues that later approaches try to solve, and therefore makes it easier to understand their design choices. Finally, the the entire ZMI of Zope 2 is implemented in DTML. The ZMI is a veritable goldmine of information about Zope, so it pays to study its crusty old DTML pages closely. If there's something you need to do (change security settings, add a folder, edit a property) you can often find a place to do it in the ZMI. When you're there, look at the source of the page and pay particular attention to the form actions.

Let's start with a definition. A template is: “A gauge, pattern, or mold, [...] used as a guide to the form of the work to be executed. [Webster's Revised Unabridged Dictionary (1913)]” A DTML page, therefore, is intended to serve as a fixed mold into which content is poured to be given a consistent shape. The purpose of templating systems is to decouple the data to be presented from the form, or template, into which it's cast before being returned to the browser [1].

Q:

Why would you want to do this?

So what is templating not? Let's look at a couple of the techniques that were around when DTML was conceived.

The simplest way to send HTML to a browser is to spit out a bunch of strings, that together make up an HTML page. I'm sure we've all done this kind of thing:


FORM_FOOT='''
</form>
'''

PAGE_FOOT='''
</body>
</html>
'''

def showFormFoot(self,session):
    self.wfile.write( FORM_FOOT )
    self.wfile.write( PAGE_FOOT )

That's embedding the HTML into the code. What are the problems with that? I'll let you count the ways.

Q:

What's wrong with embedding your HTML into the code?

Another approach is to take the HTML out of the application code, and to instead insert scripting into the page. Here's an example:


<%@ LANGUAGE="VBSCRIPT" %>
<html>
<head><title>ASP Chatback</title></head>
<body>
<p>Welcome back, <%=Request.form("yourname")%>.</p>
<p>
<%
' Declare some variables for sending email
Dim DestinationEmail
Dim OriginatingEmail
Dim Subject
Dim Body

OriginatingEmail = Request.form("email")
DestinationEmail = "you@email.com"
Subject = "The Subject Line Goes Here"
Body = "Name = " & Request.form("yourname") & "
' Send email via the SendMail utility
Set email = Server.CreateObject("MPS.SendMail")
retCode = email.SendMail(OriginatingEmail, DestinationEmail, Subject, Body)

IF NOT True = retCode THEN
Response.write("Failed to send form results to
"&DestinationEmail&".<P>")
ELSE
Response.write("Your form Results have been sent.<P>")
END IF

' Give some feedback, depending on choice
IF Request.form("answer") = "yes" THEN
Response.write("You seem very positive today.")
ELSE
Response.write("Why are you so negative?")
END IF
%>
</body>
</html>

You'll notice that this page contains almost no HTML: viewing the unprocessed page renders a single paragraph tag, and the embedded code reverts to the first approach mentioned: piecing together a page by outputting snippets of HTML depending on processing. The unprocessed page can't be viewed to gain any impression of the final page, and the side effect of viewing the page (sending email) is tightly coupled to the presentation.

These two approaches can not be said to be templating. In summary, templating is:

  • A way of abstracting the constant part of the static page

  • A way of filling in the bits that change

And the reason for doing it is to separate logic and presentation.

Why template?

When designing for the web, HTML is an interface where different roles meet.

  • The application programmer needs to elicit input from users and send them information to view,

  • The visual designer needs to convey a brand or a style,

  • The end user needs to consume the content using their client of choice (which may be a browser on a PDA, for all you know).

The ideal is that the concern of the coder stops when she's sent off the information to be displayed; that the designer can prototype the HTML without waiting for the coder, and can redesign without the coder's involvement; and that he can provide multiple templates for different end users, again without involving the coder. You'll agree that the above two primitive approaches doesn't help us with this at all. We'll see how far we get toward this goal using DTML.

Getting started with templates

OK, let's start with a really simple DTML page. Here it is:


<html>
<head><title>Directory: Jean's Homepage</title>
  <style> .sidebar {float: right} </style> 
</head>
<body>
<div class="sidebar">
  <ul><li>My Homepage</li></ul>
</div>

<h1>Jean's Homepage</h1>
<p>We'll get to this later.</p>
</body>
</html>

This might be a layout delivered by the design team. Let's say we're going to build a directory of homepages. First, strip this page down to its static template, X'ing out the dynamic parts. Here we are:


<html>
<head><title>XXX: XXX</title>
  <style> .sidebar {float: right} </style> 
</head>
<body>
<div class="sidebar">
  <ul><li>XXX</li>
  XXX
  </ul> 
</div>
<h1>XXX</h1>
XXX
</body>
</html>

Now, let's make the page dynamic:


<html>
<head>
  <title><dtml-var directory_title missing="Directory">:
         <dtml-var title></title> 
  <style> .sidebar {float: right} </style>
</head>
<body>
<div class="sidebar">
  <ul> 
  <dtml-in homepages>
    <li><dtml-var title></li>
  </dtml-in>
  </ul>
</div>

<h1><dtml-var title></h1>
<dtml-var body missing="Nothing yet!">
</body>
</html>

As you can see, DTML has a lot in common with good old "tag soup" 'HTML': e.g. the unpaired/unclosed elements and informal usage of attributes. If you try that template immediately, it'll fail. Before making that template work for us, we're going to take a step back and work up to it. At first glance, in keeping with the goal of sticking to templating, DTML is a really simple language, providing a limited number of new tags. Here they are:

DTML tags

call

Call a method

comment

Comments DTML

if/elif/else

Tests Conditions

in

Loops over sequences

let

Defines DTML variables

return

Returns data

tree

Inserts a tree widget

unless

Tests a condition

var

Inserts a variable

with

Controls DTML variable look up

Error handling

try/except/finally

Handles exceptions

raise

Raises an exception

Sending mail

sendmail

Sends email with SMTP

mime

Formats data with MIME

RDBMS interfacing

sqlgroup

Formats complex SQL expressions

sqltest

Formats SQL condition tests

sqlvar

Inserts SQL variables

Of these, var, if, in, let and with account for about 95% of templating needs between them ( unless is just shorthand for an if with a false condition). (Sadly, DTML has lots of implicit behaviour, and it doesn't stay this tidy for very long.)

Let's start again with one of the simplest DTML pages possible, the default content Zope gives a newly created template:


<dtml-var standard_html_header>

<h2><dtml-var title_or_id> <dtml-var document_title></h2>
<p>
This is the <dtml-var document_id> Document
in the <dtml-var title_and_id> Folder.
</p>

<dtml-var standard_html_footer>

You can view this one without errors immediately. (Go ahead and create it via the ZMI; call it default_content and browse to its View tab. I put mine in a Folder called templating)

Figure 2.1. Default content

Default content

Rendered, on my machine, that turns into:


<html>
<head><title>templating</title></head>
<body bgcolor="#FFFFFF">
<h2>templating Default Method</h2>

<p> This is the default Document in the templating Folder. </p>
</body>
</html>

You can see easily enough what content the DTML elements correspond to. The question that arises next is what the attribute names in the DTML tags refer to. While most DTML tags have a number of attributes, that you can look up in the online reference among other places, these aren't among them (go check, if you like!). In other words, these are not element attributes; let's call them element targets. They are looked up by acquisition (which we'll get to later) in the context of the DTML page containing them. The first two are quite easy to find: visit the root of your Zope site, and you'll find them there, large as life. The others you won't find so easily. Welcome to the dark side of DTML!

Three levels of complexity

DTML can be regarded in terms of three levels of complexity. Here they are:

DTML's levels of complexity

Simple: the DTML keywords

We've encountered them. They make it easy to turn an HTML page into a template without obscuring the page structure (as long as you don't try to code logic in them).

More complex: namespaces

This is where we're at now. The target of a DTML tag is looked up in a stack of namespaces, which starts at the page context by default, but which can be changed in various ways.

The floodgates open: Python expressions

For DTML tags that accept targets, the target may also be specified in the form expr="Expression", where Expression is any valid Python expression. (To make it interesting, the expr= is optional. When a target is quoted, it's assumed to be an expression.) While there are restrictions on the Python code that is allowed here, we are firmly back in "embedded code city". The code may have side effects, and may be tightly coupled with application code.

Another important factor, one that has caused many a newbie to boggle, is the fact that a Python expression introduces a completely different syntax into the mix [2]. The crowning glory of this is that there are whole classes of DTML identifiers containing characters such as -, which are not legal in Python and that requires arcane (but perfectly rational) lookup syntax to access.

To return to the question of where title_or_id and document_title come from, we'll be going on a little trip to find them a little later. For now, have a little faith. Before we go there, let's explore the use of DTML for templating a bit more.

Filling out the template

Let's go back to our directory template above. As simple as it is, it forms the basis of a simple, navigable directory of homepages. To start making it work, create a new Folder called directory, for example (this name doesn't matter). In it, create a DTML Method [3] called index_html (this name matters; it is the default view rendered by Zope).

Figure 2.2. Directory created

Directory created

If we create a new DTML Method with that content and view it, we'll get an easily parseable error message:


KeyError: 'homepages'  

What could be simpler? Nothing called homepages could be found. (We'll get a nicely obscure message in a moment; that script has an error planted in it.) It's worth noting that this is a not a Zope-specific error message. It's a Python error. For comparison's sake, fire up a Python interpreter and do this:


>>> dictionary = {'key': 'value'}
>>> dictionary['homepages']
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
KeyError: homepages

It's important to remember that you can use your Python knowledge to puzzle out Zope errors.

A quick detour: debugging a bit deeper

To sort this one out, let's create the folder for homepages.

Figure 2.3. Homepages created

Homepages created

Great, let's try to view our directory again. Now we have a more challenging error:


Error Type: TypeError
Error Value: unsubscriptable object

It's easy enough to figure out what's happening:


>>> title = 'homepages'
>>> title[0]              # Strings can be subscripted
'h'
>>> 0[0]                  # Numbers can not
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: unsubscriptable object

But what object is it talking about, and who's trying to subscript it? Is anything in our DTML code doing any subscription? The error page doesn't give us any of this information. To find out more, visit the error_log object in the root of your Zope site. The TypeError will be heading the list. There, we can find out the following:


Traceback (innermost last):
  Module ZPublisher.Publish, line 100, in publish
  Module ZPublisher.mapply, line 88, in mapply
  Module ZPublisher.Publish, line 40, in call_object
  Module OFS.DTMLMethod, line 130, in __call__
  - <DTMLMethod instance at 40f9c5c0>
  - URL: http://blommie:17060/tests/index_html/manage_main
  - Physical Path: /tests/index_html
  Module DocumentTemplate.DT_String, line 474, in __call__
  Module DocumentTemplate.DT_In, line 676, in renderwob
  Module AccessControl.ImplPython, line 161, in guarded_getitem
  Module AccessControl.ZopeGuards, line 67, in guarded_getitem
  Module OFS.ObjectManager, line 649, in __getitem__
  Module OFS.ObjectManager, line 242, in _getOb
TypeError: unsubscriptable object

Ack. To interpret this traceback we're going to go on a bit of a spelunking expedition here. Don't worry if this gets a bit obscure, everything will be illuminated, and though it's interesting, it's seldom really necessary to dig this deep to get your templates working.

Scanning through from the top, everything looks pretty generic until we get to DT_In. This is the implementation of the dtml-in tag. From that, we can deduce that the problematic statement in the template is the opening dtml-in statement, and the object causing the trouble is homepages, which is not in fact a sequence, but a Folder instance. But why that particular error, though? Inquiring minds want to know. Since Zope is Open Source and Python is the most readable language known to man, we can go take a look. Knowing Python's packaging rules, we can figure out that the relevant file is .../Zope-2.7.0/lib/python/DocumentTemplate/DocumentTemplate.py, and based on the line number in the traceback, the relevant code is:


for index in range(l):
    if index==last: pkw['sequence-end']=1
    if guarded_getitem is not None:
        try: client = guarded_getitem(sequence, index)

l is the length of the sequence over which dtml-in iterates, and range starts from zero. Therefor, when guarded_getitem is called for the first time, index is 0. It is this value which is passed along to the end of the traceback, where the error occurs. The code at that line, in OFS.ObjectManager._getOb, is:


if id[:1] != '_' and hasattr(aq_base(self), id):

The only subscription happening in that line is id[:1], and the value of id is the integer zero, which, as we saw at the Python prompt above, is not subscriptable.

All perfectly reasonable! But you can see that DTML can dump you off the deep end very suddenly. Here is the corrected version of the template:


<html>
<head>
  <title><dtml-var directory_title missing="Directory">:
         <dtml-var title></title>
  <style> .sidebar {float: right} </style> 
</head>
<body>
<div class="sidebar">
  <ul>
  <dtml-with homepages>
    <dtml-in objectValues>
      <li><dtml-var title></li>
    </dtml-in>
  </dtml-with>
  </ul>
</div>

<h1><dtml-var title></h1>
<dtml-var body missing="Nothing yet!">
</body>
</html>

To correct it, we use dtml-with to put the homepages folder at the top of the namespace stack in which DTML looks for names via acquisition. Then, we give dtml-in a proper sequence to iterate over: the objectValues of the homepages folder. Since we've just added homepages the sequence is empty, but we can view the directory now:

Figure 2.4. Empty homepage

Empty homepage

(I've left the frames-based view of the ZMI.)

With that, we've just plucked another name out of a hat. So far, we have title_or_id, document_title and now objectValues. The mystery has to end! Let's go looking for the provenance of these names.

Asking the Oracle: what is in scope, and where is it documented?

This is one of Zope's trickier aspects. On every Zope object that subclasses Acquisition.Implicit (which is pretty much everything you can add via the ZMI, as well as the execution context of scripts and templates) all the objects, methods and properties of its containing objects, up to the ZODB root object, are in scope. In addition to that, of course, all the methods and properties that each of these objects has by virtue of inheritance is also in scope. Access to this cornucopia of functionality is mediated by Zope's security machinery, which protects access on class, method and property level.

Acquisition

Acquisition is an inheritance-like abstraction mechanism, where objects are bound to methods and attributes based on their containment context. Where inheritance is static and identical for all instances of a class, however, acquisition is dynamic: if you move the object, its context will change.

An analogy from the real world: You acquire your telephone number from your environment. When you're at home, your phone number is aquired from the phone in your house, and when you're at work, your telephone number is acquired from the one in your office.

Acquisition and inheritance are not opposed, but they have different areas of applicability. Inheritance belongs in the domain of the application developer who creates Python packages, while acquisition is convenient for configuration, TTW (Through The Web) work or for rapid prototyping.

Exploring the Zope API

Zope follows a common trend in the Open Source world: while there is a shortage of good, current books and HowTo's, the software is largely self-documenting if you are prepared to do a bit of looking. In this section, we'll look at two complementary ways of browsing Zope's API.

DocFinderTab

The first one we'll look at is DocFinderTab, a Zope Product that started out as Dieter Maurer's DocFinder, and was integrated into the ZMI by Stefan H. Holek. It uses Python's introspection facilities to derive the available documentation immediately from Zope's source. As Dieter says: “Unlike the API documentation which describes how thing should be, the DocFinder documentation describes (at least partially) sic how they really are: complete and precise.[4] We'll need to install it on the server before using it, so we'll do that in a moment.

Zope API in the online Help system

Zope provides an online help facility as part of its ZMI (you can see it in Figure 2.3, “Homepages created”; look for Help! at the top right). For various reasons it never really caught on as the authoritative channel for providing Zope documentation, but if you know what to look for it is useful.

Installing DocFinderTab

To do the installation, we need to log in on the server ..

For now, we just want to install DocFinderTab. Unpack it in the right place, and restart Zope:


zope-user@klippie zope-user $ cd zope-instances/zope1/Products
zope-user@klippie Products $ tar xzf ~/downloads/DocFinderTab-0.5.0.tar.gz
zope-user@klippie Products $ cd ..
zope-user@klippie zope1 $ ./bin/zopectl restart

The ZMI now sports a new tab:

Figure 2.5. DocFinderTab

DocFinderTab

You can go and look up the DocFinderTab product in the Control_Panel.

Using DocFinderTab

When you browse to the Doc tab, you'll see a long slew of class names: everything that is in scope from here. To narrow this down to what we're interested in, fill in objectValues in the "Filter" box (check "Expert" while you're at it). Expand the ObjectManager section to find the matching functions.

Figure 2.6. objectValues documentation

objectValues documentation

Now you know that objectValues is a method of the ObjectManager class from the OFS.ObjectManager module, that it's accessible to users with either Manager or Anonymous roles and the Access contents information permission, and that its call signature is self, spec=None. Knowing Python, you can tell that self is the object instance the method is being called on, and is passed automatically by Python, and that spec is an optional keyword argument. Most importantly, perhaps, you know which objectValues is being called (there are four in total).

Q:

Exercise: What don't you know yet?

A:

Docstring, what spec means.

Q:

Which four? How do they differ?

A:

Grep the source. ObjectManager (with a variant), SimpleItem, ZClass (two).

Using the online Zope API

Armed with this knowledge, let's consult the Zope online help. Follow the Help! link, go to the Search tab, search for "objectValues", and in the list of matching topics, choose "ObjectManager".

Figure 2.7. objectValues online help

objectValues online help

Here we find all the rest of the information we need in order to use objectValues.

You'll notice that DocFinderTab reveals the existence of objectValues_d, which isn't mentioned by the API documentation. You should also see that the signature looks different: what DocFinderTab reported as spec is called type here. Since DocFinderTab is generated from the code, you know that spec is the correct name.

Exploring further

Now we know that objectValues comes from the ObjectManager class, which "provides core behavior for collections of heterogeneous objects". What other core behavior do we have at our disposal? Remove the filter on objectValues from the DocFinderTab view, and browse to ObjectManager again. This time, you'll see a long list of methods, including, if you have "Expert" checked, the private methods that aren't accessible to untrusted (TTW) code. This list is a lot longer than the page of the online API, and many of the entries even have documentation :)

Q:

Find title_or_id and document_title.

A:

title_or_id is findable via the Doc tab and the online Help. It's implemented by OFS.Application, OFS.SimpleItem, Shared.DC.ZRDB.Connection and Products.ZGadflyDA.DA.

document_title is findable via the Doc tab and the online Help. Since it's a variable (set by the DTMLMethod class) it doesn't show up for DocFinderTab.

Templating continued: building a simple user directory.

To get a taste of TTW development using templates, some Python code, and a lot of acquisition, we're going to build a really simple user directory. It illustrates all of the mechanisms we've mentioned above. Actually, we'll build two variations of the directory: a flat one, and a nested one. And then we'll build them in Zope Page Templates.

OK, where did we leave off? Oh yes: Dynamic Homepage View that page again; the URL should be something like http://server:8080/templating/directory/index_html depending on what we called our server and how closely you've been following along. Let's give the template some content to render. The first blank to fill is directory_title. Add that as a string property of the directory object, from its Properties tab:

Figure 2.8. Added 'directory_title' property

Added 'directory_title' property

View the template again, and the browser's title bar will have changed.

Next, add some content in homepages. We can see that the template expects title and body, so we'll have to provide them. Add some Folder instances.

Figure 2.9. Some new homepage folders created

Some new homepage folders created

View them, and you'll see the sidebar appear:

Figure 2.10. Viewing homepages

Viewing homepages

Browse to Anne's folder, and add a DTML Method called body, and give it some content.

Figure 2.11. Added 'body'

Added 'body'
Q:

Can you think of another method we might have used to provide body?

A:

It could simply have been set as a property of the folder. The only thing to watch out for in that case, is that it will no longer be a child of the homepage folder.

Now, visit the View tab on Anne's folder (not the body), and you'll see the template with all three content areas in index_html filled.

How this works

The default view of a folder is index_html. Zope doesn't find this in anne (the folder we're viewing), so it travels up the acquisition tree, looking in homepages and finally finding it in directory.

When it renders this template, it does so in the context of Anne's folder: it starts looking for body, and finds it immediately contained in the folder. title is found as a property of the folder. directory_title is found two folders up, as a property of the directory folder.

Making the sidebar active

You'll have noticed the sidebar is pretty useless at the moment: it isn't linked. Let's link it up. Edit index_html so that the sidebar looks like this:

 
<ul> 
  <dtml-let my_id="id">
    <dtml-with homepages> 
      <dtml-in objectValues>
        <li>
        <dtml-if "id == my_id">
          <dtml-var title_or_id>
        <dtml-else>
          <a href="&dtml-absolute_url;"><dtml-var title_or_id></a> 
        </dtml-if>
        </li>
      </dtml-in>
    </dtml-with>
  </dtml-let>
</ul>
Q:

Explain what that dtml-let does, and why it's necessary.

A:

It pushes an identifier onto the namespace stack. Outside the dtml-with, id resolves to the currently browsed homepage folder.

In this snippet we see the DTML entity notation syntax for the first time. This is an alternative notation for the dtml-var tag that's useful for compact value interpolation, e.g. for filling in the values of HTML attributes.

Changing the format of body

Above, we added the content of body as HTML, and it was rendered as such. That's OK if we have savvy users who know markup. For those who don't --- Zope supports StructuredText, a Wiki-like plaintext format that uses significant whitespace and a couple of other conventions to guide formatting. We'd like to provide both options. Change index_html to accommodate this. Here's one possibility:

 
<dtml-if body>
  <dtml-if stx> <dtml-var body fmt="structured-text">
  <dtml-else>   <dtml-var body>
  </dtml-if> 
<dtml-else> Nothing yet
</dtml-if>

Test this with a body such as:


Heading

    The indented text following a heading, is a paragraph.

    - Lists

    - are

    - possible.
Q:

Why isn't there a missing attribute on the dtml-var anymore?

A:

We're using 'dtml-if' instead to check for 'body'. The missing would have had to be repeated for each dtml-var.

Q:

Where should the stx property be defined?

A:

On the folder (to only set the formatting for this user); on homepages (to set the formatting for all users).

Make body editable in the directory view

First, we need to offer users the option to edit their homepages. Revisit 'index_html':

 
<dtml-if body>
  <dtml-if stx> <dtml-var body fmt="structured-text"> 
  <dtml-else>   <dtml-var body>
  </dtml-if>
  <form action="edit" method="POST">
    <button name="edit" type="submit">Edit</button></form> 
<dtml-else> Nothing yet
  <form action="add" method="POST">
    <button name="add" type="submit">Add</button></form>
</dtml-if>

You can try this out. Can you predict what you'll see?

When you've done that, add the add and edit forms as DTML Methods, siblings of index_html, with content such as:

 
<html> 
<title> Adding
</title>
<body>
    <form action="do_add" method="POST">
    <textarea name="edited_body"></textarea>
    <button type="submit">Submit</button> 
    </form>
</body>
</html>
Q:

The edit method is almost identical, just change the wording and make it sticky (prefill the current value).

A:

<textarea name="edited_body"><dtml-var body></textarea>.

Q:

Why isn't a missing necessary?

A:

The dtml-if in index_html guards.

The last step is to do the work. Since this entails a couple of Python calls, we'll do it in a Script (Python)[5], which we'll also add as sibling of index_html. Here's do_edit:


## Script (Python) "do_edit"
##bind container=container
##bind context=context
##bind namespace=
##bind script=script
##bind subpath=traverse_subpath
##parameters=REQUEST
##title=
##
context.body.manage_edit(REQUEST.edited_body, context.body.title)

REQUEST.RESPONSE.redirect(context.absolute_url())

And here's do_add:

 
## Script (Python) "do_add"
##bind container=container
##bind context=context
##bind namespace=
##bind script=script
##bind subpath=traverse_subpath
##parameters=REQUEST
##title=
##
context.manage_addProduct['OFSP'].addDTMLMethod('body')
context.body.manage_edit(REQUEST.edited_body, context.body.title)

REQUEST.RESPONSE.redirect(context.absolute_url())

We're done!

So far, we've only done this (adding and editing) via the ZMI; here we're doing it programmatically. Everything the ZMI can do, we can do in a script.

Figure 2.12. Editing Anne's homepage

Editing Anne's homepage
Q:

Where does addDTMLMethod come from? What is its signature?

A:

We could find it by examining the source of the "Add DTML Method" screen in the ZMI.

Q:

Where in the ZMI could we find out about addDTMLMethod and manage_edit?

Q:

What does context.body.title mean, and why do we include it?

A:

Acquires the container's title. It's compulsory in the signature of manage_edit.

Q:

Restrict edit rights to a user's own folder.

A:

There are various possibilities: match username in do_edit (if we do this, we'll still be vulnerable to a user who can guess the URL to call manage_edit) - only add user in an acl_users in their folder - add User role; give user local role in their own folder.

Q:

Split out header/footer/sidebar. Override the looks per user. Use header/footer in add/edit forms.

Q:

Add siblings to body

Q:

Handle submitting an empty body

Q:

Delete body

Zope security

At the moment, our edit form works because we're logged in as users with the Manager role. For the course, we setup the server to be reachable under different names, that reflect the kind of role we'd like to exercise. Now, we'd like to visit as anonymous users. The URL for anonymous usage should be something like: http://anon:8080/templating/directory/homepages/ben

Trying to add or edit now pops up an authentication dialog, and upon canceling returns an error such as this:


Error Type: Unauthorized
Error Value: You are not allowed to access 'manage_edit' in this context

The last time we encountered such an error, we found more information in the error_log. However, since users will often be unauthorized to access resources in the course of normal operation, this is one of three exception types that are ignored by default. If you like, you can remove Unauthorized from this list --- it won't leave you much the wiser, though. In order to debug security problems, run, don't walk, and install Shane Hathaway's VerboseSecurity product. Restart Zope, and retry. Now, after canceling the dialog, we have a much more helpful error message:


Error Type: Unauthorized
Error Value: Your user account does not have the required
permission. Access to 'manage_edit' of (DTMLMethod instance at
40edc1a0) denied. Your user account, Anonymous User, exists at
/acl_users. Access requires Change_DTML_Methods_Permission,
granted to the following roles: ['Manager']. Your roles in this
context are ['Anonymous'].

I don't think anyone could wish for more!

You can browse the configuration of roles and permissions at the "Security" tab of any folder. Going to e.g. http://server:8080/templating/manage_access does not help much, since only the "Acquire permission settings?" column is checked. To see what the setting are that are being acquired, we need to visit the "Security" tab of the root Zope folder, e.g. http://server:8080/manage_access. Here, you'll see the Change DTML Methods permission with an empty checkbox in the column for Anonymous.

Note

Roles are cumulative, and everyone has at least the Anonymous role. In the default configuration, therefore, the Authenticated role doesn't have any more permissions that the Anonymous one.

Figure 2.13. The Security tab of the root folder

The Security tab of the root folder

If you check that box, users with the Anonymous role will be able to change any DTML Method --- which is not what we want. We can apply security in a much more precise way by configuring individual Zope objects, instead of roles globally. To do this, browse to the Python scripts do_add and do_edit, and visit their "Proxy" tab. In each case, assign Manager as proxy role for the method. (This is similar to sudo or setuid functionality on Linux.)

Figure 2.14. Adding a proxy role for do_add

Adding a proxy role for do_add

Now, anonymous users can only effect changes to body objects via the two do_ methods. They can still, of course, edit each other's content. How do we stop this?

Configuring permissions in context (local roles)

We'll add a user folder to contain all the users of the directory. Don't give the users any roles.

Figure 2.15. Adding a user folder for the directory

Adding a user folder for the directory

Next, remove the proxy role from the do_edit method. To give them back the ability to edit their own pages, browse to each user's folder, visit the "Security" tab, and give the Change DTML Methods permission to the Owner role.

Figure 2.16. Adding a local role to a folder

Adding a local role to a folder

Now, these folders were created before there were user objects corresponding to them, so they'll be owned by a manager user. Follow the "local roles" link, and give the user the Owner local role. As you do this for each user, they'll be able to authenticate and edit in their own folder.

Q:

What happens when there is a body method in homepages or in directory?

A:

Users who don't have a homepage won't be able to add one, and won't be able to edit the acquired one.

Q:

Fix index_html to check the containment of the acquired body.

Explicitly checking for a permission

OK, now editing is restricted to the owner of the folder. However, if a user hasn't added a page for themselves yet ("Nothing yet" is still being displayed), anyone can still do it for them. We could address this issue by removing the proxy role from do_add and assigning the requisite permission to Owner in the user's folder.

Q:

What is the requisite permission?

A:

"Add Documents, Images, and Files" (from VerboseSecurity)

We don't necessarily want to do this, as it is much less restrictive than what we currently have (the ability to only add one particular DTML Method): it would mean that a clever user could add any number of Documents, Images, and Files to their folder. The quickest way to deal with this in the TTW paradigm that we're working in now, is to check for the Change DTML Methods permission in the do_add method. The relevant API for this belongs to the user's class, which is AuthenticatedUser (you can look it up in the Zope online help, or at http://zope.org/Documentation/Books/ZopeBook/2_6Edition/AppendixB.stx). A check that works is:


if not REQUEST.AUTHENTICATED_USER.has_permission(
        'Change DTML Methods', context):
    raise 'Unauthorized'

context.manage_addProduct['OFSP'].addDTMLMethod('body')
context.body.manage_edit(REQUEST.edited_body, context.body.title)

REQUEST.RESPONSE.redirect(context.absolute_url())

A nested directory

Our directory only makes provision for a flat collection of homepages. Let's change it so that they can be nested. We want to keep the flat one around, though, so let's rename it, and add a sibling folder. While we're at it, tidy away the default_content method and copy over the existing homepages (we might as well use them). Further, move up the user folder to extend the access of the users to both directories. Lastly, we're going to hope that nothing changes as far as editing and adding is concerned, and move those templates and scripts up one directory too, so that they can be acquired from both directories.

For variety's sake, let's leave the ZMI for a moment and inspect it via FTP. Here's what the hierarchy should look like once we're done:


zope-user@klippie zope1 $ ncftp -u jean -P 8021 server
NcFTP 3.1.2 (Jan 28, 2002) by Mike Gleason (ncftp@ncftp.com).
Connecting to server
......
Logged in to server
ncftp / > cd templating/
ncftp /templating > ls -R
acl_users
add
directory_deep
directory_deep/homepages/anne/body
directory_deep/homepages/ben
directory_deep/homepages/cyril/body
directory_deep/homepages/denise
directory_deep/index_html
directory_flat
directory_flat/homepages/anne/body
directory_deep/homepages/ben
directory_flat/homepages/cyril/body
directory_deep/homepages/denise
directory_flat/index_html
do_add
do_edit
edit

It so happens that all that is required is for three lines to be deleted, and four to be added. Here is the difference:


diff -ru .../directory_flat/index_html .../directory_deep/index_html
--- .../directory_flat/index_html 2004-07-05 14:54:44.000000000 +0200
+++ .../directory_deep/index_html 2004-07-05 14:54:44.000000000 +0200
@@ -8,15 +8,16 @@
 <div class="sidebar">
 <ul> 
 <dtml-let my_id="id">
-<dtml-with homepages>
-<dtml-in objectValues>
+<dtml-unless "id=='homepages'">
+<li><a href="..">Up</a></li>
+</dtml-unless>
+<dtml-in "objectValues('Folder')">
 <li>
 <dtml-if "id == my_id"><dtml-var title_or_id>
 <dtml-else><a href="&dtml-absolute_url;"><dtml-var title_or_id></a>
 </dtml-if>
 </li>
 </dtml-in>
-</dtml-with>
 </dtml-let>
 </ul>
 </div>
Q:

This is true as far as it goes, but it does not address the role of acquisition when adding or editing subfolders. What bug remains? Fix the bug.

A:

A nested folder inherits the homepage of its parent; you cannot add a body for it.

A:

Use aq_explicit.

Q:

Split out sidebar as a separate DTML Method (one for each variant), use it from index_html, and share index_html between the deep and the flat directories.

There's one line that deserves a second look in the above changes, namely the objectValues call. At issue are the differences between these four lines:


<dtml-in objectValues>
<dtml-in "objectValues">
<dtml-in "objectValues()">
<dtml-in expr="objectValues()">

Remember, if the target parameter is a name, not an expression, DTML will try to call it if it is callable. If the target is passed as an expression (either by using the expr attribute or simply by quoting it) it is evaluated as a Python expression. The second line above, therefore, will return a method object, instead of the result of calling the method.

Introducing the prompt: Zope from interactive Python

Let's make this difference visible. To do this, let's drop all the intervening layers of the ZMI and DTML, and get down to plain Python. Stop Zope, and climb in:


zope-user@klippie zope1 $ ./bin/zopectl stop
zope-user@klippie zope1 $ ./bin/zopectl debug
Starting debugger (the name "app" is bound to the top-level Zope object)
>>> app.templating.directory_flat.homepages.objectValues  
<bound method Folder.objectValues of <Folder instance at 40f15f50>>
>>> app.templating.directory_flat.homepages.objectValues()
[<Folder instance at 40f1f140>, <Folder instance at 40f15e60>,
<Folder instance at 40f1f1a0>, <Folder instance at 40f1f110>]
>>> folders = app.templating.directory_flat.homepages.objectValues() 
>>> [f.title_or_id() for f in folders]
["Anne's page", 'The Page Of Ben', 'In the middle',
'Oh Denise doo-be-do']
>>> [f.title_or_id for f in folders]
[<bound method Folder.title_or_id of <Folder instance at
40f1f140>>,
<bound method Folder.title_or_id of <Folder instance at 40f15e60>>,
<bound method Folder.title_or_id of <Folder instance at 40f1f1a0>>,
<bound method Folder.title_or_id of <Folder instance at 40f1f110>>]
>>> 

Note

We may be running our own ZEO instances by now. If so, we won't need to stop Zope before connecting to it, and Zope will continue to serve web requests while we're browsing it.

Reference

The best HowTo on this topic is Ken Manheimer's ConversingWithZope though it refers in part to older versions of Zope.

Failings of DTML

Looking back, we can evaluate the extent to which DTML fulfills the goals of templating. I'd have to say it's a qualified success.

  • While DTML provides the mechanisms for seperating presentation and logic, it leaves the door wide open: it includes branching functionality, and it's necessary to use Python expressions to accomplish many things.

  • While DTML provides a low barrier to entry for a designer needing to turn a static design into a template, it requires knowledge of the Zope API to make it work.

  • It looks like markup, but it isn't. It's neither well-formed nor valid XML or HTML. This means that general-purpose tools can't be used to edit or parse it.

  • Its use of acquisition is pervasive and implicit, and its calling conventions are rather magic. It's easy when using DTML to be confused about where exactly a value is coming from.



[1] DTML is regarded as a failed attempt, and a lot of the DTML that you'll encounter in the wild flout this ideal, but that's how the "templating" bit should be understood, anyway.

[2] We're already dealing with the Web technologies ( HTML, CSS, Javascript), and DTML, which is subtly different.

[3] What's the difference between a DTML Document and a DTML Method? The Document represents a content object that may have properties of its own, while the Method represents a method of its parent object.

[4] This will show you all the classes, methods, security configuration and signatures, but you won't see any attributes.

[5] A content type for storing restricted Python code in the ZODB. Since Scripts may be edited TTW, unlike filesystem Python code, this code is restricted (e.g. recursion depth and loop size is checked) and only has access to authorised modules. See: http://server:8080/Control_Panel/Products/PythonScripts/Help/ModuleAccess.stx