Programming Language for Old Timers

by David A. Moon
February 2006 .. September 2008

Comments and criticisms to dave underscore moon atsign alum dot mit dot edu.

Previous page   Table of Contents   Next page

Template Syntax

You can construct a P-expression any way you want. One way is to construct the objects directly by calling constructor functions. Another way is to construct a sequence of tokens, wrap it in a token-stream, then feed that to parse-expression. An easy way to do the latter is using a template. A template can construct any sequence of tokens to be parsed. The ` macro accepts a template enclosed in a pair of ` characters, expands the template into a sequence of tokens, and returns the sequence. The sequence can then be wrapped in a token-stream and fed to any parse function to produce an expression or other desired object. Typically, the token sequence is returned as the result of a macro's parse function. The macro expander will wrap it in a token-stream and pass it to parse-expression.

Most macros use templates to construct the P-expression that will be the expansion of the macro.

Note that the ` macro is not typically used to produce the direct result of a defparser or defsyntax, since that result is usually expected to be a P-expression, not a sequence of tokens waiting to be parsed. Typically the body of a defsyntax will call the constructor function for a P-expression. But it could also use the ` macro to construct a sequence of tokens, wrap it in a token-stream, and pass it to a parser function.

In a template the following have special meaning, while other tokens merely insert themselves. A newline in a template inserts a newline. Indentation and source-locations in templates are preserved as explained below.

{ ... }* repeat the "..." subtemplate zero or more times
{ ... }+ repeat the "..." subtemplate one or more times
& delimits the separator part of a repeated subtemplate; higher binding power than { }. If an additional repeat will occur, insert the separator first. Otherwise skip the separator.
^ line break -- insert a newline.
? followed by a name. Insert the definition of that name.
?= followed by a name. Insert a name-in-context with that spelling in the previous context, which is obtained by calling get-previous-context.
` not valid in a template, so the template ends here.
\ prevents the next token from being recognized as one of the special ones listed here. Also causes assignment of a context to the next token to be delayed.

When a value inserted by ? is false, nothing is inserted, which might not be what you want. You can use def var2 = var or default, and use ?var2 rather than ?var in the template.

When a value inserted by ? is a sequence, each element of the sequence is inserted in turn. If the sequence is empty, nothing is inserted. In the case of nested sequences, this flattening process is recursive.

When ? appears inside of a repeat, then if the value is a sequence each repetition uses one element of the sequence, otherwise each repetition uses the same value. When repeats are nested, sequences are nested to the same number of levels. A repetition repeats as many times as necessary to exhaust all the sequences used with ? inside it. If one sequence is smaller than another, the smaller sequence is treated as an empty sequence once it has been exhausted. If all ? values are sequences and all sequences are empty, then the * or + following the } takes effect and controls whether there are no repetitions or one.

A simple-name in a template becomes a name-in-context in the expansion of the template. The context is the context of the macro definition, not the context of the macro call. This applies to punctuation as well. Each expansion of a macro creates a new context. A template does not itself create a new context, it depends on having a visible definition of the name macro-context. The defmacro macro provides that definition.

Every occurrence of a given spelling in templates during a given macro expansion is a name in the same context. But each expansion of a macro creates a new context. You can also create a new context explicitly using the unique-macro-context function and locally define macro-context to it. You can also call the name-in-context constructor with a string and a context.

A template only converts a simple-name to a name-in-context in the macro-context context. When a name-in-context appears in a template, it retains its existing context. This makes macro-defining macros work.

A name-in-context produced by a template will be reduced to a simple-name with the same spelling if it is parsed in a syntactic position that requires an absolute particle or literal data rather than requiring a name that identifies a definition. (See the Names, Particles, and Tokens section for the definition of particle.)

To create a visible name in a template, use name(spelling, get-previous-context()) to create a name that will be visible to the macro's caller. Define a local variable to be that name object, then substitute it into a template via ?. As an abbreviation, you can use ?=name in a template to do the same thing. This works when the macro's caller is a template, where a name in the correct context is required.

The \ is especially useful when using a ` template to construct a ` template. It allows special tokens such as ? to be inserted into the constructed template. It also allows names to be inserted with the macro context of the inner template rather than the macro context of the outer template.

The code generated by the ` macro expands the template to a sequence of tokens.

The indentation for each newline token in a template expansion depends on the origin of that token:

The source-location for each token in a template expansion depends on the origin of that token:

When the source-location in a template expansion changes, and the next token does not implement the source-locator protocol, the template inserts a special source-location-marker object that implements the source-locator protocol. A token-stream skips over source-location-marker objects, only using them to update its current source-location.

Previous page   Table of Contents   Next page