Macros are the units of language definition. A macro defines the syntax of a construct by specifying the procedure for parsing that construct, and defines the semantics of a construct by expanding into the implementation of that construct.
The expansion of a macro can be a sequence of tokens to be parsed, or a single expression ( see Expressions ) which does not need to be parsed again. Because a token is an expression, a newline, or a keyword, the expansion can be a sequence of already-parsed expressions.
If the expansion of a macro is a definition, or a sequence of expressions that includes some definitions, the appropriate scoping will be imposed by parse_body when the macro call is inside a body.
The typical macro uses a pattern ( see Patterns ) as a concise way to specify how to parse the syntax of the construct, and uses a template ( see Templates ) as a concise way to construct the expansion. However, either or both parts of a macro can be written as procedural code instead.
The syntax of a prefix macro definition is
$defmacro anyname [ pattern ] "=>" bodyWithin the body, definitions corresponding to pattern variables in pattern are in scope, along with the following hard-wired definitions. All of these definitions are constants.
| Name | Type | Purpose |
|---|---|---|
| lexer | token_stream | source of tokens |
| indentation | integer | indentation at start of macro call |
| scope | scope | current scope |
| modifiers | set![name] | modifiers preceding macro call |
| context | macro_context | macro expansion's hygienic context |
| previous_context | macro_context | module | false | macro call's hygienic context |
If you prefer to write the parsing as procedural code, simply omit the pattern and parse from lexer in the body.
The modifiers set contains names of modifiers, rather than the modifier keywords themselves, for ease of lookup. The modifiers set does not include modifiers specially handled at top level, such as export:. Any modifiers handled by the macro must be removed from modifiers using remove!. Any modifiers remaining after all rounds of macro expansion have been completed will be reported as an error.
defmacro could have been defined by
defmacro defmacro anyname [ pattern ] "=>" body =>
;; Create anaphoric names
def lexer = name("lexer", previous_context)
def indentation = name("indentation", previous_context)
def scope = name("scope", previous_context)
def modifiers = name("modifiers", previous_context)
def new_context = name("context", previous_context)
def previous_context = name("previous_context", previous_context)
;; Translate the pattern into parsing code
def parser = pattern and translate_pattern(pattern, lexer, indentation, scope)
;; Construct the expansion
`def \\$anyname ; ensure it's defined as a bundle
add_macro_nature(\\$anyname, #prefix,
fun \\$anyname($lexer token_stream,
$indentation integer,
$scope lunar.scope,
$modifiers set![name],
$previous_context macro_context | module | false)
def $new_context = local_macro_context()
$parser
$body)`
Here translate_pattern does all the heavy lifting,
and add_macro_nature makes the bundle a prefix operator macro.
An infix macro is defined by defoperator with a macro: lhs_name pattern => body clause. The same definitions are in scope in the body as in a defmacro body.
In addition, lhs_name is in scope; its value is the expression on the left-hand side of the operator.
In addition, method_head? is in scope; its value is false if this infix macro is part of a normal expression, or true if it is part of a method head. In the latter case, the expander can choose to parse a different syntax (with type suffixes) and return a method_head datum. If it instead returns a call_expression it will be translated into a method_head but cannot contain type suffixes on the right-hand side.
At compile time there is a name_in_context subclass of name used by hygienic macros. The result of name(spelling, context) is a name_in_context unless context is false or a name with no context.
The macro_context class provides a unique context for name_in_context and ensures that there is only one name_in_context with any given spelling in any given context.
These could have been defined by
defclass name_in_context \
constructor: _name_in_context(spelling string,
context macro_context | module) \
name(spelling)
context = context
;; The context of a name that is not a name_in_context is false
def (name name).context false
defclass macro_context(parent_scope scope | module)
parent_scope = parent_scope
names = map![string, name_in_context]()
defmacro local_macro_context "(" ")" =>
quotation(macro_context(scope))
;; Name 2-parameter pseudo-constructors
def name(spelling name, context) name(spelling.spelling, context)
def name(spelling sequence[character], context) name(string(spelling), context)
def name(spelling string, context name) name(spelling, context.context)
def name(spelling string, context false) name(spelling)
def name(spelling string, context macro_context | module)
def adjusted_spelling = string(downcase(spelling))
context.names[adjusted_spelling, default: false] or
context.names[adjusted_spelling] := _name_in_context(spelling, context)
Lunar macros are "hygienic," which means that they provide referential transparency. In other words, when a name in the expansion of a macro came from the macro call, the name refers to the same definition in the expansion as it would at the call site. When a name in the expansion of a macro came from the macro definition, the name refers to the same definition in the expansion as it would at the macro definition site. Each name is evaluated in the context appropriate to its place in the original source code before macro expansion.
The purpose of referential transparency (or hygiene) is to prevent the meaning of programs from being changed by accidental name clashes where a macro and its caller both happen to choose the same name for different things. Thus local definitions introduced or referenced by macros are independent of local definitions introduced or referenced by callers of macros. Furthermore, free variables in the expansion of a macro placed there by the macro itself refer to definitions in the scope where the macro was defined, not in the scope where it was called.
When a macro expansion contains a call to another macro, more than two contexts are in play. A name can be in the context of the original macro call, in the context of the definition of the first macro, or in the context of the definition of the other macro.
When macro-defining macros are used, more than two contexts are also in play. A name can be in the context of the final macro call, in the context of the call to the macro-defining macro which is a macro definition site, or in the context of the site where the macro-defining macro was defined.
It is not as simple as that when a macro expansion introduces local definitions. A local definition can be invisible, visible, or anaphoric.
An invisible local definition is one that a macro introduces for its own purposes. Names in the context of the macro call should not be affected by this definition. This is accomplished by naming the definition with a name in a unique context that is created fresh for each macro invocation. A typical example would be the or infix macro, which defines temp to the value of the left-hand side so it can reference it twice, once to test it against false and again to return it.
A visible local definition is one that a macro introduces for a purpose that is part of the external interface of the macro. Names in the context of the macro call should be affected by this definition. This is accomplished by naming the definition with a name in the context of the macro call site which is supplied as part of the macro call. A typical example would be the for macro, which defines iteration variables named in the clauses of the macro call.
An anaphoric local definition is the same as a visible local definition except that the name to use is a convention of the macro rather than being supplied explicitly as part of the macro call. This is accomplished by constructing a name with a known spelling and the context of the macro call site. A typical example would be the defmacro macro, which defines lexer, etc.
Those are the requirements. The implementation is:
The contexts necessary to implement referential transparency are represented by instances of the class macro_context. The choice of context allows definitions to be visible or invisible. When looking up a name in a local scope, a definition is visible if the definition's name has the same spelling (case-independent string equality) and has the same (eq) context. When no definition is found in any of the current nested local scopes, and the name's context is a macro_context, lookup ignores the current global scope and continues looking up the name stripped of its context in the scope contained in a slot of the macro_context. This is the scope where the macro was defined.
In the body of a macro, context is defined as a unique macro_context that is a suitable context for invisible local definitions.
In the body of a macro, previous_context is defined as the appropriate context for anaphora. This is the context of the name of the macro in the macro call. It is false unless the macro call was generated by a macro call.
The pseudo-function local_macro_context returns a newly created macro_context that captures the local scope in which the function call was compiled. This is implemented by defining local_macro_context as a macro with the syntax of a function call.
A module can also be used in place of a macro_context object as the context of a name and and definition lookup goes directly to that module.
The value false can be used as a context. When a definition is not found in the current local scope and its parents, false context means to look for a global definition in the current module. This is the default behavior, of course.
Every name in the source code model carries a context, which is false, a macro_context, or a module. Names that appear directly in source code have false context. Names with a macro_context are only created by macros. Each macro call creates a new, unique macro_context object whose referenced scope is that of the macro definition site.
Can a name_in_context in a local macro capture the local definitions in scope where the macro was defined? Yes, but it is an error to reference such local definitions when they are not accessible, other than macros and constants. A name_in_context must only reference a local definition created by the macro expansion, or a macro or constant locally defined in a scope containing the macro definition, or a global definition in the module current when the macro was defined, if the macro is called from outside the scope where it was defined.
It is fairly easy to see that this mechanism is sufficient to make macros hygienic without being verbose or getting in the way.
Previous page Table of Contents Next page