ACS Java Precompiler

(necessary infrastructure for the ACS database api)

by Luke Pond

I. Introduction

This document outlines the requirements for an extensibility mechanism for the java language, allowing the development of macros that generate java code. The system will be sufficient to implement valuable ACS innovations such as the database api and the page contract, so that productivity and legibility do not decline as we port TCL code to Java.

II. Vision Statement

By providing the capability to reuse common paradigms from ACS/TCL directly in Java, this system leverages our existing investment in the ACS, makes the tasks of porting and learning the new environment easier for developers, and does not interfere with any of the tenets or advantages of the J2EE standard platform. We believe that although there is great demand for an intermediate layer to improve the ease of use of JDBC, there is no other system with the power or simplicity of our TCL database api.

III. System/Application Overview

Translation
The system is in the java package com.arsdigita.db. There is a compiler which may be invoked from the command line to translate macros it finds within files containing java code. The same compiler is invoked by the ACS Request Processor when it receives a request that resolves to a file with the .ajp extension, in which case the generated file is given the .jsp extension.
Writing a new macro
The compiler takes as input a grammar file that specifies the macros that it should expand. The java package com.arsdigita.db.ast contains classes that perform the code generation for each macro at compile time.

IV. Use-cases and User-scenarios

An ACS developer will peruse the list of macros available to them when writing either Java library code or JSP business-logic pages. As they write code, they will make use of macros that implement the database API in order to query or update the database. When writing a business-logic page, they will use the ad_page_contract and ad_return_template macros to specify the behavior of each page.

When testing or deploying new code, they will use a build process that invokes a command-line version of the precompiler to prepare their java library code for execution.

V. Competitive Analysis

JSP tag extension mechanism
The JSP environment provides a means by which to implement libraries of custom tags that have a well-defined runtime interface. When serving a page, the custom tags can write to the output, create scripting variables for use on the rest of the page, or perform looping or control structures. We created an extensive custom tag library for ACS Java 3.4, but we were disappointed by it in the following ways:
- No ability for the tag to inspect variables in the scripting environment.
- No interface with the JSP-translation mechanism, meaning that nothing interesting can be done at compile time.
- JSP-only solution: if the tags implement a database API, that API is not available to programmers working on java libraries.
- Awkward syntax when XML tags are mixed with JSP scriptlets
WebMacro

VI. Related Links

ACS Database API

VII.A Functional Requirements

10.0 On the fly translation

The request processor will be one client of the system, and it must be able to use the system to translate a page containing macros into a legal JSP.

20.0 Command-line translation

The macros will also be used in java library code, and there must be a straightforward process to translate a file containing macros into a legal .java file.

22.0 Build process

There must be a simple process to build all library code in the ACS, which includes the macro translation step. Any extension to the ACS must be able to easily plug into the existing ACS build system.

30.0 Metadata for macros: the grammar file

The canonical list of macros and the syntax with which they may be invoked must be stored in a text file with a simple layout.

32.0 Generating documentation for the macros

An HTML document specifying the syntax for all macros must be automatically generated from the grammar file.

34.0 Macro syntax

The macros are intended to look like commands in ACS/TCL, so the contents of the grammar file are roughly equivalent to TCL procedure declarations: a macro has a name and a list of arguments. Arguments can be optional, and named arguments are supported.

40.0 Macro implementation class

There must be one java class that implements code generation for each macro. The mapping between a macro and its implementation class is in the grammar file.

50.0 Requirements for the parser

The system must contain a parser which builds an abstract syntax tree from an input file. A tree structure is necessary because we intend to create macros for control structure, so a macro's arguments may contain other macro invocations.

50.20 Reliance on the grammar file
The parser must obtain its information about available macros by reading the grammar file, and must not contain special cases for parsing any particular macro.
50.30 Code between the macros
Any part of the input file that is not part of a macro invocation must remain unchanged in the output file.
50.40 Recognizing the beginning of a macro
Following the rules for TCL commands, the parser must recognize a macro invocation if the macro name is the first word on a line, or if it appears elsewhere on the line following a square bracket "[".
50.50 Recognizing the end of a macro
As in TCL, a macro invocation must be on a single line, except for the caveat that it may have arguments that are quoted with curly braces and span multiple lines.
50.60 Validating macro syntax
After reading a macro invocation, the parser must ensure its arguments are valid according to the grammar file.
50.70 Recursion
The grammar file may specify that one or more of a macro's arguments is a code block, in which case the parser builds an abstract syntax tree for the contents of that argument.

VII.B Quality requirements

The precompiler will be a fundamental part of the entire ACS Java system. We must be certain that bugs don't creep into the system and waste the time of developers using it.

10.0 Regression test suite

There should be a complete set of tests of the functionality of both the macro parser and the generated code. These tests should be run before any checkin to the precompiler code.

20.0 Performance

High performance is not absolutely necessary, but the precompiler should manage to translate a file at least as fast as javac. Like javac, it has to accept multiple files on the command-line so that a build script doesn't have to invoke a new java process for each file.

VIII. Revision History

Document Revision #	Action Taken, Notes	When?	By Whom?
0.2	Sharpened requirements for precompiler, removed design info.	10/31/2000	Luke Pond
0.1	Creation	10/31/2000	Luke Pond

luke@arsdigita.com

Last modified: 2001/01/21 01:38:18