git » jacl.git » commit 05878bf

diff --git a/paper/jacl-els-2020.tex b/paper/jacl-els-2020.tex
index 0597683..9bfcdf9 100644
--- a/paper/jacl-els-2020.tex
+++ b/paper/jacl-els-2020.tex
@@ -153,11 +153,6 @@ by a JavaScript program. In order to focus my comparison, I will
 survey only Lisps that have either demonstrated industrial utility or
 that implement a significant subset of Common Lisp.
 
-\begin{itemize}
-  \item They implement a significant subset of Common Lisp.
-  \item They approach the challenges of large-scale development.
-\end{itemize}
-
 \subsection{Parenscript}
 
 % https://web.archive.org/web/20051122141019/http://blogs.bl0rg.net/netzstaub/archives/000525.html
@@ -183,31 +178,12 @@ JavaScript-behaviors to static Web sites.
 
 SLip [ref] is perhaps the most ambitious Common Lisp-on-JavaScript
 system created to date. It offers a stunning array of powerful
-features.
+features including a self-hosting compiler, a full set of control
+operators, JavaScript FFI, tail-call optimization, green threads, and
+perhaps most impressively, a resident Emacs clone.
 
-Based originally on the compiler and bytecoded VM presented in PAIP
-[ref], SLip features include:
-
-\begin{itemize}
-  \item Self-hosting compiler
-  \item \texttt{TAGBODY}, \texttt{THROW}, \texttt{CATCH}, \texttt{BLOCK}, \texttt{RETURN}, \texttt{UNWIND-PROTECT}
-  \item Foreign Function Interface (FFI) to JavaScript
-  \item Tail-call optimization
-  \item Green threads
-  \item Package system
-  \item TinyCLOS [ref]
-  \item Resident Emacs-like editor and interactive development
-    environment
-\end{itemize}
-
-Because of interpretation overhead, performance relative to JavaScript
-is necessarily worse.
-
-The compiler features a peephole optimizer based on the one in
-PAIP. There is no way to tree-shake the Lisp image given an
-entrypoint, or to otherwise produce standalone JavaScript executable
-that a static whole-program optimizer such as Google Closure could
-work with.
+SLip is based originally on the compiler and bytecode interpreter
+presented in PAIP [ref].
 
 Lisp files may be batch-compiled to FASLs. FASLs represent code as
 JavaScript code instead of as Lisp data. The browser is able to load
@@ -224,108 +200,13 @@ project goal of facilitating large-scale industrial SPA development.
 Of existing Common Lisps, JSCL is the one aligned most closely with
 the JACL project goal. Unlike Parenscript, JSCL is a genuine Lisp
 system. And unlike SLip, JSCL compiles directly to JavaScript instead
-of to an interpreted bytecode.
-
-Its features include the following:
-
-\begin{itemize}
-  \item Self-hosting compiler targeting JavaScript
-  \item \texttt{TAGBODY}, \texttt{THROW}, \texttt{CATCH}, \texttt{BLOCK}, \texttt{RETURN}, \texttt{UNWIND-PROTECT}
-  \item Tight integration with JavaScript
-  \item Multiple values
-  \item \texttt{SETF} places
-  \item \texttt{CLOS}
-\end{itemize}
-
-The JSCL system and its features are organized hierarchically. I will
-now investigate how the lowest level Lisp and JavaScript operations
-are supported by JSCL.
-
-\subsubsection{JavaScript interoperation}
-
-JSCL's support for JavaScript interoperation consists of the following
-conventions and facilities:
-
-\begin{itemize}
-  \item \textbf{Unboxed numbers}: JSCL fixnums and flonums both map to
-    JavaScript's native \texttt{number} type. This allows for numeric
-    performance competitive with JavaScript.
-  \item \textbf{Native functions}: JSCL functions compile to concrete
-    JavaScript functions that may be invoked from JavaScript.
-  \item \textbf{\texttt{\#\textbackslash j} syntax}: The pseudo-symbol
-    reader syntax, which I will elaborate on below.
-\end{itemize}
-
-\subsubsection{\texttt{\#\textbackslash j} syntax}
-
-The JSCL reader parses tokens like \linebreak\texttt{\#\textbackslash
-  j:window:console:log} as case-sensitive references to hierarchical
-objects within JavaScript's global environment. Case preservation is
-necessary because JavaScript identifiers are case-sensitive.
-
-In a value context, references evaluate to the referenced JavaScript
-object. For example, \texttt{\#\textbackslash j:window:console:log} evaluates to
-the same object that the JavaScript expression
-\texttt{window.console.log} does.
-
-In function position of operation forms --- as in \linebreak
-\texttt{(\#\textbackslash j:window:console:log $"$hello$"$)} ---
-references are taken as the name of a foreign function to call. The
-compiler generates code that coerces the arguments from Lisp to
-JavaScript types and applies them to the named foreign function.
-
-In the preceding example, the Lisp string \texttt{$"$hello$"$} is
-converted to a JavaScript string before being passed to the function \linebreak
-\texttt{window.console.log} for display in the JavaScript console.
-
-JSCL has a distinct string type because unlike Common Lisp strings,
-JavaScript strings are immutable.
-
-Argument type coercion is automatic, cannot be disabled, and can be
-defeated by compound structures. For example, should a JavaScript
-array containing Lisp strings be passed as an argument, the inner Lisp
-strings would not be converted to JavaScript strings.
-
-\subsubsection{Multiple values}
-
-Despite being native JavaScript function objects, JSCL functions
-require a special calling convention. As JavaScript functions, JSCL
-functions always require one more argument than they declare. This
-required first argument is the function to which any returned values
-should be applied.
-
-This argument is the basis of a clever and robust mechanism for
-supporting multiple value returns.
-
-The JSCL compiler supplies this argument at function call sites
-automatically. The value of the supplied function depends on the
-calling context. If the called function is called in a context in
-which only a single value is required, the first argument is a
-function that returns its first argument and discards all others.
-
-If the function is called in a context that demands multiple values,
-the compiler supplies as the first argument a function that returns
-all of its argument as a tagged array.
-
-JSCL functions that appear as arguments in a \texttt{\#\textbackslash
-  j} syntax foreign function call are wrapped in a JavaScript function
-that supplies the first required argument. In this way, JavaScript
-code may call the functions it receives through FFI without supplying
-the first argument.
-
-This arrangement implies the reasonable limitation that JavaScript
-functions will never receive a tagged multiple values array from
-called JSCL functions.
-
-However, it does require that all JSCL functions passed to JavaScript,
-by any means, must be suitably wrapped.
-
-\subsubsection{REPL}
+of to an interpreted bytecode. It is self-hosting, includes the major
+control operators, and integrates tightly with JavaScript.
 
 JSCL includes a reader, compiler, and printer, and evaluation is
 performed by JavaScript's \texttt{eval} function. Between these, a
 Read Eval Print Loop (REPL) is possible, and the JSCL distribution
-includes a reference implementation.
+includes a reference implementation of one.
 
 However, JSCL only supports reading from string-backed input streams.
 Input streams are not an abstraction supported by JavaScript in Web
@@ -342,10 +223,15 @@ response to asynchronous input events, and invokes the reader only
 once a complete form --- as a string --- has been accumulated.
 
 Such a pre-reader can be found in the JSCL REPL implementation. It
-handles standard Lisp atoms and list syntax, but would be stymied by
-reader macros. As such, the pre-reader is a separate, degenerate
-reader that limits what's capable of being read by the underlying,
-full-featured synchronous reader.
+handles standard Lisp atoms and list syntax, but has the potential to
+be stymied by extended syntax, such as that added by reader macros. As
+such, the pre-reader is a separate, degenerate reader that limits
+what's capable of being read by the underlying, full-featured
+synchronous reader.
+
+Incidentally, SLip's reader is also synchronous, and its Emacs clone,
+ymacs, also includes a pre-reader as part of its Lisp interaction
+mode.
 
 Because JSCL includes no demonstration of embedding the REPL in an
 application, and because JSCL itself is compiled in batch through a
@@ -368,9 +254,9 @@ The first stage, the conversion from Lisp to JavaScript Abstract
 Syntax Trees (AST), is where the implementation of Lisp's special
 forms in terms of JavaScript language constructs and runtime support
 is performed. This is done in a single pass in which macro expansion,
-lexical analysis, and JavaScript AST generation all happen. The
-lexical environment is maintained in a special variable as the
-compiler descends into Lisp code.
+lexical analysis, and JavaScript AST generation all occur. The lexical
+environment is maintained in a special variable as the compiler
+descends into Lisp code and produces JavaScript AST.
 
 Code for \texttt{TAGBODY}/\texttt{GO} is generated in the first
 stage, and the generated code is much slower than comparable
@@ -378,7 +264,8 @@ JavaScript code for looping.
 
 Only the general, dynamic case of \texttt{TAGBODY}/\texttt{GO} is
 implemented. Every control transfer initiated by \texttt{GO} results
-in a JavaScript exception being thrown, an expensive operation.
+in a JavaScript exception being thrown, which is an expensive
+operation.
 
 Since many Common Lisp operators have \emph{implicit tagbodies}, and
 since most other iteration operators are expressed in terms of
@@ -402,6 +289,42 @@ optimizers excel at.
 If JSCL oriented itself toward generating code optimizable by third
 party tools, then its own internals could be simplified to a degree.
 
+\subsection{ClojureScript}
+
+A discussion of industrial Lisp technology in the SPA setting would be
+incomplete without mention of ClojureScript. ClojureScript is, by a
+wide margin, the most successful Lisp dialect for building SPAs, at
+least with respect to public lists of known users [ref].
+
+ClojureScript targets JavaScript, and is a dialect of an earlier
+language, Clojure, which targets Java Virtual Machine (JVM)
+bytecode. Both were designed and initially developed by the same
+person, Rich Hickey. ClojureScript's reader and macro systems were
+both originally hosted in Clojure, in a manner similar to Parenscript.
+
+Since its introduction, ClojureScript has heavily promoted and
+prioritized the ability to produce high-performance deliverables. It
+has always been capable of generating JavaScript deliverables amenable
+to aggressive optimization by the Google Closure Compiler.
+
+In this respect, ClojureScript aligns closely with the JACL project
+goal of competitive application performance. In fact, the author's
+experience with, and admiration for, ClojureScript is one of the
+primary motivations for the JACL project.
+
+However, ClojureScript's support for interactive development as
+traditionally performed in Lisp systems is relatively
+weak. Interactive REPLs exist, but they are neither as featureful nor
+as central to the development workflow as they are in traditional
+Common Lisp environments. Most of the tooling associated with
+industrial ClojureScript SPAs is developed on the host, as part of a
+JVM-based compiler toolchain.
+
+ClojureScript's success --- as a mostly static language oriented
+around batch compilation and whole-program reloading --- casts the
+most doubt on a central claim of this paper: that Common Lisp and
+prioritizing interactive development will make SPA development easier.
+
 \section{Design and Implementation}
 
 From a design perspective, JACL is an effort to balance the
@@ -501,7 +424,7 @@ JACL's reader is far from supporting all of Common Lisp syntax, but
 its fundamental design is complete and unlikely to substantially
 change.
 
-\subsection{Chrome Devtools REPL}
+\subsection{Chrome DevTools REPL}
 
 A browser-based REPL facilitates experimentation with the language by
 interested people, from the comfort of their Web browsers. It's also a
@@ -528,7 +451,7 @@ command-line REPL client that may be run on the developer's host
 machine. The workflow is the following:
 
 \begin{enumerate}
-  \item Run Google Chrome from the shell with the
+  \item Run Google Chrome from the shell with the \linebreak
     \texttt{--remote-debugging-port} parameter.
   \item Navigate to the Web site hosting the JACL system you wish to
     interact with.
@@ -562,7 +485,7 @@ extensible print protocol. Currently, object string representations
 are obtained by calling the generic JavaScript \texttt{toString()}
 method, which produces a representation that can't be read back in.
 
-\subsection{High-level optimizing compiler}
+\subsection{High-level analyzing compiler}
 
 Unlike JSCL, the JACL compiler is organized to facilitate high-level
 optimizations such as those that would support efficient compilation
@@ -610,16 +533,87 @@ following JACL code displays the number 3 in an alert box:
 (JACL:%JS "window.alert(~{})" 3)
 \end{verbatim}
 
-The character sequence \texttt{~\{\}} is distinct from any plausible
+The character sequence \texttt{\textasciitilde\{\}} is distinct from any plausible
 JavaScript sequence and so is used as placeholder syntax.
 
 \subsection{Delivery}
 
-TODO
+All of the JACL features I have described so far are in the very early
+stages of implementation, but JACL's delivery capabilities have yet to
+be implemented at all.
+
+Fortunately, since JACL development is image-based, JACL should
+support the traditional approach of specifying a Lisp function
+entrypoint and dumping the Lisp image to native code. SBCL's
+\texttt{SAVE-LISP-AND-DIE} and LispWork's \texttt{DELIVER} function
+are two examples of this in other implementations.
+
+It is imagined that JACL's delivery function will be called in a REPL
+at development time by a developer, or by an automated build
+script. When the function is called, a JavaScript deliverable will be
+produced, and the Web browser will either prompt the user to accept a
+download, or the JavaScript code will be written to standard output.
+
+JavaScript optimization tools such as Babel or Google Closure Compiler
+may then be applied to the executable. Such tools perform
+sophisticated optimizations such as dead-code elimination, loop
+invariant code motion, and constant folding.
+
+It is anticipated that one important high-level optimization must be
+performed in order for dumped images to be amenable to further
+optimization by JavaScript tools: Global symbol and function
+references must be converted to static references in order for
+JavaScript tools to successfully identify and eliminate dead code.
+
+This required optimization implies that JACL must retain the ASTs
+associated with global definitions, so that statically-linked code for
+these definitions can be generated at delivery time. Alternatively,
+statically-linked code could be created automatically in addition to
+dynamically-linked code.
+
+Without experience with the feature, it's hard to know which approach
+to code generation will ultimately be best.
 
 \section{Conclusions and Future Work}
 
-TODO
+I have presented a new Common Lisp, JACL, which was created to ease
+SPA development by applying the power of Common Lisp and interactive
+development to contemporary Web development challenges.
+
+I have described four novel aspects of JACL with respect to other
+comparable, contemporary Common Lisp implementations:
+
+\begin{enumerate}
+  \item Asynchronous reader.
+  \item Analyzing compiler.
+  \item Chrome DevTools REPL.
+  \item Delivery facility design.
+\end{enumerate}
+
+While most of Common Lisp that JACL will be capable of supporting
+remains to be implemented, JACL is nearly in a state in which it can
+be used for application development.
+
+The most immediate body of remaining work is population of the
+\texttt{COMMON-LISP} package, towards the end goal of supporting MIT
+\texttt{LOOP}. Such support would imply comprehensive support of many
+fundamental Lisp operators.
+
+A longer term but equally important goal is CLOS support. Key
+historical Lisp innovations were achieved with CLOS or one of its
+predecessors, and so its ultimate inclusion is an important project
+goal.
+
+Many other design and implementation tasks remain, such as support for
+special variables in lambda lists, \texttt{EVAL-WHEN}, macro lambda
+lists, \texttt{DECLARE} et al, various other data types, compiler
+macros, etc. The list of tasks is enormous, but it is anticipated that
+these features can be implemented over time, in the order demanded by
+application development.
+
+Bootstrapping is a desirable milestone, but since the fact that JACL
+is not currently bootstrapped doesn't preclude its use for application
+development, this work is low priority.
 
 \section{Acknowledgments}