\documentclass[sigconf]{acmart}
\usepackage{booktabs}
\AtBeginDocument{%
\providecommand\BibTeX{{%
\normalfont B\kern-0.5em{\scshape i\kern-0.25em b}\kern-0.8em\TeX}}}
\setcopyright{acmcopyright}
\copyrightyear{2020}
\acmYear{2020}
\acmDOI{10.5281/zenodo.3764494}
\acmConference[ELS '20]{ELS '20: European Lisp Symposium}{April 27--28, 2020}{Z\"{u}rich, Switzerland}
%%\acmSubmissionID{123-A56-BU3}
\begin{document}
\title{JACL: A Common Lisp for Developing Single-Page Web Applications}
\author{Alan Dipert}
\email{alan@dipert.org}
\begin{abstract}
This paper demonstrates JavaScript-Assisted Common Lisp (JACL), an
experimental Web-browser based implementation of an extended subset
of Common Lisp. JACL, which is in the early stages of development,
is an effort to explore new techniques for large-scale Single-page
Web Application (SPA) development in Lisp. JACL includes an
optimizing Lisp-to-JavaScript compiler and interoperates with
JavaScript. JACL promotes interactive, \emph{residential}
development in the Web browser environment with its
\emph{asynchronous reader} and Chrome DevTools-based REPL client.
\end{abstract}
\begin{CCSXML}
<ccs2012>
<concept>
<concept_id>10011007.10011006.10011041.10011045</concept_id>
<concept_desc>Software and its engineering~Dynamic compilers</concept_desc>
<concept_significance>500</concept_significance>
</concept>
<concept>
<concept_id>10011007.10011006.10011041.10011048</concept_id>
<concept_desc>Software and its engineering~Runtime environments</concept_desc>
<concept_significance>500</concept_significance>
</concept>
</ccs2012>
\end{CCSXML}
\ccsdesc[500]{Software and its engineering~Dynamic compilers}
\ccsdesc[500]{Software and its engineering~Runtime environments}
\keywords{Common Lisp, JavaScript, web applications}
\maketitle
\section{Introduction}
The demand for SPAs in the past decade has only grown, and users and
stakeholders continually expect larger and more sophisticated
applications. Unfortunately, large-scale development on the Web
browser platform presents a particular set of challenges that are not
easily overcome. Developers have responded to these challenges by
creating a widening variety of special-purpose programming languages
that compile to JavaScript
\cite{Somasegar12,Czaplicki12,wiki:ReasonML}. Each new language
promotes one or more paradigms, application architectures, or
development workflows, and claims some advantage relative to the
status quo.
This paper demonstrates one new such language, JavaScript-Assisted
Common Lisp (JACL), an experimental implementation of an extended
subset of Common Lisp. JACL was created to explore new techniques for
applying Common Lisp --- a proven\cite{Cannon07,Garnet90,Action}
substrate for UI innovation --- to SPA development.
Many projects involving compilation of Lisp to JavaScript precede
JACL. Lisps that have either demonstrated industrial utility or that
implement a significant subset of Common Lisp are surveyed in appendix
\ref{appendix:lisps}. Like many of these related efforts, JACL
includes an online, optimizing compiler and supports interoperation
with JavaScript. JACL distinguishes itself from these efforts by
placing special emphasis on the value of \emph{residential}
development style, where both applications and the tools used to
create them co-evolve in a shared environment. JACL provides
fundamental support for residential development with its
\emph{asynchronous reader}.
\section{Interoperation with JavaScript}
JACL integrates tightly with JavaScript and depends heavily on the
JavaScript runtime. As a result, JACL enjoys roughly the same
applicability and performance characteristics as the JavaScript
platform. However, this high degree of integration is at odds with
comformance to the Common Lisp specification, and so JACL will never
strictly conform.
\subsection{Object Types}
JACL introduces several of its own object types, currently implemented
in JavaScript, including \texttt{Cons}, \texttt{LispSymbol}, and
\texttt{LispString}. \texttt{Cons} and \texttt{LispSymbol} are
introduced because JavaScript does not include direct
equivalents. \texttt{LispString} is introduced because the native
JavaScript \texttt{String} is immutable, whereas Lisp strings are
mutable.
JACL includes support for only one numeric type, the JavaScript
\texttt{Number} object. The JavaScript \texttt{Number} is a
double-precision 64-bit IEEE 754 value. The JACL reader interprets
integers as \texttt{Number} objects. In the future, JACL will also
interpret floating-point numbers as \texttt{Number}. This decision
trades ANSI conformance for performance. If either type were boxed,
arithmetic performance would suffer intolerably. JSCL\cite{JSCLGitHub}
and Valtan\cite{valtanGitHub} make the same tradeoff.
JACL functions are JavaScript functions, and may be invoked by
JavaScript callbacks without a special calling convention. JavaScript
functions named as Lisp values may be invoked with \texttt{FUNCALL} or
\texttt{APPLY}. Neither arguments nor return values are automatically
coerced to or from any particular set of object types.
\subsection{Operators}
The JACL compiler supports a special operator for constructing
fragments of JavaScript code, verbatim, from Lisp. The semantics of
this operator, \texttt{JACL:\%JS}, are inspired by a similar feature
of ClojureScript\cite{Cljs}, \texttt{js*}. For example, the following
JACL code displays the number 3 in an alert box:
\begin{verbatim}
(JACL:%JS "window.alert(~{})" 3)
\end{verbatim}
The character sequence \texttt{\textasciitilde\{\}} is distinct from
any plausible \linebreak JavaScript syntax and so is used as
placeholder syntax. There must be as many placeholders as there are
arguments to \texttt{JACL:\%JS}.
In addition to \texttt{JACL:\%JS}, the JACL compiler currently
supports three more special operators for interacting with the host
platform: \texttt{JACL:\%NEW}, \texttt{JACL:\%DOT} and
\texttt{JACL:\%CALL}. These operators perform JavaScript object
instantiation, field access, and function calls, respectively. Since
JACL functions are JavaScript functions, \texttt{JACL:\%CALL} is the
basis for \texttt{FUNCALL} in JACL, and for function calls generally.
JACL also supplies a convenience macro, \texttt{JACL:\textbackslash.}
or ``the dot macro'' for performing a series of field accesses and
method calls\footnote{Strictly speaking, JavaScript ``method calls''
are normal function calls but with a particular value of
\texttt{this}.} concisely. The dot macro takes direct inspiration
from the \texttt{..} macro of
Clojure\cite{Clojure}. \texttt{JACL:\textbackslash.} expands to zero
or more nested \texttt{JACL:\%DOT} or \texttt{JACL:\%CALL} forms. Here
is an example of a \texttt{JACL:\textbackslash.} form --- equivalent
to the JavaScript expression \texttt{(123).toString().length} --- and
its corresponding expansion:
\begin{verbatim}
(\. 123 (|toString|) |length|)
(%DOT (%CALL 123 |toString|) |length|)
\end{verbatim}
\noindent Note that JavaScript identifiers are case sensitive, and so
case-preserving, pipe-delimited Lisp symbols must be used to refer to
JavaScript object field and method names. The \emph{readtable case} of
the JACL reader cannot currently be modified. The dot macro also
recognizes Lisp or JavaScript strings as JavaScript identifiers.
\subsection{Reader Macros}
JACL includes two reader macros to support interoperation with
JavaScript. These macros may be added to the \texttt{*READTABLE*} by
calling the function
\texttt{(JACL:ENABLE-JS-SYNTAX)}. \texttt{@\char`\"} denotes
JavaScript \texttt{String} objects and \texttt{@|} denotes JavaScript
identifiers.
For example, the following two forms, which both evaluate to a JavaScript
\texttt{String}, are equivalent:
\begin{verbatim}
@"Hello"
(\. "Hello" (|toString|))
\end{verbatim}
\noindent \texttt{@|} may generally be used in place of the
\texttt{JACL:\%JS} special form to refer to JavaScript
identifiers. \texttt{(JACL:\%JS \char`\"alert\char`\")} and
\texttt{@|alert|} are equivalent.
\section{Running JACL Programs}
Currently, JACL programs may be evaluated in the Web browser in two
ways: by adding Lisp \texttt{<script>} tags to the \texttt{<head>} of
a Web page that also includes \texttt{jacl.js}, or by using the
\texttt{jacl} tool included in the JACL
distribution\cite{JACLDistribution} to connect to a running Web
browser.
\subsection{Lisp Scripts}
Development of JACL itself is currently driven primarily by modifying
\texttt{jacl.js} and the \texttt{boot.lisp} and
\texttt{jacl-tests.lisp} Lisp scripts. The Lisp scripts are included
in the \texttt{jacl.html} file in the JACL
distribution\cite{JACLDistribution}. After each modification, the Web
browser is reloaded, and test results are displayed.
This Lisp script-based workflow is similar to the traditional
JavaScript development workflow and has served JACL development so
far. However, Lisp scripts require runtime parsing and compilation of
JACL source code, among other inefficiencies. Reloading the Web
browser also destroys the entire runtime environment.
The easiest way to create JACL programs in this manner is to start
with the \texttt{jacl.html} Web page provided by JACL and then modify
it by removing or adding new Lisp scripts.
It is imagined that ultimately, Lisp sources will be incorporated into
the Lisp \emph{image} exclusively by the REPL client tool. An
arrangement such as this decouples source code loading from the Web
browser lifecycle. Production executables may then be produced at any
time from the Lisp image using a Lisp function in a manner similar to
the \texttt{SAVE-LISP-AND-DIE}\cite{SBCLManual} function in SBCL or
the \texttt{DELIVER}\cite{LispWorksDeliver} function in LispWorks.
\subsection{REPL}
JACL includes a REPL client program, \texttt{jacl}, that may be used
to execute JACL programs in a Web browser from a terminal on the
host. This process is described in detail in the \texttt{RUN.md}
document included in the JACL distribution\cite{JACLDistribution}, but
is summarized here.
In order to use the REPL, the user must first start either the Google
Chrome or Chromium browser with the remote debugging feature enabled.
With remote debugging enabled, the Web browser may be controlled using
a client program over a WebSocket connection. Then, the user must
navigate to a Web page that includes at least \texttt{jacl.js} and
\texttt{boot.lisp}.
Finally, the user must start the \texttt{jacl} REPL client in a
terminal. \texttt{jacl} leverages the remote debugging feature as a
REPL transport, using it to send and receive characters between the
host and the remote JACL runtime. The \texttt{jacl} tool is currently
written in R\cite{Rstats} and uses the
\texttt{chromote}\cite{Rchromote} package for interacting with the
remote Chrome or Chromium browser.
The \texttt{jacl} program has no knowledge of JACL syntax or
semantics; it merely sends and receives characters. The intentional
simplicity of \texttt{jacl} is part of the larger project goal of
promoting residential-style tool and program development in the target
environment. The simplicity of \texttt{jacl} is possible because of
the asynchronous nature of the JACL reader. Incoming characters
delivered over the WebSocket debugging connection are received by
callback functions in the Web browser. The received characters are
asynchronously and incrementally parsed into Lisp data. When a
complete datum is formed, the compiler is called, and the resulting
JavaScript is evaluated. Finally, any output is sent back over the
debugger connection and received and printed by the \texttt{jacl}
program.
\section{Conclusion}
We introduced JACL, a new and experimental Common Lisp created to
explore techniques for building sophisticated SPAs. JACL integrates
tightly with the Web browser platform and interoperates directly with
JavaScript. Compared to other browser-based Lisps, JACL promotes
residential development, and introduces a new technique for
integrating the REPL into the development workflow.
\section{Future Work}
JACL currently lacks many basic Common Lisp data types, functions, and
operators. Ultimately, JACL should support as much of Common Lisp as
is possible, accounting for the severe limitations imposed by
JavaScript and the Web platform. Fortunately, the many other existing
Common Lisps that compile to JavaScript demonstrate that a compelling
implementation is achievable.
An in-browser REPL and other tools for interacting with the JACL
runtime in the Web browser would be desirable. Such tools could
optionally remain as parts of deployed applications and provide a
degree of introspection and extension capability even after the
application has been deployed.
Other than work related to missing features such as multiple values,
CLOS, and the conditions system, much design work remains with regard
to the specific affordances of the \texttt{jacl} tool. For example,
it's unclear how a large JACL project involving library dependencies
and multiple source files should be managed and loaded.
\section{Acknowledgments}
The author wishes to thank Micha Niskin, Bart Botta, Kevin Lynagh,
Lionel Henry, and Andy Keep for invaluable feedback on early versions
of this paper. The author wishes to express particular thanks to
Robert Strandh not only for his feedback, but also for his guidance on
the writing process.
The author also wishes to express special gratitude to his beautiful
wife, Sandra Dipert, for her encouragement and support.
Finally, the author wishes to express his deepest thanks to his
father, the late Randall R. Dipert, for first telling him about Lisp
and many other things besides.
\bibliographystyle{ACM-Reference-Format}
\bibliography{jacl-els-2020}
\appendix
\section{Survey of Related Lisps}
\label{appendix:lisps}
\subsection{Parenscript}
Released in 2005\cite{Baringer05}, Parenscript\cite{ParenScriptManual}
was the first Common Lisp compiler to target JavaScript. Parenscript
is not bootstrapped and its compiler is not written in JavaScript, and
so it relies on a hosting Common Lisp system for compilation. Only
JavaScript types are available to Parenscript programs at runtime, and
so Parenscript is more of a syntax frontend for JavaScript than it is
an interactive Lisp system. While Parenscript is not positioned to
facilitate large-scale SPA development, it remains a popular way to
add dynamic JavaScript-based behaviors to static Web sites.
\subsection{SLip}
SLip\cite{SLipHome,SLipImpl} is arguably the most ambitious Common
Lisp-on-JavaScript system created to date, even though it
intentionally diverges\cite{SLipVsCl} from Common Lisp in certain
ways. It offers a stunning array of powerful features including a
self-hosting compiler, a full set of control operators, JavaScript
Foreign-Function Interface (FFI), tail-call optimization, green
threads, and perhaps most impressively, a resident Emacs clone,
\emph{Ymacs}. SLip is based originally on the compiler and bytecode
interpreter presented in Chapter 23 of \emph{Paradigms of Artificial
Intelligence Programming: Case studies in Common
Lisp}\cite{Norvig92}.
\subsection{JSCL}
JSCL\cite{JSCLGitHub,JSCLTalk} compiles directly to JavaScript and is
self-hosting, includes the major control operators, and integrates
tightly with JavaScript. JSCL includes a reader, compiler, and
printer, and evaluation is performed by the JavaScript \texttt{eval()}
function. Between these, a Read Eval Print Loop (REPL) is possible,
and the JSCL distribution includes an implementation of one.
\subsection{ClojureScript}
ClojureScript \cite{CljsRelease,Cljs} is probably the most successful
Lisp dialect for building SPAs by number of commercial users
\cite{CljsUsers}. ClojureScript is a dialect of an earlier language,
Clojure\cite{Clojure}, which targets Java Virtual Machine (JVM)
bytecode. The ClojureScript reader and macro systems were both
originally hosted in Clojure, in a manner similar to Parenscript.
ClojureScript prioritizes the ability to produce high-performance
deliverables.
\subsection{Valtan}
Valtan\cite{valtanGitHub} compiles to JavaScript and includes a suite
of FFI operators for interoperating with JavaScript. It is
self-hosting and features a sophisticated, CLOS-based compiler
architecture. It also includes a REPL and several example
applications.
\end{document}
\endinput