<!DOCTYPE html>
<script>var __pbpa = true;</script><script>var translated_warning_string = 'Warning: Never enter your Tumblr password unless \u201chttps://www.tumblr.com/login\u201d\x0ais the address in your web browser.\x0a\x0aYou should also see a green \u201cTumblr, Inc.\u201d identification in the address bar.\x0a\x0aSpammers and other bad guys use fake forms to steal passwords.\x0a\x0aTumblr will never ask you to log in from a user\u2019s blog.\x0a\x0aAre you absolutely sure you want to continue?';</script><script type="text/javascript" language="javascript" src="http://assets.tumblr.com/assets/scripts/pre_tumblelog.js?d9f4c92b186c422f85b52215bfcb76af"></script>
<!DOCTYPE html>
<html>
<head prefix="og: http://ogp.me/ns# fb: http://ogp.me/ns/fb# blog: http://ogp.me/ns/blog#">
<meta charset="UTF-8">
<title>The S-expression Editor - alan dipert</title>
<link rel="shortcut icon" href="http://24.media.tumblr.com/avatar_4ae064934cfd_16.png">
<link rel="alternate" type="application/rss+xml" href="http://alan.dipert.org/rss">
<meta name="description" content="The S-expression Editor The other week I was reading about Bard, a new Lisp dialect, and ran into this: “Historically, and in most other Lisp dialects, source code is explicitly or implicitly not..." />
<meta name="if:Show notes" content="1">
<meta name="text:Disqus Shortname" content="" />
<meta name="color:Post Background" content="#F2F2F2" />
<link rel="stylesheet" href="http://static.tumblr.com/26l8vc7/ej3lr0jwd/style.css">
<style type="text/css">
.post{background: #F2F2F2;}
</style>
<script type="text/javascript" src="http://shjs.sourceforge.net/sh_main.min.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_css.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_html.min.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_java.min.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_sh.min.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_c.min.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_scala.min.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_cpp.min.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_php.min.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_ruby.min.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_javascript.min.js">
</script>
<link type="text/css" rel="stylesheet" href="http://shjs.sourceforge.net/sh_style.css">
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-965358-1']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
<!-- BEGIN TUMBLR FACEBOOK OPENGRAPH TAGS -->
<!-- If you'd like to specify your own Open Graph tags, define the og:url and og:title tags in your theme's HTML. -->
<!-- Read more: http://ogp.me/ -->
<meta property="fb:app_id" content="48119224995" />
<meta property="og:title" content="The S-expression Editor" />
<meta property="og:url" content="http://alan.dipert.org/post/445462664/the-s-expression-editor" />
<meta property="og:description" content="The other week I was reading about Bard, a new Lisp dialect, and ran into this:
“Historically, and in most other Lisp dialects, source code is explicitly or implicitly not text; it is s-expressions...." />
<meta property="og:type" content="tumblr-feed:entry" />
<!-- END TUMBLR FACEBOOK OPENGRAPH TAGS -->
<!-- TWITTER TAGS -->
<meta charset="utf-8">
<meta name="twitter:card" content="summary" />
<meta name="twitter:description" content="The other week I was reading about Bard, a new Lisp dialect, and ran into this:
&#8220;Historically, and in most other Lisp dialects, source code is explicitly or implicitly not text; it is s-expressions. Text is just a print representation of the s-expressions.&#8221;
Emphasis added. This got me thinking.
for lisp, text editors are overkill
Good text editors like vim and Emacs are really good at everything having to do with text editing. In fact, they&#8217;re so good at such a wide variety of things, that their features wildly exceed the sort of operations one really needs to do when working on Lisp programs.
For instance, even the idea of a &#8220;newline&#8221; (or any other character the average Reader considers whitespace) doesn&#8217;t really have any meaning in s-expression land outside of quoted strings. Everything in the editor about them needn&#8217;t be accessible to the programmer.
So, what if instead of accepting text as our source editing medium, we operated directly on data structures? That is, what if key strokes performed some operation directly on the program&#8217;s data, and the editor simply pretty printed a nicely formatted representation of the current program following edits?
This is my idea, and I&#8217;m not sure it hasn&#8217;t been done before.
some pros
Plugins: Plugin functions don&#8217;t have to worry at all about dealing with or parsing text. They&#8217;re handed the data structure representing the current program (or arbitrary s-expression), along with a function representing the &#8220;point&#8221; or cursor location. They transform it somehow, and return the new data structure. The point might be stored as a zipper partial.
Visualization: By separating editing from representation, it becomes simpler to develop alternative visualizations. I think there might be a lot of interesting work to be done here, with things like contours, bubbles, and snippets. For some cool ideas along these lines, check out Michael Weber&#8217;s Emacs hackery page.
Universal formatting: Since the programmer is no longer dealing directly with shareable, revisionable text, the text visualizer would serve also as the exporter when it came to source control and storage. In the case of Clojure, this would effectively make the *code-dispatch* dispatch table for clojure.contrib.pprint the gofmt of the language.
Evaluation: SLIME-like evaluation for Clojure would be trivial, assuming this editor was written in Clojure.
some cons
Comments: Comments, and anything else in source code normally stripped by the Reader, would be completely lost. This means that any such data would have to be metadata, something which seems cumbersome at best. Maybe not.
Another editor?: Both vim and Emacs, and particularly Emacs with Paredit, have served me exceptionally well when it comes to editing Clojure. I&#8217;m not sure a new editor is warranted; but the idea of an editor written in Clojure that takes advantage of STM (for undo) and zippers (for commands and plugins) sounds pretty appealing.
status
I&#8217;ve started playing with SWT in an effort to build a prototype. I&#8217;ll github it soon. In the meantime, does anyone know if this has this been done before? Are there unsurmountable pitfalls that make the idea pointless? Thanks in advance!
*update*
I learned recently that the INTERLISP programming environment featured just such a structural editor. Code was data, all the way down, and this was the late 1960s. Screen display and output files were pretty-printed serializations. Based on what I&#8217;ve read, this capability took INTERLISP the IDE to a level of capability that blows away anything we have today.
The wisdom of the ancients&#8230;" />
<meta name="twitter:title" content="The S-expression Editor" />
<meta name="twitter:url" content="http://alan.dipert.org/post/445462664/the-s-expression-editor" />
<meta name="twitter:site" content="tumblr" />
<meta name="twitter:app:name:iphone" content="Tumblr" />
<meta name="twitter:app:name:ipad" content="Tumblr" />
<meta name="twitter:app:name:googleplay" content="Tumblr" />
<meta name="twitter:app:id:iphone" content="305343404" />
<meta name="twitter:app:id:ipad" content="305343404" />
<meta name="twitter:app:id:googleplay" content="com.tumblr" />
<meta name="twitter:app:url:iphone" content="tumblr://x-callback-url/blog?blogName=alandipert&postID=445462664&referrer=twitter-cards" />
<meta name="twitter:app:url:ipad" content="tumblr://x-callback-url/blog?blogName=alandipert&postID=445462664&referrer=twitter-cards" />
<meta name="twitter:app:url:googleplay" content="tumblr://x-callback-url/blog?blogName=alandipert&postID=445462664&referrer=twitter-cards" />
<script type="text/javascript" language="javascript" src="http://assets.tumblr.com/assets/scripts/tumblelog.js?b32c40067ef5fa460dd9a1eda92ee368"></script><meta http-equiv="x-dns-prefetch-control" content="off"/></head>
<body>
<div id="wrapper">
<div id="header" role="banner">
<h1><a href="/">alan dipert</a></h1>
<div id="description">
<p><p><a href = "http://twitter.com/alandipert/">twitter</a> <a href = "http://github.com/alandipert/">github</a> <a href = "mailto:adipert@acm.org">email</a></p></p>
</div>
</div>
<div id="content" role="main">
<div class="post text">
<div class="date">
<p><span class="day">13</span> 3 / 2010</p>
</div>
<div class="content">
<h2>The S-expression Editor</h2>
<p>The other week I was reading about <a href="http://bywicket.com/users/mikel/weblog/126a4/Bard.html">Bard</a>, a new Lisp dialect, and ran into this:</p>
<p>“Historically, and in most other Lisp dialects, source code is explicitly or implicitly <em>not</em> text; it is s-expressions. <strong>Text is just a print representation of the s-expressions.</strong>”</p>
<p>Emphasis added. This got me thinking.</p>
<h3>for lisp, text editors are overkill</h3>
<p>Good text editors like vim and Emacs are really good at everything having to do with text editing. In fact, they’re so good at such a wide variety of things, that their features wildly exceed the sort of operations one really needs to do when working on Lisp programs.</p>
<p>For instance, even the idea of a “newline” (or any other character the average <a href="http://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node13.html">Reader</a> considers whitespace) doesn’t really have any meaning in s-expression land outside of quoted strings. Everything in the editor about them needn’t be accessible to the programmer.</p>
<p>So, what if instead of accepting text as our source editing medium, we operated directly on data structures? That is, what if key strokes performed some operation directly on the program’s data, and the editor simply pretty printed a nicely formatted representation of the current program following edits?</p>
<p>This is my idea, and I’m not sure it hasn’t been done before.</p>
<h3>some pros</h3>
<ul><li><em>Plugins</em>: Plugin functions don’t have to worry at all about dealing with or parsing text. They’re handed the data structure representing the current program (or arbitrary s-expression), along with a function representing the “point” or cursor location. They transform it somehow, and return the new data structure. The point might be stored as a <a href="http://www.fatvat.co.uk/2009/01/functional-tree-editing.html">zipper</a> partial.</li>
<li><em>Visualization: </em>By separating editing from representation, it becomes simpler to develop alternative visualizations. I think there might be a lot of interesting work to be done here, with things like contours, bubbles, and snippets. For some cool ideas along these lines, check out Michael Weber’s <a href="http://www.foldr.org/~michaelw/emacs/">Emacs hackery page</a>.</li>
<li><em>Universal formatting:</em> Since the programmer is no longer dealing directly with shareable, revisionable text, the text visualizer would serve also as the exporter when it came to source control and storage. In the case of Clojure, this would effectively make the *code-dispatch* dispatch table for clojure.contrib.pprint the <a href="http://gofmt.com/">gofmt</a> of the language.</li>
<li><em>Evaluation: </em>SLIME-like evaluation for Clojure would be trivial, assuming this editor was written in Clojure.</li>
</ul><h3>some cons</h3>
<ul><li><em>Comments:</em> Comments, and anything else in source code normally stripped by the Reader, would be completely lost. This means that any such data would have to be metadata, something which seems cumbersome at best. Maybe not.</li>
<li><em>Another editor?: </em>Both vim and Emacs, and particularly Emacs with Paredit, have served me exceptionally well when it comes to editing Clojure. I’m not sure a new editor is warranted; but the idea of an editor written in Clojure that takes advantage of STM (for undo) and zippers (for commands and plugins) sounds pretty appealing.</li>
</ul><h3>status</h3>
<p>I’ve started playing with SWT in an effort to build a prototype. I’ll github it soon. In the meantime, does anyone know if this has this been done before? Are there unsurmountable pitfalls that make the idea pointless? Thanks in advance!</p>
<p><em><strong>*update*</strong></em></p>
<p>I learned recently that the INTERLISP programming environment featured just such a structural editor. Code was data, all the way down, and this was the late 1960s. Screen display and output files were pretty-printed serializations. Based on what I’ve read, this capability took INTERLISP the IDE to a level of capability that blows away anything we have today.</p>
<p>The wisdom of the ancients…</p>
</div>
<div class="info">
<p><a href="http://alan.dipert.org/post/445462664/the-s-expression-editor">Permalink</a> <span style="float: right;"><a class="dsq-comment-count" href="http://alan.dipert.org/post/445462664/the-s-expression-editor#disqus_thread"></a></span></p>
</div>
</div>
<div class="notecontainer">
<div id="disqus_thread"></div>
<script type="text/javascript" src="http://disqus.com/forums/alandipert/embed.js"></script>
<noscript><a href="http://alandipert.disqus.com/?url=ref">View the discussion thread</a></noscript>
</div>
<div id="pagination" role="navigation">
<p>
<a href="http://alan.dipert.org/post/1588976216" class="newer">Next post »</a>
<a href="http://alan.dipert.org/post/435071716" class="older">« Previous post</a>
</p>
</div>
</div>
</div>
<div id="footer">
<div class="wrapper">
<ul class="links" role="navigation">
<li><a href="/archive">Archive</a></li>
<li><a href="http://alan.dipert.org/rss">RSS</a></li>
</ul>
<p class="info-theme" role="contentinfo">Powered by <a href="http://www.tumblr.com">Tumblr</a> and designed by <a href="http://kcmr.tumblr.com">Kcmr</a>.</p>
</div>
</div>
<script type="text/javascript">
//<![CDATA[
(function() {
var links = document.getElementsByTagName('a');
var query = '?';
for(var i = 0; i < links.length; i++) {
if(links[i].href.indexOf('#disqus_thread') >= 0) {
query += 'url' + i + '=' + encodeURIComponent(links[i].href) + '&';
}
}
document.write('<script charset="utf-8" type="text/javascript" src="http://disqus.com/forums/alandipert/get_num_replies.js' + query + '"></' + 'script>');
})();
//]]>
</script>
<!-- BEGIN TUMBLR CODE -->
<iframe src="http://assets.tumblr.com/iframe.html?9f9b05bc6d86a119a5f73c15523c270b&src=http%3A%2F%2Falan.dipert.org%2Fpost%2F445462664%2Fthe-s-expression-editor&pid=445462664&rk=Kr6PgsD5&lang=en_US&name=alandipert&brag=false&avatar=http%3A%2F%2F24.media.tumblr.com%2Favatar_4ae064934cfd_40.png&title=alan+dipert&url=http%3A%2F%2Falan.dipert.org%2F" width="330" height="25" scrolling="no" frameborder="0" style="position:absolute; z-index:2147483647; top:0px; right:0px; border:0px; background-color:transparent; overflow:hidden;" id="tumblr_controls"></iframe>
<!--[if IE]><script type="text/javascript">document.getElementById('tumblr_controls').allowTransparency=true;</script><![endif]-->
<script type="text/javascript">var s=new Image(1,1);s.src=((r='http://www.tumblr.com/impixu?T=1367904275&J=eyJ0eXBlIjoiYmxvZyIsImJsb2dpZCI6IjQzOTQ2OCIsInVybCI6Imh0dHA6XC9cL2FsYW4uZGlwZXJ0Lm9yZ1wvcG9zdFwvNDQ1NDYyNjY0XC90aGUtcy1leHByZXNzaW9uLWVkaXRvciIsInJlcXR5cGUiOjB9&U=OGMJKABPDH&K=eb819d4f43069c80c21afd962f256849a862e5470affc200004ca63994d314b0&R=http%3A%2F%2Falan.dipert.org%2Fpost%2F47444634908%2Fstructural-editing-revisited%3Fa50cdc10'.replace(/&R=[^&$]*/,'')) + ('&R='+escape(document.referrer)).substr(0, 2000-r.length).replace(/%.?.?$/,''));</script><noscript><img style="position:absolute;z-index:-3334;top:0px;left:0px;visibility:hidden;" src="http://www.tumblr.com/impixu?T=1367904275&J=eyJ0eXBlIjoiYmxvZyIsImJsb2dpZCI6IjQzOTQ2OCIsInVybCI6Imh0dHA6XC9cL2FsYW4uZGlwZXJ0Lm9yZ1wvcG9zdFwvNDQ1NDYyNjY0XC90aGUtcy1leHByZXNzaW9uLWVkaXRvciIsInJlcXR5cGUiOjAsIm5vc2NyaXB0IjoxfQ==&U=OGMJKABPDH&K=35843114de8488a44bfae304fd7fab416f5a85b4c7d75b350badc3342759617c&R=http%3A%2F%2Falan.dipert.org%2Fpost%2F47444634908%2Fstructural-editing-revisited%3Fa50cdc10"></noscript> <!-- END TUMBLR CODE -->
<iframe src="http://assets.tumblr.com/analytics.html?dcba2423e6f5653ee72915980432c1c3" scrolling="no" width="1" height="1" frameborder="0" style="background-color:transparent; overflow:hidden; position:absolute; top:0; left:0 z-index:9999;" id="ga_target"></iframe>
<script>
var COMSCORE = true;
window.setTimeout(function() {
var analytics_frame = document.getElementById('ga_target');
analytics_frame.contentWindow.postMessage('enable_comscore', analytics_frame.src.split('/analytics.html')[0]);
}, 1000);
</script>
<script type="text/javascript">
var _qevents = _qevents || [];
(function() {
var elem = document.createElement('script');
elem.src = (document.location.protocol == "https:" ? "https://secure" : "http://edge") + ".quantserve.com/quant.js";
elem.async = true;
elem.type = "text/javascript";
var scpt = document.getElementsByTagName('script')[0];
scpt.parentNode.insertBefore(elem, scpt);
})();
</script>
<script type="text/javascript">
_qevents.push( { qacct: 'p-19UtqE8ngoZbM' } );
</script>
<noscript>
<div style="display: none;"><img src="//pixel.quantserve.com/pixel/'p-19UtqE8ngoZbM'.gif" height="1" width="1" alt="Quantcast"/></div>
</noscript>
<script type="text/javascript">var s=new Image(1,1);s.src=((r='http://www.tumblr.com/impixu?T=1367904275&J=eyJ0eXBlIjoidXJsIiwidXJsIjoiaHR0cDpcL1wvYWxhbi5kaXBlcnQub3JnXC9wb3N0XC80NDU0NjI2NjRcL3RoZS1zLWV4cHJlc3Npb24tZWRpdG9yIiwicmVxdHlwZSI6MH0=&U=HGKECJOAEL&K=4a17e72e5267440469554035e2f2879c7ff8dc06db4a6c4b62f10ed04d6d3c6c&R=http%3A%2F%2Falan.dipert.org%2Fpost%2F47444634908%2Fstructural-editing-revisited%3Fa50cdc10'.replace(/&R=[^&$]*/,'')) + ('&R='+escape(document.referrer)).substr(0, 2000-r.length).replace(/%.?.?$/,''));</script><noscript><img style="position:absolute;z-index:-3334;top:0px;left:0px;visibility:hidden;" src="http://www.tumblr.com/impixu?T=1367904275&J=eyJ0eXBlIjoidXJsIiwidXJsIjoiaHR0cDpcL1wvYWxhbi5kaXBlcnQub3JnXC9wb3N0XC80NDU0NjI2NjRcL3RoZS1zLWV4cHJlc3Npb24tZWRpdG9yIiwicmVxdHlwZSI6MCwibm9zY3JpcHQiOjF9&U=HGKECJOAEL&K=5464391bc701e47eb01ddfaaa5907029249740b82ff56285c09beeddada7d543&R=http%3A%2F%2Falan.dipert.org%2Fpost%2F47444634908%2Fstructural-editing-revisited%3Fa50cdc10"></noscript><script type="text/javascript">var s=new Image(1,1);s.src=((r='http://www.tumblr.com/impixu?T=1367904275&J=eyJ0eXBlIjoicG9zdCIsInVybCI6Imh0dHA6XC9cL2FsYW4uZGlwZXJ0Lm9yZ1wvcG9zdFwvNDQ1NDYyNjY0XC90aGUtcy1leHByZXNzaW9uLWVkaXRvciIsInJlcXR5cGUiOjAsInBvc3RzIjpbeyJwb3N0aWQiOiI0NDU0NjI2NjQiLCJibG9naWQiOiI0Mzk0NjgiLCJzb3VyY2UiOjMzfV19&U=GIMNMKJBLN&K=9aba35ae9aaace3f300be97dcd668f955d2ab7efb6cdbfc7a5a087deb0ea35e8&R=http%3A%2F%2Falan.dipert.org%2Fpost%2F47444634908%2Fstructural-editing-revisited%3Fa50cdc10'.replace(/&R=[^&$]*/,'')) + ('&R='+escape(document.referrer)).substr(0, 2000-r.length).replace(/%.?.?$/,''));</script><noscript><img style="position:absolute;z-index:-3334;top:0px;left:0px;visibility:hidden;" src="http://www.tumblr.com/impixu?T=1367904275&J=eyJ0eXBlIjoicG9zdCIsInVybCI6Imh0dHA6XC9cL2FsYW4uZGlwZXJ0Lm9yZ1wvcG9zdFwvNDQ1NDYyNjY0XC90aGUtcy1leHByZXNzaW9uLWVkaXRvciIsInJlcXR5cGUiOjAsInBvc3RzIjpbeyJwb3N0aWQiOiI0NDU0NjI2NjQiLCJibG9naWQiOiI0Mzk0NjgiLCJzb3VyY2UiOjMzfV0sIm5vc2NyaXB0IjoxfQ==&U=GIMNMKJBLN&K=739ef1bd168c4eaf3919ba529e115833b8aa85774c55b050c876421b812ac579&R=http%3A%2F%2Falan.dipert.org%2Fpost%2F47444634908%2Fstructural-editing-revisited%3Fa50cdc10"></noscript></body>
</html>