<!DOCTYPE html>
<script>var __pbpa = true;</script><script>var translated_warning_string = 'Warning: Never enter your Tumblr password unless \u201chttps://www.tumblr.com/login\u201d\x0ais the address in your web browser.\x0a\x0aYou should also see a green \u201cTumblr, Inc.\u201d identification in the address bar.\x0a\x0aSpammers and other bad guys use fake forms to steal passwords.\x0a\x0aTumblr will never ask you to log in from a user\u2019s blog.\x0a\x0aAre you absolutely sure you want to continue?';</script><script type="text/javascript" language="javascript" src="http://assets.tumblr.com/assets/scripts/pre_tumblelog.js?_v=289ca146962d7f79251f9df647c84ce1"></script>
<!DOCTYPE html>
<html>
<head prefix="og: http://ogp.me/ns# fb: http://ogp.me/ns/fb# blog: http://ogp.me/ns/blog#">
<meta charset="UTF-8">
<title>Structural Editing Revisited - alan dipert</title>
<link rel="shortcut icon" href="http://24.media.tumblr.com/avatar_4ae064934cfd_16.png">
<link rel="alternate" type="application/rss+xml" href="http://alan.dipert.org/rss">
<meta name="description" content="Structural Editing Revisited It’s been a little over three years since I first stumbled on the idea of a structural Lisp editor, and over a year since I last blogged about the idea. Well, I have good..." />
<meta name="if:Show notes" content="1">
<meta name="text:Disqus Shortname" content="" />
<meta name="color:Post Background" content="#F2F2F2" />
<link rel="stylesheet" href="http://static.tumblr.com/26l8vc7/ej3lr0jwd/style.css">
<style type="text/css">
.post{background: #F2F2F2;}
</style>
<script type="text/javascript" src="http://shjs.sourceforge.net/sh_main.min.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_css.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_html.min.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_java.min.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_sh.min.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_c.min.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_scala.min.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_cpp.min.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_php.min.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_ruby.min.js">
</script>
<script type="text/javascript"
src="http://shjs.sourceforge.net/lang/sh_javascript.min.js">
</script>
<link type="text/css" rel="stylesheet" href="http://shjs.sourceforge.net/sh_style.css">
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-965358-1']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
<!-- BEGIN TUMBLR FACEBOOK OPENGRAPH TAGS -->
<!-- If you'd like to specify your own Open Graph tags, define the og:url and og:title tags in your theme's HTML. -->
<!-- Read more: http://ogp.me/ -->
<meta property="fb:app_id" content="48119224995" />
<meta property="og:title" content="Structural Editing Revisited" />
<meta property="og:url" content="http://alan.dipert.org/post/47444634908/structural-editing-revisited" />
<meta property="og:description" content="It’s been a little over three years since I first stumbled on the idea of a structural Lisp editor, and over a year since I last blogged about the idea.
Well, I have good news: Micha Niskin and I have..." />
<meta property="og:type" content="tumblr-feed:entry" />
<!-- END TUMBLR FACEBOOK OPENGRAPH TAGS -->
<!-- TWITTER TAGS -->
<meta charset="utf-8">
<meta name="twitter:card" content="summary" />
<meta name="twitter:description" content="It&#8217;s been a little over three years since I first stumbled on the idea of a structural Lisp editor, and over a year since I last blogged about the idea.
Well, I have good news: Micha Niskin and I have overcome the technical challenges that I was unable to solo, and work toward a realtime structural Clojure editor suitable for our own professional use is underway. We call it &#8220;stoke&#8221;, and I posted a short demo of our prototype on YouTube:
The editor consists of these main pieces:
reader: For code reading purposes, Clojure&#8217;s native reader does too much. For instance, scalars like &#8220;+1.0e3&#8221; are read as a Double and later print as &#8220;100.0&#8221;. Unordered collection types like sets and maps behave similarly, and the result is a mismatch between read and printed representations. Our reader deals in only two types, symbols and sequences, and preserves enough information (using metadata) to print appropriately.
printer: We were able to extend Brandon Bloom&#8217;s excellent Fast Idiomatic Pretty Printer to print Clojure code idiomatically. FIPP&#8217;s intermediate representation, the &#8220;document&#8221;, gives us a data representation of the formatted code that doesn&#8217;t necessarily need to be printed character-wise, and opens the door to interesting alternative views of code.
meta-zip: This is less a piece and more an abstraction that we have found crucial, and zippers will be the editor&#8217;s primary extension point. Zipper navigation and manipulation of our symbol/sequence representation is trivial. Meta-zip is a &#8220;zipper of zippers&#8221; that lets us cleanly model tree undo/redo.
Our immediate goal was to have a terminal-based structural editor with paredit commands, REPL, and file interaction, and as of this writing we are just about done with it. Next up are a Swing frontend, network/multi-user editing, API cleanup, and possibly a public release.
However, if we stopped here we&#8217;d fail to take advantage of one of the greatest things about having our own structural editor: files are no longer essential. What does this mean? I think the implications are vast, incredible, and since the death of Interlisp have been largely unrealized.
We&#8217;re used to working in and around files and filesystems, and much of our programming tooling is file-oriented - package systems, revision control systems, editors, versioning idioms. I think that collectively these systems represent an attempt to turn filesystems into the databases we wished they were, and I think that by storing Clojure code in Datomic we can throw most of them out.
I want to live in a world where code is stored in a temporal database and I build new programs with the combination of query and structural edit, and this is where we ultimately want to take stoke. You can follow our progress at http://tailrecursion.com.
*update 4/8/13*
Beyond a code database, other implications of the symbols+sequences intermediate representation (and zipper navigation model) include:
Editing HTML as haml, and vice versa
Editing Lisp as Sweet Expressions, and vice versa
*update 4/11/13*
Posted Clojure Structural Editor, Take 2 to YouTube" />
<meta name="twitter:title" content="Structural Editing Revisited" />
<meta name="twitter:url" content="http://alan.dipert.org/post/47444634908/structural-editing-revisited" />
<meta name="twitter:site" content="tumblr" />
<meta name="twitter:app:name:iphone" content="Tumblr" />
<meta name="twitter:app:name:ipad" content="Tumblr" />
<meta name="twitter:app:name:googleplay" content="Tumblr" />
<meta name="twitter:app:id:iphone" content="305343404" />
<meta name="twitter:app:id:ipad" content="305343404" />
<meta name="twitter:app:id:googleplay" content="com.tumblr" />
<meta name="twitter:app:url:iphone" content="tumblr://x-callback-url/blog?blogName=alandipert&postID=47444634908&referrer=twitter-cards" />
<meta name="twitter:app:url:ipad" content="tumblr://x-callback-url/blog?blogName=alandipert&postID=47444634908&referrer=twitter-cards" />
<meta name="twitter:app:url:googleplay" content="tumblr://x-callback-url/blog?blogName=alandipert&postID=47444634908&referrer=twitter-cards" />
<script type="text/javascript" src="http://assets.tumblr.com/assets/scripts/tumblelog.js?_v=dae58aa57407c7d7e8848a80145008fa"></script><meta http-equiv="x-dns-prefetch-control" content="off"/></head>
<body>
<div id="wrapper">
<div id="header" role="banner">
<h1><a href="/">alan dipert</a></h1>
<div id="description">
<p><p><a href = "http://twitter.com/alandipert/">twitter</a> <a href = "http://github.com/alandipert/">github</a> <a href = "mailto:adipert@acm.org">email</a></p></p>
</div>
</div>
<div id="content" role="main">
<div class="post text">
<div class="date">
<p><span class="day">08</span> 4 / 2013</p>
</div>
<div class="content">
<h2>Structural Editing Revisited</h2>
<p>It’s been a little over three years since I <a href="http://alan.dipert.org/post/445462664/the-s-expression-editor">first stumbled on the idea</a> of a structural Lisp editor, and over a year since I <a href="http://alan.dipert.org/post/17375691572/structural-editing-redux">last blogged</a> about the idea.</p>
<p>Well, I have good news: <a href="https://twitter.com/michaniskin">Micha Niskin</a> and I have overcome the technical challenges that I was unable to solo, and work toward a realtime structural Clojure editor suitable for our own professional use is underway. We call it “stoke”, and I posted a short demo of our prototype on YouTube:</p>
<p><iframe frameborder="0" height="315" src="http://www.youtube.com/embed/__ERGJ5EzAc" width="420"></iframe></p>
<p>The editor consists of these main pieces:</p>
<ul><li><em>reader</em>: For code reading purposes, Clojure’s native reader does too much. For instance, scalars like “+1.0e3” are read as a Double and later print as “100.0”. Unordered collection types like sets and maps behave similarly, and the result is a mismatch between read and printed representations. Our reader deals in only two types, symbols and sequences, and preserves enough information (using metadata) to print appropriately.</li>
<li><em>printer</em>: We were able to extend <a href="https://twitter.com/BrandonBloom">Brandon Bloom</a>’s excellent <a href="https://github.com/brandonbloom/fipp">Fast Idiomatic Pretty Printer</a> to print Clojure code idiomatically. FIPP’s intermediate representation, the “document”, gives us a data representation of the formatted code that doesn’t necessarily need to be printed character-wise, and opens the door to interesting alternative views of code.</li>
<li><em>meta-zip</em>: This is less a piece and more an abstraction that we have found crucial, and zippers will be the editor’s primary extension point. <a href="http://richhickey.github.io/clojure/clojure.zip-api.html">Zipper</a> navigation and manipulation of our symbol/sequence representation is trivial. Meta-zip is a “zipper of zippers” that lets us cleanly model tree undo/redo.</li>
</ul><p>Our immediate goal was to have a terminal-based structural editor with paredit commands, REPL, and file interaction, and as of this writing we are just about done with it. Next up are a Swing frontend, network/multi-user editing, API cleanup, and possibly a public release.</p>
<p>However, if we stopped here we’d fail to take advantage of one of the greatest things about having our own structural editor: files are no longer essential. What does this mean? I think the implications are vast, incredible, and since the death of Interlisp have been largely unrealized.</p>
<p>We’re used to working in and around files and filesystems, and much of our programming tooling is file-oriented - package systems, revision control systems, editors, versioning idioms. I think that collectively these systems represent an attempt to turn filesystems into the databases we wished they were, and I think that by storing Clojure code in <a href="http://www.datomic.com/">Datomic</a> we can throw most of them out.</p>
<p>I want to live in a world where code is stored in a temporal database and I build new programs with the combination of query and structural edit, and this is where we ultimately want to take stoke. You can follow our progress at <a href="http://tailrecursion.com"><a href="http://tailrecursion.com">http://tailrecursion.com</a></a>.</p>
<p><img alt="image" src="https://dl.dropbox.com/u/12379861/editingmm.png"/></p>
<p><strong>*update 4/8/13*</strong></p>
<p>Beyond a code database, other implications of the symbols+sequences intermediate representation (and zipper navigation model) include:<strong><br/></strong></p>
<ul><li><span>Editing HTML as <a href="http://haml.info/">haml</a>, and vice versa</span></li>
<li>Editing Lisp as <a href="http://readable.sourceforge.net/">Sweet Expressions</a>, and vice versa</li>
</ul><p><strong>*update 4/11/13*</strong></p>
<p>Posted <a href="http://www.youtube.com/watch?v=6VxTE1iAWyg">Clojure Structural Editor, Take 2</a> to YouTube</p>
</div>
<div class="info">
<p><a href="http://alan.dipert.org/post/47444634908/structural-editing-revisited">Permalink</a> <span style="float: right;"><a class="dsq-comment-count" href="http://alan.dipert.org/post/47444634908/structural-editing-revisited#disqus_thread"></a></span></p>
</div>
</div>
<div class="notecontainer">
<div id="disqus_thread"></div>
<script type="text/javascript" src="http://disqus.com/forums/alandipert/embed.js"></script>
<noscript><a href="http://alandipert.disqus.com/?url=ref">View the discussion thread</a></noscript>
</div>
<div id="pagination" role="navigation">
<p>
<a href="http://alan.dipert.org/post/47461498337" class="newer">Next post »</a>
<a href="http://alan.dipert.org/post/44050917245" class="older">« Previous post</a>
</p>
</div>
</div>
</div>
<div id="footer">
<div class="wrapper">
<ul class="links" role="navigation">
<li><a href="/archive">Archive</a></li>
<li><a href="http://alan.dipert.org/rss">RSS</a></li>
</ul>
<p class="info-theme" role="contentinfo">Powered by <a href="http://www.tumblr.com">Tumblr</a> and designed by <a href="http://kcmr.tumblr.com">Kcmr</a>.</p>
</div>
</div>
<script type="text/javascript">
//<![CDATA[
(function() {
var links = document.getElementsByTagName('a');
var query = '?';
for(var i = 0; i < links.length; i++) {
if(links[i].href.indexOf('#disqus_thread') >= 0) {
query += 'url' + i + '=' + encodeURIComponent(links[i].href) + '&';
}
}
document.write('<script charset="utf-8" type="text/javascript" src="http://disqus.com/forums/alandipert/get_num_replies.js' + query + '"></' + 'script>');
})();
//]]>
</script>
<!-- BEGIN TUMBLR CODE -->
<iframe id="tumblr_controls" width="330" height="25" style="position:absolute; z-index:2147483647; top:0; right:0; border:0; background-color:transparent; overflow:hidden; " src="http://assets.tumblr.com/assets/html/iframe.html?src=http%3A%2F%2Falan.dipert.org%2Fpost%2F47444634908%2Fstructural-editing-revisited%3F6aee29e0&pid=47444634908&rk=vFmbZAkc&lang=en_US&name=alandipert&brag=false&avatar=http%3A%2F%2F25.media.tumblr.com%2Favatar_4ae064934cfd_64.png&title=alan+dipert&url=http%3A%2F%2Falan.dipert.org%2F&_v=dcf1c0a578dbfd6678efa6e9b3e760bc"></iframe>
<!--[if IE]><script type="text/javascript">document.getElementById('tumblr_controls').allowTransparency=true;</script><![endif]-->
<!-- END TUMBLR CODE -->
<iframe src="http://assets.tumblr.com/analytics.html?4344dcb6da7e8742c4c5ab1d76ffafae" scrolling="no" width="1" height="1" frameborder="0" style="background-color:transparent; overflow:hidden; position:absolute; top:0; left:0 z-index:9999;" id="ga_target"></iframe>
<script>
var COMSCORE = true;
window.setTimeout(function() {
var analytics_frame = document.getElementById('ga_target');
analytics_frame.contentWindow.postMessage('enable_comscore;' + window.location, analytics_frame.src.split('/analytics.html')[0]);
}, 1000);
</script>
<script type="text/javascript">
var _qevents = _qevents || [];
(function() {
var elem = document.createElement('script');
elem.src = (document.location.protocol == "https:" ? "https://secure" : "http://edge") + ".quantserve.com/quant.js";
elem.async = true;
elem.type = "text/javascript";
var scpt = document.getElementsByTagName('script')[0];
scpt.parentNode.insertBefore(elem, scpt);
})();
</script>
<script type="text/javascript">
_qevents.push( { qacct: 'p-19UtqE8ngoZbM' } );
</script>
<noscript>
<div style="display: none;"><img src="//pixel.quantserve.com/pixel/'p-19UtqE8ngoZbM'.gif" height="1" width="1" alt="Quantcast"/></div>
</noscript>
<script type="text/javascript">var s=new Image(1,1);s.src=((r='http://www.tumblr.com/impixu?T=1370231716&J=eyJ0eXBlIjoidXJsIiwidXJsIjoiaHR0cDpcL1wvYWxhbi5kaXBlcnQub3JnXC9wb3N0XC80NzQ0NDYzNDkwOFwvc3RydWN0dXJhbC1lZGl0aW5nLXJldmlzaXRlZD82YWVlMjllMCIsInJlcXR5cGUiOjB9&U=GLMCFCOHAL&K=e0d3a01c7ecb10e3a5c3408944bb2e2175cf4cb1dae06ee7331ffd4d8baeb94d&R=http%3A%2F%2Falan.dipert.org%2Fpost%2F47444634908%2Fstructural-editing-revisited'.replace(/&R=[^&$]*/,'')) + ('&R='+escape(document.referrer)).substr(0, 2000-r.length).replace(/%.?.?$/,''));</script><noscript><img style="position:absolute;z-index:-3334;top:0px;left:0px;visibility:hidden;" src="http://www.tumblr.com/impixu?T=1370231716&J=eyJ0eXBlIjoidXJsIiwidXJsIjoiaHR0cDpcL1wvYWxhbi5kaXBlcnQub3JnXC9wb3N0XC80NzQ0NDYzNDkwOFwvc3RydWN0dXJhbC1lZGl0aW5nLXJldmlzaXRlZD82YWVlMjllMCIsInJlcXR5cGUiOjAsIm5vc2NyaXB0IjoxfQ==&U=GLMCFCOHAL&K=cbd6a0c39fc6bdf25980b1d6aafaf772af8f91f694959df16d0c241a34519393&R=http%3A%2F%2Falan.dipert.org%2Fpost%2F47444634908%2Fstructural-editing-revisited"></noscript><script type="text/javascript">var s=new Image(1,1);s.src=((r='http://www.tumblr.com/impixu?T=1370231716&J=eyJ0eXBlIjoicG9zdCIsInVybCI6Imh0dHA6XC9cL2FsYW4uZGlwZXJ0Lm9yZ1wvcG9zdFwvNDc0NDQ2MzQ5MDhcL3N0cnVjdHVyYWwtZWRpdGluZy1yZXZpc2l0ZWQ/NmFlZTI5ZTAiLCJyZXF0eXBlIjowLCJwb3N0cyI6W3sicG9zdGlkIjoiNDc0NDQ2MzQ5MDgiLCJibG9naWQiOiI0Mzk0NjgiLCJzb3VyY2UiOjMzfV19&U=IOELMELINN&K=956d5330aef7a6af7f41ff3e1bc5393f2efe289850c62289ce4fe8104154b705&R=http%3A%2F%2Falan.dipert.org%2Fpost%2F47444634908%2Fstructural-editing-revisited'.replace(/&R=[^&$]*/,'')) + ('&R='+escape(document.referrer)).substr(0, 2000-r.length).replace(/%.?.?$/,''));</script><noscript><img style="position:absolute;z-index:-3334;top:0px;left:0px;visibility:hidden;" src="http://www.tumblr.com/impixu?T=1370231716&J=eyJ0eXBlIjoicG9zdCIsInVybCI6Imh0dHA6XC9cL2FsYW4uZGlwZXJ0Lm9yZ1wvcG9zdFwvNDc0NDQ2MzQ5MDhcL3N0cnVjdHVyYWwtZWRpdGluZy1yZXZpc2l0ZWQ/NmFlZTI5ZTAiLCJyZXF0eXBlIjowLCJwb3N0cyI6W3sicG9zdGlkIjoiNDc0NDQ2MzQ5MDgiLCJibG9naWQiOiI0Mzk0NjgiLCJzb3VyY2UiOjMzfV0sIm5vc2NyaXB0IjoxfQ==&U=IOELMELINN&K=14ec97880b3284a122c2c2a171a8c57c20ab91944a8e07bf75e9701d1b996249&R=http%3A%2F%2Falan.dipert.org%2Fpost%2F47444634908%2Fstructural-editing-revisited"></noscript></body>
</html>