Welcome everyone to our talk. I am Alan
Diper. This is my long-time friend and
open source collaborator Misha Niskin.
Uh we're here to
uh give a short a short talk about kind
of the dark arts. Maybe some of you have
seen Army of Darkness. Familiar with the
book, the Necronomicon?
Um
No, we're actually not going to talk
about these kinds of things. We're going
to talk about computers. In particular,
we're going to talk about web
application programming. So, there have
been uh more than a handful of of talks
about this topic. We're just uh
another way to make web pages. So, uh
advance me there.
Cool. There's our
info. If you'd like to pull up any of
our information right now, hoplon.io is
the place to go to learn.
Why are we interested in the
Necronomicon? Dark spells, evil magics
with the CK ending.
Um
Well, as web application programmers,
we're looking at an increasingly hostile
platform space. By hostile, I mean more
complicated. All kinds of different
deployment targets now. It used to be
just, you know, IE 5.5 on Windows XP and
it was good. Now it's, you know, it
needs to work on Chrome 33, Firefox 25,
WebKit 2.whatever.
So, we're in uh facing a a wall of
complexity when it comes to platforms
we're expected to deliver to.
Um we're bad at estimating how long it
will take to build any given application
to target any particular set of
platforms. I guess by we, I just mean me
and and Misha. He's worse than I am.
But, we're both pretty bad at it. And
honestly, I even the best developers,
programmers I've ever met are usually
off by an order of magnitude. Maybe
maybe maybe three to four times.
Not quite order of magnitude, but very
few people are really good at
estimating. Even even the best of us.
So, this is kind of scary. We don't know
how long anything we do will take.
That's weird. Why do people hire us?
Um
once we do have something figured out,
like we're happy with something, we've
shipped it. Oh yeah, we shipped the app,
works great, Android 4.5. Awesome. You
know, snowball or whatever it's called.
Snow ice cream. Ice cream.
Um
but then the next day somebody says,
"Hey, does it work on iOS?"
It should. I mean, you know, one's web
kit, the other they're both sort of web
kit based, should work. But of course it
doesn't because, you know, one is a
different aspect ratio or whatever. So
now we're facing again this this wall of
complexity and we
are not able to estimate how long it
would take to port one app to a
different platform.
So all of these battles are you would
think are easy ones to fight. Like I
have an existing app, I need to get it
to work on a slightly different
platform. No, it's not easy, it's
incredibly hard. And um
to solve this problem we need to look
towards the dark arts, dead
technologies. Spreadsheets are one of
those technologies.
Um
spreadsheets are not just notorious, I'd
say spreadsheets are infamous. I mean,
these are
We've all had experiences with
spreadsheets. Um
they've been around forever. We kind of
count on them as or we we see them as
just a part of the computing landscape.
Um this is an example of VisiCalc from
1979. This was the killer app for the
Apple II.
Um
so that's what a spreadsheet looked like
in 1979.
This is what the spreadsheet looks in
2014.
Uh
what Anybody notice anything?
Yeah.
The colors changed a little. What does
this look like though? Hit us, Misha.
Patrick Stewart.
Patrick Stewart has not aged.
Neither have spreadsheets. They're the
same thing. They have been forever. For
all times, Patrick Stewart has been
Patrick Stewart and spreadsheets have
been spreadsheets.
Like Patrick Stewart, spreadsheets are
amazing. They're an amazing technology.
To the extent that
Patrick Stewart is a technology.
Should have thought about that slide.
But they're both amazing.
Um
unlike Patrick Stewart, a lot of
programmers don't like spreadsheets.
Uh I don't know what it is. You know,
they have a long time association with
Windows and sort of business stuff that
seems stupid.
So, we disparage them openly. I mean,
it's I honestly I it's rare that you run
into a programmer who's like, "Dude, I'm
into VB macros."
Yeah, I'm talking to a strange loop
about VB4 macros and Excel 97. Cool.
Like, you don't hear that. No. We think
they suck.
So, we see them as an open problem, like
something that we need to solve. Like,
the fact that we have a close friend who
uses a spreadsheet. Like, let's
intervene. I'm going to make this person
a program. Like, I'm going to
Whoa.
Um
Part of that is that they're not sort of
computer science-y enough. There aren't
really open problems around them.
Um
But, the reality of spreadsheets are
that they're what people use when we let
them down. When we fail someone, when
they come to us and say, "Oh, hey, you
know, what would it take to visualize
these sales figures?" And you're like,
"Well, I need a team of five, a budget
of 3 million, and 4 months." You're
like, "Okay, spreadsheet time." 20
minutes later, done.
They're the technology that people go to
to solve their problems when we're not
able to. So, if we're going to learn
from anything, I think spreadsheets are
a great place to look.
Um I took a stab here at describing
We're kind of taking a look at
spreadsheets from the from the eyes of a
programming language and environment.
And I think
uh spreadsheets are amazing for roughly
these three reasons. I mean, they suck
for a lot more, but for these three
reasons, I think they're particularly
incredible and we should we should dive
in. I think they're reactive out of the
box. I mean, there's a there's like a
reactive manifesto now. We're looking at
FRP and all this stuff, but spreadsheets
have been they've been doing reactive
programming since 1979.
So, this is uh something pretty
incredible.
They're transparent. They're transparent
in the way
um your program is transparent when you
have a good debugger attached.
It's easy to see, at least for
some size of spreadsheets, what's going
on.
And finally, and I think most
importantly, they're layered. They
present a layered approach to computing
that allows small groups of people and
teams and businesses to to work on each
other to extend each other's work easily
in ways that we have a lot of difficulty
doing with our programming
uh constructs.
So, reactivity is the first thing I'm
going to hit on here. Uh this is a
screenshot from a interview that uh Rich
Hickey did with Brian Beckman a couple
years ago.
Ironically, at Microsoft on the
spreadsheet.
Um
One of the things Rich says uh when the
topic of uh
the so-called epical time model comes up
is that you can have a lot of semantics
for managing time once you separate
value from identity. So, this is the
separation that Clojure uh gives us
where we separate, you know, the name of
something with the value it may have at
a point in time.
Um in spreadsheet world, we do have
something like that, a reference type,
if you will, that uh
sort of fits into this this worldview.
It It It's fine.
And that's the cell.
The cell in a lot of ways, and by cell I
mean, you know, the little box in a
spreadsheet where you type in a value or
a formula, is one of these reference
types. It's a reactive unit of time
management. Uh it's what gives
spreadsheets this magical property of
reactivity.
The result of using cells in
spreadsheets is that time does not exist
in the spreadsheet. Um you know, if
anybody's like seen the SICP lectures,
they go
three lectures without even mutating
anything. And then suddenly, you know,
you define something a second time and
time is introduced. Spreadsheets, there
is no time. They are timeless and
creepy, same way Patrick Stewart is.
They're transparent. Um
So, again, you know, the the spreadsheet
interface is basically the an incredible
debugger. It's an awesome debugging
environment. You can see every the value
of every variable.
And when you change the values of these
constituent variables, things change
reactively.
You You easily see what's going on.
You have data, you have formulas, you
have visual visualizations backed by
either of those things.
Um
just a great a great debugging They're
they're just a natural debugging tool.
Uh the third point I mentioned about
layering, uh I think we can we can start
to see here. So, we're looking at a
spreadsheet and we've got a data grid
and then we've got a visualization. The
incredible thing about what we're seeing
here is that you can change
attributes of the way the data is
displayed, like those red uh horizontal
bars. Maybe I want them to be blue
without interacting at all with the
underlying data.
Likewise, or or symmetrically, I could
change the data without changing the
chart.
And in fact,
uh
just get uh yeah, look at this next
screen here.
I mean
you can add new charts. You can have N
charts, all different kinds of charts. I
here I'm showing off like the pie chart,
which is like
probably not the one you want, but um
you know, there are lots of charts and
we can attach them or plug them into
this existing data and formula without
changing the data or the formulas. And
this is incredible power.
So, that's the tech. The sort of human
side of it is that spreadsheets give us
this natural boundary to divide labor.
One person can enter the data.
Someone who knows the business or knows
the what relationships to look for in
the data can write formulas.
And then others can make visualizations
without necessarily writing formulas or
entering new data.
So, what we see in organizations that
use spreadsheets is these spreadsheets
sort of circulate
like, you know, trash around the oceans
from different
uh business units inside the company.
And people tack stuff on. Maybe they'll
want to use it for a report. Oh, maybe I
have a better idea for how to do this
formula.
Um
I mean, that's that's from a like a
change management perspective that's
bad, but it's amazing from the
perspective of everybody has a way to
contribute something they're good at to
this to this programming and this
environment.
So, hoplon this web
thing it's a it's a framework a set of
libraries that we've developed working
closure and closure script. It's if it's
influenced by any one thing I'd say that
thing is spreadsheets.
So, at the heart of our framework is a
closure script library called Javelin.
And Javelin is an abstract spreadsheet.
It's a way to do spreadsheet like
programming in a closure script
environment. And our goal is to try and
deliver all of the things that are good
about spreadsheets to the closure script
programmer.
So, like normal spreadsheets Excel 97
we have input cells.
And one creates input cells
by
one one of two methods. The first method
is you can call the cell constructor.
And this is this is just a function you
call that returns a reference type of
cell. This is similar to how you would
create an atom or an agent or any kind
of closure reference type.
So, here we're def defining a thing to
be the cell.
Another way to make a cell in with
Javelin is we have a macro def C. And
all that does is create a cell named N
and you don't have to type the
constructor in. We just have this
because our programs tend to have lots
of cells.
We have formula cells. Formula cells are
different from input cells and that they
shouldn't really be mutated. You don't
change the contents of them dynamically
necessarily. What they do contain are
closure script expressions which are
analogous to the formula values in you
know an Excel spreadsheet. So, in this
case we're defining an input cell called
N giving an initial value of zero.
Then we're defining a formula cell
called N * 5 and we're using cell equal
which is the cell a formula cell
constructor macro that takes an
expression.
And at this point we can differ
dereference either n or n * 5 and we'll
see values that are consistent.
If we swap bang on n, incrementing the
its value,
and then dereference n and dereference n
* 5, we'll see that n has been
incremented and n * 5 is somewhat
magically, but in that awesome
spreadsheet way,
uh reflects the value. And that's the
value we established through our
formula.
Uh the the same way there's a defc,
there's a defc= and again this is just
sort of a
having fun with macros little exercise.
Javelin is different in I think some
interesting ways from spreadsheets that
you may be used to. The first and
maybe coolest difference is that cells
in a spreadsheet, like we have in Excel,
can't take collection values. You you
can put numbers and strings, but putting
collection values in the cell is not
even really possible. So, people have
other mechanisms to deal with
collections like ranges and pivot
tables.
Cells in in ClojureScript are not that
way. Like any other Clojure reference
type, that you can put any value in
them, including other cells, functions,
I mean anything that's a value, which is
everything in ClojureScript, can go in a
cell.
And the other difference is that unlike
the spreadsheets, um
uh
grid, the global scope of a spreadsheet
is this 2D grid. The scopes and names in
Javelin follow ClojureScript's. So, you
can create let bound cells, that you
know, cells that are only available
inside of closure. You can have package
cells in namespaces.
So, um
that approach that we take with Javelin
is kind of similar to the way people
work with sheets or worksheets in in um
spreadsheet programs.
So, that was undead technology number
one, spreadsheets. Uh the other undead
technology which we're we're probably
more familiar with is Lisp. Lisp, like
spreadsheets, is most programmers think
sucks. Of course, we know otherwise.
Um so, I'm going to let Misha uh
teach us some things about uh part two.
Yeah, so uh H Lisp is kind of the
charting and forms the charts and forms
layer of the spreadsheet.
Uh a cool thing about spreadsheets is
that given a spreadsheet that has
formulas you might not even understand,
you might be presenting something to the
board of directors or, you know, like in
the on TV, somebody always has to make a
make a report by Friday or whatever and
they have to stay up late. But, what
they're doing is they're taking a
spreadsheet and they have a Word
document, they're putting content in
there, making graphs, things like that,
and pulling them into their
into their Word document that they're
going to show to somebody to hopefully
elicit some kind of action.
And that's really what, you know, a web
application is too, the UI of it.
And so, H Lisp is that part of it, the
the charting part.
So, HTML,
uh when you load index.html into a
browser, the browser normally reads a
bunch of HTML markup
and loads it into the DOM.
But, this HTML is already a programming
language. It's not a very good one, but
it definitely is. Like, the browser has
to parse this HTML markup, which is kind
of weird, and then it has to build a
tree and it has to walk the tree, create
DOM elements, and uh you know, put them
in the DOM.
And so, what's, I mean, luckily, thank
God,
uh
well, sorry. So,
unfortunately, uh it doesn't have a
means of abstraction. So, there's no way
to define something new in HTML markup.
There's no access to the JavaScript
environment, uh which was kind of bolted
on after HTML already existed. So, you
can create all this stuff in the DOM,
but there's no real linkage to the
dynamic environment, JavaScript, and
there are no scopes. So, that means that
you just are basically dumping this
everything into this global tree that
you then have to
fish around in using selectors or
something to find things.
But thank God,
HTML is kind of lispy, so uh you know
what to do.
Um
these this HTML markup over here, you
can see that it's syntactically
equivalent to this Lisp markup. I mean,
people have suggested S-expressions
instead of, you know, XML syntax for a
long time.
But what's interesting in ClojureScript
is that you can actually implement these
things, implement the semantic as well.
So,
uh in H Lisp, div is actually a function
which returns a DOM element.
And uh this is all done by the Hoplon
compiler, which takes uh you know, .hl
files, and for each one of those, which
is uh you know, a page, the unit of
compilation is a page, uh it emits an
HTML bootstrapping kind of minimal file
just to load a JavaScript file, which
contains all the compiled ClojureScript,
and the DOM's actually built at run time
via a program. And the front end
compiles to just static HTML and
JavaScript, so you could like host it on
S3 or whatever.
So, this is an example page.
Uh this is kind of what, you know, a
page normally looks like simplified, but
uh
you can actually also do this in HTML
syntax. The Hoplon compiler just
compiles it all to ClojureScript anyway.
So, that's cool. You could use your
tools, like if you use Haml or
something.
You could do that.
Uh a tricky thing about, you know,
making HTML into Lisp or Lisp into HTML
is uh how to deal with attributes, which
is sort of like metadata on functions in
a way, uh on function calls, rather. And
so, the way we do it is you can either
pass in a map as the first argument, or
just some keyword-value pairs, and it'll
It's unambiguous because we know that
keywords can't be children of an
element.
Uh but they don't have to be strings.
The values could be other things, which
we'll see later.
Uh there's no boxing or indirection or
virtual DOMs or anything like that. When
you call the div function, you get a
native HTML div element that you can do
stuff to.
But uh the elements implement uh I fun,
which is the protocol to make something
invocable in ClosureScript. So, you
know, on the the invocation semantic of
HTML is append child or set attribute.
So, we implement that, so you can
basically use you can construct custom
DOM elements, meaning things that have
the same semantic as uh a built-in
uh native DOM element like div or span,
uh but you can construct it yourself and
then use it in markup.
And so, this suggests uh
you know, you can now use functions in
place of templates. So, no more printf
or printf-like things. You're dealing
with actual live objects that you can
compose uh in the same way that you
compose your markup originally. And
those things become essentially custom
elements that can then be in your
markup, and you know, somebody can use
those to build other things. So, this is
an example of that. We're making a foo
element here,
which takes any number of children
and makes a div with a a heading element
and then puts the children in.
So,
uh sequences get spliced in as children.
We have this uh this little macro here,
which allows you because there's it
supports two different um kinds of
attributes,
uh two different syntaxes kind of. Uh
you know, the key value uh pairs and the
map. So, in order to deal with that
efficiently, we have this little macro
that just collects the attributes into a
map and the children into a seek, and
then calls your body. So, th- this is an
example of that. So, here we have a
quick list uh custom element, which
basically rolls up all of its children
into an unordered list and wraps them
with uh list items.
And uh if I set the class on this thing,
it gets passed down to the unordered
list
that that it returns. So, this is
useful.
Uh
so, every page, which is the unit of
compilation for this thing,
uh
it has a thing sort of like a namespace
declaration. Um except instead of
specifying the namespace name, because
the page is the end result, you're not
going to be using it as a library, so uh
you actually specify where you want the
the page to reside in your document
root.
Uh
and H list uh hoplon refers in all the
things like div, span, and all the
things you want, so you don't have to,
you know, painstakingly refer them into
your namespace.
Uh
may I?
Absolutely.
For us something again.
Please.
Um so, Micha pointed out that we include
all of those things automatically for
you, the HTML5 in particular,
uh sort of built-ins. But, we also have
an SVG namespace, and some of our more
interesting work lately has been
working with SVG as if it were a Lisp,
which has been pretty interesting, and
that stuff is in hoplon.svg namespace.
So, here's a sample uh page declaration,
and you can see the refer hoplon on the
second line there. That's basically like
refer closure. If there's something that
hoplon's bringing in that's, you know,
annoying you, you can just exclude it,
and you won't see the warnings. So, like
div and span and stuff.
DOM basically, the document object
model, is an IO mechanism.
Input output. That's the only reason it
exists, and
uh events are input. In other words, the
user clicks on something,
and uh elements and their properties
provide output. You know, you might make
something red by changing a property,
you might change the contents of a text
node.
But, what's interesting about this is,
if you think about it in terms of like
one-way data flow,
input and output never have to mix. So,
you could, you know, have one
uh
you could have one operation that's
setting
properties without knowing at all about
what events are are listened to and vice
versa.
So,
basically callbacks trigger updates to
input cells.
Meaning I click on a button, that causes
some input cell to change its value. We
swap on it in the callback.
And that eventually may or may not
percolate through formula cells
to cause an update to the DOM. In other
words, some DOM element might be hooked
up to some wired up to some to some
formula cell, and that formula cell, you
know, computes that it should be red.
So, here's an example of that.
I'm defining a cell here, an input cell
called clicks, whose initial value is
zero. This is how many times I've
clicked on this button.
Then I have a button, and I have the
style attribute, which is being hooked
up reactively to that clicks cell over
here.
So, I'm making anonymous
formula cell here,
and I want to set the color of this
thing to be red if the number of clicks
is even, and blue if it's odd.
Then I set a an on-click handler.
So, here is the callback, which swaps on
clicks, adding one to it, incrementing
it.
And then I set the text of this button
to inform us how many times it's been
clicked on.
And it rules. Totally. And it does
things.
So, what's interesting here is that um
you know, these attributes, basically
they're dispatching on value here. So,
if it sees a cell as the value, it knows
hook this up reactively to this cell.
If it's a function, then it says, "Okay,
this is a callback, so I'm going to
treat this, you know, I'm going to look
for the click event."
And
if it's neither of those, if it's a
string, then it just sets the attribute
on that element.
Uh the attributes,
these special attributes can be
extended. There are two multimethods, do
bang and on!bang, which handle input and
output. So, you can add your own fancy
thing. So, here's an example of like the
slide attribute. So, you can make
something like slide up or down. Or, you
know, the swipe left uh event. You might
hook up to like Hammer.js or something.
So, you could just implement that with
this uh
with this multimethod, and then you can
use those things as if they were already
included in Hoplon.
One cool thing about that is that what
he just showed that multimethod, it lets
you add attributes to components or
object custom elements that you don't
own. This differs from Angular where you
can add custom attributes, but it
involves scopes and various machinery.
And web component specs, which also
require that you own the element. So, in
this example, he's actually
teaching elements that had no idea what
swipe or any of this stuff is, how to
respond to these kinds of events uh
without modifying them. So, this is kind
of an example of a
What's that like problem?
The expression problem. Yes.
That thing.
And when HTML is a program, which it is
here, you can do all kinds of neat
things. So, here's a a function called
defvalues, which is basically like in
let you can destructure, you know, the
binding form and the expression. So,
this is basically just a thing that does
that, but it defines the things instead
of binding them locally. So, we have a
function called tabify, which I'm not
showing here, but basically what it does
is it takes pairs of things. And if you
think about tabs like on the web, you
click on something and it shows content
certain content and hides other content,
right? And then you click on another
thing and it shows the content
associated with that tab, right? So,
there's like a
an interesting way to generalize this.
So, if you imagine that the real
abstraction here is a number of triggers
and a number of contents, and you just
want to set a relationship between these
things that when you click on a trigger,
the the associated content shows itself
and the other contents hide themselves.
And uh when you click on the other
trigger
you know same thing happens. So here we
have
basically we call this this function to
tabify these things, but it returns them
back. It doesn't compose them.
And this is an interesting thing because
we've been able to decouple
the relationship between the things and
their internal state from their
composition in the DOM.
So the UI guy can use this this function
to create the tabs kind of relationship
between elements, but he might not want
to put them into an unordered list here
like I did. He might want to put like
one trigger way over here and one
trigger way over here, and it might not
look like tabs at all. It's just, you
know, essentially a tab-like
relationship, and he can express that,
and he can make his own custom elements
that, you know, use this as an
underlying lower-level thing. So you can
progress from simple things to more
complex in a in a nice way.
You can also do things like this. So
here, a lot of times people will do
like, you know, div and give it a class
red, and that means that, you know, it
should be red, all the things with class
red.
And but that's like a weird kind of, you
know, that that CSS, especially with the
cascading, is like a weird kind of
scoping system and aspect-oriented
programming. You know, you're setting up
a scope in which, you know, and and
really the the purpose of it is to
manipulate properties of objects, you
know, like I want to set the style.color
property.
So a more direct and powerful way is to
actually define a thing called red div.
And since all these things implement I
fun, and they're all invocable,
we can just use that in the DOM. And so
that can you can use lexical scope
there. You know, red div might mean a
different thing in one context than
another. You can use dynamic scope to
figure out what color it should be. All
the things that that Closure lets you do
already.
It's more powerful than CSS, I believe.
So Castra is the back end that we use.
And this is a an RPC
an RPC
uh middleware.
and a ClojureScript client library.
Uh it uses EDN back and forth, so you
call it like functions on the client,
and uh you know, you give it
ClojureScript data.
You don't it's pretty much transparent.
There's no
uh
you're not making anything be JSON or
anything like that. Um it has a foreign
assist per CQRS state machine pattern,
and uh it's great with Datomic because
the CQRS pattern is uh
really amenable to polling, which
Datomic is really good at. Like, reads
are free, so polling is a lot cheaper
than if you're calling into, you know,
like MySQL or something.
Here's the picture, basically, of what's
going on. In the client, you call this
function.
Goes through
uh our little servlet uh
our middleware in Ring.
And the RPC handler
looks at authentication annotations,
calls an RPC function in your API
namespace,
and that's basically for side effects
only. It's going to manipulate the
database in some way.
It always returns the same exact query.
So, if you were a shopping cart site,
this query might be what's your user ID
and what's in your shopping cart, say.
But, that doesn't depend It doesn't
matter what what the uh endpoint was
that you called. You call login, you
call, you know, buy a thing. Whatever
endpoint you call, it always returns the
same exact query.
This makes things very simple because
now your your AJAX call doesn't have to
know what to do with the response.
It just fires and forgets. All the
responses come back to the same handler,
which updates the global state of of the
um
of the client.
So, the API endpoints are just regular
functions, but they have these
annotations on them, which are similar
to Clojure pre- and post-conditions.
Uh
this allows you to to compose API
functions uh without mocking any state.
So, it separates the authentication from
the invocation. So, the authentication
only happens when it comes in on the
HTTP
uh as an HTTP endpoint.
Uh and also exceptions can percolate out
to the client and be caught.
So, this is an example of that.
It's a map where you give it a
precondition and a query, which is what
it returns at the end.
And you can call these things from the
REPL, and they work just fine. Uh they
work exactly the same way. You don't
have to mock any state at all.
This is a example application of what it
would look like.
And on the client side, you create
generally three Javelin cells, one for
the the state of your little state
machine, one for any errors that come
back, exceptions, and one for how many,
you know, a a sequence of currently in
progress async calls. Cuz you might want
to know if something's still loading or
not.
Uh you construct the remote endpoints
and call them like functions. So,
basically, there's this make remote
function that allows you to construct
the endpoint and hook it up to the
cells.
Then, you just call it
like over here, you call it like a
function. And you don't have to worry
about what it returns. You don't have to
This part right here doesn't have to
know how the rest of the application is
affected. That percolates up through
formula cells automatically.
Here's a a picture of of the
architecture of what I just described.
Does anyone have questions?
Thumbs up, thumbs down. I don't know.
How do we feel? Yes.
Is Castra a little bit like Short Leaf?
Can you say that again?
Is Castra a little bit like Short Leaf?
The question is, is Castra like Short
Leaf?
Well, it's an RPC framework, but
um
the um the annotations that we do to
separate all the things that have to do
with the session, basically,
into a separate layer that
I think that's a little bit different.
Um so, like when you're at the REPL,
there's no session that you worry about.
You don't have to mock anything. And
everything runs the same. So, you can
test things. You can You can work the
app from the REPL, which I do a lot when
I'm developing.
Funny side note, the original version of
Castra was written in PHP. Is that true?
Oh, we won't talk about that. Sorry. No,
other other questions? Yes.
Uh you seem to have gone over like the
example code.
Can you go back to that?
Sure. This one?
No, this
This one?
Yeah.
Okay. I'm sorry we don't have Let me try
and
There we go. All right.
So, Castra define uh
exports this uh defn macro, which is
really the same as defn, but it just
implements these uh you know, these
annotations here.
And the precondition, generally the way
we do it is we set up a separate
namespace for HTTP rules, and it's
basically like Turing complete, you
know, authentication. You just call this
thing just like a precondition, and you
can throw an exception from those rules,
which will be caught in the client.
But, that only happens at the acts at
the actual endpoint. In other words, if
I call the the login function from the
client,
it will run this precondition.
But, if login itself calls another API
function
internally, like sometimes when when you
register someone, you want to log them
in then, right? So, it'd be nice to be
able to just call that API function, so
it doesn't trigger any of the
authentication stuff there.
You know, it just calls right through.
And uh so, the query uh the query
annotation here is so that you could
return that CQRS state query.
And this allows you at the REPL uh this
function could actually return one thing
at the REPL, which might be useful for
you when you're coding it, but when an
HTTP client connects, they get back this
state, you know, this state map.
Yes.
What's the CQRS?
Question was, what is CQRS? I can take
that one.
Yeah, please.
Yeah, so um all of this is to serve one
end, which is basically to be able to
conceive of when we're programming the
server as part of our spreadsheet. So,
the same way um someone who's interested
in making charts over an existing
spreadsheet can just use the existing
formulas without necessarily
manipulating any of the data or
formulas.
We want to look at the server the same
way. So, in our case here, the server is
the source of a value, which is the
current user's state.
Um we would like to be able to take that
value and aim it at UI elements, like
this little box here should be the
person's name and this should be the
number of buddies or whatever.
Um we want to con- consider the server
as like kind of a remote cell.
CQRS uh stands for command query
response separation.
Is that ish?
Martin Fowler. Martin Fowler has a good
Well,
Yeah, Martin Fowler has the write-up. Uh
Anyway, if you type in CQRS Martin
Fowler, you'll you'll get to the one of
the the source documents. But,
basically, it's a it's a fancy term for
the same kind of pattern. Where I have a
source of events, which in our case is
the user. Those events are streaming
into some remote remote data store,
Datomic. And we have formula cells,
which are calculating the user's state.
Um and those would be Datomic queries.
We periodically want to update the UI
based on what we pull from Datomic. We
want to do that in a a unidirectional
data flow kind of way.
Yeah, I think I think to think about is
like if you have a a function that does
an Ajax call to the server, like the
login function, a lot of times you'll be
saying, "Okay, here's a callback and,
you know, if the server returns true,
then I'm going to do this. And if it
returns false, then I'm going to do
that, right?" So, but the things you
generally want to do is, you know,
change the state of the application,
like I want to show you uh an error, a
flash message, like incorrect password
or something, right? But, that means
that like this piece of code that sends
off the request to login, now needs to
know about what to do
with the response.
And it has to know where to go to do
that, right? And it has to now cooperate
with these other parts of your
application. And now imagine that you
have 10 buttons on the page, right? And
you could click any one of them in any
order. So, now you run into all kinds of
synchronization issues like, you know,
should it Do I need to disable this
button when when I press this button and
so on? So, now all these things have to
know about each other very intimately.
And then you change something and you
don't know what's going to happen.
Whereas with CQRS,
things just fire and forget. You send a
message to the server and what comes
back is the next state of a state
machine, which could actually be the
same as the current state. Like, nothing
might happen on the server that you care
about, right? And so then nothing will
change. Or something could happen, but
you don't need to know about it here.
There's something that's looking at our
state model and saying, "Okay, when
there's stuff in the shopping cart, I
want to show the shopping cart."
Yes.
Can you
talk
briefly about
how that process works? So, when you get
that new state, how does that then get
reflected
in the document?
Question, I think was paraphrasing how
to integrate the server's state with the
client state. How do How do we do that?
Like, how does it get into the How does
it reflect How is it reflected in the
document? Is what you want to know.
Right. So,
Right.
in the demo yesterday, you know, that
was using React
and they're figuring out this
integration strategy.
I use Do you re-render?
Right. Yeah, I can I can go for
Yeah. Yeah, so in general, there are two
approaches to
doing this thing that we need to do. And
I I think React represents one approach
and everything we're showing you
represents another.
The basic bifurcation is between two
ways to repaint the UI.
One is faster, one is slower.
One may may or may not be inconsistent,
one is guaranteed to be consistent.
Those are the trade-offs.
The React approach is the same one that
Angular uses, but slightly
refined. It's called dirty checking. And
historically,
people who have been writing UIs using
this approach, I think there's a common
list system called Garnet or there's
whatever, there's papers there's papers.
Check them out. Um
Uh
the thing you get is speed because you
don't need to check everything that
you're trying to update for equality.
Like that's the essence of data flow is
we spin through these values that are
coming in and if they're different than
ones we knew about, we need to do stuff.
With dirty checking, you just say, "Hey,
if anything in the tree changed, we know
that stuff below we're just going to
blow away anything that existed below it
without checking it for equality."
So you get speed there.
On the other side, which is what Javelin
does and which is what spreadsheets do,
we do equality checking for every value
that gets pushed in. And we don't
calculate diffs. There's really no the
same way there's not really a concept of
a diff in a spreadsheet, there isn't in
Javelin.
What we've found in in our efforts
around this and the reason we went with
spreadsheet style data flow instead of
dirty checking is that
you need to be able to write components
that compose with each other under a
semantic.
Like the reason we invented this whole
thing is because we had a product with a
front end. And we had the it was for a
meal delivery application.
We needed to rebrand the the
Uh well actually, this is stuff Misha
worked on before I joined. I I just
showed up at the very end and like took
all the credit. But it was an
application it was like a 10 billion
line PHP jQuery shenanigan. And um
it it it had all the logic in it
encoded, obviously, but it was
impossible to maintain. And then the
requirement came down, "Hey, take this
business logic, but let's white white
label it. Like instead of this being
meals for, you know, this demographic,
we want to ship meals to these people.
Is that possible?" This came down to
Misha
and
I guess your short answer was no, it's
not possible to pull anything reusable
out of this app because the logic and
the view are just so deeply intertwined.
Yeah, and I think to point out also is
that so about the dirty checking
the way Javelin works is you can imagine
a graph of cells, you know, that are
kind of organized by the relationships
to each other. In other words, a formula
cell might depend it might refer to
other cells. So, that forms a graph, but
it's this this directed graph and
no no cell will will trigger any updates
if its value doesn't change. So, there's
like very aggressive pruning that goes
on just naturally. In other words, if if
an input cell changes it might cause one
formula cell to change, but if that's
where it ends then that's where it ends.
So, if you have a lot of DOM elements
that are looking at all these other
formula cells
nothing happens because no values
changed. However, if the values did
change, then you definitely need to
update those DOM elements anyway, no
matter what because they're no longer,
you know,
consistent. So,
I think
another thing to look at
comparing these two approaches is like I
look at the DOM as this input output
device.
And React seems more to me like a buffer
kind of like Emacs. You can type around
in Emacs and Emacs has, I'm sure, all
kinds of clever tricks to avoid
repainting the entire thing when you
type,
but it's still the abstraction is still
a buffer, you know, that's a sequential
thing.
The DOM is a little bit richer than that
in that it gives you kind of more like a
file system of buffers. You know, you
have this tree of buffers that you can
kind of read from and write to
individually, right? And so, now you can
you can be treating each each DOM
element as a buffer and doing things to
them individually. And once you have if
you're working with the the DOM element
itself, right? And like everything
involving IO is stateful. When you have
a file handle, that's a stateful thing
because you might write to it and it
needs to keep track of its buffers. It
needs to keep track of, you know,
whether things are flushed to disk or
whether you could read from it or write
to it or what. So,
DOM elements actually act the same kind
of way and so you can do like in that
tab example where you can pass DOM
elements to a function that doesn't know
what they are at all.
And they just apply magic to them, and
then these DOM elements remember the
magic. In other words, attaching an
event handler to something is a mutation
cuz it has to keep track of what it
needs to notify.
So, that's a very powerful thing when
you're working with the actual DOM
elements because you pass them to this
one function that just arranges some
relationship between them and passes
them back. You pass them to something
else. You might be mixing elements
around, and then you compose them later.
So, you're not, you know, you're not
bound to any particular composition of
them.
I think actually we're out of time. Um
I think we should leave you with what
Misha just said, which was remember the
magic. That sounds cool.
Um but we'll be around if you have
questions, want to chat. Uh hoplon.io is
the place to go and learn.
Um thank you so much for your time, and
thank you Alex and everybody else.