[Ur] Ur/Web in production

[Vladimir Shabanov]

Hi,

I want to share my experience with Ur/Web in BazQux Reader.

BazQux Reader is an advanced RSS reader that thousands of paying customers
are using everyday (https://bazqux.com). It's not an academic or school
project. Yet it is written in experimental language. Why?

Here's a common code pattern frequently found in my app:

val main = (* function-page accessed from example.com/main *)
  model <- source "Model"; (* dynamic client-side data *)
  return <xml>
    This is a view with typed server-side templating
    <dyn signal={m <- signal model; return <xml>
      And dynamic client-side templating.
      Current model value: {[m]}</xml>} />
    <div class="myButton" onclick={
      fn _ =>
        x <- rpc (getNewDataFromServer "param1");
        (* AJAX looks like usual function call *)
        set model x
        (* updated model, view will be updated automatically *)
        }>Click me</div>
    <a link={otherPage "param2"}>My link</a>
    (* link is also a usual function call,
       checked for existence and types *)
    {myWidget} (* nested template -- just call another function *)
  </xml>

>From my point of view it is excellent approach for programming interactive
web apps. No difference in programming client or server side. No
JavaScript. Everything is statically typed (easier to refactor, no stupid
errors). XML is a first-class entity that can be arbitrary mixed with code.

Above example can be easily used as an interactive widget:

val anotherPage =
  x <- signal <xml/>;
  return <xml>
    <dyn signal={signal x} />
    <button value="Click me!" onclick={fn _ =>
      interactiveWidget <- main;
      set x interactiveWidget
    }/>
  </xml>

 It's very clean and simple way to structure web app. No templating
engines. No low-level imperative DOM modifications. No need to build XML
string, change DOM, assign event handlers. Single function can create
complex interactive web fragment that can be used later in a functional
way. Most of my app is structured using this approach.

And bonus -- SQL is also statically typed first-class citizen here!

fun getNewDataFromServer x =
  r <- oneRow (SELECT t.Field1 FROM t WHERE t.Field2 = {[x]});
  return r.T.Field1

Ur/Web main page http://www.impredicative.com/ur/ focuses on safety,
metaprogramming and speed. But I think it is the simplicity of writing
plain web code that makes Ur/Web so cool. Advanced types and
metaprogramming are cool things too. But high-level functional language
with algebraic data types, pattern matching, first-class XML, SQL and very
simple AJAX is the thing I love the most. After half an hour of looking at
Ur/Web demos at http://www.impredicative.com/ur/demo/ I was sure in what
language I'm going to write my app.

There is an interesting parallel with Haskell. It mostly advertised for its
advanced types but it is conciseness and easy refactoring of code that
makes it appeal for me.

Hope it's clean now why I chose Ur/Web. But how it plays in the real world?

First problem that is visible even before you start coding is libraries.
There aren't many of them. At first I've solved this problem by writing
feeds fetcher in Haskell and making Ur/Web part only serving ready data
from Postgres. Ur/Web is very good at UI, Haskell is very good with
concurrency and complex data processing. Best of both worlds.

Then I've switched from Postgres to Riak (needed to scale writes since RSS
reader is very write heavy thing and to simplify cluster operations) and
started to need more data processing and 3rd-party integration in frontend.
So my Ur/Web app now links with Haskell and calls its functions via FFI.

I've published Ur/Web sources and Haskell interface on GitHub
https://github.com/bazqux/bazqux-urweb
so you could look how it's done (local fake 'gcc' script that calls GHC and
data types/FFI/serialization generator).

Again, best of both worlds. Need some library (authentication, mail, Riak)
or complex code -- use Haskell. Need to write interactive UI without
JavaScript -- use Ur/Web.

Ur/Web doesn't have ready to use client-side widgets. But it's very simple
to create your own. Anyway, no web framework has widgets for subscriptions
list or news feed with multiple viewing modes.

One client-side problem I've had is too many <dyn> elements in
subscriptions widget. It's OK to have thousands <dyn>s when they're added
incrementally (like in news feed). But it makes too many DOM modifications
when all of them are added at once (OK on desktop but slow on mobile
browsers). So I'm generating subscriptions list HTML on server and using
some JavaScript to work with it.

I have quite a lot of JavaScript (subscriptions list with drag and drop,
article HTML postprocessing, autocomplete and little utilities) but it's
still only 1/3 size of Ur/Web sources (and Ur is a more compact language).
And I've got few mysterious errors here (sorted ints like strings, forgot
argument in rarely called function) that were found only weeks later.
Happily, most JavaScript FFI functions are small utility ones and are
easily tested. But it's hard to imagine how much work is needed to build
app of my scale in pure JavaScript. Ur/Web static typing really helps a lot
at scale.

Few major problems with Ur/Web:

Slow compilation speed. There was a moment when it took 10 minutes to
compile few thousand lines of code. After some patches by Adam compilation
time improved and it now takes about a minute to compile several times more
code. Still too much (especially when you making some little layout or text
changes) but acceptable. It's better to wait a minute and be sure that you
didn't make some stupid mistakes than to continuously switch between the
editor and the JavaScript debugger.

Exponential code bloat. This is actually the main reason for long
compilation. Ur/Web inlines too much. For example I have a big function
that reloads current feed and it can be inlined (and can be not) at each
call site. It can easily double compilation time and generated JS code
size. Things got improved a bit lately but I'm still putting this function
to variable and reading it before call instead of direct call to disable
inlining.

One more negative part of superfluous inlining is a big generated
JavaScript strings with Ur/Web bytecode. Some browsers do not love to eval
0.5M string full of nested JS expressions. I'm using a small hack to
overcome it. Few big functions (subscriptions and feed widgets) are made
recursive with always false condition (to never actually recurse). That
forces Ur/Web compiler to not inline them and put their bytecode to a
separate strings.

I hope that with some more tweaks from Adam this compilation issues will go
away completely.

Compiler bugs. As you can expect from a new and complex tool there were
several bugs in compiler and runtime library. Most of them are in the past
now. However I'm still not sure about one nasty bug when some effectful
computation were optimized out (click on button and part of event handler
is not executed at all, whoops). It's the most annoying bug I've met in
Ur/Web. Usually solved by some code reordering or by calling it via
JavaScript FFI function (look for 'forceImpure' in the sources). Hope it's
fixed since I haven't seen it for a while.

During these long and buggy compilations I've sometimes thought to rewrite
everything in pure JavaScript. But every time I've decided than it's better
to ask Adam or to fix the bug myself than to implement "ad hoc,
informally-specified, bug-ridden, slow implementation of half of Ur/Web" ;)

Other problems are less significant but still quite inconvenient:

Error messages can be huge. Full types of big records when you've just
mistyped field name. AST of a big chunk of code with all type annotations
when you have small type error in the middle and so on. It usually helps to
add some type annotations when you see that the error "is somewhere here".
But I would like to see more readable error messages. Haskell had the same
problem and solved it. Hope Ur/Web will solve it too.

Weak pattern matching. There are no named patterns, no pattern guards, no
pattern matching in "do"-notation (impossible to write (a,b,c) <-
someCode), no view patterns.

FFI requires too much friction. It would be great to see Fay-like FFI. Or
at least have a possibility to define foreign functions inside .ur-file
(where I can put any local datatype as an argument) instead of interface
file.

Ur/Web speed? Wasn't an issue at all. Some early load testing have shown 2K
requests/sec for a dynamic page that makes a few requests to database.
That's about 100 times more than Hacker News have so it's more than enough.
There were performance issues with database and feeds fetcher but not with
Ur/Web itself.

In conclusion I would say that Ur/Web is a great idea and has a great
implementation of its core. But there are some rough edges. With faster
compilation speed, improved FFI, better pattern matching and error messages
it could become a choice for discriminating hackers. But why wait? Try it
today just to see how the future of web development can look like.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.impredicative.com/pipermail/ur/attachments/20140117/6676bdb4/attachment.html>