Graftid

From Impredicative Wiki
Revision as of 13:23, 11 February 2010 by Adam Chlipala (Talk | contribs)

Jump to: navigation, search

Graftid is a platform for developing and hosting web applications using Ur/Web, targeted at both programmers and the general public. These two audiences naturally need very different views of the service. The standard view is meant to be self-explanatory; this wiki page documents the picture for developers.

Graftid is in a closed alpha right now. The main web site is password-protected, while hosted applications can be accessed by anyone. If you are interested in testing, please e-mail Adam to ask for access. Note that things are slightly complicated so that testing can exercise the account creation code: the initial password you get will allow you into the site, but once there you will still need to create a Graftid account.

Contents

Overview

At a high level, Graftid is a repository of application designers, which are like browser-based GUI wizards for building particular kinds of web applications. Developers upload designers, which are written in Ur/Web and also generate Ur/Web code. Ideally these designers do very little ad-hoc code generation, instead serving merely to build calls to metaprogramming libraries, which can be implemented in Ur/Web such that static types guarantee they work properly on any valid inputs. To layperson end users, these details are invisible. They just see web forms that can be used to bring new database-backed web applications into existence.

Graftid can store a tree of libraries, owned by different users and with different versions available at once. This tree is much like a standard filesystem. Its main aspects are:

  • Users, corresponding to accounts created by people
  • Groups, named sets of users
  • Directories, in the usual UNIX-y sense, with every directory but the root directory having another directory as its parent. Different users and groups may be assigned read, write, or admin rights on a directory. Admin rights make it possible to edit the permission matrix. Groups themselves belong to directories, so that directory permissions control who may modify a group's member set. All of the following kinds of things that belong to directories have similar inheritance of directory permissions.
  • Libraries, named groups of Ur/Web modules, situated in directories
  • Library versions, particular sets of source files that may be linked to build concrete applications. Each version is associated with a particular library, but no formal connection is enforced between different versions of one library.
  • Designers, GUIs for building applications from particular libraries, situated in directories
  • Designer versions
  • Applications, specific persistent applications, situated in directories
  • Application versions

Graftid seems to be a new kind of service, to the extent that it's hard to describe succinctly. I recommend getting your site access password and exploring for a little while without logging in, to see what the average (non-programmer) visitor would experience. After that, create an account and come back here to pick up with the next section. Keep in mind that nothing that follows needs to be understandable by non-programmers; such folks will only see what you saw in your initial explorations while not logged in.

Programming GUIs

To get started coding Graftid application designers, you will want to download and install (i.e., unpack somewhere) the GUI library. This library defines some types and module signatures that you will need to use or implement. First, since the mission of a designer is to generate Ur/Web code, we have the Ur module, which gives a representation of a subset of Ur/Web syntax.

(* Constructors, i.e., type-level data *)
datatype constr =
             TFun of constr * constr
             (* The usual function arrow *)
           | TRecord of constr
             (* Builds a record type from a constructor of soome kind {K} *)

           | CVar of list string * string
             (* Variable reference, with CVar (m1 :: ... :: mn :: nil, x)
              * meaning m1.[...].mn.x (for module names mi) *)
           | CApp of constr * constr
             (* Constructor-level function application *)

           | CName of string
             (* Field name literal *)

           | CRecord of list (constr * constr)
             (* Explicit record fields, as pairs of names and values *)

           | CUnit
             (* The trivial constructor *)

           | CTuple of list constr
             (* Constructor-level tuple construction *)

(* Value-level constants *)
datatype prim =
         Int of int
       | Float of float
       | String of string
       | Char of char

(* Value-level expressions *)
datatype exp =
         EPrim of prim
         (* Constant *)
       | EVar of list string * string
         (* Variable reference, encoded as for CVar *)
       | EApp of exp * exp
         (* Function application *)
       | EAbs of string * exp
         (* Function abstraction *)
       | ECApp of exp * constr
         (* Applying a polymorphic function *)
       | EDisjointApp of exp
         (* Applying an abstraction over a row disjointness constraint *)
       | ERecord of list (constr * exp)
         (* Record construction *)
       | EField of exp * constr
         (* Record projection *)
       | ELet of string * exp * exp
         (* Local definition *)

(* Declarations *)
datatype decl =
         DCon of string * constr
         (* Constructor synonym *)
       | DVal of string * exp
         (* Value-level binding *)
       | DValRec of list (string * exp)
         (* Mutually-recursive value bindings (each exp must begin with EAbs) *)
       | DStr of string * str
         (* Structure (i.e., module) definition *)
       | DOpen of list string * string
         (* Import a module path *)
       | DOpenApp of list string * string * str
         (* Import a functor application *)

(* Modules *)
     and str =
         StrConst of list decl
         (* Structure with known contents *)
       | StrVar of string
         (* Variable reference *)
       | StrProj of str * string
         (* Project out a sub-structure *)
       | StrApp of str * str
         (* Apply a functor *)

type file = list decl

The other piece of the picture is a signature for widgets, which are the building blocks of designer GUIs. They come from the Gui module.

(* Signature for widgets that may be used in designers *)
signature WIDGET = sig
    type ast
    (* What sort of syntax does this widget generate? *)

    type widget
    (* Internal state type *)

    type persistent
    (* Persistent state type, to serialize/deserialize for long-term storage *)

    val default : persistent
    (* What state should a fresh widget of this variety have? *)
    val create : persistent -> transaction widget
    (* Initialize an ephemeral version of some persistent state. *)

    val persistent : widget -> signal persistent
    (* Use this to listen for changes to a widget's persistent state. *)
    val push : widget -> persistent -> transaction unit
    (* Update an existing widget to match some new persistent state. *)

    val render : widget -> xbody
    (* Display the GUI for a widget. *)
    val generate : persistent -> ast
    (* Translate state into Ur syntax. *)

    val testCase : persistent
    (* A good example state to use in testing that the widget is implemented
     * properly.  When you upload your widget, Graftid will run a test
     * compilation using this state, rejecting your upload if anything goes
     * wrong. *)
end

signature EWIDGET = WIDGET where type ast = Ur.exp
signature FWIDGET = WIDGET where type ast = Ur.file

Setting Up a Development Environment

Grab the grmake script and install it somewhere in your PATH. Edit the file ~/.graftidrc and add definitions for the following variable, on separate NAME=VALUE lines.

  • GUI: the directory where you installed the GUI library (that is, this directory should contain the full contents of the unpacked archive that you downloaded)
  • MODULES: a directory where grmake should maintain a local cache of the source to libraries pulled from the Graftid web site
  • BASIC: a pair USER:PASS, giving the username and password you were assigned for accessing the main Graftid site, not your Graftid account information

The curious can poke around the grmake source to find some other variables that you might want to set, but these three should be enough to get started.

You will also need to have the normal Ur/Web distribution installed. Sometimes the latest release will be missing some important features, so it's best to keep up to date with the version in the public Mercurial repository. The usual ./configure; make; sudo make install should work for installation. If you run into problems, see the Ur/Web manual.

Testing Graftid GUIs requires that you have a supported database client library installed. By default, grmake uses SQLite, which, for example, requires package libsqlite3-dev in Debian or Ubuntu. You can set the variable DBMS to postgres or mysql to use of one those systems instead. While these latter options require a database server in general, grmake will work fine without any database server running.

One last thing you'll want to make sure to do is set your Graftid account to use "Programmer" mode. You were given the option to select this on sign-up, and you can change it later via the "Preferences" link at the top of each page.

Examples

Hello World

Let's build a module for a family of web applications that just print "Hello" followed by an arbitrary string. Create a new (empty) directory where you will develop the source and create a file hello.urs with this content:

val main : string -> unit -> transaction page

In other words, given a string to display after "Hello", our main function returns an entry point to the application.

Now create hello.ur:

fun main s () = retun <xml>Hello {[s]}!</xml>

Finally, create lib.urp, the project file tying together the Ur sources:

hello

Now we are ready to try building our library.

$ grmake
/dir/hello.ur:1:16-1:21: Unbound expression variable retun

Oops! We made a typo in the hello.ur code. Change retun to return and try again.

$ grmake

This time no messages are printed, so our library checked out. Now we can package it for uploading to Graftid.

$ grmake package

A file lib.tgz appears in the current directory. To upload it, log into Graftid and follow the "Home" link in the top toolbar of any page. This brings you to the main page for your user's home directory. Under the "Libraries" heading, follow the "New library" link. On that page, choose "hello" as the library name and enter whatever descriptions you want for the library overall and for this first version of it. Select the lib.tgz file we just built as the package to upload. Submitting the form should create the library quickly, and it will then show up in your home directory's "Libraries" list.

Hello World Designer

The last example was just the first half of building a "Hello World" application designer. In some other new directory, let's create a GUI that builds programs based on Hello.main. We will build and upload a new library handling the details, and then we will combine this library with the Hello library to create a designer.

In file main.urs:

include Gui.FWIDGET

In other words, the Main module will be a widget for building Ur/Web source files.

In main.ur:

open Ur

type ast = file
type widget = source string
type persistent = string

val default = "world"
fun create s = source s

fun persistent w = signal w

fun push w s = set w s

fun render w = <xml><ctextbox source={w}/></xml>

fun generate s = DVal ("main", EApp (EVar ("Graftid" :: "User" :: "$You" :: "Hello" :: "Hello" :: [],
                                           "main"),
                                     EPrim (String s))) :: []

val testCase = "test case"

This widget appears only as a textbox, and the value entered in the textbox is what will be displayed after "Hello" in the final application. The generate function is most interesting, as it demonstrates how to reference another Graftid module. In Ur syntax, if your username is you, then the name of the Hello module we just created is Graftid.User.You.Hello.Hello. To get the above code to work, you must replace $You with your username, capitalized. Hello appears twice: first as the library name and second as the module name. In general, the whole Graftid directory structure appears as an Ur module called Graftid, with directories included recursively as sub-modules.

Finally, we create lib.urp:

library $GRAFTID/gui

main

We explicitly import the Graftid GUI library, as well as including our Main module. Every GUI must have a Main module that implements FWIDGET. Note that the text $GRAFTID should be included literally in this file.

Let's try to build our designer.

$ grmake gui
/tmp/.graftid_test/main.ur:1:12-1:47: Unbound structure Graftid

The error message you see might reference a different unbound module, depending on the state of your local module cache.

grmake used the testCase value in our designer to try to build a real application. Unfortunately, it encountered an error. The generated code references a module Graftid.User, but somehow the Ur/Web compiler is unaware of that module. The solution is for us to be explicit about which modules generated code will need. In a file deps in the same directory, write this line:

user/$you/hello/1

This indicates the path to a library, followed by the version number you want. You must replace $you with your username, not capitalized.

Next we ask grmake to make sure this version's source is in our local cache:

$ grmake login $you
Password: 
$ grmake deps

Since your home directory will be private by default, you need to log into Graftid first to be able access your libraries. This leaves a cookie stored in ~/.graftid_cookies by default, and you can change that location with the variable COOKIES in ~/.graftidrc. The grmake deps command does the actual fetching of package source.

Here a brief digression on security is warranted: Ur/Web and grmake are designed so that no serious security problems can result from pulling arbitrary libraries from Graftid. The basic Ur/Web language abstractions limit what applications can do, and grmake should rule out potentially dangerous features like the foreign function interface. Barring tool bugs, the worst that can happen is that an Ur/Web application runs for too long, uses too much memory, etc..

Now we can build, test, and package the GUI without incident.

$ grmake gui
$ grmake package

As before, this creates lib.tgz, and you can upload it to Graftid in the same way as in the last example. The following instructions assume that you call the new module ghello.

On your home directory's page, follow the link "New designer". Make up whatever name you like for your "Hello World" designer. For "Library/version", enter user/$you/ghello/1. Next, click the "Add blank" button to add user/$you/hello/1 as an app dependency. Add any description you care to add and then submit the form.

If all goes well, you will be taken to the page for your new designer, listing a single version. You can follow the link from the version number and then follow the link to create a demo application. From that point, everything should be self-explanatory, as it's designed to be usable by random web surfers.

Compile-Time Dependencies

The last example was a designer that produced code which referenced a particular module. It is also useful to reference Graftid modules in the code generator itself. For instance, here is how we could build a thin wrapper around hello, adding an extra exclamation point.

In hello2.urs:

val main : string -> unit -> transaction page

In hello2.ur:

fun main s = Graftid.User.$You.Hello.Hello.main (s ^ "!")

In lib.urp:

library $ROOT/user/$you/hello/1

hello2

Note that the text $ROOT should be included literally in this file.

We can build like in the last example, fetching the source to dependencies before proceeding. This time, it's the contents of lib.urp rather than a file deps that determine what to fetch.

$ grmake deps
$ grmake
$ grmake package

A Simple Database Client: Increment

Here is the source for two modules that can be combined in the same way as above to create a designer. This designer produces applications that access persistent databases, incrementing a server-side counter on each hit. The same invocations of grmake will work with this example.

For the first module, incr:

In main.urs:

val main : unit -> transaction page

In main.ur:

sequence s

fun incr () =
    n <- nextval s;
    return <xml><body>
      Current: {[n]}
<form> <submit action={incr}/> </form> </body></xml> fun main () = return <xml><body> <form> <submit action={incr}/> </form> </body></xml>

In lib.urp:

main

For the second module, gincr:

In main.urs:

include Gui.FWIDGET

In main.ur:

open Ur

type ast = file
type widget = unit
type persistent = unit

val default = ()
fun create () = return ()

fun persistent () = return ()

fun push () () = return ()

fun render () = <xml>Nothing to see here.</xml>

fun generate () = DVal ("main", EVar ("Graftid" :: "User" :: "$You" :: "Incr" :: "Main" :: [],
                                      "main")) :: []

val testCase = ()

In lib.urp:

library $GRAFTID/gui

main

Getting Help

There are two mailing lists for Graftid discussion and help.

Personal tools