add webmail

it was far down on the roadmap, but implemented earlier, because it's interesting, and to help prepare for a jmap implementation. for jmap we need to implement more client-like functionality than with just imap. internal data structures need to change. jmap has lots of other requirements, so it's already a big project. by implementing a webmail now, some of the required data structure changes become clear and can be made now, so the later jmap implementation can do things similarly to the webmail code. the webmail frontend and webmail are written together, making their interface/api much smaller and simpler than jmap. one of the internal changes is that we now keep track of per-mailbox total/unread/unseen/deleted message counts and mailbox sizes. keeping this data consistent after any change to the stored messages (through the code base) is tricky, so mox now has a consistency check that verifies the counts are correct, which runs only during tests, each time an internal account reference is closed. we have a few more internal "changes" that are propagated for the webmail frontend (that imap doesn't have a way to propagate on a connection), like changes to the special-use flags on mailboxes, and used keywords in a mailbox. more changes that will be required have revealed themselves while implementing the webmail, and will be implemented next. the webmail user interface is modeled after the mail clients i use or have used: thunderbird, macos mail, mutt; and webmails i normally only use for testing: gmail, proton, yahoo, outlook. a somewhat technical user is assumed, but still the goal is to make this webmail client easy to use for everyone. the user interface looks like most other mail clients: a list of mailboxes, a search bar, a message list view, and message details. there is a top/bottom and a left/right layout for the list/message view, default is automatic based on screen size. the panes can be resized by the user. buttons for actions are just text, not icons. clicking a button briefly shows the shortcut for the action in the bottom right, helping with learning to operate quickly. any text that is underdotted has a title attribute that causes more information to be displayed, e.g. what a button does or a field is about. to highlight potential phishing attempts, any text (anywhere in the webclient) that switches unicode "blocks" (a rough approximation to (language) scripts) within a word is underlined orange. multiple messages can be selected with familiar ui interaction: clicking while holding control and/or shift keys. keyboard navigation works with arrows/page up/down and home/end keys, and also with a few basic vi-like keys for list/message navigation. we prefer showing the text instead of html (with inlined images only) version of a message. html messages are shown in an iframe served from an endpoint with CSP headers to prevent dangerous resources (scripts, external images) from being loaded. the html is also sanitized, with javascript removed. a user can choose to load external resources (e.g. images for tracking purposes). the frontend is just (strict) typescript, no external frameworks. all incoming/outgoing data is typechecked, both the api request parameters and response types, and the data coming in over SSE. the types and checking code are generated with sherpats, which uses the api definitions generated by sherpadoc based on the Go code. so types from the backend are automatically propagated to the frontend. since there is no framework to automatically propagate properties and rerender components, changes coming in over the SSE connection are propagated explicitly with regular function calls. the ui is separated into "views", each with a "root" dom element that is added to the visible document. these views have additional functions for getting changes propagated, often resulting in the view updating its (internal) ui state (dom). we keep the frontend compilation simple, it's just a few typescript files that get compiled (combined and types stripped) into a single js file, no additional runtime code needed or complicated build processes used. the webmail is served is served from a compressed, cachable html file that includes style and the javascript, currently just over 225kb uncompressed, under 60kb compressed (not minified, including comments). we include the generated js files in the repository, to keep Go's easily buildable self-contained binaries. authentication is basic http, as with the account and admin pages. most data comes in over one long-term SSE connection to the backend. api requests signal which mailbox/search/messages are requested over the SSE connection. fetching individual messages, and making changes, are done through api calls. the operations are similar to imap, so some code has been moved from package imapserver to package store. the future jmap implementation will benefit from these changes too. more functionality will probably be moved to the store package in the future. the quickstart enables webmail on the internal listener by default (for new installs). users can enable it on the public listener if they want to. mox localserve enables it too. to enable webmail on existing installs, add settings like the following to the listeners in mox.conf, similar to AccountHTTP(S): WebmailHTTP: Enabled: true WebmailHTTPS: Enabled: true special thanks to liesbeth, gerben, andrii for early user feedback. there is plenty still to do, see the list at the top of webmail/webmail.ts. feedback welcome as always.
2025-07-13 03:34:39 +03:00 · 2023-08-07 21:57:03 +02:00
parent 141637df43
commit 849b4ec9e9
106 changed files with 25741 additions and 734 deletions
--- a/vendor/github.com/mjl-/sherpats/LICENSE
+++ b/vendor/github.com/mjl-/sherpats/LICENSE
@ -0,0 +1,7 @@
+Copyright (c) 2018 Mechiel Lukkien
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
--- a/vendor/github.com/mjl-/sherpats/Makefile
+++ b/vendor/github.com/mjl-/sherpats/Makefile
@ -0,0 +1,24 @@
+SHELL=/bin/bash -o pipefail
+
+build:
+	go build ./...
+	go vet ./...
+
+test:
+	golint
+	go test -cover ./...
+
+coverage:
+	go test -coverprofile=coverage.out -test.outputdir . --
+	go tool cover -html=coverage.out
+
+fmt:
+	go fmt ./...
+
+clean:
+	go clean
+
+# for testing generated typescript
+setup:
+	-mkdir -p node_modules/.bin
+	npm install typescript@3.0.1 typescript-formatter@7.2.2
--- a/vendor/github.com/mjl-/sherpats/README.md
+++ b/vendor/github.com/mjl-/sherpats/README.md
@ -0,0 +1,31 @@
+# Sherpats
+
+Sherpats reads the (machine-readable) documentation for a [sherpa API](https://www.ueber.net/who/mjl/sherpa/) as generated by sherpadoc, and outputs a documented typescript module with all functions and types from the sherpa documentation.  Example:
+
+	sherpadoc MyAPI >myapi.json
+	sherpats < myapi.json >myapi.ts
+
+Read the [sherpats documentation](https://godoc.org/github.com/mjl-/sherpats).
+
+
+# Tips
+
+At the beginning of each call of an API function, the generated
+typescript code reads a localStorage variable "sherpats-debug". You
+can use this to simulate network delay and inject failures into
+your calls. Example:
+
+	localStorage.setItem('sherpats-debug', JSON.stringify({waitMinMsec: 0, waitMaxMsec: 1000, failRate: 0.1}))
+
+
+# Info
+
+Written by Mechiel Lukkien, mechiel@ueber.net, MIT-licensed, feedback welcome.
+
+# Todo
+
+- linewrap long comments for fields in generated types.
+- check if identifiers (type names, function names) are keywords in typescript. if so, rename them so they are not, and don't clash with existing names.
+- better error types? how is this normally done in typescript? error classes?
+- add an example of a generated api
+- write tests, both for go and for the generated typescript
--- a/vendor/github.com/mjl-/sherpats/cmd/sherpats/main.go
+++ b/vendor/github.com/mjl-/sherpats/cmd/sherpats/main.go
@ -0,0 +1,50 @@
+// Command sherpats reads documentation from a sherpa API ("sherpadoc")
+// and outputs a documented typescript module, optionally wrapped in a namespace,
+// that exports all functions and types referenced in that machine-readable
+// documentation.
+//
+// Example:
+//
+//	sherpadoc MyAPI >myapi.json
+//	sherpats -bytes-to-string -slices-nullable -nullable-optional -namespace myapi myapi < myapi.json > myapi.ts
+package main
+
+import (
+	"flag"
+	"log"
+	"os"
+
+	"github.com/mjl-/sherpats"
+)
+
+func check(err error, action string) {
+	if err != nil {
+		log.Fatalf("%s: %s\n", action, err)
+	}
+}
+
+func main() {
+	log.SetFlags(0)
+
+	var opts sherpats.Options
+	flag.StringVar(&opts.Namespace, "namespace", "", "namespace to enclose generated typescript in")
+	flag.BoolVar(&opts.SlicesNullable, "slices-nullable", false, "generate nullable types in TypeScript for Go slices, to require TypeScript checks for null for slices")
+	flag.BoolVar(&opts.MapsNullable, "maps-nullable", false, "generate nullable types in TypeScript for Go maps, to require TypeScript checks for null for maps")
+	flag.BoolVar(&opts.NullableOptional, "nullable-optional", false, "for nullable types (include slices with -slices-nullable=true), generate optional fields in TypeScript and allow undefined as value")
+	flag.BoolVar(&opts.BytesToString, "bytes-to-string", false, "turn []uint8, also known as []byte, into string before generating the api, matching Go's JSON package that marshals []byte as base64-encoded string")
+	flag.Usage = func() {
+		log.Println("usage: sherpats [flags] { api-path-elem | baseURL }")
+		flag.PrintDefaults()
+	}
+	flag.Parse()
+	args := flag.Args()
+	if len(args) != 1 {
+		log.Print("unexpected arguments")
+		flag.Usage()
+		os.Exit(2)
+	}
+	apiName := args[0]
+
+	err := sherpats.Generate(os.Stdin, os.Stdout, apiName, opts)
+	check(err, "generating typescript client")
+}
--- a/vendor/github.com/mjl-/sherpats/sherpats.go
+++ b/vendor/github.com/mjl-/sherpats/sherpats.go
@ -0,0 +1,617 @@
+package sherpats
+
+import (
+	"bufio"
+	"encoding/json"
+	"fmt"
+	"io"
+	"os"
+	"strings"
+
+	"github.com/mjl-/sherpadoc"
+)
+
+// Keywords in Typescript, from https://github.com/microsoft/TypeScript/blob/master/doc/spec.md.
+var keywords = map[string]struct{}{
+	"break":       {},
+	"case":        {},
+	"catch":       {},
+	"class":       {},
+	"const":       {},
+	"continue":    {},
+	"debugger":    {},
+	"default":     {},
+	"delete":      {},
+	"do":          {},
+	"else":        {},
+	"enum":        {},
+	"export":      {},
+	"extends":     {},
+	"false":       {},
+	"finally":     {},
+	"for":         {},
+	"function":    {},
+	"if":          {},
+	"import":      {},
+	"in":          {},
+	"instanceof":  {},
+	"new":         {},
+	"null":        {},
+	"return":      {},
+	"super":       {},
+	"switch":      {},
+	"this":        {},
+	"throw":       {},
+	"true":        {},
+	"try":         {},
+	"typeof":      {},
+	"var":         {},
+	"void":        {},
+	"while":       {},
+	"with":        {},
+	"implements":  {},
+	"interface":   {},
+	"let":         {},
+	"package":     {},
+	"private":     {},
+	"protected":   {},
+	"public":      {},
+	"static":      {},
+	"yield":       {},
+	"any":         {},
+	"boolean":     {},
+	"number":      {},
+	"string":      {},
+	"symbol":      {},
+	"abstract":    {},
+	"as":          {},
+	"async":       {},
+	"await":       {},
+	"constructor": {},
+	"declare":     {},
+	"from":        {},
+	"get":         {},
+	"is":          {},
+	"module":      {},
+	"namespace":   {},
+	"of":          {},
+	"require":     {},
+	"set":         {},
+	"type":        {},
+}
+
+type sherpaType interface {
+	TypescriptType() string
+}
+
+// baseType can be one of: "any", "int16", etc
+type baseType struct {
+	Name string
+}
+
+// nullableType is: "nullable" <type>.
+type nullableType struct {
+	Type sherpaType
+}
+
+// arrayType is: "[]" <type>
+type arrayType struct {
+	Type sherpaType
+}
+
+// objectType is: "{}" <type>
+type objectType struct {
+	Value sherpaType
+}
+
+// identType is: [a-zA-Z][a-zA-Z0-9]*
+type identType struct {
+	Name string
+}
+
+func (t baseType) TypescriptType() string {
+	switch t.Name {
+	case "bool":
+		return "boolean"
+	case "timestamp":
+		return "Date"
+	case "int8", "uint8", "int16", "uint16", "int32", "uint32", "int64", "uint64", "float32", "float64":
+		return "number"
+	case "int64s", "uint64s":
+		return "string"
+	default:
+		return t.Name
+	}
+}
+
+func isBaseOrIdent(t sherpaType) bool {
+	if _, ok := t.(baseType); ok {
+		return true
+	}
+	if _, ok := t.(identType); ok {
+		return true
+	}
+	return false
+}
+
+func (t nullableType) TypescriptType() string {
+	if isBaseOrIdent(t.Type) {
+		return t.Type.TypescriptType() + " | null"
+	}
+	return "(" + t.Type.TypescriptType() + ") | null"
+}
+
+func (t arrayType) TypescriptType() string {
+	if isBaseOrIdent(t.Type) {
+		return t.Type.TypescriptType() + "[] | null"
+	}
+	return "(" + t.Type.TypescriptType() + ")[] | null"
+}
+
+func (t objectType) TypescriptType() string {
+	return fmt.Sprintf("{ [key: string]: %s }", t.Value.TypescriptType())
+}
+
+func (t identType) TypescriptType() string {
+	return t.Name
+}
+
+type genError struct{ error }
+
+type Options struct {
+	// If not empty, the generated typescript is wrapped in a namespace. This allows
+	// easy compilation, with "tsc --module none" that uses the generated typescript
+	// api, while keeping all types/functions isolated.
+	Namespace string
+
+	// With SlicesNullable and MapsNullable, generated typescript types are made
+	// nullable, with "| null". Go's JSON package marshals a nil slice/map to null, so
+	// it can be wise to make TypeScript consumers check that. Go code typically
+	// handles incoming nil and empty slices/maps in the same way.
+	SlicesNullable bool
+	MapsNullable   bool
+
+	// If nullables are optional, the generated typescript types allow the "undefined"
+	// value where nullable values are expected. This includes slices/maps when
+	// SlicesNullable/MapsNullable is set. When JavaScript marshals JSON, a field with the
+	// "undefined" value is treated as if the field doesn't exist, and isn't
+	// marshalled. The "undefined" value in an array is marshalled as null. It is
+	// common (though not always the case!) in Go server code to not make a difference
+	// between a missing field and a null value
+	NullableOptional bool
+
+	// If set, "[]uint8" is changed into "string" before before interpreting the
+	// sherpadoc definitions. Go's JSON marshaller turns []byte (which is []uint8) into
+	// base64 strings. Having the same types in TypeScript is convenient.
+	// If SlicesNullable is set, the strings are made nullable.
+	BytesToString bool
+}
+
+// Generate reads sherpadoc from in and writes a typescript file containing a
+// client package to out.  apiNameBaseURL is either an API name or sherpa
+// baseURL, depending on whether it contains a slash. If it is a package name, the
+// baseURL is created at runtime by adding the packageName to the current location.
+func Generate(in io.Reader, out io.Writer, apiNameBaseURL string, opts Options) (retErr error) {
+	defer func() {
+		e := recover()
+		if e == nil {
+			return
+		}
+		g, ok := e.(genError)
+		if !ok {
+			panic(e)
+		}
+		retErr = error(g)
+	}()
+
+	var doc sherpadoc.Section
+	err := json.NewDecoder(os.Stdin).Decode(&doc)
+	if err != nil {
+		panic(genError{fmt.Errorf("parsing sherpadoc json: %s", err)})
+	}
+
+	const sherpadocVersion = 1
+	if doc.SherpadocVersion != sherpadocVersion {
+		panic(genError{fmt.Errorf("unexpected sherpadoc version %d, expected %d", doc.SherpadocVersion, sherpadocVersion)})
+	}
+
+	if opts.BytesToString {
+		toString := func(tw []string) []string {
+			n := len(tw) - 1
+			for i := 0; i < n; i++ {
+				if tw[i] == "[]" && tw[i+1] == "uint8" {
+					if opts.SlicesNullable && (i == 0 || tw[i-1] != "nullable") {
+						tw[i] = "nullable"
+						tw[i+1] = "string"
+						i++
+					} else {
+						tw[i] = "string"
+						copy(tw[i+1:], tw[i+2:])
+						tw = tw[:len(tw)-1]
+						n--
+					}
+				}
+			}
+			return tw
+		}
+
+		var bytesToString func(sec *sherpadoc.Section)
+		bytesToString = func(sec *sherpadoc.Section) {
+			for i := range sec.Functions {
+				for j := range sec.Functions[i].Params {
+					sec.Functions[i].Params[j].Typewords = toString(sec.Functions[i].Params[j].Typewords)
+				}
+				for j := range sec.Functions[i].Returns {
+					sec.Functions[i].Returns[j].Typewords = toString(sec.Functions[i].Returns[j].Typewords)
+				}
+			}
+			for i := range sec.Structs {
+				for j := range sec.Structs[i].Fields {
+					sec.Structs[i].Fields[j].Typewords = toString(sec.Structs[i].Fields[j].Typewords)
+				}
+			}
+			for _, s := range sec.Sections {
+				bytesToString(s)
+			}
+		}
+		bytesToString(&doc)
+	}
+
+	// Validate the sherpadoc.
+	err = sherpadoc.Check(&doc)
+	if err != nil {
+		panic(genError{err})
+	}
+
+	// Make a copy, the ugly way. We'll strip the documentation out before including
+	// the types. We need types for runtime type checking, but the docs just bloat the
+	// size.
+	var typesdoc sherpadoc.Section
+	if typesbuf, err := json.Marshal(doc); err != nil {
+		panic(genError{fmt.Errorf("marshal sherpadoc for types: %s", err)})
+	} else if err := json.Unmarshal(typesbuf, &typesdoc); err != nil {
+		panic(genError{fmt.Errorf("unmarshal sherpadoc for types: %s", err)})
+	}
+	for i := range typesdoc.Structs {
+		typesdoc.Structs[i].Docs = ""
+		for j := range typesdoc.Structs[i].Fields {
+			typesdoc.Structs[i].Fields[j].Docs = ""
+		}
+	}
+	for i := range typesdoc.Ints {
+		typesdoc.Ints[i].Docs = ""
+		for j := range typesdoc.Ints[i].Values {
+			typesdoc.Ints[i].Values[j].Docs = ""
+		}
+	}
+	for i := range typesdoc.Strings {
+		typesdoc.Strings[i].Docs = ""
+		for j := range typesdoc.Strings[i].Values {
+			typesdoc.Strings[i].Values[j].Docs = ""
+		}
+	}
+
+	bout := bufio.NewWriter(out)
+	xprintf := func(format string, args ...interface{}) {
+		_, err := fmt.Fprintf(out, format, args...)
+		if err != nil {
+			panic(genError{err})
+		}
+	}
+
+	xprintMultiline := func(indent, docs string, always bool) []string {
+		lines := docLines(docs)
+		if len(lines) == 1 && !always {
+			return lines
+		}
+		for _, line := range lines {
+			xprintf("%s// %s\n", indent, line)
+		}
+		return lines
+	}
+
+	xprintSingleline := func(lines []string) {
+		if len(lines) != 1 {
+			return
+		}
+		xprintf("  // %s", lines[0])
+	}
+
+	// Type and function names could be typescript keywords. If they are, give them a different name.
+	typescriptNames := map[string]string{}
+	typescriptName := func(name string, names map[string]string) string {
+		if _, ok := keywords[name]; !ok {
+			return name
+		}
+		n := names[name]
+		if n != "" {
+			return n
+		}
+		for i := 0; ; i++ {
+			n = fmt.Sprintf("%s%d", name, i)
+			if _, ok := names[n]; ok {
+				continue
+			}
+			names[name] = n
+			return n
+		}
+	}
+
+	structTypes := map[string]bool{}
+	stringsTypes := map[string]bool{}
+	intsTypes := map[string]bool{}
+
+	var generateTypes func(sec *sherpadoc.Section)
+	generateTypes = func(sec *sherpadoc.Section) {
+		for _, t := range sec.Structs {
+			structTypes[t.Name] = true
+			xprintMultiline("", t.Docs, true)
+			name := typescriptName(t.Name, typescriptNames)
+			xprintf("export interface %s {\n", name)
+			names := map[string]string{}
+			for _, f := range t.Fields {
+				lines := xprintMultiline("", f.Docs, false)
+				what := fmt.Sprintf("field %s for type %s", f.Name, t.Name)
+				optional := ""
+				if opts.NullableOptional && f.Typewords[0] == "nullable" || opts.NullableOptional && (opts.SlicesNullable && f.Typewords[0] == "[]" || opts.MapsNullable && f.Typewords[0] == "{}") {
+					optional = "?"
+				}
+				xprintf("\t%s%s: %s", typescriptName(f.Name, names), optional, typescriptType(what, f.Typewords))
+				xprintSingleline(lines)
+				xprintf("\n")
+			}
+			xprintf("}\n\n")
+		}
+
+		for _, t := range sec.Ints {
+			intsTypes[t.Name] = true
+			xprintMultiline("", t.Docs, true)
+			name := typescriptName(t.Name, typescriptNames)
+			if len(t.Values) == 0 {
+				xprintf("export type %s = number\n\n", name)
+				continue
+			}
+			xprintf("export enum %s {\n", name)
+			names := map[string]string{}
+			for _, v := range t.Values {
+				lines := xprintMultiline("\t", v.Docs, false)
+				xprintf("\t%s = %d,", typescriptName(v.Name, names), v.Value)
+				xprintSingleline(lines)
+				xprintf("\n")
+			}
+			xprintf("}\n\n")
+		}
+
+		for _, t := range sec.Strings {
+			stringsTypes[t.Name] = true
+			xprintMultiline("", t.Docs, true)
+			name := typescriptName(t.Name, typescriptNames)
+			if len(t.Values) == 0 {
+				xprintf("export type %s = string\n\n", name)
+				continue
+			}
+			xprintf("export enum %s {\n", name)
+			names := map[string]string{}
+			for _, v := range t.Values {
+				lines := xprintMultiline("\t", v.Docs, false)
+				s := mustMarshalJSON(v.Value)
+				xprintf("\t%s = %s,", typescriptName(v.Name, names), s)
+				xprintSingleline(lines)
+				xprintf("\n")
+			}
+			xprintf("}\n\n")
+		}
+
+		for _, subsec := range sec.Sections {
+			generateTypes(subsec)
+		}
+	}
+
+	var generateFunctionTypes func(sec *sherpadoc.Section)
+	generateFunctionTypes = func(sec *sherpadoc.Section) {
+		for _, typ := range sec.Structs {
+			xprintf("	%s: %s,\n", mustMarshalJSON(typ.Name), mustMarshalJSON(typ))
+		}
+		for _, typ := range sec.Ints {
+			xprintf("	%s: %s,\n", mustMarshalJSON(typ.Name), mustMarshalJSON(typ))
+		}
+		for _, typ := range sec.Strings {
+			xprintf("	%s: %s,\n", mustMarshalJSON(typ.Name), mustMarshalJSON(typ))
+		}
+
+		for _, subsec := range sec.Sections {
+			generateFunctionTypes(subsec)
+		}
+	}
+
+	var generateParser func(sec *sherpadoc.Section)
+	generateParser = func(sec *sherpadoc.Section) {
+		for _, typ := range sec.Structs {
+			xprintf("	%s: (v: any) => parse(%s, v) as %s,\n", typ.Name, mustMarshalJSON(typ.Name), typ.Name)
+		}
+		for _, typ := range sec.Ints {
+			xprintf("	%s: (v: any) => parse(%s, v) as %s,\n", typ.Name, mustMarshalJSON(typ.Name), typ.Name)
+		}
+		for _, typ := range sec.Strings {
+			xprintf("	%s: (v: any) => parse(%s, v) as %s,\n", typ.Name, mustMarshalJSON(typ.Name), typ.Name)
+		}
+
+		for _, subsec := range sec.Sections {
+			generateParser(subsec)
+		}
+	}
+
+	var generateSectionDocs func(sec *sherpadoc.Section)
+	generateSectionDocs = func(sec *sherpadoc.Section) {
+		xprintMultiline("", sec.Docs, true)
+		for _, subsec := range sec.Sections {
+			xprintf("//\n")
+			xprintf("// # %s\n", subsec.Name)
+			generateSectionDocs(subsec)
+		}
+	}
+
+	var generateFunctions func(sec *sherpadoc.Section)
+	generateFunctions = func(sec *sherpadoc.Section) {
+		for i, fn := range sec.Functions {
+			whatParam := "pararameter for " + fn.Name
+			paramNameTypes := []string{}
+			paramNames := []string{}
+			sherpaParamTypes := [][]string{}
+			names := map[string]string{}
+			for _, p := range fn.Params {
+				name := typescriptName(p.Name, names)
+				v := fmt.Sprintf("%s: %s", name, typescriptType(whatParam, p.Typewords))
+				paramNameTypes = append(paramNameTypes, v)
+				paramNames = append(paramNames, name)
+				sherpaParamTypes = append(sherpaParamTypes, p.Typewords)
+			}
+
+			var returnType string
+			switch len(fn.Returns) {
+			case 0:
+				returnType = "void"
+			case 1:
+				what := "return type for " + fn.Name
+				returnType = typescriptType(what, fn.Returns[0].Typewords)
+			default:
+				var types []string
+				what := "return type for " + fn.Name
+				for _, t := range fn.Returns {
+					types = append(types, typescriptType(what, t.Typewords))
+				}
+				returnType = fmt.Sprintf("[%s]", strings.Join(types, ", "))
+			}
+			sherpaReturnTypes := [][]string{}
+			for _, a := range fn.Returns {
+				sherpaReturnTypes = append(sherpaReturnTypes, a.Typewords)
+			}
+
+			name := typescriptName(fn.Name, typescriptNames)
+			xprintMultiline("\t", fn.Docs, true)
+			xprintf("\tasync %s(%s): Promise<%s> {\n", name, strings.Join(paramNameTypes, ", "), returnType)
+			xprintf("\t\tconst fn: string = %s\n", mustMarshalJSON(fn.Name))
+			xprintf("\t\tconst paramTypes: string[][] = %s\n", mustMarshalJSON(sherpaParamTypes))
+			xprintf("\t\tconst returnTypes: string[][] = %s\n", mustMarshalJSON(sherpaReturnTypes))
+			xprintf("\t\tconst params: any[] = [%s]\n", strings.Join(paramNames, ", "))
+			xprintf("\t\treturn await _sherpaCall(this.baseURL, { ...this.options }, paramTypes, returnTypes, fn, params) as %s\n", returnType)
+			xprintf("\t}\n")
+			if i < len(sec.Functions)-1 {
+				xprintf("\n")
+			}
+		}
+
+		for _, s := range sec.Sections {
+			generateFunctions(s)
+		}
+	}
+
+	xprintf("// NOTE: GENERATED by github.com/mjl-/sherpats, DO NOT MODIFY\n\n")
+	if opts.Namespace != "" {
+		xprintf("namespace %s {\n\n", opts.Namespace)
+	}
+	generateTypes(&doc)
+	xprintf("export const structTypes: {[typename: string]: boolean} = %s\n", mustMarshalJSON(structTypes))
+	xprintf("export const stringsTypes: {[typename: string]: boolean} = %s\n", mustMarshalJSON(stringsTypes))
+	xprintf("export const intsTypes: {[typename: string]: boolean} = %s\n", mustMarshalJSON(intsTypes))
+	xprintf("export const types: TypenameMap = {\n")
+	generateFunctionTypes(&typesdoc)
+	xprintf("}\n\n")
+	xprintf("export const parser = {\n")
+	generateParser(&doc)
+	xprintf("}\n\n")
+	generateSectionDocs(&doc)
+	xprintf(`let defaultOptions: ClientOptions = {slicesNullable: %v, mapsNullable: %v, nullableOptional: %v}
+
+export class Client {
+	constructor(private baseURL=defaultBaseURL, public options?: ClientOptions) {
+		if (!options) {
+			this.options = defaultOptions
+		}
+	}
+
+	withOptions(options: ClientOptions): Client {
+		return new Client(this.baseURL, { ...this.options, ...options })
+	}
+
+`, opts.SlicesNullable, opts.MapsNullable, opts.NullableOptional)
+	generateFunctions(&doc)
+	xprintf("}\n\n")
+
+	const findBaseURL = `(function() {
+	let p = location.pathname
+	if (p && p[p.length - 1] !== '/') {
+		let l = location.pathname.split('/')
+		l = l.slice(0, l.length - 1)
+		p = '/' + l.join('/') + '/'
+	}
+	return location.protocol + '//' + location.host + p + 'API_NAME/'
+})()`
+
+	var apiJS string
+	if strings.Contains(apiNameBaseURL, "/") {
+		apiJS = mustMarshalJSON(apiNameBaseURL)
+	} else {
+		apiJS = strings.Replace(findBaseURL, "API_NAME", apiNameBaseURL, -1)
+	}
+	xprintf("%s\n", strings.Replace(libTS, "BASEURL", apiJS, -1))
+	if opts.Namespace != "" {
+		xprintf("}\n")
+	}
+
+	err = bout.Flush()
+	if err != nil {
+		panic(genError{err})
+	}
+	return nil
+}
+
+func typescriptType(what string, typeTokens []string) string {
+	t := parseType(what, typeTokens)
+	return t.TypescriptType()
+}
+
+func parseType(what string, tokens []string) sherpaType {
+	checkOK := func(ok bool, v interface{}, msg string) {
+		if !ok {
+			panic(genError{fmt.Errorf("invalid type for %s: %s, saw %q", what, msg, v)})
+		}
+	}
+	checkOK(len(tokens) > 0, tokens, "need at least one element")
+	s := tokens[0]
+	tokens = tokens[1:]
+	switch s {
+	case "any", "bool", "int8", "uint8", "int16", "uint16", "int32", "uint32", "int64", "uint64", "int64s", "uint64s", "float32", "float64", "string", "timestamp":
+		if len(tokens) != 0 {
+			checkOK(false, tokens, "leftover tokens after base type")
+		}
+		return baseType{s}
+	case "nullable":
+		return nullableType{parseType(what, tokens)}
+	case "[]":
+		return arrayType{parseType(what, tokens)}
+	case "{}":
+		return objectType{parseType(what, tokens)}
+	default:
+		if len(tokens) != 0 {
+			checkOK(false, tokens, "leftover tokens after identifier type")
+		}
+		return identType{s}
+	}
+}
+
+func docLines(s string) []string {
+	s = strings.TrimSpace(s)
+	if s == "" {
+		return nil
+	}
+	return strings.Split(s, "\n")
+}
+
+func mustMarshalJSON(v interface{}) string {
+	buf, err := json.Marshal(v)
+	if err != nil {
+		panic(genError{fmt.Errorf("marshalling json: %s", err)})
+	}
+	return string(buf)
+}
--- a/vendor/github.com/mjl-/sherpats/ts.go
+++ b/vendor/github.com/mjl-/sherpats/ts.go
@ -0,0 +1,387 @@
+package sherpats
+
+const libTS = `export const defaultBaseURL = BASEURL
+
+// NOTE: code below is shared between github.com/mjl-/sherpaweb and github.com/mjl-/sherpats.
+// KEEP IN SYNC.
+
+export const supportedSherpaVersion = 1
+
+export interface Section {
+	Name: string
+	Docs: string
+	Functions: Function[]
+	Sections: Section[]
+	Structs: Struct[]
+	Ints: Ints[]
+	Strings: Strings[]
+	Version: string // only for top-level section
+	SherpaVersion: number // only for top-level section
+	SherpadocVersion: number // only for top-level section
+}
+
+export interface Function {
+	Name: string
+	Docs: string
+	Params: Arg[]
+	Returns: Arg[]
+}
+
+export interface Arg {
+	Name: string
+	Typewords: string[]
+}
+
+export interface Struct {
+	Name: string
+	Docs: string
+	Fields: Field[]
+}
+
+export interface Field {
+	Name: string
+	Docs: string
+	Typewords: string[]
+}
+
+export interface Ints {
+	Name: string
+	Docs: string
+	Values: {
+		Name: string
+		Value: number
+		Docs: string
+	}[] | null
+}
+
+export interface Strings {
+	Name: string
+	Docs: string
+	Values: {
+		Name: string
+		Value: string
+		Docs: string
+	}[] | null
+}
+
+export type NamedType = Struct | Strings | Ints
+export type TypenameMap = { [k: string]: NamedType }
+
+// verifyArg typechecks "v" against "typewords", returning a new (possibly modified) value for JSON-encoding.
+// toJS indicate if the data is coming into JS. If so, timestamps are turned into JS Dates. Otherwise, JS Dates are turned into strings.
+// allowUnknownKeys configures whether unknown keys in structs are allowed.
+// types are the named types of the API.
+export const verifyArg = (path: string, v: any, typewords: string[], toJS: boolean, allowUnknownKeys: boolean, types: TypenameMap, opts: ClientOptions): any => {
+	return new verifier(types, toJS, allowUnknownKeys, opts).verify(path, v, typewords)
+}
+
+export const parse = (name: string, v: any): any => verifyArg(name, v, [name], true, false, types, defaultOptions)
+
+class verifier {
+	constructor(private types: TypenameMap, private toJS: boolean, private allowUnknownKeys: boolean, private opts: ClientOptions) {
+	}
+
+	verify(path: string, v: any, typewords: string[]): any {
+		typewords = typewords.slice(0)
+		const ww = typewords.shift()
+
+		const error = (msg: string) => {
+			if (path != '') {
+				msg = path + ': ' + msg
+			}
+			throw new Error(msg)
+		}
+
+		if (typeof ww !== 'string') {
+			error('bad typewords')
+			return // should not be necessary, typescript doesn't see error always throws an exception?
+		}
+		const w: string = ww
+
+		const ensure = (ok: boolean, expect: string): any => {
+			if (!ok) {
+				error('got ' + JSON.stringify(v) +  ', expected ' + expect)
+			}
+			return v
+		}
+
+		switch (w) {
+		case 'nullable':
+			if (v === null || v === undefined && this.opts.nullableOptional) {
+				return v
+			}
+			return this.verify(path, v, typewords)
+		case '[]':
+			if (v === null && this.opts.slicesNullable || v === undefined && this.opts.slicesNullable && this.opts.nullableOptional) {
+				return v
+			}
+			ensure(Array.isArray(v), "array")
+			return v.map((e: any, i: number) => this.verify(path + '[' + i + ']', e, typewords))
+		case '{}':
+			if (v === null && this.opts.mapsNullable || v === undefined && this.opts.mapsNullable && this.opts.nullableOptional) {
+				return v
+			}
+			ensure(v !== null || typeof v === 'object', "object")
+			const r: any = {}
+			for (const k in v) {
+				r[k] = this.verify(path + '.' + k, v[k], typewords)
+			}
+			return r
+		}
+
+		ensure(typewords.length == 0, "empty typewords")
+		const t = typeof v
+		switch (w) {
+		case 'any':
+			return v
+		case 'bool':
+			ensure(t === 'boolean', 'bool')
+			return v
+		case 'int8':
+		case 'uint8':
+		case 'int16':
+		case 'uint16':
+		case 'int32':
+		case 'uint32':
+		case 'int64':
+		case 'uint64':
+			ensure(t === 'number' && Number.isInteger(v), 'integer')
+			return v
+		case 'float32':
+		case 'float64':
+			ensure(t === 'number', 'float')
+			return v
+		case 'int64s':
+		case 'uint64s':
+			ensure(t === 'number' && Number.isInteger(v) || t === 'string', 'integer fitting in float without precision loss, or string')
+			return '' + v
+		case 'string':
+			ensure(t === 'string', 'string')
+			return v
+		case 'timestamp':
+			if (this.toJS) {
+				ensure(t === 'string', 'string, with timestamp')
+				const d = new Date(v)
+				if (d instanceof Date && !isNaN(d.getTime())) {
+					return d
+				}
+				error('invalid date ' + v)
+			} else {
+				ensure(t === 'object' && v !== null, 'non-null object')
+				ensure(v.__proto__ === Date.prototype, 'Date')
+				return v.toISOString()
+			}
+		}
+
+		// We're left with named types.
+		const nt = this.types[w]
+		if (!nt) {
+			error('unknown type ' + w)
+		}
+		if (v === null) {
+			error('bad value ' + v + ' for named type ' + w)
+		}
+
+		if (structTypes[nt.Name]) {
+			const t = nt as Struct
+			if (typeof v !== 'object') {
+				error('bad value ' + v + ' for struct ' + w)
+			}
+
+			const r: any = {}
+			for (const f of t.Fields) {
+				r[f.Name] = this.verify(path + '.' + f.Name, v[f.Name], f.Typewords)
+			}
+			// If going to JSON also verify no unknown fields are present.
+			if (!this.allowUnknownKeys) {
+				const known: { [key: string]: boolean } = {}
+				for (const f of t.Fields) {
+					known[f.Name] = true
+				}
+				Object.keys(v).forEach((k) => {
+					if (!known[k]) {
+						error('unknown key ' + k + ' for struct ' + w)
+					}
+				})
+			}
+			return r
+		} else if (stringsTypes[nt.Name]) {
+			const t = nt as Strings
+			if (typeof v !== 'string') {
+				error('mistyped value ' + v + ' for named strings ' + t.Name)
+			}
+			if (!t.Values || t.Values.length === 0) {
+				return v
+			}
+			for (const sv of t.Values) {
+				if (sv.Value === v) {
+					return v
+				}
+			}
+			error('unknkown value ' + v + ' for named strings ' + t.Name)
+		} else if (intsTypes[nt.Name]) {
+			const t = nt as Ints
+			if (typeof v !== 'number' || !Number.isInteger(v)) {
+				error('mistyped value ' + v + ' for named ints ' + t.Name)
+			}
+			if (!t.Values || t.Values.length === 0) {
+				return v
+			}
+			for (const sv of t.Values) {
+				if (sv.Value === v) {
+					return v
+				}
+			}
+			error('unknkown value ' + v + ' for named ints ' + t.Name)
+		} else {
+			throw new Error('unexpected named type ' + nt)
+		}
+	}
+}
+
+
+export interface ClientOptions {
+	aborter?: {abort?: () => void}
+	timeoutMsec?: number
+	skipParamCheck?: boolean
+	skipReturnCheck?: boolean
+	slicesNullable?: boolean
+	mapsNullable?: boolean
+	nullableOptional?: boolean
+}
+
+const _sherpaCall = async (baseURL: string, options: ClientOptions, paramTypes: string[][], returnTypes: string[][], name: string, params: any[]): Promise<any> => {
+	if (!options.skipParamCheck) {
+		if (params.length !== paramTypes.length) {
+			return Promise.reject({ message: 'wrong number of parameters in sherpa call, saw ' + params.length + ' != expected ' + paramTypes.length })
+		}
+		params = params.map((v: any, index: number) => verifyArg('params[' + index + ']', v, paramTypes[index], false, false, types, options))
+	}
+	const simulate = async (json: string) => {
+		const config = JSON.parse(json || 'null') || {}
+		const waitMinMsec = config.waitMinMsec || 0
+		const waitMaxMsec = config.waitMaxMsec || 0
+		const wait = Math.random() * (waitMaxMsec - waitMinMsec)
+		const failRate = config.failRate || 0
+		return new Promise<void>((resolve, reject) => {
+			if (options.aborter) {
+				options.aborter.abort = () => {
+					reject({ message: 'call to ' + name + ' aborted by user', code: 'sherpa:aborted' })
+					reject = resolve = () => { }
+				}
+			}
+			setTimeout(() => {
+				const r = Math.random()
+				if (r < failRate) {
+					reject({ message: 'injected failure on ' + name, code: 'server:injected' })
+				} else {
+					resolve()
+				}
+				reject = resolve = () => { }
+			}, waitMinMsec + wait)
+		})
+	}
+	// Only simulate when there is a debug string. Otherwise it would always interfere
+	// with setting options.aborter.
+	let json: string = ''
+	try {
+		json = window.localStorage.getItem('sherpats-debug') || ''
+	} catch (err) {}
+	if (json) {
+		await simulate(json)
+	}
+
+	// Immediately create promise, so options.aborter is changed before returning.
+	const promise = new Promise((resolve, reject) => {
+		let resolve1 = (v: { code: string, message: string }) => {
+			resolve(v)
+			resolve1 = () => { }
+			reject1 = () => { }
+		}
+		let reject1 = (v: { code: string, message: string }) => {
+			reject(v)
+			resolve1 = () => { }
+			reject1 = () => { }
+		}
+
+		const url = baseURL + name
+		const req = new window.XMLHttpRequest()
+		if (options.aborter) {
+			options.aborter.abort = () => {
+				req.abort()
+				reject1({ code: 'sherpa:aborted', message: 'request aborted' })
+			}
+		}
+		req.open('POST', url, true)
+		if (options.timeoutMsec) {
+			req.timeout = options.timeoutMsec
+		}
+		req.onload = () => {
+			if (req.status !== 200) {
+				if (req.status === 404) {
+					reject1({ code: 'sherpa:badFunction', message: 'function does not exist' })
+				} else {
+					reject1({ code: 'sherpa:http', message: 'error calling function, HTTP status: ' + req.status })
+				}
+				return
+			}
+
+			let resp: any
+			try {
+				resp = JSON.parse(req.responseText)
+			} catch (err) {
+				reject1({ code: 'sherpa:badResponse', message: 'bad JSON from server' })
+				return
+			}
+			if (resp && resp.error) {
+				const err = resp.error
+				reject1({ code: err.code, message: err.message })
+				return
+			} else if (!resp || !resp.hasOwnProperty('result')) {
+				reject1({ code: 'sherpa:badResponse', message: "invalid sherpa response object, missing 'result'" })
+				return
+			}
+
+			if (options.skipReturnCheck) {
+				resolve1(resp.result)
+				return
+			}
+			let result = resp.result
+			try {
+				if (returnTypes.length === 0) {
+					if (result) {
+						throw new Error('function ' + name + ' returned a value while prototype says it returns "void"')
+					}
+				} else if (returnTypes.length === 1) {
+					result = verifyArg('result', result, returnTypes[0], true, true, types, options)
+				} else {
+					if (result.length != returnTypes.length) {
+						throw new Error('wrong number of values returned by ' + name + ', saw ' + result.length + ' != expected ' + returnTypes.length)
+					}
+					result = result.map((v: any, index: number) => verifyArg('result[' + index + ']', v, returnTypes[index], true, true, types, options))
+				}
+			} catch (err) {
+				let errmsg = 'bad types'
+				if (err instanceof Error) {
+					errmsg = err.message
+				}
+				reject1({ code: 'sherpa:badTypes', message: errmsg })
+			}
+			resolve1(result)
+		}
+		req.onerror = () => {
+			reject1({ code: 'sherpa:connection', message: 'connection failed' })
+		}
+		req.ontimeout = () => {
+			reject1({ code: 'sherpa:timeout', message: 'request timeout' })
+		}
+		req.setRequestHeader('Content-Type', 'application/json')
+		try {
+			req.send(JSON.stringify({ params: params }))
+		} catch (err) {
+			reject1({ code: 'sherpa:badData', message: 'cannot marshal to JSON' })
+		}
+	})
+	return await promise
+}
+`