Plugin system

Plugins are .xs files that get access to a special plugin object and can inject globals, add methods to built-in types, define new syntax, and hook into evaluation and parsing.

What is a plugin

A plugin is a regular XS file. When loaded, it gets a plugin variable that exposes surfaces into the host runtime: the global scope, the parser, the lexer, and the AST constructor table. Whatever the plugin registers through those surfaces affects every subsequent piece of code in the host program.

-- my_plugin.xs
plugin.meta = #{ name: "my_plugin", version: "0.1.0" }

plugin.runtime.global.set("greet", fn(name) {
    return "Hello, {name}!"
})

-- main.xs
load "my_plugin.xs"
println(greet("world"))  -- Hello, world!

Loading plugins

load "path/to/plugin.xs" is the canonical form. The path is resolved relative to the loading file. Plugins execute top-to-bottom at load time, before the rest of the program. If a plugin fails for any reason, the host stops - no half-loaded state. Multiple plugins load in order; later ones can override globals from earlier ones.

When two plugins introduce colliding names, use the with rename form:

load "plugin-a" with { ARROW: "pipe_arrow" }
load "plugin-b"

The plugin object

Every plugin file gets a plugin variable with these surfaces:

plugin.runtime - inject globals, add methods, eval/import/error hooks
plugin.parser - syntax handlers, parser overrides, parser primitives
plugin.lexer - register keywords, source transforms
plugin.ast - constructors for every AST node type
plugin.meta - plugin metadata map
plugin.requires(name) - declare a dependency on another loaded plugin
plugin.teardown(fn) - run cleanup when the interpreter exits
plugin.hooks() - inspect all currently registered hooks

Metadata

Set plugin.meta to a map with at least name and version. Only name is used by the runtime (for dependency checking). Everything else in the map is for humans.

plugin.meta = #{ name: "router", version: "2.1.0" }

Injecting globals

The most common plugin operation. Injects a name into the host's global scope.

plugin.runtime.global.set("clamp", fn(val, lo, hi) {
    if val < lo { return lo }
    if val > hi { return hi }
    return val
})

Also: plugin.runtime.global.get(name) to read an existing global, and plugin.runtime.global.names() to list all global names.

Adding methods to types

plugin.runtime.add_method(type, name, fn) adds a method to every value of that type. The self parameter receives the receiver.

plugin.runtime.add_method("str", "excited", fn(self) {
    return self ++ "!!!"
})

plugin.runtime.add_method("array", "sum", fn(self) {
    var total = 0
    for x in self { total = total + x }
    return total
})

-- in the host:
"hello".excited()    -- "hello!!!"
[1, 2, 3].sum()      -- 6

Valid type names: "str", "int", "float", "array", "map", "bool".

Eval hooks

before_eval fires before a node is evaluated; after_eval fires after. Both take an optional tag filter to restrict which node types trigger the hook.

let handle = plugin.runtime.before_eval("call", fn(node) {
    println("about to call")
    return node  -- must return the node
})

plugin.runtime.after_eval("call", fn(node, result) {
    println("call returned: {result}")
    return result  -- must return the result
})

Common tags: "call", "binop", "ident", "let", "assign", "if", "for", "while", "fn", "return", "block". Omit the tag to hook every node (slow).

Hook handles

Every hook registration returns a handle with a .remove() method. Once removed, the hook stops firing. Works for before_eval, after_eval,on_unknown, on_unknown_expr, on_postfix, resolve_import, on_error, and transform.

let trace = plugin.runtime.before_eval("call", fn(node) {
    println("trace: {node}")
    return node
})

trace.remove()  -- disable it

Syntax extension

For new statement-level syntax, use the declarative plugin "name" { parser { production NAME(...) { ... } } } form. This is the only path that works at parse time, so the new keyword is available in the same file that defines it.

plugin "unless" {
  meta { id: "unless"; version: "0.1.0" }

  parser {
    production unless(parser, token) {
      let cond = parser.expr()
      let body = parser.block()
      plugin.ast.if_expr(plugin.ast.unary("!", cond), body)
    }
  }
}

unless x > 10 {
    println("x is small")
}

Parser access inside handlers

Inside any parser callback you can call:

plugin.parser.expr() - parse and consume one expression
plugin.parser.block() - parse and consume a block
plugin.parser.ident() - consume an identifier
plugin.parser.expect(kind) - consume a specific token kind
plugin.parser.at(kind) - peek without consuming
plugin.parser.peek(offset) - look ahead by offset

Parser override

Override how a built-in keyword is parsed. The previous function chains back to the default or the previous override.

plugin.parser.override("fn", fn(previous) {
    let node = previous()  -- parse normally
    -- inspect or transform node
    return node
})

Import hooks

Intercept import statements to provide virtual modules. Always delegate unrecognized names to previous.

plugin.runtime.resolve_import(fn(name, previous) {
    if name == "server" {
        return #{
            start: fn(port) { println("listening on :{port}") }
        }
    }
    if previous != null { return previous(name) }
    return null
})

Lexer transforms

Transform the entire source string before parsing. Use sparingly - this is a blunt instrument that runs before any tokenization.

plugin.lexer.transform(fn(source) {
    return source.replace("MAGIC", "42")
})

AST constructors

When writing syntax handlers, use plugin.ast.* to build AST nodes. Available constructors include literals (int_node, str_node,bool_node, ident), operators (binop, unary), calls (call, method_call), control flow (if_expr, if_else, for_loop, while_loop), declarations (let_decl, var_decl, fn_decl, lambda), and structure (block, array, map, return_node, assign). Temporal constructors (every, after, timeout, debounce) build the desugared form for scheduling constructs.

Sandboxing

Restrict what a plugin can do with sandbox { flags }:

load "sketchy.xs" sandbox { inject_only }
load "another.xs" sandbox { no_override }
load "strict.xs"  sandbox { inject_only, no_override, no_eval_hook }

inject_only - global.set can only create new names, not overwrite existing ones
no_override - plugin.parser.override is disabled
no_eval_hook - before_eval and after_eval are disabled

Violated sandbox rules silently fail with a stderr warning rather than crashing the host.

Dependencies and teardown

Declare a dependency on another plugin by name (matched against plugin.meta.name). If the dependency was not loaded first, the load fails.

plugin.requires("base_framework")

plugin.teardown(fn() {
    println("plugin shutting down")
})

Gotchas

Hook registrations through plugin.parser.on_unknown and plugin.lexer.add_keyword happen at runtime (after the file is parsed), so they cannot introduce new statement keywords in the same file that calls load. Use the declarative plugin "name" { ... } form for parse-time syntax extension.

Plugin files run in a temporary interpreter; their closures still capture plugin-local variables.
Load order matters. Later plugins can overwrite globals from earlier ones.
If a plugin fails, the host does not execute. No partial state.
Hook registries have fixed caps (64 eval hooks, 16 syntax handlers, etc.). Hitting a cap drops the hook with a stderr warning.

Edit this page on GitHub ->