Wasmer Pack

The Wasmer Pack project is a code generator that takes in a WebAssembly library and the WAI files defining its interface, and generates packages for using it natively in your favourite programming language.

Useful links:

Installation

The WAPM backend automatically runs Wasmer Pack over any packages that are published to the registry, so most users won't need to interact with it directly.

That said, the wasmer-pack CLI is available on crates.io for those wanting to run it locally (e.g. during testing).

$ cargo install wasmer-pack-cli

The same CLI is published to WAPM as a WASI executable, meaning you can use wasmer run to automatically fetch and run the latest version.

$ wasmer run wasmer/wasmer-pack-cli --dir=. -- --help
wasmer-pack-cli 0.5.2

USAGE:
    wasmer-pack.wasm <SUBCOMMAND>

OPTIONS:
    -h, --help       Print help information
    -V, --version    Print version information

SUBCOMMANDS:
    help      Print this message or the help of the given subcommand(s)
    js        Generate bindings for use with NodeJS
    python    Generate Python bindings
    show      Show metadata for the bindings that would be generated from a Pirita file

NOTE: the --dir=. flag is important! This tells the wasmer CLI to let wasmer/wasmer-pack-cli access the current directory.

WebAssembly is sandboxed by default, so all file system access must be explicitly provided.

Wasmer Pack Quickstart Guide

This guide walks you through creating, publishing, and using a simple library with Wasmer Pack, WAI, and WAPM.

Creating the Library

Create a new Rust project and add the wai-bindgen-rust crate as a dependency:

$ cargo new --lib quickstart
$ cd quickstart
$ cargo add wai-bindgen-rust

Create a WAI file to define the calculator interface:

// calculator.wai

/// Add two numbers.
add: func(a: float32, b: float32) -> float32

Implement the interface in Rust:

#![allow(unused)]
fn main() {
// src/lib.rs

wai_bindgen_rust::export!("calculator.wai");
struct Calculator;

impl crate::calculator::Calculator for Calculator {
    fn add(a: f32, b: f32) -> f32 { a + b }
}
}

Publishing the Library

Install wapm using the Wasmer installer:

$ curl https://get.wasmer.io -sSfL | sh

Next, install the cargo wapm helper:

$ cargo install cargo-wapm

Log in to your WAPM account:

$ wapm login

Update Cargo.toml to configure the package for WAPM publication:

# Cargo.toml

[package.metadata.wapm]
namespace = "<YOUR_USERNAME>"
abi = "none"
bindings = { wai-version = "0.2.0", exports = "calculator.wai" }

[lib]
crate-type = ["cdylib", "rlib"]

Publish the package:

$ cargo wapm

Using the Library

JavaScript

Create a new JavaScript project and add the calculator package:

$ yarn init --yes
$ wasmer add --yarn wai/tutorial-01

Import and use the package in your JavaScript code:

// index.js

import { bindings } from "wai/tutorial-01";

async function main() {
    const calculator = await bindings.calculator();
    console.log("2 + 2 =", calculator.add(2, 2));
}

main();

Python

Add the calculator package to your Python project:

$ wasmer add --pip wai/tutorial-01

Import and use the package in your Python code:

# main.py

from tutorial_01 import bindings

calculator = bindings.calculator()
print("2+2 = ", calculator.add(2.0, 2.0))

Conclusion

Congratulations, you have successfully completed the Wasmer Pack Quickstart Guide! In this tutorial, you learned how to:

  • Create a simple calculator library using Rust and WAI.
  • Implement the interface using Rust and wai-bindgen-rust.
  • Publish the library to WAPM.
  • Use the library in JavaScript and Python projects.

Now that you have a basic understanding of how to create, publish, and use a Wasmer Pack library, you can explore more advanced topics and features.

Here are some suggestions for further exploration:

  • Learn about more advanced WAI features, such as error handling and custom types.
  • Discover how to optimize your WebAssembly modules for performance and size.
  • Explore the Wasmer ecosystem and learn about other Wasmer tools and libraries.
  • Create and publish more complex libraries, experimenting with different use cases and applications.

For more tutorials, guides, and resources, visit the Wasmer Pack documentation and the Wasmer Pack GitHub repository. You can also join the Wasmer community to ask questions, share your projects, and connect with other developers.

Good luck with your future projects, and happy coding!

Hello, World!

Like all good tutorials, let's start with WIT Pack's equivalent of "Hello, World!" - a library that adds two numbers together.

By the end, you should know how to define a simple WIT interface and implement it in Rust. We will also publish the package to WAPM and use it from JavaScript.

You can check WAPM for the package we'll be building - it's called wasmer/hello-world.

Installation

You will need to install several CLI tools.

  • The Rust toolchain so we can compile Rust code (curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh)
  • the wasm32-unknown-unknown target so Rust knows how to compile to WebAssembly (rustup target add wasm32-unknown-unknown)
  • The Wasmer runtime so we can interact with WAPM (curl https://get.wasmer.io -sSfL | sh)
  • the cargo wapm sub-command for publishing to WAPM (cargo install cargo-wapm)

Once you've installed those tools, you'll want to create a new account on wapm.io so we have somewhere to publish our code to.

Running the wapm login command will let you authenticate your computer with WAPM.

The WIT File

We want to start off simple for now, so let's create a library that just adds two 32-bit integers.

First, let's create a new Rust project and cd into it.

$ cargo new --lib tutorial-01
$ cd tutorial-01

(you can remove all the code in src/lib.rs - we don't need the example boilerplate)

Now we can add a hello-world.wai file to the project. The syntax for a WIT file is quite similar to Rust.

// hello-world.wai

/// Add two numbers
add: func(a: u32, b: u32) -> u32

This defines a function called add which takes two u32 parameters (32-bit unsigned integers) called a and b, and returns a u32.

You can see that normal comments start with a // and doc-comments use ///. Here, we're using // hello-world.wai to indicate the text should be saved to hello-world.wai.

One interesting constraint from the WIT format is that all names must be written in kebab-case. This lets wai-bindgen convert the name into the casing that is idiomatic for a particular language in a particular context.

For example, if our WIT file defined a hello-world function, it would be accessible as hello_world in Python and Rust because they use snake_case for function names, whereas in JavaScript it would be helloWorld.

Writing Some Rust

Now we've got a WIT file, let's create a WebAssembly library implementing the hello-world.wai interface.

The wai-bindgen library uses some macros to generate some glue code for our WIT file, so add it as a dependency.

$ cargo add wai-bindgen-rust

Towards the top of your src/lib.rs, we want to tell wai-bindgen that this crate exports our hello-world.wai file.

#![allow(unused)]
fn main() {
// src/lib.rs

wai_bindgen_rust::export!("hello-world.wai");
}

(note: hello-world.wai is relative to the crate's root - the folder containing your Cargo.toml file)

Now let's run cargo check to see what compile errors it shows.

$ cargo check
error[E0412]: cannot find type `HelloWorld` in module `super`
 --> src/lib.rs:1:1
  |
1 | wai_bindgen_rust::export!("hello-world.wai");
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ not found in `super`
  |

This seems to fail because of something inside the wai_bindgen_rust::export!() macro, but we can't see what it is.

The cargo expand tool can be really useful in situations like these because it will expand all macros and print out the generated code.

To use cargo expand, you'll need to make sure it's installed (cargo install cargo-expand) and that you have the nightly toolchain available (rustup toolchain install nightly).

$ cargo expand
mod hello_world {
    #[export_name = "add"]
    unsafe extern "C" fn __wai_bindgen_hello_world_add(arg0: i32, arg1: i32) -> i32 {
        let result = <super::HelloWorld as HelloWorld>::add(arg0 as u32, arg1 as u32);
        wai_bindgen_rust::rt::as_i32(result)
    }
    pub trait HelloWorld {
        /// Add two numbers
        fn add(a: u32, b: u32) -> u32;
    }
}

There's a lot going on in that code, and most of it isn't relevant to you, but there are a couple of things I'd like to point out:

  1. A hello_world module was generated (the name comes from hello-world.wai)
  2. A HelloWorld trait was defined with an add() method that matches add() from hello-world.wai (note: HelloWorld is hello-world in PascalCase)
  3. The __wai_bindgen_hello_world_add() shim expects a HelloWorld type to be defined in the parent module (that's the super:: bit), and that super::HelloWorld type must implement the HelloWorld trait

From assumption 3, we know that the generated code expects us to define a HelloWorld type. We've only got 1 line of code at the moment, so it shouldn't be surprising to see our code doesn't compile (yet).

We can fix that by defining a HelloWorld type in lib.rs. Adding two numbers doesn't require any state, so we'll just use a unit struct.

#![allow(unused)]
fn main() {
pub struct HelloWorld;
}

Looking back at assumption 3, our code still shouldn't compile because we haven't implemented the HelloWorld trait for our HelloWorld struct yet.

$ cargo check
error[E0277]: the trait bound `HelloWorld: hello_world::HelloWorld` is not satisfied
 --> src/lib.rs:1:1
  |
1 | wai_bindgen_rust::export!("hello-world.wai");
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `hello_world::HelloWorld` is not implemented for `HelloWorld`

The fix is pretty trivial.

#![allow(unused)]
fn main() {
impl hello_world::HelloWorld for HelloWorld {
    fn add(a: u32, b: u32) -> u32 { a + b }
}
}

If the naming gets a bit confusing (that's a lot of variations on hello-world!) try to think back to that output from cargo expand. The key thing to remember is the HelloWorld type is defined at the root of our crate, but the HelloWorld trait is inside a hello_world module.

Believe it or not, but we're done writing code for now. Your crate should now compile 🙂

Compiling To WebAssembly

At the moment, running cargo build will just compile our crate to a Rust library that will work on your current machine (e.g. x86-64 Linux), so we'll need to cross-compile our code to WebAssembly.

Rust makes this cross-compilation process fairly painless.

First, we need to install a version of the standard library that has already been compiled to WebAssembly.

$ rustup target add wasm32-unknown-unknown

We'll go into target triples a bit more when discussing WASI, but wasm32-unknown-unknown basically means we want generic 32-bit WebAssembly where the OS is unknown (i.e. we know nothing about the underlying OS, so we can't use it).

Next, we need to tell rustc that we want it to generate a *.wasm file.

By default, it will only generate a rlib (a "Rust library"), so we need to update Cargo.toml so our crate's crate-type includes a cdylib (a "C-compatible dynamic library").

# Cargo.toml

[lib]
crate-type = ["cdylib", "rlib"]

Now, we should be able to compile our crate for wasm32-unknown-unknown and see a *.wasm file.

$ cargo build --target wasm32-unknown-unknown
$ file target/wasm32-unknown-unknown/debug/*.wasm
target/wasm32-unknown-unknown/debug/tutorial_01.wasm: WebAssembly (wasm) binary module version 0x1 (MVP)

The wasmer CLI also has an inspect command which can be useful for looking at our *.wasm file.

$ wasmer inspect target/wasm32-unknown-unknown/debug/tutorial_01.wasm
Exports:
  Functions:
    "add": [I32, I32] -> [I32]

You'll notice that, besides a bunch of other stuff, we're exporting an add function that takes two i32s and returns an i32.

This matches the __wai_bindgen_hello_world_add() signature we saw earlier.

Publishing to WAPM

Now we've got a WebAssembly binary that works, let's publish it to WAPM!

The core component in a WAPM package is the wapm.toml file. This acts as a "manifest" which tells WAPM which modules are included in the package, and important metadata like the project name, version number, and repository URL.

You can check out the docs for a walkthrough of the full process for packaging an arbitrary WebAssembly module.

However, while we could create this file ourselves, most of the information is already available as part of our project's Cargo.toml file. The cargo wapm sub-command lets us automate a lot of the fiddly tasks like compiling the project to wasm32-unknown-unknown, collecting metadata, copying binaries around, and so on.

To enable cargo wapm, we need to add some metadata to our Cargo.toml.

# Cargo.toml

[package]
...
description = "Add two numbers"

[package.metadata.wapm]
namespace = "wasmer"  # Replace this with your WAPM username
abi = "none"
bindings = { wai-bindgen = "0.1.0", exports = "hello-world.wai" }

Something to note is that all packages on WAPM must have a description field.

Other than that, we use the [package.metadata] section to tell cargo wapm a couple of things:

  • which namespace we are publishing to (all WAPM packages are namespaced)
  • The ABI being used (none corresponds to Rust's wasm32-unknown-unknown, and we'd write wasi if we were compiling to wasm32-wasi), and
  • The location of our hello-world.wai exports, plus the version of wai-bindgen we used

Now we've updated our Cargo.toml, let's do a dry-run to make sure the package builds.

$ cargo wapm --dry-run
Successfully published package `wasmer/hello-world@0.1.0`
[INFO] Publish succeeded, but package was not published because it was run in dry-run mode

If we dig around the target/wapm/ directory, we can see what cargo wapm generated for us.

$ tree target/wapm/tutorial-01
target/wapm/tutorial-01
├── tutorial_01.wasm
├── hello-world.wai
└── wapm.toml

0 directories, 3 files

$ cat target/wapm/tutorial-01/wapm.toml
[package]
name = "wasmer/tutorial-01"
version = "0.1.0"
description = "Add two numbers"
repository = "https://github.com/wasmerio/wasmer-pack-tutorial"

[[module]]
name = "tutorial-01"
source = "tutorial_01.wasm"
abi = "none"

[module.bindings]
wai-exports = "hello-world.wai"
wai-bindgen = "0.1.0"

This all looks correct, so let's actually publish the package!

$ cargo wapm

If you open up WAPM in your browser, you should see a new package has been published. It'll look something like wasmer/tutorial-01.

Using the Package from Python

Let's create a Python project that uses the bindings to double-check that 1+1 does indeed equal 2.

First, create a new virtual environment and activate it.

$ python -m venv env
$ source env/bin/activate

Now we can ask the wapm CLI to pip install our tutorial-01 package's Python bindings.

$ wapm install --pip wasmer/tutorial-01
...
Successfully installed tutorial-01-0.1.0 wasmer-1.1.0 wasmer-compiler-cranelift-1.1.0

Whenever a package is published to WAPM with the bindings field set, WIT Pack will automatically generate bindings for various languages in the background. All the wapm CLI is doing here is asking the WAPM backend for these bindings - you can run the query yourself if you want.

The tutorial_01 package exposes a bindings variable which we can use to create new instances of our WebAssembly module. As you would expect, the object we get back has our add() method.

# main.py

from tutorial_01 import bindings

instance = bindings.hello_world()
print("1 + 1 =", instance.add(1, 1))

Let's run our script.

$ python ./main.py
1 + 1 = 2

Conclusions

Hopefully you've got a better idea for how to create a WebAssembly library and use it from different languages, now.

To recap, the process for publishing a library to WAPM is:

  1. Define a *.wai file with your interface
  2. Create a new Rust crate and add wai-bindgen as a dependency
  3. Implement the trait defined by wai_bindgen_rust::export!("hello-world.wai")
  4. Add [package.metadata.wapm] table to your Cargo.toml
  5. Publish to WAPM

We took a bit longer than normal to get here, but that's mainly because there were plenty of detours to explain the "magic" that tools like wai-bingen and cargo wapm are doing for us. This explanation gives you a better intuition for how the tools work, but we'll probably skip over them in the future.

Some exercises for the reader:

  • If your editor has some form of intellisense or code completion, hover over things like bindings.hello_world and instance.add to see their signatures
  • Add an add_floats function to hello-world.wai which will add 32-bit floating point numbers (f32)

Strings and Lists

Now we know how to write a WebAssembly library and add two numbers, let's work with something slightly more interesting - strings and lists!

First, we need to create a new project to hold our code. We'll also remove the existing code, so we start from blank slate.

$ cargo new --lib tutorial-02
$ cd tutorial-02 && rm src/lib.rs

The WIT File

Just like last time, our first step is to define a WIT file for our interface.

This file has two functions, the first function will create a string that greets a person by name (i.e. "Hello, Michael!")...

// strings-and-lists.wai

/// Greet a person by name.
greet: func(name: string) -> string

... and the other function will take a list of people's names, greeting them all at the same time.

/// Say hello to multiple people.
greet-many: func(people: list<string>) -> string

Writing Some Rust

Now we've defined our strings-and-lists.wai file, let's implement the crate.

The first thing we need to do is add wai-bindgen as a dependency.

$ cargo add wai-bindgen-rust

We also need to tell wai-bindgen that we're implementing strings-and-lists.wai.

#![allow(unused)]
fn main() {
// src/lib.rs

wai_bindgen_rust::export!("strings-and-lists.wai");
}

Next, we need to define a StringsAndLists type and implement the strings_and_lists::StringsAndLists on it.

#![allow(unused)]
fn main() {
struct StringsAndLists;

impl strings_and_lists::StringsAndLists for StringsAndLists {
    fn greet(name: String) -> String {
        format!("Hello, {name}!")
    }

    fn greet_many(people: Vec<String>) -> String {
        match people.as_slice() {
            [] => "Oh, nobody's there...".to_string(),
            [person] => format!("Hello, {person}!"),
            [people @ .., last] => {
                let people = people.join(", ");
                format!("Hello, {people}, and {last}!")
            }
        }
    }
}
}

The implementation of these functions is fairly straightforward, so we don't need to go into too much detain about it other than pointing out greet_many()'s use of Slice Patterns.

A Note on Ownership

While our code wasn't overly complex, there is something that needs to be pointed out,

Both functions use owned values for their arguments and results

This may seem odd, because it's idiomatic in normal Rust to pass strings and lists around by reference (i.e. &str and &[String]) so the caller can maintain ownership of the original value and doesn't need to make unnecessary copies.

This comes back to one of the design decisions for WebAssembly, namely that a guest (our tutorial-02 crate) is completely sandboxed and unable to access memory on the host.

That means when the host calls our WebAssembly function, arguments will be passed in by

  1. Allocate some memory inside the guest's linear memory
  2. Copy the value into this newly allocated buffer
  3. Hand ownership of the buffer to the guest function (i.e. our greet() method)

Luckily wai-bindgen will generate all the code we need for this, but it's something to be aware of.

Another thing to keep in mind is that all return values must be owned, too.

WebAssembly doesn't have any concept of ownership and borrowing, so it'd be easy for the host to run into use-after-free issues and dangling pointers if we were allowed to return non-'static values.

Publishing

Similar to last time, if we want to publish our package to WAPM, we'll need to update our Cargo.toml file.

# Cargo.toml
[package]
...
description = "Greet one or more people"

[lib]
crate-type = ["cdylib", "rlib"]

[package.metadata.wapm]
namespace = "wasmer"
abi = "none"
bindings = { wai-bindgen = "0.1.0", exports = "strings-and-lists.wai" }

Now, we can publish it to WAPM.

$ cargo wapm
Successfully published package `wasmer/tutorial-02@0.1.0`

Using The Bindings From TypeScript

For a change, let's use our bindings from TypeScript. First, we need to create a basic package.json file.

$ yarn init --yes
success Saved package.json

We'll need to install TypeScript and ts-node.

$ yarn add --dev ts-node typescript @types/node

The TypeScript compiler will need a basic tsconfig.json file.

// tsconfig.json
{
  "compilerOptions": {
    "target": "es2016",
    "module": "ESNext",
    "moduleResolution": "node",
    "strict": true,
    "skipLibCheck": true
  }
}

Now, we can use wapm install to add our tutorial-02 package as a dependency.

$ wapm install --yarn wasmer/tutorial-02

Finally, we're able to start writing some code.

// index.ts

import { bindings } from "@wasmer/tutorial-02";

async function main() {
  const strings = await bindings.strings_and_lists();
  console.log(strings.greet("World"));
  console.log(strings.greetMany(["a", "b", "c"]));
}

main();

If we run it using the ts-node loader, we'll see exactly the output we expect.

$ node --loader ts-node/esm index.ts
Hello, World!
Hello, a, b, and c!

Conclusion

Strings and lists are the building blocks of all meaningful applications, so it's important to know how to use them.

Our first foray into non-primitive types has also introduced us to the repercussions of running your code inside a fully sandboxed virtual machine - any data received from the outside world must be copied into linear memory.

Records

You're just doing great! Now, let us introduce record types, often called "structs" in other languages.

Project Setup

Let's clear our ./src/lib.rs and start from scratch

Introduction

Record is a combination of data that structurally belongs together such as:

Syntax for a record

record-item ::= 'record' id '{' record-fields '}'

record-fields ::= record-field | record-field ',' record-fields?

record-field ::= id ':' ty

A Record lets you pass data between the guests and the hosts

The WIT File

Now let us define our WIT file for our interface.

//geometry.wai

/// A point coordinate structure with { x, y }
record point {
    x: float32,
    y: float32
}

Note: As you use cargo expand, the generated file won't contain the Point Geometry 🙁

#![feature(prelude_import)]
#[prelude_import]
use std::prelude::rust_2021::*;
#[macro_use]
extern crate std;
#[allow(clippy::all)]
mod geometry {}

This happens because we didn't use our Point geometry in any function/interface so it is not compiled to an underlying struct for Rust.

So now let's use our Point geometry in a function.

// geometry.wai

/// A point coordinate structure with { x, y }
record point {
    x: float32,
    y: float32
}

/// Calculate distance between two points
distance-between: func(p1: point, p2: point) -> float32

cargo expand

#![feature(prelude_import)]
#[prelude_import]
use std::prelude::rust_2021::*;
#[macro_use]
extern crate std;
use crate::geometry::{Circle, MultiLine, Point};
#[allow(clippy::all)]
mod geometry {
    /// A point coordinate structure with { x, y }
    #[repr(C)]
    pub struct Point {
        pub x: f32,
        pub y: f32,
    }
    #[automatically_derived]
    impl ::core::marker::Copy for Point {}
    #[automatically_derived]
    impl ::core::clone::Clone for Point {
        #[inline]
        fn clone(&self) -> Point {
            let _: ::core::clone::AssertParamIsClone<f32>;
            *self
        }
    }
    impl core::fmt::Debug for Point {
        fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
            f.debug_struct("Point").field("x", &self.x).field("y", &self.y).finish()
        }
    }
    #[export_name = "distance-between"]
    unsafe extern "C" fn __wai_bindgen_geometry_distance_between(
        arg0: f32,
        arg1: f32,
        arg2: f32,
        arg3: f32,
    ) -> f32 {
        let result = <super::Geometry as Geometry>::distance_between(
            Point { x: arg0, y: arg1 },
            Point { x: arg2, y: arg3 },
        );
        wai_bindgen_rust::rt::as_f32(result)
    }
    pub trait Geometry {
        /// Calculate distance between two points
        fn distance_between(p1: Point, p2: Point) -> f32;
    }
}

Here, we see that our Point struct can be seen distance_between function is present and is exported as an external variable for use in our Rust file. We also see the Debug trait being implemented for the Point record.

This file has a record and a function. The record is a structure for a point in a coordinate system (x,y). The function performs the distance calculation between two points as arguements.

Writing Some Rust

Now we've defined our geometry.wai file, let's implement the crate.

The first thing we need to do is add wai-bindgen as a dependency

$ cargo add wai-bindgen-rust

We also need to tell wai-bindgen that we're implementing geometry.wai.

#![allow(unused)]
fn main() {
// src/lib.rs

wai_bindgen_rust::export!("geometry.wai");
}

Next, we need to define a geometry struct and implement the geometry::Geometry Trait on it.

#![allow(unused)]
fn main() {
struct Geometry;

impl geometry::Geometry for Geometry {
    fn distance_between(p1: Point, p2: Point) -> f32 {
        let Point { x: x1, y: y1 } = p1;
        let Point { x: x2, y: y2 } = p2;

        ((x2 - x1).powi(2) + (y2 - y1).powi(2)).sqrt()
    }
}
}

Note: This may seem comfusing so I've boiled it down:

  • geometry is the crate
  • Geometry is the struct
  • geometry::Geometry is the Trait that implements the function distance_between on Geometry

Explaination

Here, the function distance_between takes two arguement of the Point type. For simplicity we destructure it for a clear distinction between the x1, x2 and y1,y2 as opposed to writing p1.x or p1.y everytime.

We then find the distance between the two points using the distance formula.

Note📝

As .wai files only accept kebab-casing. The function distance_between in the geometry.wai will convert to the default casings for the respected language.

//change here after formatting!! i.e: snake_case for rust, CamelCase for Javascript,

Nested Records

As we saw, the use of simpler identifiers to create a Point record. we can further extend this functionality using records or other valid WAI types to specify the record arguments to create more complex and nested records.

⚠️ Recursive types are explicitly forbidden in WAI.

record tree-node {
    children: list<tree-node>
}

👆🏼, is not allowed.

Let's futher explain Nested Records this with an example:

WAI file with nested records :

/// A line geometry to represent a line segment with starting and ending point
record line-segment {
    start: point,
    end: point
}

/// A structure to represent a circle with center point and a radius
record circle {
    center: point,
    radius: float32
}

/// Arbitary shape - represent a shape with n number of points using a list of points
record multi-line{
    points: list<point>,
}

/// Calculate the perimeter of a Circle using 2*π*r.
perimeter-of-circle: func(c: circle) -> float32

/// Calculate the area of a Circle using π*r*r.
area-of-circle: func(c: circle) -> float32

/// Calculate the length of the multi-line by iterating over all the points and adding the result
multi-line-length: func(l: multi-line) -> float32

Here we used the point struct that we created earlier to futher define records (i.e. line-segment, circle and shape).

  • line segment uses points to define starting and ending of the line
  • Circle uses the point record for defining a center
  • An Arbitrary shape can also be represented as a list of points

If we had x,y for representing points in each of these geometries it would have no structure and code readability. Thus, we define nested records using a previously existing record.

Note📝

Records can further have type identifiers such as u8, u16, float32, enum, tuple, etc.

Writing Some Rust Again

use crate::geometry::{Circle, MultiLine, Point};

wai_bindgen_rust::export!("geometry.wai");

struct Geometry;

impl geometry::Geometry for Geometry {
    fn distance_between(p1: Point, p2: Point) -> f32 {
        let Point { x: x1, y: y1 } = p1;
        let Point { x: x2, y: y2 } = p2;

        ((x2 - x1).powi(2) + (y2 - y1).powi(2)).sqrt()
    }
    fn perimeter_of_circle(c: Circle) -> f32 {
        let Circle { center: _, radius } = c;
        (2.0 * 22.0 * radius as f32) / 7.0
    }
    fn area_of_circle(c: Circle) -> f32 {
        let Circle { center: _, radius } = c;
        (22.0 * (radius * radius) as f32) / 7.0
    }
    fn multi_line_length(l: MultiLine) -> f32 {
        if l.points.len() == 0 {
            return 0.0;
        }
        let mut result = 0.0;
        for i in 1..l.points.len() {
            let p1 = l.points[i - 1];
            let p2 = l.points[i];
            result += Geometry::distance_between(p1, p2);
        }
        result
    }
}

Here, we defined multiple functions such as:

  • perimeter_of_circle
  • area_of_circle
  • multi_line_length

All of these functions show how the nested records can be used to perform operations.

Publishing

Similar to last time, if we want to publish our package to WAPM, we'll need to update our Cargo.toml file.

# Cargo.toml
[package]
...
description = "Geometrical representations using points"

[lib]
crate-type = ["cdylib", "rlib"]

[package.metadata.wapm]
namespace = "wasmer"
abi = "none"
bindings = { wai-bindgen = "0.1.0", exports = "geometry.wai" }

Now, we can publish it to WAPM.

$ cargo wapm
Successfully published package `wasmer/tutorial-03@0.1.0`

Exposing Resources

So far we've covered basic functions, numbers, strings, lists, records, and variants, but there's one key aspect in programming we haven't touched on yet - objects with behaviour and internal state!

In WIT, we use a resource to give the host access to an "object" that has behaviour without exposing how that object is implemented (or even which language it is implemented in).

To explore this concept, we'll create a basic Calculator resource which lets the caller do various operations like addition and multiplication.

Like always, let's start off with a new Rust project.

$ cargo new --lib tutorial-05

The WIT File

The key thing to understand about a resource is that it only defines methods.

// resources.wai

resource calculator {
    static new: func(initial-value: float32) -> calculator
    current-value: func() -> float32
    add: func(value: float32)
    multiply: func(value: float32)
}

Prefixing a function with the static keyword will turn it into a static method. This is useful for defining constructors or factory functions.

Resource methods also allow the async modifier (i.e. add: async func(value: float32)), however that will require your runtime to support async functions.

Writing Some Rust

As always, we need to add wai-bindgen as a dependency.

$ cargo add wai-bindgen-rust

We also need to ask wai-bindgen to generate exports for our resources.wai interface.

#![allow(unused)]
fn main() {
// src/lib.rs

wai_bindgen_rust::export!("resources.wai");
}

If we run cargo check, we'll see that - besides the missing Resources type we expected - it complains about not being able to find Calculator.

$ cargo check
error[E0412]: cannot find type `Calculator` in module `super`
...
error[E0412]: cannot find type `Resources` in module `super`

We can create the Resources struct and implement the resources::Resources trait for it. This module won't have any top-level functions, so the trait implementation can stay empty.

#![allow(unused)]
fn main() {
pub struct Resources;

impl resources::Resources for Resources {}
}

The way the resources::Calculator trait was designed adds certain constraints to our Calculator struct, though.

#![allow(unused)]
fn main() {
pub struct Calculator(Mutex<f32>);

impl resources::Calculator for Calculator {
    fn new(initial_value: f32) -> Handle<Calculator> {
        Handle::new(Calculator(Mutex::new(initial_value)))
    }

    fn current_value(&self) -> f32 {
        *self.0.lock().unwrap()
    }

    fn add(&self, value: f32) {
        *self.0.lock().unwrap() += value;
    }

    fn multiply(&self, value: f32) {
        *self.0.lock().unwrap() *= value;
    }
}
}

A Note On Mutability

You'll notice that all methods in the resources::Calculator trait take an immutable &self. This means we'll need to use interior mutability if we want to update internal state.

While this might feel a bit awkward, there is a very good reason for requiring all state in a resource to synchronise its mutations - WebAssembly makes no guarantees that the caller will respect the borrow checker.

Publishing

We need to publish this package to WAPM, so let's update Cargo.toml with the relevant metadata.

# Cargo.toml
[package]
...
description = "A basic calculator"

[lib]
crate-type = ["cdylib", "rlib"]

[package.metadata.wapm]
namespace = "wasmer"
abi = "none"
bindings = { wai-bindgen = "0.1.0", exports = "resources.wai" }

Now, we can publish it to WAPM.

$ cargo wapm
Successfully published package `wasmer/tutorial-05@0.1.0`

Using The Bindings From Python

Once the package has been published, create a new Python virtual environment and import the package into Python.

$ python -m venv env
$ source env/bin/activate
$ wapm install --pip wasmer/tutorial-05

Next, let's create our test script.

# main.py

from tutorial_05 import bindings
from tutorial_05.bindings.resources import Calculator

resources = bindings.resources()
calculator = Calculator.new(resources, 3)

calculator.add(5)
calculator.multiply(2)
print("(3 + 5)*2 =", calculator.current_value())

Running it from the command-line should show us that (3+5)*2 = 16.

$ python main.py
(3+5)*2 = 16

Conclusion

The resource is a useful tool for exposing stateful objects that have behaviour, and should be familiar to

With the addition of resources, we've introduced most of the fundamental constructs in WIT.

Exercises for the reader:

  • Expand the calculator resource to be a fully fledged calculator
  • Try to create your own Regex package using the regex crate

WebAssembly Interfaces (WAI)

The WebAssembly spec that was first released in 2017 was only a minimum viable product and deliberately left several features incomplete to be iterated on by the ecosystem.

Arguably the most important functionality gap is the fact that only WebAssembly primitives can be passed between the host and guest. That means imports and exports can only use the following data types,

  • i32 - signed 32-bit integers
  • i64 - signed 64-bit integers
  • f32 - a 32-bit float
  • f64 - a 64-bit float (often called a double)
  • funcref - a reference to a WebAssembly function
  • externref - a reference to some opaque object owned by the WebAssembly virtual machine

You'll notice this list doesn't even include strings or boolean values!

The WebAssembly Interfaces project (WAI for short) provides a polyfill for passing around higher-level objects. It lets developers define their imports and exports in a *.wai file, then uses wai-bindgen to generate glue which automagically passes things around within the constraints of WebAssembly.

There are four main parts to WAI:

  • The *.wai file
  • The WAI Bindgen code generator
  • The guest
  • The host

The Wasmer Pack project provides a convenient way to consume WebAssembly packages which implement a WAI interface.

Some useful links:

The *.wai File

WAI uses a file (or files) with the *.wai extension to define the host-guest interface for an application that uses WebAssembly.

The items in a *.wai file map closely to concepts shared by most programming languages. It has types, interfaces ("Resources"), structs ("Records"), functions, enums, and so on.

The precise syntax is defined in the WAI repository and a parser, wai-parser, is available as a Rust crate.

The Guest

In an application using WebAssembly, the "guest" is the code that has been compiled to WebAssembly and is being loaded into a WebAssembly virtual machine.

The Host

In an application using WebAssembly, the "host" is the code that uses a WebAssembly virtual machine (like wasmer) to load a guest and use functionality it provides.

The WebAssembly spec refers to the host in some places as the embedder.

WAI Bindgen

The WAI Bindgen code generator consumes *.wai files and generates glue code that the host can use for using functionality from a WebAssembly module or the guest can use for implementing that functionality.

There are two primary ways users will interact with WAI Bindgen, the wai-bindgen CLI, and the wai-bindgen-* family of crates.

The wai-bindgen CLI provides a command-line interface to the wai-bindgen-* crates, and is often used for once-off investigation or integration with a non-Rust build system.

On the other hand, the wai-bindgen-* crates allow users to generate bindings programmatically and give them much more control over the generation process.

LanguageDirectionCode GeneratorProcedural Macro
RustGuestwai-bindgen-gen-rust-wasmwai-bindgen-rust
RustHost (Wasmer)wai-bindgen-gen-wasmerwai-bindgen-wasmer
RustHost (Wasmtime)wai-bindgen-gen-wasmtimewai-bindgen-wasmtime
CGuestwai-bindgen-gen-c
JavaScriptHostwai-bindgen-gen-js
PythonHost (Wasmer)wai-bindgen-gen-wasmer-py
PythonHost (Wasmtime)wai-bindgen-gen-wasmtime-py

Builtin Types

All types that can be used in a *.wai file are intended to mappable to native types in a general purpose programming language.

The core basic types are

  • Unsigned integers (u8, u16, u32, u64)
  • Signed integers (s8, s16, s32, s64)
  • Floating point numbers (float32, float64)
  • UTF-8 Strings (string)
  • UTF-8 code points (char)
  • Void or nothing (unit)

For a more precise definition, consult the Types section in the *.wai format.

Other Builtin Types

Besides the basic builtin types, there are several "generic" types built into WAI which let users express common concepts.

Tuples

The tuple is equivalent to a record that has numeric fields.

Code generators may be able to express tuples as a first-class concept. For example, tuple<string, float32, float32> would be expressed as (String, f32, f32) in Rust.

Lists

Lists are dynamically-sized sequences of the same element type. Often called a "list", "vector", or "array", a list<string> would be expressed as Vec<String> in Rust.

Option

The option type is used to express a value that may or may not be present.

In Rust, an option<T> is expressed as std::option::Option<T>, while other languages may choose to use null to represent the missing value.

It is semantically equivalent to the following variant:

variant option {
    none,
    some(T),
}

Expected

The expected type is used to express the result of a fallible operation.

In Rust, an expected<T, E> is expressed as std::result::Result<T, E>, although other languages may choose to convert errors into exceptions.

It is semantically equivalent to the following variant:

variant expected {
    ok(T),
    err(E),
}

Futures & Streams

The future<T> and stream<T, E> types are used to represent the result of asynchronous operations.

Functions

Functions are one of the most important concepts in WAI. They are what guests and hosts use to expose functionality to each other, and have a name, parameters, and results.

Here are some examples:

thunk: func()
fibonacci: func(n: u32) -> u32
sleep: async func(ms: u64)

Most guests will map functions to top-level functions, while most hosts will expose functions as some sort of callable object which eventually calls into the relevant part of the WebAssembly virtual machine.

For a more details, consult the Item: func section in the *.wai format.

Records

A record is an abstract data type containing a series of named fields. It has no associated behaviour and acts as a way to group data together. In C++, this would be referred to as a plain old data type.

Depending on the language, records may be expressed in in different ways.

LanguageEquivalent Construct
RustStruct
CStruct
PythonData Class
TypeScriptType alias

Syntax

A record contains a list of fields, where each field must be given a type.

record person {
    name: string,
    age: u32,
    has-lego-action-figure: bool,
}

For a more details, consult the Item: record section in the *.wai format.

Enums, Flags, Variants, and Unions

the concept of "this value can be X or Y or Z" can be expressed in several ways depending on the context.

Enum

The enum keyword can be used to introduce a C-style enum. This is a type of named constant where each constant has its own value.

enum ordering {
    less-than,
    equal-to,
    greater-than,
}

For a more details, consult the Item: enum section in the *.wai format.

Flags

The flags keyword can be used to introduce a bitflags variable. The easiest way to think of this is as a "bag of bools" where each variant can be set independently.

flags text-style {
    bold,
    italics,
    underline,
    strikethrough,
}

The flags type is a separate concept from enum because multiple flag variants can be set at a time, whereas an enum can be only one thing at a time. Different languages are often able to express this in a very efficient form, typically an integer where each bit represents a different flag.

For a more details, consult the Item: flags section in the *.wai format.

Variant

A variant lets you express something that is one of a set of types. This is similar to an enum, except each variant may have some associated data.

variant error {
    file-not-found(string),
    parse(parse-failed),
    other,
}

record parse-failed {
    message: string,
    line-number: u32,
}

Variants are implemented very differently based on what is idiomatic for a particular language.

In Rust, a variant is just a normal enum.

In TypeScript, the variant is implemented as a tagged union.

type error = { type: "file-not-found", value: string }
    | { type: "parse", value: ParseFailed }
    | { type: "other" };

For a more details, consult the Item: variant section in the *.wai format.

Union

A union is very similar to a variant, except it drops the type tag.

union configuration {
    string,
    list<string>,
}

This is distinct from a variant because some languages may be able to represent an union in a way that is more efficient or idiomatic.

For a more details, consult the Item: union section in the *.wai format.

Resources

A resource represents an opaque object where the representation and underlying implementation is completely hidden from the outside world. Resources may have associated methods, static methods, or no methods at all.

Depending on the language, records may be expressed in in different ways.

LanguageEquivalent Construct
RustTrait object
Pythonclass
JavaScriptclass

Resources can only be used through a "handle" and can be owned by either the host or the guest. Resource lifetimes are managed manually, although most languages provide some sort of reference counting or garbage collection mechanism.

Syntax

The simplest resource is an opaque "token". Users can pass this value around, but have no other way to interact with it.

resource file-descriptor

Resources can also have methods. These are functions which are associated with the resource and are implicitly given a reference to the resource when they are invoked.

resource writer {
  write: func(data: list<u8>) -> expected<unit, error>
  flush: func() -> expected<unit, error>
}

Additionally, resources can have static methods. These are often used to implement constructors.

resource request {
    static new: func() -> request

    body: async func() -> list<u8>
    headers: func() -> list<string>
}

For a more details, consult the Item: resource section in the *.wai format.

Choosing Between Resources and Records

The difference between a resource and a record can be subtle when first starting out, but there is a simple rule of thumb that will work 90% of the time:

Records contain data, resources encapsulate behaviour.

Typical Examples

A person is a good example of a record.

record person {
    name: string,
    age: u8,
    address: option<address>,
}

record address {
    number: u32,
    street: string,
    state: string,
    country: string,
    postcode: u32,
}

On the other hand, a database connection would be best represented using a resource.

resource database {
    static connect: func(connection_string: string) -> expected<database, error>
    query: func(sql: string) -> expected<list<record>, error>
    close: func() -> expected<unit, error>
}

Key Considerations

When deciding between using a resource or a record, consider the following:

  • Performance: Records require deep copying when passed between guest and host, which can be expensive for large or complex records. Consider using resources for objects with significant amounts of data or complex structures to mitigate performance issues.
  • Immutability: Records provide a level of immutability due to their pass-by-value nature. If immutability is a priority, records can be a suitable choice. However, if you need to frequently modify an object's state, a resource might be more appropriate.
  • Encapsulation: For objects with both data and behavior, consider whether separating the data and behavior into different objects—a record for data and a resource for behavior—adds value or complexity to your code.
  • Data Sharing: If data sharing or synchronization across components or instances is important, resources are a better choice since they use references, while records are not ideal for sharing data.

Edge Cases

While the "Records contain data, resources encapsulate behaviour" rule works for most cases, you will almost certainly run into situations where something has both data and behaviour.

This happens a lot when wrapping a "normal" library with a WAI interface so it can be used from WebAssembly. The distinction between "object" and "data" is more fluid in most general purpose programming languages, so it can be common to encounter something that doesn't neatly fall into the "record" or "resource" categories.

Workaround 1: Getters & Setters

If something would normally have publicly accessible fields and methods which might modify those fields, the best solution is to make that thing a resource with getters and setters.

For example, a virtual machine might normally expose its instruction pointer and any global variables that are defined, while also having an eval() method for evaluating code snippets.

resource virtual-machine {
    instruction-pointer: func() -> u32
    set-instruction-pointer: func(ip: u32)
    global-variables: func() -> list<tuple<string, value>>
    set-global-variable: func(name: string, value: value)
    eval: func(code: string) -> expected<unit, error>
}

This approach works particularly well when the methods will update state because all resources are reference types, meaning any modifications made to a resource through one handle (e.g. via a method) will be seen by all other handles to the same resource.

One downside of this approach is that each getter or setter is implemented using a method. When you have a large number of fields to expose, these extra methods can become hard to maintain or make it easy to lose interesting functionality within a wall of boilerplate.

Workaround 2: Move Methods to Top-Level Functions

Going in the other direction, sometimes it might be better to turn methods into top-level functions and use a record.

One example of this could be the satellite object used in a library that predicts the motion of a satellite.

/// An element
record satellite {
    object-name: optional<string>,
    norad-id: u64,
    inclination: float64,
    right-ascension: float64,
    ..
}

/// Parse a satellite from its JSON representation.
satellite-from-json: func(json: string) -> expected<satellite, error>

/// Predict where a satellite will be at a particular time.
predicted-location: func(s: satellite, ts: timestamp) -> position

This works well when the thing being expressed is mostly data, with only a couple of pieces of associated behaviour.

Records are passed around by value, meaning any operations that would normally modify a field will need to return a new value with the updated field, instead. This can be quite expensive when the record is large, because passing a record from guest to host (or host to guest) will often mean the entire object is serialized recursively and copied across the host-guest boundary. Consider the trade-offs between performance and immutability when deciding whether to use records or resources in these edge cases.

Contributing

Design Goals

This project is developed under several assumptions,

  • Most of the time spent using wasmer-pack will actually go into the IO before and after using it rather than running the code generator (i.e. because you have to download large inputs from a server), so performance is a non-goal for this project
  • The core library should be usable by itself, but everything else can be tailored to Wasmer's use cases

As such, we prefer developer experience over flexibility and raw performance.

Goal 1: Fast Compile Times

A clean build of the entire workspace shouldn't take any longer than 30 seconds and all CI jobs should finish within 5 minutes.

This isn't actually too difficult to achieve as long as you follow some guidelines:

  • Don't add dependencies unless you absolutely need to
  • Trim out unnecessary features
  • Periodically use cargo clean && cargo build --timings to see where compile time is spent
  • Don't import crates that pull in half of crates.io

The rationale behind this is simple - a short edit-compile-test cycle is a force multiplier. If you have fast compile times then developers can recompile and re-run the test suite after every change.

On the other hand, if CI takes 30 minutes to complete, developers will avoid your project like the plague because getting even the most trivial changes merged becomes a multi-hour chore.

To help this, we have a GitHub Action which will post comments on each PR to let you know how much your changes have affected CI times.

Goal 2: It Just Works

Users of wasmer-pack should be able to compile the project from scratch and use the generated bindings without needing to mess around with dependencies or configuration.

To achieve this,

  • Avoid dependencies that link to native libraries because they need a working C toolchain and often require installing system libraries
  • Avoid conditional compilation (including feature flags) because they tend to introduce edge cases that are hard to test and debug
  • Be opinionated - don't give the end user unnecessary flags or options unless it's part of the core functionality

Release Process

This is the process if you ever need to cut a release:

  • Make a PR which mentions the new version in its title (e.g. "Release v1.2.3" on a releases branch)
  • Update CHANGELOG.md to include any user-facing changes since the last release (the [Unreleased] link at the bottom is particularly helpful here)
  • Run cargo release. This will...
    • Promote the change log's [Unreleased] items to a named version
    • Bump version numbers in all Cargo.toml files
    • Tag the commit (e.g. v1.2.3)
    • Publish to crates.io, and
    • Push all commits and tags to GitHub
  • Wait for the "Releases" job to pass. This will...
    • Publish WebAssembly binaries to WAPM
    • Use cargo xtask set-generator to make the WAPM backend generate bindings with the new version of wasmer-pack-cli
  • Merge the "Release v1.2.3" PR into the master branch!

Cargo xtask

We use the cargo xtask pattern for any project automation more complex than 1 or 2 lines of shell. This means we get access to any library on crates.io, and having everything in pure Rust means you don't need to manually install anything or worry about OS-specific weirdness.

Currently, there are only a couple major tasks,

  • cargo xtask set-generator calls the mutation for setting a bindings generator
  • cargo xtask sync-schema will make sure the schema.graphql file is in sync with the WAPM backend, automatically updating the file if necessary

You can run cargo xtask --help to find out more details.

Project Architecture

The wasmer-pack project is split across several crates depending on the various ways it might be used.

The main crates are:

  • crates/wasmer-pack - this is the meat and potatoes of wasmer-pack. It contains all the code for generating bindings to WebAssembly modules, plus templates for any glue code that will be needed along the way
  • crates/cli - this is a CLI tool that lets wasmer-pack generate bindings using the commands and libraries inside a Pirita file
  • crates/wasm - this is a wrapper that makes wasmer-pack available as a WebAssembly module. The functions and data types that are exposed are defined in crates/wasm/wasmer-pack.exports.wai (see WIT.md for the syntax)

Architecture Decision Records

An architectural decision record (ADR) is a document that describes a choice the team makes about a significant aspect of the software architecture they’re planning to build. Each ADR describes the architectural decision, its context, and its consequences.

The goal is to get knowledge about a decision out of a developer's head so it doesn't get lost to time.

ADRs aren't big documents - if you are writing more than a couple paragraphs, you are probably doing it wrong!

(click to see the template)
# (short title of solved problem and solution)

| Metadata | Value                                                                               |
| -------- | ----------------------------------------------------------------------------------- |
| Status   | *proposed, rejected, accepted, deprecated, superseded by [ADR-123](123-example.md)* |

## Context and Problem Statement

*(Describe the context and problem statement, e.g., in free form using two to three sentences. You may want to articulate the problem in form of a question.)*

## Decision Drivers <!-- optional -->

1. *(driver 1, e.g., a force, facing concern, …)*
2. … <!-- numbers of drivers can vary -->

## Considered Options

1. option 1
2. option 2
4. … <!-- numbers of options can vary -->

## Decision Outcome

Chosen option: "option 1", because (justification. e.g., only option, which meets k.o. criterion decision driver | which resolves force force | … | comes out best).

### Positive Consequences <!-- optional -->

- (e.g., improvement of quality attribute satisfaction, follow-up decisions required, …)
- …

### Negative Consequences <!-- optional -->

- (e.g., compromising quality attribute, follow-up decisions required, …)
- …

## Pros and Cons of the Options <!-- optional -->

### option 1

*(example | description | pointer to more information | …)* <!-- optional -->

- Good, because X
- Good, because Y
- Bad, because Z
- … <!-- numbers of pros and cons can vary -->

## Links <!-- optional -->

- []()

Generated Package Naming

MetadataValue
Statusproposed

Context and Problem Statement

Currently, generated packages will derive their name from the WAPM package name, so wasmer/wasmer-pack-cli gets turned into wasmer_pack_cli for Python and @wasmer/wasmer-pack-cli for JavaScript.

Eventually, we'd like WAPM to automatically publish these packages to PyPI or NPM, so we need to come up with names that are unique.

Decision Drivers

  1. Negligible chance of collisions
  2. Generated package names are similar to the package on WAPM

Considered Options

  1. Put all packages under a @wasmer-package organisation and use __ for delimiters
  2. Host our own private registries

Decision Outcome

TODO: make a decision

Pros and Cons of the Options

Option 1

  • Good, because it's practically guaranteed to not have collisions
  • Good, because we can publish to PyPI/NPM and be used by other packages
  • Good because there is an obvious way to transform a package name back and forth
  • Bad, because the names become very verbose and unwieldy
    • wasmer/wasmer-pack becomes wasmer_package__wasmer__wasmer_pack on Python and @wasmer-package/wasmer__wasmer-pack on JavaScript

Option 2

  • Good, because we don't have to worry about colliding with existing packages
  • Good, because we get complete control over the registry
  • Bad, because it's more infrastructure to manage (operations costs, expertise, , etc.)
  • Bad, because most package managers don't let you publish packages that depend on something from another registry

Change Log

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Changelog entries will contain a link to the pull request implementing that change, where applicable.

Note: The project was renamed from wit-pack to wasmer-pack in version 0.5.0. Changelog entries from 0.4.2 and earlier use the old name.

Unreleased - ReleaseDate

0.7.2 - 2024-02-12

0.7.2 - 2024-02-12

0.7.2 - 2024-02-12

0.7.2 - 2024-02-12

0.7.2 - 2024-02-12

0.7.2 - 2024-02-12

0.7.2 - 2024-02-12

Changed

  • Allow packages with . in their names. This is useful for packages my-website.com. Internally, the . is converted into a - to make it a valid binding name.

Added

  • Added ability to pass in a user-specified name for the generated bindings. This can be done by passing in the --name flag in the CLI or by passing the name option when calling the generate_* functions.

Fixed

  • Fixed the python-wasi test in wasmer-pack crate to use poetry instead of pipenv. This is because poetry is used everywhere else, except in this one place.

  • Fixed flaky integration tests by specifying a valid webc file for wasmer-pack in both test_wasmer_pack.py (pytest) and index.test.ts (jest)

  • Pass in BindingsOptions by reference so it's easier to implement builder pattern later on.

0.7.1 - 2023-06-12

Added

  • Upgraded the webc crate to 5.0.4 so we now have support for *.webc files in both the v1 and v2 formats (#131)

Changed

  • Raised the MSRV from 1.64.0 to 1.67.0 to match wapm-targz-to-pirita (#131)

Fixed

  • Fixed the "Releases" job in CI so wasmer/wasmer-pack-cli would be published and not just wasmer/wasmer-pack (#124)

0.7.0 - 2023-02-10

💥 Breaking Changes 💥

  • Restructured the wasmer/wasmer-pack WAPM package's API (#118)
    • The Package type has been changed from a record to a resource with methods
    • There are now convenience functions for loading a Package from known binary formats (e.g. WEBC)

Added

  • Moved the logic for loading a wasmer_pack::Package from a WEBC binary out of wasmer-pack-cli into the main wasmer-pack crate (#118)

Fixed

  • Replaced the naive ABI detection routine with something that properly checks which namespaces are imported by a module (#118)

Removed

  • Removed the TestEnvironment construct from wasmer-pack-testing in favour of the autodiscover() tests (#121)

0.6.0 - 2022-12-28

Added

  • Generated JavaScript packages can now import host functions (#109)
  • Introduce the wasmer-pack-testing crate for testing generated bindings (#112)
  • Generated Python packages can now import host functions (#113)

Changed

  • Renamed wasmer-pack's sub-commands to be called javascript and python, with js and py as aliases (i.e. running wasmer-pack js is equivalent to wasmer-pack javascript) #111

0.5.3 - 2022-12-02

Fixed

  • The wasmer-pack CLI wasn't extracting the *.wai export files correctly (#105)

0.5.2 - 2022-11-24

Added

  • Users can now also generate bindings from a directory containing a wapm.toml file or a *.tar.gz downloaded from WAPM (#80)
  • Added a tutorial on using records (#83)

0.5.1 - 2022-11-18

Fixed

  • Running wasmer run --mapdir .:. wasmer/wasmer-pack-cli ... would fail because the [fs] table was being used incorrectly in wapm.toml (#91)
  • Added a workaround for dealing with *.webc files that were generated by a buggy version of wapm2pirita (#92)

Changed

  • The wasmer-pack tool now generates CommonJS packages instead of ES Modules (#89)

0.5.0 - 2022-11-08

Changed

  • Switched from Wasmer's fork of wit-bindgen on GitHub to the wai-bindgen crate on crates.io (#71)

Fixed

  • Update PackageName validation to accept the _ namespace and global packages (#74)
  • Package bindings will no longer have naming conflicts when importing a binding type with the same name as one of types we generate (#75)

💥 Breaking Changes 💥

  • The project has been renamed from wit-pack to wasmer-pack

0.4.2 - 2022-10-30

Fixed

  • Put all generated JavaScript inside a package/ folder to match the logic used by npm pack when consuming tarballs (#66)
  • Update MANIFEST.in to include py.typed in the package, meaning MyPy can now typecheck the generated bindings (#66)
  • Don't assume atoms will have the same name as their commands (#64)
  • Some JavaScript bindings wouldn't run because the bindings always import @wasmer/wasi, while the dependency was only added when one or more libraries/commands was compiled to WASI (#58)

0.4.1 - 2022-10-24

Added

  • User-facing documentation and a tutorial series are now available under the doc/ folder (#47)
  • Mention the wit-pack version in each generated package (#54)

Fixed

  • Fixed a bug where *.wasm files weren't being installed with the Python bindings from WAPM (#52)

0.4.0 - 2022-10-12

Added

  • To facilitate caching or different means of distribution, users are now able to provide their own pre-compiled WebAssembly module when initialising libraries or running commands (#45)

Changed

  • Removed the LoadArgs type from the Python bindings in favour of named arguments (#45)
  • Raised the MSRV from 1.59.0 to 1.61.0 to match minijinja
  • Removed the top-level class from the generated bindings, so now you just need to do something like from wit_pack import bindings, commands to use the package's libraries or commands (#40)

Fixed

  • Make the current directory available to the CLI when run by wasmer (#37)

0.3.0 - 2022-09-27

Added

  • Set up CI to automatically deploy to wapm.dev whenever GitHub receives a tagged commit (#24)
  • Fleshed out the repo's documentation (#25)
    • Populated the CHANGELOG.md
    • Wrote up CONTRIBUTING.md
    • Rewrote the README.md walkthrough
  • Added a "Time Reporter" task to CI so we can keep an eye on CI times (#25)
  • Generate wrappers for calling WASI executables from JavaScript (#26)
  • Generate wrappers for calling WASI executables from Python (#27)
  • Detect all available WASI executables in a Pirita file (#28)
  • Add a top-level facade to the generated Python bindings so libraries and commands can be accessed through one common object (#30)
  • Add a top-level facade to the generated JavaScript bindings so libraries and commands can be accessed through one common object (#34)
  • Added a wit-pack show sub-command to show which libraries and commands would be generated from a Pirita file (#35)

Fixed

  • Inspect each atom's kind when discovering the commands in a Pirita file instead of blindly assuming everything is a command (#32)

0.2.3 - 2022-09-15

Fixed

  • When run as a WASI program, the wit-pack CLI would unconditionally fail to load inputs because mmap isn't available (#24)

0.2.2 - 2022-09-15

(no user-facing changes)

0.2.1 - 2022-09-15

(no user-facing changes)

0.2.0 - 2022-09-15

Added

  • The wit-pack crate now allows packages to contain multiple WebAssembly modules (#22)

💥 Breaking Changes 💥

  • The wit-pack CLI now takes a Pirita file as its only input (#20)

    • This means the commandline interface has changed

      # Instead of this
      $ wit-pack js --exports exports.wit --name hello_world --version 0.1.1 --module wit.wasm -o=wit-js --abi=none
      
      # you should now do this
      $ wit-pack js -o=wit-js ./hello-world.webc
      

0.1.5 - 2022-09-12

Added

  • Introduced support for WASI libraries (#12)

Changed

  • The crates/wit-pack-cli and crates/wit-pack-wasm crates are now published to WAPM under the wasmer namespace instead of Michael-F-Bryan

0.1.4 - 2022-08-25

(no user-facing changes)

0.1.3 - 2022-08-25

(no user-facing changes)

0.1.2 - 2022-08-24