Wasmer Pack
The Wasmer Pack project is a code generator that takes in a WebAssembly library and the WAI files defining its interface, and generates packages for using it natively in your favourite programming language.
Useful links:
- The Wasmer Pack repository on GitHub
- This user guide
- The WebAssembly Package Manager
- The
*.wai
format
Installation
The WAPM backend automatically runs Wasmer Pack over any packages that are published to the registry, so most users won't need to interact with it directly.
That said, the wasmer-pack
CLI is available on crates.io for those wanting to
run it locally (e.g. during testing).
$ cargo install wasmer-pack-cli
The same CLI is published to WAPM as a WASI executable, meaning
you can use wasmer run
to automatically fetch and run the latest version.
$ wasmer run wasmer/wasmer-pack-cli --dir=. -- --help
wasmer-pack-cli 0.5.2
USAGE:
wasmer-pack.wasm <SUBCOMMAND>
OPTIONS:
-h, --help Print help information
-V, --version Print version information
SUBCOMMANDS:
help Print this message or the help of the given subcommand(s)
js Generate bindings for use with NodeJS
python Generate Python bindings
show Show metadata for the bindings that would be generated from a Pirita file
NOTE: the
--dir=.
flag is important! This tells thewasmer
CLI to letwasmer/wasmer-pack-cli
access the current directory.WebAssembly is sandboxed by default, so all file system access must be explicitly provided.
Wasmer Pack Quickstart Guide
This guide walks you through creating, publishing, and using a simple library with Wasmer Pack, WAI, and WAPM.
Creating the Library
Create a new Rust project and add the wai-bindgen-rust crate as a dependency:
$ cargo new --lib quickstart
$ cd quickstart
$ cargo add wai-bindgen-rust
Create a WAI file to define the calculator interface:
// calculator.wai
/// Add two numbers.
add: func(a: float32, b: float32) -> float32
Implement the interface in Rust:
#![allow(unused)] fn main() { // src/lib.rs wai_bindgen_rust::export!("calculator.wai"); struct Calculator; impl crate::calculator::Calculator for Calculator { fn add(a: f32, b: f32) -> f32 { a + b } } }
Publishing the Library
Install wapm using the Wasmer installer:
$ curl https://get.wasmer.io -sSfL | sh
Next, install the cargo wapm
helper:
$ cargo install cargo-wapm
Log in to your WAPM account:
$ wapm login
Update Cargo.toml
to configure the package for WAPM publication:
# Cargo.toml
[package.metadata.wapm]
namespace = "<YOUR_USERNAME>"
abi = "none"
bindings = { wai-version = "0.2.0", exports = "calculator.wai" }
[lib]
crate-type = ["cdylib", "rlib"]
Publish the package:
$ cargo wapm
Using the Library
JavaScript
Create a new JavaScript project and add the calculator package:
$ yarn init --yes
$ wasmer add --yarn wai/tutorial-01
Import and use the package in your JavaScript code:
// index.js
import { bindings } from "wai/tutorial-01";
async function main() {
const calculator = await bindings.calculator();
console.log("2 + 2 =", calculator.add(2, 2));
}
main();
Python
Add the calculator package to your Python project:
$ wasmer add --pip wai/tutorial-01
Import and use the package in your Python code:
# main.py
from tutorial_01 import bindings
calculator = bindings.calculator()
print("2+2 = ", calculator.add(2.0, 2.0))
Conclusion
Congratulations, you have successfully completed the Wasmer Pack Quickstart Guide! In this tutorial, you learned how to:
- Create a simple calculator library using Rust and WAI.
- Implement the interface using Rust and wai-bindgen-rust.
- Publish the library to WAPM.
- Use the library in JavaScript and Python projects.
Now that you have a basic understanding of how to create, publish, and use a Wasmer Pack library, you can explore more advanced topics and features.
Here are some suggestions for further exploration:
- Learn about more advanced WAI features, such as error handling and custom types.
- Discover how to optimize your WebAssembly modules for performance and size.
- Explore the Wasmer ecosystem and learn about other Wasmer tools and libraries.
- Create and publish more complex libraries, experimenting with different use cases and applications.
For more tutorials, guides, and resources, visit the Wasmer Pack documentation and the Wasmer Pack GitHub repository. You can also join the Wasmer community to ask questions, share your projects, and connect with other developers.
Good luck with your future projects, and happy coding!
Hello, World!
Like all good tutorials, let's start with WIT Pack's equivalent of "Hello, World!" - a library that adds two numbers together.
By the end, you should know how to define a simple WIT interface and implement it in Rust. We will also publish the package to WAPM and use it from JavaScript.
You can check WAPM for the package we'll be building - it's called
wasmer/hello-world
.
Installation
You will need to install several CLI tools.
- The Rust toolchain so we can compile Rust code
(
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
) - the
wasm32-unknown-unknown
target so Rust knows how to compile to WebAssembly (rustup target add wasm32-unknown-unknown
) - The Wasmer runtime
so we can interact with WAPM (
curl https://get.wasmer.io -sSfL | sh
) - the
cargo wapm
sub-command for publishing to WAPM (cargo install cargo-wapm
)
Once you've installed those tools, you'll want to create a new account on wapm.io so we have somewhere to publish our code to.
Running the wapm login
command will let you authenticate your computer with
WAPM.
The WIT File
We want to start off simple for now, so let's create a library that just adds two 32-bit integers.
First, let's create a new Rust project and cd
into it.
$ cargo new --lib tutorial-01
$ cd tutorial-01
(you can remove all the code in src/lib.rs
- we don't need the example
boilerplate)
Now we can add a hello-world.wai
file to the project. The syntax for a WIT
file is quite similar to Rust.
// hello-world.wai
/// Add two numbers
add: func(a: u32, b: u32) -> u32
This defines a function called add
which takes two u32
parameters (32-bit
unsigned integers) called a
and b
, and returns a u32
.
You can see that normal comments start with a //
and doc-comments use ///
.
Here, we're using // hello-world.wai
to indicate the text should be saved to
hello-world.wai
.
One interesting constraint from the WIT format is that all names must be
written in kebab-case. This lets wai-bindgen
convert the name into the casing
that is idiomatic for a particular language in a particular context.
For example, if our WIT file defined a hello-world
function, it would be
accessible as hello_world
in Python and Rust because they use snake_case for
function names, whereas in JavaScript it would be helloWorld
.
Writing Some Rust
Now we've got a WIT file, let's create a WebAssembly library implementing the
hello-world.wai
interface.
The wai-bindgen
library uses some macros to generate some glue code for our
WIT file, so add it as a dependency.
$ cargo add wai-bindgen-rust
Towards the top of your src/lib.rs
, we want to tell wai-bindgen
that this
crate exports our hello-world.wai
file.
#![allow(unused)] fn main() { // src/lib.rs wai_bindgen_rust::export!("hello-world.wai"); }
(note: hello-world.wai
is relative to the crate's root - the folder
containing your Cargo.toml
file)
Now let's run cargo check
to see what compile errors it shows.
$ cargo check
error[E0412]: cannot find type `HelloWorld` in module `super`
--> src/lib.rs:1:1
|
1 | wai_bindgen_rust::export!("hello-world.wai");
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ not found in `super`
|
This seems to fail because of something inside the
wai_bindgen_rust::export!()
macro, but we can't see what it is.
The cargo expand
tool can be really useful in situations like
these because it will expand all macros and print out the generated code.
To use cargo expand
, you'll need to make sure it's installed
(cargo install cargo-expand
) and that you have the nightly toolchain
available (rustup toolchain install nightly
).
$ cargo expand
mod hello_world {
#[export_name = "add"]
unsafe extern "C" fn __wai_bindgen_hello_world_add(arg0: i32, arg1: i32) -> i32 {
let result = <super::HelloWorld as HelloWorld>::add(arg0 as u32, arg1 as u32);
wai_bindgen_rust::rt::as_i32(result)
}
pub trait HelloWorld {
/// Add two numbers
fn add(a: u32, b: u32) -> u32;
}
}
There's a lot going on in that code, and most of it isn't relevant to you, but there are a couple of things I'd like to point out:
- A
hello_world
module was generated (the name comes fromhello-world.wai
) - A
HelloWorld
trait was defined with anadd()
method that matchesadd()
fromhello-world.wai
(note:HelloWorld
ishello-world
in PascalCase) - The
__wai_bindgen_hello_world_add()
shim expects aHelloWorld
type to be defined in the parent module (that's thesuper::
bit), and thatsuper::HelloWorld
type must implement theHelloWorld
trait
From assumption 3, we know that the generated code expects us to define a
HelloWorld
type. We've only got 1 line of code at the moment, so it shouldn't
be surprising to see our code doesn't compile (yet).
We can fix that by defining a HelloWorld
type in lib.rs
. Adding two numbers
doesn't require any state, so we'll just use a unit struct.
#![allow(unused)] fn main() { pub struct HelloWorld; }
Looking back at assumption 3, our code still shouldn't compile because we
haven't implemented the HelloWorld
trait for our HelloWorld
struct yet.
$ cargo check
error[E0277]: the trait bound `HelloWorld: hello_world::HelloWorld` is not satisfied
--> src/lib.rs:1:1
|
1 | wai_bindgen_rust::export!("hello-world.wai");
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `hello_world::HelloWorld` is not implemented for `HelloWorld`
The fix is pretty trivial.
#![allow(unused)] fn main() { impl hello_world::HelloWorld for HelloWorld { fn add(a: u32, b: u32) -> u32 { a + b } } }
If the naming gets a bit confusing (that's a lot of variations on
hello-world
!) try to think back to that output from cargo expand
. The key
thing to remember is the HelloWorld
type is defined at the root of our crate,
but the HelloWorld
trait is inside a hello_world
module.
Believe it or not, but we're done writing code for now. Your crate should now compile 🙂
Compiling To WebAssembly
At the moment, running cargo build
will just compile our crate to a Rust
library that will work on your current machine (e.g. x86-64 Linux), so we'll
need to cross-compile our code to WebAssembly.
Rust makes this cross-compilation process fairly painless.
First, we need to install a version of the standard library that has already been compiled to WebAssembly.
$ rustup target add wasm32-unknown-unknown
We'll go into target triples a bit more when discussing WASI, but
wasm32-unknown-unknown
basically means we want generic 32-bit WebAssembly
where the OS is unknown (i.e. we know nothing about the underlying OS, so we
can't use it).
Next, we need to tell rustc
that we want it to generate a *.wasm
file.
By default, it will only generate a rlib
(a "Rust library"), so we need to
update Cargo.toml
so our crate's crate-type
includes a
cdylib
(a "C-compatible dynamic library").
# Cargo.toml
[lib]
crate-type = ["cdylib", "rlib"]
Now, we should be able to compile our crate for wasm32-unknown-unknown
and
see a *.wasm
file.
$ cargo build --target wasm32-unknown-unknown
$ file target/wasm32-unknown-unknown/debug/*.wasm
target/wasm32-unknown-unknown/debug/tutorial_01.wasm: WebAssembly (wasm) binary module version 0x1 (MVP)
The wasmer
CLI also has an inspect
command which can be useful for looking
at our *.wasm
file.
$ wasmer inspect target/wasm32-unknown-unknown/debug/tutorial_01.wasm
Exports:
Functions:
"add": [I32, I32] -> [I32]
You'll notice that, besides a bunch of other stuff, we're exporting an add
function that takes two i32
s and returns an i32
.
This matches the __wai_bindgen_hello_world_add()
signature we saw earlier.
Publishing to WAPM
Now we've got a WebAssembly binary that works, let's publish it to WAPM!
The core component in a WAPM package is the wapm.toml
file. This acts as a
"manifest" which tells WAPM which modules are included in the package, and
important metadata like the project name, version number, and repository URL.
You can check out the docs for a walkthrough of the full process for packaging an arbitrary WebAssembly module.
However, while we could create this file ourselves, most of the information is
already available as part of our project's Cargo.toml
file. The
cargo wapm
sub-command lets us automate a lot of the fiddly
tasks like compiling the project to wasm32-unknown-unknown
, collecting
metadata, copying binaries around, and so on.
To enable cargo wapm
, we need to add some metadata to our Cargo.toml
.
# Cargo.toml
[package]
...
description = "Add two numbers"
[package.metadata.wapm]
namespace = "wasmer" # Replace this with your WAPM username
abi = "none"
bindings = { wai-bindgen = "0.1.0", exports = "hello-world.wai" }
Something to note is that all packages on WAPM must have a description
field.
Other than that, we use the [package.metadata]
section
to tell cargo wapm
a couple of things:
- which namespace we are publishing to (all WAPM packages are namespaced)
- The ABI being used (
none
corresponds to Rust'swasm32-unknown-unknown
, and we'd writewasi
if we were compiling towasm32-wasi
), and - The location of our
hello-world.wai
exports, plus the version ofwai-bindgen
we used
Now we've updated our Cargo.toml
, let's do a dry-run to make sure the package
builds.
$ cargo wapm --dry-run
Successfully published package `wasmer/hello-world@0.1.0`
[INFO] Publish succeeded, but package was not published because it was run in dry-run mode
If we dig around the target/wapm/
directory, we can see what cargo wapm
generated for us.
$ tree target/wapm/tutorial-01
target/wapm/tutorial-01
├── tutorial_01.wasm
├── hello-world.wai
└── wapm.toml
0 directories, 3 files
$ cat target/wapm/tutorial-01/wapm.toml
[package]
name = "wasmer/tutorial-01"
version = "0.1.0"
description = "Add two numbers"
repository = "https://github.com/wasmerio/wasmer-pack-tutorial"
[[module]]
name = "tutorial-01"
source = "tutorial_01.wasm"
abi = "none"
[module.bindings]
wai-exports = "hello-world.wai"
wai-bindgen = "0.1.0"
This all looks correct, so let's actually publish the package!
$ cargo wapm
If you open up WAPM in your browser, you should see a new package has been
published. It'll look something like wasmer/tutorial-01
.
Using the Package from Python
Let's create a Python project that uses the bindings to double-check that 1+1
does indeed equal 2
.
First, create a new virtual environment and activate it.
$ python -m venv env
$ source env/bin/activate
Now we can ask the wapm
CLI to pip install
our tutorial-01
package's
Python bindings.
$ wapm install --pip wasmer/tutorial-01
...
Successfully installed tutorial-01-0.1.0 wasmer-1.1.0 wasmer-compiler-cranelift-1.1.0
Whenever a package is published to WAPM with the bindings
field set, WIT Pack
will automatically generate bindings for various languages in the background.
All the wapm
CLI is doing here is asking the WAPM backend for these bindings -
you can run the query yourself if you want.
The tutorial_01
package exposes a bindings
variable which we can use to
create new instances of our WebAssembly module. As you would expect, the object
we get back has our add()
method.
# main.py
from tutorial_01 import bindings
instance = bindings.hello_world()
print("1 + 1 =", instance.add(1, 1))
Let's run our script.
$ python ./main.py
1 + 1 = 2
Conclusions
Hopefully you've got a better idea for how to create a WebAssembly library and use it from different languages, now.
To recap, the process for publishing a library to WAPM is:
- Define a
*.wai
file with your interface - Create a new Rust crate and add
wai-bindgen
as a dependency - Implement the trait defined by
wai_bindgen_rust::export!("hello-world.wai")
- Add
[package.metadata.wapm]
table to yourCargo.toml
- Publish to WAPM
We took a bit longer than normal to get here, but that's mainly because there
were plenty of detours to explain the "magic" that tools like wai-bingen
and
cargo wapm
are doing for us. This explanation gives you a better intuition
for how the tools work, but we'll probably skip over them in the future.
Some exercises for the reader:
- If your editor has some form of intellisense or code completion, hover over
things like
bindings.hello_world
andinstance.add
to see their signatures - Add an
add_floats
function tohello-world.wai
which will add 32-bit floating point numbers (f32
)
Strings and Lists
Now we know how to write a WebAssembly library and add two numbers, let's work with something slightly more interesting - strings and lists!
First, we need to create a new project to hold our code. We'll also remove the existing code, so we start from blank slate.
$ cargo new --lib tutorial-02
$ cd tutorial-02 && rm src/lib.rs
The WIT File
Just like last time, our first step is to define a WIT file for our interface.
This file has two functions, the first function will create a string that greets a person by name (i.e. "Hello, Michael!")...
// strings-and-lists.wai
/// Greet a person by name.
greet: func(name: string) -> string
... and the other function will take a list of people's names, greeting them all at the same time.
/// Say hello to multiple people.
greet-many: func(people: list<string>) -> string
Writing Some Rust
Now we've defined our strings-and-lists.wai
file, let's implement the crate.
The first thing we need to do is add wai-bindgen
as a dependency.
$ cargo add wai-bindgen-rust
We also need to tell wai-bindgen
that we're implementing
strings-and-lists.wai
.
#![allow(unused)] fn main() { // src/lib.rs wai_bindgen_rust::export!("strings-and-lists.wai"); }
Next, we need to define a StringsAndLists
type and implement the
strings_and_lists::StringsAndLists
on it.
#![allow(unused)] fn main() { struct StringsAndLists; impl strings_and_lists::StringsAndLists for StringsAndLists { fn greet(name: String) -> String { format!("Hello, {name}!") } fn greet_many(people: Vec<String>) -> String { match people.as_slice() { [] => "Oh, nobody's there...".to_string(), [person] => format!("Hello, {person}!"), [people @ .., last] => { let people = people.join(", "); format!("Hello, {people}, and {last}!") } } } } }
The implementation of these functions is fairly straightforward, so we don't
need to go into too much detain about it other than pointing out
greet_many()
's use of Slice Patterns.
A Note on Ownership
While our code wasn't overly complex, there is something that needs to be pointed out,
Both functions use owned values for their arguments and results
This may seem odd, because it's idiomatic in normal Rust to pass strings and
lists around by reference (i.e. &str
and &[String]
) so the caller can
maintain ownership of the original value and doesn't need to make unnecessary
copies.
This comes back to one of the design decisions for WebAssembly, namely that a
guest (our tutorial-02
crate) is completely sandboxed and unable to access
memory on the host.
That means when the host calls our WebAssembly function, arguments will be passed in by
- Allocate some memory inside the guest's linear memory
- Copy the value into this newly allocated buffer
- Hand ownership of the buffer to the guest function (i.e. our
greet()
method)
Luckily wai-bindgen
will generate all the code we need for this, but it's
something to be aware of.
Another thing to keep in mind is that all return values must be owned, too.
WebAssembly doesn't have any concept of ownership and borrowing, so it'd be easy
for the host to run into use-after-free issues and dangling pointers if we were
allowed to return non-'static
values.
Publishing
Similar to last time, if we want to publish our package to WAPM, we'll need to
update our Cargo.toml
file.
# Cargo.toml
[package]
...
description = "Greet one or more people"
[lib]
crate-type = ["cdylib", "rlib"]
[package.metadata.wapm]
namespace = "wasmer"
abi = "none"
bindings = { wai-bindgen = "0.1.0", exports = "strings-and-lists.wai" }
Now, we can publish it to WAPM.
$ cargo wapm
Successfully published package `wasmer/tutorial-02@0.1.0`
Using The Bindings From TypeScript
For a change, let's use our bindings from TypeScript. First, we need to create
a basic package.json
file.
$ yarn init --yes
success Saved package.json
We'll need to install TypeScript and ts-node
.
$ yarn add --dev ts-node typescript @types/node
The TypeScript compiler will need a basic tsconfig.json
file.
// tsconfig.json
{
"compilerOptions": {
"target": "es2016",
"module": "ESNext",
"moduleResolution": "node",
"strict": true,
"skipLibCheck": true
}
}
Now, we can use wapm install
to add our tutorial-02
package as a dependency.
$ wapm install --yarn wasmer/tutorial-02
Finally, we're able to start writing some code.
// index.ts
import { bindings } from "@wasmer/tutorial-02";
async function main() {
const strings = await bindings.strings_and_lists();
console.log(strings.greet("World"));
console.log(strings.greetMany(["a", "b", "c"]));
}
main();
If we run it using the ts-node
loader, we'll see exactly the output we expect.
$ node --loader ts-node/esm index.ts
Hello, World!
Hello, a, b, and c!
Conclusion
Strings and lists are the building blocks of all meaningful applications, so it's important to know how to use them.
Our first foray into non-primitive types has also introduced us to the repercussions of running your code inside a fully sandboxed virtual machine - any data received from the outside world must be copied into linear memory.
Records
You're just doing great! Now, let us introduce record types, often called "structs" in other languages.
Project Setup
Let's clear our ./src/lib.rs
and start from scratch
Introduction
Record is a combination of data that structurally belongs together such as:
Syntax for a record
record-item ::= 'record' id '{' record-fields '}'
record-fields ::= record-field | record-field ',' record-fields?
record-field ::= id ':' ty
A Record lets you pass data between the guests and the hosts
The WIT File
Now let us define our WIT file for our interface.
//geometry.wai
/// A point coordinate structure with { x, y }
record point {
x: float32,
y: float32
}
Note: As you use
cargo expand
, the generated file won't contain thePoint
Geometry 🙁
#![feature(prelude_import)]
#[prelude_import]
use std::prelude::rust_2021::*;
#[macro_use]
extern crate std;
#[allow(clippy::all)]
mod geometry {}
This happens because we didn't use our
Point
geometry in any function/interface so it is not compiled to an underlying struct forRust
.
So now let's use our Point
geometry in a function.
// geometry.wai
/// A point coordinate structure with { x, y }
record point {
x: float32,
y: float32
}
/// Calculate distance between two points
distance-between: func(p1: point, p2: point) -> float32
cargo expand
#![feature(prelude_import)]
#[prelude_import]
use std::prelude::rust_2021::*;
#[macro_use]
extern crate std;
use crate::geometry::{Circle, MultiLine, Point};
#[allow(clippy::all)]
mod geometry {
/// A point coordinate structure with { x, y }
#[repr(C)]
pub struct Point {
pub x: f32,
pub y: f32,
}
#[automatically_derived]
impl ::core::marker::Copy for Point {}
#[automatically_derived]
impl ::core::clone::Clone for Point {
#[inline]
fn clone(&self) -> Point {
let _: ::core::clone::AssertParamIsClone<f32>;
*self
}
}
impl core::fmt::Debug for Point {
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
f.debug_struct("Point").field("x", &self.x).field("y", &self.y).finish()
}
}
#[export_name = "distance-between"]
unsafe extern "C" fn __wai_bindgen_geometry_distance_between(
arg0: f32,
arg1: f32,
arg2: f32,
arg3: f32,
) -> f32 {
let result = <super::Geometry as Geometry>::distance_between(
Point { x: arg0, y: arg1 },
Point { x: arg2, y: arg3 },
);
wai_bindgen_rust::rt::as_f32(result)
}
pub trait Geometry {
/// Calculate distance between two points
fn distance_between(p1: Point, p2: Point) -> f32;
}
}
Here, we see that our
Point
struct can be seendistance_between
function is present and is exported as an external variable for use in our Rust file. We also see the Debug trait being implemented for the Point record.
This file has a record and a function. The record is a structure for a point in a coordinate system (x,y). The function performs the distance calculation between two points as arguements.
Writing Some Rust
Now we've defined our geometry.wai
file, let's implement the crate.
The first thing we need to do is add wai-bindgen
as a dependency
$ cargo add wai-bindgen-rust
We also need to tell wai-bindgen
that we're implementing
geometry.wai
.
#![allow(unused)] fn main() { // src/lib.rs wai_bindgen_rust::export!("geometry.wai"); }
Next, we need to define a geometry
struct and implement the
geometry::Geometry
Trait on it.
#![allow(unused)] fn main() { struct Geometry; impl geometry::Geometry for Geometry { fn distance_between(p1: Point, p2: Point) -> f32 { let Point { x: x1, y: y1 } = p1; let Point { x: x2, y: y2 } = p2; ((x2 - x1).powi(2) + (y2 - y1).powi(2)).sqrt() } } }
Note: This may seem comfusing so I've boiled it down:
geometry
is the crateGeometry
is the structgeometry::Geometry
is the Trait that implements the functiondistance_between
onGeometry
Explaination
Here, the function distance_between
takes two arguement of the Point type. For simplicity we destructure it for a clear distinction between the x1, x2 and y1,y2 as opposed to writing p1.x
or p1.y
everytime.
We then find the distance between the two points using the distance formula.
Note📝
As
.wai
files only accept kebab-casing. The functiondistance_between
in thegeometry.wai
will convert to the default casings for the respected language.//change here after formatting!! i.e: snake_case for rust, CamelCase for Javascript,
Nested Records
As we saw, the use of simpler identifiers to create a Point
record. we can further extend this functionality using records or other valid WAI types
to specify the record arguments to create more complex and nested records.
⚠️ Recursive types are explicitly forbidden in WAI.
record tree-node {
children: list<tree-node>
}
👆🏼, is not allowed.
Let's futher explain Nested Records
this with an example:
WAI file with nested records :
/// A line geometry to represent a line segment with starting and ending point
record line-segment {
start: point,
end: point
}
/// A structure to represent a circle with center point and a radius
record circle {
center: point,
radius: float32
}
/// Arbitary shape - represent a shape with n number of points using a list of points
record multi-line{
points: list<point>,
}
/// Calculate the perimeter of a Circle using 2*π*r.
perimeter-of-circle: func(c: circle) -> float32
/// Calculate the area of a Circle using π*r*r.
area-of-circle: func(c: circle) -> float32
/// Calculate the length of the multi-line by iterating over all the points and adding the result
multi-line-length: func(l: multi-line) -> float32
Here we used the point
struct that we created earlier to futher define records (i.e. line-segment
, circle
and shape
).
- line segment uses points to define starting and ending of the line
- Circle uses the point record for defining a center
- An Arbitrary shape can also be represented as a list of points
If we had x,y for representing points in each of these geometries it would have no structure and code readability. Thus, we define nested records using a previously existing record.
Note📝
Records can further have type identifiers such as u8, u16, float32, enum, tuple, etc.
Writing Some Rust Again
use crate::geometry::{Circle, MultiLine, Point};
wai_bindgen_rust::export!("geometry.wai");
struct Geometry;
impl geometry::Geometry for Geometry {
fn distance_between(p1: Point, p2: Point) -> f32 {
let Point { x: x1, y: y1 } = p1;
let Point { x: x2, y: y2 } = p2;
((x2 - x1).powi(2) + (y2 - y1).powi(2)).sqrt()
}
fn perimeter_of_circle(c: Circle) -> f32 {
let Circle { center: _, radius } = c;
(2.0 * 22.0 * radius as f32) / 7.0
}
fn area_of_circle(c: Circle) -> f32 {
let Circle { center: _, radius } = c;
(22.0 * (radius * radius) as f32) / 7.0
}
fn multi_line_length(l: MultiLine) -> f32 {
if l.points.len() == 0 {
return 0.0;
}
let mut result = 0.0;
for i in 1..l.points.len() {
let p1 = l.points[i - 1];
let p2 = l.points[i];
result += Geometry::distance_between(p1, p2);
}
result
}
}
Here, we defined multiple functions such as:
perimeter_of_circle
area_of_circle
multi_line_length
All of these functions show how the nested records can be used to perform operations.
Publishing
Similar to last time, if we want to publish our package to WAPM, we'll need to
update our Cargo.toml
file.
# Cargo.toml
[package]
...
description = "Geometrical representations using points"
[lib]
crate-type = ["cdylib", "rlib"]
[package.metadata.wapm]
namespace = "wasmer"
abi = "none"
bindings = { wai-bindgen = "0.1.0", exports = "geometry.wai" }
Now, we can publish it to WAPM.
$ cargo wapm
Successfully published package `wasmer/tutorial-03@0.1.0`
Exposing Resources
So far we've covered basic functions, numbers, strings, lists, records, and variants, but there's one key aspect in programming we haven't touched on yet - objects with behaviour and internal state!
In WIT, we use a resource
to give the host access to an "object" that has
behaviour without exposing how that object is implemented (or even which
language it is implemented in).
To explore this concept, we'll create a basic Calculator
resource which lets
the caller do various operations like addition and multiplication.
Like always, let's start off with a new Rust project.
$ cargo new --lib tutorial-05
The WIT File
The key thing to understand about a resource
is that it only defines methods.
// resources.wai
resource calculator {
static new: func(initial-value: float32) -> calculator
current-value: func() -> float32
add: func(value: float32)
multiply: func(value: float32)
}
Prefixing a function with the static
keyword will turn it into a
static
method. This is useful for defining constructors or
factory functions.
Resource methods also allow the async
modifier (i.e.
add: async func(value: float32)
), however that will require your runtime to
support async
functions.
Writing Some Rust
As always, we need to add wai-bindgen
as a dependency.
$ cargo add wai-bindgen-rust
We also need to ask wai-bindgen
to generate exports for our resources.wai
interface.
#![allow(unused)] fn main() { // src/lib.rs wai_bindgen_rust::export!("resources.wai"); }
If we run cargo check
, we'll see that - besides the missing Resources
type
we expected - it complains about not being able to find Calculator
.
$ cargo check
error[E0412]: cannot find type `Calculator` in module `super`
...
error[E0412]: cannot find type `Resources` in module `super`
We can create the Resources
struct and implement the resources::Resources
trait for it. This module won't have any top-level functions, so the trait
implementation can stay empty.
#![allow(unused)] fn main() { pub struct Resources; impl resources::Resources for Resources {} }
The way the resources::Calculator
trait was designed adds certain constraints
to our Calculator
struct, though.
#![allow(unused)] fn main() { pub struct Calculator(Mutex<f32>); impl resources::Calculator for Calculator { fn new(initial_value: f32) -> Handle<Calculator> { Handle::new(Calculator(Mutex::new(initial_value))) } fn current_value(&self) -> f32 { *self.0.lock().unwrap() } fn add(&self, value: f32) { *self.0.lock().unwrap() += value; } fn multiply(&self, value: f32) { *self.0.lock().unwrap() *= value; } } }
A Note On Mutability
You'll notice that all methods in the resources::Calculator
trait take an
immutable &self
. This means we'll need to use interior mutability
if we want to update internal state.
While this might feel a bit awkward, there is a very good reason for requiring all state in a resource to synchronise its mutations - WebAssembly makes no guarantees that the caller will respect the borrow checker.
Publishing
We need to publish this package to WAPM, so let's update Cargo.toml
with the
relevant metadata.
# Cargo.toml
[package]
...
description = "A basic calculator"
[lib]
crate-type = ["cdylib", "rlib"]
[package.metadata.wapm]
namespace = "wasmer"
abi = "none"
bindings = { wai-bindgen = "0.1.0", exports = "resources.wai" }
Now, we can publish it to WAPM.
$ cargo wapm
Successfully published package `wasmer/tutorial-05@0.1.0`
Using The Bindings From Python
Once the package has been published, create a new Python virtual environment and import the package into Python.
$ python -m venv env
$ source env/bin/activate
$ wapm install --pip wasmer/tutorial-05
Next, let's create our test script.
# main.py
from tutorial_05 import bindings
from tutorial_05.bindings.resources import Calculator
resources = bindings.resources()
calculator = Calculator.new(resources, 3)
calculator.add(5)
calculator.multiply(2)
print("(3 + 5)*2 =", calculator.current_value())
Running it from the command-line should show us that (3+5)*2 = 16
.
$ python main.py
(3+5)*2 = 16
Conclusion
The resource is a useful tool for exposing stateful objects that have behaviour, and should be familiar to
With the addition of resources, we've introduced most of the fundamental constructs in WIT.
Exercises for the reader:
- Expand the
calculator
resource to be a fully fledged calculator - Try to create your own Regex package using the
regex
crate
WebAssembly Interfaces (WAI)
The WebAssembly spec that was first released in 2017 was only a minimum viable product and deliberately left several features incomplete to be iterated on by the ecosystem.
Arguably the most important functionality gap is the fact that only WebAssembly primitives can be passed between the host and guest. That means imports and exports can only use the following data types,
i32
- signed 32-bit integersi64
- signed 64-bit integersf32
- a 32-bit floatf64
- a 64-bit float (often called adouble
)funcref
- a reference to a WebAssembly functionexternref
- a reference to some opaque object owned by the WebAssembly virtual machine
You'll notice this list doesn't even include strings or boolean values!
The WebAssembly Interfaces project (WAI for short) provides a polyfill for
passing around higher-level objects. It lets developers define their imports and
exports in a *.wai
file, then uses wai-bindgen
to generate glue which
automagically passes things around within the constraints of WebAssembly.
There are four main parts to WAI:
- The
*.wai
file - The WAI Bindgen code generator
- The guest
- The host
The Wasmer Pack project provides a convenient way to consume WebAssembly packages which implement a WAI interface.
Some useful links:
The *.wai
File
WAI uses a file (or files) with the *.wai
extension to define the host-guest
interface for an application that uses WebAssembly.
The items in a *.wai
file map closely to concepts shared by most programming
languages. It has types, interfaces
("Resources"), structs ("Records"),
functions, enums, and so on.
The precise syntax is defined in the WAI repository and a parser,
wai-parser
, is available as a Rust crate.
The Guest
In an application using WebAssembly, the "guest" is the code that has been compiled to WebAssembly and is being loaded into a WebAssembly virtual machine.
The Host
In an application using WebAssembly, the "host" is the code that uses a
WebAssembly virtual machine (like wasmer
) to load a guest and use
functionality it provides.
The WebAssembly spec refers to the host in some places as the embedder.
WAI Bindgen
The WAI Bindgen code generator consumes *.wai
files and generates glue code
that the host can use for using functionality from a WebAssembly
module or the guest can use for implementing that functionality.
There are two primary ways users will interact with WAI Bindgen, the
wai-bindgen
CLI, and the wai-bindgen-*
family of crates.
The wai-bindgen
CLI provides a command-line interface to the wai-bindgen-*
crates, and is often used for once-off investigation or integration with a
non-Rust build system.
On the other hand, the wai-bindgen-*
crates allow users to generate bindings
programmatically and give them much more control over the generation process.
Language | Direction | Code Generator | Procedural Macro |
---|---|---|---|
Rust | Guest | wai-bindgen-gen-rust-wasm | wai-bindgen-rust |
Rust | Host (Wasmer) | wai-bindgen-gen-wasmer | wai-bindgen-wasmer |
Rust | Host (Wasmtime) | wai-bindgen-gen-wasmtime | wai-bindgen-wasmtime |
C | Guest | wai-bindgen-gen-c | |
JavaScript | Host | wai-bindgen-gen-js | |
Python | Host (Wasmer) | wai-bindgen-gen-wasmer-py | |
Python | Host (Wasmtime) | wai-bindgen-gen-wasmtime-py |
Builtin Types
All types that can be used in a *.wai
file are intended to mappable to native
types in a general purpose programming language.
The core basic types are
- Unsigned integers (
u8
,u16
,u32
,u64
) - Signed integers (
s8
,s16
,s32
,s64
) - Floating point numbers (
float32
,float64
) - UTF-8 Strings (
string
) - UTF-8 code points (
char
) - Void or nothing (
unit
)
For a more precise definition, consult the Types section in the
*.wai
format.
Other Builtin Types
Besides the basic builtin types, there are several "generic" types built into WAI which let users express common concepts.
Tuples
The tuple is equivalent to a record that has numeric fields.
Code generators may be able to express tuples as a first-class concept. For
example, tuple<string, float32, float32>
would be expressed as
(String, f32, f32)
in Rust.
Lists
Lists are dynamically-sized sequences of the same element type. Often called
a "list", "vector", or "array", a list<string>
would be expressed as
Vec<String>
in Rust.
Option
The option type is used to express a value that may or may not be present.
In Rust, an option<T>
is expressed as std::option::Option<T>
,
while other languages may choose to use null
to represent the missing value.
It is semantically equivalent to the following variant:
variant option {
none,
some(T),
}
Expected
The expected type is used to express the result of a fallible operation.
In Rust, an expected<T, E>
is expressed as
std::result::Result<T, E>
, although other languages may choose
to convert errors into exceptions.
It is semantically equivalent to the following variant:
variant expected {
ok(T),
err(E),
}
Futures & Streams
The future<T>
and stream<T, E>
types are used to represent the result of
asynchronous operations.
Functions
Functions are one of the most important concepts in WAI. They are what guests and hosts use to expose functionality to each other, and have a name, parameters, and results.
Here are some examples:
thunk: func()
fibonacci: func(n: u32) -> u32
sleep: async func(ms: u64)
Most guests will map functions to top-level functions, while most hosts will expose functions as some sort of callable object which eventually calls into the relevant part of the WebAssembly virtual machine.
For a more details, consult the Item: func
section in the *.wai
format.
Records
A record is an abstract data type containing a series of named fields. It has no associated behaviour and acts as a way to group data together. In C++, this would be referred to as a plain old data type.
Depending on the language, records may be expressed in in different ways.
Language | Equivalent Construct |
---|---|
Rust | Struct |
C | Struct |
Python | Data Class |
TypeScript | Type alias |
Syntax
A record contains a list of fields, where each field must be given a type.
record person {
name: string,
age: u32,
has-lego-action-figure: bool,
}
For a more details, consult the Item: record
section in the
*.wai
format.
Enums, Flags, Variants, and Unions
the concept of "this value can be X or Y or Z" can be expressed in several ways depending on the context.
Enum
The enum
keyword can be used to introduce a C-style enum. This is a type of
named constant where each constant has its own value.
enum ordering {
less-than,
equal-to,
greater-than,
}
For a more details, consult the Item: enum
section in the *.wai
format.
Flags
The flags
keyword can be used to introduce a bitflags variable. The easiest
way to think of this is as a "bag of bools" where each variant can be set
independently.
flags text-style {
bold,
italics,
underline,
strikethrough,
}
The flags
type is a separate concept from enum
because multiple flag
variants can be set at a time, whereas an enum
can be only one thing at a
time. Different languages are often able to express this in a very efficient
form, typically an integer where each bit represents a different flag.
For a more details, consult the Item: flags
section in the *.wai
format.
Variant
A variant
lets you express something that is one of a set of types. This is
similar to an enum
, except each variant may have some associated data.
variant error {
file-not-found(string),
parse(parse-failed),
other,
}
record parse-failed {
message: string,
line-number: u32,
}
Variants are implemented very differently based on what is idiomatic for a particular language.
In Rust, a variant
is just a normal enum
.
In TypeScript, the variant is implemented as a tagged union.
type error = { type: "file-not-found", value: string }
| { type: "parse", value: ParseFailed }
| { type: "other" };
For a more details, consult the Item: variant
section in the
*.wai
format.
Union
A union
is very similar to a variant, except it drops the type tag.
union configuration {
string,
list<string>,
}
This is distinct from a variant
because some languages may be able to
represent an union
in a way that is more efficient or idiomatic.
For a more details, consult the Item: union
section in the
*.wai
format.
Resources
A resource represents an opaque object where the representation and underlying implementation is completely hidden from the outside world. Resources may have associated methods, static methods, or no methods at all.
Depending on the language, records may be expressed in in different ways.
Language | Equivalent Construct |
---|---|
Rust | Trait object |
Python | class |
JavaScript | class |
Resources can only be used through a "handle" and can be owned by either the host or the guest. Resource lifetimes are managed manually, although most languages provide some sort of reference counting or garbage collection mechanism.
Syntax
The simplest resource is an opaque "token". Users can pass this value around, but have no other way to interact with it.
resource file-descriptor
Resources can also have methods. These are functions which are associated with the resource and are implicitly given a reference to the resource when they are invoked.
resource writer {
write: func(data: list<u8>) -> expected<unit, error>
flush: func() -> expected<unit, error>
}
Additionally, resources can have static
methods. These are often used to
implement constructors.
resource request {
static new: func() -> request
body: async func() -> list<u8>
headers: func() -> list<string>
}
For a more details, consult the Item: resource
section in the
*.wai
format.
Choosing Between Resources and Records
The difference between a resource and a record can be subtle when first starting out, but there is a simple rule of thumb that will work 90% of the time:
Records contain data, resources encapsulate behaviour.
Typical Examples
A person is a good example of a record.
record person {
name: string,
age: u8,
address: option<address>,
}
record address {
number: u32,
street: string,
state: string,
country: string,
postcode: u32,
}
On the other hand, a database connection would be best represented using a resource.
resource database {
static connect: func(connection_string: string) -> expected<database, error>
query: func(sql: string) -> expected<list<record>, error>
close: func() -> expected<unit, error>
}
Key Considerations
When deciding between using a resource or a record, consider the following:
- Performance: Records require deep copying when passed between guest and host, which can be expensive for large or complex records. Consider using resources for objects with significant amounts of data or complex structures to mitigate performance issues.
- Immutability: Records provide a level of immutability due to their pass-by-value nature. If immutability is a priority, records can be a suitable choice. However, if you need to frequently modify an object's state, a resource might be more appropriate.
- Encapsulation: For objects with both data and behavior, consider whether separating the data and behavior into different objects—a record for data and a resource for behavior—adds value or complexity to your code.
- Data Sharing: If data sharing or synchronization across components or instances is important, resources are a better choice since they use references, while records are not ideal for sharing data.
Edge Cases
While the "Records contain data, resources encapsulate behaviour" rule works for most cases, you will almost certainly run into situations where something has both data and behaviour.
This happens a lot when wrapping a "normal" library with a WAI interface so it can be used from WebAssembly. The distinction between "object" and "data" is more fluid in most general purpose programming languages, so it can be common to encounter something that doesn't neatly fall into the "record" or "resource" categories.
Workaround 1: Getters & Setters
If something would normally have publicly accessible fields and methods which might modify those fields, the best solution is to make that thing a resource with getters and setters.
For example, a virtual machine might normally expose its instruction pointer
and any global variables that are defined, while also having an eval()
method
for evaluating code snippets.
resource virtual-machine {
instruction-pointer: func() -> u32
set-instruction-pointer: func(ip: u32)
global-variables: func() -> list<tuple<string, value>>
set-global-variable: func(name: string, value: value)
eval: func(code: string) -> expected<unit, error>
}
This approach works particularly well when the methods will update state because all resources are reference types, meaning any modifications made to a resource through one handle (e.g. via a method) will be seen by all other handles to the same resource.
One downside of this approach is that each getter or setter is implemented using a method. When you have a large number of fields to expose, these extra methods can become hard to maintain or make it easy to lose interesting functionality within a wall of boilerplate.
Workaround 2: Move Methods to Top-Level Functions
Going in the other direction, sometimes it might be better to turn methods into top-level functions and use a record.
One example of this could be the satellite object used in a library that predicts the motion of a satellite.
/// An element
record satellite {
object-name: optional<string>,
norad-id: u64,
inclination: float64,
right-ascension: float64,
..
}
/// Parse a satellite from its JSON representation.
satellite-from-json: func(json: string) -> expected<satellite, error>
/// Predict where a satellite will be at a particular time.
predicted-location: func(s: satellite, ts: timestamp) -> position
This works well when the thing being expressed is mostly data, with only a couple of pieces of associated behaviour.
Records are passed around by value, meaning any operations that would normally modify a field will need to return a new value with the updated field, instead. This can be quite expensive when the record is large, because passing a record from guest to host (or host to guest) will often mean the entire object is serialized recursively and copied across the host-guest boundary. Consider the trade-offs between performance and immutability when deciding whether to use records or resources in these edge cases.
Contributing
Design Goals
This project is developed under several assumptions,
- Most of the time spent using
wasmer-pack
will actually go into the IO before and after using it rather than running the code generator (i.e. because you have to download large inputs from a server), so performance is a non-goal for this project - The core library should be usable by itself, but everything else can be tailored to Wasmer's use cases
As such, we prefer developer experience over flexibility and raw performance.
Goal 1: Fast Compile Times
A clean build of the entire workspace shouldn't take any longer than 30 seconds and all CI jobs should finish within 5 minutes.
This isn't actually too difficult to achieve as long as you follow some guidelines:
- Don't add dependencies unless you absolutely need to
- Trim out unnecessary features
- Periodically use
cargo clean && cargo build --timings
to see where compile time is spent - Don't import crates that pull in half of crates.io
The rationale behind this is simple - a short edit-compile-test cycle is a force multiplier. If you have fast compile times then developers can recompile and re-run the test suite after every change.
On the other hand, if CI takes 30 minutes to complete, developers will avoid your project like the plague because getting even the most trivial changes merged becomes a multi-hour chore.
To help this, we have a GitHub Action which will post comments on each PR to let you know how much your changes have affected CI times.
Goal 2: It Just Works
Users of wasmer-pack
should be able to compile the project from scratch and
use the generated bindings without needing to mess around with dependencies or
configuration.
To achieve this,
- Avoid dependencies that link to native libraries because they need a working C toolchain and often require installing system libraries
- Avoid conditional compilation (including feature flags) because they tend to introduce edge cases that are hard to test and debug
- Be opinionated - don't give the end user unnecessary flags or options unless it's part of the core functionality
Release Process
This is the process if you ever need to cut a release:
-
Make a PR which mentions the new version in its title (e.g.
"Release v1.2.3" on a
releases
branch) -
Update
CHANGELOG.md
to include any user-facing changes since the last release (the[Unreleased]
link at the bottom is particularly helpful here) -
Run
cargo release
. This will...- Promote the change log's
[Unreleased]
items to a named version - Bump version numbers in all
Cargo.toml
files - Tag the commit (e.g.
v1.2.3
) - Publish to crates.io, and
- Push all commits and tags to GitHub
- Promote the change log's
-
Wait for the "Releases" job to pass. This will...
- Publish WebAssembly binaries to WAPM
- Use
cargo xtask set-generator
to make the WAPM backend generate bindings with the new version ofwasmer-pack-cli
-
Merge the "Release v1.2.3" PR into the
master
branch!
Cargo xtask
We use the cargo xtask
pattern for any project automation more
complex than 1 or 2 lines of shell. This means we get access to any library
on crates.io, and having everything in pure Rust means you don't need to
manually install anything or worry about OS-specific weirdness.
Currently, there are only a couple major tasks,
cargo xtask set-generator
calls the mutation for setting a bindings generatorcargo xtask sync-schema
will make sure theschema.graphql
file is in sync with the WAPM backend, automatically updating the file if necessary
You can run cargo xtask --help
to find out more details.
Project Architecture
The wasmer-pack
project is split across several crates depending on the various
ways it might be used.
The main crates are:
crates/wasmer-pack
- this is the meat and potatoes ofwasmer-pack
. It contains all the code for generating bindings to WebAssembly modules, plus templates for any glue code that will be needed along the waycrates/cli
- this is a CLI tool that letswasmer-pack
generate bindings using the commands and libraries inside a Pirita filecrates/wasm
- this is a wrapper that makeswasmer-pack
available as a WebAssembly module. The functions and data types that are exposed are defined incrates/wasm/wasmer-pack.exports.wai
(seeWIT.md
for the syntax)
Architecture Decision Records
An architectural decision record (ADR) is a document that describes a choice the team makes about a significant aspect of the software architecture they’re planning to build. Each ADR describes the architectural decision, its context, and its consequences.
The goal is to get knowledge about a decision out of a developer's head so it doesn't get lost to time.
ADRs aren't big documents - if you are writing more than a couple paragraphs, you are probably doing it wrong!
(click to see the template)
# (short title of solved problem and solution)
| Metadata | Value |
| -------- | ----------------------------------------------------------------------------------- |
| Status | *proposed, rejected, accepted, deprecated, superseded by [ADR-123](123-example.md)* |
## Context and Problem Statement
*(Describe the context and problem statement, e.g., in free form using two to three sentences. You may want to articulate the problem in form of a question.)*
## Decision Drivers <!-- optional -->
1. *(driver 1, e.g., a force, facing concern, …)*
2. … <!-- numbers of drivers can vary -->
## Considered Options
1. option 1
2. option 2
4. … <!-- numbers of options can vary -->
## Decision Outcome
Chosen option: "option 1", because (justification. e.g., only option, which meets k.o. criterion decision driver | which resolves force force | … | comes out best).
### Positive Consequences <!-- optional -->
- (e.g., improvement of quality attribute satisfaction, follow-up decisions required, …)
- …
### Negative Consequences <!-- optional -->
- (e.g., compromising quality attribute, follow-up decisions required, …)
- …
## Pros and Cons of the Options <!-- optional -->
### option 1
*(example | description | pointer to more information | …)* <!-- optional -->
- Good, because X
- Good, because Y
- Bad, because Z
- … <!-- numbers of pros and cons can vary -->
## Links <!-- optional -->
- []()
Generated Package Naming
Metadata | Value |
---|---|
Status | proposed |
Context and Problem Statement
Currently, generated packages will derive their name from the WAPM package name,
so wasmer/wasmer-pack-cli
gets turned into wasmer_pack_cli
for Python and
@wasmer/wasmer-pack-cli
for JavaScript.
Eventually, we'd like WAPM to automatically publish these packages to PyPI or NPM, so we need to come up with names that are unique.
Decision Drivers
- Negligible chance of collisions
- Generated package names are similar to the package on WAPM
Considered Options
- Put all packages under a
@wasmer-package
organisation and use__
for delimiters - Host our own private registries
Decision Outcome
TODO: make a decision
Pros and Cons of the Options
Option 1
- Good, because it's practically guaranteed to not have collisions
- Good, because we can publish to PyPI/NPM and be used by other packages
- Good because there is an obvious way to transform a package name back and forth
- Bad, because the names become very verbose and unwieldy
wasmer/wasmer-pack
becomeswasmer_package__wasmer__wasmer_pack
on Python and@wasmer-package/wasmer__wasmer-pack
on JavaScript
Option 2
- Good, because we don't have to worry about colliding with existing packages
- Good, because we get complete control over the registry
- Bad, because it's more infrastructure to manage (operations costs, expertise, , etc.)
- Bad, because most package managers don't let you publish packages that depend on something from another registry
Links
- Original ticket - wasmerio/wasmer-pack#100
Change Log
All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
Changelog entries will contain a link to the pull request implementing that change, where applicable.
Note: The project was renamed from
wit-pack
towasmer-pack
in version 0.5.0. Changelog entries from 0.4.2 and earlier use the old name.
Unreleased - ReleaseDate
0.7.2 - 2024-02-12
0.7.2 - 2024-02-12
0.7.2 - 2024-02-12
0.7.2 - 2024-02-12
0.7.2 - 2024-02-12
0.7.2 - 2024-02-12
0.7.2 - 2024-02-12
Changed
- Allow packages with
.
in their names. This is useful for packagesmy-website.com
. Internally, the.
is converted into a-
to make it a valid binding name.
Added
- Added ability to pass in a user-specified name for the generated
bindings. This can be done by passing in the
--name
flag in the CLI or by passing thename
option when calling thegenerate_*
functions.
Fixed
-
Fixed the
python-wasi
test inwasmer-pack
crate to use poetry instead of pipenv. This is because poetry is used everywhere else, except in this one place. -
Fixed flaky integration tests by specifying a valid webc file for wasmer-pack in both
test_wasmer_pack.py
(pytest) andindex.test.ts
(jest) -
Pass in
BindingsOptions
by reference so it's easier to implement builder pattern later on.
0.7.1 - 2023-06-12
Added
- Upgraded the
webc
crate to5.0.4
so we now have support for*.webc
files in both the v1 and v2 formats (#131)
Changed
- Raised the MSRV from
1.64.0
to1.67.0
to matchwapm-targz-to-pirita
(#131)
Fixed
- Fixed the "Releases" job in CI so
wasmer/wasmer-pack-cli
would be published and not justwasmer/wasmer-pack
(#124)
0.7.0 - 2023-02-10
💥 Breaking Changes 💥
- Restructured the
wasmer/wasmer-pack
WAPM package's API (#118)- The
Package
type has been changed from arecord
to aresource
with methods - There are now convenience functions for loading a
Package
from known binary formats (e.g. WEBC)
- The
Added
- Moved the logic for loading a
wasmer_pack::Package
from a WEBC binary out ofwasmer-pack-cli
into the mainwasmer-pack
crate (#118)
Fixed
- Replaced the naive ABI detection routine with something that properly checks which namespaces are imported by a module (#118)
Removed
- Removed the
TestEnvironment
construct fromwasmer-pack-testing
in favour of theautodiscover()
tests (#121)
0.6.0 - 2022-12-28
Added
- Generated JavaScript packages can now import host functions (#109)
- Introduce the
wasmer-pack-testing
crate for testing generated bindings (#112) - Generated Python packages can now import host functions (#113)
Changed
- Renamed
wasmer-pack
's sub-commands to be calledjavascript
andpython
, withjs
andpy
as aliases (i.e. runningwasmer-pack js
is equivalent towasmer-pack javascript
) #111
0.5.3 - 2022-12-02
Fixed
- The
wasmer-pack
CLI wasn't extracting the*.wai
export files correctly (#105)
0.5.2 - 2022-11-24
Added
- Users can now also generate bindings from a directory containing a
wapm.toml
file or a*.tar.gz
downloaded from WAPM (#80) - Added a tutorial on using records (#83)
0.5.1 - 2022-11-18
Fixed
- Running
wasmer run --mapdir .:. wasmer/wasmer-pack-cli ...
would fail because the[fs]
table was being used incorrectly inwapm.toml
(#91) - Added a workaround for dealing with
*.webc
files that were generated by a buggy version ofwapm2pirita
(#92)
Changed
- The
wasmer-pack
tool now generates CommonJS packages instead of ES Modules (#89)
0.5.0 - 2022-11-08
Changed
- Switched from Wasmer's fork of
wit-bindgen
on GitHub to thewai-bindgen
crate on crates.io (#71)
Fixed
- Update
PackageName
validation to accept the_
namespace and global packages (#74) - Package bindings will no longer have naming conflicts when importing a binding type with the same name as one of types we generate (#75)
💥 Breaking Changes 💥
- The project has been renamed from
wit-pack
towasmer-pack
0.4.2 - 2022-10-30
Fixed
- Put all generated JavaScript inside a
package/
folder to match the logic used bynpm pack
when consuming tarballs (#66) - Update
MANIFEST.in
to includepy.typed
in the package, meaning MyPy can now typecheck the generated bindings (#66) - Don't assume atoms will have the same name as their commands (#64)
- Some JavaScript bindings wouldn't run because the bindings always import
@wasmer/wasi
, while the dependency was only added when one or more libraries/commands was compiled to WASI (#58)
0.4.1 - 2022-10-24
Added
- User-facing documentation and a tutorial series are now available under the
doc/
folder (#47) - Mention the
wit-pack
version in each generated package (#54)
Fixed
- Fixed a bug where
*.wasm
files weren't being installed with the Python bindings from WAPM (#52)
0.4.0 - 2022-10-12
Added
- To facilitate caching or different means of distribution, users are now able to provide their own pre-compiled WebAssembly module when initialising libraries or running commands (#45)
Changed
- Removed the
LoadArgs
type from the Python bindings in favour of named arguments (#45) - Raised the MSRV from
1.59.0
to1.61.0
to matchminijinja
- Removed the top-level class from the generated bindings, so now you just need
to do something like
from wit_pack import bindings, commands
to use the package's libraries or commands (#40)
Fixed
- Make the current directory available to the CLI when run by wasmer (#37)
0.3.0 - 2022-09-27
Added
- Set up CI to automatically deploy to wapm.dev whenever GitHub receives a tagged commit (#24)
- Fleshed out the repo's documentation (#25)
- Populated the
CHANGELOG.md
- Wrote up
CONTRIBUTING.md
- Rewrote the
README.md
walkthrough
- Populated the
- Added a "Time Reporter" task to CI so we can keep an eye on CI times (#25)
- Generate wrappers for calling WASI executables from JavaScript (#26)
- Generate wrappers for calling WASI executables from Python (#27)
- Detect all available WASI executables in a Pirita file (#28)
- Add a top-level facade to the generated Python bindings so libraries and commands can be accessed through one common object (#30)
- Add a top-level facade to the generated JavaScript bindings so libraries and commands can be accessed through one common object (#34)
- Added a
wit-pack show
sub-command to show which libraries and commands would be generated from a Pirita file (#35)
Fixed
- Inspect each atom's kind when discovering the commands in a Pirita file instead of blindly assuming everything is a command (#32)
0.2.3 - 2022-09-15
Fixed
- When run as a WASI program, the
wit-pack
CLI would unconditionally fail to load inputs becausemmap
isn't available (#24)
0.2.2 - 2022-09-15
(no user-facing changes)
0.2.1 - 2022-09-15
(no user-facing changes)
0.2.0 - 2022-09-15
Added
- The
wit-pack
crate now allows packages to contain multiple WebAssembly modules (#22)
💥 Breaking Changes 💥
-
The
wit-pack
CLI now takes a Pirita file as its only input (#20)-
This means the commandline interface has changed
# Instead of this $ wit-pack js --exports exports.wit --name hello_world --version 0.1.1 --module wit.wasm -o=wit-js --abi=none # you should now do this $ wit-pack js -o=wit-js ./hello-world.webc
-
0.1.5 - 2022-09-12
Added
- Introduced support for WASI libraries (#12)
Changed
- The
crates/wit-pack-cli
andcrates/wit-pack-wasm
crates are now published to WAPM under thewasmer
namespace instead ofMichael-F-Bryan
0.1.4 - 2022-08-25
(no user-facing changes)
0.1.3 - 2022-08-25
(no user-facing changes)