Pyodide 0.22 is a major release focused on improving the foreign function interface between Python and JavaScript. We also improved the systems for building, loading, and testing packages. In particular, we added support for creating virtual environments that execute code using Pyodide in Node.js.

We unvendored the Pyodide package manager and matplotlib backend into separate repositories. We also added a new command line interface for common tasks involved in building and testing packages with Pyodide.

Additionally, we added 16 new packages to Pyodide, including geopandas and fiona, which have been a longstanding request from the geospatial community.

For a complete list of changes, please see the changelog.

New JavaScript APIs

We added support for mounting a folder from the local filesystem into the Pyodide filesystem using the Chrome File System Access API. If you acquire a directory handle (say from showDirectoryPicker) then you can mount this directory handle.

We added new API to set the standard stream handlers from JavaScript and we improved the defaults for stream handling. The new stream APIs are particularly useful if you want to make a terminal-like interface to Python.

Pyodide virtual environments

We added support for creating virtual environments that run Python using Pyodide in Node.js. This is particularly exciting because it makes it simpler than ever to test pure Python packages against Pyodide. In many cases it is possible to run the tests in the same way as in native Python (though building and installing the dependencies may take extra care). As a proof of concept, we added Pyodide to numpy CI.

Unvendoring Pyodide-specific packages

Pyodide includes a few pure Python packages written specifically for use with Pyodide. We have unvendored these packages into standalone repositories.

The following packages have been unvendored:

By separating these packages from the main Pyodide repository, we hope to facilitate their development and make it easier for the Python-in-the-browser community to use and improve them. They depend on the Pyodide foreign function interface so for now it is still difficult to use them separate from Pyodide. In the future we hope to also unvendor the foreign function interface so that other Emscripten builds of CPython can use it.

Improvements to the foreign function interface

We made a large number of minor improvements to the foreign function interface, particularly to the use of JavaScript objects in Python. There is a lot of work left to do here, particularly to improve static type analysis, but these improvements filled a lot of minor holes.

Improved static and runtime type analysis

We split the pyodide.ffi.JsProxy class into several subclasses e.g., JsMap, JsBuffer, etc. These classes can be used for static or runtime type analysis of JavaScript objects. We made major improvements to the static type analysis system, including a typeshed for some commonly-used objects in the JavaScript global scope.

JavaScript objects as drop-in replacements for Python ones

We improved the comprehensiveness of the APIs for using several common JavaScript objects. JavaScript Arrays now implement the full collections.abc.MutableSequence API and so can be used as drop-in replacements for lists in most circumstances. For instance, they can now be used in match blocks. Similarly, JavaScript Maps implement the MutableMap API. We also added an as_object_map method, which can be used to treat a JavaScript object as a dictionary. For example,

run_js("({a:2, b:3})").as_object_map()["a"]

returns 2 (whereas

run_js("({a:2, b:3})")["a"]

raises TypeError: 'JsProxy' object is not subscriptable).

Iterators and Generators

The foreign function interface now has bidirectional support for async iterators and async iterables. Python generators and async generators can now be used as drop-in replacements for JavaScript generators and vice versa. There is new lifetime management when using JavaScript generators from Python which should make them much easier to use them.

Cross-language bound methods

We also added a __get__ descriptor to JsCallable so that it is possible to use a JavaScript function as a method for a Python class:

class T:
    def __init__(self, a, b):
        self.a = a
        self.b = b
    
    js_get_attr = run_js(
        """
        (function(name){
            // `this` binds the `self` parameter of the method!
            return this[name];
        })
        """
    )

t = T(3, 5)
t.js_get_attr("a") # 3
t.js_get_attr("b") # 5

In most cases this is probably not useful, but it is cool that it is possible.

In the reverse direction, it is possible to request that invocations of a PyProxy pass this as the first argument. This makes it possible to use a Python method with a JavaScript object:

let o = {a:3, b:5};
o.py_get_attr = pyodide.runPython(`
def py_get_attr(self, name):
    return getattr(self, name)
py_get_attr
`).captureThis();
o.py_get_attr("a"); // 3
o.py_get_attr("b"); // 5
let f = o.py_get_attr.bind(o);
f("a"); // 3

The new Pyodide CLI interface

We added a new CLI interface to Pyodide. This is a work in progress, but it already has a few useful features, including a pyodide venv command for creating virtual environments mentioned above. This new CLI interface will eventually replace the old pyodide-build interface.

The new CLI interface is based on the Python entrypoints system, which allows developers to add new commands by creating a Python package with an entrypoint. To add a new command, you can include the following in your setup.cfg file:

[options.entry_points]
pyodide.cli =
 do_something = "<your-package>.cli:main"

Alternatively, you can use the following in your pyproject.toml file:

[project.entry-points."pyodide.cli"]
do_something = "<your-package>.cli:main"

We hope that this new CLI interface will make it easier for developers to use and contribute to Pyodide.

Acknowledgements

Thanks to everyone who contributed code to this release and all users who reported issues and provided feedback. Thanks to the Jupyterlite and PyScript developers for their feedback and contributions. Thanks to the Emscripten team for their helpful and responsive support. Thanks to Brett Cannon and Christian Heimes for continuing to work on improving core Python support for the Emscripten platform. We would also like to thank the entire Python-in-the-browser community for experimentation and discussion which have helped determine the direction both of Pyodide and the broader Python-in-the-browser project.

The following people commited to Pyodide in this release:

Aierie, dataxerik, David Lechner, Deepak Cherian, Filipe, Gyeongjae Choi, Hood Chatham, H.Yamada, Jacques Boscq, Jeremy Tuloup, Joe Marshall, John Wason, Loïc Estève, partev, Patrick Arminio, Péter Ferenc Gyarmati, Prete, Qijia Liu, Roman Yurchak, skelsec, Starz0r, Will Lachance, YeonWoo, Yizhi Liu