Pyodide 0.19 is a major release focused on improving the build system, updating scientific computing packages, and improving the workflow for loading Python code.

Here are a few highlights. For the complete list of changes see the changelog.

Removal of function pointer cast emulation

Some Python C extensions use function pointer casting, which is an unspecified behavior according to the C specification. However, all common native C compilers generate code that works fine. This is not the case with Emscripten when building for WebAssembly, which will fail on function pointer casts. As a workaround we have been using Emscripten’s -s EMULATE_FUNCTION_POINTER_CASTS flag, which emits code to emulate function pointer casts at runtime. This has significant overhead.

In version 0.19 we were able to remove the function cast emulation by using JavaScript method call trampolines in the few places where the cast function pointers are called. This allowed us to increase the maximum supported recursion depth to the default value of 1000, decrease the compiled code size by ~25% and make Python run faster by ~15%. For more details about this work, see our post on Function Pointer Cast Handling in Pyodide.

SciPy Upgrade

The initial version of SciPy added to Pyodide was 0.17.1 from 2016. Since then we have had major difficulties updating it because SciPy is large and difficult to build. We particularly struggle with the use of Fortran which has no reliable compiler supporting WebAssembly. Our inability to update SciPy in turn prevented us from updating other scientific Python packages, including scikit-learn, scikit-image and statsmodels.

SciPy was finally updated in pyodide/pyodide#2065 after a great deal of effort. There are still known issues with SciPy in Pyodide (you can see the current status of the scipy test suite here). However, this update is a major milestone in getting us closer to the development version of SciPy and allowing us to contribute fixes upstream.

New unpackArchive and pyimport apis

We added new APIs pyodide.unpackArchive and pyodide.pyimport.

These are part of our goal of encouraging people to develop their Python code for use in Pyodide like normal Python packages and to call into their Python packages directly from JavaScript, rather than using pyodide.runPython.

Users can develop a Python package, then zip it and install it with pyodide.unpackArchive, import it with pyodide.pyimport and then call their Python functions directly from Javascript.

See the discussion here: pyodide/pyodide#1940.

New pyfetch API

We added the pyfetch API to which is a convenience wrapper to make fetch requests from Python and convert the response directly into various Python types while avoiding copying data as much as possible.

Micropip improvements

The observability of micropip was improved with better error messages, an easier way to debug missing wheels, and a way to list the packages micropip has installed.

Type conversions improvements and a new JS calling convention

We added polish to core features based on user feedback, including bugfixes and convenience improvements.

We added several functions that allow conversion of JavaScript buffers to various Python objects with as few copies as possible, which are useful internally to Pyodide and should also help applications processing data to do faster conversions in certain cases.

We also changed the calling convention for invoking a JavaScript function from Python so that in some cases no special care needs to be taken to call JavaScript functions from Python without leaking Python objects. We also made it much easier to call asynchronous Python functions from JavaScript.

Acknowledgements

We thank all contributors to this release as well as all users who provided feedback and reported issues.

The following people commited to Pyodide in this release:

Alexey Ignatiev, Alex Hall, Bart Broere, Cyrille Bogaert, etienne, Grimmer, Grimmer Kang, Gyeongjae Choi, Hao Zhang, Hood Chatham, Ian Clester, Jan Max Meyer, LeoPsidom, Liumeo, Michael Christensen, Owen Ou, Roman Yurchak, Seungmin Kim, Sylvain, Thorsten Beier, Wei Ouyang, Will Lachance