Pyodide 0.24 is a major release focused on fine tuning public APIs and
performance. It includes a major rework of the streams APIs to be faster and
more flexible. We increased the consistency of the foreign function interface.
We also added
micropip support for the Python simple repository API.
Up to this point, we had not devoted much energy to performance improvements so there were many low-hanging fruit.
We finally added a
packages argument to
loadPyodide so that packages can be
downloaded during the Pyodide bootstrap. We also fixed the problem that
pyodide.asm.wasm only began downloading after
pyodide.asm.js was finished
loading. Loading them simultaneously can improve the load time by as much as a
second depending on network conditions. There is a lot more work to do in
improving load time but both of these changes are significant improvements.
Foreign Function Interface performance
Several major bottlenecks were identified. The biggest performance problem was
with the hiwire map. This map is needed because C cannot directly handle
taking almost all of the execution time in some profiles. By changing the data
structures involved we managed to improve the performance of
to_js by a factor
of 15 on a 10kb json object.
Also, rendering the error messages for
PyProxy use-after-free errors takes a
huge percentage of the runtime in some use cases. We fixed this by pregenerating
the strings involved and interning the results, and by only enabling more
detailed error messages if
pyodide.setDebug(true) is run first.
As another time saver, we avoid registering
PyProxy arguments with the
FinalizationRegistry unless necessary.
FinalizationRegistry.unregister are expensive calls and in some cases we know
PyProxy is automatically destroyed by the Pyodide foreign function
interface so these calls can be avoided.
The internal implementation of the
pyodide.setStdxxx APIs was completely
reworked. It is now faster and allows correct handling of keyboard interrupts
EAGAIN. We also added a new way of implementing standard
streams by defining a
read function for
stdin or a
write function for
stdout/stderr. These are the simplest APIs to implement internally, and the
older APIs are implemented in terms of them.
We’ve made some useful improvements to micropip.
Simple Repository API support
micropip now supports the Simple Repository API (PEP 503, PEP 691), which is a
standard way of querying packages to the Python package repository. We also
added supports for using package repositorys other than PyPI via
micropip.set_index_urls(). This means
that you can host and use a collection of webassembly-emscripten Python packages
that you have built in your local environment, using
Other API improvements
It is now possible to remove packages at runtime using
We also added the
verbose option to
micropip.install. Previously, micropip
was totally silent and it was hard to tell which versions of which packages were
being installed from which indices. Now, when run in verbose mode, pip-like
installation messages are printed to stdout.
For a detailed list of changes, see the v0.24 changelog.
- Recently clang and Emscripten have added much better support for
externrefs will allow further performance improvements and a significant reduction in code size.
- Cibuildwheel support and a PEP for Emscripten wheels on PyPI support.
- More work on load time and ffi performance.
Thanks to everyone who contributed code to this release and all users who reported issues and provided feedback. Thanks to the PyScript and Jupyterlite developers and community for their feedback and contributions. Particularly, we would like to thank Andrea Giammarchi, Antonio Cuni, Chris Laffner, and Der Thorsten. Thanks to Brett Cannon and Mark Shannon for core Python support.
Thanks to the Emscripten team for their helpful and responsive support.
The following people commited to Pyodide in this release:
Adam Tombleson, Alexey Ignatiev, Andrea Giammarchi, Angus Hollands, Arpit, Bart Broere, Christian Clauss, chrysn, David Lechner, Deepak Cherian, Eli Lamb, Feodor Fitsner, Guillaume Lemaitre, Gyeongjae Choi, Hannes Krumbiegel, Hanno Rein, Henry Schreiner, Hood Chatham, Ian Thomas, Jeff Glass, Jo Bovy, Joe Marshall, Joseph Rocca, Juniper Tyree, Kai Tietz, Kevin Hill, Loïc Estève, Luiz Irber, Maks Bialas, Martin Renou, Martoxa, messense, Michael Weinold, Milan Raj, nascheinkman, Neil Stoker, Nicholas Bollweg, Owen Lamont, Ralf Gommers, Roman Yurchak, Tamás Nepusz, TheOnlyWayUp, Tim Paine, Tim Sherratt, Tomas R, Tom Nicholas, Victor Blomqvist, Yan Wong, Ye Joo Park, Yossi Shirizli, कारतोफ्फेलस्क्रिप्ट