Happy New Year! Pyodide 0.25 is a major release with some exciting and experimental features. This major release includes requests library support, experimental support for JS Promise integration, and build system improvements.

requests Support in Pyodide

For a long time, the Pyodide community has been requesting support for the requests, Python’s popular HTTP client library. However, due to the limitations of Pyodide running in a browser environment, integrating networking libraries presented challenges. So, traditionally, if you needed to do something networking-related in Pyodide, you would need to use the pyfetch API, which is a wrapper around the browser’s fetch API.

Another workaround was to use the pyodide-http by @koenvo, which provides a patch mechanism for common HTTP libraries, including requests, urllib.

With this release, Pyodide now provides native support for requests without the need for patches. How does it work? @joemarshall has contributed a new adapter for urllib3, a backend of requests, that uses Web APIs supported by Pyodide instead of native sockets when running in a Pyodide environment.

Try the following code snippet to see requests in action:

await pyodide.loadPackage("requests")
pyodide.runPython(`
  import requests
  license = requests.get("https://raw.githubusercontent.com/pyodide/pyodide/main/LICENSE").text
  print(license)
`)

Please note that requests in Pyodide are still subject to browser constraints, including CORS policies and limited access to low-level networking features not accessible from the browser. However, we hope that this new feature will make Pyodide more useful for many users.

Experimental Support for JS Promise Integration

A very common problem encountered using Pyodide is the need to call an asynchronous API in a synchronous context. For example, the requests library would be most naturally implemented in terms of the browser fetch API, but requests is a synchronous library and fetch is asynchronous so this is impossible.

An experimental new browser feature will hopefully change this. The JavaScript Promise integration API allows us to make async calls from a synchronous context. For example:

// run Python code in an environment that can stack switch
py.runPythonSyncifying(`
  from js import fetch

  def sync_fetch(url):
    resp = fetch(url).syncify()
    text = resp.text().syncify()
    return text

  print(sync_fetch("https://example.com"))
`);

This will work in node v20 with the --experimental-wasm-stack-switching flag or in Chrome with Experimental WebAssembly JavaScript Promise Integration (JSPI) enabled in chrome://flags.

This feature is a work in progress, and in particular it will take some time to smooth out the interaction between the event loop and syncify(). We are very excited about the possible applications.

Build System Improvements

Pyodide’s package build system is continually evolving to enhance user experience and compatibility. In this release, we introduce the following improvements:

  • Meson Backend Support: We have added support for the meson backend, which numpy and scipy have recently adopted.
  • Enhanced Support for Conda environment: We have improved the build process within the conda environment.

We are actively working on improved support for out-of-tree builds to make it easier to build packages. We hope that these improvements will make it easier for users to build packages for Pyodide.

Acknowledgements

Thanks to everyone who contributed code to this release and all users who reported issues and provided valuable feedback. Particularly, we would like to thank Joe Marshall and Koen Vossen for their substantial contributions to bring networking libraries to Pyodide and Loïc Estève for assisting us in debugging scipy and scikit-learn issues.

Additionally, we always appreciate the support and assistance from the Emscripten team.

The following people commited to Pyodide in this release:

Alex Bolenok, Andrew Moon, Angus Hollands, Christian Clauss, David Lechner, Deepak Cherian, Gyeongjae Choi, Hood Chatham, Ian Thomas, Jo Bovy, Joe Marshall, Kai Mühlbauer, Lewis Cowles, Loïc Estève, Luiz Irber, Matthew Taylor, Matty, Max Halford, Nicholas Bollweg, pfebrer, Roman Yurchak, Steven Weaver, Szabolcs Dombi, Toshiki Teramura