-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Description
Summary
Currently, RustPython does not provide the os module for wasm32-unknown-unknown. The module is gated behind the host_env feature, and the wasm32 example project disables it (default-features = false, features = ["compiler"]). However, CPython's Emscripten build provides an os module with a reduced function set, and many Python packages do import os at the top level. Without the os module, even os.path.join() (pure Python) fails with ImportError.
Motivation
- Many Python packages unconditionally
import osat the top level — they currently fail entirely on wasm32-unknown-unknown os.pathfunctions (join,dirname,basename, etc.) are pure Python and would work fine if the module existed- Constants like
os.sep,os.name,os.O_RDONLYare useful even without real I/O - Functions like
os.fspath(),os.getpid(),os.strerror()can return sensible values - CPython's Emscripten build takes this approach — provide the module, let unsupported operations raise
OSErrorat runtime
Problem Analysis
The os module (specifically _os in os.rs) fails to compile on wasm32-unknown-unknown due to several dependencies:
1. crt_fd.rs depends on libc
mod c { pub(super) use libc::*; }—libcexports nothing forwasm32-unknown-unknown- Functions like
c::open(),c::close(),c::read(),c::write(),c::fsync(),c::ftruncate()don't exist - Constants like
c::EBADF,c::off_tdon't exist - Currently gated:
#[cfg(all(feature = "std", any(unix, windows, target_os = "wasi")))]inlib.rs
2. fileutils.rs depends on libc::stat
pub use libc::stat as StatStructon non-windows — nolibc::staton wasm32fstat()useslibc::fstat()— doesn't exist- Currently gated:
#[cfg(all(feature = "std", any(not(target_arch = "wasm32"), target_os = "wasi")))]
3. os.rs uses libc directly
#[pyattr] use libc::{O_APPEND, O_CREAT, O_EXCL, O_RDONLY, O_RDWR, O_TRUNC, O_WRONLY}— empty on wasm32libc::EINTRin read/write retry loopslibc::isatty()inisatty()libc::strerror()instrerror()libc::lseek()inlseek()libc::nl_langinfo()indevice_encoding()libc::stat/lstat/fstat/fstatatinstat_inner()
4. num_cpus crate — not available for wasm32-unknown-unknown
- Used in
cpu_count()
5. os_open() is gated behind cfg(any(unix, windows, target_os = "wasi"))
Proposed Solution
Approach: Compile-only support — the os module compiles and is importable on wasm32-unknown-unknown. Functions that can't work on wasm32 will raise OSError at runtime (since std::fs returns errors). Constants and path functions work normally.
Files to Modify
crates/common/src/lib.rs
Expand cfg gates to include wasm32-unknown-unknown:
// crt_fd: include wasm32
#[cfg(all(feature = "std", any(unix, windows, target_os = "wasi", target_arch = "wasm32")))]
pub mod crt_fd;
// fileutils: remove wasm32 exclusion
#[cfg(feature = "std")]
pub mod fileutils;crates/common/src/crt_fd.rs
Add a wasm32-specific mod c block (similar to the existing Windows-specific sections):
#[cfg(all(target_arch = "wasm32", not(any(unix, windows, target_os = "wasi"))))]
mod c {
pub type off_t = i64;
pub type c_int = i32;
pub type c_char = i8;
pub const EBADF: c_int = 9;
// Stub functions that always return -1
pub unsafe fn open(_: *const c_char, _: c_int, _: c_int) -> c_int { -1 }
pub unsafe fn close(_: c_int) -> c_int { -1 }
pub unsafe fn read(_: c_int, _: *mut u8, _: usize) -> isize { -1 }
pub unsafe fn write(_: c_int, _: *const u8, _: usize) -> isize { -1 }
pub unsafe fn fsync(_: c_int) -> c_int { -1 }
pub unsafe fn ftruncate(_: c_int, _: off_t) -> c_int { -1 }
}std::os::fd types (OwnedFd, BorrowedFd, RawFd) are available on wasm32 since Rust 1.84, so the existing #[cfg(not(windows))] imports work.
crates/common/src/fileutils.rs
Add wasm32-specific StatStruct and fstat stub:
#[cfg(not(any(unix, windows, target_os = "wasi")))]
pub struct StatStruct { /* minimal fields matching libc::stat layout */ }
#[cfg(not(any(unix, windows, target_os = "wasi")))]
pub fn fstat(_: crate::crt_fd::Borrowed<'_>) -> std::io::Result<StatStruct> {
Err(std::io::Error::new(std::io::ErrorKind::Unsupported, "fstat not supported"))
}crates/vm/src/stdlib/os.rs
~10 changes needed:
- O_* constants: Define manually for wasm32 (
O_RDONLY=0,O_WRONLY=1,O_RDWR=2, etc.) open/os_open: Add wasm32 path returning errorread/write: Replacelibc::EINTRwith cfg-gated constantisatty: Returnfalseon wasm32strerror: Basic error-number-to-string mappinglseek: Return error on wasm32stat_inner: Usestd::fs::metadataor return errorStatResultData::from_stat: Handle wasm32 StatStruct fieldscpu_count: Return 1 on wasm32device_encoding: Return "UTF-8" on wasm32utime_impl: Return "not supported" error on wasm32
crates/vm/src/stdlib/posix_compat.rs
Add environ for wasm32 (currently only defined for target_os = "wasi"):
#[cfg(all(target_arch = "wasm32", not(target_os = "wasi")))]
#[pyattr]
fn environ(vm: &VirtualMachine) -> crate::builtins::PyDictRef {
vm.ctx.new_dict() // Empty dict — no env vars on bare wasm32
}Cfg Pattern
The recurring condition is:
#[cfg(all(target_arch = "wasm32", not(any(unix, windows, target_os = "wasi"))))]This matches wasm32-unknown-unknown specifically (not emscripten, not wasi, not windows).
Verification Plan
cargo check --target wasm32-unknown-unknown -p rustpython-vm --features "compiler,host_env"— must compilecargo test -p rustpython-vmon host — no regressions- Update
example_projects/wasm32_without_js/to optionally enablehost_env - Runtime: build wasm32 binary with
import os; print(os.name)— should print"posix"without error
References
- CPython Emscripten support: https://docs.python.org/3/using/wasm.html
std::os::fdportable stabilization (Rust 1.84): std::os::fd module missing on Hermit rust-lang/rust#126198libccrate wasm32-unknown-unknown: exports nothing — https://docs.rs/libc/latest/libc/