Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add script to automate WebAssembly builds #95853

Closed
tiran opened this issue Aug 10, 2022 · 0 comments
Closed

Add script to automate WebAssembly builds #95853

tiran opened this issue Aug 10, 2022 · 0 comments
Labels
OS-wasi type-feature A feature request or enhancement

Comments

@tiran
Copy link
Member

tiran commented Aug 10, 2022

Feature or enhancement

I propose to add a tool that automates WebAssembly builds and guides developers through installation process of SDKs.

Pitch

It is not trivial to build Python on WebAssembly platforms. Developers have to install the correct SDKs, active them or set env vars. They also have to build a build Python interpreter before they can compile Python to WASM. Each variant requires wrappers for configure and make and several configure arguments.

I'm going to automate the steps with a Python script. The script features several build profiles and can run tests, too.

  $ ./Tools/wasm/wasm_builder.py emscripten-browser compile
  $ ./Tools/wasm/wasm_builder.py emscripten-node-dl test
  $ ./Tools/wasm/wasm_builder.py wasi test
@tiran tiran added the type-feature A feature request or enhancement label Aug 10, 2022
@bedevere-bot bedevere-bot mentioned this issue Aug 10, 2022
Merged
tiran added a commit that referenced this issue Aug 13, 2022
@tiran @brettcannon
gh-95853: Add script to automate WASM build (GH-95828)
32ac98e 
Automate WASM build with a new Python script. The script provides
several build profiles with configure flags for Emscripten flavors
and WASI. The script can detect and use Emscripten SDK and WASI SDK from
default locations or env vars.

``configure`` now detects Node arguments and creates HOSTRUNNER
arguments for Node 16. It also sets correct arguments for
``wasm64-emscripten``.

Co-authored-by: Brett Cannon <[email protected]>
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Aug 13, 2022
@tiran @miss-islington
pythongh-95853: Add script to automate WASM build (pythonGH-95828)
96c378a 
Automate WASM build with a new Python script. The script provides
several build profiles with configure flags for Emscripten flavors
and WASI. The script can detect and use Emscripten SDK and WASI SDK from
default locations or env vars.

``configure`` now detects Node arguments and creates HOSTRUNNER
arguments for Node 16. It also sets correct arguments for
``wasm64-emscripten``.

Co-authored-by: Brett Cannon <[email protected]>
(cherry picked from commit 32ac98e)

Co-authored-by: Christian Heimes <[email protected]>
@bedevere-bot bedevere-bot mentioned this issue Aug 13, 2022
Closed
tiran added a commit to tiran/cpython that referenced this issue Aug 14, 2022
@tiran
pythongh-95853: Address wasm build and test issues
1719a26 
- pyexpat.c: readinst(): Use correct format char for int.
- Add workaround for Emscripten timezone issue with calculating tm_yday.
- Skip a decimal test that allocates a lot of memory. It sometimes fails
  on Emscripten.
tiran added a commit to tiran/cpython that referenced this issue Aug 14, 2022
@tiran
pythongh-95853: Address wasm build and test issues
91ce8dc 
- pyexpat.c: readinst(): Use correct format char for int.
- Add workaround for Emscripten timezone issue with calculating tm_yday.
- Skip a decimal test that allocates a lot of memory. It sometimes fails
  on Emscripten.
@bedevere-bot bedevere-bot mentioned this issue Aug 14, 2022
Merged
tiran added a commit that referenced this issue Aug 15, 2022
@tiran
gh-95853: Address wasm build and test issues (GH-95985)
4a7f5a5
tiran added a commit to tiran/cpython that referenced this issue Aug 17, 2022
@tiran
pythongh-95853: WASM: better version and asset handling in scripts
46e0998
@bedevere-bot bedevere-bot mentioned this issue Aug 17, 2022
Merged
@ezio-melotti ezio-melotti mentioned this issue Aug 18, 2022
Closed
tiran added a commit to tiran/cpython that referenced this issue Aug 18, 2022
@tiran
pythongh-95853: WASM: better version and asset handling
0a95b55 
- support EMSDK tot-upstream and git releases
- allow WASM assents for wasm64-emscripten and WASI. This makes single
  file distributions on WASI easier.
- decouple WASM assets from browser builds
tiran added a commit that referenced this issue Aug 19, 2022
@tiran
gh-95853: WASM: better version and asset handling in scripts (GH-96045)
6087f49 
- support EMSDK tot-upstream and git releases
- allow WASM assents for wasm64-emscripten and WASI. This makes single file distributions on WASI easier.
- decouple WASM assets from browser builds
tiran added a commit to tiran/cpython that referenced this issue Aug 29, 2022
@tiran
pythongh-95853: Improve WASM build script
292fca0 
- pre-build Emscripten ports and system libraries
- check for broken EMSDK versions
- use EMSDK's node for wasm32-emscripten
- warn when PKG_CONFIG_PATH is set
- add support level information
@bedevere-bot bedevere-bot mentioned this issue Aug 29, 2022
Merged
tiran added a commit that referenced this issue Aug 30, 2022
@tiran
gh-95853: Improve WASM build script (GH-96389)
2eb9008 
- pre-build Emscripten ports and system libraries
- check for broken EMSDK versions
- use EMSDK's node for wasm32-emscripten
- warn when PKG_CONFIG_PATH is set
- add support level information
tiran added a commit to tiran/cpython that referenced this issue Sep 3, 2022
@tiran
pythongh-95853: Multiple ops and logging for wasm_build
470028c
tiran added a commit to tiran/cpython that referenced this issue Sep 4, 2022
@tiran
pythongh-95853: Multiple ops and debug for wasm_build.py
693ce8e 
- `wasm_build.py` now supports multiple operations.
- only `build` (default) performs auto-building
- new ops `configure` and `compile`.
- better help
- `repl` starts `wasm_webserver` and opens browser repl
tiran added a commit to tiran/cpython that referenced this issue Sep 6, 2022
@tiran
pythongh-95853: Multiple ops and debug for wasm_build.py
bd8fff6 
- `wasm_build.py` now supports multiple operations.
- only `build` (default) performs auto-building
- new ops `configure` and `compile`.
- better help
- `repl` starts `wasm_webserver` and opens browser repl
- add `make hostrunnertest`
tiran added a commit to tiran/cpython that referenced this issue Sep 6, 2022
@tiran
pythongh-95853: Multiple ops and debug for wasm_build.py
54782ec 
- `wasm_build.py` now supports multiple operations.
- only `build` (default) performs auto-building
- new ops `configure` and `compile`.
- better help
- `repl` starts `wasm_webserver` and opens browser repl
- `test` and `hostrunnertest` now support custom testopts
tiran added a commit to tiran/cpython that referenced this issue Sep 6, 2022
@tiran
pythongh-95853: Multiple ops and debug for wasm_build.py
4b2dd01 
- `wasm_build.py` now supports multiple operations.
- only `build` (default) performs auto-building
- new ops `configure` and `compile`.
- better help
- `repl` starts `wasm_webserver` and opens browser repl
- `test` and `hostrunnertest` now support custom testopts
tiran added a commit to tiran/cpython that referenced this issue Sep 6, 2022
@tiran
pythongh-95853: Multiple ops and debug for wasm_build.py
bca78f3 
- `wasm_build.py` now supports multiple operations.
- only `build` (default) performs auto-building
- new ops `configure` and `compile`.
- better help
- `repl` starts `wasm_webserver` and opens browser repl
- `test` and `hostrunnertest` now support custom testopts
tiran added a commit to tiran/cpython that referenced this issue Sep 6, 2022
@tiran
pythongh-95853: Multiple ops and debug for wasm_build.py
105673f 
- `wasm_build.py` now supports multiple operations.
- only `build` (default) performs auto-building
- new ops `configure` and `compile`.
- better help
- `repl` starts `wasm_webserver` and opens browser repl
- `test` and `hostrunnertest` now support custom testopts
tiran added a commit to tiran/cpython that referenced this issue Sep 8, 2022
@tiran
pythongh-95853: Multiple ops and debug for wasm_build.py
4054552 
- `wasm_build.py` now supports multiple operations.
- only `build` (default) performs auto-building
- new ops `configure` and `compile`.
- better help
- `repl` starts `wasm_webserver` and opens browser repl
- `test` and `hostrunnertest` now support custom testopts
tiran added a commit to tiran/cpython that referenced this issue Sep 8, 2022
@tiran
pythongh-95853: Multiple ops and debug for wasm_build.py
cdeee67 
- `wasm_build.py` now supports multiple operations.
- only `build` (default) performs auto-building
- new ops `configure` and `compile`.
- better help
- `repl` starts `wasm_webserver` and opens browser repl
- `test` and `hostrunnertest` now support custom testopts
tiran added a commit to tiran/cpython that referenced this issue Sep 8, 2022
@tiran
pythongh-95853: Multiple ops and debug for wasm_build.py
16ee5b2 
- `wasm_build.py` now supports multiple operations.
- only `build` (default) performs auto-building
- new ops `configure` and `compile`.
- better help
- `repl` starts `wasm_webserver` and opens browser repl
- `test` and `hostrunnertest` now support custom testopts
tiran added a commit to tiran/cpython that referenced this issue Sep 8, 2022
@tiran
pythongh-95853: Multiple ops and debug for wasm_build.py
5583674 
- `wasm_build.py` now supports multiple operations.
- only `build` (default) performs auto-building
- new ops `configure` and `compile`.
- better help
- `repl` starts `wasm_webserver` and opens browser repl
- `test` and `hostrunnertest` now support custom testopts
tiran added a commit to tiran/cpython that referenced this issue Sep 10, 2022
@tiran
pythongh-95853: Multiple ops and debug for wasm_build.py
0779369 
- `wasm_build.py` now supports multiple operations.
- only `build` (default) performs auto-building
- new ops `configure` and `compile`.
- better help
- `repl` starts `wasm_webserver` and opens browser repl
- `test` and `hostrunnertest` now support custom testopts
tiran added a commit to tiran/cpython that referenced this issue Sep 10, 2022
@tiran
pythongh-95853: Multiple ops and debug for wasm_build.py
4166618 
- `wasm_build.py` now supports multiple operations.
- only `build` (default) performs auto-building
- new ops `configure` and `compile`.
- better help
- `repl` starts `wasm_webserver` and opens browser repl
- `test` and `hostrunnertest` now support custom testopts
tiran added a commit to tiran/cpython that referenced this issue Sep 10, 2022
@tiran
pythongh-95853: Multiple ops and debug for wasm_build.py
ecc8bf6 
- `wasm_build.py` now supports multiple operations.
- only `build` (default) performs auto-building
- new ops `configure` and `compile`.
- better help
- `repl` starts `wasm_webserver` and opens browser repl
- `test` and `hostrunnertest` now support custom testopts
@bedevere-bot bedevere-bot mentioned this issue Sep 10, 2022
Merged
tiran added a commit that referenced this issue Sep 11, 2022
@tiran
gh-95853: Multiple ops and debug for wasm_build.py (#96744)
1fc8bd3
tiran added a commit to tiran/cpython that referenced this issue Sep 11, 2022
@tiran @brettcannon
pythongh-95853: Add script to automate WASM build (pythonGH-95828)
4fc1e0d 
Automate WASM build with a new Python script. The script provides
several build profiles with configure flags for Emscripten flavors
and WASI. The script can detect and use Emscripten SDK and WASI SDK from
default locations or env vars.

``configure`` now detects Node arguments and creates HOSTRUNNER
arguments for Node 16. It also sets correct arguments for
``wasm64-emscripten``.

Co-authored-by: Brett Cannon <[email protected]>
tiran added a commit to tiran/cpython that referenced this issue Sep 11, 2022
@tiran
pythongh-95853: Address wasm build and test issues (pythonGH-95985)
1a3c5a1
tiran added a commit to tiran/cpython that referenced this issue Sep 11, 2022
@tiran
pythongh-95853: WASM: better version and asset handling in scripts (p…
d72298d 
…ythonGH-96045)

- support EMSDK tot-upstream and git releases
- allow WASM assents for wasm64-emscripten and WASI. This makes single file distributions on WASI easier.
- decouple WASM assets from browser builds
tiran added a commit to tiran/cpython that referenced this issue Sep 11, 2022
@tiran
pythongh-95853: Improve WASM build script (pythonGH-96389)
ccc643c 
- pre-build Emscripten ports and system libraries
- check for broken EMSDK versions
- use EMSDK's node for wasm32-emscripten
- warn when PKG_CONFIG_PATH is set
- add support level information
tiran added a commit to tiran/cpython that referenced this issue Sep 11, 2022
@tiran
pythongh-95853: Multiple ops and debug for wasm_build.py (python#96744)
5939c34
@bedevere-bot bedevere-bot mentioned this issue Sep 11, 2022
Merged
tiran added a commit to tiran/cpython that referenced this issue Sep 12, 2022
@tiran
Merge branch '3.11' into backport-pythongh-95853-3.11
2ff3ed2
tiran added a commit that referenced this issue Sep 13, 2022
@tiran
[3.11] gh-95853: Add script to automate WASM build (GH-95828, GH-95985,
4958820 
GH-96045, GH-96389, GH-96744) (GH-96749)

Automate WASM build with a new Python script. The script provides
several build profiles with configure flags for Emscripten flavors
and WASI. The script can detect and use Emscripten SDK and WASI SDK from
default locations or env vars.

``configure`` now detects Node arguments and creates HOSTRUNNER
arguments for Node 16. It also sets correct arguments for
``wasm64-emscripten``.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
OS-wasi type-feature A feature request or enhancement
Projects
No open projects
WASI to tier 2
Status: Done
Milestone
No milestone
Development

No branches or pull requests
2 participants
@brettcannon @tiran