Use javascript emulator in blog posts

[robert-w-gries]

Edit

I have decided that my original post is out of scope of this repository. The goal is now to use a javascript emulator in the blog posts to allow readers to compile the code blocks within the blog and run the blog_os kernel within an element in the web page.

Here are the steps that would need to be taken:

1. Use jslinux or, more likely, v86 to run our blog_os kernel
2. Submit a PR to replace all code blocks in blog posts with blog_os.js

---

Original Post

Rust has the ability to compile to WebAssembly and all relevant browsers now ship support for the WebAssembly standard

This WebAssembly target could have several use cases for blog_os

Run the kernel in a browser

Worth it for the cool factor alone :-)

Allow the compilation and running of the kernel at different stages of your blog posts

After you explain a complex topic, such as paging, you generally provide code as a reference. It'd be nice to be able to compile that code within the blog post's webpage and run the full kernel. The reader could even fiddle with the code, re-run it, and see how their modifications affect the kernel.

The Rust by Example website has the functionality I'm talking about.

Experiment with the concept of kernels hosted on browsers.

I know that javascript PC emulators exist, but WebAssembly seems to have more potential in terms of performance and codability.

Extremely portable emulation

Everyone has a browser and everyone could run the kernel regardless of which OS they're using.

Random Notes

• There's a WIP wasm rustc target that looks relevant: std: Add a new wasm32-unknown-unknown target rust-lang/rust#45905
• Here's a blog post from Jan 2017 that details how to build a webasm project
• Here's a recent, interesting forum topic from last month that discusses the state of webasm in Rust

[robert-w-gries]

I haven't followed WebAssembly for a year, so there might be some aspects I'm missing. I will do more research into this concept and post updates on info I've found and progress I've made.

Let me know if you have interest in such a project.

[steveklabnik]

I personally am working towards integrating wasm into intermezzos, that is, as the userland format for programs.

[robert-w-gries]

I've put more thought into this, and I think there are two separate projects in my original idea:

• Project 1 - Use a javascript emulator in the blog posts to let readers compile code and run the blog_os kernel within the browser
• Project 2 - Make an x86_64 emulator that compiles to wasm

Project 2 seems massively out of scope of this repository, so I'll shelve that for now :-)

Project 1 could be very useful for blos_os. Here are the steps that need to be taken:

1. Hack up jslinux to be able to run our blog_os kernel
2. Find a way to insert js-blos_os into blog post
3. Submit a PR to add js-blog_os to the published blog posts

[robert-w-gries]

@steveklabnik If you don't mind me picking your brain, what would using wasm as the userland format for apps accomplish? It sounds like an interesting idea

[4e554c4c]

Seems like little gain for much complication to me. Much of this blog is about getting an OS started on x86_64, and therefore mostly deals with x86_64 initialization and structures. A wasm target would have to be made completely different.
I can see the benefit of being able to see what each step in the OS should look like. I'd be a proponent of replacing the screenshots of QEMU with copyable text, but modifying the actual code of the operating system so that it can compile to wasm seems completely out of scope.
Then again, I'm not Phil so it's not my decision.

[steveklabnik]

@robert-w-gries i will be writing something up about it someday, let's not clutter up phil's tracker with my silly ideas :)

[robert-w-gries]

@4e554c4c You're right, the original suggestion was out of scope.

I have modified the original post to include only the suggestion of an external x86 javascript emulator that would run the code blocks within the blog posts.

[Evrey]

Project 1 - Use a javascript emulator in the blog posts to let readers compile code and run the blog_os kernel within the browser

... why? Running a full x86 emulator in a browser tab seems scary. Like... freezing browser tabs scary. One might as well just assume a reader having QEMU installed, and then...

• ... provide a readily compiled blog_os binary and instructions on how to run it in QEMU.
• ... provide a Git clone URL pointing to a release which represents the state of the code at the point of that blog post, together with build instructions. Just as Rustendo64 has releases per released video episode.
• ... maybe even provide readily compiled cross tools for beginners.

This also avoids possible (security) problems of having a build server compiling whatever the reader might have typed into the browser. Unless... you know... you want to also include a complete cross toolchain into the WAsm or JS stuff compiling things on the client side.

[andre-richter]

... maybe even provide readily compiled cross tools for beginners.

I think this is indeed the first and easiest step to get readers and author of the blog aligned. Ensure that the toolchains are in sync and accessible easily.

In #222, Docker has been suggested, which is the way to go nowadays imho. For example, Redox does that too: https://github.com/redox-os/redox/tree/master/docker
Besides the redox container, I've tinkered a lot with dockerizing rust-toolchains since then, so I'm happy to provide some input.

Switching to browser-based also sounds interesting, but really needs a sophisticated server infrastructure as pointed out by @Evrey.

[robert-w-gries]

The work required vs benefit gained seems less worth it as we discuss this, so I'll close the issue.

I still think there's value in running the kernel on a js emulator, so I will get it working and share my efforts.

[phil-opp]

Running the kernel directly in the browser seems like a great idea for trying out precompiled images. We could provide several ISOs for the different stages of the kernel, so that users can get the "feel" without needing to install anything. Also, it would be completely platform independent. Allowing the user to modify the kernel however, would require some server side recompilation, which is more difficult (but I think it would be great to have).

I still think there's value in running the kernel on a js emulator, so I will get it working and share my efforts.

Awesome, please keep me posted :).

[phil-opp]

@andre-richter

... maybe even provide readily compiled cross tools for beginners.

I think this is indeed the first and easiest step to get readers and author of the blog aligned. Ensure that the toolchains are in sync and accessible easily.

Agreed! I plan a second edition of the blog with a custom bootloader, written in inline assembly as a normal rust crate. This would replace grub-mkrescue, which causes the most cross-platform problems. I also plan to use the LLVM linker lld and hope that it is soon distributed with the Rust compiler. For the debug tools such as objdump or gdb, we could use travis/appveyor to create binaries for Windows, Mac, and Linux.

Docker might be an alternative, but requiring that all users install and learn the basics of docker does not seem like a good solution. However, providing a docker image for those who use docker already seems like a good idea. We could also add a short docker tutorial for those who are interested.

[phil-opp]

@Evrey

... provide a readily compiled blog_os binary and instructions on how to run it in QEMU.

This would allow users to try it out without installing all build dependencies and building it manually. The only hurdle is to get travis to produce an ISO, which is difficult due the dependency on grub-mkrescue. Maybe we could switch travis to a docker based build to fix the dependency issues.

... provide a Git clone URL pointing to a release which represents the state of the code at the point of that blog post, together with build instructions.

There are already post_N tags in the repo that contain the state of the kernel after each post. Maybe we should link them more prominently. At the moment, the "Github" link at the beginning of each post and the "(source)" links in the Readme point to these tags. They have an own Readme that lists the dependencies, but no detailed build instructions yet.

[andre-richter]

@phil-opp

You can do this with a very minimal learning overhead.
In fact, the rust docker containers I built so far are designed such that they act as an "interactive overlay" to local directory, just providing the toolchain. With this approach, apart from the single command to spawn them, there is no difference to using a native environment. Even the same Makefile can be used.

https://github.com/andre-richter/docker-rust-persistent
https://github.com/redox-os/redox/tree/master/docker

The one I did for redox is slightly newer. I think looking at the gifs in these repos could give you an idea about the approach.

[Evrey]

@phil-opp

I also plan to use the LLVM linker lld and hope that it is soon distributed with the Rust compiler.

This. Do it. It's great. I never understood why compilers and linkers don't just include all the available backends by default. Everyone can just pacman -S lld or whatever and use it out of the box. Cross compiling binutils on the other hand is a pain and takes forever.

Maybe we should link them more prominently.

Seems like it. I didn't know. =D

[phil-opp]

@andre-richter

You can do this with a very minimal learning overhead.

Maybe, but you have to install and set up a various tools, download relatively large external images, and issue very long command line commands such as:

docker run --cap-add MKNOD --cap-add SYS_ADMIN --device /dev/fuse \
    -e LOCAL_UID="$(id -u)" -e LOCAL_GID="$(id -g)" \
    -v redox-"$(id -u)-$(id -g)"-cargo:/usr/local/cargo \
    -v redox-"$(id -u)-$(id -g)"-rustup:/usr/local/rustup \
    -v "$(pwd):$(pwd)" -w "$(pwd)" --rm redoxos/redox make fetch all

If you already know docker, this is all fine. For new users, however, it makes things much more complicated. One important goal of the blog is that you can start without setting up an elaborate development environment, so that there are no unnecessary hurdles in the beginning.

I think the best solution is to add optional docker support, so that the users can decide if they want to use docker or install the tools normally. What do you think?

I created #369 as a more focused issue, so let's continue the discussion there.

[phil-opp]

@Evrey I opened #370 and #371 to track these issues.

[nebrelbug]

I can use my simple InterMezzOS kernel (with just the "hello world") here. I think it would be cool to be able to preview the 32-bit stuff there