Running Webservers in your Browser

A simple proposal

A while back I had an idea - make it easier for data science web applications to offload their compute to individual client laptops. These applications generally used Python on the server side because of the existing libraries for data science. I knew that Web Assembly (wasm) was a pretty mature concept for running high performance binary code directly in the browser. I also knew that Python distributions like Pyodide could use it to run directly in the browser.

Given that it was well supported, I thought any heavy data computations that would typically be handed off to a Python worker in a SAAS product could now be handed back to the user's browser instead. The only downside seemed to be data transfer to the browser, but in many applications this happens to some degree anyways.

As a proof of concept, I started writing a simple Streamlit component that could do this. Streamlit is a library that runs a Python script provided by the user to generate a webpage. User interactions on the webpage automatically re-run the Python script on the server and update the webpage based on the results. This allows simple interfaces to be designed quickly for relatively complex Python functionality. Moreover, there is strong support for hosting and sharing these applications in the cloud.

Ultimately I saw a few long term design benefits to moving costly Python functions to run in the user's browser:

1. Reducing hosting costs since each user provides their own machine.
2. Reduce system complexity since the frontend and backend can use the same data processing code.
3. Keep more sensitive data on the user's machine rather than in the cloud.

I quickly got a simple example running that could pass a Python function to the front-end, run it there, and send serialized results back to the server. The initial results verified that my assumptions above were likely reasonable. However, as I got ready to share the prototype, I stumbled across an existing project called stlite.

Taking it too far

stlite had taken the idea a step further and ported the entire streamlit library to wasm. What blew my mind about this was that this meant stlite - a frontend library - was running a local web server that hosted its own web application.

On the surface, this worked shockingly well. The streamlit application ran directly in the browser and there was zero outside network traffic to run complex data science workloads. I tried a few streamlit programs and they all ran as expected. The developer even had a free IDE in the web available for writing and running streamlit programs.

As a systems developer though, the whole thing made me feel uneasy. I would go so far as to describe my feelings as an identity crisis - is this where SAAS applications are headed? Why shouldn't something complex like Youtube's video encoding pipeline simply run in the uploading browser instead of servers? I kept coming back to that this was not the "right" way to design a system like this. But it worked... so why wasn't it?

Meeting in the middle

Eventually I took a step back and think about how I would architect stlite from scratch. I figured this would shed some light on why the whole library worked so well but felt so poorly designed. I came up with two design requirements:

1. Allow the page to be hosted statically - without a server.
2. Allow data processing code to be written in Python.

At first I thought my unease came from the fact that Python was still making UI decisions instead of the web app itself. That meant the data processing code and UI code were deeply coupled, which would traditionally imply unnecessary complexity in traditional applications. However, any attempt to decouple these parts proved frustratingly difficult. I eventually realized that this coupling was actually a key usability feature since it allowed the UI to change dynamically based on the data processing results. Intuitively, this still felt sub-optimal because it meant the Python script would (in the general case) need to be run directly as-written, but it was also a blessing in disguise since it limited the scope of any reasonable redesign.

After a lot of thought, I concluded there was only one significant change to make - cut out the Python webserver.

A simple version might look like this:

1. Update the streamlit front-end to run a .py file initially on loading the page.
2. Update any signals from the python code to the front-end to use wasm signaling techniques instead of network requests.
3. Add some translation code so that updated variables in the web app are injected into the wasm .py execution directly.
4. Move caching and server state to be stored in the web app (or a dedicated storage system) instead of the web server.

Superficially the changes are only to the communication protocol between the Python code and the frontend code. However, if you dig one level deeper, there are significant architectural improvements over simply running Streamlit as-is in the browser.

1. There is no persistent web server running in your browser. This reduces resources wasted on a user's machine.
2. The system is much simpler to maintain - a few changes that completely cut out the web server.
3. The system should be much faster since none of the server dependencies or startup times are necessary.
4. The new system no longer has server state, and so can be easily extended to offload the Python compute to serverless functions in the future.

Reflecting on stlite

When I think about stlite after this exercise, I still view it as a very cool demonstration of what wasm can do. I don't think stlite should become the defacto way anyone uses streamlit, but it is incredibly creative. Especially considering that the project was done by a single developer, I'm incredibly impressed. Yes, there could be improvements, but that doesn't diminish how cool the project is.

That being said, I can't help but feel that the library reflects an unfortunate trend in system design. I've worked with many developers who are unwilling to tackle existing complexity to improve a system. Instead they treat the system as a black-box to be worked around rather than with. Startups re-invent the wheel all the time, not realizing that existing companies have decades of learnings on the exact problem the startup is tackling. During my early years as a software developer, I was always told to read orders of magnitude more code than I write. I feel like fewer and fewer developers see their roles this way.