Browsers and the Web

Twitter · Blog · Patreon · Discussions

Why study web browsers? The way I see it, browsers are fundamental to the web, to modern computing, and even to the economy broadly—so it’s worth knowing how they work. And in fact, cool algorithms, tricky data structures, and fundamental concepts come alive inside a web browser. This book walks you through building a browser, from scratch. I hope, as you read it, that you fall in love with web browsers, just like I did.

The browser and me

I—this is Chris speaking—have known the webBroadly defined, the web is the interlinked network (“web”) of web pages on the internet. If you’ve never made a web page, I recommend MDN’s Learn Web Development series, especially the Getting Started guide. This book will be easier to read if you’re familiar with the core technologies. for all of my adult life. Since I first encountered the web and its predecessors,For me, BBS systems over a dial-up modem connection. A BBS is not all that different from a browser if you think of it as a window into dynamic content created somewhere else on the internet. in the early 90s, I’ve been fascinated by browsers and the concept of networked user interfaces. When I surfed the web, even in its earliest form, I felt I was seeing the future of computing. In some ways, the web and I grew together—for example, in 1994, the year the web went commercial, was the same year I started college; while there I spent a fair amount of time surfing it, and by the time I graduated in 1999, the browser had fueled the famous dot-com speculation gold rush. The company for which I now work, Google, is a child of the web and was founded during that time. The web for me is something of a technological companion, and I’ve never been far from it in my studies or work.

In my freshman year at college, I attended a presentation by a RedHat salesman. The presentation was of course aimed at selling RedHat Linux, probably calling it the “operating system of the future” and speculating about the “year of the Linux desktop”. But when asked about challenges RedHat faced, the salesman mentioned not Linux but the web: he said that someone “needs to make a good browser for Linux.”Netscape Navigator was available for Linux at that time, but it wasn’t viewed as especially fast or featureful compared to its implementation on other operating systems. Even back then, in the very first year or so of the web, the browser was already a necessary component of every computer. He even threw out a challenge: “how hard could it be to build a better browser?” Indeed, how hard could it be? What makes it so hard? That question stuck with me for a long time.Meanwhile, the “better Linux browser than Netscape” took a long time to appear….

How hard indeed! After seven years in the trenches working on Chrome, I now know the answer to his question: building a browser is both easy and incredibly hard, both intentional and accidental. And everywhere you look, you see the evolution and history of the web wrapped up in one codebase. But most of all, it’s fun and endlessly interesting.

So that’s how I fell in love with web browsers. Now let me tell you why you will, too.

The web in history

The web is a grand, crazy experiment. It’s natural, nowadays, to watch videos, read news, and connect with friends on the web. That can make the web seem simple and obvious, finished, already built. But the web is neither simple nor obvious. It is the result of experiments and research reaching back to nearly the beginning of computingAnd the web also needed rich computer displays, powerful UI-building libraries, fast networks, and sufficient CPU power and information storage capacity. As so often happens with technology, the web had many similar predecessors, but only took its modern form once all the pieces came together. about how to help people connect and learn from each other.

In the early days, the internet was a world wide network of computers, largely at universities, labs, and major corporations, linked by physical cables and communicating over application-specific protocols. The early web built on this foundation. Web pages were files in a specific format stored on specific computers, and web browsers used a custom protocol to request them. URLs for web pages named the computer and the file, and early servers did little besides read files from a disk. The logical structure of the web mirrored its physical structure.

A lot has changed. HTML is now usually dynamically assembled on the fly“Server-side rendering” is the process of assembling HTML on the server when loading a web page. Server-side rendering often uses web tech like JavaScript, and even a headless browser. Yet one more place browsers are taking over! and sent on demand to your browser. The pieces being assembled are themselves filled with dynamic content—news, inbox contents, and advertisements adjusted to your particular tastes. Even URLs no longer identify a specific computer—content distribution networks route a URL to any of thousands of computers all around the world. At a higher level, most web pages are served not from someone’s home computerPeople actually did this! And when their website became popular, it often ran out of bandwidth or computing power and became inaccessible. but from a social media platform or cloud computing service.

With all that’s changed, some things have stayed the same, the core building blocks that are the essence of the web:

As a philosophical matter, perhaps one or another of these principles is secondary. One could try to distinguish between the networking and rendering aspects of the web. One could abstract linking and networking from the particular choice of protocol and data format. One could ask whether the browser is necessary in theory, or argue that HTTP, URLs, and hyperlinking are the only truly essential parts of the web.

Perhaps.It is indeed true that one or more of the implementation choices could be replaced, and perhaps that will happen over time. For example, JavaScript might eventually be replaced by another language or technology, HTTP by some other protocol, or HTML by its successor. Certainly all of these technologies have been through many versions, but the web has stayed the web. The web is, after all, an experiment; the core technologies evolve and grow. But the web is not an accident; its original design reflects truths not just about computing, but about how human beings can connect and interact. The web not only survived but thrived during the virtualization of hosting and content, specifically due to the elegance and effectiveness of this original design.

The key thing to understand is this grand experiment is not over. The essence of the web will stay, but by studying web browsers you have the chance to contribute and to shape its future.

Real browser codebases

So let me tell you what it’s like to contribute. Some time during my first few months of working on Chrome, I came across the code implementing the <br> tag—look at that, the good-old <br> tag, which I’ve used many times to insert newlines into web pages! And the implementation turns out to be barely any code at all, both in Chrome and in this book’s simple browser.

But Chrome as a whole—its features, speed, security, reliability—wow. Thousands of person-years went into it. There is a constant pressure to do more—to add more features, to improve performance, to keep up with the “web ecosystem”—for the thousands of businesses, millions of developers,I usually prefer “engineer”—hence the title of this book—but “developer” or “web developer” is much more common on the web. One important reason is that anyone can build a web page—not just trained software engineers and computer scientists. “Web developer” also is more inclusive of additional, critical roles like designers, authors, editors, and photographers. A web developer is anyone who makes web pages, regardless of how. and billions of users on the web.

Working on such a codebase can feel daunting. I often find lines of code last touched 15 years ago by someone I’ve never met; or even now discover files and code that I never knew existed; or see lines of code that don’t look necessary, yet seem to be important. How do I understand that 15-year-old code? Or learn the purpose of these new files? Can I delete those lines of code, or are they there for a reason?

What’s amazing is that, despite the scale and the pace and the complexity, there is still room to contribute. Every browser has thousands of unfixed bugs, from the smallest of mistakes to myriad mix ups and mismatches. Every browser must be endlessly tuned and optimized to squeeze out that last bit of performance. Every browser requires painstaking work to continuously refactor the code to reduce its complexity, often through the carefulBrowsers are so performance-sensitive that, in many places, merely the introduction of an abstraction—the function call or branching overhead—has an unacceptable performance cost! introduction of modularization and abstraction.

What makes a browser different from most massive code bases is their urgency. Browsers are nearly as old as any “legacy” codebase, but are not legacy, not abandoned or half-deprecated, not slated for replacement. On the contrary, they are vital to the world’s economy. Browser engineers must therefore fix and improve rather than abandon and replace. And since the character of the web itself is highly decentralized, the use cases met by browsers are to a significant extent not determined by the companies “owning” or “controlling” a particular browser. Other people—you—can contribute ideas and proposals and implementations.

What’s amazing is that, despite the scale and the pace and the complexity, there is still plenty of room to contribute. Every browser today is open-source, which opens up its implementation to the whole community of web developers. Browsers evolve like giant R&D projects, where new ideas are constantly being proposed and tested out. As you would expect, some features fail and some succeed. The ones that succeed end up in specifications and are implemented by other browsers. That means that every web browser is open to contributions—whether fixing bugs or proposing new features or implementing promising optimizations.

And it’s worth contributing, because working on web browsers is a lot of fun.

Browser code concepts

HTML, CSS, HTTP, hyperlinks, and JavaScript—the core of the web—are approachable enough, and if you’ve made a web page before you’ve seen that programming ability is not required. That’s because HTML & CSS are meant to be black boxes—declarative APIs—where one specifies what outcome to achieve, and the browser itself is responsible for figuring out the how to achieve it. Web developers don’t, and mostly can’t, draw their web page’s pixels on their own.

As a black box, the browser is either magical or frustrating (depending on whether it is working correctly or not!). But that also makes a browser a pretty unusual piece of software, with unique challenges, interesting algorithms, and clever optimizations. Browsers are worth studying for the pure pleasure of it.

There are practical reasons for the unusual design of a browser. Yes, developers lose some control and agency—when pixels are wrong, developers cannot fix them directly.Loss of control is not necessarily specific to the web—much of computing these days relies on mountains of other peoples’ code. But they gain the ability to deploy content on the web without worrying about the details, to make that content instantly available on almost every computing device in existence, and to keep it accessible in the future, mostly avoiding the inevitable obsolescence of most software.

What makes that all work is the web browser’s implementations of inversion of control, constraint programming, and declarative programming. The web inverts control, with an intermediary—the browser—handling most of the rendering, and the web developer specifying parameters and content to this intermediary.For example, in HTML there are many built-in form control elements that take care of the various ways the user of a web page can provide input. The developer need only specify parameters such as button names, sizing, and look-and-feel, or JavaScript extension points to handle form submission to the server. The rest of the implementation is taken care of by the browser. Further, these parameters usually take the form of constraints over relative sizes and positions instead of specifying their values directly;Constraint programming is clearest during web page layout, where font and window sizes, desired positions and sizes, and the relative arrangement of widgets is rarely specified directly. A fun question to consider: what does the browser “optimize for” when computing a layout? the browser solves the constraints to find those values. The same idea applies for actions: web pages mostly require that actions take place without specifying when they do. This declarative style means that from the point of view of a developer, changes “apply immediately,” but under the hood, the browser can be lazy and delay applying the changes until they become externally visible, either due to subsequent API calls or because the page has to be displayed to the user.For example, when exactly does the browser compute which CSS styles apply to which HTML elements, after a web page changes those styles? The change is visible to all subsequent API calls, so in that sense it applies “immediately.” But it is better for the browser to delay style re-calculation, avoiding redundant work if styles change twice in quick succession. Maximally exploiting the opportunities afforded by declarative programming makes real-world browsers very complex.

To me, browsers are where algorithms come to life. A browser contains a rendering engine more complex and powerful than any computer game; a full networking stack; clever data structures and parallel programming techniques; a virtual machine, an interpreted language, and a JIT; a world-class security sandbox; and a uniquely dynamic system for storing data.

And the truth is—you use the browser all the time, maybe for reading this book! That makes the algorithms more approachable in a browser than almost anywhere else: the web is already familiar. After all, it’s at the center of modern computing.

The role of the browser

Every year the web expands its reach to more and more of what we do with computers. It now goes far beyond its original use for document-based information sharing: many people now spend their entire day in a browser, not using a single other application! Moreover, desktop applications are now often built and delivered as web apps: web pages loaded by a browser but used like installed applications.Related to the notion of a web app is a Progressive Web App, which is a web app that becomes indistinguishable from a native app through progressive enhancement. Even on mobile devices, apps often embed a browser to render parts of the application UI.The fraction of such “hybrid” apps that are shown via a “web view” is likely increasing over time. In some markets like China, “super-apps” act like a mobile web browser for web-view-based games and widgets. Perhaps in the future both desktop and mobile devices will largely be a container for web apps. Already, browsers are a critical and indispensable part of computing.

So given this centrality, it’s worth knowing how the web works. And in fact, the web is built on simple concepts: open, decentralized, and safe computing; a declarative document model for describing UIs; hyperlinks; and the User Agent model.The User Agent concept views a computer, or software within the computer, as a trusted assistant and advocate of the human user. It’s the browser that makes these concepts real. The browser is the User Agent, but also the mediator of the web’s interactions and the enforcer of its rules. The browser is the implementer of the web: Its sandbox keeps web browsing safe; its algorithms implement the declarative document model; its UI navigates links. Web pages load fast and react smoothly only when the browser is hyper-efficient.

Such lofty goals! How does the browser deliver on them? It’s worth knowing. And the best way to understand that question is to build a web browser.

Browsers and you

This book explains how to build a simple browser, one that can—despite its simplicity—display interesting-looking web pages and support many interesting behaviors.You might relate this to the history of the web and the idea of progressive enhancement. As you’ll see, it’s surprisingly easy, and it demonstrates all the core concepts you need to understand a real-world browser. You’ll see what is easy and what is hard; which algorithms are simple, and which are tricky; what makes a browser fast, and what makes it slow.

The intention is for you to build your own browser as you work through the early chapters. Once it is up and running, there are endless opportunities to improve performance or add features. Many of these exercises are features implemented in real browsers, and I encourage you to try them—adding features is one of the best parts of browser development!

The book then moves on to details and advanced features that flesh out the architecture of a real browser’s rendering engine, based on my experiences with Chrome. After finishing the book, you should be able to dig into the source code of Chromium, Gecko, or WebKit, and understand it without too much trouble.

I hope the book lets you appreciate a browser’s depth, complexity, and power. I hope the book passes along its beauty—its clever algorithms and data structures, its co-evolution with the culture and history of computing, its centrality in our world. But most of all, I hope the book lets you see in yourself someone building the browser of the future.