The Cathedral and The Bazaar

This week in class we examined Eric Raymond’s essay “The Cathedral and The Bazaar.” It is filled with pithy aphorisms about open source, which we spent our discussion unpacking. Here are a few that stood out to me and my open source experience.

“Good programmers know what to write. Great ones know what to rewrite (and reuse).”

Recently, while working on Preswald, we were trying to implement a requested feature mentioned in an issue. The exact details aren’t terribly important, but on a basic level, we were trying to add a “datasource selection” menu to an existing AI chat component. In the existing implementation, the AI chat was locked to a single CSV or SQL table. We spend a lot of time modifying the code of the AI chat widget, fiddling with connections to the frontend and backend, trying to read and prepare extra data sources, all to come up with a mostly functional prototype of the feature that greatly increased the complexity of the code and the user experience when trying to add this widget to a dashboard.
While looking for a solution to something else, I found myself reading through the documentation, and I noticed a component called “selectbox” that created drop down menus in the project. Recalling the other issue I was working on, I tried to use this existing component to implement the requested feature. With just a small amount of tweaking, it worked! This method greatly reduced the code complexity of the solution, and kept things much more clean and modular. This experience showed us that just writing a bunch of code isn’t always the solution— oftentimes problems can be solved much more easily and simply by understanding and reusing the existing tools available in a project.

“Plan to throw one away; you will, anyhow.”

This quote doesn’t just resonate with me in terms of my contributions to Preswald— I’ve been having this feeling since the moment I started in Computer Science. It feels like almost an inevitability, no matter what you’re working on, that your first solution will fail. Often, its only after seeing why your first solution fails that you truly understand the depth of the problem you’re trying to solve! I can think of many times I’ve been trying to solve a problem for an assignment, see that my test cases aren’t passing, and only then realize I missed a part of the problem description or misunderstood an assumption.
Now, when a solution does fail, a valuable skill is knowing how to modify it to make it work. An even more valuable skill, though, as this quote alludes to, is knowing when to completely throw it away. It can be hard to throw away a solution to a problem, especially when you invested a lot of time into it, but, sometimes, if it doesn’t work, it’s because your approach was fundamentally flawed. It’s important to recognize, though, that the time on your first solution wasn’t wasted— you learned about the problem and what won’t work, and your next solution will likely come faster than the first.

“Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix obvious to someone.”

We discussed in class that this quote can be dangerous. You shouldn’t always assume that someone else will find the solution to a challenging problem— that’s a good way to let bugs linger and never get solved. In Preswald, however, this quote certainly feels relevant. As I’ve discussed in previous blog posts, many, many times we have gotten deep into work on (what seems to us, at least) a challenging issue, just to get beaten to the PR by someone who seemingly came out of nowhere. This is, of course, frustrating, but looking on the bright side, it is cool to see the way bugs and new feature requests can be knocked out so quickly by a team of volunteer open source developers. It is also useful to see the ways they got past roadblocks in the issue that stopped us. Still annoying, though!!

Claude Warren’s Coffeehouse

In the lecture from Claude Warren we watched this week as homework, he discusses the idea of a “coffeehouse.” His idea takes inspiration from Edward Lloyd’s 1688 coffeehouse in London. At this coffeehouse, his customers were primarily merchants, and in order to reduce the individual risk of a sunk ship and a lost shipment, the patrons of Lloyd’s shop agreed to split the profits of their successful voyages and split the loss of a lost one. In a way, this was one of the first instances of “insurance.”
Warren argues that we (particularly large companies that rely on the technology) should take a coffeehouse-like approach to open source software. Like merchant ships, open source projects are hugely valuable, but invite catastrophic risk should they fail. To prevent this, a coffeehouse-like community should be formed to support open source, in which companies pool their resources to support the open source projects that are the foundation of their infrastructure.
One such way this can be done is through an “OSPO,” an open source program office. This is an office within an organization that studies and supports the open source projects that your product relies on. Critically, you don’t show the workers in your OSPO your proprietary code, so they can’t accidentally leak that information to the open source project. The OSPO allows your company to have a direct link to experts in the open source project you use, providing tech support, bug fixes, or even development of new features.

OSPOs in Action

Many companies have taken Warren’s advice and created OSPOs of their own, including Microsoft, Block, and many more. These offices do exactly what Warren says: they research and support the thousands, if not hundreds of thousands, of upstream open source projects these companies rely on. They provide technical support to engineers at the company, contribute to the open source projects they use, facilitate communication between open source maintainers and engineers at the company, and even provide legal insight into the (very complex!) world of open source licensing. Of course, all the information I could find on these offices was in pretty self-congratulatory articles written by the same company the OSPO is in, so I do wonder just how effective these OSPOs actually are at supporting the open source community, or if they’re instead more performatory and self-serving. Either way, I am interested to learn more about this branch of software development that is new to me.