The Cathedral and the Bazaar
I like to work in the open and when I say open, I don’t mean people shoulder surfing me watching what I do and judging me, but open as in being honest, welcoming, expressing desires, sharing success, sharing failures, and keeping people abreast of things they may be interested in and chasing my interests. I try and do this for my job, my life, my friends, and my family. Working in the open helps me feel confident that feedback is accepting, intentions are honest, and goodness is rewarded and that I care!
Today, was not one of those days where my goals lined up with reality and in stepping back from making a mountain out of a mole hill, I’m reminded of a book appropriately titled “The Cathedral and the Bazaar” and how I work with open source projects and communities that support open source concepts.
PREFACE – SPEAKING TO THE POINT OF THE BAZAAR AS AN OSS MOVEMENT I AM NOT IN SUPPORT OF ESR OR HIS PERSONAL VIEWS. Thx for letting me clear that out of the way. THIS IS NOT IN SUPPORT OF ESR for those in the back.
From Wikipedia – https://en.wikipedia.org/wiki/TheCathedralandtheBazaar
The Cathedral and the Bazaar
The essay contrasts two different free software development models:
- The Cathedral model, in which source code is available with each software release, but code developed between releases is restricted to an exclusive group of software developers. GNU Emacs and GCC were presented as examples.
- The Bazaar model, in which the code is developed over the Internet in view of the public. Raymond credits Linus Torvalds, leader of the Linux kernel project, as the inventor of this process. Raymond also provides anecdotal accounts of his own implementation of this model for the Fetchmail project.
But what are the important learnings from the model of the Bazaar? As Wikipedia mentions, I believe very much that they are the following lessons for creating good open-source software.
Now, let’s dive into the juicy nineteen lessons for creating good practice.
Lessons for creating good open source software (or communities of practice)
Raymond points to 19 “lessons” learned from various software development efforts, each describing attributes associated with good practice in open source software development:(Wikipedia [3])
Every good work of software starts by scratching a developer's personal itch.
We love mastodon, we love that Eugen scratched this itch – so many people are inspired by it and enjoy consuming it and want to see it grow in new and interesting ways.
Good programmers know what to write. Great ones know what to rewrite (and reuse).
Open Protocols, Open Source, embracing frameworks/tools/services that are open source. I can see other developers are working on go and rust versions of Mastodon compatible services. This is amazing that it grows in so many directions.
Plan to throw one [version] away; you will, anyhow (copied from Frederick Brooks's [The Mythical Man-Month](https://en.wikipedia.org/wiki/TheMythicalMan-Month “The Mythical Man-Month”)).
Part of growing up and learning. Changing your future does not mean failure. Technology is enabling new ways we could have never conceived of when the project was launched and years from now, that will be even more true.
If you have the right attitude, interesting problems will find you.
This is painfully apparent. Who knew that an OSS (open source software) *software* project would be something that would take on Billionaire’s services in addition to proprietary systems?
When you lose interest in a program, your last duty to it is to hand it off to a competent successor.
Not so much a problem here per se, but if people find new ways to use the software to solve modern problems, our duty isn’t to get in their way either because we did things differently.
Treating your users as co-developers is your least-hassle route to rapid code improvement and effective debugging.
We have forks, we have forks of forks, we have opinions, we have challenges – everyone is trying to build their communities and chase their dreams. Embrace them if they’re not acting in bad faith.
Release early. Release often. And listen to your customers.
Smaller & more frequently smaller releases are wise. It means less change per release, and it means more practice as we release so admins don’t become stale in their practice.
For example, in the release from 4.0 to 4.1 – which was mostly a lot of patches vs user features a lot of admins had to re-learn and re-experience things that would have been more practiced had we seen more 4.0.4 or 4.0.5 for smaller bug fixes.
Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix obvious to someone.
Talent everywhere – explore & embrace when people offer it up!
Smart data structures and dumb code works a lot better than the other way around.
See #7 ;)
If you treat your beta-testers as if they're your most valuable resource, they will respond by becoming your most valuable resource.
We 💖 mastodon!
The next best thing to having good ideas is recognizing good ideas from your users. Sometimes the latter is better.
This is where I hit the wall and where there is a disconnect between admins and the project and the project folks between admins at times.
There is a huge and rightful concern for how things are and have been and how the future should be – but not necessarily based on our own actions or our own experiments and search is one of these.
As someone who believes in open source, who wants to be a great steward of open source who also works for the world’s largest open-source company, I thought it was pretty clear that me working out in the open to deliver functionality to my users and doing so with honesty, integrity and passion and according to the Bazaar that I would have fit right in.
That isn’t necessarily true… and its demoralizing when not true.
Often, the most striking and innovative solutions come from realizing that your concept of the problem was wrong.
Search… It’s how humans deal with massive amount of information.
It’s also how people can be abusive.
When studying joint cognitive systems, the relationship of “Artefact” (computer) and human is where you research on inputs and outputs of your system(s) are having the desire defects.
We do a terrible job on this. (More on it later.)
Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away. (Attributed to Antoine de Saint-Exupéry)
See #7
Any tool should be useful in the expected way, but a truly great tool lends itself to uses you never expected.
It me. It pains me that this is a punishable offense on Mastodon.
When writing gateway software of any kind, take pains to disturb the data stream as little as possible—and never throw away information unless the recipient forces you to!
This one is where we need to “crap or get off the pot” for lack of better phrase. Small instances must abuse this stream as much as possible to be small and throw away data to avoid costs.
Larger instances can preserve things, make data streams available as they are and not throw away information unless the recipient or sender asks to.
Do we continue to have problems with mobile phones to appease small instances who can’t handle large datasets, or do we agree that completeness, robustness, and feature richness are a desirable goal along with safety and privacy?
When your language is nowhere near Turing-complete, syntactic sugar can be your friend.
Ruby is doing great powering our platforms.
A security system is only as secure as its secret. Beware of pseudo-secrets.
I’m happy here with mastodon as far as security is implemented from a web service standpoint but feel we’re severely hand-wavy in how users feel safe within our ecosystem.
Beware of security through obscurity.
To solve an interesting problem, start by finding a problem that is interesting to you.
For me, discoverability.
Provided the development coordinator has a communications medium at least as good as the Internet, and knows how to lead without coercion, many heads are inevitably better than one.
Hello fediverse!
The challenge
Let’s choose an easy target – Quote boosts/Quote Toots – whatever we want to call it. People want it, people didn’t want it. Lines were drawn and sides were chosen, and you know what. We did NONE of the 1-19 above to explore it.
Well… some people did.
There are mastodon instances (forks) with Quote Posts.
There are fediverse instances (non mastodon fediverse tools) with Quote Posts.
There have been PRs submitted for review for Quote Posts.
Instances that have done this on their own – They connect freely and openly with our existing Mastodon instances and so far, – the sky hasn’t fallen.
This is data we have – that we don’t act on. Because we believe our motives to be true because we don’t want to be like some other site that rhymes with bitter.
Why is this?
The bazaar worked for some, but not for others and in the end, everyone kind of suffered unless you knew.
And we perpetuate the QT thing to this day as something that has to wait for purity. Something that has to be perfect. Something that has to be official. Something that if you do on your own – you’re assuming the risk of being seen as an aggressor. Danger.
The Danger
Take my quote from earlier:
It me. It pains me that this is a punishable offense on Mastodon.
In the Bazaar, we wouldn’t deny QT. It may not live in our official branch for reasons, but we wouldn’t judge others for QT. The people who use the software would be judged for their use of it.
Users with good intentions are accepted and do so responsibly.
Users with bad intentions reject everything we want to normalize and pose a challenge.
We’d realize well moderated instance probably have QT and probably still have a great community. If we don’t realize this, then we can explore how to improve it. Do we need more rules? Do we need more moderators? What are our inputs and ouputs and how do we measure success? It is extremely hard in a world where our competitors measure everything, and we measure nothing.
Search
For me, search was more important than QT. Search is how I discover things. Search is how I find things. Search is how I can connect the “Artifacts” created by humans with the “desires of other humans”. There are 50 million posts and if we’re successful that 50 million will be an HOUR. You can’t make sense of this scale of information without searching.
So, I implemented a search engine.
I had the itch
Reused code that was already developed and used open source projects as the foundation
Threw out the old search. I’m sorry, it’s junk.
I saw other people were solving these problems. I created tickets. I chased down people. I surveyed the community. I asked questions. I told people I was doing this and that this was a problem that needed to be solved.
N/A
Asked community for feedback, shared code, asked world for feedback (the announcement is on our blog here)
Users submitted feedback – we’re adding more operators.
Other people have chimed in on code to seek more nuances/features – this puts us above and beyond what search does elsewhere. COOL!
It’s all about our data. What is a public Toot I mean Post if it weren’t meant to be discovered with good intentions?
We love you #universeodon
Users asked. Users voted. We’ve had people leave because search didn’t do what they wanted. We had people say it was empty because search didn’t show them 50 million posts. We had users wonder where everyone is because people can’t be found. We have solutions to all these problems right under our nose.
It was convenient we didn’t have search when we were smaller because dang it, it’s cheaper/easier to run mastodon without search. It’s easier to moderate, setup and manage. I totally get it. There is some necessary expertise here, but I see my role as providing the expertise and delivering the value – yet somehow, this has been demonized as providing weapons to the enemy.
You have to start somewhere. Waiting until perfection means we’re losing against those who did something… and let me tell you, nothing is perfect… ever…
Think of the possibilities of extending what mastodon means when mobile clients and advanced UI’s can build upon search operators. I imagine lists that are based on search operators such as creating a list with “domain:mastodon.art has: media” so i have a list of images shared from mastodon.art
Should I not have search because it’s too hard for small instances to do? Is this what really divides the community? Is this what drove some pushback? Could I not offer my search as an API for discovery to smaller instances or is that too much centralization? Could I not build a search service small instances could join/participate in as a Co-op? The fact everyone jumps into demonizing solutions vs asking probable and possible future outcomes is extremely disappointing
Search operators and search functionality could get infinitely more valuable here with more operators/functions/stemming/languages/parsers/validators and safety mechanisms as well as permission boundaries.
Right now, I don’t search anything that isn’t public. What is the safety people want?
Discoverability
I didn’t feel I was leading by coercion, but i get the impression this is how people responded to search.
The solution
I have no idea.
In the Bazaar, I feel no one should have attacked me for doing this. But here we are.
I say things like ‘good intentions` but I also know abusers abuse what `good intentions` are framed like and i didn’t like “Assume best intentions” when I heard it either (some people are just abusers/abusive – i get it) – so the argument is lost, and it turns into more arguments over solutions. I’m tired of fighting every side of the argument, so here I’m hedging on the Bazaar.
but
I’d like to say – why aren’t we talking Cybernetics? Why aren’t we using Sense-Making? Why aren’t we writing things doing and describing how we want our Joint Cognitive Systems to be and how we can improve our JCS to serve humans better?
There are theories of work, theories of sense making, theories of experimentation and theories of learning that people don’t seem to accept and flat out deny when they deny the Bazaar.
I don’t want what I think of search today as being the only extent as to what I can do with search. I want to use the Bazaar to make it better.
I want to offer a pivot page that shows what’s trending
I want to offer people the chance to follow sports, see what scores are, find news and be better aware of everything that is shared – for people to use. More relevant/more trending/more related. These things are trivial to improve upon when we have search, do smaller releases and more frequent releases.
Discovery shouldn’t be friction by design to achieve security through obscurity.
It’s not a fight
That’s all I ask. I’m not here to fight. I’m not here to pick sides. I’m not here to devalue opinions. I’m not here to say I’m right, you’re wrong. I will be wrong. I will make mistakes. I will continuously strive to learn and improve because that Is what I do.
Without the bazaar, I can’t do that.
And I find that problematic. Are we developing the fediverse from the pulpit in the Cathedral or are we bringing out the best of the Bazaar?