What Open Source shares with science

Posted by conz, community.zdnet.co.uk

I had a lengthy discussion with a new acquaintance the other day, and as it so often does in conversations I'm involved in, topics eventually shifted to the philosophy of knowledge and ideas, and their impact on the improvements within human societies over time.

My conversation partner knew something of these topics, but didn't know much about the computer industry. She was quite surprised when I introduced her to that concept of open source software, and how it was merely an extension of, if not the Scientific Method per-se, then at least the underpinning drive and methodology of Science. This in turn got me to digging up an essay from many years back, which I thought may be of interest to this forum's members.

One of the overlooked advantages that Open Source development affords, is that it imitates perhaps the most fruitful and beneficial of all human endeavours: Science. How has the scientific-method evolved, and what can it teach us about the future possibilities of software construction?

Science, in its clearly understood modern guise, is unique. This essentially Western tradition of open inquiry is believed to have developed only one instantiation throughout the whole period of human history. While almost all human societies have developed language, art, and music, open inquiry into the natural and philosophical world sprung only from the eastern rim of the Mediterranean sea, in a number of ancient Greek states, approximately 27 centuries ago.

Helped along by the advantages provided by the recently formulated Greek alphabet, the people of this region bought forth the makings of the primary conceptual and philosophical machinery that was necessary to develop an understanding of Nature that surrounded them. In short order, they had conceived ideas which led them to believe that the Universe was understandable, that it was measurable and that it could undergo rational analysis.

Through the advantages bestowed upon the Greeks by virtue of their written language and undoubtedly their open, democratic political environment, these concepts spread. Where once these people would have been manipulators purely of the physical world (pottery, sculpture) they now also became masters if symbolic manipulation. Mathematics, logic, geometry, geography, mechanics, hydraulics, medicine, architecture, astronomy and cosmology, optics and dozens of other disciplines flowered. There has never been a similar period in human history, with the possible exception of the 18th century 'age of reason', the Enlightenment, interceded by the Dark Ages following the fall of Rome and the burning of the great library in Alexandria.

Luckily for all of us, these same ideas were storehoused and enhanced by the Islamic world. Eventually, after the fall of Muslim-held Toledo in the 11th century, these same ideas resurfaced slowly but steadily into Western and Southern Europe, to precipitate the Renaissance; the re-birth (of civilisation.) The arrival of these Greek texts coincided with the development of the university as a legal entity with political and intellectual autonomy. Once again, the openness to new forms of thought, the cheap, efficient and accurate transmission of ideas through the wonderful machinery of technology (Gutenberg's printing press,) brought forth an explosion of creativity and propelled Western civilisation forward. Open Source software is a direct descendant of this culture of open thought, as it prizes the same properties and philosophy which form the basis of the driving force of Science.

Openness is a key area of the scientific process which holds our interest here for comparison with Open Source development. Science is the process of verifying or culling hypothesis, and is in essence an open and self correcting system. Because of this, progress occurs at a much faster rate and in a more dependable/trusted fashion. This doesn't mean that the self correction happens in minute, continuously flowing 'chunks'. In reality, corrections arise as mini-revolutions, characterised by philosopher of science, Thomas Kuhn, as paradigm shifts. Nonetheless, over longer periods of time, progress does occur. In many ways, this progress is accidental, as there is often no 'vision' or nomenclature to describe where Science is heading, until after it has arrived.

The speed of progress is greatly enhanced by virtue of the fact the practitioners of Science publish not only results, but methodology, and techniques. In programmatic terms, this is equivalent to both the binary and the source code. This not only helps 'bootstrap' others into the field, to learn from the examples set, but makes it possible for others to verify or refute the results (or techniques) under investigation. In an almost guided-Darwinian evolutionary fashion, this makes the scientific process a powerful tool for the highlighting, analysis and possible culling of ideas and concepts; less useful ideas and hypothesis die, and likely contenders come sharply into focus. Newton made his famous comment about 'standing on the shoulders of giants', in part, to indicate that his contributions to human knowledge could not have been achieved solely. He needed the 'firmament' beneath him hypothesised, tested and confirmed by generations of scientists, philosophers and thinkers before him, over centuries. With Science, in the medium to long run, all other issues ('marketing' and previous acceptance) fall by the wayside, and merit alone is the main attribute of the victorious memes.

By analogy, making the source code available for peer review and extension, is perhaps Open Source's most powerful advantage as a development methodology. Besides the verification provided by peer review, refutation for issues such as security is possible, learning of techniques by new practitioners is a great advantage, as is modification and redistribution of the code under similar conditions. It makes it possible for anyone who has a background in, or can acclimatise to the technology and skills required, to continue development, extending the code into whatever direction that they need. This, in turn can be plowed back into the original system, causing eddies of strengthening feedback. This results in an ever growing base (or firmament) of quality code, upon which more and more programmers can benefit from, and contribute back to. The open communication of source and ideas, while progressing slowly at first, builds momentum as more and more practitioners learn from, extend or revive from obscurity, more and more code.

Another facet of similarity between Open Source and Science, is the respective cultures. Both are strongly technical, perhaps verging on the geeky, and both are meritocracies. A large part of the impetus of scientists is the applause of others. Peer recognition is also perhaps the single most important reason attributed by Open Source advocates and developers as to the reason why they pursue Open Source methods of software construction. Finally, much like the contemporaneous invention of the printing press helping propel Science, the Internet has provided both the human inter-communication and resultant software distribution vehicle, to help catalyse the Open Source process.

This chain of attributes and contingencies which made Science perhaps the most effective and pervasive of successful human endeavours, is finding resonant similarities in the burgeoning success of the Open Source process and may help explain its rise and rise.