To the fundamental question of how do we know what is true, there are three basic answers: authority, reason, and experience [1]. An epistemology based on authority states that truth is given to us by someone more knowledgeable than ourselves. The two primary variations of authority-based epistemologies are omniscient authority (the authority is God), and human authority (the authority is a human expert).
An epistemology based on reason claims that what is true is that which can be proven using the rules of deductive logic. Finally, an epistemology based on experience claims that what is true is that which can be encountered through one or more of the senses.
Several different variations of experience-based epistemologies exist. The two variations relevant to this discussion are anecdotal experience and experimental evidence. The first states that truth for any particular individual (or group of individuals) is that which the individual (or group) personally experiences. The second states that truth is that which can be verified through carefully controlled experiments.
The relative strengths of these epistemological approaches are as follows. Omniscient authority provides absolute truth; if there is a God and He has spoken on something, then what He says must, by definition, be true [2]. Reason yields conditional absolute truth; if the premises of a valid deductive argument are known to be true, then the conclusion of the argument must also be true. Experimental evidence provides probable truth; if controlled experiments are designed properly and repeated enough times, then it is highly probable that the results accurately describe reality. Anecdotal experience yields possible truth; if something happened for one person, it is possible it might happen to others also. Finally, human authority provides opinion.
On which of these approaches to epistemology is software engineering mostly based? Well, the software engineering literature is filled with pronouncements about how software should be developed. Representative comments include the following:
Rarely, if ever, are these pronouncements augmented with anything remotely resembling either logical or experimental evidence. Thus, one can only conclude that software engineering is based on a combination of anecdotal experience and human authority. That is, we know that a particular technique is good because John Doe, who is an authority in the field, says that it is good (human authority); John Doe knows that it is good because it worked for him (anecdotal experience). Resting an entire discipline on such a shaky epistemological foundation is absurd, but, ubiquitous nevertheless.
In a time in which software is being used in more life-critical systems than ever before, one would expect that those designing such systems would be extra careful to insure that they knew what they were doing, and why they were doing it. However, this is too often not the case. With few exceptions, the vast majority of software engineers seem to be woefully ignorant of their own ignorance.
This ignorance manifests itself in the plethora of people who jump on the latest-and-greatest "methodology" bandwagon (functional decomposition, rapid prototyping, object-orientation, CASE tools, reuse techniques, and process maturity are just a few such bandwagons) on the basis of "success stories" and slick sales pitches. The notion of requesting actual logical or experimental evidence of success seems not to enter the picture.
This ignorance manifests itself in other ways also: in the proliferation of mushy, content-free "lessons-learned" papers that litter the software engineering landscape, in the resistance to any form of meaningful data collection in software projects, and in the insistence on hiring undereducated, poorly-trained "programmers" to specify, design, code, and test important software systems.
I pray that it will not take the loss of hundreds of lives in an airplane crash, or even the loss of millions of dollars in a financial system collapse, before we acknowledge our ignorance, and redirect our efforts away from "methodologies" and "processes" and towards developing a valid epistemological foundation. If we do not do this, the loss of lives and dollars will come; the only questions will be, "when?" and "how many?"
Notes:
[1] Epistemology is a complex subject, one to which many philosophers and theologians have devoted their entire careers. The discussion here is necessarily brief and incomplete; however, it should be sufficient to demonstrate the critical importance of the subject to software engineering.
[2] In truth, I believe that the revelation that God has given in the Bible provides the only intellectually defensible fundamental epistemology; however, that is a subject for a theological or philosophical journal, not for a computer journal.
A presentation I gave on this topic is available, too.