Tag Archive: falsifiability

Inductive psychology vs deductive physics

Contrast this bizarre state of affairs with the state of affairs in physics. While there are of course a few exceptions, the usual situation in the experimental testing of a physical theory at least involves the prediction of a form of function (with parameters to be fitted); or, more commonly, the prediction of a quantitative magnitude (point-value). Improvements in the accuracy of determining this experimental function-form or point-value, whether by better instrumentation for control and making observations, or by the gathering of a larger number of measurements, has the effect of narrowing the band of tolerance about the theoretically predicted value. What does this mean in terms of the significance-testing model? It means: In physics, that which corresponds, in the logical structure of statistical inference, to the old-fashioned point-null hypothesis H0 is the value which flows as a consequence of the substantive theory T; so that an increase in what the statistician would call “power” or “precision” has the methodological effect of stiffening the experimental test, of setting up a more difficult observational hurdle for the theory T to surmount. Hence, in physics the effect of improving precision or power is that of decreasing the prior probability of a successful experimental outcome if the theory lacks verisimil­itude, that is, precisely the reverse of the situation obtaining in the social sciences.

As techniques of control and measurement improve or the number of observations increases, the methodological effect in physics is that a successful passing of the hurdle will mean a greater increment in corroboration of the substantive theory; whereas in psychology, comparable improvements at the experimental level result in an empirical test which can provide only a progressively weaker corroboration of the substantive theory.

In physics, the substantive theory predicts a point-value, and when physicists employ “significance tests,” their mode of employment is to compare the theoretically predicted value x0 with the observed mean x0, asking whether they differ (in either direction!) by more than the “probable error” of determination of the latter. Hence H : H0 = μx functions as a point-null hypothesis, and the prior (logical, antecedent) probability of its being correct in the absence of theory approximates zero. As the experimental error associated with our determination of x0 shrinks, values of x0 consistent with x0 (and hence, compatible with its implicans T) must lie within a narrow range. In the limit (zero probable error, corresponding to “perfect power” in the significant test) any non-zero difference (x0 – x0) provides a modus tollens refutation of T. If the theory has negligible verisimilitude, the logical probability of its surviving such a test is negligible. Whereas in psychol­ogy, the result of perfect power (i.e., certain detection of any non-zero difference in the predicted direction) is to yield a prior probability p = ½ of getting experimental results compatible with T, because perfect power would mean guaranteed detection of whatever difference exists; and a difference [quasi] always exists, being in the “theoretically expected direc­tion” half the time if our substantive theories were all of negligible verisimilitude (two-urn model). [112-3]

Methodological confirmation bias

Inadequate appreciation of the extreme weakness of the test to which a substantive theory T is subjected by merely pre­dicting a directional statistical difference d > 0 is then compounded by a truly remarkable failure to recognize the logical asymmetry between, on the one hand, (formally invalid) “confirmation” of a theory via affirming the consequent in an argument of form: [T ⊃ H1, H1, infer T], and on the other hand the deductively tight refutation of the theory modus tollens by a falsified prediction, the logical form being: [T ⊃ H1, ~H1, infer ~T].

While my own philosophical predilections are somewhat Popperian, I daresay any reader will agree that no full-fledged Popperian philosophy of science is presupposed in what I have just said. The destruction of a theory modus tollens is, after all, a matter of deductive logic; whereas that the “confirmation” of a theory by its making successful predictions involves a much weaker kind of inference. This much would be conceded by even the most anti-Popperian “inductivist.” The writing of behavior scientists often reads as though they assumed—what it is hard to believe anyone would ex­plicitly assert if challenged—that successful and unsuccessful predictions are practically on all fours in arguing for and against a substantive theory. [112]

The soft corroboration of psychology

Isn’t the social scientist’s use of the null hypothesis simply the application of Popperian (or Bayesian) thinking in con­texts in which probability plays such a big role? No, it is not. One reason it is not is that the usual use of null hypothesis testing in soft psychology as a means of “corroborating” substantive theories does not subject the theory to grave risk of refutation modus tollens, but only to a rather feeble danger. The kinds of theories and the kinds of theoretical risks to which we put them in soft psychology when we use significance testing as our method are not like testing Meehl’s theory of weather by seeing how well it forecasts the number of inches it will rain on certain days. Instead, they are depressingly close to testing the theory by seeing whether it rains in April at all, or rains several days in April, or rains in April more than in May. [821-2]

More falsificationism strawmen going up in flames

A second familiar approach from the same period is Karl Popper’s ‘falsificationist’ criterion, which fares no better. Apart from the fact that it leaves ambiguous the scientific status of virtually every singular existential statement, however well supported (e.g., the claim that there are atoms, that there is a planet closer to the sun than the Earth, that there is a missing link), it has the untoward consequence of countenancing as ‘scientific’ every crank claim which makes ascertainably false assertions. Thus flat Earthers, biblical creationists, proponents of laetrile or orgone boxes, Uri Geller devotees, Bermuda Triangulators, circle squarers, Lysenkoists, charioteers of the gods, perpetuum mobile builders, Big Foot searchers, Loch Nessians, faith healers, polywater dabblers, Rosicrucians, the-world-is-about-to-enders, primal screamers, water diviners, magicians, and astrologers all turn out to be scientific on Popper’s criterion – just so long as they are prepared to indicate some observation, however improbable, which (if it came to pass) would cause them to change their minds. [121]

Popper on Duhem–Quine’s naive falsificationism

The falsifying mode of inference here referred to—the way in which the falsification of a conclusion entails the falsifi­cation of the system from which it is derived—is the modus tollens of classical logic. It may be described as follows:

Let p be a conclusion of a system t of statements which may consist of theories and initial conditions (for the sake of simplicity I will not distinguish between them). We may then symbolize the relation of derivability (analytical implication) of p from t by ‘t ➙ p’ which may be read: ‘p follows from t ’. Assume p to be false, which we may write ‘p’, to be read ‘not-p’. Given the relation of deducibility, t ➙ p, and the assumption p, we can then infer t  (read ‘not-t ’); that is, we regard t as falsified. If we denote the conjunction (simultaneous assertion) of two statements by putting a point between the symbols standing for them, we may also write the falsifying inference thus: ((t ➙ p).p) ➙ t , or in words: ‘If p is derivable from t, and if p is false, then t also is false’.

By means of this mode of inference we falsify the whole system (the theory as well as the initial conditions) which was required for the deduction of the statement p, i.e. of the falsified statement. Thus it cannot be asserted of any one statement of the system that it is, or is not, specifically upset by the falsification. Only if p is independent of some part of the system can we say that this part is not involved in the falsification.* With this is connected the following possibility: we may, in some cases, perhaps in consideration of the levels of universality, attribute the falsification to some definite hypothesis—for instance to a newly introduced hypothesis. This may happen if a well-corroborated theory, and one which continues to be further corroborated, has been deductively explained by a new hypothesis of a higher level. The attempt will have to be made to test this new hypothesis by means of some of its consequences which have not yet been tested. If any of these are falsified, then we may well attribute the falsification to the new hypothesis alone. We shall then seek, in its stead, other high-level generalizations, but we shall not feel obliged to regard the old system, of lesser generality, as having been falsified.

* Thus we cannot at first know which among the various statements of the remaining sub-system t ′ (of which p is not independent) we are to blame for the falsity of p; which of these statements we have to alter, and which we should retain. (I am not here discussing interchangeable statements.) It is often only the scientific instinct of the investigator (influenced, of course, by the results of testing and re-testing) that makes him guess which statements of t ′ he should regard as innocuous, and which he should regard as being in need of modification. Yet it is worth remembering that it is often the modification of what we are inclined to regard as obviously innocuous (because of its complete agreement with our normal habits of thought) which may produce a decisive advance. A notable example of this is Einstein’s modification of the concept of simultaneity. [55-6]

The logic of discovering our errors

If the purpose of an argument is to prove its conclusion, then it is difficult to see the point of falsifiability. For deductive arguments cannot prove their conclusions any more than inductive ones can.

But if the purpose of the argument is to force us to choose, then the point of falsifiability becomes clear.

Deductive arguments force us to question, and to reexamine, and, ultimately, to deny their premises if we want to deny their conclusions. Inductive arguments simply do not.

This the real meaning of Popper’s Logic of Scientific Discovery—and it is the reason, perhaps, why so many readers have misunderstood its title and its intent. The logic of discovery is not the logic of discovering theories, and it is not the logic of discovering that they are true.

Neither deduction nor induction can serve as a logic for that.

The logic of discovery is the logic of discovering our errors. We simply cannot deny the conclusion of a deductive argu­ment without discovering that we were in error about its premises. Modus tollens can help us to do this if we use it to set problems for our theories. But while inductive arguments may persuade or induce us to believe things, they cannot help us discover that we are in error about their premises. [113-4]

Never put to a fair test

Of course, like the most enduring monarchies, the scientific establishment continues to enjoy widespread public sup­port on most matters, including the tinge of divine inspiration that has traditionally legitimated royalty. It might therefore be claimed that science already represents ‘the will of the people’, and hence requires no further philosophical schemes for democratisation. Here Popper’s anti-majoritarian approach to democracy – what I would call his ‘civic republican’ sensibility – comes to the fore. Many authoritarian regimes, especially the 20th-century fascist and com­munist ones, could also persuasively claim widespread popular support, at least at the outset and in relation to the available alternatives. For Popper, however, the normative problem posed by these regimes is that their performance is never put to a fair test. Kuhn suffers from the same defect: a paradigm is simply an irrefutable theory that becomes the basis for an irreversible policy. [47−8]

Accountable to more than just themselves

Popper and his followers were unique in seizing a glaring weakness in Kuhn’s theory: Kuhnian normal science was a politically primitive social formation that combined qualities of the Mafia, a royal dynasty and a religious order. It lacked the sort of constitutional safeguards that we take for granted in modern democracies that regularly force politicians to be accountable to more people than just themselves. Scientists should be always trying to falsify their theories, just as people should be always invited to find fault in their governments and consider alternatives – and not simply wait until the government can no longer hide its mistakes. This notoriously led Popper and his students to be equal opportunity fault-finders across the natural and social sciences. [46]

Intellectual honesty

Intellectual honesty does not consist in trying to entrench, or establish one’s position by proving (or ‘probabilifying’) it — intellectual honesty consists rather in specifying precisely the conditions under which one is willing to give up one’s position. [92]

The stimulus to further development

As Max Planck once remarked in a lecture,

A living and flourishing theory does not avoid its anomalies but searches them out, for the stimulus to fur­ther development comes from contradiction, not from confirmations.

The point to keep in mind is that although falsifiability, rather than verifiability, is the most important criterion in determin­ing whether a theory is scientifically meaningful, its usefulness for the greater task of building confidence in a theory is limited. A theory is accepted not simply because it has withstood many attempts at falsification, the need for such tests notwithstanding, but because it leads to predictions that are experimentally verified. After all, the purpose of a theory is to be productive and not just to fail to be wrong. [115]

Older posts «

» Newer posts