The misuse of significance tests

The examples elaborated in the foregoing sections of numerical discrepancies arising from tbe rigid formulation of a rule, which at first acquaintance it seemed natural to apply to all tests of significance, constitute only one aspect of the deep-seated difference in point of view which arises when Tests of Significance are reinterpreted on the analogy of Acceptance Decisions. It is indeed not only numerically erroneous conclusions, serious as these are, that are to be feared from an uncritical acceptance of this analogy.

An important difference is that Decisions are final, while the state of opinion derived from a test of significance is provisional, and capable, not only of confirmation, but of revision. An acceptance procedure is devised for a whole class of cases. No particular thought is given to each case as it arises, nor is the tester’s capacity for learning exercised. A test of significance on the other hand is intended to aid the process of learning by observational experience.[100]

Unnatural science

If a broad line of demarcation is drawn between the natural sciences and what can only be described as the unnatural sciences, it will at once be recognized as a distinguishing mark of the latter that their practitioners try most painstakingly to imitate what they believe—quite wrongly, alas for them—to be the distinctive manners and observances of the natural sciences. Among these are:

(a) the belief that measurement and numeration are intrinsically praiseworthy activities (the worship, indeed, of what Ernst Gombrich calls idola quantitatis);

(b) the whole discredited farrago of inductivism—especially the belief that facts are prior to ideas and that a sufficiently voluminous compilation of facts can be processed by a calculus of discovery in such a way as to yield general prin­ciples and natural-seeming laws;

(c) another distinguishing mark of unnatural scientists is their faith in the efficacy of statistical formulas, particularly when processed by a computer—the use of which is in itself interpreted as a mark of scientific manhood. There is no need to cause offense by specifying the unnatural sciences, for their practitioners will recognize themselves easily: the shoe belongs where it fits. [167]

The merely rational

Es ist sehr interessant, daß die geschichtliche Periode, in der bei uns die Tradition des kritischen Denkens zum ersten Male einen entscheidenden Durchbruch erzielte, nämlich die Aufklärung, im deutschen Sprachbereich nur selten eine positive Würdigung erfährt. Man versieht sie gerne mit schmückenden Beiworten pejorativen Charakters wie „flach“, „unhistorisch“, „trocken“, bescheinigt ihr eine Überschätzung des bloß Vernünftigen und dokumentiert die eigene Über­legenheit damit, daß man bereit ist, dunklen, unklaren, widerspruchsvollen und vieldeutigen Ergebnissen geistiger An­strengungen wegen ihrer angeblichen Tiefe den Vorzug vor den Resultaten klaren, nüchternen und kritischen Denkens zu geben.

A whiff of Pangloss

Democracies have serious drawbacks. They certainly are not better than they ought to be. But corruption can occur under any kind of government. And I think that every serious student of history will agree, upon consideration, that our Western democracies are not only the most prosperous societies in history – that is important, but not so very important – but the freest, the most tolerant, and the least repressive large societies of which we have historical knowledge.

I have said this before, of course. It would be almost criminal not to say it if one believes it. One must fight those who make so many young people unhappy by telling them that we live in a terrible world, in a kind of capitalist hell. The truth is that we live in a wonderful world, in a beautiful world, and in an astonishingly free and open society. Of course it is fashionable, it is expected, and it is almost demanded from a Western intellectual to say the opposite, to lament loudly about the world we live in, about our social ills, about the inherent injustice of our society, and especially about the alleged terrible inequalities, and the impending day of reckoning.

I do not think that any of this is true. It is true that there are a people who are very rich. But what does it matter to me or to you? It is most certainly not true that anybody suffers because a few are very rich, not to mention that quite a few of the few who are rich spend much of their money on such things as founding universities and lectureships and on scholarships and cancer research.

The truth is that Western democracies are the only societies in which there is much freedom, much welfare, and much equality before the law. Of course, our society is very far from perfect. There is much misuse of drugs, of tobacco, and of alcohol. [335]

A party in the interest of all

And so it becomes clear that the political aims of us all must be the containment of power rather than its attainment; the control of and resistance to power rather its use for some ideological purpose. My very simple and hopeful proposal is this: At least one of the major politcal parties in a state should declare its interest in trying to surmount the dominance of ideologies, and its readiness to try to replace them by a straightforward programme of serving the most urgent needs of all. [390]

Bold piecemeal engineering

I do not suggest that piecemeal engineering cannot be bold, or that it must be confined to ‘smallish’ problems. But I think that the degree of complication which we can tackle is governed by the degree of our experience gained in con­scious and systematic piecemeal engineering. [ch. 9, n3]

The scope of piecemeal engineering

The difference between Utopian and piecemeal engineering turns out, in practice, to be a difference not so much in scale and scope as in caution and in preparedness for unavoidable surprises. … [W]hile the piecemeal engineer can attack his problem with an open mind as to the scope of the reform, the holist cannot do this; for he has decided before­hand that a complete reconstruction is possible and necessary. [63]

Scientific politics

According to this piecemeal view, there is no clearly marked division between the pre-scientific and the scientific ex­perimental approaches, even though the more and more conscious application of scientific, that is to say, of critical methods, is of great importance. Both approaches may be described, fundamentally, as utilizing the method of trial and error. We try; that is, we do not merely register an observation, but make active attempts to solve some more or less practical and definite problems. And we make progress if, and only if, we are prepared to learn from our mistakes: to recognize our errors and to utilize them critically instead of persevering in them dogmatically. Though this analysis may sound trivial, it describes, I believe, the method of all empirical sciences. This method assumes a more and more scien­tific character the more freely and consciously we are prepared to risk a trial, and the more critically we watch for the mistakes we always make. …

For the piecemeal technologist or engineer these views mean that, if he wishes to introduce scientific methods into the study of society and into politics, what is needed most is the adoption of a critical attitude, and the realization that not only trial but also error is necessary. And he must learn not only to expect mistakes, but consciously to search for them. We all have an unscientific weakness for being always in the right, and this weakness seems to be particularly common among professional and amateur politicians. But the only way to apply something like scientific method in politics is to proceed on the assumption that there can be no political move which has no drawbacks, no undesirable consequen­ces. To look out for these mistakes, to find them, to bring them into the open, to analyse them, and to learn from them, this is what a scientific politician as well as a political scientist must do. Scientific method in politics means that the great art of convincing ourselves that we have not made any mistakes, of ignoring them, of hiding them, and of blaming others for them, is replaced by the greater art of accepting the responsibility for them, of trying to learn from them, and of applying this knowledge so that we may avoid them in future. [80-1]

Praise is of no intellectual value

We should be thankful for every serious attempt to criticize our positions. Praise is of no intellectual value. Victories in debates are of no intellectual value. Yet if we can succeed even in getting a little clearer about our problems, we should be very happy with that intellectual achievement. [205]

Every policy is a solution to a problem

Every reputable political or social policy is a proposed solution to a problem; and we always need to be clear about the problem before we can propose the solution. We must always be able to ask of a policy: “To what problem is this the solution?” If there is no problem to which a given policy is a solution then the policy is superfluous, and therefore harm­ful, if only because it consumes resources to no purpose. … The whole notion that you can start with policies is deeply erroneous, and very damaging in practice. [151]

Older posts «