The critical attitude, the tradition of free discussion of theories with the aim of discovering their weak spots so that they may be improved upon, is the attitude of reasonableness, of rationality. It makes far-reaching use of both verbal argument and observation—of observation in the interest of argument, however. The Greeks’ discovery of the critical method gave rise at first to the mistaken hope that it would lead to the solution of all the great old problems; that it would establish certainty; that it would help to prove our theories, to justify them. But this hope was a residue of the dogmatic way of thinking; in fact nothing can be justified or proved (outside of mathematics and logic). The demand for rational proofs in science indicates a failure to keep distinct the broad realm of rationality and the narrow realm of rational certainty: it is an untenable, an unreasonable demand. 
The recklessly critical quest for truth
With the idol of certainty (including that of degrees of imperfect certainty or probability) there falls one of the defences of obscurantism which bar the way of scientific advance. For the worship of this idol hampers not only the boldness of our questions, but also the rigour and the integrity of our tests. The wrong view of science betrays itself in the craving to be right; for it is not his possession of knowledge, of irrefutable truth, that makes the man of science, but his persistent and recklessly critical quest for truth.
Has our attitude, then, to be one of resignation? Have we to say that science can fulfill only its biological task; that it can, at best, merely prove its mettle in practical applications which may corroborate it? Are its intellectual problems insoluble? I do not think so. Science never pursues the illusory aim of making its answers final, or even probable. Its advance is, rather, towards an infinite yet attainable aim: that of ever discovering new, deeper, and more general problems, and of subjecting our ever tentative answers to ever renewed and ever more rigorous tests. 
Step-by-step approximations to truth
The degree of corroboration of two statements may not be comparable in all cases, any more than the degree of falsifiability: we cannot define a numerically calculable degree of corroboration, but can speak only roughly in terms of positive degree of corroboration, negative degrees of corroboration, and so forth. Yet we can lay down various rules; for instance the rule that we shall not continue to accord a positive degree of corroboration to a theory which has been falsified by an inter-subjectively testable experiment based upon a falsifying hypothesis. (We may, however, under certain circumstances accord a positive degree of corroboration to another theory, even though it follows a kindred line of thought. An example is Einstein’s photon theory, with its kinship to Newton’s corpuscular theory of light.) In general we regard an inter-subjectively testable falsification as final (provided it is well tested): this is the way in which the asymmetry between verification and falsification of theories makes itself felt. Each of these methodological points contributes in its own peculiar way to the historical development of science as a process of step by step approximations. [266-7]
The necessity of pushing freedoms to the extreme
Strange it is, that men should admit the validity of the arguments for free discussion, but object to their being ‘pushed to an extreme’; not seeing that unless the reasons are good for an extreme case, they are not good for any case. Strange that they should imagine that they are not assuming infallibility, when they acknowledge that there should be free discussion on all subjects which can possibly be doubtful, but think that some particular principle or doctrine should be forbidden to be questioned because it is so certain, that is, because they are certain that it is certain. To call any proposition certain, while there is any one who would deny its certainty if permitted, but who is not permitted, is to assume that we ourselves, and those who agree with us, are the judges of certainty, and judges without hearing the other side. [ch. II, 28-9]
The sole way of attaining a better truth
The beliefs which we have most warrant for, have no safeguard to rest on, but a standing invitation to the whole world to prove them unfounded. If the challenge is not accepted, or is accepted and the attempt fails, we are far enough from certainty still; but we have done the best that the existing state of human reason admits of; we have neglected nothing that could give the truth a chance of reaching us: if the lists are kept open, we may hope that if there be a better truth, it will be found when the human mind is capable of receiving it; and in the meantime we may rely on having attained such approach to truth, as is possible in our own day. This is the amount of certainty attainable by a fallible being, and this the sole way of attaining it. [ch. II, 28]
Precision for its own sake
[B]oth precision and certainty are false ideals. They are impossible to attain, and therefore dangerously misleading if they are uncritically accepted as guides. The quest for precision is analogous to the quest for certainty, and both should be abandoned.
I do not suggest, of course, that an increase in the precision of, say, a prediction, or even a formulation, may not sometimes be highly desirable. What I do suggest is that it is always undesirable to make an effort to increase precision for its own sake—especially linguistic precision—since this usually leads lo loss of clarity, and to a waste of time and effort on preliminaries which often turn out to be useless, because they are bypassed by the real advance of the subject: one should never try to be more precise than the problem situation demands.
I might perhaps state my position as follows. Every increase in clarity is of intellectual value in itself; an increase in precision or exactness has only a pragmatic value as a means to some definite end—where the end is usually an increase in testability or criticizability demanded by the problem situation (which for example may demand that we distinguish between two competing theories which lead to predictions that can be distinguished only if we increase the precision of our measurements). [24-5]
The subject-matter of mathematics
Abstract entities that are complex and autonomous exist objectively and are part of the fabric of reality. There exist logically necessary truths about these entities, and these comprise the subject-matter of mathematics. However, such truths cannot be known with certainty. Proofs do not confer certainty upon their conclusions. The validity of a particular form of proof depends on the truth of our theories of the behaviour of the objects with which we perform the proof. Therefore mathematical knowledge is inherently derivative, depending entirely on our knowledge of physics. [256-7]
Gould on fact and theory
Well, evolution is a theory. It is also a fact. And facts and theories are different things, not rungs in a hierarchy of increasing certainty. Facts are the world’s data. Theories are structures of ideas that explain and interpret facts. Facts do not go away when scientists debate rival theories to explain them. Einstein’s theory of gravitation replaced Newton’s, but apples did not suspend themselves in mid-air, pending the outcome. And humans evolved from apelike ancestors whether they did so by Darwin’s proposed mechanism or by some other, yet to be discovered.
Moreover, “fact” does not mean “absolute certainty.” The final proofs of logic and mathematics flow deductively from stated premises and achieve certainty only because they are not about the empirical world. Evolutionists make no claim for perpetual truth, though creationists often do (and then attack us for a style of argument that they themselves favor). In science, “fact” can only mean “confirmed to such a degree that it would be perverse to withhold provisional assent.” I suppose that apples might start to rise tomorrow, but the possibility does not merit equal time in physics classrooms. [254-5]
Science: more or less likely
It is not unscientific to make a guess, although many people who are not in science think it is. Some years ago I had a conversation with a layman about flying saucers—because I am scientific I know all about flying saucers! I said “I don’t think there are flying saucers”. So my antagonist said, “Is it impossible that there are flying saucers? Can you prove that it’s impossible?” “No”, I said, “I can’t prove it’s impossible. It’s just very unlikely”. At that he said, “You are very unscientific. If you can’t prove it impossible then how can you say that it’s unlikely?” But that is the way that is scientific. It is scientific only to say what is more likely and what less likely, and not to be proving all the time the possible and impossible. [165-6]