The critical attitude, the tradition of free discussion of theories with the aim of discovering their weak spots so that they may be improved upon, is the attitude of reasonableness, of rationality. It makes far-reaching use of both verbal argument and observation—of observation in the interest of argument, however. The Greeks’ discovery of the critical method gave rise at first to the mistaken hope that it would lead to the solution of all the great old problems; that it would establish certainty; that it would help to prove our theories, to justify them. But this hope was a residue of the dogmatic way of thinking; in fact nothing can be justified or proved (outside of mathematics and logic). The demand for rational proofs in science indicates a failure to keep distinct the broad realm of rationality and the narrow realm of rational certainty: it is an untenable, an unreasonable demand. 
But if ‘2 + 2 = 4’ is true, then it follows that ‘2 + 2 = 5’ is false—and that ‘2 + 2 = 6’ is false, and that ‘2 + 2 = 7’ is false, and so on. But if these statements are false, then the statements ‘“2 + 2 = 5” is false’ and ‘“2 + 2 = 6” is false’ and ‘“2 + 2 = 7” is false’ are one and all true. And it is easy to see that we can, in this way, generate an infinite number of true statements that are of no interest to science at all, or to anyone else for that matter. 
Thus we accept the idea that the task of science is the search for truth, that is, for true theories (even though as Xenophanes pointed out we may never get them, or know them as true if we get them). Yet we also stress that truth is not the only aim of science. We want more than mere truth: what we look for is interesting truth – truth which is hard to come by. And in the natural sciences (as distinct from mathematics) what we look for is truth which has a high degree of explanatory power, which implies that it is logically improbable.
For it is clear, first of all, that we do not merely want truth – we want more truth, and new truth. We are not content with ‘twice two equals four’, even though it is true: we do not resort to reciting the multiplication table if we are faced with a difficult problem in topology or in physics. Mere truth is not enough; what we look for are answers to our problems.
Only if it is an answer to a problem – a difficult, a fertile problem, a problem of some depth – does a truth, or a conjecture about the truth, become relevant to science. This is so in pure mathematics, and it is so in the natural sciences. And in the latter, we have something like a logical measure of the depth or significance of the problem in the increase of logical improbability or explanatory power of the proposed new answer, as compared with the best theory or conjecture previously proposed in the field. [311-2]
So, a computation or a proof is a physical process in which objects such as computers or brains physically model or instantiate abstract entities like numbers or equations, and mimic their properties. It is our window on the abstract. It works because we use such entities only in situations where we have good explanations saying that the relevant physical variables in those objects do indeed instantiate those abstract properties.
Consequently, the reliability of our knowledge of mathematics remains for ever subsidiary to that of our knowledge of physical reality. Every mathematical proof depends absolutely for its validity on our being right about the rules that govern the behaviour of some physical objects, like computers, or ink and paper, or brains. So, contrary to what Hilbert thought, and contrary to what most mathematicians since antiquity have believed and believe to this day, proof theory can never be made into a branch of mathematics. Proof theory is a science: specifically, it is computer science.
The whole motivation for seeking a perfectly secure foundation for mathematics was mistaken. It was a form of justificationism. Mathematics is characterized by its use of proofs in the same way that science is characterized by its use of experimental testing; in neither case is that the object of the exercise. The object of mathematics is to understand – to explain – abstract entities. Proof is primarily a means of ruling out false explanations; and sometimes it also provides mathematical truths that need to be explained. But, like all fields in which progress is possible, mathematics seeks not random truths but good explanations. [188-9]
Abstract entities that are complex and autonomous exist objectively and are part of the fabric of reality. There exist logically necessary truths about these entities, and these comprise the subject-matter of mathematics. However, such truths cannot be known with certainty. Proofs do not confer certainty upon their conclusions. The validity of a particular form of proof depends on the truth of our theories of the behaviour of the objects with which we perform the proof. Therefore mathematical knowledge is inherently derivative, depending entirely on our knowledge of physics. [256-7]
Thanks to Gödel, we know that there will never be a fixed method of determining whether a mathematical proposition is true, any more than there is a fixed way of determining whether a scientific theory is true. Nor will there ever be a fixed way of generating new mathematical knowledge. Therefore progress in mathematics will always depend on the exercise of creativity. It will always be possible, and necessary, for mathematicians to invent new types of proof. They will validate them by new arguments and by new modes of explanation depending on their ever improving understanding of the abstract entities involved. Gödel’s own theorems were a case in point: to prove them, he had to invent a new method of proof. I said the method was based on the ‘diagonal argument’, but Gödel extended that argument in a new way. Nothing had ever been proved in this way before; no rules of inference laid down by someone who had never seen Gödel’s method could possibly have been prescient enough to designate it as valid. Yet it is self-evidently valid. Where did this self-evidentness come from? It came from Gödel’s understanding of the nature of proof. Gödel’s proofs are as compelling as any in mathematics, but only if one first understands the explanation that accompanies them.
So explanation does, after all, play the same paramount role in pure mathematics as it does in science. Explaining and understanding the world – the physical world and the world of mathematical abstractions – is in both cases the object of the exercise. Proof and observation are merely means by which we check our explanations. [235-6]
Do abstract, non-physical entities exist? Are they part of the fabric of reality? I am not interested here in issues of mere word usage. It is obvious that numbers, the laws of physics, and so on do ‘exist’ in some senses and not in others. The substantive question is this: how are we to understand such entities? Which of them are merely convenient forms of words, referring ultimately only to ordinary, physical reality? Which are merely ephemeral features of our culture? Which are arbitrary, like the rules of a trivial game that we need only look up? And which, if any, can be explained only in a way that attributes an independent existence to them? Things of this last type must be part of the fabric of reality as defined in this book, because one would have to understand them in order to understand everything that is understood. [222-3]
The main problem with school mathematics is that there are no problems. Oh, I know what passes for problems in math classes, these insipid “exercises.” “Here is a type of problem. Here is how to solve it. Yes it will be on the test. Do exercises 1−35 odd for homework.” What a sad way to learn mathematics: to be a trained chimpanzee.
But a problem, a genuine honest-to-goodness natural human question—that’s another thing. How long is the diagonal of a cube? Do prime numbers keep going on forever? Is infinity a number? How many ways can I symmetrically tile a surface? The history of mathematics is the history of mankind’s engagement with questions like these, not the mindless regurgitation of formulas and algorithms (together with contrived exercises designed to make use of them).
A good problem is something you don’t know how to solve. That’s what makes it a good puzzle, and a good opportunity. A good problem does not just sit there in isolation, but serves as a springboard to other interesting questions. A triangle takes up half its box. What about a pyramid inside its three-dimensional box? Can we handle this problem in a similar way?
I can understand the idea of training students to master certain techniques—I do that too. But not as an end in itself. Technique in mathematics, as in any art, should be learned in context. The great problems, their history, the creative process—that is the proper setting. Give your students a good problem, let them struggle and get frustrated. See what they come up with. Wait until they are dying for an idea, then give them some technique. But not too much. [40-2]