A serious empirical test always consists in the attempt to find a refutation, a counterexample. In the search for a counterexample, we have to use our background knowledge; for we always try to refute first the most risky predictions, the ‘most unlikely … consequences’ (as Peirce already saw); which means that we always look in the most probable kinds of places for the most probable kinds of counterexamples—most probable in the sense that we should expect to find them in the light of our background knowledge. Now if a theory stands up to many such tests, then, owing to the incorporation of the results of our tests into our background knowledge, there may be, after a time, no places left where (in the light of our new background knowledge) counter examples can with a high probability be expected to occur. But this means that the degree of severity of our test declines. This is also the reason why an often repeated test will no longer be considered as significant or as severe: there is something like a law of diminishing returns from repeated tests (as opposed to tests which, in the light of our background knowledge, are of a new kind, and which therefore may still be felt to be significant). These are facts which are inherent in the knowledge-situation; and they have often been described—especially by John Maynard Keynes and by Ernest Nagel—as difficult to explain by an inductivist theory of science. But for us it is all very easy. And we can even explain, by a similar analysis of the knowledge-situation, why the empirical character of a very successful theory always grows stale, after a time. We may then feel (as Poincaré did with respect to Newton’s theory) that the theory is nothing but a set of implicit definitions or conventions—until we progress again and, by refuting it, incidentally re-establish its lost empirical character. (De mortuis nil nisi bene: once a theory is refuted, its empirical character is secure and shines without blemish.) [325-6]
Category: “Truth, Rationality, and the Growth of Scientific Knowledge”
Conjectures and Refutations, pp. 291-338
The objective theory of truth leads to a very different attitude. This may be seen from the fact that it allows us to make assertions such as the following: a theory may be true even though nobody believes it, and even though we have no reason for accepting it, or for believing that it is true; and another theory may be false, although we have comparatively good reasons for accepting it.
Clearly, these assertions would appear to be self-contradictory from the point of view of any subjective or epistemic theory of truth. But within the objective theory, they are not only consistent, but quite obviously true.
A similar assertion which the objective correspondence theory would make quite natural is this: even if we hit upon a true theory, we shall as a rule be merely guessing, and it may well be impossible for us to know that it is true. 
Thus we accept the idea that the task of science is the search for truth, that is, for true theories (even though as Xenophanes pointed out we may never get them, or know them as true if we get them). Yet we also stress that truth is not the only aim of science. We want more than mere truth: what we look for is interesting truth – truth which is hard to come by. And in the natural sciences (as distinct from mathematics) what we look for is truth which has a high degree of explanatory power, which implies that it is logically improbable.
For it is clear, first of all, that we do not merely want truth – we want more truth, and new truth. We are not content with ‘twice two equals four’, even though it is true: we do not resort to reciting the multiplication table if we are faced with a difficult problem in topology or in physics. Mere truth is not enough; what we look for are answers to our problems.
Only if it is an answer to a problem – a difficult, a fertile problem, a problem of some depth – does a truth, or a conjecture about the truth, become relevant to science. This is so in pure mathematics, and it is so in the natural sciences. And in the latter, we have something like a logical measure of the depth or significance of the problem in the increase of logical improbability or explanatory power of the proposed new answer, as compared with the best theory or conjecture previously proposed in the field. [311-2]
When the judge tells a witness that he should speak ‘The truth, the whole truth, and nothing but the truth’, then what he looks for is as much of the relevant truth as the witness may be able to offer. A witness who likes to wander off into irrelevancies is unsatisfactory as a witness, even though these irrelevancies may be truisms, and thus part of ‘the whole truth’. It is quite obvious that what the judge — or anybody else — wants when he asks for ‘the whole truth’ is as much interesting and relevant true information as can be got; and many perfectly candid witnesses have failed to disclose some important information simply because they were unaware of its relevance to the case.
Thus when we stress, with Busch, that we are not interested in mere truth but in interesting and relevant truth, then, I contend, we only emphasize a point which everybody accepts. And if we are interested in bold conjectures,even if these should soon turn out to be false, then this interest is due to our methodological conviction that only with the help of such bold conjectures can we hope to discover interesting and relevant truth.
There is a point here which, I suggest, it is the particular task of the logician to analyse. ‘Interest’, or ‘relevance’, in the sense here intended, can be objectively analysed; it is relative to our problems; and it depends on the explanatory power, and thus on the content or improbability, of the information. The measures alluded to earlier […] are precisely such measures as take account of some relative content of the information — its content relative to a hypothesis or to a problem. [312-3]