There are no sensory “data”. Rather, there is an incoming challenge from the sensed world, which then puts the brain, or ourselves, to work on it, to try to interpret it. Thus, at first, there are no data: there is, rather, a challenge to do something; namely to interpret. Then we try to match the so-called sense data. I say “so-called” because I don’t think there are sense “data”. What most people hold to be simple sense “datum” is in fact the outcome of a most elaborate process. Nothing is directly “given“ to us: perception is arrived at only as a result of many steps involving interaction between the stimuli which reach the senses, the interpreting apparatus of the senses, and the structure of the brain. So, while the term “sense datum” suggests primacy in the first step, I would suggest that, before I can realize what is a sense datum for me (before it is ever “given” to me), there are a hundred steps of give and take which result from the challenge presented to our senses and our brain. 
Science begins with observation, says Bacon, and this saying is an integral part of the Baconian religion. It is still widely accepted, and still repeated ad nauseam in the introductions to even some of the best textbooks in the field of the physical and biological sciences.
I propose to replace this Baconian formula by another one.
Science, we may tentatively say, begins with theories, with prejudices, superstitions, and myths. Or rather, it begins when a myth is challenged and breaks down – that is, when some of our expectations are disappointed. But this means that science begins with problems, practical problems or theoretical problems. 
The belief that science proceeds from observation to theory is still so widely and so firmly held that my denial of it is often met with incredulity. I have even been suspected of being insincere – of denying what nobody in his senses can doubt.
But in fact the belief that we can start with pure observations alone, without anything in the nature of a theory, is absurd; as may be illustrated by the story of the man who dedicated his life to natural science, wrote down everything he could observe, and bequeathed his priceless collection of observations to the Royal Society to be used as inductive evidence. This story should show us that though beetles may profitably be collected, observations may not.
Twenty-five years ago I tried to bring home the same point to a group of physics students in Vienna by beginning a lecture with the following instructions: ‘Take pencil and paper; carefully observe, and write down what you have observed!’ They asked, of course, what I wanted them to observe. Clearly the instruction, ‘Observe!’ is absurd. (It is not even idiomatic, unless the object of the transitive verb can be taken as understood.) Observation is always selective. It needs a chosen object, a definite task, an interest, a point of view, a problem. And its description presupposes a descriptive language, with property words; it presupposes similarity and classification, which in its turn presupposes interests, points of view, and problems. 
Thus Kant’s reply to Hume came near to being right; for the distinction between an a priori valid expectation and one which is both genetically and logically prior to observation, but not a priori valid, is really somewhat subtle. But Kant proved too much. In trying to show how knowledge is possible, he proposed a theory which had the unavoidable consequence that our quest for knowledge must necessarily succeed, which is clearly mistaken. When Kant said, ‘Our intellect does not draw its laws from nature but imposes its laws upon nature’, he was right. But in thinking that these laws are necessarily true, or that we necessarily succeed in imposing them upon nature, he was wrong. Nature very often resists quite successfully, forcing us to discard our laws as refuted; but if we live we may try again.