Can definitions be experimentally tested?

Question

The present Wikipedia article on the Scientific method, section "Elements of the scientific method" states:

Four essential elements of the scientific method are iterations, recursions, interleavings, or orderings of the following:

• Characterizations (observations, definitions, and measurements of the subject of inquiry)
• Hypotheses (theoretical, hypothetical explanations of observations and measurements of the subject)
• Predictions (reasoning including logical deduction from the hypothesis or theory)
• Experiments (tests of all of the above)

[Emphasis mine. Markup referring to several wikilinks and references omitted. Notably, there was no specific reference attached to the emphasized phrase.]

In this quote I find the final parenthetical emphasized phrase questionable:
because apparently it implies that "characterizations (observations, definitions, measurements)" are subject to experimental tests.

In contrast, I'd say that

• definitions of notions and terminology by which to express particular hypotheses and predictions endure any tests and possibly resulting refutation of those hypotheses and predictions,

• definitions of notions and terminology by which to express particular measurement operations endure any application of those measurement operations to given observational data regardless of resulting measurement values, and

• observational data collected in previous trials and derived measured values pertaining to those previous trials endure the possible collection of observational data and derivation of measurements in any subsequent trials.

Focussing especially on definitions, therefore I'd like to know:

Is there anyone besides Wikipedia known to claim explicitly that definitions can be experimentally tested?

Is there anyone known who would follow my claim instead, as sketched above, that definitions cannot be experimentally tested?

Is there any possible resolution of this conflict, for instance by considering different notions of "definition" and/or of "experimental test"?

Answer 1

You can test whether a definition is a good one by assessing whether it parsimoniously divides things where distinctions need to be made.

For example, let's say a restaurant is "swell" if it has a rating in the top half (of all ratings of restaurants), and it is "lame" if it is not. It will turn out that there are a huge number of restaurants that are essentially indistinguishable in quality (those that are about average) but are labeled "swell" or "lame". This means our definitions are--at least for the purposes of distinguishing restaurants we wish to go to from those we do not--poor. It's not false, since you can define anything you want (as long as it's not logically inconsistent), but it's wrong in that it doesn't map onto the structure of the world the way we (at least implicitly) want it to.

Also, some definitions make implicit truth-claims, e.g. if you have procedural vs. declarative memory, you are implying that a memory cannot simultaneously be both--and that implicit claim is subject to test, and if a memory can be both, then the definition needs at least to be amended.

Whether a definition is good or not depends also on whether it is good for us even if it isn't universal. For example, we say that yellow and violet are complementary colors not because of the properties of the electromagnetic spectrum at wavelengths that appear to us to be yellow and violet, but because of how our three-receptor color vision works. It is true that yellow and violet mixed together appears uncolored (gray) when mixed together in appropriate proportions, while if you mix yellow and blue in various proportions you will get a spectrum of non-gray colors. The definition of complementary colors therefore can be tested and found to be sound (i.e. there is actually an interesting phenomenon there that it captures, even if it is specific to human perception of color).

Answer 2

There is a scene in Lakatos's Proofs and Refutations that goes something like this:

"We define a polyhedron to be a three dimensional solid with only straight edges."

Someone draws a cube with another cube inside it -- "This is a polyhedron?"

"Oh, I guess we need to add the fact that the object is connected."

"But this is a cube with a cube cut out of the middle of it. It is connected."

"I mean the edges must be connected when they are drawn out, We need to add an edge that connects the outer cube to the inner cube."

"In what way is that an edge? It lies along the side of no face."

"Well, then we will reduce the definition of a polyhedron for our purposes to be a connected drawing of a network of polygons."

"Well we just had trouble agreeing on what an edge was, so we should define a polygon."

"A polygon is a connected, closed sequence of non-intersecting line segments embedded in a plane."

(Several attempts to define a polygon fail.)

In what way is this not experimental evidence disproving a succession of definitions?

You might consider drawing or building geometrical objects to be just thought experiments, and not real experiments. But either thought-experiment data is data or Relativity was not science until the 40's. And constructions or direct representations are awfully concrete for thought-experiments.

Further, in practice these objections do not arise a priori like this, at the very start of a theory. The counterexamples are found when predictions fail and we realize the flaw was in an overly-aggressive definition that is in fact not properly fit to do the job of defining.

So, yes, data tests definitions, there is not a nice neat waterfall that goes define -> predict -> evaluate -> refine -> next theory.

(I would suggest you read the actual Proofs and Refutations just because it is simple, short, entertaining, and makes some very good points.)

Another approach is to look at something from a weaker science like psychology. Look at intelligence or personality theory.

Every definition of intelligence begs the question as to whether it is stable enough to mean anything. Half the theoretical literature in the field for about 25 years, was using experimental data to prove or disprove the well-foundedness and non-circularity of a given definition of intelligence. (If a definition is not well-founded, or is ultimately circular, then is it a definition, or just a logical structure that will hopefully converge upon one?)

(You cannot give up and claim you have a test that measures success on the test, without discarding the usefulness of the test. That is simply defensive nonsense. The test is constructed from a theory, with specific defined attributes dictated by the theory.)

Either none of this is science, or a whole lot of empirical data may be needed to establish whether a definition actually defines anything. This cannot be done with mere thought-experiments.

Likewise, every personality theory has a totally different definition of personality, and proceeds to continually refine that definition based upon observations.

Are the Jungian types: Introverted vs Extroverted, Sensing vs Intuitive, Thinking vs Feeling, defined? You can read the definitions, from back in the 1930's, but they won't really help unless you run some experiments, or you are Jung. The definitions are observations of what kinds of communication techniques address different kinds of people in different states. So is there no definition here? Are these undefined, yet useful distinctions? To what degree does that leave meaning to the word definition?

Answer 3

Lets try another route. Yes, there are definitely different kinds of definitions, in mathematics, where such things tend to be most restrictive, there are at least three:

1. As compressive notation: We can define the real numbers as distances along a chosen line. This is, in the end, completely circular and does not define anything. We already had the notion of the real numbers, or we would not be measuring things, but it makes a new way of saying things that is more compact when we are focussed on points rather than measures.

   An example from outside math of this kind of definition is defining "work" as the product of force and distance, just so we can refer to it efficiently.

2. Via distinction and equivalence: We can define the real numbers as the equivalence classes of convergent sequences of rational numbers the unions of which still converge. This brings up background questions. We have to establish that the condition on the equivalence satisfies the requirements of an equivalence relation. That can be work. If it failed, we would not have a definition. We would not have accidentally defined something else, other than the ordinary reals, we would in fact have a statement that if treated as a definition, would give us false proofs.

   An example from outside of math is the definition of "species". What defines the word is the ability to describe how to tell whether two animals are of the same species, and just that. (I don't mean the standards for each species, but the idea that those standards exist generally.) It is important that the distinguishing criterion not get too vague or become ambivalent about a too many pairs of animals, or we have to start questioning the concept of species itself.

3. Via models of axioms: We can define real numbers as the minimal ordered field with consistent least upper bounds. Besides being oddly abstruse, this also pushes a horde of assumptions. It is the minimal model of this set of axioms, there are not thousands of unrelated models that all behave differently -- they all contain this one, and that takes a lot of proving.

   An example of this kind of definition from outside math is that of 'the alpha of a troupe'. It picks out a specific kind of animal from a group by its behavior and that of those around it. The individual models an archetype or a stereotype made up of general rules. If we find different applications of the same checklist of features leads to disagreement too often, we have to discard the definition, and perhaps question the applicability of the concept itself.

All three of these would be considered genuine definitions by almost any mathematician. So anything you say about all definitions must apply to all of them, or you need to go scold the entire field of math for not knowing what a definition is.

Logical positivists like to imagine that all definitions really take the first form, at root, in some indirect way, and that all of language goes back to symbolic logic. But they are wrong: their enterprise fell apart under the weight of formal proof and observational data long ago. Even to the degree mathematics can be reduced to symbolic logic, the axioms of set theory are not strictly unique and therefore purely logical.

Even if that were somehow true, it is not what the practitioners mean when they use the word 'definition'. They do not unwind every case into set theory. Instead, they prove the internal consistency and non-circularity of their definition as a statement, the same way they would prove any other theorem. In most higher mathematics, the very first thing you have to do after giving a definition is prove it is a definition.

When these approaches are used in an experimental science, those proofs of applicability need not be formal mathematics, but are usually made on the basis of observations, instead. I know when I am defining a new species of bird, not because I have formally plumbed the separation criteria between animals and the equivalence classes created by enumerating traits, but because I can look at my clade diagram and see where the overlaps with other species are ruled out. But all of that on the chart is data, not math, and any part might easily have been misinterpreted.

Then when the observation offered instead of formal proof is challenged by new data the definition is tested by experimental evidence.