3
$\begingroup$

My little nephew asked me a question about biased/unbiased samples in which is teachers answer is something I disagree with to say the least (I don't agree with the assumption made by the teacher nor the lack of critical thinking within his answer).

The question is, "If Johnny took a handful of photographs and threw them up in the air, and selected 7 that landed on the rug- would this be a biased sampling?" The teachers answer is simply, "No, each photograph has an equal chance of landing on the rug therefore it is an unbiased sample."

I couldn't disagree more. The equal probability of each photographs landing on the rug has many dependent factors, how big is the rug compared to the room, where are you in the room when tossing the photos in the air, what is your current position relative to the rug, in what way are you tossing the photos into the air, and how were you holding them when do did it...

In fact I think we could continue to give factors that would be relevant, was the window open and a breeze coming through, in what direction etc, but in only a few case of the vastly many are there the situation where the probability of landing on the rug is equal for all photographs given that the size of the rug doesn't take up the entire room.

Since the question give very little to no information required to actually determine if the sampling would be biased or unbiased, can we not infer this is likely a biased sample since perfect circumstances would be needed in the room to guarantee each photograph having an equal chance of landing on the rug (because there are infinitely more ways the perfect circumstance could fail to exist if the rug isn't as big as the room)?

My nephew couldn't follow what I was trying to explain so I gave him this to think about.

If I stood in a big room with the rug across the room and toss the pictures in the air, there is zero probability that any will land on the rug.. I repeat this each time, inching closer to the rug, until eventually seven cards landed on the rug. Now just because the sample amount you wish to take has finally landed on the rug- is that enough to guarantee that all photographs tossed up in air during this iteration of the experiment had an equal probability of landing on the rug?

If given the situation where there is enough information loss so that determining the equal distribution of probabilities for an event to occur is impossible then isn't it reasonable to assume there there may exist some bias in the result?

$\endgroup$
4
  • $\begingroup$ If you can't predict what will fall on the rug and what won't, isn't that equivalent to a random sample? That is, there are non-random phenomena that can be modeled effectively as random. All models are wrong, but some are useful. $\endgroup$
    – soakley
    Commented Dec 20, 2014 at 23:47
  • $\begingroup$ @soakley: I like to view a random sample as a guarantee that in an event each item had an equal probability of occurring. In your example non-random phenomena, pseudo random number generation, or other seemingly chaotic behavior (like weather patterns) random models shouldn't model these systems well. For instance Markov chains can accurately predict weather within a 5-7 day forecast, pseudo random number generators can be cracked, and the truth behind non-random phenomena can be described given enough empirical data but treating any of these as random models would mask their natural behavior. $\endgroup$
    – seeksUnderstanding
    Commented Dec 21, 2014 at 1:34
  • $\begingroup$ I'm thinking of something much simpler. Toss a coin. In theory it's all physics and perhaps someday we will have a way to predict what the outcome will be. In the meantime I will make my guesses using pseudo-random numbers. It doesn't mask the natural behavior to think of the result as random. As Monty Python says, "it's only a model." $\endgroup$
    – soakley
    Commented Dec 21, 2014 at 1:53
  • $\begingroup$ @soakley: In theory yes, the certainty of a coin toss is intractable given all the data, but if the coin is fair it is hardly random. The coin only acts random in large samples and a coin flip is now be usually not used as a randomizing technique. In fact for large samples a fair coin as three outcomes, heads, tails, and edge. Because the each outcome is not equally likely it isn't "truly" a random event in my opinion. For an event to be random, for no matter how many iterations occur, any outcome needs to be equally as likely. But for all practical purposes you're right. $\endgroup$
    – seeksUnderstanding
    Commented Dec 21, 2014 at 2:16

2 Answers 2

Reset to default
2
$\begingroup$

You are right in saying that the teacher did not strictly define the experiment and thus it is hard to tell whether or not there is a bias. For example you could also ask was the selection process random. However I think the idea the teacher was trying to get across was that if I have $n$ objects and randomly partition the set into $A$ and $B$ with the same probability for each object of being in either set. Then I take set $A$ and repeat the process into sets $X$ and $Y$, such that set $X$ has 7 objects and $Y$ has the remainder. Is set $X$ an unbias sample of the original set of objects. In which case the answer is yes. The analogy just wasn't ideal.

$\endgroup$
4
  • $\begingroup$ I understnad that the analogy isn't ideal but two things- "n objects and RANDOMLY partition the set into A and B ..." This is exactly the point, there exists no reason to believe that n entities were randomly partitioned into two sets where each entity had an equal probability of being in each set. Also, if n entities were partitioned into two sets, if at least one of the sets were built or constructed with a bias, even if I randomly choose entities from only one of the two sets my sampling will be biased. So I still completely disagree with what the teacher was trying to get at. $\endgroup$
    – seeksUnderstanding
    Commented Dec 21, 2014 at 1:52
  • $\begingroup$ @seeksUnderstanding I totally see where you are coming from. I am not entirely sure that the teacher is to blame though as it seems to be the way the education system is going - they are will sacrifice rigour for the sake of trying to build intuition. There are many ways of stating this problem but I think teachers are afraid that student won't be able to relate this to "real world" scenarios. I think it is ok to do as long as they put in the caveats that allow the students to make fully informed conclusions. In this case those caveats were missing. $\endgroup$
    – j__
    Commented Dec 21, 2014 at 1:58
  • $\begingroup$ I agree, looking at the other examples he had it was clear as day what was biased and not, this one simply was an edge case. I would have loved for him to be able to answer free for to state that it couldn't be determined and that he needed more information and why he needed it etc. Randomness is a little over middle schooler's heads I think, and even mine from time to time. $\endgroup$
    – seeksUnderstanding
    Commented Dec 21, 2014 at 2:08
  • $\begingroup$ @seeksUnderstanding to be honest I think he will do fine - keep asking him to question and understand and he will do more than fine! If you are satisfied with our joint conclusion please accept the answer :) $\endgroup$
    – j__
    Commented Dec 21, 2014 at 2:14
1
$\begingroup$

I understand what you're trying to point out about the factors, but I have to say that I agree with teacher, the factors you mentioned do affect the experiment but without being directly biased to a certain photograph/s since the photos are randomaly held as it's not mentioned otherwise thus I think that it's an unbiased experiment.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .