For example using a neural network to predict a coin toss. Can a trained neural network to predict it with more than 50% accuracy?
8
-
13$\begingroup$ The short answer is "No." :) $\endgroup$– David HoelzerCommented Sep 1, 2022 at 21:12
-
2$\begingroup$ Could you give more details? Do you have any information about the process of the event before it starts that could feasibly be used to make a prediction, as opposed to simply knowing that the event has been observed in the past (or is expected due to its nature) to occur with $p=0.5$? For instance, are sequential events not independent (a theoretical coin toss is independent of previous coin tosses)? $\endgroup$– Neil SlaterCommented Sep 1, 2022 at 22:37
-
$\begingroup$ @NeilSlater Knowing more information also does not prove that the additional information is related to the probability of the event occurring, as I know the quality of the dice, the material from which it is made, etc. I put this information into the neural network and it doesn't increase the accuracy either. $\endgroup$– huangCommented Sep 1, 2022 at 22:44
-
$\begingroup$ If you're evaluating your neural network on the same set of data used to train the model, you could definitely get higher than 50% "accuracy". But, the accuracy would go down to 50% once evaluated on unseen data if the event is truly random (i.e. truly unrelated to any features fed into the neural network). $\endgroup$– clementzachCommented Sep 2, 2022 at 14:25
-
2$\begingroup$ @clementzach True randomness may be a false proposition, since we can't find features that accurately predict it, it's true randomness, and once we find features that accurately predict it, it's no longer true randomness. $\endgroup$– huangCommented Sep 2, 2022 at 14:31
|
Show 3 more comments
6 Answers
$\begingroup$
$\endgroup$
6
This is a question of marginal vs. conditional distribution
The marginal distribution of the coin may be a Bernoulli random variable with 50% probability for either outcome. However, the conditional distribution of the outcome given information about other factors (e.g. the angle, throw height, ... see other answers) may look entirely different. Provided these features determine the outcome in some way, a neural network can absolutely predict the outcome with more than 50% accuracy.
A neural network could not exceed 50% accuracy, if
- The information determining the throw outcome is not available
- The function is of a nature that can not be learnt by the neural network
- The coin toss is truly random
A coin toss is often used as a casual example of a "truly random" event, so in this sense the answer to your question is "No". In reality however, it is very hard to find any truly random events (at least outside quantum mechanics), which is why random number generation is a big challange and neural networks can predict a lot of things.
-
$\begingroup$ What nature of function can prevent neural networks from learning? $\endgroup$– huangCommented Sep 2, 2022 at 14:19
-
1$\begingroup$ @JoeHuang For example, a neural network with only linear layers can only predict linear functions. (Nobody uses this type of neural network). Or the neural network could be too small for the complexity of the prediction. $\endgroup$ Commented Sep 2, 2022 at 14:22
-
2$\begingroup$ @JoeHuang, another example is the sorts of functions you'd find in modern cryptography. Although they can support some non-linearities, a neural network still relies on at least local smoothness for gradient descent and generalization. Hash or encryption functions, by design, do not have that. (In principle a sufficiently large neural network could encode such a function, just by memorising the mapping, but it wouldn't do any better than a lookup table.) $\endgroup$– JosiahCommented Sep 2, 2022 at 21:48
-
2$\begingroup$ This is the best answer, hands down. The question cannot be answered with "Yes" or "No" without knowing the nature of randomness. I find it misleading to even begin or end any answer with such a definitive summarization, even if it is clarified in the details (which not everybody reads carefully after seeing a simple, bold answer). $\endgroup$ Commented Sep 3, 2022 at 18:02
-
$\begingroup$ What's the logical nature of your bulleted list? Do all these conditions need to be satisfied, or any of them? $\endgroup$ Commented Sep 7, 2022 at 9:00
$\begingroup$
$\endgroup$
2
No.
If there are no patterns, relations or correlations in your data, AI can do nothing to improve what essentially is just guessing.
My last 5 tosses were Heads, Tails, Tails, Heads, Tails. Can you predict the next toss outcome? How would you explain your guess? If you give AI this same data, it cannot do better than just guessing.
The question changes if you have data that is related to the outcome of the coin toss, such as the direction and force the coin was tossed before it lands. In this case, it isn't "an event with a statistical probability of 50%" anymore. If you measured everything perfectly, you could have 99.9% accuracy on what the outcome of the coin toss would be.
AI can only produce accurate results if a super smart human could theoretically also produce accurate results.
-
$\begingroup$ I do agree with the first part (no patterns = guessing), but I don't think the second part is strictly speaking correct. You don't need more types of data (like force or direction), you just need any patterns at all. While I can't say much from your dataset of five throws (neither can AI), if you gave me results of many more of your throw, it might show that you have a slight tendency for heads. Maybe that's the side you intuitively put the coin on your hand and you don't throw with many rotations. Maybe the coin is not perfectly balanced. $\endgroup$ Commented Sep 7, 2022 at 8:55
-
$\begingroup$ I'm not saying that the probability of a right prediction will be 90% in such case, but it still might be some number above like 50,01% (which is > 50%) $\endgroup$ Commented Sep 7, 2022 at 8:56
$\begingroup$
$\endgroup$
7
YES
If you obtain information about the force and angle of the throwers thumb striking the coin at release, that would give insight into how many times the coin would be expected to rotate. Combine this with what faces up when the coin releases, and you should be able to do better than 50/50.
I don’t have a firm source (perhaps there is something on Skeptics), but it seems that people have trained themselves to flip coins to reliably land on one of the sides, so there are some features that dictate how the coin rotates.
Really, this is kind of the point of regression. You think some process has a 50/50 chance of the two outcomes, but once you know a bit more (features), you can sharpen that estimate. Formalizing this mathematically involves the conditional vs marginal distribution discussed in the answer by Scriddie.
-
3$\begingroup$ this is cherry picking an example. To be more precise, I think OP's example relates more to e.g. a Random Variable $X \sim \mbox{Bernoulli}(0.5)$, in which case a NN cannot predict $x \sim X$ with accuracy (significantly) different to 50%. $\endgroup$– DavidCommented Sep 2, 2022 at 9:23
-
1$\begingroup$ @DavidIreland Maybe, but the whole point of doing a regression model like a neural network is to take a process that seems unpredictable (e.g., flipping a coin) and use related information (e.g., force of the flip) to reliably make better predictions. $\endgroup$– DaveCommented Sep 2, 2022 at 11:30
-
$\begingroup$ This does not predict statistical probability. AI may highlight patterns that imply deviation from pure luck, e.g. a roulette wheel being imbalanced or a person being more likely to spin full circles with the coin, but this is not of statistical significance, rather priorly unknown factors that cause deviations from expected probability. $\endgroup$ Commented Sep 2, 2022 at 12:35
-
$\begingroup$ @user3819867 That’s what regression is: using outside factors to predict something better than you could without that outside factor. If you’re unclear about this, you might consider asking for clarification on a statistics website like Cross Validated Stack Exchange. $\endgroup$– DaveCommented Sep 2, 2022 at 12:39
-
3$\begingroup$ @Dave As you say Dave, this is an issue of statistics. You are assuming the existence of the extra information that you can condition on knowing. Assume this extra information is denoted by $Z$, then you're actually proposing to learn approximate the distribution of $X | Z$, which is a totally different random variable to simply $X$. So, again, referring to my original comment, it depends on what exactly OP is asking. (If this is confusing, perhaps you can ask on the CVSE that you linked to earlier). $\endgroup$– DavidCommented Sep 2, 2022 at 14:01
$\begingroup$
$\endgroup$
1
You need to ask yourself, what is the limiting factor in the accuracy for whatever you are trying to predict.
- If the limiting factor is in the quality of the algorithm being used to calculate the prediction, then perhaps you could find a better algorithm that would improve the accuracy.
- If the limiting factor is in the very nature of the problem itself,
such as a coin flip, then there is no method of calculation that
could improve the accuracy.
-
$\begingroup$ Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. $\endgroup$– Community BotCommented Sep 2, 2022 at 6:14
$\begingroup$
$\endgroup$
Although the question is a little vague, I'll treat it as a statement about the mapping of inputs and outputs in the underlying random process - no matter what conditions/inputs/features we observe, there is not a consistent mapping from input to output. A statistical probability of 50% suggests in two cases with identical inputs, we may find different outputs. A traditional deterministic neural network cannot do this, as it is really just a mathematical function, which by definition maps every possible input to exactly one output - it is not possible to use the same inputs and get different outputs. Because of this, a deterministic neural network can't achieve more than 50% accuracy in the long run in this case. No matter what set of features is input, there are in reality two possible outcomes, but the neural network can only return one outcome for any particular input. On average, the neural network will return the correct output only half the time - it can't achieve more than 50% accuracy.
$\begingroup$
$\endgroup$
If a neural network was only able to be as reliable as random guessing, they wouldn't be much use!
Let's suppose there's an election on, and the result is finely balanced between the yellow party and the purple party. At a top level, 50% of people will vote for each colour. If you know nothing else about the people, "Who will the next person in the polling station vote for?" is intrinsically an even guess.
It would still be possible to use a neural network (or a decision tree, or a human!) to predict at much better than 50% if you have additional input. For example, if you look at them and can read their apparent wealth, race, gender presentation, or whether they come accompanied or alone, it may then be possible to identify membership of a sub-population which is more likely to vote yellow.
The fundamental limit on performance isn't the baseline 50:50 probability. Instead it is the component which is caused by some inputs (which may but doesn't have to be true randomness) that are simply not available to the network. Suppose for example that there's a 5% chance that someone has been bribed to flip their vote and the network can't know that. In this case it won't get to more than 95% reliable predictions, but can still do much better than 50% with demographic data.
