1
$\begingroup$

I'm having a hard time finding the constants/witnesses $C_0$ and $k_0$ that show $n^d \in O(b^n)$ ($b > 1$ and $d$ is positive). That is $|n^d| \leq C_0|b^n|$ for $n > k_0$

I understand you can prove the Big-O relation through L-Hopital's rule, but I'm mainly interested in the constants/witnesses.

I'm not sure how I could do this since I tried to solve for the intersections but don't know how to, and I'm just really stuck honestly

Kindly please do help. A solution similar to Determining the witnesses (constants $C_0$ and $k_0$) when showing $(log_b n)^c$ is $O(n^d)$ (b > 1 and c,d are positive) would work (doesn't have to be similar though lol)

$\endgroup$

1 Answer 1

2
$\begingroup$

Before you try to prove a statement, the most important thing to ask yourself is:

do I even believe that this statement is true?

Not in the sense of, has some authority figure has told me it's true, but in the sense of: do I, myself, feel with my own soul that this statement is true, or not?

My experience on math.SE has been that students are never trained to ask this question, and if asked in detail would probably admit that they don't even really understand what the statement is saying. And if you don't even really understand what it's saying of course you can't feel whether it's true or not!

So, before we try to prove anything let's try to feel it. The first problem is that there are too many variables which makes it hard to tell what is happening. In fact you are using big-O notation in a way which disguises which variable is the asymptotic variable and where it's going; from context it's $n \to \infty$ but if you just read the statement $n^d \in O(b^n)$ without thinking about it too hard there are just three variables arranged in some unclear way. So, let's pick specific values of $d$ and $b$. This is an infinite collection of statements such as

$$n^3 \in O(2^n)$$ $$n^8 \in O(3^n)$$ $$n^{26} \in O(15^n)$$

etc. etc. The general claim is:

exponentials always grow faster than polynomials.

This requires no calculus to state and it requires no calculus to understand or prove. So, do you believe it's true? Why is it true?

Here's something we can learn from getting more concrete about what is being claimed: it suffices to assume WLOG that $d$ is a positive integer, since we can always round up. That is, if we know that $n^{\lceil d \rceil} \in O(b^n)$ then since $n^d \le n^{\lceil d \rceil}$ it follows that $n^d \in O(b^n)$. This should be clear from looking at the specific numbers above; e.g. once we know that $n^{26} \in O(15^n)$ we also know that $n^{25.534 \dots} \in O(15^n)$ or whatever. (This will simplify an argument later.)

Now let's think. What's the defining feature of an exponential sequence like $15^n$? It's that its values get bigger by a factor (in this case, of $15$) every time we increment $n$. So if we want to understand why polynomials can never grow as fast as an exponential, that suggests we look at the ratio of consecutive terms. I am going to continue to pick specific numbers here to reduce the amount of abstraction happening, so let's take $a_n = n^{26}$ and consider the ratio

$$\frac{a_{n+1}}{a_n} = \frac{(n+1)^{26}}{n^{26}} = \left( 1 + \frac{1}{n} \right)^{26}.$$

What do you notice about this? Here are some values to be really concrete:

$n$ $\left( 1 + \frac{1}{n} \right)^{26}$
$1$ $2^{26} \approx 6.7 \times 10^7$
$2$ $\left( \frac{3}{2} \right)^{26} \approx 3.8 \times 10^4$
$3$ $\left( \frac{4}{3} \right)^{26} \approx 1.7 \times 10^3$
$10$ $\left( \frac{11}{10} \right)^{26} \approx 12$
$15$ $\left( \frac{16}{15} \right)^{26} \approx 5.3$
$50$ $ \left( \frac{51}{50} \right)^{26} \approx 1.7$

It's always getting smaller! In fact as $n \to \infty$ it's clear that $1 + \frac{1}{n} \to 1$ and so does this ratio. But, again, we don't even need the explicit concept of a limit here: the point is that this thing is getting arbitrarily small, and the table even shows us that once $n \ge 10$ it is clearly smaller than $15$. So that is the conceptual explanation for why $n^{26} \in O(15^n)$, and furthermore there's nothing special about these two numbers and the argument is completely general. The general idea, independent of the details of any particular proof, is:

exponentials always grow faster than polynomials because if $a_n$ is a polynomial the ratio $\frac{a_{n+1}}{a_n}$ eventually becomes arbitrarily close to $1$.

Before we move on to the proof, check: is this convincing to you? Do you believe the statement now? It's okay if you don't! You can try to play around with specific values of $d, b$ to gain some intuition here; I suggest picking $b = 2$ and then continually increasing the value of $d$, so for example you can start with $n^2 \in O(2^n)$, compute some numbers to see why it's true, move on to $n^3 \in O(2^n)$, eventually to $n^{10} \in O(2^n)$, as high up as you like.


Now, the technical statement we need to establish to turn the above into a proof is:

For any positive integer $d$ and real $b > 1$, the ratio $\left( 1 + \frac{1}{n} \right)^d$ eventually becomes smaller than $b$, once $n$ is large enough.

This is where I'm going to use that WLOG assumption above that $d$ is a positive integer: there's a completely elementary way to prove this that doesn't require knowing anything about limits or logarithms or anything like that. We can simply perform the binomial expansion:

$$\left( 1 + \frac{1}{n} \right)^d = 1 + \frac{d}{n} + \frac{ {d \choose 2} }{n^2} + \frac{ {d \choose 3} }{n^3} + \dots + \frac{1}{n^d}.$$

For fixed $d$, this is a sum of finitely many terms each of which decays at least as fast as $\frac{1}{n}$, so hopefully it should be really clear now that this gets arbitrarily small. A nice elementary way to rigorously finish the argument from here is to use the very simple inequality ${d \choose k} \le d^k$, which gives

$$\left( 1 + \frac{1}{n} \right)^d \le 1 + \frac{d}{n} + \frac{d^2}{n^2} + \frac{d^3}{n^3} + \dots + \frac{d^d}{n^d}.$$

This is a geometric series! In fact it's harmless to take the corresponding infinite geometric series, which gives the completely elementary inequality

$$\boxed{ \left( 1 + \frac{1}{n} \right)^d \le \frac{1}{1 - \frac{d}{n}} }.$$

Now if we want this to be $\le b$ all we have to do is set $\frac{1}{1 - \frac{d}{n}} = b$ and solve for $n$; this gives $n = \frac{d}{1 - \frac{1}{b}}$, so from this value of $n$ onwards (so we can take $\boxed{ k_0 = \left\lceil \frac{d}{1 - \frac{1}{b}} \right\rceil }$), $n^d$ grows at most as fast as $b^n$. As a sanity check, this behaves in the intuitive way as we change the values of $d$ and $b$: it gets larger if $d$ gets larger (since $n^d$ is growing faster, so it takes longer for the exponential to overtake it) and it also gets larger as $b$ gets smaller (since $b^n$ is growing slower, so again it takes longer to overtake the polynomial).

This tells us that $n^d \le C_0 b^n$ for $n \ge k_0$ and a constant $C_0$ which we can take to be the constant such that this inequality is an equality when $n = k_0$. This gives $\boxed{ C_0 = \frac{k_0^d}{b^{k_0}} }$.

But honestly these explicit expressions make the situation look more complicated than it is; big-O notation is exactly designed to suppress these details so that the underlying asymptotic statement can be made more clearly.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .