Why is the Fibonacci Sequence Big O(2^n) instead of O(logn)?

Question

I took discrete math (in which I learned about master theorem, Big Theta/Omega/O) a while ago and I seem to have forgotten the difference between O(logn) and O(2^n) (not in the

Answer 1

The other answers are correct, but don't make it clear - where does the large difference between the Fibonacci algorithm and divide-and-conquer algorithms come from? Indeed, the shape of the recursion tree for both classes of functions is the same - it's a binary tree.

The trick to understand is actually very simple: consider the size of the recursion tree as a function of the input size n.

Recall some basic facts about binary trees first:

• The number of leaves n is a binary tree is equal to the the number of non-leaf nodes plus one. The size of a binary tree is therefore 2n-1.
• In a perfect binary tree, all non-leaf nodes have two children.
• The height h for a perfect binary tree with n leaves is equal to log(n), for a random binary tree: h = O(log(n)), and for a degenerate binary tree h = n-1.

Intuitively:

• For sorting an array of n elements with a recursive algorithm, the recursion tree has n leaves. It follows that the width of the tree is n, the height of the tree is O(log(n)) on the average and O(n) in the worst case.

• For calculating a Fibonacci sequence element k with the recursive algorithm, the recursion tree has k levels (to see why, consider that fib(k) calls fib(k-1), which calls fib(k-2), and so on). It follows that height of the tree is k. To estimate a lower-bound on the width and number of nodes in the recursion tree, consider that since fib(k) also calls fib(k-2), therefore there is a perfect binary tree of height k/2 as part of the recursion tree. If extracted, that perfect subtree would have 2^k/2 leaf nodes. So the width of the recursion tree is at least O(2^{k/2}) or, equivalently, 2^O(k).

The crucial difference is that:

• for divide-and-conquer algorithms, the input size is the width of the binary tree.
• for the Fibonnaci algorithm, the input size is it the height of the tree.

Therefore the number of nodes in the tree is O(n) in the first case, but 2^O(n) in the second. The Fibonacci tree is much larger compared to the input size.

You mention Master theorem; however, the theorem cannot be applied to analyze the complexity of Fibonacci because it only applies to algorithms where the input is actually divided at each level of recursion. Fibonacci does not divide the input; in fact, the functions at level i produce almost twice as much input for the next level i+1.

Answer 2

To address the core of the question, that is "why Fibonacci and not Mergesort", you should focus on this crucial difference:

• The tree you get from Mergesort has N elements for each level, and there are log(n) levels.
• The tree you get from Fibonacci has N levels because of the presence of F(n-1) in the formula for F(N), and the number of elements for each level can vary greatly: it can be very low (near the root, or near the lowest leaves) or very high. This, of course, is because of repeated computation of the same values.

To see what I mean by "repeated computation", look at the tree for the computation of F(6):

^{_{Fibonacci tree picture from: http://composingprograms.com/pages/28-efficiency.html}}

How many times do you see F(3) being computed?

Answer 3

Consider the following implementation

int fib(int n)
{
    if(n < 2)
      return n;
    return fib(n-1) + fib(n-2)
}

Let's denote T(n) the number of operations that fib performs to calculate fib(n). Because fib(n) is calling fib(n-1) and fib(n-2), it means that T(n) is at least T(n-1) + T(n-2). This in turn means that T(n) > fib(n). There is a direct formula of fib(n) which is some constant to the power of n. Therefore T(n) is at least exponential. QED.

Answer 4

Merge sort time complexity is O(n log(n)). Quick sort best case is O(n log(n)), worst case O(n^2).

The other answers explain why naive recursive Fibonacci is O(2^n).

In case you read that Fibonacci(n) can be O(log(n)), this is possible if calculated using iteration and repeated squaring either using matrix method or lucas sequence method. Example code for lucas sequence method (note that n is divided by 2 on each loop):

/* lucas sequence method */
int fib(int n) {
    int a, b, p, q, qq, aq;
    a = q = 1;
    b = p = 0;
    while(1) {
        if(n & 1) {
            aq = a*q;
            a = b*q + aq + a*p;
            b = b*p + aq;
        }
        n /= 2;
        if(n == 0)
            break;
        qq = q*q;
        q = 2*p*q + qq;
        p = p*p + qq;
    }
    return b;
}

Answer 5

To my understanding, the mistake in your reasoning is that using a recursive implementation to evaluate f(n) where f denotes the Fibonacci sequence, the input size is reduced by a factor of 2 (or some other factor), which is not the case. Each call (except for the 'base cases' 0 and 1) uses exactly 2 recursive calls, as there is no possibility to re-use previously calculated values. In the light of the presentation of the master theorem on Wikipedia, the recurrence

f(n) = f (n-1) + f(n-2)

is a case for which the master theorem cannot be applied.

Answer 6

With the recursive algo, you have approximately 2^N operations (additions) for fibonacci (N). Then it is O(2^N).

With a cache (memoization), you have approximately N operations, then it is O(N).

Algorithms with complexity O(N log N) are often a conjunction of iterate over every item (O(N)) , split recurse, and merge ... Split by 2 => you do log N recursions.