Order Of Growth complicated for loops

Question

For the following code fragment, what is the order of growth in terms of N?

int sum = 0;
for (int i = 1; i <= N; i = i*2)
  for (int j = 1; j <= N; j = j*2)
    for (int k = 1; k <= i; k++)
        sum++;

I have figured that there is lgN term, but I am stuck on evaluating this part : lgN(1 + 4 + 8 + 16 + ....). What will the last term of the sequence be? I need the last term to calculate the sum.

Answer 1

You have a geometric progression in your outer loops, so there is a closed form for the sum of which you want to take the log:

1 + 2 + 4 + ... + 2^N = 2^(N+1) - 1

To be precise, your sum is

1 + ... + 2^(floor(ld(N))

with ld denoting the logarithm to base 2.

The outer two loops are independent from each other, while the innermost loop only depends on i. There is a single operation (increment) in the innermost loop, which means that the number of visits to the innermost loop equals the summation result.

  \sum_i=1..( floor(ld(N)) ) {
      \sum_j=1..( floor(ld(N)) ) {
          \sum_k=1..2^i { 1 }
      }
  }

    // adjust innermost summation bounds   
= \sum_i=1..( floor(ld(N)) ) {
      \sum_j=1..( floor(ld(N)) ) {
          -1 + \sum_k=0..2^i { 1 }
      }
  }

    // swap outer summations and resolve innermost summation
= \sum_j=1..( floor(ld(N)) ) {
      \sum_i=1..( floor(ld(N)) ) {
          2^i
      }
  }

   // resolve inner summation
= \sum_j=1..( floor(ld(N)) ) {
      2^(floor(ld(N)) + 1) - 2
  }

   // resolve outer summation
= ld(N) * N - 2 * floor(ld(N))

This amounts to O(N log N) ( the second term in the expression vanishes asymptotically wrt to the first ) in Big-Oh notation.

Answer 2

To my understanding, the outer loop will take log N steps, the next loop will also take log N steps, and the innermost loop will take at most N steps (although this is a very rough bound). In total, the loop has a runtime complexity of at most ((log N)^2)*N, which can probably be improved.