Role of lower order terms in big O notation

一笑奈何 提交于 2019-12-18 18:08:37

问题


In big O notation, we always say that we should ignore constant factors for most cases. That is, rather than writing,

3n^2-100n+6

we are almost always satisfied with

n^2

since that term is the fastest growing term in the equation.

But I found many algorithm courses starts comparing functions with many terms

2n^2+120n+5 = big O of n^2

then finding c and n0 for those long functions, before recommending to ignore low order terms in the end.

My question is what would I get from trying to understand and annalising these kinds of functions with many terms? Before this month I am comfortable with understanding what O(1), O(n), O(LOG(n)), O(N^3) mean. But am I missing some important concepts if I just rely on this typically used functions? What will I miss if I skipped analysing those long functions?


回答1:


Ever have intermediate steps in your work? That is what this likely is as when you are computing a big O, chances are you don't already know for sure what the highest order term is and thus you keep track of them all and then determine which complexity class makes sense in the end. There is also something to be said for understanding why the lower order terms can be ignored.


Take some graph algorithms like a minimum spanning tree or shortest path. Now, can just looking at an algorithm you know what the highest term will be? I know I wouldn't and so I'd trace through the algorithm and collect a bunch of terms.

If you want another example, consider Sorting Algorithms and whether you want to memorize all the complexities or not. Bubble Sort, Shell Sort, Merge Sort, Quick Sort, Radix Sort and Heap Sort are a few of the more common algorithms out there. You could either memorize both the algorithm and complexity or just the algorithm and derive the complexity from the pseudo code if you know how to trace them.




回答2:


Let's first of all describe what we mean when we say that f(n) is in O(g(n)):

... we can say that f(n) is O(g(n)) if we can find a constant c such that f(n) is less than c·g(n) or all n larger than n0, i.e., for all n>n0.

In equation for: we need to find one set of constants (c, n0) that fulfils

f(n) < c · g(n), for all n > n0,                        (+)

Now, the result that f(n) is in O(g(n)) is sometimes presented in difference forms, e.g. as f(n) = O(g(n)) or f(n) ∈ O(g(n)), but the statement is the same. Hence, from your question, the statement 2n^2+120n+5 = big O of n^2 is just:

f(n) = 2n^2 + 120n + 5

a result after some analysis: f(n) is in O(g(n)), where

    g(n) = n^2

Ok, with this out of the way, we look at the constant term in the functions we want to analyse asymptotically, and let's look at it educationally, using however, your example.


As the result of any big-O analysis is the asymptotic behaviour of a function, in all but some very unusual cases, the constant term has no effect whatsoever on this behaviour. The constant factor can, however, affect how to choose the constant pair (c, n0) used to show that f(n) is in O(g(n)) for some functions f(n) and g(n), i.e., the none-unique constant pair (c, n0) used to show that (+) holds. We can say that the constant term will have no effect of our result of the analysis, but it can affect our derivation of this result.

Lets look at your function as well as another related function

f(n) = 2n^2 + 120n + 5                                        (x)
h(n) = 2n^2 + 120n + 22500                                    (xx)

Using a similar approach as in this thread, for f(n), we can show:

linear term: 

    120n < n^2 for n > 120 (verify: 120n = n^2 at n = 120)    (i)

constant term:

    5 < n^2 for e.g. n > 3 (verify: 3^2 = 9 > 5)              (ii)

This means that if we replace both 120n as well as 5 in (x) by n^2 we can state the following inequality result:

Given that n > 120, we have:
2n^2 + n^2 + n^2 = 4n^2 > {by (ii)} > 2n^2 + 120n + 5 = f(n)  (iii)

From (iii), we can choose (c, n0) = (4, 120), and (iii) then shows that these constants fulfil (+) for f(n) with g(n) = n^2, and hence

result: f(n) is in O(n^2)

Now, for for h(n), we analogously have:

linear term (same as for f(n)) 

    120n < n^2 for n > 120 (verify: 120n = n^2 at n = 120)    (I)

constant term:

    22500 < n^2 for e.g. n > 150 (verify: 150^2 = 22500)      (II)

In this case, we replace 120n as well as 22500 in (xx) by n^2, but we need a larger less than constraint on n for these to hold, namely n > 150. Hence, we the following holds:

Given that n > 150, we have:
2n^2 + n^2 + n^2 = 4n^2 > {by (ii)} > 2n^2 + 120n + 5 = h(n)  (III)

In same way as for f(n), we can, here, choose (c, n0) = (4, 150), and (III) then shows that these constants fulfil (+) for h(n), with g(n) = n^2, and hence

result: h(n) is in O(n^2)

Hence, we have the same result for both functions f(n) and h(n), but we had to use different constants (c,n0) to show these (i.e., somewhat different derivation). Note finally that:

  • Naturally the constants (c,n0) = (4,150) (used for h(n) analysis) are also valid to show that f(n) is in O(n^2), i.e., that (+) holds for f(n) with g(n)=n^2.
  • However, not the reverse: (c,n0) = (4,120) cannot be used to show that (+) holds for h(n) (with g(n)=n^2).

The core of this discussion is that:

  1. As long as you look at sufficiently large values of n, you will be able to describe the constant terms in relations as constant < dominantTerm(n), where, in our example, we look at the relation with regard to dominant term n^2.
  2. The asymptotic behaviour of a function will not (in all but some very unusual cases) depend on the constant terms, so we might as well skip looking at them at all. However, for a rigorous proof of the asymptotic behaviour of some function, we need to take into account also the constant terms.


来源:https://stackoverflow.com/questions/34421620/role-of-lower-order-terms-in-big-o-notation

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!