3. Asymptotic Analysis
Using Limits
Use of basic definition for determining the asymptotic behavior is often awkward. It
involves ad hoc approach or some kind of manipulation to prove algebraic relations.
Calculus provides an alternative method for the analysis. It depends on evaluating
the following limit.
f(n)
lim = c
n→∞ g(n)
where f(n) is a growth function for an algorithm and g(n) is a standard function
Depending upon the value c , the relation between f(n) and g(n) can be expressed in terms
of asymptotic notations. In most cases it is easier to use limits, compared to basic method, to
determine asymptotic behavior of growth functions.
It will be seen that the Calculus notation n→ ∞ is equivalent to the algebraic condition
for all n ≥ n0 . Either of these conditions implies large input
4. O-Notation Using Limit
Definition
If f(n) is running time of an algorithm and g(n) is some standard growth function such
that
f(n)
lim —— = c , where c is a positive constant such that 0 ≤ c < ∞
n → ∞ g(n)
then f(n) = O ( g(n) )
Note that infinity is excluded from the range of permissible values for the constant c
6. O-Notation
Examples
Example(3): lg n = O(n)
lim
n → ∞
lg n
n
= ∞ / ∞ ( Need to apply the L’ Hopital Rule )
In order to compute differential of lg n we first convert binary logarithm to natural logarithm.
Converting lg n (binary log ) to ln( n) ( natural log) , by the using formula lg n = ln n / ln 2
lim
lg n
= (ln n) (Converting to natural log)
n → ∞ n (ln 2) n
lim
n → ∞
1
ln 2. n
( Differentiating numerator and denominator)
= 0 ( Evaluating limits)
Therefore, lg n = O(n)
7. O-Notation
Examples
Example(4): n2 = O(2n)
lim
n → ∞
n2
=∞ / ∞ ( Need to apply the L’ Hopital Rule )
2n
Since d ( 2n ) = ln 2 . 2n, ( Calculus rule for differentiating the exponential functions)
dn
lim 2n (Differentiating numerator and denominator )
n → ∞ ln 2 2n
= ∞ / ∞ (Need again to apply the L’Hopital rule)
lim
n → ∞
2
(ln2)2. 2n
(Again differentiating numerator and denominator )
= 0 (Evaluating the limits )
Therefore, n2 = O(2n)
8. Ω-Notation Using Limit
Definition
If f(n) is running time of an algorithm and g(n) is some standard growth function such
that
f(n)
lim —— = c , where c is a constant, such that 0 < c ≤ ∞
n → ∞ g(n)
then f(n) = Ω( g(n) ).
Note that zero value for the constant c is excluded from the permissible range, but
infinity is included.
10. Ө-Notation Using Limit
Definition
If f(n) is running time of an algorithm and g(n) is some standard growth function such
that
f(n)
lim —— = c , where c is a constant, such that 0 < c < ∞
n → ∞ g(n)
then f(n) = Ө ( g(n) ).
Note that zero value and infinity are excluded from the permissible values for the
range of constant c
11. Θ-Notation
Examples
Example(1): 45n3 - 3n2 - 5n+ 20 = θ(n3)
lim
n → ∞
45n3 -3n2 - 5n + 20
n3
= 45-3 / n -5 / n2 +20/n3
= 45 - 0 – 0 + 0
=45 (non-zero constant)
Therefore, 45n3 - 3n2 - 5n+ 20 = θ(n3)
Example(2): n lg n + n + n2 = θ(n2)
lim
n → ∞
n lg n + n + n2
n2
= lg n / n +1/n +1
= 0 +0 + 1
=1 (non-zero constant)
Thus, n lg n + n + n2 = θ(n2)
12. Θ-Notation
Examples
Example(3): 45n3 - 3n2 - 5n+ 20 ≠ θ(n4)
lim
n → ∞
45n3 -3n2 - 5n + 20
n4
= 45 / n - 3 / n2 - 5 / n3 + 20 / n4
= 0 - 0 - 0 + 0
= 0 (zero is excluded from the permissible range for Ө-notation)
Therefore, 45n3 - 3n2 - 5n+ 20 ≠ θ(n3)
Example(4): n lg n + n ≠ θ(n2)
lim
n → ∞
n lg n + n
n2
= lg n / n + 1/n ( Using the L’Hopital Rule to evaluate the limit of lgn /n)
= 0 +0
=0
Thus, n lg n + n ≠ θ(n2)
13. o-Notation Using Limit
Definition
If f(n) is running time and g(n) is some standard growth function such that
f(n)
lim —— = 0 ( zero )
n → ∞ g(n)
then f(n) = o( g(n) ) (Read f (n) is small-oh of g(n))
o( g(n) ) is referred to as the loose upper bound for f(n)
15. o-Notation
Examples
Example(3): lg n = o (n)
lim lg n
=
∞ ( Need to use L’Hopital Rule)
n → ∞ n ∞
lim ln n
( Converting to natural log ,lg n = log2 n =log e n / loge 2 )
n → ∞ n ln 2
lim 1
( Differentiating numerator and denominator )
n → ∞ n ln2
= 0
Therefore, lg n = o (n)
16. ω-Notation
Definition
If f(n) is running time and g(n) is some standard growth function such that
f(n)
lim —— = ∞
n → ∞ g(n)
then f(n) = ω(g(n)) (Read f(n) is small-omega of g(n))
ω(g(n)) is referred to as the loose lower bound for f(n)
18. ω-Notation
Examples
Example(3): n! =ω(2n)
lim
n → ∞
n!
= ∞ / ∞ (Need to use the LHpoital Rule
2n
The function n! cannot be differentiated directly. We first use Stirling’s approximation:
lim
n! = √ 2πn ( n /e ) n for large n
n!
n → ∞
=
2n
√ 2πn ( n / e ) n
2n
= √ 2πn ( n /2 e ) n
= ∞
Thus, n! = ω( 2n )
19. Asymptotic Notation
Summary
Let f(n) be time complexity and g(n) standard function, such that lim
n→∞
f(n)
——— = α
g(n)
Table below summarizes the asymptotic behavior of f(n) in terms of g(n)
Notation Using Basic Definition Using Limits Asymptotic Bound
f(n)=O( g(n) ) f(n) ≤ c.g(n) for some c>0, and n ≥ n0 0 < α < ∞ upper tight
f(n)=o( g(n) ) f(n) <c.g(n) for all c>0, and n ≥ n0 α = 0 upper loose
f(n)=Ω( g(n) ) f(n) ≥ c.g(n) for some c>0 and n ≥ n0 0 < α ≤ ∞ lower tight
f(n)=ω( g(n) ) f(n) > c.g(n) for all c>0 and n ≥ n0 α = ∞ lower loose
f(n)=θ( g(n) ) c1.g ≤ f(n) ≤ c2.g(n) for some c1>0, c2>0
and n ≥ n0
0 < α < ∞ tight
20. Asymptotic Set Notations
Definitions
The asymptotic notation can also be expressed in Set notations by using the limits
f(n)
Let f(n) be time complexity and g(n) standard function, such that lim ——— = c
where c is zero, positive constant or infinity.
n→∞ g(n)
The set notations for the asymptotic behavior is are defined as follows
(i) O(g(n) ) = { f(n): 0 ≤ c < ∞ }
(ii) Ω( g(n) ) = { f(n) : 0 < c ≤ ∞ }
(iii) Ө( g(n) ) = { f(n): 0 < c < ∞ }
(iv) o( g(n) ) = { f(n): c = 0 }
(v) ω( g(n) ) = { f(n): c = ∞ }
21. Asymptotic Set Notations
Examples
Example (1): Let f(n)O ( g(n) ), f(n) Ω( f(n) ) . Then
O(g(n) ) = { f(n): 0 ≤ c < ∞ } (definition of O-notation)
Ω( g(n) ) = { f(n) : 0 < c ≤ ∞ } (definition of Ω-notation )
O(g(n) ) ∩ Ω ( g(n) ) = { f(n) : 0 < c < ∞ } ( Performing Set intersection operation )
= Ө (g(n)) (definition of Ө-notation }
Thus, O(g(n) ) ∩ Ω ( g(n) ) = Ө (g(n))
Example (2):Suppose f(n) o(g(n)), and f(n)ω(g(n)). Then
o(g(n)) = { f(n): c = 0 } (definition of o-notation)
Ω(g(n))= { f(n): c = ∞ } (definition of ω-notation)
Therefore, o(g(n)) ∩ ω( g(n)) =φ ( Empty set )
23. Arithmetic Summation
Asymptotic Behavior
The sum of first of n terms of arithmetic series is :
1 + 2 + 3………..+n = n(n+1)/2
Let f(n)= n(n+1)/2
and g(n)= n2
f(n)
lim
n(n+1)/2 n2 /2 + n/2
= = = 1 /2 + 1/2n =1/2 + 0=1/2
n→∞ g(n) n2 n2
Since the limit is non-zero and finite it follows
f(n) = θ( g(n)) = θ(n2)
Or, 1 + 2 + ………..+ n = θ(n2)
24. Geometric Summation
Asymptotic Behavior
The asymptotic behavior of geometric series
1 +r + r2+……..+rn
depends on geometric ratio r. Three cases need to be considered
Case r > 1: It can be shown that sum f(n) of first n terms is as follows
rn+1 - 1
f(n) = 1 +r + r2+……..+rn = —————
r - 1
Case r = 1: This is trivial
f(n)= 1 + 1 + 1+ ……+1 = n =Ө(n)
Case r < 1: It can be shown that
2 n
1 - rn+1
f(n) = 1 +r + r +……..+r = —————
1 - r
The asymptotic behavior in first case and third case is explored by computing limits.
25. Geometric Summation
Case r > 1
2 n
rn+1 - 1
Let f(n) = 1 +r + r
Let g(n)= rn
+……..+r = —————
r - 1
Consider, the limit
lim
n→∞
f(n)
g(n)
rn+1 - 1
=
(r - 1).rn
r - 1/rn
=
(r - 1)
Since r>1 , 1/rn →0 as n→ ∞
lim
n→∞
f(n)
g(n)
=
r
(r - 1)
>0, since r > 1
Therefore, f(n) = θ(rn) for r>1
Or, 1 +r + r2+……..+rn = θ(rn) for r > 1
26. Geometric Summation
Case r < 1
2 n
rn+1 - 1
Consider 1 +r + r +……..+r = —————
r - 1
Let g(n)=c where c is some positive constant
Taking the limit
lim
f(n) rn+1 - 1
=
1- rn+1
=
n→∞ g(n) (r - 1).c (1- r )c
Since r<1 , rn+1 →0 as n→ ∞
lim
n→∞
f(n)
g(n)
=
1
(1 - r). c
>0, since r < 1
Therefore, f(n)= θ(g(n))=θ(c) = θ(1) for r <1
Or, 1 +r + r2+……..+rn
=θ(1) for r <1
27. Geometric Summation
Asymptotic Behavior
From the preceding analysis it follows that the asymptotic behavior of geometric summation is
S(n) =1+r+r2+.........+ rn =
Ө(rn) when r>1
Ө(1) when r<1
When r <1, the largest term in the geometric summation would be 1 ( the first term) , and
S(n) =Ө(1) . On the other hand if r>1, the largest term would be rn ( the last term), and
S(n)=Ө(rn) In the light of this observation it can said that asymptotic behavior of geometric
summation is determined by the largest term i.e S(n)=Ө (largest term).
Example(1): The geometric series
1+21+ 22 +….+ 2n
has geometric ratio r=2>1 ,
Thus, 1+21+ 22 +….+ 2n =Ө(2n) , ► Ө(largest term)
Example(2): The geometric series
1+(2/3)1+ (2/3)2 +….+ (2/3)n
has geometric ratio r=2/3 <1 ,
Thus, 1+(2/3)1+ (2/3)2 +….+ (2/3)n =Ө(1) , ►Ө(largest term)
28. Logarithm Summation
Asymptotic Behavior
The logarithmic series has the summation
lg(1)+lg(2)+ lg(3)+….. +lg(n)
Let f(n) = lg(2)+ lg(3)+….+.lg(n) = lg(2.3……..n) = lg( n!)
and g(n) = n lg n
lim
n→∞
f(n)
g(n) =
lg n!
n lg n =
lg ( √(2πn)(n/e)n )
n lg n
(Using Stirling’s approximation)
Now, lg ( √(2πn)(n/e)n ) = (1+lg π + lg n)/2 + n lg n - n lg e , therefore
lim lg ( √(2πn)(n/e)n ) = ( 1+lg π + lg n ) /(2 nlg n) + 1 - lg e / lg n = (0+ 1- 0)=1
n→∞ n lg n
Since limit is non-zero and finite, it follows
f(n)= θ(g(n))
Or, lg(1)+lg(2)+ lg(3)+….. +lg(n) = θ(n lg n) …
29. Harmonic Summation
Asymptotic Behavior
The sum of first n terms of Harmonic series is
1+ 1/2 + 1/3+…..+1/n
Let f(n) = 1+ 1/2 + 1/3+…..+1/n
and g(n) = lg n
lim
n→∞
f(n)
g(n)
1+1/2+1/3+ …..+1/n
=
lg n
It can be shown that 1+ 1/2+ 1/3+…+1/n = lg (n) + γ + 1/2n – 1/12n2+..where γ ≈ 0.5772
lim
n→∞
f(n)
g(n)
lg n + γ + 1/2n – 1/12n2 +…
=
lg n
= 1+0+0-0+0+…=1
Since limit is finite and non-zero, it follows
f(n) = θ( g(n))
Or, 1+ 1/2 + 1/3+…..+1/n = θ (lg n )
