#02 (06/18/2025)

Taylor series

If all one wants is the formula for the Taylor series of f(x) about x = a, one can easily derive such a formula by assuming

f(x) = C0 +C1 (x − a) + C2 (x − a)2 + C3 (x − a)3 + …
(1)
where the coefficients can be determined by substituting x = a on the both sides and subsequent differentiation as
f(x) = f(a) + f′(a)(x − a) + f"(a)
2!
 (x − a)2 + f"′(a)
3!
 (x − a)3 + …
(2)
Example: Expand f(x) = exp (−1/x2) about x = 0.
One is tempted to use the Taylor series formula, i.e. f(x) = f(0) + f′(0) x + f"(0)/2! x2 + f"′(0)x3/3! + …. However, it turns out that all the coefficients are zero as shown in class, i.e. f(0) = f′(0) = f"(0) = … = 0 so the Taylor series for e−1/x2 is
exp⎛
⎝ 		− 1
x2
⎞
⎠ 		= 0 + 0 x + 0 x2
2!
 + …
(3)
This leads us to investigate under what condition the Taylor series is valid. In fact, this example is a rather pathetic one as f(z) = e−1/z2 in the complex plane has an essential singularity at z = 0 as shown in class.
We begin with an identity
⌠
⌡ 		x

a 
 f′(x) dx = f(x) − f(a),
(4)
or
f(x) = f(a) + ⌠
⌡ 		x

a 
 f′(x) dx.
(5)
By replacing f′(x) in Eq.(4) by f"(x), one obtains
⌠
⌡ 		x

a 
 f"(x) dx = f′(x) − f′(a),
(6)
or
f′(x) = f′(a) + ⌠
⌡ 		x

a 
 f"(x) dx,
(7)
which is substituted into Eq.(5) to yield
f(x) = f(a) + f′(a) (x − a) + ⌠
⌡ 		x

a 
 ⌠
⌡ 		x

a 
 f"(x) dx dx.
(8)
Using this relation recursively, one obtains
f(x)
=
f(a) + f′(a) (x − a) + f"(a)
2!
 (x − a)2 + f"′(a)
3!
 (x − a)3 + …+ ⌠
⌡ 		x

a 
 ⌠
⌡ 		x

a 
 …⌠
⌡ 		x

a 
 f(n)(x) dx dx …dx
=
f(a) + f′(a) (x − a) + f"(a)
2!
 (x − a)2 + f"′(a)
3!
 (x − a)3 + …+Rn,
(9)
where
Rn =

⌠
⌡ 		x

a 
 ⌠
⌡ 		x

a 
 …⌠
⌡ 		x

a 
n
 
 f(n)(x)

dx dx …dx
n 
 ,
(10)
is called the remainder of the Taylor series. Note that Eq.(9) is an identity which holds regardless of the condition of f(x) (no approximation).
If Rn → 0 as n→ ∞, we say that f(x) can be expanded by the Taylor series. In the example above for f(x) = e−1/x2, Rn does not go to 0 as n→ ∞.
The remainder of the Taylor series, Rn, can be assessed if one assumes that f(n)(x) is bounded as
m ≤ f(n)(x) ≤ M.
(11)
Substituting this into Eq.(10) yields
m (x − a)n
n!
 ≤ Rn ≤ M (x − a)n
n!
 .
(12)
From the inequality above, it follows that there must be a point ξ in (a, x) such that
Rn = f(n)(ξ) (x − a)n
n!
        a ≤ ξ ≤ x.
(13)
When n = 1, Eq.(13) is known as the mean value theory, i.e.
f(x) = f(a) + f′(ξ) (x − a),
(14)
or
f′(ξ) = f(x) − f(a)
x − a
 .
(15)
Exercise: Prove
1
x
 < ln x − ln a
x−a
 < 1
a
 .
(16)

Taylor series examples

Examples

  1. ln (1 + x)
    By integrating the both sides of
    1
    1 + x
    		 = 1 − x + x2 − x3 + x4 − x5 + …,
    		(17)
    one obtains
    ln (1 + x) = x − x2
    2
    		 + x3
    3
    		 − x4
    4
    		 + x5
    5
    		 − ….1
    		(18)
    Note that the convergence range of 1/(1 + x) is |x| < 1 so that the convergence region of ln x is also |x| < 1.
  2. arctan x
    By integrating
    1
    1+x2
    		 = 1 − x2 + x4 − x6 + …,
    		(19)
    one obtains
    arctan x = x − x3
    3
    		 + x5
    5
    		 − x7
    7
    		 +….
    		(20)
  3. Expand ln x about x = 2.

    ln x
    =
    ln ( (x−2)+ 2 )
    =
    ln ⎛
    ⎝     		2 (1+ x − 2
    2
    		 )⎞
    ⎠
    =
    ln 2 + ln ⎛
    ⎝     		1 + x−2
    2
    		⎞
    ⎠
    =
    ln 2 + ⎛
    ⎝     		x − 2
    2
    		⎞
    ⎠     		−
    ⎛
    ⎝     		x − 2
    2
    		⎞
    ⎠     		2

     
    2
    		 +
    ⎛
    ⎝     		x − 2
    2
    		⎞
    ⎠     		3

     
    3
    		 −….
  4. (1 + x)n (binomial expansion)

    (1+x)n = 1+ n x + n (n−1)
    2!
    		 x2 + n (n−1) (n−2)
    3!
    		 x3+….
    		(21)
    (Example):
    3
    √
     
    1003
     
    =
    (1000+3)1/3
    =
    10001/3 (1+0.003)1/3
     ∼ 
    10 ⎛
    ⎝     		1+0.003 × 1
    3
    		⎞
    ⎠
    =
    10.01.

Taylor series for multivariable functions

The Taylor series for a single variable function is rewritten as
f(x)
=
f(a) + f′(a) (x−a) + f"(a)
2!
 (x−a)2+ f"′(a)
3!
 (x−a)3 +…
=
f(a) + ⎛
⎝ 		(x−a) d
dx
⎞
⎠ 		f|a + 1
2!
 ⎛
⎝ 		(x−a) d
dx
⎞
⎠ 		2

 
 f|a + 1
3!
 ⎛
⎝ 		(x−a) d
dx
⎞
⎠ 		3

 
 f|a +…
(22)
When a function, f, has two variables, x and y, the Taylor series for f(x,y) can be rewritten by replacing ( (x−a)d/dx ) by ( (x−a)∂/∂x +(y−b) ∂/ ∂y) as
f(x,y)
=
f(a,b) + ⎛
⎝ 		(x−a) ∂
∂x
 +(y−b) ∂
∂y
⎞
⎠ 		f|(a,b)
+
1
2!
 ⎛
⎝ 		(x−a) ∂
∂x
 +(y−b) ∂
∂y
⎞
⎠ 		2

 
 f|(a,b)
+
1
3!
 ⎛
⎝ 		(x−a) ∂
∂x
 +(y−b) ∂
∂y
⎞
⎠ 		3

 
 f|(a,b)
+
…
(23)

Examples

As noted in class, the formula for the Taylor series can be bypassed in many cases:
  1. Expand sin (x + y) about (0, 0).
    Remembering that sin x = x − x3/3! + x5/5! − …,
    sin (x + y) = (x + y) − (x + y)3
    3!
    		 + (x + y)5
    5!
    		 −…
    		(24)
  2. Expand sin (x + y) about (π, π/2). In this case, the expansion must contain (x − π)(y − π/2) terms.

    sin (x + y)
    =
    sin ⎛
    ⎝     		(x − π) +π+ (y − π
    2
    		 )+ π
    2
    		⎞
    ⎠
    =
    sin ⎛
    ⎝     		(x − π) + (y − π
    2
    		 ) + 3π
    2
    		⎞
    ⎠
    =
    sin ⎛
    ⎝     		(x − π) + (y − π
    2
    		 )⎞
    ⎠     		cos 3π
    2
    		 +cos ⎛
    ⎝     		(x − π) +(y − π
    2
    		 )⎞
    ⎠     		sin 3π
    2
    =
    (−1) cos ⎛
    ⎝     		(x − π) + (y − π
    2
    		 )⎞
    ⎠
    =
    (−1)⎛
    ⎜
    ⎝     		1−
    ⎛
    ⎝     		(x − π) + (y − π
    2
    		 )⎞
    ⎠     		2

     
    2!
    		 +
    ⎛
    ⎝     		(x − π) + (y − π
    2
    		 )⎞
    ⎠     		4

     
    4!
    		 − …⎞
    ⎟
    ⎠     		,
    where sin (A + B) = sin A cos B + cos A sin B was used.
  3. Expand x2 y about (1, 2).

    x2 y = ( (x−1) + 1)2 ( (y − 2) + 2).
    		(25)
  4. Expand ln (x + y) about (1, 2).

    ln (x + y)
    =
    ln ⎛
    ⎝     		(x − 1) + 1 + (y − 2) + 2⎞
    ⎠
    =
    ln ⎛
    ⎝     		3 +(x − 1) + (y − 2)⎞
    ⎠
    =
    ln 3 ⎛
    ⎝     		1 + (x − 1) + (y − 2)
    3
    		⎞
    ⎠
    =
    ln 3 + ln⎛
    ⎝     		1+ (x − 1) + (y − 2)
    3
    		⎞
    ⎠
    =
    ln 3 + ⎛
    ⎝     		(x − 1) + (y − 2)
    3
    		⎞
    ⎠     		−
    ⎛
    ⎝     		(x − 1) + (y − 2)
    3
    		⎞
    ⎠     		2

     
    2
    		 +
    ⎛
    ⎝     		(x − 1) + (y − 2)
    3
    		⎞
    ⎠     		3

     
    3
    		 − …
Example of two variable functions
f(x, y) = ⎧
⎪
⎨
⎪
⎩
x y
x2 + y2
(x, y) ≠ (0, 0)
0
(x,y) = (0,0).
(26)
To find out what is lim(x, y)→ (0, 0) f(x,y), let (x, y) approach to (0, 0) along the line y = x. By setting x = t and y = t, f(x, y) = t2/(t2 + t2) = 1/2. However, if one approaches to (0, 0) along the x axis, f(x, y) = x ×0/(02 + x2) = 0 and they don't match. In short, f(x, y) does not have the limit as (x, y)→ (0,0).
taylor2var.jpg

Chain Differentiation

When f(x, y) is a composite function (i.e. x and y are also functions of other variables), the derivative of f with respect to the implicit variable can be achieved by using the chain differentiation rule, i.e.
df
dt
 = ∂f
∂x
 dx
dt
 + ∂f
∂y
 dy
dt
 .
(27)
An example was shown in class, i.e. this is a variation of derivative of composite functions.

Integration

Some of important integrals that you may have forgotten:


⌠
⌡ 		xα dx
xα+ 1
α+ 1
⌠
⌡ 		1
x
 dx
ln x
⌠
⌡ 		dx
1 + x2
arctan x
⌠
⌡ 		dx

√
a2 − x2
arcsin (x/a)
⌠
⌡ 		f(x) g′(x) dx
f(x)g(x) −⌠
⌡ 		f′(x) g(x) dx
⌠
⌡ 		f(x) dx
x f(x) − ⌠
⌡ 		x f′(x) dx

Exercises

Evaluate the following integrals:

(a) ⌠
⌡ 		dx
ex + e−x
(b)
⌠
⌡
√
 
a2 − x2
 
 dx
(c) ⌠
⌡ 		eax cos b x dx
(d)
⌠
⌡
√
 
x2 + 1
 
 dx
Solution for (d)  
I ≡ ⌠
⌡
√
 
x2 + 1
 
 dx.
Use integration by parts:
I
=
x
√
 
x2 + 1
 
 − ⌠
⌡ 		x ⎛
⎝
√
 
x2 + 1
 
 ⎞
⎠ 		′
 
 dx
=
x
√
 
x2 + 1
 
 − ⌠
⌡ 		x2

√
x2 + 1
 dx
=
x
√
 
x2 + 1
 
 − ⌠
⌡ 		x2 + 1 − 1

√
x2 + 1
 dx
=
x
√
 
x2 + 1
 
 − ⌠
⌡
√
 
x2 + 1
 
 dx + ⌠
⌡ 		dx

√
x2 + 1
=
x
√
 
x2 + 1
 
 − I + ⌠
⌡ 		dx

√
x2 + 1
 ,
(28)
so
I = 1
2
 ⎛
⎜
⎝ 		x
√
 
x2 + 1
 
 + ⌠
⌡ 		dx

√
x2 + 1
⎞
⎟
⎠ 		.
(29)
Now, the integral, J, defined as
J ≡ ⌠
⌡ 		dx

√
x2 + 1
 ,
(30)
is a challenging integral. The recipe is to change the variable from x to t as2

√
 
x2 + 1
 
 = t − x.
(31)
With this substitution,
1
=
t2 −2 t x,
(32)
dx
=
t − x
t
 dt,
(33)
so
J
=
⌠
⌡ 		1
t − x
 t − x
t
 dt
=
⌠
⌡ 		dt
t
=
lnt
=
ln⎛
⎝ 		x +
√
 
x2 + 1
 
⎞
⎠ 		,
(34)
and finally we obtain
I = 1
2
 ⎛
⎝ 		x
√
 
x2 + 1
 
 + ln ⎛
⎝ 		x +
√
 
x2 + 1
 
⎞
⎠ 		⎞
⎠ 		.
(35)
It is, however, more practical these days to carry out integrations like the one above via computer.
Enter the following line 
Sqrt[x^2 + 1]
exactly (case sensitive, square bracket) in the first box and x (lower case) in the second box, then press the query button.
Integrate with respect to .

Numerical Integration

You can differentiate almost any function exactly but very few functions can be integrated analytically which is why numerical integrations are more important than numerical differentiation.
numint.jpg
  • (Left) Rectangular rule:

    I  ∼  h (f0 + f1 + f2 + …+ fn−1).
    		(36)
  • Trapezoidal rule:
    I  ∼   h
    2
    		 (f0 + 2 f1 + 2f2 + …+ 2 fn−1 + fn).
    		(37)
  • Simpson rule:
    I  ∼   h′
    3
    		 (f0 +4 f1 + 2 f2 + 4 f3 + 2 f4 + …+ 4 f2n−1 + f2n).
    		(38)
where
h = b − a
n
 ,     h′ = b − a
2 n
 .
(39)
It is possible to estimate possible numerical errors for each scheme.
Derivation

Rectangular rule

The rectangular rule is to approximate ∫at f(x)dx by a rectangle with f(t) as the height and t − a ( = h) as the base.
num_rec.jpg
The error is then defined as

ϵ(t) = (t − a) f(t) −⌠
⌡ 		t

a 
 f(x) dx.
(40)
To see how this function behaves, take the derivative of ϵ(t) with respect to t as
ϵ′(t)
=
f(t) + (t − a) f′(t) − f(t)
=
f′(t) (t − a)
 ∼ 
f′(ξ) (t − a),
(41)
where ξ is some constant value assuming that f′(t) does not fluctuate much between a and t. By integrating Eq. (41) with respect to t, we have

ϵ(t)
 ∼ 
f′(ξ)
2
 (t − a)2
=
f′(ξ)
2
 h2,
(42)
where h = (t − a).
When adding up Eq. (42) over a finite interval of (a,b) for n times, the total error is

ϵtotal
 ∼ 
n h2 f′(ξ)
2
=
(b − a) f′(ξ)
2
 h,
(43)
where b − a = n h was used in Eq. (43). Equation (43) implies that when the rectangular rule is used to approximate
⌠
⌡ 		b

a 
 f(x) dx,
(44)
the numerical error involved is in the order of
ϵtotal  ∼ A h,
(45)
where A is a constant and h is the step size. In other words, to get 10−5 accuracy for b − a = 1, one would need n = (b − a)/h  ∼ 105 partitions over the interval.

Trapezoidal Rule

num_trap.jpg
The error involved in the trapezoidal rule is given

ϵ(t) = ⎛
⎝ 		f(t) + f(a)
2
⎞
⎠ 		(t − a) −⌠
⌡ 		t

a 
 f(x) dx.
(46)
Take the derivative of Eq. (46) as
ϵ′(t) = f′(t)
2
 (t − a) + f(t) + f(a)
2
 − f(t),
(47)
which does not yield much insight. Try taking another derivative of Eq. (47) as

ϵ"(t)
=
f"(t)
2
 (t − a) + f′(t)
2
 + f′(t)
2
 − f′(t)
 ∼ 
f"(ξ)
2
 (t − a).
(48)
Integrating Eq. (48) yields

ϵ′(t)  ∼  f"(ξ)
4
 (t − a)2.
(49)
Integrating Eq. (49) again yields

ϵ(t)
 ∼ 
f"(ξ)
12
 (t − a)3
=
f"(ξ)
12
 h3.
(50)
By adding up Eq. (50) over the interval of (a,b), we obtain
ϵtotal
 ∼ 
n h3 f"(ξ)
12
=
(b − a) f"(ξ)
12
 h2
 ∼ 
A h2,
(51)
where A is a constant. This conclusion implies that we need √{105}  ∼ 316 partitions to obtain 10−5 accuracy.

Simpson's Rule

From the result of error analysis of the trapezoidal rule, it was found that

I  ∼ Tn + A h2,
(52)
where I is the true value, Tn is the approximation by the trapezoidal rule, h is the step size and n is the number of partitions. If one makes the stepsize h one-half, Eq. (52) becomes

I  ∼ T2n + A⎛
⎝ 		h
2
⎞
⎠ 		2

 
 ,
(53)
or

I
 ∼ 
Tn + A h2,
(54)
4I
 ∼ 
4 T2n + A h2,
(55)
from which it follows

I  ∼  4T2n − Tn
3
 + (higher order terms than h2).
(56)
Actual computation of Eq. (56) is
I
 ∼ 
1
3
 ⎛
⎝ 		4 h
2
 (f0 + 2 f1 + 2 f2 + …+ 2 f2n−1 + f2n)− (2 h)
2
 (f0 + 2 f2 + 2 f4 + …+ f2n)⎞
⎠
=
h
3
 ⎛
⎝ 		f0 + 4 f1 + 2 f2 + 4 f3 + 2 f4 + …+2 f2n−2 + 4 f2n−1 + f2n⎞
⎠ 		.
(57)
Equation (57) is called the Simpson rule as you already figured it out.
An example of numerical integration with poor convergence:  

Using the relationship
4 ⌠
⌡ 		1

0 
√
 
1 − x2
 
 dx = π,
(58)
to compute an approximation of π.
numint_ex.jpg
Numerical integration of this function results in poor convergence. Why ?

Footnotes:

1 If one substitutes x=1 on the both sides, one obtains
ln 2 = 1 − 1
2
 + 1
3
 − 1
4
 + 1
5
 − …
(59)
Note that
1 + 1
2
 + 1
3
 + 1
4
 + 1
5
 + …,
(60)
diverges.
2For an integral of the type
⌠
⌡ 		F(x,
√
 
a x2 + b x + c
 
 ) dx     a > 0
the following change of variable usually works:

√
 
a x2 + b x + c
 
 = √a x − t.

File translated from TEX by TTH, version 4.03.
On 17 Jun 2025, 17:52.