#15 (03/19/2025)

Numerical differentiation

Analytical differentiation

First, some non-numerical (analytical) differentiation


f (x) f ′(x) Examples
(f ± g)′ f ′± g ′ (x + ex) ′
(f g) ′ f ′g + f g ′ (f g h) ′
(f/g) ′ (f ′g − f g ′)/g2 x/sin x
f (g (x)) ′ f ′(g (x)) g ′(x) (x2 + 1)10
xα αxα−1 x5, √x, √{x3}
ex ex ex2, 2x
ln x 1/x ln (x3)
sin x cos x sin (1+x2)
cos x −sin x cos (x2)
tan x 1/cos 2 x tan (x2+1)
arcsin x 1/√{1−x2} arcsin (arcsin (x))
arccos x −1/√{1−x2}
arctan x 1/(1+x2)

Some exercise:

    (a)
    (2x2 + 3)2 (3 x + 1)2

    (b)
    c x + d
    a x + b

    (c)

    √
     
    x2 − a2
     

    (d)
    1

    √
    a2 − x2

    (e)
    exp(−h2 x2)

    (f)
    cos2  3 x

    (g)
    ln (tan x)

    (h)
    tan−1   1 + x
    1 − x

Numerical differentiation

The only occasion where numerical differentiation is needed is when data are given numerically only, i.e.
x f (x)
1 1
2 8
3 27
4 64
numdiff.gif

Forward difference


f (x + h)
=
f (x) + h f ′(x) + h2
2!
 f  " (x) + h3
3!
 f  "′(x)+…
(1)
≈
f (x) + h f ′(x),
(2)
hence
f ′(x) ≈ f (x + h) − f (x)
h
 .
(3)

Backward difference


f ( x − h )
=
f (x) − h f ′(x) + h2
2!
 f  "(x) − h3
3!
 f  "′(x) + …
≈
f (x) − h f ′(x),
(4)
hence
f ′(x) ≈ f (x) − f (x − h)
h
 .
(5)

Central difference


f (x + h)
=
f (x) + h f ′(x) + h2
2!
 f  "(x) + h3
3!
 f  "′(x)+…
(6)
f (x − h)
=
f (x) − h f ′(x) + h2
2!
 f  "(x) − h3
3!
 f  "′(x)+…
(7)
from which
f ( x + h ) − f ( x − h ) = 2 h f ′(x) + 2 h3
3!
 f  "′(x)+ …
(8)
hence

f ′(x) ≈ f ( x + h )− f ( x − h )
2h
 .
(9)
It is seen from the above that the central difference should yield better approximation as the truncation error is in the order of h2 while the truncation error for the forward and backward difference methods is in the order of h.

Second order derivative


f (x + h)
=
f (x)+h f ′(x) + h2
2!
 f  "(x) + h3
3!
 f  "′(x)+…
(10)
f (x − h)
=
f (x)−h f ′(x) + h2
2!
 f  "(x) − h3
3!
 f  "′(x)+…
(11)
Adding the above two yields

f (x + h)+f (x − h) = 2 f (x) + h2 f  "(x) + …
(12)
so

f  "(x) ≈ f (x + h) + f (x − h) − 2 f (x)
h2
 .
(13)

Example

The program below is to numerically differentiate a function given by the table below (numerical values of sin x for x = 0.0  ∼ 1.0). for x = 0 to x = 1 with the step size of h = 0.1 excluding at x=0 and x=1.0 where the central difference scheme does not work.
x 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
f(x) 00.09980.19860.29550.38940.47940.56460.64420.71730.78330.8414 


#include <stdio.h>
#define N 11

int main()
{
double y[N]={0, 0.0998, 0.1986, 0.2955, 0.3894, 0.4794, 0.5646, 0.6442, 0.7173, 0.7833, 0.8414};
double central[N], h=0.1;
int i;

/* Excluding x=0 and x=1.0 */
for (i=1;i<N-1;i++) central[i]= (y[i+1]-y[i-1])/(2*h);

printf ("   x    Central \n---------------------------\n");

for (i=1;i<N-1;i++)
 printf ("%lf %lf\n", i*h, central[i]);

return 0;
}

What if you want to obtain f′(x) at x = a with the same accuracy as the central difference ?

Obviously you cannot use the central difference scheme at x=a. Using the forward difference scheme, one can get

f′(x)  ∼  f(x+h) − f(x)
h
which is a poor approximation at best.
Consider

f(x + h)
=
f(x) + h f′(x) + h2
2!
 f"(x) + …
(14)
f(x + 2 h)
=
f(x) + 2 h f′(x) + 4 h2
2!
 f"(x) + …
(15)
The h2 term can be eliminated if one subtracts Eq.(15) from 4 times Eq.(14) as

4 f(x + h) − f(x + 2 h) = 3 f(x) + 2 h f′(x) + (higher order) …
(16)
from which one obtains

f′(x)  ∼  4 f(x+h) − f(x+2h) − 3 f(x)
2 h
(17)
Eq.(17) has the same order of accuracy as the central difference scheme.
So the program above can be modified as: 


#include <stdio.h>
#define N 11

int main()
{
double y[N]={0, 0.0998, 0.1986, 0.2955, 0.3894, 0.4794, 0.5646, 0.6442, 0.7173, 0.7833, 0.8414};
double central[N], h=0.1;
int i;

for (i=1;i<N-1;i++) central[i]= (y[i+1]-y[i-1])/(2*h);

central[0]=(4*y[1]-y[2]-3*y[0])/(2*h);

printf ("   x    Central \n---------------------------\n");

for (i=0;i<N-1;i++)
 printf ("%lf %lf\n", i*h, central[i]);

return 0;
}


File translated from TEX by TTH, version 4.03.
On 17 Mar 2025, 13:16.