General difference formulas for the second derivative

Numerical differentiation: Difference formulas for the second derivative

General difference formulas for the second derivative

The general approach for $n+1$ different grid points $x_0, x_1, \ldots x_n$ , which differ by a multiple of step size $h$ , is to find coefficients $c_0, c_1, \ldots, c_n$ such that the expression

$Q(h)\stackrel{\mathrm{def}}{=}\sum_{i=0}^n c_if(x_i)$ approximates the second derivative $f''(x_0)$ with the largest possible order of the truncation error. We find the coefficients $c_0, c_1, \ldots, c_n$ by developing each $f(x_i)$ into a Taylor polynomial about $x_0$ with sufficiently high degree and by constructing and solving equations for the unknown $c_i$ 's.

We take 3 grid points $x_0-h$ , $x_0$ and $x_0+h$ . We define the formula

$Q(h)=A\cdot f(x_0-h)+B\cdot f(x_0)+C\cdot f(x_0+h)$ and determine the Taylor approximation of $Q(h)$ . We know that

$\begin{aligned} f(x_0-h) &= f(x_0)-f'(x_0)h+\tfrac{1}{2}f''(x_0)h^2+O(h^3)\\ f(x_0+h) &= f(x_0)+f'(x_0)h+\tfrac{1}{2}f''(x_0)h^2+O(h^3)\end{aligned}$ Therefore:

$\begin{aligned}Q(h) &= A\cdot ( f(x_0)-f'(x_0)h+\tfrac{1}{2}f''(x_0)h^2)+B\cdot f(x_0)\\ &\phantom{=}{}+C\cdot (f(x_0)+f'(x_0)h+\tfrac{1}{2}f''(x_0)h^2) + O(h^3)\\ \\ &= (A+B+C)f(x_0)+(C-A)f'(x_0)h+\tfrac{1}{2}(A+C)f''(x_0)h^2+O(h^3)\end{aligned}$ The equations that $A, B, C$ must satisfy in order to maximise the order of the error in $Q(h)$ are

$A+B+C = 0,\quad (C-A) = 0,\quad \tfrac{1}{2}(A+C)h^2 = 1$ These equations are easy to solve:

$A=\frac{1}{h^2}, B=-\frac{2}{h^2}, C=\frac{1}{h^2}$ So we have now found

$Q(h)=\frac{f(x_0+h)-2f(x_0)+f(x_0-h)}{h^2}$ and

$f''(x_0)\approx\frac{f(x_0+h)-2f(x_0)+f(x_0-h)}{h^2}$ This is equal to the earlier found central difference formula for the second derivative. It can be verified that the truncation error is of the order of $h^4$ .

One-sided 3-point formulas Left-sided 3-point difference formula:

$f''(x_0)\approx\frac{f(x_0)-2f(x_0+h)+f(x_0+2h)}{h^2}$ Right-sided 3-point difference formula:

$f''(x_0)\approx\frac{f(x_0)-2f(x_0-h)+f(x_0-2h)}{h^2}$ These one-sided 3-point difference formulas are useful to approximate second derivatives at the edges of a finite discrete signal.

We take 3 grid points $x_0$ , $x_0+h$ and $x_0+2h$ . We define the formula

$Q(h)=A\cdot f(x_0)+B\cdot f(x_0+h)+C\cdot f(x_0+2h)$ and determine the Taylor approximation of $Q(h)$ . We know that

$\begin{aligned} f(x_0+h) &= f(x_0)+f'(x_0)h+\tfrac{1}{2}f''(x_0)h^2+O(h^3)\\ f(x_0+2h) &= f(x_0)+2f'(x_0)h+\tfrac{1}{2}f''(x_0)(2h)^2+O(h^3)\end{aligned}$ Thus:

$\begin{aligned}Q(h) &=A\cdot f(x_0) + B\cdot ( f(x_0)+f'(x_0)h+\tfrac{1}{2}f''(x_0)h^2)\\ &\phantom{=}{}+C\cdot (f(x_0)+2f'(x_0)h+\tfrac{4}{2}f''(x_0)h^2) + O(h^3)\\ \\ &= (A+B+C)f(x_0)+(B+2C)f'(x_0)h+\tfrac{1}{2}(B+4C)f''(x_0)h^2+O(h^3)\end{aligned}$ The equations that $A, B, C$ must satisfy in order to maximise the order of the error in $Q(h)$ are

$A+B+C = 0,\quad B+2C = 0,\quad \tfrac{1}{2}(B+4C)h^2 = 1$ These equations are easy to solve:

$A=\frac{1}{h^2}, B=-\frac{2}{h^2}, C=\frac{1}{h^2}$ So now we have found

$Q(h)=\frac{f(x_0)-2f(x_0+h)+f(x_0+2h)}{h^2}$ and

$f''(x_0)\approx\frac{f(x_0)-2f(x_0+h)+f(x_0+2h)}{h^2}$ This is a left sided 3 point difference formula. Similarly, you can also find the right-tailed 3-point difference formula:

$f''(x_0)\approx\frac{f(x_0)-2f(x_0-h)+f(x_0-2h)}{h^2}$

Central 5-point difference

$f''(x_0)\approx\frac{-f(x_0-2h)+16f(x_0-h)-30f(x_0)+16f(x_0+h)-f(x_0+2h)}{12h^2}$ The truncation error is generally smaller than with the central 3-point difference formula.

We take 5 grid points $x_0-2h$ , $x_0-h$ , $x_0$ , $x_0+h$ and $x_0+2h$ . We define the formula

$\begin{aligned}Q(h)&=A\cdot f(x_0-2h)+B\cdot f(x_0-h)+C\cdot f(x_0)\\ &\phantom{=}{}+ D\cdot f(x_0+h)+E\cdot f(x_0+2h)\end{aligned}$ and determine the Taylor approximation of $Q(h)$ . There is

$\begin{aligned} f(x_0-2h) &= f(x_0)-2f'(x_0)h+2f''(x_0)h^2-\tfrac{4}{3}f'''(x_0)h^3+\tfrac{2}{3}f''''(x_0)h^4+O(h^5)\\ f(x_0-h) &= f(x_0)-f'(x_0)h+\tfrac{1}{2}f''(x_0)h^2-\tfrac{1}{6}f'''(x_0)h^3+\tfrac{1}{24}f''''(x_0)h^4+O(h^5)\\ f(x_0+h) &= f(x_0)+f'(x_0)h+\tfrac{1}{2}f''(x_0)h^2+\tfrac{1}{6}f'''(x_0)h^3+\tfrac{1}{24}f''''(x_0)h^4+O(h^5)\\ f(x_0+2h) &= f(x_0)+2f'(x_0)h+2f''(x_0)h^2+\tfrac{4}{3}f'''(x_0)h^3+\tfrac{2}{3}f''''(x_0)h^4+O(h^5)\end{aligned}$ and therefore

$\begin{aligned}Q(h) &= (A+B+C+D+E)f(x_0)+(-2A-B+D+2E)f'(x_0)h\\&\phantom{=}{}+\tfrac{1}{2}(4A+B+D+4E)f''(x_0)h^2+\tfrac{1}{6}(-8A-B+D+8E)f'''(x_0)h^3\\&\phantom{=}{}+\tfrac{1}{24}(16A+B+D+16E)f''''(x_0)h^4+O(h^5)\end{aligned}$ The equations that $A, B, C, D, E$ must satisfy in order to maximise the order of the error in $Q(h)$ are

$\begin{aligned}A+B+C +D+\;\;E&= 0\\ -2A-B+\phantom{+C}D+\;2E &= 0\\ 4A+B+\phantom{+C}D+\;4E &= \frac{2}{h^2}\\ -8A-B+\phantom{+C}D+\;8E &= 0\\ 16A+B+\phantom{+C}D+16E &= 0\end{aligned}$ These equations are relatively easy to solve:

$A=-\frac{1}{12h^2}, B=\frac{4}{3h^2}, C=-\frac{5}{2h^2}, D=\frac{4}{3h^2}, E=-\frac{1}{12h^2}$ So we have found that

$Q(h)=\frac{-f(x_0-2h)+16f(x_0-h)-30f(x_0)+16f(x_0+h)-f(x_0+2h)}{12h^2}$ and

$f''(x_0)\approx\frac{-f(x_0-2h)+16f(x_0-h)-30f(x_0)+16f(x_0+h)-f(x_0+2h)}{12h^2}$ This is the 5-point central difference formula.