Numerical differentiation: Difference formulas for the second derivative
General difference formulas for the second derivative
The general approach for \(n+1\) different grid points \(x_0, x_1, \ldots x_n\), which differ by a multiple of step size \(h\), is to find coefficients \(c_0, c_1, \ldots, c_n\) such that the expression \[Q(h)\stackrel{\mathrm{def}}{=}\sum_{i=0}^n c_if(x_i)\] approximates the second derivative \(f''(x_0)\) with the largest possible order of the truncation error. We find the coefficients \(c_0, c_1, \ldots, c_n\) by developing each \(f(x_i)\) into a Taylor polynomial about \(x_0\) with sufficiently high degree and by constructing and solving equations for the unknown \(c_i\) 's.
The 3-point central formula We take 3 grid points \(x_0-h\), \(x_0\) and \(x_0+h\). We define the formula \[Q(h)=A\cdot f(x_0-h)+B\cdot f(x_0)+C\cdot f(x_0+h)\] and determine the Taylor approximation of \(Q(h)\). We know that \[\begin{aligned} f(x_0-h) &= f(x_0)-f'(x_0)h+\tfrac{1}{2}f''(x_0)h^2+O(h^3)\\ f(x_0+h) &= f(x_0)+f'(x_0)h+\tfrac{1}{2}f''(x_0)h^2+O(h^3)\end{aligned}\] Therefore: \[\begin{aligned}Q(h) &= A\cdot ( f(x_0)-f'(x_0)h+\tfrac{1}{2}f''(x_0)h^2)+B\cdot f(x_0)\\ &\phantom{=}{}+C\cdot (f(x_0)+f'(x_0)h+\tfrac{1}{2}f''(x_0)h^2) + O(h^3)\\ \\ &= (A+B+C)f(x_0)+(C-A)f'(x_0)h+\tfrac{1}{2}(A+C)f''(x_0)h^2+O(h^3)\end{aligned}\] The equations that \(A, B, C\) must satisfy in order to maximise the order of the error in \(Q(h)\) are \[A+B+C = 0,\quad (C-A) = 0,\quad \tfrac{1}{2}(A+C)h^2 = 1\] These equations are easy to solve: \[A=\frac{1}{h^2}, B=-\frac{2}{h^2}, C=\frac{1}{h^2}\] So we have now found \[Q(h)=\frac{f(x_0+h)-2f(x_0)+f(x_0-h)}{h^2}\] and \[f''(x_0)\approx\frac{f(x_0+h)-2f(x_0)+f(x_0-h)}{h^2}\] This is equal to the earlier found central difference formula for the second derivative. It can be verified that the truncation error is of the order of \(h^4\).
One-sided 3-point formulas Left-sided 3-point difference formula: \[f''(x_0)\approx\frac{f(x_0)-2f(x_0+h)+f(x_0+2h)}{h^2}\] Right-sided 3-point difference formula: \[f''(x_0)\approx\frac{f(x_0)-2f(x_0-h)+f(x_0-2h)}{h^2}\] These one-sided 3-point difference formulas are useful to approximate second derivatives at the edges of a finite discrete signal.
We take 3 grid points \(x_0\), \(x_0+h\) and \(x_0+2h\). We define the formula \[Q(h)=A\cdot f(x_0)+B\cdot f(x_0+h)+C\cdot f(x_0+2h)\] and determine the Taylor approximation of \(Q(h)\). We know that \[\begin{aligned} f(x_0+h) &= f(x_0)+f'(x_0)h+\tfrac{1}{2}f''(x_0)h^2+O(h^3)\\ f(x_0+2h) &= f(x_0)+2f'(x_0)h+\tfrac{1}{2}f''(x_0)(2h)^2+O(h^3)\end{aligned}\] Thus: \[\begin{aligned}Q(h) &=A\cdot f(x_0) + B\cdot ( f(x_0)+f'(x_0)h+\tfrac{1}{2}f''(x_0)h^2)\\ &\phantom{=}{}+C\cdot (f(x_0)+2f'(x_0)h+\tfrac{4}{2}f''(x_0)h^2) + O(h^3)\\ \\ &= (A+B+C)f(x_0)+(B+2C)f'(x_0)h+\tfrac{1}{2}(B+4C)f''(x_0)h^2+O(h^3)\end{aligned}\] The equations that \(A, B, C\) must satisfy in order to maximise the order of the error in \(Q(h)\) are \[A+B+C = 0,\quad B+2C = 0,\quad \tfrac{1}{2}(B+4C)h^2 = 1\] These equations are easy to solve: \[A=\frac{1}{h^2}, B=-\frac{2}{h^2}, C=\frac{1}{h^2}\] So now we have found \[Q(h)=\frac{f(x_0)-2f(x_0+h)+f(x_0+2h)}{h^2}\] and \[f''(x_0)\approx\frac{f(x_0)-2f(x_0+h)+f(x_0+2h)}{h^2}\] This is a left sided 3 point difference formula. Similarly, you can also find the right-tailed 3-point difference formula: \[f''(x_0)\approx\frac{f(x_0)-2f(x_0-h)+f(x_0-2h)}{h^2}\]
Central 5-point difference \[f''(x_0)\approx\frac{-f(x_0-2h)+16f(x_0-h)-30f(x_0)+16f(x_0+h)-f(x_0+2h)}{12h^2}\] The truncation error is generally smaller than with the central 3-point difference formula.
We take 5 grid points \(x_0-2h\), \(x_0-h\), \(x_0\), \(x_0+h\) and \(x_0+2h\). We define the formula \[\begin{aligned}Q(h)&=A\cdot f(x_0-2h)+B\cdot f(x_0-h)+C\cdot f(x_0)\\ &\phantom{=}{}+ D\cdot f(x_0+h)+E\cdot f(x_0+2h)\end{aligned} \] and determine the Taylor approximation of \(Q(h)\). There is \[\begin{aligned} f(x_0-2h) &= f(x_0)-2f'(x_0)h+2f''(x_0)h^2-\tfrac{4}{3}f'''(x_0)h^3+\tfrac{2}{3}f''''(x_0)h^4+O(h^5)\\ f(x_0-h) &= f(x_0)-f'(x_0)h+\tfrac{1}{2}f''(x_0)h^2-\tfrac{1}{6}f'''(x_0)h^3+\tfrac{1}{24}f''''(x_0)h^4+O(h^5)\\ f(x_0+h) &= f(x_0)+f'(x_0)h+\tfrac{1}{2}f''(x_0)h^2+\tfrac{1}{6}f'''(x_0)h^3+\tfrac{1}{24}f''''(x_0)h^4+O(h^5)\\ f(x_0+2h) &= f(x_0)+2f'(x_0)h+2f''(x_0)h^2+\tfrac{4}{3}f'''(x_0)h^3+\tfrac{2}{3}f''''(x_0)h^4+O(h^5)\end{aligned}\] and therefore \[\begin{aligned}Q(h) &= (A+B+C+D+E)f(x_0)+(-2A-B+D+2E)f'(x_0)h\\&\phantom{=}{}+\tfrac{1}{2}(4A+B+D+4E)f''(x_0)h^2+\tfrac{1}{6}(-8A-B+D+8E)f'''(x_0)h^3\\&\phantom{=}{}+\tfrac{1}{24}(16A+B+D+16E)f''''(x_0)h^4+O(h^5)\end{aligned}\] The equations that \(A, B, C, D, E\) must satisfy in order to maximise the order of the error in \(Q(h)\) are \[\begin{aligned}A+B+C +D+\;\;E&= 0\\ -2A-B+\phantom{+C}D+\;2E &= 0\\ 4A+B+\phantom{+C}D+\;4E &= \frac{2}{h^2}\\ -8A-B+\phantom{+C}D+\;8E &= 0\\ 16A+B+\phantom{+C}D+16E &= 0\end{aligned}\] These equations are relatively easy to solve: \[A=-\frac{1}{12h^2}, B=\frac{4}{3h^2}, C=-\frac{5}{2h^2}, D=\frac{4}{3h^2}, E=-\frac{1}{12h^2}\] So we have found that \[Q(h)=\frac{-f(x_0-2h)+16f(x_0-h)-30f(x_0)+16f(x_0+h)-f(x_0+2h)}{12h^2}\] and \[f''(x_0)\approx\frac{-f(x_0-2h)+16f(x_0-h)-30f(x_0)+16f(x_0+h)-f(x_0+2h)}{12h^2}\] This is the 5-point central difference formula.
