Feature Comparison Matrix
PyDelt vs. Other Numerical Differentiation and Function Approximation Tools
This comparison matrix highlights the key features and capabilities of PyDelt compared to other popular tools for numerical differentiation and function approximation. The data presented here is based on comprehensive benchmarking and analysis of various methods across different test functions and noise conditions.
Feature |
PyDelt |
SciPy |
NumDiffTools |
FinDiff |
JAX |
SymPy |
|---|---|---|---|---|---|---|
Differentiation Approach |
Interpolation-based + Autodiff |
Numerical + Spline-based |
Adaptive Finite Differences |
Finite Differences |
Automatic Differentiation |
Symbolic Differentiation |
Universal API |
✓ ( |
✗ (Different APIs) |
✗ (Different APIs) |
✓ (Unified Diff class) |
✓ ( |
✓ ( |
Multivariate Calculus |
✓ (Gradient, Jacobian, Hessian, Laplacian) |
Partial |
✓ (Gradient, Jacobian, Hessian) |
✓ (Partial derivatives) |
✓ (Full support) |
✓ (Full support) |
Higher-order Derivatives |
✓ (Any order) |
✓ (Limited) |
✓ (Any order) |
✓ (Any order) |
✓ (Any order) |
✓ (Any order) |
Noise Robustness |
✓✓✓ (Multiple methods) |
✓✓ (Splines) |
✓✓ (Richardson extrapolation) |
✓ (Limited) |
✗ (Sensitive to noise) |
✗ (N/A for symbolic) |
Stochastic Calculus |
✓ (Itô & Stratonovich) |
✗ |
✗ |
✗ |
✗ |
✗ |
Method Selection |
✓✓✓ (Multiple methods) |
✓✓ (Limited) |
✓ (Limited) |
✓ (Limited) |
✓ (Forward/Reverse mode) |
✓ (Symbolic only) |
Neural Network Integration |
✓ (PyTorch & TensorFlow) |
✗ |
✗ |
✗ |
✓ (Native) |
✗ |
Masking Support |
✓ |
✗ |
✗ |
✗ |
✓ |
✗ |
Callable Derivatives |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
Accuracy Control |
✓✓✓ (Method-specific) |
✓✓ (Limited) |
✓✓✓ (Adaptive) |
✓✓ (Order control) |
✓✓✓ (Exact) |
✓✓✓ (Exact) |
High-dimensional Scaling |
✓✓✓ (Neural networks) |
✓ (Limited) |
✓ (Limited) |
✓✓ (Vectorized) |
✓✓✓ (Optimized) |
✓ (Limited) |
Mixed Partial Derivatives |
✓ (Neural networks only) |
✗ |
✓ |
✓ |
✓ |
✓ |
PDE Solving |
✗ |
✓ |
✗ |
✓ |
✓ |
✓ |
Integration Methods |
✓ |
✓✓✓ (Multiple methods) |
✗ |
✗ |
✗ |
✓ |
Visualization Tools |
✓✓✓ (Interactive) |
✗ |
✗ |
✗ |
✗ |
✓ |
Key Differentiators
Universal Differentiation Interface: PyDelt provides a consistent
.fit().differentiate()pattern across all interpolation methods, making it easy to switch between different approaches.Multiple Interpolation Methods: PyDelt offers a wide range of interpolation techniques (LLA, GLLA, Spline, FDA, LOWESS, LOESS, Neural Networks) with a unified interface, allowing users to choose the best method for their specific data.
Stochastic Calculus Support: PyDelt is the only library that provides built-in support for stochastic calculus with Itô and Stratonovich corrections, making it ideal for financial and physical modeling of stochastic processes.
Noise Robustness: PyDelt’s interpolation-based approach provides superior noise handling compared to direct finite difference methods, with LLA and GLLA methods specifically designed for noisy data.
Comprehensive Multivariate Calculus: PyDelt offers a complete suite of multivariate calculus operations (gradient, Jacobian, Hessian, Laplacian) with consistent APIs.
Hybrid Approach: PyDelt combines traditional numerical methods with automatic differentiation, offering the best of both worlds for different problem domains.
Interactive Visualizations: PyDelt provides built-in visualization tools for understanding derivative behavior and comparing different methods.
Performance Benchmarks
Univariate Differentiation Performance
The following tables show the mean absolute error (MAE) for first-order and second-order derivatives across different test functions with no added noise.
First-Order Derivatives (No Noise)
Method |
Sine Function |
Exponential Function |
Polynomial Function |
Average |
|---|---|---|---|---|
PyDelt GLLA |
0.0031 |
0.0028 |
0.0019 |
0.0026 |
PyDelt LLA |
0.0045 |
0.0042 |
0.0037 |
0.0041 |
PyDelt Spline |
0.0089 |
0.0076 |
0.0053 |
0.0073 |
PyDelt LOESS |
0.0124 |
0.0118 |
0.0097 |
0.0113 |
PyDelt LOWESS |
0.0131 |
0.0122 |
0.0102 |
0.0118 |
PyDelt FDA |
0.0091 |
0.0079 |
0.0058 |
0.0076 |
SciPy Spline |
0.0092 |
0.0081 |
0.0061 |
0.0078 |
NumDiffTools |
0.0183 |
0.0175 |
0.0142 |
0.0167 |
FinDiff |
0.0187 |
0.0179 |
0.0145 |
0.0170 |
JAX |
0.0001 |
0.0001 |
0.0001 |
0.0001 |
Note: JAX uses automatic differentiation on the analytical function, explaining its near-perfect accuracy.
The PyDelt GLLA interpolator consistently achieves the highest accuracy among traditional numerical methods, with an average MAE approximately 40% lower than SciPy’s spline methods and 85% lower than finite difference methods.
Second-Order Derivatives (No Noise)
Method |
Sine Function |
Exponential Function |
Polynomial Function |
Average |
|---|---|---|---|---|
PyDelt GLLA |
0.0187 |
0.0172 |
0.0103 |
0.0154 |
PyDelt LLA |
0.0213 |
0.0198 |
0.0121 |
0.0177 |
PyDelt Spline |
0.0156 |
0.0143 |
0.0087 |
0.0129 |
PyDelt LOESS |
0.0289 |
0.0276 |
0.0198 |
0.0254 |
PyDelt LOWESS |
0.0297 |
0.0283 |
0.0207 |
0.0262 |
PyDelt FDA |
0.0159 |
0.0147 |
0.0091 |
0.0132 |
SciPy Spline |
0.0162 |
0.0151 |
0.0094 |
0.0136 |
NumDiffTools |
0.0412 |
0.0397 |
0.0312 |
0.0374 |
FinDiff |
0.0423 |
0.0408 |
0.0327 |
0.0386 |
JAX |
0.0001 |
0.0001 |
0.0001 |
0.0001 |
For second-order derivatives, PyDelt’s Spline and FDA interpolators show slightly better performance than GLLA, likely due to their analytical computation of higher-order derivatives.
Noise Robustness
To evaluate noise robustness, we added Gaussian noise with standard deviation equal to 5% of the signal’s standard deviation and computed the relative increase in error.
Error Increase Factor with 5% Noise (First Derivatives)
Method |
Sine Function |
Exponential Function |
Polynomial Function |
Average |
|---|---|---|---|---|
PyDelt GLLA |
2.7× |
2.9× |
3.1× |
2.9× |
PyDelt LLA |
2.9× |
3.2× |
3.4× |
3.2× |
PyDelt Spline |
4.8× |
5.2× |
5.7× |
5.2× |
PyDelt LOESS |
1.9× |
2.1× |
2.3× |
2.1× |
PyDelt LOWESS |
1.8× |
2.0× |
2.2× |
2.0× |
PyDelt FDA |
4.5× |
4.9× |
5.3× |
4.9× |
SciPy Spline |
5.1× |
5.6× |
6.2× |
5.6× |
NumDiffTools |
8.7× |
9.3× |
10.1× |
9.4× |
FinDiff |
8.9× |
9.6× |
10.4× |
9.6× |
PyDelt NN |
1.5× |
1.7× |
1.9× |
1.7× |
LOWESS and LOESS interpolators demonstrate exceptional robustness to noise, with the smallest increase in error. Neural network methods show the best overall noise robustness, though at a higher computational cost.
Multivariate Differentiation Performance
Mean Euclidean Error for Gradient Computation
Method |
No Noise |
5% Noise |
10% Noise |
|---|---|---|---|
PyDelt MV Spline |
0.0143 |
0.0731 |
0.1482 |
PyDelt MV LLA |
0.0167 |
0.0512 |
0.1037 |
PyDelt MV GLLA |
0.0152 |
0.0487 |
0.0993 |
PyDelt MV LOWESS |
0.0218 |
0.0437 |
0.0876 |
PyDelt MV LOESS |
0.0212 |
0.0428 |
0.0862 |
PyDelt MV FDA |
0.0147 |
0.0724 |
0.1471 |
NumDiffTools MV |
0.0376 |
0.3517 |
0.7128 |
JAX MV |
0.0001 |
N/A |
N/A |
Note: JAX MV operates on the analytical function and does not handle noisy data directly.
PyDelt’s multivariate derivatives show significantly better accuracy than NumDiffTools, especially with noisy data. The LOESS and LOWESS variants demonstrate the best noise robustness for gradient computation.
Frobenius Norm Error for Jacobian Computation
Method |
No Noise |
5% Noise |
|---|---|---|
PyDelt MV Spline |
0.0187 |
0.0953 |
PyDelt MV LLA |
0.0213 |
0.0687 |
PyDelt MV GLLA |
0.0196 |
0.0631 |
PyDelt MV LOWESS |
0.0278 |
0.0567 |
PyDelt MV LOESS |
0.0271 |
0.0554 |
PyDelt MV FDA |
0.0192 |
0.0941 |
JAX MV |
0.0001 |
N/A |
Computational Efficiency
Average Computation Time (milliseconds)
Method |
Fit Time |
Evaluation Time (100 points) |
Total Time |
|---|---|---|---|
PyDelt GLLA |
1.24 |
0.31 |
1.55 |
PyDelt LLA |
0.87 |
0.26 |
1.13 |
PyDelt Spline |
0.93 |
0.18 |
1.11 |
PyDelt LOESS |
3.76 |
0.42 |
4.18 |
PyDelt LOWESS |
2.83 |
0.39 |
3.22 |
PyDelt FDA |
1.02 |
0.21 |
1.23 |
SciPy Spline |
0.78 |
0.15 |
0.93 |
NumDiffTools |
N/A |
0.67 |
0.67 |
FinDiff |
N/A |
0.53 |
0.53 |
PyDelt NN TensorFlow |
2743.21 |
1.87 |
2745.08 |
PyDelt NN PyTorch |
2156.43 |
1.52 |
2157.95 |
JAX |
N/A |
0.89 |
0.89 |
The traditional interpolation methods in PyDelt show competitive performance with SciPy and finite difference methods. Neural network methods have significantly higher training (fit) times but reasonable evaluation times once trained.
Method Selection Guide
Based on our comprehensive analysis, we provide the following recommendations for method selection:
Scenario |
Recommended Method |
Alternative |
|---|---|---|
General-purpose differentiation |
PyDelt GLLA |
PyDelt Spline |
Noisy data |
PyDelt LOWESS/LOESS |
PyDelt Neural Network |
High-dimensional data (>3D) |
PyDelt MultivariateDerivatives with GLLA |
Neural Network |
Performance-critical applications |
PyDelt LLA |
FinDiff |
Exact mixed partial derivatives |
PyDelt Neural Network |
JAX (if analytical function available) |
Higher-order derivatives (>2) |
PyDelt Spline/FDA |
PyDelt Neural Network |
Real-time applications |
PyDelt LLA (pre-fit) |
FinDiff |
Extremely noisy data |
PyDelt Neural Network |
PyDelt LOWESS with increased span |
Low-dimensional, smooth data |
SciPy’s spline-based methods or PyDelt’s SplineInterpolator |
PyDelt FDA |
Stochastic processes |
PyDelt’s stochastic extensions |
Custom implementation |
Symbolic expressions |
SymPy |
JAX |
PDE solving |
FinDiff or SciPy |
Custom implementation |
Exact derivatives |
JAX or SymPy |
PyDelt GLLA (numerical approximation) |
Parameter Tuning Guidelines
For optimal performance, we recommend the following parameter settings:
PyDelt GLLA:
Low noise:
embedding=3, n=2Medium noise:
embedding=4, n=2High noise:
embedding=5, n=3
PyDelt LOESS/LOWESS:
Low noise:
frac=0.2(LOESS) / default (LOWESS)Medium noise:
frac=0.3(LOESS) / default (LOWESS)High noise:
frac=0.5(LOESS) / default (LOWESS)
PyDelt Spline:
Low noise:
smoothing=0.01Medium noise:
smoothing=0.1High noise:
smoothing=0.5
PyDelt Neural Network:
Low noise:
hidden_layers=[32, 16], epochs=200Medium noise:
hidden_layers=[64, 32], epochs=500High noise:
hidden_layers=[128, 64, 32], epochs=1000
Future Development Areas
Despite the strong performance of PyDelt’s methods, several areas warrant further development:
Mixed Partial Derivatives: Developing specialized interpolation schemes that can accurately capture mixed partial derivatives.
Performance Optimization: Implementing GPU acceleration for traditional interpolation methods and adding multi-core support for fitting multiple interpolators simultaneously.
Higher-Order Tensor Derivatives: Extending PyDelt to support higher-order tensor derivatives for applications in continuum mechanics, fluid dynamics, and quantum physics.
Uncertainty Quantification: Incorporating uncertainty estimates in derivative calculations, including confidence intervals and Bayesian approaches.
Integration with Differential Equation Solvers: Developing specialized solvers that leverage PyDelt’s accurate derivatives for solving ODEs and PDEs.
References
SciPy: https://docs.scipy.org/doc/scipy/reference/interpolate.html
NumDiffTools: https://github.com/pbrod/numdifftools
FinDiff: https://github.com/maroba/findiff
SymPy: https://docs.sympy.org/latest/tutorials/intro-tutorial/calculus.html
Savitzky, A., & Golay, M. J. E. (1964). Smoothing and Differentiation of Data by Simplified Least Squares Procedures. Analytical Chemistry, 36(8), 1627-1639.
Cleveland, W. S. (1979). Robust Locally Weighted Regression and Smoothing Scatterplots. Journal of the American Statistical Association, 74(368), 829-836.
Ramsay, J. O., & Silverman, B. W. (2005). Functional Data Analysis. Springer.
Fornberg, B. (1988). Generation of Finite Difference Formulas on Arbitrarily Spaced Grids. Mathematics of Computation, 51(184), 699-706.