Chapter 7: Complex Analysis

“Complex numbers are not complicated—they’re just 2D. Euler’s formula connects exponentials, trigonometry, and rotation in one beautiful equation.”

Why Complex Numbers for ML?

You might wonder: why do ML practitioners need complex numbers? Here’s why:

  1. Fourier transforms use complex exponentials

  2. Signal processing is naturally complex

  3. Quantum computing is built on complex numbers

  4. Some optimizations are easier in the complex plane

  5. Eigenvalues of real matrices can be complex

This chapter gives you the intuition you need—not a full course in complex analysis.

Complex Numbers as 2D Vectors

The Basic Idea

A complex number z = a + bi is just a point in 2D:

  • Real part: a (x-coordinate)

  • Imaginary part: b (y-coordinate)

  • i: The “imaginary unit” where i² = -1

import numpy as np

# Complex numbers in Python
z = 3 + 4j  # a = 3, b = 4

print(f"Real part: {z.real}")       # 3.0
print(f"Imaginary part: {z.imag}")  # 4.0
print(f"Magnitude: {abs(z)}")       # 5.0 (distance from origin)
print(f"Angle: {np.angle(z)}")      # 0.927 radians

Polar Form

Any complex number can be written as:

\[z = r(\cos\theta + i\sin\theta) = re^{i\theta}\]

where:

  • r = |z| is the magnitude (distance from origin)

  • θ is the argument (angle from positive real axis)

r = abs(z)
theta = np.angle(z)
z_polar = r * np.exp(1j * theta)
print(f"z = {z}, z_polar = {z_polar}")  # Same number!

Euler’s Formula: The Most Beautiful Equation

\[e^{i\theta} = \cos\theta + i\sin\theta\]

This single equation connects:

  • Exponentials (e)

  • Trigonometry (sin, cos)

  • Complex numbers (i)

  • Rotation (θ)

Special Case: Euler’s Identity

When θ = π:

\[e^{i\pi} + 1 = 0\]

This equation contains the five most important constants in mathematics: e, i, π, 1, and 0.

Why It Matters

Euler’s formula means:

  • Multiplication by e^(iθ) = rotation by angle θ

  • Sinusoids are complex exponentials: cos(θ) = Re(e^(iθ))

  • Fourier analysis becomes multiplication

Complex Differentiation

Analytic Functions

A function f: ℂ → ℂ is analytic (or holomorphic) if it’s differentiable in the complex sense:

\[f'(z) = \lim_{h \to 0} \frac{f(z+h) - f(z)}{h}\]

where h can approach 0 from any direction in the complex plane.

The Cauchy-Riemann Equations

If f(z) = u(x,y) + iv(x,y) is analytic, then:

\[\frac{\partial u}{\partial x} = \frac{\partial v}{\partial y}, \quad \frac{\partial u}{\partial y} = -\frac{\partial v}{\partial x}\]

These constraints are very restrictive—analytic functions are extremely smooth.

Why This Matters

Analytic functions have remarkable properties:

  • Infinitely differentiable (unlike real functions)

  • Equal to their Taylor series (no approximation error!)

  • Determined by values on boundary (holographic principle)

The Fourier Transform

From Time to Frequency

The Fourier transform converts a signal from time domain to frequency domain:

\[\hat{f}(\omega) = \int_{-\infty}^{\infty} f(t) e^{-i\omega t} dt\]

The inverse:

\[f(t) = \frac{1}{2\pi} \int_{-\infty}^{\infty} \hat{f}(\omega) e^{i\omega t} d\omega\]

Intuition

  • e^(iωt) is a rotating complex number at frequency ω

  • The integral measures how much f(t) “correlates” with each frequency

  • The result is a complex number encoding amplitude and phase

In Code

import numpy as np
from numpy.fft import fft, fftfreq

# Create a signal: sum of two frequencies
t = np.linspace(0, 1, 1000)
signal = np.sin(2 * np.pi * 10 * t) + 0.5 * np.sin(2 * np.pi * 25 * t)

# Compute FFT
spectrum = fft(signal)
frequencies = fftfreq(len(t), t[1] - t[0])

# The spectrum shows peaks at 10 Hz and 25 Hz
import matplotlib.pyplot as plt
plt.plot(frequencies[:500], np.abs(spectrum[:500]))
plt.xlabel('Frequency (Hz)')
plt.ylabel('Magnitude')
plt.title('Frequency Spectrum')
plt.show()

ML Applications

  • Audio processing: Speech recognition, music generation

  • Image processing: Convolution in frequency domain

  • Time series: Spectral analysis, periodicity detection

  • Positional encodings: Transformers use sinusoidal embeddings

Convolution and the Fourier Transform

The Convolution Theorem

Convolution in time domain = multiplication in frequency domain:

\[\mathcal{F}(f * g) = \mathcal{F}(f) \cdot \mathcal{F}(g)\]

This is why FFT-based convolution is fast:

  1. FFT both signals: O(n log n)

  2. Multiply: O(n)

  3. Inverse FFT: O(n log n)

Total: O(n log n) instead of O(n²) for direct convolution.

ML Connection

CNNs use convolution extensively. For large kernels, FFT-based convolution can be faster.

Residues and Contour Integration

The Residue Theorem

For analytic functions with isolated singularities:

\[\oint_C f(z) dz = 2\pi i \sum \text{Res}(f, z_k)\]

The integral around a closed curve equals 2πi times the sum of residues inside.

Why It Matters

This lets you compute difficult real integrals by:

  1. Extending to complex plane

  2. Choosing a clever contour

  3. Using residues instead of direct integration

Example Application

The integral ∫₀^∞ sin(x)/x dx = π/2 is hard to compute directly but easy with contour integration.

Complex Numbers in Neural Networks

Complex-Valued Networks

Some applications benefit from complex-valued neural networks:

# Complex linear layer (conceptual)
def complex_linear(z, W_real, W_imag, b_real, b_imag):
    """
    Complex matrix multiplication:
    (W_r + iW_i)(z_r + iz_i) = (W_r z_r - W_i z_i) + i(W_r z_i + W_i z_r)
    """
    z_real, z_imag = z.real, z.imag
    out_real = W_real @ z_real - W_imag @ z_imag + b_real
    out_imag = W_real @ z_imag + W_imag @ z_real + b_imag
    return out_real + 1j * out_imag

Applications

  • MRI reconstruction: Data is inherently complex

  • Radar/sonar: Phase information matters

  • Audio synthesis: Complex spectrograms

  • Quantum ML: Quantum states are complex

Conformal Maps

Angle-Preserving Transformations

Analytic functions preserve angles—they’re conformal maps. This means:

  • Local shapes are preserved (circles map to circles, infinitesimally)

  • Useful for solving PDEs by mapping to simpler domains

Example: The Joukowski Transform

\[w = z + \frac{1}{z}\]

This maps a circle to an airfoil shape—used in aerodynamics!

Connecting to PyDelt

While PyDelt primarily works with real-valued functions, complex analysis concepts appear in:

  1. Fourier-based smoothing: Some interpolation methods use frequency domain

  2. Spectral analysis: Analyzing periodic components in time series

  3. Signal processing: Derivatives of complex signals

import numpy as np
from pydelt.interpolation import SplineInterpolator

# Complex signal (e.g., from FFT)
t = np.linspace(0, 2*np.pi, 100)
signal_complex = np.exp(1j * t)  # Unit circle in complex plane

# Work with real and imaginary parts separately
real_part = signal_complex.real
imag_part = signal_complex.imag

# Differentiate each part
interp_real = SplineInterpolator(smoothing=0.01)
interp_real.fit(t, real_part)
deriv_real = interp_real.differentiate(order=1)(t)

interp_imag = SplineInterpolator(smoothing=0.01)
interp_imag.fit(t, imag_part)
deriv_imag = interp_imag.differentiate(order=1)(t)

# Combine back
deriv_complex = deriv_real + 1j * deriv_imag

# For e^(it), derivative should be i*e^(it)
expected = 1j * np.exp(1j * t)
print(f"Max error: {np.max(np.abs(deriv_complex - expected)):.6f}")

Key Takeaways

  1. Complex numbers are 2D vectors with special multiplication

  2. Euler’s formula connects exponentials, trig, and rotation

  3. Fourier transform converts time ↔ frequency using complex exponentials

  4. Analytic functions are extremely smooth and well-behaved

  5. Convolution theorem makes FFT-based convolution fast

  6. Complex networks are useful for inherently complex data

Exercises

  1. Euler’s formula: Verify that e^(iπ/4) = (1+i)/√2 using Python.

  2. FFT analysis: Create a signal with 3 frequency components. Use FFT to identify them.

  3. Complex derivative: For f(z) = z², verify the Cauchy-Riemann equations.

  4. Rotation: Show that multiplying by e^(iπ/2) rotates a complex number by 90°.

Previous: ← Multivariate Calculus | Next: Applications to ML →