Particle Swarm Optimization

HumpDay Implementation Documentation

Abstract

Particle Swarm Optimization (PSO) is a population-based stochastic optimization technique inspired by the social behavior of bird flocking and fish schooling. Originally developed by Kennedy and Eberhart (995), PSO optimizes a problem by iteratively improving candidate solutions (particles) with regard to a given measure of quality. Each particle adjusts its position based on its own best-known position and the swarm's best-known position, guided by velocity vectors that balance eploration and eploitation.

Algorithm Description

The Particle Swarm Optimization algorithm maintains a swarm of particles, where each particle $i$ has a position $mathbf_i$ and velocity $mathbfv_i$ in the $D$-dimensional search space. The algorithm updates these parameters according to the following equations:

Velocity Update:

$$mathbfv_i^t+ = wmathbfv_i^t + c_r_(mathbfp_i - mathbf_i^t) + c_2r_2(mathbfg - mathbf_i^t)$$

Position Update:

$$mathbf_i^t+ = mathbf_i^t + mathbfv_i^t+$$

where:

$w$ is the inertia weight
$c_, c_2$ are acceleration coefficients (cognitive and social parameters)
$r_, r_2$ are random numbers in [,]
$mathbfp_i$ is the personal best position of particle $i$
$mathbfg$ is the global best position found by the swarm

Implementation Details

Component	Description	Reference
Original Algorithm	Kennedy, J. & Eberhart, R. (995). Particle swarm optimization technique introduced as population-based stochastic optimization inspired by social behavior of bird flocking.	Kennedy & Eberhart (995)
Python Implementation	HumpDay modular implementation in evolutionary_algorithms.py	Source Code
JavaScript Implementation	Browser-compatible implementation in modular structure	JavaScript Source
Reference Package	Cross-validated against PySwarms implementation	PySwarms Documentation

Implementation Notes: This implementation includes adaptive parameters, velocity clamping, and diversification mechanisms for improved convergence and global search capabilities. The swarm size scales with problem dimensionality, and the algorithm includes early termination criteria for efficient resource utilization.

Algorithmic Parameters

Parameter	Value/Range	Description
Swarm Size	min(, ma(5, 3×D))	Number of particles, scaled by problem dimension
Inertia Weight ($w$)	.9 → .	Decreasing linearly with iteration progress
Cognitive Coefficient ($c_$)	2.5 → .5	Decreasing emphasis on personal best
Social Coefficient ($c_2$)	.5 → 2.5	Increasing emphasis on global best
Velocity Clamping	±2% of search range	Decreasing with iteration progress

References

Kennedy, J., & Eberhart, R. (995). Particle swarm optimization. Proceedings of ICNN'95 - International Conference on Neural Networks, , 92-98. IEEE.
Shi, Y., & Eberhart, R. (998). A modified particle swarm optimizer. 998 IEEE international conference on evolutionary computation proceedings, 9-73. IEEE.
Clerc, M., & Kennedy, J. (22). The particle swarm-eplosion, stability, and convergence in a multidimensional comple space. IEEE transactions on Evolutionary Computation, (), 58-73.
Poli, R., Kennedy, J., & Blackwell, T. (27). Particle swarm optimization: an overview. Swarm intelligence, (), 33-57.

Part of the HumpDay optimization library | Documentation | Source Code