Metadata-Version: 2.4
Name: qddate
Version: 1.0.17
Summary: Quick and dirty date parsing Python library to parse HTML dates really fast
Author-email: Ivan Begtin <ivan@begtin.tech>
Maintainer-email: Ivan Begtin <ivan@begtin.tech>
License: BSD
Project-URL: Homepage, https://github.com/ivbeg/qddate
Project-URL: Documentation, https://ivbeg.github.io/qddate/
Project-URL: Repository, https://github.com/ivbeg/qddate
Project-URL: Issues, https://github.com/ivbeg/qddate/issues
Keywords: date,datetime,parsing,html,scraping
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: BSD License
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Text Processing :: General
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: AUTHORS.md
Requires-Dist: pyparsing>=3.1.0
Provides-Extra: test
Requires-Dist: pytest>=8.0.0; extra == "test"
Requires-Dist: pytest-cov>=4.0.0; extra == "test"
Provides-Extra: bench
Requires-Dist: dateparser>=1.2.0; extra == "bench"
Requires-Dist: python-dateutil; extra == "bench"
Requires-Dist: arrow; extra == "bench"
Requires-Dist: pendulum; extra == "bench"
Provides-Extra: dev
Requires-Dist: ruff; extra == "dev"
Requires-Dist: tox; extra == "dev"
Requires-Dist: twine; extra == "dev"
Requires-Dist: build; extra == "dev"
Dynamic: license-file


# qddate — quick and dirty date parsing for scraped HTML

[![CI](https://github.com/ivbeg/qddate/workflows/CI/badge.svg)](https://github.com/ivbeg/qddate/actions)
[![PyPI Version](https://img.shields.io/pypi/v/qddate.svg?style=flat-square)](https://pypi.python.org/pypi/qddate)
[![Docs](https://img.shields.io/badge/docs-GitHub%20Pages-blue)](https://ivbeg.github.io/qddate/)
[![Coverage](https://codecov.io/gh/ivbeg/qddate/branch/master/graph/badge.svg)](https://codecov.io/gh/ivbeg/qddate)
[![Gitter](https://badges.gitter.im/qddate/Lobby.svg)](https://gitter.im/qddate/Lobby)

`qddate` is a Python 3 library focused on parsing date strings extracted from messy HTML pages as fast as possible. It was born out of long-term news aggregation work and is intentionally pragmatic: fewer abstractions, lots of hard-coded optimizations, and enough language coverage to process millions of strings at scale. The parser ships as part of the "news reconstruction" toolchain that can build RSS feeds for sites that never offered one.

If you need broader language coverage (but can trade speed for flexibility), check out [dateparser](https://github.com/scrapinghub/dateparser) or [dateutil](https://launchpad.net/dateutil).

## Documentation

Full documentation is published at [ivbeg.github.io/qddate](https://ivbeg.github.io/qddate/). The site lives in `docs/` (Docusaurus); run `make docs-serve` to preview it locally.

## Features

<!-- BEGIN qddate-stats -->
- 1,072 generated date patterns (from 134 base patterns)
- 14 supported languages
<!-- END qddate-stats -->
- Multi-language parsing (English, Romanian, Ukrainian, Russian, Spanish, and more)
- Handles left-aligned dates with trailing text: `12.03.1999 some text here`
- Configurable language subsets via the `languages=` parameter
- Prioritizes speed via pyparsing, hard-coded constants, and dirty tricks

## Recent updates

**1.0.17** — a weekday before a month-first English date is recognised: `Monday, January 31, 2022`, `Mon, Jan 31, 2022`.

**1.0.16** — pyparsing snake_case API (no deprecation warnings on pyparsing 3.3), signed timezone offsets (`-0500`), whole-word language detection (Portuguese `maio` dates parse again). First PyPI release since 1.0.10.

**1.0.15** — `DateParser(use_fingerprint=False)` is now deprecated; the legacy 6-level filter will be removed in `v2.0.0`. The fingerprint-based matcher (`1.0.14`) has been the default since the previous release.

**1.0.14** — Fingerprint-based matcher is now the default. The 6-level filter (`length → charset → separator → language → year_format → prefix`) collapses into a single-walk intersection over a precomputed fingerprint index. Exactly equivalent to the legacy path on the extended probe corpus (165 strings).

**1.0.13** — Single source of truth for month names: a new `qddate.patterns.months.MONTHS_BY_LANGUAGE` table is the canonical source for every language's month-name variants. `qdparser._detect_language` and all 14 per-language pattern modules consume from it. Legacy `*_MONTHS` / `*_mname2mon` constants are preserved as re-exports.

**1.0.12** — `DateParser.parse_relative()` for English and Russian relative-date phrases (`today`, `yesterday`, `N days ago`, `через N дней`, …). Filter pipeline documentation under `docs/docs/api/filter-pipeline.md`.

**1.0.11** — Auto-generated README pattern counts; opt-in relative-date parsing; new `format_date()` round-trip helper; type hints on the public API; `py.typed` marker; `noyear=` deprecated in favour of `allow_no_year=`.

**1.0.10** — English weekday with abbreviated month-first dates, e.g. `Thursday, Jun 25, 2026` and `Monday, Jun 22, 2026 - 13:46`.

**1.0.9** — Spanish article dates with a comma before the year, e.g. `03 de Julio, 2026` and `30 de junio, 2026` (common on Latin American government sites).

See [CHANGELOG.md](CHANGELOG.md) for the full release history.

## Limitations

- Limited language coverage compared to larger projects
- Adding new languages requires editing pattern tables manually
- Rare/odd formats may still slip through
- No calendar support

## Speed Optimization

- All constants are embedded directly; no runtime configuration files
- Only depends on `datetime` and `pyparsing` (plus packaging tooling)
- Avoids regular expressions in favor of pre-generated pyparsing patterns
- Aggressive pattern filtering using min/max length filters and prefix heuristics
- Nothing is loaded from disk at runtime besides the Python modules themselves

## Benchmark & Performance Notes

- **Standard benchmarks**: `benchmarks/bench.py` compares qddate with dateparser on the bundled corpus. Run `python benchmarks/bench.py` to execute.
- **Webpage data benchmarks**: `benchmarks/benchmark_webpage_data.py` tests qddate against other libraries using real-world text snippets extracted from webpages. Use `scripts/generate_webpage_test_data.py` to generate test data from government and international organization websites.
- **Comprehensive performance tests**: `benchmarks/comprehensive_performance_test.py` provides extensive performance testing with detailed metrics, pattern analysis, and regression detection. See [`benchmarks/README_PERFORMANCE_TEST.md`](benchmarks/README_PERFORMANCE_TEST.md) for details.
- [`PERFORMANCE_ANALYSIS.md`](PERFORMANCE_ANALYSIS.md) documents the 2025 profiling work (length-based indexing, prefix caching, packrat parsing, etc.) and how to evaluate future optimizations.
- Benchmark numbers depend on hardware, Python version, and workload. Please share your results in issues if you run the benchmark elsewhere.

## Usage

The simplest way to work with qddate is through `qddate.DateParser` and its `parse` method:

```python
import qddate

parser = qddate.DateParser()
print(parser.parse('2012-12-15'))
print(parser.parse('Fri, 12 Dec 2014 10:55:50'))
print(parser.parse('Thursday, Jun 25, 2026'))
print(parser.parse('03 de Julio, 2026'))
print(parser.parse('пятница, июля 17, 2015'))
```

The parser auto-detects languages for each string and returns `datetime.datetime` instances when it finds a match.

To restrict parsing to a subset of languages (faster, and avoids cross-language
ambiguity), pass the `languages` parameter:

```python
import qddate

# Only English and German patterns are loaded
parser = qddate.DateParser(languages=["en", "de"])
print(parser.parse("28. Juli 2015"))   # 2015-07-28 00:00:00
print(parser.parse("6 Jan 2009"))       # 2009-01-06 00:00:00
print(parser.parse("3 Января 2003 года"))  # None (Russian not in the allow-list)
```

`languages` accepts a single code (`"ru"`), a list (`["en", "de"]`), or `None`
(default: all supported languages).

## Dependencies

- [pyparsing](https://pypi.python.org/pypi/pyparsing) — the only runtime dependency.

Benchmarking against other date libraries is optional. Install the `bench` extra to
run the comparison scripts in `benchmarks/`:

```bash
pip install -e ".[bench]"   # adds dateparser, python-dateutil, arrow, pendulum
```

## Supported Languages

- Bulgarian
- Czech
- Dutch
- English
- French
- German
- Italian
- Polish
- Portuguese
- Romanian
- Russian
- Spanish
- Turkish
- Ukrainian

## Thanks

The original parser dates back to 2008 and evolved from regular expressions into pyparsing over time. Thanks to the [ScrapingHub](https://github.com/scrapinghub/dateparser) team for the inspiration to clean up the code, documentation, and build tooling, and for motivating the public release.

<!-- BEGIN qddate-stats -->
- 1,072 generated date patterns (from 134 base patterns)
- 14 supported languages
<!-- END qddate-stats -->
