Metadata-Version: 2.4
Name: npm-miner
Version: 1.0.2
Summary: Mine and extract complete package lists from NPM registry
Author-email: Rinalic <rinalic39@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/oraoraoraaa/Package-List-Miner
Project-URL: Repository, https://github.com/oraoraoraaa/Package-List-Miner
Project-URL: Issues, https://github.com/oraoraoraaa/Package-List-Miner/issues
Keywords: npm,javascript,nodejs,package-mining,data-mining,registry
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: requests>=2.31.0
Requires-Dist: tqdm>=4.66.0

# NPM Package Miner

A Python tool to mine and extract complete package lists from the NPM registry.

## Features

- Fetches all ~2-3 million NPM packages from the official registry
- Retrieves package metadata including homepage and repository URLs
- Parallel processing with 50 workers for efficient data collection
- Progress tracking with visual feedback
- Outputs standardized CSV format for cross-ecosystem analysis

## Installation

```bash
pip install npm-miner
```

## Quick Start

```bash
npm-miner
```

Or use as a Python module:

```python
from npm_miner import mine_npm
mine_npm()
```

## Output

Generates a CSV file with package information:
- Package ID, Platform, Name
- Homepage URL, Repository URL

## Performance

- Runtime: 10-20 hours for complete dataset
- Uses 50 parallel workers
- Network-dependent processing speed

## Data Source

- NPM Registry: https://registry.npmjs.org/
- All packages: https://replicate.npmjs.com/_all_docs

## License

MIT License - see LICENSE file for details
