Metadata-Version: 2.4
Name: ml-research-trends
Version: 0.1.1
Summary: A Python toolkit for collecting academic papers from Semantic Scholar and analyzing/visualizing publication trends in machine learning research
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: dotenv>=0.9.9
Requires-Dist: feedparser>=6.0.12
Requires-Dist: ipykernel>=7.2.0
Requires-Dist: ollama>=0.6.1
Requires-Dist: pandas>=3.0.1
Requires-Dist: requests>=2.32.5
Requires-Dist: seaborn>=0.13.2
Requires-Dist: matplotlib>=3.7.0
Requires-Dist: tqdm>=4.67.3
Requires-Dist: sentence-transformers>=3.0.0
Requires-Dist: torch>=2.2.0
Requires-Dist: scikit-learn>=1.4.0
Requires-Dist: umap-learn>=0.5.5
Requires-Dist: numpy>=1.26.0
Requires-Dist: plotly>=5.20.0

# ML Research Trends

A Python toolkit for collecting academic papers from Semantic Scholar and analyzing/visualizing publication trends in machine learning research.

## Features

- **Data Collection**: Collect papers from Semantic Scholar API based on keywords and date ranges
- **Trend Analysis**: Analyze publication trends, citation patterns, and temporal distributions
- **Visualization**: Create publication trend plots, embedding visualizations, and landmark timelines
- **Embedding Analysis**: Generate and visualize paper abstract embeddings using transformer models
- **Flexible API**: Easy-to-use functions for data collection, analysis, and visualization

## Installation

```bash
pip install ml-research-trends
```

Or with uv:
```bash
uv add ml-research-trends
```

## Requirements

- Python >= 3.11
- Semantic Scholar API key (set as environment variable `SEMANTIC_SCHOLAR_API_KEY`)

## Quick Start

```python
from ml_research_trends import (
    collect_topic_data,
    summarize_topic_data,
    analyze_topic_trends,
    plot_topic_counts_by_year,
)

# Collect papers on a topic
df = collect_topic_data(
    keywords=["retrieval augmented generation", "RAG"],
    max_results_per_keyword=100,
    min_year=2020,
)

# Summarize the data
summary = summarize_topic_data(df)
print(f"Total papers: {summary['total_papers']}")

# Analyze trends
trends = analyze_topic_trends(df)

# Visualize publication trends
plot_topic_counts_by_year(trends, topic_name="RAG")
```

## API Reference

### Data Collection & Analysis

- `collect_topic_data()` - Collect papers from Semantic Scholar
- `summarize_topic_data()` - Generate summary statistics
- `analyze_topic_trends()` - Analyze per-year trends

### Visualization

- `plot_topic_counts_by_year()` - Plot publication counts over time
- `embed_and_plot_abstracts()` - Create embedding visualizations
- `plot_landmark_timeline()` - Plot timelines with landmark events

## Examples

The package includes several example topics:

- **RAG (Retrieval-Augmented Generation)**: Track the evolution of RAG research
- **Chain-of-Thought Reasoning**: Analyze reasoning capabilities in LLMs
- **Multimodal Vision-Language Models**: Study multimodal learning trends
- **Transformer Architectures**: Examine transformer model developments

## Configuration

Set your Semantic Scholar API key:

```bash
export SEMANTIC_SCHOLAR_API_KEY="your_api_key_here"
```

Or create a `.env` file:
```
SEMANTIC_SCHOLAR_API_KEY=your_api_key_here
```

## Dependencies

- pandas: Data manipulation
- requests: API calls
- matplotlib/seaborn: Plotting
- sentence-transformers: Text embeddings
- torch: Machine learning
- scikit-learn: Dimensionality reduction
- umap-learn: Embedding visualization
- plotly: Interactive plots

## License

MIT License
