Metadata-Version: 2.4
Name: deepfeature
Version: 0.1.0
Summary: AI Data Scientist agent that automates end‑to‑end ML workflows
Author-email: Your Name <you@example.com>
License: MIT
Project-URL: Homepage, https://github.com/Mindlord-rex/deepfeature
Project-URL: Issues, https://github.com/Mindlord-rex/deepfeature/issues
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=2.0.0
Requires-Dist: scikit-learn>=1.3.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: langgraph>=0.0.20
Requires-Dist: litellm>=1.0.0
Requires-Dist: joblib>=1.2.0
Provides-Extra: openai
Requires-Dist: openai>=1.0.0; extra == "openai"
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.18.0; extra == "anthropic"
Provides-Extra: groq
Requires-Dist: groq>=0.4.0; extra == "groq"
Provides-Extra: ollama
Provides-Extra: notebook
Requires-Dist: matplotlib>=3.5.0; extra == "notebook"
Requires-Dist: seaborn>=0.12.0; extra == "notebook"
Requires-Dist: missingno>=0.5.0; extra == "notebook"
Provides-Extra: all
Requires-Dist: openai>=1.0.0; extra == "all"
Requires-Dist: anthropic>=0.18.0; extra == "all"
Requires-Dist: groq>=0.4.0; extra == "all"
Requires-Dist: matplotlib>=3.5.0; extra == "all"
Requires-Dist: seaborn>=0.12.0; extra == "all"
Requires-Dist: missingno>=0.5.0; extra == "all"
Dynamic: license-file

# DeepFeature

**AI Data Scientist – Automate End‑to‑End Machine Learning Pipelines**

[![PyPI version](https://badge.fury.io/py/deepfeature.svg)](https://badge.fury.io/py/deepfeature)
[![Python versions](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12-blue)](https://pypi.org/project/deepfeature/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

DeepFeature is an open‑source Python framework that acts as an **autonomous AI Data Scientist**. Just give it a dataset and a target column, and it will:

- Infer the problem type (classification/regression)
- Perform exploratory data analysis (EDA)
- Preprocess data (impute, scale, encode)
- Suggest and train multiple ML models (with optional LLM support)
- Evaluate and select the best model
- Export the entire pipeline as a **Jupyter notebook** or **Python script**
- Save trained model artifacts for later use

All decisions are transparent, and the generated code is editable – so you can run, modify, and reuse it.

---

## Features

- **Automatic Planning** – detects problem type and suggests a workflow.
- **EDA** – profiles columns, finds issues (missing values, constants, high cardinality).
- **Preprocessing** – handles numeric (median imputation + scaling) and categorical (most‑frequent imputation + one‑hot encoding) features.
- **LLM‑Powered Code Generation** – uses OpenAI, Anthropic, Groq, Ollama, etc. to write all pipeline code.
- **Any Model** – supports scikit‑learn, XGBoost, LightGBM, CatBoost, and more, dynamically loaded from import paths.
- **Artifact Saving** – automatically saves the best model, metrics, and metadata to disk.
- **Export** – export the generated code as a `.ipynb` notebook or `.py` script.
- **Optional Module Installation** – if a model library is missing, asks the user to install it.
- **Provider‑Agnostic** – works with any LLM provider via `litellm`.

---

## Installation

```bash
pip install deepfeature
```

### Optional Dependencies

For specific LLM providers:

```bash
pip install "deepfeature[openai]"
pip install "deepfeature[anthropic]"
pip install "deepfeature[groq]"
```

For notebook visualisations (when exporting):

```bash
pip install "deepfeature[notebook]"
```

To install everything:

```bash
pip install "deepfeature[all]"
```

---

## Quickstart

```python
from deepfeature import DeepFeature

# Initialize the agent (using Groq as the LLM provider)
agent = DeepFeature(
    llm_provider="groq",
    llm_config={"model": "mixtral-8x7b-32768"},
    api_key="gsk_...",          # or set GROQ_API_KEY env var
    verbose=True,
)

# Run the full pipeline
result = agent.run(
    dataset="path/to/titanic.csv",
    target="Survived",
)

# Check the results
print(result["report"]["summary"])
print(result["evaluation"])

# Export the generated code as a Python script
agent.export_file(result, "output/pipeline.py", format="script")
```

---

## Configuration

### LLM Providers

| Provider | `llm_provider` | Required env var |
| :--- | :--- | :--- |
| OpenAI | `"openai"` | `OPENAI_API_KEY` |
| Anthropic | `"anthropic"` | `ANTHROPIC_API_KEY` |
| Groq | `"groq"` | `GROQ_API_KEY` |
| Ollama (local) | `"ollama"` | none (uses `http://localhost:11434`) |
| Together AI | `"together"` | `TOGETHER_API_KEY` |

You can also pass an existing `LLMClient` instance:

```python
from deepfeature.llm import LLMClient
client = LLMClient(provider="openai", model="gpt-4")
agent = DeepFeature(llm_client=client)
```

### Saving Artifacts

By default, the best model, metrics, and metadata are saved under `artifacts/run_<timestamp>/`.  
You can change the directory or disable saving:

```python
agent = DeepFeature(artifacts_dir="my_models", save_artifacts=False)
```

### Exporting Code

```python
# Export as Jupyter notebook (default)
agent.export_file(result, "output/analysis.ipynb", format="notebook")

# Export as Python script
agent.export_file(result, "output/pipeline.py", format="script")
```

---

## How It Works

The pipeline is orchestrated with **LangGraph** and consists of these steps:

1. **Loader** – reads the CSV.
2. **Planner** – infers problem type (LLM‑assisted if available).
3. **EDA** – profiles columns, detects issues.
4. **Preprocessor** – cleans and transforms data.
5. **Training** – suggests models via LLM (or fallback), cross‑validates, selects best.
6. **Evaluation** – computes metrics on the full dataset (or a hold‑out set if you add it).
7. **Reporting** – compiles a summary report.
8. **Export** – generates code blocks and saves them as a notebook/script.

All generated code is produced by the LLM and cleaned for execution.

---

## Code Structure

```
deepfeature/
├── agents/            # modular pipeline steps
├── exporters/         # notebook/script export
├── llm/               # unified LLM client (litellm)
├── models/            # Pydantic models for state
├── nodes/             # LangGraph nodes
├── utils/             # helpers (permission, module install)
├── agent.py           # main DeepFeature class
├── graph.py           # LangGraph workflow
├── saver.py           # artifact saving/loading
└── prompts.py         # (coming soon) centralised prompt templates
```

---

## Example Notebook

You can also explore the exported notebook interactively:

```python
agent.export_file(result, "analysis.ipynb", format="notebook")
```

Then open it in Jupyter and run it step by step – the code is fully editable.

---

## Development

Install the package in development mode:

```bash
git clone https://github.com/Mindlord-rex/deepfeature.git
cd deepfeature
pip install -e ".[all]"
```

Run tests (coming soon):

```bash
pytest
```

---

## License

This project is licensed under the MIT License – see the [LICENSE](LICENSE) file for details.

---

## Contributing

Contributions are welcome! Please open an issue or pull request on GitHub.

---

## Roadmap (Future Ideas)

- [ ] Train/test split for more realistic evaluation.
- [ ] Hyperparameter tuning.
- [ ] SHAP / LIME explanations.
- [ ] Web UI.
- [ ] Support for time‑series and NLP tasks.
- [ ] Memory and human‑in‑the‑loop with LangGraph `interrupt()`.

---

## Acknowledgements

- Built with [LangGraph](https://www.langchain.com/langgraph) and [LiteLLM](https://github.com/BerriAI/litellm).
- Inspired by AutoML and agentic AI workflows.

---

**Made with ❤️ by Mindlord-rex**
