Metadata-Version: 2.4
Name: mlflow-langsmith
Version: 0.1.2
Summary: Import LangSmith traces into MLflow as durable local MLflow runs and traces.
Project-URL: Homepage, https://github.com/dej-h/mlflow-langsmith
Project-URL: Repository, https://github.com/dej-h/mlflow-langsmith
Project-URL: Issues, https://github.com/dej-h/mlflow-langsmith/issues
Project-URL: Documentation, https://github.com/dej-h/mlflow-langsmith/blob/main/docs/pypi-release.md
Author: dej-h
License: MIT License
        
        Copyright (c) 2026 Dejan
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Keywords: langsmith,mlflow,observability,python,tracing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Requires-Dist: langsmith>=0.8
Requires-Dist: mlflow>=3.12
Requires-Dist: packaging>=24.0
Requires-Dist: pydantic<3.0,>=2.11
Description-Content-Type: text/markdown

# mlflow-langsmith

[![pytest](https://github.com/dej-h/mlflow-langsmith/actions/workflows/pytest.yml/badge.svg)](https://github.com/dej-h/mlflow-langsmith/actions/workflows/pytest.yml)
[![PyPI version](https://img.shields.io/pypi/v/mlflow-langsmith.svg)](https://pypi.org/project/mlflow-langsmith/)
[![Python versions](https://img.shields.io/pypi/pyversions/mlflow-langsmith.svg)](https://pypi.org/project/mlflow-langsmith/)
[![License: MIT](https://img.shields.io/github/license/dej-h/mlflow-langsmith.svg)](LICENSE)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)

Imports a LangSmith run into MLflow as a native MLflow trace. Run tree reconstructed as MLflow spans, source timestamps preserved, raw payload stored as an artifact, lookup tags written.

Use it when you want a LangSmith run to outlast LangSmith retention, sit next to your native MLflow runs in one UI, or act as a stable baseline for an eval suite.

## Install

```bash
pip install mlflow-langsmith
# or
uv add mlflow-langsmith
```

Python 3.10, 3.11, 3.12, or 3.13. Depends on `langsmith>=0.8`, `mlflow>=3.12`, `pydantic>=2.11`.

## Configure

```bash
LANGSMITH_API_KEY=...
MLFLOW_TRACKING_URI=http://127.0.0.1:5000
LANGSMITH_PROJECT=my-project            # optional default for list-langsmith-runs
MLFLOW_EXPERIMENT_NAME=agent-baselines  # optional default for imports
```

CLI flags and Python kwargs override the env.

## CLI

Find a recent LangSmith run:

```bash
mlflow-langsmith list-langsmith-runs --limit 10 --success-only
```

Import it into MLflow:

```bash
mlflow-langsmith import-run \
    --langsmith-run-id 019e5a23-a0b8-7491-848f-b251a011e23a \
    --mlflow-experiment-name agent-baselines \
    --baseline-key nightly-regression
```

Look it back up later:

```bash
mlflow-langsmith list-imports  --mlflow-experiment-name agent-baselines
mlflow-langsmith show-import   --langsmith-run-id 019e5a23-a0b8-7491-848f-b251a011e23a
```

The import shows up in the MLflow UI as a normal trace.

## Python

```python
from mlflow_langsmith import import_langsmith_run

result = import_langsmith_run(
    "019e5a23-a0b8-7491-848f-b251a011e23a",
    mlflow_experiment_name="agent-baselines",
    baseline_key="nightly-regression",
    tags={"owner": "eval-platform"},
)

print(result.mlflow_run_id, result.mlflow_trace_id)
```

Already have a LangSmith `Run` object (e.g. inside your own eval harness)? Skip the second API call:

```python
from mlflow_langsmith import import_langsmith_run_object
result = import_langsmith_run_object(run, mlflow_experiment_name="agent-baselines")
```

Resolve a prior import by baseline key. No LangSmith call needed:

```python
from mlflow_langsmith import find_imported_run, load_imported_payload

ref = find_imported_run(baseline_key="nightly-regression")
payload = load_imported_payload(ref.mlflow_run_id)
```

## What gets written

One MLflow run plus one MLflow trace per import, in your chosen experiment.

Spans: LangSmith `chain`, `llm`, `chat_model`, `tool`, `retriever`, `parser`, `embedding`, `agent` runs map to the matching MLflow span types. Parent-child relationships and start/end timestamps come from LangSmith, so step durations are real and not import wall-clock time.

Tags: source run id, source trace id, source project, baseline key. Indexable in MLflow search, and what `find_imported_run` queries against.

Artifacts under `langsmith/`:

- `raw-run.json` (full LangSmith payload, audit copy)
- `normalized-run-tree.json`
- `span-manifest.json`
- `import-summary.json`
- `messages.json`, `tool-calls.json`, `token-usage.json`, `attachments-manifest.json` when present

Per-span attributes: token usage, model name, model provider, cost, status. Pulled from LangSmith fields where they exist.

## Re-imports

`duplicate_mode="skip"` is the default. Importing the same LangSmith run twice returns the existing MLflow run. Pass `"raise"` to fail, `"import"` to force a new MLflow run.

## Sensitive data

Imported artifacts contain the same prompts, responses, tool inputs, tool outputs, and metadata as the original LangSmith trace. Don't import into an MLflow store you wouldn't keep that data in.

---

## For contributors

`src` layout. `uv`, `pytest`, `ruff`, `mypy`, `pre-commit`. CI runs the test suite on Python 3.10, 3.11, 3.12, 3.13.

```bash
uv sync
uv run pytest
uv run mypy src
uv run ruff check .
```

Local MLflow server (`sqlite:///experiments/mlflow/mlflow.db`):

```bash
bash scripts/start_mlflow_dev.sh
```

End-to-end against a real LangSmith run:

```bash
uv run --env-file .env mlflow-langsmith import-run --langsmith-run-id <RUN_ID>
```

Deeper docs:

- [docs/langsmith-to-mlflow-trace-strategy.md](docs/langsmith-to-mlflow-trace-strategy.md): span mapping, timestamp handling, normalization rules
- [docs/local-mlflow-dev.md](docs/local-mlflow-dev.md): local tracking store layout
- [docs/pypi-release.md](docs/pypi-release.md): TestPyPI / PyPI release workflow
