Metadata-Version: 2.4
Name: baseten_inference_client
Version: 0.0.1rc1
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
Summary: A ultra-high performance package for sending requests to Baseten Embedding Inference'
Author-email: Michael Feil <no-reply@baseten.co>
License: MIT
Requires-Python: >=3.8
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Issues, https://github.com/basetenlabs/truss/issues
Project-URL: Repository, https://github.com/basetenlabs/truss.git

# High performance client for Baseten-Embeddings

Usage with Baseten Embeddings
```
pip install truss_client_bei
```

```python
from truss_client_bei import OpenAIEmbeddingsResponse, RerankResponse, SyncClient

api_key = os.environ.get("BASETEN_API_KEY")
api_base_embed = "https://model-yqv0rjjw.api.baseten.co/environments/production"

client = SyncClient(api_base=api_base_embed, api_key=api_key)

assert client.api_key == api_key
response = client.embed(
    ["Hello world", "Hello world 2", ".."],
    model="my_model",
    # mini batch size
    batch_size=4,
    # send up to 32 mini-batch-sizes at once.
    max_concurrent_requests=32,
    # set a timeout
    timeout_s=360,
)
```

### Develop

```
apt-get install patchelf
pip install maturin[patchelf]
maturin develop
pytest tests
```

