Metadata-Version: 2.4
Name: llm-togetherai
Version: 0.1.0
Summary: LLM plugin for models hosted by Together AI
Author: Rajashekar Chintalapati
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/rajashekar/llm-togetherai
Project-URL: Changelog, https://github.com/rajashekar/llm-togetherai/releases
Project-URL: Issues, https://github.com/rajashekar/llm-togetherai/issues
Project-URL: CI, https://github.com/rajashekar/llm-togetherai/actions
Keywords: llm,ai,together,language-models,plugin
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: llm>=0.23
Requires-Dist: httpx
Requires-Dist: openai>=1.57.0
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Requires-Dist: pytest-recording; extra == "test"
Requires-Dist: inline-snapshot; extra == "test"
Dynamic: license-file

# llm-togetherai

[![PyPI](https://img.shields.io/pypi/v/llm-togetherai.svg)](https://pypi.org/project/llm-togetherai/)
[![Changelog](https://img.shields.io/github/v/release/rajashekar/llm-togetherai?include_prereleases&label=changelog)](https://github.com/rajashekar/llm-togetherai/releases)
[![Tests](https://github.com/rajashekar/llm-togetherai/workflows/Test/badge.svg)](https://github.com/rajashekar/llm-togetherai/actions?query=workflow%3ATest)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/rajashekar/llm-togetherai/blob/main/LICENSE)

[LLM](https://llm.datasette.io/) plugin for models hosted by [Together AI](https://www.together.ai/)

## Installation

First, [install the LLM command-line utility](https://llm.datasette.io/en/stable/setup.html).

Now install this plugin in the same environment as LLM.
```bash
llm install llm-togetherai
```

## Configuration

You will need an API key from Together AI. You can [obtain one here](https://api.together.xyz/settings/api-keys).

You can set that as an environment variable called `TOGETHER_API_KEY`, or add it to the `llm` set of saved keys using:

```bash
llm keys set together
```
```
Enter key: <paste key here>
```

## Usage

To list available models, run:
```bash
llm models list
```
You should see a list that looks something like this:
```
together: together/meta-llama/Llama-2-7b-chat-hf
together: together/meta-llama/Llama-2-13b-chat-hf
together: together/meta-llama/Llama-2-70b-chat-hf
together: together/mistralai/Mistral-7B-Instruct-v0.1
together: together/NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO
...
```

To run a prompt against a model, pass its full model ID to the `-m` option, like this:
```bash
llm -m together/meta-llama/Llama-2-7b-chat-hf "Five creative names for a pet robot"
```

You can set a shorter alias for a model using the `llm aliases` command like so:
```bash
llm aliases set llama2-7b together/meta-llama/Llama-2-7b-chat-hf
```
Now you can prompt the model using:
```bash
cat llm_togetherai.py | llm -m llama2-7b -s 'write some pytest tests for this'
```

### Vision models

Some Together AI models can accept image attachments. Run this command:

```bash
llm models --options -q together
```
And look for models that list these attachment types:

```
  Attachment types:
    image/gif, image/jpeg, image/png, image/webp
```

You can feed these models images as URLs or file paths, for example:

```bash
curl https://static.simonwillison.net/static/2024/pelicans.jpg | llm \
    -m together/meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo 'describe this image' -a -
```

### Listing models

The `llm models -q together` command will display all available models, or you can use this command to see more detailed information:

```bash
llm together models
```
Output starts like this:
```yaml
- id: meta-llama/Llama-2-7b-chat-hf
  name: Llama-2-7b-chat-hf
  context_length: 4,096
  type: chat
  organization: Together
  pricing: input $0.2/M, output $0.2/M

- id: meta-llama/Llama-2-13b-chat-hf
  name: Llama-2-13b-chat-hf
  context_length: 4,096
  type: chat
  organization: Together
  pricing: input $0.3/M, output $0.3/M
```

Add `--json` to get back JSON instead:
```bash
llm together models --json
```

### Refreshing the model cache

The plugin caches the list of available models for 1 hour. To refresh this cache manually:

```bash
llm together refresh
```

This will fetch the latest models from the Together AI API and update the local cache.

## API Endpoint

This plugin uses the Together AI API endpoint:
```
https://api.together.xyz/v1/models
```

The models are cached locally in your LLM user directory to improve performance and reduce API calls.

## Development

To set up this plugin locally, first checkout the code. Then create a new virtual environment:
```bash
cd llm-togetherai
python3 -m venv venv
source venv/bin/activate
```
Now install the dependencies and test dependencies:
```bash
llm install -e '.[test]'
```
To run the tests:
```bash
pytest
```

## License

Apache 2.0
