Metadata-Version: 2.4
Name: jupyterlite_ai
Version: 0.9.1
Dynamic: Keywords
Summary: AI code completions and chat for JupyterLite
Project-URL: Homepage, https://github.com/jupyterlite/ai
Project-URL: Bug Tracker, https://github.com/jupyterlite/ai/issues
Project-URL: Repository, https://github.com/jupyterlite/ai.git
Author: JupyterLite Contributors
License: BSD 3-Clause License
        
        Copyright (c) 2024, JupyterLite Contributors
        Copyright (c) 2024, Jeremy Tuloup
        All rights reserved.
        
        Redistribution and use in source and binary forms, with or without
        modification, are permitted provided that the following conditions are met:
        
        1. Redistributions of source code must retain the above copyright notice, this
           list of conditions and the following disclaimer.
        
        2. Redistributions in binary form must reproduce the above copyright notice,
           this list of conditions and the following disclaimer in the documentation
           and/or other materials provided with the distribution.
        
        3. Neither the name of the copyright holder nor the names of its
           contributors may be used to endorse or promote products derived from
           this software without specific prior written permission.
        
        THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
        AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
        IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
        DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
        FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
        DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
        SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
        CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
        OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
        OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
License-File: LICENSE
Classifier: Framework :: Jupyter
Classifier: Framework :: Jupyter :: JupyterLab
Classifier: Framework :: Jupyter :: JupyterLab :: 4
Classifier: Framework :: Jupyter :: JupyterLab :: Extensions
Classifier: Framework :: Jupyter :: JupyterLab :: Extensions :: Prebuilt
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.9
Requires-Dist: jupyter-secrets-manager<0.5,>=0.4
Requires-Dist: jupyterlab-diff<0.7,>=0.6.0
Provides-Extra: jupyter
Requires-Dist: jupyterlab>=4.4.0; extra == 'jupyter'
Requires-Dist: jupyterlite>=0.6.0a0; extra == 'jupyter'
Requires-Dist: notebook>=7.4.0; extra == 'jupyter'
Provides-Extra: test
Requires-Dist: fastmcp<3,>=2.12.4; extra == 'test'
Requires-Dist: starlette<0.49,>=0.48.0; extra == 'test'
Description-Content-Type: text/markdown

# jupyterlite-ai

[![Github Actions Status](https://github.com/jupyterlite/ai/workflows/Build/badge.svg)](https://github.com/jupyterlite/ai/actions/workflows/build.yml)
[![lite-badge](https://jupyterlite.rtfd.io/en/latest/_static/badge.svg)](https://jupyterlite.github.io/ai/lab/index.html)

AI code completions and chat for JupyterLab, Notebook 7 and JupyterLite ✨

[a screencast showing the Jupyterlite AI extension in JupyterLite](https://github.com/user-attachments/assets/e33d7d84-53ca-4835-a034-b6757476c98b)

## Requirements

- JupyterLab >= 4.4.0 or Notebook >= 7.4.0

## ✨ Try it in your browser ✨

You can try the extension in your browser using JupyterLite:

[![lite-badge](https://jupyterlite.rtfd.io/en/latest/_static/badge.svg)](https://jupyterlite.github.io/ai/lab/index.html)

See the [Usage](#usage) section below for more information on how to provide your API key.

## Install

To install the extension, execute:

```bash
pip install jupyterlite-ai
```

To install requirements (JupyterLab, JupyterLite and Notebook):

```bash
pip install jupyterlite-ai[jupyter]
```

## Usage

> [!NOTE]
> This documentation applies to the upcoming **0.9.0** release.
> For the latest stable version, please refer to the [0.8.x branch](https://github.com/jupyterlite/ai/tree/0.8.x).

AI providers typically require using an API key to access their models.

The process is different for each provider, so you may refer to their documentation to learn how to generate new API keys.

### Using a provider with an API key (e.g. Anthropic, MistralAI, OpenAI)

1. Open the AI settings and
2. Click on "Add a new provider"
3. Enter the details for the provider
4. In the chat, select the new provider

![screenshot showing the dialog to add a new provider](https://github.com/user-attachments/assets/823c71c6-5807-44c8-80b6-2e59379a65d5)

### Using a generic OpenAI-compatible provider

The Generic provider allows you to connect to any OpenAI-compatible API endpoint, including local servers like Ollama and LiteLLM.

1. In JupyterLab, open the AI settings panel and go to the **Providers** section
2. Click on "Add a new provider"
3. Select the **Generic (OpenAI-compatible)** provider
4. Configure the following settings:
   - **Base URL**: The base URL of your API endpoint (suggestions are provided for common local servers)
   - **Model**: The model name to use
   - **API Key**: Your API key (if required by the provider)

### Using Ollama

[Ollama](https://ollama.com/) allows you to run open-weight LLMs locally on your machine.

#### Setting up Ollama

1. Install Ollama following the instructions at <https://ollama.com/download>
2. Pull a model, for example:

```bash
ollama pull llama3.2
```

3. Start the Ollama server (it typically runs on `http://localhost:11434`)

#### Configuring `jupyterlite-ai` to use Ollama

1. In JupyterLab, open the AI settings panel and go to the **Providers** section
2. Click on "Add a new provider"
3. Select the **Generic (OpenAI-compatible)** provider
4. Configure the following settings:
   - **Base URL**: Select `http://localhost:11434/v1` from the suggestions (or enter manually)
   - **Model**: The model name you pulled (e.g., `llama3.2`)
   - **API Key**: Leave empty (not required for Ollama)

### Using LiteLLM Proxy

[LiteLLM Proxy](https://docs.litellm.ai/docs/simple_proxy) is an OpenAI-compatible proxy server that allows you to call 100+ LLMs through a unified interface.

Using LiteLLM Proxy with jupyterlite-ai provides flexibility to switch between different AI providers (OpenAI, Anthropic, Google, Azure, local models, etc.) without changing your JupyterLite configuration. It's particularly useful for enterprise deployments where the proxy can be hosted within private infrastructure to manage external API calls and keep API keys server-side.

#### Setting up LiteLLM Proxy

1. Install LiteLLM:

Follow the instructions at <https://docs.litellm.ai/docs/simple_proxy>.

2. Create a `litellm_config.yaml` file with your model configuration:

```yaml
model_list:
  - model_name: gpt-5
    litellm_params:
      model: gpt-5
      api_key: os.environ/OPENAI_API_KEY

  - model_name: claude-sonnet
    litellm_params:
      model: claude-sonnet-4-5-20250929
      api_key: os.environ/ANTHROPIC_API_KEY
```

3. Start the proxy server, for example:

```bash
litellm --config litellm_config.yaml
```

The proxy will start on `http://0.0.0.0:4000` by default.

#### Configuring `jupyterlite-ai` to use LiteLLM Proxy

Configure the [Generic provider (OpenAI-compatible)](#using-a-generic-openai-compatible-provider) with the following settings:

- **Base URL**: `http://0.0.0.0:4000` (or your proxy server URL)
- **Model**: The model name from your `litellm_config.yaml` (e.g., `gpt-5`, `claude-sonnet`)
- **API Key (optional)**: If the LiteLLM Proxy server requires an API key, provide it here.

> [!IMPORTANT]
> The API key must be configured on the LiteLLM Proxy server (in the `litellm_config.yaml` file). Providing an API key via the AI provider settings UI will not have any effect, as the proxy server handles authentication with the upstream AI providers.

> [!NOTE]
> For more information about LiteLLM Proxy configuration, see the [LiteLLM documentation](https://docs.litellm.ai/docs/simple_proxy).

## Custom Providers

`jupyterlite-ai` supports custom AI providers through its provider registry system. Third-party providers can be registered programmatically in a JupyterLab extension.

Providers are based on the [Vercel AI SDK](https://sdk.vercel.ai/docs/introduction), which provides a unified interface for working with different AI models.

### Registering a Custom Provider

#### Example: Registering a custom OpenAI-compatible provider

```typescript
import {
  JupyterFrontEnd,
  JupyterFrontEndPlugin
} from '@jupyterlab/application';
import { IProviderRegistry } from '@jupyterlite/ai';
import { createOpenAI } from '@ai-sdk/openai';

const plugin: JupyterFrontEndPlugin<void> = {
  id: 'my-extension:custom-provider',
  autoStart: true,
  requires: [IProviderRegistry],
  activate: (app: JupyterFrontEnd, registry: IProviderRegistry) => {
    const providerInfo = {
      id: 'my-custom-provider',
      name: 'My Custom Provider',
      apiKeyRequirement: 'required' as const,
      defaultModels: ['my-model'],
      supportsBaseURL: true,
      factory: (options: {
        apiKey: string;
        baseURL?: string;
        model?: string;
      }) => {
        const provider = createOpenAI({
          apiKey: options.apiKey,
          baseURL: options.baseURL || 'https://api.example.com/v1'
        });
        return provider(options.model || 'my-model');
      }
    };

    registry.registerProvider(providerInfo);
  }
};
```

The provider configuration object requires the following properties:

- `id`: Unique identifier for the provider
- `name`: Display name shown in the settings UI
- `apiKeyRequirement`: Whether an API key is `'required'`, `'optional'`, or `'none'`
- `defaultModels`: Array of model names to show in the settings
- `supportsBaseURL`: Whether the provider supports a custom base URL
- `factory`: Function that creates and returns a language model (the registry automatically wraps it for chat usage)

#### Example: Using a custom fetch function

You can provide a custom `fetch` function to the provider, which is useful for adding custom headers, handling authentication, or routing requests through a proxy:

```typescript
factory: (options: { apiKey: string; baseURL?: string; model?: string }) => {
  const provider = createOpenAI({
    apiKey: options.apiKey,
    baseURL: options.baseURL || 'https://api.example.com/v1',
    fetch: async (url, init) => {
      // Custom fetch implementation
      const modifiedInit = {
        ...init,
        headers: {
          ...init?.headers,
          'X-Custom-Header': 'custom-value'
        }
      };
      return fetch(url, modifiedInit);
    }
  });
  return provider(options.model || 'my-model');
};
```

## API key management

To avoid storing the API keys in the settings, `jupyterlite-ai` uses [jupyter-secrets-manager](https://github.com/jupyterlab-contrib/jupyter-secrets-manager) by default.

The secrets manager get the API keys from a connector in a secure way.\
The default connector of the secrets manager is _in memory_, which means that **the API keys are reset when reloading the page**.

To prevent the keys from being reset on reload, there are two options:

1. use a connector that fetches the keys on a remote server (using secure rest API, or web socket)

This is the recommended method, as it ensures the security of the keys and makes them accessible only to logged-in users. \
But it requires some frontend and backend deployments:

- a server that can store and send the keys on demand
- a way to get authenticated to the server
- a frontend extension providing the connector, able to connect to the server side

2. disable the use of the secrets manager from the AI settings panel

> [!WARNING]
> The API keys will be stored in plain text using the settings system of Jupyterlab
>
> - using Jupyterlab, the settings are stored in a [directory](https://jupyterlab.readthedocs.io/en/stable/user/directories.html#jupyterlab-user-settings-directory) on the server
> - using Jupyterlite, the settings are stored in the [browser](https://jupyterlite.readthedocs.io/en/latest/howto/configure/storage.html#configure-the-browser-storage)

## Uninstall

To remove the extension, execute:

```bash
pip uninstall jupyterlite-ai
```

## Contributing

See [CONTRIBUTING](CONTRIBUTING.md)
