Metadata-Version: 2.4
Name: shidoshi
Version: 0.0.1
Summary: An opinionated way to augment Jupyter Lab for iterative work
Project-URL: Homepage, https://github.com/tdchaitanya/shidoshi
Project-URL: Repository, https://github.com/tdchaitanya/shidoshi
Project-URL: Issues, https://github.com/tdchaitanya/shidoshi/issues
Author-email: Chaitanya <tdchaitanya@gmail.com>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: ai-assistant,jupyter,jupyterlab,llm,openai
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Jupyter
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.13
Requires-Dist: ipylab>=0.6
Requires-Dist: ipynbname>=2025.8.0.0
Requires-Dist: ipython>=8.39.0
Requires-Dist: openai>=2.41.1
Requires-Dist: trio>=0.25
Requires-Dist: typing-extensions>=4.0
Provides-Extra: notebook
Requires-Dist: ipywidgets>=8.0; extra == 'notebook'
Description-Content-Type: text/markdown

# shidoshi

[![PyPI](https://img.shields.io/pypi/v/shidoshi.svg)](https://pypi.org/project/shidoshi/)
[![CI](https://github.com/tdchaitanya/shidoshi/actions/workflows/test.yml/badge.svg)](https://github.com/tdchaitanya/shidoshi/actions/workflows/test.yml)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)

An opinionated way to augment Jupyter Lab for iterative work.

shidoshi adds `%ask` / `%%ask` magics to Jupyter that let you talk to an LLM
from inside a notebook — using the notebook itself, in order, as the
conversation history. No separate chat pane, no copy-pasting context: your
code cells, their outputs, and your notes are the context.

## Install

Requires Python ≥3.13 and JupyterLab. Set an API key before use:

```bash
export OPENAI_API_KEY=sk-...       # for the openai provider (default)
export OPENAI_BASE_URL=...         # optional, e.g. to point at a proxy
export OPENROUTER_API_KEY=...      # for the openrouter provider
```

### Installing across Jupyter environments

`%load_ext shidoshi` runs `import shidoshi` **inside the running kernel
process**. That means shidoshi has to be installed into whichever Python
environment the kernel you're using actually runs in — it is not a
standalone CLI, so `uvx` / `uv tool install` (which run a tool in an
isolated subprocess, separate from any kernel) don't apply here.

- **Per-project venv with its own JupyterLab** (e.g. a `uv`-managed project):
  add shidoshi as a normal dependency of that project.

  ```bash
  uv add shidoshi
  # or: pip install shidoshi
  ```

- **One shared JupyterLab, many kernels** (each notebook's kernel points at a
  different project venv registered via `ipykernel install`): install
  shidoshi into *each* kernel's venv. Installing it only where JupyterLab
  itself lives will not make it importable from other kernels.

  ```bash
  # inside the venv backing a given kernel
  uv add shidoshi
  # or: pip install shidoshi
  ```

- **Auto-load without typing `%load_ext` every time**: add to
  `~/.ipython/profile_default/ipython_config.py` (create it first with
  `ipython profile create` if it doesn't exist):

  ```python
  c.InteractiveShellApp.extensions = ["shidoshi"]
  ```

  `~/.ipython` is a single directory shared by every Python environment
  under your `$HOME` — it is **not** per-venv. So this line will make every
  kernel try to load shidoshi at startup, regardless of which venv it's
  running in. Two ways to keep that safe:

  1. Only add it once shidoshi is installed in **every** environment you use
     for Jupyter on that machine, or
  2. Scope it to specific projects: create a named profile
     (`ipython profile create shidoshi`), put the `extensions` line in that
     profile's config instead of `profile_default`, and point only the
     relevant kernel(s) at it (add `"--profile=shidoshi"` to that kernel's
     `argv` in its `kernel.json`, or install with
     `ipython kernel install --profile=shidoshi ...`).

## Quickstart

```
%load_ext shidoshi
```

```
%%ask
What does the `history.build_history` function in this file do?
```

The response streams into the cell's output as Markdown.

## Magics reference

- **`%ask <prompt>`** — line magic for a one-line prompt.
  - Prefix with `model|` or `provider:model|` to override the configured
    default model for just this call, e.g. `%ask openrouter:openai/gpt-4o|summarize this`.
  - Add `--debug` anywhere on the line to also show the full request/response
    payload.
- **`%%ask [model]`** — cell magic; the whole cell body is the prompt
  (multi-line is fine, and it can reference images via Markdown
  `![]()`/`<img>` syntax or bare local file paths — they're inlined as
  base64). An optional model name on the magic line overrides the default
  for this call. Also supports `--debug`.
- **`%%skip`** — runs the cell normally, but the cell is left out of the
  context sent to the model entirely. Use it for scratch or exploratory
  cells you don't want the model to see.
- **`%%pin`** — runs the cell normally; its content and output are *always*
  included in context and are exempt from the auto-trim behavior below. Use
  it to protect a fact, constant, or definition you don't want dropped over
  a long session.

## How context is built

Every prior cell in the notebook — up to the one you're currently running,
and accounting for kernel restarts — is turned into conversation history
automatically:

- **Markdown cells** become background text/image context (treated as notes
  or reference material, not instructions).
- **Regular code cells** appear as fenced code plus their text/image
  outputs.
- **Prior `%ask` / `%%ask` cells** become real user/assistant turns. Their
  responses are reused from a per-cell cache rather than re-sent, so
  replaying history doesn't resend answers the model already produced.
- **`%%skip` cells** are dropped entirely.
- **`%%pin` cells** are always kept.

## Automatic context-length handling

If a request is rejected for exceeding the model's context window, shidoshi
automatically retries, dropping the oldest trimmable history units first
(markdown cells, then plain code cells, then whole ask+response pairs —
`%%pin` cells are never dropped), up to 20 times. A banner reports how many
cells were dropped so you know context shrank.

## Providers & tools

- **`openai`** (default) — uses the OpenAI Responses API.
- **`openrouter`** — uses OpenRouter's chat-completions API; select it with
  the `provider:model` prefix, e.g. `openrouter:anthropic/claude-3.5-sonnet`.

Every request currently has the built-in `web_search` tool attached, so the
model can search the web when it needs current information. (A `web_fetch`
tool also exists in the codebase but isn't wired into the magics yet — not
available today.)

## Debug mode

Add `--debug` to `%ask`/`%%ask` to render a collapsible, syntax-highlighted
panel showing exactly what was sent (system prompt, full message history,
tools) and every raw event streamed back — useful when the model's behavior
is surprising and you want to see the actual payload.

## Configuration

shidoshi reads TOML config, layered as defaults → `~/.shidoshi/config.toml`
→ `./.shidoshi/config.toml` (project config overrides user config):

```toml
default_model = "gpt-5.5"   # model used when none is specified
reasoning_effort = "low"    # OpenAI only
ask_color = "#eafbea"       # highlight color for %ask/%%ask cells
skip_color = "#ececec"      # highlight color for %%skip cells
```

## Development

```bash
uv sync
uv run pytest tests/unit tests/btp -v
```

Integration tests under `tests/integration/` require a live `OPENAI_API_KEY`
(or a proxy via `OPENAI_BASE_URL`) and are run with:

```bash
uv run pytest tests/integration/ -v -m integration
```

## License

Apache License 2.0 — see [LICENSE](LICENSE).
