Metadata-Version: 2.4
Name: ietf-notebook
Version: 0.3.1
Summary: Gather resources for an IETF Working Group for use in NotebookLM
Author-email: Mark Nottingham <mnot@mnot.net>
License: Copyright (c) Mark Nottingham
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in
        all copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
        THE SOFTWARE.
        
Project-URL: homepage, https://github.com/mnot/ietf-notebook
Classifier: Operating System :: OS Independent
Classifier: Development Status :: 4 - Beta
Classifier: License :: OSI Approved :: MIT License
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE.md
Requires-Dist: requests
Requires-Dist: beautifulsoup4
Requires-Dist: google-auth
Requires-Dist: google-auth-oauthlib
Requires-Dist: google-auth-httplib2
Provides-Extra: certs
Requires-Dist: pip-system-certs; extra == "certs"
Provides-Extra: dev
Requires-Dist: mypy; extra == "dev"
Requires-Dist: black; extra == "dev"
Requires-Dist: pylint; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-md; extra == "dev"
Requires-Dist: validate-pyproject; extra == "dev"
Requires-Dist: build; extra == "dev"
Requires-Dist: types-requests; extra == "dev"
Requires-Dist: types-beautifulsoup4; extra == "dev"
Dynamic: license-file

# ietf-notebook

Automate gathering of [NotebookLM](https://notebooklm.google.com/)-ready documents for an [IETF](https://www.ietf.org/) Working Group.

This tool gathers Working Group charters, drafts, meeting minutes, PDF slides, meeting transcripts, mailing list archives, and GitHub issues into a set of clean text files and PDFs suitable for ingestion into NotebookLM.

## Installation

```bash
pipx install ietf-notebook
```

### Certificate Errors

If you encounter SSL or certificate errors (common behind corporate firewalls), install with the `certs` option:

```bash
pipx install ietf-notebook[certs]
```

## Usage

### First Run

To start collecting documents for a Working Group, use the `--destination` flag to specify where the documents should be stored. This will create a directory with the WG name and populate it with the WG charter, meeting minutes, slides, transcripts, mailing list archives, and GitHub issues.

Because `ietf-notebook` persists Working Group configuration options, you don't need to specify them again for that Working Group. Use `--clear-config` to reset a group's configuration.

```bash
ietf-notebook [OPTIONS] --destination _destination_ _wg_shortname_
```

Then, upload all of the files in _destination_ to NotebookLM.

### Subsequent Updates

To update the documents, run the same command again. _destination_ will only contain files that have changed since the last run. Upload the new and updated files to NotebookLM.

```bash
ietf-notebook _wg_shortname_
```


### Options

Working Group-specific:
- `wg_shortname`: IETF Working Group short name (e.g., `httpbis`).
- `--destination`: Folder to populate with group records (NOTE: contents will be deleted on each run).
- `--github`: GitHub org/repo for issues (e.g., `ietf-wg-httpbis/wg-materials`).
- `--github-label`: Include only GitHub issues with this label (can be specified multiple times).
- `--exclude-github-label`: Exclude GitHub issues with this label (can be specified multiple times).
- `--months`: Number of months of mailing list history to fetch (default: 12).
- `--create`: See "NotebookLM Export" below.
- `--clear-config`: Clear the persisted configuration for this Working Group.
- `--clear-cache`: Clear the local file cache and re-download everything from scratch.

General options:
- `--quiet`: No messages except for errors and the final resource summary.
- `--verbose`: Detailed progress reporting.


### Default Behavior

- **Selective Mirroring**: The `--destination` folder is cleared at the start of each run. It is then populated **only** with files that were updated or newly created in the local cache during that run.
- **File Caching**: All documents are collected in `~/.cache/ietf-notebook/[wg]/files/` to avoid redundant downloads.
- **Charters, Meetings, and Documents**: Existing files in the cache are skipped unless `--clear-cache` is used.
- **Mailing List Discovery**: The tool automatically finds the mailing list for the WG from the Datatracker.
- **IMAP Retrieval**: Mailing list archives are fetched via IMAP from `imap.ietf.org` and cached locally in `~/.cache/ietf-notebook/{wg_name}/imap-cache/`.
- **GitHub Strategy**: The tool first checks for `archive.json` on the `gh-pages` branch.
- **Transcripts**: Meeting transcripts are fetched from the `ietf-minutes-data` repository and cached locally in `~/.cache/ietf-notebook/{wg_name}/transcript-cache/`.
- **GitHub Auth**: To avoid rate limits when fetching from the API, set the `GITHUB_TOKEN` environment variable.
- **NotebookLM Export**: Use the `--create` flag to automatically create a new notebook in NotebookLM Enterprise and upload all generated archives as sources.

### NotebookLM Export (Enterprise only)

If you have a Google Workspace Enterprise account with NotebookLM enabled, you can programmatically create a notebook and upload your gathered resources.

```bash
ietf-notebook httpbis --create [MY_PROJECT_ID]
```

**Requirements:**
1.  **Google Cloud Project**: You must have a GCP project with the **Discovery Engine API** enabled.
2.  **OAuth Credentials**: You need an "OAuth 2.0 Client ID" (Type: Desktop App) from the [Google Cloud Console](https://console.cloud.google.com/apis/credentials).
3.  **Client Secrets**: Save the JSON file as `client_secrets.json` in `~/.config/ietf-notebook/` (or specify its path with `--credentials-file`).

The first time you run this, a browser window will open to authorize the application. Your access permissions will be cached in `~/.config/ietf-notebook/token.json` (or you can specify with `--token-file`).

## Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
