Metadata-Version: 2.4
Name: tagpipeline
Version: 0.0.2
Summary: Attribute tagging pipeline built on the tagstudio and identity_mapping modules
Author: sanchi
Author-email: sanchi@flixstock.com
License: LicenseRef-Flixstock-Proprietary
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: POSIX :: Linux
Requires-Python: >=3.11,<3.13
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: google-genai<2.0,>=1.52
Requires-Dist: google-api-core<3.0,>=2.24
Requires-Dist: Pillow<12.0,>=11.0
Requires-Dist: jsonschema<5.0,>=4.20
Requires-Dist: pypdf<6,>=4
Requires-Dist: opencv-python>=4.8.0
Requires-Dist: numpy<2.0.0,>=1.24.0
Requires-Dist: pandas>=2.0.0
Requires-Dist: mediapipe>=0.10.0; python_version < "3.13"
Requires-Dist: requests>=2.31.0
Requires-Dist: tqdm>=4.0.0
Requires-Dist: termcolor>=2.0.0
Requires-Dist: boto3>=1.34
Provides-Extra: excel
Requires-Dist: openpyxl>=3.1; extra == "excel"
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: license
Dynamic: license-file
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# tagpipeline

Turns a brand's SKU image bundle into structured, webshop-ready product tags.

Given one folder of images per SKU and a brand metadata CSV, `tagpipeline` runs
three stages in order — keyed by SKU folder name — and merges their results into
a single per-SKU record plus a flat CSV/Excel deliverable.

## The workflow

```
SKU images + metadata CSV
        |
        v
1. identity mapping   detects and crops faces locally, then matches them against
                      reference embeddings served by the description-model API.
                      Yields the model featured in the SKU, or None.
        |
        v
2. tag generation     three LLM steps via the tagstudio driver (Gemini):
                      file_descriptions -> taxonomy_generation -> attribute_generation.
                      Yields a gender/category/subcategory triple plus a set of
                      validated product attributes.
        |
        v
3. taxonomy mapping   deterministic, no LLM. Resolves the predicted triple into
                      the client's own webshop categories via a reverse mapping.
        |
        v
combined_results.json + final_results.csv + final_results.xlsx
```

## Outputs

- **`combined_results.json`** — the merged per-SKU record (`model_identity`,
  `tags`, `taxonomy_mapping`).
- **`final_results.csv`** — long/tidy format, one row per SKU *and* attribute:
  `sku_id, model_name, model_height, pred_gender, pred_category,
  pred_subcategory, webshop_gender, webshop_sub_gender, descriptive_categories,
  attribute_name, attribute_value`.
- **`final_results.xlsx`** — the same rows as a banded Excel view (needs the
  `excel` extra).
- Per-SKU intermediates under `tag_gen/` and `identity/`.

Output accumulates: re-running a SKU replaces only that SKU's entry, leaving
every other SKU already in the output directory untouched.

## SKU boundaries

A SKU is a folder, and the folder's name is its id. Every file inside it belongs
to that SKU — loose images and nested subfolders alike, at any depth — so
bundles that group shots into `angle_image_urls/` or `detail/closeup/` are
handled without those subfolders being mistaken for separate SKUs.

## Configuration

The brand framework, taxonomy reverse-mapping, tag-generation pipeline config
and both API keys are fetched from S3 at runtime and cached locally. The cache is
hash-verified (S3 ETag for upstream changes, SHA-256 for local integrity), and a
cached asset costs no S3 call at all — so a warm run needs no network and no
credentials. Credentials are never bundled: supply them via `AWS_ACCESS_KEY_ID` /
`AWS_SECRET_ACCESS_KEY` or `$TAGPIPELINE_AWS_SECRETS`.

## Entry points

```bash
# full run over a bundle, assets pulled from S3
python pipeline.py --skus-dir /abs/skus --metadata-csv /abs/skus.csv \
                   --s3-client BST --out ./runs

# fetch or inspect the S3 config assets on their own
python -m tagpipeline.s3_assets --client BST --cache-status

# rebuild the flat CSV from an existing run
python build_final_csv.py --combined ./runs/combined_results.json
```

`run_pipeline()` is the programmatic API; `pipeline_api.main()` adds structural
validation and a per-SKU PASS/FAIL report.

## Requirements

Python 3.11–3.12 (MediaPipe does not support 3.13). Tag generation makes real
Gemini API calls and identity mapping calls the description-model API; both incur
cost. Access to the configured S3 bucket is required.

---

Proprietary software of Flixstock, Inc. — see `LICENSE`. Publication on a package
index does not place it in the public domain or license it to the general public.
