Metadata-Version: 2.4
Name: pygx
Version: 0.4.8
Summary: PyGX: A library for manipulating Python objects.
Home-page: https://github.com/google/pygx
Author: PyGX Authors
Author-email: pygx-authors@google.com
License: Apache License 2.0
Keywords: ai machine learning automl mutable symbolic framework meta-programming
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Human Machine Interfaces
Classifier: Topic :: Software Development :: Code Generators
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: docstring-parser>=0.12
Requires-Dist: pygx-core==0.4.7
Requires-Dist: termcolor>=1.1.0
Provides-Extra: all
Requires-Dist: docstring-parser>=0.12; extra == "all"
Requires-Dist: pygx-core==0.4.7; extra == "all"
Requires-Dist: termcolor>=1.1.0; extra == "all"
Requires-Dist: fsspec>=2023.3.0; extra == "all"
Requires-Dist: anyio>=4.0; extra == "all"
Requires-Dist: tqdm>=4.0; extra == "all"
Requires-Dist: cloudpickle>=3.0; extra == "all"
Requires-Dist: pyyaml>=6.0; extra == "all"
Provides-Extra: io
Requires-Dist: fsspec>=2023.3.0; extra == "io"
Provides-Extra: concurrent
Requires-Dist: anyio>=4.0; extra == "concurrent"
Requires-Dist: tqdm>=4.0; extra == "concurrent"
Provides-Extra: serialization
Requires-Dist: cloudpickle>=3.0; extra == "serialization"
Provides-Extra: yaml
Requires-Dist: pyyaml>=6.0; extra == "yaml"
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: keywords
Dynamic: license
Dynamic: license-file
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

<div align="center">


# Symbolic Object Model for Python

[**Documentation**](https://free-solo.github.io/pygx/)
| [**Search spaces**](#a-class-that-is-also-a-space-of-programs)
| [**Events**](#an-edit-re-initializes-what-it-touched)
| [**Evolution**](#programs-written-by-algorithms)
| [**Performance**](#performance)
| [**Install**](#install)

</div>

**A symbolic object can be both executed and manipulated, and the two stay in
sync.** It behaves like any Python object — methods, attributes, validation —
and it simultaneously exposes the structure it was built from, so your program
is also data your own code can query, edit, diff, and search over.

Ordinary objects are built and then sealed. `Trainer(model=Model(units=128))`
runs, and the call that produced it is gone — you can read the attributes back,
but the *structure* of how it was assembled is nowhere.

For most objects that's fine. It stops being fine the moment your program is
also something you need to **operate on**: to sweep, to diff, to patch from a
flag, to generate, to hand to a search.

A `pg.Object` keeps the call.

```python
import pygx as pg

class Model(pg.Object):
    units: int = 8

class Trainer(pg.Object):
    model: Model
    lr: float = 0.01

t = Trainer(model=Model(units=128))
t.sym_init_args          # {'model': Model(units=128), 'lr': 0.01}
```

That one difference is the whole library. Everything below is a consequence of
it — and each is something that has no straightforward answer in plain Python.

The idea has a name: **symbolic programming**, a paradigm where a program can
manipulate its own components as if they were plain data. PyGX brings it to
ordinary `class` definitions.

---

## A class that is also a space of programs

You have a config three levels deep and you want to sweep two of its values.

Ordinarily you write a loop that reconstructs the whole config per variant,
with the paths hard-coded — and every new parameter means editing that loop.

In PyGX you say it where the value lives:

```python
space = Exp(
    model=Model(
        units=pg.oneof([8, 16]),
        opt=Opt(lr=pg.oneof([0.1, 0.01])),
    )
)

list(pg.iter(space))     # 4 programs: (8, 0.1) (8, 0.01) (16, 0.1) (16, 0.01)
pg.dna_spec(space)       # the space itself, as an introspectable genotype
```

No separate search-space schema that has to be kept in sync with the config
class. No loop to rewrite when a parameter is added. The class *is* the space,
and a space is a value you can pass around, serialize, and search over — which
is what drives Google Cloud Vertex AI NAS, Pax, and Vizier.

## Programs that aren't finished yet

A hole is usually `None`, which already means four other things. Here it is a
value with a name, that you can pass around and fill in later:

```python
p = Model.partial()
pg.is_partial(p)         # True
p.sym_missing()          # {'units': MISSING_VALUE}
```

An abstract object's `__init__` body does not even run until it becomes
concrete. That is what lets a program be assembled in stages — by a user, a
config file, a search, or a model — and still be a real, checkable object at
every stage.

## Edits addressed at the whole program

An override arrives as a string: from a CLI flag, a config file, an experiment
sheet. Ordinarily you parse the path, walk it with `getattr`, `setattr` the
leaf, and validate by hand.

```python
exp.sym_rebind({'model.opt.lr': 0.5})            # validated, at any depth
exp.sym_rebind(lambda k, v, p: v * 2 if isinstance(v, int) else v)
pg.patch(exp, ['scale_lr?factor=3'])             # named, composable, serializable
```

The rule form is what makes bulk transformation possible at all: *every integer
under this tree, doubled* is one line rather than a bespoke recursive walk that
has to know your class layout.

## An edit re-initializes what it touched

This is what makes the edit above safe rather than merely convenient. A rebind
is not a `setattr` that leaves you to figure out the consequences — every
object the change reached is re-initialized, so derived state cannot go stale:

```python
class Model(pg.Object, topo=True):
    units: int = 8

    def on_sym_ready(self):              # runs at construction…
        super().on_sym_ready()
        self.scale = self.units * 2      # …and again after any rebind

m = Model(units=8)                       # scale == 16
m.sym_rebind(units=64)                   # scale == 128, without being asked
```

`on_sym_ready` fires whenever the object is concrete — at the end of `__init__`
and after every rebind that reaches it, including one addressed at an ancestor
three levels up. Anything computed from fields belongs here, and it stays
correct for free.

Three hooks, by how much you need to know:

- **`on_sym_ready`** — recompute derived members. Fires only when every field
  is present, so you never guard for half-built state.
- **`on_sym_bound`** — same timing, but fires even while the object is still
  partial. For logic that must run on an incomplete program.
- **`on_sym_change`** — receives the exact `field_updates`, so an expensive
  derivation can refresh only what the change actually touched.

Notification travels **upward**: an edit deep in a tree notifies the object,
then each ancestor, keyed by the path it saw the change at — so a holder can
invalidate a cache it computed from a child it doesn't directly own.

> **Call `super()` in these hooks.** Overriding `on_sym_change` without it
> swallows the cascade — `on_sym_ready` stops firing and derived state silently
> goes stale.

For code that cares about *position* rather than value, `on_topo_parent_change`
and `on_topo_path_change` fire when an object is adopted, moved, or detached —
the advanced end, for caches keyed on where a node sits.

## Two programs, structurally compared

```python
pg.diff(baseline, candidate)
# Exp(model=Model(units=Diff(left=8, right=16)))
```

Not a text diff of two dumps — a structural one that answers *which knob
differs*, at the position it differs. When runs are configs, this is the
question you actually ask.

## Values that resolve from where they sit

```python
class Layer(pg.Object, topo=True):
    dropout: Any = pg.symbolic.ValueFromParentChain()

Net(dropout=0.5, layer=Layer()).layer.dropout    # 0.5 — read from the ancestor
```

The alternative is threading the argument through every constructor between the
two, or a global. Here the field is optional at construction and resolved at
read time from its position in the tree.

## Code you don't own

```python
Sym = pg.symbolize(ThirdPartyClass)     # make it symbolic, no source edits

with pg.detour([(Adam, LAMB)]):         # change what a LIBRARY constructs
    third_party_training_loop()         # → returns a LAMB
```

`detour` redirects construction inside code you cannot edit — nested,
transitive, outer-scope-wins. The usual answers are monkeypatching or forking.

For functions, `pg.functor` gives you the thing Python has no construct for: a
**bound function you can hold, inspect, and rebind before calling it.**

## Provenance for generated programs

```python
with pg.track_origin():
    variant = base.sym_clone()

variant.sym_origin.source is base       # True
variant.sym_origin.tag                  # 'clone'
```

When programs are produced by other programs — mutated, evolved, sampled — the
bookkeeping of *where did this one come from* is otherwise yours to build.

## Programs written by algorithms

Once a program is a value, an algorithm can produce one. Evolution is the
clearest case: mutation and crossover are ordinary operations on the structure,
so a search algorithm needs to know nothing about *your* classes.

```python
algo = evo.regularized_evolution(evo.mutators.Uniform())

for net, feedback in pg.iter(space, 30, algo):
    feedback(score(net))          # your objective; the algorithm does the rest
```

Each `net` is a real, validated instance of your class — not a parameter vector
you have to decode. The algorithm mutates the *representation*, so the same
`Uniform()` mutator works on a neural architecture, a tour of cities, or a
symbolic expression, without being written for any of them.

Worked examples:
[OneMax](docs/notebooks/evolution/onemax.ipynb) ·
[Traveling Salesperson](docs/notebooks/evolution/tsp.ipynb) ·
[Function Regression](docs/notebooks/evolution/function_regression.ipynb) ·
[writing your own operations](docs/notebooks/intro/search/evolution_ops.ipynb)

## Seeing it

```python
pg.to_html(trainer)      # an interactive, collapsible tree
```

A nested program is unreadable as `repr` output. Any symbolic value renders as
a browsable tree, in a notebook or a file.

---

## The shape of the thing

Everything above comes from one property — the object retains its own
structure — and they compose, because they all speak about the same tree:

- a **search space** is a program with holes that stand for many values
- **materializing** one is narrowing those holes to a program
- a **partial** is a program with holes that are not yet decided
- a **patch** is a rule from program to program
- a **diff** is the difference between two of them

So a program stops being only *what your code runs* and becomes *what your code
produces*. That is the category PyGX is in — not a faster way to validate a
dataclass, but a way to write programs whose subject is other programs.

---

## Living with it

The above is the reason to reach for PyGX. This is what it's like once you have.

**It is an ordinary Python class.** No registration, no separate schema, no
config DSL. `class Model(pg.Object)` gives you a keyword-only `__init__`,
validation on construct and on assignment, JSON round-trip that restores the
real class, value equality, and `pg.diff` — and everything above already works
on it.

**The tree is opt-in.** `topo=True` adds tree positions (`topo_path`,
`topo_parent`), change notification that travels *up* through parents, and
contextual values. Same class, same callers, same field declarations — one
keyword. Objects that never need a position never pay for one.

## Performance

The symbolic model used to be a tax; it isn't anymore. The hot paths run in a
native Rust core (`pygx-core`, installed automatically), while the pure-Python
implementation remains the executable specification — the full suite runs
against **both** cores on every PR, so they cannot diverge.

Median ns/op on a 3-field object (Apple Silicon; read ratios, not absolutes —
the [full report](docs/reports/perf.md) covers ~50 operations across scales):

| operation | pygx (default) | pygx `topo=True` | `@dataclass` | pydantic v2 |
| --- | --- | --- | --- | --- |
| construct (kwargs) | 292 | 283 | 192 | 522 |
| attr get | 45 | 45 | 40 | 45 |
| clone (deep) | 915 | 882 | 2,270 | 1,620 |
| to dict/json | 389 | 388 | 631 | 532 |
| from dict/json | 977 | 982 | 196 | 656 |

Validated construction beats pydantic v2 *with the whole symbolic model
attached*, and overtakes a plain dataclass past ~8 fields; attribute reads are
at parity. Deserialization and hashing are slower — PyGX emits and dispatches a
`_type` tag so JSON round-trips back to the real class, which is strictly more
work than producing a bare dict. The report tracks all of it honestly.

Wheels ship for Linux (glibc + musl, x86_64 + aarch64), macOS (Intel + Apple
Silicon), and Windows on CPython 3.12–3.14 — plus a genuinely free-threaded
3.14t wheel (the GIL stays off; crash-freedom and per-operation atomicity per
the threading contract in [`docs/design/gil-free.md`](docs/design/gil-free.md)
§3). On anything else PyGX falls back to the pure-Python core with identical
behavior.

## When you don't need it

If your objects are only ever *built and read* — request payloads, plain
records, a config that is loaded once and never manipulated — you don't need
any of this, and a dataclass or pydantic model is the better tool. Keeping the
structure costs memory and construct time you won't spend.

PyGX earns its keep at the point where a program becomes something you operate
on. If you have never wanted to ask a program a question, you don't need it.

## Install

```bash
pip install pygx
```

Nightly build:

```bash
pip install pygx --pre
```

Optional extras:

```bash
pip install "pygx[io]"          # fsspec-backed remote IO (GCS, S3, ...)
pip install "pygx[concurrent]"  # parallel execute/map with retries + progress
```

Requires Python 3.12+.

> Upgrading from an earlier version? See the
> [0.4 migration guide](docs/guide/general/migration_0_4.md) — the `pg.Object`
> default changed from `topo=True` to `topo=False`, and the position API moved
> to `topo_*`.

## Background

PyGX was originally built at **Google Brain / DeepMind** by Daiyi Peng to power
automated machine learning (AutoML), under the name
[**PyGlove**](https://github.com/google/pyglove). The abstraction underneath —
symbolic object-oriented programming — turned out to be much more general than
AutoML, and PyGlove grew into a toolkit for advanced Python programming used
well beyond ML. The original PyGlove paper was
[published](https://proceedings.neurips.cc/paper/2020/file/012a91467f210472fab4e11359bbfef6-Paper.pdf)
at NeurIPS 2020.

## Documentation & examples

- [Bird's-eye view of PyGX](https://free-solo.github.io/pygx/notebooks/intro/birdview/)
  — 5–10 minute tour of the core ideas.
- [User Guide](https://free-solo.github.io/pygx/guide/) — topic-specific tracks
  for Python, ML, AutoML, and Evolution.
- [Learning PyGX](https://free-solo.github.io/pygx/learn/) — conceptual
  material on Symbolic Object-Oriented Programming and Symbolic Detour.
- [Authoring `pg.Object` subclasses](docs/guide/general/pg_object_style.md) —
  style guide for field defaults, `ClassVar` config, and `__init__` shape.
- [API Reference](https://free-solo.github.io/pygx/api/) — generated from
  source.

Runnable notebooks live in [`docs/notebooks/`](docs/notebooks/) and source
examples in [`examples/`](examples/).

## Citing PyGX

```
@inproceedings{peng2020pyglove,
  title={PyGlove: Symbolic programming for automated machine learning},
  author={Peng, Daiyi and Dong, Xuanyi and Real, Esteban and Tan, Mingxing and Lu, Yifeng and Bender, Gabriel and Liu, Hanxiao and Kraft, Adam and Liang, Chen and Le, Quoc},
  booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
  volume={33},
  pages={96--108},
  year={2020}
}
```

## License

Apache License 2.0. PyGX is derived from PyGlove (also Apache 2.0); see
[`LICENSE`](LICENSE) and the per-file copyright headers for attribution.

PyGX is developed by Daiyi Peng.
