Usage
Optimize a representative signature.
Supply the model and example positional or keyword tensors exactly as serving code will call them. Selection remains specific to those shapes, dtypes, device, and software versions.
from custom_dl_optimizer import OptimizationConfig, Optimizer
optimizer = Optimizer(
device="cuda",
config=OptimizationConfig(
enable_compile=True,
min_speedup=1.02,
),
)
result = optimizer.optimize(model.eval(), sample)
output = result(sample)
result.save_report("artifacts/optimization.json")
Read the decision
Each candidate records steady-state latency, setup time, calls per second, speedup against eager and native, parity, numerical error, and any failure. The parent report records graph rewrites and runtime provenance.
Correctness boundary
Candidates must preserve output structure and satisfy torch.allclose using configured tolerances. Always run task-level accuracy evaluation before deployment.
Candidate providers
Compare another compiler under the same policy.
A provider declares a unique name, availability check, and builder. The builder receives an isolated model plus prepared example inputs and returns a callable nn.Module.
from custom_dl_optimizer import FunctionCandidateProvider, Optimizer
provider = FunctionCandidateProvider(
name="my_backend",
availability=lambda context: context.device.type == "cuda",
builder=lambda model, context: compile_model(
model,
context.example_args,
),
)
result = Optimizer(
device="cuda",
providers=(provider,),
).optimize(model, sample)
Fair comparison
Keep transfers and precision consistent, separate engine-build cost from steady-state latency, warm caches, expose provider configuration, and never silently run another backend under the same provider name.
Agent toolkit
Register objects locally; expose names remotely.
Live modules and tensors stay in the host process. The toolkit exposes only runtime inspection, workload listing, optimization, and report retrieval.
from custom_dl_optimizer import OptimizationAgentToolkit, Optimizer
toolkit = OptimizationAgentToolkit(Optimizer(device="cuda"))
toolkit.register_workload(
"encoder-b16",
encoder,
tokens,
attention_mask=mask,
)
schemas = toolkit.tool_schemas()
report = toolkit.invoke(
"custom_dl_optimize",
{"workload": "encoder-b16"},
)
Security boundary
There is no tool argument for a file path, import, download, shell command, Python expression, model weight, or credential. The host remains responsible for deciding which workload names an agent may access.
Migration
Move from a tuple to one result object.
Version 1 constructed AutoOptimizer around a model and returned a module/report tuple. Version 2 constructs reusable optimizer policy and returns OptimizationResult.
# Version 1
engine = AutoOptimizer(model, config=config)
module, report = engine.optimize_with_report(sample)
# Version 2
optimizer = Optimizer(config=config)
result = optimizer.optimize(model, sample)
module = result.module
report = result.report
AutoOptimizer remains available for one transition release. New integrations should use the version 2 result contract.