polgrad
Copyright 2026 Saif Punjwani

Licensed under the Apache License, Version 2.0.

This product vendors code from the following Apache-2.0 licensed projects for
conformance testing, under src/polgrad/conformance/_vendor/. Each vendored
file's header records the pinned upstream commit, source paths, upstream file
SHA256s, the definitions kept, and every edit made (import/decorator stripping
only; kept bodies are byte-identical to upstream).

1. verl
   https://github.com/volcengine/verl
   (redirects to https://github.com/verl-project/verl after the project rename)
   Copyright 2024 Bytedance Ltd. and/or its affiliates
   Copyright 2022 The HuggingFace Team. All rights reserved.
   Licensed under the Apache License, Version 2.0.
   Vendored file: src/polgrad/conformance/_vendor/verl_core_algos.py
   Sources: verl/trainer/ppo/core_algos.py and verl/utils/torch_functional.py
   at commit 74a718a492092312f1004fe25369975137388849.
   SHA256 of the vendored file as committed:
   54332baf8da9e821bdb3cd5738d7c74018f25d0df51e4d91e193b82db29bbe3a

2. OpenRLHF
   https://github.com/OpenRLHF/OpenRLHF
   Licensed under the Apache License, Version 2.0 (the upstream LICENSE keeps
   the unmodified Apache-2.0 template copyright line and the source files carry
   no per-file license header; copyright rests with the OpenRLHF contributors).
   Vendored file: src/polgrad/conformance/_vendor/openrlhf_loss.py
   Sources: openrlhf/models/loss.py and openrlhf/models/utils.py
   at commit bc71bb19464aca306b33080b2d2bb45d154e2f49.
   SHA256 of the vendored file as committed:
   caad2b3b7cd6f0d873b71f765d0ad94e7d5da4e19b9a8e0186ed105b9c6ba9f0

This product also contains a labeled reimplementation (not vendored code) of
the TRL GRPO loss in src/polgrad/conformance/harness.py whose clipped-surrogate
arithmetic is reproduced verbatim from trl/trainer/grpo_trainer.py
(https://github.com/huggingface/trl) at commit
95809b942eb5d11d0b06d749510d88be99230b73 (upstream file SHA256
52d9a6c1e298df35d0da4a6fa17874d750ee627f6ac15393c8860d74d1ba4917).
Copyright 2020-2026 The HuggingFace Team. All rights reserved.
Licensed under the Apache License, Version 2.0.

This product also contains labeled reimplementations (not vendored code) of
verl's GSPO and CISPO policy losses in src/polgrad/conformance/harness.py
(_verl_gspo_loss and _verl_cispo_loss), whose arithmetic is reproduced verbatim
from compute_policy_loss_gspo and compute_policy_loss_cispo in
verl/trainer/ppo/core_algos.py (https://github.com/volcengine/verl) at the same
pinned commit 74a718a492092312f1004fe25369975137388849 (upstream file SHA256
9114f9e16c87e4c9ebf2fa016baf733c9bbc819766b53c8968aaa9e8abcd7916).
Copyright 2024 Bytedance Ltd. and/or its affiliates.
Licensed under the Apache License, Version 2.0.
