Metadata-Version: 2.4
Name: lograder
Version: 0.2.0
Summary: A general-purpose autograder written primarily in Python 3.
Author-email: Logan Dapp <logan@logand.app>
License: MIT
Project-URL: Homepage, https://github.com/lognd/lograder
Project-URL: Documentation, https://github.com/lognd/lograder
Project-URL: Source, https://github.com/lognd/lograder
Project-URL: Issues, https://github.com/lognd/lograder/issues
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: tomli>=2.0.0; python_version < "3.11"
Requires-Dist: StrEnum>=0.4.15; python_version < "3.11"
Requires-Dist: pydantic>=2.12.0
Requires-Dist: ansi2html>=1.9.2
Requires-Dist: colorama>=0.4.6
Requires-Dist: types-colorama>=0.4.15
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: black; extra == "dev"
Requires-Dist: isort; extra == "dev"
Dynamic: license-file

# `lograder`

This repository contains the source code for
a general-purpose autograder. The idea from a
high-level view is everything belongs to a pipeline,
which can be found in `src/lograder/pipeline/pipeline.py`.
The main type that is of note is `Step`, found in
`src/lograder/pipeline/step.py`. The steps that make up the
pipeline have a `__call__` which is a generator.
The generator may `yield` values (both errors and info)
which are non-fatal, while `return` values are either
fatal errors or the data to pass onto the next step.

The central mantra to this library is validation,
validation, validation (because repetition legitimizes).
All calls to processes should be through a
`TypedExecutable`, which contains validated input
models via `CLIArgs`. Additionally, any output
needs to be a validated model and needs to be parsed
programmatically and completely. Either the program
must have an API, or the text parser must be bullet-proof.

Additionally, passing errors as values rather than
exception based programming (except for developer bugs, 
staff errors, or validation errors) is preferred.

Finally, all the output is generated by the steps and
will be handled by the loggers, which are responsible for
producing logging output (obviously), but also the `.json`
or `.html` or whatever is used by the underlying application.

With that out of the way, we can get a little more into the
details.

---
## Installation

```bash
pip install -e ".[dev]"
```

Run tests with `pytest`.

---
# "The Pipeline"

The autograder must be able to take student files of any type,
such as source files for C, C++, Python, Rust, Assembly, and 
so forth; build the project in as configurable manner (with
a natural default, ONLY IF there is an intuitive default);
create any sort of artifact, whether that be text output, file
output, etc.; and then "test" the artifacts whatever that may
mean.

The primary subpackage for this is located in `src/lograder/pipeline`.

## Step generics

Every concrete `Step` subclass must declare exactly five type parameters:

```python
class MyStep(Step[InputT, OkOutputT, ErrOutputT, OkDisplayT, ErrDisplayT]): ...
```

`OkOutputT` becomes the input for the next step. `ErrOutputT` and `ErrDisplayT`
are fatal and non-fatal error types respectively, sent to the packet logger.
`__init_subclass__` enforces this at class definition time.

## Result type

`Result[T, E]` (see `src/lograder/common/result.py`) is a Rust-inspired
error-as-value container. Use `Ok(value)` and `Err(value)` constructors.
`danger_ok` / `danger_err` are assertion-guarded unwrappers. Do not raise
exceptions for expected failure modes — return `Err(...)` instead.
`DeveloperException` and `StaffException` are reserved for internal bugs
and course staff configuration errors respectively.

## Manifest

`Manifest` (see `src/lograder/pipeline/types/parcels.py`) represents a
project's file/directory tree and is the primary type flowing between
Input and Check steps. It can be constructed from a directory scan
(`from_directory`), a TOML file (`from_toml`), or a flat list of paths
(`from_flat`). `==` checks exact equality; `<=` checks subset (received
contains at least the expected files).

## Input

These will be the very beginning of the pipeline, taking "no input"
and producing some output (if no error, of course)! For example,
the *Gradescope* autograder puts uploaded files into a fixed directory,
so a natural input is to enumerate a local directory into a `Manifest`.
However, there are other examples of inputs, such as a single file
upload.

## Check

These are mostly validation that happens before build. An example
could be to make sure the project matches a specified manifest. Another
example is to do a source file parse to ensure that an illegal library or
operator is not used or is used under a certain number of times. The input
and output type of these should be able to be chained together.

The `simple_project.py` check module generates named manifest-check classes
(`CMakeManifestCheck`, `MakefileManifestCheck`, `PyProjectManifestCheck`)
at import time, so they can be used directly.

## Mixin

These will be used to inject staff code (such as specified header files)
into the students code, or if the student is expected to write some file
but not the surrounding build system, then it will inject the necessary files.

## Build

These steps actually build the library, such as via `cmake` or `make`
or `pip install` and produce artifacts that are testable or usable in the
next steps.

Implemented builders: `CMakeBuild` (configure + build + File API artifact
parsing), `MakefileBuild` (runs `make`; artifact parsing is not yet
implemented).

## Test

These are mostly post-build testing. Examples could be running a number
of instances of the student's executable, passing different arguments and
seeing if the output matches. Another example is checking the bytes of a
binary file to see if they match or running `valgrind` to check for memory
leaks.

---
# Ideas / Future Work
- Implement `Mixin.__call__` (currently a stub).
- Add Makefile artifact parsing in `MakefileBuild` (returns empty list today).
- Create `Layout` for `lograder.pipeline.build.makefile.MakefileBuildOutput`.
- Implement `Pipeline.validate_step_types()` call and validation in `Pipeline.__call__`.
