A tool for lecturers to automatically generate assessment scripts for Grading a Programming Assignment on Gradescope. It supports filling out an interactive web form or uploading a YAML configuration, which then generates a packaged ZIP file ready to be uploaded to Gradescope.
The diagram below illustrates how a lecturer configures and exports their assessment rules, packages the autograder, deploys it to Gradescope, and how students' submissions are automatically evaluated in isolation.
Select one of the validated YAML configuration examples below to view its structure. You can import any example directly into the builder interface.
A lightweight setup that evaluates basic function creation and arithmetic operations with simple Python unit tests.
version: '1.0'
language: python
files_necessary:
- solution.py
questions:
- name: Math Test
description: "Implement a simple calculator. Specifically, create a function `add(a, b)` that returns the sum of two integers. This exercise evaluates your ability to define basic functions and perform arithmetic operations in Python. \n\nExample:\nInput: `add(1, 2)`\nOutput: `3`"
marking_items:
- target_file: solution.py
total_mark: 10
type: function_test
function_name: add
test_cases:
- args:
- 1
- 2
expected: '3'
- args:
- 5
- 5
expected: '10'Validates both python function signatures (parameter names, count) and logic verification in an integrated test suite.
version: '1.0'
language: python
files_necessary:
- solution.py
questions:
- name: Function Signature and Logic
description: "This task requires you to implement a robust `multiply` function. You must ensure the function is named exactly `multiply` and accepts two arguments: `a` and `b`. The function should return the mathematical product of these two values. The autograder will verify both the interface (signature) and the implementation (logic). \n\nExample:\nInput: `multiply(2, 3)`\nOutput: `6`"
marking_items:
- target_file: solution.py
total_mark: 5
type: signature_check
function_name: multiply
expected_parameters: a, b
- target_file: solution.py
total_mark: 5
type: function_test
function_name: multiply
test_cases:
- args:
- 2
- 3
expected: '6'
- args:
- 0
- 10
expected: '0'A fully featured scenario validating environment installation commands, required file checks, standard IO comparison tests, type-hint signature checks, edge case testing, and mixed evaluation items.
version: '1.0'
language: python
global_time_limit: 600
setup_commands:
- pip install numpy pandas matplotlib requests
- pip install scipy scikit-learn
files_necessary:
- basic_operations.py
- math_functions.py
- data_processing.py
- advanced_algorithms.py
questions:
- name: File Existence Validation
description: "Organizational check. Ensure that the project structure is correct by including the required files: `basic_operations.py`, `math_functions.py`, and `data_processing.py`. This verifies that you have correctly partitioned your logic according to the project specifications. \n\nExpected files: `basic_operations.py`, `math_functions.py`, `data_processing.py`."
marking_items:
- target_file: basic_operations.py
total_mark: 1
type: file_exists
time_limit: 5
visibility: visible
- target_file: math_functions.py
total_mark: 1
type: file_exists
time_limit: 5
visibility: visible
- target_file: data_processing.py
total_mark: 1
type: file_exists
time_limit: 5
visibility: hidden
- name: Basic Output Comparison Tests
description: "Standard input/output test. Your script `basic_operations.py` must read from standard input and produce a formatted report. This test checks your ability to handle string formatting and basic data ingestion. \n\nExample:\nInput: `42` on stdin\nOutput: `Number: 42` on stdout"
marking_items:
- target_file: basic_operations.py
total_mark: 5
type: output_comparison
time_limit: 30
visibility: visible
expected_input: 'Hello World
42
3.14'
expected_output: 'Hello World
Number: 42
Float: 3.14'
- target_file: basic_operations.py
total_mark: 5
type: output_comparison
time_limit: 45
visibility: after_due_date
expected_input: 'Second Test
100
-5.5'
expected_output: 'Second Test
Number: 100
Float: -5.5'
- name: Basic Function Signature Validation
description: "Interface contract check. Implement `add_numbers(a: int, b: int) -> int` and `multiply(x: float, y: float) -> float` in `math_functions.py`. The autograder will inspect these functions to ensure they match the requested type hints and parameter names exactly. \n\nExample:\nSignature: `def add_numbers(a: int, b: int) -> int:`"
marking_items:
- target_file: math_functions.py
function_name: add_numbers
total_mark: 3
type: signature_check
time_limit: 10
visibility: visible
expected_parameters: 'a: int, b: int'
expected_return_type: int
- target_file: math_functions.py
function_name: multiply
total_mark: 3
type: signature_check
time_limit: 10
visibility: visible
expected_parameters: 'x: float, y: float'
expected_return_type: float
- name: Advanced Signature Validation with Defaults
description: "Complex interface check. This question tests your knowledge of default arguments and keyword parameters. You must implement several utility functions in `data_processing.py` and `advanced_algorithms.py` that support optional configuration parameters."
marking_items:
- target_file: data_processing.py
function_name: process_data
total_mark: 5
type: signature_check
time_limit: 15
visibility: visible
expected_parameters: 'data: list, threshold: float = 0.5, normalize: bool = True'
expected_return_type: dict
- target_file: data_processing.py
function_name: filter_values
total_mark: 4
type: signature_check
time_limit: 15
visibility: after_due_date
expected_parameters: values, min_val=0, max_val=100, inclusive=True
- target_file: advanced_algorithms.py
function_name: complex_calculation
total_mark: 5
type: signature_check
time_limit: 20
visibility: hidden
expected_parameters: input_data, algorithm='default', precision=2, debug=False
- name: Simple Function Testing
description: "Unit testing basic logic. We will run your `add_numbers` and `multiply` functions from `math_functions.py` against hidden test cases to ensure they return the mathematically correct results. \n\nExample:\nInput: `add_numbers(2, 3)`\nOutput: `5`"
marking_items:
- target_file: math_functions.py
function_name: add_numbers
total_mark: 6
type: function_test
time_limit: 30
visibility: visible
test_cases:
- args:
- 2
- 3
expected: '5'
- target_file: math_functions.py
function_name: multiply
total_mark: 6
type: function_test
time_limit: 25
visibility: visible
test_cases:
- args:
- 3
- 4
expected: '12'
- name: Advanced Function Testing with Keywords
description: "High-level data processing. Implement `calculate_statistics` and `transform_data` in `data_processing.py`. These functions must handle lists of data and dictionary-based configurations respectively. Accuracy in floating-point calculations is required. \n\nExample:\nInput: `calculate_statistics([1, 2, 3])`\nOutput: `{'mean': 2.0, ...}`"
marking_items:
- target_file: data_processing.py
function_name: calculate_statistics
total_mark: 9
type: function_test
time_limit: 45
visibility: after_due_date
test_cases:
- args:
- [1, 2, 3, 4, 5]
expected: '{''mean'': 3.0, ''median'': 3.0, ''std'': 1.58}'
- target_file: data_processing.py
function_name: transform_data
total_mark: 8
type: function_test
time_limit: 60
visibility: after_published
test_cases:
- args:
- a: 1
b: 2
expected: '{''a'': 2, ''b'': 4}'
- name: Edge Case Testing
description: "Robustness check. Your `handle_edge_cases` function in `advanced_algorithms.py` must gracefully handle empty input lists by returning `None`. This evaluates your defensive programming skills. \n\nExample:\nInput: `handle_edge_cases([])`\nOutput: `None`"
marking_items:
- target_file: advanced_algorithms.py
function_name: handle_edge_cases
total_mark: 12
type: function_test
time_limit: 90
visibility: hidden
test_cases:
- args:
- []
expected: None
- name: Mixed Testing Scenarios
description: "Integration challenge. This final question combines file checks, signature validation, logic testing, and output comparison for the `main_algorithm` in `advanced_algorithms.py`. This represents a complete module implementation. \n\nExample:\nInput: `main_algorithm([1, 2])`\nOutput: `[1, 4]`"
marking_items:
- target_file: advanced_algorithms.py
total_mark: 2
type: file_exists
time_limit: 5
visibility: visible
- target_file: advanced_algorithms.py
function_name: main_algorithm
total_mark: 5
type: signature_check
time_limit: 20
visibility: visible
expected_parameters: input_list, config=None, verbose=False
- target_file: advanced_algorithms.py
function_name: main_algorithm
total_mark: 12
type: function_test
time_limit: 120
visibility: after_due_date
test_cases:
- args:
- [1, 2, 3, 4, 5]
expected: '[1, 4, 9, 16, 25]'
- target_file: advanced_algorithms.py
total_mark: 7
type: output_comparison
time_limit: 60
visibility: after_published
expected_input: '5'
expected_output: ''A simple Java setup that evaluates basic static function logic inside class files with GradeScope compatible execution templates.
version: '1.0'
language: java
files_necessary:
- Solution.java
questions:
- name: Question 1
description: "Implement a function `add(a, b)` in `Solution.java` that returns the sum of two doubles."
marking_items:
- target_file: Solution.java
total_mark: 10
type: function_test
function_name: add
test_cases:
- args:
- 1.0
- 2.0
expected: '3.0'
- args:
- 5.0
- 5.0
expected: '10.0'