Metadata-Version: 2.4
Name: CSTester
Version: 2.0.0a0
Summary: Computer Science programming submission testing tool
Author-email: Chase Phelps <admin@chaseleif.tech>
Maintainer-email: Chase Phelps <admin@chaseleif.tech>
Project-URL: Homepage, https://github.com/chaseleif/test_output
Project-URL: Repository, https://github.com/chaseleif/test_output.git
Project-URL: Issues, https://github.com/chaseleif/test_output/issues
Keywords: testing,education,cstester
Classifier: Environment :: Console :: Curses
Classifier: Intended Audience :: Education
Classifier: Operating System :: OS Independent
Classifier: Topic :: Education :: Testing
Requires-Python: >=3.9
Description-Content-Type: text/x-rst
License-File: LICENSE
Requires-Dist: PyYAML>=5.4
Dynamic: license-file

CSTester
========

.. BEGINDOCS

``CSTester`` is a Python curses package
to automate Computer Science programming assignment evaluation.



Configuration options
~~~~~~~~~~~~~~~~~~~~~

- phasedir

  The directory containing groups' submissions

- phasezip

  A zip containing groups' zipfiles

- keyfile

  Keys to associate with group numbers to aid extraction

- groupre_str

  A regex to extract group numbers from group zip filenames

- zipexclude_strs

  A list of regex patterns to exclude files if any portion of
  the filename matches a pattern

- zipinclude_strs

  A map of files to explicitly place at a location
  relative to the group's root directory

- allgroups

  A calculated number/range list and includes all group numbers

- include

  A number/range list of groups to **only** include

- exclude

  A number/range list of groups to exclude

- freezefiles

  A pattern for files to "freeze"

- cleanfiles

  A pattern for files to remove with the clean command

- casedir

  The directory containing input test cases

- caseext

  The file extension of input test cases

- cases

  A calculated list of input test cases

- expdir

  The directory containing expected output files

- expext

  The file extension of expected output files

- exps

  A calculated list of expected output files

- testcmd

  The testing command to run from each group's directory

- prepcmds

  A list of commands to run from selected group directories

- readmename

  The name of a text file that should be included in submissions

- searchstrs

  A list of regex patterns to search for within searchfiles

- searchfiles

  A pattern for files to include within the strings search

----

Phase directory
~~~~~~~~~~~~~~~

One of the first things wich must be done is setting up the phase directory.

The phase directory contains each group's submission in a subdirectory
in the form ``group_n``, where n is a non-negative integer.

.. note::
  Group numbers
    - Are **non-negative**, ``0|[1-9][0-9]*``
    - Do **not** have to be consecutive
    - Must **not** have leading zeroes

Extraction
~~~~~~~~~~

The extractor was made to extract ``submissions.zip``,
where each group submits a zip.

``submissions.zip`` is a zip containing other zips.

Group numbers are obtained through a group regex,
or, optionally, using a keyfile.

The group regex extracts a number from a filename,
using ``re.match``, e.g.,

.. code::

  ^group_(0|[1-9][0-9]*).*\.zip$

The key-value pairs taken from the keyfile are of the form

.. code::

  ^(0|[1-9][0-9]*)(?:,.*):([^:]+)$

That is, lines start with a valid group number,
optionally followed by a comma and some series of text before a trailing colon,
finally, the key to search for in the filename is after the final colon.

For example, with

.. code::

  42,Group leader: A name, Members: Another name:groupname

if **groupname** exists within the filename
then the group number will be inferred to be **42**.

All regex matches performed during extraction are all **case-insensitive**.
*Regexes* in a keyfile **must be unique**, *group numbers* **need not be**.

Group directories are of the form ``group_n``, where ``n`` is the group number

Within the phase directory, an extraction log is created: ``phasedir/x.log``

The extraction log records general information about extraction
  - If a keyfile was used and how many keys were obtained from the keyfile
  - If files were skipped/ignored
  - If a group number was inferred from keyfile and why
  - If a group number was manually entered
    (decisions in subsequent extractions *of the same zipfile* can be repeated)
  - Which file was chosen among multiple submissions
  - How many group->file pairs were found
  - If duplicate files were ignored
  - If a submission zip had symlink files
  - If extraction of a submission zip had an exception
  - If a submission wasn't extracted due to a suspicious compression ratio
  - If duplicate files were ignored
  - If extracted files needed to be renamed

Within group directories, e.g., ``phasedir/group_1/``
  - ``group1.zip``--contains the original zipfile
  - ``sub.nfo``--contains details of the original zipfile
    The original zip filename
    The compressed size, extracted size, and compression ratio
    Whether symlink files existed (and were skipped)
    List of extracted files
    List of files excluded

**Extracting individual zip files**

The submissions zip is extracted to a temporary directory.

We examine the zip memberlist against exclude patterns,
and do not extract files which match (case-insensitive)
an exclude pattern.

For files we want at a specific location,
these can be specified in the include pattern map:

  - The ``key`` is a regex to match the file
  - The ``val`` is a relative location from the group directory

.. hint::
  include={'Makefile','','driver\.c','src}
  Will put Makefile in the group's root directory and driver.c in src/


We
exclude (and print a warning) any symlinks within the archive. We
extract the contents of the zip to a new temporary directory. We ensure
all files are chmod 600 (**owner rw**), and all directories are chmod
700 (**owner rwx**).

We move all files to the final extraction directory,
we remove any special characters from file/directory names,
and spaces are converted to underscores.
We put the original zip in the group’s directory.
A text file, ``./sub.nfo``,
will contain the zip filename,
original content list with sizes (compressed/decompressed),
and compression ratio of extracted files.

----

Making patches
~~~~~~~~~~~~~~

Patches can be made which can be shared
  - So that they can be applied with the ``patch`` command
  - To show changes made

To make patches, relevant files must be first frozen in their initial state.

Freezing files requires a regex to match files of interest.

All files within group directories which match the regex
  - Are duplicated in a zip archive
  - Their hash is calculated and stored in a text file

The frozen zip and hashes are stored in the root of the phase directory.
