Development#

Git#

If have accidentally committed to wrong branch need these series of commands

# Note: Any changes not committed will be lost.
git branch newbranch # Create a new branch, saving the desired commits
git checkout master # checkout master, this is the place you want to go back
git reset --hard HEAD~3 # Move master back by 3 commits (Make sure you know how many commits you need to go back)
git checkout newbranch # Go to the new branch that still has the desired commits

Docstring coverage#

(locpix-env) $ pip install docstr-coverage
(locpix-env) $ docstr-coverage src/locpix

Sphinx documentation#

Ran

(locpix-env) $ sphinx-quickstart docs

Choose yes for separate docs and src directories.

Then followed: sphinx auto summary

Had to install sphinx-autoapi & pydata theme

(locpix-env) $ pip install sphinx-autoapi
(locpix-env) $ pip install pydata-sphinx-theme

Then need to make sure installed torch, torchvision, cellpose as according to README and prettytable

First delete folder docs/source/_autosummary

Then run

(locpix-env) $ make clean_templates
(locpix-env) $ make templates
(locpix-env) $ make clean
(locpix-env) $ make html

YAML files#

Have yaml files with configuration - need to convert these to .rst

(locpix-env) $ pip install yaml2rst

Then for all the template files convert to .rst , add heading and put in docs/source

(locpix-env) $ yaml2rst src/locpix/templates/*.yaml docs/source/*.rst

To support CI, we add this to the makefile for sphinx

To generates templates run

(locpix-env) $ make clean_templates
(locpix-env) $ make templates

Then can run

(locpix-env) $ make clean
(locpix-env) $ make html

Warning

You get an error when running

(locpix-env) $ make templates

This is because the makefile contains a catch all, so it will run make templates then trys to run templates into Sphinx but this doesn’t work!

Warning

Note that to get the templates.rst in the correct format we had to edit yaml2rst. The additions are on lines 46-48

title = os.path.basename(infilename).removesuffix('.yaml')
print(title, file=outfh)
print('='*len(title), file=outfh)

Note this will fail on github actions - so need to include these functions as part of this package - not rely on yaml2rst!

PyPI#

See this documentation for how to integrate publishing to PyPI using GitHub actions.

PyPI

However, as I protected master branch this was causing issues therefore moved towards this instead

GitHub PyPI

#. Removed version from pyproject.toml as setuptools_scm finds this - however Sphinx needs this - therefore follow version under usage from sphinx - requires adding to the docs/conf.py file #. Went up to “You could stop here” - could later implement automatic version increasing

Linting#

In line with GitHub actions run the following, note the GitHub editor is 127 chars wide

Python syntax errors or undefined names

(locpix-env) $ flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics

Note that exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide

(locpix-env) $ flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics

GitHub#

CI#

Master branch is protected therefore have to checkout new branch and then merge this instead.

Steps for CI

  1. Commit changes to working_branch and push this up to GitHub, no actions will run

  2. When happy create pull request - this will trigger tests to run - if tests successful then merge to master

  3. When merges, will generate documentation for master branch also runs CI again (maybe remove the latter). Could do this by removing on push from CI (but need to check that when the tests run on a pull request are they running on the new branch or the merged branch?)

  4. When happy can create release on master and will only run publish workflow

Skip actions#

When push can choose not to run actions by including string

[skip actions]

To publish to pypi needs a tag therefore do in sequence

git checkout -b <branch-name>

Make changes then run (if don’t want to push to pypi)

git add -A
git commit
git push origin <branch-name>

Then to push to pypi have to just push tagged master branch, where tag must start with v

git checkout master
git tag <tag-name>
git push origin <tag-name>

Code coverage#

(locpix-env) $ pip install pytest
(locpix-env) $ pip install pytest-cov
(locpix-env) $ pytest --cov=src tests/

Cellpose train#

To train cellpose first need .npy files of imgs and labels

Therefore use src/locpix/scripts/img_seg/custom_train.py to convert all images to .npy

Then move images to folder with Fov1->6 in train and move masks into this folder as well Fov 7,8,9,10 in test and move masks into this folder as well

Had to append name _masks to each of the masks

Then run

(locpix-env) $ python -m cellpose --train --dir ~/imgs/train/ --test_dir ~/imgs/test/ --pretrained_model LC1 --chan 0 --chan2 0 --learning_rate 0.1 --weight_decay 0.0001 --n_epochs 3
(locpix-env) $ python -m cellpose --dir ~/imgs/test/ --pretrained_model model --chan 0 --chan2 0 --save_png

python -m cellpose –train –dir “C:UsersoliveOneDrive - University of LeedsProjectoutputlocpix_projectcellpose_traintrain” –test_dir “C:UsersoliveOneDrive - University of LeedsProjectoutputlocpix_projectcellpose_traintest” –pretrained_model LC1 –chan 0 –chan2 0 –learning_rate 0.1 –weight_decay 0.0001 –n_epochs 10 –min_train_masks 1 –verbose

python -m cellpose –dir “C:UsersoliveOneDrive - University of LeedsProjectoutputlocpix_projectcellpose_traintest” –pretrained_model “C:UsersoliveOneDrive - University of LeedsProjectoutputlocpix_projectcellpose_traintrainmodelscellpose_residual_on_style_on_concatenation_off_train_2022_12_20_14_56_59.377919” –chan 0 –chan2 0 –save_png