Development#
Git#
If have accidentally committed to wrong branch need these series of commands
# Note: Any changes not committed will be lost.
git branch newbranch # Create a new branch, saving the desired commits
git checkout master # checkout master, this is the place you want to go back
git reset --hard HEAD~3 # Move master back by 3 commits (Make sure you know how many commits you need to go back)
git checkout newbranch # Go to the new branch that still has the desired commits
Docstring coverage#
(locpix-env) $ pip install docstr-coverage
(locpix-env) $ docstr-coverage src/locpix
Sphinx documentation#
Ran
(locpix-env) $ sphinx-quickstart docs
Choose yes for separate docs and src directories.
Then followed: sphinx auto summary
Had to install sphinx-autoapi & pydata theme
(locpix-env) $ pip install sphinx-autoapi
(locpix-env) $ pip install pydata-sphinx-theme
Then need to make sure installed torch, torchvision, cellpose as according to README and prettytable
First delete folder docs/source/_autosummary
Then run
(locpix-env) $ make clean_templates
(locpix-env) $ make templates
(locpix-env) $ make clean
(locpix-env) $ make html
YAML files#
Have yaml files with configuration - need to convert these to .rst
(locpix-env) $ pip install yaml2rst
Then for all the template files convert to .rst , add heading and put in docs/source
(locpix-env) $ yaml2rst src/locpix/templates/*.yaml docs/source/*.rst
To support CI, we add this to the makefile for sphinx
To generates templates run
(locpix-env) $ make clean_templates
(locpix-env) $ make templates
Then can run
(locpix-env) $ make clean
(locpix-env) $ make html
Warning
You get an error when running
(locpix-env) $ make templates
This is because the makefile contains a catch all, so it will run make templates then trys to run templates into Sphinx but this doesn’t work!
Warning
Note that to get the templates.rst in the correct format we had to edit yaml2rst. The additions are on lines 46-48
title = os.path.basename(infilename).removesuffix('.yaml')
print(title, file=outfh)
print('='*len(title), file=outfh)
Note this will fail on github actions - so need to include these functions as part of this package - not rely on yaml2rst!
PyPI#
See this documentation for how to integrate publishing to PyPI using GitHub actions.
However, as I protected master branch this was causing issues therefore moved towards this instead
#. Removed version from pyproject.toml as setuptools_scm finds this - however Sphinx needs this - therefore follow version under usage from sphinx - requires adding to the docs/conf.py file #. Went up to “You could stop here” - could later implement automatic version increasing
Linting#
In line with GitHub actions run the following, note the GitHub editor is 127 chars wide
Python syntax errors or undefined names
(locpix-env) $ flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
Note that exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
(locpix-env) $ flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
GitHub#
CI#
Master branch is protected therefore have to checkout new branch and then merge this instead.
Steps for CI
Commit changes to working_branch and push this up to GitHub, no actions will run
When happy create pull request - this will trigger tests to run - if tests successful then merge to master
When merges, will generate documentation for master branch also runs CI again (maybe remove the latter). Could do this by removing on push from CI (but need to check that when the tests run on a pull request are they running on the new branch or the merged branch?)
When happy can create release on master and will only run publish workflow
Skip actions#
When push can choose not to run actions by including string
[skip actions]
To publish to pypi needs a tag therefore do in sequence
git checkout -b <branch-name>
Make changes then run (if don’t want to push to pypi)
git add -A
git commit
git push origin <branch-name>
Then to push to pypi have to just push tagged master branch, where tag must start with v
git checkout master
git tag <tag-name>
git push origin <tag-name>
Code coverage#
(locpix-env) $ pip install pytest
(locpix-env) $ pip install pytest-cov
(locpix-env) $ pytest --cov=src tests/
Cellpose train#
To train cellpose first need .npy files of imgs and labels
Therefore use src/locpix/scripts/img_seg/custom_train.py to convert all images to .npy
Then move images to folder with Fov1->6 in train and move masks into this folder as well Fov 7,8,9,10 in test and move masks into this folder as well
Had to append name _masks to each of the masks
Then run
(locpix-env) $ python -m cellpose --train --dir ~/imgs/train/ --test_dir ~/imgs/test/ --pretrained_model LC1 --chan 0 --chan2 0 --learning_rate 0.1 --weight_decay 0.0001 --n_epochs 3
(locpix-env) $ python -m cellpose --dir ~/imgs/test/ --pretrained_model model --chan 0 --chan2 0 --save_png
python -m cellpose –train –dir “C:UsersoliveOneDrive - University of LeedsProjectoutputlocpix_projectcellpose_traintrain” –test_dir “C:UsersoliveOneDrive - University of LeedsProjectoutputlocpix_projectcellpose_traintest” –pretrained_model LC1 –chan 0 –chan2 0 –learning_rate 0.1 –weight_decay 0.0001 –n_epochs 10 –min_train_masks 1 –verbose
python -m cellpose –dir “C:UsersoliveOneDrive - University of LeedsProjectoutputlocpix_projectcellpose_traintest” –pretrained_model “C:UsersoliveOneDrive - University of LeedsProjectoutputlocpix_projectcellpose_traintrainmodelscellpose_residual_on_style_on_concatenation_off_train_2022_12_20_14_56_59.377919” –chan 0 –chan2 0 –save_png