homelette.evaluation

The homelette.evaluation submodule contains different classes for evaluating homology models.

It is possible to implement custom Evaluation building blocks and use them in the homelette framework.

Tutorials

Working with model evaluations in homelette is discussed in detail in Tutorial 3. Implementing custom evaluation metrics is discussed in Tutorial 4. Assembling custom pipelines is discussed in Tutorial 7.

Classes

The following evaluation metrics are implemented:


class homelette.evaluation.Evaluation_dope(model: Type[Model], quiet: bool = False)

Class for evaluating a model with DOPE score.

Will dump the following entries to the model.evaluation dictionary:

  • dope

  • dope_z_score

Parameters
  • model (Model) – The model object to evaluate

  • quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores

Variables
  • model (Model) – The model object to evaluate

  • output (dict) – Dictionary that all outputs will be dumped into

Raises

ImportError – Unable to import dependencies

Notes

DOPE is a staticial potential for the evaluation of homology models 1. For further information, please check the modeller documentation or the associated publication.

References

1

Shen, M., & Sali, A. (2006). Statistical potential for assessment and prediction of protein structures. Protein Science, 15(11), 2507–2524. https://doi.org/10.1110/ps.062416606

evaluate() → None

Run DOPE evaluation. Automatically called on object initialization

Returns

Return type

None

class homelette.evaluation.Evaluation_soap_protein(model: Type[Model], quiet: bool = False)

Class for evaluating a model with the SOAP protein protential.

Will dump the following entries to the model.evaluation dictionary:

  • soap_protein

Parameters
  • model (Model) – The model object to evaluate

  • quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores

Variables
  • model (Model) – The model object to evaluate

  • output (dict) – Dictionary that all outputs will be dumped into

Raises

ImportError – Unable to import dependencies

Notes

SOAP is a statistical potential for evaluating homology models 2. For more information, please check the modeller and SOAP documentations or the associated publication.

References

2

Dong, G. Q., Fan, H., Schneidman-Duhovny, D., Webb, B., Sali, A., & Tramontano, A. (2013). Optimized atomic statistical potentials: Assessment of protein interfaces and loops. Bioinformatics, 29(24), 3158–3166. https://doi.org/10.1093/bioinformatics/btt560

evaluate() → None

Run SOAP protein evaluation. Automatically called on object initialization

Returns

Return type

None

class homelette.evaluation.Evaluation_soap_pp(model: Type[Model], quiet: bool = False)

Class for evaluating a model with SOAP interaction potentials. This is used for the evaluation of models of protein complexes.

Will dump the following entries to the model.evaluation dictionary:

  • soap_pp_all

  • soap_pp_atom

  • soap_pp_pair

Parameters
  • model (Model) – The model object to evaluate

  • quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores

Variables
  • model (Model) – The model object to evaluate

  • output (dict) – Dictionary that all outputs will be dumped into

Raises

ImportError – Unable to import dependencies

Notes

SOAP is a statistical potential for evaluating homology models 3. For more information, please check the modeller and SOAP documentations or the associated publication.

References

3

Dong, G. Q., Fan, H., Schneidman-Duhovny, D., Webb, B., Sali, A., & Tramontano, A. (2013). Optimized atomic statistical potentials: Assessment of protein interfaces and loops. Bioinformatics, 29(24), 3158–3166. https://doi.org/10.1093/bioinformatics/btt560

evaluate() → None

Run SOAP interaction evaluation. Automatically called on object initialization

Returns

Return type

None

class homelette.evaluation.Evaluation_qmean4(model: Type[Model], quiet: bool = False)

Class for evaluating a model with the QMEAN4 potential.

Will dump the following entries to the model.evaluation dictionary:

  • qmean4

  • qmean4_z_score

Parameters
  • model (Model) – The model object to evaluate.

  • quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores

Variables
  • model (Model) – The model object to evaluate

  • output (dict) – Dictionary that all outputs will be dumped into

Raises

ImportError – Unable to import dependencies

Notes

QMEAN is a statistical potential for evaluating homology models 4 5.

Briefly, QMEAN is a combination of different components. Four compoenents (interaction, cbeta, packing and torsion) form the qmean4 score.

For more information, please check the QMEAN documentation or the associated publications.

References

4

Benkert, P., Tosatto, S. C. E., & Schomburg, D. (2008). QMEAN: A comprehensive scoring function for model quality assessment. Proteins: Structure, Function and Genetics, 71(1), 261–277. https://doi.org/10.1002/prot.21715

5

Benkert, P., Biasini, M., & Schwede, T. (2011). Toward the estimation of the absolute quality of individual protein structure models. Bioinformatics, 27(3), 343–350. https://doi.org/10.1093/bioinformatics/btq662

evaluate() → None

Run QMEAN4 protein evaluation. Automatically called on object initialization :returns: :rtype: None

class homelette.evaluation.Evaluation_qmean6(model: Type[Model], quiet: bool = False)

Class for evaluating a model with the QMEAN6 potential.

Will dump the following entries to the model.evaluation dictionary:

  • qmean6

  • qmean6_disco

Requires the following valid entries in the model.info dictionary:

  • accpro_file (.acc file)

  • psipred_file (.horiz file)

Parameters
  • model (Model) – The model object to evaluate.

  • quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores

Variables
  • model (Model) – The model object to evaluate

  • output (dict) – Dictionary that all outputs will be dumped into

Raises

ImportError – Unable to import dependencies

Notes

QMEAN is a statistical potential for evaluating homology models 6 7.

QMEAN6 is a combination of six different components (interaction, cbeta, packing, torsion, ss_agreement, acc_agreement). It is an extension to the QMEAN4 score, which additionally evaluates the agreement of the model to secondary structur predictions from PSIPRED 8 and solvent accessiblity predictions from ACCpro 9.

For more information, please check the QMEAN documentation or the associated publications.

References

6

Benkert, P., Tosatto, S. C. E., & Schomburg, D. (2008). QMEAN: A comprehensive scoring function for model quality assessment. Proteins: Structure, Function and Genetics, 71(1), 261–277. https://doi.org/10.1002/prot.21715

7

Benkert, P., Biasini, M., & Schwede, T. (2011). Toward the estimation of the absolute quality of individual protein structure models. Bioinformatics, 27(3), 343–350. https://doi.org/10.1093/bioinformatics/btq662

8

Jones, D. T. (1999). Protein secondary structure prediction based on position-specific scoring matrices. Journal of Molecular Biology, 292(2), 195–202. https://doi.org/10.1006/JMBI.1999.3091

9

Magnan, C. N., & Baldi, P. (2014). SSpro/ACCpro 5: almost perfect prediction of protein secondary structure and relative solvent accessibility using profiles, machine learning and structural similarity. Bioinformatics, 30(18), 2592–2597. https://doi.org/10.1093/BIOINFORMATICS/BTU352

evaluate() → None

Run QMEAN6 protein evaluation. Automatically called on object initialization

Returns

Return type

None

class homelette.evaluation.Evaluation_qmeandisco(model: Type[Model], quiet: bool = False)

Class for evaluating a model with the QMEAN DisCo potential.

Will dump the following entries to the model.evaluation dictionary:

  • qmean6

  • qmean6_z_score

  • qmean_local_scores_avg

  • qmean_local_scores_err

Requires the following valid entries in the model.info dictionary:

  • accpro_file (.acc file)

  • psipred_file (.horiz file)

  • disco_file (generated by qmean.DisCoContainer.Save)

Parameters
  • model (Model) – The model object to evaluate.

  • quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores

Variables
  • model (Model) – The model object to evaluate

  • output (dict) – Dictionary that all outputs will be dumped into

Raises

ImportError – Unable to import dependencies

Notes

QMEAN is a statistical potential for evaluating homology models 10 11.

QMEAN DisCo is an extension of QMEAN by the inclusion of homology derived DIStance COnstraints 12. These distance contraints do not influence the six component of the QMEAN6 score (interaction, cbeta, packing, torsion, ss_agreement, acc_agreement), but only the local scores.

The distance contraints for the target have to be generated before and saved to a file.

For more information, please check the QMEAN documentation or the associated publications.

References

10

Benkert, P., Tosatto, S. C. E., & Schomburg, D. (2008). QMEAN: A comprehensive scoring function for model quality assessment. Proteins: Structure, Function and Genetics, 71(1), 261–277. https://doi.org/10.1002/prot.21715

11

Benkert, P., Biasini, M., & Schwede, T. (2011). Toward the estimation of the absolute quality of individual protein structure models. Bioinformatics, 27(3), 343–350. https://doi.org/10.1093/bioinformatics/btq662

12

Studer, G., Rempfer, C., Waterhouse, A. M., Gumienny, R., Haas, J., & Schwede, T. (2020). QMEANDisCo-distance constraints applied on model quality estimation. Bioinformatics, 36(6), 1765–1771. https://doi.org/10.1093/bioinformatics/btz828

evaluate() → None

Run QMEAN DisCo protein evaluation. Automatically called on object initialization

Returns

Return type

None

class homelette.evaluation.Evaluation_mol_probity(model: Type[Model], quiet: bool = False)

Class for evaluating a model with the MolProbity validation service.

Will dump the following entries to the model.evaluation dictionary:

  • mp_score

Parameters
  • model (Model) – The model object to evaluate

  • quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores

Variables
  • model (Model) – The model object to evaluate

  • output (dict) – Dictionary that all outputs will be dumped into

Notes

Molprobity is a program that evaluates the quality of 3D structures of proteins based on structural features 13 14 15. For more information, please check the MolProbity webpage or the associated publications.

References

13

Davis, I. W., Leaver-Fay, A., Chen, V. B., Block, J. N., Kapral, G. J., Wang, X., Murray, L. W., Arendall, W. B., Snoeyink, J., Richardson, J. S., & Richardson, D. C. (2007). MolProbity: all-atom contacts and structure validation for proteins and nucleic acids. Nucleic Acids Research, 35(suppl_2), W375–W383. https://doi.org/10.1093/NAR/GKM216

14

Chen, V. B., Arendall, W. B., Headd, J. J., Keedy, D. A., Immormino, R. M., Kapral, G. J., Murray, L. W., Richardson, J. S., & Richardson, D. C. (2010). MolProbity: All-atom structure validation for macromolecular crystallography. Acta Crystallographica Section D: Biological Crystallography, 66(1), 12–21. https://doi.org/10.1107/S0907444909042073

15

Williams, C. J., Headd, J. J., Moriarty, N. W., Prisant, M. G., Videau, L. L., Deis, L. N., Verma, V., Keedy, D. A., Hintze, B. J., Chen, V. B., Jain, S., Lewis, S. M., Arendall, W. B., Snoeyink, J., Adams, P. D., Lovell, S. C., Richardson, J. S., & Richardson, D. C. (2018). MolProbity: More and better reference data for improved all-atom structure validation. Protein Science, 27(1), 293–315. https://doi.org/10.1002/pro.3330

evaluate() → None

Run MolProbity evaluation. Automatically called on object initialization

Returns

Return type

None