homelette.evaluation
The homelette.evaluation
submodule contains different classes for
evaluating homology models.
It is possible to implement custom Evaluation building blocks and use them in the homelette framework.
Tutorials
Working with model evaluations in homelette is discussed in detail in Tutorial 3. Implementing custom evaluation metrics is discussed in Tutorial 4. Assembling custom pipelines is discussed in Tutorial 7.
Classes
The following evaluation metrics are implemented:
-
class
homelette.evaluation.
Evaluation_dope
(model: Type[Model], quiet: bool = False) Class for evaluating a model with DOPE score.
Will dump the following entries to the model.evaluation dictionary:
dope
dope_z_score
- Parameters
model (Model) – The model object to evaluate
quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores
- Variables
model (Model) – The model object to evaluate
output (dict) – Dictionary that all outputs will be dumped into
- Raises
ImportError – Unable to import dependencies
Notes
DOPE is a staticial potential for the evaluation of homology models 1. For further information, please check the modeller documentation or the associated publication.
References
- 1
Shen, M., & Sali, A. (2006). Statistical potential for assessment and prediction of protein structures. Protein Science, 15(11), 2507–2524. https://doi.org/10.1110/ps.062416606
-
evaluate
() → None Run DOPE evaluation. Automatically called on object initialization
- Returns
- Return type
None
-
class
homelette.evaluation.
Evaluation_soap_protein
(model: Type[Model], quiet: bool = False) Class for evaluating a model with the SOAP protein protential.
Will dump the following entries to the model.evaluation dictionary:
soap_protein
- Parameters
model (Model) – The model object to evaluate
quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores
- Variables
model (Model) – The model object to evaluate
output (dict) – Dictionary that all outputs will be dumped into
- Raises
ImportError – Unable to import dependencies
Notes
SOAP is a statistical potential for evaluating homology models 2. For more information, please check the modeller and SOAP documentations or the associated publication.
References
- 2
Dong, G. Q., Fan, H., Schneidman-Duhovny, D., Webb, B., Sali, A., & Tramontano, A. (2013). Optimized atomic statistical potentials: Assessment of protein interfaces and loops. Bioinformatics, 29(24), 3158–3166. https://doi.org/10.1093/bioinformatics/btt560
-
evaluate
() → None Run SOAP protein evaluation. Automatically called on object initialization
- Returns
- Return type
None
-
class
homelette.evaluation.
Evaluation_soap_pp
(model: Type[Model], quiet: bool = False) Class for evaluating a model with SOAP interaction potentials. This is used for the evaluation of models of protein complexes.
Will dump the following entries to the model.evaluation dictionary:
soap_pp_all
soap_pp_atom
soap_pp_pair
- Parameters
model (Model) – The model object to evaluate
quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores
- Variables
model (Model) – The model object to evaluate
output (dict) – Dictionary that all outputs will be dumped into
- Raises
ImportError – Unable to import dependencies
Notes
SOAP is a statistical potential for evaluating homology models 3. For more information, please check the modeller and SOAP documentations or the associated publication.
References
- 3
Dong, G. Q., Fan, H., Schneidman-Duhovny, D., Webb, B., Sali, A., & Tramontano, A. (2013). Optimized atomic statistical potentials: Assessment of protein interfaces and loops. Bioinformatics, 29(24), 3158–3166. https://doi.org/10.1093/bioinformatics/btt560
-
evaluate
() → None Run SOAP interaction evaluation. Automatically called on object initialization
- Returns
- Return type
None
-
class
homelette.evaluation.
Evaluation_qmean4
(model: Type[Model], quiet: bool = False) Class for evaluating a model with the QMEAN4 potential.
Will dump the following entries to the model.evaluation dictionary:
qmean4
qmean4_z_score
- Parameters
model (Model) – The model object to evaluate.
quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores
- Variables
model (Model) – The model object to evaluate
output (dict) – Dictionary that all outputs will be dumped into
- Raises
ImportError – Unable to import dependencies
See also
Notes
QMEAN is a statistical potential for evaluating homology models 4 5.
Briefly, QMEAN is a combination of different components. Four compoenents (interaction, cbeta, packing and torsion) form the qmean4 score.
For more information, please check the QMEAN documentation or the associated publications.
References
- 4
Benkert, P., Tosatto, S. C. E., & Schomburg, D. (2008). QMEAN: A comprehensive scoring function for model quality assessment. Proteins: Structure, Function and Genetics, 71(1), 261–277. https://doi.org/10.1002/prot.21715
- 5
Benkert, P., Biasini, M., & Schwede, T. (2011). Toward the estimation of the absolute quality of individual protein structure models. Bioinformatics, 27(3), 343–350. https://doi.org/10.1093/bioinformatics/btq662
-
evaluate
() → None Run QMEAN4 protein evaluation. Automatically called on object initialization :returns: :rtype: None
-
class
homelette.evaluation.
Evaluation_qmean6
(model: Type[Model], quiet: bool = False) Class for evaluating a model with the QMEAN6 potential.
Will dump the following entries to the model.evaluation dictionary:
qmean6
qmean6_disco
Requires the following valid entries in the model.info dictionary:
accpro_file (.acc file)
psipred_file (.horiz file)
- Parameters
model (Model) – The model object to evaluate.
quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores
- Variables
model (Model) – The model object to evaluate
output (dict) – Dictionary that all outputs will be dumped into
- Raises
ImportError – Unable to import dependencies
See also
Notes
QMEAN is a statistical potential for evaluating homology models 6 7.
QMEAN6 is a combination of six different components (interaction, cbeta, packing, torsion, ss_agreement, acc_agreement). It is an extension to the QMEAN4 score, which additionally evaluates the agreement of the model to secondary structur predictions from PSIPRED 8 and solvent accessiblity predictions from ACCpro 9.
For more information, please check the QMEAN documentation or the associated publications.
References
- 6
Benkert, P., Tosatto, S. C. E., & Schomburg, D. (2008). QMEAN: A comprehensive scoring function for model quality assessment. Proteins: Structure, Function and Genetics, 71(1), 261–277. https://doi.org/10.1002/prot.21715
- 7
Benkert, P., Biasini, M., & Schwede, T. (2011). Toward the estimation of the absolute quality of individual protein structure models. Bioinformatics, 27(3), 343–350. https://doi.org/10.1093/bioinformatics/btq662
- 8
Jones, D. T. (1999). Protein secondary structure prediction based on position-specific scoring matrices. Journal of Molecular Biology, 292(2), 195–202. https://doi.org/10.1006/JMBI.1999.3091
- 9
Magnan, C. N., & Baldi, P. (2014). SSpro/ACCpro 5: almost perfect prediction of protein secondary structure and relative solvent accessibility using profiles, machine learning and structural similarity. Bioinformatics, 30(18), 2592–2597. https://doi.org/10.1093/BIOINFORMATICS/BTU352
-
evaluate
() → None Run QMEAN6 protein evaluation. Automatically called on object initialization
- Returns
- Return type
None
-
class
homelette.evaluation.
Evaluation_qmeandisco
(model: Type[Model], quiet: bool = False) Class for evaluating a model with the QMEAN DisCo potential.
Will dump the following entries to the model.evaluation dictionary:
qmean6
qmean6_z_score
qmean_local_scores_avg
qmean_local_scores_err
Requires the following valid entries in the model.info dictionary:
accpro_file (.acc file)
psipred_file (.horiz file)
disco_file (generated by
qmean.DisCoContainer.Save
)
- Parameters
model (Model) – The model object to evaluate.
quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores
- Variables
model (Model) – The model object to evaluate
output (dict) – Dictionary that all outputs will be dumped into
- Raises
ImportError – Unable to import dependencies
See also
Notes
QMEAN is a statistical potential for evaluating homology models 10 11.
QMEAN DisCo is an extension of QMEAN by the inclusion of homology derived DIStance COnstraints 12. These distance contraints do not influence the six component of the QMEAN6 score (interaction, cbeta, packing, torsion, ss_agreement, acc_agreement), but only the local scores.
The distance contraints for the target have to be generated before and saved to a file.
For more information, please check the QMEAN documentation or the associated publications.
References
- 10
Benkert, P., Tosatto, S. C. E., & Schomburg, D. (2008). QMEAN: A comprehensive scoring function for model quality assessment. Proteins: Structure, Function and Genetics, 71(1), 261–277. https://doi.org/10.1002/prot.21715
- 11
Benkert, P., Biasini, M., & Schwede, T. (2011). Toward the estimation of the absolute quality of individual protein structure models. Bioinformatics, 27(3), 343–350. https://doi.org/10.1093/bioinformatics/btq662
- 12
Studer, G., Rempfer, C., Waterhouse, A. M., Gumienny, R., Haas, J., & Schwede, T. (2020). QMEANDisCo-distance constraints applied on model quality estimation. Bioinformatics, 36(6), 1765–1771. https://doi.org/10.1093/bioinformatics/btz828
-
evaluate
() → None Run QMEAN DisCo protein evaluation. Automatically called on object initialization
- Returns
- Return type
None
-
class
homelette.evaluation.
Evaluation_mol_probity
(model: Type[Model], quiet: bool = False) Class for evaluating a model with the MolProbity validation service.
Will dump the following entries to the model.evaluation dictionary:
mp_score
- Parameters
model (Model) – The model object to evaluate
quiet (bool) – If True, will perform evaluation with suppressing stdout (default False). Needs to be False for running it asynchronously, as done when running Task.evaluate_models with multple cores
- Variables
model (Model) – The model object to evaluate
output (dict) – Dictionary that all outputs will be dumped into
Notes
Molprobity is a program that evaluates the quality of 3D structures of proteins based on structural features 13 14 15. For more information, please check the MolProbity webpage or the associated publications.
References
- 13
Davis, I. W., Leaver-Fay, A., Chen, V. B., Block, J. N., Kapral, G. J., Wang, X., Murray, L. W., Arendall, W. B., Snoeyink, J., Richardson, J. S., & Richardson, D. C. (2007). MolProbity: all-atom contacts and structure validation for proteins and nucleic acids. Nucleic Acids Research, 35(suppl_2), W375–W383. https://doi.org/10.1093/NAR/GKM216
- 14
Chen, V. B., Arendall, W. B., Headd, J. J., Keedy, D. A., Immormino, R. M., Kapral, G. J., Murray, L. W., Richardson, J. S., & Richardson, D. C. (2010). MolProbity: All-atom structure validation for macromolecular crystallography. Acta Crystallographica Section D: Biological Crystallography, 66(1), 12–21. https://doi.org/10.1107/S0907444909042073
- 15
Williams, C. J., Headd, J. J., Moriarty, N. W., Prisant, M. G., Videau, L. L., Deis, L. N., Verma, V., Keedy, D. A., Hintze, B. J., Chen, V. B., Jain, S., Lewis, S. M., Arendall, W. B., Snoeyink, J., Adams, P. D., Lovell, S. C., Richardson, J. S., & Richardson, D. C. (2018). MolProbity: More and better reference data for improved all-atom structure validation. Protein Science, 27(1), 293–315. https://doi.org/10.1002/pro.3330
-
evaluate
() → None Run MolProbity evaluation. Automatically called on object initialization
- Returns
- Return type
None