Principles of Protein Structure - G. Schultz 1982

Secondary structure prediction from amino acid sequence
Evaluation of predictive methods
Arbitrariness of parameter selection

By introducing a sufficient number of parameters, an excellent fit can be achieved with any benchmark dataset. Attempts to improve predictions must always rely on assessing the accuracy of the method. With this in mind, many authors have tested their method on a benchmark set of Proteins with known structures. However, such a Procedure is of very limited value. It is quite obvious that the more internal information (rules and parameters) a method incorporates, the better the agreement with the benchmark set will be; yet, this by no means determines the predictive value of the method for proteins with unknown structures.

To somehow address this issue, some authors initially exclude a certain protein from the benchmark set and perform parametrization with a reduced benchmark set. The derived parameters are then used to predict the Introduction/11.html">Secondary Structure of the isolated protein. In this way, the absolute values of the fitted parameters appear to become independent (the propensities given in Tables 6.1 and 6.2). However, the rules for applying these parameters, which constitute the core of each method, are still based on all known proteins, including the one being predicted.

Some Methods are not defined with sufficient clarity. Aside from the difficulties associated with the independent verification of a method's predictive power, in some cases it turns out that the method is simply irreproducible [381, 382]. These reasons explain the widespread skepticism regarding secondary structure prediction methods. It should be noted, however, that good reproducibility and agreement with a benchmark set are still not equivalent to successful predictions of unknown protein structures. This occurs mainly because authors fail to define their method with sufficient clarity, thereby leaving room for a high degree of arbitrariness in its application. Critical attitudes toward predictive methods were shaken to some extent following comparative tests on adenylate kinase [383] and T4 Lysozyme [384].