A Cookbook for Using Model Diagnostics in Integrated Stock Assessments

Integrated analysis has increasingly been the preferred approach for conducting stock assessments and providing the basis for management advice for fish and invertebrate stocks around the world. Many decisions are required when developing integrated stock assessments. For example, the analyst needs to decide whether the model fits the data, if the optimization was successful, if estimates are consistent retrospectively, and if the model is suitable to predict future stock responses to fishing. This study provides practical guidelines for implementing selected diagnostic tools that can assist analysts in identifying problems with model specifications and alternatives that can be explored to minimize or eliminate such problems. Emphasis is placed on reviewing the implementation and interpretation of contemporary model diagnostic tools. We first describe each diagnostic approach and its utility. We then proceed by providing a "cookbook recipe" on how to implement each of the diagnostics, together with an interpretation of the results, using two worked examples of integrated stock assessments with Stock Synthesis. Further, we provide a conceptual flow chart that lays out a generic process of model development and selection using the presented model diagnostics. Based on this, we propose the following four properties as objective criteria for evaluating the plausibility of a model: (1) model convergence, (2) fit to the data, (3) model consistency, and (4) prediction skill. It would greatly benefit the stock assessment community if the next generation of stock assessment models could include the diagnostic tests presented in this study as a set of open source tools.