Discriminating between high- and low-quality field depletion experiments through simulation analysis

Between 1997 and 2011, the National Marine Fisheries Service conducted 50 depletion experiments to examine efficiency of survey gear for capturing clam species and to estimate stock density for populations of Atlantic surfclams (Spisula solidissima) and ocean quahogs (Arctica islandica) by using commercial hydraulic dredges. The Patch model was formulated to estimate gear efficiency and organism density from the depletion experiment data. The range of efficiencies estimated is substantial, leading to uncertainty in the application of these estimates in stock assessment. Known values of 4 measures of experimental performance for each of the 50 depletion experiments conducted in the field were compared with values of those same characteristics from 9000 simulated depletion experiments, values that were assumed to represent a suite of conditions that might also occur in the corresponding field experiment. These comparisons allow analyses of the quality of field experiments that would otherwise not be possible and provide evidence for weighting the results of field experiments beyond traditional measures of uncertainty. The characteristics of performance were used to identify a subset of field experiments that were more likely to have produced inaccurate estimates of gear efficiency, potentially introducing bias and, as a result, lowering the efficiency estimates in the entire depletion data set.