Shellfish Quality Assurance programs rely on the use of bacteriological fecal pollution indicators as a routine monitoring tool for risk assessment and management in shellfish growing areas. Internationally, shellfish programs vary in detail but are typically based on either the United States model, as in Australia, which relies on the enumeration of indicator bacteria in water as a measure of fecal pollution, or the European Union model, which assesses the exposure of production sites to fecal pollution by determining indicator bacteria present in shellfish flesh. The equivalency of these approaches for delivering the same level of public health protection is not immediately apparent Both indicator tests are used in combination in a very conservative approach adopted in the Australian state of New South Wales (NSW) by the Shellfish Program for the oyster (predominately Sydney rock oyster) fishery, the country's oldest and the state's most productive fishery. This study reports on the analysis of a large data set of these two indicators taken at multiple sites within three NSW estuaries between 2000 and 2005. We report on performance measures used routinely in epidemiological studies to compare the two tests. These measures demonstrate poor agreement between the two tests. When the harvest area is deemed closed by the water test the odds of a negative test based on shellfish flesh is approximately one in four. Conversely, the odds of a positive test based on shellfish flesh occurring during extended periods of dry weather when the water test is clear exceed one in five and are seemingly random in occurrence. We are able to demonstrate a highly significant statistical relationship between the water test and environmental covariates of rainfall, salinity, and water temperature. We use k-fold cross validation methods to develop predictive models using these environmental covariates that can account for over 90% of the variation in water test readings. In contrast we are unable to demonstrate a valid predictive model for the oyster flesh test using these covariate. We discuss the results of these analyses and suggest that there are significant issues with the efficacy of the shellfish flesh test and, in particular, the unexplained systematic error that occurs at a high rate in the current program. The dual use of these tests in harvest management results in routine uncertainty that must create significant unpredictability and costs in trade. In turn, this uncertainty and the costs of testing result in levels of commercial risk that could prove unsustainable. The study highlights the need for the performance of tests to be rigorously appraised in shellfish quality assurance programs. In so doing it may be possible to maintain public health standards while minimizing unnecessary disruptions and costs in the trade of fresh oysters.