Habitat models are often used to describe species distributions, but they need to be tested to evaluate their predictive performance. We investigated the importance of model evaluation in a study of habitat selection by Saltmarsh Sharp-tailed Sparrows (Ammodramus caudacutus), a species of conservation concern in eastern North America. We estimated the number of birds, nests, and fledglings produced in thirty 1-ha study plots spread across multiple marshes and used an information-theoretic approach to select among explanatory habitat models. Model performance was evaluated using both the original data set and data from another 30 plots from the same set of marshes. Our models indicate that both female and male Saltmarsh Sharp-tailed Sparrows increase in abundance as one moves away from the marsh's upland edge and in areas where there is deep thatch and uniform vegetation height. In contrast to the results for adult birds, number of nests and production of fledglings were positively associated with Saltmeadow Rush (Juncus gerardii). These models suggest that where adults spend the most time is influenced by the vertical structure of the vegetation but that nesting activity is determined more by plant compositions that indicate subtle variations in marsh elevation. Despite the fact that we found good internal consistency for our models during model evaluation, model performance worsened considerably when used to make predictions about new sites. Thus, although our study identifies several new factors influencing habitat selection in Saltmarsh Sharp-tailed Sparrows, it also highlights the need to be cautious when making predictions from habitat models.