An important driver of the evolution of animal coloration is sexual selection operating on traits that are used to transmit information to rivals and potential mates, which has a major impact on fitness. Reflectance spectrometry has become a standard color-measuring tool, especially after the discovery of tetrachromacy in birds and their ability to detect UV light. Birds' plumage patterns may be invisible to humans, and therefore the establishment of reliable and quantitatively objective ways of assessing coloration not dependent on human vision is a technical need of primary importance. Plumage coloration measurements can be taken directly on live birds in the field, or in the laboratory (e.g., on collected feathers). However, which of these 2 approaches offers a more reliable, repeatable sampling method remains an unsolved question. Using a spectrophotometer, we measured melanin-based coloration in the plumage of Barn Swallows (Hirundo rustica). We assessed the repeatability of measures obtained with both traditional sampling methods to quantitatively determine their reliability. We used an ANOVA-based method for calculating the repeatability of measurements from 2 years separately, and a GLMM-based method to calculate overall adjusted repeatabilities for both years. The results of our study indicate a great disparity between color measurements obtained using both sampling methods and a low comparability across them. Assuming that measurements taken in the field reflect the real or “true” color of plumage, we may conclude that there is a lack of reliability of the laboratory method to reflect this true color in melanin-based plumages. Likewise, we recommend the use of the GLMM-based statistical method for repeatability calculations, as it allows the inclusion of random factors and the calculation of more realistic, adjusted repeatabilities. It also reduces the number of necessary tests, thereby increasing power, and it allows easy calculation of 95% CIs, a measure of the reliability and precision of effect-size calculations.