In this work we consider the geometrical model of R. A. Fisher, in which individuals are characterized by a number of phenotypic characters under optimizing selection. Recent work on this model by H. A. Orr has demonstrated that as the number of characters increases, there is a significant reduction in the rate of adaptation. Orr has dubbed this a “cost of complexity.” Although there is little evidence as to whether such a cost applies in the natural world, we suggest that the prediction is surprising, at least naively. With this in mind, we examine the robustness of Orr's prediction by modifiying the model in various ways that might reduce or remove the cost. In particular, we explore the suggestion that modular pleiotropy, in which mutations affect only a subset of the traits, could play an important role. We conclude that although modifications of the model can mitigate the cost to a limited extent, Orr's finding is robust.