There has been increased interest in the immune stimulatory properties of ionizing radiation based on several preclinical models and recently completed clinical studies performed in combination with checkpoint inhibitors. This is a paradigm shift in that it considers the role of radiation beyond its direct cytotoxic effects, however, the factors that promote or limit radiation-induced immunogenicity are still unclear. Here we review the role of radiation in modulating the various aspects of the tumor immune microenvironment and discuss in particular the direct effects of radiation on the DNA damage response and its immediate consequences to neighboring cells. The latter “danger response” in particular can enhance recruitment of dendritic and macrophage cells to the tumor microenvironment, which in turn can activate or diminish subsequent T-cell priming. Identification of the critical factors that modulate the interaction between radiation-induced cell damage and the immune system will allow for rational combinational therapy design and the development of biomarkers that predict effective immune responses.