Accurate condition assessment of in-service infrastructure systems is critical for system-wide prioritization decisions. Manual visual inspection is currently the main form of assessing the physical and functional conditions of civil infrastructure at regular intervals in order to ensure the infrastructure still meets its expected service requirements. During a routine inspection, qualified and trained inspectors visually observe and manually record their observations, a costly and time-consuming process that often results in subjective and variable final reports. Thus, making the inspection process less costly, less obtrusive, more quantitative, and more consistent is a major research need. In particular, there is a need for better methods of recording the visual representation of a structure at a given inspection interval, in order to create a more consistent and repeatable record of structural health. This dissertation presents a contact-less and nondestructive computational framework which attempts to integrate computer vision, robotics, and remote sensing to provide a quantitative inspection methodology that can decrease cost, expedite inspection and facilitate access in comparison with the current inspection routine. First, it uses a set of two-dimensional (2D) digital images to produce a high-resolution and scale-accurate photorealistic three-dimensional (3D) model through a multi-scale and adaptive photogrammetric approach, followed by a fully automated robust segmentation of structural elements (e.g., columns and beams) in 3D models and a systematic and autonomous damage detection method. The accuracy, effectiveness, adaptability, and feasibility of the presented framework were evaluated by comparing its performance against conventional methods on large-scale infrastructure systems.