The high seismic vulnerability of unreinforced masonry (URM) buildings has been observed during earthquakes. Many URM buildings still exist and are susceptible to failure during future earthquakes, especially in areas of low to moderate seismicity. To aid in quantifying this risk, the broad objective of this dissertation is to develop and demonstrate a numerical analysis framework for assessing the seismic performance and expected vulnerability of URM building structures. A robust and efficient panel macro-element model for the numerical analysis of in-plane URM walls is presented and validated. Importantly, this model can quantify the contribution of different damage modes at different locations in a wall, including the effects of perforations from doors and windows. The behaviors of a two-story residential prototype wall and a five-story mixed-use wall are characterized and compared under monotonic and reversed-cyclic lateral loading. Four wall model scenarios that represent assumptions regarding out-of-plane wall interaction and loading from flexible floor and roof diaphragms on the in-plane wall are investigated. Then, the demands from Incremental Dynamic Analysis (IDA) on the four model scenarios of the prototype two-story wall are used to obtain estimates of damage considering scaled ground motion suites for Boston, Seattle, and Los Angeles. Monte Carlo simulation is used to probabilistically sample material properties and modeling assumptions to capture both the aleatory uncertainty from the building and epistemic uncertainty from the models. Fragility curves and damage functions are developed for each model scenario and compared against each other and across the three ground motion suites. It is found that the impact of the different ground motion suites on the view of vulnerability for the prototype two-story wall is small compared to the impact of the different assumptions inherent to the four model scenarios. Importantly, an in-plane wall that is isolated from the out-of-plane walls and the floor/roof diaphragms will exhibit the lowest seismic vulnerability of the four scenarios analyzed. For this scenario, the estimated damage is concentrated in the first story due to rocking of the piers and diagonal tension/compression in the coupling beams, which can be mitigated by efficient retrofitting measures that address these damage modes.