The effect of large displacement rotational motion of the shafting system on the higher frequency, small displacement vibratory motion primarily excited by gear transmission error and variation of gear mesh stiffness is examined in this paper. Traditional hypoid gear dynamic analysis based on a pure vibration model assumes that the system perturbs about its mean position without coupling to the large displacement rotational motion. To improve on this approach and understanding of the influences of the dynamic interaction, a coupled multi-body dynamic and vibration simulation of the hypoid geared rotor system is performed. In the proposed formulation, a multi-degrees-of-freedom, multi-body hypoid geared rotor system dynamic model is developed to calculate the combined motion of the large displacement rotation of the shaft and small vibratory motion of the gear pair. The formulation may be generalized to other forms of gearing because hypoid gears have more complicated geometry and time-varying mesh characteristics when compared to parallel axis gears. Numerical simulation results are compared to those derived from the classical analytical method that only considers pure vibration effect. The proposed theory also provides new approaches to investigate both steady-state and transient geared rotor system dynamics.