Precision hypoid gears are widely used in power transmission systems to transmit large torque between two perpendicular, non-intersecting shafts at very high speed. However, this class of geared rotor systems is often plagued by excessive vibration and noise problems even at light load condition. To examine the root cause of this problem, a nonlinear time-varying torsional dynamic model is formulated. The model is applied to examine the fundamental physical phenomenon governing the generation of dynamic mesh force and subsequent nonlinear tooth impact response. Solutions are presented in the forms of semi-analytical and numerical integration results. Parametric studies reveal new, interesting findings of the controlling factors related to the out-of-phase gear pair torsion mode. Also, the occurrences of single-sided and double-sided tooth impacts leading to high degree of nonlinearity in the gear pair system can be clearly observed in the predicted dynamic response.