Building simulation models play a vital role in optimal building climate control, energy audit, fault detection and diagnosis, continuous commissioning, and planning. Real system parameters are often unknown or partially unknown and need to be identified through historical data, which are currently acquired by heuristically designed experiments. Without quality sensor data, model calibration is prone to fail, even if the calibration algorithm is appropriate. In this paper, we propose a Fisher-information-matrix (FIM)-based metric to examine the sensor data measurements and how their quality is related to the model calibration quality. It aims to provide quantitative guidance in the calibration cycle of a whole building model that takes as many variables as possible into consideration for the sake of accuracy. Our concerned model is based on well-known physical laws and tries to avoid simplification, thereby leading to a highly discontinuous system with model switches due to the seasonal or daily variation and other reasons. Such a model is implemented in the form of a software package. Hence, no explicit mathematical expression can be given. A key technical challenge is that the complexity of the model prohibits the analytical derivation of FIM, while the numeric calculation is sensitive to sensor noise and model switches. We, hence, propose to adopt an automatic differentiation method, which exploits the operator overload feature of object oriented programming language, for robust numerical FIM calculation.