Propositional representation for graphical knowledge

Multi-media interfaces with a graphics and a natural language component can be viewed as a Natural Language Graphics systems without a host program. We will investigate a theory of Natural Language Graphics that is based on the notion of "Graphical Deep Knowledge" defined in this research. Graphical Deep Knowledge is knowledge that can be used for display purposes as well as reasoning purposes and we describe the syntax and semantics of its constructs. This analysis covers forms, positions, attributes, parts, classes, reference frames, inheritability, etc. Part hierarchies are differentiated into three sub-types. The usefulness of inheritance along part hierarchies is demonstrated, and criticism of inheritance-based knowledge representation formalisms with a bias towards class hierarchies is derived from this finding. The presented theory has been implemented as a generator program that creates pictures from knowledge structures, and as an augmented transition network grammar that creates knowledge structures from limited natural language input. The function of the picture generation program Tina as a user interface for a circuit board maintenance system and as part of a CAD-like layout system is demonstrated.