While machine learning algorithms are being successfully employed for a wide variety of high level cognitive tasks, their wide-spread adoption on mobile platforms faces challenges because of the large network sizes and extremely long training times involved in learning underlying features of user-dependent data. In this paper, we first discuss the requirements and target specifications of nanoscale devices that could enable hardware platforms supporting on-chip, real-time learning based on third generation spiking neural networks. We then survey some recent developments from the field of emerging non-volatile memory technologies such as chalcogenide based phase change memory and other memristive devices that could be used to build cognitive learning systems. We will then discuss how we have implemented synaptic plasticity in a Cu/SiO2/W memristive device at voltages below 500 mV using bio-mimetic waveforms. These devices hold promise to herald the next generation of information processing systems that are inspired by the brain.