Learning and Leveraging Neural Memories