Introduction -- A quick tour of traditional NLP -- Foundational components of neural networks -- Feed-forward networks for natural language processing -- Embedding words and types -- Sequence modeling for natural language processing -- Intermediate sequence modeling for natural language processing -- Advanced sequence modeling for natural language processing -- Classics, frontiers, and next steps.