Hesburgh Libraries

Learn about how to apply recurrent neural networks with PyTorch to text-based applications.

Those who study words as a stream of rapid sound, as objects which fulfill roles in a larger discourse structure, or as an interpretable vessel for abstract ideas know that, in all these cases, language is far from simple. Yet somehow neural networks manage to capture elements of them, sometimes mimicking or even surpassing human performance on language-based tasks.

Recurrent neural networks have played a critical role in this process; up until recently, they were the de facto standard in the field of natural language processing. This workshop aims to provide participants with an in-depth exploration of recurrent neural networks.

On the one hand, it provides a full, hands-on introduction to the code which represents and uses a recurrent neural network, going all the way from loading data to evaluating a trained model. On the other hand, it also accompanies the abstract architecture and its programmatic form with linguistic examples to provide a stronger intuition about the networks’ capabilities.

After completing this workshop, participants will:

1. Know the general formulation and inductive bias of recurrent neural networks and be able to describe common variants of these networks.
2. Identify and differentiate common task setups for recurrent neural networks.
3. Recognize many critical components of the PyTorch library and how they can formulate elements such as automated data loading, a general neural network architecture, and both the training and evaluation processes for a neural network.
4. Connect neural networks with ideas from formal language theory in order to have a better sense of the capacity for neural networks to learn complex phenomena.
5. Understand the role of hyperparameters in creating and applying a neural network.

Prerequisites:

1. Please bring your laptop or visit the Circulation Desk before the session for a short-term laptop loan.
2. An intermediate understanding of Python is expected; in particular, participants should have facility with the syntax and some familiarity with built-in libraries. Proficiency in other libraries--especially those for scientific computing (e.g., numpy, scikit-learn) or for neural network development (e.g., PyTorch, TensorFlow)--is helpful but not required.
3. A familiarity with vector and matrix mathematics (e.g., addition, multiplication, dot products) is expected, and an understanding of calculus concepts like derivatives is recommended. Having knowledge of tensors is also helpful, but it is not required.

This session will be presented by one of the NFCDS Pedagogy Fellows. Visit this page for more information about this fellowship program.