Dive Into Deep Learning - Lecture 3: Build a Simple Neural Network from Scratch with PyTorch

In this video, we defining a utility function to shuffle the dataset and access it in minibatches using and DataLoader. We also discuss how to use by reviewing background information regarding derivatives, partial derivatives, and gradients. We also discuss how to define a fully connected layer using . Before, optimizing our model’s parameters by minibatch stochastic gradient descent, we show how to access model parameters, i.e., weights and bias, and different ways to initialize them. After initializing our parameters, our next task is to update them until they fit our data sufficiently well. We also talk about defining the squared loss function in PyTorch, which is essential for computing gradients and updating model parameters. Using one minibatch randomly drawn from our dataset, we will estimate the gradient of the loss with respect to our parameters. In each iteration, we get a minibatch of training examples, and pass them through our model to obtain a set of pre