RNN language model
Revisiting MLPs
Section titled “Revisiting MLPs”RNNs are universal approximators and can implement all of the algorithms. But not very easy to train.
A limitation of MLPs is that they have a fixed-size input and output. This makes it hard to model sequences of varying lengths, such as sentences or time series data. RNNs fix this by using a recurrent structure that allows them to process inputs of arbitrary lengths.