Feed Forward Networks: Only One Way To Go From Here

Unlike Recurrent Neural Networks, which have many problems and deficiencies, where information moves backwards and forth throughout the network, a feed forward network only moves information in one direction: forwards.

This network will take the output of the self attention layer and transform it into the final output of the model.