Axolotl: Feet First Into Dependency Hell
With Axolotl you can create custom configs, mess with your optimizer, and so much more. It offers an amazing, albeit confusing way to train and fine tune Large Language Models.
You can access the code for it here: https://github.com/OpenAccess-AI-Collective/axolotl (opens in a new tab)