After remedying the smelly towel situation, I decided to go ahead and just clean my top loading washing machine to make sure it wasn't contributing to the problem. I found a solution on pinterest on how to clean your top loader washing machine that worked well.
First I filled the washer with hot water (the hottest water my washer offers). I then added 4 cups of bleach with no detergent. Then I set it to the longest wash and spin cycle. After a couple of minutes of letting it agitate I stopped the cycle and let it sit for one hour. After one hour of sitting I let the cycle finish.
Right after the cycle was finished I filled the washer with hot water again and this time I added 4 cups of my trusted white distilled vinegar (again no detergent or bleach). I repeated the process of setting it to the longest wash and spin cycle, letting it agitate for a couple of minutes, stopping the cycle for an hour, and then letting the cycle finish.
During this whole process I also cleaned off the entire washing machine outwardly and its dispensers where the water wouldn't reach to clean.
And voila - I now have a clean and sparkling washing machine.
This is obviously not something you want to do super often as it requires much water etc., but it is a good idea to do once in a while.