I'm new to Ubuntu and my question is pretty much the title. I just need help with this because I don't want to go through and specifically name ALL of my songs.
No, it wont, by default. I am not aware of any plugin to do the task either. The plugin that is with Rhythmbox, might get you the covers for it. If you want to tag your songs, I recommend you use MusicBrainz Picard. Open a terminal from Menu > Accessories > Terminal and type the following:
To get started with tagging, read the manual (with screenshots) given at the Picard website.