There is another way if you want to get rid of it permanently. If you are happy with Matt2's solution there is no need to do so. I wanted remove it totally, because the mouse gestures provide by the extension All-in-one Gestures don't work on new tabs with the new startpage (by the way, I think its official name is new tab page).
Step 1
Enter this address to the location bar and confirm that you will be careful:
about:config

Step 2
Search for the preference with name:
browser.newtab.url

Step 3
Double click to modify and set the value to
about:blank
or
about:startpage
if you prefer to see your own startpage (set its url in the standard preferences dialog).
Close the window, you're done!