According to Wikipedia it's probably the chipset or BIOS.
Intel states the Atom supports 64-bit operation only "with a processor, chipset, BIOS" that all support Intel 64. Those Atom systems not supporting all of these cannot enable Intel 64.[26] As a result, the ability of an Atom-based system to run 64-bit versions of operating systems such as Ubuntu or Debian GNU/Linux may vary from one motherboard to another.
I actually eventually found this answer while writing the question but figured it was worth posting since I couldn't find an answer here. It might not be specific enough to Ubuntu though?