I dropped my Nexus 4 and the screen now is completely broken, and yet I have all these files on my Nexus 4 that I need recovered.
I had ubuntu 16.04 installed on it (Yes, the ubuntu phone, not android). What can I do to get say, my contacts back?
If you had Ubuntu installed on your phone (wow! good job!), you may be able to open a SSH connection... and then recover any file you need.
Else, you could try to plug your phone to an external screen, keyboard and even mouse. I'm not sure how it can handle peripherals, but since it's Ubuntu i'm quite optimistic. (However, it should be wired since wireless needs configuration on phone's side, which you can't control yet).
If you can plug a screen, you may be able to duplicate the your phone display. If the tactile is broken, you'll need a keyboard to navigate. Then you could either install a SSH server (and copy any file from your computer) or send them to a cloud service like Dropbox or Ubuntu One or Google Drive or even Git or whatever else, or even copy them to a USB key, but for that you need a OTA cable.
So I guess you'd need to find an adapter to plug USB (with OTA support for the key) and HDMI devices.
One other solution to consider is to try to boot from USB. I don't even know if it's possible but I guess you know if your BIOS is configured to launch on USB. If so, you could boot on a USB with a light distro executing a script when it starts, to mount your phone's partition and then copy files from it.
Lastly, one question : what happens when you plug your phone to your computer? Does it just not mount as any USB key?
TL;DR
Can you plug peripherals on your phone, do you know how they are handled?
Have you installed a SSH server on your phone?
What if you just connect your phone to your computer?
Do you have a dual boot on your phone? I'm not sure I understood what you mean.