What is the output of a plain
lspci -v ? (without the grep mentioned above?)
Make sure your BIOS is set to use the card you want to use as the VGA primary card. It probably has an option like "Primary video: PCIE16-slot1, PCIE-slot2, .. ". Make sure this is set to the slot of the NVidia card.
Blacklisting the modules might work but I imagine that the Xorg server will still try and run a vesa driver on the other card. Another option may be to use the pci-stub driver to essentially 'hide' the device from Linux. See here:
.. You might need to add these commands to scrips that run early in startup.
Failing those, you'd need a custom /etc/X11/xorg.conf to force the xserver to run on only one GPU. Otherwise, the Xorg server will scan the PCI bus and helpfully load any drivers it finds and (presumably) try to use them. Do
man xorg.conf and then run
Xorg -configure to generate an xorg.conf.new in the current directory. Then edit from the generated data, paying special attention to the BusID field under section Device. Then look at the "screen" section of the config, and remove any screens with a reference to the device (probably Card0 or Card1) that you don't want to use.
However, you may want to reconsider your use case. From your comment:
I'd like to use the HD 4770 and the radeon drivers for X and the HD 7770 for a Windows Virtual Machine or just the HD 4770 and ignore the HD 7770.
That's a very, very tall order despite how simple it seems. The problem is that either you need a system that has an IOMMU that can support re-routing the PCI(-E) bus using hardware virtualization (this needs CPU, bios and chipset support). Very few motherboards have the full support necessary (most have borked BIOS tables). Here's an account of someone trying to get their asus motherboard to work with virtualbox and running into BIOS issues:
Even systems that have an IOMMU working enough to support things like network card passthrough fail when working with video cards since the video cards are still treated as 'special' by the pci bus for legacy reasons (particularly the primary card as marked by the bios).
Otherwise, you're stuck using experimental hacks to various hypervisors trying to make things like this work using paravirt PCI-passthrough support. Xen claims to support using radeon cards in guests that are non-primary (and hidden using their PCIback driver). They also claim to be able to passthrogh an intel card to a guest even if its the primary card on the host. I personally never got it to work. NVidia cards are always harder to deal with in this case due to lack of documentation. It's quite possible that VMWare, KVM, or virtualbox may have better support for this, i don't know.
You can find information on Xen's graphics passthrough here:
Really, if you don't have functioning IOMMU support, give up trying to run a VM with video card passthrough, save yourself the pain.