H264 is a high quality video codec but (as you refer), it makes "heavy" size video files.
I am not sure if your question is related to a GUI or Command Line task. But, assumming that you are using "Transmaggedon" I will write this considering the options for GUI applications.
In my case, I prefer the usage of "Mobile Media Converter" which works pretty fine in 32bit systems (both Ubuntu and MS based systems). And the creators have already released a 64 bit version nowadays.
What you wish is to make smaller size video files. Which can be done by changing the "video bit rate", which will give you the best quality, according to the video bit rate you request. In example: a 40MB video file (uncompressed, AVI) will fit in a 5MB video file (any codec) if you use a 800kbps video bit rate. But -of course- it may harm the quality.
I strongly suggest you to perform some testing using the Mobile Media Converter, which can be reached (.deb) by clicking the next link:
After installing, just drop any video files (preferably small for the test) into the drag and drop area, then open the Advanced options by clicking a small arrow that is placed at the very right of the window, below the "browse" button. "[ ... ]"
Next, click in "Advanced" and you will have all the options that you need in order to transcode your video and make small and good quality transcodification.
Remember: The higher video bit rate is, you will have better quality but the file size will become "heavier".
BTW: You can also transcode videos by using a non linear editing system, such as "Kdenlive" (http://www.kdenlive.org/) or "Openshot" (http://www.openshotvideo.com/) and many more. A non linear editing system will also be useful to edit the video (trim, crop, add text and more).
Here is a screenshot for you to see a few options of Mobile Media Converter in action.