If converting to djvu would also be ok and if no colors are involved, you could try the following:
Convert the pdf to jpg files using pdfimages -j
If you get pbm files instead, you should do the intermediate step:
for FILENAME in $(ls *.pbm); do convert $FILENAME ${FILENAME%.*}.jpg ;done
The convert command is from the imagemagick package.
Then use scantailor to make tif's out of it.
In a last step you go to scantailors out direcory (where the tif's are located) and apply djvubind to that directory.
This should reduce the filesize drastically without big quality loss of the text. If you want finer control over the ocr-backend, you may try djvubind --no-ocr and use ocrodjvu to add the ocr layer afterwards.
If you have color's in your document things get a bit more complicated. Instead of djvubind you could use didjvu and in scantailor you have to change to mixed mode and select sometimes color-images manually.