10

I have a PDF file that I want to compress to less than 100 kB. It has a scan of two pages that I scanned from my mobile scanner. Post scan it is 338 kB (with minimum quality to view the pages). Basically I want to upload this file to a government portal which only allows upload of a maximum 100 kB in just one file. This is my primary purpose. The below are the methods that I have tried now and it did not do the needed task:

  • using a simple wrapper around Ghostscript to shrink PDF files "./shrinkpdf.sh in.pdf out.pdf xx". I set xx to 90 and it gives me 282 kB. Below the 90 value, the text in PDF document is not visible clearly and I am sure my application will be rejected then.

  • I also tried "gs -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dPDFSETTINGS=setting -sOutputFile=output.pdf input.pdf" with setting value as /screen which I think is minimum setting and it gives me 232 kB and the text is fairly visible.

  • I also tried converting it to JPEG using "pdftoppm" method like - "pdftoppm compressed.pdf jpeg -r 75 -jpeg" which gives me 141 kB for 1 page and 128 kB for page 2. I am not sure how I will get these two JPEG files to upload as one file, but I guess my primary aim should be now to get it less than 100 kB first?

I use Ubuntu 20.04.2.

4
  • There are many online websites that compress pdfs – Jdeep Apr 10 at 7:37
  • 4
    Hi Jdeep thanks, but I am afraid to use those as this is my home givernment document which is not supposed to leak! So not sure if I should take this chance... Hence I was looking for something within os. – Prathik M. Apr 10 at 7:38
  • Running image-to-text (OCR, optical character recognition) detection would embed the text as text, making it readable at any quality. – knallfrosch Apr 13 at 9:12
25

Below 90 value the text in pdf is not visible clearly and I am sure my application will be rejected then.

It's a scanned document. That means it's not text, it's an image of the page. PDF supports multiple image compression schemes, including lossless, but quality and degradation suggest that you're using JPEG.

This is probably the most efficient way to store it.

You want to store two pages in <100kB. That's 50kB per page. That's a tall order - but probably possible.

I would export the pages as JPG's and play with quality settings and resolution until you get your result:

convert file.pdf file.jpg

This will give you file-0.jpg and file-1.jpg for page 1 and two respectively.

Now we can try to reduce the resolution of the pages:

mogrify -resize 600x700 -quality 45 file-0.jpg

By this measure, I managed to get an A4 page down to 28kB. It's legible, but not very clear:

Unclear image

To convert your files back to a PDF after playing with them to reduce size, run

convert file-?.jpg file.pdf

In addition to mogrify to modify files, you can use tools such as gimp.

6
  • 6
    Another option to try is resampling to change the DPI. The scanner might have been set to a high DPI like 600x600, when something like 200x200 is plenty good. – muru Apr 10 at 8:51
  • You mean scanning it once more? Otherwise the DPI is just a integer in the file, not relevant to the size. – vidarlo Apr 10 at 8:52
  • 1
    No, I mean convert -resample X – muru Apr 10 at 8:55
  • 3
    That's basically the same as resizing, and won't really do anything to the filesize that resizing won't do. It may impact how the picture is presented, however. – vidarlo Apr 10 at 10:32
  • True, but sometimes you're forced to maintain some minimum size (e.g., pic must be at least X pixels wide or something - I don't remember what it was exactly, but there was case when I had to resort to resampling instead of resizing for some web form), in which case resampling can be of help. – muru Apr 10 at 10:34
13

Since a filesize-optimized scanned document is probably going to be black and white, .pbm is a monochrome bitmap format that seems perfect for this, and you can covert that back into a png to embed in a pdf.

Using a sample document1 (imgur permalink: https://i.imgur.com/Ak2kVGD.jpg)

Original Document

Its a 1751x2451 jpg scan of a document, size 1.71MB, black and white with some blue accents

convert document_scanner_sample_scan_00_zoom.jpg -resize 1000 intermediate.pbm
convert intermediate.pbm page1.png # 1000x1436, 46kb

page1.png looks quite presentable for 46kb (https://i.imgur.com/gYwtipQ.png)

Optimized Document

As pointed out in comments, the png needs to be transcoded to be embedded in the pdf. convert uses /FlateDecode pdf compression format by default (convert page1.png page1.pdf) and the resulting pdf is 67kb. Use the /CCITTFaxDecode format instead, which is for monochrome images, to get that down to 57kb

convert page1.png -alpha off -monochrome -compress fax page1.pdf

See the image magick documentation for a mapping of command line options to pdf compression format: https://legacy.imagemagick.org/Usage/formats/#pdf_compression

For documentation of the pdf compression filters, see section 7.4 of the pdf reference (version 1.7). An introduction is provided in section 7.4.1 Table 6.

https://www.adobe.com/content/dam/acom/en/devnet/pdf/pdfs/PDF32000_2008.pdf


1. Original sample document found here:

https://s1-www.scan2docx.com/img/samples/document_scanner_sample_scan_00_zoom.jpg

6
  • 1
    FYI, the PDF spec (ISO-32000-1) does not support the PNG image format directly, but any decent PDF tool (ncluding GhostScript, used by the OP) will automagically convert PNG to a compatible compressed raster image (with the the /FlateDecode filter) before embedding into PDF. See stackoverflow.com/a/20675801/766786 – Amedee Van Gasse Apr 12 at 22:05
  • @AmedeeVanGasse yeah good point. If you do convert img.png img.pdf and inspect the contents it looks like it automatically uses Flate for the conversion. It does add some bloat though so the pdf I created from the 46kb png ended up being 67kb. – chiliNUT Apr 12 at 23:50
  • 2
    @AmedeeVanGasse you can use the /CCITTFaxDecode filter for monochrome images to get the pdf down to 57kb, see updated answer – chiliNUT Apr 13 at 4:48
  • 1
    Your updated answer has improved significantly in quality, upvoting! – Amedee Van Gasse Apr 13 at 7:14
  • 1
    @AmedeeVanGasse thanks! I had some passing knowledge of pdf image filters but now I've learned more about them today after researching your comment – chiliNUT Apr 13 at 8:01
7

Thank you all for helping me! Thanks @vidarlo! Your suggestion and ideas really helped me get through this and also a little bit of luck did the trick. I will mark your input as solution since it really helps in this task.

For me the luck part was the government site allowing us to upload two files each of 100kb separately. This was no where mentioned! It only shows up a second dialog box in site post uploading one page. Waht!

So now the idea was to compress each page to less than 100kb. I decided to scan each page separately:

Page 1 144kb.pdf, and Page 2 165kb.pdf

I found that (for my document atleast) convert file.pdf file.jpg performs less than pdftoppm file.pdf jpeg -r 75 -jpeg. I am not sure why but the output of convert gave me 258kb.jpg for page one (144kb.pdf) and from pdftoppm it gave me 130.6kb .jpg for the same page, but a better looking jpeg file! I decided to go ahead with that.

pdftoppm 144kb.pdf jpeg -r 75 -jpeg  --> 130.6kb.jpg
pdftoppm 165kb.pdf jpeg -r 75 -jpeg  --> 134.4kb.jpg

Then like @vidarlo suggested, I tried mogrify but without resizing option, so mogrify -quality 50 page1.jpg and page2.jpg gave me 91kb and 96kb for two files! 45 does blur out things little more, and >50 increases the file size to > 100kb.

Just in case, while trying convert file.pdf file.jpg i got below error -

convert-im6.q16: attempt to perform an operation not allowed by the security policy `PDF' @ error/constitute.c/IsCoderAuthorized/408.
convert-im6.q16: no images defined `file.jpg' @ error/convert.c/ConvertImageCommand/3258.

For this i modified the policy.xml file located in /etc/ImageMagick-6. I had the line <policy domain="coder" rights="read | write" pattern="PDF" /> added before </policymap>, which was previously set to none (rights). This solution bypassed this error.

Also I had an issue modifying this policy.xml file and it was opening in read only mode (though i was logged in as admin). For that I used gedit admin:///etc/ImageMagick-6/policy.xml from terminal to make the file writable.

For both of this, I have to thank these,

Again, thank you all very much!

4
  • 7
    Unless your government is unreasonable (ok, no jokes about this statement...), usually the upload limit is a lot more than what they expect of you. Otherwise, they will get a lot of people complaining to them, and they wouldn't want that. While you know about computers, the average person does not. So, if you've hit this limit, it's possible you've done something wrong, perhaps? For example, maybe 100 kb is their way of asking you to just lower the resolution of your scan. – Ray Apr 10 at 15:13
  • 3
    @Ray More likely it's not supposed to be a scan or they just reused old form infrastructure. – David Mulder Apr 11 at 8:43
  • 1
    @Ray I agree. At first I thought it might have been some error from our side, but also I could see a lot of people struggling to achieve this and to our surprise the gov not changing this system online. I am not sure if I can show a snapshot in the comments here of where I was stuck, here is one from a site which shows the same: link I am talking about the electricity bill and the consent letter document type, max size is either jpg ot pdf is 100kb. – Prathik M. Apr 11 at 9:38
  • 2
    The ImageMagick policy was changed for a reason. It's much better to go directly to the underlying tools in this situation. (Or, alternatively, use img2pdf which won't undo various optimizations you ran previous steps and tries to optimize for size. For example, you could try making a grayscale JPEG, then feeding it to jpegoptim, then img2pdf.) – ssokolow Apr 12 at 21:37
6

Most likely the best way would be to scan it in black and white (two colors, not grayscale). A form for a government agency is unlikely to need full color or grayscale.

A decent scanner will choose a compression option other than jpeg for black and white images which will lead to much smaller files.

If that's not enough, then manually compressing the bw images with jbig2 can lead to even smaller files, although the savings only really start adding up for longer documents with many pages.

3
  • To compress a scanned file from 300kb to 100kb is impossible. Complain to the recipient.. Incidientally, this has nothing to do with printing. – brian_p Apr 10 at 19:35
  • 1
    @brian_p It's absolutely possible. That's a compression ratio of just 3. I have a large scanned document collection where I achieved additional compression ratios around 10 for documents that already came out of the scanner compressed with ccitt (i.e. much better than jpeg). – Nobody Apr 11 at 6:50
  • I considered this option initially, but since in my case it had to be in color because of stamp paper on it. But I think this idea should work for other requirements needing compression! – Prathik M. Apr 11 at 9:41
1

After a lot of experimentation with this I find the easiest method is to load the PDF into LibreOffice Writer (this may take some time and consume memory with large PDFs - so close unnecessary apps). Once loaded 'Export to PDF...' setting jpeg compression to 50% & image resolution to 150dpi - you can play with the compression & dpi settings to suit. Mike

1

It's your scanner settings.

You are scanning your documents like photos. By photo standards, text is lots of intricate and nuanced detail that takes lots of storage space.

The pages you are scanning are monochrome, black and white, ink present or absent. Monochrome scans are perfectly legible at 75 DPI (3 pixels per mm) but let's go a standard fax format, 100 DPI (4px per mm).

At that resolution, there are less than a million pixels on an entire sheet of paper. 1 bit per pixel, so 125kb raw. Two pages are 250kb raw.

Now, all you need is modestly competent compression, noting that white areas of the page will compress to virtually zero. At this point, this is within the range of lossless compression to get you under 100kb.

If there are shaded areas of the page, tell your scanner software not to "dither". Dithering uses a random pattern of on/off pixels to approximate grayscale. This noise is randomized (is entropy), so it is very hard to compress and takes a lot of space for no good reason.

Depending on the page content of your forms (which I can't know), they may compress to much better than 100k. If so, you may be able to get away with higher resolution, so redo it at 150 dpi (6 px/mm) and see if that makes it. Do not use software to downgrade resolutions unless they are exact multiples. (300 dpi to 150 dpi ok... 200 dpi to 100 dpi bad).

600 dpi is a nice "universal donor" as it downgrades to 200, 150, 120 or 100.

2
  • 1
    FYI, the PDF spec (ISO-32000-1) does not support the PNG image format directly, but any decent PDF tool (ncluding GhostScript, used by the OP) will automagically convert PNG to a compatible compressed raster image (with the the /FlateDecode filter) before embedding into PDF. See stackoverflow.com/a/20675801/766786 – Amedee Van Gasse Apr 12 at 22:05
  • @AmedeeVanGasse Oh thanks, PDFs from PNGs work so well that I presume it did. But PNG is lossless, so at least the input doesn't degrade with conversion. – Harper - Reinstate Monica Apr 12 at 22:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.