This is a copy-paste from another answer ( link here ). All credits to the original author of the question.
You will have to use a brute-force software to open it. These programs often use brute-force or dictionary-based attacks. I would not use such a closed-source programs from random, untrusted sources as suggested. Virus scanners do not find everything, so you still cannot trust the executable because ClamAV detects nothing.
Since LibreOffice is open source, I'd start with looking up what kind of encryption it uses. Ubuntu 12.04 ships with LibreOffice 3.5. According to http://wiki.documentfoundation.org/ReleaseNotes/3.5#Different_Encryption_Algorithm, it uses a 256-bits AES cipher.
This mailing list post also makes clear that the password is derived using PBKDF2 which means that brute-forcing will be much slower with a sufficient high iteration count.
Since .ods files are just Zip files, I tried to encrypt the file and extracted the contents. As expected, the contents of the document are encrypted and indistinguishable from random bytes. Of course there exist some unencrypted metadata, one of them being
META-INF/manifest.xml. My example encrypted spreadsheet contained the following interested parts:
<manifest:encryption-data manifest:checksum-type="urn:oasis:names:tc:opendocument:xmlns:manifest:1.0#sha256-1k" manifest:checksum="48KzqP1PL7Wu/YTtHzlN0buJeUmigGT247dZ6Wrj10s=">
<manifest:algorithm manifest:algorithm-name="http://www.w3.org/2001/04/xmlenc#aes256-cbc" manifest:initialisation-vector="82mrg52Yifh1iIye5W0xuw=="/>
<manifest:key-derivation manifest:key-derivation-name="PBKDF2" manifest:key-size="32" manifest:iteration-count="1024" manifest:salt="hUZrwD1BWkODYVklZiScqA=="/>
<manifest:start-key-generation manifest:start-key-generation-name="http://www.w3.org/2000/09/xmldsig#sha256" manifest:key-size="32"/>
From that we can learn that LibreOffice uses a SHA256 hashing algorithm for checking data integrity, AES256 in CBC mode. The 32-byte password is derived from your password using PBKDF2 with 1024 iterations.
A paper on decrypting ODF files is available here, these contain nice information to craft your own brute-forcer but are probably not suitable for the average user.
As with most encryption products, password recovery is near impossible. I suggest:
- If the file just contains bookkeeping for a week, just start over and do not waste time on decrypting the file.
- Try to recall the password if you want to decrypt the file.
- If you are going to use a brute-force program, do not use a random program found on the internet. Try to find an open-source program. If you only find closed-source programs, download it from a reliable source and ensure that it has good reviews (from several sources), put it through https://www.virustotal.com and check again that the program you are trying is legit.