Sometime ago I created an
.ods file with Libreoffice Calc which I password protected. I have now forgotten the password and as a result I am unable to access the file. Is there any way of bypassing or recovering the password, e.g. as root using the terminal?
Opening the .ods file as root will still not allow you to retrieve the file contents because those are encrypted which is beyond the power of a root user.
MiJyn suggested the use of password crackers for OpenOffice.org. These programs often use brute-force or dictionary-based attacks. I would not use such a closed-source programs from random, untrusted sources as suggested. Virus scanners do not find everything, so you still cannot trust the executable because ClamAV detects nothing.
Since LibreOffice is open source, I'd start with looking up what kind of encryption it uses. Ubuntu 12.04 ships with LibreOffice 3.5. According to http://wiki.documentfoundation.org/ReleaseNotes/3.5#Different_Encryption_Algorithm, it uses a 256-bits AES cipher.
Since .ods files are just Zip files, I tried to encrypt the file and extracted the contents. As expected, the contents of the document are encrypted and indistinguishable from random bytes. Of course there exist some unencrypted metadata, one of them being
From that we can learn that LibreOffice uses a SHA256 hashing algorithm for checking data integrity, AES256 in CBC mode. The 32-byte password is derived from your password using PBKDF2 with 1024 iterations.
A paper on decrypting ODF files is available here, these contain nice information to craft your own brute-forcer but are probably not suitable for the average user.
As with most encryption products, password recovery is near impossible. I suggest:
Answer was based on the conversation I had with the OP in the comments
There is no way of bypassing the password because if it was possible, it would completely defeat the purpose of having a password-protected document (the only people who would not want to use the password by-passer is the ones who have the password). The only option is to actually use a password cracker. This method is legal if, and only if you are the owner of the document, or the person who sent you the document gave you permission to crack the password. But, of course, these days no one cares about being on the right side of the law :P
So if you are still interested in doing so, here is another drawback of this technique (yay!): It takes a very long time because it has to find every single possible password combination. Just some very basic math here, if it takes 1 millisecond to try a password, and the password is ASCII encoded, 5 characters long, then it would take 56 weeks to find it. If you have a very fast computer (maybe even a supercomputer), it would take around 5-10 microseconds per password, so it would take 1.5-3 days to calculate it. If you use a graphics card instead of your CPU, it will be able to calculate passwords much faster, as GPUs usually have around 32-64 cores.
Another problem, specific to OpenOffice passwords, is that OpenOffice has very good encryption methods, making it much longer and harder to find passwords. This means that only a brute-force attack (the slowest attack) will work!
EDIT: Seems like that one only supports Writer passwords. Here is the same program, apparently engineered for Calc passwords. Notice that I have not tried that one either, but I have virus checed that one with ClamAV too: http://www.sharewareconnection.com/download-openoffice-calc-password-recovery-from-sharecon.html
EDIT 2: Lekensteyn posted a way better way of doing this than using an untrusted .exe file. I highly recommend using his way instead!
EDIT 3: Based on the method Lekensteyn posted, I wrote a little tool that implemented it, located here: https://github.com/MiJyn/ooo-pass-recover/downloads. It is, of course, open-source (just click on "code" from the download link). All it is is just an interface to ODFJlib by Ringlord (same person that made the document Lekensteyn posted). Notice that this tool is written in Java, and is rather slow (243 keys/sec on my computer). I will try to optimize it for multiple cores though.
Example of usage: