47

Duplicity does already a great job of making efficient backups, but I want still to minimize the amount of files being back-upped.

What files or directories can be excluded in a home directory? Please name these explicitly, wildcards allowed, paths are relative to the home directory.

Improve this question
1
  • Great question! I find it useful to analyze my home directory (the one I back up) using Disk Usage Analyzer and make sure to ignore anything that shows up in the top and which I don't mind losing in case of data loss. For instance, I'm not interested in backing up my src directory containing local copies of source code available online.
    – joelpet
    Jan 12, 2014 at 17:58

4 Answers 4

Reset to default
44

Most of these files and directories are recreated if missing. They are mostly intended for lowering the startup time of an application (Firefox extensions cache), indicating locks (.gksu.lock), or containing session-specific information (e.g. ID of dbus session). Recent documents changes frequently and is generally not that sensible to backup.

These directories may be excluded:

.gvfs                           # contains mounted file systems?
.local/share/gvfs-metadata
.Private                        # contains the actual encrypted home directory
.dbus                           # session-specific
.cache
.Trash                          # do I need to say more?
.local/share/Trash
.cddb                           # cached info about audio CDs
.aptitude                       # cached packages lists

Flash-specific:

.adobe        # Cache for flash, maybe others?
.macromedia   # except for Flash persistence, there is no reason to keep this

Files:

.xsession-errors            # contains errors from the current graphical session
.recently-used              # recently used files
.recently-used.xbel
.thumbnails
.Xauthority                 # session-specific
.ICEauthority
.gksu.lock
.pulse                      # directory
.pulse-cookie
.esd_auth

KDE specific:

.kde/share/apps/RecentDocuments # Recent documents on KDE
.kde/share/apps/klipper         # Contains a history of the Klipper clipboard (KDE)
.kde/share/apps/okular/docdata  # you will loose saved scrolling positions of PDFs
.kde/share/apps/gwenview/recentfolders
.kde/share/apps/kmess/displaypics  # cached other users' profile pics
.kde/share/apps/kmess/customemoticons  # cached emoticons of others

Firefox-specific (see also Profile folder):

.mozilla/firefox/*/Cache
.mozilla/firefox/*/minidumps    # in case Fx crashes dumps will be stored in this
.mozilla/firefox/*/.parentlock       # session-specific           
.mozilla/firefox/*/urlclassifier3.sqlite  # phishing database, recreated
.mozilla/firefox/*/blocklist.xml   # blacklisted extensions
.mozilla/firefox/*/extensions.sqlite  # extension database, recreated on startup
.mozilla/firefox/*/extensions.sqlite-journal
.mozilla/firefox/*/extensions.rdf
.mozilla/firefox/*/extensions.ini
.mozilla/firefox/*/extensions.cache
.mozilla/firefox/*/XUL.mfasl     # cached UI data, recreated
.mozilla/firefox/*/XPC.mfasl
.mozilla/firefox/*/xpti.dat
.mozilla/firefox/*/compreg.dat

Opera-specific (related question on Superuser.com: Is documentation available on files and directories in the Opera profile folder?):

.opera/temporary_downloads
.opera/cache
.opera/thumbnails
.opera/opcache
.opera/icons
.opera/application_cache
.opera/widgets/*/cache
.opera/lock

Komodo Edit:

.komodoedit/*/codeintel/db
.komodoedit/*/host-*/*/codeintel
.komodoedit/*/XRE/Cache
.komodoedit/*/XRE/.activatestate/komodo edit/Crash Reports
.komodoedit/*/XRE/.activatestate/komodo edit/*/Cache
.komodoedit/*/XRE/.activatestate/komodo edit/*/minidump
.komodoedit/*/XRE/.parentlock
.komodoedit/*/XRE/extensions.rdf
.komodoedit/*/XRE/extensions.ini
.komodoedit/*/XRE/extensions.cache
.komodoedit/*/XRE/XPC.mfasl
.komodoedit/*/XRE/XUL.mfasl
.komodoedit/*/XRE/xpti.dat
.komodoedit/*/XRE/pluginreg.dat
.komodoedit/*/XRE/compreg.dat
.komodoedit/*/XRE/*.sqlite-journal
.komodoedit/*/pystdout.log
.komodoedit/*/pystderr.log
.komodoedit/*/history.sqlite.bak
.komodoedit/*/running.lock
.komodoedit/*/mutex.lock
.komodoedit/*/*.xmlc
.komodoedit/*/startup-env.tmp
.komoeoedit/*/commandments.fifo
.komoeoedit/*/history.sqlite

GnuPG:

.gnupg/rnd
.gnupg/random_seed
.gnupg/.#*
.gnupg/*.lock
.gnupg/gpg-agent-info-*

Google Chrome:

.config/google-chrome/Default/Local Storage
.config/google-chrome/Default/Session Storage
.config/google-chrome/Default/Application Cache
.config/google-chrome/Default/History Index *

Other apps:

.pulse/icons                  # Pidgin
.java/deployment/cache        # Cached applets
.icedteaplugin
.icedtea
.gnome2/epiphany/favicon_cache
.dropbox                      # to avoid problems, let Dropbox re-create these
.dropbox-dist
Improve this answer
2
  • Strange, the raw link doesn't work, when you call it directly, but it works with the git.io shortener: You can download the ignorelist from my gist with wget http://git.io/vmQZ2 -O /var/tmp/ignorelist Then start the rsync with rsync -a --progress --exclude-from=/var/tmp/ignorelist /home/$USER/ /media/$USER/linuxbackup/home/ see askubuntu.com/a/545676/34298
    – rubo77
    Jul 19, 2015 at 22:50
  • HEAD and master both do not work, but this link is valid: wget https://gist.githubusercontent.com/rubo77/8ffaadbc58ab099d2bc3/raw/ignorelist
    – Lekensteyn
    Jul 20, 2015 at 8:14
11

Good question. I like these -- where we can gather community input/lists and cherry pick relevant stuff. Here's my .exclude file. I feed this file to rsync as an --exclude-file= parameter in multiple scripts (which you can also do with duplicity). Note that the question implies "what do you want/need to save". For example, I backup my complete .mozilla DIR as it contains multiple profiles and tweaks and it's easier to dump it back on disk as a whole. Also implied in my config: I do a lot of "build from the ground up installs" for testing. So I have a clonezilla image of a known-good-clean-base install, and I add in my personal (/home) data stored on USB after it's on disk; sometimes selectively, sometimes as a whole.

# Filename: admin/dotfiles/exclude
# Create Date: 20110307-23:36HRS
# ln -s admin/dotfiles/exclude ~/.exclude

.cache
.adobe
.aptitude
.compiz # no settings here; processes
.dbus
.dropbox # don't try to restore and reinitialize either of
.dropbox-dist # these DIRs; very messy results can happen
.elegant-gnome-backup
.fontconfig
.gnome2_private # MT; at least on my system
.pulse*
.gconfd
.gvfs
.macromedia
.local/share/Trash/
.thumbnails
Downloads/
Dropbox/ # recreated on re-initialization
Public/
Music/
Templates/ # stored on U1
Ubuntu One/ # recreated on re-initialization
wdir/iso/
wdir/backups/
Improve this answer
1
  • If I use duplicity through DejaDup, is there such a file, too? (as of Ubuntu 16.04, there's no ~/.exclude for a DejaDup user like me)
    – Frank N
    Nov 16, 2016 at 14:10
5

I created an ignorelist at https://github.com/rubo77/rsync-homedir-excludes

You can use it with Duplicity or rsync like:

rsync -aP --exclude-from=rsync-homedir-excludes.txt /home/$USER/ /media/$USER/linuxbackup/home/$USER/

this is regularely updated, at the moment we have this so far:

################################################
# These directories may definitely be excluded #
################################################

# Contains mounted file systems:

.gvfs
.local/share/gvfs-metadata
# contains the actual encrypted home directory
/.Private

# Session-specific:
/.dbus
/.cache
/.Trash
.Trash-1*
.local/share/Trash
/Trash

# Cached info about audio CDs:
.cddb

# Cached packages lists:
.aptitude

# Node Package Manager cache
.npm

#Flash-specific:

# Cache for flash, maybe others?
.adobe
# except for Flash persistence, there is no reason to keep this
.macromedia

#Files:

# Contains errors from the current graphical session
.xsession-errors
.wayland-errors

# Recently used files
.local/share/RecentDocuments
.recently-used
.recently-used.xbel
.thumbnails
.thumb
Thumbs.db
.DS_Store
.localised
.bash_history
.CFUserTextEncoding

# Common Unix Printing System
/.cups

# run-time configuration information for Subversion
/.subversion

# Python virtual environments
/.virtualenvs

# Session-specific
.Xauthority
.ICEauthority
.gksu.lock
.pulse
.pulse-cookie
.esd_auth

#KDE specific:

# Recent documents on KDE
.kde/share/apps/RecentDocuments
.kde4/share/apps/RecentDocuments
# Contains a history of the Klipper clipboard (KDE)
.kde/share/apps/klipper
.kde4/share/apps/klipper
# You will loose saved scrolling positions of PDFs
.kde/share/apps/okular/docdata
.kde/share/apps/gwenview/recentfolders
.kde4/share/apps/okular/docdata
.kde4/share/apps/gwenview/recentfolders
# Cached other users' profile pics
.kde/share/apps/kmess/displaypics
.kde4/share/apps/kmess/displaypics
# Cached emoticons of others
.kde/share/apps/kmess/customemoticons
.kde4/share/apps/kmess/customemoticons

#Firefox-specific (see also Profile folder):

.mozilla/firefox/*/Cache
# in case Fx crashes dumps will be stored in this
.mozilla/firefox/*/minidumps
# session-specific 
.mozilla/firefox/*/.parentlock
# phishing database, recreated
.mozilla/firefox/*/urlclassifier3.sqlite
# blacklisted extensions
.mozilla/firefox/*/blocklist.xml
# extension database, recreated on startup
.mozilla/firefox/*/extensions.sqlite
.mozilla/firefox/*/extensions.sqlite-journal
.mozilla/firefox/*/extensions.rdf
.mozilla/firefox/*/extensions.ini
.mozilla/firefox/*/extensions.cache
# cached UI data, recreated
.mozilla/firefox/*/XUL.mfasl
.mozilla/firefox/*/XPC.mfasl
.mozilla/firefox/*/xpti.dat
.mozilla/firefox/*/compreg.dat
# cached plugin data, recreated
.mozilla/firefox/*/pluginreg.dat

#SeaMonkey-specific (see also Profile folder):

.mozilla/seamonkey/*/Cache
# in case Fx crashes dumps will be stored in this
.mozilla/seamonkey/*/minidumps
# session-specific 
.mozilla/seamonkey/*/.parentlock
# blacklisted extensions
.mozilla/seamonkey/*/blocklist.xml
# extension database, recreated on startup
.mozilla/seamonkey/*/extensions.sqlite
.mozilla/seamonkey/*/extensions.rdf
.mozilla/seamonkey/*/extensions.ini
# cached UI data, recreated
.mozilla/seamonkey/*/xpti.dat
.mozilla/seamonkey/*/compreg.dat
# cached plugin data, recreated
.mozilla/seamonkey/*/pluginreg.dat

# Thunderbird Cache in your profile folder
.thunderbird/*/Cache

#Opera-specific (related question on Superuser.com: Is documentation available on files and directories in the Opera profile folder?):

.opera/temporary_downloads
.opera/cache
.opera/thumbnails
.opera/opcache
.opera/icons
.opera/application_cache
.opera/widgets/*/cache
.opera/lock

# Keychain dir (manage SSH and GPG keys, front-end to ssh-agent and ssh-add)
/.keychain

#Komodo Edit:

.komodoedit/*/codeintel/db
.komodoedit/*/host-*/*/codeintel
.komodoedit/*/XRE/Cache
.komodoedit/*/XRE/.activatestate/komodo edit/Crash Reports
.komodoedit/*/XRE/.activatestate/komodo edit/*/Cache
.komodoedit/*/XRE/.activatestate/komodo edit/*/minidump
.komodoedit/*/XRE/.parentlock
.komodoedit/*/XRE/extensions.rdf
.komodoedit/*/XRE/extensions.ini
.komodoedit/*/XRE/extensions.cache
.komodoedit/*/XRE/XPC.mfasl
.komodoedit/*/XRE/XUL.mfasl
.komodoedit/*/XRE/xpti.dat
.komodoedit/*/XRE/pluginreg.dat
.komodoedit/*/XRE/compreg.dat
.komodoedit/*/XRE/*.sqlite-journal
.komodoedit/*/pystdout.log
.komodoedit/*/pystderr.log
.komodoedit/*/history.sqlite.bak
.komodoedit/*/running.lock
.komodoedit/*/mutex.lock
.komodoedit/*/*.xmlc
.komodoedit/*/startup-env.tmp
.komoeoedit/*/commandments.fifo
.komoeoedit/*/history.sqlite

#GnuPG:

.gnupg/rnd
.gnupg/random_seed
.gnupg/.#*
.gnupg/*.lock
.gnupg/gpg-agent-info-*

#Google Chrome:

.config/google-chrome/Default/Local Storage
.config/google-chrome/Default/Session Storage
.config/google-chrome/Default/Application Cache
.config/google-chrome/Default/History Index *

#Chromium:

.config/chromium/Default/Local Storage
.config/chromium/Default/Session Storage
.config/chromium/Default/Application Cache
.config/chromium/Default/History Index *

#Local repositories (added by errantlinguist on 2015-04-13):
.gradle/caches
/.m2/repository

#indexer
.local/share/baloo
.local/share/zeitgeist
.local/share/akonadi

#Other apps:

# Pidgin
.pulse/icons
# Cached applets
.guayadeque/cache.db
.java/deployment/cache
.icedteaplugin
.icedtea
.gnome2/epiphany/favicon_cache

# other temporary files 
nohup.out

###############################################
# directories, probably not worth a backup    #
# (uncomment the files you don't need)        #
###############################################

#.android
#.AndroidStudio*/
#Android/Sdk
#.gradle
#.gvm
#.grails/
#.grails_history
#.kde/share/apps/nepomuk
#.local/share/notbit
#.local/libvirt
#.vagrant
#.vagrant.d
#.wine
#.twister
#/twister
#Applications/eclipse*
#/Downloads
#*Popcorntime*
#/Scribus
#/VirtualBox*
#
## Add Game Folders here:
#
#/games
#.local/share/Steam
#.steam
#Steam
#.minecraft
#.PlayOnLinux
Improve this answer
-1

I've recently been looking at doing this sort of thing, and I've noticed that many of the files in my home directory that I don't want to back up seem to not have group permissions set. If the same holds on other systems, then a likely list of suspects might be obtained with:

find ~ -not -perm -g+r

(Although check the output, because although mine seemed mostly good, my e-mail inbox also appeared on the list, and I don't know how well it will work for others.)

This is particularly appealing to me, because my home directory is encrypted (including filenames), so a manual exclusions list would be a PITA, and instead I should be able to use group permissions to auto-generate an exclusion list.

(I know the thread is old, but I thought this might be useful to someone.)

Improve this answer
2
  • 3
    The permission of a file is a bad guide to determine whether a file is needed or not. Private SSH keys for example should have restrictive permissions, but surely you want to backup them.
    – Lekensteyn
    Aug 17, 2014 at 9:49
  • Permissions are not designed to be used as backup flags. You may have a lot of very good reasons to set custom permissions on files and directories, and your approach could break something. If you really don't like exclusion lists (why not?), instead use the dump/nodump flag of ext3 and ect4 file system to mark files for backup/no backup. This flag has been designed exactly for this purpose. See cyberciti.biz/faq/…
    – gerlos
    May 4, 2015 at 13:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .