0

I read quite a lot of posts, wiki pages and web pages about backup tools including Bacula and backupPc. For BackupPC I read the official documentation. I am looking for a automatic backup solution as replacement for Clonezilla backups.

It strikes me as weird that I did not come across any references to system consistency in the tutorials. Wouldn't that be one of the first things coming up when when thinking about automated server backup? What about MySQL databases, what about large content (email, blog, shops, git) that might change while being backed up. I would except the tools provide features to automate DB dumps, to start and stop services when their files are about to be backed up. As services distribute their files all over the place I would expect that to be a non-trivial and very time consuming planning. Further some services like NTP, LDAP, Samba might be involved in user and services authentication and can be shut down only when a backup server takes over (which should not be backed up at the same time...). Then to include user warnings that something is going to be offline in 5-4-3-2-1-0 minutes, etc. I think you get the point.

So now I have the feeling that I am overlooking some really trivial solution here. I would be very glad if someone could release me of that mental lock and point me to some place where I could follow a start-to-end tutorial covering to backup of a server including services.

EDIT:
The comments show me that my question could be misunderstood. Please let me rephrase the question to:
"With what I said above, how did you solve those issues for your environment?"
I am happy for any information about stuff that actually worked in the field.

thanks for reading.

CatMan

  • You have a good set of questions, but your question is too broad and there are multiple solutions from scripts to graphical tools. Whole pages are devoted to general backkups - help.ubuntu.com/community/BackupYourSystem , help.ubuntu.com/lts/serverguide/backup-shellscripts.html , digitalocean.com/community/tutorials/… . Everyone's server is different , tolerance for down time , etc. With virtual machines it gets more interesting. – Panther Apr 3 '17 at 18:32
  • @bodhi.zazen I meant the question to be general and not broad. You are correct that there are many aspects to it, but most are for illustrating the point. One of your links I can not access, but the others do not contain any reference to that topic (did you read them?). A helpful answer could already be: "Yes, many tools offer specific support for those things, but every tool in an other way" or "No, none of the tools offer any support for that, you need to write your own scripts". You see, clear follow up on "yes.." would be pls show me one example. – CatMan Apr 3 '17 at 18:40
  • What servers are you running ? What is your tolerance for down time ? Virtual servers ? do you want a graphical tool, web tool, or script ? To where do you want to save the backup ? What is your plan to restore ? What do you want to back up, data only, entire system ? There is no way to answer your question. – Panther Apr 3 '17 at 19:04
  • @bodhi-zazen I do appreciate your efforts, but your question show me that we are talking about very different things. I tried to modify my question to clear up that misunderstanding. – CatMan Apr 4 '17 at 7:46
  • @CatMan We need more details from you. Specific questions are better responded on askubuntu. – ankit7540 Apr 4 '17 at 7:52
0

If you use a virtual machine you copy the whole virtual machine and be done with it. We use custom scripts though for our google instances and do not rely on backup software.

Basically: gather all data to backup and place it into a directory. Let's call this /backup/. Within /backup/ we have directories named 01 to 31. In those directories we copy all files that need to be saved. So we have 1 month of backups on the machine itself. If space is limited you can also do this on number of the day (so 7 days worth of backups).

For mysql we have a script to stop the database and use mysqldump to generate 1 big file and store it in /backup/[01-31] on that specific number of the month. The configuration files are also copied over. Jasperserver (a tool to create templates mainly for PDFs) has an export script to export all the custom content into 1 big zip file. For apache we copy over the website software into a zip file and also copy the configuration files. We do not use mail on our systems (we use 2 remote systems called flowmailer and postmark to mail) but something similar can be done here too.

And that data is then copied to another machine that is not in the same building.

Our downtime needs to be close to 0. We have clients that have locations all over the world (from China to America). The MySQL dump takes about 90 minutes (in those 90 minutes all the other software is done). To solve this we use 2 google instances and the backup is done from the slave so the downtime is 0. All we need to do afterwards is update the slave.

  • That is exactly the kind of reply to learn from that I was looking for. Thanks a lot. Your approach to snapshot VMs is very interesting I did not think about setting up the server in a VM. Just to understand, I would install server to a machine, add KVM and install server again on a VM. The server inside VM gets all services. Why do you not just snapshot the VM and store that? I would think it includes disks and memory, so no need to dump databases via script. I guess space reasons. Did you have a specific reason to skip backup tools at all? – CatMan Apr 4 '17 at 9:09
  • Yeah my boss is of the type: if I can not see what is going on inside I will code it myself ;-) We are not using snapshots though; google instances is our thing. And we have a google instance, a 2nd one that gets replicated on the fly as a 1st fall back and an amazon instance that gets replicated on the fly as a 2nd fall back if google decides to fall over (happend 2 times the last 5 years ;-) ) "I would think it includes disks and memory, so no need to dump databases via script. " yes. – Rinzwind Apr 4 '17 at 9:22
  • ... If you can virtualize do that. A couple of our clients demanded in-house servers so those are virtual. Creating a backup is copying the VM. Creating a test server is copying the vm and connect it to a new IP. Easy :) – Rinzwind Apr 4 '17 at 9:22
  • Sorry space running out. I did not understand how you have 0 downtime when you need 90min to dump you Mysql. Even when a backup is taking over, how do you sync primary and backup when primary went back online for 90min? Your fully self made solution could seem a bit daunting for a beginner. For transfer it would be great if you could add a bit of information what disadvantages backup tools would have in your environment. – CatMan Apr 4 '17 at 9:22
  • Because we backup from the fall-back server. Not from the live system. All that happens when we do a backup is that the fall-back server is out of sync for 90 minutes. And mysql fixes that by itself when the slave is reconnected. "Your fully self made solution could seem a bit daunting for a beginner. " Nope! It is all done with baby steps. Make a list of services that you have that need backing up and start with 1 and set it up. We (you too?) already have a list of things we alter on a machine (config changes) so those are includes in the backups. – Rinzwind Apr 4 '17 at 9:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.