...making Linux just a little more fun!

### Backup Strategy

Amit Saha [amitsaha.in at gmail.com]

Sat, 6 Mar 2010 20:29:43 +0530

Hello all:

I have a simple requirement- I want to back up all my lab's data (~ 1TB and increasing) to a 2TB hard-disk on another machine. How best should I partition/create volumes on my 2 TB hard-disk so as to enable my backups? I have no special requirements. Just simple dumping of the contents from one to another. In the same vein, I would like to hear how you do something similar.

(I plan to use rsync for the same)

Best,

Amit

    --
    Journal: http://amitksaha.wordpress.com,
    µ-blog: http://twitter.com/amitsaha

    Freenode: cornucopic in #scheme, #lisp, #math,#linux, #python

Top Back

Mulyadi Santosa [mulyadi.santosa at gmail.com]

Sat, 6 Mar 2010 22:07:15 +0700

On Sat, Mar 6, 2010 at 9:59 PM, Amit Saha <amitsaha.in@gmail.com> wrote:

    > Hello all:
    >
    > I have a simple requirement- I want to back up all my lab's data (~
    > 1TB and increasing) to a 2TB hard-disk on another machine. How best
    > should I partition/create volumes on my 2 TB hard-disk so as to enable
    > my backups? I have no special requirements.

Sounds like rsync would do the job for you, something rsync -avPH to maintain all the permission and the ownership to the target disk.

And you said "increasing"? You mean, you plan to implement to create LVM? Then go ahead...it's a good choice IMO. IMO Double it with RAID 1 for data redundancy...

    --
    regards,

    Mulyadi Santosa
    Freelance Linux trainer and consultant

    blog: the-hydra.blogspot.com
    training: mulyaditraining.blogspot.com

Top Back

Thomas Adam [thomas at xteddy.org]

Sat, 6 Mar 2010 16:08:55 +0000

On Sat, Mar 06, 2010 at 10:07:15PM +0700, Mulyadi Santosa wrote:

    > On Sat, Mar 6, 2010 at 9:59 PM, Amit Saha <amitsaha.in@gmail.com> wrote:
    > > Hello all:
    > >
    > > I have a simple requirement- I want to back up all my lab's data (~
    > > 1TB and increasing) to a 2TB hard-disk on another machine. How best
    > > should I partition/create volumes on my 2 TB hard-disk so as to enable
    > > my backups? I have no special requirements.
    >
    > Sounds like rsync would do the job for you, something rsync -avPH to
    > maintain all the permission and the ownership to the target disk.

rdiff-backup. The tool of choice for this sort of thing.

-- Thomas Adam

    --
    "It was the cruelest game I've ever played and it's played inside my head."
    -- "Hush The Warmth", Gorky's Zygotic Mynci.

Top Back

Amit Saha [amitsaha.in at gmail.com]

Sat, 6 Mar 2010 21:40:55 +0530

On Sat, Mar 6, 2010 at 9:38 PM, Thomas Adam <thomas@xteddy.org> wrote:

    > On Sat, Mar 06, 2010 at 10:07:15PM +0700, Mulyadi Santosa wrote:
    >> On Sat, Mar 6, 2010 at 9:59 PM, Amit Saha <amitsaha.in@gmail.com> wrote:
    >> > Hello all:
    >> >
    >> > I have a simple requirement- I want to back up all my lab's data (~
    >> > 1TB and increasing) to a 2TB hard-disk on another machine. How best
    >> > should I partition/create volumes on my 2 TB hard-disk so as to enable
    >> > my backups? I have no special requirements.

Pls. direct your answers more towards the partitioning of the 2 TB disk

    >>
    >> Sounds like rsync would do the job for you, something rsync -avPH to
    >> maintain all the permission and the ownership to the target disk.
    >
    > rdiff-backup. The tool of choice for this sort of thing.

Thanks for the help, already!

Best, Amit

    --
    Journal: http://amitksaha.wordpress.com,
    µ-blog: http://twitter.com/amitsaha

    Freenode: cornucopic in #scheme, #lisp, #math,#linux, #python

Top Back

John Karns [johnkarns at gmail.com]

Sat, 6 Mar 2010 13:57:58 -0700

On Sat, Mar 6, 2010 at 9:10 AM, Amit Saha <amitsaha.in@gmail.com> wrote:

    > Pls. direct your answers more towards the partitioning of the 2 TB disk
    >
    > Thanks for the help, already!
    >
    > Best,
    > Amit

You should be a little more specific in wording your question. IOW, specify whether:

- you will be using the 2 TB drive as a secondary drive, (dedicated to backups),

or,

- you are asking for recommendations for a partitioning scheme which will serve the dual purpose of using the 2TB drive as both a system drive, and a backup drive?

    --
    John

Top Back

Amit Saha [amitsaha.in at gmail.com]

Sun, 7 Mar 2010 02:53:20 +0530

On Sun, Mar 7, 2010 at 2:27 AM, John Karns <johnkarns@gmail.com> wrote:

    > On Sat, Mar 6, 2010 at 9:10 AM, Amit Saha <amitsaha.in@gmail.com> wrote:
    >> Pls. direct your answers more towards the partitioning of the 2 TB disk
    >>
    >> Thanks for the help, already!
    >>
    >> Best,
    >> Amit
    >
    > You should be a little more specific in wording your question. IOW,
    > specify whether:
    >
    > - you will be using the 2 TB drive as a secondary drive, (dedicated to
    > backups),

I am going to use it as a dedicated backup drive. Thanks in advance!

Best, Amit

    --
    Journal: http://amitksaha.wordpress.com,
    µ-blog: http://twitter.com/amitsaha

    Freenode: cornucopic in #scheme, #lisp, #math,#linux, #python

Top Back

Karl-Heinz Herrmann [kh1 at khherrmann.de]

Fri, 19 Mar 2010 22:09:24 +0100

On Sun, 7 Mar 2010 02:53:20 +0530 Amit Saha <amitsaha.in@gmail.com> wrote:

    > I am going to use it as a dedicated backup drive. Thanks in advance!

Well then use one 2TB partition. You might need cfdisk or similar instead of the regular fdisk which cant handle filesystems beyond a certain size.

If you would like to keep the option of further enlarging the backup storage beyond the 2TB and still see it as one huge partition use lvm, use the 2TB drive as the first physical member of that LVM volume and create your filesystem on that virtual disk.

Once the 2TB are used up buy a 4TB drive and add it as a second physical drive to the volume, enlarge LVM and filesystem and go on.

With that amount of data you do want protection against single disk failure. So if the primary disk is not already a raid system you should make the backup a raid 1 or 5.

If your amount of data ist typically growing by adding new files you might want to have a look into backuppc which can give you daily views of the data (if they change) while storing identical files only once aand hardlinking them.

K.-H.

Top Back

René Pfeiffer [lynx at luchs.at]

Fri, 19 Mar 2010 22:15:41 +0100

On Mar 19, 2010 at 2209 +0100, Karl-Heinz Herrmann appeared and said:

    > On Sun, 7 Mar 2010 02:53:20 +0530
    > Amit Saha <amitsaha.in@gmail.com> wrote:
    >
    > > I am going to use it as a dedicated backup drive. Thanks in advance!
    >
    > [...]
    > With that amount of data you do want protection against single disk
    > failure. So if the primary disk is not already a raid system you should
    > make the backup a raid 1 or 5.

Don't use RAID5 with disks containing a large block number. The probability for a double-disk failure gets higher with the amount of blocks.

http://www.tomshardware.com/news/RAID-5-Doomed-2009,6525.html http://en.wikipedia.org/wiki/Standard_RAID_levels#RAID_5_disk_failure_rate

Consider RAID6 or using RAID1 block devices as building blocks for logical volumes.

Best,

René,

jumping to mention this, no harm intended.

Top Back

Jim Jackson [jj at franjam.org.uk]

Sat, 20 Mar 2010 22:07:15 +0000 (GMT)

On Fri, 19 Mar 2010, Ren? Pfeiffer wrote:

    > On Mar 19, 2010 at 2209 +0100, Karl-Heinz Herrmann appeared and said:
    >> On Sun, 7 Mar 2010 02:53:20 +0530
    >> Amit Saha <amitsaha.in@gmail.com> wrote:
    >>
    >>> I am going to use it as a dedicated backup drive. Thanks in advance!
    >>
    >> [...]
    >> With that amount of data you do want protection against single disk
    >> failure. So if the primary disk is not already a raid system you should
    >> make the backup a raid 1 or 5.
    >
    > Don't use RAID5 with disks containing a large block number. The
    > probability for a double-disk failure gets higher with the amount of
    > blocks.
    >
    > http://www.tomshardware.com/news/RAID-5-Doomed-2009,6525.html

I think you will find, if you read all the comments on this article, that many people, who appear reasonably knowledgable, think this is bullshit^H^H^H^H^H^H^H^Hfanciful.

Given the article was written in 2008 and disks have indeed got a lot bigger, and are used in large raid5 arrays, I think we'd be hearing a lot of squealing in the industry right now!

Top Back

René Pfeiffer [lynx at luchs.at]

Sun, 21 Mar 2010 12:47:23 +0100

On Mar 20, 2010 at 2207 +0000, Jim Jackson appeared and said:

    > On Fri, 19 Mar 2010, Ren? Pfeiffer wrote:
    > > [...]
    > >Don't use RAID5 with disks containing a large block number. The
    > >probability for a double-disk failure gets higher with the amount of
    > >blocks.
    > >
    > >http://www.tomshardware.com/news/RAID-5-Doomed-2009,6525.html
    >
    > I think you will find, if you read all the comments on this article,
    > that many people, who appear reasonably knowledgable, think this is
    > bullshit^H^H^H^H^H^H^H^Hfanciful.
    >
    > Given the article was written in 2008 and disks have indeed got a lot
    > bigger, and are used in large raid5 arrays, I think we'd be hearing a
    > lot of squealing in the industry right now!

The source might not have been the best one, but I cooperate with a data recovery company whose "war stories" tell a different tale. Of course the URE event is rare, but it does happen.

Best, René.

Top Back

Anderson Silva [afsilva at gmail.com]

Sat, 6 Mar 2010 12:54:22 -0500

On Sat, Mar 6, 2010 at 9:59 AM, Amit Saha <amitsaha.in@gmail.com> wrote:

    > Hello all:
    >
    > I have a simple requirement- I want to back up all my lab's data (~
    > 1TB and increasing) to a 2TB hard-disk on another machine. How best
    > should I partition/create volumes on my 2 TB hard-disk so as to enable
    > my backups? I have no special requirements. Just simple dumping of the
    > contents from one to another. In the same vein, I would like to hear
    > how you do something similar.
    >
    > (I plan to use rsync for the same)

If you want to try rsync with a GUI, try backintime, it's pretty cool.

AS

    --
    http://www.the-silvas.com

Top Back