...making Linux just a little more fun!

### USB Drive Bad Sectors

Fri, 27 Oct 2006 17:04:01 -0400

Greetings,

I have Linux (2.6.15.6) booting from a USB drive. My application is machine control and I wanted to use something solid-state. A USB stick seemed like a good idea because it is cheap and readily available and reasonably fast.

My problem is that I get file system corruption and "bad sectors" some times, presumably due to an unclean shut down. I am using an ext2 filesystem with the thinking that a journal would just wear out the drive in a certain spot. Maybe that's not correct -- I am not sure if the drive has a wear-leveling algorithm implemented or not.

Questions:

1. Is ext2 the right choice? Or would reiserfs be better? Vfat??

2. Is their a quick way to detect bad sectors? I tried running e2fsck -c but it has been 45 minutes so far!

I am thinking if I can detect the corruption, I can just reformat the partition and restore it from another read-only partition. The kernel seems to report them:

    sd 5:0:0:0: SCSI error: return code = 0x08000002
    sdb: Current: sense key=0x3
        ASC=0x11 ASCQ=0x0
    end_request: I/O error, dev sdb, sector 598730

I guess I could grep the messages log, but I don't think that is a conclusive check.

Any pointers would be appreciated.

    --
    Brandon Reynolds                Ph: 330-644-3059
    Systems Engineer                Fax: 330-644-8110
    CTI, a Poling Group Company     Email: bmr at comtime.com

Top Back

Rick Moen [rick at linuxmafia.com]
Tue, 31 Oct 2006 10:24:33 -0800

Quoting Brandon M. Reynolds (breynolds@comtime.com):

    > I have Linux (2.6.15.6) booting from a USB drive.  My application is
    > machine control and I wanted to use something solid-state.  A USB stick
    > seemed like a good idea because it is cheap and readily available and
    > reasonably fast.
    >
    > My problem is that I get file system corruption and "bad sectors" some
    > times, presumably due to an unclean shut down.  I am using an ext2
    > filesystem with the thinking that a journal would just wear out the
    > drive in a certain spot.  Maybe that's not correct -- I am not sure if
    > the drive has a wear-leveling algorithm implemented or not.
    >
    > Questions:
    >
    > 1. Is ext2 the right choice?  Or would reiserfs be better? Vfat??

Flash-memory drives have some peculiar physical characteristics, and choice of filesystem for them depends on whether you value multi-OS compatibility (which suggests VFAT) or whether you value drive life and even-ness of wear (which suggests JFFS2 or YAFFS). Essentially, they have a limited number of erase/write cycles per sector, and each sector must be read, erased, and rewritten any time any data in the sector changes. The reasons why VFAT is bad for drive life, not to mention atime updating, follow logically from that, and should be obvious.

I imagine that the wear resulting from ext2/ext3 or ReiserFS would be absolutely excessive.

    > 2. Is their a quick way to detect bad sectors?  I tried running e2fsck
    > -c but it has been 45 minutes so far!

I imagine that the wear resulting from e2fsck would be absolutely frightful, sorry to say.

More at: http://www.linuxjournal.com/article/6867 http://en.wikipedia.org/wiki/Flash_memory

    --
    Cheers,             The genius of you Americans is that you never make
    Rick Moen           clear-cut stupid moves, only complicated stupid moves
    rick@linuxmafia.com that make us wonder at the possibility that there may be
                        something to them that we are missing. --Gamel Abdel Nasser

Top Back

Brandon M. Reynolds [breynolds at comtime.com]
Tue, 31 Oct 2006 15:31:53 -0500

The original e-mail was top-posted, and contained all of Rick's previous answer. PLEASE do not top-post, and please do snip quoted material down to relevant text! - Kat

Rick,

Thanks a lot for the help! I will look into those other filesystems. The other thing I forgot to mention is that if you write over that sector again it seems to work. So I don't think we are hitting the 10000 writes, but maybe not. I wonder if it is possible that the drive is marking a sector bad because it was in the middle of zeroing the sector or something?

I do have the partitions mounted with noatime. I think part of my problem maybe that I am using sqlite which seems to want to sync everything to disk all the time. What I will probably end up with is using a RAM disk and then syncing that to USB drive on a shutdown.

Regards,

    --
    Brandon Reynolds                Ph: 330-644-3059
    Systems Engineer                Fax: 330-644-8110
    CTI, a Poling Group Company     Email: bmr@comtime.com

Top Back

Rick Moen [rick at linuxmafia.com]
Tue, 31 Oct 2006 14:08:51 -0800

Quoting Brandon M. Reynolds (breynolds@comtime.com):

    > Thanks a lot for the help!  I will look into those other filesystems.
    > The other thing I forgot to mention is that if you write over that
    > sector again it seems to work.  So I don't think we are hitting the
    > 10000 writes, but maybe not.  I wonder if it is possible that the
    > drive is marking a sector bad because it was in the middle of zeroing
    > the sector or something?

I'm thinking it smells a bit like borderline filesystem corruption, e.g., a given sector has existing coherency problems, which get cured in the act of going through an erase/rewrite cycle. That might be just a SWAG (silly wild-assed guess) on my part, of course.

    > I do have the partitions mounted with noatime.  I think part of my
    > problem maybe that I am using sqlite which seems to want to sync
    > everything to disk all the time.  What I will probably end up with is
    > using a RAM disk and then syncing that to USB drive on a shutdown.

I do think that sounds like a winner. It's so tempting to just treat these NAND flash-memory devices as if they were just tiny, electonic hard disks, but that's really not quite the right model, as it turns out.

    --
    Cheers,      "Transported to a surreal landscape, a young girl kills the first
    Rick Moen     woman she meets, and then teams up with three complete strangers
    rick@linuxmafia.com       to kill again."  -- Rick Polito's That TV Guy column,
                  describing the movie _The Wizard of Oz_

Top Back