cksum and FAT32

Custom / CUSTOM
March 25, 2010 at 09:29:38
Specs: linux red hat, 2.401 GHz / 2047 MB
I'm having a FAT32 disk attached to my Red Hat linux, and I'm checking the CRC, by means of tool CKSUM, on the files I'm putting on this mounted FAT32 volume.

The problem is that the data returned by CKSUM is not consistent, even when the files (on FAT32) are not being touched. Sometimes they are the same, sometimes they change!

What can be the problem here ?


See More: cksum and FAT32

Report •

#1
March 25, 2010 at 13:25:11
Try a file and copy it binary or make it contigious and try your test again. Only two ways I suspect would be fragmentation is causing this error. The other being actual file damage.


Playing to the angels
Les Paul (1915-2009)


Report •

#2
March 25, 2010 at 14:24:38
I'm further checking, but note there is no FTP involved here. The disk is FAT32, by I connect it either to one Linux, then to another Linux. I always use CP to get the files onto the disk, no FTP anywhere.

I can see the files in Windows, but I don't plan to do anything with them there, except backing up.

I was indeed thinking on fragmentation ... but how does that corrupt files ? There's one thing I didn't mention, but one Linux is actually a virtual Linux in VMware.


Report •

#3
March 25, 2010 at 14:29:43
Usually bad hardware like bad cpu, memory controller, memory, bad drive or even virus or some software with memory leak or other fault.

I guess I may not know the exact test you are using to state this problem. Relate the exact set of steps that happen.

Could it be that the checksum is simply relaying the filesystem issues such as permissions, attributes or even file structure that don't translate to the new system?


Playing to the angels
Les Paul (1915-2009)


Report •

Related Solutions

#4
March 26, 2010 at 04:18:01
It's possible. This is what I'm seeing right now (the USB drive in question is Windows formatted in FAT32) :

Creating a file on the local filesystem on Linux 1
Finding checksum of that file (using CKSUM)
Copying that file to an attached USB drive (using CP command)
Checking checksum : still OK
Switchig to Linux 2, attaching same USB drive there
Checking checksum of files using the CKSUM of that Linux, straight on the USB drive still
MOST files are OK, but SOME have another result from CKSUM
Filesizes are always exact
Permissions are sometimes very weird, and often they vary between the files, for unclear reason


Report •

#5
March 26, 2010 at 13:59:00
OK try this just to see if you have time.

Check the file.

Tar and zip the file, transfer all you want in some compressed form. Then unzip and check files. Be sure to read the man page on how which compression to keep as many attributes.

I can't otherwise explain the errors. I don't think it is normal.

Compressed files almost all have a checksum feature so if the transfer is the problem it will fail on unzip.

Playing to the angels
Les Paul (1915-2009)


Report •

#6
March 26, 2010 at 14:48:53
The files are already compressed, and it is indeed on uncompressing, that they appear to be corrupt. Given the behaviour I noticed, it is expected that uncompression fails, but I indeed need to find why files are corrupt.

Next thing I'll be trying is to perform a FAT-format, but in Linux this time.

It's either that, or it is the USB-release-media issue (you know, when you unplug USB's in Windows, files can be corrupted if you don't "eject" properly). My USB is always auto-mounted, and I suspect the same occurs on Linux shutdown, but maybe I need to dismount manually ?!


Report •

#7
March 26, 2010 at 18:18:40
Try 7-zip and see if that fails too maybe??

Playing to the angels
Les Paul (1915-2009)


Report •

#8
March 27, 2010 at 09:28:44
No, the issue is that the checksum of a set of files is correct on one Linux, and when mounted into another one, some files are corrupt (most not). Using a different zipping tool is not going to change that.

I've formatted the USB drive with Linux FS, same issue. Things which remain are:
- hardware failure of the USB drive itself (but I don't see any errors when I use it on Windows, neither on Linux 1)
- a VMware issue, since he has got a layer which allows the files being transported into the Linux image. I'm gonna see if I can do the reverse test


Report •

#9
April 27, 2010 at 05:54:51
I have not found the exact cause, but the below behaviour has been noted various times, is consistently there, no way to avoid:

I create files on my linux FS, no problem
I copy these files to a USB disk attached to this Linux OS, no problem (FS of USB is FAT32)
I double-check the CRC of the files, all OK

Then, I switch to another Linux installation (which works OK), I connect the drive, I can read the files, but SOME of them (about 50%) have bad CRC.
The disk is OK, I checked and double-checked for bad sectors and such. The files are there in their full size, only the content is not correct.

I can connect the same USB to Windows, and copy them there, no problem (this indicates that FAT thinks the files are OK).

Solution I applied is to FTP the same files from same source OS to same target OS, via an intermediate Windows OS, and then it works just fine. (using same mechanism, split them up in 2Gig files, which is not necessary, but to show that my splitting system works OK).


Report •

Ask Question