Computing.Net > Forums > Linux > TAR file size over 2GB limit

Computer Problems? Computing.Net has over 1,000,000 posts about all things technology related! Over 90% answered within 24 hours! Click here to start participating now! Also, be sure to check out the New User Guide.

TAR file size over 2GB limit

Reply to Message Icon

Name: Danny Larouche
Date: December 10, 2002 at 07:19:31 Pacific
OS: LINUX rh8
CPU/Ram: 256
Comment:

With tar, it is possible to create a file over the FS limit of 2147483647bytes. However, most tool can't deal with such file. We can split such file using the "split" tool, however its impossible to restore the splited file components together.

The -M and -L option in tar are for stream tape, not for file. Does anyone got this problem and found a way to get rid of this stupid fs limit.

I need TAR to automaticaly split the archive when 2GB size is reached and continue on the next file.tar.1 file.tar.2 ...

I can't believe i'm the first that complaint about that!!!



Sponsored Link
Ads by Google

Response Number 1
Name: Dlonra
Date: December 10, 2002 at 07:33:58 Pacific
Reply:

If you have root permission, man ulimit, man bash (contains ulimit)

Are you using compression?
tar -j or
tar -z


0

Response Number 2
Name: Danny Larouche
Date: December 10, 2002 at 07:37:28 Pacific
Reply:

i am using the -z

This is not a OS limit, but File system limit, i don't think ulimit will help.


0

Response Number 3
Name: 3Dave
Date: December 10, 2002 at 08:04:05 Pacific
Reply:

I had the same problem trying to backup a system to a file. Ended up having to use ghost instead!=o(

ulimit looks as though it might help, if only I had known about that before


0

Response Number 4
Name: Dlonra
Date: December 10, 2002 at 08:21:20 Pacific
Reply:

On my Mandrake 8.2, ulimit -a shows
core file size (blocks) 1000000
data seg size (kbytes) unlimited
file size (blocks) unlimited
max locked memory (kbytes) unlimited
max memory size (kbytes) unlimited
open files 1024
pipe size (512 bytes) 8
stack size (kbytes) 8192
cpu time (seconds) unlimited
max user processes 4095
virtual memory (kbytes) unlimited

I had an empty 3.8G partition so I cat'ed a file
-rw-r--r-- 1 root root 3937685504 Dec 10 11:10 xx
Size Used AVAIL
/dev/hdc10 on /mnt/hdc10 RW EXT3 3.7G 3.8G 0 100%



0

Response Number 5
Name: Danny Larouche
Date: December 10, 2002 at 08:22:06 Pacific
Reply:

I never got problem TARing my system since the past 5 years even with tar.gz file over 10GB. However i now need to crypt them with ccrypt.

Since the file is over the FS limit, all tools and server such as binsplit, ccrypt, cat, apache will stop when the 2147483747 bytes limit is reached.

I am able to split with "split" the encrypt them, but unable to restore them toghether in the tar ball.

I will take a look to ulimit today, but i doubt it can't change anything to the problem.


0

Related Posts

See More



Response Number 6
Name: Danny Larouche
Date: December 10, 2002 at 08:26:07 Pacific
Reply:

Hi Dlonra,

I am curently use this file on a RH6 ext2, but will verify on my RH8 ext3 file system this afternoon


0

Response Number 7
Name: Jake
Date: December 10, 2002 at 10:35:41 Pacific
Reply:

I have no trouble with large files in ReiserFS. I have 2 tarballs over 8Gb, and one almost 5Gb that's gzipped.

Would it be possible to upgrade to ReiserFS? It would require a new kernel and some data shifting, which may just be more trouble than it's worth. Just upgrading to Ext3 will not fix your problem.


0

Response Number 8
Name: Danny Larouche
Date: December 10, 2002 at 11:32:30 Pacific
Reply:

Jake:
Even on a ext2, tar can create such large file, but could you please try to "cat" the file to a test file.

If it work on a riserfs, i will consider this as an interesting solutions.



0

Response Number 9
Name: Jake
Date: December 10, 2002 at 14:05:17 Pacific
Reply:

The gzipped file wasn't created with tar z; it was gzipped later, so I would assume other system tools also work. I tried cat, redirecting the output to /dev/null, and there were no errors, so I'm assuming it worked.


0

Response Number 10
Name: Danny Larouche
Date: December 10, 2002 at 15:12:11 Pacific
Reply:

Jake:

cat will not return error message, but silently assume the job is done when reaching 2147MB. Should cat to a file instead of /dev/null in order to test this fs limit.



0

Response Number 11
Name: Jake
Date: December 10, 2002 at 20:49:29 Pacific
Reply:

Ok, this time I dumped the output into another file. When cat was done, ls -l reported exactly the same size for both files, and both files had the same MD5 hash.


0

Sponsored Link
Ads by Google
Reply to Message Icon






Post Locked

This post is quite old and has been locked from receiving new replies. Please create a new posting instead.


Go to Linux Forum Home


Sponsored links

Ads by Google


Results for: TAR file size over 2GB limit

how to break a 2GB tar file? www.computing.net/answers/linux/how-to-break-a-2gb-tar-file/2042.html

file-size limit in 'ext3' www.computing.net/answers/linux/filesize-limit-in-ext3/23344.html

Maximum files per directory? www.computing.net/answers/linux/maximum-files-per-directory/5879.html