List large no of files

November 30, 2010 at 01:59:14
Specs: Windows XP
I am getting 'no space' error when trying
below command
##############
for file in 'ls -rt'
do
something
################

This happens when the no of files in the directory exceeds 20000 files.
How to solve this issue


See More: List large no of files

Report •

#1
November 30, 2010 at 17:29:48
First, I am assuming this is a copy error:

for file in 'ls -rt'
do

you probably meant to use back tics:

for file in `ls -rt`
do

Second, an 'out of space' error normally means you are running out of space. Are you certain that the "something" executing in the for loop isn't causing you ro run out of space.

Third, you don't mention which version of Unix you are using, but it's possible to overflow the command line buffer. This article discusses it:

http://www.theillien.com/Sys_Admin_...

This find/xargs script might fix your problem:

find . -type f -print|xargs ls -rt1|while read file
do
   echo "$file"
done


Report •

#2
December 2, 2010 at 22:22:13
What is happening is that you are running out of stack space. When you use a command such as:

ls -rt *

each file is being parsed and passed to the command line as an argument. Every Unix/Linux shell has limitations regarding how much memory is allocated to the shell. In AIX for example, the limitation is imposed by the NCARGS variable. Once this stack space is used up, you will receive an error such as:

arg list too long

I believe that the out of space message is the result of running out of stack space.



Report •

#3
December 15, 2010 at 23:56:36
If you are working on HP-UX. Please try the following command:

find . \( -type d ! -name . -prune \) -o \( -type f -name "*" -print \)

The above find will not traverse into the sub directories which I guess you don't want to.


Report •

Related Solutions

#4
December 16, 2010 at 03:05:18
The original question dealt with directories that contained more than 20,000 files. Did not seem to be an issue with traversing into subdirectories, particularly since ls -rt will not traverse subdirectories.

Report •

#5
December 16, 2010 at 05:22:48
find command by default traverse to sub directories. Hence we should be careful with selecting the right option for avoiding it. This is why I provided a command with -prune option.

Report •

Ask Question