I'm having problems sorting a 2Mb log file by date. I have combined two sorted logs into one file, with the one being added to the end of the other. I have tried all kinds of sort commands to arrange the combined file chronologically, and the result is still always one log followed by the other. I know my commands are entered correctly because I've taken sample entries from the two logs, combined them into a smaller file and run sort, and it does just fine. Is 2Mb really too much for the sort command to handle in one go?

Any ideas?

Please post as sample of the file(s) so we can give you a better answer.

Here is a sample line from one log:

216.161.16.41 - - [15/Oct/2002:14:39:12 -0500] "GET /images/specphoto/hitchphoto3.jpg HTTP/1.1" 200 22630 "http://www.domain.com/hitchphoto.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"

and from the secure log...

208.238.142.5 - - [14/Oct/2002:10:28:49 -0500] "GET /images/bkwhtc.jpg HTTP/1.1" 200 19436 "https://www.domain.com/cart/shopper.cgi?preadd=action&key=HR001" "Mozilla/4.0 (compatible; MSIE 5.12; Mac_PowerPC) OmniWeb/4.1-v422"

I have two days wort of entries (Oct 14 and 15) represented in both logs. A merge simply produces a file with the content of the first log sorted chronologically (Oct14-15) followed by the content of the second log starting with its earliest Oct 14 entry through to the end of the 15th.
No sorts on the entire merged file have yet been successful.

I am sure my commands are good because of the success I had in sorting a set of randomly selected entries. Could there be a bad entry somewhere that screws up the sort? This one has had me stumped for the past 3 days.

This would get a lot easier by specifying a custom format for the apache log however...

With a log file names f1.txt:

#!/bin/sh
cat f1.txt | \
awk '{ print $4 " " $5 " " $1 " " $2 " " $3 " " $6 " " $7 " " $8 " " $9 " " $10 " " %11 " " $12 " " $13 " " $14 " " $15 " " $16 " " $17 " " $18 " " $19}' | \
sed -e 's/\[//' -e 's/\]//' | sort -M

If you need to have the date back in position 4 and 5 pass it through awk again after the sort.

Oops. typo. that should be $11 not %11