Tom's Guide | Tom's Hardware | Tom's Games
![]() |
![]() |
![]() |
OK, no longer an issue, posting it in here in case someone in the future comes across this, it might help them. The search on this page has been a lifesaver in the past for me :)
This should also work if you delete the e0xx.log file in MDBDATA directory of exchange, and cannot start Exchange.
*************************
OK, so Microsoft Exchange 2k creates log file that records every transaction. Saves it in the MDBDATA directory, generaly as E0000##.log. When it gets to 5,120k, it stops, and creates another file sequentially numbered. When the backups run at night, Veritas deletes the files so they don't fill up the hard drive. My server usually generates about 25 of these files an hour during the busy parts of the day. (Figure it should be lower, but it must be spam coming in, I've checked and we're not an open relay)It's a normal routine, something that doesn't need daily monitoring or anything.
On Tuesday I'm doing some poking around the server, we use one of the drives for saving setup files and a few other things that aren't used too often, and I like to go in and probe around and see if there are any obsolete versions of setup files that I can delete, and I notice the server only has 400MB of free space on the D drive (where exchange is installed) and on the M drive (Exchange's virtual drive)
This is alarming, since there is normally about 8 gigs free, so I check the information store, it's at 11 gigs. I check to see who the largest mailboxes are (marketing department) and I ask them to clear out some old e-mails, especially back-and-forth conversations that carried imbedded images each time it was replied to, stuff like that. As she does it I'm watching the free disk space drop and drop.
I investigate, and the E00000X.log file is at 7 gigs. And climbing. For some reason, it stopped breaking itself into new files after 5 megs. And because it was still in use, Veritas couldn't delete it. Strangely, nothing in the Backup logs indicating the file was skipped for being in use.
s---. This can't be good.
Wednesday I do some research, and run an extra database backup, but it was too late, on Thursday just after lunch the server ran out of hard drive space and the stores dismounted. I dropped the Swap File size dedicated to the D drive from 1024mb to 512 mb, this would free up 500 MB to get everything back up so people can go about their business while I search for a solution.
This requires a reboot, so I warned everyone that the IM system would be down for a few minutes, and no one was bothered, since E-mail was already down. I reboot, and Exchange doesn't come back up. I go to My Computer, and sure enough, there is 500+ MB free. But my heart sank as I saw there was no M drive. I go to services, restart the Information Store. No joy, it fails. I look in event viewer, it says that log file E00000xx2.log is not found. I look, the super large file was e000000xx1.log. It's looking for a file that was never created.
I went through all 4 weeks of daily backups that I have, the file was already corrupted. I went back to the Feb 1st backup, the file was already corrupt. (I have daily backups for 4 weeks, monthly for 1 year)
The best backup I have is January 1st. That's a s---load of e-mails to be lost. I did some more research, and everything pointed to 'restore from backup'. f---. We had a new CEO start in February. All the stuff the old CEO left for him, all the stuff he brought from his old place, we had auditors here 2 weeks ago, all the corrospondence, all the s--- that we desperately need.
I am desperate to avoid having to restore. I can't delete that corrupt log file, since Exchange cannot load without it. Delete that file, and might as well delete the database, since that file tells the information store what mailbox is what datafile, ect. If the information store is corrupted, goodnight gracie. I have Exchange backed up nightly, and each morning I restore a random e-mail to a random user to verify that it worked, but that's only the mesages, there's no way of verifying if the information store is backup up correctly.
If the information store backup is corrupted in any way, then having all those e-mails in the other backup is useless, since I'd have to create a NEW information store, and the e-mails would be looking to restore to old, obsolete unique ID's that were in the old database.
I'm not panicking, but there's a s---storm of worse-case scenarios running through my brain.
I learn about ESEUTIL.exe, a program in the BIN directory. I copy the information store to a different server, since there was no breathing room on the Exchange server's hard drive, and this prgram needs room to run.
I run eseutil/r which restores the database from the logfile. But it can't find e000000xxx2.log, only xx1.log. I try to restore from that, it laughs at me.
f---.
I try eseutil/p which repairs the database. It warns me that it will probably corrupt the database if there are no accurate .log files. Which is what I'm missing, but what the f---.
Each one of these steps take about 2-3 hours, since the IS is 11 gigs.
By 3am friday I'd had enough, call it a night.
Friday people here are cool, I warned them Monday at earliest. I can't do much since running this program drags down the server it's running on, which is heavily used, so I work on some other issues. Friday night I run eseutil /d, which defrags the Information Store.
I copy to another server and delete all the files in MDBDATA, including the unrepaired and undefragged information store, and copy the repaired version in. Reboot.
M drive mounts.
Everything is back to how it should be.
Thank f---ing god for eseutil.
I run the same process on the public information store, and it's happy. Finally, at about 11:30 saturday, after many hours of lost sleep to be here, and many more lost thinking about "what if's", the e-mail is restored.
However, there are warnings that doing the /p to repair without a valid log file (which again, I didn't have) could corrupt the database and I would need to create a new one and move the mailboxes to the new one.
So that was Sunday. I created a whole new information store, and used the mailbox/move utility to move everything to a new mailbox. I restart the information store, and I get flooded with all the spam and messages I haven't received since Thursday. Life is good. I send a mail out to everyone to let me know of any issues, and go home. At about 6 Sunday night I dial in, reply to a few e-mails about customer's monthyl statements that I uploaded to the printers Saturday while I was here, and logged off and had some sleep for the first time. Still not a good sleep, still worried that it LOOKED good, but problems would emerge once it was put through the rigors of daily use.
Sure enough, come in yesterday, get several IM's with people who were THRILLED to see an online connection to exchange, and able to get to their e-mails. I'm feeling pretty good, go into my mailbox, and see no spam in the spam folder since Sunday. ALl my messages sitting in Outbox.
s---.
Check it out, SMTP service is not responding. Restart it, it fails to shut down. Oh f---.
Restart server, smooth sailing, everything's gold since.
Man I dodged a bullet. I spent all day documenting what happened, and I still don't know how that file corrupted like that. I DID find, however, a setting in veritas that was unchecked that told it to log skipped/in use files. I don't remember turning that off, or if the intern I had over Xmas break might have, since I had him verifying backups as part of his daily task so I could read the paper in the mornings, but whatever.
Thought my head was going to roll this week, or that they would come in monday to find me hanging from the rafters.
Thank god for the eseutil.exe program. I had never heard of it before, so if you haven't yet, keep it in the back of your mind somewhere, just in case. Figure I'd share my story in case it happens to one of you tools, and to make you aware of the program. It's a lifesaver!

![]() |
![]() |
![]() |

This post is quite old and has been locked from receiving new replies. Please create a new posting instead.
| Ads by Google |