|Why are you removing the keyboard when the system is running?|
Or are you running the computer with no keyboard attached? If so, check the 'halt on' options in cmos/bios setup. You can also lock out the keyboard with the key switch that's usually on the front panel of those older PCs. Or open the case and jumper the 'keyboard lock' pins on the motherboard.
Ding dong the witch is dead. . . .