I recently realized I had exposed my server's ssh keys. This was a beginner's blunder.
So the question then becomes how do I securely:
1) change delete the old ssh key (so whoever has access will no longer be able to get in)
2) add a fresh new ssh key
To do this I had to first generate a new ssh key on my client, and make sure my ~/.ssh/config file was appropriately pointing to the new key files.
On the server side, just assume someone is logged in. Type:
users | wc -w
If the result is greater than 1, someone else is logged in. Even if nobody else is logged in, just assume that at any moment someone may attempt to log in.
To kill all other connections, I rebooted the server. [Maybe there's a better way?]
After reboot, log back in.
Now edit the /etc/passwd file so that the current user you are logged in as cannot log in again.
Change this line:
ec2-user:x:500:500:EC2 Default User:/home/ec2-user:/bin/bash
to look like this:
ec2-user:x:500:500:EC2 Default User:/home/ec2-user:/sbin/nologin
Save the file. [I do all file editing in vi.]
If this worked correctly, you can verify that attempting to login won't work in another shell. Also verify that scp no longer works. If these still work as before, then the method I've documented here is not for you.
Verify only 1 user logged in.
[EDIT.140626] Verify /etc/init.d and /etc/rc.local are unchanged (that is, they aren't starting an unexpected script).
At this point you can edit the server's user account ~/.ssh/authorized_keys file to delete the old (exposed) key, and paste in the new public key you generated earlier.
Once the old key is deleted, it's safe to revert the /etc/passwd file back to the way it was, and log in with the new key.
