How to Set up your Ubuntu Server with a new or updated SSH key

Backdrop

For some reason this particular skill has been on my ‘fear list’ for many years. I had created ssh keys before, I had ssh’d into lots of servers, but one day, I migrated from one laptop to another and tried to ssh into my work computer and… boom:

ssh root@123.123.123.123 root@123.123.123.123: Permission denied (publickey)

In the past, I just quit and went back to the old machine and ssh’d in from there but this time I decided I need to level-up and defeat this pesky fear…

Turns out, it was pretty easy.. but there wasn’t a very simple blog like this one you’re reading.

First, I watched this video which helped but started to go a little ‘scary’ with all the ‘cats’. Hiss….

Then I found this blog which was informative but didn’t really tell me how to practically move my key from A to B.

Hence this blog you’re reading now.

Goal

To be able to connect to your Ubuntu server from a different machine from the one you originally set your SSH keys up with and not get the ‘permission denied(publickey)’ message

Steps

1. Create the new key on the new remote machine.

• Open a terminal
• type ssh-keygen
• do the prompts. I create passwords when prompted but many people do not. You choose.
• check to make sure it was created by typing ls -al ~/.ssh You should see something called ‘id_rsa.pub’ which is the default but if you manually changed things in the steps above it might say something different.

2. Move the SSH Key to your Old Machine (which has SSH access to the server)

• Make the cotents of the newly-created ssh public key (id_rsa.pub) file ‘copy-able and paste-able’ with this command cat ~/.ssh/id_rsa.pub
• highlight the text starting with ssh-rsa and ending in the name of your machine
• copy with your right click mouse or by pressing control+shift+c
• Paste the contents first into your keepassxc password manager and if you don’t know what that is you should find out 🙂 I just realized I should have always been backing up my ssh keys to a password manager so starting now that’s what I’ll be doing
• Decide on a method of sending this text to your server. In my case, I still had access to the old local machine which had SSH access to the server so I will use that method. If you lost access to that then you might have to physically move the data to the server on a USB drive in a text file, or, if you are on something like Digital Ocean they have a virtual terminal when you log in and you should be able to move the text contents in that way too. But in either case, we will assume you still have access to the first machine that can still SSH into your server.
• Send the contents of the ssh key to your machine in the way you decide. I am lazy and foolish so I sent it to myself over a chat messenger and then deleted it. A usb drive with a text file is better….

3. SSH into your Ubuntu Server and paste in the key

• ssh into your server the way you always do from the old machine
• Enter sudo nano ~/.ssh/authorized_keys which will open up the file on the server that has the authorized ssh keys listed. they are listed as ‘one key per line’ in the terminal
• put the cursor at the bottom of that file
• paste in the contents of the new SSH key into that
• control x and yes to save the changes

4. Test the SSH connection from the new machine

Now you should be able to log in from the new remote machine into your server. Give it a test by trying to ssh in.

Follow up thoughts

Now that I’ve learned this new skill I realized that if you are a system administrator, it’s probably wise to have an old laptop just to have SSH access to all your servers you maintain in this way so that if you lose your main machine you can just open the second machine and start SSH’ing in and doing work…

Hope some or all of this helps one or more of you.