Is your home drive on the Ubuntu server encrypted? If
If so then it is decrypted when you log in to the local account. When you try and ssh in before logging in locally then ssh can not read your ~username/.ssh/authorized_keys~username/.ssh/authorized_keys
file and will ask for your password.
A solution would be to change the location that ssh looks for authorized_keys. As root you can edit the /etc/ssh/sshd_config/etc/ssh/sshd_config
file and change the AuthorizedKeysFileAuthorizedKeysFile
line.
For example if you change it to -
AuthorizedKeysFile /etc/ssh/authorized_keys.%u
AuthorizedKeysFile /etc/ssh/authorized_keys.%u
then copy your ~username/.ssh/authorized_keys~username/.ssh/authorized_keys
to /etc/ssh/authorized_keys/etc/ssh/authorized_keys.username
.username ssh will be able to read it without logging into the machine locally.