Copy files using WinSCP: Permission denied

Question

Platform: Google Compute Engine

VM instance: pytorch-latest-gpu-20190911

login command: $ gcloud compute ssh jupyter@my-fastai-instance -- -L 8080:localhost:8080

In order to save GCP billing time, I cloned the fastai course v3 and libraries on my local machine which allows me to play with non-compute intensive operations like data set creation, etc.

So after creating several datasets locally, I tried to copy the folders to my vm instance using WinSCP. But I got ‘Permission denied’ error. After much frustration, I found this was due to the /home directories being owned by ‘jupyter’. I then used $ sudo chown -R my-username ~/jupyter to change ownership to my username. This allowed me to copy all my local data to gce via WinSCP. But when I attempted to run my notebooks in gce, they would not run. In order to run them, I had to revert ownership back to ‘jupyter’.

So my question is this: how can I change /home owner to my_username but still allow Jupyter to run my notebooks? Is this a setting within Jupyter or do I need to do something else? My goal is to freely transfer files with WinSCP without messing with changing jupyter ownership of the directories.

Answer 1

WinSCP.net has published a detailed description of the procedure to follow in the article
Connecting Securely to Google Compute Engine Server with SFTP.

The article itself is much too long and detailed to reproduce here. In a nutshell, it describes how to:

• Generate your private key
• Collect information about your GCE instance
• Set up SSH keys for your Google Compute account
• Setup the WinSCP login

Answer 2

never worked in that same scenario, but I would guess that jupyter is just a typical user with a home folder. You could have added your username to the jupyter group instead of taking ownership.

When you run ls -l /home you can see the current permissions of the user folders. Take for instance:

ls -l /home | grep jupyter
drwxr-xr-x 13 jupyter jupyter   4096 Sep  8 10:26 jupyter

The first jupyter is the username, the second jupyter is the group name. You can see the permissions are:

• rwx for user jupyter.
• r-x for members of group jupyter.
• r-x for the rest of the users.

In that case you could add your user to the group jupyter editing /etc/group/:

jupyter:x:1001:bill

and then add the appropriate permissions for the group:

chmod -R g+w /home/jupyter

Which basically means 'recursively change permissions in /home/jupyter to add write permissions to the group'. Now you should see the permissions like:

ls -l /home | grep jupyter
drwxrwxr-x 13 jupyter jupyter   4096 Sep  8 10:26 jupyter

Note the difference, now the UGO permissions (User, group, other) are 'drwxrwxr-x'

Answer 3

When getting a "Permission denied" error while transferring files to a path that requires a 'super-user' permission.

typically, there are two possible methods that I know:

Method 1: Use sudo with SCP from the Command Line:

sudo scp -r /path/to/local/files username@remote:/var/www/

Method 2: Temporarily Change Permissions (Not Recommended for Production)

Change the permissions of the target directory:

sudo chmod 777 /var/www/

Restore the original permissions after transferring the files:

sudo chmod 755 /var/www/

How to deal with this error message, in case the two above methods are not an option and I want to configure the WinSCP to do the file-transfer with a sudo without prompting me for a password and with no error?

ANSWER:

1. Create a Custom SFTP Script on the Remote Server:

ssh username@remote
sudo touch /usr/local/bin/sftp-server-sudo.sh
sudo chmod +x /usr/local/bin/sftp-server-sudo.sh
sudo nano /usr/local/bin/sftp-server-sudo.sh

Add the following content to the script:

#!/bin/bash
sudo /usr/lib/openssh/sftp-server

2.Ensure Proper Permissions and Test the Setup: Make sure that the script has the correct permissions and that your user has sudo permissions without needing a password for /usr/lib/openssh/sftp-server. You can test the script manually by SSHing into the remote server and running:

sudo /usr/local/bin/sftp-server-sudo.sh

IF YOU SEE THAT YOU ARE PROMPTED FOR PASSWORD, follow the step 3. otherwise proceed with step 4.

3. Steps to Allow without Password:

ssh adminuser@remotehost
sudo visudo

Add the No-Password Entry at the end:

# User privilege specification without password
YouUserName  ALL=(ALL) NOPASSWD:ALL

4. Configure your WinSCP:

1. From the menu Tabs go to Sites > Site Manager....
2. Go to Advanced Site Settings, select SFTP in the Environment and add the value /usr/local/bin/sftp-server-sudo.sh for the SFTP server:.
3. Go to the Shell and add the value sudo su - for the Shell:.
4. Save and exit.

Now, you should be able to transfer the files with a super user privilege.