Managing your Files

Users on Shamu are automatically granted several locations to store their files. Most users will be storing their files in one of two locations, $HOME or /work.

• $HOME directory - /home-new/abc123 - This directory is entirely controlled by you and the default permissions are that nobody else can see or access your files. There are no quotas limits and this location is backed up nightly.
• Work directory - /work/abc123 - This directory is where you should place any input/output files as well as logs for your running jobs. This directory is NOT backed up.

Transferring Files to Shamu

Shamu uses the Secure FTP protocol to tranfer files. Below are instructions for the various operating systems.

Windows

On Microsoft Windows, an SFTP client must be downloaded to transfer files to Shamu. This guide will use the “MobaXterm” application, which was also used in the connecting (ssh) and X-forwarding guides. Other common SFTP applications are listed below; all of these will work fine with Shamu.

SFTP Clients

• MobaXterm
• FileZilla
• WinSCP
• PSFTP

File Transfer using MobaXterm

When you log in to a remote Shamu session using SSH, a graphical SFTP (Secure File Transfer Protocol) browser appears in the left sidebar allowing you to drag and drop files directly to or from Shamu using the SFTP connection. To manually open a new SFTP session:

1. Open a new session

Note: Windows-based editors generally put an extra “carriage return” (also referred to as control-M/^M) character at the end of each line of text. This will cause problems for most Linux-based applications. To correct this problem, execute the built-in utility dos2unix on each ASCII file you upload from your Windows machine to Shamu:

    [abc123@login01 ~]$ dos2unix filename

Linux/Mac

There are several ways to transfer files from a Linux or Mac machine to Shamu. The easiest methods are available right from your command line.

File Transfer

SCP

Open a Terminal and use the scp command, executed on your local machine:

localMachine% scp -P1209 LocalFile abc123@login.shamu.utsa.edu:/path/to/destination

Below is a specific example

localMachine% scp -P1209 hello_world.py abc123@login.shamu.utsa.edu:/work/abc123/.

scp works like cp but uses ssh to connect, so you will be asked for your campus passphrase again.

To copy a directory from a local to remote system use the -r option:

localMachine% scp -P1209 -r /local/directory abc123@login.shamu.utsa.edu:/remote/directory

-- AdminUser - 17 May 2016