Managing your Files
Users on Shamu are automatically granted several locations to store their files. Most users will be storing their files in one of two locations, $HOME
- $HOME directory - /home-new/abc123 - This directory is entirely controlled by you and the default permissions are that nobody else can see or access your files. There are no quotas limits and this location is backed up nightly.
- Work directory - /work/abc123 - This directory is where you should place any input/output files as well as logs for your running jobs. This directory is NOT backed up.
Transferring Files to Shamu
Shamu uses the Secure FTP protocol to tranfer files. Below are instructions for the various operating systems.
On Microsoft Windows, an SFTP client must be downloaded to transfer files to Shamu. This guide will use the “MobaXterm” application, which was also used in the connecting (ssh) and X-forwarding guides. Other common SFTP applications are listed below; all of these will work fine with Shamu.
File Transfer using MobaXterm
When you log in to a remote Shamu session using SSH, a graphical SFTP (Secure File Transfer Protocol) browser appears in the left sidebar allowing you to drag and drop files directly to or from Shamu using the SFTP connection. To manually open a new SFTP session:
- Open a new session
Windows-based editors generally put an extra “carriage return” (also referred to as control-M/^M) character at the end of each line of text. This will cause problems for most Linux-based applications. To correct this problem, execute the built-in utility
on each ASCII file you upload from your Windows machine to Shamu:
[abc123@login01 ~]$ dos2unix filename
There are several ways to transfer files from a Linux or Mac machine to Shamu. The easiest methods are available right from your command line.
Open a Terminal and use the scp
command, executed on your local machine:
localMachine% scp -P1209 LocalFile email@example.com:/path/to/destination
Below is a specific example
localMachine% scp -P1209 hello_world.py firstname.lastname@example.org:/work/abc123/.
to connect, so you will be asked for your campus passphrase again.
To copy a directory from a local to remote system use the
localMachine% scp -P1209 -r /local/directory email@example.com:/remote/directory
- 17 May 2016