The Research Computation Cluster (RCC) provides a stable and current Red Hat Enterprise Linux environment for computation and data analysis.  The primary means of access is Secure Shell (SSH) and the usual Linux command-line environment.

The RCC provides a full suite of open-source scientific and programming tools, including:

• R
• Macaulay2
• Pari/GP
• Sage
• Python, NumPy, SciPy
• Octave
• Java
• traditional C, C++, Fortran compilers and libraries
• git, mercurial, subversion, CVS, and many others

Some commercial software is also possible, but we are working on licenses. This includes:

• MatLab
• Mathematica
• Maple
• SPSS
• SAS

At the network level, the current “test harness” servers are at these addresses:
150.108.20.61 (ace-math01.ace.fordham.edu)
150.108.20.62 (ace-math03.ace.fordham.edu)
150.108.68.34 (math department only)
As hardware expands through the summer, the number and power of the machines will grow. At the physical level, the hardware is in the basement of Keating Hall. The hardware was purchased as part of the Academic Computing Environment [ACE].

The /app directory is shared between the RCC servers (except burr, which is independent in the mathematics department). All user home directories are in /app/home/, so your files should be available on any machine. Custom-installed software is also under /app, and old versions are preserved. For example, versions of Sage are installed at /app/sage-5.8/sage and at /app/sage-5.10/sage, et cetera.

This is a multi-user system, and many users may be running jobs at once. Of course, the system is made to be used. There are two things to balance: Processor allocation and memory allocation.

Processor time is easy to sub-divide, as it is divided into time-steps called "jiffies". During a single jiffy on a single processor core, a single process may be run. Thus, if there are two jobs with equal priority are running on the same processor core, the Linux kernel allocates every-other jiffy to each job. Thus, in real time, each job runs half as quickly as it could on its own core. In that sense of time-sharing, CPU time is infinitely divisible.

Memory allocation is more difficult. The machine has a fixed amount of fast physical memory as well as some slow overflow space on disk that is called "swap". Jobs request memory as they run, and there is no way to forcibly remove memory from a job without killing that job. Of course, the Linux kernelneeds some memory to keep essential processes running, too. Therefore, if too much memory is requested by running jobs, the Linux kernel enters the dreaded Out-Of-Memory [OOM] condition and risks crashing. This is like having too much cargo on a sinking ship --- if you can quickly throw some containers overboard, the rest of the ship might be saved. The kernel must decide which jobs to kill.

To deal with both of these problems, there is an hourly process that examines all of the running jobs and classifies them as "young" or "old". Once a job has used 12 processor-hours, it is classified as "old," and thereafter receives 50% of the share of a young job. So, instead of each job getting every-other jiffy, the young job gets two jiffies of every three. The goal is to get short jobs out of the queue as rapidly as possible. Also, "old" jobs are the first to have their active memory pushed to slow disk swap in the event that system memory is low.

However, old jobs are increasingly protected from being killed by the Out-Of-Memory killer. Obviously, your fellow researchers get very upset when a 2-month-old job was killed because your runaway 10-second job ate all the memory. Under the current configuration, by the time a job has been running for 10 CPU-days, it should never be killed by an Out-Of-Memory condition (unless the entire machine crashes for some horrible reason). On the system status pages (linked above), you can see the "OOM Score". The higher the score, the more likely a job is to be killed in an OOM condition. A score of -1000 means "never kill"; a score of "0" is neutral. Even with this fix, we have the potential problem that many 10-day-old jobs could eat all the memory and sink the ship, so please be careful. There is essentially no way to guarantee that OOM will never happen.

The goal of SSH is to allow secure communication between computers on the Internet, even if the intermediate network has been compromised. To accomplish this, three things must be accomplished once a basic network connection is established:

1. Your workstation must know that the server is actually the machine it claims to be, to detect whether some malicious third party has replaced the server. This is the role of the server fingerprint.
2. The server must know that the person sitting at the client machine is authorized to access to requested account. This is the role of the keypair that you generate with ssh-keygen and unlock with ssh-agent on your workstation.
3. Once identities have been established, all network communications must be encrypted with a strong cipher. This is auto-generated for each connection, using Diffie--Hellman-style key-exchange to make a shared 128-bit AES key.

(If you have already made a keypair for other purposes, you can skip this step. Just send the public part of your keypair to the administrator.)

1. Open a Terminal window. If you cannot find a launch icon on MacOS, try hitting Command+Space to search and type "terminal". If you cannot find a launch icon on Linux, try hitting Alt+F2 and try "gnome-terminal" or "konsole" or the old-fashioned "xterm".
   
2. In the terminal, type "cd ~" to make sure you are in your home directory. (Remember that "~" is always the alias for your home directory.)
   
3. In the terminal, type "ssh-keygen". Do not change the default filename. You should use a long passphrase to protext your keypair. You don't have to enter the passphrase very often (only once per day or once per reboot, probably), so make it very long and easy to remember. For passphrases, length is more important than crazy characters. Try something like a complete sentence telling a little story: "When Sally was an 8-year-old, her Labrador puppy Henry ate the birthday cake."
   
4. After you have generated your keypair, type "cat ~/.ssh/id_rsa.pub" to see the Public portion of your keypair. It will look something like this:
   ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnTaP2CdEB7qxtEPNEIviiZPftqiu79tgFsUA8VgU0qagUQljMBgAixw5Zmu7bLsF+ILFvHxjr1OAOOlE79xY
   uqYBy8GFeFTiJzznzkeJzAYK/rKfTcrRO8EYLuIgPvTy8/bKrbCwzIKZu8aJ8k1GF9QshVTn/2UKC6ejRip3XDPmfrM7zg0qVkL5ruoT4VNqdhhdnl6hL8cxI8Y
   F+Kb9zRQgiwgtXJWlwsDpcKXsqbohvfw/GuJv3e6l7Z/l2susqlvm7xzUMWf2sdW5mc45Lqac/aVyLsoYBJV158caD6MAxeaWrsN9An+I+Fqk7Bb4kjFUPPiCIl
   Qe0VZtX93Jkd smith@quark.cis.fordham.edu
   
5. Copy the text and paste it into the RCC Account Application Form or email it to the administrator at rcc@fordham.edu. You will have to wait until the administrator makes the account, but that usually takes less than a day.

This step is optional, but it makes life more convenient, since it allows you to type your long passphrase less frequently and therefore have automated connections. Note! Many modern Linux desktops (KDE, Gnome, Ubuntu's Unity) will automatically run ssh-agent for you, so you may not need to worry about this. As far as I know, MacOS does not automatically run ssh-agent, so these steps are necessary there.

1. Open a terminal and type ssh-add. If ssh-agent is already running, it should ask for your passphrase and unlock your keypair.
2. If ssh-add failed with "No ssh agent found" or a similar message, then run eval $(ssh-agent), which starts the ssh-agent and loads its identifier into your Terminal session. Then, try ssh-add again.
3. Try logging in, by typing ssh username@150.108.20.61. The first time you try, it will warn you that the server is not known and offer a fingerprint. Please check the server fingerprint against the list on this page. If the fingerprint ever changes after this time, SSH will warn you that something malicious may be happening.
   
4. If the you see a login message and a command prompt saying "ace-math01", then everything worked!

1. Open a terminal window and type ssh username@150.108.20.61. If ssh-agent is running properly, it should not ask for your passphrase, as your keypair is already unlocked.
   
2. If you want graphical programs to run on the server and display their windows on your workstation, then try the "-X" switch.
   
   

SSH allows you to move files over the network using the scp or sftp commands. A more efficient way for bulk transfers is to use rsync, which can recursively transfer entire directories while only moving the changed parts of files. For example:

• scp my_file username@150.180.20.61: This will copy the local file my_file (in your current working directory) to the location ~/my_file on the server.
• rsync my_file username@150.180.20.61: Same thing for rsync.
• rsync --archive my_directory/ username@150.108.20.61:my_new_directory/ Note the trailing slashes, they are very important!! This will recursively copy the contents of the local directory my_directory/ to the contents of the remote directory ~/my_new_directory/. If you run the command a second time, it will only send the changes. When using rsync, try adding the -n option to see what would be sent without actually sending it (so that you don't accidentally clobber files at the destination). My favorite options for rsync are first to check with -Pain then to run it with -Pai. Of course, read the man page.

A good rule of thumb for computers is "if you are typing a complex command more than twice, you are doing it wrong." SSH allows several options to be set for every connection on a per-server basis, using the configuration file ~/.ssh/config. Here is an example:

Host ace1
     Hostname 150.108.20.61
     User asmith122
     BatchMode yes
     Compression yes
     FordwardX11 yes
     DynamicForward 9051
     LocalForward 8080:localhost:8085

This means that I can type "ssh ace1" and many of my favorite options are automatically used. In this case, it saves me from typing "ssh -C -X -D 9051 -L 8080:localhost:8085 asmith122@150.108.20.61".

It is wise to read the System Manual Pages obtained from the commands "man 1 ssh" and "man 5 ssh_config" to learn what is possible with SSH.

Secure Shell (SSH) is the standard protocol for secure network access in the world of UNIX, Linux, and BSD (including MacOS systems). On these systems, SSH is built into the basic system utilities, as described in the Instructions for SSH on Linux and MacOS workstations. However, Microsoft Windows is separate from the UNIX tradition, and some extra tools are necessary to use SSH on Windows. If you're curious about the theoretical underpinnings of SSH (which are indeed fascinating!), read the Theoretical Background section of the Instructions for SSH on Linux and MacOS workstations.

For complete functionality, three programs are used. However, some of them are optional:

1. The PuTTY suite, which is necessary.
   • PuTTY itself, which provides command-line access to the server.
   • PuTTYgen, which creates a cryptographically secure keypair that will be used for your logins.
   • Pagent, which conveniently manages your keypair.
2. WinSCP is a nice program to send files back-and-forth between your workstation and the servers. If you don't mind using the command-line file-copying tools in PuTTY, this is optional.
3. The Cygwin/X module of Cygwin (or the alternative Xming which works very well but requires a small donation to the developer), which allows your workstation to draw windows for graphical programs that are running on the server. If you only use command-line tools, this is optional.

1. Download the Installation Package from WinSCP.net
   
2. Run the program, and fill in the same account information as you used in Putty. Do NOT set a `private key file' or `password'. The running Pagent process will automatically authenticate you!
   
3. Drag-and-drop files between your workstation (left) and the server (right).
   

1. Download and run the setup program from cygwin.com
   
2. Cygwin has many components it can install (effectively reproducing a complete UNIX environment in Windows). We need "X11", so make sure that is selected. Cygwin downloads a lot of packages and will take a long time to install, perhaps an hour.
   
3. Run XLaunch
   
5. There are several display styles. Most people prefer the "Multiple windows" one.
   
6. Select "Start no client" so that Cygwin/X will run in the tray as a service.
   
7. Log into the server using PuTTY, and try to run a graphical command, such as "xclock" or "firefox". The software will run on the server, but its window will be drawn on the screen at your workstation. This should be reasonably responsive at Fordham, but it may be unbearably slow from a home DSL connection.
   
   

The default shell environment is Bash, which is a complete computer language designed for rapid access to frequent file and navigation operations. You should read about it at The Advanced Bash Scripting Guide

Normally, if your network connection closes, then the shell dies, and all of the child processes spawned from it will die, too. To avoid this for long-running jobs, try using GNU Screen, which allows you to "detach" the shell session from a specific login, so you can "reattach" once your network connection is re-established. Although the manual is very complicated, basic usage is very simple. Start screen by running screen. It will give you a new shell, which you can detach with CTRL-a d. Later, from a different SSH session on a different workstation, you can re-attach by running screen -xR. Tmux is an alternative to GNU Screen that is easier to use but is sometimes more limited in its abilities.

The server has rudimentary email sending capabilities, but it can only send to @fordham.edu addresses. For example, you set your jobs to run the command echo "all done" | mail username@fordham.edu when they finish.

Google (or bing or duckduckgo) is your friend, as are your colleagues and administrators. Don't be afraid to ask! Also, try man foo if you have questions about a program named "foo", or try apropos foo if you want to search the system manual pages for the word "foo". For example, you may want to look at man ssh to learn all the details of setting up SSH, and apropos ssh suggests that you also look at man ssh_config, man ssh-keygen, man ssh-keyscan, and more.

The server has subversion, git and mercurial installed for version control of code. Repositories can be accessed over SSH. If you do programming and don't know what these programs are, look them up! Mercurial is probably the easiest to use.