Home News Course Calendar Homework Projects Lecture Slides Suggested Readings Darwin Cluster Amazon EC2 Julia Docs Previous Classes

Our Amazon EC2 Environment

Log in here to access our account (password will be provided in lecture).

Using our account provides access to some free time from Amazon, totaling roughly 12 days of CPU time on a large instance per student, or 48 days on a small instance.

Terminology

Instance - A virtual machine

Image - A copy of all the data on a disk

EBS - Amazon's Elastic Block Store, a storage service with a disk-like interface

S3 - Amazon's Simple Storage Service, a scalable key-value store

EBS-backed instance - An instance whose boot disk is an EBS volume, allowing it to be turned on and off and keep data (like a normal computer)

Stopping an instance - Turning off the power. Only available for EBS-backed instances.

Terminating an instance - Powering off and deleting, losing all data.

S3- or instance-store-backed - An instance that uses no EBS space, and can only be terminated (not stopped).

Instance types

These are the EC2 instance types we will generally use. "Small" should be the default choice for testing and experimenting. Use "large" if you need more memory per node. Other instance types can be considered in special cases (micro might be good for scalability testing).

Name	Arch	CPU	Memory	cost $/hr
micro	32- or 64-bit	~ 1GHz	613MB	0.02
small	32- or 64-bit	1GHz	1.7GB	0.08
medium	32- or 64-bit	2GHz	3.75GB	0.16
large	64-bit	2x 2GHz	7.5GB	0.32
high-CPU medium	32- or 64-bit	2x 2.6GHz	1.7GB	0.165

How to create an instance

1. Select "Instances" in the sidebar of the AWS Management Console.
2. Click the "Launch Instance" button.
3. Click "Continue" for "Classic Wizard"
4. Select the "My AMIs" tab.
5. Select an image to use. The image determines what instance types will be available, since it contains software for either x86 or x86/64. Look under "Root Device" to see which images are on EBS. "julia64_ebs" is our default image for x86/64, with Julia pre-built.
6. Select an instance type. Click "Continue".
7. Click "Continue" for "Advanced Instance Options".
8. Click "Continue" for "Storage Device Configuration".
9. The next screen lets you assign key/value tags to your instance. Fill in a value for the "Name" key; you can name your machine anything, or use your own name.
10. Under "Create Key Pair" select "Proceed without a Key Pair".
11. Click "Continue" for "Security Groups".
12. Hit "Launch"!
13. Wait for your instance to start (watch under the "Status" column).

Connecting to your instance

1. Our default images already contain a key pair allowing passwordless SSH access. Download the private key here to a Unix machine (e.g. your Athena account), and run chmod 400 julia64.pem. This step only needs to be done once.
2. Click on your instance in the management console, and scroll down in the lower pane until you see the line "Public DNS:". This will tell you the instance's public address. It will look something like "ec2-50-17-79-44.compute-1.amazonaws.com".
3. Run ssh -i julia64.pem ec2-user@ADDRESS. Type "yes" if asked "Are you sure you want to continue connecting?".
4. The Julia environment is in the home directory, in a directory called "julia". We can run it and use the peakflops() function to see how good a machine we got:

   [ec2-user@domU-12-31-39-0A-8C-72 ~]$ julia/julia
                  _
      _       _ _(_)_     |
     (_)     | (_) (_)    |  A fresh approach to technical computing
      _ _   _| |_  __ _   |
     | | | | | | |/ _` |  |  Version 0.0.0-prerelease
     | | |_| | | | (_| |  |  Commit ea80f93d30 (2011-09-08 02:56:32)
    _/ |\__'_|_|_|\__'_|  |
   |__/                   |
   
   julia> peakflops()
   The peak flop rate is 7.17063066823053408 gigaflops
   7.17063066823053360e+09
   

Terminating instances

If you want to pause work for a while and pick up later, you can stop your instance(s) (this will cut down on costs). But, if you're done with an instance, please terminate it.

1. Right click on your instance and pick "Terminate" or "Stop".
2. Click "Yes" to confirm.

NOTE: When an instance is terminated, any changes you might have written to disk are lost! You can use scp to move files on and off.

From Amazon:

scp file username@linux.mit.edu:~/path/file

To Amazon:

scp -i julia64.pem file ec2-user@ADDRESS:~/path/file