CSE 6230

Programming for high-performance computing systems.
Fall 2014, Professor Richard (Rich) Vuduc.

Overview

This course gives you hands-on experience with programming high-performance computer systems, from single-processor multicore platforms up to large-scale distributed memory multi-CPU/GPU systems.

Unlike CSE 6220: Intro to HPC, which presents theory, this course explores practice: current and emerging programming models, performance analysis, and low-level performance engineering both for parallelism and for locality.

Ideally, you will emerge knowing how to write blazingly fast and scalable code for modern machines.

Schedule

The class meets Tuesdays and Thursdays from 4:35-5:55pm in Klaus 2447.

Each weekly lab assignment is due before class (at 4:30pm) on the day of the next lab.


Module 1: HPC Programming 101

Week Tuesday Thursday Notes
1: Aug 19 & 21 Introduction
[PDF (15 MiB)]
(no class) Read “Dynamic multithreading” (PDF) thru p. 795
2: Aug 26 & 28 Dynamic MT
[PDF (819 KiB)]
Cilk Plus (dynamic MT)
[PDF (2.8 MiB)]
Optional reading: Hager Ch. 5
3: Sep 2 & 4 Lab 1 OpenMP 101 (shared memory)
[PDF (2.3 MiB)]
Hager Ch. 6 & 7
4: Sep 9 & 11 Lab 2
OpenMP + NUMA
[PDF (842 KiB)]
CUDA 101 (GPUs)
[PDF (5.5 MiB)]
Hager Ch. 8
5: Sep 16 & 18 Lab 3 MPI 101 (distributed memory)
[PDF (37 MiB)]
Hager Chap. 9 & 10
6: Sep 23 & 25 Broadcast algorithms Lab 4 See van de Geijn's notes on Piazza

Module 2: GPU deep dive

Week Tuesday Thursday Notes
7: Sep 30 & Oct 2 GPU Tuning, Part 1
[PDF (11 MiB)]
Lab 5 Project proposals due on Fri Oct 3
8: Oct 7 & 9 GPU Tuning, Part 2
[PDF (11 MiB)]
Matrix multiply notes for Lab 6
[PDF (15 MiB)]
Lab 6
9: Oct 14 & 16 (no class) Advanced CUDA features No lab; Project survey due on Fri Oct 17

Module 3: CPU deep dive

Week Tuesday Thursday Notes
10: Oct 21 & 23 Compilers: Polyhedral model Lab 7 (optional)
11: Oct 28 & 30 Microarchitecture Lab 8 Readings TBD (will come from textbook)
12: Nov 4 & 6 SIMD Lab 9
13: Nov 11 & 13 Caches & NUMA Lab 10 (in-class only) Hager Ch. 8
Project checkpoint due Fri Nov 7

Module 4: Topics

Week Tuesday Thursday Notes
14: Nov 18 & 20 No class due to Supercomputing (SC) '14 Conference
15: Nov 25 & 27 Hybrid programming (no class–Thanksgiving)
16: Dec 2 & 4 Lab 11 (in-class only) TBD Project report due Fri Dec 5

Poster Session: Tu Dec 9, 2:50-5:40pm (final exam day)

Logistics

Textbook(s)

Grading

This course is taught under the Georgia Tech Honor Code. In the context of this course, this means that all work that a student submits for individual assignments and exams will be his or her own work. For group assignments, project submissions, or of projects that are part of a larger on-going project, the final report has a required section in which each student must detail which portions of the work were completed individually and which portions were the work of others.

The course has two components: weekly labs, which collectively account for 50% of your grade, and the final project, which has graded checkpoints and collectively accounts for the remaining 50% fo your grade.
Assignments due dates and times are strictly enforced. Any assignment you submit late gets 0 credit. To give you some slack, we will drop the lowest two lab assignments. (So, effectively, you could just skip two labs with no penalty.)

Prerequisites

You should be comfortable with sequential programming in “mainstream” general-purpose compiled languages, such as Fortran, C, C++, or Java. Familiarity with basic algorithm analysis (i.e., big-O notation) is also helpful.

Contact

We will use Piazza for class discussion: https://piazza.com/gatech/fall2014/cse6230. The system enables you to get help quickly and efficiently from classmates and instructors. Rather than emailing questions to the teaching staff, post your questions on Piazza.

Professor Richard (Rich) Vuduc
Klaus 1334
Office hours: Mon 2-3p & Wed 4-5p

Ms. Jiajia Li
Klaus 1343 (“HPC Lab”)
Office hours: Th 3:30-4:30p

cse6230-fa14--09-polyopt.pdf