Parallel Computing 048874

Spring 2012

Last Updated: 21/04/2012

Time and Place

        Sunday 12:30-14:30 EE (room 3xx)

Lecturer

        Prof. Ran Ginosar (ran@ee.technion.ac.il)

Syllabus

This is a course in ARCHITECTURE (what is architecture? what questions do we ask and what answers do we produce and how? what issues does the architect rely on others to investigate?)

 

(Preliminary plan)

Parallel computing architecture

Warehouse Scale Computing versus on-chip parallelism (multicores, manycores)

History of the field and what we can learn from it (what we could adapt and what we'd better avoid)

Interconnection networks and NoC

Various manycore architectures: clustered, tiled, shared memory, GPU

The PLURAL architecture

Programming models, languages, algorithms, applications, patterns

Performance evaluation: How to modify and forget Amdahl's Law, contrast with Gustafson, etc. What about power / energy / cost / complexity

Sources

  1. What is parallel computing? Short summary / intro in http://www.llnl.gov/computing/tutorials/parallel_comp/
  2. A good background for parallel programming is taught in CS by Assaf Schuster and Mark Silberstein: http://webcourse.cs.technion.ac.il/236370. We are not practicing actual programming, except indirectly (how can we do this indirectly?)
  3. Check Wikipedia on Amdahl's Law, Gustafson's Law, Karp-Flatt Metric, Speedup, Parallel Computing, and various related topics linked from these pages
  4. Watch Google-Tech-Talk lecture by Mark Hill (U. Wisconsin) on Amdahl's Law for Multicores and visit his Amdahl's page. He starts in the right direction but then veers off and we will have issue with his vision.
  5. Watch Google-Tech-Talk lecture by Jack Dongarra (UTK) on HPC (high performance computing). He has been a key leader in developing linear algebra libraries for parallel computing. The lecture provides a good background on supercomputers and then suggests one of the best ways for fighting Amdahl's Law on them.
  6. Watch Google-Tech-Talk by Dave Patterson (UC Berkeley) on The View from Berkeley and visit their web page. It attempts to review the entire field, but Dave makes a gloomy outlook which we should take as a challenge.
  7. See the on-line slides accompanying Introduction to Concurrency in Programming Languages by Sottile, Mattson and Rasmussen on http://www.parlang.com/  (slides under “course material”). The presentation is useful from an architect's point of view.
  8. Another interesting resource: Parallel patterns (Mark Snir page).

Class Schedule

Lecture 1 March 11 Introduction    
Lecture 2 March 18 Architecture Basics Williams on Roofline  
  March 25 NO CLASS    
Lecture 3 April 1 Many-core Examples Devadas on EM2 HW1 due
  April 8 NO CLASS    
Lecture 4 April 15   Corporaal on GPU  
Lecture 5 April 22 Patterns   HW2 due
Lecture 6 April 29 PRAM, PRAM Architectures    
  May 6 NO CLASS    
Lecture 7 May 13      
Lecture 8 May 20      
  May 27 NO CLASS    
Lecture 9 June 3      
Lecture 10 June 10      
Lecture 11 June 17      
Lecture 12 June 24      

 

Homework Assignments

1. Measure performance of your own machine (as a preparation for measuring its performance and speedup when running parallel code). See HW-1. Due 1 April 2012.

2. Repeat HW-1 on a multi-core and show the speed-up. See HW-2. Due 22 April 2012.