Parallel computing

Page contents

• Official course description
• Tentative contents
• Project
• Registering
• Schedule
• Grading
• Workload
• Literature
• WWW-links
• Tools
• Exercises
• Examples

• MSc -level elective course
• Introduction to parallel computing and parallel algorithms. Models of parallel computing, parallel architectures, design principles of parallel algorithms. Implementability of parallel algorithms in different architectures.
  Literature: Jaja: Introduction to Parallel Algorithms. Almasi, Gottlieb: Highly Parallel Computing, 2nd.
  Prerequisites: Programming language C, Computer Systems, Data Structures and Algorithms.
• Course intersects by 2cp with Seminar in Parallel Programming 3621550. Thus, if you have passed 3621550, you will get net 3cp of this course.

Parallel computation is about achieving more computing power for a single task.

Parallel computing is mainstream at 2018!

Subjects

• An introduction to parallel computing
  • What, Why, How
  • Examples
  • Benefits, drawbacks
  • Parallel computers
  • Difficulties, Design issues
• Models
  • Theoretical models
  • Standards
  • Accepting physical and financial limitations
• Parallel computer architectures
  • SMP, MPP, NOW
• Parallel algorithms
  • Simple algorithms using PRAM-model
  • Sorting, etc.
• Parallel programming
  • MPI, HPF, OpenMP, Shared memory (SMP), GPU computing (CUDA/OpenCL), ...
• Other topics
  • Concurrent programming (depending on time and interests)
    • pthreads
    • processes
    • interprocess (&thread) synchronization & communication
  • Distributed computing

• 08.01.2018 -- 27.02.2018
  Mon, Tue 12-14 (Except Tue -> Wed 14.02.)
  Joensuu: TB248, Kuopio: F213 (Except 23.01., 14.02. at F211)
• 2015 lecture recordings.
  • Username is the five first letters of the first word of the course name, password is the five first letters of the second word of the course name.
• Slide copies (updated during the course)

• 12.01.2018 -- 02.03.2018
  Fri 10-12 (Except Fri -> Thu 08.02.)
  Joensuu: TB248, Kuopio: F213 (Video-exercises).
• On select weeks, Kuopio group will be moved to from Fri to Monday 10-12 (and I'll be there live).

• Please register to the course through WebOodi.
• "Separate" courses at Joensuu and Kuopio because of Lukari. In reality, the same course.
• Register for the course and one exercise group.

• Course exam Fri 16.3.2018 12:00-16:00, Joensuu: C2 (Carelia), Kuopio MD100
  Re-try 4.5.2018 M100 / CA101

• X-exercises have to be made&passed before participating.
• Contact me to enroll.

Exercises

• Traditional
  • Make beforehand, mark as done at the beginning, present when requested.
  • It is not possible to get marks without participating the class.
  • Part are traditional paper-whiteboard exercises, part has to be done using computer, compiled, will be presented using data projector and ssh to cs.
  • At least 1/3 has to be done.
• Obligatory X-exercises
  • 2-4 on key subjects
  • Will be graded (coarsely)
  • Missing (not submitted) will be graded negatively
  • Has to be submitted by email on previous day, lateness will reduce a point.
  • A unique solution for every student (co-operation is allowed, but the same answer from two or more students is not)
  • Automated processing and comparison, strict submitting rules
  • Not counted as traditional exercises when calculating the 1/3 rule.

• X-exercises: 30-40%
• Exams: 60-70%
• Extra points for making more than 1/3 of standard exercises (up to 10% of maximum).
  ((n-N/3)/(2N/3))*(max/10) N is # of std ex, n is # of your marks).
• No rounding.

• Slide copies (updated during the course)
• JaJa: An Introduction to Parallel Algorithms
• Akl: The Design and Analysis of Parallel Algorithms
• Other

• Internet Parallel Computing Archieve
• Nan's Parallel Computing Page
• ParaScope
• www.corba.com/
• BSP Worldwide
• MPI standard (1.1)
• LAM / MPI
• LAM/MPI function bindings
• Beej's Guide to Unix IPC
• Lectures given by Guy Keren
• Tutorial on Unix Multi-Process Programming and Inter-Process Communications (IPC)
• Tutorial on Multi-Threaded Programming With POSIX Threads
• Beej's guide to Network Programming: Using Internet Sockets
• Top 500 supercomputer sites
• Fortran 90: An introduction to the language for beginners (slides)
• Introduction to FORTRAN 90
• g95: free, open source Fortran 95 compiler
• gfortran documentation
• Nvidia CUDA
• OpenMP.org
• Intel Software Network: Getting Started with OpenMP (See also the two Related Links (More Work-Sharing with OpenMP* and Advanced OpenMP* Programming) at the end of article)
• BLAS

Submit your solutions to Moodle 1 hour before the exercise class (09 am).
• Exercise 1 (12.1.)
• Exercise 2 (19.1.)
• Exercise 3 (26.1.)
• Exercise 4 (2.2.) (Includes first obligatory X-task).
• Exercise 5 (8./12.2) (Thu at Joensuu, Mon at Kuopio)
• Exercise 6 (16.2.)
• Exercise 7 (23.2.) (Includes second obligatory X-task).
• Exercise 8 (12.3.) (Includes third obligatory X-task).