[an error occurred while processing this directive] [an error occurred while processing this directive]
[an error occurred while processing this directive]
[an error occurred while processing this directive]
Monash University

FIT5139 Advanced distributed and parallel systems - Semester 1, 2015

Modern computer systems contain parallelism in both hardware and software. This unit covers parallelism in both general purpose and application specific computer architectures and the programming paradigms that allow parallelism to be exploited in software. This unit examines both shared memory and message passing paradigms in both hardware and software; concurrency, multithreading and synchronicity; parallel, clustered and distributed supercomputing models and languages. Students will program in these paradigms. This unit draws on units in distributed databases and grid computing. It will also cover the technology and application of cloud computing with particular reference to programming frameworks (e.g. MapReduce, Hadoop etc).

Mode of Delivery

Caulfield (Day)

Workload Requirements

Minimum total expected workload equals 12 hours per week comprising:

(a.) Contact hours for on-campus students:

  • Two hours of lectures
  • One 2-hour tutorial

(b.) Additional requirements (all students):

  • A minimum of 8 hours independent study per week for completing tutorial and project work, private study and revision.

See also Unit timetable information

Unit Relationships



Chief Examiner

Campus Lecturer


Asad I. Khan

Consultation hours: By appointment via e-mail: Asad.Khan@monash.edu

Your feedback to Us

Monash is committed to excellence in education and regularly seeks feedback from students, employers and staff. One of the key formal ways students have to provide feedback is through the Student Evaluation of Teaching and Units (SETU) survey. The University’s student evaluation policy requires that every unit is evaluated each year. Students are strongly encouraged to complete the surveys. The feedback is anonymous and provides the Faculty with evidence of aspects that students are satisfied and areas for improvement.

For more information on Monash’s educational strategy, see:

www.monash.edu.au/about/monash-directions/ and on student evaluations, see: www.policy.monash.edu/policy-bank/academic/education/quality/student-evaluation-policy.html

Academic Overview

Learning Outcomes

On successful completion of this unit students should be able to:
  • solve basic problems in distributed computing, especially in relation to synchronisation, distributed transactions, concurrency control, distributed consensus;
  • explain the differences between various distributed computing models and widely used distributed computing schemes;
  • describe a variety of parallel programming paradigms, synchronisation and parallelisation primitives, message passing, data parallel, tuple space;
  • identify computational tasks that benefit from parallelism;
  • design and implement a parallel-distributed software system.

Unit Schedule

Week Activities Assessment
0 Unit Introduction on web (no lecture). Register for lab sessions No formal assessment or activities are undertaken in week 0
1 Introduction to parallel patterns, distributed systems, and important metrics No lab
2 Big data and cloud computing Weekly lab-group submissions from Week 3 to Week 10 (both Weeks inclusive). Lab0: Programming and OS Primer (not graded)
3 Map-reduce applications Lab1: Hadoop system
4 GPU processing model and programming with Open CL Lab2: Hadoop application
5 Parallel processing models and languages Lab3: GPU Programming
6 Message Passage Library (MPI) Lab4: Open MP Exercise
7 Interprocess communication and remote procedure call Lab5: MPI Data Structures 1
8 Synchronisation, MUTEX, Deadlocks Lab6: MPI Data Structures 2
9 Election Algorithms, Distributed Transactions, Concurrency Control Lab7: MPI Data Structures 3
10 Faults, Distributed Consensus, and Security Lab8: MPI Master Slave
11 Instruction Level Parallelism (1) Assignment project written submission in Moodle is due on Mon 18/May, 2PM. (2) Assignment project code demonstrations during Week 11 labs (attendance is essential)
12 Vector and superscaler architectures Assignment project code demonstrations during Week 12 labs (attendance is essential)
  SWOT VAC No formal assessment is undertaken during SWOT VAC
  Examination period LINK to Assessment Policy: http://policy.monash.edu.au/policy-bank/

*Unit Schedule details will be maintained and communicated to you via your learning system.

Teaching Approach

Lecture and tutorials or problem classes
This teaching and learning approach helps students to initially encounter information at lectures, discuss and explore the information during lecture and lab classes, and practice in a hands-on lab environment.

Assessment Summary

Examination (3 hours): 60%, In-semester assessment: 40%

Assessment Task Value Due Date
Assignment Project 25% Code demonstrations during Week 11 or 12 labs. Assignment Project write-up due in Week 11, Mon 18/May, 2PM
The assessed laboratory/tutorial work 12% Weekly group submissions Week 3 to Week 10 (both Weeks inclusive)
Lecture feedback 3% During weekly lectures
Examination 1 60% To be advised

Assessment Requirements

Assessment Policy

Assessment Tasks


  • Assessment task 1
    Assignment Project
    The students will develop parallel program/s that correctly and effieciently meet with the specfied task/s using the programming model/s of their choice. The program/s will be demonstrated in week11 and 12. They will explain the programming methodology and the code in the form of a write-up to be sumbitted in Moodle. The students will demonstrate their practical skills in developing and documenting parallel distributed applications through this assessment.
    Criteria for assessment:

    Individual assessment. This work will be assessed on a mix of programming tasks and theoretical write-up.

    Due date:
    Code demonstrations during Week 11 or 12 labs. Assignment Project write-up due in Week 11, Mon 18/May, 2PM
  • Assessment task 2
    The assessed laboratory/tutorial work
    Assessed laboratory/tutorial work submissions. The code developed during the lab from week 3 to week 10 (both weeks inclusive) will be demonstrated and the results submitted at the end of each laboratory/tutorial session or as advised by the tutor.
    Criteria for assessment:

    The laboratory work is group-based and it is assessed on correctness and on the quality of the solutions and on the quality of presentation/documentation. Individual marks for each group member will be derived from (i) the peer assessements made by the group under the supervision of the tutor and (ii) the overall marks achieved by the group.

    The students will email their group's working to their respective tutor at the end of each lab or as specified by the tutor.

    Due date:
    Weekly group submissions Week 3 to Week 10 (both Weeks inclusive)
  • Assessment task 3
    Lecture feedback
    Students will be expected to remain engaged during all the lectures. The engagment will be measured by occasional feedback exercises, where students will be asked to discuss selected concepts covered in the current or previous lectures.
    Criteria for assessment:

    This is an individual assessment, which aims to gauge a student's level of engagement in this unit over the semester. Regular lecture attendance and participation in Q&A will be the two key indicators for measuring engagement and assigning credit.

    Due date:
    During weekly lectures


  • Examination 1
    3 hours
    Type (open/closed book):
    Electronic devices allowed in the exam:

Learning resources

Reading list

A. H. Amin, A.I. Khan, B.B. Nasution: Internet-Scale Pattern Recognition: New Techniques for Voluminous Data Sets and Data Clouds, Chapman and Hall/CRC Press, ISBN-13: 978-1466510968, 2012.

A.S. Tanenbaum, T. Austin: Structured Computer Organization, 6th Ed, Prentice Hall (PEARSON), 2012.

G.R. Andrews: Foundations of Multithreaded, Parallel and Distributed Programming, Addison-Wesley, 2000.

Advanced Computer Architectures: A Design Space Approach, Sima, Fountain and Kacsuk , Addison Wesley Publishers.

W. Stallings: Computer Organization Architecture, 9th Ed, Prentice Hall (Pearson Hall), 2013.

Monash Library Unit Reading List (if applicable to the unit)

Feedback to you

Types of feedback you can expect to receive in this unit are:

  • Informal feedback on progress in labs/tutes
  • Graded assignments with comments

Extensions and penalties

Returning assignments

Assignment submission

It is a University requirement (http://www.policy.monash.edu/policy-bank/academic/education/conduct/student-academic-integrity-managing-plagiarism-collusion-procedures.html) for students to submit an assignment coversheet for each assessment item. Faculty Assignment coversheets can be found at http://www.infotech.monash.edu.au/resources/student/forms/. Please check with your Lecturer on the submission method for your assignment coversheet (e.g. attach a file to the online assignment submission, hand-in a hard copy, or use an electronic submission). Please note that it is your responsibility to retain copies of your assessments.

Online submission

If Electronic Submission has been approved for your unit, please submit your work via the learning system for this unit, which you can access via links in the my.monash portal.

Other Information


Monash has educational policies, procedures and guidelines, which are designed to ensure that staff and students are aware of the University’s academic standards, and to provide advice on how they might uphold them. You can find Monash’s Education Policies at: www.policy.monash.edu.au/policy-bank/academic/education/index.html

Faculty resources and policies

Important student resources including Faculty policies are located at http://intranet.monash.edu.au/infotech/resources/students/

Graduate Attributes Policy

Student Charter

Student services

Monash University Library

Disability Liaison Unit

Students who have a disability or medical condition are welcome to contact the Disability Liaison Unit to discuss academic support services. Disability Liaison Officers (DLOs) visit all Victorian campuses on a regular basis.

[an error occurred while processing this directive]