Introduction

Overview

Teaching: 20 min
Exercises: 0 min

Questions

• How shared memory parallel programs work?

• What is OpenMP?

• How to write and compile parallel programs in C?

Objectives

• Understand the shared memory programming environment provided by OpenMP

• Learn how to write and compile simple parallel programs in C

Shared Memory OpenMP Programming Overview

• The OpenMP standard outlines how parallel programs should be written for shared memory systems.

• The majority of compilers support OpenMP.

View OpenMP specifications
View Full List of Compilers supporting OpenMP
For an overview of the past, present and future of the OpenMP read the paper “The Ongoing Evolution of OpenMP”.

OpenMP Execution Model

• OpenMP programs realize parallelism through the use of threads.

• Parallel OpenMP execution is based on a concept called fork-join:

  1. An OpenMP program starts as a single thread, the master thread.
  2. The master thread creates a team of parallel threads.
  3. Team threads execute statements in parallel.
  4. Team threads synchronize and terminate.

• Memory is categorized into two types: shared and thread-local.

Compiling OpenMP Programs

Create a working directory in ~/scratch:

cd ~/scratch

Code examples on the training cluster can be obtained by following these steps:

cp /tmp/omp_workshop_2024.tar .
tar -xf omp_workshop_2024.tar

If you are using a real cluster, you can download examples from the web:

wget https://github.com/acenet-arc/ACENET_Summer_School_OpenMP_ACC/raw/gh-pages/code/omp_workshop_2024.tar 
tar -xf omp_workshop_2024.tar

A Very Quick Introduction to C

Preprocessor Directives

• The #include directive inserts the contents of a file.

#include <header_file.h> 

• The #define directive declares constant values.

# define SIZE 1000

• The conditional group #ifdef.

#ifdef MACRO
controlled code
#endif 

Basic Syntax

• { ... } Curly braces are used to group statements into blocks.
• ; The semicolon operator is used to separate statements

• , The comma operator separates expressions (which have value)

• For Loop

  for ([initialization]; [condition test]; [increment or decrement])
  {
         //Statements to be executed repeatedly
  }
  

• Example for loop:

  int i,c;
  for(i=10,c=1;i<100;i++) {
    c += 2*i; }
  

• Any statements in a for loop can be skipped, but semicolons must remain. For example, you can do initialization before loop:

  i=10, c=1;
  for(; i<100; i++) {
    c += 2*i; }
  

• Skipping all loop statements will result in an infinite loop: for(;;)

Functions

• The format of function definitions is as follows:

  return_type function_name( parameters ) {
      body
  }
  

• Here is an example of a function that finds the maximum of two numbers:

  int max(int num1, int num2) {
    int result;
    if (num1 > num2)
      result = num1;
    else
      result = num2;
    return result;
  }
  

Pointers

• Pointers are special variables used to store addresses of variables rather than values.

• The & operator (reference) returns the variable’s address: address of A is &A.

• The * symbol has two meanings
  1. The * operator (dereference) allows you to access an object through a pointer. *X, for example, will give you the value of variable X if X is the memory address where it is stored.
  2. In the statement float* my_array; the variable my_array; is defined as a pointer to a location in memory containing floating-point numbers.

Arrays

• Static arrays.

int A[500]; 

• Dynamic arrays.

int * A;  
size = 500; // Define array size
A = (int *)malloc(size*sizeof(int)); // Allocate memory
A = (int *)realloc(A,1000);  // Increase size
A[i]=10;  // Set array elements 
free(A); // Free memory 

Compiling C Code

Using C compilers on Compute Canada Systems

• Currently the default environment on all general purpose clusters (Beluga, Cedar, Graham, Narval) is StdEnv/2023.
• The default compilers available in this environment on Graham and Beluga are intel/2023.2.1 and gcc/12.3.
• To load another compiler you can use the command module load. For example, the command to load gcc/10.3.0 is:

module load StdEnv/2020 gcc/10.3.0

Training cluster

The default environment on the training cluster is StdEnv/2020. We will use gcc/9.3.0

module load StdEnv/2020 gcc/9.3.0

• Intel compilers are not available on the traiining cluster

The following is an example of a simple hello world program written in C.

/* --- File hello_serial.c --- */
#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv) {
  printf("Hello World\n");
}

In order to compile this code, you would need to use the following command:

gcc -o hello hello_serial.c

This gives you an executable file hello that will print out the text “Hello World”. You run it with the command:

./hello

Hello World

• If you don’t specify the output filename with the -o option compiler will use the default output name a.out (assembler output).

Key Points

• Shared-memory parallel programs break up large problems into a number of smaller ones and execute them simultaneously

• OpenMP programs are limited to a single physical machine

• OpenMP libraries are built into all commonly used C, C++, or Fortran compilers