Lesson 2: shmem_put
Ok. So we now have a way to grab data from another pe. But, we need a way to put data onto other pes. And one question pops out in my mind: can we do so without shaking hands prior?Similar to shmem_get, the put command allows a pe to put data onto other pes without receiving permission. BUT, with much freedom comes much responsibility ... We must be careful that we don't overwrite data that is needed - in other words, we need to avoid race conditions, and this is where you and I come in as programmers.
Today, we are only going to cover three of the basic put commands:
- copying only one element onto another pe
- copying an array of elements onto another pe
- copying strided data onto another pe
---Copying only one element onto another pe---
To do so, we use the following command:void shmem_<TYPENAME>_p(TYPE *dest, TYPE value, int pe);
- arg0 - *dest: a pointer to an element; this element must be allocated in the shared memory space of each PE.
- arg1 - value: the value to be placed at *dest; can be either locally or globally defined memory
- arg2 - pe: the PE on which the value is to be placed.
- returns - none
| TYPE | TYPENAME |
|---|---|
| short | short |
| int | int |
| long | long |
| long long | longlong |
| unsigned short | ushort |
| unsigned int | uint |
| unsigned long | ulong |
| unsigned long long | ulonglong |
| int32_t | int32 |
| int64_t | int64 |
| uint32_t | uint32 |
| uint64_t | uint64 |
| size_t | size |
| ptrdiff_t | ptrdiff |
//put_data_ex0.c
#include <stdio.h>
#include <shmem.h>
int main(){
int my_pe, num_pe; //declare variables for both pe id of processor and the number of pes
shmem_init();
num_pe = shmem_n_pes(); //obtain the number of pes that can be used
my_pe = shmem_my_pe(); //obtain the pe id number
int pe_source = 0; //pe where the desired data is residing
int pe_dest = 1;
double source = 1;
static double dest; //variable to be filled
//if main node, put data onto destination node
if (my_pe == pe_source){
shmem_double_p(&dest, source, pe_dest);
}
shmem_barrier_all();
if (my_pe == pe_dest){
printf("%d: Data at dest is now %f\n", my_pe, dest);
}
shmem_finalize();
return 0;
}To compile:
oshcc put_data_ex0.c -o put_data_ex0
To run:
oshrun -np 10 put_data_ex0
Lets first take a look at the code
if (my_pe == pe_source){
shmem_double_p(&dest, source, pe_dest);
}
Note that PE 1 skips this command and would just go on to the next. So, what would happen
then if the next command was to print the data? Do you think the correct data would be
there yet? Unless PE 1 is super lazy and moving ultra slow, the answer is no. The data it
would print would be the data from of old. Haha. I just imagine an assembly line in a
cartoon with Tom and Jerry. Tom (the cat) would just be redoing his work over and over
again --- Jerry in this instance would probably be slow just to spite Tom.
So, how do we ensure that PE 1 (or Tom) has the correct piece? This is where a sync comes into play. In the above, I used the most basic: shmem_barrier_all(), which makes all the PEs stop until both the local and remote data are in their proper locations. But, you are probably already asking as I was, "wait. What about the scenario where you only want to wait for a certain PE's data to finish moving? Or what if you don't care if all of the data is in the correct location before moving on?" Well, thankfully, there are commands that cover these scenarios and more. The next lesson will go into more detail on these.
---Copying an array of elements onto another pe---
I don't know about you, but memorization is not my forte. And my short term memory is very similar to Dory's from Finding Nemo. But, this is where I am super thankful for understandable commands and similar commands!!!!!!Think back to the last lesson where we learned shmem_g and shmem_get. Notice that for only one element, there is only one single letter "g"; whereas for the grabbing an array of elements we use the whole word "get". Well, thankfully, we are in a similar situation. To put an array of elements onto another pe, the whole word "put" is used and the arguments are identical to that of the "get" command.
To put an array of elements onto another pe, we can use one of the two following commands:
- shmem_<TYPENAME>_put(TYPE *dest, const TYPE *source, size_t nelems, int pe);
- arg0 - *dest: a pointer to an array of elements; this array must be allocated in the shared memory space of each PE.
- arg1 - *source: the pointer to an array of values to be placed at *dest; can be either locally or globally defined memory
- arg2 - nelems: the number of elements in the array.
- arg3 - pe: the PE on which the value is to be placed.
- returns - none
- shmem_put<SIZE>(void *dest, const void *source, size_t nelems, int pe)
- SIZE: number of bits
- arg0 - *dest: a pointer to an array of elements; this array must be allocated in the shared memory space of each PE.
- arg1 - *source: the pointer to an array of values to be placed at *dest; can be either locally or globally defined memory
- arg2 - nelems: the number of elements in the array.
- arg3 - pe: the PE on which the value is to be placed.
- returns - none
shmem_put(TYPE *dest, const TYPE *source, size_t nelems, int pe) To be able to use the above, you need a C11 compiler. In that case, you can use the shortened version of all of the commands (just eliminate the TYPENAME or SIZE).
//put_data_ex1.c
#include <stdio.h>
#include <shmem.h>
int main(){
int my_pe, num_pe; //declare variables for both pe id of processor
//and the number of pes
shmem_init();
num_pe = shmem_n_pes(); //obtain the number of pes that can be used
my_pe = shmem_my_pe(); //obtain the pe id number
int pe_source = 0; //pe where the desired data is residing
int pe_dest = 1;
double source[10] = {1,2,3,4,5,6,7,8,9,10};
static double dest[10]; //array to be filled
//if main node, put data onto destination node
if (my_pe == pe_source){
shmem_double_put(dest, source, 10, pe_dest);
}
shmem_barrier_all();
if (my_pe == pe_dest){
printf("%d: Data at dest is now ", my_pe);
for (int i=0; i<10; i++)
printf(" %d," dest[i]);
printf("\n");
}
shmem_finalize();
return 0;
}To compile:
oshcc put_data_ex1.c -o put_data_ex1
To run:
oshrun -np 10 put_data_ex1