SCOREC core
Parallel unstructured mesh tools
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Macros Pages
Functions
pcu.c File Reference

The PCU communication interface. More...

Go to the source code of this file.

Functions

int PCU_Comm_Init (void)
 Initializes the PCU library. More...
 
int PCU_Comm_Free (void)
 Frees all PCU library structures. More...
 
int PCU_Comm_Self (void)
 Returns the communication rank of the calling thread. More...
 
int PCU_Comm_Peers (void)
 Returns the number of threads in the program. More...
 
void PCU_Comm_Begin (void)
 Begins a PCU communication phase. More...
 
int PCU_Comm_Pack (int to_rank, const void *data, size_t size)
 Packs data to be sent to to_rank. More...
 
int PCU_Comm_Send (void)
 Sends all buffers for this communication phase. More...
 
bool PCU_Comm_Listen (void)
 Tries to receive a buffer for this communication phase. More...
 
int PCU_Comm_Sender (void)
 Returns in * from_rank the sender of the current received buffer. More...
 
bool PCU_Comm_Unpacked (void)
 Returns true if the current received buffer has been unpacked. More...
 
int PCU_Comm_Unpack (void *data, size_t size)
 Unpacks a block of data from the current received buffer. More...
 
void PCU_Barrier (void)
 Blocking barrier over all threads.
 
void PCU_Add_Doubles (double *p, size_t n)
 Performs an Allreduce sum of double arrays. More...
 
void PCU_Min_Doubles (double *p, size_t n)
 Performs an Allreduce minimum of double arrays.
 
void PCU_Max_Doubles (double *p, size_t n)
 Performs an Allreduce maximum of double arrays.
 
void PCU_Add_Ints (int *p, size_t n)
 Performs an Allreduce sum of integers.
 
void PCU_Add_Longs (long *p, size_t n)
 Performs an Allreduce sum of long integers.
 
void PCU_Add_SizeTs (size_t *p, size_t n)
 Performs an Allreduce sum of size_t unsigned integers.
 
void PCU_Min_SizeTs (size_t *p, size_t n)
 Performs an Allreduce minimum of size_t unsigned integers.
 
void PCU_Max_SizeTs (size_t *p, size_t n)
 Performs an Allreduce maximum of size_t unsigned integers.
 
void PCU_Exscan_Ints (int *p, size_t n)
 Performs an exclusive prefix sum of integer arrays. More...
 
void PCU_Exscan_Longs (long *p, size_t n)
 See PCU_Exscan_Ints.
 
void PCU_Min_Ints (int *p, size_t n)
 Performs an Allreduce minimum of int arrays.
 
void PCU_Max_Ints (int *p, size_t n)
 Performs an Allreduce maximum of int arrays.
 
int PCU_Or (int c)
 Performs a parallel logical OR reduction.
 
int PCU_And (int c)
 Performs a parallel logical AND reduction.
 
int PCU_Proc_Self (void)
 Returns the unique rank of the calling process.
 
int PCU_Proc_Peers (void)
 Returns the number of processes.
 
int PCU_Comm_Rank (int *rank)
 Similar to PCU_Comm_Self, returns the rank as an argument.
 
int PCU_Comm_Size (int *size)
 Similar to PCU_Comm_Peers, returns the size as an argument.
 
bool PCU_Comm_Initialized (void)
 Returns true iff PCU has been initialized.
 
int PCU_Comm_Start (PCU_Method method)
 Deprecated, see PCU_Comm_Begin.
 
int PCU_Comm_Packed (int to_rank, size_t *size)
 Returns in * size the number of bytes being sent to to_rank. More...
 
int PCU_Comm_Write (int to_rank, const void *data, size_t size)
 Packs a message to be sent to to_rank. More...
 
bool PCU_Comm_Receive (void)
 Convenience wrapper over Listen and Unpacked.
 
bool PCU_Comm_Read (int *from_rank, void **data, size_t *size)
 Receives a message for this communication phase. More...
 
void PCU_Debug_Print (const char *format,...)
 like fprintf, contents go to debugN.txt
 
int PCU_Comm_From (int *from_rank)
 Similar to PCU_Comm_Sender, returns the rank as an argument.
 
int PCU_Comm_Received (size_t *size)
 Returns in * size the bytes in the current received buffer. More...
 
void * PCU_Comm_Extract (size_t size)
 Extracts a block of data from the current received buffer. More...
 
void PCU_Switch_Comm (MPI_Comm new_comm)
 Reinitializes PCU with a new MPI communicator. More...
 
MPI_Comm PCU_Get_Comm (void)
 Return the current MPI communicator. More...
 
double PCU_Time (void)
 Return the time in seconds since some time in the past.
 

Detailed Description

The PCU communication interface.

Definition in file pcu.c.

Function Documentation

void PCU_Add_Doubles ( double *  p,
size_t  n 
)

Performs an Allreduce sum of double arrays.

This function must be called by all ranks at the same time. p must point to an array of n doubles. After this call, p[i] will contain the sum of all p[i]'s given by each rank.

Definition at line 256 of file pcu.c.

257 {
258  if (global_state == uninit)
259  reel_fail("Add_Doubles called before Comm_Init");
260  pcu_allreduce(&(get_msg()->coll),pcu_add_doubles,p,n*sizeof(double));
261 }
void PCU_Comm_Begin ( void  )

Begins a PCU communication phase.

This function must be called by all threads in the MPI job at the beginning of each phase of communication. After calling this function, each thread may call functions like PCU_Comm_Pack or PCU_Comm_Write.

Definition at line 122 of file pcu.c.

123 {
124  if (global_state == uninit)
125  reel_fail("Comm_Begin called before Comm_Init");
126  pcu_msg_start(get_msg());
127 }
void* PCU_Comm_Extract ( size_t  size)

Extracts a block of data from the current received buffer.

This function should be called after a successful PCU_Comm_Receive. The next size bytes of the current received buffer are unpacked, and an internal pointer to that data is returned. The returned pointer must not be freed by the user.

Definition at line 707 of file pcu.c.

708 {
709  if (global_state == uninit)
710  reel_fail("Comm_Extract called before Comm_Init");
711  pcu_msg* m = get_msg();
712  if (m->order)
713  return pcu_order_unpack(m->order,size);
714  return pcu_msg_unpack(m,size);
715 }
int PCU_Comm_Free ( void  )

Frees all PCU library structures.

This function must be called by all MPI processes after all other calls to PCU, and before calling MPI_Finalize.

Definition at line 77 of file pcu.c.

78 {
79  if (global_state == uninit)
80  reel_fail("Comm_Free called before Comm_Init");
81  if (global_pmsg.order)
82  pcu_order_free(global_pmsg.order);
83  pcu_free_msg(&global_pmsg);
84  pcu_pmpi_finalize();
85  global_state = uninit;
86  return PCU_SUCCESS;
87 }
int PCU_Comm_Init ( void  )

Initializes the PCU library.

This function must be called by all MPI processes before calling any other PCU functions. MPI_Init or MPI_Init_thread should be called before this function.

Definition at line 58 of file pcu.c.

59 {
60  if (global_state != uninit)
61  reel_fail("nested calls to Comm_Init");
62  pcu_pmpi_init(MPI_COMM_WORLD);
63  pcu_set_mpi(&pcu_pmpi);
64  pcu_make_msg(&global_pmsg);
65  global_state = init;
66  /* turn ordering on by default, call
67  PCU_Comm_Order(false) after PCU_Comm_Init
68  to disable this */
69  PCU_Comm_Order(true);
70  return PCU_SUCCESS;
71 }
bool PCU_Comm_Listen ( void  )

Tries to receive a buffer for this communication phase.

Either this function or PCU_Comm_Read should be called at least once by all threads during the communication phase, after PCU_Comm_Send is called. The result will be false if and only if the communication phase is over and there are no more buffers to receive. Otherwise, a buffer was received. Its contents are retrievable through PCU_Comm_Unpack, and its metadata through PCU_Comm_Sender and PCU_Comm_Received. Users should unpack all data from this buffer before calling this function again, because the previously received buffer is destroyed by the call.

Definition at line 171 of file pcu.c.

172 {
173  if (global_state == uninit)
174  reel_fail("Comm_Listen called before Comm_Init");
175  pcu_msg* m = get_msg();
176  if (m->order)
177  return pcu_order_receive(m->order, m);
178  return pcu_msg_receive(m);
179 }
int PCU_Comm_Pack ( int  to_rank,
const void *  data,
size_t  size 
)

Packs data to be sent to to_rank.

This function appends the block of size bytes starting at data to the buffer being sent to to_rank. This function should be called after PCU_Comm_Start and before PCU_Comm_Send.

Definition at line 135 of file pcu.c.

136 {
137  if (global_state == uninit)
138  reel_fail("Comm_Pack called before Comm_Init");
139  if ((to_rank < 0)||(to_rank >= pcu_mpi_size()))
140  reel_fail("Invalid rank in Comm_Pack");
141  memcpy(pcu_msg_pack(get_msg(),to_rank,size),data,size);
142  return PCU_SUCCESS;
143 }
int PCU_Comm_Packed ( int  to_rank,
size_t *  size 
)

Returns in * size the number of bytes being sent to to_rank.

Returns the size of the buffer being sent to to_rank. This function should be called after PCU_Comm_Start and before PCU_Comm_Send.

Definition at line 546 of file pcu.c.

547 {
548  if (global_state == uninit)
549  reel_fail("Comm_Packed called before Comm_Init");
550  if ((to_rank < 0)||(to_rank >= pcu_mpi_size()))
551  reel_fail("Invalid rank in Comm_Packed");
552  *size = pcu_msg_packed(get_msg(),to_rank);
553  return PCU_SUCCESS;
554 }
int PCU_Comm_Peers ( void  )

Returns the number of threads in the program.

when called from a non-threaded MPI process, this function is equivalent to MPI_Comm_size(MPI_COMM_WORLD,size).

Definition at line 109 of file pcu.c.

110 {
111  if (global_state == uninit)
112  reel_fail("Comm_Peers called before Comm_Init");
113  return pcu_mpi_size();
114 }
bool PCU_Comm_Read ( int *  from_rank,
void **  data,
size_t *  size 
)

Receives a message for this communication phase.

This function tries to receive a message packed by PCU_Comm_Write. If a the communication phase is over and there are no more messages to receive, this function returns false. Otherwise, * from_rank will be the rank which sent the message, data will point to the start of the message data, and size will be the number of bytes of message data. If this function is used, PCU_Comm_Receive should not be used. Note that the address * data points into a PCU buffer, so it is strongly recommended that this data be read and not modified.

Definition at line 598 of file pcu.c.

599 {
600  if (global_state == uninit)
601  reel_fail("Comm_Read called before Comm_Init");
602  if (!PCU_Comm_Receive())
603  return false;
604  *from_rank = PCU_Comm_Sender();
605  PCU_COMM_UNPACK(*size);
606  *data = PCU_Comm_Extract(*size);
607  return true;
608 }
PCU_Comm_Receive
bool PCU_Comm_Receive(void)
Convenience wrapper over Listen and Unpacked.
Definition: pcu.c:578
PCU_Comm_Extract
void * PCU_Comm_Extract(size_t size)
Extracts a block of data from the current received buffer.
Definition: pcu.c:707
PCU_Comm_Sender
int PCU_Comm_Sender(void)
Returns in * from_rank the sender of the current received buffer.
Definition: pcu.c:184
int PCU_Comm_Received ( size_t *  size)

Returns in * size the bytes in the current received buffer.

This function should be called after a successful PCU_Comm_Receive. The size returned will be the total received size regardless of how much unpacking has been done.

Definition at line 689 of file pcu.c.

690 {
691  if (global_state == uninit)
692  reel_fail("Comm_Received called before Comm_Init");
693  pcu_msg* m = get_msg();
694  if (m->order)
695  *size = pcu_order_received_size(m->order);
696  else
697  *size = pcu_msg_received_size(m);
698  return PCU_SUCCESS;
699 }
int PCU_Comm_Self ( void  )

Returns the communication rank of the calling thread.

when called from a non-threaded MPI process, this function is equivalent to MPI_Comm_rank(MPI_COMM_WORLD,rank).

Ranks are consecutive from 0 to $pt-1$ for a program with $p$ processes and $t$ threads per process. Ranks are contiguous within a process, so that the $t$ threads in process $i$ are numbered from $ti$ to $ti+t-1$.

Definition at line 98 of file pcu.c.

99 {
100  if (global_state == uninit)
101  reel_fail("Comm_Self called before Comm_Init");
102  return pcu_mpi_rank();
103 }
int PCU_Comm_Send ( void  )

Sends all buffers for this communication phase.

This function should be called by all threads in the MPI job after calls to PCU_Comm_Pack or PCU_Comm_Write and before calls to PCU_Comm_Listen or PCU_Comm_Read. All buffers from this thread are sent out and receiving may begin after this call.

Definition at line 152 of file pcu.c.

153 {
154  if (global_state == uninit)
155  reel_fail("Comm_Send called before Comm_Init");
156  pcu_msg_send(get_msg());
157  return PCU_SUCCESS;
158 }
int PCU_Comm_Sender ( void  )

Returns in * from_rank the sender of the current received buffer.

This function should be called after a successful PCU_Comm_Listen.

Definition at line 184 of file pcu.c.

185 {
186  if (global_state == uninit)
187  reel_fail("Comm_Sender called before Comm_Init");
188  pcu_msg* m = get_msg();
189  if (m->order)
190  return pcu_order_received_from(m->order);
191  return pcu_msg_received_from(m);
192 }
int PCU_Comm_Unpack ( void *  data,
size_t  size 
)

Unpacks a block of data from the current received buffer.

This function should be called after a successful PCU_Comm_Listen. data must point to a block of memory of at least size bytes, into which the next size bytes of the current received buffer will be written. Subsequent calls will begin unpacking where this call left off, so that the entire received buffer can be unpacked by a sequence of calls to this function. Users must ensure that there remain size bytes to be unpacked, PCU_Comm_Unpacked can help with this.

Definition at line 217 of file pcu.c.

218 {
219  if (global_state == uninit)
220  reel_fail("Comm_Unpack called before Comm_Init");
221  pcu_msg* m = get_msg();
222  if (m->order)
223  memcpy(data,pcu_order_unpack(m->order,size),size);
224  else
225  memcpy(data,pcu_msg_unpack(m,size),size);
226  return PCU_SUCCESS;
227 }
bool PCU_Comm_Unpacked ( void  )

Returns true if the current received buffer has been unpacked.

This function should be called after a successful PCU_Comm_Listen.

Definition at line 197 of file pcu.c.

198 {
199  if (global_state == uninit)
200  reel_fail("Comm_Unpacked called before Comm_Init");
201  pcu_msg* m = get_msg();
202  if (m->order)
203  return pcu_order_unpacked(m->order);
204  return pcu_msg_unpacked(m);
205 }
int PCU_Comm_Write ( int  to_rank,
const void *  data,
size_t  size 
)

Packs a message to be sent to to_rank.

This function packs a message into the buffer being sent to to_rank. Messages packed by this function can be received using the function PCU_Comm_Read. This function should be called after PCU_Comm_Start and before PCU_Comm_Send. If this function is used, PCU_Comm_Pack should not be used.

Definition at line 565 of file pcu.c.

566 {
567  if (global_state == uninit)
568  reel_fail("Comm_Write called before Comm_Init");
569  if ((to_rank < 0)||(to_rank >= pcu_mpi_size()))
570  reel_fail("Invalid rank in Comm_Write");
571  pcu_msg* msg = get_msg();
572  PCU_MSG_PACK(msg,to_rank,size);
573  memcpy(pcu_msg_pack(msg,to_rank,size),data,size);
574  return PCU_SUCCESS;
575 }
void PCU_Exscan_Ints ( int *  p,
size_t  n 
)

Performs an exclusive prefix sum of integer arrays.

This function must be called by all ranks at the same time. p must point to an array of n integers. After this call, p[i] will contain the sum of all p[i]'s given by ranks lower than the calling rank.

Definition at line 392 of file pcu.c.

393 {
394  if (global_state == uninit)
395  reel_fail("Exscan_Ints called before Comm_Init");
396  int* originals;
397  NOTO_MALLOC(originals,n);
398  for (size_t i=0; i < n; ++i)
399  originals[i] = p[i];
400  pcu_scan(&(get_msg()->coll),pcu_add_ints,p,n*sizeof(int));
401  //convert inclusive scan to exclusive
402  for (size_t i=0; i < n; ++i)
403  p[i] -= originals[i];
404  noto_free(originals);
405 }
MPI_Comm PCU_Get_Comm ( void  )

Return the current MPI communicator.

Returns the communicator given to the most recent PCU_Switch_Comm call, or MPI_COMM_WORLD otherwise.

Definition at line 736 of file pcu.c.

737 {
738  if (global_state == uninit)
739  reel_fail("Get_Comm called before Comm_Init");
740  return pcu_pmpi_comm();
741 }
void PCU_Switch_Comm ( MPI_Comm  new_comm)

Reinitializes PCU with a new MPI communicator.

All of PCU's logic is based off two duplicates of this communicator, so you can safely get PCU to act on sub-groups of processes using this function. This call should be collective over all processes in the previous communicator.

Definition at line 724 of file pcu.c.

725 {
726  if (global_state == uninit)
727  reel_fail("Switch_Comm called before Comm_Init");
728  pcu_pmpi_switch(new_comm);
729 }
Generated for SCOREC core by doxygen 1.8.5