dmpar WRF crashes with large model wrfinput when using a serial I/O format

[JTOFH]

In dmpar configuration and when using a serial I/O format, WRF reads in the initial conditions on rank 0 and then distributes the values to the other ranks using an MPI_scatterV call. MPI has an array limit of 2**31-1 (2GB). For any model using REAL*4, the array will overflow above 500M cells.

The overflow beings in frame/module_io.F in the subroutine call_pkg_and_dist_real. This allocates the buffer (globbuf) and then passes it to wrf_scatterv_real (via call_pkg_and_dist_generic). A similar problem would also occur with REAL*8 at half the number of cells through call_pkg_and_dist_double -> call_pkg_and_dist_generic -> wrf_scatterv_double.

This problem was identified in WRF 3.8.1, but appears to be the same limitation in all versions.

[benkirk]

I believe this is similar to an issue encountered in #2156, with a proposed fix in #2157.

I'm not sure if this exact change will fix your issue, the approach should work. In #2156 the offending call was the MPI_Scatterv in frame/collect_on_comm.c