Skip to content

Decompose across panels #220

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
thomasmelvin wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Decompose across panels #220

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
daf65ec
Copy of core code for decomposing across panels from srs lfric_core#4699
thomasmelvin Jan 6, 2026
5781c76
Remove accidental changes to wqtrunk code
thomasmelvin Jan 6, 2026
63a802d
Fixes for style checker
thomasmelvin Jan 7, 2026
5345287
Update contributers
thomasmelvin Jan 15, 2026
8ec2d52
Merge branch 'main' into decompose_across_panels
thomasmelvin Jan 16, 2026
38b5741
Addition of rose stem tests for new decomposiiton strategy
thomasmelvin Jan 19, 2026
d297a80
add in kgos
thomasmelvin Jan 19, 2026
2748b11
Merge branch 'decompose_across_panels' of github.com:thomasmelvin/lfr…
thomasmelvin Jan 19, 2026
f7dfa28
Add some documentation for the new partitioning
thomasmelvin Jan 20, 2026
2c1af4f
Merge branch 'main' into decompose_across_panels
thomasmelvin Jan 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CONTRIBUTORS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,4 @@
| harry-shepherd | Harry Shepherd | Met Office | 2026-01-08 |
| EdHone | Ed Hone | Met Office | 2026-01-09 |
| tom-j-h | Tom Hill | Met Office | 2026-01-19 |
| thomasmelvin | Thomas Melvin | Met Office | 2026-01-15 |
6 changes: 3 additions & 3 deletions components/driver/source/mesh/runtime_partition_mod.f90
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,15 +78,15 @@ subroutine get_partition_strategy( mesh_selection, total_ranks, partitioner_ptr
call log_event( "Using serial cubed sphere partitioner", &
log_level_debug )

else if (mod(total_ranks,6) == 0) then
else if (mod(total_ranks,3) == 0 .or. mod(total_ranks,2) == 0) then
! Paralled run job
partitioner_ptr => partitioner_cubedsphere
call log_event( "Using parallel cubed sphere partitioner", &
log_level_debug )

else
call log_event( "Total number of processors must be 1 (serial) "// &
"or a multiple of 6 for a cubed-sphere domain.", &
call log_event( "Total number of processors must be 1 (serial) "// &
"or a multiple of 2 or 3 for a cubed-sphere domain.", &
log_level_error )
end if

Expand Down
8 changes: 6 additions & 2 deletions documentation/source/how_to_use_it/technical_articles/lfric_distmem_impl.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -177,8 +177,12 @@ supplied to the partitioner through the use of a function passed in via
a function pointer.

For efficiency reasons, the algorithm for partitioning a cubed-sphere
mesh is a specific algorithm that will only work for a cubed-sphere mesh
(each panel of the cubed-sphere is split into rectangular partitions).
mesh is a specific algorithm that will only work for a cubed-sphere mesh.
By default each panel of the cubed-sphere is split into rectangular partitions,
alternatively if the 'custom' decomposition type is used then sets of 2 or 3
cubed-sphere panels can be grouped together before these sets are decomposed into
a number of rectangular panels. These restrictions mean that the total number of
partitions for cubed-sphere meshes needs to be a multiple of 2 or 3.
This specific partitioner could be replaced with a more general
partitioner for use on fully unstructured meshes.

Expand Down
95 changes: 60 additions & 35 deletions infrastructure/source/mesh/global_mesh_mod.F90
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1019,79 +1019,104 @@ end function get_equatorial_latitude
!>
!> @note For a cubed -sphere mesh, this will only return correct cell IDs if
!> the offset cell remains on same the cubed-sphere "face" as the
!> start cell.
!> start cell unless check_orientation is specified.
!>
!> @param[in] cell_number ID of the anchor element.
!> @param[in] x_cells Offset in the E/W direction.
!> @param[in] y_cells Offset in the N/S direction.
!> @param[in] check_orientation Switch to check for orientation changes
!! such as when crossing panel boundaries
!>
!> @return cell_id ID of the cell at the given offset to the start cell.
!>
function get_cell_id( self, cell_number, &
x_cells, y_cells ) result ( cell_id )
x_cells, y_cells, &
check_orientation ) result ( cell_id )

use reference_element_mod, only : W, S, E, N

implicit none

class(global_mesh_type), intent(in) :: self

integer(i_def), intent(in) :: cell_number
integer(i_def), intent(in) :: x_cells, y_cells
integer(i_def), intent(in) :: cell_number
integer(i_def), intent(in) :: x_cells, y_cells
logical(l_def), optional, intent(in) :: check_orientation

integer(i_def) :: cell_id
integer(i_def) :: cell_id, old_cell_id

integer(i_def) :: index_x, dist_x
integer(i_def) :: index_y, dist_y
integer(i_def) :: i
integer(i_def) :: x_index, y_index, x_dist, y_dist, i, j
integer(i_def) :: opposite(4), rotate(4)

cell_id = cell_number
logical(l_def) :: check

! Determine march along local x-axis
if (x_cells > 0) then
index_x = E
dist_x = x_cells
else if (x_cells < 0) then
index_x = W
dist_x = abs(x_cells)
opposite = (/ E, N, W, S /)
rotate = (/ S, E, N, W /)
if ( present( check_orientation ) ) then
check = check_orientation
else
index_x = W
dist_x = 0
check = .false.
end if

! Determine march along local y-axis
if (y_cells > 0) then
index_y = N
dist_y = y_cells
else if (y_cells < 0) then
index_y = S
dist_y = abs(y_cells)
cell_id=cell_number

if (x_cells >= 0 )then
x_index = E
x_dist = x_cells
else
x_index = W
x_dist = abs(x_cells)
end if
if (y_cells >= 0 )then
y_index = N
y_dist = y_cells
else
index_y = S
dist_y = 0
y_index = S
y_dist = abs(y_cells)
end if

!========================================
! March from anchor along local x/y axes.
!========================================
do i=1, dist_x
! x_dist cells in the x-direction
do i = 1, x_dist
old_cell_id = cell_id
cell_id = self%cell_next_2d(x_index,cell_id)
if (cell_id == self%void_cell) then
! The current cell is not on the domain
write(log_scratch_space,'(A)') &
'No adjacent cell, the current cell is not on mesh domain'
call log_event(log_scratch_space, LOG_LEVEL_ERROR)
end if
cell_id = self%cell_next_2d(index_x, cell_id)
! Check if we've changed direction
if ( check .and. self%cell_next_2d(opposite(x_index),cell_id) /= old_cell_id ) then
! We have changed direction so we need to find the correct
! index and reset
do j = 1,4
if ( self%cell_next_2d(opposite(j),cell_id) == old_cell_id ) x_index = j
end do
end if
end do

do i=1, dist_y
! y_dist cells in the y-direction
! Since the direction may have changed we need to recompute
y_index = rotate(x_index)
if ( y_cells < 0 ) y_index = opposite(y_index)

! y_index and y_dist
do i = 1,y_dist
old_cell_id = cell_id
cell_id = self%cell_next_2d(y_index,cell_id)
if (cell_id == self%void_cell) then
! The current cell is not on the domain
write(log_scratch_space,'(A)') &
'No adjacent cell, the current cell is not on mesh domain'
call log_event(log_scratch_space, LOG_LEVEL_ERROR)
end if
cell_id = self%cell_next_2d(index_y, cell_id)
! Check if we've changed direction
if ( check .and. self%cell_next_2d(opposite(y_index),cell_id) /= old_cell_id ) then
! We have changed direction so we need to find the correct
! index and reset
do j = 1,4
if ( self%cell_next_2d(opposite(j),cell_id) == old_cell_id ) y_index = j
end do
end if
end do

end function get_cell_id
Expand Down
Loading