Update how we specify symmetry of matrix in linear problem

[tommbendall]

From https://petsc.org/release/changes/318/ it appears that the pc_gamg_sym_graph option was removed (a couple of years ago) from petsc. I think the purpose of this was to communicate some symmetry of the problem to petsc, so it doesn't spend time trying to work it out itself.

We should update how this is done, which might be an optimisation.

This option appears in our compressible solver parameters

[JHopeCollins]

I think that option was to tell GAMG to symmetrize the graph of the matrix before coarsening, rather than to say something about the original matrix itself. This should (I think) ensure that the coarse operator is symmetric.

It seems like this option does the same thing (force the graph to be symmetrised): https://petsc.org/release/manualpages/PC/PCGAMGSetGraphSymmetrize/, so I'm not sure if it's quietly meant to replace the removed option, or if there was previously duplication.

Setting Mat options should probably be handled further upstream in Firedrake or pyop2(3?), but if it's a noticeable performance gain then I think it should be sufficient to just set the option manually after creating the solver but before calling solve, something like:

ksp = variational_solver.snes.ksp
A, P = ksp.getOperators()
A.setOption(PETSc.Mat.Option.SYMMETRIC, True)
P.setOption(PETSc.Mat.Option.SYMMETRIC, True)