Vignette

[paul-vdb]

@paciorek I've made a vignette that is a beginners guide to nimbleQuad with lots of little details about what the user can change and hopefully some context for them. I'm happy for it not to go into the package but end up somewhere else. I think it still needs some editing and probably should add some references to it.

See the branch vignette_quadGrids

[paciorek]

Thanks @paul-vdb . This looks like a very nice start, and I'm hopeful we can also use this as the basis for a blog post too. If we do come up with a better example, I guess some of the following specific details will be moot.

I've been working through this in some detail and have the following partial/tentative thoughts:

• I'm wary of only comparing against INLA. From the tests in test-nested-fit.R, our performance is almost as good as INLA's, apart (to some degree) for the fixed effects. So I think we should compare against HMC output too.
• Awkwardly, in some cases, using INLA's marginal density gives a somewhat unnormalized density (e.g., for tau_re, a numerical estimate of the integral is 1.034, compared to 0.9996554 for us). That is part of the explanation for the difference between the red and black lines. I'm not sure what to do about it. We don't want to come across as criticizing a minor aspect of the INLA post-processing and also I'd simply prefer not to spend time on it. I guess we could simply make a side comment that INLA's density values don't quite integrate to one and that is why the red is always above the black. And if we have HMC in there as well, the red-black difference won't be as prominent.
• We should use inla.dmarginal and not directly plot based on their more coarse grid.
• I don't think the marginal log-lik is very meaningful for diffuse priors. So I wouldn't show results from that. We might just comment off-hand when showing the result of runNestedApprox about how the marg log lik with diffuse priors integrates over very bad values of the hyperparameters.
• Somewhat minor point about the overplotting in the grid figures. How about we have open circles for AGHQ and closed circles for AGHQ sparse/pruned. That will make more clear what has been omitted and reduce overplotting.
• I think the discussion of sampling the parameters should not be part of the "Improving the parameter marginals" section as it is a different idea.

I have some partial code that I can use to make edits to the vignette qmd, but thought I would post these bigger picture comments first. I can also take a first pass at editing the text. But I'll wait for the moment while we brainstorm a possible different example.

[paciorek]

One advantage of a simple 2-d example is we can show the grids easily.

We could also consider just getting the vignette and blog post out soon and if we come up with a more compelling example we'll have plenty of opportunities to use that down the road (ISBA Bulletin, trainings, etc.).

[paul-vdb]

@paciorek Let's stick with Salamanders. Have a go at edits as you have lots of ideas. I have no problem with those comments above. Better to get this out with a blog post sooner than later anyways. Let me know when you want me to have another look at it.

I almost prefer not comparing with INLA as the priors are annoying and it would be nicer to avoid the logistic normal prior on p that makes reporting convenient. But maybe that is just another useful tidbit of information for the user in a vignette to get a custom prior and include the bounds.

[paciorek]

Ok, I've pushed a bunch of wording edits and some rearrangement of material to the branch. I've also removed the calculation of the marginal likelihood given the improper and vague priors.

I have some questions for you inline in the Rmd. So please take a look at those when you have a chance.

There's still some monkeying to be done with the plots, but I did make some changes there already, including the main thing of adding the HMC comparison.

Logistics - I'd like to convert the Rmd to qmd as that is the format we are using for the website and is the "next generation" of Rmd. I can do that unless you have concerns. (For the moment, I commented out the knitr config chunk as it wasn't running for me (resolve was not found), and rendered with quarto render Quadrature.Rmd.) (That said, I'm not sure what the requirements are, if any, for file formats in a vignettes directory in an R package...)

I had a typo that causes the chunk that creates PCA_AGHQ to have a problem that is seen in the current html. I've fixed that in the Rmd, but I haven't re-rendered.

[paul-vdb]

@paciorek This looks a lot better. I made some minor edits and created a quarto version. I realized I should have made the RMD edits and then the Quarto so github would track them. My apologies.

Edits were a couple of spelling problems. Added "quadrature methods for outer marginalization of the parameter nodes" in point 2 of the introduction. I also added a paragraph where you had asked if I could give more detail about why AGHQ-Sparse had more points (nQuad == levels and refers to accuracy not number of quad points). Please have a read of this paragraph and make sure I got the wording on principle axis correct for the PCA-AGHQ method :)

Happy with 7 quad points instead of 11 for the pruning example and not comparing marginal likelihoods as per your valid point.

On formatting, feel free to change whatever makes sense to you. Reading about Quarto it seems like if we wanted this on CRAN then we want a 'lightweight' format but otherwise can make it look quite a bit nicer. https://quarto-dev.github.io/quarto-r/articles/advanced-vignettes.html But I am also no expert on this. It is definitely an improvement on how plots look on render etc.

Also note that the new pruning example with 49 points and removing 50% leaves a small corner point removed. This is fine and can be resolved by asking for 50+% removed etc but is really an issue with 'ties' versus a proportion cut-off. Not strictly wrong but maybe something to think about.

Should we provide a full reference section to this vignette? Currently it is just packages and a comment about the Howes paper, but it has a fair amount of detail in it that might be nice to formally reference. Or we can update when the ISBA bulletin is out to refer to for technical details.

[paciorek]

Ok, I think this is nearly done.

I changed the pruning to better illustrate how one might use it such that one gets points intermediate between two full grids.

As far as references, yes, I think it would be good to add references when you have time.

CRAN is back from break today, so I will resubmit the package. I've also fixed a few fiddly things that have come up since I initially submitted, so maybe it's not so bad we got a second chance.

[paul-vdb]

@paciorek I put some references in there. Feel free to put any others you see fit. I figured it was key to put in the 2009 INLA paper and the Stringer paper plus reference the integration free method.

Fingers crossed nimbleQuad comes out tomorrow!