sensitivity of Laplace optimization in case of 1 RE to parameter starting value #64
Description
When optimizing the 1 RE example in the Laplace test "Laplace simplest 1D (constrained) with multiple data works", whether the optimization converges to the maximum depends on the starting value in some cases.
This might relate to the tolerance for the outer optimization as well as to the exact behavior of the inner optimization (e.g., when it decides it has converged).
I'm reporting this but not planning to investigate further.
set.seed(1)
m <- nimbleModel(
nimbleCode({
mu ~ dnorm(0, sd = 5)
a ~ dexp(rate = exp(mu))
for (i in 1:5){
y[i] ~ dnorm(a, sd = 2)
}
}), data = list(y = rnorm(5, 1, 2)), inits = list(mu = 2, a = 1),
buildDerivs = TRUE
)
mLaplace <- buildLaplace(model = m)
cm <- compileNimble(m)
cLaplace <- compileNimble(mLaplace, project = m)
xs <- seq(.289, .290, len=100)
ll <- sapply(xs, function(x) cLaplace$calcLogLik(x))
gr <- sapply(xs, function(x) cLaplace$gr_logLik(x))
opt <- cLaplace$findMLE(2) # seems to find the optimum
opt2 <- cLaplace$findMLE(.289) # doesn't move from starting point
options(digits=15)
opt$par
# [1] 0.2896864856623
opt2$par
# [1] 0.289
abs(opt$value-opt2$value) # close to `rel.tol` of `nlminb`
# [1] 8.59552216070369e-08
Note that other starting points do move and converge, but during other experiments I did find other starting values that didn't result in convergence.