Trying to understand refit function

[nickcox896]

Dear professor Hyndman,

The following code is take from your text book

recent_production <- aus_production |>
  filter(year(Quarter) >= 1992)
beer_train <- recent_production |>
  filter(year(Quarter) <= 2007)


test <- recent_production |>
  filter(year(Quarter) > 2007)


beer_fit <- beer_train |>
  model(
    Mean = MEAN(Beer),
    `Naïve` = NAIVE(Beer),
    `Seasonal naïve` = SNAIVE(Beer),
    Drift = RW(Beer ~ drift())
  )

beer_fc <- beer_fit |>
  forecast(h = 10)

accuracy(beer_fc, recent_production)

# A tibble: 4 × 10
  .model         .type    ME  RMSE   MAE    MPE  MAPE  MASE RMSSE    ACF1
  <chr>          <chr> <dbl> <dbl> <dbl>  <dbl> <dbl> <dbl> <dbl>   <dbl>
1 Drift          Test  -54.0  64.9  58.9 -13.6  14.6  4.12  3.87  -0.0741
2 Mean           Test  -13.8  38.4  34.8  -3.97  8.28 2.44  2.29  -0.0691
3 Naïve          Test  -51.4  62.7  57.4 -13.0  14.2  4.01  3.74  -0.0691
4 Seasonal naïve Test    5.2  14.3  13.4   1.15  3.17 0.937 0.853  0.132 

I can not understand the difference between the last line of code about accuracy and the following code

beer_fit  |>
  refit(test) |>
 forecast:: accuracy()

# A tibble: 4 × 10
  .model         .type        ME  RMSE   MAE    MPE  MAPE  MASE RMSSE    ACF1
  <chr>          <chr>     <dbl> <dbl> <dbl>  <dbl> <dbl> <dbl> <dbl>   <dbl>
1 Mean           Training -13.8   38.4 34.8  -3.97   8.28  4.45  3.45 -0.0691
2 Naïve          Training  -5.11  53.0 46.9  -2.01  10.9   5.99  4.75 -0.178 
3 Seasonal naïve Training  -2.17  11.2  7.83 -0.618  2.00  1     1     0.120 
4 Drift          Training  -5.59  53.0 46.9  -2.13  10.9   5.99  4.75 -0.178 

I understant that both outputs refer to the test data set.
The first one is about iterated pseudo out of sample forecasts and the last one about rolling pseudo out of sample forecasts or about
cross-validation forecasts?

I am a bit confused.

Many thanks

[robjhyndman]

The first one is a standard training/test split, where the accuracy measures are computed over the test set. The test set consists of 10 rows, so the accuracy measures are averaged over the 1-step, 2-step, ..., 10-step forecast horizons.

The second one applies the trained model to the test set (rather than forecasting the test set). When you apply accuracy() to a trained model (even a refit model), it will give results based on the one-step errors from the training set. So the accuracy measures here are all based on 1-step forecasts over the 10 rows of the test set.

[nickcox896]

Many many thanks!