adaption in load.timeseries_opsd ...

[martacki]

... according to new format of opsd output file (_load_old is not up to date, new input data for GB and additional years). Won't be compatible with the current version of pypsa-eur (data/bundle should be updated simultaneously when merging into master)

[coroa]

Thanks for taking care of this!

Fine by me! All other countries are represented accurately?

To coordinate the merge with pypsa-eur it would be best to:

• release a new vresutils minor version (0.3.1), since there have been a couple of commits since
• lock pypsa-eur to the previous release (ie. pip version constraint instead of git+https)
• then merge this PR
• release a new vresutils version
• update the pypsa-eur data bundle with the new data files and add a lower bound version constraint instead for vresutils

@fneum I just released the vresutils version 0.3.1, it would be good to pin this version in the next pypsa-eur release to be able to merge this PR to master, or have some other work around!

You could also decide to copy this code straight to pypsa-eur master, and slowly get rid of yet another vresutils dependency?

[martacki]

Fine by me! All other countries are represented accurately?

Actually, there are few other "holes", but those were present in the old load data as well and not corrected by interpolation or copying slices from previous weeks in CH, IE, IS and NO.
Total demand seems also higher in the new data (up to 20%), but closer to what I found on other sources (like statista)

GR and EE have the same missing hours, no changes there.

Other few have still missing data for 2019.

[coroa]

Actually, there are few other "holes", but those were present in the old load data as well and not corrected by interpolation or copying slices from previous weeks in CH, IE, IS and NO.

It probably makes sense to stitch those as well. Not a prerequisite for this change.

The problem with interpolation is that for gaps larger than maybe 2-3 hours, it produces wrong results.

One easy improvement would be to blanket apply interpolate(limit=2) to not touch gaps larger than to 2 hours, i think. Please check!

[martacki]

It probably makes sense to stitch those as well. Not a prerequisite for this change.

The problem with interpolation is that for gaps larger than maybe 2-3 hours, it produces wrong results.

One easy improvement would be to blanket apply interpolate(limit=2) to not touch gaps larger than to 2 hours, i think. Please check!

I adapted those for the years [2010,2018], interpolation is now for max. 4 hours, larger timeframes are copied from couple days ahead. Only GB is missing data for a whole month, GR is missing a week or so, other "holes" are maximal 1-2 days.

[coroa]

Great improvement. A couple of comments.

[coroa]

@eb5194 Another last look before you would be happy to merge this?

[martacki]

@eb5194 Another last look before you would be happy to merge this?

should be all right