Using hourly spaced data in Pyleoclim

[bdamir5]

Is there any good way to use hourly or even less spaced data in Pyleoclim? In my example CO2 is measured and I will like to see daily periodicity.

To be more precise, my dates are of the form 2022-08-21 20:00:00 sampled hourly with some "holes" so I'm using Lomb scargle. I can convert it to Julian date or decimal year. Lomb scargle will gives me presumably right solution, but WWZ scalogram is having X axis not expressed in normal dates.

Here is WWZ scalogram:

Periodogram when using Julian date seems ok

But when using numerical year I'm unable to see one day periodicity

Excel file with data:

df_senzor2_cvetkovic.xlsx

Juper notebook used:

example.zip

[khider]

Pyleotutorial issue, transferring

[tanaya-g]

@bdamir5 We are creating a tutorial on this issue, as several users have been using Pyleoclim for sub-monthly data. Would you allow us to use your data in this tutorial? If so, can you provide the source as well?

[bdamir5]

@tanaya-g Sure, you can freely use the data. The data are my own measurements of CO2 concentration in the urban area of city Zagreb. They are sampled at a two-second frequency with the common low-cost CO2 sensor Senseair K30 and saved in .txt or .log files. Then I did some Python gymnastics to load all those log files to Pandas dataframe and extract whole-hour measurements. I exported the data to Excel to have them in one place when uploading here. Because there are a lot those log files. If you want I can provide you with a code for all that.

For your convenience, I attached even nicer and longer data that have this periodogram.

File with CO2 concentration (ppm)
df_senzor1_prozordolje.xlsx

Also here you can find one of many of those log files from which I extracted the data

logfile_2022-03-25_13.20.44.log

[tanaya-g]

@bdamir5 Great. Is this format close to what you are looking for? This is the Series object plotted with no modifications made to the axes labels or titles. The format of the x-axis is the same as this one when doing the WWZ scalogram as well.

[bdamir5]

@tanaya-g That format would be fine. It is certainly better than what I was getting. But I am wondering, is there an option to get classical date format like when you plot Pandas date-time object like this

[bdamir5]

@tanaya-g One more question. Can we read peak positions in periodograms precisely? It is hard to see where peaks are positioned on X axis. Or there is some option to export periodogram data to evaluate it in something other?

[bdamir5]

@bdamir5 Great. Is this format close to what you are looking for? This is the Series object plotted with no modifications made to the axes labels or titles. The format of the x-axis is the same as this one when doing the WWZ scalogram as well.

Also, one more thing to consider when using decimal year is that a small periodicity (like a day) will be incorporated on a high decimal place in decimal year number which can lead to potential calculation problems. I'm not sure is that the reason I'm not getting one-day periodicity in following (Jungfraujoch) CO2 data when using decimal year or it is just X-axis resolution problem?

Here you can see the periodogram when using decimal year (the period is in years):

And here you can see when using Julian date (in Julian date a day is the first digit before the decimal point)

Also, I attached used data in this example but it is the same problem in the data before

JungfraujochCO2.xlsx