Updated changes from main #14
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…262) * Read Simple Reco info to PhaseIITreeMaker and add Simple Tank Track Length variable * Updated Reweight Tool and added separate Reweight Flux Tool --------- Co-authored-by: James Minock <jminock@anniegpvm02.fnal.gov>
…er to fix charge omission seen for MC Hits (#259) * Update ClusterFinderConfig Narrow cluster window - typical use is 5 hits in 50ns, not 5 hits in 100ns * Update EventSelectorConfig New timing offsets moved the PMTs "back" by an average of ~10ns. When looking for PMT/MRD coincidence, we this need to adjust this offset value. * Update ClusterClassifiers.cpp With new gains calibration we now have 121 active PMTs instead of 123. Adjust charge balance parameter to reflect the number of active PMTs that could capture light. In the future, will alter this to read in the channel keys so that it automatically grabs the correct number of active PMTs. * Update ClusterFinder.cpp Fixed issues seen with ClusterFinder and the omission of charge for MC Hits: - Change MC hit time away from 2ns discretization, as the new data hit times are float values. - As a result, need to change for loop to not just grab integer hit times (old), but also grab float hit times. - MC Hits with times < 0 were being ignored due to a variable set to 0. Tool now keeps all hit times, even if they are negative due to time smearing. - Reduce first-photon effect by modifying how hit timing is assigned for a "pulse"/digit. Followed what the tool DigitBuilder used: median photon hit time as the "pulse" time. Also, - In Data, mandate hits need to be > 0 (there is a weird spike at 0 in cluster times that are from this and are likely unphysical) TODO: Potentially expand the MC "pulse" integration window from 10ns to something larger. Average charge per hit is lower and total number of hits slightly higher in MC than data, indicating the pulse window may be too narrow and is cutting what should be one pulse into two. Need to check with calibration to see what the average pulse window length is for ~SPE pulses. * Update ClusterFinderConfig Same as data - cluster window 50ns, looking for at least 5 hits * Update ClusterFinderConfig Dropped line pointing to an external WCSim channel key ID (not really an issue cause they are the same) * Update DeadPMTIDs_p2v7.txt Updated mask to omit additional PMT to reflect new gains data. (PMT 337, aka 6 here, no longer active in data) * Update LoadWCSimConfig Added a small description on where to find collaboration WCSim samples. Also cut down on commented lines * Update LoadGenieEventConfig Updated tool config to read in new collaboration samples. Added description to each variable per doc 5230-v2 (LoadGenieEvent Tutorial) * Update PhaseIITreeMakerConfig In prep for whenever we have tank reco available * Update LoadGeometryConfig new gains * Update DeadPMTIDs_p2v7.txt Updated new inactive PMT from the gains calibration * Update LoadWCSimConfig Updated config to include paths to latest collaboration samples * Add files via upload PMT timing offsets derived from Laser calibration data. These offsets should be considered precision software corrections on top of the previous firmware corrections at the ADC level to correct for the large cabling delays. * Update LoadGeometry.h Updated header file with PMT timing offsets * Update LoadGeometryConfig Path to PMT timing offset .csv file * Update EventSelector.cpp 1. Updated default offset value (745) to 755 to account for new timing offsets of the PMTs. 2. In the new gains file, there are measurements for the 121 active PMTs. Two PMTs were deemed "inactive" and are no longer included. (Old file used to have 123 active PMTs). These PMTs were omitted as no data/hits were recorded on them during the source calibration. In principle there should only be hits in the active PMTs. However, even though there were no data/hits recorded in those tubes for the source calibration, for very energetic events occasionally one of the since "inactived" PMTs registers a hit for some reason and tries to access the map, which crashes this tool. Just added a condition that the channel must exist in the map in order to count that hit. * Update LoadGeometry.cpp Adjust LoadGeometry to include new PMT Timing offsets from .csv file, derived from the laser source calibration. * Update PhaseIIADCHitFinder.cpp - Grab new PMT timing offset values from .csv file to subtract from the hit times --> added this to all methods of hit finding - Instead of grabbing the x tick with the maximum ADC value as the hit time (always binned to 2ns), assign float hit times based when the pulse crosses 50% of its maximum above threshold. Currently only implemented for the Pulse Finding Approach "Threshold" and Pulse Window Type "Dynamic" (the default, used configuration of the code). TODO: - Add this hit timing approach to all configurations of the code. - (Maybe) per Bob's recommendation, it might be better to instead assign the hit time using the following: 1. get the time at 50% between peak and threshold 2. get the time at 20% " " " " 3. Draw a straight line between these two points to extrapolate the time to zero threshold This way it would lessen the effect of time shifts between large and small pulses. * Update PhaseIIADCHitFinder.h Added offset map to header * Add files via upload Old tool, new toolchain: output raw waveforms of PMT and aux channels from built, processed ANNIEEvents (that contain the raw waveforms). Useful to analyze PMT waveforms, as well as look at aux channel waveforms like the BoosterRWM and BoosterRF. * Add files via upload In case users want to eventbuild part files with raw waveforms (useful for source analysis), this toolchain complements the official event building toolchain that is used to create our processed data. Configurations for the tools to extract raw waveforms successfully from the PMTs are taken from Gian's AmBe analysis work. Currently, the event building for this toolchain has only been tested for Tank + CTC events. * Add files via upload Added README for event building toolchain, with reference to the ANNIE wiki page on how to properly event build. * Update README.md small syntax fix for EventBuilder README file * Update ClusterFinder.cpp Missed a bracket which caused the CI build error * Update LoadWCSim.cpp When using the "automatic" method for loading in GENIE files in the LoadGenieEvent, this was causing that tool to crash as it couldn't find the proper path based on the WCSim file name. Removed the to_string function * Update EventSelector.cpp Removed the charge threshold (> 200 pe) needed to consider tank/MRD coincidence. Given the README states "PMTMRDCoincCut: Flags events that do not have a certain time offset between PMT & MRD cluster" this should include all PMT clusters, and not just some that are over a certain charge threshold. This change in reality likely won't change much, as most muon tracks in the MRD deposit enough light to leave more than 200 pe worth of charge, but per our Slack convo it's probably best to just remove this condition as its not obvious.
…268) * Update DNNTrackLengthTrainConfig Changed the storing location of ScalingVarsStore.bs since the Data_Energy_Reco directory doesn't exist in the main ToolAnalysis repository. * Update EventSelectorConfig Applied the cuts that we use currently in the Energy Reconstruction. * Update HitCleanerConfig Lowered the verbosity * Update LoadWCSimConfig Upped the verbosity * Update LoadWCSimLAPPDConfig Upped the verbosity * Update TimeClusteringConfig Lowered the verbosity * Adding the PhaseIITreeMaker Tool Adding a config file for the PhaseIITreeMaker Tool so we can use it in the Energy Reconstruction ToolChain. * Rename PhaseIITreeMakerConfig.txt to PhaseIITreeMakerConfig Removed the .txt from the file name * Update ToolsConfig Added a line for the PhaseIITreeMaker Tool * Update DNNTrackLengthPredictConfig Changed the path of ScalingVarsStore.bs according to the change I made in the Train_Test/DNNTrackLengthTrainConfig * Update EventSelectorConfig Applied the current cuts used in the Energy Reconstruction * Update HitCleanerConfig Lowered the verbosity * Update LoadWCSimConfig Upped the verbosity * Update LoadWCSimLAPPDConfig Upped the verbosity * Update ToolsConfig Added a line for the PhaseIITreeMaker Tool * Adding PhaseIITreeMaker in /Predict Adding a config file for PhaseIITreeMaker just like I did for the Train_Test ToolChain * Rename PhaseIITreeMakerConfig.txt to PhaseIITreeMakerConfig Removed the .txt from the file name * Update PlotsTrackLengthAndEnergy.cpp Added a histogram that plots the Energy Relative Deviation between E_true and E_reco * Update PlotsTrackLengthAndEnergy.cpp Added a missing line * Updating PhaseIITreeMaker to include Digit info Added lines of code to the .h and .cpp files to include the digitX, digitY, digitZ and digitT variables in the root ouput * Update PhaseIITreeMaker.cpp re-trigger workflow --------- Co-authored-by: marc1uk <marc1uk_@hotmail.com>
…ool: (#267) - Renamed load_from_file -> load_from_csv - Introduced variable save_to_csv - Updated documentation at beginning of RingCounting.py to reflect changes - Updated README.md to reflect changes - Updated UserTools/RingCounting/RingCountingConfig to reflect changes Users using the RingCounting tool need to update their RingCountingConfig files following this change.
* Read Simple Reco info to PhaseIITreeMaker and add Simple Tank Track Length variable * Updated Reweight Tool and added separate Reweight Flux Tool * Updated reading in new ANNIE flux histograms --------- Co-authored-by: James Minock <jminock@anniegpvm02.fnal.gov>
* Read Simple Reco info to PhaseIITreeMaker and add Simple Tank Track Length variable * Updated Reweight Tool and added separate Reweight Flux Tool * Added RingCounting variables to PhaseIITreeMaker --------- Co-authored-by: James Minock <jminock@anniegpvm02.fnal.gov>
* Update Makefile PythonScript lives /ToolAnalysis. It can't be edited, so 'make clean' break at that point. * Update Makefile rework solution to make it less specific. this version removes all files that can be removed, skipping those that aren't writable. --------- Co-authored-by: marc1uk <marc1uk_@hotmail.com>
* Update BeamFetcherV2.cpp - fixed a bug where not all timestamps were being saved to the output ROOT file - clear beamdatamap after writing it to the root file. This increases the speed of the tool by ~2x - change the beam triggerword from 5 to 14 (undelayed beam trigger). This doesn't really affect the entries we get (< 1% difference in total counts and total recorded POT), but Yue's new event building (coming soon) builds on 14, so trying to stay consistent with that * Update BeamFetcherV2.cpp Missed two small changes * Update BeamFetcherV2.h
* Update ClusterFinderConfig With the new timing offsets for data, cluster window can be reduced from 50ns to 40ns, as MC shows all reflected + scattered light resulting is captured within 40ns of the initial hit. * Update ClusterFinderConfig New PMT timing offsets reduced spread in hit times that were a result of unsubtracted TOF differences. The cluster window can be reduced from 50ns to 40ns; this narrower window still captures all light from the interaction according to the MC.
* Update ChannelSPEGains2023.csv this channel does not have a gains measurement as its off/inactive. However, for some reason in very rare instances a hit is registered (not sure why this is the case). Set the gains to 0 in case a hit is registered, as to assign it 0 ADC value and not contribute any charge. * Update TankPMTTimingOffsets.csv Same as the gains, but assign a hit time offset of 0 (no measurement data on this channel from the laser calibration)
Add "pulse width" as a configuration variable in MC hits. It is traditionally 10ns in width, but this width is yet to be tuned to data.
If database entry for a particular device doesn't return any values, fill that device branch in the output root file with -9999 instead of leaving it empty. We use this tool in the BeamFetcherV2 toolchain to produce beam root files that are then used by the event building toolchain. The beam root files contain a branch for each device. Without this modification, as the BeamFetcherV2 toolchain loops over RAWData part files, grabs CTC timestamps and queries for DB values, if a device entry is empty the toolchain will fail to fill the entire device branch as the entries between branches will be unequal.
Co-authored-by: James Minock <jminock@anniegpvm02.fnal.gov>
…ct x,y,z conventions (#284) Co-authored-by: James Minock <jminock@anniegpvm02.fnal.gov>
…produced it. (#285) * A new tool that links vectors of MCHits to the particle that deposited the most energy to the cluster. * Changing variables over to pointers * Making this thing actually work. LoadWCSim outputs were not as expected... * Some final adjustments and finally a README
Co-authored-by: James Minock <jminock@anniegpvm02.fnal.gov>
…d momentum transfer, bjorken x, elasticity y, and target nuclei (#280) Co-authored-by: James Minock <jminock@anniegpvm02.fnal.gov>
…Gs and momentum, energy and momentum transfer, bjorken x, elasticity y, and target nuclei (#281) Co-authored-by: James Minock <jminock@anniegpvm02.fnal.gov>
Co-authored-by: James Minock <jminock@anniegpvm02.fnal.gov>
Expand file limit to allow tools like LoadANNIEEvent to loop over all part files of a run
* Update PhaseIIADCHitFinder.cpp Assigned correct 50% interpolation time instead of max ADC time * Update PhaseIIADCHitFinder.cpp clean up verbose of the hit finding tool
* Add files via upload New BeamQuality tool (from Andrew) to produce BeamStatuses from the BeamFetcherV2 that the EventBuilding can interface with * Update Factory.cpp * Update Unity.h * Update BeamQuality.cpp change beam trigger to 14 for the new event builder --------- Co-authored-by: marc1uk <marc1uk_@hotmail.com>
* Delete configfiles/EventBuilder/BeamDecoderConfig BeamFetcherV2 and BeamQuality tools replace BeamDecoder functionality * Add files via upload BeamFetcherV2 device list (currently set to read all of them) * Update MRDDataDecoderConfig drop verbose * Update README.md * Update ToolsConfig * Add files via upload
* Add check for TriggerWord being set * Set StopLoop to 1 when failed to get TriggerWord
* Update PMTDataDecoder.cpp Add functionality for reading the RWM + BRF waveforms * Update PMTDataDecoder.h * Add files via upload New Fit RWM+BRF tool that will find the rising edge of the auxiliary channel waveforms * Create README.md README for the new RWM fitting tool * Update FitRWMWaveform.cpp delete histograms to prevent potential memory leakage
* Update FindMrdTracks.cpp Don't hault the toolchain if there are no MRD tracks * Update TriggerDataDecoder.cpp Triggers are stored to the buffer - clear it to speed up the tool. Also adding verbosity conditions on std out * Update LoadRawDataConfig Wrong config variables - now it should work out of the box as intended * Update CreateMyList.sh Rather than passing the path of the RAWData file to the bash script, add it hard coded. Not great practice but this script is used for a very specific purpose to generate a list file containing part files from the RAWData, and the data location has not changed in years (and likely won't change for now). * Update FindMrdTracks.cpp revert changes
* Added MuonFitter Tool * Fixed minor bug for string delimination --------- Co-authored-by: James Minock <jminock@anniegpvm02.fnal.gov> Co-authored-by: marc1uk <marc1uk_@hotmail.com>
* Add FitRWM tool to Factory * Add FitRWM to Unity
This reverts commit 0d9aca0. Testing by James M. indicates that this PR introduced breaking changes. While we wait for assistance from Daniel in correcting them, revert to the previous working version. Co-authored-by: Marcus O'Flaherty <moflaher@fnal.gov>
* add .csv output for storing run metrics
Unsmeared hits are relative to MC global time. Adding in the event time was causing an artificial shift in the hit times if smeared hit times was turned off.
* Initial toolchain upload New toolchain to extract raw small AmBe PMT waveforms from RWM aux channel and store them into a root file * Create README.md * Update README.md * Create my_inputs.txt
Co-authored-by: James Minock <jminock@anniegpvm02.fnal.gov>
Co-authored-by: James Minock <jminock@anniegpvm02.fnal.gov>
) * Update hit finding to reflect Gains integration window Added another hit finding technique that reflects the same integration window used in the 2023 Gains calibration by Gian. * Update README.md * Update PhaseIIADCHitFinder.cpp Replace std::cout with Log * Update PhaseIIADCHitFinder.cpp Calling log within a const method raises errors, changing back to std:out
* Fixing bug in PhaseIITreeMaker.cpp NDigitsPMTs and NDigitsLAPPDs are initialized to -9999. Then, they are incremented in order to count the number of PMT/LAPPD digits in the event. But this way, there is a -9999 offset to their values. So I initialize them to 0 before incrementing. Also I removed the totalPMTs and totalLAPPDs variables because they are basically not used. * Fixing typo * Update PhaseIITreeMaker.cpp Initialize NDigitsPMTs and NDigitsLAPPDs variables in ResetVariables() function.
* update makefile to allow building a specific toolchain. Associated changes to support this; move classes shared by multiple tools to DataModel, update PlotWaveforms library Makefile for new locations, change pre-processor defines duplicated in multiple headers to consts in implemenetation file. * make MyFactory if it doesn't exist * missed Tool define collisions. Note these same defines are made in other Tools so should be moved to DataModel/Constants.h * add notes on building to README * remove duplicate libraries in target dependencies --------- Co-authored-by: Marcus O'Flaherty <moflaher@fnal.gov> Co-authored-by: Marcus O'Flaherty <marcus.o-flaherty@warwick.ac.uk>
remove unneeded #include "DummyTool.h" from main.cpp
* Update DeadPMTIDs_p2v7.txt Add two PMTs to the deadlist based on observed occurrence of hits in data * Update DeadPMTIDs_p2v7.txt 359 shows some activity for higher-energy events. We can remove it from the dead mask but more investigation is needed
* Added LAPPDWaveformDisplayTool * Update ConfigGeo it has been updated to the latest geometry file. --------- Co-authored-by: Mustafa Kandemir <kandemir@anniegpvm01.fnal.gov>
* Update BFV2 Add a memory clean for BeamDataQuery map, also clean the TimeToTriggerWordMapComplete from trigger data decoder. Now we can run BFV2 for the whole run to avoid the entry lost issue at the beginning of processing * Update for BFV2 Cleaning the BeamDataQuery doesn't change too much, removed
Usage: ./CreateMyList.sh RUN will populate my_files.txt with all the raw part files from the run
Co-authored-by: James Minock <jminock@anniegpvm02.fnal.gov>
stupid security measures
* Adjusted logic to handle MC waveforms From Andrew's changes, MC now has a waveform simulator that ClusterFinder handles as data-like hits. As a result we must adjust some of the logic used by downstream tools. * Update README.md * Update PhaseIITreeMaker.cpp Add option to skip execution step in case PMTWaveform has no MCHits * Update PhaseIITreeMaker.cpp typo
…ry within configfiles (#360) * update new makefile to support ToolChain folders embedded in a hierarchy (e.g. EnergyReco/Predict) * make it more robust - need to specify hierarchy not just last folder name * update readme * actually fix it to support toolchain hierarchies for real this tiem * Update Makefile re-trigger CI as workflow needed updating * correct default ToolChain and re-enable debug symbols * re-add excludes, we do need them --------- Co-authored-by: Marcus O'Flaherty <moflaher@fnal.gov>
The LAPPDHit and LAPPDPulse.h are for LAPPD so they are included in this PR Co-authored-by: marc1uk <marc1uk_@hotmail.com>
* Update FullTankPMTGeometry.csv Add offline / dead PMTs * Update FullTankPMTGeometry.csv * Dead channel flexibility in the data and MC * Update README.md Add info about ApplyDeadMask * Update README.md ApplyDeadMask info for TreeMaker * Update ClusterFinder.cpp Given the tool fetches from Detector.h, need to use "detectorstatus" rather than "channelstatus" * Update PhaseIITreeMaker.cpp Seems like the goto is illegal because goto skip_channel skips SPE_available’s initialization. Declare it first. * Update PhaseIITreeMaker.cpp forgot to change this to detectorstatus * Extra whitespace at the end (row) leading to crash * Update PhaseIITreeMaker.cpp Final modification - remove hits from being included in the cluster tree
* Add EBPMT for PMT pairing in EventBuilding * Modifications based on comments from PR#332: 1. I move the get and check of `RWMRawWaveforms` and `BRFRawWaveforms` to line 75 to 85. 2. I move the check and return to line 44-49. 3. The line 122 was still at 122, but only print after the `[]` 4. For line 125 and 137, I change the usage of `[]` to a int AlmostCompleteWaveforms_CountHere, initialized to be zero. 5. I would want to use that but may want to leave this to future optimization... since everything is working right now. (not perfectly because I do see zero PMT waveform from some timestamps, but they can be removed in later steps no matter where they are from) 6. Now at line 339 to 341, changed to emplace as you suggest. * Fix a bug about NumWavesInCompleteSet. Previously, if NumWavesInCompleteSet was set to 140 and there are only ~133 PMT waveforms exist in data, the AlmostCompleteWaveforms[PMTCounterTimeNs] won't be created. But the usage of AlmostCompleteWaveforms[PMTCounterTimeNs] in Log will create an entry there. This won't affect the later code because there are checks to require AlmostCompleteWaveforms[PMTCounterTimeNs] > 5. Now the Log was removed and the tolerance for PMT number is changed to 10 at: ChannelKey.size() >= (NumWavesInCompleteSet - 10) * The correct file for previous commit --------- Co-authored-by: marc1uk <marc1uk_@hotmail.com>
* Update README.md Beef up Hit finding README (yay more documentation) * Update PhaseIIADCHitFinder.cpp Add entire ADC trace to the pulse class * Update ADCPulse.cpp Expand ADCpulse class to include entire trace * Update ADCPulse.h Extend ADCpulse class to include traces * Update ADCPulse.h bug fix and additional documentation for the newly added pulse features * Update ADCPulse.cpp more bug fixes to ADCpulse (should work now!) * Update PhaseIIADCHitFinder.cpp bug fix for storing traces * Update ADCPulse.h trying to make it as backwards compatible as possible
* Update Factory.cpp Add PrintADCTraces to factory * Update Unity.h add to unity * New PrintADCTraces tool * PrintADCTraces working toolchain * Update PrintADCTraces.cpp - condensed some code - added a name to the TGraph - removed the double delete
* Adjust EventSelector logic to handle MC waveforms Another downstream tool modification to adjust how the tool uses MC hits that are coming from the waveform generator. * Update README.md Added config variable that tells EventSelector to use MC waveform hits rather than the default MC hits. * Update EventSelector.cpp Added skip event for PMTWaveformSim * Update EventSelector.cpp Cleaned up logic * Update EventSelector.cpp - code condensed by merging data and MC logic together - removed the event skipping check
* Remove skipping of events Instead, we can fill one of the dead PMTs with a "minimal" waveform to pass to the hit finding tool * Remove event skipping for hit finder tool * Fix skip event * Add files via upload * Add files via upload * Add files via upload * Update PMTWaveformSim.cpp forgot to set the channel ID to a dead PMT * Update README.md Quick disclaimer about the dummy channel
…368) * Add CalcMinMaxAmp method to LAPPDTraceMax Added CalcMinMaxAmp method to LAPPDTraceMax class. * Implement CalcMinMaxAmp function for waveform analysis Added a new function to calculate minimum and maximum amplitudes along with RMS, standard deviation, variance, and integral of the waveform. * Add noise study variables and metadata timestamps Added new variables for noise study and metadata timestamps also adding RMS, min and max waveform amplitudes fro dark room data * Add new timestamp metadata and related branches * Clean up LAPPDClusterTree.cpp formatting Removed unnecessary closing brace and adjusted formatting. * Fix missing newline at end of LAPPDClusterTree.cpp * Improve ACDCmetadata handling and timestamp extraction Added error handling for ACDCmetadata retrieval and refactored timestamp extraction from ACDC metadata. * Add timestamps_meta to LAPPDClusterTree * Add endl to OUT OF RANGE error messages * Fix min/max assignments and removing double mean calculation Refactor CalcMinMaxAmp function to correct min/max assignments and removing double mean calculation. * Add meta_timestamp_int variable for timestamp handling
* Add BRF / beam jitter to the MC Include additional jitter / uncertainty observed in the data to correctly model the BNB spill structure * Update AssignBunchTimingMCConfig * Update AssignBunchTimingMCConfig
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.