Tool Audit

[tdiethe]

We currently have a long list of (poorly documented) tools:

aggregate
aggregate_into_dict_and_apply
aggregate_plate
aligned_merge
aligning_window
apply
asset_plate_generator
asset_splitter
asset_writer
clock
component
component_filter
component_set
dict_argmax
dict_values_to_const
histogram_from_list
histograms_to_csv
index_of
index_of_by_stream
jsonify
list_dict_mean
list_dict_sum
list_length
list_mean
list_sum
meta_instance
meta_instance_from_list
percentiles_from_list
percentiles_to_csv
plate_sink
product
relative_apply
relative_apply2
relative_window
sink
slice
sliding_apply
sliding_listify
sliding_sink
sliding_window
splitter
splitter_from_stream
splitter_of_dict
splitter_of_list
splitter_time_aware
splitter_time_aware_from_stream
splitter_with_unlist
stream_broadcaster
stream_broadcaster_from_stream

We should attach usage numbers to each of these for sphere-hyperstream, and determine whether any should be pruned/renamed. For those that the remain the documentation should be improved.

[perellonieto]

I see in the BaseTool https://github.com/IRC-SPHERE/HyperStream/blob/master/hyperstream/tool/base_tool.py that at least a logging message in the debug level is generated every time that one of the tools is created.

Would it be enough setting the logging level at least to DEBUG and grep the standard output to see how many occurrences are of each tool?

Would it be better to augment the BaseTool in a way that the count is stored in the MongoDB or memory?

[tdiethe]

Interesting - hadn't thought of automating it - was just thinking to go through all of the SPHERE workflow scripts by hand. Trouble is you'd have to execute every workflow to be sure?