-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
enhancementNew feature or requestNew feature or request
Description
-
solsitce.reduce(): this should be a function reducing a list of pytrees to a single pytree. This should be a single dispatch generic function which calls.merge()on anyMetricsobjects (a bit like the CLU implementation) and has customisable behaviour for other types of pytrees (e.g. mean leaves, sum leaves etc...). Could add an implementation for model parallel training ??? -
solstice.gather(): possibly unnecessary. Utility for reducing and gathering sharded pytrees across devices. This will be most useful for accumulatingMetrics, but maybe also for model parallel training ??
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request