How to calculate FLOPS?

[albert-666]

In the version of global threshold, how to calculate the flops? Is there an example code to calculate the theoretical FLOPS?

[hellozhuo]

In the version of global threshold, how to calculate the flops? Is there an example code to calculate the theoretical FLOPS?

Hi, to calculate the actual flops when using global threshold, we need to run the model over the whole testing set (here we refer to the validation set), and track the everage pruning rate in each layer. Because for each layer, the actual pruning rate in different iterations (i.e., within different minibatchs) can also be different.

Once we get the actual pruning rate for each layer, the calculation of flops is similar to the way as using the head-wise threshold.

[albert-666]

Thanks for your nice answer. Is there any demo code for calculating FLOPS when using global threshold?

[hellozhuo]

Once we got the pruning rate for each layer, the calculation of flops can refer to utils.py, by changing the target_pruning_rate to the actual pruning rate we got. The difference is that we have to calculate each layer's flops by using different pruning rates.

[albert-666]

Get it! Thanks!