第一博客
公众号：深度学习视觉

---

由于近期看见了BN中方差新的计算方法，不明白为什么要用那种计算方式，所以做了一次简单的性能测试。
两种方差计算公式：
1/n$\sum_{i=0}^n x_i^2 - \sum_{i=0}^n x_i * u$

1/n$\sum_{i=0}^n (x_i - u) ^2$

$u = \frac{1}{n}\sum_{i=0}^n x_i$

方法名	1	2	3	4	5	mean	备注	备注
input_.sum()	0.005	0.005	0.005	0.004	0.005	0.0047	np	sum+int
input_.mean()	0.012	0.009	0.109	0.109	0.009	0.0107	np	mean+int
np.power(input_,2).sum()	0.051	0.046	0.045	0.044	0.046	0.047	np	power+sum+int
np.power(input_,2)	0.044	0.041	0.040	0.040	0.040	0.041	np	power+int
_.sum	0.004	0.004	0.005	0.005	0.005	0.0045	np:上面数值求和	sum+int
np.power(input_ - mean,2)	0.44	0.44	0.44	0.44	0.44	0.44	np	power减法+int
_.sum()	0.004	0.004	0.004	0.005	0.004	0.0041	np:上面数值求和	sum+int
np.power(input_ + mean,2)	0.40	0.41	0.41	0.41	0.42	0.412	np,mean=3.5	power加法+float
sum() / len()	0.006	0.005	0.004	0.005	0.007	0.0055	自定义均值：先求和再均值	自定义sum+div+len+int
np.power(input_,2)	0.043	0.052	0.045	0.041	0.042	0.0446	变快：input在整型的情况下	power+int
np.power(input_,2)	0.32	0.32	0.32	0.32	0.32	0.32	float64	power+float
input_ + mean	0.0418	0.0418	0.0418	0.043	0.0418	0.0418	float64	np_array+float
np.power(input_+mean,2)	0.411	0.412	0.412	0.411	0.417	0.412	float64	power加法+float
自定义方差	0.35	0.34	0.36	0.36	0.34	0.35	float64	power + sum+div+len+float
np.power(input_,2).sum()	0.329	0.342	0.33	0.33	0.337	0.33	float64	power + sum+float

1. 主要消耗在power中有运算操作。比如power(a+b)

以下是测试代码，需要的自取测试。

import numpy as np
import time
input_ = np.random.randint(0,10,(10000000))

定义时间消耗计时器

def getTimeCost(f):def wrapper(*args,**kwargs):# x是从被修饰函数中传入的一个参数start = time.time()result = f(*args,**kwargs)end = time.time()print("{}:  costTime:{}".format(f.__name__,(end - start)))return end - startreturn wrapper

每部分时间消耗

@getTimeCost
def getVar_fast2(input_):st1 = time.time()numel = len(input_)et1 = time.time()print("len(input_):",et1-st1)sum_ = input_.sum()et2 = time.time()print("input_.sum():",et2-et1)mean = sum_ / numelet3 = time.time()print("sum div numel:",et3-et2)square = np.power(input_,2)et4 = time.time()print("np.power(input_,2):",et4-et3)sum_of_square = square.sum()et5 = time.time()print("square.sum():",et5-et4)sumvar = sum_of_square - sum_ * meanet6 = time.time()print("sum_of_square:",et6-et5)return sumvar@getTimeCost
def getVar_np2(input_):st1 = time.time()mean = input_.mean()et1 = time.time()print("input_.mean:",et1-st1)et2 = time.time()minus = input_ - meanprint("minus=input_ - mean:",et2-et1)power = np.power(minus,2).sum()et3 = time.time()print("input_.power(minus,2):",et3-et2)sumvar = power.sum()et4 = time.time()print("power.sum():",et4-et3)return sumvarprint("1.",40*"=")
x = input_ * 1.1
r = [getVar_fast2(x) for i in range(1)]
print("all mean:",np.array(r).mean())print("2.",40*"=")
r = [getVar_np2(x) for i in range(1)]
print("all mean:",np.array(r).mean())

时间消耗数值

1. ========================================
len(input_): 0.0
input_.sum(): 0.014959335327148438
sum div numel: 0.0
np.power(input_,2): 0.33510398864746094
square.sum(): 0.012965917587280273
sum_of_square: 0.0
getVar_fast2:  costTime:0.3700096607208252
all mean: 0.3700096607208252
2. ========================================
input_.mean: 0.023946523666381836
minus=input_ - mean: 0.0
input_.power(minus,2): 0.46773767471313477
power.sum(): 0.0
getVar_np2:  costTime:0.5006604194641113
all mean: 0.5006604194641113

两种方差计算方法如下

@getTimeCost
def getVar_fast2(input_):numel = len(input_)sum_ = input_.sum()mean = sum_ / numelsum_of_square = np.power(input_,2).sum()sumvar = sum_of_square - sum_ * meanreturn sumvar@getTimeCost
def getVar_np2(input_):mean = input_.mean()sumvar = np.power(input_ - mean,2).sum()return sumvargetVar_fast2(input_)
getVar_np2(input_)#cost time: 0.05784344673156738
#cost time: 0.4697434902191162

计算求和的时间消耗

@getTimeCost
def getSum(input_):sum_ = input_.sum()getSum(input_)
# out: getSum:  costTime:0.006983041

计算方差计算部分的时间消耗

numel = len(input_)
sum_ = input_.sum()
mean = sum_ / numel@getTimeCost
def getVar_fast1(input_1):sum_of_square = np.power(input_,2).sum()sumvar = sum_of_square - sum_ * meanreturn sumvarmean = input_.mean()
@getTimeCost
def getVar_np1(input_1):sumvar = np.power(input_ - mean,2).sum()return sumvargetVar_fast1(input_)
getVar_np1(input_)# getVar_fast1:  costTime:0.048866748
# getVar_np1:  costTime:0.4687469

计算均值部分的时间消耗

@getTimeCost
def getMean_fast1(input_):numel = len(input_)sum_ = input_.sum()mean = sum_ / numelreturn mean@getTimeCost
def getMean_np1(input_):mean = input_.mean()return meangetMean_fast1(input_)
getMean_np1(input_)# getMean_fast1:  costTime:0.0059847831
# getMean_np1:  costTime:0.010972023