吴恩达第二课第二周编程实例

吴恩达第2课第2周编程习题

目标：使用mini―batch来加快学习速度；比较梯度下降，momentum，adam的效果

核心：指数加权平均值得计算及其意义，它是momentum，RMSProp，Adam算法的基石

不足：本例程没有使用学习率衰减的步骤，同时本例程只适于3层的二分法的神经网络

常记点：

1. 偏差修正时是除以，此处是－，t从1开始；

2. L=len(parameters) //2 ，这个L不等于网络层数，range(1,L+1)=range(1,len(layers_dims))

3. Adam算法求s时，需要平方(np.square)，便于后面分母除根号(np.sqrt)

4. np.random.permutation(m)，把range(m)重排列，用于把样本打乱，每一代都要打乱一次

5. arr[:,:]：逗号前面表示行的选取，后面表示列的选取

1. ‘‘‘‘‘
4. 1.
5. 2.mini-batch
6. 3.momentum
7. 4.Adam
8. ‘‘‘
9. import
10. import
11. import
12. import
13. import
14. import
16. import
17. import
19. plt.rcParams[‘figure.figsize‘
20. plt.rcParams[‘image.interpolation‘]=‘nearest‘
21. plt.rcParams[‘image.cmap‘]=‘gray‘
23. #,
25. def
26. #parametersWb
27. forin#L1L
28. ‘W‘+str(l)]=parameters[‘W‘+str(l)]-learning_rate*grads[‘dW‘
29. ‘b‘+str(l)]=parameters[‘b‘+str(l)]-learning_rate*grads[‘db‘
31. return
32. ‘‘‘‘‘
33. mini-batch
34. ‘‘‘
36. def
44. forin
50. #mini_batch_size
51. if
56. return
58. ‘‘‘‘‘
59. momentum
60. ‘‘‘
61. #v
62. def
65. forin
66. ‘dW‘+str(l)]=np.zeros_like(parameters[‘W‘
67. ‘db‘+str(l)]=np.zeros_like(parameters[‘b‘
68. return
70. #
71. def
73. forin
74. ‘dW‘+str(l)]=beta*v[‘dW‘+str(l)]+(1-beta)*grads[‘dW‘
75. ‘db‘+str(l)]=beta*v[‘db‘+str(l)]+(1-beta)*grads[‘db‘
77. ‘W‘+str(l)]=parameters[‘W‘+str(l)]-learning_rate*v[‘dW‘
78. ‘b‘+str(l)]=parameters[‘b‘+str(l)]-learning_rate*v[‘db‘
80. return
82. ‘‘‘‘‘
83. Adam
84. ‘‘‘
85. #vs
86. def
89. forin
90. ‘dW‘+str(l)]=np.zeros_like(parameters[‘W‘
91. ‘db‘+str(l)]=np.zeros_like(parameters[‘b‘
93. ‘dW‘+str(l)]=np.zeros_like(parameters[‘W‘
94. ‘db‘+str(l)]=np.zeros_like(parameters[‘b‘
95. return
97. #
98. def
99. #t
102. forin
103. #
104. ‘dW‘+str(l)]=beta1*v[‘dW‘+str(l)]+(1-beta1)*grads[‘dW‘
105. ‘db‘+str(l)]=beta1*v[‘db‘+str(l)]+(1-beta1)*grads[‘db‘
106. #
107. ‘dW‘+str(l)]=v[‘dW‘
108. ‘db‘+str(l)]=v[‘db‘
109. #
111. ‘dW‘+str(l)]=beta2*s[‘dW‘+str(l)]+(1-beta2)*np.square(grads[‘dW‘
112. ‘db‘+str(l)]=beta2*s[‘db‘+str(l)]+(1-beta2)*np.square(grads[‘db‘
113. #
114. ‘dW‘+str(l)]=s[‘dW‘
115. ‘db‘+str(l)]=s[‘db‘
117. ‘W‘+str(l)]=parameters[‘W‘+str(l)]-learning_rate*(v_corrected[‘dW‘+str(l)]/np.sqrt(s_corrected[‘dW‘
118. ‘b‘+str(l)]=parameters[‘b‘+str(l)]-learning_rate*(v_corrected[‘db‘+str(l)]/np.sqrt(s_corrected[‘db‘
119. #vss=0sepsilon
120. return
122. ‘‘‘‘‘
124. ‘‘‘
127. def
134. #paramvs
136. if‘gd‘
137. pass
138. elif‘momentum‘
140. elif‘adam‘
142. else
143. print(
145. #
146. forin
149. #mini_batches
150. forin
151. #mini_batchX,Y
152. #
155. #
158. #
161. #
162. if‘gd‘
164. elif‘momentum‘
166. elif‘adam‘
170. if
172. ifand
173. print(+str(i)+‘:‘
174. if
176. ‘cost‘
177. ‘epoch‘
180. return
182. ‘‘‘‘‘
184. ‘‘‘
186. "gd"
187. "momentum"
188. "adam"
189. ‘‘‘‘‘
191. adam
192. ‘‘‘

原文：https://www.cnblogs.com/sytt3/p/9363326.html