# DeepLearning.ai作业:(4-1)-- 卷积神经网络（Foundations of CNN）

• 卷积神经网络的模型搭建
• 用TensorFlow来训练卷积神经网络

# Part1：Convolutional Neural Networks: Step by Step

• convolution funtions:
• Convolve window
• Convolution forward
• Convolution backward (optional)
• Pooling functions：
• Pooling forward
• Distribute value
• Pooling backward (optional)

## Convolutional Neural Networks

2.Single step of convolution

3.Convolutional Neural Networks - Forward pass

$$n_H = \lfloor \frac{n_{H_{prev}} - f + 2 \times pad}{stride} \rfloor +1$$
$$n_W = \lfloor \frac{n_{W_{prev}} - f + 2 \times pad}{stride} \rfloor +1$$
$$n_C = \text{number of filters used in the convolution}$$

## Pooling layer

$$n_H = \lfloor \frac{n_{H_{prev}} - f}{stride} \rfloor +1$$
$$n_W = \lfloor \frac{n_{W_{prev}} - f}{stride} \rfloor +1$$
$$n_C = n_{C_{prev}}$$

## Backpropagation in convolutional neural networks

### 1.Convolutional layer backward pass

1.计算dA

$$dA += \sum _{h=0} ^{n_H} \sum_{w=0} ^{n_W} W_c \times dZ_{hw}$$

2.计算dW

$dW = A \times dZ$，而更具体一点，因为W对Z的每一个像素都是有作用的，所以就等于每一个输入图片的切片乘以对应输出图片像素的导数，然后再求和！

$$dW_c += \sum _{h=0} ^{n_H} \sum_{w=0} ^ {n_W} a_{slice} \times dZ_{hw}$$

3.计算db

$$db = \sum_h \sum_w dZ_{hw}$$

### Pooling layer - backward pass

1. Max pooling - backward pass

$$X = \begin{bmatrix}1 && 3 \\ 4 && 2 \end{bmatrix} \quad \rightarrow \quad M =\begin{bmatrix} 0 && 0 \\ 1 && 0 \end{bmatrix}$$

2. Average pooling - backward pass

$$dZ = 1 \quad \rightarrow \quad dZ =\begin{bmatrix} 1/4 && 1/4 \\ 1/4 && 1/4 \end{bmatrix}$$

# Part2：Convolutional Neural Networks: Application

## 2.Initialize parameters

W = tf.get_variable("W", [1,2,3,4], initializer = ...)

initializer 用tf.contrib.layers.xavier_initializer

init = tf.global_variables_initializer()

sess_test.run(init)

## 3. Forward propagation

• tf.nn.conv2d(X,W1, strides = [1,s,s,1], padding = ‘SAME’): given an input $X$ and a group of filters $W1$, this function convolves $W1$’s filters on X. The third input ([1,f,f,1]) represents the strides for each dimension of the input (m, n_H_prev, n_W_prev, n_C_prev). You can read the full documentation here

• tf.nn.max_pool(A, ksize = [1,f,f,1], strides = [1,s,s,1], padding = ‘SAME’): given an input A, this function uses a window of size (f, f) and strides of size (s, s) to carry out max pooling over each window. You can read the full documentation here

• tf.nn.relu(Z1): computes the elementwise ReLU of Z1 (which can be any shape). You can read the full documentation here.

• tf.contrib.layers.flatten(P): given an input P, this function flattens each example into a 1D vector it while maintaining the batch-size. It returns a flattened tensor with shape [batch_size, k]. You can read the full documentation here.

• tf.contrib.layers.fully_connected(F, num_outputs): given a the flattened input F, it returns the output computed using a fully connected layer. You can read the full documentation here.

In the last function above (tf.contrib.layers.fully_connected), the fully connected layer automatically initializes weights in the graph and keeps on training them as you train the model. Hence, you did not need to initialize those weights when initializing the parameters.

## 4. Compute cost

• tf.nn.softmax_cross_entropy_with_logits(logits = Z3, labels = Y): computes the softmax entropy loss. This function both computes the softmax activation function as well as the resulting loss. You can check the full documentation here.这个函数已经包含了计算softmax，还有求cross-entropy两件事了。
• tf.reduce_mean: computes the mean of elements across dimensions of a tensor. Use this to sum the losses over all the examples to get the overall cost. You can check the full documentation here.

## 5. Model

optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cost)