Question

conceptual & architectural questions

1. fully-connected vs. convolutional layers: explain how a convolutional (conv) layer takes advantage of the fact that its inputs are images compared to a regular fully-connected (fc) layer. in your explanation, highlight how local connectivity and parameter sharing help protect the model against overfitting.

Explanation:

Analyze spatial structure of images

Images possess strong local spatial correlation. Nearby pixels are highly related, whereas distant pixels are mostly independent. Fully-connected (FC) layers ignore this structure by flattening the input, treating all pixels equally regardless of their spatial arrangement.

Define local connectivity

A convolutional (CONV) layer connects each neuron only to a local region of the input volume (the receptive field). This local connectivity drastically reduces the number of connections compared to an FC layer, where every neuron connects to every input pixel.

Define parameter sharing

CONV layers apply the assumption that if a feature is useful to compute at some spatial position, it is also useful to compute at other positions. Thus, the same set of weights (the filter) is shared across the entire input width and height, reducing the parameter count.

Explain protection against overfitting

By combining local connectivity and parameter sharing, the total number of learnable parameters is dramatically reduced. Fewer parameters restrict the model's capacity to memorize noise, acting as a strong regularizer that protects against overfitting.

Answer:

A Convolutional (CONV) layer exploits the spatial structure of images through two main mechanisms:

1. Local Connectivity: Neurons connect only to a small, localized region of the input (receptive field) rather than the entire image. This preserves spatial relationships and reduces the number of weights per neuron.
2. Parameter Sharing: The same filter weights are applied across the entire input space. This assumes features (like edges) are translation-invariant.

Overfitting Protection: Together, these properties drastically reduce the total number of learnable parameters compared to a Fully-Connected (FC) layer. This lower model capacity prevents the network from memorizing training noise, thereby reducing overfitting.