Dynamics in Deep Classifiers Trained with the Square Loss: Normalization, Low Rank, Neural Collapse, and Generalization Bounds

Keyphrases

• Low-rank 100%
• Deep Network 100%
• Generalization Bounds 100%
• Loss Normalization 100%
• Deep Learning Classifier 100%
• Neural Collapse 100%
• Weight Matrix 50%
• Stochastic Gradient Descent 50%
• Weight Decay 50%
• Network Architecture 25%
• Order of Magnitude 25%
• Convolutional Neural Network 25%
• Lagrange multiplier 25%
• Convolutional Layer 25%
• Expected Error 25%
• Frobenius Norm 25%
• Analysis Support 25%
• Deep Learning Architectures 25%
• Gradient Descent 25%
• Rectified Linear Unit (ReLU) 25%
• Curse of Dimensionality 25%
• Norm-based 25%
• Target Function 25%
• Gradient Flow 25%
• Ranked Positional Weight 25%
• Classical Bounds 25%
• Absolute Minimum 25%

Mathematics

• Square Loss 100%
• Stochastics 66%
• Weight Matrix 66%
• Lagrange Multiplier 33%
• Gradient Flow 33%
• Frobenius Norm 33%
• Convolutional Neural Network 33%
• Absolute Minimum 33%
• Convolutional Layer 33%
• Curse of Dimensionality 33%