Gradient Descent
An optimisation algorithm that iteratively updates model parameters to reduce prediction error.
Gradient Descent
An optimisation algorithm used to minimise the error of machine learning models by iteratively adjusting model parameters.
What Is Gradient Descent
Gradient descent is a foundational optimisation method used to improve machine learning models. It updates weights in the direction that most reduces the loss function, helping models learn patterns that better match expected outputs.
How Gradient Descent Works
During training, the model computes prediction error, then calculates gradients that indicate how each parameter affects that error. Parameters are adjusted by a learning rate multiplied by the gradient. Repeating this process over many iterations progressively lowers overall loss.
Types of Gradient Descent Algorithms
- Batch gradient descent: uses the full dataset for each update.
- Stochastic gradient descent (SGD): updates after each sample.
- Mini-batch gradient descent: updates using small subsets of data.
- Adaptive variants: methods such as Adam and RMSprop adjust update behaviour dynamically.
Role in Training Neural Networks
Neural networks with millions or billions of parameters rely on gradient descent and backpropagation to converge toward useful representations. Without this optimisation cycle, modern deep learning systems for translation, speech, and language understanding could not be trained effectively.
Importance for Machine Learning Systems
Gradient descent directly affects model quality, training stability, and compute efficiency. Choosing suitable learning rates, batch sizes, and optimiser strategies is critical for achieving reliable performance in production AI systems, including document-level language and translation models.
Related Terms
GDPR (General Data Protection Regulation)
EU legislation regulating the processing and protection of personal data.
Gender Bias in AI
Differences in how AI treats or represents genders due to training-data patterns.
Generative AI
AI systems that generate new text, images, audio, code, or other content from learned patterns.
GPU (Graphics Processing Unit)
A specialised processor that accelerates parallel computations for training and running AI models.
Glossary-driven Translation
Translation guided by predefined terminology lists.
Related Resources
Machine Learning
A field of AI where systems learn patterns from data to make predictions or generate content.
Supervised Learning
A machine learning method in which models are trained using labelled data.
Reinforcement Learning
A machine learning method in which an agent learns decision-making through rewards and penalties.
Backpropagation
A learning process used in neural networks to adjust internal weights after comparing predictions with expected results.
Fine-Tuning
The adaptation of a pre-trained model for a specific task through additional training on focused data.
Quantisation
A model optimisation technique that reduces the numerical precision of neural network parameters to decrease memory usage and improve inference speed.