Dynamic data processing: Recursive least-squares

P. J. G. Teunissen’s Dynamic Data Processing: Recursive Least-Squares develops estimation methods for measurements that arrive sequentially and for parameters that change over time. The six chapters move from batch least-squares through recursive updates, polynomial motion, state-space dynamics, random functions, and dynamic filtering. The treatment assumes linear algebra, calculus, statistics, and prior adjustment theory. This guide follows the second-edition source and emphasizes the connection between a physical model, an observation model, and the uncertainty attached to an estimate.

Chapter-wise summaries and reading tasks

Chapter 1 Least-squares: a review

The review establishes the mathematical language needed for the later recursive development. Observation equations relate measurements to unknown parameters through a design matrix, and consistency concerns whether the observed vector lies within the model’s range. Least-squares replaces an exact fit with a weighted compromise when observations are inconsistent. A stochastic model then distinguishes an estimate computed from data from the estimator whose uncertainty can be analysed. Nonlinear observation models require linearization and iteration, while condition equations provide an alternative formulation. Rank, redundancy, covariance, and the meaning of a residual deserve particular attention: a numerical solution is not automatically unique or statistically meaningful merely because a solver returns it.

Study task. Build a small overdetermined linear system. Identify its dimensions and rank, compute a weighted estimate, and explain the roles of the residual vector and estimated-parameter covariance.

Chapter 2 Recursive least-squares: the static case

The static recursive case keeps the unknown parameter fixed while adding new observations. Rather than storing and resolving the entire historical dataset, the method carries forward the information needed to update the estimate and its uncertainty. The A-form and B-form provide two formulations of that measurement update. The new data matter through how they compare with the model’s current prediction and how their uncertainty relates to existing information. This is not the same operation as averaging successive estimates without regard to precision. The discussion of nonlinear models also separates iterative refinement around an approximation from recursion through arriving measurements, two processes that can otherwise be confused.

Study task. Process a small dataset both in one batch and in sequential groups under identical assumptions. Compare the resulting estimate and covariance, then explain why an unweighted average of the group estimates need not agree.

Chapter 3 Recursive least-squares: the time-varying case

Time-varying parameters require a model that connects states at different epochs. The chapter begins with polynomial equations of motion and develops the distinction between prediction, filtering, and smoothing. Prediction uses information available before the target epoch’s measurement update, filtering incorporates measurements through that epoch, and smoothing also uses later data to improve an earlier state estimate. The recursive procedure therefore combines a time update with a measurement update. The polynomial assumption constrains possible trajectories and is not simply a convenient way of drawing a curve. Choosing the state components and their time relationships is essential before interpreting any estimated position, velocity, or other changing quantity.

Study task. Construct a constant-velocity state model and label its time and measurement updates. Compare what information is available for prediction, filtering, and smoothing at one selected epoch.

Chapter 4 State-space models for dynamic systems

State-space descriptions broaden the dynamic model beyond simple polynomial motion. Kinematics specifies movement, while dynamics introduces the physical causes of that movement. The chapter develops state vectors, state equations, linearization of nonlinear systems, and the propagation of linear time-varying and time-invariant models. The matrix exponential supplies an important transition operator in the time-invariant case, with alternative evaluation methods considered. A state must contain enough information to support the assumed future evolution, but it should not be confused with the complete measurement vector. Understanding those roles makes the later addition of stochastic disturbances much clearer and prevents a physical modelling assumption from disappearing inside matrix notation.

Study task. Define state and observation vectors for a simple moving object. Explain the dimensions of the transition and observation matrices and identify where a nonlinear model would require linearization.

Chapter 5 Random functions

Random functions allow uncertainty to enter the description of a dynamic system rather than being attached only to its measurements. Mean functions and covariance relationships describe ensemble properties and dependence across time. The chapter derives propagation relationships for linear systems, including cross-covariance and output autocovariance, and considers white-noise and correlated inputs. White noise is an idealized statistical model, not a statement that real disturbances are literally identical or physically unstructured. The examples of random motion models show how assumptions about driving disturbances alter uncertainty in the evolving state. Tracking correlation is therefore as important as calculating a single variance at one time.

Study task. Compare independent measurement noise with a disturbance correlated across time. Sketch plausible covariance patterns and explain why the two assumptions lead to different confidence in a sequence of state estimates.

Chapter 6 Recursive least-squares: the dynamic case

The dynamic case combines uncertain state evolution with sequential observation updates. It addresses the limitation of treating a trajectory as completely determined by one initial state, an assumption that can make a filter increasingly confident in a model that no longer fits reality. Process uncertainty permits departures from that idealized evolution, while measurements contribute new information with their own covariance. The resulting prediction and filtering equations connect model propagation, innovation, gain, and updated uncertainty. State augmentation can represent additional correlated processes, including velocity or acceleration disturbances. The central lesson is that filter quality depends on credible deterministic and stochastic models, not on recursion alone or on obtaining a small reported covariance.

Study task. Explain how underestimating process uncertainty could contribute to filter divergence. Design a simulated tracking comparison with two process-noise assumptions and evaluate errors alongside the reported uncertainty.

How to study this book

1. Check matrix dimensions, rank assumptions, and covariance symmetry before attempting a derivation or implementation.
2. Derive each update first in a scalar example, then identify its matrix counterpart.
3. Keep time updates and measurement updates separate in notes and code; record which epoch every vector describes.
4. Compare a sequential solution against a batch reference when the two models are mathematically equivalent.
5. Use simulations with known states to assess both numerical error and whether the uncertainty estimates are credible.

Questions for reflection

• What information must a recursive solution preserve to avoid retaining all observations?
• Why are iteration and recursion different operations in a nonlinear estimation problem?
• How can a small covariance coexist with a poor physical estimate?

Source edition and coverage

The guide covers all six numbered chapters in the linked second edition, published by TU Delft OPEN Publishing in 2024. It assumes the mathematical prerequisites described in the source foreword. This original study guide explains the source in new wording and adds learning activities. Read the publisher’s book and complete exercises.