Numerical optimization is a major aspect of applied mathematics and there have been many methods developed that are far more advanced than what will be discussed here, but least-squares fitting has been used traditionally in crystallographic fitting and a slightly improved method is used in GSAS-II. During my career in science, I have grown increasingly impressed with the field of applied math, but before I explore the topic in this chapter, I will point out that I am not very skilled in the area.
Is it necessary that you fully understand the math behind least-squares fitting in order to use it? Probably not, but there are some key concepts that I do think are very important to discuss, which include the affects of parameter correlation and how GSAS-II switches between least-squares minimization and steepest descent minimization, when correlation becomes problematic.
While least-squares is old, it does have one significant advantage over many some minimization approaches, in that it provides error estimates (standard uncertainty values) for the fitted parameters. We will see how that works, as it is also important.