The matrix equations we have created define two matrices, \(\textbf {D}\) which is called the design matrix and matrix \(\textbf {H}\) which we call the Hessian matrix, though I will note that our applied math friends will tell us that this is only an approximation to the Hessian, as the actual definition of the Hessian has higher order derivatives in it. Note that what we are calling the Hessian matrix is dependent on the partial derivatives of the model function evaluated and summed at every data point. This matrix is diagonally symmetric in that \(H_{kj} = H_{jk}\). The diagonal values of the Hessian are only dependent on the derivative of each parameter, but the off-diagonal values depend on the product two different parameter’s derivatives. In this way, the Gauss-Newton method takes into account how changing one parameter can effect another.
Note that should the derivatives of two parameters be too similar, then we have that problem I discussed before, where the Hessian becomes nearly singular and matrix inversion becomes increasingly inaccurate. When this happens, minimization becomes poor and eventually fails. GSAS-II uses two “tricks” to help deal with singularities. The first is that if the derivatives for a parameter are all zero, which means that the parameter has no effect on the quality of the fit, then the minimizer simply omits that parameter from the refinement. Secondly, it uses a process called singular value decomposition (SVD) to replace a set of parameters that have derivatives that are too similar with a smaller set of composite variables. If minimization still fails, then yet another trick, automatic Marquardt damping is applied, as described below in §7.8. How that works will make more sense after we discuss correlation and steepest descent minimization.