Linear Algebra provides a compact notation to deal with simultaneous equations:
where the problem is how to find the values for \(x_j\) when we know the \(A_{jk}\) and the \(b_k\) values. Note that these \(n\) equations can also be written in summation form with n equations \[\sum _{k=1}^m A_{jk} x_k = b_j\] Using linear algebra notation we can simplify the \(n\) equations into a single one:
\[\mathbf {A} \mathbf {x} = \mathbf {b}\] where \(\mathbf {A}\) is matrix with \(n\) rows and \(m\) columns, \(\mathbf {x}\) is a column vector of length \(m\) or equivalently a matrix with \(n\) rows and one column, and \(\mathbf {b}\) is a row vector of length \(n\) or equivalently a matrix with 1 rows and and \(n\) columns. We also need to define how matrix multiplication works, \[\mathbf {A} \mathbf {B} = \mathbf {C}\] then \(C_{jk} = \sum _\eta A_{j\eta } B_{\eta k}\). Note that in general, \(\mathbf {A} \mathbf {B} \neq \mathbf {B} \mathbf {A}\).
One more important property is needed, which is that of matrix inversion. We define the inverse of matrix \(\mathbf {A}\) as \(\mathbf {A}^{-1}\) such that \(\mathbf {A}^{-1} \mathbf {A} = I\), where \(I\) is an identity matrix, where \(I_{jj} = 1\) and \(I_{jk} = 0\) for \(j \neq k\), or equivalently, \[ I = \begin {bmatrix} 1 & 0 & \cdots & 0 \\ 0 & 1 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 1 \end {bmatrix} \] Note that matrix inversion can only be performed with a square matrix. Thus, we can solve the previous equation for \(\mathbf {x}\) using: \[\mathbf {A}^{-1} \mathbf {A} \mathbf {x} = \mathbf {A}^{-1} \mathbf {b}\] \[\mathbf {x} = \mathbf {A}^{-1} \mathbf {b}\] So how do we determine \(\mathbf {A}^{-1}\)? For that in the general case, I will refer you to an applied mathematician, but the \(3\times 3\) example is “simple” enough that I can write that in closed form. If we define \(\mathbf {A}\) using \[ \mathbf {A} = \begin {bmatrix} a & b & c \\ d & e & f \\ g & h & i \end {bmatrix} \] then we get the inverse as \[\mathbf {A}^{-1} = \begin {bmatrix} a & b & c \\ d & e & f \\ g & h & i \end {bmatrix}^{-1} = \frac {1}{|\textbf {A}|} \begin {bmatrix} ei-fh & ch-bi & bf-ce \\ fg -di& ai-cg & cd-af \\ dh-eg & bg-ah & ae-bd \end {bmatrix} \] where the determinant of \(\textbf {A}\) which is written \(|\textbf {A}|\) is given by \(|\textbf {A}| = a(ei-fh) + b(fg -di) + c(dh-eg)\). Not simple!
Not every matrix can be inverted. If any row or column of the matrix is repeated, even if multiplied by a constant, we get a zero determinant. For example, let us look at \[ \mathbf {A} = \begin {bmatrix} a & b & na \\ d & e & nd\\ g & h & ng \end {bmatrix} \]
Then we get \begin{align} |\textbf {A}| &= a(eng-ndh) + b(ndg -dng) + na(dh-eg)\\ &= n(aeg-adh + bdg -bdg + adh-aeg)\\ &= 0 \end{align}
Likewise, a matrix where all the elements in a row or column are 0 will also have \(|\textbf {A}| = 0\) and also cannot be inverted. A matrix that cannot be inverted is called a singular matrix.
One other comment here related to matrix inversion is that if you look at the \(3\times 3\) example above, you will see that the terms in the inverse matrix involve lots of difference terms, where the product of two terms is subtracted from the product of two other terms. This throws a red flag to people used to doing numerical analysis with computers. Computers usually store numbers to only limited precision. When subtracting two numbers that are very close in size, the difference becomes more and more influenced by the precision used to store the numbers. The closer the two numbers are that are subtracted, the more the resulting value is determined by what is called the “round-off” error. This means that as as two rows or columns in the matrix become similar, the matrix becomes nearly singular and the computed inverse to the matrix becomes dominated by round-off errors. GSAS-II is a major advance over GSAS/EXPGUI, in that GSAS-II uses twice as many digits (64 bits) to store numbers and did GSAS/EXPGUI (which used 32-bit math). That reduces the impact of this problem but does not eliminate it.
Finally, a few more linear algebra concepts. The transpose of a matrix swaps the rows and columns of the matrix, where \(\mathbf {A}^T\) is the transpose of \(\mathbf {A}\) so that \[ \mathbf {A}^T = \begin {bmatrix} a & d & g \\ b & e & h\\ c & f & i \end {bmatrix} \] In addition, we can make a distinction between the matrix elements, \(A_{ij}\), where \(i=j\) from those where \(i\neq j\). Where \(i=j\), corresponding to the \(a\), \(e\), and \(i\) elements in \(\textbf {A}\) and \(\textbf {A}^T\), we call those the diagonal terms and the other terms are the off-diagonal terms. When all the off-diagonal terms in matrix are 0, such as in the Identity matrix, we call that a diagonal matrix.