Total derivatives

Let \(\require{physics} S\) be an open subset of \(\mathbb{R}^n\) and \(\vb{a}\), such that we define the scalar function \(f\): \[\begin{equation} \vb{f}: S \rightarrow \mathbb{R}: \vb{x} \mapsto f(\vb{x}) \thinspace , \end{equation}\] Let \(\vb{a}\) be an interior point of \(S\), and let’s choose an \(\vb{r}\) such that the open \(n\)-ball is contained by \(S\): \[\begin{equation} \mathcal{B}(\vb{a},\vb{r}) \subseteq S \thinspace . \end{equation}\] Now let’s choose a \(\vb{v}\) with \(||\vb{v}|| < ||\vb{r}||\) such that \[\begin{equation} \vb{a} + \vb{v} \in \mathcal{B}(\vb{a},\vb{r}) \thinspace . \end{equation}\] These are all the ingredients we need to define differentiability. We call the function \(f\) differentiable at \(\vb{a}\) if there exists a linear transformation \(\vb{T}_{\vb{a}}\) \[\begin{equation} \vb{T}_{\vb{a}}: \mathbb{R}^n \to \mathbb{R} \end{equation}\] such that \(f\) admits a first-order Taylor formula: \[\begin{equation} f(\vb{a} + \vb{v}) = f(\vb{a}) + \vb{T}_{\vb{a}}(\vb{v}) + ||\vb{v}|| E(\vb{a},\vb{\vb{v}}) \thinspace , \end{equation}\] in which \(E(\vb{a},\vb{\vb{v}})\) is a scalar function with the behaviour that \(E(\vb{a},\vb{\vb{v}}) \to 0\) if \(||\vb{v}|| \to 0\). We call this linear map the total derivative and we can equivalently write: \[\begin{equation} \lim_{h \to 0} \qty( \frac{f(\vb{a} + h \vb{y}) - f(\vb{a}) - h \vb{T}_{\vb{a}}(\vb{y})}{h} ) = 0 \thinspace . \end{equation}\]

The total derivative is related to the directional derivative: \[\begin{equation} \vb{T}_{\vb{a}}(\vb{y}) = f'(\vb{a};\vb{y}) = \grad_{\vb{y}} f(\vb{a}) = \lim_{h \to 0} \qty( \frac{f(\vb{a} + h\vb{y}) - f(\vb{a})}{h} ) \thinspace , \end{equation}\] and the following useful formula holds: \[\begin{equation} \vb{T}_{\vb{a}} (\vb{y}) = \grad{f(\vb{a})} \cdot \vb{y} \thinspace , \end{equation}\] in which we have introduced the gradient of \(f\) at the point \(\vb{a}\). This is the vector of partial derivatives: \[\begin{equation} \Big( \grad{f(\vb{x})} \Big)_i \equiv \Big( \pdv{f}{\vb{x}} \Big)_i = \pdv{f(\vb{x})}{x_i} \thinspace . \end{equation}\]

The directional derivative of a function \(f\) along a vector \(\vb{a}\) is defined as \[\begin{equation} \grad_{\vb{a}} f(\vb{x}) = \lim_{h \to 0} \qty( \frac{f(\vb{x} + h \vb{a}) - f(\vb{x})}{h} ) \end{equation}\] and can be calculated using \[\begin{equation} \grad_{\vb{a}} f(\vb{x}) = \grad{f(\vb{x})} \cdot \vb{a} \end{equation}\] for functions that are differentiable at \(\vb{x}\).

Some useful formulas concerning the derivative of scalar fields with respect to a vector are: \[\begin{align} & \pdv{\vb{x}} (\vb{x} \cdot \vb{y}) = \vb{y} \\ & \pdv{\vb{x}} (\vb{x} \cdot \vb{x}) = 2 \vb{x} \\ & \pdv{\vb{x}} (\vb{x} \cdot (\vb{A} \vb{y})) = \vb{A} \vb{y} \\ & \pdv{\vb{x}} (\vb{y} \cdot (\vb{A} \vb{x})) = \vb{A}^\text{T} \vb{y} \\ & \pdv{\vb{x}} (\vb{x} \cdot (\vb{A} \vb{x})) = (\vb{A} + \vb{A}^\text{T}) \vb{x} \thinspace . \end{align}\]