Given two real-valued symmetric positive-definite matrices $A$ and $B$. Assume that $A\succeq I$. Let $\Delta = B-A$. We are interested in a bound on $\Vert B^{-1}-A^{-1}\Vert$ in terms of $\Vert \Delta\Vert$.
One attempt from Upper bound of a norm of an inverse of a matrix says: $$ \Vert A^{-1} -B^{-1} \Vert \leq \frac{\|A^{-1}\|^2\|\Delta\|}{1-\|A^{-1}\Delta\|}, $$ given the condition that $\Vert A^{-1}\Delta\Vert <1$. But in our case, we assume $A, B$ are positive definite. Can we obtain a tighter bound on the norm by leveraging this assumption?
And we are interested in a bound taking a particular form: $\Vert B^{-1}-A^{-1}\Vert \leq c(A)\Vert \Delta\Vert$, where $c$ is a scalar dependent on $A$.
One way is to assume $\Vert \Delta\Vert$ is small enough such that $\Vert \Delta\Vert \leq 1$ and also $\Vert A^{-1}\Vert\leq1$. Then, we obtain $\Vert B^{-1} - A^{-1}\Vert\leq \frac{\Vert A^{-1}\Vert^2\Vert\Delta\Vert}{1-\Vert A^{-1}\Vert}$.
But I still could figure out how to leverage the positive definite assumption to obtain a tighter bound or a bound with less assumption on $\Vert A^{-1}\Vert$ and $\Vert \Delta\Vert$.
