I want to find the equations governing the TE and TM modes on the following three-layered slab waveguide:
I know how to use the Helmholtz equation and solve for the boundary conditions on normal isotropic waveguides, but I have no idea how to procceed when the permittivities are given by matrices. Any advice would help, thank you very much!
I don't know if this will fully answer your question, but as a suggestion, you may want to try to solve the problem component-wise assuming that the permittivity matrices are isotropic first (i.e. same value on each diagonal).
In other words, assume something like:
$$ \begin{bmatrix} D_{x} \\ D_{y} \\ D_{z} \end{bmatrix} = \begin{bmatrix} \epsilon & 0 & 0 \\ 0 & \epsilon & 0 \\ 0 & 0 & \epsilon \\ \end{bmatrix} \begin{bmatrix} E_{x} \\ E_{y} \\ E_{z} \end{bmatrix} $$
but keep each component separate in your calculations. Once you have convinced yourself that you get the correct answer (which I believe you already know how to do), go back and make the matrices anisotropic and redo the calculations. Since you have already seen how the process works component-wise in the isotropic case, this may help guide your calculations in the other case.
As another suggestion, you may want to consult a book like Born and Wolf Optics (https://archive.org/details/principlesofopti00born/mode/1up) Section 14, where they discuss electromagnetic solutions in anisotropic crystals. This may be of some use as well.
In theory it is not uniaxial slab. Wide isotropy is not good permittivity.
In helmholtz equations there is no good approach for matrices.
