This is my PCB stackup.
In the top layer, I have a 50 Ω controlled RF trace. The bottom reference of the trace is in L3.
I have a cutout under the trace on L2 to maintain the 50 ohm impedance to the reference L3.
Between top to L3, there are two different dielectrics with different Dk (pp1=3.66, pp2=4.4.)
What will be the effective dielectric constant for the RF trace to L3? How can I calculate it?
If you have vertically stacked dielectrics then this effectively forms two capacitors in series: One with \$\epsilon_r=3.66\$, and the other with \$\epsilon_r=4.4\$.
If the thicknesses are equal then the effective dielectric constant will be
$$ \epsilon=2 \ \frac{\epsilon_{r1} \ \epsilon_{r2}}{\epsilon_{r1} + \epsilon_{r2}} $$
If the thicknesses are different then you need to work out from
$$ C =\epsilon_r \ \epsilon \frac{A}{d} $$
for each dielectric and its thickness, and
$$ \frac{1}{C}=\frac{1}{C_1}+\frac{1}{C_2} $$
to obtain the final dielectric constant.
If your stackup/trace impedance tool can't handle that topology, I would just calculate an average Dk based on the thickness and Dk of the two materials.
