I set up a new Lenovo Windows 11 Pro ThinkPad in early March this year (2022).
Windows Update installed most of the new Lenovo Drivers at that time. Ordinary drivers were installed. Firmware (IME, UEFI BIOS) were not.
The problem is when I install windows with internet connection windows
automatically install all OEM drivers right after the first boot up.
There is no problem doing this - it works just fine.
Is it fine to just download and install laptop manufacturer drivers on
top of OEM drivers installed by windows?
There is no need to do this. The Windows Drivers come from the Windows Driver Catalogue which is populated by the Manufacturers with the correct drivers.
In fact, a good quality Driver Update App (e.g. Lenovo) will not duplicate the installation.
I'm curious if there will be some left over files if I just install
laptop manufacturer drivers on top of windows OEM drivers.
There should not be many. Lenovo puts most installations in C:\Drivers, for example, and once installation is complete, these can be deleted.
Summary: You are fine to let Windows Update do Driver updates. Check occasionally with the Manufacturer's Driver Update App for Firmware (IME, UEFI BIOS and like firmware).
Also, it is possible that Drivers appear on the Manufacturer's website before being in the Manufacturer's Driver Update App and possibly before being in the Windows Update Catalogue. There is nothing you can do about this, so probably just wait for Windows to offer the update.
Some of the Driver Updates will be in Windows Optional Updates, so check that as well.