In general, The connection should be like PC---SW---ROUTER---Many ISP router---Router---SW---PC
so the packet will be from layer 2 ---layer 3---layer 2
What if I put one layer 2 switch in the network
PC---SW---ROUTER---MANY ISP ROUTER---LAYER 2 SW---MANY ISP ROUTER---ROUTER---SW---PC
Can any network form between 2 PC? What should be configured on the layer 2 SW in the middle of the network ? I would say default gateway is not enough ? Of course , no normal people would do this ,but what-if ?
I can assure you ISP infrastructure contains switches, just like your LAN contains a switch. All layer-3 protocols (IP, IPv6, IPX, Appletalk, etc.) have to move over some layer-2 protocol, most commonly ethernet. The IP(L3) traffic from "PC" flows over an ethernet(L2) network to get to the router. Similarly there are several possible L2's from the router out to the ISP -- ethernet to a modem, and then DSL, xPON, cellular, etc. from the modem to the ISP hardware. (the device your modem connects to very likely isn't a layer-3 "router", but a bridge to yet other layer-2's.)
In your diagram, assuming 12.0.0.0/8 and 14.0.0.0/8 IP subnets, it doesn't matter if they have an ethernet interface in the same broadcast domain (i.e. VLAN 20), they will never attempt to communicate directly via that L2 path. (without "interface routes", not to muddy the water.) Even with that switch out in the middle, the PC's don't have a direct (layer-2) path between them; they still have numerous routers (layer-3) to get through.
There are various VPN / Tunneling methods that can bridge layer-2 networks across layer-3 networks. (VXLAN is the current "shinny thing" for doing that.)
From a PC to another PC in another subnet, that's layer 3. When two layer 3 router sandwiches a layer 2 switch, the destination IP is, of course, the destination PC IP address. But on the mac address, it will be the next hop mac address, not the destination mac address. From the packet gets to the first router it will have a packet sent with the destination mac address of the next router, so when the middle switch has the packet with the destination mac-address of the next hop it can use the layer 2 mac-address-table to send to next hop.
A network (L2 segment / IP subnet) is connected by a switch or VLAN. Routers route between networks.
An additional switch in that place allows you to connect additional devices to the transfer network. You've also added another "many ISP router", please check whether that's intentional.
There are some device connected to that additional sw which cannot be migrated or moved to other place
That's not a good reason to add device to a transfer network.
I want to make sure that when some layer 3 packet flowing through the additional sw won't block it or drop the traffic accidentally
That depends entirely on the IP configuration that you told us nothing about.
when 2 OSPF router can form neighbourhood when they both connect to the same router(same VLAN), so that the additional sw is ignored in this situation ?
OSPF uses multicast or explicit unicast to discover neighbors and form adjacencies. As long as you don't change the IP configuration, an additional switch doesn't make a difference.
From the diagram, it looks like you're connecting a switch to the routers' WAN uplinks. That doesn't make sense in the scenario that you describe.
What you seem to want(?) is to use additional, routed ports on the routers and route between them using a private transfer network - no switch necessary. With static routes on either side (or OSPF) you could transparently route between the PCs connected to both routers. If that is what you're trying to achieve.
