It feels like you're confusing two different kinds of public keys in SSH. There are host keys and there are user keys – they both use the same key format, but they have complete opposite purposes: host keys are received from the server (like TLS certificates); user keys are sent to the server (for user login purposes).

After hitting the login button, Host Key Verification window shows up with host key info. (Not generated by me. Should this be happening by default.)

Yes, because this is the host key – it is received from the server, not sent by you to the server.

(SSH servers do generate their host keys by default, without admin involvement. But that has nothing to do with your SSH client software.)

For Public Key Authentication, I tried generating keypair from client key manager and then I exported public key to my webserver and it said key invalid. (What are the things that could be wrong here?)

There are several formats to represent the same keys – the one-line "OpenSSH" format that you'd use in authorized_keys; the "SSH2" format; even the "PEM" format on occasion. If the server expects an OpenSSH-format public key, you must select that format when exporting.

(This is assuming you tried to use the public key in a place that expects SSH keys, and not mixing it up with some other feature that expects TLS keys.)

If I generate or import keypair, the Host Key Verification window does not show I generated. (This is where you are prompted to accept fingerprint info.)

You are generating user keys, but the "Host Key Verification" window is not about that at all – it's not showing you your own key; it's showing you the key that identifies the server.

It is bad to accept Host key without verifying that your fingerprints matches because bad actors could mess around with your system such as sending virus to your computer or stealing sensitive information.

They cannot mess around with your system (assuming a non-buggy SSH client).

It is bad to accept Host key without verifying for other reasons – it functions much like TLS certificates in that it verifies which host you're connecting to. If you skip verification, and you're connecting from an untrusted location, then you might be connecting to an attacker's MITM host which will happily steal your SSH passwords and everything else that you send through that SSH connection.

I am assuming that is a dummy Host key info from the software itself. Maybe to insure you have matching fingerprints.

No, it's not. It's real host key information from the server that you're connecting to.

Software is encouraging you generate a host key as an added security measure other than password or you can just login with just the Host key.

Host keys have absolutely nothing to do with user logins. Logging in requires a user key – same concept, same algorithms, but the usage is the complete opposite. (Host keys let the client verify the server; user keys let the server verify the client.)

In most SSH servers, the user keypairs work instead of a password. It is possible to require both a key and a password, but it's uncommon. (I know OpenSSH supports that, I believe Bitvise WinSSHd also does.)

It sounds like you're confusing two different kinds of public keys in SSH. 

There are host keys and there are user keys – they both use the same key format, same algorithms, but they have complete opposite purposes: host keys are received from the server (like TLS certificates); user keys are the ones that would be sent to the server (for user login purposes).

After hitting the login button, Host Key Verification window shows up with host key info. (Not generated by me. Should this be happening by default.)

Yes, because this is the host key – it is received from the server, not sent by you to the server. Hence the name "host key".

(SSH servers indeed generate their host keys by default, without admin involvement. But that has nothing to do with your SSH client software.)

For Public Key Authentication, I tried generating keypair from client key manager and then I exported public key to my webserver and it said key invalid. (What are the things that could be wrong here?)

There are several formats to represent the same keys – the one-line "OpenSSH" format that you'd use in authorized_keys; the multi-line "SSH2" format; even the "PEM" format on occasion. If the server expects an OpenSSH-format public key, you must select that format when exporting.

If I generate or import keypair, the Host Key Verification window does not show I generated. (This is where you are prompted to accept fingerprint info.)

You are generating user keys, but the "Host Key Verification" window is not about that at all – it's not meant to be showing you your own key (there is no point in verifying that); it's showing you the key that identifies the server.

It is bad to accept Host key without verifying that your fingerprints matches because bad actors could mess around with your system such as sending virus to your computer or stealing sensitive information.

They cannot mess around with your system (assuming a non-buggy SSH client).

It is bad to accept Host key without verifying for other reasons – it functions much like TLS certificates in that it verifies which host you're connecting to. If you skip verification, and you're connecting from an untrusted location, then you might be connecting to an attacker's MITM host which will happily steal your SSH passwords and everything else that you send through that SSH connection.

I am assuming that is a dummy Host key info from the software itself. Maybe to insure you have matching fingerprints.

No, it's not. It's real host key information from the server that you're connecting to.

Software is encouraging you generate a host key as an added security measure other than password or you can just login with just the Host key.

Host keys have absolutely nothing to do with user logins. Logging in requires a user key – same concept, same algorithms, but the usage is the complete opposite. (Host keys let the client verify the server; user keys let the server verify the client.)

In most SSH servers, the user keypairs work instead of a password. It is possible to require both a key and a password, but it's uncommon. (I know OpenSSH supports that, I believe Bitvise WinSSHd also does.)