What are ways besides the hosts file, that I can redirect or otherwise block websites entirely through the use of software?
Malwarebytes Premium is able to redirect websites it detects are bad and redirects them to their block page - without extensions or seemingly obvious methods.
How does something like Malwarebytes redirect the request to a different page?
I'd thought I'd ask here due to it being more on the internal workings of Windows than a coding issue.
I don't know about Malwarebytes' HTTP filter specifically, but I assume it works in a similar way to Fiddler.
Fiddler itself is a HTTP Proxy that handles requests. It works by running a HTTP server on port 8888 that accepts both HTTP and HTTPS connections.
When you run Fiddler, it reconfigures Windows' systemwide Internet connection settings to set itself as a systemwide HTTP Proxy. You can see this if you look at Control Panel > Internet Options > Connections > LAN Settings > Proxy server > Advanced > HTTP: 127.0.0.1:8888. When you close Fiddler it reverts the settings back to their previous values.
These settings inside Internet Options are automatically used by any application that uses WinINet and WinHTTP (these are Windows components that handle HTTP requests for userland application software). Software that doesn't use WinINet or WinHTTP (such as Java and .NET programs which have their own HTTP client libraries) often respect the WinINet/WinHTTP system settings anyway (in most cases, anyway). Chrome also has its own HTTP client stack but also respects system settings too. I believe Firefox is the same.
These HTTP proxies then receive the requests that would normally go directly to the intended website, they can then inspect the request and handle it (e.g. to reject it) or forward it on to the intended destination. HTTPS can be handled with a private X.509 certificate that is trusted and valid for all hosts - or by forwarding those requests on. Note that if your certificate is not truly private and specific to your machine then this is a massive security liability and you should remove the certificate immediately.
Fiddler, and how it works, is described in this blog article: http://www.mehdi-khalili.com/fiddler-in-action/part-1
Besides HTTP and HTTPS proxies, SOCKS proxies also provide an injection route - though they're an older approach not seen much thesedays (as SOCKS is for all TCP and UDP traffic, whereas HTTP proxies only handle HTTP specifically).
Windows Vista introduced a new extensibility point in Windows called the Windows Filtering Platform (WFP) which provides a nicer API than a manual HTTP/HTTP proxy configuration, and it allows software to filter more things than just HTTP traffic. It's also built right-in to the networking stack which means that it should be able to intercept all HTTP traffic, even from applications that don't respect the WinINet or WinHTTP configuration.
This API was updated in successive versions of Windows so it's actively supported and it seems like it's built specifically for security products:
With the WFP API, developers can implement firewalls, intrusion detection systems, antivirus programs, network monitoring tools, and parental controls.
Windows Filtering Platform is a development platform and not a firewall itself. The firewall application that is built into Windows Vista, Windows Server 2008, and later operating systems – Windows Firewall with Advanced Security (WFAS) – is implemented using WFP.
It is worth pointing out that WFP is a kernel-mode API - which means it will be harder to program against, and that if a WFP implementation has a bug and crashes then it has the potential to cause a BUGCHECK (Blue Screen of Death), so it's possible that developers might prefer to use a HTTP proxy instead of a WFP because it's easier and less likely to result in a poor user-experience.
This is documented here: https://msdn.microsoft.com/en-us/library/windows/desktop/aa366510%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396
