A notable property of frictional forces is that they resist motion (as opposed to other types of forces, which might resist displacement, for example, which is how a spring behaves). As a result, the brakes on your car slow down the motion of your wheels that produces forward movement of your car—but they would also slow down motion that produces reverse movement.
If instead you used another type of force-applying system to slow down your car (e.g., a giant spring), then your car would slow down, then stop, and then start moving backwards.