First, you should understand that .gitignore doesn't mean "ignore these files". Git should maybe have named this file .git-dont-complain-about-these-files-if-they-are-untracked-and-dont-automatically-add-these-files-to-the-index-if-they-are-untracked-but-note-that-this-has-no-effect-at-all-on-tracked-files, rather than just .gitignore, but who wants to type all that in every time?
Next, you—and your entire team, for that matter—need to understand what "untracked files" really means. Fortunately, the definition of an untracked file, in Git, is very simple: An untracked file is a file that is not in the index. If a file is in the index, it is tracked. If it is not in the index, it is untracked.
Alas, this usually leaves everyone wondering about "the index". This is especially bad because the index is a central concept in Git, that everyone must understand in order to use Git correctly. Git tutorials always mention the index, which goes by three different names in Git: index, staging area, and cache. But many leave it as a vague, puzzling entity, or do not explain it very well. Git's index has several different roles, and explaining all of them is a little bit tricky, especially because the index is hard to see; but to a first approximation, the index is simply a copy of every file that will go into the next commit you make.
(This is where the index gets the name staging area. If a file is in the index right now, and you run git commit, the copy that is in the index right now is the one that goes into the commit. Hence the one that's in the index is staged. Note that when git status says "staged", it means specifically that the file is (a) in the index and (b) different, in the index, from what's in the current commit.)
This is why Git keeps making you git add the same files over and over again. When you have an existing commit, it has a bunch of files in it. Those files are also in the index, with the same contents as they have in the comment. And, those same files are in your work-tree—the place where you do your work on files—with (initially anyway) the same contents as they have in the commit and in the index. You have to copy the new contents over the old ones, using git add, to stage newer versions of the file.
Anyway, to make the file be not in the index, you must remove it from the index, using git rm filename. By default, this removes the file from both the index and your work-tree. Using git rm --cached filename, you can tell Git to remove the file from the index, while leaving it alone in the work-tree. Now that it's not in your index, it's an untracked file, so now .gitignore can affect it.
When you make a new commit now, since the file is not in the index, it won't be in the new commit either. This produces a problem.
Suppose you, or your co-workers, have an old commit checked out, and that old commit has the file in it. Because you have that old commit checked out, the file is in your index—remember, your index matches your checked-out commit—and also in your work-tree: Git fills your work-tree with the files it copies into your index based on the commit you're checking out. You now have the file tracked (so it's not ignored). If you git checkout the new commit, where the file isn't tracked, Git will immediately remove the file from your index ... and remove the file from your work-tree too.
The good news is that you're now on a commit that doesn't have the file committed, and hence doesn't have the file in the index. So now the file is untracked. The bad news is, the file isn't even there, becuase Git removed it.
Git will do this every time you switch from a commit that has the file, to a commit that doesn't have the file, because when you're on a commit that does have the file, the file is in the commit and hence copied into the index and hence copied into your work-tree, but you are switching from that commit to one that doesn't have the file, so Git removes it from your index and removes it from your work-tree.
Unfortunately, this is the path you've chosen by having the file in some commits, and not having the file in other commits. To keep the file in your work-tree, you will have to rename it out of the way, then switch from "commit that has the file" to "commit that doesn't have the file". Git will remove the file from the index (great) and try to remove it from the work-tree (which fails, but Git doesn't care because the result is correct). Then you can safely rename it back. But you have to do this all the time you switch in and out of these old commits.
Meanwhile, everyone who's going to make a new commit must make sure they also remove the file from their index if necessary, so that all the new commits omit the file, so that it doesn't go back into anyone else's index when they fetch those new commits and then check them out.
There is one other option
Instead of making everyone remove the file from each index they have, you can choose to direct your Git to skip your work-tree version of your file when comparing your index to your work-tree. That lets you leave the file in your index (so that the file isn't ignored), but also keeps Git from adding updated versions to your index (so that your index version stays as-is). To do this, you can use git update-index --skip-worktree filename.
Note: I don't recommend this method of working. It can work, but every user has to do this for each index file that is to be skipped this way.
