SortedSet<string> sorted = new SortedSet<string>();
for (int i = 0; i < testList.Count; i++)
{
  if (sorted.Contains(testList[i].TestValue))
    testList[i].IsDuplicate = true;
  else
    sorted.Add(testList[i].TestValue);
}

A few things:

1. As you have allowed in the question, I'd change testList to be an array instead of a list, to make indexer faster.
2. ImmutableSortedSet can also be used to make things even faster with a multi-threaded solution.

SortedSet<string> sorted = new SortedSet<string>();
for (int i = 0; i < testList.Count; i++)
  testList[i].IsDuplicate = !sorted.Add(testList[i].TestValue);

As you have allowed in the question, I'd change testList to be an array instead of a list, to make indexer faster.