8

This TED presentation says that the New York Department of Education is using a secret algorithm to evaluate teachers:

(..) teachers were being scored with a complex, secret algorithm called the "value-added model (..)

I have looked for this algorithm and there is some criticism surrounding it (Huffington Post):

An algorithm is an automatic way to determine results. No need to have those pesky human factors like judgment, emotion, discernment, understanding, empathy, sympathy, critical reasoning involved.

While using or not an algorithm for evaluation is clearly debatable, I am interested mostly in the not being public part. This sounds strange as teachers do not know what they should improve if they are penalized by it.

Question: Why was the "value-added model" algorithm kept secret?

Improve this question
16
  • 2
    @KDog it is a policy of public employee evaluations.
    – user9389
    Commented Jul 26, 2018 at 15:57
  • 2
    And who owns the IP on the algo?
    – user9790
    Commented Jul 26, 2018 at 16:00
  • 8
    @KDog In general, there is no real IP on algorithms (although implementations can be copyrighted). It's probably a secret to prevent teachers from attempting to game the algorithm artificially inflate their score.
    – user11249
    Commented Jul 26, 2018 at 16:46
  • 6
    I dont know enough details to know if/why this specifically would have been done. But one reason why you would hide the details of an imperfect algorithm is to prevent people from "gaming the system" or "playing to the test". Basically if there is no knowledge of what is analyzed, people have to act normally and just hope that the magic black-box doesnt overlook their work. Thats all they can do. If they know the algorithm, they can focus on only things that the algorithm grades while ignoring other things, even if those other things are actually really important.
    – Tal
    Commented Jul 26, 2018 at 17:43
  • 4
    @KDog It seems clear that this is a public policy question. According to the article, the New York Department of Education used this approach to score teachers. So the policy question is, "why was it a secret?".
    – indigochild
    Commented Jul 26, 2018 at 20:18

2 Answers 2

Reset to default
4

Algorithm is kept secret because it's developed by a private company and as such proprietary (source).

As far as I know this is the system used by NY Board of Education, however I was not able to confirm it yet. But if it is then it answers your question: it is a product on which many fortunes depend.

Improve this answer
3
  • Same reason why we can't (or "couldn't," if that has changed) look at source code in voting machines to make sure they are honestly tabulating votes.
    – PoloHoleSet
    Commented Aug 1, 2018 at 17:20
  • @PoloHoleSet - Well, in some countries there is a requirement to keep votes on paper, too (in case of machines printouts). But from experiences in voting in my home country I can tell you that doesn't matter. Stalin said that who's counting the votes matters, not who votes. You wouldn't believe what's going on there. One can also asks Scots - their independence vote was as fine a racket as possible...
    – user10424
    Commented Aug 2, 2018 at 10:39
  • Believe me, I'm not claiming that the USA way for stuff like this is the desirable mode, at all.
    – PoloHoleSet
    Commented Aug 2, 2018 at 15:33
2

One reason to keep scoring algorithms like this secret is to reduce the ability for people to "game" your system by performing poorly but scoring highly. By keeping an algorithm like this a secret, it hopefully encourages people to simply perform their best and hope the algorithm reflects that.

Improve this answer
2
  • If the algorithm can be gamed in such a manner, that indicates that it's not fit for purpose and should be fixed.
    – Vikki
    Commented Jul 21, 2022 at 20:16
  • @Vikki All possible algorithms are likely to be gameable, it'd be very difficult to come up with something that's impossible to game. Consider how many people work on ensuring that actual games are balanced, and yet people still manage to min-max them in undesirable and unexpected ways even without significant financial advantages.
    – David Rice
    Commented Jul 23, 2022 at 21:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .