A/B testing to create a heuristic pain scale
A good way of predicting pain eludes modern science, not because we don't know how to test it, but because we can not ethically test it. The reason modern pain scales are so vague is because when a doctor sees a patient for a neuroma, he can't then also break the person's arm to determine which one is actually more painful.
Imagine you have a pool of 100 pain experiments: broken arms, stubbed toes, loud noises, root canals, jellyfish stings... all sorts of stuff. Using humans as test subjects, you inflict 2 forms of pain from the list on a person and ask them which one hurt more. Repeating each possible pair 10 times (using different test subjects for each repeated pairing), you will end up with 1000 total tests for each kind of pain. Yes this is going to need a BIG sample pool... but that's okay, because your society is already okay with inflicting lots of pain on lots of people, and the outcome of theses experiments is important enough that someone will be more than happy to fund the project.
Charting your findings
When you are done, you tally up how many times each form of pain "won" and you will have yourself all the heuristic data you need to create an objective pain scale. You can then lay out your pain scale either as a normal distribution or a uniform distribution, depending on how you want to present your findings.
![enter image description here](https://cdn.statically.io/img/i.sstatic.net/vk3fxQo7.png)
If you want to present it as a normal distribution, you score each form of pain 0 to 1000 to measure how many times it won its experiments. This will likely create a bell curve where most forms of pain are somewhere around 500 points, and only extreme outliers will be anywhere near 0 or 1000. You can then, if you so choose, apply a linear transformation to scale the score to easier to work with numbers; like if you want a 0-10 scale, just divide the score by 100.
The other way to chart it would be if you want to measure pain types by what percentile they fall in. So, you still score each pain type, but instead of a uniform transformation, you assign each experiment a score based on how it ranks against other pain experiments. So, if you want to do a 1-10 pain scale, the lowest 10% of scores are a 1, the next 10% are a 2, etc. This will help people using the scale better rank all of those medium kinds of pain which will likely have very similar scores near the middle, but offers less distinction at extreme ends of the scale.
If your goal is pick the best pain method for torture, I'd suggest using a normal distribution, but if you want to use it for medical purposes, like figuring out what order to triage patients in, I'd suggest a uniform distribution chart; so, it's possible that your society will use this data to compile 2 different scales depending on when they are doing torture or medicine.
Estimating New kinds of pain
The obvious issue here is that once your chart is done, it's hard to add new kinds of pain without redoing the whole thing... but there are good methods for estimating new kinds of pain. Let's say you have a new idea for torture that you expect to score around an 8. You AB test it against a known 8 point torture, and the test subjects either rank it overall as higher or lower than the control. Continue doing this until your new test falls between 2 known tests and you will have a good estimate of where on the pain scale your new experiment belongs.