SURVEILLE Scoring Methodology

[Appendix to Guest Post by Martin Scheinin, EU-Funded Study: Electronic Mass Surveillance Fails – Drastically]

While the ethicists use color coding (green = low ethical risk, amber = intermediate ethical risk, red = severe ethical risk), the technology assessment team produced a numerical score for ‘usability’. The scores range from 0 to 10 where the highest scores identify the surveillance methods that are assessed as most usable, on the basis of a simple point system related to ten different factors. Three of the ten factors relate to the capacity of the surveillance method to achieve a security benefit (delivery, simplicity and sensitivity). Another three points could be obtained through factors related to financial cost (purchase cost, staff cost, external personnel). Further three points were allocated on the basis of factors that relate to the capacity of the technology in question to accommodate privacy concerns (subject-specificity, collateral intrusion, privacy-by-design), and a tenth discretionary point is awarded by the expert team for ‘excellence’.

The third set of assessments was conducted by a team of constitutional and international lawyers who assessed the intrusiveness of each surveillance technology or method upon privacy and other human rights. Building upon Robert Alexy’s Weight Formula, they assessed two separate factors using a logarithmic scale (1, 2, 4), multiplied the scores with each other, and then applied a third multiplicator (1/2, ¾, 1) to adjust the outcome according to the reliability of our assessment. Basically, the last-mentioned factor was related to the question whether clear case law by the European Court of Human Rights or the European Court of Justice was available. The two main factors assessed were the importance of privacy (or another human right) in the given context, and the depth of intrusion into that right, as resulting from surveillance. As the scores were multiplied with each other, the outcome was anywhere between 0 and 16.

A more detailed explanation of the scoring system can be found in SURVEILLE’S Matrix of Scoring Technologies.