Measuring “what matters” in 21st-century policing: partnering with civilian oversight to assess procedurally just policing in Philadelphia

Abstract

Listen

Translate article

ABSTRACT Police oversight bodies routinely audit body-worn camera (BWC) footage for binary policy compliance (e.g., Was the camera activated per policy?), and police performance assessments traditionally focus on quantifiable outputs. In both cases, the ‘process domain’ that reflects service quality is largely ignored, resulting in impoverished metrics. In partnership with the Philadelphia Police Department and the Citizens Police Oversight Commission, we conducted a systematic social observation (SSO) of BWC footage from 233 randomly selected traffic stops to evaluate a protocol for measuring procedurally just policing (PJP). Rater consistency and validity assessments showed that coders provided reliable ratings, capturing elements of participation, dignity/respect, and trustworthy motives, with lower correlations for neutrality. Mean PJP scores varied with suspect tone, supporting predictive validity and aligning with procedural justice theory. The PJP protocol enables oversight entities to assess service quality and alignment with democratic policing values, though refinements are needed to address the range of police–citizen interactions. We discuss our findings and their implications for automation.