One of the problems of data analysis is being able to trust the results in the face of questionable data collection and coverage practices. For example, WaPo has a longish article on the problems of understanding the range and character of deaths of people by police force:
The Post started tracking fatal shootings by on-duty police officers after a Ferguson police officer killed Michael Brown, an unarmed black teenager, during an altercation after a convenience store reported a robbery in August 2014. That shooting set off demonstrations and sparked calls for reform.
Amid the turmoil, nobody could answer a simple question: How often do police shoot and kill someone? No one knew for sure, because no government agency kept a comprehensive count.
When The Post began tracking these shootings, it became clear that police were shooting and killing people about twice as often as numbers reported by the FBI, which collected voluntary reports from police departments. The Post’s database, which is regularly updated, relies on a collection of news media accounts, social media posts and police reports. …
The Post’s database relies significantly on reporting from local media outlets on shootings in their own communities. The amount of reporting done on individual shootings has declined, probably a victim of the continued cuts by local media outlets.
But fatal shootings by police have not slowed — even though the pandemic closed businesses, shuttered schools and effectively shut down much of American life for weeks on end. In May 2019, police shot and killed 74 people. In May of this year, police shot and killed 109 people.
WaPo maintains its own database because no one seems to have a reliable and official database available.
And:
Fatal shootings by police are a limited metric for answering larger questions about how police use their powers, experts said. Whether a shooting is fatal may depend entirely on a few centimeters in the trajectory of a bullet.
No nationwide data exists on how often police shoot and wound someone, or how often they fire and miss. And no comprehensive national data exists on how other kinds of force — such as chokeholds or the use of batons or stun guns — are used.
“The fatalities is a very good measure of some things, but doesn’t include the kinds of events and activities that we’re seeing all over the country that normally don’t lead to death,” said Alpert, the criminology professor. “Unless there’s an injury or unless there’s a complaint that gets traction, either we don’t know, or it doesn’t matter.”
So when we want to analyze the situation, it almost seems hopeless – the data is not trustable, not granular enough, and how do we measure the racism inherent in redlining and profiling and, say, the casual racism of this woman:
Amy Cooper said she would be calling the police instead.
“I’m going to tell them there’s an African American man threatening my life,” the white woman told him, pulling out her cellphone and dialing 911.
Less than 24 hours later after a video of their exchange went online, she has lost her dog, her anonymity, and her job — the latest incident in a long, too-familiar pattern of white people calling the police on black people for any number of everyday activities: Barbecuing. Playing golf. Swimming at a pool.
My point? Be careful when citing figures concerning police shootings – not only might they not be trustable, they’re probably not even applicable to the situation.
