The Cochrane Schizophrenia Group

Trust the machine? A keen eye is what you need

on August 9, 2013

By Drew Davey, Research Assistant for the CSzG

I recently encountered Cyndy Green’s blog Thinking Visually and realised that her Jan 6 2013 posting “Battling the modern mindset and its deeply rooted trust of technology” had massive reference to my own work.

English: Magnifying glass with focus on paper....

English: Magnifying glass with focus on paper. Text in the background is from the public domain work Die Baukunst, issue 11, page 8, written by . Rendered with a development version of Cycles in Blender. Deutsch: Lupe mit Fokus auf dem Papier. Der im Hintergrund verwendete Text stammt aus gemeinfreien Werk Die Baukunst, Heft 11, Seite 8, das von Max Hasak geschrieben wurde. Gerendert mit einer Entwickerversion von Cycles in Blender. (Photo credit: Wikipedia)

My name is Drew Davey and I have been working with the Cochrane Schizophrenia Group since 1998, fetching published articles of treatment trials, and assessing their designs. The Cochrane Register of Studies is ostensibly composed of random controlled trials (RCTs) which Archie Cochrane recognized as the gold standard of medical evidence. Since ’98, I have learned to use numerous database programs and am aware that my work would now be impossible without computerized search and storage. The painful need to revisit and read each library’s catalogue every time a new search list appeared on my desk could never work now. The proliferation of journals alone would overwhelm manual searching. No problem, you say, just use the specialized electronic search engine just as my colleague does on a monthly basis, and search for “schiz*” and “rand*”.  If it was that simple, I would be unemployed and chiselling dovetails in my workshop. That wonderful search produces a minimum of 600 new references every month and I am tasked with checking them for inclusion in a shortlist—usually reduced by a factor of 10. I have found that the electronic search is a brilliant tool for the first sweep, but it lacks the precision needed to maintain the legitimacy of a register of RCTs.

By definition, our electronic search has to include all possible trials or we risk losing evidence which would be vital for a review. At this point in the process, I read through titles and abstracts of the monthly 600. If there is a chance that they are about schizophrenia and have a proper randomized design, I flag them and track down the entire article online or in a library. At this point, most make the cut and are included in our register. So far—so obvious, but so time-consuming that I fear for any organization doing evidence-based research that “trusts the machine.”  If they accept the original 600, their evidence will be truly corrupted, and force an impossible task onto a reviewer who is attempting a review or meta-analysis. Cyndy Green’s thesis? Good enough isn’t good enough. Close enough isn’t close enough. It is a pleasure and privilege to be a small part of an organization that endorses precision and rigour.

Check out our post on Technology; making the review writing process more efficient and accurate to see how we’re combining Drew’s skill and precision with a program called Ferret.  Technology is not the be all and end all but it can help, especially when you’ve got someone on the other end to identify and remove the fluff.

Related Articles

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: