Fourth Annual DNA Grantees' Workshop

Wednesday, June 25, 2003

AFTERNOON SESSION

Technology Solutions in the Crime Lab
Roger Kahn
Biography

MS. BASHINSKI: Our next speaker is a man who needs no introduction, but I'm going to introduce him anyway. Roger Kahn, as I'm sure most of you know, was a pioneer in this field in Miami-Dade County, Florida, and then he jumped ship not too long ago to go to the DNA program in Ohio. Now he has been kicked upstairs and is an administrator. Will the real Roger Kahn please stand up?

DR. KAHN: Thank you, Jan [Bashinski]. Kahn: Slide 1

It's nice to present near the end because you get a chance to be provocative without being intentionally substantive, so I'll try to do that for you. I thought about this in a variety of different terms and tried to think about what the folks who spoke before me would say, so that I could talk about something that hasn't been said. So I took a different track on this.

Instead of looking at bottlenecks, which have been addressed to some extent, and trying to make worksheets that are smart or that use robotics to fill themselves in, I just tried to stand back to look at improving productivity in a broad sense. Kahn: Slide 2

So I'm just going to mention a few things, as the others have. I've promised to only speak for a few minutes.

First off, as you've heard and we've done some of this and not just in DNA, I'll talk about automation to improve throughput and about a LIMS that we have just recently implemented that actually helps us. I think that's novel in some labs because not all LIMS systems have improved labs, but we think ours did and we're excited about it enough to keep working on it. So I can talk about that for a moment.

I want to talk about something I learned from Susan Johns: Making sure that the laboratory is equipped with enough low-cost equipment that people don't get stuck waiting for the things that they have to share. I was struck by your comment, Lisa, about a $16,000 microscope so they wouldn't have to reset everything. We just won't permit that in our lab. Our operation is too expensive to be stuck on a microscope. So I'll say one or two things about that.

I also want to say something about trying to reduce the burden of quality control (QC). This is a touchy audience to talk about this with. If anything, we've brought quality control to forensic science and some of us in this room do a great deal of checking on that. We audit the labs and make sure it's all there. But I think there are opportunities to reduce the QC burden, because I've seen some folks go way over the top on this, thus creating their own bottlenecks.

Lastly—and this may be a lifetime project—we're trying to find out how you can use the casework approach to work smarter, not by top-down management but just by finding what good ideas are out there and asking people how they could be more productive instead of insisting that they be.

Let me just briefly throw this one away, because it's been mentioned. Just like Jeff Ban, we have a Biomek robot. Frankly, we're not as far along with it as Jeff is, but we expect to do just what he's doing. I can tell you right now that we've discussed with analysts the idea of batching the standards on the robot, and every one of them is against it. Kahn: Slide 3

Now, you don't work in my lab so it doesn't affect you, but if you stand back a little bit, you know that it's a good idea to do that. It separates samples in space and time and gets them done faster, even with different hands handling them. And yet, it's not case ownership. Everybody's a little queasy about that. But we're going to try to move in that direction: To batch these things and let the analysts work where their hands are needed most—not on the standards.

We're working on a system for real-time PCR (polymerase chain reaction) for human DNA quantitation. It's similar to the one that Eric Buel described and that others have talked about. Unfortunately, I'm not allowed to talk about it just yet. I'll just tell you that we're doing it. We're doing it because it's easy. You just make PCR reactions, put the tubes in, get the numbers that come out at the end, and throw away the tubes. Because the numbers are already there, they can be connected back to LIMS and hopefully to the Biomek to make the dilutions and so on.

Now, when we bought our LIMS, we did what you all have done if you've shopped for one: We asked people to show us about them. We looked at screen after screen, and it was hard to figure out which one to pick. They looked great, and we knew that they were surely an improvement over what we had. Kahn: Slide 4

Off the shelf, these system mainly track evidence. As a DNA examiner that's not your biggest problem, but as a lab director it's one of mine. If I can't keep track of the evidence, all sorts of things can go wrong to things that I may have to one day provide answers. So I was dazzled by that.

When we actually had an opportunity to more carefully look and see what the system did, we were fortunate enough to be handed a project director who said, "Well, what do you want this to do?" We said that we wanted it to keep track of evidence, write reports, monitor analyst productivity, and make worksheets.

What we found out was that it kept track of evidence. If we wanted it to do those other things, we had to create that for the company, essentially doing that with them and paying extra for it. There's not a system out there, at least as far as I know, that will do that off the shelf, and if it does, it will do it my way. It will write reports that look like Ohio's reports.

So we went back to the drawing board and spent many, many months designing reports. Eventually we asked the company to do the programming for us.

Giving the analyst the ability to produce a report in a single pass is a big advantage of this. Believe it or not, we were doing this with handwritten drafts. A secretary was typing them and sending them back to us, where analysts would review and correct them. That iterative process would occur over and over again.

This also helps to standardize the language. People don't want to say things in precisely the same way. We appreciate that, but we're trying hard to standardize the language.

We've heard complaints that people don't really know how many LIMS stations you need to keep things moving and people from waiting in line for computers. So, we bought enough stations to reduce the chances of backlogs. It's no fun if all your tubes are barcoded and you can't get to the barcode reader.

This is what the report writer looks like. What you have here are a series of wizards; for example, this is the general DNA wizard, DNA table (14 loci) non-semen DNA report, and no DNA profile obtained report. Kahn: Slide 5

As you can imagine, when you push the button associated with these things you're going to get some sort of a report that's just about done. You can imagine, of course, that the recipient's name is going to be there, the formatting will all be there, and the list of items will all be there. There is no transposing or creating this de novo.

But more importantly, the complex DNA reports even include the loci listed and the results under each one. And yes, I know it's controversial if we tell them the actual findings other than that it matched, but we still do that because we believe that these reports should be decipherable by people who might not ask for anything else.

So if we asked for that and you pushed the button under "DNA table 14 loci," this is what you would see. You would have the description of the item come up automatically, and you would type in the allele numbers on this screen. Now, you would do this for each sample, and the table comes up automatically in Microsoft Word. It's actually kind of fun to watch. It's not instantaneous. It looks like it's busy and then this beautiful table comes up that you never actually typed. Language associated with it that explains the matches and so on comes up automatically, too. Kahn: Slide 6

It isn't completely trouble free, but I will say this: This has eliminated the need for the secretaries. We had six people in three labs in our organization who are very, very concerned about their jobs because they used to type these reports on a full-time basis. But I can say this: Our evidence vaults are much more organized, the evidence is being returned to the agencies more regularly, and little projects are getting done. I think their jobs are truly safe, and we're operating more efficiently.

Even more interesting is that it doesn't take the analyst any longer to fill out this table than it used to to make the draft. The analysts are spending just as much time as they always did but the secretaries are no longer needed. So we're actually getting our reports for free.

We created the entire setup on our own. We told the programmers what we wanted the system to do. It's de novo programming essentially. The system we have didn't do this until we asked for it.

It does provide low-cost equipment contributions. The best example I could come up with is from the chemistry section and stems from a conversation that I had with Susan some years ago. I had massive problems in the chemistry section. We were nearly a year behind. As you can imagine, if multiagency drug task forces go out and confiscate drugs and they don't get the analysis reports back for a year, then they don't prosecute the cases and they get an attitude. So we knew we had to do something. We thought that if we bought everybody a balance and a dissecting scope to look at marijuana and an analytical scope to do crystals—yes, we still do crystals—that they would keep going and take better care of the stuff. It's an old story. It's what Miami-Dade did to justify distributing cars to its officers. If you give them the car, they'll take it home and wash it. But if they have a pool car, they'll leave their Wendy's wrappers in the back seat. So if it's your balance or your scope or your pipette pan, it's going to be used more efficiently, and people will take better care of it. Kahn: Slide 7

Wherever we've been able to find a place for automation, we have used it; for example, autosamplers on GC/MS (gas chromatograph/mass spectrometer) applications. I want to say one thing as a caveat though, and this is a confession of a recently promoted lab director. It occurred to me when I first got there that technology would fix everything, and we've been fortunate. We have a senator who likes us. We've been able to get earmarks. We've also had a very supportive State administration, so we've bought a lot of stuff.

Instead of improving productivity, however, it reduced it to some extent. We're far more powerful now, but it takes more to do a case. For example, we bought an AFIS (automated fingerprint identification system) that automatically identifies fingerprints. But to put the fingerprints into AFIS, you have to actually mark them up, edit them, and so on so that you can be submitted for searching. That takes a whole lot longer than not doing it. Then we bought the MOREHITS system in the latents section. MOREHITS eliminates background on surfaces, so if you have a plastic bag, for example, the repetitive texture goes away and it takes the wavy lines out of a check. It just takes time to use it.

We also bought a microspectrophotometer so we can distinguish one fiber from another. When they were the same fiber content but they seemed to be red in two instances, but in fact the dyes used to create the color weren't the same. Without the microspectrophotometer, we couldn't distinguish between them and we might have called them the same or similar. Now we can tell fibers apart, but it still takes time to use the microspec.

The same can be said of the Graphical Image Database Management system. I can go through thing after thing where adding new technology created more work. It made us more potent and we do better work, but it just takes longer. So in terms of improving productivity, it takes more than just technology. It takes specific technology to speed things up. I'll come back to that in a moment.

Let me talk about reducing extraordinary quality control. This came from a conversation that Cecelia Crouse had with Mark Nelson, who is sometimes my boss when I audit with NFSTC (National Forensic Science Technology Center). She asked why some are labs taking daily temperature checks on things like refrigerators. If it's not in the refrigerator, then it's still going to be okay. It's probably good for several weeks. I mean, if you leave the milk out for an hour, you can still drink it, right? Kahn: Slide 8

It may not be necessary to be so diligent, but its up to you whether you want to perform temperature checks only on certain types of refrigerators or to define certain things as not being important to maintaining temperature. We're going to hold you to whatever you say when we audit, but if you define these things broadly, then there's no basis for us to say you really have to keep the temperature of that refrigerator between 3 and 4°C. I think that lots of people go way overboard when it comes to monitoring things like temperature.

Another thing that you can do is reduce the number of critical reagents. Now, maybe you'll boo and hiss at this one. There is a lab I know of that thinks every single reagent in the DNA test is critical; they pretest everything. I think that's kind of cool, actually. But we have so few failures. The standards require that you perform QC tests only on the kits and the kit components. But if you choose to do other things and you have to hold your processing up so that you can make sure they're all tested, maybe you're doing too much.

When I was in graduate school, we had this tank of water that sat above the sink that was like God's water. Everything you could do to water, we did to that water because that was critical stuff. But once something was wrong with that water, we didn't know what to do. So we started to look at processes and more processes, and the only thing that we could think of was to do more.

I had a conversation with a professor in the lab next door and he said he just used what came out of the tap. I asked if that was enough, and he said that he was just trying to demystify things. Actually, we started doing that. I don't know what was wrong with that carboy water, but, whatever it was, we couldn't fix it, never understood it, and frankly didn't care.

It may be that you spend so much time worrying if you're meeting the standard that you're actually stepping on your own toes and tiring your group out by burdening yourself with unnecessary QC. Just meet the standard. If your lab needs more, that's fine. But there's a lot of leeway in these standards. They don't specify a lot of things that you must test.

We also encourage the analysts to study their own productivity. What we did was rather simple. It actually started in the chemistry section, but we have done it with DNA and I'll tell you about that in a minute. We made a copy of one proficiency test from each chemist and gave the entire set to each chemist for review. We asked them to critique the set, looking at the number of tests done, the amount of work that was done, legibility, and the clarity of the conclusions. Kahn: Slide 9

They did a far better and much more credible job than we would have ever done through peer review, because they were commenting on each other's work—they didn't have to actually face each other. It wasn't a secret. The names were on everything. They concluded that some analysts were doing way too much documentation, some were too sloppy, and so on. They were able to communicate with each other.

Actually, the guy who was way too thorough was working harder than anybody else and putting out fewer cases than anyone else. The only person who had been able to reach him was his own supervisor, but he couldn't calm him down. We'd tried everything but still couldn't get him to calm down, because he had 20 years of success doing it his way. When his own analyst peers told him that they also had 20 years and didn't do it that way, he finally listened. Even the fellow who was a little slovenly cleaned up his act when the entire group let him know that his work wasn't up to snuff.

Also, you could administratively limit marginal testing, like residues in crack pipes and so on.

We did a tremendous amount of outsourcing as part of our effort to get rid of a chemistry backlog that I mentioned before. One of the interesting things is that we learned some things from the private laboratory that we outsourced to. We currently do Fourier Transform Infrared Spectroscopy (FTIR), which means you have to work with one sample at a time. If you want to identify crack cocaine and distinguish it from cocaine hydrochloride, using FTIR is one way to do it. Kahn: Slide 10

The company took an ordinary hotplate, set it at 100°C, drew a grid on the top with a felt-tip marker, and put a little bit of each sample in each square. If it melted at 100°C, then it was crack. Gee, that's much cheaper than FTIR, and you don't have to calibrate it too often.

So out of the woods came a brilliant idea. We've checked with SWGDRUG (Scientific Working Group on the Analysis of Seized Drugs) on this, and it's legitimate. I think some of the best ideas are here in the room. We don't often get a chance to share our little tricks. This one will save us hundreds and hundreds of hours a year.

Now, what do I want? I want to automate the screening process. I think this is Cecelia Crouse's idea: Having sperm search be a computerized function, where it searches field by field and finds them for you. We've actually had to limit the number of hours people spend doing this on slides that don't have any sperm because they won't give up. Kahn: Slide 11

Then we had the old argument whether one is enough or you have to find two. You know, I love that argument. It's hopeless. You know already whether it's one or two before you came in today, and if you think it's two you won't be able to convince the person who thinks it's one, and so on. So what if the machine did it? I would just love that.

We would like to robotically analyze every sample possible, not so we can eliminate 90 percent of our examiners, but so we can test 90 percent more samples. I really would like to get away from being the one who says we can't do that, unless it's truly redundant and dumb. I would like to test what we're sent, and when it becomes apparent that it's dumb later in court, that's the best trainer I know of. It's the long way around, I know, but it's very hard to win these arguments when you've got to get everyone in the lab to just agree to it.

I really want DNA interpretation to go away from the craft stage and become an automated function. We're seeing glimmers of hope now. Dr. [Mark] Perlin, from his talk earlier, seems to be making strides in that regard. I don't know who else is doing this, but I'm very, very anxious to have this important function become a computer function.

Really, if I had my way, I would like questioned documents, firearm comparisons, and latent print comparisons to go the same way. It's much more easy to become statistical when you're not just coming to the conclusion that they match through some sort of Tom Terrific process, where you stare at it long enough and convince yourself of a match. All of this is possible, but just not today.

So, for the discussion portion of this session, I ask what are your dreams. Let's try to communicate them to NIJ, and I just have a sneaking suspicion they'll find some way to fund them and get them back to us. Kahn: Slide 12

That's all. Thanks.

MS. BASHINSKI: Thank you very much, Roger. Can I ask you a question? I have to ask a question about your automated report system. Did you try or consider directly importing the allele calls into the report from the software you're using to make the calls? Is that possible?

DR. KAHN: We know it's doable. We haven't tried it because we've had the LIMS system only since the end of last year [2002], but yes, that's the plan. Also, we want to automate a lot of intermediate analytical data. For example, there must be people here whose labs automatically write the weight of the drug into the case notes so you don't have to. Basically, our plan is to eliminate as many points as possible where you would ordinarily pick up a pen.