Fourth Annual DNA Grantees' Workshop
Tuesday, June 24, 2003
MORNING SESSION
Question-and-Answer Session
Dr. SOZER: Now I'd like to ask a couple questions of the audience. How many people here are looking at new technologies and doing the research and development?
Okay. We have about half an hour and I'd like to hear from a lot of those people. We've talked now about some of the issues that the end user has. For the people in the audience who are working with the new technologies, what can the end user do that would help the researcher?
DR. BALLANTYNE: If you're developing technology, methods, and technologies, one of the issues from the research side that is quite difficult is applying them to casework and specimens. One of the biggest problems for a researcher is getting access to casework specimens. We can prepare casework-type specimens but it's not quite the same thing.
In some crime labs, of course, you couldn't touch a casework sample. In other labs, however, they're slightly more accessible. But trying to gain access to so-called non-probative casework specimens is an issue for researchers. I think that's a big issue, because at the end of the day you have to prove the worth of your technology and your methods with bona fide casework specimens. I think that's one particular area that could be addressed in forums such as this.
Anyone willing to give some casework specimens? Put your hands up.
One. Seriously, how many crime lab people here could give genuine casework specimens—genuine ones, non-probative ones—to researchers such as myself and others? How many could do that? Good. I know exactly who to speak to.
DR. SOZER: Lucy, would you like to comment on that? And I think Dave [Foran] has a comment also.
MS. DAVIS HOUCK: To point out one other thing you said: Different laboratories have different procedures. For some of us, it is an issue with the legal court system as to whether or not you can release some of that stuff or keep it. We're starting to come under retaining-of-sample issues. Where with the Colt case and the reexamination of cases, need I say it, and Project Innocence coming in, we're scared to death to overuse any of the evidence, even once it's adjudicated, because you never know when they're going to come back and get you on it.
QUESTION: Coming from the community diagnostic side of DNA, it's actually pretty striking, especially with this panel, to see the correlation between the need for clinical diagnostics and forensics. The documentation needs are very similar: quality and quality assurance and validation.
Actually, the words are exactly the same that we hear on the clinical diagnostic side, I think. I'm involved in developing technology, and for the people doing the research in developing technology, I think it would be helpful if individuals in forensics would take a look at what's already available and what has been made available for community diagnostics. There are a lot of software packages and things that already exist that might be adaptable. There are certainly special needs for the forensics community, but I do think that there's a significant amount of overlap that could be leveraged to move things along.
DR. WEEDN: Steve Niezgoda commented that this was a change-averse community and that there are clear reasons for why this has to be a conservative community. But I wanted to make the point that one of the inhibitors of research is the fact that this community largely is not an early adopter. Many of the speakers commented on reasons why we shouldn't be early adopters or why we have problems with early adoption. I thought that I would speak up to give some reasons of why somebody might be an early adopter.
The most obvious reason is the earliest adopters tend to get the grant money for the early adoption. You see some of the more progressive laboratories are in fact better supported and have the tools and the instruments, so it really did inure to their benefit.
I think a very strong argument in my own mind is the fact that early adopters really have to learn the technology. That means that there's a very strong understanding, and I believe that quality is tied to a true understanding of what it is that you're doing. If all that you're doing is buying a new instrument and following the protocol, then there is not the same level of understanding.
Third is the issue of motivating your personnel. I think that that's one problem we have in forensic science. People come in wanting to solve crimes and then they find themselves pushing buttons. I think personnel motivation is an issue for us. This is one of the ways that we address it.
I think that government leadership loves to chortle about the good things being done in the forensic science lab, and if they're early adopters, then they can say that they were out in front doing it. That is often neglected as a real value.
Of course, when we try to adopt something we're trying to address a need. That is to say that the adoption can actually address needs. We could presumably solve cases and save money.
Lastly, I'll just say that if nobody steps out, then the community will not progress.
DR. SOZER: Thank you, Victor. Would someone else like to comment?
MS. HART JOHNS: I guess I do have some perspective on that—and I would maybe ask Lois to talk a little bit too—because I think that we're going to see more and more partnerships being formed between organizations that have different origins. I know our agency has recently formed some partnerships to provide direct input to research. Lois, would you like to talk about that?
DR. TULLY: I could talk for just a minute, yes. That's actually something I'm going to talk a little bit about during my lunchtime presentation, but we have been talking. I've also talked with all of you guys, as well as others in this room, about the importance of collaborations and what different people with different backgrounds and areas of expertise can bring to these collaborative studies.
We've been working with some people in Susan's lab, some of our researchers, and others in other Federal agencies. So far it has been very productive, and it's something that we would like to pursue maybe in a more structured level. Like I said, I'm going to talk about it in my presentation.
I'll be coming to you, the practitioners, as well as the researchers, to help us determine the best structure so we can get the most out of it. At the same time, we don't want to overburden anyone, because everyone's busy, but we want it to be the most productive and most effective. We all recognize that need, and we'll be working to determine the best way to pursue it and implement it.
VOICE: I just want to comment and perhaps follow up to Victor Weedn's comments, particularly the one on motivation factor. I've traveled quite a bit and visited a lot of laboratories, and one thing that I really get a sense of is that there are a lot of practitioners who would really like to interact with companies and manufacturers, but you feel that you're constrained. I would like to encourage you—and I know that I've spoken to a lot of the analysts; that is, the people who are at the bench—that if you do reach out to the manufacturers, there are enough manufacturers who are probably willing to work with you—either through designing experiments or working some of the needs both from researchers and the practitioner—on coordinating some of these things.
I think the benefit is really a win-win for private industry and practitioners and researchers alike. I can't emphasize enough the fact that through the early adopters, we have made a lot of progress in implementation and practical application.
I hope that management really listens to the analysts, because there are a lot of motivated people out there. They just need to be given a chance to pick up the ball and run.
Thank you.
DR. McCORD: I just have another comment; it actually goes the other way. As an academician, one of the things that I find the most helpful is taking my advanced students and getting them opportunities to develop joint research projects or to visit other forensic laboratories.
We've farmed out students to various laboratories or developed partnerships with such agencies as the Federal Bureau of Investigation (FBI), Drug Enforcement Agency, laboratories in Ohio, and so on. You'll see situations where a research project is proposed as so high tech or something that it's absolutely ridiculous to implement.
Developing these lines of communications—that is developing opportunities for internships or a visiting scientist program—is critical to making sure that the right kind of research is getting done.
DR. SOZER: Are there some comments that people would like to talk about regarding building bridges between researchers and end users, in addition to internships? Are there some researchers that are sitting in their labs saying, if only this could happen?
VOICE: Based on what you said about interactions between researchers and some of the other laboratories, I'd like to mention a new program that's starting up at NIST (National Institute of Science and Technology). It's an interactive postdoctoral program that involves postdocs working with NIST and the National Institutes of Health (NIH). Actually, this just recently happened, and applications have to be in by August 1st [2003]. So if you know anybody who would like to do a combined postdoc with NIH and NIST, they can contact me. I actually put out an advertisement about it in the current issue of Science. I'll put copies out on the table.
This is a great opportunity and the salary is pretty good, $55,000. So if you know anybody who's finishing up in the fall and getting their postdoc, this would be a great opportunity to work with both agencies.
DR. SOZER: Based on the response, you may want to check later on this afternoon to make sure that you have enough of those out there.
VOICE: I haven't thought this totally through yet. There's a level of frustration I have in trying to get out what we feel may be useful technology to the forensic community. Some sort of guidelines or structure could be implemented or facilitated by the National Institute of Justice (NIJ) that could help us bridge the gap and get a research product out for use by the forensic community. That would be helpful, I guess, from my standpoint.
DR. PRINZ: I actually wanted to change the topic, but I can answer Eric: If it's not a kit, then it's not going to be adopted. That's my experience with the YM1 system.
Again, I wanted to change the subject. I wanted to ask about publications, because they very often seem to make the difference between developmental and internal validation. I know that we do a lot of validation and never have time to write about it. Plus, some of it is extremely boring and not acceptable in scientific journals.
I don't have a solution, but maybe we should start thinking about it. Maybe a writing workshop could be organized by somebody that would help people write up internal validation results. The results could still be peer reviewed but not be so scientific that it couldn't get published.
I know, for example, that sometimes robotics companies publish their own validations, but then judges frown on that because they don't think it's objective. So maybe the forensic community could start thinking about it. I think we're already moving in the right direction with the collaborations among companies and different laboratories and interlaboratory studies. I know that people have some things validated, but why does everybody have to revalidate it? We could share protocols.
VOICE: Here are a few comments very specific to Eric's, and Dr. Prinz said the same thing: If it's not a kit, then it's not going to be adopted.
Eric and I had a good collaboration going until we became aware of a kit. Without even really having a chance to use his system, I chose the kit. Well, that's not very aggressive. It doesn't help Eric, it doesn't help the community, and it doesn't support NIJ and its initiatives. I really think there is a disconnect that we need to address, and it would be helpful if NIJ could help do it.
How does Eric's work become captured in some type of kit? Some form of technology transfer link is missing. It's a tremendous amount of work. I saw the data. It's brilliant. But I would prefer not to have to ask Eric for opportunities in the courtroom for quality assurance of my own samples or for the kit materials and so on.
It's just so much more risk-free to use a product and then have the backup behind it.
Bruce McCord mentioned it again and again regarding dye blobs from his own purification of primers. He said, "When I purchase them, I have much less of them." Why would I want to buy my own labeled primers if I can't get them as clean as the commercial company? That's a serious issue. And if the money is being spent on developmental systems like Eric's and nobody adopts it, then it doesn't seem like very good use of Federal dollars.
I want to encourage NIJ and the community here to help with the link between development and distribution of these products in some form of kit that prevents laboratories from having to face risk and quality problems that go along with using something where a lot of materials are made in house.
Let me change gears for a bit. Lucy [Houck], I want to go to the very first bullet from your talk. I know you only had three or four [bullets], but I was intrigued by the first one and want to raise a little bit of an issue.
I think it said something like casework productivity versus probative value.
MS. DAVIS HOUCK: Analyst time versus probative value.
VOICE: Yes, that's it: analyst time versus probative value. There's a group here that started DNA testing in crime labs in the late 1980s. We had a very slow system in the beginning. It was impossible to get data out. Things have progressed, now, with hints of automation. But not long ago at a Scientific Working Group on DNA Analysis Methods meeting, guests, who were prosecutors, were talking about how prosecutors should select the samples and that the term "probative value" shouldn't really be used in the crime lab because you don't know what it is. Then, everybody there bristled, "how dare you? Of course we know." And I think we do know. But truly, the prosecutors have a special take on this.
Well, now I have a boss who's a prosecutor, and he has simply commandeered the selection process from me. We're doing a lot more testing than we would have. He challenged my backlog of samples, which, for a sample that's not forced through the lab in some sort of rush, is probably 2 to 4 months. "Not good enough," he said. "I'd like to see 2 to 4 weeks." To which I said, "Well look, sometimes if you don't even start a case for a month, it goes away," and he responded with, "That might be true but it also may be true that you are receiving samples for unfounded cases or that because DNA testing results aren't coming back in a timely way, the investigators have turned their attention to cases they can actually make progress on. In fact, you're the reason why justice isn't being served."
The challenge that I'm facing in my world is getting my turnaround time down to a couple of weeks. That's a tough problem. Until I'm there, I have a backlog that's unacceptable.
So my comment is forget probative value. Test what you can and try not to limit your system. Try to build up your systems so that you can test things as fast as they need to be tested. Let the trial prosecuting team select the samples. If trial strategy wants this or that tested, even though you don't understand why, you should be able to do it by ramping up and not by saying the laboratory can't keep up because of resource limitations. That may sound radical, but in an automated world, I think not.
MS. DAVIS HOUCK: Let me explain a little bit about what I meant. I didn't quite mean probative value. There's a lot of research out there. Take cat hair for example. The cat hair DNA testing is a wonderful technique and it has added to the field of forensic science. But if I'm running a small local laboratory and I look at the number of cases coming in, how many times am I going to have a DNA case where I need to do cat hair?
I think it's wonderful that we have some of this high-end technology, but how many times are we going to have a cat hair in a case that will deserve all of my DNA testing efforts and resources? Those are the types of issues that I'm talking about.
DR. SOZER: We have another comment here, and then we'll go back to the technology transfer link.
DR. HERR: One observation regarding the technology transfer side: I think the forensic community faces a problem very similar to the problems faced by the drug industry. That is to say, the market for forensic products is relatively small. The same can be said about the drug markets for certain diseases.
It may be useful that the forensic community and NIJ work to create what might be called a Forensic Orphan Reagent and Instrumentation Act, which would attempt to encourage more industries to get involved in the development of forensic reagents. That may be one strategy.
On the subject of deployment of new technologies, it seems to me that it might be an appropriate time for NIJ, ASCLD (American Society of Crime Laboratory Directors), and the FBI to maybe combine some resources and create a forensic technology deployment team. Such a team could attend meetings like this, pick up the new technologies, send a team to the labs where they're being developed, and then go around the country and deploy this technology with the appropriate instrumentation.
DR. SOZER: Thank you.
DR. FORMAN: There are routes for commercialization, but unless we have a commercialization branch like NIH, NASA (National Aeronautics and Space Administration), and all of the bigger research-focused institutions, I don't see that we would be in the position to support the entire commercialization process. As Dr. Herr pointed out, a lot of these processes truly are orphan processes. No one is going to get rich doing forensics.
Forensic science does not have the kind of money found in drug companies, car companies, or even rocket ship companies. We're a pretty closed system in terms of how much money can be made. About the biggest money on the horizon is the potential of analyzing all arrestees, which would bring in 12 million samples a year.
So in terms of commercialization at this point, I think that NIJ can help the community develop the best research areas that are important to the forensic community. Even if the forensic community doesn't recognize certain areas as being instantly important, they may see that there's a future benefit down the road.
One other issue, though. We have been trying to create a parallel universe that can act quickly. That parallel universe to us is called the Forensic Resource Network. There are people who are fast on their feet and quick in their brains. In fact, the idea of taking these orphan technologies across the country has actually been one that they've discussed. But we're not there yet. We're just starting all of these things, and I think having this kind of discussion in this kind of environment really adds to the way that these ideas can come to fruition. So I'm really glad that this conversation is being had.
DR. TULLY: I was just going to add one more comment. Eric Buel has been approached by at least one commercial vendor with an interest in his assay. Like Lisa [Forman] was saying, the commercialization process itself is one that NIJ probably can't play as big a role in as you would like to see. But I encourage the researchers to publish, to get their work out there, and to get buy-in by the community. The practitioners can demonstrate that this is a valuable technology that you would want and use and that would lead us in the right direction toward commercialization.
MR. NIEZGODA: Well, if it's not NIJ, who would take this commercialization step? Do you have any thoughts on that?
VOICE: I guess my comments were really intended to be more positive. I know that NIJ's budget is limited, but I've also heard about a $1 billion, 5-year presidential initiative. Somehow that sounds like a lot of money to me. This makes me enthusiastic and optimistic, and I'd like to see if the money really is going to go strictly for DNA—the way the initiative reads. With $1 billion, we can do a lot of good. We can automate things, increase output, and increase capability. What couldn't you buy with $1 billion?
VOICE: One of the strategies that NIH and some of the other defense branches have adopted is the 5-percent SBIR (Small Business Innovation Research) set-aside funding mechanism. I guess the question is, is NIJ operating under the same guidelines as the other agencies, so that 5 percent of the budget is dedicated to the translational research element? That may be one of the mechanisms that can allow kit formation research to go on—if in fact the study sections are willing to focus on product development for the grants that are submitted.
The problem has always been that panels that review the translational research tend to have a lot of academicians on them, and when it comes to actually having a grant that's actually going to put a product on the market, they tend not to fund it. Getting the technology transfer to work effectively has been the problem on the NIH side.
But if there's appropriate review of the translational components, then I think that kind of a set-aside—in conjunction with some support from the Small Business Administration—can be very effective.
DR. SOZER: Victor?
DR. WEEDN: First, is there an answer? Is there an SBIR? My understanding is that all Federal agencies are supposed to have that 5 percent.
DR. FORMAN: We don't take 5 percent out of the $5 million program fund for research and development in DNA forensics. There is a 5-percent set-aside and there's actually a commercialization component to NIJ through the National Law Enforcement and Corrections Technology Center in West Virginia. Several successful commercializations have come out of that particular shop.
So there is money, a 5-percent set-aside, but it's put aside in the technology assistance division of NIJ.
DR. WEEDN: I was going to have two comments, one of which was along the SBIR. In fact, I think SBIR and STTR (Small Business Technology Transfer) are the two mechanisms involved. Once you get to the $1 billion DNA initiative, I think that will become a much bigger issue.
As my second comment, I also wanted to talk about this orphan situation. You're talking about people who have a disease that affects too few people, for example, Canavan's disease. Because not enough people have the disease, there is no market to support the development of the product. It's so small, similar to what Lucy [Houck] said about cat hairs.
Even if you consider forensic science a reasonable market, which is an arguable point, the cat hair is always going to be an orphan for us. Just recently, for example, the Federal Government's Bioshield program guaranteed a market for vaccine manufacturers. (Bioshield is one of President Bush's major bioterrorism initiatives.) You can imagine the problem here. Why develop a melioidosis vaccine when you don't have a melioidosis outbreak? Anthrax is going to be small. You can imagine there are a lot of bioterrorism things and some of the vaccines you're not going to use unless there's an outbreak. Having a market for the product is the fundamental issue, and the government has addressed that by guaranteeing a market for bioterrorism vaccines.
DR. SOZER: We are out of time. I would encourage people over lunch to continue these discussions. I feel like it's such a privilege to work with so many bright people. This room is filled with so many bright and thoughtful people. I thank you all for your participation and I thank the panel members.