" ... Gang prevention programs that may be systematically excluding many of the kids that most need their help and intervention programs that are based on a model that has little or no proven success. Further, the programs may fail to emphasize the most basic services that have been shown to help the men and women in L.A.'s most violent, troubled neighborhoods leave gang life behind."

"Some kids who deserve attention don't get anything," Terry Dunworth, the Urban Institute's researcher for evaluating the city gang program tells Witness LA. "It raises concerns. All of the kids have problems. They all deserve help. But GRYD wants to seek out only those most at risk."

FOLLOW THE GANG MONEY: PART ONE:

Are LA's Gang Prevention Strategies Excluding the Kids Who Most Need Our Help?

by Matthew Fleischer

On a hot day in early May, nearly 200 gang-reduction experts under the umbrella of the city of Los Angeles' Gang Reduction and Youth Development program, or GRYD, gathered in the LA City Council chambers to fight for their jobs. There were too many intervention workers, some of them former gang members with extravagant tattoos and shaved heads, to cram into the rows of seats in the City Council chambers, so they spilled into the hallways instead, greeting each other fondly and chatting nervously about their fates. With the city facing a $212 million budget shortfall, the City Council was looking to do some serious fiscal trimming, and GRYD's $26 million in operating funds were slated for the shears.

As the council meeting came to order and the public comment period began, these men and women stepped to the microphone at the center front of the chambers and told stories of bullets whizzing, children dying and the great risks they took in their daily lives to keep their communities safe. In between their testimonies, a sprinkling of tweedy academic types from the administrative ranks of these same gang-reduction programs came forward to bolster the street workers' pleas with facts and figures.

No money should be slashed from GRYD, each of them said, in one impassioned way or another. Despite its budget woes, this is one program cut Los Angeles cannot afford.

“We're saving lives,” was the common refrain.

Last to speak, and most eloquent, was civil rights attorney and gang intervention expert Connie Rice, whose 2007 Advancement Project report, “A Call to Action,” was part of what triggered the formation of GRYD in the first place. More recently, Rice and her Advancement Project have been tapped to run the city's Los Angeles Violence Intervention Training Academy (LAVITA)—which is attempting to train and professionalize gang intervention workers. “We are celebrating low crime, but in the hot zones, kids still dodge bullets,” said Rice. “These [gang workers] are the people who keep the kids safe. The GRYD office is absolutely essential. We just spent $7 million for a reptile enclosure. I'm happy for Reggie [the alligator], but we need to save our kids first.”

Although some of the city council members fully intended to snip GRYD's funds, Rice made her pitch with the knowledge that the program enjoys the unequivocal backing of LA Mayor Antonio Villaraigosa. Ever since his school reform efforts sputtered and stalled, Villaraigosa has taken to GRYD as his new flagship policy effort. He routinely touts it as “among the most innovative in the U.S.,” and has the habit of making lofty claims about GRYD's impact: “The program has reclaimed our city for our citizens.”

Within days of the City Council hearing, the mayor, Connie Rice and the rest of the GRYD network got their way: GRYD would receive full funding for another year, which in 2009-10 amounted to $26 million, $18.5 million of which came directly from LA's general fund. In the following weeks, virtually every other program in the city would be cut amid LA's budget crunch—the library system, city attorney's office and even the LAPD's counterterrorism task force among them. GRYD was among the few allowed to remain intact.

It was a major political victory for Villaraigosa and Gang Reduction and Youth Development.

The mayor reacted to the news with a celebratory tweet: “Our GRYD programs WORK—gang crime is way down and more kids have a way out of the gang life.”

A two-month investigation by the LA Justice Report, however, has revealed that the mayor and the City Council's confidence in GRYD's central programs isn't grounded in quantifiable facts. In truth, no one knows if, how well or how poorly GRYD is working—not the mayor, not the police, not GRYD itself.

Power and accountability have been consolidated in the mayor's office, but there is still no way of determining whether the program is effective. And there are many indications that methodological errors have been made that have cost—and continue to cost—the city millions of dollars.

A recent audit by LA City Controller Wendy Greuel stated that, after nearly two years, GRYD, much like LA Bridges, still has no adequate evaluation of its effectiveness, or lack thereof—despite the city's spending $525,000 (with another $375,000 soon to be paid out) for an assessment report from the Urban Institute (UI).

“We had years of a feel-good program under LA Bridges,” Greuel says. “Now we've spent more than $500,000 on a tool to see what's working, but we still don't have that yet.

“Transparency is the biggest problem we face.”

But while Greuel placed most of the blame on the irritatingly secretive assessment conducted by the UI, the Justice Report found the real failings to be not with the UI researchers' evaluation of the GRYD programs, but with the programs themselves. Though it took weeks and multiple California Public Records Act requests, we acquired a copy of the UI's 60-page evaluation and found it most revealing. After speaking with the UI head evaluator and two independent evaluation experts, we have learned that UI had a perfectly acceptable methodology in place. GRYD, however, has been hampered by serious bureaucratic blunders, prime among them poorly negotiated contracts that resulted in the loss of a year of data.

But beyond pure evaluation and data-collection screw-ups —of which there have been plenty—the Justice Report discovered gang prevention programs that may be systematically excluding many of the kids that most need their help and intervention programs that are based on a model that has little or no proven success. Further, the programs may fail to emphasize the most basic services that have been shown to help the men and women in LA's most violent, troubled neighborhoods leave gang life behind.

As with many city and county problems, the situation is complex, so bear with us. Policy analysis can be wonky at times. But this is no academic exercise. LA is the gang capital of America, and the stakes of the gang-reduction debate are measured in blood.

So what is Gang Reduction and Youth Development, and how does it work?

GRYD was born in 2008 after an audit by then City Controller Laura Chick, who, along with Rice's Advancement Project report, pointed out that GRYD's predecessor, LA Bridges, was a bureaucratic mess in which money was doled out to politically favored programs in each council district with no central authority, no means of evaluating the various programs' efficacy, little transparency and no accountability about how the millions flowing out of the city's coffers were being spent. To remedy this, Chick and Rice recommended that all the city's gang money be moved under a single roof.

The city took Chick's advice, and the program was consolidated under the exclusive control of the mayor's office. LA Bridges was scrapped and GRYD replaced it.

Officially begun in the summer of 2008, GRYD operates from the premise that not all gang life is created equal in Los Angeles. Some neighborhoods are far more dangerous than others. (In other words, Larchmont and Brentwood don't need lots of gang reduction services.) So GRYD focuses its resources in 12 zones that have been judged to be extraordinarily gang intensive: 77th Division, Baldwin Village/Southwest, Boyle Heights/Hollenbeck, Cypress Park/Northeast, Florence-Graham/77th, Newton, Pacoima/Foothill, Panorama City/Mission, Ramona Gardens/Hollenbeck, Rampart, Southwest, and Watts/Southeast.

So far so good. Maximizing this city's slim resources by focusing on LA's hottest zones is a wise strategy that may be marked as one of GRYD's true accomplishments.

Within those zones, programs are further divided into two categories: prevention and intervention. Accordingly, each of these zones is served by gang prevention and gang intervention agencies.

Gang prevention efforts are just that. They are aimed at 10- to 15-year-olds who are at high risk of joining a gang but who haven't yet taken that plunge. Prevention programs work to steer those kids in a positive direction.

Gang intervention targets teenagers and young adults who are already in gangs, strongly associated with gangs, or in tagging crews, and offers them help in leaving violence behind through services such as job training, mentoring and counseling. Intervention, especially as the city defines it, also means “proactive peacemaking”—reaching out to gangs to broker peace treaties and stem retributive gang violence.

In addition, GRYD also includes the mayor's popular Summer Night Lights program, which keeps neighborhood parks open well past normal hours, and provides activities to keep kids out of trouble. The program also serves as a recruiting tool to help make contact with troubled kids and funnel them into the appropriate prevention or intervention program.

Classically, gang-violence reduction is divided into three categories: prevention, intervention and suppression—law enforcement. Politicians, gang workers and police have come to agree that all three elements are needed to address the complex problem of gang violence. Police can put out fires, but—as Sheriff Lee Baca and LAPD chiefs Bill Bratton and Charlie Beck have repeatedly pointed out—arrests alone won't end gang violence. Without programs that address the sociological side of the issue, a new generation of alienated youth will always spring up to pick up where their parents, siblings, cousins and friends left off.

But despite the importance attached to the notion of gang-reduction work, figuring out what actually makes a measurable change in gang-afflicted communities isn't easy. Like LA Bridges, plenty of well-intentioned but ineffective programs have come and gone throughout the country. But assessing any of these programs' on-the-ground success requires the systematic analysis of an intricate array of variables.

For instance, Mayor Villaraigosa likes to point out that crime is down by 10.7 percent in GRYD zones over the past year. But without rigorous comparisons to nationwide and/or statewide crime trends (both of which have also taken a big dive) and microspecific information about whether more police were deployed to those same areas, it's impossible to say what led to the crime decrease.

Real assessment, the kind that can't be spun to suit political needs, is essential—without it, the mayor's stat is nothing more than a political talking point and millions of dollars are being spent in the dark.

With this problem in mind, the prevention side of GRYD's program has been the focus of a concerted effort to standardize its methods into an effective model that can later be replicated. To do so, all of the GRYD's 12 contracted prevention agencies are required to use the Youth Services Eligibility Tool (YSET), a lengthy questionnaire designed by a team of experts at the University of Southern California. In theory, the YSET gauges which children are most likely to be headed for gang membership based on nine “risk factors.” In other words, instead of judging a kid by his or her tattoos or behavior at school, the test isolates a set of psycho-social components that may lead to future gang membership; home life, past trauma, drug use and general mental health are among those taken into consideration. GRYD gang prevention agencies administer the 60-plus-question test to the kids who agency personnel deem good candidates, and then send the resulting data to USC for scoring and analysis. Kids who show four or more risk factors are considered to be adequately “at risk” and are therefore eligible for GRYD prevention services.

The YSET is designed to be highly selective.

“In the past, the paradigm was the Boys and Girls Club-type model, which was open to anyone and everyone,” explains Karen Hennigan, who heads USC's involvement with GRYD. “But that model hasn't worked in gang reduction. There are plenty of youth with family issues, but it's not one or two issues that pose the greatest risk. It's the piling on of risk factors.

“The YSET creates a sample of youth with higher needs.”

OK, fair enough. But here's where things get tricky.

For one thing, kids with serious problems who agency workers feel sure are headed straight for gang membership but who don't show an overwhelming “piling on” of issues are denied services and made to serve as the “control group.” (We'll get back to the “control group” in a minute.)

In other words, LA children who may very well be in need of help —who experts in the field may even think are in desperate need—are tossed aside in favor of kids who match the academically generated model. For instance, according to an to an April 2009 Daily News story, GRYD had, at that time, tested 364 at-risk kids citywide. Out of that 364, only 100 were deemed acceptable. Today, GRYD's numbers have improved somewhat. However, this seems to be mostly because program workers have begun recruiting from youth populations who are already in trouble with the law—not kids teetering on the edge.

“Some kids who deserve attention don't get anything. It raises concerns, ” says Terry Dunworth, the Urban Institute's researcher for the GRYD evaluation. “All of the kids have problems. They all deserve help. But GRYD wants to seek out only those most at risk.”

“Establishing a cut-off point is always difficult,” says Hennigan in regard to ethics of the YSET methodology. “But it's just as unethical to spread the money to everyone and have a program that's ineffective.”

She has a point. GRYD prevention programs have limited resources , so it undeniably makes sense to target the kids with the highest risk for gang membership.

The problem, say critics, is that the YSET's methodology itself may inadvertently eliminate a large number of the troubled young people it most aims to serve.

“It's not that the general concept of YSET isn't useful,” said one GRYD insider who asked to remain anonymous. “It's that the academics are basically running the program—and that's a big problem.” The academics involved were really smart people, the insider explained. But they didn't have experience in on-the-ground application of their theories.

The theory-versus-practice discrepancy, say some GRYD critics, is where these prevention programs are running aground.

In addition to the problems of eliminating kids in need of the program, no one knows how effective the Youth Services Eligibility Tool is. Despite two years' worth of high-minded academic pronouncements, the Justice Report has found that data collection on the YSET was riddled with screw-ups. Until last month, the contracts that Gang Reduction and Youth Development negotiated with prevention providers did not contain provisions to retest children who initially failed the YSET. So there is no control group—the entire basis for a true evaluation.

A little background on how all this is supposed to work: To set up an effective evaluation, you need to study both a test group that's receiving services and a control group that's not and compare the results. The environmental conditions of both the test subjects and the control group need to be as identical as possible—so for gang-prevention testing, that means establishing a control group of kids from the same neighborhood, with similar problems and backgrounds, as the kids who are receiving services. However, there are ethical dilemmas with this method. Namely, how in the world can you choose which troubled children from poor, dangerous LA neighborhoods get services and which land in the control group and get stuck with nothing?

The YSET was designed to solve this quandary by tracking and retesting kids who initially failed the test and were subsequently denied services. Again, the ethics of this are questionable. If these kids are troubled enough to serve as a control group, why are they being denied full services?

But that's not the end of the YSET mess: Making matters worse, those promised retests never happened. So not only does that mean a year of lost data, it means that hundreds of children who may have been in desperate need of what GRYD purported to offer were screened out to serve as a control group that never materialized.

Is the YSET too restrictive? Does it even work in identifying high-risk kids? Are GRYD prevention services philosophically on target? No one knows.

Dunworth says that when the UI's annual report is released sometime later this month, it will show progress in GRYD's prevention efforts. YSET retests of 150 kids who went through the prevention program showed that “pro-social attitudes” improved—meaning kids in the program started to feel better about themselves. However, their behavior did not improve.

“Bottom line,” says Dunworth, “without the control group, it made the test nothing more than a pilot.”

In other words, no matter what cheery statistics are trotted out in the coming months after the UI's year-end report is made public, there is no conclusive evidence that gang prevention efforts, or the Youth Services Eligibility Tool, are working.

From its inception, the YSET was highly controversial among the gang-prevention experts who actually had to use it on live human beings. In the first six months of the YSET's implementation, GRYD's prevention service providers had a difficult time giving services to anyone—because, they said, the YSET was screening out too many kids. This meant that, while $12 million was being spent on GRYD prevention programs in LA's hottest gang zones, YSET-hampered providers couldn't manage to identify the requisite 200 kids per zone worthy of its services.

“We had to test 100 kids to get 30 enrolled,” says Kathy Houston-Berryman, director of the Community Build GRYD prevention program in the Baldwin Village zone. And remember those 100 kids tested were not children with mild problems. They were all young people who were deemed to be likely candidates for help. “I think everyone was skeptical about the YSET,” says Houston-Berryman. “Time and again, kids we all thought had to be at risk were failing the test.”

Community Build has since gotten its YSET pass rate up to 68 percent. (According to Hennigan, across GRYD the average is around 60 percent, up from 30 percent in the initial six months of the program). They've increased their numbers by targeting kids who are already in trouble in the system—like those in the LAPD's Boot Camp program.

“It's too early to comment on whether this thing is working,” says Houston-Berryman. “I'm sure in the months ahead there will be dialogue—but we have seen some value in the YSET.”

But others (who are not receiving GRYD paychecks) are a great deal more skeptical. The problem, they say, is that GRYD eligibility is entirely dependent on a troubled kid's willingness to self-report shame-inducing and/or criminal behavior—a methodology that, by its structure, is likely to result in the screening out of the very young people whom the YSET is theoretically designed to identify.

“The fact is, middle school kids lie,” says Jorja Leap, a gang expert, researcher at UCLA and the senior policy adviser on gangs and youth violence for Los Angeles County Sheriff Lee Baca. “How many kids do you know that would implicate themselves for marijuana or heroin use?

“How many adults do you know who would sit down for a test so long and incredibly invasive?”

UI's Dunworth says concerns about kids lying have validity: “Self-reported information is always subject to manipulation. There is no way to say self-reporting is 100 percent accurate.”

He adds the caveat, however, that despite its flaws self-reporting is considered a valid methodology.

Dunworth says that as more YSET data come in, the UI plans to go back and interview, in person, a cross section of kids, some who passed and some who failed the YSET—to determine whether they were lying on the tests. Only then can we begin to tell how effective the YSET is, he says. Until that study is completed—and hopefully yields useful data—the cumbersome YSET will continue to determine which of LA's kids get access to GRYD prevention services and which are left to fend for themselves. Whether the kids who do get the services actually benefit from them is also a question that awaits an answer.

Guillermo Cespedes, director of Gang Reduction and Youth Development, says his office acknowledges that contract snafus and data-collection flaws were the primary culprits for the lack of an adequate gang-prevention evaluation. He says the problem has been fixed, and that, as of several weeks ago, retesting of YSET ineligibles is mandated for all prevention providers in the GRYD system.

“I'm not going to deny mistakes were made,” says Cespedes. “But mistakes will happen in a program of this magnitude. LA Bridges was given 11 years of funding. We've only had one year of full funding.” [Editors note: Actually, GRYD has been under the mayor's control for two years.]

As for the YSET itself: “The old method of screening didn't work. We were told to try something new, and that's what we're doing. Something like this has never been done before. We're confident it's working, but we need time.”

In Part Two, due out around the end of August, we tackle an entirely different set of problems with the intervention side of the GRYD equation.