Tap 20 — TAPs <<<Documents<<<Home

Internal Program Evaluation Techniques In an Adolescent Substance Abuse Treatment Program in Rural Illinois

Pamela P. Irwin, M.A.
Executive Director
Central East Alcoholism and Drug Council
Charleston, Illinois

Abstract The purpose of this internal evaluation study is to examine various characteristics of the Model Comprehensive Treatment Program for Critical Populations—Rural Youth of the Central East Alcoholism and Drug Council, in Charleston, Illinois. An eclectic evaluation design was utilized that examined program implementation efforts across three components of: (1) program implementation efforts; (2) client demographics; and (3) treatment issues. A program description of goals and attainments that could be utilized in service replication efforts is given. Methods of analysis included review of agency documents, analysis of demographic characteristics and symptom indicators of adolescent clients in the program, and quantitative analysis of Personal Experience Inventory (Winters et al. 1988) testing results. The testing data were gathered on a group sample of 30 adolescent clients who successfully completed the treatment program compared through discriminant analysis with a sample of 30 clients matched by age and gender who had voluntarily terminated services against staff advice. The results indicate a statistically significant relationship between a complex set of client characteristics and program completion outcomes. The program's successful attainment of goals beyond initially projected expectations is documented.

A growing trend in the substance abuse treatment field is the great concern for effectiveness, efficiency, and accountability of agencies that provide services. The increasing costs to society related to addiction have also been of concern. A recent study by Merrill et al. (1994) revealed over 70 conditions requiring hospitalization that are attributable in whole or in part to substance abuse, including tobacco. According to this study, males under 15 years of age with substance abuse as a primary or secondary diagnosis stay four times longer than those with no such diagnosis (16.4 days compared with 3.9 days). Females in the same age group stay almost three times longer (9.8 days compared with 3.6 days). The cost benefits of providing treatment services have also been documented in the literature. Gerstein et al. (1994) determined that for a cost of $200 million for treating 150,000 individuals in California, benefits received during treatment and in the first year afterwards were worth approximately $1.5 billion in savings to taxpaying citizens. According to the study, each day of treatment paid for itself (the benefits to taxpaying citizens equaled or exceeded the costs) on the day it was received, primarily through an avoidance of crime.

Treatment systems are currently burdened with inadequate fiscal resources to meet the existing demands for treatment. Of concern is a trend noted by Gfroerer (1994) that suggests that there are increasing rates of illicit drug use among youth and that their perceived risk of use of illicit substances is decreasing. Society's failure to concentrate significant efforts in understanding the treatment needs for the future for our young people will only bring disastrous results.

Purpose of the Evaluation

The literature lacks intensive analyses of the characteristics, clinical issues, and substance abuse patterns of rural youth in need of substance abuse treatment. Furthermore, minimal research exists that examines those characteristics of adolescent clients that might have predictive relevance to the course of treatment. According to Rog (1995), intervention programs are often developed in a political and social environment in which the human urgency for development of services precedes the explicit theoretical understanding of the problem or the intervention such that an appropriate goal for evaluation is to develop better understanding of the phenomena itself. Such has been the case in development of rural adolescent substance abuse services. The current study is an initial step in delineating the phenomena of adolescent substance abuse treatment.

The purpose of this study was to investigate the provision of substance abuse treatment services to youth in a rural outpatient setting through use of an internal evaluation design. The evaluation examined three elements of the Model Comprehensive Treatment Program for Critical Populations—Rural Youth of the Central East Alcoholism and Drug (CEAD) Council. The evaluation investigated (1) program attainment of goals; (2) demographic characteristics of the population that was served; and (3) client characteristics, including use patterns, concomitant issues, and tendencies predictive of early termination. According to Love (1991), the advantages of use of internal evaluation include the evaluator's firsthand knowledge of the organization's philosophy and policies and procedures. The internal evaluator can also communicate evaluation information to staff in a timely manner and participate in long-range planning.

Program Background and Description

CEAD Council is a community-based, not-for-profit corporation offering a full range of services to address chemical abuse and dependency in a four-county rural area of central East Illinois. Although the agency has provided traditional core programming for adolescents (as well as adults) since the early 1970s, the Adolescent Program has been significantly enhanced with multiple innovative program components through having been awarded a 3-year Federal grant from the Center for Substance Abuse Treatment.

Philosophy

The service philosophy of the agency as expressed in the original grant application included a conceptualization of programmatic development grounded in historical experience in the provision of treatment services to the target population and ongoing awareness of therapeutic techniques and theories as delineated in the research literature. Primary program innovations were precipitated by three guiding research foundations:

• Theories that explicate adolescent developmental factors related to decision making (Kohlberg 1964)
• Research drawing from ecological systems approaches (Hartford and Grant 1987; McLaughlin et al. 1985)
• Research examining "host" factors that increase risk of substance abuse (Pandina and Schule 1983; Hawkins et al. 1985).

Program enhancement goals and objectives were related to the above research and the agency's clinical experience in working with adolescents in the rural population.

Utilizing research concerned with adolescent development, the agency created age- and gender-specific group formats and contents.

Using research concerned with ecological systems, the agency created the Youth Leadership Center for enhancement of social support systems during treatment and in continuing care maintenance.

Using research regarding "host" factors, the agency developed enhanced family services, specialized linkage networking for outreach and ancillary services, enhanced assessment, and more holistic treatment services.

The Federal grant request for proposals had sought innovative strategies for the development of model treatment services for various segments of targeted populations. The CEAD Council had chosen to develop Outpatient Treatment services for adolescents and their families in the context of a rural community. To complement the traditional core adolescent treatment programming with the grant-provided program enhancements, the agency proposed the following five specific aims.

• Increase the clinical staff to allow greater accessibility of clinical staff to community referral sources for case finding, case coordination, and long-term care planning.
• Increase client accessibility to a broad spectrum of health, mental health, social, educational, vocational, primary medical care, HIV/AIDS services, and acute care through acquisition of qualified, specialized staff and consultants, as well as enhanced networking approaches to improve long-term client self-sufficiency and address overall client health.
• Make transportation to and from the treatment facility available to clients and families whose low income and rural location have previously complicated full engagement in treatment components, thereby negatively influencing program retention and recidivism as well as client relapse.
• Develop an innovative Youth Leadership Center for adolescent specific socialization, recovery maintenance, and/or recreational resources that will foster a therapeutic social milieu of recovering teens in order to devalue and eliminate drug use, reduce incidence of client conflict with the criminal justice system, and promote overall client health and self-esteem as well as decrease the negative social stigma associated with drug treatment services.
• Increase availability of family services, enhancing existing education and consciousness raising with state of the art family therapy services both separate from and conjointly with the adolescent client.

Evaluation Methods

An eclectic approach to the design of the evaluation was adopted to best obtain information that could be utilized by current program stakeholders for purposes of ongoing program improvement and for future potential replication projects. The evaluation questions included:

• Programmatic issues
  Was the program implemented as planned?
  Could the program be replicated?
• Demographics
  What were the demographics of the population that was served?
• Treatment issues
  What patterns of addiction existed in the adolescent clients that were served?
  Were there patterns of concomitant diagnoses present in the clinical population?
  Was there a significant difference between those clients who successfully completed the program and those who voluntarily terminated services prior to program completion?

Methods for data gathering included the following: (1) a review of agency documents; (2) staff interviews and observations of program activities; and (3) quantitative data gathering and analysis that included a comparison of Personal Experience Inventory (Winters et al. 1988) testing results gathered on a matched group sample of 30 adolescents who successfully completed the treatment program compared through discriminant analysis with 30 adolescents who voluntarily terminated against staff advice.

Evaluation Results and Discussion

Programmatic Issues

WAS THE PROGRAM IMPLEMENTED AS PLANNED AND COULD IT BE REPLICATED?

Throughout the 3-year implementation period of the program enhancements, CEAD Council monitored accomplishment of program objectives through quarterly reports as well as various other evaluation measures. The reports included service statistics as well as fiscal accountability measures. A summation of the quarterly reports reflects the accomplishments of the program.

The original goals and objectives were categorized as a multi-focused strategy toward remediation in issues that had previously negatively influenced adolescent treatment access, therapeutic processes, and client retention. Each of the goals was correspondingly defined by various objectives, enhancement implementation efforts, programmatic aims, methodologies, and activities as the development of the project continued. Process evaluation measures included documentation of attainment of service statistics for each of the areas of remediation. A summary of the data from reporting periods of January 1, 1991, through March 31, 1993, is given below as an example of the program's attainment of objectives.

Access and Assessment

Case finding. The enhanced multidisciplinary staffing pattern of the grant award provided for a sufficient number and quality of staff to respond to needs for rapid assessments, assessments within schools and community settings, and extensive contacts with community and agency personnel to enhance early identification, intervention, referral, and education of significant community "gatekeepers." During the above stated time period, 700 community contacts were made by project staff. During that same time, screening and assessments were accomplished with 338 adolescent clients, of which 238 completed full admission processes and engaged in treatment services.

Transportation. The program had been targeted toward adolescent clients and their families in a rural four county area that covered approximately 2,100 square miles without the availability of public transportation. Program staff had reported that lack of viable transportation was frequently instrumental in creating client's early termination from treatment. Consistent utilization of the CEAD Council transportation services occurred during the reporting time period with a total of 1,729 uses of transportation to or from program services.

Physical Assessments. During the reporting period, 123 clients were seen by the CEAD Council Medical Director. Clients not seeing the CEAD Council Medical Director had regular physicians within the community or recent physicals from local care providers, and their issues pertinent to substance abuse treatment were reviewed by the Medical Director.

Psychological Assessments. A total of 428 cases were reviewed with a consulting psychologist. The consulting psychologist participated in weekly review sessions with the clinical staff of the program to offer input to specific case treatment planning, additional clinical supervision, and occasional inservice training.

Treatment Process

Staff additions. The staffing complement of the program was increased by the equivalent of six FTE positions. The grant allowed a more multifaceted staff to meet the multiple treatment needs of adolescents, including biological/physical, psychological, informational, vocational, educational, social, cultural, and adjunct issues needing attention within the primary addiction services. Having the additional staff capabilities also influenced enhanced age-specific programming and extensively increased family involvement in the program. Prior to the enhanced programming, less than 10 percent of the clients had significant family involvement in any area of their treatment programming. Family issues had been frequently identified as a problem area in clinical supervision meetings. Through the enhanced staffing, weekly joint family therapy groups as well as weekly gender-specific therapy sessions were added to the program. The core program, which had included one therapy group per week and an individual counseling session, was expanded to a daily availability of multidimensional programming from which counselors could individually tailor treatment planning on an ongoing basis as clients and families progressed through recovery.

The original grant application did not call for an increase in numbers of clients to be served by the program, but was rather a mechanism to increase the level of intensity of services offered to the same number of people as had been the program's normal experience. However, as the program developed and matured through the grant cycle, increased clients and family members were readily subsumed into the program without adding measurably to the program costs. As can be seen in table 1, a comparison of projected to actual services reveals an increase in services provided over the original expectations.

In summary, a review of the process evaluation measures revealed that the program had met objectives as planned. Additionally, the program exceeded projected figures for numbers of clients to be served without any additional increases in program staffing patterns or funding. The attainment of the goals and objectives of the program with the addition of exceeding annual projections of clinical services is an indication that similar programs could be implemented in other locations. The program staff complied with and exceeded activities specified in their goals. With appropriate fiscal resources for sufficiently qualified staff and the additional equipment/facility resources such as were made available through Federal funding, it appears that similar programmatic structures could be replicated and implemented in other areas.

Demographics

WHAT WERE THE DEMOGRAPHICS OF THE POPULATION THAT WAS SERVED?

A sample of 97 records was reviewed and data were gathered regarding demographic information taken from the client application forms. The application forms included multiple demographic questions that revealed characteristics of the population that was served.

The average age of clients was 17.4 years with ages in the sample group as follows: 10-15 years, 14.4 percent; 16 years, 19.6 percent; 17 years, 15.5 percent; 18 years, 19.6 percent; 19 years, 16.5 percent; and 20-22 years, 14.4 percent. The primary racial group was Caucasians, with one American Indian, three Hispanics, and two clients from other racial groups represented in the sample. Clients were 77.3 percent males and 22.7 percent females. Most of the clients (85.5 percent) lived with their family. Low income was a pervasive issue in the sample, with about two-thirds of the families (66.3 percent) having an annual income of less than $7,401; 17.9 percent had incomes between $7,401 and $19,644.

While about a third of the families did not report a source of income, 45.3 percent indicated having some income through wages or salary and 22.1 percent indicated receiving some form of public assistance or other source of income. In reporting their employment status, 63.9 percent of the clients were currently enrolled students, while 14.4 percent reported being employed on a full or part time basis. Almost 10 percent of the sample had not yet even begun high school and were already in need of substance abuse treatment. About 2 percent of the sample had completed 2 years of college, but the largest representation was the 88.6 percent that had completed somewhere between the 9th and 12th grades.

A large proportion of these adolescents had already found themselves in trouble with the legal system: 67.7 percent had previous arrests of from 1 to 5 times and 5.4 percent had been arrested between 6 and 10 times; 26.9 percent had never been arrested. Even with the multiple indicators of problems existing in the lives of these adolescents, 61.5 percent reported that they had never previously received treatment, while 34.4 percent had been in treatment one or two times prior to their current admission and 4.1 percent had been in treatment for from three to five prior admissions. Multiple referral sources had facilitated the adolescents' entry to treatment with the primary source of referral from the courts at 47.9 percent. Other referral resources included hospitals/physicians, Alcoholics Anonymous (AA) and Narcotics Anonymous (NA), State Corrections, family, self, schools, Protections and Advocacy, the Department of Children and Family Services, and TASC.

The data discussed in this section were gathered in the first admission interview with the clients. We frequently found that later clinical interviews reported far greater instances of substance use and abuse than had been revealed in either the admission interview or in the standardized testing. It appeared that as the clients became better acquainted with treatment and the clinical staff, they became more open about more extensive use patterns than had been revealed at admission. Keeping in mind this potential minimization of reporting, a full 50.8 percent of the clients who answered the question on first use of substance—63 of 97 clients—reported that they initially had begun use of substances at or below the age of 13. Another 41.2 percent began their substance use at or below the age of 16. Put in the context of adolescent development, by the time these adolescents were just able to begin driving, 92 percent had begun use of substances.

Substances that were reported as being the "primary substance of choice" included: alcohol, 72.9 percent; marijuana, 19.8 percent; amphetamines, 5.2 percent; PCP, 1 percent; and inhalants, 1 percent. The reported frequency of use for the primary substance of abuse was categorized as follows: No use during the preceding month, 29.2 percent; less than or once per week, 40.6 percent; and several times a week through more than three times daily, 30.2 percent. Additionally, 66 of the clients also reported a secondary substance of abuse as follows: marijuana, 53 percent; alcohol, 33.3 percent; amphetamines, 10.6 percent; and inhalants, 3 percent. Frequency of use of the secondary substance of choice was as follows: no use in the preceding month, 37.9 percent; less than or once per week, 25.8 percent; several times a week through more than three times daily, 33.3 percent; and frequency unknown, 3 percent.

The demographic information of these clients is indicative that rural America has not escaped the problems of substance abuse among adolescents. The treatment program is located in a primarily rural area within a central Illinois four-county catchment area of approximately 2,100 square miles and a population base of about 107,000 people. The two largest towns in the area have less than 20,000 people, and the closest metropolitan area of 50,000 or above population is over 50 miles away from the major population centers of the program.

Treatment Issues

Examination of the treatment issues was handled by data analysis of psychological testing using the Personal Experience Inventory (PEI) (Winters et al. 1988) that is completed on all clients entering the program. First, a sample of 30 records of clients who had voluntarily terminated services against staff advice prior to program completion was chosen. A matched sample by age and gender of 30 client records was then chosen to be the comparison group who had successfully completed program services. Length of stay in program services for the entire sample varied from 8 to 363 days of enrollment with an average length of stay of 128 days. Initial expected length of stay in program services is determined at the beginning of treatment based on clinical needs and was documented in the treatment plans of the client records. Determination of early termination and or successful program completion is assessed by the counselors at the time of discharge, approved by the clinical supervisor, and documented in the discharge summary. Due to the low number of early terminations, records were chosen throughout the 3-year duration of the grant program enhancements.

WHAT PATTERNS OF ADDICTION EXISTED IN THE ADOLESCENT CLIENTS WHO WERE SERVED?

The substance abuse scales of the PEI classify reported usage patterns of the following substances: alcohol, marijuana, LSD, PCP, cocaine, amphetamine, Quaalude (methaqualone), tranquilizers, heroin, opiates, and inhalants. The usage patterns examined for this study include those reported as having been used in the past year and those in the past 3 months.

We noted that frequently the clients' scores on this subset of items indicated lower reported patterns of usage than that which was reported in the initial clinical assessment process or in later progress notes.

The usage patterns shown in table 2 reveal significant substance abuse among the 60 clients and widespread abuse of multiple substances, which is not the common perception of "rural America." Unfortunately, it appears that the societal misconception of rural areas being less affected by substance abuse than the urban counterparts is quite inaccurate according to the testing reports of these adolescents. Of equal concern is that the average age of this sample of clients was 16; ages ranged from 13 to 19. The average age of the first use of substances was 13; ages of first use ranged from 5 to 16. In this samples, 16.7 percent reported their age of initial use to be at or below the age of 11.

WHERE THERE PATTERNS OF CONCOMITANT DIAGNOSES PRESENT IN THE CLINICAL POPULATION?

The individual client profiles on the PEI denote six scales that encompass various psychological concomitant issues. The results of examination of these scales across the full sample of clients are shown in table 3.

Although a full 20 percent of the subjects attested to scaled items on many subtests, by far the most predominant concomitant psychological issue was chemical dependency in the family. Such information is consistent with the literature in the substance abuse field that has traditionally considered genetic elements to enhance risk factors among the children of alcoholics. With the current shortage of treatment availability for both adults and adolescents, the occurrence of the high incidence of reported familial addiction is of greater concern for the future. If treatment availability continues to lag painfully behind the demand for services, it is predictable that the problems evidenced by these adolescents will only be repeated in the years to come in the next generations. Additionally, the data revealed disturbing rates of concomitant problems among these adolescents that further enhance their needs for treatment. The significance of family involvement in the treatment process is underscored by these data. The likelihood of significant changes in the adolescents' lives is certainly more at risk if dysfunctional families are not also brought into the treatment process for the needed opportunities for recovery of the whole family constellation.

IS THERE A SIGNIFICANT DIFFERENCE BETWEEN THOSE CLIENTS WHO SUCCESSFULLY COMPLETED THE PROGRAM AND THOSE WHO VOLUNTARILY TERMINATED SERVICES PRIOR TO PROGRAM COMPLETION?

The final question for this evaluation was concerned with examination of the PEI testing results to determine if significant differences were present among those clients who successfully completed the treatment program and those who voluntarily terminated against staff advice. Client retention problems have frequently been noted in the treatment field literature and have presented issues of concern for practitioners. The clinical staff of the Adolescent Program had wrestled with this issue in their staff meetings regarding program quality assurance even though their client retention rates across the 3-year time span were frequently higher than those quoted in the general substance abuse field literature (which quotes early termination rates of anywhere from 20 to 50 percent as not uncommon for adults and higher expectancies for adolescent programs). If the program evaluation was able to determine client characteristics that differentiated potential ASA clients from program completers, the utilization of evaluative results could be enhanced by the existing staff interest in this question.

A sample of 30 client records of male and female adolescent clients who had voluntarily terminated program services was matched by age and gender with a same size sample of clients who had successfully completed program services according to their individualized treatment plan. The PEI is delineated into various categorical areas (Validity Scales, Basic Scales, Clinical Scales, Substance Abuse Frequency Scales, Personal Risk Factors, and Environmental Risk Factors) that—other than the validity scales—are each then further defined by various subscale groupings. Additionally, each individual client profile of testing results classifies potential concomitant clinical issues such as sexual abuse, familial chemical dependency, and suicide risk, among other issues that were discussed in the section on concomitant disorders.

Data were obtained from each of the client records using the T-score given in the individual client profile. Each client profile gives T-scores as compared with a normative group of high school students or a normative group based on chemical dependency drug clinic adolescents. The T-scores based on the high school students normative group were chosen for this analysis as perhaps being more sensitive to the outpatient sample of clients seen by the program, many of whom may have been in early stages of addictive patterns and still been eligible for program services. T-scores were recorded across each of the subscales for a total variable pool of 61 clinical subscales, 6 validity scales, and a variable for whether or not the client was a currently enrolled student. Discriminant analysis was utilized to obtain the differentiating characteristics of the two groups. The discriminant analysis obtained a canonical correlation value of R=.92, d.f. =30, p = .0000.

The data revealed significant differences between the two groups to obtain predictive group membership of 100 percent. Characteristics of each of the two groups as delineated in the discriminant analysis are portrayed in table 4. The characteristics have been ranked in order of the magnitude of the standardized canonical discriminant function coefficient. Although all scales that contributed to the equation are listed, those scales that contributed minimally (standardized coefficient of < +1.00) have been noted with an asterisk.

Gleaning meaningful information for use by practitioners from the above data is complex due to the multiple characteristics determined as significant in the discriminant analysis equation. Attempts to reduce the characteristics to the top five, and/or to categorize by substance use patterns only or clinical scale indicators only, created reduced levels of prediction available in the equation and much greater probabilities of incorrectly assigning group membership. The indicators listed in table 4 are evidence of the complexity faced by practitioners in their attempts to complete accurate clinical assessments, match clients with the appropriate treatment levels of care, and to be able to predict potential tendencies toward early termination.

*Minimal contribution to the equation (see text).
All characteristics p<.005.

The characteristics of the ASA group (the early terminators) suggest an increased likelihood of early termination in adolescents who have used multiple substances in the past year, including opiates, cocaine, inhalants, and barbiturates, but are more likely to have used only amphetamines and alcohol in the past 3 months prior to treatment. It is likely that these clients have extensive personal involvement with drugs, suggested by using at inappropriate times such as early in the morning or at school; they probably use for psychological benefit and restructure activities to accommodate use. Their response to testing may indicate questionable validities in results and high indices of defensiveness. Adolescents in this group would tend to associate with chemical abusing peers; however, they probably perceive high levels of social discomfort, incompetence, and feelings of mistrust toward others. They would tend to be socially isolated and feel socially inept. Adolescents in this group would also be more likely to have participated in unlawful, delinquent, or oppositional behavior.

Adolescents in the Program Completers group also present a fairly complex picture of clinical need. In differentiating between the two groups, the clinical similarities of the entire population should be kept in mind, in that both groups are by definition adolescents who had been involved in substance abuse outpatient treatment. It is not as if the comparison is between adolescents in treatment and those who present without dysfunctional symptomatology or comparison between significantly differing levels of pathology along the continuum of chemical dependency progression. The group differentiations are derived characteristics from a supposedly homogenous subpopulation of adolescents.

Suggested characteristics of the Program Completers group include similar multisubstance use patterns in the past year of such substances as amphetamines, PCP, and LSD, but much more extensive use of substances in the 3 months prior to treatment, including opiates, cocaine, inhalants, and marijuana. This group is characterized by more extensive severity symptoms of chemical dependency, including transituational drug use, loss of control, and clinical preoccupation with drug use. Adolescents in this group are more likely to have high scores in the indicators for psychiatric referral and to have been victims of sexual abuse. They may also show symptoms of response biases in infrequency and defensiveness. Adolescents in this group may be likely to reject convention, fail to endorse traditional beliefs about right and wrong, and tend toward antisocial and unconventional moral beliefs. These adolescents tend toward absence of spirituality, spiritual beliefs, and the use of prayer in their life. This group is more likely to be comprised of currently enrolled students.

Another interesting comparison that can be derived from the total psychological characteristics of each of the two groups is that the ASA characteristics tend toward external interpersonal elements while the PC characteristics are more of an internal intrapersonal nature. Additionally, the differences in substance use characteristics between the two groups show increased recent use of multiple substances in the PC group, as well as characteristics of more extensive chemical dependency patterns such as loss of control and clinical preoccupation with drugs.

Perhaps what the Adolescent Program has begun to discover is empirical evidence of that which substance abuse practitioners have intuitively known for years as "readiness" or "hitting bottom." Could it be that the internal emotional struggle as the disease of chemical dependency progresses—as opposed to external problems that signify initial stages of abusive patterns—is composed of such significantly dichotomous processes that ASA/PC group memberships can be predictable? Can corresponding interventions be created to intervene in tendencies toward ASA prior to early program termination and further progression of the illness process? Is it the lesser amount of internal struggle and crises in the ASA group that has reduced their motivation to complete program services? Will further progression of the illness within these individuals reflect corresponding similarities to the PC group?

The internal evaluation of this program reflects extensive levels of program goal attainment, a delineation of program enhancements and client demographics, and an explication of characteristics of significant group predictability between early terminations and program completers. Multiple avenues for further research, including larger samples and questions of relevance to practitioners, the scientific community, and State and Federal policymakers, are suggested by this study.

Recommendations

The various data gathering mechanisms utilized in this study revealed high levels of program accomplishment. The program met or exceeded its original program expansion goals. Staff were cooperative and interested in the evaluation study as well as hopeful about continuing program improvements. A major barrier to ongoing provision of successful services occurred with the expiration of the Federal funding at the end of the 3-year demonstration period. Replacement funding that had originally been expected from State mechanisms also was not available. Although no ready source of funding to continue the program at full capacity with the multiple enhancements was available, the Board of Directors of the agency supported continuation of a reduced structure program that retained many of the critical enhancements such as the Leadership Center and transportation. The agency continued to seek replacement funding, as community need for service was not reduced.

The complex patterns and issues suggested by this analysis call into question the trend toward simplistic screening devices as opposed to thorough assessment. Further avenues of research are suggested by this study that could assist in bridging the gap between practitioners and the scientific community. For too long, practitioners in the substance abuse field have been left without viable evaluative research to assist in their understanding of potential program improvements or to give them opportunities for replication of models that show promise. It is hoped that this study is a step in the right direction for a future that elucidates greater understanding of the complex nature of substance abuse and recovery for adolescents.

References

Gerstein, D.R.; Johnson, R.A.; Harwood, H.J.; Fountain, D.; Suter, N.; and Malloy, K. Evaluating Recovery Services: The California Drug and Alcohol Treatment Assessment (CALDATA). Sacramento, CA: California Department of Alcohol and Drug Programs, Resource Center, 1994.

Gfroerer, J. Advance Report No. 10: Preliminary Estimates from the 1994 National Household Survey on Drug Abuse. Washington, D.C.: Substance Abuse and Mental Health Services Administration, 1995.

Hartford, T.C., and Grant, B.F. Psychosocial factors in adolescent drinking contexts. Journal of Studies on Alcohol 48:551-557, 1987.

Hawkins, J.D.; Lishner, D.; and Catalano, R.F. Childhood predictors and the prevention of adolescent substance abuse. NIDA Research Monograph No. 56. Etiology of Drug Abuse: Implications for Prevention. Rockville, MD: National Institute on Drug Abuse, 1985. pp. 75-126.

Kohlberg, L. Development of moral character. Review of Child Development Research, Vol. I. New York: Russell Sage Foundation, 1964.

Love, A.J. Internal Evaluation: Building Organizations From Within. Newbury Park: Sage Publishers, 1991.

McLaughlin, R.J.; Baer, P.E.; Burnside, M.A.; and Pokorny, A.D. Psychosocial correlates of alcohol use at two levels during adolescence. Journal of Studies on Alcohol 46:212-218, 1985.

Merrill, J.; Fox, K.; and Chang, H. The Cost of Substance Abuse to America's Health Care System. Report 1: Medicaid Hospital Costs. New York: The Center on Addiction and Substance Abuse at Columbia University, 1994.

Pandina, R.J., and Schule, J.A. Psychosocial correlates of alcohol and drug use of adolescent students and adolescent clients in treatment. Journal of Studies on Alcohol 44:950-973, 1983.

Rog, D.J. Reasoning in evaluation: Challenges for the practitioner. New Directions for Evaluation 68:93-100, 1995.

Tatsouka, M.M. Discriminant Analysis: The Study of Group Differences. Champaign, IL: Institute for Personality and Ability Testing, 1970.

Winters, K.C.; Henly, G.A.; Rauch, J.; and Huba, G.J. Personal Experience Inventory. Los Angeles: Western Psychological Services, 1988.

Last Updated 11-7-02