Abstract

Key words: Internet, alcohol abuse, psychotherapy

Introduction

Web-based programs that deliver an automated form of psychotherapy have the potential to reach populations of people with mental illnesses who do not access currently available care. The development and testing of automated or computer-assisted psychotherapy is in a state of early development. Most systems are used exclusively in a research setting, and the few that have been introduced into clinical practice are not yet being used widely. It is not yet clear if this technology will lead to a new era in psychotherapy or will remain simply a curiosity.

Computer-aided psychotherapy is different from other forms of telepsychiatry such as video-conferencing, email consultations, chat rooms, and support groups. Those make communication more convenient, and can help to overcome obstacles of distance, but they require the full time and attention of a therapist. Computer-aided psychotherapy, by contrast, transfers some or all of the actual therapeutic work to the computer. The advantage is that the therapy can be made more widely available by reducing the barriers of cost and limited availability of trained therapists.

In addition to increased access and lower cost, there are other advantages to computer-aided psychotherapy as well. Some patients may prefer working on problems with an automated program compared to a therapist. They can work at their own pace, and they may feel more comfortable dealing with sensitive issues such as substance abuse or sexual activity. It also eliminates the need to schedule appointments, and lessens the stigma of having to visit a mental health professional.

Although the intensity of a computerized therapy cannot be compared to therapy with a live person, it is often difficult to find therapists with specific expertise outside of academic medical centers. There are shortages of therapists who have specialized training in cognitive behavioral therapy, for example, to say nothing of those with specialized training in evidence-based, disease-specific psychotherapies such as for eating disorders, bipolar disorder, and substance use disorders. Computerized therapy can multiply the expertise of a single specialist, and make an attenuated form of these therapies broadly available.

Computerized therapies have been developed for a wide range of illnesses including simple phobias, panic disorder, obsessive compulsive disorder, generalized anxiety disorder, depression, eating problems, and substance abuse (Marks, Cavanagh, & Gega, 2007). Some have been only been tested in small, open, pilot studies (Lieberman, 2003a), while others have been extensively tested in multiple randomized controlled trials (Gega, Norman, & Marks, 2007). One system in particular, FearFighter (Kenwright & Marks, 2004), has been approved by the National Institute for Clinical Excellence (NICE). This English regulatory body publishes clinical appraisals of whether particular treatments should be considered worthwhile by the National Health Service (NHS). These appraisals are based on effectiveness and cost. Treatments, including drugs, that are approved by NICE generally become eligible for coverage under the NHS.

Good Days Ahead, was developed Jesse H. Wright who is one of the leading experts in the field of cognitive behavioral therapy (Wright & Beck, 1983), and is the first multimedia program for computer-assisted therapy of depression (Wright et al., 2002). A randomized controlled trial of this program compared to standard therapy and a wait list group found that computer-assisted cognitive therapy and standard cognitive therapy were superior to the wait list control group for treatment of depression and did not differ from each other (Wright et al., 2005).

Substance abusers represent a population that may derive particularly large benefit from computerized therapy, especially when it is accessible over the Internet (Copeland & Martin, 2004; Lieberman, 2005). The treatment rates of substance use disorders are low despite the high degree of morbidity associated with these illnesses, and the demonstrated efficacy of currently available interventions (Mojtabai, Olfson, & Mechanic, 2002). Reasons for low treatment rates include the shame and stigma of substance abuse, denial of the problem, ambivalence about changing behavior, and preconceptions about available treatment modalities (Cunningham, Sobell, Sobell, Agrawal, & Toneatto, 1993). New and innovative methods are needed to engage these individuals.

Web-based programs can reach large numbers of people in a private and easily accessible manner (Greist et al., 2002; Lenert, Munoz, Perez, & Bansod, 2004; Lieberman, 2003a), however among the disadvantages of this modality is a lack of flexibility. In traditional treatment, as a clinician learns about the important psychological features of a patient, the clinician is able to adapt therapeutic interventions to address the patient’s specific circumstances. A computer program or other self-help modality, on the other hand, typically delivers a standardized intervention of unvarying content.

It may be possible to reduce this problem by utilizing tailoring techniques so that the program better fits individual patient needs. Tailored treatments that use traditional forms of self-help therapy have been shown to increase the efficacy of the treatment of depression, self-defeating behavior, and substance abuse (Beutler, Zetzer, & Yost, 1997). A web-based intervention is a good candidate for a tailored approach because it has the capacity to collect large amounts of data from a series of user interactions, and can change the content of the material in real time. As a user interacts with the web application, and more information is gathered, the personalization of the program can become increasingly more sophisticated and extensive.

This concept of website individualization has received the most attention by companies that produce commercial websites (Riemer & Totz, 2003). Recommender systems, a specific type of information filtering technique, attempt to present the user with items such as music, movies, or books that the user is most likely to purchase (Hanani, Shapira, & Shoval, 2001). The recommender system compares data from the user’s entries and web browsing patterns to data collected from others, then develops a list of recommended items (Adomavicius & Tuzhilin, 2005; Parsons, Ralph, & Gallagher, 2004).

Applying a similar strategy to a web-based program designed for individuals with drinking problems was examined. The ability to predict what types of psychoeducational material each individual user would be most interested in viewing was evaluated based on responses to questionnaires that were obtained during the assessment phase of the program.

Methods

An Internet program was developed (Lieberman, 2003b) that was based on the “drinker’s checkup,” which is a component of motivational interviewing, a form of psychotherapy that facilitates behavior change (Miller & Rolllnick, 1991). The drinker’s check-up is a set of measures that helps a patient to see how alcohol consumption is negatively affecting his or her life, thereby increasing awareness of and concern about these consequences (Miller, Sovereign, & Krege, 1988). The software which presented the questionnaires and determined the patient feedback was written using an open source model. The source code is freely available for use and modification under a Creative Commons license (http://creativecommons.org/licenses/by-nc-sa/3.0/), and can be obtained from the corresponding author.

The application, which was designed to increase internal motivation for change, emphasized a process that was non-judgmental, non-threatening, and objective, rather than value-laden. This approach allowed us to recruit a broad sample of drinkers, not just those who had already reached the stage at which they identified themselves as being alcoholics. Reaching a broad population of drinkers who had some level of concern about their behavior was achieved by making the application easily accessible, completely anonymous, and emphasizing that it was a way to learn about personal behavior related to alcohol without requiring a commitment to long-term treatment or behavior change. Subjects were recruited via text advertisements that were placed on Google search results pages. The advertisement was displayed in response to searches of relevant terms such as “alcohol,” “drinking problem,” and “alcoholics anonymous.”

Inclusion criteria were broad. Study participants were required to have current drinking problems, as measured by an AUDIT score of 8 or more (Babor, de la Fuente, Saunders, & Grant, 1992); have enough experience with the Internet to find the site; and have the ability to successfully navigate the simple user interface. Participants were not offered any kind of incentive or payment for filling out the questionnaires, except the opportunity to receive scores and individualized feedback based on the results of the assessment.

Potential study participants were required to give informed consent by reading an information screen and agreeing to participate in the study. Written documentation of the informed consent process was waived because obtaining written informed consent while preserving anonymity would have created obstacles that would have made the study impossible to carry out. The study was approved by the George Washington University Institutional Review Board.

Based on a user’s responses to specific items in the questionnaires, predictions were made as to which educational modules he or she would view. The items that were evaluated for their ability to predict module choice came from the Stages of Change Readiness and Treatment Eagerness Scale (SOCRATES) (Miller, 1992), the Alcohol Use Disorders Identification Test (AUDIT) (Babor & Grant, 1989), and the Decisional Balance Questionnaire (Carey, Maisto, Carey, & Purnine, 2001).

The SOCRATES evaluates a patient’s readiness to change in the context of the transtheoretical model of health behavior change (Prochaska & Velicer, 1997). Patients rate their agreement or disagreement with statements that reflect, 1) a recognition of having an alcohol problem, 2) concern about whether alcohol is having a negative effect on their lives, and 3) the degree to which they have initiated behavior change.

The AUDIT is a ten-item questionnaire designed to distinguish normal drinkers from those with problem drinking. The items included in the AUDIT were chosen to reflect three dimensions of drinking: alcohol intake (items 1–3), alcohol dependence, such as difficulty in controlling drinking, neglect of alternative interests, and physiological withdrawal (items 4–6), and adverse consequences (items 7–10).

The Alcohol Decisional Balance Scale allows subjects to rate the importance of various consequences of drinking. This scale includes both positive and negative features of alcohol use. Subjects rate each item on a five-point Likert scale ranging from “Not important at all” to “Extremely important.”

Subjects registered for the study anonymously by using a pseudonym and password. They did not provide any identifying information or personal health information as described in the Health Insurance Portability and Accountability Act (HIPAA). Because all data from the internet was collected anonymously, the degree of correlation between alcohol consumption and the AUDIT score was analyzed to evaluate the putative validity of the data. Although there is not an exact relationship between the two variables, a general correlation would be expected. Such a correlation would support the likelihood that the participants entered information with some degree of accuracy rather than at random. Similarly, alcohol consumption reported by women was compared to men. Most studies find higher levels of consumption by men. Replication of this finding would support the validity of the anonymously reported data.

After receiving personalized feedback on the results of the assessment, users were offered educational modules that addressed four areas of potential interest:

Users were able to view all, some, or none of the modules. Using information obtained during the assessment phase, predictions were made regarding which modules each user would select. Affirmative responses to three items from the SOCRATES, one item from the AUDIT, and two items from the Decisional Balance Scale were hypothesized to predict viewing of the Social and Psychological Problems Associated With Alcohol module. One item from the AUDIT was hypothesized to predict viewing of the Health Risks module, two items from the Decisional Balance Scale were hypothesized to predict viewing of the Having Fun Without Alcohol module, and one item from the Decisional Balance Scale was hypothesized to predict viewing of the Managing Craving module.

Results

Between December 2005 and May 2006 283 subjects completed the program. Demographic and clinical characteristics of the sample are described in Table 1. The level of alcohol consumption and pathology was high. The mean AUDIT score was 19, and the number of drinks per week was 38. Nevertheless, only 12% of the sample was unemployed. Slightly more than half of the participants had a first degree relative with a known drinking problem.

Table 1. Demographic and clinical characteristics of participants

About one third of the subjects viewed each of the clinical modules. In general “Managing Craving” and “Health Risks” were viewed more frequently than “Having Fun Without Alcohol” and “Social and Psychological Problems.” The percent of subjects viewing each educational module is shown in Table 2.

Table 2. Number of subjects viewing each educational module from alcoholcheckup.com

Table 3 shows the mean score and standard deviation of items from the SOCRATES, AUDIT, and Decisional Balance Scale of those who did, and those who did not, view modules predicted to be associated with each item. In general, subjects who viewed a specific educational module more strongly endorsed the putatively related items.

Table 3. Comparisons of scores on the SOCRATES, AUDIT, and Decisional Balance Scale between those who did and did not view educational modules on alcoholcheckup.com

Findings that have been observed in other populations were tested in this population in order to help evaluate the validity of the data collected from anonymous study participants. The number of drinks per week correlated with the AUDIT score (r=.34, P<.001). Men reported consuming more drinks per week than did women (39 vs. 25, P<.001), and they reported an earlier age of onset of problem drinking (24 vs. 29, P=.005).

Discussion

One of the greatest obstacles to engaging substance abusers in treatment is overcoming denial, ambivalence, and shame. A web-based form of treatment, which is private and easily accessible, substantially reduces the barriers to initiation compared to traditional forms of care. Once a substance abuser enters treatment, the challenge shifts to retention. A program that can be customized in real time may increase user engagement with the treatment, and this effect can build upon the ease of initiation by increasing the amount of time a user is willing to spend working on the program.

In the current study significant associations were observed between responses to items taken from standardized assessment instruments and the educational modules that were viewed. These associations suggest that an online program designed to reduce problem drinking may have some features in common with recommender systems developed for commercial websites. Rather than seeking to promote purchasing activity, the goal would be to maximize the amount of time a user spent using the program. One of the strongest predictors of outcome in substance abuse treatment is the length of participation (Gossop, Marsden, Stewart, & Rolfe, 1999). In some cases, participation alone can be enough to result in favorable outcomes even among poorly motivated individuals, such as those who were coerced into treatment by the legal system (Miller & Flaherty, 2000). Although it was not tested in this study, accurate predictions about what modules would be of greatest interest to a user would be expected to extend the duration of treatment participation.

The technique used in this study to predict user preference was to estimate the personal relevance of each module based on content criteria from the evaluation portion of the program. The results show that this strategy was only modestly effective. Four items demonstrated statistical significance (P<.05), another three demonstrated a trend toward significance (P<.1), and three did not show a trend. While these results support the potential value of this type of approach, more refined criteria are necessary. The items used to predict module preferences came from standard drinking assessment instruments. These instruments were selected because they have been well-tested for validity and reliability (Carey, Purnine, Maisto, & Carey, 1999). However, their ability to accurately assess drinking behaviors may not necessarily be related to their success in predicting content preference. A more effective strategy might be to develop new instruments that are more specifically targeted at predictive accuracy.

Other predictive approaches besides content criteria are also possible, and might lead to better results. Recommendations based on popularity would recommend modules most frequently viewed by all other users. A more refined strategy would be what is sometimes called “collaborative filtering” (Herlocker, Konstan, Terveen, & Riedl, 2004). Collaborative filtering is the method of making automatic predictions (filtering) about the interests of a user by collecting interest information from many users who have things in common with the current user. The underlying assumption of the collaborative filtering approach is that those who agreed in the past tend to agree again in the future (Hanani, Shapira, & Shoval, 2001).

An example of collaborative filtering can be seen on the movie rental site Netflix. A member who has signed in is presented with a list of movies that are predicted to be of interest. Netflix not only uses rental history to develop this list, but also gives users the opportunity to rate movies which allows for more sophisticated mathematical algorithms. Recently, Netflix has taken this process a step further by informing the user of the specific movie ratings that were used to generate a recommendation. If members find that the recommendations are not accurate, or that their tastes have changed, they can immediately adjust the ratings of the movies that triggered the recommendation.

On average, Netflix members have approximately 200 rated movies in their profiles, and the system matches these to 1.7 billion total ratings (Netflix Consumer Press Kit). According to Netflix, 60 percent of subscribers add these suggested movies to their queues. One advantage of this system is that it reduces the tendency of members to simply request new releases. By encouraging the viewing of older, less expensive movies, Netflix is able to increase its profits (Netflix Investor Press Kit). This recommendation system has significantly changed movie preferences. It has given independent releases a wider distribution. As more Netflix subscribers see and rate these lesser-known films, the recommender system suggests them to more people (Leonhardt, 2006).

Collaborative filtering systems also have significant weaknesses. The primary liability is that very popular items are recommended indiscriminately (Herlocker, Konstan, Terveen, & Riedl, 2004). For example, a collaborative filtering program to help people select books might recommend “Harry Potter” in response to a high rating of any fantasy novel or any children’s book. This occurs because large numbers of people with diverse tastes and interests rate popular items highly. The problem is that every book lover knows about “Harry Potter,” and so the recommendation is useless. As a result, designers of collaborative filtering programs must also focus on introducing novelty and non-obviousness into the resulting recommendations (Li, Lu, & Xuefeng, 2005).

Internet-based behavioral interventions, like other forms of self-help treatment, are low intensity treatments. The goal of the drinking intervention was not so much to bring about full abstinence, but rather to move users closer to the decision to seek definitive care in a traditional face-to-face clinic setting (Lieberman & Massey, 2008). The low intensity is offset by the low cost, the ability to reach large numbers of users, and the minimization of barriers to treatment initiation. Although an automated approach may never reach the intensity of a clinician-delivered treatment, an approach which is similar to the preference prediction strategies utilized in this study may lead to a process of continuous improvement.

Commercial Internet companies, which have invested considerable resources to better understand online behavior, have implemented methods that suggest how this might be accomplished. Online advertising campaigns routinely use algorithms to optimize advertisement performance. An optimizer may run thousands of calculations each time it evaluates an ad campaign. The optimizer collects data on hundreds or even thousands of ads, and analyzes which ones are working, and why. As data is collected, the optimizer discontinues underperforming ads, and then adjusts the appearance, and even modifies the content of the other ads. This approach has been shown to improve response rates, reduce server storage requirements and improve ad efficiency (Karuga, Khraban, Nair, & Rice, 2001).

Unlike traditional therapy, automated therapy can collect data on every interaction that occurs. Measurements can include user responses, which materials are viewed, how long the material is viewed, and the result of the viewing experience. Analysis can identify which parts of an intervention are most effective, and this analysis can potentially lead to incremental improvement of the web-based therapy over time.

The automated nature of the intervention also helps to overcome a problem that frequently affects evidence-based psychotherapies: diminishing fidelity to the therapy manual (Gotham, 2004). Therapist adherence to evidence-based treatment manuals tends to be low, and it continues to diminish over time. The modified therapy that is actually delivered may be as effective, or in some cases even more effective (Henry, Strupp, Butler, Schacht, & Binder, 1993) than the manual protocol, but it is difficult to know for sure because it is no longer the same as the one tested in clinical trials. An automated intervention, by contrast, is delivered in a uniform manner so that a clinician who recommends an automated intervention to a patient knows what will be done with a high degree of precision. This knowledge may allow a behavioral intervention to be “prescribed” in a way that is similar to the prescription of a pharmaceutical agent that has shown clinical efficacy.

The current study was a retrospective one in which a post hoc evaluation of predictive variables was performed. It is difficult to draw firm conclusions based on this design, however the results support the utility of carrying out a randomized, prospective study in order to assess whether a recommender system will increase time spent with a program of this nature. Further development of automated interventions will help increase access to evidence-based behavioral therapies, and reach populations of people with psychopathology that are not currently well-served by the existing healthcare system.

References

Adomavicius, G., & Tuzhilin, A. (2005). Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. Institute of Electrical and Electronics Engineers, Inc.(IEEE) Transactions On Knowledge And Data Engineering, 17, 734-749.

Babor, T. F., de la Fuente, J. R., Saunders, J., & Grant, M. (1992). The alcohol use disorders identification test: Guidelines for use in primary health care. Geneva, Switzerland: World Health Organization.

Babor, T. F., & Grant, M. (1989). From clinical research to secondary prevention: International collaboration of the Alcohol Use Disorders Identification Test (AUDIT). Alcohol Health & Research World, 13, 371-374.

Beutler, L. E., Zetzer, H., & Yost, E. (1997). Tailoring interventions to clients: effects on engagement and retention. NIDA Research Monographs, 165, 85-109.

Carey, K. B., Maisto, S. A., Carey, M. P., & Purnine, D. M. (2001). Measuring readiness-to-change substance misuse among psychiatric outpatients: I. Reliability and validity of self-report measures. Journal of Studies on Alcohol, 62(1), 79-88.

Copeland, J., & Martin, G. (2004). Web-based interventions for substance use disorders: a qualitative review. Journal on Substance Abuse Treatment, 26(2), 109-116.

Cunningham, J. A., Sobell, L. C., Sobell, M. B., Agrawal, S., & Toneatto, T. (1993). Barriers to treatment: why alcohol and drug abusers delay or never seek treatment. Addictive Behaviors, 18, 347-353.

Gega, L., Norman, I. J., & Marks, I. M. (2007). Computer-aided vs. tutor-delivered teaching of exposure therapy for phobia/panic: randomized controlled trial with pre-registration nursing students. International Journal of Nursing Studies, 44, 397-405.

Gossop, M., Marsden, J., Stewart, D., & Rolfe, A. (1999). Treatment retention and 1 year outcomes for residential programmes in England. Drug and Alcohol Dependence, 57(2), 89-98.

Gotham, H. J. (2004). Diffusion of mental health and dsbstance abuse treatments: Development, dissemination, and implementation. Clinical Psychology: Science and Practice, 11(2), 160-176.

Greist, J. H., Marks, I. M., Baer, L., Kobak, K. A., Wenzel, K. W., Hirsch, M. J., et al. (2002). Behavior therapy for obsessive-compulsive disorder guided by a computer or by a clinician compared with relaxation as a control. Journal of Clinical Psychiatry, 63(2), 138-145.

Hanani, U., Shapira, B., & Shoval, P. (2001). Information filtering: Overview of issues, research and systems. User Modeling and User-Adapted Interaction, 11, 203-259.

Henry, W. P., Strupp, H. H., Butler, S. F., Schacht, T. E., & Binder, J. L. (1993). Effects of training in time-limited dynamic psychotherapy: changes in therapist behavior. Journal of Consulting and Clinical Psychology, 61, 434-440.

Herlocker, J. L., Konstan, J. A., Terveen, L. G., & Riedl, J. T. (2004). Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems, 22(1), 5-53.

Karuga, G. G., Khraban, A. M., Nair, S. K., & Rice, D. O. (2001). AdPalette: an algorithm for customizing online advertisements on the fly. Decision Support Systems, 32(2), 85-106.

Kenwright, M., & Marks, I. M. (2004). Computer-aided self-help for phobia/panic via internet at home: a pilot study. British Journal of Psychiatry, 184, 448-449.

Lenert, L., Munoz, R. F., Perez, J. E., & Bansod, A. (2004). Automated e-mail messaging as a tool for improving quit rates in an internet smoking cessation intervention. Journal of the American Medical Informormatics Association, 11, 235-240.

Leonhardt, D. (2006, June 7). What netflix could teach Hollywood. The New York Times.

Li, Y., Lu, L., & Xuefeng, L. (2005). A hybrid collaborative filtering method for multiple-interests and multiple-content recommendation in e-commerce. Expert Systems with Applications, 28(1), 67-77.

Lieberman, D. Z. (2003a). An automated treatment for jet lag delivered through the internet. Psychiatric Services, 54, 394-396.

Lieberman, D. Z. (2003b). Determinants of satisfaction with an automated alcohol evaluation program. CyberPsychology and Behavior, 6, 677-682.

Lieberman, D. Z. (2005). Clinical characteristics of individuals using an online alcohol evaluation program. American Journal on Addictions, 14(2), 155-165.

Lieberman, D. Z., & Massey, S. H. (2008). Pathways to change: the effect of a Web application on treatment interest. American Journal on the Addictions, 17, 265-270.

Marks, I. M., Cavanagh, K., & Gega, L. (2007). Hands-on help: Computer-aided psychotherapy. East Sussex, UK: Psychology Press.

Miller, N. S., & Flaherty, J. A. (2000). Effectiveness of coerced addiction treatment (alternative consequences): a review of the clinical research. Journal of Substance Abuse Treatment, 18(1), 9-16.

Miller, W. R. (1992). Stages of change readiness and treatment eagerness scale (SOCRATES). Albuquerque, NM: Center on Alcoholism, Substance Abuse, and Addictions.

Miller, W. R., & Rolllnick, S. (1991). Motivational interviewing. New York: The Guilford Press.

Miller, W. R., Sovereign, R. G., & Krege, B. (1988). Motivational interviewing with problem drinkers: II. the drinker's check up as a preventive intervention. Behavioral Psychotherapy, 16, 251-268.

Mojtabai, R., Olfson, M., & Mechanic, D. (2002). Perceived need and help-seeking in adults with mood, anxiety, or substance use disorders. Archives of General Psychiatry, 59(1), 77-84.

Netflix Consumer Press Kit. (n.d.). Retrieved October 31, 2008, from http://cdn-0.nflximg.com/us/pdf/Consumer_Press_Kit.pdf

Netflix Investor Press Kit. (n.d.). Retrieved October 31, 2008, from http://files.shareholder.com/downloads/NFLX/...

Parsons, J., Ralph, P., & Gallagher, K. (2004). Using viewing time to infer user preference in recommender systems. Paper presented at the AAAI Workshop in Semantic Web Personalization, San Jose, California.

Prochaska, J. O., & Velicer, W. F. (1997). The transtheoretical model of health behavior change. American Journal of Health Promotion, 12(1), 38-48.

Riemer, K., & Totz, C. (2003). The many faces of personalization - An integrative economic overview of mass customization and personalization. In M. M. Tseng & F. T. Piller (Eds.), The customer centric enterprise: advances in mass customization and personalization (pp. 35-70). Heidelberg, Germany: Springer.

Wright, J. H., & Beck, A. T. (1983). Cognitive therapy of depression: Theory and practice. Hospital & Community Psychiatry, 34, 1119-1127.

Wright, J. H., Wright, A. S., Albano, A. M., Basco, M. R., Goldsmith, L. J., Raffield, T., et al. (2005). Computer-assisted cognitive therapy for depression: maintaining efficacy while reducing therapist time. American Journal of Psychiatry, 162, 1158-1164.

Wright, J. H., Wright, A. S., Salmon, P., Beck, A. T., Kuykendall, J., Goldsmith, L. J., et al. (2002). Development and initial testing of a multimedia program for computer-assisted cognitive therapy. American Journal of Psychotherapy, 56(1), 76-86.

Please address correspondence to:
Daniel Z. Lieberman, M.D.
2150 Pennsylvania Avenue, NW Rm 8-421
Washington. DC 20037
(202) 741-2899
daniel.z.lieberman@gmail.com