A PRELIMINARY STUDY OF FACULTY AND GRADUATES PERCEPTIONS OF VOCATIONAL EVALUATION COMPETENCIES IN THE EDUCATIONAL CURRICULUM Except where reference is made to the work of others, the work described in this thesis is my own or was done in collaboration with my advisory committee. This thesis does not include any proprietary or classified information. ____________________________________ Melanie Thompson McAlister CERTIFICATION OF APPROVAL: Randall McDaniel Vivian M. Larkin, Chair Professor Assistant Professor Rehabilitation and Special Education Rehabilitation and Special Education Everett D. Martin Stephen L. McFarland Professor Dean Rehabilitation and Special Education Graduate School A PRELIMINARY STUDY OF FACULTY AND GRADUATES PERCEPTIONS OF VOCATIONAL EVALUATION COMPETENCIES IN THE EDUCATIONAL CURRICULUM Melanie McAlister A Thesis Submitted to the Graduate Faculty of Auburn University in Partial Fulfillment of the Requirements for the Degree of Master of Science Auburn, Alabama May 11, 2006 iii A PRELIMINARY STUDY OF FACULTY AND GRADUATES PERCEPTIONS OF VOCATIONAL EVALUATION COMPETENCIES IN THE EDUCATIONAL CURRICULUM Melanie Thompson McAlister Permission is granted to Auburn University to make copies of this thesis at its discretion, upon the request of individuals or institutions at their expense. The author reserves all publication rights. ________________________________ Signature of Author ________________________________ Date of Graduation iv THESIS ABSTRACT A PRELIMINARY STUDY OF FACULTY AND GRADUATES PERCEPTIONS OF VOCATIONAL EVALUATION COMPETENCIES IN THE EDUCATIONAL CURRICULUM Melanie Thompson McAlister Master of Science, May 11, 2006 (B.A. University of Georgia 1993) (B.S. Medical College of Georgia 1999) 97 Typed Pages Directed by Dr. Vivian M. Larkin This study investigated faculty and graduates perception of vocational evaluation competencies in the curriculum. The study surveyed 55 faculty and practitioners in regards to 51 competencies found credible by the Newman and Waechter (1997) study entitled Commissioned Assessment of Competencies Prepared for the Commission on Certification of Work Adjustment and Vocational Evaluation Specialists. The top ranked competencies were reported for faculty and vocational evaluation practitioners. Differences in means were compared between the following groups: vocational evaluation faculty, vocational evaluation graduates, rehabilitation counseling graduates with a specialty in vocational evaluation, rehabilitation counseling graduates, and other unspecified graduates. Twenty-four top ranked competencies listed by faculty as most emphasized in their curriculums were compared to rankings of all graduates. Vocational v evaluation graduates and rehabilitation counseling graduates with a specialty in vocational evaluation had the most similar ratings with faculty. Rehabilitation counseling graduates with no specialty differed most from faculty members in terms of ratings. These findings suggest that a difference exists between curricula in terms of obtaining vocational evaluation competencies at the university level, and between the two professions of rehabilitation counseling and vocational evaluation. Significant differences were found between the faculty members and other unspecified graduates for the competencies of job analysis and vocational interviewing skills. In addition, the competency of awareness/inclusion of cultural diversity received a lower rating by vocational evaluation graduates as compared to the ratings of rehabilitation counseling graduates and faculty. vi VITA The author is the daughter of Dan Thompson and Sylvia Thompson (deceased) of St. Petersburg, Florida. She graduated from Dunwoody High School in 1989, and received a Bachelors of Arts in French from the University of Georgia in 1993. She received a Bachelors of Science in Occupational Therapy from the Medical College of Georgia in 1999. She began the Auburn University graduate program in Rehabilitation Counseling with a specialty in Vocational Evaluation in August of 2002. She currently works with a pediatric population in a community setting providing occupational therapy services. She is married to Robert B. McAlister and has one daughter, Zoe Elizabeth McAlister. vii ACKNOWLEDGEMENTS The author would like to recognize the faculty and vocational evaluation practitioners who responded to the survey. This study would have been impossible without their input and time. She would like to recognize the staff at VECAP as well as CCWAVES who were very helpful in providing information and encouraging the study. She would like to give most thanks to Dr. Vivian Larkin who gave direction to the study. She would also like to thank Dr. Randall McDaniel and Dr. E. Davis Martin who provided valuable input. She would also like to thank Dr. David Shannon and Asim Ali who gave statistical and technical support. The author would like to thank her husband who provided encouragement as well as support throughout the completion of the study. In addition, their beautiful daughter, now in her toddler years, provided endless laughter and joy throughout this tedious but worthwhile process. viii Style manual or journal used: Publication Manual of the American Psychological Association (5 th ed.) Washington, DC: Author Computer software used: Microsoft Excel XP, Microsoft Word XP, Microsoft FrontPage, Statistical Package for the Social Sciences (SPSS) Program ix TABLE OF CONTENTS LIST OF TABLES..................................................................................................... xi I. INTRODUCTION ......................................................................................... 1 Statement of the Research Problem ............................................................... 6 Purpose of the Study ...................................................................................... 6 Research Questions........................................................................................ 6 Significance of the Study............................................................................... 8 Limitations and Assumptions ........................................................................ 8 Definition of Terms........................................................................................ 9 Summary........................................................................................................ 11 II. REVIEW OF LITERATURE ........................................................................ 12 History............................................................................................................ 12 The Profession ............................................................................................... 18 Standards and Certification............................................................................ 21 Facing Criticism............................................................................................. 24 Competency Studies....................................................................................... 27 University Programs Offering Courses.......................................................... 35 Summary........................................................................................................ 40 III. A PRELIMINARY STUDY OF FACULTY AND GRADUATES? PERCEPTIONS OF VOCATIONAL EVALUATION COMPETENCIES IN REGARDS TO EMPHASIS IN THE CURRICULUM........................... 42 History and Becoming a Profession............................................................... 42 Facing Criticism............................................................................................. 44 Competency Studies....................................................................................... 45 Curriculum and Competencies....................................................................... 47 Statement of the Research Problem ............................................................... 51 Purpose of Study............................................................................................ 52 Research Questions........................................................................................ 52 Methodology.................................................................................................. 53 Participants......................................................................................... 53 x Procedures.......................................................................................... 58 Survey Development.............................................................. 58 Survey Distribution................................................................ 58 Response Rate........................................................................ 59 Survey Questions ........................................................................................... 60 Analysis of Data............................................................................................. 61 IV. CONCLUSION AND DISCUSSION............................................................ 74 V. IMPLICATIONS AND RECOMMENDATIONS???............................ 79 Recommendations.......................................................................................... 80 REFERENCES .......................................................................................................... 82 APPENDICES ........................................................................................................... 87 Appendix A: Information Letter for Participation in Study ......................... 88 Appendix B: Vocational Evaluator Competencies Survey........................... 90 xi LIST OF TABLES Table 1 Demographic Characteristics of Faculty and Graduates by Frequency and Percent............................................................................. 55 Table 2 Top 5 Ranked Competencies and Means/Standard Deviations of Vocational Evaluation Faculty.......................................... 63 Table 3 Top 5 Ranked Competencies and Means/Standard Deviations of Vocational Evaluation Graduates...................................... 64 Table 4 Top 5 Ranked Competencies and Means/Standard Deviations of Rehabilitation Graduates with Vocational Evaluation Specialty.............................................................. 66 Table 5 Top 5 Ranked Competencies and Means/Standard Deviations of Rehabilitation Counseling Graduates ? No Specialty............................................................................................. 67 Table 6 Top and Lowest 5 Ranked Competencies and Means/ Standard Deviations of Other Graduates ................................................. 68 Table 7 Differences between Means of Faculty and Graduates ? Top 10 Ranked Competencies ................................................................. 70 Table 8 Analysis of Variance for Differences in Means Between Groups (One Way ANOVA) ................................................................... 72 Table 9 Follow Up- Multiple Comparison Analysis Between Faculty................. 73 and Graduate Means 1 I. INTRODUCTION Vocational evaluation is an essential part of the rehabilitation process; yet competencies in the curriculum have not been well defined. Although the field of vocational evaluation has existed since the late 1940s, competencies continue to evolve. Shumate, Hamilton, and Fried (2004) believe that exploration of competencies represents a crucial step for vocational evaluation as it expands the field, develops standards, and updates the certification exam. Pruitt (1977) identifies vocational evaluation as a unique field which uses work as the main focal point in the assessment process. The rehabilitation process involves providing services such as medical restoration, counseling, training, and placement in order to assist a person with a disability to find employment (Rubin & Roessler, 2001). One of the first steps in the rehabilitation process may often be a vocational evaluation. The evaluation is intended to assist a rehabilitation counselor in answering questions regarding where to begin and how to proceed. Despite vocational evaluation?s importance within the vocational rehabilitation process, minimal research has examined the efficacy of curriculum to prepare vocational evaluation practitioners (Taylor & Pell, 1993). In addition, there appear to be no studies of practitioners? perceptions in regards to vocational evaluation competencies and university curricula. 2 Shumate et al. (2004) believe that the field of vocational evaluation is in a process of evolution, making the exploration of competencies a critical process. The current exploration of vocational evaluation competencies is important for the expansion of the field, updating standards, and updating the certification exam (Shumate et al., 2004). A review of competency literature suggests that competency exploration is important. The initial establishment of competencies occurred with Coffey?s 1978 study. This study gave credibility to the field of vocational evaluation and allowed for later development of the certification exam (Zwyghuizen, 1980). Newman and Waechter (1997) explored competencies for the expressed purpose of updating the current certification exam. Revision of the certification exam is always important to the profession as work environments, and various tasks or functions of the vocational evaluation change continually (Shumate et al., 2004). Despite the importance of competency studies, review of the literature finds a lack of recent studies. Previous studies (Boyer-Stephens, Waechter, & Newman, 1999; Coffey, 1978; Leahy & Wright, 1988; Taylor, Bordieri & Lee, 1993) have ranged from national role and function studies to competency studies. The most recent study is by Hamilton (2003) and was sponsored by the Commission on Certification of Work Adjustment and Vocational Evaluation Specialists (CCWAVES). It is entitled The Role and Function of Certified Vocational Evaluation Specialists: A Survey of Practice in North America. This study investigated the roles of vocational evaluators and found that the overall role and functions of vocational evaluators to be similar across employment settings. Hamilton (2003) did find differences between the two groups of private sector vocational evaluators and evaluators working in public agencies, schools, or 3 not-for-profit settings. Those respondents in the private sector rated less intensive evaluation techniques as related to occupational analysis or information to be more important. They rated certain techniques to be more utilized as well. These techniques include transferable skills analysis, job matching, and labor market research. Moreover, there have been no competency studies which assess curriculum and perceptions of vocational evaluation practitioners. Thomas and Sigmon (1989) completed a competency study which reviewed the curriculums of 12 graduate specialty vocational evaluation programs. They found a lack of uniform competencies within these graduate specialty vocational evaluation programs. The problems in curriculums were attributed to decreased federal funding for research, university training, and in-service training opportunities in vocational evaluation. Overall vocational evaluation curriculums were found to be lacking. Areas of insufficiency were not identified by the authors; rather, a new curriculum was proposed focusing on the dynamic of the following two areas: knowledge of instruments and clinical competencies; and skill with instruments and clinical competencies. Shumate et al. (2004) concur that ?While CCWAVES requires graduate level coursework in specified knowledge and performance areas, most graduate programs offering curricula in vocational evaluation lack many key knowledge and content areas? (p. 34). These key knowledge and content areas have not been identified due to a lack of research. Thomas and Sigmon (1989) advocated for establishing vocational evaluation curriculum standards. The importance of updating curricula to match expanding service markets and settings cannot be understated (Thomas & Sigmon, 1989). 4 Competency studies are needed to not only improve the curriculum but also to promote the professional nature of vocational evaluation (Taylor & Pell, 1993). Hamilton (2003) states ?There exists a critical need for a commitment to research that will facilitate continuity of educational preparation and standards to enhance the overall discipline of vocational evaluation? (p. 9). A survey conducted by Saxon, Spitznagel, and Kennison (1999) found that the field of vocational evaluation lacks professional status amongst other related fields. The survey concluded that college students view the field of vocational evaluation as lacking status. They placed vocational evaluation last in terms of status amongst a list of 18 other allied health careers (Saxon et al., 1999). There is a concern by some in the field that vocational evaluation lacks status and a certain professional image. Public image is being questioned as vocational evaluators in many states have expressed concern about the impact public image has had on the profession (Fried, Harrand, Dowd, & Schuster, 1994). Public image and professional status are important as vocational evaluators are currently being utilized less by some referral sources. An unpublished document entitled ?A New Paradigm for Vocational Evaluation: Empowering the VR Consumer Through Vocational Information? reports on the decreased utilization of vocational evaluation services. This document is a result of meetings held in May of 2004 by the 30 th Institute on Rehabilitation Issues and sponsored by Institute on Rehabilitation Issues (IRI). In this document, it is reported that some state departments are choosing to reduce their use of vocational evaluators. Shumate et al. (2004) concur that on a national level the purchase of vocational evaluation services are on the decline due to cost containment efforts. In addition, Taylor and Bordieri (1993) found that rehabilitation counselors are not always 5 satisfied with the reports that they receive from vocational evaluators as some important factors were not included in vocational evaluation reports. These reports traditionally include work personality, physical and cognitive aspects, specific job selection, and formal education and training. In this study, a total of 374 rehabilitation counselors from 4 Midwestern states were surveyed to determine their perceptions on the information received from a vocational evaluation. Taylor and Bordieri (1993) noted that responding rehabilitation counselors found information from vocational evaluations was important to vocational planning. They found that rehabilitation counselors purchased vocational evaluation services for 20% to 80% of their clients. Vocational evaluation is a unique field which uses work as its main focal point of assessment (Pruitt, 1972). Today, vocational evaluators are concerned about support from the public and from legislation. The evolving discipline of vocational evaluation is facing criticism by some. Current exploration of vocational evaluation competencies is important for the field. Hamilton surmises ?paramount to the profession is the need for ongoing investigation and empirical validation of competencies and work roles of practitioners to insure consistency and defensibility of vocational evaluation practices of the 21 st century? (p. 9). Enhancement and increased credibility, which are aided by established competencies, are two important goals for the field of vocational evaluation (Hamilton, 2003). Establishing competencies is also necessary for ensuring effective teaching at the university level (Taylor & Pell, 1993). Vocational evaluation educational standards are lacking resulting in some programs which have outdated curricula (Thomas & Sigmon, 1989). 6 Statement of the Research Problem Although current research studies have identified vocational evaluation competencies and their importance to the profession, there remains an absence in research of vocational evaluation competencies in relation to curriculum. Critical questions regarding the actual teaching of essential competencies in the curriculum exist (Taylor & Pell, 1993). The field of vocational evaluation is experiencing criticism and a lack of educational standards (Thomas & Sigmon, 1989). Research is needed to address the emphasis of vocational evaluation competencies in the curriculum. Purpose of the Study The purpose of this study was to examine faculty and graduates? perceptions of vocational evaluation competencies in the educational curriculum. Perceptions of the amount of emphasis placed on vocational evaluation competencies were examined. The broad question to be answered was whether or not competencies were perceived as being adequately included in university programs. Specific questions concerned the following: faculty members? ratings versus vocational evaluation graduates? ratings of emphasis of competencies in curriculum and faculty members? ratings versus non-vocational evaluation graduates? ratings of emphasis of competencies in curriculum. Research Questions For this study, the following research questions were developed: 7 1. Is there a difference between the perceptions of vocational evaluation faculty and graduates on their ratings of vocational evaluation competencies in the educational curriculum? a. Are there differences among the perceptions of vocational evaluation faculty and vocational evaluation graduates, vocational rehabilitation graduates with a specialty in vocational evaluation, vocational rehabilitation graduates with no specialty in vocational evaluation, and unspecified (other) graduates? 2. Are there significant differences between the perceptions of vocational evaluation faculty and graduates on their ratings of vocational evaluation competencies in the educational curriculum? a. Is there a difference between the perceptions of vocational evaluation faculty and vocational evaluation graduates on their ratings of vocational evaluation competencies in the educational curriculum? b. Is there a difference between the perceptions of vocational evaluation faculty and vocational rehabilitation graduates (with a specialty in vocational evaluation) on their ratings of vocational evaluation competencies in the educational curriculum? c. Is there a difference between the perceptions of vocational evaluation faculty and vocational rehabilitation graduates (with no specialty in vocational evaluation) on their ratings of vocational evaluation competencies in the educational curriculum? 8 d. Is there a difference between the perceptions of vocational evaluation faculty and unspecified (other) graduates on their ratings of vocational evaluation competencies in the educational curriculum? Significance of Study Results of the study may assist university programs in assessing their curriculums in terms of vocational evaluation competencies. It will also provide information about differing emphasis of competencies in various university programs and differing perceptions between vocational evaluation faculty and graduates. Limitations and Assumptions Participants used in this study were selected based on their affiliation with two organizations. First, faculty members were identified only from the 12 universities listed on a Commission on Certification of Work Adjustment and Vocational Evaluation Specialists? (CCWAVES) website (www.ccwaves.org). In order to maintain anonymity of the participants, only faculty members listed on this site were emailed the link to the survey. The surveyed faculty members were listed by CCWAVES as being identified with universities offering degrees in vocational evaluation and assessment. Faculty members from other rehabilitation counseling programs with an emphasis in vocational evaluation were not surveyed. Secondly, the survey was sent to members of Vocational Evaluation and Career Assessment Professionals (VECAP). The many vocational evaluation practitioners who are not members of VECAP and who did not receive university training were not 9 included in this study. The results of this study are limited to CCWAVES and VECAP, and they should not be generalized to other populations. As noted in the conclusion, the type of participants used limit the generalizability of the results. Currently VECAP consists of 170 members overall with less than 25 percent of these members holding certification. The organization entitled Vocational Evaluation and Work Adjustment Association (VEWAA) was contacted but did not provide access to contact information for this study. Using participants from VEWAA could have added to the significance of the study. Finally, this study assumed that graduates and faculty who responded to the survey answered questions correctly. Definition of Terms Certification: a certified practitioner has demonstrated possession of at least an acceptable minimal level of performance with regards to the practice of vocational evaluation. The purpose of certification is to establish standards that may be used by any interest group, agency or individual with regard to vocational evaluation services (CCWAVES, 2005). Commission on Certification of Work Adjustment and Vocational Evaluation Specialists (CCWAVES): This non-profit, independent certification body was formed in 1981 with the purpose of identifying acceptable minimal levels of knowledge of the practice of vocational evaluation and work adjustment (CCWAVES, 2005). Specific academic training, practicum experience, and knowledge are proscribed for practitioners by this commission (Fry & Garner, 1993). 10 Commission on Accreditation of Rehabilitation Facilities (CARF): This commission is known today as the Rehabilitation Accreditation Commission. CARF was the first regulatory body to accept Vocational Evaluation and Work Adjustment Association (VEWAA) standards for accredited facilities offering vocational evaluation services. CARF supports the standards of practice for vocational evaluation services (Hamilton, 2003). Competencies: ?Competency is the knowledge, skills, or abilities for a task performance, frequently related to a performance or behavioral standard; a task is what a worker does in performing his or her job, and a function is a similar tasks grouped together to constitute a worker function? (Pruitt, 1986, p. 266). Rehabilitation Process: This process involves those services provided to the consumer which are designed to increase personal satisfaction and adjustment of the consumer. The services relate to the total needs of the consumer and are intended to increase role fulfillment in family, social, and occupational areas (Hamilton, 2003). Standards of Practice: Standards of practice concerns recognized codes to be used in vocational evaluation as determined by CCWAVES. CCWAVES currently recognizes standards set forth by the Interdisciplinary Council on Vocational Evaluation and Assessment which was formed in 1992 (CCWAVES, 2005). Vocational Evaluation:? Vocational (work) evaluation is a comprehensive process that utilizes work, real or simulated, as the focal point for assessment and vocational counseling to assist individuals in vocational development. Vocational (work) evaluation incorporates medical, psychological, social, vocational, educational, cultural 11 and economic data to assist in the attainment of goals of the evaluative process? (Zwyghuizen, 1980, p. 127). Vocational Evaluation and Work Adjustment Association (VEWAA): This Association is a division of the National Rehabilitation Association. This national Association has members with interest and engagement in vocational evaluation and/or the adjustment of persons with disabilities (Dowd, 1993). Work Adjustment Specialists: These specialists are responsible for carrying out a work adjustment program. Work adjustment refers to the training or process involving work related activities. This process assists consumers in understanding work and in adapting attitudes and behavior in order to develop functional capacities (Dowd, 1993). Summary The field of vocational evaluation is continuing to develop with periods of expansion and periods of contraction in terms of utilization of services. Some vocational evaluators find themselves today concerned about their credibility and professional identity. Vocational evaluators would benefit from continued exploration of competencies (Hamilton, 2003). University programs that offer courses in vocational evaluation would likewise benefit from assessing curriculum in terms of meeting minimal level competencies (Taylor & Pell, 1993). Many studies have been completed over the last decade in regards to vocational evaluation competencies. However, investigations into vocational evaluation curriculum and currently established competencies have been few. 12 II. REVIEW OF LITERATURE The purpose of this study was to investigate faculty and graduates? perceptions of competencies in vocational evaluation curriculums across the country. This chapter presents a review of literature on the topic of competencies of vocational evaluators. This chapter is divided into the following sections: (a) history, (b) the profession, (c) standards and certification, (d) facing criticism, (e) competency studies, (f) university programs offering courses, (g) summary of the literature. History Vocational evaluation formed officially as a discipline during the second half of the last century. The field today is an essential part of the rehabilitation process (Hamilton, 2003). The rehabilitation process involves providing services such as medical restoration, counseling, training, guidance and placement in order to find employment for persons with disabilities. According to Rubin and Roessler (2001), the rehabilitation process can be divided into four stages to include evaluation, planning, treatment, and termination of services. The first step in the vocational rehabilitation process may be a formal vocational evaluation. This evaluation is intended to answer questions regarding where to begin and proceed in the rehabilitation process. The evaluation gives information regarding vocational choices and alternatives, an individual?s competencies, 13 and services needed by the individual to realize their vocational goals (Rubin & Roessler, 2001). Vocational evaluation itself can be defined as a general assessment of an individual?s vocational potential. This assessment consists of interviews, testing, work tasks, and behavioral observation. Medical and psychological data is used by the evaluator to develop conclusions and provide recommendations to the rehabilitation counselor (Rubin & Roessler, 2001). The first formal definition of vocational evaluation was provided in 1972 at the Tenth Institute on Rehabilitation Services meeting. At this Institute meeting, vocational evaluation was defined as: Vocational (work) evaluation is a comprehensive process that utilizes work, real or simulated, as the focal point for assessment and vocational counseling to assist individuals in vocational development. Vocational (work) evaluation incorporates medical, psychological, social, vocational, educational, cultural and economic data to assist in the attainment of goals of the evaluative process. (Zwyghuizen, 1980, p. 127) As a professional discipline within the fields of rehabilitation and psychology, vocational evaluation is relatively young. Vocational evaluation formed during the rehabilitation facility movement of the 1950s (Pruitt, 1977). The passage of the Social Security Act in 1935 established vocational rehabilitation as a government program. This program was poorly funded until after World War II which began to provide persons with disabilities increased opportunities for participation in the workforce (Rubin & Roessler, 2001). Vocational rehabilitation entered a rapid period of expansion in the 1950s. The Vocational Rehabilitation Act of 1954 provided massive amounts of funding for state 14 rehabilitation programs (Rubin & Roessler, 2001). Vocational evaluation saw a rapid expansion as well as it progressed from being relatively non-existent to an essential service in the rehabilitation process (Shumate, Hamilton, & Fried, 2004). The need for vocational evaluation services expanded significantly due to legislation passed after 1950. Shumate et al. surmise ?The urgent need for practitioners specializing in vocational evaluation evolved primarily from significant changes to rehabilitation legislation during the 1950?s and 1960?s? (p. 29). The 1965 amendments to the Vocational Rehabilitation Act expanded the scope of services to include broader populations such as persons with substance abuse and behavioral disorders. Extended evaluation services were authorized under this amendment. Applicant eligibility now was mandated from 6 to 18 months allowing for greater services to be given to the individual (Rubin & Roessler, 2001). These services included vocational evaluations. As a result of the 1965 amendment, vocational evaluation developed into an essential diagnostic tool used by rehabilitation counselors (Hamilton, 2003). The disability consumer movement of the 1970s brought about focus on the successful integration of individuals with severe disabilities into typical work environments. Persons with disabilities began to fight for their equal rights in the 1970s. They fought for equality in their rights to life, liberty, and the pursuit of happiness which in turn played a significant role in legislation reform (Rubin & Roessler, 2001). Vocational evaluation was seen increasingly as an essential service to help with this integration of persons with disabilities. The Rehabilitation Act of 1973 and its subsequent amendments mandated the service of individuals with severe disabilities, consumer involvement, and program evaluation (Shumate et al., 2004). 15 The Education for All Handicapped Children Act of 1975 (Public Law 94-142) mandated that all students with disabilities be provided with vocational education programs. In addition to these programs, every student now had educational rights in terms of appropriate placement in educational programs (Rubin & Roessler, 2001). Vocational evaluation was identified as one type of service that could aid in the appropriate placement of students (Hamilton, 2003). In 1984, the Carl D. Perkin?s Act mandated that all students enrolled in vocational education programs be provided with vocational evaluation services (Taylor & Pell, 1993). The Americans with Disabilities Act of 1990 (Public Law 101-336) was passed and became a progressive law. It intended to increase the employment opportunities for persons with disabilities. It intended specifically to eliminate environmental and societal barriers to successful employment and to increase the participation of persons with disabilities in the workforce (Rubin & Roessler, 2001). The act increased the need for vocational evaluation services due to increasing demands for identification of potential vocations. Persons with disabilities deserved adequate representation in the workforce, and vocational evaluation services could aid in this goal (Hamilton, 2003). President Clinton signed into law two initiatives entitled the Workforce Investment Act of 1998 and the Ticket to Work and Work Incentives Improvement Act (1999). The Workforce Investment Act of 1998 (Public Law 105-220) revised the rehabilitation act of 1973 in order to streamline services. Persons with disabilities now had direct access to employment services. The act called for a one-stop service provision for vocational rehabilitation services (Rubin & Roessler, 2001). Centers called one-stop providers were created to offer services for individuals. Vocational evaluation was 16 included as one of the services (Shumate et al., 2004). The Ticket to Work Program began in 2002 under this program. Eligible social security beneficiaries with disabilities were now able to receive free tickets. These tickets could be used to obtain vocational rehabilitation services, including vocational evaluation services (Rubin & Roessler, 2001). The field of vocational evaluation expanded significantly due to passages in legislation. The development of university programs in vocational evaluation are a direct result of this increased need for vocational evaluators (Shumate et al., 2004). The first program in vocational evaluation developed at Stout State University in 1966. Concurrently, the actual formation of a professional organization called Vocational Evaluation and Work Adjustment Association (VEWAA) began in 1966 at an ad hoc committee meeting at the National Rehabilitation Association (NRA) national conference (Hoffman, 1971). VEWAA was officially established in 1968 when it was declared as a division of the NRA. At the time of its formation, VEWAA had a roster of 470 members representing 42 states plus Washington, DC and Canada. By 1971, this number had risen to just over 1,000 members (Hoffman, 1971). The actual formation of VEWAA can be traced back to the American Association of Work Evaluators (AAWE). The AAWE held meetings in the mid 1960s and grew to an organization involving six states. The membership at the time of its disassembly was 70 members. The AAWE organization dissolved in the 1960s, because it did not have sufficient resources to become a national organization. It was replaced before the end of the decade by VEWAA, and the name was changed from work evaluators to vocational evaluators (Hoffman, 1971). 17 VEWAA developed task forces along the way as a product of the vocational evaluation research project (Hamilton, 2003). The project lasted from 1972 to 1975, and the first task force established a three phase rehabilitation model. This model included the phases of interview and screening, vocational counseling, and when necessary, evaluation. Vocational evaluation was described as an alternative assessment or an assessment of last resorts. The second task force defined the tools used by a vocational evaluator. The third task force defined the roles of the vocational evaluator, and the fourth task force identified barriers to success (Piccari, LeBlanc, Kells, Baker, Meyer, King, & Bleeker, 1975). The fourth task force of this project identified barriers to success for vocational evaluation. These barriers included lack of communication with other professionals, lack of a professional image for vocational evaluators, and lack of a common language used amongst evaluators. The fifth task force helped to establish standards for vocational evaluation. The sixth task force of the project explored the relationship of vocational evaluation to other organizations and educational institutions. The public recognition of the field of vocational evaluation as a whole was called into question. National organizations and federal agencies were described as lacking in recognition of vocational evaluation. Suggestions for improvement in recognition of the field by others were generated by this task force (Piccari et al., 1975). Today the Commission on the Certification of Work Adjustment and Vocational Evaluation Specialists (CCWAVES) oversees standards of practice and other topics addressed by the vocational evaluation projects. CCWAVES was formed in 1981 with the purpose of identifying acceptable minimal levels of knowledge of the practice of 18 vocational evaluation (CCWAVES, 2005). Specific academic training, practicum experience, and knowledge are proscribed for practitioners by this commission (Fry & Garner, 1993). In addition, it is the sole certification body for vocational evaluators. Standards of practice and a code of ethics are published by this commission (CCWAVES, 2005). Vocational evaluation experienced significant growth due to legislation passed after the last half of the last century (Hamilton, 2003). The list of significant legislation includes the Social Security Act of 1935, the Vocational Rehabilitation Act of 1954 and its subsequent amendments, the Rehabilitation Act of 1973, the Education for All Handicapped Children Act of 1975, the Individuals for Disabilities Education Act, the Carl D. Perkins Act of 1984, the Americans with Disabilities Act of 1998, the Workforce Investment Act of 1998, and the Ticket to Work and Work Incentives Improvement Act of 1999 (Rubin & Roessler, 2001). University programs and VEWAA were formed in order to give training and standards to the significantly expanded field of vocational evaluation (Hamilton, 2003). The services of VEWAA were later augmented in the 1980?s by the formation of CCWAVES which provides standards, a code of ethics, and a body of knowledge for practitioners (Fry & Garner, 1993). The Profession Vocational evaluation is a relatively new field with its professional formation beginning in the 1960s (Hoffman, 1971). The foundation of vocational evaluation is rooted in various fields including the U.S. military, medicine, psychology, education, occupational therapy, and disability and rehabilitation legislation (Shumate et al., 2004). 19 Despite these roots, vocational evaluation finds itself today to be a unique practice as it uses work as a primary tool for determining potential (Pruitt, 1977). In addition, its certification body, CCWAVES, proscribes specific academic training, supervised practical experience, and a unique code of ethics (Fry & Garner, 1993) Although vocational evaluation finds itself today to be a critical component of the rehabilitation process, it was not until 1967 that vocational evaluators actually began essential training and preparation in their careers. Prior to this time, vocational evaluation practitioners evaluated consumers using few tested research methods and utilizing no specific academic preparation (Shumate et al., 2004). By the 1960s, federal grants began to be awarded to colleges and universities in order to train rehabilitation professionals in vocational evaluation (Shumate et al., 2004). The first university program offering vocational evaluation was at Stout State University which is now called University of Wisconsin-Stout (Hamilton, 2003). These programs were primarily developed in terms of curriculum to meet regional needs (Thomas & Sigmon, 1989). Professional formation evolved into certain steps as all professions have an evolutionary process (McDaniel, 1978). The field of vocational evaluation has followed a different process from other professions (Shumate et al., 2004). The rapid need for vocational evaluation services due to legislation and growth of rehabilitation services changed the process. The more traditional and predictable stages of professional development were averted due to this rapid growth (Hamilton, 2003). However, the growth of the field of vocational evaluation can still be applied to predictable stages of professional growth. Shumate et al. (2004) surmise, ?? vocational evaluation during the past half century has proceeded through predictable stages of professional status 20 development. However, as a relatively young profession, vocational evaluation continues to evolve and define its scope of practice? (p. 27). Professionalization has four stages according to Caplow (1966). The distinct four stages are as follows: forming a professional organization, changing the professions name, developing a code of ethics, and garnering support from the public and legislation (Caplow, 1966). The first step towards becoming a profession involves the formation of a professional organization. Inherent in the organization?s formation is the specificity that membership is open to a select few and is not available to those who are unqualified. The organization individualizes itself and begins the road to professional recognition (Caplow, 1966). The second step towards becoming a profession involves changing of the actual name of the profession. This change in name serves to differentiate the new professionals from the unqualified workers in the field (Caplow, 1966). Development of a code of ethics providing rules and benefits is the third step towards becoming a profession (Caplow, 1966). The fourth step towards becoming a profession involves getting support from the public and the law so that new occupational barriers are maintained (Caplow, 1966). Vocational evaluation continues to be in pursuit of support from the public and from legislation (Fried, Harrand, Dowd, & Schuster, 1994). In addition, the field of vocational evaluation as a whole is still in a state of evolution as it continues to try to define its field of practice (Shumate et al., 2004). In 1994, Fried et al. reported on viewpoints of vocational evaluators. They compared the viewpoints between practitioners residing in three different states. One of the key concerns of all practitioners interviewed involved legislation and the impact others outside the field of vocational evaluation had on 21 traditional standards and best practices. These concerns are well founded. As a field still in a state of evolution, vocational evaluation may be significantly impacted by legislation and public perception (Fried et al., 1994). The field of vocational evaluation is relatively young and is still in a state of evolution (Shumate et al., 2004). Although professions can be charted into specific developments (Caplow, 1966), vocational evaluation is paving its own path partly due to its significantly rapid growth after the last half of the last century (Shumate et al., 2004). It is still defining scopes of practice. In addition, practitioners are concerned about public perception and future legislation (Fried et al., 1994). Standards and Certification Standards and certification set rules and boundaries for the practice of vocational evaluation and help to further the field on the track towards becoming a profession (Zwyghuizen, 1980). At the time of VEWAA?s formation, a set of ethics was in the process of being developed (Couch, 1971). The ethics were presented for consideration in 1969 and they were finally published in 1971 (Couch, 1971). The identification of ethics is closely related to the formation of standards. The development of standards is crucial for any profession. They are necessary for development of certifications. In addition, standards shape the identity of a profession and ensure distinction of skills (Taylor & Pell, 1989). The first developed standards applied only to vocational evaluators in vocational rehabilitation facilities and also in non-rehabilitation facility programs. New standards were soon developed which made a 22 clear distinction between two similar services: vocational evaluation and work adjustment (Fry & Garner, 1993). Even after these standards were developed, they were not mandated. It was not until 1980 that final implementation of certification and standards procedures were mandated by VEWAA. Before 1980, states had set up their own certification and criteria which presented threats to the profession of vocational evaluation as a whole. During that time, the standards also varied between states (Zwyghuizen, 1980). The development of standards continues today as establishment of standards needs to be an ongoing process (Hamilton, 2003). Vocational evaluation standards were formed beginning in the 1970s (Baker, 1977). In 1972, VEWAA applied for a grant from the Rehabilitation Services Administration (RSA). This grant, along with assistance from the National Accrediting Council (NAC), allowed for the development of standards in vocational evaluation (Baker, 1977). First, the Commission on Accreditation of Rehabilitation Facilities (CARF) standards for accreditation were studied and used as a model. A freestanding set of standards for vocational evaluation was developed at this time (Baker, 1977). The final recommendation for standards was submitted, and the standards were adopted by the CARF board in 1976. These standards were published in conjunction with a 1978 edition of the Standards Manual for Rehabilitation Facilities by CARF (Baker, 1977). The first standards committee recognized four significant tools used in vocational evaluation. These tools included psychometrics, work samples, simulated job stations, and on the job evaluations (Piccari et al., 1975). The new standards specified requirements for each tool or technique. They also mandated that the vocational evaluator 23 use at least two or more of the tools for each evaluation. Other standards included specific written plans for each individual served (Piccari et al., 1975). VEWAA developed task forces along the way as a product of the vocational evaluation research project. This project helped to aggressively promote standards and certification for vocational evaluators (Piccari et al., 1975). The project lasted from 1972 to 1975, and the fifth task force was the most significant in terms of standards. The first task force established a three phase rehabilitation model. This model included the phases of interview and screening, vocational counseling, and when necessary, evaluation. Vocational evaluation was described as an alternative assessment or an assessment of last resorts. The second task force defined the tools used by a vocational evaluator. The third task force defined the roles of the vocational evaluator, and the fourth task force identified barriers to success. Finally the fifth task force helped to establish standards for vocational evaluation (Piccari et al., 1975). Standards and certification are interrelated as certification procedures may not be developed without the aid of established standards (Taylor & Pell, 1989). The first certification committee was held by VEWAA in 1975. The VEWAA board of directors voted in this year to have competency based certification. The competency based certification was changed to a minimum standards certification in 1978. Coffey?s (1978) survey, which identified competency statements for vocational evaluation, provided material for the competency exam. Coffey (1978) identified nine competency domains which formed a foundation for the present certification of vocational evaluators. These nine areas included (a) professional background; (b) inter-agency relationship; (c); initial evaluation procedures; (d) determination of vocational direction; (e) analysis and 24 synthesis of evaluation data; (f) communication; (g) adjustment; (h) referral and placement; and (i) administration. The 1979 studies completed by Sink and Porter gave fuel to the certification procedures as the comparison between competencies of rehabilitation counseling and vocational evaluation proved that vocational evaluation was a specific and unique field (Coffey & Mason, 1980). Competency studies, using survey research methods, helped give universal standards and furthered along of the development of national certification. In 1979 a study by Pruitt defined universal tasks of the evaluator, also giving clarification about appropriate material for the certification exam (Zwyghuizen, 1980). In addition, the study completed in 1997 by Newman and Waechter was intended to aid in the updating of the vocational evaluation certification exam. Standards were developed to set rules and boundaries for the practice of vocational evaluation and to further the field on the track towards becoming a profession (Hamilton, 2003). The development of standards is crucial for any profession as they determine skills, identity, and certification of professionals (Taylor & Pell, 1989). Certification is not possible without establishing standards (Taylor & Pell, 1989). The development of standards continues today as establishment of standards needs to be an ongoing process (Hamilton, 2003). Facing Criticism Vocational evaluators are concerned that the profession as a whole is lacking in public recognition (Fried, 1994). A survey was conducted by Saxon, Spitznagel, and Kennsion (1999) to determine the perceived professionalism of certain allied health 25 careers. The allied health careers, numbering 19 in all, included vocational evaluation, occupational therapy, rehabilitation engineer, rehabilitation nurse, respiratory therapy, and clinical social worker to name a few. Each career was listed with a description of job duties to aid the respondents in rating the careers. Nearly 100 University of Florida undergraduate students were surveyed. The students rated each career based on their perceived status of the career. Vocational evaluation rated 19th in the total rankings of occupations based on perceived status. The five topped ranked careers included physical therapy, respiratory therapy, clinical/counseling psychologist, practical nurse, and physician assistant (Saxon et al., 1999). In addition, the profession of vocational evaluation is currently experiencing criticism involving its perceived value to the field of rehabilitation. Vocational evaluation in public vocational rehabilitation (VR) agencies is suffering from cost-containment efforts originating from funding limitation and managed care (Shumate et al., 2004). Many state departments have chosen to reduce their use of vocational evaluators. This reduction may be due to varying perceptions of vocational evaluation. An unpublished document entitled ?A New Paradigm for Vocational Evaluation: Empowering the VR Consumer Through Vocational Information? reports on the decreased utilization of vocational evaluation services. This document is a result of meetings held in May of 2004 by the 30 th Institute on Rehabilitation Issues. Concerns were raised at these meetings about some rehabilitation professionals? narrow scope of vocational evaluation, and the perception of vocational evaluation as merely involving paper and pencil or dexterity testing. 26 Taylor and Bordieri (1993) surveyed 374 rehabilitation counselors from four Midwestern states to determine their perceptions of reports received from vocational evaluators. The respondents to the survey identified three factors which are important in a vocational evaluation report. These three factors include (a) work personality, physical and cognitive, (b) specific job selection, and (c) formal education and training. Many of the respondents to the survey communicated that these factors were not always covered in the evaluation reports. In other words, they indicated that there was information they believed important to vocational planning which was not included in the vocational evaluation results they received. However, it should be noted that overall respondents believed vocational evaluation to be a valuable service. The study concluded that rehabilitation counselors found information from evaluations to be important in terms of vocational planning (Taylor & Bordieri, 1993). In addition, vocational evaluation services appear to be expanding in the private sector (Shumate et al., 2004). Public image is being questioned as vocational evaluators in many states have expressed concern about the impact public image has had on the profession (Fried, Harrand, Dowd, & Schuster, 1994). This concern is expressed clearly by the Saxon, Spitznagel, and Kennison (1999) survey which found that vocational evaluation lacks professional status amongst other related fields. Public image and professional status are important as vocational evaluators are currently being utilized less by some referral sources. On a national level, the purchase of vocational evaluation services are on the decline at the state level due to cost containment efforts (Shumate et al., 2004). 27 Competency Studies Previous competency studies have ranged from national role and function studies to competency studies. The list includes Coffey (1978), Leahy and Wright (1988), Taylor, Bordieri and Lee (1993) and Boyer-Stephens, Waechter, and Newman (1999). The most recent study is by Hamilton (2003) and sponsored by the Commission on Certification of Work Adjustment and Vocational Evaluation Specialists (CCWAVES). It is entitled The Role and Function of Certified Vocational Evaluation Specialists: A Survey of Practice in North America. A variety of research methods have been used to identify competencies of the vocational evaluator. The first documented competency study utilized task analysis. Pruitt (1972) asked 45 evaluators to complete job analyses which resulted in 67 identifiable tasks performed by the evaluator. The identified functions included (a) evaluation; (b) counseling and interviewing; (c) training; (d) administration; (e) occupational analysis; (f) communicating and relating; and (g) research and development (Leahy & Wright, 1988). The competency study completed by Coffey in 1978 provided invaluable contributions to the field of vocational evaluation. Coffey surveyed a large number of evaluators on a considerable number of competencies. After gathering a list of over 2,500 competency statements, Coffey synthesized a list of 175 primary vocational evaluation competencies. He surveyed three separate groups of vocational evaluators to include 96 students or practitioners and 20 rehabilitation facility educators in the southeastern region of the United States. Each participant rated competencies using a five point Likert-type scale in terms of importance to practice. Coffey then found the mean ranking of 28 competencies by category. The correlation of competency rankings amongst the groups was found to be .80 to .90. Sink and Porter (1978) examined the competencies as they related to the rehabilitation counselor and the vocational evaluator. They found that indeed sufficient differences existed amongst competencies of the two professions, rehabilitation counseling and vocational evaluation. Similarities were found between the two groups; however, enough difference existed suggesting that vocational evaluation warranted its own curriculum and training programs (Sink & Porter, 1978). The identification of competencies of the two professions was a pressing issue at the time. In 1978, the National Association of Rehabilitation Counseling (NRCA) and VEWAA published a joint issue defining roles of the vocational evaluator and the rehabilitation counselor. A couple of studies thereafter took the initiative to define the similarities and differences between the two professions. Bozarth was amongst one of the members in the rehabilitation field who advocated that vocational evaluation was not a profession in its own right (Leahy & Wright, 1988). While some in the rehabilitation field believed that vocational evaluation was too similar to vocational rehabilitation in order to constitute a separate profession, a study by Sink, Porter, Rubin, and Painter (1979) found otherwise. This survey, which took place at a national seminar in Atlanta, was undertaken in part by the National Consortium on Competency Based Education. Sink et al. (1979) reviewed the literature and identified over 1,000 competencies of rehabilitation counselors and vocational evaluators. These competency statements were consolidated to 298 and then reviewed by a panel of experts. The panel of experts consisted of 27 participants. They analyzed the competencies in 29 terms of being equally needed by the rehabilitation counselor and vocational evaluator, or by being needed by just one of the professions. They found that 55 percent of the competencies identified were needed by both rehabilitation counselors and vocational evaluators. These shared areas included diagnostic, counseling, and job placement related competencies. Exactly 12 percent of the competencies identified were specifically the domain of the vocational evaluator. These competencies included assessment related activities such as selection and utilization of standardized instruments and work samples (Gannaway & Sink, 1979). Sigmon (1982) surveyed vocational evaluators regarding competencies, but chose to use alternative survey methods to the typical Likert-type scale. He used the Ebel (1972) and Angoff (1971) methods. Using Ebel?s method, the participants in the study were asked to rate Coffey?s (1978) 175 competency statements in terms of being essential, important-easy, important-medium, acceptable-easy, acceptable-medium, acceptable-hard, questionable easy, questionable-medium, and questionable-hard. The participants determined the relevance as well as the difficulty level of each competency. The Angoff procedure asked for the evaluators to estimate the percentage of entry level vocational evaluators they thought would possess the specific competency. The reliabilities were found to be .97 for the Ebel method and .93 for the Angoff methods (Sigmon, Couch, & Halpin, 1987). Sigmon et al. (1987) completed a correlational study between the two studies. The Coffey study (1978) and the Sigmon study (1982) were compared. The two studies were chosen, because they used entirely different survey methods. The two studies also differed in the population surveyed. Coffey?s sample consisted of three groups: 30 practitioners, educators, and graduate students. Sigmon used two groups: field personnel and educators. An emerging consensus of the role and function of the evaluator was found. Using the two studies, the researchers found correlation coefficients that ranged between .63 to .88. Sigmon?s results using the Angoff method showed the least amount of agreement (.63) while the highest coefficient (.88) existed between Sigmon?s Ebel method and Coffey?s survey (Sigmon et al., 1987). Although these studies helped clarify the vocational evaluator?s role, a large scale study was still needed using sound research methods to further define the competencies (Leahy & Wright, 1988). In the latter half of the 1980s, a nationwide research effort was sponsored by the National Council on Research Education (NCRE) and funded by the National Institute on Disability and Rehabilitation Research. In 1986, a large scale survey was conducted with 3,614 practitioners being surveyed including 803 vocational evaluators. The surveyors of this study reviewed ten previous rehabilitation competency studies and formalized a list of 114 competency statements. In the end, a total of 270 usable questionnaires were completed by vocational evaluators. These evaluators rated the competencies using a five point Likert-type scale based on importance to effective service delivery. The six important competency clusters found included (a) assessment planning and interpretation, (b) vocational counseling, (c) assessment administration, (d) job analysis, (e) case management, and (f) personal adjustment counseling. Four competency areas were judged to be insignificant in terms of effective service delivery. These included (a) job placement, (b) professional and community involvement, (c) group and behavioral techniques, and (d) consultation (Leahy & Wright, 1988). 31 Taylor, Bordieri, and Lee (1993) undertook a national study by surveying evaluators about their job tasks. This survey utilized the largest sample of evaluators to date but had a low response rate of 18.8%. Evaluators, amounting to 526 participants, came from the following job settings: state rehabilitation offices, public schools, non- profit facilities, and the private sector. The survey entitled The Vocational Evaluator Job Task Inventory?2 (VEJTI?2) was formulated using previous role and function surveys. The job tasks were weaned down from 182 to 84 tasks and were evaluated by the CCWAVES Research Committee (Taylor et al., 1993). Taylor et al. (1993) found that most of the job tasks between evaluators in different job settings were similar. Overall, there were six factor-derived domains which included (a) vocational counseling; (b) behavioral observation; (c) occupational development; (d) standardized assessment; (e) professionalism; and (f) case management. All surveyed evaluators determined the job tasks in the vocational counselor category to be important. However, differences were found in the other five categories. For example, private vocational evaluators perceived behavioral observation as less important than public school vocational evaluators did. In addition, private evaluators found occupational development as more important than their public school colleagues did (Taylor, Bordieri, Crimando, & Janikowski, 1993). In 1997, CCWAVES contracted with Research and Evaluation Associates Inc. to identify competency areas which are necessary for entry level vocational evaluators to successfully perform and fulfill their job responsibilities. The study was commissioned to further promote the field of vocational evaluation. Shumate et al. (2004) summarize the importance of commissioning this type of study when they state 32 With the expansion of venues in which CVEs work, the various tasks and functions they perform frequently must be verified, updated and revised in order for the credentialing process to remain relevant. This allows the CCWAVES to identify new or outdated competency domains for the profession, and to revise the CVE examination. (p. 32) At the time of the study, CCWAVES had determined that the 14 presently identified content areas forming the basis of the certification exam could be condensed. The focus of the study concerned minimal competency in terms of application, skills, knowledge, and ability. The main goal of the study was to identify minimum competency areas particularly as they relate to the certification exam. In addition, the results of the study were found to be potentially useful to identify courses at the university level which are mandatory for vocational evaluation curricula. It was also determined that a handbook for best practice or best use in vocational evaluation could be formulated using the results of the study (Newman, Waechter, Nolte, & Boyer-Stephens, 1998). Important skills of the entry level vocational evaluator were first identified. The CCWAVES board then categorized the skills into five domains to include: The Professional, Tools and Techniques, The Vocational Evaluation Process, Characteristics of Consumers, and The World of Work (Newman et al., 1998). Twenty-three experts were chosen by the board after reviewing experts? degrees, experiences, and reputations. The experts completed a questionnaire developed by the researchers and then rated and ranked each ability using a scale of 1 to 5 and a ranking from most to least important. After rating and ranking each item, the experts were asked to include any comments. 33 Data was formulated using a formula for both ratings and rankings. The relative value was found by averaging ratings and rankings (Newman & Waechter, 1997). Interjudge agreement was estimated using Kendall?s Coefficient of Concordance. There was found to be significant interjudge agreement in only one out of the five domains which was entitled characteristics of consumers. While there was not an overall agreement on the relative ranking of importance of the items, the average ratings for items within all questionnaires was high. In addition, the major cause of disagreement in terms of ranking could be attributed to only one judge per domain (Newman et al., 1998). It was also noted that ?The objectives identified were all considered to be important by 23 nationally recognized experts, but as one would expect due to differences in training philosophies, education, experiences, etc., there was a lack of agreement on relative importance of objectives? (Newman et al., 1998, p. 78). In addition, the experts were questioned using a qualitative design revealing several new potential competencies. These new competencies included multicultural issues, computer literacy, and case management. There was general agreement amongst the experts that entry level evaluators needed basic skills in such areas as behavioral observation, report writing, interviewing, test statistics, counseling, knowledge of legislation, and good communication skills. However, the qualitative comments concluded that entry level evaluators usually do not possess the synthesis skills for effective report writing and that more training was needed in the areas of behavioral observation, legal/ethical issues, cultural diversity and computer literacy. Given the purpose of the study in mind, it was essential to find that most experts believed the competencies needed for entry level evaluators to be consistent with the CCWAVES 34 Knowledge and Performance Areas. It was noted in conclusion that the qualitative research methods used and the limited number of experts may limit the generalizability of the results (Boyer-Stephens, Waechter, & Newman, 1999). The most recent study conducted by Hamilton (2003) and sponsored by CCWAVES is a dissertation entitled The Role and Function of Certified Vocational Evaluation Specialists: A Survey of Practice in North America. Results of this role and function study may help to reinforce the interdisciplinary and collaborative nature of vocational evaluation. Hamilton (2003) found certain primary knowledge areas possessed by vocational evaluators. These areas include the following categories or factors: (a) Foundations of Vocational Evaluation; (b) Standardized Assessment; (c) Occupational Information; (d) Implications of Disability; (e) Communication; and (f) Professional Networking and Coordination. A survey instrument entitled the Vocational Evaluation? Job Task and Knowledge Inventory (VE?JTAKI) was formulated. Based on qualitative and quantitative results, the final VE?JTAKI instrument contained 85 job task items and 55 knowledge areas. Certified Vocational Evaluators, 800 in number, were randomly selected and surveyed This study investigated the roles of vocational evaluators and found that the overall role and functions of vocational evaluators to be similar across employment settings. Hamilton (2003) did find differences between the two groups of private sector vocational evaluators and evaluators working in public agencies, schools, or not-for- profit settings. Those respondents in the private sector rated less intensive evaluation techniques as related to occupational analysis or information to be more important. They 35 rated certain techniques to be more utilized as well. The techniques include transferable skills analysis, job matching, and labor market research. University Programs Offering Courses In 1966, the first graduate degree program in vocational evaluation was established at Stout State University. Other university programs followed while even more added vocational evaluation concentrations to their rehabilitation counseling curricula (Shumate et al., 2004). There are currently 12 universities in the United States offering degrees in vocational evaluation. Many more universities offer courses or specialties in vocational evaluation to their rehabilitation counseling students. Vocational evaluation is taught in the universities at the master?s degree level. Some universities offer degrees in vocational evaluation while others offer specialties or coursework in vocational evaluation. A survey was conducted by Taylor, Pell, Chan (1998) concerning the teaching of specialized courses in vocational evaluation. Thirty-nine directors of master?s level rehabilitation counseling education programs were surveyed. Of these programs, only five offered an entire degree in vocational evaluation. Nine of the programs reported that they offered more then one course in vocational evaluation. The courses devoted entirely to vocational evaluation ranged from 1 to 15 in number. Thirteen of the programs offered only one course in vocational evaluation. Seventeen of the programs reported having no specific courses in vocational evaluation, but rather having vocational evaluation content which is spread across the main curriculum (Taylor et al., 1998). 36 Taylor and Pell (1993) conducted a study of ten university programs offering degrees in rehabilitation counseling with specialties in vocational evaluation. The intent of the study was to distill standards as set forth by CCWAVES, CARF, and CORE. In terms of demographics, Taylor and Pell (1993) found that the surveyed graduate programs dedicated between 3 to 30 semester hours to vocational evaluation coursework. Some programs offered a varied vocational evaluation curriculum while some only offered one to two generic assessment courses. Total program length varied from 42 to 60 semester hours to 65 to 72 quarter hours. Six of the programs were funded by Rehabilitation Services Administration (RSA) grant funds. Two of the ten programs were not actively recruiting students due to lack of interest and funds (Taylor & Pell, 1993). The training of vocational evaluators to obtain competencies comes from different sources. Some vocational evaluators receive their primary training at the university level while some receive no university or college training at all. Some sources state that universities and training programs are responsible for teaching most of the essential skills and competencies. Other sources state that the majority of competencies are obtained through on the job or short term workshops (Newman et al., 1998). The majority of research tends to indicate that most evaluators do not receive their training at the university level. Taylor, Bordieri, Crimando, and Janikowski (1993) completed two studies using a survey entitled the Vocational Evaluator Job Task Inventory. The first study solicited participants in Florida through mail outs to CARF Florida facilities listings, Florida?s Division of Workers Compensation, VEWAA and CCWAVES listing of CVE?s, and University of South Florida?s Vocational Evaluation Project. Of the 188 vocational evaluators who participated in the study, little of them had formal training in 37 vocational evaluation. Only five of the participants reported earning a degree in vocational evaluation (Taylor et al., 1993). For the second study, Taylor, Bordieri, and Lee (1993) surveyed evaluators on a national scale. Six mailing sources were used to identify participants in the study. These included (a) CCWAVES, (b) CARF, (c) VEWAA (d) National Association of Vocational Special Needs Personnel, (e) National Association of Vocational Assessment in Education, and the (f) Council for Exceptional Children ? Division of Career Development. Over 2,500 vocational evaluators were identified through randomized sampling. Of the 526 respondents to the survey, only 8.5% earned degrees with an emphasis in vocational evaluation. In addition, these evaluators cited on the job training or short term workshops as their most widely used source of training (Taylor et al., 1993). The national study completed by Leahy and Wright (1988) found that of the 270 viable respondents, 110 of them received academic training specifically in rehabilitation. Only 40 of the 110 respondents received academic coursework specific to vocational evaluation. Only 14.8% of the total sample received university training in vocational evaluation. Of the total respondents, 160 of them had a major in the field outside of rehabilitation (Leahy & Wright, 1988). Research indicates that competencies are best met through university or college level preparation. Masters trained vocational evaluators have significantly higher attainment of professional competencies than their counterparts who have rehabilitation related or unrelated degrees (Shapson, Wright, & Leahy, 1987). Shapson et al. (1987) found that higher levels of satisfaction, higher numbers of certified and higher perceived professional competencies were found in master?s level graduates. 38 Questions have arisen in regards to the quality of vocational evaluation education at the university level. Some have questioned the ability of universities to accurately prepare graduates to engage in effective and competent vocational evaluation practice. ?While CCWAVES requires graduate level coursework in specified knowledge and performance areas, most graduate programs offering curricula in vocational evaluation lack many key knowledge and content areas? (Shumate et al., 2004, p. 34). Thomas and Sigmon (1989) completed a competency study that reviewed the curriculums of 12 graduate specialty vocational evaluation programs. They found a lack of uniform competencies within these graduate specialty vocational evaluation programs. The problems in curriculums were attributed to decreased federal funding for research, university training, and in-service training opportunities in vocational evaluation. Overall vocational evaluation curriculums were found to be lacking. Areas of insufficiency were not identified by the authors; rather, a new curriculum was proposed focusing on the dynamic of the following two areas: knowledge of instruments and clinical competencies; and skill with instruments and clinical competencies. Furthermore, Thomas and Sigmon (1989) found that some graduate programs in vocational evaluation had developed without the aid of uniform standards or competencies. It was surmised that university programs at the time of their formation beginning in the 1960s were influenced by regional needs or by the knowledge and expertise of their faculty members. Taylor and Pell (1993) state that ?? vocational evaluation education has not followed a consistent or orderly process in the development of educational standards which parallel professional developments? (p. 185). Standards in vocational evaluation education have been lacking (Thomas & Sigmon, 1989). This lack 39 of standards poses a threat to the profession as ?? the lack of available coursework emphasized by CCWAVES can undermine the certification process and compromise the profession? (Shumate et al., 2004, p. 34). The importance of keeping vocational evaluation curriculum up to date with current competencies cannot be underestimated. The field as a whole benefits from relating curriculum to pressing issues for the profession. It is imperative to update curriculums to include expanding service markets and settings (Thomas & Sigmon, 1989). Vocational evaluation curriculum needs to be constantly modified to reflect specific competencies as they relate to specific job areas. Taylor, Bordieri, and Lee (1993) state ?Educators should differentiate their curriculum based on the individual service populations and should prepare students to approach their future careers with the understanding of different roles and functions of vocational evaluation? (p. 153). A survey was conducted by Taylor, Pell, and Chan (1998) to determine the capacity of rehabilitation educators to teach specialized courses in vocational evaluation. The subject matter of the survey included course work, curricular content, faculty expertise, and clinical learning experiences related to vocational evaluation. A four-point scale was used. The curricular content found to be most addressed by rehabilitation programs, in fact 90 to 100% of them, included the following: occupational information, job analysis, job development and placement, medical/psychosocial aspects of disability, standardized testing, assessment, functional aspects of disability, job modification and accommodation, and vocational interviewing. Topics addressed by less than half of the programs included the following: ergonomic principles and practices, assessment of learning, research in vocational evaluation, vocational evaluation credentialing, expert 40 witness testimony, and vocational evaluation program design and development (Taylor et al., 1998). An informal comparison was completed between the CCWAVES Knowledge and Performance Areas and Council on Rehabilitation Education (CORE) required course content. The CCWAVES and CORE requirements overlapped considerably. However, certain CCWAVES knowledge and performance areas were not adequately covered by programs. One area includes assessment of learning which was covered by less than half of the programs (Taylor et al., 1998). In terms of faculty expertise, the survey found that each rehabilitation program averaged 1.2 faculty members with master?s-level preparation in vocational evaluation and 1.1 faculty members with direct job experience in vocational evaluation. Over half of the programs reported that none of the faculty members held the Certified Vocational Evaluator (CVE) credential. Over twenty-five percent of the programs reported one faculty member with a CVE, and the rest had two to three faculty members with a CVE. The average rehabilitation program has approximately 3.7 faculty per this survey. Therefore, the amount of vocational evaluation expertise was found to be adequate amongst the panel of experts associated with this study. However, the amount of faculty members holding the CVE certification was found to be low (Taylor et al., 1998). Summary Vocational evaluation is a unique profession which uses work as its main focal point of assessment (Pruitt, 1972). Today, vocational evaluators are concerned about support from the public and from legislation as the field of vocational evaluation is facing criticism. Vocational evaluation lacks a perception of professionalism among some 41 persons and it is sometimes equated with simple pencil or dexterity testing. In addition, rehabilitation counselors are not always satisfied with the vocational evaluation reports they receive (Taylor & Bordieri, 1993). The field itself is in a process of evolution (Shumate et al., 2004). Current exploration of vocational evaluation competencies is important for the field. Enhancement and increased credibility, which are aided by established competencies, are two important goals for the field of vocational evaluation (Hamilton, 2003). Establishing competencies is also necessary for ensuring effective teaching at the university level. Vocational evaluation educational standards are lacking resulting in some programs having curriculum which is outdated (Thomas & Sigmon, 1989). Following is a discussion of the results of the study and recommendations for vocational evaluation programs to improve emphasis of competencies in their curricula. 42 III. A PRELIMINARY STUDY OF FACULTY AND GRADUATES PERCEPTIONS OF VOCATIONAL EVALUATION COMPETENCIES IN REGARDS TO EMPHASIS IN THE CURRICULUM Vocational evaluation has experienced times of greater and lesser acceptance. As a relatively young field, it is still in a process of evolution (Shumate, Hamilton, & Fried, 2004). Competency studies offer greater credibility and enhancement to the field (Hamilton, 2003). In the past these studies have enabled formation of the certification exam (Zwyghuizen, 1980). The most recent study completed by Newman and Waechter (1997) entitled Commissioned Assessment of Competencies Prepared for the Commission on Certification of Work Adjustment and Vocational Evaluation Specialists specifically studied competencies to update the current certification exam. In addition to updating the certification exam, competency studies are needed to update vocational evaluation curriculums (Taylor & Pell, 1993). Vocational evaluation educational standards are lacking resulting in some programs having curriculums which are outdated (Thomas & Sigmon, 1989). History and Becoming a Profession Vocational evaluation is a unique discipline which uses work as its main focal point of assessment (Pruitt, 1972). Vocational evaluation is an essential part of the 43 rehabilitation process (Hamilton, 2003). As a professional discipline within the fields of rehabilitation and psychology, vocational evaluation is relatively young. Vocational evaluation formed during the rehabilitation facility movement of the 1950s (Pruitt, 1977). The Vocational Rehabilitation Act of 1954 provided massive amounts of funding for state rehabilitation programs (Rubin & Roessler, 2001). Vocational evaluation saw a rapid expansion as well as it progressed from being relatively non-existent to an essential service in the rehabilitation process (Shumate, Hamilton, & Fried, 2004). The 1965 amendments to the Vocational Rehabilitation Act expanded the scope of services to include broader populations such as persons with substance abuse and behavioral disorders. Extended evaluation services were authorized under this amendment. Applicant eligibility now was mandated from 6 to 18 months allowing for greater services to be given to the individual (Rubin & Roessler, 2001). As a result of the 1965 amendment, vocational evaluation developed into an essential diagnostic tool used by rehabilitation counselors (Hamilton, 2003). The disability consumer movement of the 1970s brought about focus on the successful integration of individuals with severe disabilities into typical work environments (Rubin & Roessler, 2001). Vocational evaluation was seen increasingly as an essential service to help with this integration of persons with disabilities. The Rehabilitation Act of 1973 and its subsequent amendments mandated the service of individuals with severe disabilities, consumer involvement, and program evaluation (Shumate et al., 2004). The Education for All Handicapped Children Act of 1975 (Public Law 94-142) mandated that all students with disabilities be provided with vocational education programs. In addition to these programs, every student now had educational 44 rights in terms of appropriate placement in educational programs (Rubin & Roessler, 2001). Vocational evaluation was identified as one type of service that could aid in the appropriate placement of students (Hamilton, 2003). In 1984, the Carl D. Perkin?s Act mandated that all students enrolled in vocational education programs be provided with vocational evaluation services (Taylor & Pell, 1993). The American with Disabilities Act of 1990 (Public Law 101-336) was passed and became a progressive law. It intended to eliminate environmental and societal barriers to successful employment and to increase the participation of persons with disabilities in the workforce (Rubin & Roessler, 2001). The act increased the need for vocational evaluation services due to increasing demands for identification of potential vocations (Hamilton, 2003). President Clinton signed into law two initiatives entitled the Workforce Investment Act of 1998 and the Ticket to Work and Work Incentives Improvement Act (1999). The first act called for a one-stop service provision for vocational rehabilitation services (Rubin & Roessler, 2001). Vocational evaluation was included as one of the services (Shumate et al., 2004). Under the second Act, eligible social security beneficiaries with disabilities were now able to receive free tickets. These tickets could be used to obtain vocational rehabilitation services, including vocational evaluation services (Rubin & Roessler, 2001). Facing Criticism Despite vocational evaluation?s importance within the vocational rehabilitation process, it is currently experiencing times of criticism as some state departments are choosing to reduce their use of vocational evaluators. On a national level, the purchase of 45 vocational evaluation services are on the decline at the state level due to cost containment efforts (Shumate et al., 2004). In addition, at the meetings of the 30 th Institute on Rehabilitation Issues, concerns were raised about equating the field of vocational evaluation simply with pencil and paper or dexterity testing. While some persons may have a narrow scope of vocational evaluation, others may also find the services as needing improvement. Taylor and Bordieri (1993) found that rehabilitation counselors are not always satisfied with the reports that they receive from vocational evaluators as some important factors were not included in vocational evaluation reports which cover work personality, physical and cognitive aspects, specific job selection, and formal education and training. In addition, the professional image and status of vocational evaluation have been questioned by some. A survey of nearly 100 college students conducted by Saxon, Spitznagel, and Kennison (1999) found that the field of vocational evaluation lacks professional status among other related fields. Despite the small number of persons surveyed for this study, the questioned status of vocational evaluators remains valid amongst current practitioners. Vocational evaluators in several states have expressed concern about the impact public image has had on the profession (Fried, Harrand, Dowd, & Schuster, 1994). Competency Studies The current exploration of vocational evaluation competencies is important for the expansion of the profession, updating standards, and updating the certification exam (Shumate et al., 2004). Looking at the history of competency exploration, one can see its 46 importance. The initial establishment of competencies gave credibility to the field of vocational evaluation and allowed for later development of the certification exam (Zwyghuizen, 1980). Previous studies have ranged from national role and function studies to competency studies (Boyer-Stephens, Waechter, & Newman, 1999; Coffey, 1978; Leahy & Wright, 1988; National Forum, 2004; Taylor, Bordieri & Lee, 1993). The most recent study is by Hamilton (2003) and sponsored by the Commission on Certification of Work Adjustment and Vocational Evaluation Specialists (CCWAVES). The first documented competency study was completed by Pruitt (1972) and used task analysis to identify functions of vocational evaluators. Coffey?s (1978) study gathered an initial list of 2,500 competencies later synthesized to 175 primary vocational evaluator competencies. This competency study was instrumental in the formation of the first vocational evaluator certification exam (Zwyghuizen, 1980). Sink and Porter (1978) studied competencies as they related to the vocational evaluator and rehabilitation counselor. They found significant differences between the two professions, suggesting the need for separate vocational evaluation curriculum and training. Competencies in terms of importance to effective service delivery were rated by 270 vocational evaluation practitioners in 1986. The most important competencies were determined to be job placement, professional and community involvement, group and behavioral techniques, and consultation (Leahy & Wright, 1988). Taylor, Bordieri, and Lee (1993) surveyed 526 vocational evaluators concerning job tasks and job settings. Overall job tasks between vocational evaluators in different job settings were found to be similar. In 1997, CCWAVES consulted with Research and Evaluation Associates Inc. to identify competency areas necessary for entry level 47 evaluators to successfully perform and fulfill job responsibilities. General agreement existed among a panel of vocational evaluation experts that basic skills were needed in the following areas: behavioral observation, report writing, interviewing, test statistics, counseling, knowledge of legislation, and good communication skills. Qualitative comment review revealed concerns of experts in the following areas regarding entry level vocational evaluators acquisition of: report writing, behavioral observation, legal/ethical issues, cultural diversity, and computer literacy. However, the researchers noted that the number of experts surveyed, 23 in all, may limit the generalizability of the study results (Newman & Waechter, 1997). Hamilton?s (2003) most recent study examined the role and function of certified vocational evaluators in the United States. Hamilton surveyed 800 certified vocational evaluators to find certain primary knowledge areas. She found these areas grouped into the following categories: (a) Foundations of Vocational Evaluation; (b) Standardized Assessment; (c) Occupational Information; (d) Implications of Disability; (e) Communication; and (f) Professional Networking and Coordination. Curriculum and Competencies The first degree program in vocational evaluation developed at Stout State University in 1966 (Hamilton, 2003). Other university programs followed while even more added vocational evaluation concentrations to their rehabilitation counseling curricula (Shumate et al., 2004). The development of university programs in vocational evaluation are a direct result of legislation and the increased need for vocational evaluators. Examples of laws which increased the need for vocational evaluation services 48 are the Vocational Rehabilitation Acts of 1954 and 1965.With this increased need, came a sudden expansion in vocational evaluation courses (Shumate et al., 2004). As a result, most of the graduate programs in vocational evaluation developed without the aid of uniform standards or competencies. The available programs at this time were influenced by regional needs or by the knowledge and expertise of their faculty members (Thomas & Sigmon, 1989). Curricular standards suffered as a result of this expansion. Taylor and Pell (1993) state that ??vocational evaluation education has not followed a consistent or orderly process in the development of educational standards which parallel professional developments? (p. 185). There are currently 12 universities in the United States offering degrees in vocational evaluation (CCWAVES, 2005). Many more universities offer courses in vocational evaluation to their rehabilitation counseling students. Vocational evaluation is taught in the universities at the master?s degree level. Some universities offer degrees in vocational evaluation while others offer specialties or coursework in vocational evaluation. A survey conducted by Taylor, Pell, and Chan (1998) found that of the 39 rehabilitation counseling master?s programs surveyed, only five offered an entire degree in vocational evaluation. Thirteen of the programs surveyed offered only one course in vocational evaluation. Seventeen of the programs reported having no specific courses in vocational evaluation, but rather having vocational evaluation content which is spread across the main curriculum (Taylor et al., 1998). Taylor and Pell (1993) conducted a study of ten university programs offering degrees in rehabilitation counseling with specialties in vocational evaluation and found that some programs offered a varied 49 vocational evaluation curricula while some only offered one to two generic assessment courses. Research indicates that competencies are best met through university or college level preparation. Masters trained vocational evaluators have significantly higher attainment of professional competencies than their counterparts who have rehabilitation related or unrelated degrees (Shapson, Wright, & Leahy, 1987). Shapson et al. (1987) found that higher levels of satisfaction, higher numbers of certified and higher perceived professional competencies were found in master?s level graduates. Despite this fact, the majority of research tends to indicate that most evaluators do not receive their training at the university level. Newman et al. (1998) found that the majority of competencies are obtained through on-the-job or short-term workshops. Taylor, Bordieri, Crimando, and Janikowski (1993) found that of the 188 vocational evaluators who participated in the study, few of them had formal training in vocational evaluation. Only five of the participants reported earning a degree in vocational evaluation (Taylor et al., 1993). For the second study, Taylor, Bordieri, and Lee (1993) surveyed 526 vocational evaluation practitioners and found that only 8.5% earned degrees with an emphasis in vocational evaluation. In addition, these evaluators cited on-the-job training or short-term workshops as their most widely used source of training (Taylor, Bordieri, & Lee, 1993). Leahy and Wright (1988) found that of the 270 viable respondents, only 14.8% of the total sample received university training in vocational evaluation. Of the total respondents, 160 of them had a major in the field outside of rehabilitation (Leahy & Wright, 1988). Competency studies are needed to enhance the field of vocational evaluation and to improve vocational evaluation curricula (Taylor & Pell, 1993). Hamilton (2003) states 50 ?There exists a critical need for a commitment to research that will facilitate continuity of educational preparation and standards to enhance the overall discipline of vocational evaluation? (p. 9). Minimal research has examined the efficacy of curriculum to prepare vocational evaluation practitioners (Taylor & Pell, 1993). In addition, there appears to be no studies of practitioners? perceptions that relate to vocational evaluation competencies and university curricula. Taylor and Pell (1993) found that revised competencies are not being reflected in vocational evaluation curriculum. . Thomas and Sigmon (1989) completed a competency study that reviewed the curriculums of 12 graduate specialty vocational evaluation programs. They found a lack of uniform competencies within these graduate specialty vocational evaluation programs. The problems in curriculums were attributed to decreased federal funding for research, university training, and in-service training opportunities in vocational evaluation. Overall vocational evaluation curriculums were found to be lacking. Areas of insufficiency were not identified by the authors; rather, a new curriculum was proposed focusing on the dynamic of the following two areas: knowledge of instruments and clinical competencies; and skill with instruments and clinical competencies. Shumate et al. (2004), surmise that ?while CCWAVES requires graduate level coursework in specified knowledge and performance areas, most graduate programs offering curricula in vocational evaluation lack many key knowledge and content areas? (Shumate et al., 2004, p. 34). These key knowledge and content areas have not been identified due to a lack of research. Thomas and Sigmon (1989) advocate for increasing vocational evaluation curriculum standards. The importance of updating 51 curricula to match expanding service markets and settings cannot be understated (Thomas & Sigmon, 1989). The importance of keeping vocational evaluation curricula up to date with current competencies is crucial to the profession. The field as a whole benefits from relating curricula to pressing issues for the profession. It is imperative to update curricula to include expanding service markets and settings (Thomas & Sigmon, 1989). Vocational evaluation curricula needs to be constantly modified to reflect specific competencies as they relate to specific job areas. Taylor, Bordieri, and Lee (1993) stated ?educators should differentiate their curriculum based on the individual service populations and should prepare students to approach their future careers with the understanding of different roles and functions of vocational evaluation? (p. 153). Statement of the Research Problem Although current research studies have identified vocational evaluation competencies and their importance to the profession, there remains an absence of research related to vocational evaluation competencies in educational curriculum. Critical questions regarding the actual teaching of essential competencies in the curriculum exist (Taylor & Pell, 1993). The field of vocational evaluation is experiencing criticism and a lack of educational standards (Thomas & Sigmon, 1989). Research is needed to address the emphasis of vocational evaluation competencies in curricula. 52 Purpose of the Study The purpose of this study was to examine the amount of emphasis vocational evaluation competencies are given in different types of curricula. The broad question to be answered was whether or not competencies are being taught at the university level. Specific questions concerned the following: faculty members? ratings versus vocational evaluation graduates? ratings of emphasis of competencies in curriculum and faculty members? ratings versus non-vocational evaluation graduates? ratings of emphasis of competencies in curriculum. Research Questions For this study, the following research questions were developed: 1. Is there a difference between the perceptions of vocational evaluation faculty and graduates on their ratings of vocational evaluation competencies in the educational curriculum? a. Are there differences among the perceptions of vocational evaluation faculty and vocational evaluation graduates, vocational rehabilitation graduates with a specialty in vocational evaluation, vocational rehabilitation graduates with no specialty in vocational evaluation, and unspecified (other) graduates? 2. Are there significant differences between the perceptions of vocational evaluation faculty and graduates on their ratings of vocational evaluation competencies in the educational curriculum? 53 a. Is there a difference between the perceptions of vocational evaluation faculty and vocational evaluation graduates on their ratings of vocational evaluation competencies in the educational curriculum? b. Is there a difference between the perceptions of vocational evaluation faculty and vocational rehabilitation graduates (with a specialty in vocational evaluation) on their ratings of vocational evaluation competencies in the educational curriculum? c. Is there a difference between the perceptions of vocational evaluation faculty and vocational rehabilitation graduates (with no specialty in vocational evaluation) on their ratings of vocational evaluation competencies in the educational curriculum? d. Is there a difference between the perceptions of vocational evaluation faculty and unspecified (other) graduates on their ratings of vocational evaluation competencies in the educational curriculum? Methodology Participants Participants were selected from two sources. First, participants were contacted via a professional organization entitled Vocational Evaluation and Career Assessment Professionals (VECAP). They were emailed after permission was granted from the president of VECAP. Secondly, participants were found as they were identified as faculty teaching courses in vocational evaluation via the Commission on Certification of Work Adjustment and Vocational Evaluation Specialists (CCWAVES) website. The final 54 participants were selected only if they were faculty involved in teaching vocational evaluation courses. No compensation was given to participants in the study. Demographic data regarding respondents is noted in Table 1. The major demographic characteristics in terms of type of respondent were 12 faculty, 10 graduates in vocational evaluation, 10 graduates in rehabilitation counseling with a specialization in vocational evaluation, 11 graduates in rehabilitation counseling with no specialization, and 12 graduates in other unspecified fields. The demographics in terms of vocational evaluation faculty for sex were seven males and five females; and in terms of race were 12 Caucasian. Regarding type of job position held for vocational evaluation faculty, there were 12 faculty members. In terms of years of service in the field of vocational evaluation for vocational evaluation faculty, there were two respondents with five to ten years of service, two respondents with ten to twenty years of experience, and finally seven respondents with over twenty years of experience. One of the respondents left this category blank, resulting in a total of 11 responses. 55 Table 1 Demographic Characteristics of Faculty and Graduates by Frequency and Percent (N = 55) Faculty Graduates Summary of Sample Characteristics f Percent f Percent Description Faculty Members 12 100 Graduates in Vocational Evaluation 10 23.3 Graduates in Rehab Counseling, specialty VE 10 23.3 Graduates in Rehab Counseling 11 25.6 Graduates in Other Fields 12 27.9 Current Work Position Vocational Evaluator 25 56.8 Rehab Counselor 3 6.8 Faculty Member 12 100 5 11.4 Other 11 25.0 Years of Service in Vocational Evaluation 0-3 years 7 16.3 3-5 years 4 9.3 5-10 years 2 18.2 9 20.9 10-20 years 2 18.2 11 25.6 (table continues) 56 Table 1 (continued) Faculty Graduates Summary of Sample Characteristics f Percent f Percent 20 + years 7 63.6 12 27.9 Gender Male 7 58.3 11 25.6 Females 5 41.7 32 74.4 Level of Education Bachelor?s Degree 8 18.6 M.S./M.Ed. 1 8.3 30 69.8 PhD 11 9.2 5 11.6 Degree Type Rehab Counseling Degree 8 47.1 21 34.4 Vocational Evaluation Degree 3 17.6 13 21.3 Education Degree 4 23.5 11 18.0 Other Degree 2 11.8 16 26.2 Race African American 5 11.6 Caucasian 12 100 37 86.0 Other 1 2.3 57 The demographics in terms of graduates for sex were 11 males and 32 females; and in terms of race were 5 African American, 37 Caucasian, and one other. Regarding type of job position held for graduates, there were 25 vocational evaluators, 3 rehabilitation counselors, 5 faculty members, and 11 other. Respondents were allowed to give more than one employment position resulting in a larger number of positions than total number of respondents. The population in terms of years of service in the field of vocational evaluation for graduates were 7 respondents with zero to three years of service, 4 respondents with three to five years of service, 9 respondents with five to ten years of service, 11 respondents with ten to twenty years of experience, and finally 12 respondents with over twenty years of experience. Participants were also asked to indicate their level of education and field of study. In terms of level of education for faculty, there was one respondent with a master?s degree, and 11 respondents with a doctorate degree. In terms of degree type for faculty, there were 8 respondents with a rehabilitation counseling degree, 3 respondents with a vocational evaluation degree, 4 with an education degree, and 2 with a degree other than the aforementioned degrees. In terms of level of education for graduates, there were 8 respondents with a bachelor?s degree, 30 respondents with a master?s degree, and 5 respondents with a doctorate degree. In terms of degree type for graduates, there were 21 respondents with a rehabilitation counseling degree, 13 respondents with a vocational evaluation degree, 11 with an education degree, and 16 with a degree other than the aforementioned degrees. Respondents were allowed to give more than one degree type resulting in a larger number of positions than total number of respondents. 58 Procedures Survey development. In the development of the survey form, the 1997 study entitled Commissioned Assessment of Competencies Prepared for the Commission on Certification of Work Adjustment and Vocational Evaluation Specialists was used. This study was completed by Dr. Isadore Newman and Dr. Donna Waechter of Evaluation and Research Associates, Inc. This study provided the competencies for the survey. The first section of the survey pertained to the demographics of participants. Demographic information questions concerned job title, educational background, program type, and employment background. The second section focused on identified competencies. These competencies were grouped into five sections as follows: The Professional, Tools and Techniques, The Vocational Evaluation Process, Characteristics of Consumers, The World of Work. A total of 51 competencies were listed as Likert scale items. To indicate the level of emphasis each competency was given in the curriculum, participants responded by selecting one of the following responses: 5 = greatly emphasized throughout the program 4 = major part of the program 3 = adequately included in the program 2 = minimally included in the program 1 = not part of the program Survey distribution. The subject population included vocational evaluators who are members of Vocational Evaluation and Career Assessment Professionals organization (VECAP) and faculty of vocational evaluation programs. The organization entitled Vocational Evaluation and Work Adjustment Association (VEWAA) was contacted to 59 provide contact information for participants in this study. VEWAA was unable to provide the contact information; therefore they were not utilized for this study. The VECAP members comprise over 170 members nationwide. The number of members of VECAP who hold certification in vocational evaluation are less than 25 percent. The faculty members of vocational evaluation programs average 1 to 2 members per program, totaling 17 faculty members. An introduction letter was emailed to each VECAP and faculty member with a link to the survey website. The faculty members? contact information was retrieved from the CCWAVES website which lists 12 universities offering degrees in vocational evaluation. The survey was internet-based and was formulated using Microsoft FrontPage (a copy of instrument is in Appendix B). In order to protect the privacy of responses, no attempt was made to identify individual participants. Responses were sent to an anonymous, secure server which blocked the return e-mail addresses and protected identifying information. The introduction letter with the link to the survey was sent a second time due to an inadequate number of responses (39 initial responses). Once the data collection was complete, the data were exported to the researcher?s computer, saved as a Statistical Package for Social Sciences (SPSS) file, and then analyzed. Response Rate Fifty-five persons responded to the survey (32 percent response rate). Dillman, Tortora, Conradt, and Bowker (1998) studied the response rates of differently designed web surveys and found that a typical response rate for a plainly designed survey was 41 percent while a fancy designed web survey received a 36 percent response rate. This survey best meets their definition of a plainly designed web survey. Exactly three survey 60 responses were eliminated because of inaccuracy of responses due to failed attempts to access the survey. There were fifty-eight initial responses with three of them thrown out, making a total of fifty-five responses. The response rate for specific questions varied due to either the lack of information regarding the question or the inability for the participant to answer the question for unknown reasons. Nine respondents left one question blank in the entire survey. Each of these questions was different. In other words, the nine questions left blank were not the same. Survey Questions For ease in administration, the survey was divided into two parts (see Appendix B). The first part consisted of demographic information of participants. Questions were asked regarding the type of job position held, type of degree obtained, and years of service. The survey also inquired about education level, work status, certification, gender, and race. The second section consisted of the competencies to be rated. The competencies were divided into five sections which is consistent with the Newman and Waechter (1997) study entitled Commissioned Assessment of Competencies Prepared for the Commission on Certification of Work Adjustment and Vocational Evaluation Specialists. These five sections include (a) the professional, (b) tools and techniques, (c) the vocational evaluation process, (d) characteristics of consumers, (e) the world of work. The competencies were listed. They were not defined for the participants. Some of the competencies were eliminated from the Newman and Waechter (1997) due to repetitions of competencies within the five groups. The competencies were rated in terms of the degree each was emphasized in the participant?s curriculum. The 61 participants were asked to base their ratings on how much emphasis is placed on or has been placed on each specific competency. If the participants were faculty members, they were asked to rate the university at which they were currently employed. It was asked that ratings be based on personal experience. It was explained to each participant that they were not rating the importance of each competency. The participants used a 5 point Likert-type scale to rate 51 competencies. Each group of competencies was headed by a description of this scale. The description read as follows: (1) not part of program, (2) minimally included in program, (3) adequately included in program, (4) major part of program, (5) greatly emphasized throughout the program. A score of one signified that the competency was not part of the program. A score of two meant that the competency was minimally included in the program. A score of three signified that the competency was included in the program. A score of four meant that the competency was a major part of the program. A score of five meant that the competency was greatly emphasized throughout the program. Analysis of Data The computer software program Statistical Package for the Social Sciences (SPSS) (13.0) was used to analyze the data. Analysis of the data involved comparing mean scores, examining the distribution of scores, and arranging the competencies in rank order. Data analysis began with a tabulation of the demographic information collected in section one of the survey to provide a detailed description of the respondents. Frequency distributions and percentages were then calculated for each item. Likert scale ratings were used to calculate descriptive statistics for each of the 51 competencies and 62 five domains. Statistical tests conducted included a one-way univariate analysis of variance within subjects (ANOVA). The data results from the survey were recorded using a Microsoft Excel spreadsheet. Vocational evaluation faculty rated 92% of competencies as being at least adequately included in the program. Vocational evaluation graduates rated 94% of competencies as being at least adequately included in the program. Rehabilitation counseling graduates with a specialty in vocational evaluation rated 82% of competencies as being at least adequately included in the program. Rehabilitation counseling graduates with no specialty rated 63% of competencies as being at least adequately included in the program. Other graduates rated 57% of competencies as being at least adequately included in the program. Data in Tables 2, 3, 4, 5, and 6 were presented in terms of five different groups. These groups included faculty members, vocational evaluation graduates, rehabilitation counseling graduates with a specialty in vocational evaluation, rehabilitation counseling graduates, and other graduates. The other graduates group was not specified in terms of degree type. The data presented in Tables 2, 3, 4, 5, and 6 concern the five top ranked competencies and means. Many of the competencies received the same mean score; therefore, there was a tie in the ranking. This tie is represented by the letter ?T? before the number. Table 2 concerns the top ranked competencies by faculty members. The top five ranked competencies in terms of emphasis in the curriculum included ethical practices, medical and psychosocial aspects of disability, individual planning, vocational aspects of disability, vocational interviewing skills, evaluator?s interpersonal skills, knowledge of 63 appropriate use of tests, and observation of behavior. These competencies are listed in ranked order. Table 2 Top 5 Ranked Competencies and Means/Standard Deviations of Vocational Evaluation Faculty Competencies Mean S.D. 1. Ethical Practices 4.58 .669 T2. Medical and Psychosocial Aspects of Disability 4.33 .778 T2. Individual Planning 4.33 .651 T3. Vocational Aspects of Disability 4.33 .754 T3. Vocational Interviewing Skills 4.25 .866 T3. Evaluator?s Interpersonal Skills 4.25 .754 4. Knowledge of Appropriate Use of Tests 4.18 .603 5. Observation of Behavior 4.17 1.030 Note. T means that the items listed were tied, such as T2 which means that there were 2 items ranked second Table 3 concerns the top ranked competencies by vocational evaluation graduates. The top five ranked competencies in terms of emphasis in the curriculum included the following: Medical and Psychosocial Aspects of Disability, Psychometric Principles, Knowledge of Test Administration, Observation of Behavior, Vocational Aspects of Disability, Scoring and Interpretation of Tests, Knowledge of Appropriate Use of Tests, 64 Knowledge of Process of Vocational Evaluation, Functional Aspects of Disability, Knowledge of Appropriate Use of Work in Assessment Process, Interpretation of Observation and Performance Data, Vocational Interviewing Skills, Individual Planning, Ethical Practices, Knowledge of Worker Traits, Knowledge of Physical Capabilities, Knowledge of World of Work, Work Samples and Vocational Screening Systems, Job Analysis. These competencies are listed in rank order Table 3 Top 5 Ranked Competencies and Means/Standard Deviations of Vocational Evaluation Graduates Competencies Mean S.D. T1 Medical and Psychosocial Aspects of Disability 4.70 .483 T1 Psychometric Principles 4.70 .483 T1 Knowledge of Test Administration 4.70 .483 T1 Observation of Behavior 4.70 .483 T2 Vocational Aspects of Disability 4.60 .516 T2 Scoring and Interpretation of Tests 4.60 .516 T2 Knowledge of Appropriate Use of Tests 4.60 .966 T2 Knowledge of Process of Vocational Evaluation 4.60 .516 T3 Functional Aspects of Disability 4.50 .527 T3 Knowledge of Appropriate Use of Work in Assessment Process 4.50 .707 (table continues) 65 Table 3 (continued) Competencies Mean S.D. T3 Interpretation of Observation and Performance Data 4.50 .707 T3 Vocational Interviewing Skills 4.50 .527 T4 Individual Planning 4.40 .843 T4 Ethical Practices 4.40 .843 T5 Knowledge of Worker Traits 4.30 .675 T5 Knowledge of Physical Capabilities 4.30 .483 T5 Knowledge of World of Work 4.30 .675 T5 Work Samples and Vocational Screening Systems 4.30 .823 T5 Job Analysis 4.30 .949 Note. T means that the items listed were tied, such as T2 which means that there were 2 items ranked second Table 4 concerns the top ranked competencies by rehabilitation counseling graduates with a vocational evaluation specialty. The top five ranked competencies in terms of emphasis in the curriculum included the following: Report Writing Skills, Knowledge of Process of Vocational Evaluation, Work Samples and Vocational Screening Systems, Knowledge of Test Administration, Observation of Behavior, Vocational Aspects of Disability, Medical and Psychosocial Aspects of Disability, Vocational Interviewing Skills, Knowledge of Appropriate Use of Tests, Job Analysis, and Analysis of Observation and Performance Data. These competencies are listed in rank order. 66 Table 4 Top 5 Ranked Competencies and Means/Standard Deviations of Rehabilitation Graduates with Vocational Evaluation Specialty Competencies Mean S.D. T1. Report Writing Skills 4.50 .707 T1 Knowledge of Process of Vocational Evaluation 4.50 .527 T2 Work Samples and Vocational Screening Systems 4.40 .843 T2 Knowledge of Test Administration 4.40 .699 T2 Observation of Behavior 4.40 .699 T3 Vocational Aspects of Disability 4.30 .823 T3 Medical and Psychosocial Aspects of Disability 4.30 .823 T3. Vocational Interviewing Skills 4.30 .823 T4 Knowledge of Appropriate Use of Tests 4.20 .789 T4 Job Analysis 4.20 .919 5. Analysis of Observation and Performance Data 4.11 1.269 Note. T means that the items listed were tied, such as T2 which means that there were 2 items ranked second Table 5 concerns the top ranked competencies by rehabilitation counseling graduates. The top five ranked competencies in terms of emphasis in the curriculum included the following: Knowledge of Physical Capabilities, Medical and Psychological Aspects of Disability, Functional Aspects of Individuals with Special Needs, Vocational Aspects of Disability, ADA and Reasonable Accommodations in the Workplace Culture, Social Interaction Skills. The top four ranked competencies by rehabilitation counseling 67 graduates are within the fourth group of competencies entitled the characteristics of consumers. Table 5 Top 5 Ranked Competencies and Means/Standard Deviations of Rehabilitation Counseling Graduates ? No Specialty Competencies Mean S.D. 1. Knowledge of Physical Capabilities 3.91 .944 2. Medical and Psychological Aspects of Disability 3.82 .982 3. Functional Aspects of Individuals with Special Needs 3.73 1.009 4. Vocational Aspects of Disability 3.70 1.059 T5. ADA and Reasonable Accommodations in the Workplace Culture 3.55 1.036 T5. Social Interaction Skills 3.55 .934 Note. T means that the items listed were tied, such as T2 which means that there were 2 items ranked second Table 6 concerns the top ranked competencies by other graduates. The top five ranked competencies in terms of emphasis in the curriculum included the following: Observation of Behavior, Medical and Psychological Aspects of Disability, Ethical Practices, Scoring and Interpretation of Tests, Report Writing Skills, Functional Aspects 68 of Individuals with Special Needs, Learning Styles, Knowledge of Test Administration. These competencies are listed in rank order. Table 6 Top and Lowest 5 Ranked Competencies and Means/Standard Deviations of Other Graduates Mean S.D. Top 1. Observation of Behavior 4.08 .793 2. Medical and Psychological Aspects of Disability 3.92 1.084 T3. Ethical Practices 3.83 1.115 T3. Scoring and Interpretation of Tests 3.83 1.403 T3. Report Writing Skills 3.83 1.115 T4. Functional Aspects of Individuals with Special Needs 3.75 1.138 T4. Learning Styles 3.75 .866 T5. Knowledge of Test Administration 3.67 1.435 T5. Social Interaction Skills 3.67 1.073 T5. Knowledge of Physical Capabilities 3.67 1.155 T5. Knowledge of Appropriate Use of Tests 3.67 1.435 T means that the items listed were tied such as T2 which means that there were 2 items ranked second. The competency entitled ?knowledge of process of vocational evaluation? was rated second by vocational evaluation graduates with a mean emphasis of 4.60. The rehabilitation counseling graduates with a specialty in vocational evaluation rated this 69 competency first with a mean emphasis of 4.50. This competency was rated 11 th amongst faculty members with a mean emphasis of 3.67. The competency entitled ?Work Samples and Vocational Screening Systems? was rated fifth by vocational evaluation graduates with a mean of 4.30. The rehabilitation counseling graduates with the vocational evaluation specialty rated this competency as second with a mean emphasis of 4.40. The faculty members rated this competency as 18 th with a mean competency of 3.17. The competency entitled ?job analysis? was rated fifth by vocational evaluators with a mean of 4.30 and fourth by rehabilitation counseling graduates with the vocational evaluation specialty with a mean emphasis of 4.20. The faculty members rated this competency as 11 th with a mean of 3.67. The data presented in Table 7 concerns the differences between means of competencies rated by faculty members in comparison to the graduates. The data was presented in terms of four different groups. These groups included vocational evaluation faculty members, vocational evaluation graduates (VE), rehabilitation counseling graduates (RC), and other graduates (Other). The vocational evaluation graduates included graduates with vocational evaluation degrees and graduates with rehabilitation counseling degrees with a specialization in vocational evaluation. These groups were combined due to similarity in ratings and for convenience in reporting the scores. The other graduate groups were not specified in terms of degree type. 70 Table 7 Differences between Means of Faculty and Graduates ? Top 10 Ranked Competencies Top 10 Ranked Competencies by Faculty Faculty Means VE/RC Specialty graduates RC graduates Other graduates Overall 4.02 .07 .69 .60 Ethical Practices 4.58 .38 1.13 .75 Medical and Psychosocial Aspects of Disability 4.33 .17 .51 .41 Individual Planning 4.33 .08 1.13 1.08 Vocational Aspects of Disability 4.25 .20 .55 .92 Vocational Interviewing Skills 4.25 .15 .98 1.25 Evaluator?s Interpersonal Skills 4.25 .45 1.34 .75 Knowledge of Appropriate Use of Tests 4.18 .22 1.00 .51 Observation of Behavior 4.17 .38 .97 .09 Knowledge of Community Resources 4.08 .33 1.17 .83 Interpretation of Observation and Performance Data 4.00 .25 .64 .50 Knowledge of Physical Capabilities 4.00 .15 .08 .33 Functional Aspects of Disability 4.00 .25 .27 .25 Psychometric Principles 4.00 .35 .82 .50 Knowledge of Appropriate Use of Work in Assessment Process 3.92 .38 .83 .92 Awareness/Inclusion of Cultural Diversity 3.92 .98 .47 .84 Social Interaction Skills 3.83 .18 .28 .16 Employability Factors 3.83 .08 .56 .83 Scoring and Interpretation of Tests 3.83 .52 .92 .00 Knowledge of Test Administration 3.83 .72 .56 .16 Case Management 3.83 .53 .63 1.33 ADA and Reasonable Accommodations in the Workplace Culture 3.83 .23 .28 1.25 Report Writing Skills 3.83 .47 .83 .00 Analysis of Observation and Performance Data 3.75 .36 .48 .25 Knowledge of Worker Traits 3.75 .35 .30 .50 71 The faculty members top ten ranked means were reported in Table 7. There are more than ten competencies listed due to a tie in the rankings. These top ten ranked competencies were reported in terms of differences in means. The differences between the faculty competency means and the graduates competency means are reported. The smallest difference in means existed between the faculty and the vocational evaluation graduates group. The difference between faculty and vocational evaluation graduate means was overall .07. The differences between faculty and rehabilitation counseling graduates was .69. The differences between faculty and other graduates was .60. The faculty and vocational evaluation graduates group were the most similar on 17 of 24 competencies. The faculty and other graduates group were most similar on 5 of the 24 competencies. These competencies included observation of behavior, scoring and interpretation of tests, knowledge of test administration, report writing skills, and analysis of observation and performance data. The vocational evaluation graduate group rated all of the aforementioned competencies higher than faculty members did, thereby causing the greater difference in means. The faculty and rehabilitation counseling group were most similar on 2 of the 24 competencies. These two competencies included awareness/inclusion of cultural diversity and knowledge of worker traits. Vocational evaluation graduates rated the emphasis of awareness of cultural diversity as lower than the faculty members did. They rated the emphasis of knowledge of worker traits as higher than the faculty members did, thereby causing the difference. In Table 8, the results of a one-way univariate analysis of variance within subjects (ANOVA) are listed. Three competencies were found as overall significant. Vocational interviewing skills, job analysis, and task analysis are significant in terms of differences 72 between means of faculty and graduates. The significance of these competencies are .003, .000, and .004 respectively. Job analysis was found to be most significant with a .000 significance rating. Vocational interviewing skills and task analysis followed closely behind with a significance rating of .003 and .004 respectively. All other competencies within the 51 competencies on the survey were not found to be significant in terms of differences between means of faculty and graduates. The significance was set at .005 in order to decrease the amount of error associated with running ten tests. Table 8 Analysis of Variance for Differences in Means Between Groups (One Way ANOVA) Competency df F p Vocational Interviewing Skills 4 4.66 .003* Job Analysis 4 6.48 .000* Task Analysis 4 4.44 .004* *p = .005 In Table 9, a multiple comparisons analysis was used to determine exactly which groups differed the most. Two competencies were found as significant in terms of differences between faculty and the other graduates group. The two competencies are vocational interviewing skills with a significance of .019 and job analysis with a significance of .015. Task analysis was not found to be significant between faculty and 73 the other graduates. This competency was also not found to be significant between faculty and all other graduates. Table 9 Follow Up Multiple Comparisons Analysis Between Faculty and Graduate Means Competency Vocational Evaluation Graduates Rehab Counseling Specialty Graduates Rehab Counseling Graduates Other Unspecified Graduates Vocational Interviewing Skills .951 1.000 .098 .019* Job Analysis .472 .619 .403 .015* Task Analysis .209 .141 .952 .436 *p = .05 74 IV. CONCLUSION AND DISCUSSION The results of this survey reveal interesting information about the differences in emphasis of competencies across curricula. It is noted that the types of participants used limit the generalizability of the results. The demographic makeup of the participants is varied. The degrees obtained by the participants are varied as well. The participant groups surveyed included vocational evaluation faculty, vocational evaluation graduates, rehabilitation counseling graduates with a specialty in vocational evaluation, rehabilitation counseling graduates with no specialty, and other unspecified graduates. The overall means of ratings shows that competencies are perceived as most emphasized in the following groups respectively: vocational evaluation graduates, vocational evaluation faculty, rehabilitation counseling graduates with specialty in vocational evaluation, rehabilitation counseling graduates, and other unspecified graduates. For all groups, over 50 percent of the 51 competencies were found to be adequately included in the university curricula. Rehabilitation counseling graduates with a specialty in vocational evaluation differed the most with rehabilitation counseling graduates with no specialty. The first group rated 82% of the competencies as being at least adequately included in the university curricula while the second group rated 63% of the competencies as being at least adequately included. The vocational evaluation graduates ratings were most similar 75 to faculty with percentages of 94% and 92% respectively in terms of emphasis in curricula. In addition, the top ranked competencies of vocational evaluation and rehabilitation counseling graduates with a specialty in vocational evaluation closely follow the top ranked competencies of vocational evaluation faculty. Overall means of ratings of competencies are close between these three groups. These results support Sink and Porter?s (1978) conclusions that distinct differences exist between vocational evaluation and rehabilitation counseling. The responses of vocational evaluation graduates and faculty members were most similar. This result would be expected due to similarity of programs. The competencies most similar between the other graduates group and the faculty members included observation of behavior, scoring and interpretation of tests, knowledge of test administration, report writing skills, and analysis of observation and performance data. These competencies suggest that the other graduates group might consist of social science majors, most notably psychology. It is important to note that the qualitative results of the Newman and Waechter 1997 study entitled Commissioned Assessment of Competencies Prepared for the Commission on Certification of Work Adjustment and Vocational Evaluation Specialists found that experts believed entry level evaluators to need more skills in the following areas: effective report writing, behavioral observation, legal/ethical issues, cultural diversity, and computer literacy. The faculty and rehabilitation counseling group were most similar on the competency of awareness/inclusion of cultural diversity. Vocational evaluators rated the emphasis of awareness of cultural diversity as lower than the faculty members did. The discrepancy might be due to a difference in time. The faculty members were rating 76 current programs while some of the vocational evaluation graduates were rating much earlier programs. Awareness of cultural diversity has been found to be an important issue for the field of rehabilitation in general as well as vocational evaluation. The qualitative comments gathered from the Newman and Waechter (1997) study suggest that entry level evaluators may not be equipped with enough knowledge and training in cultural diversity. Some vocational evaluation programs might take mention of the fact that vocational evaluation graduates were not as similar as rehabilitation counseling graduates in terms of emphasis of awareness of culture diversity in the curriculum. Notable differences exist in the rankings of three competencies between faculty, vocational evaluation graduates, and rehabilitation counseling graduates with a specialty in vocational evaluation. These three competencies are (1) knowledge of the process of vocational evaluation, (2) work samples and vocational screening systems, and (3) job analysis. Vocational evaluation graduates and rehabilitation graduates with a specialty in vocational evaluation rated these three competencies as being a major part of the curriculum to being greatly emphasized throughout the curriculum. Faculty rated these three competencies as being adequately included in the program to being a major part of the curriculum. They did not rate these competencies as being greatly emphasized throughout the curriculum. It should be noted that on the whole the vocational evaluation graduates rated most competencies higher than the faculty members did. These overall higher ratings occur throughout and account for the differences in ratings of these three competencies. A one way anova test found the three competencies of vocational interviewing skills, job analysis, and task analysis as significant in terms of differences between 77 faculty and graduates. All other competencies within the 51 competencies on the survey were not found to be significant in terms of differences between faculty and graduates. Two competencies were found as significant in terms of differences between faculty and other unspecified graduates. The two competencies are vocational interviewing skills and job analysis. It is worthwhile to note that Taylor, Bordieri, and Lee (1993) found that less than 10% of respondents surveyed actually earned degrees with an emphasis in vocational evaluation while most practitioners cited on the job training or short term workshops as their most widely used source of training. Future training and workshops along with possible revision of CCWAVES? CVE requirements for non-vocational evaluation graduates may be warranted with specific attention to the knowledge areas of job analysis and vocational interviewing skills. In summary, vocational evaluation graduates and faculty have the most similar perceptions in regards to emphasis of competencies in the curriculum. Rehabilitation counseling graduates with a specialty in vocational evaluation follow closely behind. The other unspecified graduates had more similar ratings to vocational evaluation faculty than the rehabilitation counseling graduates with no specialty had. The emphasis of the following competencies come into question for vocational evaluation curriculum: observation of behavior, scoring and interpretation of tests, knowledge of test administration, report writing skills, analysis of observation and performance data, and awareness/inclusion of cultural diversity. These finding are consistent with the Newman and Waechter 1997 study entitled Commissioned Assessment of Competencies Prepared for the Commission on Certification of Work Adjustment and Vocational Evaluation Specialists which found experts believed entry level evaluators to need more skills in the 78 following areas: effective report writing, behavioral observation, legal/ethical issues, cultural diversity, and computer literacy. Finally, two competencies differed significantly between faculty and the other unspecified graduates. These competencies are job analysis and vocational interviewing skills. Future training, in service, and revision of CCWAVES requirements for non-vocational evaluation graduates may be warranted to ensure competencies in these two areas. 79 VI. IMPLICATIONS AND RECOMMENDATIONS Over 50 percent of the 51 competencies received a scoring of three or higher meaning that these competencies were adequately included in the curriculums. The top ranked competencies closely match in the following three groups: vocational evaluation faculty, vocational evaluation graduates, and rehabilitation counseling graduates with a specialty in vocational evaluation. This close matching supports Sink and Porters?(1978) conclusions that distinct differences exist between vocational evaluation and rehabilitation counseling. The need for more training in cultural diversity may be evident with the lower ratings of this competency by vocational evaluation graduates. These graduates, who consistently rated most competencies higher than vocational evaluation faculty, rated the competency of awareness/inclusion of cultural diversity as lower than the faculty did. Finally, the competencies of vocational interviewing skills and job analysis were rated significantly lower by the other graduates in comparison to faculty ratings. The training of most vocational evaluation practitioners in the areas of vocational interviewing skills and job analysis needs to be investigated. CCWAVES may need to revise their requirements to sit for the CVE exam for these other graduates as a preliminary step towards ensuring competence. 80 Recommendations A recommendation for a future study involves the investigation of vocational evaluation graduates in terms of utilization of awareness/inclusion of cultural diversity in daily practice. This study suggests that these graduates did not receive emphasized training in cultural diversity at the university level. It is recommended that current vocational evaluation practitioners be polled in terms of inclusion of cultural awareness in their current practices. In addition, there is a question as to how practitioners with non- rehabilitation type degrees are using skills in certain competency areas. It is recommended that the use of vocational interviewing skills and job analysis be investigated among non-rehabilitation graduate practitioners. A detailed task analysis of competencies might be completed by vocational evaluation/rehabilitation counseling and non-rehabilitation counseling graduates to determine exactly how and if these practitioners are using these competencies actively in their practice. In addition, it would be beneficial to the field of vocational evaluation to ascertain why so few practitioners hold the title of CVE. It would also be advantageous for VECAP to survey their members about their specific occupational roles to gain more insight into the members of this organization. Recommendations for replication of this study include ascertaining emphasis as well as importance. It would be beneficial to compare respondents? ratings of emphasis to ratings of importance. Any confusion over the emphasis versus importance of each competency would be eliminated using this type of survey. In addition, the Likert-type scale should be defined more clearly. For example, the number five on the Likert-type scale is described as ?greatly emphasized throughout the program.? It would be beneficial 81 to give examples of this type of emphasis. For example, number five could have been additionally described as ?competency is infused throughout the program, emphasized in over 90 percent of classes.? In addition, the descriptors of four and five should have been defined more clearly, creating greater difference in these two ratings. For example, number four could have included a description such as ?competency is emphasized throughout the program, emphasized in 50 to 75 percent of classes.? A seven point Likert-type scale could be used giving more descriptions and further defining emphasis. 82 REFERENCES Baker, R. J. (1977). Introduction: Vocational development standards ? A joint VEWAA- CARF effort (Report No. CE025411). Menomonie, WI: University of Wisconsin- Stout, The Rehabilitation Resource. (ERIC Document Reproduction Service No. ED 186 751) Boyer-Stephens, A., Waechter, D., & Newman, I. (1999, Spring). Analysis of qualitative aspects of assessment of minimum competencies for entry level vocational evaluators: Judgments of experts. Vocational Evaluation and Work Adjustment Bulletin, 23?34. CCWAVES. (2005, August). Standards and Procedures Manual for Certification in Vocational Evaluation. Retrieved October 5, 2005, from www.ccwaves.org Caplow, T. (1966). The sequence of professionalization. In H. M. Vollmer & D. L. Mills (Eds.), Professionalization (pp. 19-21). Upper Saddle River, NJ: Prentice-Hall. Coffey, D. (1978). Vocational evaluator competencies and their relevance as perceived by practitioners and educators in vocational evaluation. (Doctoral Dissertation, Auburn University). Dissertation Abstracts International, 39, 3364?3365. Coffey, D., & Mason, V. A. (1980, Fall). Certification of vocational evaluators and work adjustment specialists: Its history and its future. Vocational Evaluation and Work Adjustment Bulletin, 83?85. 83 Couch, R. H. (1971). The involvement of VEWAA in training, standards, and research. Vocational Evaluation and Work Adjustment Bulletin, 4(2), 27?31. Dillman, D. A., Tortora, R. D., Conradt, J., & Bowker, D. (1998). Influence of plain vs. fancy design on response rates for web surveys (Washington State University Report). Retrieved March 18, 2006, from http://www.sesrc.wsu.edu/dillman/ papers/asa98ppr.pdf Dowd, L. (Ed.). 1993. Glossary of terminology for vocational assessment, evaluation, and work adjustment. Menomonie, WI: Materials Development Center, Stout Vocational Rehabilitation Institute, University of Wisconsin-Stout. Fried, J. H., Harrand, G., Dowd, L. R., & Schuster, D. (1994, Spring). Standards for best practices in vocational evaluation/assessment: Three state models. Vocational Evaluation and Work Adjustment Bulletin, 21?26. Fry, R. R., & Garner, W. E. (1993). Standards for best practice in vocational assessment and evaluation. An opportunity for interdisciplinary input. In R. Fry (Ed.), The issues papers. Sixth national forum on issues in vocational evaluation (pp.93?95). Menomonie, WI: University of Wisconsin-Stout, The Rehabilitation Resource. Gannaway, T. W., & Sink, J. M. (1979). An analysis of competencies for counselor and evaluators, Vocational Evaluation and Work Adjustment Bulletin, 12(3), 3?15. Hamilton, M. (2003). The role and function of certified vocational evaluation specialists: A survey of practice in North America. (Doctoral Dissertation, University of Northern Colorado). Dissertation Abstracts International, AAT 3099708. Hoffman, P. R. (1971). History of the vocational evaluation and work adjustment association. Vocational Evaluation and Work Adjustment Bulletin, 11(3), 7?14. 84 Leahy, M. J., & Wright, G. N. (1988). Professional competencies of the vocational evaluator. Vocational Evaluation and Work Adjustment Bulletin, 21(4), 127?132. McDaniel, R. S. (1978). Vocational evaluation: On becoming a profession. Vocational Evaluation and Work Adjustment Bulletin, 11(3), 29?38. Newman, I., & Waechter, D. (1997). Commissioned assessment of competencies prepared for the commission on certification of work adjustment and vocational evaluation specialists. Akron, OH: Evaluation and Research Associates, Inc. Newman, I., Waechter, D., Nolte, D., & Boyer-Stephens, A. (1998, Fall/Winter). An assessment of knowledge domains for vocational evaluators: A precursor to a national licensure examination. Vocational Evaluation and Work Adjustment Bulletin, 72?79. Piccari, J., LeBlanc, D., Kells, F., Baker, R., Meyer, P., King, S., & Bleeker, R. (1975). Standards for vocational evaluation. Vocational Evaluation and Work Adjustment Bulletin, 8, 68?87. Pruitt, W. A. (1972). Task analysis of vocational rehabilitation graduate major with an emphasis in work evaluation: A comparative study of two groups of work evaluators. Menomonie, WI. Graduate College, University of Wisconsin-Stout. Pruitt, W. A. (1986). Vocational Evaluation (2 nd Edition). Menomonie, WI: Walt Pruitt and Associates. Rubin, S. E., & Roessler, R. T. (2001). Foundations of the vocational rehabilitation process (4 th ed.) Austin, TX: Pro-ed, Inc. 85 Saxon, J. P., Spitznagel, R. J., & Kennison, J. E. (1999). Vocational evaluation: Perceived professional status. Vocational Evaluation and Work Adjustment Journal, 32(1), 3?13. Shumate, S., Hamilton, M., & Fried, J. (2004, Spring). Vocational evaluation: The development of a profession. National VECAP Journal, 25?39. Sigmon, G. L. (1982). Application of judgmental standard setting procedures to vocational evaluation competency statements by rehabilitation field personnel and educators. Unpublished doctoral dissertation. Auburn, AL: Auburn University. Sigmon, G. L., Couch, R. H., & Halpin, G. (1987). A comparison of competency studies in vocational evaluation. Vocational Evaluation and Work Adjustment Bulletin, 20(1), 19?21. Sink, J. M., & Porter, T. L. (1978). Convergences and divergences of the rehabilitation counselor and vocational evaluator. Vocational Evaluation and Work Adjustment Association Bulletin, 1(11), 5?20. Taylor, D. W., Bordieri, J. E., & Lee, D. (1993, Winter). Job tasks and functions of vocational evaluators: A national study. Vocational Evaluation and Work Adjustment Bulletin, 146?154. Taylor, D. W., Bodieri, J. E., Crimando, W., & Janikowski, T. P. (1993, Summer). Job tasks and functions of vocational evaluators in three sectors of practice. Vocational Evaluation and Work Adjustment Bulletin, 39?46. Taylor, D. W., & Pell, K. L. (1993). Vocational evaluation curricula in rehabilitation education. Rehabilitation Education, 7(3), 185?194. 86 Taylor, D. W., Pell, K. L, & Chan, F. (1998). Survey of CORE-accredited rehabilitation counseling programs regarding training in vocational evaluation. Vocational Evaluation and Work Adjustment Bulletin, 31(1), 15?18. Thomas, S. W., & Sigmon, G. L. (1989). Critical content areas for graduate vocational evaluation education. Rehabilitation Education, 3, 35?42. Zwyghuizen, S. A. (1980, Winter). On the certification process: As a future vocational evaluator sees it. Vocational Evaluation and Work Adjustment Bulletin, 127?128. 87 APPENDICES 88 APPENDIX A INFORMATION LETTER FOR PARTICIPATION IN STUDY 89 INFORMATION Vocational Evaluator Competencies as Perceived by Vocational Evaluation Faculty and Graduates of Programs Offering Vocational Evaluation Courses You are invited to participate in a research study to determine the amount of emphasis placed on vocational evaluation competencies in university programs. This study is being conducted by Melanie McAlister under the supervision of Dr. Vivian M. Larkin. You were selected as a possible participant because you either graduated from a program offering vocational evaluation courses and are a VECAP/VEWAA member and/or you are a faculty member in vocational evaluation. Professional vocational evaluators and faculty members will be able to use this information to enhance practice and teaching. This research will serve to advance the quality of vocational evaluation and aid in preparing vocational evaluators at the university level. Any information obtained in connection with this study and that can be identified with you will remain confidential. Once the survey is completed, it will be emailed back to a secured web page and no identifying information will be connected with the returned survey. If you have any questions we invite you to ask them now. If you have questions later, Melanie McAlister (334-844-5943, mcalimt@auburn.edu) or Dr. Vivian M. Larkin (334- 844-5943, larkivm@auburn.edu) will be happy to answer them. For more information regarding your rights as a research participant you may contact the Auburn University Office of Human Subjects Research or the Institutional Review Board by phone (334)-844-5966 or e-mail at hsubjec@auburn.edu or IRBChair@auburn.edu . Please click on the link if you decide to participate in this study. Click here to proceed 90 APPENDIX B VOCATIONAL EVALUATOR COMPETENCIES SURVEY 91 Vocational Evaluator Competencies Survey I. Mark one category: _____Graduate with Degree in Vocational Evaluation _____Graduate with Degree in Rehabilitation Counseling with a Specialty in Vocational Evaluation _____Graduate with Degree in Rehabilitation Counseling with No Specialty in Vocational Evaluation _____Graduate with Degree in Related Field with Work Experience in Vocational Evaluation _____Faculty Member What percentage of those you serve fall into these categories? _____Transition _____Adult/VR _____Nonreaders _____Sensory Impaired _____Cognitive Impaired How long have you been employed in your current position? _____years What are your degrees in? ________________________________________ II. Gender _____Male _____Female III. Years of Service in the Field of Vocational Evaluation _____0-3 years _____3-5 years _____5-10 years _____10 to 20 years _____20+ years 92 IV. Level of Education _____B.S. _____M.S./M.Ed _____PhD V. Position ____Vocational Evaluator _____Counselor _____Faculty _____Other VI. Status _____Full Time _____Part Time VII. Certification _____CVE _____CRC _____Other VIII. State Employed in _______________(please list) IX. Race _____African American _____Caucasian _____Hispanic/Latino _____Other Please review the competencies below (52 items grouped into five groups) and rate from 1 to 5. Base ratings please on how much emphasis is placed on or has been placed on each specific competency in your individual university program (if you are a faculty member rate your university at which you are employed). Please rate based on your personal experience. Once again you are rating the degree to which each competency is EMPHASIZED in vocational evaluation courses. You are not rating importance; you are rating EMPHASIS. 93 1 2 3 4 5 ____________________________________________________________________________________ Not part Minimally included Adequately Major Part Greatly Emphasized of Program in Program included in of Program throughout Program Program Group I: The Professional _____Analysis of Observation and Performance Data _____Awareness/Inclusion of Cultural Diversity _____Computer Literacy _____Counseling Services as Applied to Vocational Assessment _____Data Synthesis _____Evaluator?s Interpersonal Skills _____Ethical Practices _____Laws Governing Vocational Evaluation Services _____Report Writing Skills _____Stress Management _____Teaching Skills _____Use of Reinforcement _____Verbally Communicating Assessment Results _____Vocational Relevance of Historical Data _____Vocational Interviewing Skills 94 1 2 3 4 5 ____________________________________________________________________________________ Not part Minimally included Adequately Major Part Greatly Emphasized of Program in Program included in of Program throughout Program Program Group II: Tools and Techniques _____ADA and Reasonable Accommodations in the Workplace Culture _____Assessment of Workplace Culture _____Data Recording Functions _____Community Based Assessment _____Development of Situations for Assessment (not work samples and not situational assessment). _____Interpretation of Observation and Performance Data _____Job Analysis _____Knowledge of Appropriate Use of Tests _____Knowledge of Appropriate Use of Work in Assessment Process _____Knowledge of Assistive Technology _____Knowledge of Test Administration _____Psychometric Principles _____Scoring and Interpretation of Tests _____Situational Assessment _____Task Analysis _____Work Samples and Vocational Screening Systems 95 Group 3: the Vocational Evaluation Process _____Case Management _____Individual Planning _____Knowledge of Multiple Intelligences _____Knowledge of the Process of Vocational Evaluation _____Learning Styles _____Observation of Behavior 1 2 3 4 5 ____________________________________________________________________________________ Not part Minimally included Adequately Major Part Greatly Emphasized of Program in Program included in of Program throughout Program Program Group 4: Characteristics of Consumers _____Employability Factors _____Functional Aspects of Individuals with Special Needs _____Knowledge of Physical Capabilities _____ Medical and Psychological Aspects of Disability _____Social Interaction Skills _____Vocational Aspects of a Disability 96 1 2 3 4 5 ____________________________________________________________________________________ Not part Minimally included Adequately Major Part Greatly Emphasized of Program in Program included in of Program throughout Program Program Group 5: The World of Work _____Knowledge of Community Resources _____Knowledge of Employer Needs _____Knowledge of Government Publications _____Knowledge of Local Labor Market _____Knowledge of the Use of Natural Supports _____Knowledge of Vocational Retraining Programs _____Knowledge of Worker Traits _____Knowledge of World of Work _____Transferability of Skills Analysis THANK YOU!!!!!