Abstract
This study evaluated the impact of five alternative types of leadership development practices on ratings of leaders within a transnational organization. Impact was assessed with 360-degree ratings of leadership behaviors. Results showed that self and superior ratings of leadership behaviors were higher for leaders who had completed at least one form of leadership development while follower ratings were higher only for those leaders who had completed a developmental job assignment. Practical and theoretical implications raise questions about the usefulness of leadership development programs.
Impact of Leadership Development Experiences in a Transnational Organization
Bass (1990); Burke and Day (1986); and Clark, Clark, and Campbell (1992) have proposed that leadership makes a positive difference in organizations. Iles (1993); Story, Mabey, and Thomson (1997); and Drew and Wah (1999) observed that organizations turn to leadership development in search of effective ways to provide this needed leadership. Conger and Benjamin (1999) noted that this need has spurred the leadership development field into a billion-dollar business and Giber, Carter, and Goldsmith (2000) indicated that annual budgets for leadership development programs will continue to grow throughout the next decade. Engelbrecht and Fischer (1995); and Goldstein and Gessner (1988) have expressed dissatisfaction with existing theoretical and empirical foundations for leadership development and indicate a growing desire to validate leadership training and development efforts. Reflecting on the need for current research in the field, Collins (2001) wrote, “No comprehensive analysis of managerial or leadership training has been done since Burke and Day’s (1986) cumulative analysis” (p. 44). This is echoed by Lynham (2000), who concluded that little is known about what outcomes in leadership development initiatives contribute to exemplary leadership practices in today’s organizations. This current study sought to determine the impact of leadership development programs on leaders’, as well as their subordinates’ and superiors’ perception of the leaders’ performance.
Variables
The dependent variable for this study is exemplary leadership practices as defined by Kouzes and Posner’s (1995) Five Practices of Exemplary Leaders: (a) challenging the process; (b) inspiring a shared vision; (c) enabling others to act; (d) modeling the way; and (e) encouraging the heart. Jurkowski, (1997); Nolan (1992); Huber, Maas, McCloskey, Goode, and Watson (2000); and Leong, (1995) substantiate the validity of Kouzes and Posner’s work.
The independent variable for this study is a leadership development experience. While both “leadership development” and “experience” may suggest subjective challenges to overcome in this empirical study, the definitions below provide operational parameters for creating the self-typing paragraphs that describe these various leadership development experiences. Brungardt (1996) broadly defined leadership development to include “almost every form of growth or stage of development in the life-cycle that promotes, encourages, and assists in one’s leadership potential” (p. 83). This definition perceives leadership development as a continuous learning process that spans an entire lifetime. McCauley, Moxley, and Van Velsor (1998) provided a more focused perspective when they defined leadership development as intervention processes that seek to expand a person’s capacity to be effective in leadership roles. This study employs McCauley, Moxley, and Van Velsor’s definition that limits the discussion to structured adult learning that intervenes to alter, create, enhance, and/or accelerate effective leadership performance within individuals through a leadership development experience.
Elements of a Leadership Development Experience
Van Velsor, McCauley, and Moxley (1998) proposed three elements that combine to characterize an effective leadership development experience: (a) assessment; (b) challenge; and (c) support. Van Velsor, McCauley, and Moxley concluded, “When we look at any type of developmental experience – from training programs to job assignments – we find that they are most effective when all three elements are present” (p. 8).
Assessment
According to Van Velsor, McCauley, and Moxley (1998), the element of assessment within a leadership development experience provides relevant information to the participant from others regarding the participant’s current leadership strengths, level of leadership effectiveness, and primary developmental needs. This assessment information can be supplied by coworkers (bosses, peers, direct reports) and/or individuals external to the organization (customers, suppliers, family, and friends). Chappelow (1998) added, “People’s own views of themselves are often narrow and biased. The enhanced self-awareness created by feedback can help leaders know where to focus their developmental efforts and motivate them to better understand their strengths and improve their weaknesses” (p. 31). This study measures leadership development experiences that contain the element of assessment.
Challenge
Van Velsor, McCauley, and Moxley (1998) described the element of challenge as an opportunity for exposure, experimentation, and practice needed to master the leadership role; they wrote, “Challenging experiences force people out of their comfort zone. They create disequilibrium, causing people to question the adequacy of their skills, framework, and approaches” (p. 11). Van Velsor, McCauley, and Moxley summarized, “The element of challenge serves the dual purpose of motivating development and providing the opportunity to develop” (p. 14). This study measures leadership development experiences that contain the element of challenge.
Support
Van Velsor, McCauley, and Moxley (1998) defined the element of support within a leadership development experience as the confirmation and clarification of lessons learned. They wrote, “While the element of challenge provides the disequilibrium needed to motivate people to change, the support elements of an experience send the message that their efforts to learn and grow are valued” (p. 15). According to these researchers, support most often comes from other people (coworkers, family, friends, colleagues, coaches, mentors), but can also stem from organizational norms and procedures that attach value to continuous learning and development. They wrote, “Support serves as the social cue that puts a positive valence on where people are currently and on the direction in which they are moving” (p. 16). This study measures leadership development experiences that contain the element of support.
Types of Leadership Development Experiences
After Van Velsor, McCauley, and Moxley (1998) theorized that all effective leadership development experiences are composed of the elements of assessment, challenge, and support in varying degrees of emphasis, they explained, “There are many types of experience that develop leadership abilities in those who participate” (p. 21). Although no single type of leadership development experience currently dominates the organizational spectrum, significant types of leadership development experiences can be synthesized into five categories: (a) developmental job assignments; (b) multi-rater feedback; (c) skill-based training; (d) feedback-intensive programs; and (e) developmental relationships.
Developmental job assignments
According to Ohlott (1998), developmental job assignments are new jobs (via promotion or transfer) or responsibilities added to existing jobs that stretch participants, push them out of their comfort zones, and require them to think and act differently. Ohlott proposed that developmental job assignments are often not well defined, present responsibilities that are new to the participant, contain problems to solve, and demand the participant to make choices under conditions of risk and uncertainty. Gabarro’s (1987) research suggested that it can take three and a half years for a person to glean the important leadership lessons from a developmental job assignment; he also found that learning at a deeper level does not occur until after the first eighteen months in a job. Lombardo and Eichinger (1989) proposed that a supervisory role can be developmental because it allows the leader to test different leadership styles on group performance; moreover, work assignments that are complex, ambiguous, changing, and/or unpredictable provide opportunities for new and innovative solutions.
According to Ohlott (1998), job assignments that are most developmental incorporate the element of assessment. Whether the assessment is provided through delivery systems such as one-on-one assessment, multi-rater feedback, or written evaluations, the topics that facilitate assessment include: (a) the organization’s objectives for the job; (b) job relationship to organization’s mission and goals; (c) participant’s development needs; (d) developmental objectives of job for participant; and (e) progress made toward developmental objectives.
Ohlott (1998) asserted, “The key element in a developmental assignment is challenge. By tackling unfamiliar tasks and seeing the consequences of their actions, people learn from the challenges of their assignments” (p. 130). Whether a developmental job assignment includes transition to a new job or a task within an existing job, Ohlott identified five sources of challenge in job assignments. First, transitions are a source of challenge; these transitions propel a leader into new, different, or broader leadership responsibilities than previous ones. Second, responsibility for creating change is a source of challenge; such change could include being tasked with starting something new in the organization, carrying out a reorganization, responding to rapid changes in the environment, fixing problems created by a former incumbent, reducing staff, and taking over problem (incompetent or resistant) employees. Third, a high level of responsibility can be a source of challenge; such a high level is characterized by high visibility, pressure from senior leaders for key decisions, and responsibility to consider multiple functions and stakeholders. Fourth, non-authority relationships are a source of challenge; in such an assignment, the leader must get the job done through colleagues and other external parties over whom he/she has no direct authority. Fifth, obstacles provide a developmental challenge; these obstacles may include leadership responsibility under adverse financial conditions, working in a context where there is unclear direction from senior management, starting a new project with few resources, or working with a difficult boss.
Commenting on the element of support in developmental job assignments, Ohlott (1998) wrote, “Perhaps the most important success factor is organizational support” (p. 150). According to Ohlott, organizational support can take a variety of forms: (a) a sense of collegiality with coworkers; (b) buffering by the supervisor with permission to risk; (c) mentor(s) within the organization who help guide the participant through their phase of getting acquainted with the new assignment (d) coaching from the previous job holder; and (e) increased compensation or other forms of public recognition.
Multi-rater feedback
Multi-rater feedback (also known as 360-degree feedback) has generated much discussion and gained widespread popularity during recent years in the United Kingdom and the United States (Funderburg & Levy, 1997; Mathews & Redman, 1997). Applying multi-rater feedback to leadership, Chappelow (1998) wrote, “What we call 360-degree feedback is a method of systematically collecting opinions about a manager’s performance from a wide range of coworkers” (p. 31). These opinions are solicited from the manager, his/her peer(s), direct report(s), and supervisor(s); in addition, opinions can be collected from people outside the organization (customers, suppliers, family members).
Chappelow (1998) explained that the prominent element in multi-rater feedback is assessment. According to Chappelow, the person being assessed in multi-rater feedback process typically selects a number of coworkers to participate. Working individually, the raters and the person being assessed (the participant) complete surveys designed to collect information about the participant’s specific skills and/or behaviors that are considered important for leadership effectiveness within the organization. After the raters complete the surveys, the results are returned to the participant.
Based on Chappelow’s (1998) observations, the element of support within the multi-rater feedback experience can contain the following features: (a) maintaining the confidential nature of the feedback data; (b) giving participants access to a trained feedback facilitator for interpretation/clarification of the data; (c) allowing the participant and his/her supervisor to meet beforehand to discuss their mutual goals for the process; (d) strategizing how to receive ongoing feedback; (e) involving superiors to gain buy-in for the participant’s resulting development plan; and (f) following up periodically on the development plan.
Skill-based training
McDonald-Mann (1998) described skill-based training as “a developmental experience in which individuals gain knowledge and practice behaviors necessary to hone present skills or develop new ones” (p. 107). According to McDonald-Mann, an effective skill-based training environment facilitates participants to learn conceptual information or necessary behaviors, practice using the new information or behaviors, and receive feedback on their performance. This training can be delivered through five primary methods: (a) lectures that present content-specific information; (b) case studies that present information describing an organizational situation, how the situation was handled, and the outcome; (c) role-plays in which participants spontaneously act out characters assigned to them in a scenario based on an understanding of the theories, principles, and techniques underlying the topic at hand; (d) behavioral role-modeling that presents participants with models of appropriate behavior after which they dramatize the behavior and receive feedback on their performance; and (e) simulation that offers a realistic opportunity to practice one or more aspects of the leadership role.
McDonald-Mann (1998) identified two means of assessment in a skill-based training experience: (a) informal assessment; and (b) formal assessment. Informal assessment occurs when the participant gains self-awareness of his/her leadership strengths and weaknesses through interaction with the training program. According to McDonald-Mann, the participant typically receives impromptu feedback during the program and returns to the job with better awareness of current skill level and development needs. In the formal assessment approach, the participant completes pre-training and post-training measures of his/her skills. In the formal approach, one assessment or series of assessments is conducted before or near the beginning of the training. After the program of lectures, simulations, and ongoing feedback, the participant completes a second assessment or series of assessments and measures his/her improved leadership skills. McDonald-Mann wrote, “Once participants have successfully learned the training content, post-training assessment can validate their efforts. If they have not, post-training assessment highlights the need for either further training or a different approach to developing the skills” (p. 119).
Addressing the element of challenge through 360-degree feedback, McDonald-Mann (1998) wrote, “By definition, learning something new takes people out of their comfort zones and into new waters. Leaving this comfort zone is a challenging experience, but it is necessary if learning is to take place” (p. 119). According to this research, skill-based training is embedded with challenges through exposure to new information, confronting unfamiliar ideas, and practicing new behaviors. Role-plays, behavior modeling, and simulations are relatively safe ways to put individuals into challenging situations for practicing new skills.
According to McDonald-Mann (1998), “One factor that strongly affects transfer of training is the degree to which the organization has a supportive climate, where people are allowed and encouraged to use new skills on the job” (p. 120). Rouillier and Goldstein (1990) identified two major components of a supportive climate for individuals who have experienced skill-based training: (a) situational cues; and (b) consequences. Situational cues may involve giving participants the opportunity to use the training content on the job immediately. Another is for the supervisor to relieve pressure on the participant for a short time to allow them to practices new skills on the job. Examples of positive consequences include participants receiving acknowledgement and being rewarded when they use their new skills. Bunker and Webb’s (1992) study found that participants who start to work on incorporating newly learned skills on their job after a training program may experience an initial drop in performance; therefore, the supportive organization provides time and buffering for the new skills to be fully integrated.
Feedback-intensive programs
Guthrie and Kelly-Radford (1998) described the feedback-intensive program (FIP) as a classroom-based educational experience that typically takes place away from work; it provides intense feedback that goes beyond skills and behaviors to address deeper values and preferences within a climate that is relationship-based and support-intensive. Reflecting on the value of the FIP, Guthrie and Kelly-Radford wrote, “The process of providing a rich and intense feedback experience is grounded in our understanding that people approach leadership with frameworks built on their past experience, their values, and personality-based needs and preferences” (p. 68).
McDonald-Mann (1998) described five differences between skill-based training and feedback-intensive training. First, the purpose of skill-based training is to improve performance and specified skill areas whereas the purpose of an FIP is to provide the participant with in-depth understanding of his/her strengths and weaknesses through increased self-awareness. Second, skill-based training is narrowly focused to specified skill areas whereas an FIP maintains a wide focus on a broad range of potential strengths and weaknesses. Third, skill-based training employs feedback as a tool for assessing skill level whereas an FIP employs feedback to understand how the participant is viewed from multiple perspectives and how one’s behavior has an impact on others. Fourth, effective skill-based training provides ample opportunity for participants to practice new skills through experiential activities whereas an FIP provides little if any practice of new skills in lieu of generating data about the participant, his/her behavior, and how these behaviors are viewed by others. Fifth, the design of skill-based training includes prescriptive information and specific models whereas an FIP design includes information that helps participants organize their experiences with general concepts and models.
According to Guthrie and Kelly-Radford (1998), the element of assessment in an FIP comes in many forms and from a variety of sources: formal feedback from a supervisor, feedback from peers and subordinates, observation from trainers, consultants, and/or professors during the program, and the participants’ own assessments of their values and preferences. These researchers found that an FIP often takes advantage of different methods and techniques; these can include structured experiences (group observation, videotaping, facilitated debrief), and assessment instruments (multi-rater feedback, structured feedback from fellow participants). Leslie and Fleenor (1998) emphasized the critical importance of reliable and valid feedback that the participant can take seriously and use productively to create plans for change.
Guthrie and Kelly-Radford (1998) identified four potential sources of challenge in an FIP. First, the assessment and feedback process described above provides challenge by its very nature; the discomfort of being observed and rated by others and having weaknesses exposed is often a source of intense challenge. Second, the conceptual framework itself (classroom content) is a source of challenge. Classroom content can cover the domains of communications, interpersonal relationships, coaching, teamwork, decision making, and/or dealing with ambiguity and turmoil; these domains provide a new perspective for understanding leadership. A third source of challenge can come from engaging in unfamiliar activities; Guthrie and Kelly-Radford wrote, “For some, the challenging activity might be a simulation or an exercise in group problem solving under tight time constraints. For others, it may be an artistic activity, or an outdoor challenge course” (p. 87). A fourth source of challenge can arise from meeting diverse people with different perspectives. In an FIP, this type of challenge may occur unintentionally, or it may take place through intentional selection of participants so as to maximize racial, gender, cultural, functional, or professional diversity.
Commenting on the link between the elements of challenge and support in an FIP, Guthrie and Kelly-Radford (1998) wrote, “To hear and accept feedback in these areas, and to be motivated to change behaviors or add new skills, people need a good deal of support” (p. 89). These researchers proposed that support is created primarily through formal and informal processes enacted by the FIP staff. Behaviors and processes from staff that facilitate support include: (a) relating to each participant with authenticity; (b) being comfortable with self-disclosure; (c) understanding and acknowledging each participant’s unique needs; and (d) being nonjudgmental and non-prescriptive in working with participants. To accomplish this support, the following techniques may be present: (a) integrating elements of each participant’s situation into the program activities; (b) sharing of participant’s perspectives; (c) teaching to all learning styles and abilities, using a variety of methodologies; and (d) creating opportunities for reflection and consolidation of learning through feedback specialists, coaches, journaling, and/or group reflection.
Developmental relationships
Commenting on Bandura’s (1986) research on social cognitive theory, McCauley and Douglas (1998) wrote, “In addition, research-based advice on almost any topic related to learning and development also entails a relationship strategy” (p. 161). To define developmental relationships operationally, McCauley and Douglas identified various developmental relationships that provide the elements of assessment, challenge, and support.
McCauley and Douglas (1998) identified four developmental relationships that provide the element of assessment. First, a “feedback provider” is a source of day-to-day, ongoing information on how a participant is doing in seeking to learn new skills and perspectives. Although an FIP or 360-degree feedback may be the impetus for development goals, the continuous feedback through such a developmental relationship becomes critical. Second, a “sounding board” provides the participant with opportunities to articulate ideas and strategies for reactions and fine-tuning before implementation. Third, a “comparison point” provides the participant with a model (benchmark); the comparison point can be an individual in a similar work situation. Fourth, a “feedback interpreter” helps the participant make sense of the feedback data.
To push participants past their comfort zones, McCauley and Douglas (1998) proposed four developmental relationships that can provide the element of challenge. First, a “dialogue partner” can expose the participant to different perspectives through questioning, prodding, and reflecting on underlying assumptions and biases. Second, an “assignment broker” provides the participant with assignments (new jobs, new responsibilities added to the current job, or temporary assignments outside the participant’s normal job responsibilities) that stretch the participant’s current capacities. Third, an “accountant” assumes responsibility to monitor the participant’s progress toward development goals. Fourth, a “role model” encourages the participant to step outside his/her comfort zone and try new or more complex skills and behaviors.
Based on their theory that participants need the element of support in developmental relationships, McCauley and Douglas (1998) wrote, “There are a variety of support roles that people play for each other” (p. 166). First, a “counselor” encourages the participant to explore the emotional aspects of the development experience: fear of failing, anxiety about leaving behind the familiar, stress in trying to learn and change while carrying a heavy workload, frustration at not making progress, or anger with others who do not support the participant’s development. Second, a “reinforcer” expresses confidence, provides affirmation, and rewards the participant for making progress toward development goals. Third, a “cohort” is an individual who struggles with the same challenges and can empathize with the participant; the cohort may also provide the participant with living proof that the development challenge can be mastered.
Method
This study employed a field-based cross-sectional approach to measure the types of leadership development experiences using the self-typing paragraph approach and the leaders’ self-report, superior-report, and other-report of the leader using Kouzes and Posner’s LPI. Permission was received to use the LPI.
Participants
The participants for this study came from a transnational non-profit, faith-based organization that specializes in international Christian broadcasting in six international regions (Africa, Americas, Europe, Northeast Asia, South Asia and Southeast Asia). The transnational organization employs 1,000 full time staff and has an additional 2,000 part-time and volunteer staff.
Self-typing Paragraphs
James and Hatten (1995) described the self-typing paragraph approach as a method by which respondents read unlabeled paragraphs and identify the paragraph that best identifies his/her experience. James and Hatten proposed that there is good evidence for the reliability of the self-typing paragraph approach and has been effectively employed to gather research from organizations such as firms (Miles and Snow, 1978; Winston, 1998), hospitals (Shortell and Zajac, 1990), and HMOs (Conant, Mokwa, and Varadarajan, 1990). This study asked participants to read ten unlabeled paragraphs that describe the following: (a) an individual who has experienced a developmental job assignment that includes the elements of assessment, challenge, and support; (b) an individual who has not experienced a developmental job assignment that includes the elements of assessment, challenge, and support; (c) an individual who has experienced multi-rater feedback that includes the elements of assessment, challenge, and support; (d) an individual who has not experienced multi-rater feedback that includes the elements of assessment, challenge, and support; (e) an individual who has experienced skill-based training that includes the elements of assessment, challenge, and support; (f) an individual who has not experienced skill-based training that includes the elements of assessment, challenge, and support; (g) an individual who has experienced a feedback-intensive program that includes the elements of assessment, challenge, and support; (h) an individual who has not experienced a feedback-intensive program that includes the elements of assessment, challenge, and support; (i) an individual who has experienced a developmental relationship that includes the elements of assessment, challenge, and support; (j) an individual who has not experienced a developmental relationship that includes the elements of assessment, challenge, and support .These self-typing paragraphs are located in the chapter on methods. The respondents were instructed to identify the paragraph(s) that best identify their experiences. Based on their responses, these individuals were initially classified into two categories: (a) those respondents who indicated that they have encountered at least one of the leadership development experiences described; and (b) those who indicated they have not encountered any of the described leadership development experiences.
Leadership Practices Inventory (LPI)
Based on their conceptual research that resulted in the Five Practices of Exemplary Leaders, Kouzes and Posner (1995) developed a set of statements describing various leadership actions and behaviors reflected in the five practices: (a) challenging the process; (b) inspiring a shared vision; (c) enabling others to act; (d) modeling the way; and (e) encouraging the heart. Each statement was originally cast on a five-point Likert scale, and reformulated in 1999 into a more robust and sensitive ten-point Likert scale. A higher value represents more frequent use of a leadership behavior. In relation to the leadership behavior, the selections are: (1) almost never do what is described in the statement; (2) rarely; (3) seldom; (4) once in a while; (5) occasionally; (6) sometimes; (7) fairly often; (8) usually; (9) very frequently; and (10) almost always do what is described in the statement. Statements were modified, discarded, or included following lengthy discussions and iterative feedback sessions with respondents and subject matter experts as well as empirical analyses of various sets of behaviorally based statements. Ongoing analysis and refinements in the instrument continue with a database involving well over 100,000 respondents.
Procedures
With a cover letter, survey packets including the self-typing paragraphs survey (Appendix A), LPI Self form and LPI Observer form were sent electronically to the organization’s four regional liaisons (Africa, Americas, Northeast Asia, and Southeast Asia) who, in turn, distributed the survey packets electronically to all executives, directors, officers, and managers in their respective regions.
Table 1 – Survey Packets (LPI Surveys and Self-Typing Paragraph Surveys) Distributed and Received
| TWR region | Survey Packets Distributed | Survey Packets Received |
| Africa | 23 | 11 |
| Americas | 34 | 23 |
| Corporate | 13 | 6 |
| Northeast Asia | 26 | 25 |
| Southeast Asia | 17 | 6 |
Table 1 above provides data regarding completed survey packets distributed and collected by region. The percentage of participants by region in the data sample is as follows: (a) Africa – 15.5%; (b) Americas – 32.4%; (c) Corporate – 7.0%; (d) Europe – 0%; (e) Northeast Asia – 36.6%; (f) South Asia – 0%; and (f) Southeast Asia – 8.5%. This response rate, while lower than desired, does allow statistical analysis of the results. Due to the relatively small sample size, this research will accept an alpha level of .10 toindicate statistical significance.
Table 2 below reflects the data sample population by roles; according to this data, the total data sample percentages by role is as follows: (a) executives – 8.5%; (b) directors – 38.0%; (c) officers – 46.5%; and (d) managers – 4.0%. These distribution percentages are generally consistent with the total TWR leadership population.
Table 2 – TWR Role Representation by Region
| TWR region | Executive | Director | Officer | Manager |
| Africa | 1 | 8 | 0 | 2 |
| Americas | 1 | 7 | 12 | 3 |
| Corporate | 2 | 2 | 2 | 0 |
| Northeast Asia | 1 | 6 | 18 | 0 |
| Southeast Asia | 1 | 4 | 1 | 0 |
| Total | 6 | 27 | 33 | 5 |
Self-typing Paragraphs
SPSS (2001) statistical software was used to analyze the results of the self-typing paragraphs. Table 3 below reports the frequency distribution of leadership development experiences within the data sample. Of those who experienced any type of leadership development, the majority of participants reported having had multiple types of leadership development experiences while comparatively few of these participants reported having experienced only one type of leadership development.
Table 3 – Leadership Development Experiences Frequency Distribution
| Leadership development experience(s) type | Yes | No | Only |
| Any type | 40 | 31 | |
| Developmental job assignment | 24 | 47 | 4 |
| Multi-rater assessment | 8 | 63 | 0 |
| Skill-based training | 15 | 56 | 2 |
| Feedback-intensive program | 12 | 59 | 3 |
| Developmental relationship | 27 | 44 | 6 |
LPI Scoring
After the data was entered into the LPI Scoring Software (2001) to obtain the participant’s LPI Self and LPI Observer raw scores, a raw score across the Five Practices of Exemplary Leaders was determined for the following: (a) self; (b) manager, and (c) an average score for others (coworkers, direct reports, and others). These raw scores were entered into SPSS (2001) under the participant’s identification number for analysis.
Independent Samples T-test Analysis
Table 4 below provides t-test and significance results when comparing LPI raw scores of participants (self, manager, and others) who indicated that they have had at least one (any) type of leadership development experience and the LPI raw scores (self, manager, and others) of those who indicated that they have not experienced any type of leadership development. According to the data, raw scores of those who have experienced leadership development are consistently higher than those who have not experienced leadership development. Significance factors less than .10 appear in the LPI self and manager scores, but not in the LPI others scores.
Table 4 – Independent Samples T-test Relating Any Leadership Development Experience to LPI Scores
| LPI category | Experiencedavg. score | Not experiencedavg. score | T-test | Significance |
| Challenging the process (self) | 44.58 | 39.65 | -2.525 | .014 |
| Inspiring a shared vision (self) | 44.23 | 36.29 | -3.017 | .004 |
| Enabling others to act (self) | 47.58 | 45.06 | -1.541 | .013 |
| Modeling the way (self) | 46.40 | 42.16 | -2.545 | .013 |
| Encouraging the heart (self) | 44.23 | 39.32 | -2.143 | .036 |
| Challenging the process (manager) | 42.76 | 36.32 | -2.062 | .044 |
| Inspiring a shared vision (manager) | 42.97 | 35.92 | -2.146 | .036 |
| Enabling others to act (manager) | 44.88 | 39.40 | -1.872 | .066 |
| Modeling the way (manager) | 47.32 | 40.44 | -2.300 | .025 |
| Encouraging the heart (manager) | 42.94 | 36.84 | -2.237 | .029 |
| Challenging the process (others) | 38.79 | 36.21 | -0.869 | .388 |
| Inspiring a shared vision (others) | 38.75 | 37.08 | -0.550 | .584 |
| Enabling others to act (others) | 44.05 | 42.49 | -0.723 | .472 |
| Modeling the way (others) | 42.81 | 41.37 | -0.526 | .601 |
| Encouraging the heart (others) | 40.93 | 39.52 | -0.636 | .527 |
Table 5 below provides t-test and significance results when comparing LPI raw scores of participants (self, manager, and others) who indicated that they have experienced a developmental job assignment and the LPI raw scores (self, manager, and others) of those who indicated that they have not experienced a developmental job assignment. According to the data, raw scores of those who have experienced a developmental job assignment are consistently higher than those who have not experienced a developmental job assignment. With the exception of self-scores under challenging the process (.118), significance factors of less than.10 are reflected for all other LPI scoring categories.
Table 5 – Independent Samples T-test Analysis Relating Developmental Job Assignments to LPI Scores
| LPI category | Experiencedavg. score | Not experiencedavg. score | T-test | Significance |
| Challenging the process (self) | 44.63 | 41.30 | -1.583 | .118 |
| Inspiring a shared vision (self) | 46.17 | 38.00 | -2.955 | .004 |
| Enabling others to act (self) | 48.63 | 45.38 | -1.915 | .060 |
| Modeling the way (self) | 46.88 | 43.36 | -1.977 | .052 |
| Encouraging the heart (self) | 45.54 | 40.32 | -2.180 | .033 |
| Challenging the process (manager) | 45.24 | 37.16 | -2.550 | .013 |
| Inspiring a shared vision (manager) | 46.62 | 36.32 | -3.169 | .002 |
| Enabling others to act (manager) | 48.62 | 39.21 | -3.297 | .002 |
| Modeling the way (manager) | 49.95 | 41.34 | -2.850 | .006 |
| Encouraging the heart (manager) | 45.71 | 37.39 | -3.058 | .003 |
| Challenging the process (others) | 41.83 | 35.53 | -2.073 | .042 |
| Inspiring a shared vision (others) | 41.90 | 36.04 | -1.884 | .064 |
| Enabling others to act (others) | 46.74 | 41.65 | -2.314 | .024 |
| Modeling the way (others) | 46.21 | 40.11 | -2.192 | .032 |
| Encouraging the heart (others) | 43.61 | 38.63 | -2.211 | .030 |
Table 6 below provides t-test and significance results when comparing LPI raw scores of participants (self, manager, and others) who indicated that they have experienced multi-rater feedback and the LPI raw scores (self, manager, and others) of those who indicated that they have not experienced multi-rater feedback. According to the data, LPI self scores of those who have experienced multi-rater feedback are higher than those who have not experienced multi-rater feedback, but LPI manager and others scores for those who have experienced multi-rater feedback are consistently lower than those who have not experienced multi-rater feedback. Significance factors are less than .10 only under LPI self scores for challenging the process, inspiring a shared vision, and encouraging the heart.
Table 6 – Independent Samples T-test Relating Multi-rater Feedback to LPI Scores
| LPI category | Experiencedavg. score | Not experiencedavg. score | T-test | Significance |
| Challenging the process (self) | 48.25 | 41.68 | -2.117 | .038 |
| Inspiring a shared vision (self) | 48.00 | 39.84 | -1.907 | .061 |
| Enabling others to act (self) | 48.63 | 46.21 | -0.937 | .352 |
| Modeling the way (self) | 48.25 | 44.08 | -1.553 | .125 |
| Encouraging the heart (self) | 47.75 | 41.37 | -1.761 | .083 |
| Challenging the process (manager) | 38.14 | 40.29 | 0.434 | .666 |
| Inspiring a shared vision (manager) | 38.14 | 40.23 | 0.401 | .690 |
| Enabling others to act (manager) | 37.57 | 43.23 | 1.244 | .219 |
| Modeling the way (manager) | 44.00 | 44.66 | 0.097 | .923 |
| Encouraging the heart (manager) | 38.29 | 40.63 | 0.542 | .590 |
| Challenging the process (others) | 33.93 | 38.13 | 0.903 | .370 |
| Inspiring a shared vision (others) | 35.65 | 38.32 | 0.561 | .577 |
| Enabling others to act (others) | 40.03 | 43.79 | 1.114 | .269 |
| Modeling the way (others) | 39.28 | 42.54 | 0.762 | .448 |
| Encouraging the heart (others) | 38.81 | 40.50 | 0.487 | .628 |
Table 7 below provides t-test and significance results when comparing LPI raw scores of participants (self, manager, and others) who indicated that they have experienced skill-based training and the LPI raw scores (self, manager, and others) of those who indicated that they have not experienced skill-based training. According to the data, LPI self-scores of those who have experienced skill-based training are generally higher than those who have not experienced multi-rater feedback; the only exception is under enabling others to act. Similarly, LPI manager and others scores for those who have experienced multi-rater feedback are consistently higher than those who have not experienced multi-rater feedback; however, significance factors are less than .10 only under LPI self scores for challenging the process.
Table 7 – Independent Samples T-test Relating Skill-based Training to LPI Scores
| LPI category | Experiencedavg. score | Not experiencedavg. score | T-test | Significance |
| Challenging the process (self) | 45.73 | 41.54 | -1.729 | .088 |
| Inspiring a shared vision (self) | 45.07 | 39.61 | -1.637 | .106 |
| Enabling others to act (self) | 46.27 | 46.54 | 0.134 | .894 |
| Modeling the way (self) | 45.13 | 44.39 | -0.350 | .727 |
| Encouraging the heart (self) | 43.33 | 41.75 | -0.553 | .582 |
| Challenging the process (manager) | 42.80 | 39.09 | -1.018 | .313 |
| Inspiring a shared vision (manager) | 42.20 | 39.23 | -0.771 | .444 |
| Enabling others to act (manager) | 43.73 | 42.16 | -0.461 | .647 |
| Modeling the way (manager) | 47.40 | 43.39 | -1.143 | .258 |
| Encouraging the heart (manager) | 42.00 | 39.80 | -0.686 | .496 |
| Challenging the process (others) | 37.97 | 37.58 | -0.107 | .915 |
| Inspiring a shared vision (others) | 38.28 | 37.95 | -0.090 | .928 |
| Enabling others to act (others) | 43.13 | 43.43 | -0.116 | .908 |
| Modeling the way (others) | 43.29 | 41.88 | -0.424 | .673 |
| Encouraging the heart (others) | 40.10 | 40.37 | -0.100 | .921 |
Table 8 below provides t-test and significance results when comparing LPI raw scores of participants (self, manager, and others) who indicated that they have experienced a feedback-intensive program and the LPI raw scores (self, manager, and others) of those who indicated that they have not experienced a feedback-intensive program. According to the data, LPI self-scores of those who have experienced a feedback-intensive program are consistently higher than those who have not experienced a feedback-intensive program. However, LPI manager and others scores for those who have experienced a feedback-intensive program are consistently lower than those who have not experienced a feedback-intensive program. Significance factors are less than .10 only under LPI self scores for challenging the process, enabling others to act, and encouraging the heart.
Table 8 – Independent Samples T-test Relating Feedback-intensive Programs to LPI Scores
| LPI category | Experiencedavg. score | Not experiencedavg. score | T-test | Significance |
| Challenging the process (self) | 46.83 | 41.53 | -2.023 | .047 |
| Inspiring a shared vision (self) | 44.67 | 39.97 | -1.284 | .203 |
| Enabling others to act (self) | 50.83 | 45.59 | -2.495 | .015 |
| Modeling the way (self) | 47.58 | 43.93 | -1.613 | .111 |
| Encouraging the heart (self) | 46.83 | 41.12 | -1.874 | .065 |
| Challenging the process (manager) | 40.00 | 40.04 | 0.008 | .993 |
| Inspiring a shared vision (manager) | 39.50 | 40.06 | 0.113 | .910 |
| Enabling others to act (manager) | 41.75 | 42.69 | 0.215 | .830 |
| Modeling the way (manager) | 42.50 | 44.71 | 0.490 | .626 |
| Encouraging the heart (manager) | 38.38 | 40.67 | 0.560 | .578 |
| Challenging the process (others) | 34.93 | 38.21 | 0.835 | .406 |
| Inspiring a shared vision (others) | 35.04 | 38.63 | 0.897 | .373 |
| Enabling others to act (others) | 41.39 | 43.77 | 0.831 | .409 |
| Modeling the way (others) | 38.12 | 43.00 | 1.362 | .178 |
| Encouraging the heart (others) | 38.81 | 40.62 | 0.617 | .539 |
Table 9 below provides t-test and significance results when comparing LPI raw scores of participants (self, manager, and others) who indicated that they have experienced a developmental relationship(s) and the LPI raw scores (self, manager, and others) of those who indicated that they have not experienced a developmental relationship(s). According to the data, raw scores of those who have experienced a developmental relationship(s) are consistently higher than those who have not experienced a developmental relationship(s). Significance factors less than .10 appear in the LPI self and manager scores, but not in the LPI others scores.
Table 9 – Independent Samples T-test Relating Developmental Relationships to LPI Scores
| LPI category | Experiencedavg. score | Not experiencedavg. score | T-test | Significance |
| Challenging the process (self) | 45.89 | 40.30 | -2.835 | .006 |
| Inspiring a shared vision (self) | 46.11 | 37.48 | -3.242 | .002 |
| Enabling others to act (self) | 48.56 | 45.20 | -2.039 | .045 |
| Modeling the way (self) | 47.85 | 42.52 | -3.210 | .002 |
| Encouraging the heart (self) | 46.07 | 39.64 | -2.816 | .006 |
| Challenging the process (manager) | 43.17 | 37.89 | -1.659 | .103 |
| Inspiring a shared vision (manager) | 44.29 | 37.03 | -2.202 | .032 |
| Enabling others to act (manager) | 46.04 | 40.17 | -2.001 | .050 |
| Modeling the way (manager) | 48.25 | 41.77 | -2.140 | .037 |
| Encouraging the heart (manager) | 44.25 | 37.69 | -2.408 | .019 |
| Challenging the process (others) | 39.21 | 36.71 | -0.823 | .413 |
| Inspiring a shared vision (others) | 39.86 | 36.89 | -0.961 | .340 |
| Enabling others to act (others) | 44.58 | 42.63 | -0.886 | .379 |
| Modeling the way (others) | 44.05 | 41.03 | -1.088 | .281 |
| Encouraging the heart (others) | 42.01 | 39.27 | -1.219 | .227 |
Conclusions
The data above reports that a developmental job assignment is the only leadership development experience (or combination of experiences) that makes a significant difference from all rater perspectives related to Kouzes and Posner’s (1995) Five Practices of Exemplary Leaders. With the singular exception of t-test self scores in Table 5 relating a developmental job assignment to challenging the process, participant and their observers generally seem to concur that a leader’s competency to inspire a shared vision is best enhanced through on-the-job training that contains assessment, challenge, and support. Such a developmental job assignment seems to provide the appropriate organizational context for participants to practice exemplary leadership.
Additionally, the data seems to reveal various perspectives regarding the relationship of developmental relationships to exemplary leadership practices from participant (self) and manager rating perceptions compared to others (direct reports and coworker) perceptions.
The data also seems to indicate that no other type of leadership development (or combination of types) has a consistent significant relationship across all scores (positive or negative) to any of Kouzes and Posner’s (1995) Five Practices of Exemplary Leaders. With the exception of a developmental job assignment and developmental relationships, participants (self) generally seem to indicate a more positive perception than their managers, direct reports and coworkers regarding the relationship of their own leadership development experiences to exemplary leadership practices.
Recommendations for Further Research
This study hopefully serves as a catalyst for additional research regarding the relationship of leadership development to exemplary leadership practices. First, similar research employing a larger, randomized sample population would provide helpful data to enhance the conclusions of this study. Second, whereas this study developed self-typing paragraphs to quantify participants’ leadership development experiences that reflected the elements of assessment, challenge, and support, additional research is needed to assess the quality of assessment, challenge and support participants received through their leadership development experience(s). Third, the field of leadership development would benefit from a longitudinal study employing a control group and experimental groups (representing the five types of leadership development experiences) to observe the relationship of leadership development experience(s) to exemplary leadership practices. Fourth, studies are needed that investigate the potential effect of participants’ organizational role (executive, director, officer, or manager) on the relationship of leadership development experiences to exemplary leadership practices. Fifth, further research is needed to study the effect of participants’ region (culture) on the relationship of leadership development experiences to exemplary leadership practices. Six, additional quantitative studies are needed that investigate the reasons for the consistently significant positive relationship of developmental job assignments to exemplary leadership practices and the generally inconsistent relationship of all other types (and combination of types) of leadership development to exemplary leadership practices. Seventh, additional studies are needed that probe the reasons that developmental relationships are consistently viewed more positively related to exemplary leadership practices by participants and their managers than by other raters (direct reports and coworkers).
References
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Upper Saddle River, NJ: Prentice-Hall.
Bass, B. M. (1990). Bass and Stogdill’s handbook of leadership: theory, research, and managerial applications(3rd ed.). New York: Free Press.
Brungardt, C. L. (1996). The making of leaders: A review of the research in leadership development and education. The Journal of Leadership Studies, 3(3), 81-95.
Bunker, K., & Webb, A. (1992). Learning how to learn from experience: Impact of stress and coping. Greensboro, NC: Center for Creative Leadership.
Burke, M. J., & Day, R. R. (1986). A cumulative study of the effectiveness of managerial training. Journal of Applied Psychology, 71, 242-45.
Chappelow, C. T. (1998). 360-degree feedback. In C. D. McCauley, R. S. Moxley, & E. Van Velsor (Eds.), The Center for Creative Leadership handbook of leadership development (pp. 29-65). San Francisco: Jossey-Bass.
Clark, K. C., Clark, M. B., & Campbell, D. P. (Eds.). (1992). Impact of leadership. Greensboro, NC: Center for Creative Leadership.
Collins, D. B. (2001). Organizational performance: The future focus of leadership development programs.Journal of Leadership Studies, 7(4), 43-58.
Conant, J. S., Mokwa, M. P., & Varadarajan, P. R. (1990). Strategic types, distinctive marketing competencies and organizational performance: A multiple measures-based study. Strategic Management Journal, 11(5), 365-383.
Conger, J. A., & Benjamin, B. (1999). Building leaders: How successful companies develop the next generation. San Francisco: Jossey-Bass.
Drew, R., & Wah, L. (1999, October). Making leadership development effective. Management Review, 8.
Engelbrecht, A., & Fischer, A. (1995). The managerial performance implications of a developmental assessment center process. Human Relations, 48, 387-404.
Funderburg, S. A., & Levy, P. E. (1997). The influence of individual and contextual variables on 360-degree feedback system attitudes. Journal of Occupational and Organizational Psychology, 67, 57-67.
Gabarro, J. (1987). The dynamics of taking charge. Boston: Harvard Business School Press.
Giber, D., Carter, L., & Goldsmith, M. (2000). Best practices in leadership development handbook. San Francisco: Jossey-Bass.
Goldstein, I., & Associates. (1989). Training and development within organizations. San Francisco: Jossey-Bass.
Guthrie, V. A., & Kelly-Radford, L. (1998). Feedback-intensive programs. In C. D. McCauley, R. S. Moxley, & E. Van Velsor (Eds.), The Center for Creative Leadership handbook of leadership development (pp. 66-105). San Francisco: Jossey-Bass.
Huber, D. L., Maas, M., McCloskey, J., Goode, C. J., & Watson, C. (2000). Evaluating nursing administration instruments. Journal of Nursing Administration, 30(5), 251-272.
Iles, P. (1993). Achieving strategic coherence in HRD through competence-based management and organizational development. Personnel Review, 22(6), 63-80.
James, W. L., & Hatten, K. J. (1995). Further evidence on the validity of the self typing paragraph approach: Miles and Snow strategic archetypes in banking. Strategic Management Journal, 16, 161-168.
Jurkowski, E. T. (1997, January). Leadership/community participation in resource planning/development for individuals with physical disabilities. Unpublished doctoral dissertation. Chicago: The University of Illinois at Chicago.
Kouzes, J. M., & Posner, B. Z. (1995). The leadership challenge: How to get extraordinary things done in organizations. San Francisco: Jossey-Bass.
Leong, F. T. (1995). Review of the Leadership Practices Inventory. In J. C. Conoley & J. C. Impara (eds.), The twelfth mental measurements yearbook (pp. 555-556). Lincoln: University of Nebraska Press.
Lombardo, M. M., & Eichinger, R. W. (1989). Eighty-eight assignments for development in place: Enhancing the development challenge of existing jobs. Greensboro, NC: Center for Creative Leadership.
Lynham, S. A. (2000). Leadership development: A review of the theory and literature. In P. Kuchinke (Ed.),Proceedings of the 2000 Academy of Human Resource Development annual meeting. Baton Rouge, LA: Academy of Human Resource Development.
Matthews, B. P., & Redman, T. (1997). The attitudes of service industry managers toward upward appraisal.Career Development International, 2, 46-53.
McCauley, C. D., & Douglas, C. A. (1998). Developmental relationships. In C. D. McCauley, R. S. Moxley, & E. Van Velsor (Eds.), The Center for Creative Leadership handbook of leadership development (pp. 160-193). San Francisco: Jossey-Bass.
McCauley, C. D., Moxley, R. S., & Van Velsor, E. (Eds.) (1998). Handbook of leadership development. San Francisco: Jossey-Bass.
McDonald-Mann, D. G. (1998). Skill-based training. In C. D. McCauley, R. S. Moxley, & E. Van Velsor (Eds.), The Center for Creative Leadership handbook of leadership development (pp. 106-126). San Francisco: Jossey-Bass.
Miles, R. E., & Snow, C. C. (1978). Organizational strategy, structure, and process. New York: McGraw-Hill.
Nolan, F. L. (1992, June). Ethical leadership and school culture: An exploratory study of nine middle-level schools. Unpublished doctoral dissertation. University of Minnesota.
Ohlott, P. J. (1998). Job assignments. In C. D. McCauley, R. S. Moxley, & E. Van Velsor (Eds.), The Center for Creative Leadership handbook of leadership development (pp. 127-159). San Francisco: Jossey-Bass.
Rouillier, J. Z., & Goldstein, I. L. (1990). Determinants of the climate for transfer of training. Paper presented at the meeting of the Society of Industrial and Organizational Psychology, St. Louis: MO.
Shortell, S. M., & Zajac, E. J. (1990). Perceptual and archival measures of Miles and Snow’s Strategic Types: A comprehensive assessment of reliability and validity. Academy of Management Journal, 33(4), 817-832.
Storey, J., Mabey, C., & Thomson, A. (1997, June 12). What a difference a decade makes. People Management, 289-330.
Van Velsor, E., McCauley, C. D., & Moxley, R. S. (1998). Our view of leadership development. In C. D. McCauley, R. S. Moxley, & E. Van Velsor (Eds.), The Center for Creative Leadership handbook of leadership development (pp. 217-241). San Francisco: Jossey-Bass.
Winston, B. E. (1998). Configurations of marketing strategy and technology adoption strategies: Do they affect profit performance? Unpublished doctoral dissertation, Virginia Commonwealth University, Richmond, Virginia.