Effectiveness in Communication, Education & Training

Organizations and individuals today are saturated with communication, education and training. Nevertheless, we are confronted daily with situations indicating that the lack of communication, education and training keeps causing misbehavior, mistakes and low performance.
This shows that the traditional ways of assessing CE&T, as for example, the number of hours per employee, is not good enough. What really matters is the Effectiveness in promoting the desired changes in the business, not the number of hours people have been subject to CE&T.
Focusing on the effectiveness of CE&T leads us to a significant change in our use of CE&T, including the analysis of how the different instruments of CE&T interact and integrate to produce the desired changes.
Kirk Patrick and Phillips are two important contributors to the development of methodologies to assess effectiveness of C, E & T. Jeanenne LaMarsh and Rebecca Potts (2) have developed what I consider as the best approaches and methodology to manage changes. This article extensively benefits from their ideas and proposals.
Communication, Education and Training are all extremely valuable instruments and the dedication of attention by the employees is an even more precious element. We cannot afford using them without a clear purpose and without making sure they are being effectively used for the defined purpose.
As a principle, the objective of every and all CE&T actions needs to be justified with an objective of promoting change: change of values, beliefs, attitudes, behaviors, knowledge and/or skills. From a business perspective these changes must result in reliability, better decisions and performance.
When we force a clear definition of which changes are targeted with an action of CE&T, we promote a more responsible approach to it and we make it easier and more accurate the measurement of the effectiveness of the action.
Having a clear definition of the targeted change and of how it will be measured helps us select the most appropriate combination of actions in each case.
Measuring the effectiveness enables us to identify deviations and to learn and improve over time.
In summary, we will achieve a lot more with possibly less investment in C, E & T.
What follows equally applies to all modes in which we do CE&T.
Top-down, bottom-up, horizontal, 360º, coaching, lecturing, self-studying and other ways should all be challenged as to which changes are targeted and how effectively changes are being promoted.
Five Levels of Measurement
Adapting from Kirk Patrick and Phillips (1) proposals, the effectiveness of CE&T can be measured at 5 different levels:
Level 1 - the Quality of the event: contents, instructor, room, duration, etc. This is most frequently done through a perception survey filled by the participants at the end of each event.
Level 2 - Acquisition of knowledge and skills. This can be measured with before / after tests focused on what is targeted to change.
Level 3 - Effect on the job - how the level 2 changes produced improvements to the measured performance on the job: productivity, rate of defects, customer satisfaction.
Level 4 - Effect on the business - this is the valution, in business terms, of the level 3 effects. For example, how much money was gained or saved by the reduction in rework hours, the increase in throughput or decrease in product cost, as a result of the increased productivity of the participant, observed on the job after the event.
Level 5 - Return on the CE&T Investment. Having a monetary valuation of the benefits, analyze it against the cost of the CE&T actions.
Case as Illustration
The following case explains and illustrates the process and the main points that need to be considered.
Through the Strategic Management process, Velocity was identified by CompanyX as a desired competitive advantage and as a critical success factor; then it was converted into a Strategic Objective for CompanyX.
Through the deployment of the Strategic Objectives, the Planning & Logistics organization was confronted with the need to significantly improve the performance of material inventory management, in order to contribute to the Velocity strategic objective.
As the manager of Planning analyzed the performance of the Material Planner Stephanie, he verified that the following changes should be targeted:
safety stocks for the items managed by Stephanie then represented $20 million. Given the demand and supply characteristics of the items under her responsibility and the service level objectives, $10 million would be the ideal level; therefore a change from $20 to $10 should be promoted;
the service level then provided by most of the items managed by Stephanie ranged from 60 to 70%; a minimum of 95% was considered necessary to support production; an improvement of at least 25-35% was to be obtained;
other changes were identified as required but are disregarded in this article to simplify the explanation.
A similar analysis was made for the colleagues of Stephanie and for other related functions. This illustration will proceed with the Stephanie’s example alone.
The following actions were identified as necessary to produce the desired changes:
a.  education on the basics of supply chain management;
b.  education and training on inventory management techniques;
c.  on the job coaching by the manager and by the internal expert on Inventory Management.
The investigation of the alternative ways to execute the actions led to the selection of two courses given by an external entity as the best ways to teach the basics of supply chain and the techniques for inventory management. When discussing with the external entity the tailoring of these courses, the specific objectives targeted were explained and the entity was challenged as to the ability to produce the specified changes.
The targeted changes were made clear to the external entity and to Stephanie as well as how they would be measured afterwards.
The appropriate internal Coaches were identified, thus completing the solution package.
The internal Coaches and Stephanie agreed on how the coaching should be conducted and which objetives should be accomplished.
Upon the execution of each course, performance was measured at Levels 1 and 2 (quality of the events and the acquisition of knowledge and skills).
The execution of Coaching was monitored against the plan, with a periodical review between coaches, Stephanie and her manager (who in this case was one of the coaches).
One month after the solution package (courses and coaching) was completed, the planned safety stocks were measured based on the parameters in the system.
This showed a sum of $13 million planned.
Some change was measured on service level: 80% of the items moved from the 60-70% range to 70-75%.
The effectiveness of the solution at that point was considered to be around 60% of the expectation; it was recognized that the increased maturity in the following months would improve the results but it would not be able to complete the desired changes.
An analysis of the causes of insufficient effectiveness was made and two of the causes identified were the lack of proficiency in the supporting systems and a need for better knowledge of the involved materials and of the way they moved in the chain from the suppliers to the point of use in production.
An additional program was defined including systems training and an ‘walk through’ of the supply-and-usage chain of the different types of materials.
One month after the extended solution was completed, the parameterized safety stocks in the systems were measured again and amounted to $9.2 million, while the service level had increased to 85-95%. It was then assessed that service level would continue to increase as the actions taken in inventory management and procurement would become effective in the following 3 months.
It was concluded that the extended solution achieved the desired changes and potentially exceeded them ($9.2 currently planned vs. $10 millions target safety stocks).
The Level 3 effectiveness measurement was therefore considered 100%, to be confirmed once service level ans safety stocks would mature and demonstrate to be sustainable at or better than target level. (Remembering: Level 3 is the effectiveness in producing the desired changes in the performance at the job).
The performance in the job was then translated into the impact on the business:
a.  one time cash flow release of $10.8 million;
b.  savings in the cost of carrying inventory ($10.8 * 10%) = $1.08 million/year;
impact of improved service level: reduction of emergency actions, expediting, special freight, reduction of disruption to the processes that use the items, reduction of missed shipments to customers, etc.). The quantifiable effects in this item were valued at $3 million per year.
The total impact on the business was estimated at $10.8 million onetime cash relief plus $4.08 million per year. This was considered the effectiveness measurement at level 4 (impact on the business).
The impact on the business was then analyzed against the cost of the solution.
Because education & training needs periodical reinforcement and there is some expected turnover of personnel, a conservative approach considered only the benefits of one full year.
Costs were estimated as follows:
a.  Courses given to Stephanie:
b.  Coach time (2h/week * 12 weeks = 24h)
c.  Stephanie’s time (108h)
$ 4,900
$ 2,700

$ 2,400
Roughly calculating, the investment on Stephanie returned 4,080,000 / 10,000 = 40,800%.
The investment paid for itself since the moment she first applied the learned concepts and techniques to the parameterization of Safety Stocks. It was considered that payback time was around 1 month from initial costs.
The percentage of return on investment and the payback time were considered as the Level 5 performance measurement: Return on E&T Investment.
Measuring effectiveness of CE&T is clearly underutilized and, as Demin and Juran repeated many times, measuring is a pre-requisite to improving.
As with anything involving subjective aspects, the accuracy of the measurement of effectiveness of CE&T will always be subject to debate. Although improving the ways of measuring to obtain increased accuracy would be welcome, focusing on accuracy is the sure way to preclude the benefits.
We have observed an important change in attitude from simply challenging people with questions like:
why are you communicating this or giving that class? Is it simply to inform people?
which are the actual changes you want to promote in values, beliefs, attitudes, behaviors, knowledge and/or skills?
have you thought of all the actions that need to be combined in order to promote the desired changes? Have you thought of how to best integrate them?
are those actions the best combination and do they ensure the achievement of the desired changes? Since we do not want to just ‘do the best we can’, how can you make sure the changes will happen and be sustained?
how can we verify that the changes are being achieved?
have we gotten designers, executors and target people of CE&T to agree on the objectives, on the means and on the measurements?
Also, simply stating that effectiveness will be measured is sufficient to promote a more careful design of the actions and an increased attention by those delivering and by those receiving the CE&T.
Approaching CE&T as an instrument of change management, that will be held accountable for effectiveness, does help increase the performance of people and to convert the actions into actual business results.
Handbook of Training Evaluation and Measurement Methods, 3rd edition, Jack J. Phillips, Butterworth-Heinemann.
Master Change, Maximize Success, Rebecca Potts and Jeanenne LaMarsh, Chronicle Books, San Francisco.
Transformation Through People
English Version
Parceiros Estratégicos:
Links Recomendados:
APICS - The Association for Operations Management
LaMarsh Global
ABAI - Associação para Educação em Administração Empresarial
GPS Vale - Consultores Associados
LFC Management
Modus Logística Aplicada
TGT Consult
Trans4mar Consultoria
ACPM - Association of Change Management Professionals
TODAY - Logistics & Supply Chain
XYZ Consultoria

Conecte-se com a T2People®