Element 5: Developing a Program and Evaluation Plan
Creating a program plan
A program plan links to the strategies or broad approaches to facilitating change identified in the program plan. For example, “building healthy public policy” or “creating supportive environments” are strategies. Each strategy contains one or more activities. Planning terms may vary between organizations. One common point of confusion is the difference between “tasks” and “activities.”
According to THCU, activities are products or services that are made or held for a given audience, such as an event, a phone-in counseling service or a self-help group. Output is another fairly common term that may be used to replace activity.
Tasks are different from activities. Each activity may require multiple tasks. Tasks are a part of operational work plans. They are assigned people, resources and deadlines. They are required to make activities happen. Tasks are sometimes called action steps. Tasks include such things as hiring a consultant, researching potential speakers or booking meetings. Tasks are operational steps or actions that contribute to the development of activities for an audience.
Priority setting and decision making:
It is important to prioritize efforts. Where will resources and efforts have the most impact? What projects will require a longer-term investment? It’s important to be realistic about what can be accomplished with the human and financial resources available.
Not all activities will be compatible with the culture of an organization. Be critical when choosing activities. Many practices that are used to promote mental health are also strategies for supporting employees who have mental illness. Therefore, one benefit of implementing many of these programs is that they also aid people with mental health issues. They may allow them to be successful without requiring them to disclose their illness.
Ideally, healthy workplace plans are flexible and allow individuals to indicate the changes or supports they need in order to work well and be productive. Flexible work programs support everyone within the workplace, they enable every employee to be more productive, less stressed and have overall better mental health. The majority of Canadians are reporting higher levels of burnout, major depression, anxiety and other mental health problems. Mentally healthy workplaces provide mentally healthy options and programs for all employees. By making these programs available to everyone, employees with mental illness can access them without disclosing and potentially exposing themselves to stigma and discrimination.
Creating an evaluation plan
Evaluations of CWHP initiatives are carried out to:
- determine the effectiveness or impact of a program (i.e., the extent to which program goals and objectives were met);
- be accountable to program stakeholders such as the employers, employees, and other groups with an interest in the CWHP initiative,
- identify ways of improving a program, such as ensuring that all activities are relevant and appropriate to the health needs of employees and removing potential barriers to participation;
- assess the economic efficiency of a program through costs benefit or cost effectiveness analysis;
- compare a CWHP program with similar initiatives being implemented elsewhere;
- use the evaluation information for a range of purposes, including promotion, fundraising, attracting potential partners and advocating for policy changes to create healthier workplaces.
An evaluation plan should be created before evaluation has started. It must consider both formative and summative evaluation.
Formative evaluation plans must be created during the program planning phase, before implementation.
Summative evaluation is completed at the end of the project. This shows whether the program was successful and can be used to decide whether it should continue.
There are three types of evaluations to consider, not all of which will necessarily be used for every program.
|Type of Evaluation||Description||Examples of what it evaluates|
Formative evaluation measurement can help improve the program during and after the implementation. Summative evaluation measures provide the proof that the program works.
|Formative Evaluation – Improve||Summative Evaluation – Prove|
|Provides information that helps improve the program. Generates periodic reports. Information can be shared quickly.||Generates information that can be used to demonstrate the results of the program to funders and the community.|
|Focuses most on program activities, outputs and short-term outcomes for the purpose of monitoring progress and making mid-course corrections when needed.||Focuses most on the program’s intermediate-term outcomes and impact. Although data may be collected throughout the program, the purpose is to determine the value and worth of a program based on results.|
|Helpful in bringing suggestions for improvement to the attention of staff.||Helpful in describing the quality and effectiveness of the program by documenting its impact on participants and the community.|
When conducting evaluations, practitioners should think about expected outcomes, how achievements will be measured and the role of the workers themselves in program planning, design, implementation and evaluation. In order to begin an evaluation plan, effective process and outcome objectives need to be created.
Indicators also need to be set. Indicators are the units of measurement used to assess the extent to which objectives have been met. Outcome indicators help you decide whether your program is effective and successful. Process indicators help you decide whether your program was implemented in the way you intended.
Indicators are a building block for a comprehensive evaluation plan. They should be reliable, valid and accessible. A mix of health and business indicators should be used and if organizational levels of change have been included, indicators regarding the workplace culture, including the psychosocial environment, will be necessary.
Working life indicators and measures
|Stress||Percentage of adults who find their job very or extremely stressful|
|Work-life balance||Mean score for how satisfied adults are with their work-life balance (paid work)|
|Demand||Percentage of adults who often or always have unrealistic time pressures at work|
|Control||Percentage of adults who often or always have a choice in deciding the way they do their work|
|Manager support||Percentage of adults who strongly or tend to agree that their line manager encourages them at work|
|Colleague support||Percentage of adults who strongly or tend to agree that they get the help and support they need from colleagues|
There are many things to think about when developing an evaluation plan, only some have been explored in this Info-Pack. To learn more about the steps to evaluating a comprehensive workplace health promotion plan, review THCU’s Evaluation Info-Pack at www.thcu.ca/workplace/documents/EvaluationInfoPackFinalWeb.pdf.
Developing a work plan
An organization’s healthy workplace committee is responsible for developing a detailed work plan. The work plan should outline the program objectives, activities and evaluation methods in each phase of the plan. Developing a plan is the key to successful programs. The program work plan needs to be revisited regularly to check on progress and to make any necessary modifications.
A healthy workplace committee used an organizational culture audit ‑ a type of situational assessment. They held focus groups. One of the issues flagged was that employees felt their stress levels were almost unmanageable. Many felt they were not being supported by management. Employees said they dreaded coming to work and often spent long hours answering emails on their off hours. They would often call in sick to avoid these feelings. As a response to the assessments, the healthy workplace committee set a goal of creating a more resilient and mentally supportive workplace and decided to tackle three issues: 1) individual stress management skills, 2) management support skills, and 3) workplace policies around workload.
Online Health Program Planner provides additional information and helps users develop a full, comprehensive plan. See www.thcu.ca/ohpp.
The following examples are from a comprehensive workplace mental health promotion plan. They are practices that can promote positive mental health within an organization. The examples are matched with their corresponding psychosocial risk factor, which is an element that affects employees’ mental responses to work and the work environment and have the ability to cause mental health problems.
In Canada and the rest of the world, not enough research has been conducted to find the workplace interventions that are good practices, those that work well and those interventions that do not work as intended. The following components address the PSR factors identified by Guarding Minds @ Work. They are based on workplace health promotion concepts. For example, social and environmental support is extremely beneficial when trying to promote behaviour change. The following activities have been generated through anecdotal examples of programs that have worked well within various organizations, from research and from the activities provided by Guarding Minds @ Work PSR Action Sheets., As outlined earlier, multiple strategies should be used to make sure that the program is comprehensive, reaching the maximum number of employees and making a sustained cultural change. Also, from the onset of the program, an organization should evaluate and share its findings. More Canadian groups need to contribute to the evidence base for good practices in workplace mental health promotion.
|Psychosocial Risk Factors||Activities|
|Organizational Culture: A work environment characterized by trust, honesty and fairness.||
|Clear Leadership and Expectations: A work environment where there is effective leadership and support that helps employees know what they need to do, how their work contributes to the organization and whether there are impending changes.||
|Civility and Respect: A work environment where employees are respectful and considerate in their interactions with one another, as well as with customers, clients and the public.||
|Psychological Job Fit: A work environment where there is a good fit between employees’ interpersonal and emotional competencies, their job skills and the position they hold.||
|Growth and Development: A work environment where employees receive encouragement and support in the development of their interpersonal, emotional and job skills.||
|Recognition and Reward: A work environment where there is appropriate acknowledgement and appreciation of employees’ efforts in a fair and timely manner. These recognition practices make employees feel valued and that their effort is noticed.||Employer Recognition
|Involvement and Influence: A work environment where employees are included in discussions about how their work is done and how important decisions are made.||
|Workload Management: A work environment where tasks and responsibilities can be accomplished successfully within the time available.||
|Engagement: A work environment where employees enjoy and feel connected to their work and where they feel motivated to do their job well. This helps increase feelings of control, alignment or buy-in to the company’s mission and increases employees’ sense of pride in their work.||
|Balance: A work environment where there is recognition of the need for balance between the demands of work, family and personal life. These practices can help reduce work-conflict and unhealthy stress and can improve productivity.||
Alternative Work Schedules
|Psychological Protection: A work environment where employees’ psychological safety is ensured.||Psychological Supports
Situation: To reduce turnover, Lee Valley wanted to show employees that there was room for growth within the organization.
Action: Before jobs are advertised publicly, Lee Valley posts them internally. This gives employees first crack at these positions and many people have grown and developed their careers through this feature. For employees who want to further their education, Lee Valley has an education policy, paying the tuition as long as it has some relevance to their job or is along the lines of professional development.
Result: Employees see that opportunities for growth and development come in many shapes and forms and many employees have grown and developed their careers at Lee Valley through these measures.
To read the whole case study visit http://www.vifamily.ca/library/social/lee_valley.pdf.
- Canadian Best Practices Portal for Health Promotion and Chronic Disease Prevention, by the Public Health Agency of Canada, is a database of evaluated, best practice interventions. See www.phac.gc.ca/cbpp.
- Logic Model Workbook, by THCU, provides an overview of logic models including what they are, why they are important and how to use their four-step approach to creating program logic models. See http://www.thcu.ca/infoandresources/publications/logicmodel.wkbk.v6.1.full.aug27.pdf.
- From Strategic Plan to Program Plan and Back Again contains answers to important questions such as “Who should be involved in the planning process?” “Where does strategic planning stop and program planning begin?” “How do you keep the plan realistic?” “When should indicators be included in a plan?” “Are there templates or supports to develop a strategic plan on our own?” See http://www.thcu.ca/infoandresources/resource_display.cfm?resourceID=958&emailID=527 .
 Desjardins Financial Security. “Health is Cool! 2006 Survey on Canadian Attitudes towards Physical and Mental Health at Work and Play.” (2006) http://www.dsf-dfs.com/en-CA/_Utilitaires/Prmtns/HlthCl.htm (accessed December 2, 2009).
 THCU. “Comprehensive Workplace Health Promotion Evaluation Info-pack.” (2005) p. 6,7 http://www.thcu.ca/workplace/documents/EvaluationInfoPackFinalWeb.pdf (accessed February 12, 2010).
 Guarding Minds @ Work, www.guardingmindsatwork.ca
 Guarding Minds @ Work, “Action Sheets,” (2009) www.guardingmindsatwork.ca http://www.guardingmindsatwork.ca/resources.aspx.
 Elaine Lowe, “Working by the Golden Rule, Lee Valley Tools,” The Vanier Institute of the Family (2005), http://www.vifamily.ca/media/webfm-uploads/Publications/SocialInnovations/leevalley.pdf (accessed October 07 2009).