Program Evaluation
Evaluation allows the organisation and the participants to consider if the peer support program is meeting its objectives and should be viewed in an open, positive light. You need to formulate an evaluation plan covering both ongoing monitoring and regular reviews (about every two years). There needs to be commitment from all stakeholders to improvement, both short-term and long-term. The scope of the evaluation will depend somewhat on the number of volunteers and peers. With larger programs, the review process will most likely need to be more structured. Participants and stakeholders need to be agreeable to the process from the outset. Resources, especially time and a budget, need to be allocated for the conduct of a review. Your record keeping practices need to support ongoing monitoring and reviews.
The scope of any evaluation should be clearly defined in the planning stage. You may seek to answer questions about the processes you are following in peer support, things like:
- Are processes for recruiting volunteers being followed?
- Do all volunteers receive training per our procedures? Are they finding that it meets their needs?
- How long do peers wait to actively participate in the program after they have expressed an interest in being involved?
- Is the coordinator able to keep the necessary records? Are they proving useful in planning system improvements?
You may also wish to seek feedback on the outcomes of the program, such as:
- How many people are involved as peers and as volunteers? Is this more or less than you had planned?
- What benefits do peers tell you they are experiencing as a result of their involvement in your program (health, service use, well-being etc.)? How might this be linked to sourcing of future funding?
6.1 Who is involved in the evaluation of the program?
There should be opportunities for all stakeholders, or their representatives, to be involved to a greater or lesser extent in the program evaluation. Consider using the evaluation as a way of seeking external input on the program – from health professionals and others who may refer new members, and from other potential new members of the program, too.
The responsibility for ongoing evaluation will usually lie with the coordinator. With the conduct of a formal review, it is recommended to form a panel consisting of representatives of the volunteers, peers and management; it may also facilitate the process to appoint or involve someone who is independent of the program.
6.2 When to evaluate
There should be ongoing evaluation to improve processes especially in a developing program. Some peer support partnerships lend themselves to the cycle of “establish – activate – conclude – evaluate”. It is advisable to conduct a formal review about every two years.
6.3 Areas to be considered
With ongoing evaluation occurring, modifications to the program, mostly operational, will occur. The following areas are likely to need to be considered in a more formal review:
- Types of peer support offered,
- Coordination and governance structures,
- Support given to the program by the governing body,
- Recruitment of both volunteers and peers,
- Selection and screening of both volunteers and peers,
- Training of volunteers,
- Job descriptions,
- Operational and communication issues,
- Record keeping practices,
- Resource issues, including financial arrangements, and
- Public relations.
6.4 Collection of data
You can collect data through the use of surveys, focus groups, interviews and statistics (drawn from the records). Comparisons with other peer support programs can be made or use the key factors contained in this framework to serve as a checklist. If possible, both quantitative and qualitative data should be collected on an ongoing basis. You need to reassure individuals that their feedback will be treated in a confidential, respectful manner. Often with sensitive opinions, the material needs to be aggregated.
Possible sources of data are:
- Statistics regarding requests for peer support (nature, frequency, source),
- Volunteer availability and retention rates,
- Feedback from volunteers and peers throughout the partnership,
- Feedback from volunteers at the conclusion of training,
- Statistics and feedback from referring agencies,
- Numbers of compliments and complaints, and
- Feedback from the coordinator and the governing body.
6.5 Evaluation findings
For political and financial reasons, you may find that it is important to provide regular updates, to the governing body, on the development of the program. The findings and recommendations of the review should be reported back to the manager or governing body. Participants who are involved in the evaluation should have access to the outcomes. Responsibility for implementing change should be clearly defined. Consultation with stakeholders should occur prior to the introduction of any changes. Celebrate successes!
Key factors for successful monitoring and supporting of the program are:
- Do you have an accepted systematic process of ongoing evaluation?
- Do regular reviews occur?
- Does the evaluation process reflect the objectives of your program?
- Is there a person nominated to be responsible for the evaluation process?
- In evaluation do you consider both outcomes and processes?
- Is there an established system of record-keeping to support evaluation?
- In evaluation, do you use a range of data?
- In evaluation, do all stakeholders (or their representatives) have the opportunity to contribute?
- In evaluation, are all levels of the program covered – peers, volunteers, coordinator and any other paid support staff?
- Are the findings of the evaluation reported to all relevant stakeholders?
- Have recommendations from the evaluation or review been enacted?
