Part 2: For evaluators and evaluation commissioners
‘Good evaluation means people with disability are to be consulted at multiple stages throughout the design process of the evaluation, initially before anything is designed, looking at what the goal is, what the information they’re trying to get out of the evaluation is and helping to suggest ideas of going about this, questions to ask, etc. and then being able to come back and see the draft of what’s being created, give feedback on that’.
The information in this section is not intended to tell evaluators and evaluation commissioners how to evaluate. It is intended to support good practice from the start so that evaluation processes uphold the rights of people with disability and address barriers to inclusion.
This section gives key information and helpful resources. It is designed to help evaluators and evaluation commissioners plan for inclusive and accessible evaluation processes. More detailed information can be found in the fact sheets, links, and resources throughout.
The benefits to involve people with disability in evaluation
People with disability have been clear that they want to be involved in evaluation. 1 in 5 Australians identify with disability. This number can be much higher in some communities and groups of people. People with disability have the right to be involved in making decisions about their own lives. They should also be able to speak up and influence things that affect them.
When people with disability are involved in all stages of an evaluation, it’s more likely to focus on what really matters for the people affected. It helps make sure the right questions are asked, in the right way and to the right people.
There are many benefits to involving people with disability, including:
- supporting stronger findings by including a wide range of points of view, experiences and expertise
- promoting inclusion and equity
- more believable and trustworthy outcomes
- building confidence and skills for both people with disability and evaluators. Some evaluators may be people with disability
- supporting shared ownership of the outcomes, including with the disability community
- saving time and money in the long-term by improving design and testing of programs that impact people with disability.
To fully realise these benefits, people with disability should be leaders and have the power to make decisions from the beginning.
‘Relationships grounded in collaboration, mutual respect, and shared decision-making are key to ethical co-production. This requires a person-centred approach addressing power dynamics and accessibility so that the perspectives of co-researchers are valued and included’.
Planning and funding evaluation
You can plan from the beginning for people with disability to be a part of shaping the proposed goals of an evaluation. To do this, the tender or proposal should clearly say that evaluators need to make accommodations or adjustments. A tender is the process of asking organisations to formally submit an offer to provide goods or services. Accommodations or adjustments might include changing the location or the way the evaluation is done to help remove barriers and make sure everyone can take part equally. This may include paying people with disability for their time and work. It also means finding different ways to involve people, so everyone has a chance to take part. A good resource to help is the Good Practice Guidelines for Engaging with People with Disability.
Include people who have lived experience of disability in all stages of the process. This means involving them as evaluators and making sure they are part of decision-making groups. You can set up co-chairing arrangements, where leadership is shared. You can look for people with disability who are subject matter experts on the evaluation topics and methods.
It is important to include multiple people with disability as participants. This helps evaluators collect information from a wide range of views, experiences, and knowledge.
Why take action to make evaluation accessible
To get the most benefit of including people with disability in all stages of an evaluation, planning needs to start early. Evaluators and evaluation commissioners should think about a few key things from the beginning. This includes accessibility, inclusivity, staff capability and ethics.
Share power
The need for real collaboration is something people with disability often raise in feedback. Sharing power and working together to design and run evaluations helps make this happen. It makes sure people with disability, and other diverse groups, are truly involved. These ways of working are called co-design and co-production.
Co-design and co-production should be standard for evaluation processes. However, sharing power equally is not always possible. For example, sometimes experts need to make final decisions. In these cases, people can still work together and have a say during an evaluation. But if there is not equal power sharing, the processes can’t be called co-design or co-production.
You can learn more about collaboration in Fact sheet 2: Collaborating in evaluation. It includes definitions of co‑design and co-production.
Make sure activities are accessible
Working with people with disability at all stages of evaluation goes beyond just meeting the rules about accessibility. However, these resources are a good place to start:
- Web Content Accessibility Guidelines (WCAG)
- Good Practice Guidelines for Engaging with People with Disability.
All evaluation documents and processes should be accessible and inclusive. Engagement activities should give people with disability equal opportunity to participate and support the Universal Design Principles.
The Australian Human Rights Commission have guides that give extra advice on making activities accessible:
- Hosting accessible and inclusive in-person meetings and events
- Hosting accessible and inclusive online meetings and events.
Provide accommodations and adjustments
Even when engagement activities are planned to be inclusive and accessible, some people might need extra help to take part equally. These extra supports are sometimes called accommodations or reasonable adjustments.
Accommodations or reasonable adjustments are types of support or changes that make spaces equal for everyone. They can reduce barriers that stop people from participating. These changes can help people with disability (or anyone who needs more support) take part equally.
When planning how to implement accommodations and adjustments, evaluators and evaluation commissioners need to think carefully about the language they use when asking people what support they need. Not every person who might need extra support will say they are a person with disability. People will find it helpful to see examples of what adjustments are available.
What can you do?
You can proactively and clearly explain what access features you can offer. You can also say that if anyone needs other accommodations or adjustments to take part, they can contact you to arrange them.
For example: ‘These are the access features we have on offer to support you. They include an accessible location, hearing loop, natural lighting and an Auslan interpreter. Please let us know what support you need to take part in the evaluation.’
This will help people know you are thinking about their different needs and are willing to meet these needs. This will also help build trust with people. It will help make people feel safe to ask for the supports they need.
This is particularly important when you consider the context around disability. Some people have faced stigma and discrimination in the past. Stigma is a negative belief people share about certain traits. Discrimination is when people are treated badly because of who they are. This can make people not want to say they have disability.
Other people might have different reasons not to identify with disability. For example, because of their culture. Dr Scott Avery explains in Culture is Inclusion, there is no word for disability in First Nations languages and many First Nations people don’t identify with Western terminology about disability.
You can learn more about adjustments and supports in Fact sheet 1: Adjustments and supports. It includes examples of the types of adjustments and supports that you can offer.
Disability confidence and awareness training
Think about the skill level of the team involved and if they could benefit from disability confidence and awareness training to help them:
- feel confident in supporting people with disability
- better understand the diverse experiences of people with disability.
If people have completed this training in the past, they might need to refresh their skills. Training works best if it is led by a person with disability. It should also be flexible so it can meet people’s different needs.
There are many Disability Representative Organisations and advocacy groups across the country that support people to become more confident about disability. Those organisations are trusted by the disability community. They can help share opportunities and connect evaluators to different voices and communities. They also often publish resources that can help build understanding of disability.
Reach out to local organisations and build relationships. These organisations can suggest local training and support. They can help connect you to local people with disability who can take part in evaluation. There are also some disability awareness training programs listed on the Job Access website.
There are online courses you can undertake and videos you can watch, such as SBS’s ‘Inclusion’ disability training program.
Griffith University offers 2 micro-credential training programs through its Inclusive Futures: Reimagining Disability initiative.
The first one is called Citizen Researcher Essentials: A Short Course. It is free, online and self-paced program. It helps people learn how to take part in research. It teaches topics like consumer involvement principles, ethics, collaboration and data handling.
The second course is called Partnering with Consumers in Research. It is designed for researchers, PhD students and people starting their career in research. It provides evidence-based training aligned with the National Health and Medical Research Council and Australian Research Council. It provides evidence-based training that focuses on:
- ways to engage consumers that is authentic
- research practices that are inclusive
- diverse ways to work together with disability
- First Nations partnerships
Respect the diversity of people with disability
People with disability across Australia have different identities, points of view and experiences. It is important to make sure to work with people with disability who are most likely to be affected by the policy, program or service being evaluated.
People are more than the definition of their disability. Disability is experienced in different ways. This goes beyond disability types. Evaluators need to consider how different parts of a person’s identity can overlap and affect their experiences. This concept is known as ‘intersectionality’. Intersectionality can include a person’s experience of:
- cultural and social background
- sex and gender identity
- sexual orientation
- disability
- level of literacy
- age.
Different intersecting parts of a person’s identity can strongly affect how they can take part in an evaluation. These parts – like disability, culture and age – can shape what someone needs or prefers when they are involved. You can learn more about making an evaluation safe and inclusive in Fact sheet 3: Safe and inclusive evaluation. You can also learn more about how to make sure an evaluation supports diverse perspectives in Fact sheet 7: Making sure evaluations include diverse views.
Be trauma-informed
Evaluators and evaluation commissioners need to remember that some people with disability may have experienced trauma. Because of this, it is a good idea to take a trauma-informed approach. Trauma-informed practice means making sure people feel safe, respected and supported. It also means building trust and giving people choices in how they take part. Evaluators and evaluation commissioners need to be able to:
- know the signs that someone has experienced trauma
- avoid re-traumatisation and
- make sure that everyone can get the support they need. This includes participants and the evaluation team.
You can learn more about trauma-informed approaches in Fact sheet 3: Safe and inclusive evaluation. You can also learn more about using a trauma-informed approach on the Australian Institute of Family Studies website.
Supported decision-making
To make sure informed consent and confident decision making, some people use supported decision-making. This could mean some people have formal arrangements about making decisions, like guardians or representatives. Other people might choose to ask trusted friends or family to help them decide. This is known as supported decision-making. To learn more about supported decision-making go to the Supported Decision Making Hub.
Recognition and remuneration
When people with disability are involved in a feedback process like an evaluation project, they should expect to receive fair recognition for their time, insights and expert knowledge.
You can learn more about remuneration in Fact sheet 4: Having your time and work recognised. It includes examples of current remuneration policies and guidelines and how much people are paid under these.
We’ve heard feedback from people with disability that they prefer to be paid for their work. The income gap between people with disability and people without disability was $480 per week in 2022, according to the Australian Institute of Health and Welfare.
People with disability have different preferences for how they want to be paid. Some people would rather get a gift card instead of having money transferred into their bank account. Some people like to be paid straight away, while others are happy to wait. The way that someone wants to be paid often depends on their personal situation. For example, it might affect payments or benefits they already get.
For some people with disability, out-of-pocket costs can be a barrier to taking part in an evaluation This can include costs for travel or meals. Think about whether the evaluation can cover reasonable out-of-pocket costs. Removing the financial barrier will make it easier for people with disability to join in with an evaluation. This makes it more equal with others.
Evaluators should make sure they say what out-of-pocket costs can be paid. It is important to be clear about this from the beginning to avoid miscommunication. This will make sure all people involved can:
- be confident that their time is valued
- make an informed decision about taking part
- have the chance to talk about any concerns.
Important factors to consider when planning for remuneration include:
- Payment rates should increase in line with the level of involvement. The more time, effort and thinking people contribute, the more they should be recognised for it.
- People contributing to the design of an evaluation strategy should be paid equally to the consultants or academics involved.
- Where possible, participants should be given options for how they are paid. This may include bank transfer, visa gift cards, taxi vouchers or gift cards for specific stores. You can learn more about renumeration in Fact sheet 4: Having your time and work recognised.
Although remuneration is dependent on the engagement activities and individual circumstances, it is important that evaluators take a consistent approach. The Good Practice Guidelines for Engaging with People with Disability provide advice on remuneration that evaluators can refer to. This includes guidelines around types of remuneration and factors that should be considered when deciding.
Ethical research requirements
Evaluation processes and outcomes must meet the highest standard of ethical conduct and treatment of participants with disability. This means evaluators and evaluation commissioners need to make sure activities are safe and inclusive. There are also formal ethics requirements that need to be considered.
All research in Australia that involves people must follow the:
- National Statement on Ethical Conduct in Human Research (2025) and
- Australian Code for the Responsible Conduct of Research.
This can be relevant to research methods used in evaluation, such as interviews, focus groups, workshops, data collection and analysis. You can learn more about these requirements in Fact sheet 6: Ethics.
Engaging people with disability at every stage
This Toolkit breaks evaluation into 3 main stages: plan, do, and use. The steps within these stages include scoping and design, data collection, analysis and interpretation of outcomes, reporting findings, and making good use of evaluation.
1. Plan the evaluation
- Early planning is best to make sure an evaluation is implemented well and meets expectations. While evaluation planning can take a variety of formats, a formal planning document should include:
- the extent the evaluation will partner with people with disability. You can learn more about collaboration in Fact sheet 2: Collaborating in evaluation
- an outline of governance structures. This should include how people with disability will be involved in decision-making and management processes. This should also outline accountability and responsibility
- an outline of resources and checks to inform the scope of the evaluation. This will inform how to make sure you pay people with disability fairly for their involvement. You can learn more about renumeration in Fact sheet 4: Having your time and work recognised.
- how the evaluation design should address any limits related to budget, time and data availability
- how the evaluation is designed to be inclusive and accessible so it is relevant and useful to people with disability.
Every stage of the evaluation should be carried out in a way that respects and meets the expectations of people with disability. Evaluation processes should help everyone who takes part to feel safe, included and equally supported. For example, evaluators should:
- commit to working with people with disability in a way that is sensitive, ethical and appropriate so that they can engage with dignity and respect. You can learn more in Fact sheet 3: Safe and inclusive evaluation and in Fact sheet 6: Ethics
- make sure people can get the information they need about the evaluation and they understand their role and contribution
- ask people if they need any adjustments and supports so they can fairly take part in meetings, workshops or other ways people share ideas. You can learn more in Fact sheet 1: Adjustments and supports
- identify at least 2 people from the evaluation team who participants can feel comfortable to communicate with for any questions or concerns
- make sure there is time to create and respond to follow-up communication to provide a feedback loop – a way to give feedback
- think about using an expert panel if expert views or specialised input and opinion is needed
- work together with disability specialists and disability organisations to facilitate activities. They are experts and might have built trust and relationships with participants. Think about what services are needed, find the right person or group, and be prepared to work together
- talk with the people involved and ask for feedback. This helps makes sure everyone shares the same values when working with people with disability. This will mean the contribution of people with disability will be central and genuinely valued
- involve the right people in the recruitment process – ideally led by a person with disability. People with disability can be recruited through multiple channels, including client organisations, interviews or peer networks.
Evaluation commissioners need to commit to genuine engagement with people with disability. This includes being flexible with timeframes, resources and training considerations to support people with disability to contribute to the evaluation.
2. Do the evaluation
Evaluators have a responsibility to make sure people with disability understand how data will be collected and why. It is also important to let people help decide what questions are asked for data collection. To make sure data is collected in an ethical and transparent way, evaluators should:
- consider a formal ethics review, which is important for managing risk of harm
- be clear and upfront about decisions and processes for data collection, including which parts are fixed and which can be influenced
- provide an accessible consent form and information in advance that explains what the evaluation is about and how information will be used. This way participants can provide informed consent to be involved. Make sure participants are aware they can withdraw at any time
- set up processes, such as working groups or advisory boards, that make sure people with disability are actively involved in data collection and analysis. These working groups or advisory boards can also include other relevant participants such as family members
- be aware of adjustments that can make different types of data collection activities more accessible. For example, it might be easier to call someone to complete a survey over the phone or allow them to provide their input by video.
Analyse data and understand findings
Consultation and engagement can be undertaken during the analysis and interpretation stage, and this analysis and reflection can be led and guided by people with disability.
Involving people with disability as co-producers in analysis and reporting can result in findings that are more likely to translate into real life improvements for people with disability.
When engaging people with disability in this stage, evaluators also need to:
- consider the experience and confidence of the people with disability involved, particularly in relation to analysis and interpretation of data. Offer to provide support and capacity building if this is something people are interested to learn about. Such as, training to give people the skills to do evaluative analysis and develop recommendations
- be mindful of cultural safety and engage with data analysis that thinks about bias, also using a strengths based approach
- consider the importance that analysis of data about First Nations people with disability is guided by First Nations people and informed by the Indigenous data sovereignty principles - MnW Principles — Maiam Nayri Wingara
- make sure analysis processes are made transparent to people with disability
- give people with disability enough time or supports to respond to invitations or requests for input.
3. Use the evaluation
Report findings
One of the key things people want when they get involved in evaluation is to hear about what happened and receive the evaluation findings in a format that they can understand. Following up and ‘closing the loop’ allows people to see how their efforts have been used and a way to encourage more people to get involved.
Some things to consider when reporting evaluation findings include:
- the outcomes and findings are widely shared with people directly involved, key community stakeholders, and the broader disability community
- the evaluation outcomes are communicated in a way so everyone can access, understand, and use the report
- it is clear how evaluations involving people with disability link the outcomes and findings to improvements in the lives of people with disability.
Make good use of the evaluation
Making good use of the evaluation includes:
- learning from it
- measuring outcomes
- showing the value of a policy, program or service.
People with disability who take part in evaluation have a right to know how evaluators have used their work and what the outcomes of the evaluation were. Sharing this information shows honesty and builds trust.
When considering how to make good use of evaluation findings and recommendations, evaluators need to think about:
- sharing de-identified responses openly across relevant forums and the disability community. This includes information about why some actions have been committed to and others have not
- making the expertise of people with disability and their guidance central to translating evaluation findings into use. Ideally, a range of opportunities are provided to people with disability to consider, plan, or present actions from the evaluation findings, including in accessible forums
- giving people with disability the opportunity to provide feedback on the evaluation activities and processes, with this feedback used to inform and improve future evaluation approaches. Feedback and other responses to evaluation should be documented and published (with permission) for the purpose of transparency
- establishing an ongoing feedback loop with participants who give their consent. This can give both evaluators and participants the opportunity to see how things are changing and how well the changes are working for people with disability.
The See Me. Hear Me. Respect Me campaign by Queensland Health is an example that promotes inclusion and respect for people with disability through targeted resources and messaging. An evaluation of the campaign, From Awareness to Action by Inclusive Futures: Reimagining Disability, as an example highlighting its impact and effectiveness in fostering greater awareness and driving meaningful change.
Part 3 of this Toolkit includes a series of fact sheets and Disability inclusive evaluation checklist for evaluators to support you in evaluation.