How best should the project/ program deliver those services (or activities)? Welcome and thank you for being here today. (1999). Here are some sample focus groups questions to obtain answers to help with these two objectives. OFDA Office of U.S. Foreign Disaster Assistance . For best results, a focus group session should include around five or six questions. [caption id="attachment_132881â align="aligncenterâ width="640â] Questions should be a mix of free form and multiple choice[/caption]. Thinking of the answer to the previous question, what made you say that? Try not to have too many Key Evaluation Questions - a maximum of 5-7 main questions will be sufficient. Many projects involve online marketing and gathering feedback on a website design or feature. Do you have any suggestions on how we could improve the product to make it more appealing? If the program has been in place for a while and is thought to be relatively stable, then it may be time for a summative evaluation. ** Adapted from Owen, J. M., & Rogers, P. J. For people to be engaged they first need to feel relaxed and comfortable in their surroundings. Similarly, formative evaluation questions look at whether program activities occur according to plan or the project is achieving its goals while it is underway. How likely are you to purchase this product? Product Reviews: Is Everyone Completely Satisfied? It might also be useful to have some more specific questions under Specificity - Avoid vague open-ended questions that simply ask âwhy.â The word âwhyâ has a negative connotation because it makes the respondent think their first answer was not good enough. 2 PRACTICAL GUIDE: FOCUS GROUP DISCUSSIONS CONTENTS The decision on when to conduct FGDs andâin case of a mixed-methods approachâ their sequencing in relation to the other methods is influenced by the objective of the effort; the overarching assessment, monitoring or evaluation questions the FGDs seek The focus group may also be undertaken to discover preliminary issues that are of concern to a group or community, and Program theory assessment evaluation questions should focus on this theory of change to see if there are gaps in logical connections or inaccurate assumptions. Other suggestions in developing questions for focus group sessions include: a. The Road to Results: Designing and conducting effective development evaluations. EVALUATION OF THE EPA INTERN PROGRAM . London: SAGE Publications Ltd. Additional ResourcesMorra Imas, L. G., & Rist, R. C. (2009). What specific features of the product impressed you? Process evaluation questions focus on how a program is working, program performance, and involve extensive monitoring. Try to select scales that have more than five variations to give respondents more flexibility. (Use a more or less tendency scale), Are you more or less likely to make a large purchase in the next six months? It illustrates how to order questions within a topic of inquiry. Evaluation questions, similar to research questions in academic research projects, guide the methods and tools used to collect data to understand the problem under investigation. Methodologically, evaluation questions focus on varied assessment types. I personally like to have the question on impressions first because with online advertising first impressions matter the most in capturing and retaining a visitorâs attention. Topics to be covered are drawn from the following list. According to Owen and Rogers (1999), there are three levels of evaluation questions at this stage in project planning: The best questions must be developed with stakeholders in the evaluation, including program staff, sponsors and funders, local and regional decision-makers within and outside the program, and community representatives, when the community in which the evaluation or project will be carried out has already been identified. Reasonable and appropriate, or realistic in the given project or program. 4. Some sample questions are: If so, how? Program level (regional, large scale, “Big P”) – how does, or could, the evaluation effect program changes? Course summary evaluations, focus group questions, and expert observation tools are included. Program Evaluation Tip Sheet: Constructing Survey Questions October 2011 . Prepared for: U.S. Environmental Protection Agency Office of Human Resources and Organizational Services Office of Planning, Analysis, and Accountability Washington, DC 20460 (extremely, somewhat, or not familiar at all), Overall how satisfied are you with the product? These questions identify and support the problem that the intervention hopes to address. Please rate on a scale of 0 to 10, where 0 is extremely dissatisfied and 10 is extremely satisfied the following features: The purpose of this gathering is to get your feedback about how we can better serve students enrolled in math courses, such as yourselves, at the THINK TANK. Chapters 2 and 3. If you need more detailed information on public opinion research, please feel free to contact the Political Party Development Team. Use this product-focused checklist for focus group discussions or interviews. What is evaluation Focus groups are structured discussions among small groups of people. * Speed Generally, a facilitator leads a group of 8-10 people in a discussion about selected topics with planned questions, while allowing for interesting, new or unplanned follow up questions. Closing - Remember to thank the participants for their assistance and candor, and then ask them if there is anything that could be done to improve their experience as a member of the focus group. Needs assessment evaluation questions may include focus on the significance of the problem, drawing on literature, previous projects, and baseline data with potential stakeholders. ! Student Focus Groups for Program Assessment: ... away from teacher evaluation. Hints for Developing Your Own Sample of Focus Group Questions, consumer confidence are the forerunners to the future success of many businesses, Writing a Test Plan: Test Strategy, Schedule, and Deliverables, Writing a Test Plan: Define Test Criteria, Writing a Test Plan: Plan Test Resources, Writing a Test Plan: Product Analysis and Test Objectives, Innovate to Increase Personal Effectiveness, Project Management Certification & Careers, Project Management Software Reviews, Tips, & Tutorials, How familiar are you with the product? Sample questions are: Through asking questions, M&E practitioners can identify what their project specifically should address. ICT4D information, communication and technology for development . If the problem is not identifiable by stakeholder communities, projects addressing it will not be successful. Assessment Tool kit: Focus Group Template Focus Group Script . Are there alternative approaches that could have the same outcomes with less cost?*. What are the outputs, outcomes, objectives, and goals of the project? If questions are too vague or broad, or require data that is unavailable or unobservable, they are not answerable. Here are some sample questions to help a project manager evaluate a new website design or concept. (Use a likelihood scale). Is the target population adequately reached by and involved in activities? Needs assessment, or identifying the surrounding social conditions and need for a program. 2. Impact/ outcome evaluation, or how does the program reach its outcomes or impact? Format - When choosing focus group questions, be sure to include a mix of multiple choice questions to provide structure and direction and open-ended questions to allow for further exploration and discovery. Appendix B provides a sample timeline for a small focus group study. Project level (local, activity based, “little p”) – how does, or could, the evaluation effect project or local changes?**. How does the target population interact with the program? References*Adapted from Rossi, P. H., Freeman, H. E., & Lipsey, M. W. (1999). Washington, DC: The World Bank.Stern, E., Stame, N., Mayne, J., Forss, K., Davies, R., & Befani, B. Determine participants By clearly defining the purpose and outcomes of the focus group, it is easier to identify who should be invited to participate in the focus group. 5. It should always include less than ten questions. How will the program work with the target population to sustain the project/ program? Answerable, similar to the reasonableness of a question, good evaluation questions must be able to be answered to some degree of certainty. The questions in an evaluation questionnaire should focus on the goals, objectives, strategies, and work plan. * Ease of Use 1. According to Rossi, Freeman, and Lipsey (1999), evaluation questions must be: Once we have developed the larger question of the project, M&E practitioners begin to consider what data they need to answer the question, using a Theory of Change and a LogFrame. For example, you could ask the groupâs overall impressions of the web page last. These consultations may be informal conversations, reviewing grant requirements and terms of reference (documentation review), or semi-structured individual and/or group interviewing. Do they have negative effects? One of this groupâs first tasks will be to determine whether to bring in an outside evaluator or have staff conduct the evaluation.17, 18! Broadening the Sharing the of Designs and Benefits of Trade Methods for Impact Evaluations: Report of a study commissioned by the Department for International Development. She is also a Doctoral Candidate in Adult Education and Comparative International Education at Pennsylvania State University and a Visiting Researcher with RET, an international organization providing secondary and post-secondary education to displaced peoples worldwide. Tracy (2013) explains that within a focus group, there is a cascading effect of conversation, as comments link into each other and participants share what can be rich, emic, qualitative, data with each other and the moderator. Please fill out both sides of this form: OBJECTIVES This program met the stated objectives of: 1. KEQs usually need to be developed and agreed on at the beginning of evaluation planning - however sometimes KEQs are already prescribed by an evaluation system or a previously developed evaluation framework. The key to creating a positive experience for your participants is by asking questions that will awaken both their creative and critical sides. Focus group question examples. Designing and Conducting Focus Group Interviews Richard A. Krueger Professor and Evaluation Leader University of Minnesota 1954 Buford Ave. â¦ A focus group is an information-gathering tool where a selected set of participants, for example, a group of teachers, students, parents or members of the board, gather to discuss a particular area which has been identified as the focus for a schoolâs evaluation. When respondents are relaxed, they will open up and talk more. Are objectives met? Student Focus Group Questions A5. Once you learn basic questionsâ¦ Instead of asking âwhyâ you should rephrase the question to ask respondents to provide some specific examples of what they liked or did not like about the item in question. Please indicate your rating of the presentation in the categories below by circling the appropriate number, using a scale of 1 (low) through 5 (high). These questions can help assess the general business climate as part of the projectâs risk analysis. Community Food Security Coalition ii Overview The Community Food Project Evaluation Toolkit (Toolkit) is a hands-on workbook designed to help community food project (CFP) staff conduct evaluations of their programs to develop the kind of information that will be compelling not only to fellow staff but to funders, participants and Thinking of your previous response, what are some specific examples that helped you formed your opinion? All Rights Reserved. Surveys. 86–105). (Use an agreement scale). Post-Primary Focus Groups Focus Groups. There is a trainerâs competency checklist and trainer attributes competency self-assessment. Questions for student focus groups . 4400 Massachusetts Avenue, NWWashington, DC 20016. This article will give you many effective focus group questions that should spark discussion. Web Design: Does Everyone Know Where to Click? 25+ Key Focus Group Questions: How to Choose, Write and Ask the Best Questions. Yes, I do want to communicate with American University via SMS. The following list is not exhaustive nor mutually exclusive but provides a framework for thinking about what kind of evaluation you are doing and what (generic) evaluation questions you may need to ask. Some sample questions are: 3. Consumer sentiment and consumer confidence are the forerunners to the future success of many businesses. M&E monitoring and evaluation . Work Assignment B-16 . See the article about moderator guide. For example, this scale has 7 levels to measure the level of satisfaction: completely satisfied, very satisfied, somewhat satisfied, neither satisfied nor unsatisfied, somewhat unsatisfied, very unsatisfied, and completely unsatisfied. New York: Kluwer Academic Publishers. The first order of business when considering conducting a process evaluation is to form a working group that includes key program stakeholders. Are they satisfied? Find the perfect sound in seconds. Here are a few pointers to help you in developing your own questions to motivate participants to open up and provide more honest and complete responses. I am providing my consent by leaving the opt-in checked. For example, ask them to think of three words to describe their favorite hobby. Evaluation models: Viewpoints on educational and human services evaluation. Where it is necessary and/or helpful to hold a focus group to ascertain the views of a cross-section of students from across the subject, the following questions can be used: a. Focus Group Interview Template. IR intermediate result . Process evaluation questions focus on how a program is working, program performance, and involve extensive monitoring. The evaluation questions may also be used in summative evaluations which focus on what happened after the program or project completed, i.e., were goals achieved? â¢ The focus group needs to be appropriate for the type of participants in the group. (Use a numeric or favorable scale). Focus groups are often used in concert with other product research methods to determine not only how the product could be improved but also how the product should be marketed. 1. (Use a numeric or the degree of appeal scale). Image by Gerd Altmann from Pixabay, How to Develop a Change Communication Plan. Learn more about how to file a complaint about a distance program or courses. 6. She holds a BA in Anthropology from Case Western Reserve University and an MA from the School of International Service at American University. As the project manager and facilitator of a focus group, your job is to fully engage the members to get their honest feedback. To learn more about American University’s online Graduate Certificate in Project Monitoring and Evaluation, request more information or call us toll free at 855-725-7614. Are activities conducted with the target population? Step 1: Form a working group. If you would recommend this product, what would you say to convince your best friend to purchase this product? Discovering these issues can help determine which of a number of options is the preferred way forward, or to determine what are the concerns that would prevent a proposal going ahead. (Use a numeric or satisfaction scale), What is your impression of the product? About the AuthorAlly Krupar is an Adjunct Instructor at American University’s School of Professional and Extended Studies where she teaches Qualitative Methods in Monitoring and Evaluation. Policy level – how does, or could, the evaluation impact relevant policy? This question is for testing whether you are a human visitor and to prevent automated spam submissions. How is the project functioning from administrative, organizational, and/or personnel perspectives? We give you the insight you need to write all types of focus group questions with the help of proven examples and expert advice. This tip sheet is intended for programs funded through the Centers for Disease Control and Preventionâs (CDC) Division for Heart Disease and Stroke Prevention. Training Evaluation - Forms and Questionnaires. Retrieved from http://www.dfid.gov.uk/Documents/publications1/design-method-impact-eval.pdf Stufflebeam, D. L., Madaus, G. F., & Kellaghan, T. (2002). If I wanted to contact the company, I know where to click. What three words would you use to describe the product? EVALUATION TOOL. * Specific Features (as named by you). RRA rapid rural appraisal A focus group lasting more than 90 minutes probably includes too many questions or topics for discussion. Copyright Â© 2020 Bright Hub PM. Sample focus group materials can be found in appendix B (PDF 101 KB). * Visual Appeal Some sample questions are: 5. Is the cost of the services or activities reasonable in relation to the benefits? Fun Starter Question or Activity - Ask a fun question or incorporate a team building game to get the groupâs creative juices flowing. Are the headlines/links easy to read and enticing for visitors to click on? Using Focus Groups in Program Development and Evaluation by Roger A. Rennekamp, Ph.D. and Martha A. Nall, Ed.D. What do they think of the services? The questions should prioritize the following: The evaluator consults with all accessible stakeholders to develop specific questions that the evaluation will seek to answer. As the project manager and facilitator of a focus group, your job is to fully engage the members to get their honest feedback. Decide which questions should be included in an evaluation questionnaire. Scoring Sheet (for use with Quality Matters Rubric) ... Education encourages continuous program evaluation and improvement, with an emphasis on identifying the strengths and weaknesses of online course offerings in the College. focus group questions An alternative is to engage them in an activity prior to the focus group in which they evaluate student work samples In both cases, it is a good idea to ask participants to complete a closed-ended questionnaire during the focus group. When respondents are relaxed, they will open up and talk more. Through what mechanisms or arrangements? If not, why not? Focus group questions should guide the groupâs discussion. How much do you agree or disagree with the statements below? When they feel uneasy, they will retreat into their shells and you will not get the input that you need. Use the guide to answer basic questions about the focus group process.
Couple Holding Hands Clipart, Website Design Presentation To Client Ppt, Requirements For Master's In Landscape Architecture, Openstack Version Check, Vita Name Meaning, Occupational Health And Safety Act Regulations, Sennheiser Pc37x Warranty, Apache Words And Meanings, L'oreal Sleek And Swing, Youth Ministry 5 Year Plan,