Course Name: Evaluation and Impact Assessment
Evaluation and Impact Assessment
Teacher
Dr Ajay Kumar Prusty
Category
Core Courses
Course Attendees
Still no participant
Course Reviews
Still no reviews
Course objectives:
- To orient students on the importance of evaluation and impact assessment.
- To develop capacities for evaluation and impact assessment.
- Discuss ways of conducting evaluations and impact assessment.
Course Outcome:
| COs | Course outcomes | Mapping Cos with PSOs |
| CO1 | Students can apply the evaluation process and critically evaluate programmes for developing rural areas and the environment. | PSO2(3) |
| CO2 | Students understand impact assessment and can apply it to assess the programme’s impact and environmental impact. |
PSO2: Develop the ability to know farming practices and their scope to improve in rural areas and the environments.
Syllabus:
Theory
UNIT I
Concept of Evaluation: Meaning and concept in different contexts; Why Evaluation is Done and When? Programme planning, analyse programme effectiveness, decision making, accountability, impact assessment, policy advocacy; Objectives, types, criteria and approaches of programme evaluation, evaluation principles; the context of program evaluation in agricultural extension; Role and Credibility of Evaluator: Role as educator, facilitator, consultant, interpreter, mediator and change agent. Competency and credibility of evaluator
UNIT II
Evaluation Theories: Evaluation theory vs. practice – synergistic role between practice and theory in evaluation; Evaluation theories - Three broad categories of theories that evaluators use in their works - programme theory, social science theory, and evaluation theory (other theories/ approaches - Utilization-Focused Evaluation & Utilization-Focused Evaluation (U-FE) Checklist, Values Engaged Evaluation, Empowerment Evaluation, Theory-Driven Evaluation). Integration between theory and practice of evaluation: –evaluation forums, workshops, conferences and apprenticeship/ internship.
UNIT III
How to Conduct Evaluation: Ten Steps in programme evaluation: (1) Identify and describe programme you want to evaluate (2) Identify the phase of the programme (design, start-up, on- going, wrap-up, follow-up) and type of evaluation study needed (needs assessment, baseline, formative, summative, follow-up) (3) Assess the feasibility of implementing an evaluation (4) Identify and consult key stakeholders (5) Identify approaches to data collection (quantitative, qualitative, mixed) (6) Select data collection techniques (survey interviews and questionnaires with different types) (7) Identify population and select sample (sampling for evaluation, sample size, errors, sampling techniques (8) Collect, analyse and interpret data (qualitative and quantitative evaluation data analysis) (9) Communicate findings (reporting plan, evaluation report types, reporting results, reporting tips, reporting negative findings (10) Apply and use findings (programme continuation/ discontinuation, improve on-going programme, plan future programmes and inform programme stakeholders).
UNIT IV
Evaluating the Evaluation - 10 Steps as above with focus on conceptual clarity, representation of programme components and stakeholders, sensitivity, representativeness of needs, sample and data, technical adequacy, methods used for data collection and analysis, costs, recommendations and reports.
UNIT V
SWOT Analysis and Bar Charts: SWOT Analysis – Concept, origin and evolution; SWOT As a Programme Management Tool; Conducting SWOT Analysis - Common Questions in SWOT Analysis; Advantages and Disadvantages of SWOT; Bar Charts (Gantt Charts and Milestone Charts) - Characteristics, advantages and limitations.
UNIT VI
Networks – Introduction, origin and widely used networks (Programme Evaluation and Review Technique (PERT) and Critical Path Method (CPM), differences between PERT and CPM, advantages and disadvantages. Networks Terminology – Activity, Dummy activity, Event (predecessor event, successor event, burst event, merge event, critical event), Earliest Start Time (EST), Latest Start Time (LST), Critical Path, Critical Activity, Optimistic time (To), Pessimistic time (Po), Most likely time (TM), Expected time (TE), Float or Slack, Event Slack, Lead time, Lag time, Fast tracking, Crashing critical path, Acclivity Table, Danglers, Normal Time. Rules for Preparation of Networks and Steps in Network Preparation with example.
UNIT VII
Bennett’s Hierarchy of Evaluation: Introduction to Bennett’s hierarchy – Background and description; Relation between programme objectives & outcomes at 7 levels of Bennett’s hierarchy – Inputs, activities, participation, reactions, KASA changes, practice and behaviour changes, end results. Advantages and Disadvantages of Bennett’s hierarchy
UNIT VIII
Logic Framework Approach (LFA): Introduction to LFA – Background and description; Variations of LFA - Goal Oriented Project Planning (GOPP) or Objectives Oriented Project Planning (OOPP); LFA Four-by-Four Grid – Rows from bottom to top (Activities, Outputs, Purpose and Goal & Columns representing types of information about the events (Narrative description, Objectively Verifiable Indicators (OVIs) of these events taking place, Means of Verification (MoV) where information will be available on the OVIs, and Assumptions). Advantages and Disadvantages of LFA.
UNIT IX
Introduction to Impact Assessment: Concept of Impact Assessment: Meaning, concept and purpose in different contexts; Impact Assessment Framework: Meaning of inputs, outputs, outcomes, impacts and their relation with monitoring, evaluation and impact assessment.
UNIT X
Impact Assessment Indicators: Indicators for impact assessment – meaning and concept; Selecting impact indicators; Types of impact indicators for technology and extension advisory services - social and behavioral indicators, socio-cultural indicators, technology level indicators, environmental impact assessment indicators and institutional impact assessment indicators.
UNIT XI
Approaches for Impact Assessment: Impact assessment approaches – Quantitative, qualitative, participatory and mixed methods with their advantages and disadvantages; Quantitative Impact Assessment Types – Based on Time of Assessment (Ex-ante and ex-post), Based on Research Design (Experimental, quasi experimental, Non-experimental). Econometric Impact Assessment: - (Partial Budgeting Technique, Net Present Value, Benefit Cost Ratio, Internal Rate of Return, Adoption Quotient, etc.). Qualitative and Participatory Impact Assessment Methods.
UNIT XII
Environment Impact Assessment (EIA): Concept of EIA – Introduction, what it is? Who does it? Why it is conducted? How it is done? Benefits and important aspects of EIA-risk assessment, environmental management and post product monitoring. Environmental Components of EIA – air, noise, water, biological, land; Composition of the expert committees and Steps in EIA process - screening, scoping, collection of baseline data, impact prediction, mitigation measures and EIA report, public hearing, decision making, monitoring and implementation of environmental management plan, assessment of alternatives, delineation of mitigation measures and EIA report; Salient Features of 2006 Amendment to EIA Notification - Environmental Clearance/Rejection, participants of EIA; Shortcomings of EIA and How to improve EIA process?
Practical
Search the literature using web / printed resources and identify evaluation indicators for the following: Utilization-Focused Evaluation, Values Engaged Evaluation, Empowerment Evaluation, Theory-Driven Evaluation; Visit Directorate of Extension in your university and enquire about extension programmes being implemented / coordinated by Directorate. Develop an evaluation proposal of any one programme using ‘Ten Steps in Programme Evaluation’ discussed in the theory class; Review any comprehensive programme evaluation report from published sources. Evaluate the report and write your observations following the ‘Evaluating the Evaluation’ approach; Identify at least four agriculture development programmes and their objectives being implemented in your state. Write two attributes each on Strengths, Weaknesses, Opportunities and Threats related to the identified programme objectives in the SWOT grid.; Identify an on-going development programme and make-out 6 activities from the programme; Draw a Gantt chart for 12 months programme activities; Write a report on evaluation hierarchy levels and indicators as per Bennett’s hierarchy of evaluation for any development programme or project; Develop LFA four-by-four grid for any development programme or project with activities, outputs, purpose and goal and objectively verifiable indicators, means of verification & assumptions; Visit a nearby KVKs / ATIC. Select any agriculture technology with package of practices and extension advisory services promoted by KVK / ATIC; Identify impact assessment indicators for social and behavioral indicators, socio-cultural indicators, technology level indicators, environmental impact assessment indicators and institutional impact assessment indicators; Refer any Environment Impact Assessment report and analyse steps in EIA. Write your observations.
References:
- Adrienne M, Gundel S, Apenteng E and Pound B. 2011. Review of Literature on Evaluation Methods Relevant to Extension. Lindau, Switzerland: Global Forum for Rural Advisory Services, Lindau, Switzerland
- Bagnol B. 2014. Conducting participatory monitoring and evaluation. Pages 81-85 in FAO, Decision tools for family poultry development. FAO Animal Production and Health Guidelines, No. 1 6. Rome, Italy: FAO.
- Bennett CF. 1979. Analyzing impacts of extension programs. Washington, D.C., USA: U.S. Department of Agriculture.
- Boyle R and Le Maire D. 1999. Building effective evaluation capacity: lessons from practice. New Brunswick, NJ: Transaction Publishers.
- Bradford RW, Duncan, P.J. and Tarcy, B. 1999. Simplified Strategic Planning: A No-nonsense Guide for Busy People Who Want Results Fast. New York: Chandler House.
- Braverman MT and Engle M. 2009. Theory and rigor in Extension program evaluation planning. Journal of EXtension 47(3). www.joe.org/joe/2009june/a1.php
- Chen H. 2012. Theory-driven evaluation: Conceptual framework, application and advancement. In: Strobl R., Lobermeier O., Heitmeyer W. (eds) Evaluation von Programmen und Projekten für eine demokratische Kultur. Springer VS, Wiesbaden
- Chen, H.T. 2011. Practical program evaluation: Theory-Driven Evaluation and the Integrated Evaluation Perspective. Thousand Oaks, CA: Sage.
- Dale R. 2004. Evaluating Development Programmes and Projects, New Delhi, India: Sage Publications.
- Shadish, W. R. Jr., Cook, T. D., and Leviton, L. C. 1991. Chapter 2: Good theory for social program evaluation. Foundations of Program Evaluation: Theories of Practice (pp. 36-67). Newbury Park, CA: Sage.
- Srinath, L.S. 1975. PERT and CPM Principles and Applications, East-West Press, New Delhi.
- Suvedi M, Heinze K and Ruonavaara D. 1999. How to Conduct Evaluation of Extension Programs. ANRECS Center for Evaluative Studies, Dept of ANR Education and Communication Systems, Michigan State University Extension, East Lansing, MI, USA https://msu.edu/~suvedi/Resources/Documents/4_1_Evaulation%20manual%202000.pdf
- Suvedi M. 2011. Evaluation of agricultural extension and advisory services — A MEAS training module. Urbana Champaign, IL: Modernizing Extension and Advisory Services Project. http://www.meas-extension.org/
- Suvedi, M. and Kaplowitz, M.D. 2016. Process skills and competency tools – what every extension worker should know – Core Competency Handbook. Urbana, IL: USAID-MEAS.
- Suvedi, M and Morford S. 2003. Conducting Program and Project Evaluations: A Primer for Natural Resource Program Managers in British Columbia. Forrex-Forest Research Extension Partnership, Kamloops, B.C. Forrex Series 6.
Session Plan
Session 2: Why Evaluation is Done and When?
Session 3: Programme planning, analyse programme effectiveness
Session 4: Objectives, types, criteria and approaches of programme evaluation
Objectives, types, criteria and approaches of programme evaluation
Session 6: Role and Credibility of Evaluator
Session 7 & 8: Evaluation Theories
Sessions 9 & 10: Ten Steps in programme evaluation
Session 11: Evaluating the Evaluation
Session 13: Conducting SWOT Analysis
Sessions 15 & 16: Networks – Introduction, origin and widely used networks
Sessions 17 & 18: Bennett’s Hierarchy of Evaluation
Sessions 19 & 20: Logic Framework Approach (LFA)
Session 23: Impact assessment framework
Sessions 26 & 27: Approaches for Impact Assessment
Session 28: Quantitative Impact Assessment Types
Session 31: Composition of the expert committees and Steps in EIA process
Composition of the expert committees and Steps in EIA process
Session 32: Salient Features of 2006 Amendment to EIA Notification
Salient Features of 2006 Amendment to EIA Notification
Assignments:
- Role and Credibility of Evaluator
- Evaluation Theories
- Integration between theory and practice of evaluation
- How to Conduct Evaluation: Ten Steps in programme evaluation
- SWOT Analysis – Concept, origin and Evolution
- Bar Charts (Gantt Charts and Milestone Charts) - Characteristics, advantages and limitations
- Logic Framework Approach (LFA): Introduction to LFA – Background and description
- Variations of LFA - Goal Oriented Project Planning (GOPP) or Objectives Oriented Project Planning (OOPP); LFA Four-by-Four Grid
- Concept of Impact Assessment: Meaning, concept and purpose in different contexts,
- Selecting impact indicators, Types of impact indicators for technology and extension advisory services
- Approaches for Impact Assessment
- Composition of the expert committees and Steps in the Environment Impact Assessment process
Our Main Teachers
Dr Prusty, Ph. D. (Extension Education) from Odisha University of Agriculture and Technology, presently working as an Assistant Professor in the Department of Agricultural Extension and Communication, M.S. Swaminathan School of Agriculture, Centurion University of Technology and Management, Paralakhemundi, Gajapati, Odisha, India. He has presented more than ten research papers at national and international conferences. […]
Dr. Satarupa Modak completed Ph.D from Uttar Banga Krishi Viswavidyalaya, West Bengal during 2019, involved in three months PG internship programme of MANAGE, Hyderabad in 2017; qualified ASRB-NET in 2016; completed M.Sc (Agricultural Extension) from Anand Agricultural University, Gujarat in 2014; B.Sc (Agri) from Central Agricultural University, Imphal in 2012. Worked as Watershed Development Team Member […]