SlideShare a Scribd company logo

Evaluating online courses 25 may 2011

•Download as PPT, PDF•
•
Jisc RSC East Midlands
Jisc RSC East Midlands

This document outlines a presentation about moving learning online and evaluating online courses. It includes slides about introducing online learning, assessing where users are on the online learning spectrum, checklists for assessing online courses, examples of online course materials, the importance of interactivity, and tools for evaluating online courses including a survey monkey template and evaluation tool. The presentation also discusses observing online lessons, formative and summative assessment, staff development programs, and includes references and resources.

Related slideshows

Site Presentation 2012
Site Presentation 2012Site Presentation 2012
Site Presentation 2012
 
Apps for teaching and learning: An institutional approach
Apps for teaching and learning: An institutional approachApps for teaching and learning: An institutional approach
Apps for teaching and learning: An institutional approach
 
Approaches to changing assessment and feedback practice
Approaches to changing assessment and feedback practiceApproaches to changing assessment and feedback practice
Approaches to changing assessment and feedback practice
 
1 of 31
Moving Learning On Line - Evaluating & Reviewing Online courses Phil Hardcastle Email: Phil.Hardcastle@@rsc-em.ac.uk Adam Blackwood Email: A.Blackwood@kent.ac.uk May 25, 2011 | slide 1
Plan for the session  Ice Breaking & Introduction  Observable Benefits of MLO  Measuring Impact and Quality – Tools & Approaches  Review Session  Further Resources May 25, 2011 | slide 2
Who is here… May 25, 2011 | slide 3
Who’s here A Adult/Community Learning B Work Based Learning C FE College D Specialist College E 6th Form College B. Williams RSCEM May 25, 2011 | slide 4
Online learning – a spectrum of use A VLE is used to Used to deliver learning A fully online course make a variety of activities and to requires the tutor to documents encourage co-operation work online in the available as a between students on same way as he/she reference point for projects. Assignments would have done face the learner when may be submitted and to face. The tutor will he/she is outside the marked on-line and have to use digital classroom. there will be means to promote assessment exercises. engagement and a sense of community. B. Williams RSCEM May 25, 2011 | slide 5
Where are you on the scale? Learning activities, assessment B. Williams RSCEM May 25, 2011 | slide 6
Plan for the session  Ice Breaking & Introduction  Observable Benefits of MLO  Measuring Impact and Quality – Tools & Approaches  Review Session  Further Resources May 25, 2011 | slide 7
A checklist for assessing courses 1  Is it likely to work? – Has information been provided to enable prospective users to understand the course being offered whether it will suit them? – Have students had an introduction or orientation which prepares them to be successful in the course? – Do students have the necessary hardware & software to complete the course? – Have students the necessary study skills and IT skills to complete the course – is there opportunity to practise these skills? May 25, 2011 | slide 8
A checklist for assessing courses 2  Is it likely to work? – Is the course designed to be interactive? – Are the tutors trained in the techniques that are required for online tutoring? – Is there a system of support for students (and staff) – when does it operate 24/7? 7/5? FAQ? – Is data collected about the course so that administrators, decision makers and course designers can learn from the shared information? This checklist is adapted from the Appleton Eschool’s online evaluation programme. http://www2.ed.gov/admins/lead/academic/evalonline/evalonline.pdf May 25, 2011 | slide 9
Online materials presented to students A basic course would contain the following:  Course and module title  A reading list  Tutor name, contact details and  A glossary of terms, if photo appropriate  Brief welcome/introduction to the  A course timetable (a list or course calendar)  Structured content (chronological  Links to specific or topic order) resources/websites for research  A brief introduction to each  A link to library services and section search  Lecture  Clearly label all resources notes, presentations, hand-outs - indicating what they relate to placed in context  Assignments details including  A clear indication of where and how to submit how to get help May 25, 2011 | slide 10
Interactivity and “stickiness”  The more engaging the course, the more likely the students are to remain on it and complete assignments.  The course should therefore promote interactivity – with the materials, with the tutor and with other students.  Courses can be assessed for the interactive elements they provide.  Varied media presentation will also help – video and podcasts; short spoken introductions to materials; all these help to give a varied experience to the student. May 25, 2011 | slide 11
Plan for the session  Ice Breaking & Introduction  Observable Benefits of MLO  Measuring Impact and Quality – Tools & Approaches  Review Session  Further Resources May 25, 2011 | slide 12
 Outline of the Evaluation Tool  How it works.  Criteria  Download and use  Complete survey for May 25, 2011 | slide 13
Course Evaluation & Review Tool Students Data and Quality Assurance Teaching ROI Practitioners
May 25, 2011 | slide 15
MLOL Evaluation Tool  The Process Determine Evaluation Criteria Create a Survey Add to MLOL Provide to Sample Download as Excel Evaluation Tool in the Monkey Answer File Groups… Grid Data Sheet Tab Review Picture May 25, 2011 | slide 16
May 25, 2011 | slide 17
May 25, 2011 | slide 18
May 25, 2011 | slide 19
MLOL Evaluation Tool Poor Evaluation Picture May 25, 2011 | slide 20
MLOL Evaluation Tool Very Good Evaluation Picture May 25, 2011 | slide 21
MLOL Evaluation Tool  The Process Determine Evaluation Criteria Create a Survey Add to MLOL Provide to Sample Download as Excel Evaluation Tool in the Monkey Answer File Groups… Grid Data Sheet Tab Review Picture May 25, 2011 | slide 22
Survey Monkey Answer Grid Option May 25, 2011 | slide 23
MLOL Evaluation To  TASK 1 Your Criteria evaluation  http://tiny.cc/mlol7task1 May 25, 2011 | slide 24
TASKS  TASKS  Download the MLOL Evaluation Tool – 10 Minutes – Evaluation Tool Download at http://www.tiny.cc/mlol7tool  Consider what Criteria you might want to add to the different Sectors  Feedback May 25, 2011 | slide 25
Online Lesson Observation  The main concern when observing a lesson is to check whether learning is taking place.  How could you demonstrate that learning was taking place in an online course? – Data from online interactions – forums etc – Reports showing quiz answers, assignments completed  How do learners feel about the quality of learning? – Learner surveys by web or phone or email – Use an online live forum of learners  Feedback to tutors? May 25, 2011 | slide 26
Formative and Summative  Online systems and course design are in the very early stages – formative assessment of the course through user surveys, evaluation of online interaction etc can be very useful to help re-focus the course to meet learner needs.  Summative assessment needs to be disseminated around the organisation to make sure that success and failure can be duplicated and avoided. May 25, 2011 | slide 27
Plan for the session  Ice Breaking & Introduction  Observable Benefits of MLO  Measuring Impact and Quality – Tools & Approaches  Review Session  Further Resources May 25, 2011 | slide 28
Staff Development Programme and QA Processes  Staff Development Programme and QA Processes May 25, 2011 | slide 29
Plan for the session  Ice Breaking & Introduction  Observable Benefits of MLO  Measuring Impact and Quality – Tools & Approaches  Review Session  Further Resources May 25, 2011 | slide 30
References and Resources  Google Docs List of references at:  http://tiny.cc/mlol7  Can be added to if you have other links and resources you can suggest.  Survey Monkey Example at http://www.tiny.cc/mlol7task1  Evaluation Tool Download at http://www.tiny.cc/mlol7tool May 25, 2011 | slide 31

More Related Content

Evaluating online courses 25 may 2011

  • 1. Moving Learning On Line - Evaluating & Reviewing Online courses Phil Hardcastle Email: Phil.Hardcastle@@rsc-em.ac.uk Adam Blackwood Email: A.Blackwood@kent.ac.uk May 25, 2011 | slide 1
  • 2. Plan for the session  Ice Breaking & Introduction  Observable Benefits of MLO  Measuring Impact and Quality – Tools & Approaches  Review Session  Further Resources May 25, 2011 | slide 2
  • 3. Who is here… May 25, 2011 | slide 3
  • 4. Who’s here A Adult/Community Learning B Work Based Learning C FE College D Specialist College E 6th Form College B. Williams RSCEM May 25, 2011 | slide 4
  • 5. Online learning – a spectrum of use A VLE is used to Used to deliver learning A fully online course make a variety of activities and to requires the tutor to documents encourage co-operation work online in the available as a between students on same way as he/she reference point for projects. Assignments would have done face the learner when may be submitted and to face. The tutor will he/she is outside the marked on-line and have to use digital classroom. there will be means to promote assessment exercises. engagement and a sense of community. B. Williams RSCEM May 25, 2011 | slide 5
  • 6. Where are you on the scale? Learning activities, assessment B. Williams RSCEM May 25, 2011 | slide 6
  • 7. Plan for the session  Ice Breaking & Introduction  Observable Benefits of MLO  Measuring Impact and Quality – Tools & Approaches  Review Session  Further Resources May 25, 2011 | slide 7
  • 8. A checklist for assessing courses 1  Is it likely to work? – Has information been provided to enable prospective users to understand the course being offered whether it will suit them? – Have students had an introduction or orientation which prepares them to be successful in the course? – Do students have the necessary hardware & software to complete the course? – Have students the necessary study skills and IT skills to complete the course – is there opportunity to practise these skills? May 25, 2011 | slide 8
  • 9. A checklist for assessing courses 2  Is it likely to work? – Is the course designed to be interactive? – Are the tutors trained in the techniques that are required for online tutoring? – Is there a system of support for students (and staff) – when does it operate 24/7? 7/5? FAQ? – Is data collected about the course so that administrators, decision makers and course designers can learn from the shared information? This checklist is adapted from the Appleton Eschool’s online evaluation programme. http://www2.ed.gov/admins/lead/academic/evalonline/evalonline.pdf May 25, 2011 | slide 9
  • 10. Online materials presented to students A basic course would contain the following:  Course and module title  A reading list  Tutor name, contact details and  A glossary of terms, if photo appropriate  Brief welcome/introduction to the  A course timetable (a list or course calendar)  Structured content (chronological  Links to specific or topic order) resources/websites for research  A brief introduction to each  A link to library services and section search  Lecture  Clearly label all resources notes, presentations, hand-outs - indicating what they relate to placed in context  Assignments details including  A clear indication of where and how to submit how to get help May 25, 2011 | slide 10
  • 11. Interactivity and “stickiness”  The more engaging the course, the more likely the students are to remain on it and complete assignments.  The course should therefore promote interactivity – with the materials, with the tutor and with other students.  Courses can be assessed for the interactive elements they provide.  Varied media presentation will also help – video and podcasts; short spoken introductions to materials; all these help to give a varied experience to the student. May 25, 2011 | slide 11
  • 12. Plan for the session  Ice Breaking & Introduction  Observable Benefits of MLO  Measuring Impact and Quality – Tools & Approaches  Review Session  Further Resources May 25, 2011 | slide 12
  • 13. Outline of the Evaluation Tool  How it works.  Criteria  Download and use  Complete survey for May 25, 2011 | slide 13
  • 14. Course Evaluation & Review Tool Students Data and Quality Assurance Teaching ROI Practitioners
  • 15. May 25, 2011 | slide 15
  • 16. MLOL Evaluation Tool  The Process Determine Evaluation Criteria Create a Survey Add to MLOL Provide to Sample Download as Excel Evaluation Tool in the Monkey Answer File Groups… Grid Data Sheet Tab Review Picture May 25, 2011 | slide 16
  • 17. May 25, 2011 | slide 17
  • 18. May 25, 2011 | slide 18
  • 19. May 25, 2011 | slide 19
  • 21. MLOL Evaluation Tool Very Good Evaluation Picture May 25, 2011 | slide 21
  • 22. MLOL Evaluation Tool  The Process Determine Evaluation Criteria Create a Survey Add to MLOL Provide to Sample Download as Excel Evaluation Tool in the Monkey Answer File Groups… Grid Data Sheet Tab Review Picture May 25, 2011 | slide 22
  • 23. Survey Monkey Answer Grid Option May 25, 2011 | slide 23
  • 24. MLOL Evaluation To  TASK 1 Your Criteria evaluation  http://tiny.cc/mlol7task1 May 25, 2011 | slide 24
  • 25. TASKS  TASKS  Download the MLOL Evaluation Tool – 10 Minutes – Evaluation Tool Download at http://www.tiny.cc/mlol7tool  Consider what Criteria you might want to add to the different Sectors  Feedback May 25, 2011 | slide 25
  • 26. Online Lesson Observation  The main concern when observing a lesson is to check whether learning is taking place.  How could you demonstrate that learning was taking place in an online course? – Data from online interactions – forums etc – Reports showing quiz answers, assignments completed  How do learners feel about the quality of learning? – Learner surveys by web or phone or email – Use an online live forum of learners  Feedback to tutors? May 25, 2011 | slide 26
  • 27. Formative and Summative  Online systems and course design are in the very early stages – formative assessment of the course through user surveys, evaluation of online interaction etc can be very useful to help re-focus the course to meet learner needs.  Summative assessment needs to be disseminated around the organisation to make sure that success and failure can be duplicated and avoided. May 25, 2011 | slide 27
  • 28. Plan for the session  Ice Breaking & Introduction  Observable Benefits of MLO  Measuring Impact and Quality – Tools & Approaches  Review Session  Further Resources May 25, 2011 | slide 28
  • 29. Staff Development Programme and QA Processes  Staff Development Programme and QA Processes May 25, 2011 | slide 29
  • 30. Plan for the session  Ice Breaking & Introduction  Observable Benefits of MLO  Measuring Impact and Quality – Tools & Approaches  Review Session  Further Resources May 25, 2011 | slide 30
  • 31. References and Resources  Google Docs List of references at:  http://tiny.cc/mlol7  Can be added to if you have other links and resources you can suggest.  Survey Monkey Example at http://www.tiny.cc/mlol7task1  Evaluation Tool Download at http://www.tiny.cc/mlol7tool May 25, 2011 | slide 31