Before setting up an online summative assessment, it is imperative that you have approval from your Deputy Head of School Education and AQSC to ensure that the risks are being mitigated effectively and adequate provision for the support of all students taking the assessment are in place.

The Centre for Higher Education Practice has produced an online assessment guide which includes alternatives to short duration tests or exams. We strongly recommend that you investigate these options and use these in preference where possible. You can also contact CHEP for guidance about designing Open Book exams.

In uncontrolled environments, where students are completing computer-based assessments outside of an invigilated University computer-based exam location, there is no way to assure that problems will not occur. Students may be adversely affected as a result.

Lockdown browsers and online proctoring are measures that are not currently available at UoS, whether you are running essay, short answer format, or defined-response questions.

Creating robust, online, defined-response tests (such as MCQs) requires front-loading development time. Typically, such tests tend to focus on the ability to recall and organise information.  It is better practice to design test questions that focus more on higher-order and critical thinking and that require the application, analysis, and evaluation of learning, rather than simple knowledge recall.

In terms of security, be aware that students can take a picture of questions on their phone, or write down the questions.   While random selection can help to ensure students do not see the same questions it will not result in an even distribution of questions.  In a defined response test, students may still see the same questions as their peers, even with randomisation selected.

In terms of support, ServiceLine is available to resolve technical queries in developing assessments, but at this time we cannot guarantee dedicated and scalable support to “in flight” Short Duration Online Assessments. Blackboard tests had been recommended only for formative use previously and OnDemand for summative use.  Blackboard tests have not been tested for suitability for summative use.

• Tools to support short duration online assessments.
• Risks and possible mitigation.
• What to tell students.
• Assessment recipes.
• Checking a short duration online assessment

Tools to support short duration online assessments

Risks and possible mitigation

The risks outlined in this section apply to all forms of short duration online assessment. The assessment recipes contain additional, specific risks.

Practice makes perfect

In all circumstances, it is very beneficial to run a short formative assessment using the same methods and options that would be used in the summative assessment. This allows students to get used to the interface and process and allows staff to ensure that the results they receive are what they expect.

Be aware that cloud-based services may change quickly

Blackboard, Turnitin, and OnDemand are all “cloud based services”. We do not maintain and upgrade these services ourselves, instead, they are hosted externally.  This means that the services may change from week to week – usually for the better, but sometimes with unintended consequences.

As an example, in November a new text editor was introduced into Blackboard.  A bug was found where response feedback for the Grade Questions feature of the Grade Centre was missing the new content editor, and text entered into the box there wouldn’t save. Blackboard will be delivering a fix for this issue in December.

The lesson is,  outside of a controlled environment of campus-based exams, there is potential for new issues to be introduced in our cloud-based services.

Have you considered students in different time zones?

Be mindful of those students outside of the UK time zone when selecting certain times for tests to be available.  Allowing the assessment to be taken at any time of day will be more inclusive for those outside of the UK.

Do students have the appropriate equipment or connectivity to complete an assessment?

Part of the approval process for Short Duration Online Assessments is conditional on this risk being mitigated.

Errors found in the assessment after it has been made available.

The IT service (e.g Blackboard) becomes unavailable during the assessment window.

Create a backup plan; for example, students should send an email to the school student office with their submission attached, and ensure your students and school are aware of the plan you agree.

Guidance for students

• ALL questions about the assessment, including technical queries, should be directed to a named contact in the faculty*, such as an instructor of the course.
  • This contact will then answer questions about the assessment and, in the case of technical questions they are unable to answer, they should use the “Raise Serviceline Ticket” link in the course’s Control Panel.

• **This user guide will be available for you to share with your students.

Assessment Recipes

If you:

1. Have obtained the necessary approvals from your DHoSE and AQSC,
2. Have taken steps to ensure risks are mitigated (including, but not limited to, the information above and in the assessment recipes),
3. Have satisfied yourself that the necessary functionality can be achieved using the tools above.

Then review the following recipes which have been designed to support the following assessment types:

• My assessment is essay based:
  • Assessment Recipe: Long essay questions .
• My assessment contains many short answer questions requiring typed answers:
  • Assessment Recipe: Short essay/text answers.
• My assessment uses computer-based tests, bubble sheets, or defined response:
  •  Assessment Recipe: Blackboard Tests for Multiple Choice / Defined Response / Computer-based tests.
  • If you already were using OnDemand, contact ServiceLine to discuss continuing using OnDemand.
• My assessment requires handwritten answers; for example maths calculations and diagrams.
  • Assessment Recipe: Maths questions/answers which need to be hand written.
• My assessment is made up of multiple different types of the above.
  • Assessment Recipe: Combined formats (may use multiples of other practices).

Checking a short duration online assessment

Certain staff will be responsible for checking and verifying assessment set up.

• Checking assessment set-up in Blackboard