Bb Tests and Surveys

Assessment Recipe: School of Health Sciences timed essay questions

Bb Tests and Surveys, Blackboard

Back to Short Duration Online Assessments page | Other Assessment Recipes

This page is advice for running the School of Health Sciences Short Duration Online Assessment, if that is not the type of assessment you are running then you should not follow this guidance.

In-person arrangements

When this type of exam is sat in-person students sit spaced apart in a large hall. Students complete 3 essay questions in 90 minutes. Longer time is given to students with Additional Exam Requirements.

How this is set up online

The 3 essay questions are to be set using Blackboard Tests with a timer set for 90 minutes. There will be an alternative test of  113 minutes for students who have AER in place. Students will choose the appropriate test length. Students will have an 8 hour window to take the test (10 hours for AERs) . Students will not be able to start the test after the available time has passed, but they will be able to continue if they have already started the Test.  Test answers save automatically during the test and the test will auto-submit once the time has expired – students will see the timer counting down during the test and will get a message to notify them when they are halfway through and as they near the end of the test.

Information about the test will be available before the test in the Final Assessment area so that students have a clear understanding of what the format will be and information such as who to contact with questions about the assessment, including technical questions*. The Test will appear in the Final Assessment area during the Availability time set.

*this is based on on this section of Section IV: General Information and Regulations of the University Calendar that states that “The nominee(s) of the Associate Dean (Education) must be available during the whole of the examination period in case of queries”, to keep the Short Duration Online Assessment process aligned with this exams process that it replaces, the nominee (usually a course instructor) should be available for student queries during their working hours.

 

Exams will be scheduled by the Exams Office so you should match start and end dates/times for the assessment to match the exam schedule.

The test will be created in the Blackboard course SOHS SDOA Test Area the tests will then be copied (rolled over) from the Test Area into the Final Assessment area of the relevant Blackboard modules. Once copied over, no further set-up is needed within the Blackboard modules.

Select the ▶ symbol to expand each section.
1. Create the test

Create a Blackboard test, set the creation options, then start building the test.

When creating test questions, use the Accessibility checker for each and follow its guidance to resolve any accessibility issues.

2. Deploy the test

Once ready deploy the test to a content area using the below settings that are particularly relevant for a Short Duration Online Assessment.

Open test in new window: No – this reduces the possibility of students accidentally logging out of Blackboard in a different window and being locked out of the test

Make available to students: Yes – students will not be able to access the test unless this option is set.

Multiple Attempts: Tick one attempt.  This is the most straightforward option for students. If a student submits by mistake and needs another attempt, they should be told to contact the student office or the course instructor. Another attempt can be given to an individual student if needed. (Alternatively, choose 2 attempts to allow for technical difficulties with the first attempt – see note below. )

Score attempts using:  set this to set to First Marked Attempt if more than one attempt is set as an option (above). Students should be told that the second attempt is only to be used if they had technical issues with the first attempt (not an opportunity to improve their score). They should contact the school office, or the course instructor, if they want the second attempt counted.

Force Completion: Do not tick.  If you tick this box, then once participants have begun the Test, they may not exit and continue working on it at a later date.  This means if students lose their internet connection or have a power outage while taking the test they will not be able to complete it.

Set timer: set the timer to the number of minutes allowed for the test and select Auto-Submit to on – this will enforce submission when the time is up. Note: If a student experiences technical difficulties during the Test they may request extra time or request another attempt. If a student loses connection they can re-join the test but the timer will have continued while they were offline. 

Display after/until tick the boxes and set the dates and times to correspond with dates and times that the assessment should be available to students. – they will be able to continue the test, for the allotted time set by the Timer, if they have started it before the display until deadline.

Password: to avoid increased support requests do not set a password.

Due Date: do not set a due date, use the display until settings to set when the test is no longer accessible.

Include this Test in Grade Centre Score Calculations: Do not tick – this will prevent the score of the test from affecting the running total in the Grade Centre, potentially giving students an indication of their score even if results are hidden.

Hide results for this test completely from the instructor and the Grade Centre:  Do not tick: ticking this box will mean the results will never be recorded, once students start taking this assessment this option can never be undone.

Show test results and feedback to students: untick everything.  In most circumstances for summative assessment, the marks should not be shown to students after they have completed the assessment.  This can be adjusted later to reveal feedback at a set date if desired.

Test presentation: All at once. It is then easy for students to choose which question to start first.

Prohibit Backtracking: Do not tick.  If this is ticked students will not be able return to an earlier question.

Randomise Questions: Do not tick.

 

After submitting these options, make sure to follow the steps in this guide to ensure that the results will not be visible to students.

3. Important- Hide Blackboard GradeCentre column to ensure that results are not visible to students 

It’s really important to ensure that students do not see scores before you want them to.  Follow the steps in this guide to ensure that the results will not be visible to students.

4. Copy Tests and Grade Centre settings to Blackboard modules

Go to  the course where the Test has been set up : SOHS SDOA Test Area

Go to Course Management/Rollover(course copy)
Set the Destination Course ID = Blackboard course ID to copy the test into.
Tick options:
-Content areas/final assessment.
-Select any content area that contains a test for the particular module – if the Test is not included in the Final Assessment area.
-Grade Centre Columns and Settings.
-Tests Surveys and Pools.
-Submit

Rollover settings

 

Online considerations

This assessment will not be proctored (invigilated) which means that it is effectively open book. Students have access to all of Google and the wider web. The timer will enforce the attempts are made only within a limited time

Risks and possible mitigation

In addition to the general risks outlined on the main Online Short Duration Assessments web page, the table below covers risks specific to this scenario along with suggested mitigations.

What could go wrong? Possible Mitigation
Error found in question/answers after assessment made available. Ensure that the questions, rubric and all settings have been thoroughly proof read and reviewed for errors by yourself and relevant colleagues in good time before the assessment is due to take place.

Where a mistake is found after students have started the test please follow the appropriate AQSC Policy.
Student is disconnected and cannot continue assessment. Note: This advice may change subject to AQSC policy

Students will be able to re-join their attempt if the time has not run out in the meantime If they experience time out of the test they can contact the office to request another attempt or more time. Alternatively they can be asked to submit Special Considerations.

Do not select Force Completion. For details of these and other settings view this page.
An assessment has been scheduled to be available for the wrong time, or for too little time. Scheduling can be adjusted when the test is deployed. If this needs to be done after the test you must refer to the AQSC Policy which will explain how respond to this should it arise.
Blackboard becomes unavailable during an assessment. Allow at least two attempts on the assessment when deploying the assessment.

Do not select Force Completion. For details of these and other settings view this page.
If Blackboard does not become available again within the time window available for the student to complete the assessment it may be a question of an appeal to go via the School’s standard process.
A student sees a score or correct answer feedback when they should not. Leave the “show test results and feedback to students” options empty in the Test Options. Set the ‘hide from students’ setting for the Grade Centre column and make sure the score is not included in the running total.
A student was still doing a test when the deadline was reached and did not submit on time, leaving their attempt marked “in progress” You can submit a test on behalf of a student if required. This will not occur if the Auto -submit is enabled.

Marking and giving feedback

To replicate the exams experience it is expected that no feedback will be given to students for the submissions other than a final score which would be delivered via Banner Self Service and the “Marks Release” process.  More information will be placed here shortly.  In the meantime it is important to ensure that marks and feedback are hidden from students and ensure when deploying the test that no feedback options are ticked.

Back to Short Duration Online Assessments page | Other Assessment Recipes

Checking assessment set-up in Blackboard

Bb Assessment, Bb assignments, Bb Tests and Surveys, Blackboard

Contents

  1. Accessing and identifying assessments
      1. Checking options for a Blackboard test
      2. Checking options for a Turnitin Assignment
    1. Checking options for a Blackboard Assignment
  2. When the assessment is uploaded
    1. Is the assessment hidden from students?
    2. Is the assessment set to be published to students on the right date and at the right time? Is the assessment set to close on the right date and at the right time?
      1. Blackboard tests
      2. Turnitin Assignments
      3. Blackboard Assignments
    3. Are the assessment rubric and any other instructions or documents attached to the assessment?
    4. Is there an assessment submission link?
    5. Are all the instructions and links a student will need to access, carry out and submit the assessment in the same place or clearly linked and are the instructions for accessing and submitting the assessment clear?
    6. Does the rubric for the assessment allow multiple submissions within the assessment window?
      1. Multiple submissions in Blackboard tests
      2. Multiple submissions in Turnitin assignments
      3. Multiple submissions in Blackboard assignments
    7. Is the assessment set up correctly to meet the rubric?
    8. Is the assessment set to prevent submission after the submission deadline?
      1. Ensuring submission is not possible after the deadline in Blackboard tests
      2. Ensuring submission is not possible after the deadline in Turnitin assignments
      3. Ensuring submission is not possible after the deadline in Blackboard assignments
    9. Are scores hidden from students?
    10. Is an announcement set up to alert students when the assessment opens and an hour before it will close? Are these set for the correct days and times?
  3. Checking the assessment release
    1. Has the assessment been successfully published to students?

Accessing and identifying assessments

For most of the example below you will need to know where the assessments are.  Follow these steps:

1. Enter the Blackboard course in question and ensure that Edit Mode is set to On.

Edit mode is On.

2. Open the Final Assessment menu item within the course menu.  The course menu appears on the left side of a Blackboard course.

3. Assessments are likely to be tests, Turnitin assignments or Blackboard Assignments.  The screenshot below shows an example of each.

Example assessments

4. The next guides below show how to access the options for each type of assessment.

Return to the contents page.

Checking options for a Blackboard test

1. Select the action button that appears to the right of the test name.

2. Select Edit the Test Options from the menu that appears.

Editing test options

Return to the contents page.

Checking options for a Turnitin Assignment

1. Select the action button that appears to the right of the assignment name.

2. Select Edit from the menu that appears.

Editing Turnitin assignment options

Return to the contents page.

Checking options for a Blackboard Assignment

1. Select the action button that appears to the right of the assignment name.

2. Select Edit from the menu that appears.

Editing Blackboard assignment options

Return to the contents page.

When the assessment is uploaded:

Is the assessment hidden from students?

It is vital that the assessment is not visible to students until it is due to be published.  To check this identify the assessment/s within the content area and ensure you see that it is hidden to students.  This screenshot shows examples of tests, Turnitin assignments and Blackboard assignments that are hidden from students.

Three different types of assessment, all state they are hidden from students

Return to the contents page.

 

Is the assessment set to be published to students on the right date and at the right time? Is the assessment set to close on the right date and at the right time?

Having identified the assessment, edit its settings. The following screenshots show where the date and time of release to students is set:

Blackboard tests

In Blackboard tests the Display After date and time sets when the test will be accessible to students, the Display Until date sets when students can no longer access it.

Start and end times of a Blackboard test

It is also important to verify that the Make available to students setting is set to Yes.

If this option is not set students will not be able to access the test once the display after date and time has been reached.

Return to the contents page.

Turnitin Assignments

In Turnitin the Start date and time sets when the assignment will be accessible to students.  The due date sets when the assignment is due.

Turnitin start and end times

The option to allow submissions after the due date should be set to No.  This is found under Optional Settings.  When submissions are not allowed after the due date then once the due date and time has been reached students will no longer be able to submit.

Turnitin due date

Return to the contents page.

Blackboard Assignments

In Blackboard assignments the Display After date and time sets when the assignment will be accessible to students, the Display Until date sets when students can no longer access it.

Blackboard assignment start and end date

Return to the contents page.

Are the assessment rubric and any other instructions or documents attached to the assessment?

The assignment rubric could have been added in a variety of places, such as:

  1. Additional content items within the same folder on Blackboard.
  2. The description, instructions or Content Link Description of a Blackboard test.
  3. The Special instructions section of a Turnitin assignment.
  4. The instructions of a Blackboard assignment.

Return to the contents page.

The encircled parts of the screenshots below show the link that students should select in order to submit their assignment, depending on whether it is a Test, Turnitin assignment or Blackboard assignment.

Submission links

Return to the contents page.

Are all the instructions and links a student will need to access, carry out and submit the assessment in the same place or clearly linked and are the instructions for accessing and submitting the assessment clear?

The instructions and links a student will need to access may vary depending on the assignment and could have been added in a variety of places, such as:

  1. As additional content items within the same folder on Blackboard.
  2. In the description, instructions or Content Link Description of a Blackboard test.
  3. In the Special instructions section of a Turnitin assignment.
  4. In the instructions of a Blackboard assignment.

Return to the contents page.

Does the rubric for the assessment allow multiple submissions within the assessment window?

Multiple submissions in Blackboard tests

Within the test options Multiple Attempts can be allowed – either an unlimited number of a maximum number.  The score attempts using setting determines which attempt should be scored.  This may be set to the last marked attempt.

test allowing three attempts, the last attempt will be used

Return to the contents page.

Multiple submissions in Turnitin assignments

Within the optional settings, set for similarity reports to be created on the due date.  This will mean students can submit as many times as they like up to the due date.  On the due date they will no longer be able to submit and their most recent submission will have a similarity report generated.  Neither staff nor students will be able to access earlier submissions if they had been made.

Turnitin - multiple attempts allowed via similarity report setting

Return to the contents page.

Multiple submissions in Blackboard assignments

Within the test options Multiple Attempts can be allowed – either an unlimited number of a maximum number.  The score attempts using setting determines which attempt should be scored.  This may be set to the last marked attempt.

Assignment allowing 3 attempts, the last will be marked

Return to the contents page.

Is the assessment set up correctly to meet the rubric?

Since the rubrics may vary between assignments it is harder to give specific steps.  Outline steps are:

  1. Review the rubric.
  2. Edit the settings for the assessment and check that the settings chosen match those described in the rubric.

Return to the contents page.

Is the assessment set to prevent submission after the submission deadline?

Edit the assignment and check the options.  These are described for each type of assessment below.

Ensuring submission is not possible after the deadline in Blackboard tests

Within Blackboard test options a due date should be set appropriately.  The tick box “Do not allow students to start the Test if the due date has passed” should be selected.

Due date in a Blackboard test

Return to the contents page.

Ensuring submission is not possible after the deadline in Turnitin assignments

The option to allow submissions after the due date should be set to No.  This is found under Optional Settings within the Turnitin assignment in Blackboard.  When submissions are not allowed after the due date then once the due date and time has been reached students will no longer be able to submit.

Turnitin due date

Return to the contents page.

Ensuring submission is not possible after the deadline in Blackboard assignments

In Blackboard assignment options the due date is used to set when the deadline is.  It is not possible to disallow attempts entirely after the due date.  If a student starts an assignment at 11:59 in the below example they may still submit, but the assignment will be marked as late.  This can later be given zero points manually.

Due date in a Blackboard assignment

Note that the Blackboard assignment date restrictions will prevent a student from being able to start a submission after the due date.

Return to the contents page.

Are scores hidden from students?

It’s important to verify that students will not be able to find out their assessment results and scores before you want them to.

Follow the steps in this guide to ensure that scores are not shown to students.

Extra step for Blackboard tests.

If the assessment is a Blackboard test, edit the test options and verify that under “Show test results and feedback to students” no tick boxes are selected.

Untick an options that would show students their scores or feedback.

Return to the contents page.

Is an announcement set up to alert students when the assessment opens and an hour before it will close? Are these set for the correct days and times?

Announcements are found in the announcements menu of a course.  With Edit Mode set to On, check that two announcement have been set up, one stating that the assessment is open and the other that it is an hour before the assessment will close.

1. Edit the announcement to check the date ranges chosen for the announcement. To edit an announcement select the modify button that appear to the right of it and choose Edit from the menu that appears.

Editing an announcement

2. Scroll down to the Web Announcement Options section and verify that Date Restricted is set for the Duration.

3. Check that the display after date matches either the start time of the assessment, or one hour before the end time of the assessment as appropriate.

announcement display after time

4. Selecting “Send a copy of the announcement” will generate an error. This setting cannot be used for email announcements set to appear in the future.

Warning when overriding email preferences on an announcement due to be displayed at a future date

Notes

When an announcement is created and set to display only after a future date and time, an email will be sent to all students who were enrolled on the Blackboard course at the date and time that the announcement was originally created who have not edited their settings to decline announcement emails. The email will be sent usually within 30 minutes after the ‘display after’ date. More information about announcements is available. Note therefore that not all students may receive the email and information should avoid very specific information about timings (eg: will close shortly, not: will close in 1 hour).

If you want to ensure that all students taking the assessment are alerted to its release and/or its impending closure at the right time it is preferable to use the Send Email tool within Blackboard. You cannot send emails in advance using the Sent Email tool.

Return to the contents page.

Checking the assessment release

Within one hour of the assessment opening:

Has the assessment been successfully published to students?

1. To ensure that tests can be accessed by students use the Student Preview button to view the course as though you were a student.

Student preview icon is highlighted

2. Find the assessment in the course.  This screenshot matches the other examples used in this page.  Ensure that phrases like “Availability: Item is hidden” are not shown.

Visible SDOAs

3. Select the submission links to ensure the assignments are accessible.

 

Return to the contents page.

Using TeX in Blackboard

Bb Content, Bb Tests and Surveys, Blackboard

There are a number of ways to use mathematical notation in Blackboard.  In this page I share a technique to use the Display mathematic mode of TeX within Blackboard.

This guide refers to typing and displaying mathematical notation.  This method is not suitable for building formulas for use in Calculated Formula question types.

For more information refer to:

Contents of this page

  1. Basic operation
    1. Superscript example
    2. Arithmetic Example
    3. Fractions
    4. Integrals
    5. Matrix
  2. Issues to be aware of
    1. If you go back to edit, be careful that your TeX is not adjusted.
    2. Adding text before and after the notation may not end up how you intend.
  3. Alternative approaches
    1. Use the Mathtype interface within Blackboard.
    2. Create your notation in a different application, take a screenshot of it, and add it within Blackboard.

Basic operation

When using the Blackboard text editor, for example when creating a content item, or a test question, you can use the $$..$$ TeX primitive syntax.

Back to top.

Superscript example

$$e^{e^{e^{e^{x}}}}$$

in Blackboard:

Blackboard text box. Entered is: $$e^{e^{e^{e^{x}}}}$$

Results in:

e to the power of e to the power of e to the power of e to the power of x end exponent end exponent end exponent

In the Blackboard App it looks like this:

Example in the Blackboard app

Accessibility

The notation is rendered as a graphic.  Importantly users with a screen reader will see accurate alternate text, which in this example is:

“e to the power of e to the power of e to the power of e to the power of x end exponent end exponent end exponent”.

Back to top.

Arithmetic example

$$\sqrt[3]{8}=8^{\frac{1}{3}}=2$$

in Blackboard:

Results in:

In the Blackboard App it looks like this:

Example from the Blackboard app

Accessibility

The notation is rendered as a graphic.  Importantly users with a screen reader will see accurate alternate text, which in this example is:

“cube root of 8 equals 8 to the power of 1 third end exponent equals 2”.

Back to top.

Fractions

$$\left.\frac{1}{1+x^2}\right|_0^\infty$$

in Blackboard:

Results in:

In the Blackboard App it looks like this:

Example from the bb app

Accessibility

The notation is rendered as a graphic.  Importantly users with a screen reader will see accurate alternate text, which in this example is:

“open fraction numerator 1 over denominator 1 plus x squared end fraction close vertical bar subscript 0 superscript infinity”.

Back to top.

Integrals

$$\int_0^2x^2dx$$


in Blackboard:


$$\int_0^2x^2dx$$ is shown in the Blackboard text editor


Results in:


Blackboard screenshot, shows "integral subscript 0 superscript 2 x squared d x"


In the Blackboard App it looks like this:




Accessibility


The notation is rendered as a graphic.  Importantly users with a screen reader will see accurate alternate text, which in this example is:


“integral subscript 0 superscript 2 x squared d x”.


Back to top.


Matrix


$$\begin{bmatrix}a & b\\c & d\end{bmatrix}$$


in Blackboard:




Results in:




In the Blackboard App it looks like this:


Example from the blackboard app


Accessibility


The notation is rendered as a graphic.  Importantly users with a screen reader will see accurate alternate text, which in this example is:


“open square brackets table row a b row c d end table close square brackets”.


Back to top.


Issues to be aware of


When you go back to edit, be careful that your TeX is not adjusted.


For example


$$\begin{bmatrix}a & b\\c & d\end{bmatrix}$$


may be changed to


$$\begin{bmatrix}a §amp; b\\c §amp; d\end{bmatrix}$$.




Back to top.


Adding text before and after the notation may not end up how you intend


For example


$$e^{e^{e^{e^{x}}}}$$ and then $$\sqrt[3]{8}=8^{\frac{1}{3}}=2$$


Results in:




Back to top.


Alternative approaches


Use the Mathtype interface within Blackboard


Select the Mathtype option from the text editor:




and refer to the Blackboard guide on using MathType.


Back to top.


Create your notation in a different application, take a screenshot of it, and add it within Blackboard


1. Crop and save your screenshot to a png file on your computer.


2. Use the Plus icon to add it within your text editor:




3. Select Insert Local File.




4. Right click on the image and open the Image… menu.




5. Enter and submit an alternative description that explains your notation in written terms, to make it accessible to those using screen readers.




Back to top.



    


		
		


		
	

	
	



							
						
											



	
	
	


					

		
					 
Short duration online assessments 
		
					


				
				Bb assignments, Bb Tests and Surveys, Blackboard, eAssignments, QM Perception, Support, TurnItIn
											

		
		


		
			
Before setting up an online summative assessment, it is imperative that you have approval from your Deputy Head of School Education and AQSC to ensure that the risks are being mitigated effectively and adequate provision for the support of all students taking the assessment are in place.


The Centre for Higher Education Practice has produced an online assessment guide which includes alternatives to short duration tests or exams. We strongly recommend that you investigate these options and use these in preference where possible. You can also contact CHEP for guidance about designing Open Book exams.


In uncontrolled environments, where students are completing computer-based assessments outside of an invigilated University computer-based exam location, there is no way to assure that problems will not occur. Students may be adversely affected as a result.


Lockdown browsers and online proctoring are measures that are not currently available at UoS, whether you are running essay, short answer format, or defined-response questions.


Creating robust, online, defined-response tests (such as MCQs) requires front-loading development time. Typically, such tests tend to focus on the ability to recall and organise information.  It is better practice to design test questions that focus more on higher-order and critical thinking and that require the application, analysis, and evaluation of learning, rather than simple knowledge recall.


In terms of security, be aware that students can take a picture of questions on their phone, or write down the questions.   While random selection can help to ensure students do not see the same questions it will not result in an even distribution of questions.  In a defined response test, students may still see the same questions as their peers, even with randomisation selected.


In terms of support, ServiceLine is available to resolve technical queries in developing assessments, but at this time we cannot guarantee dedicated and scalable support to “in flight” Short Duration Online Assessments. Blackboard tests had been recommended only for formative use previously and OnDemand for summative use.  Blackboard tests have not been tested for suitability for summative use.



Tools to support short duration online assessments







Turnitin via Blackboard
Blackboard Tests
Blackboard Assignments
OnDemand




Recommended for
Long essay and typed short essay/text answers
Short answer computer-based tests, bubble sheets, or defined response.
Short answers, uploading handwritten answers & images
Only if you have used it before




Features
Plagiarism check
No plagiarism check.

Variety of question types inc multiple-choice, fill in the blanks & short answer

Randomised questions

Math editor		
No plagiarism check
Variety of questions types inc multiple-choice, short answer, essay & file response




Marking tools available
Rubric

Quick marks

Inline marking		
Automated marking Yes – except for File Response, Essay, and Short Answer.
Direct annotation

Inline marking

Quick marks

Rubric		
Automated marking – except for File Response, Essay, and Short Answer.




Set up
Multiple assignments need to be created if more than one document is to be uploaded
Time needed to develop robust, potentially single-use, defined-response questions.

Only to be used if you have used this before. Contact ServiceLine for technical checking and scheduling.




How will the assessment be completed?
Students advised to download the question paper, work offline in Office 365 using autosave. Save and upload a Word document.**
Online via Blackboard
Students advised to download the question paper, work offline in Office 365 using autosave. Save and upload a document. For handwritten answers, scan paper to PDF using a scanner or the Office Lens or OneDrive app on a mobile**
Online




Risks
Low.

Users’connections issues – mitigated if they use O365 with autosave		
High* – dependent on users’ internet connection. See Risks and Mitigation section
Low.

Users’connections issues.

No access/knowledge of tools needed for uploads such as smartphone or scanner		
High* – dependent on users’ internet connection. See Risks and Mitigation section






Risks and possible mitigation


The risks outlined in this section apply to all forms of short duration online assessment. The assessment recipes contain additional, specific risks.


Practice makes perfect


In all circumstances, it is very beneficial to run a short formative assessment using the same methods and options that would be used in the summative assessment. This allows students to get used to the interface and process and allows staff to ensure that the results they receive are what they expect.


Be aware that cloud-based services may change quickly


Blackboard, Turnitin, and OnDemand are all “cloud based services”. We do not maintain and upgrade these services ourselves, instead, they are hosted externally.  This means that the services may change from week to week – usually for the better, but sometimes with unintended consequences.


As an example, in November a new text editor was introduced into Blackboard.  A bug was found where response feedback for the Grade Questions feature of the Grade Centre was missing the new content editor, and text entered into the box there wouldn’t save. Blackboard will be delivering a fix for this issue in December.


The lesson is,  outside of a controlled environment of campus-based exams, there is potential for new issues to be introduced in our cloud-based services.


Have you considered students in different time zones?


Be mindful of those students outside of the UK time zone when selecting certain times for tests to be available.  Allowing the assessment to be taken at any time of day will be more inclusive for those outside of the UK.


Do students have the appropriate equipment or connectivity to complete an assessment?


Part of the approval process for Short Duration Online Assessments is conditional on this risk being mitigated.


Errors found in the assessment after it has been made available.




Ensure that the questions, rubric and all settings have been thoroughly proof read and reviewed for errors by yourself and relevant colleagues in good time before the assessment is due to take place.


Where a mistake is found after students have started the test please follow the appropriate AQSC Policy.




The IT service (e.g Blackboard) becomes unavailable during the assessment window.


Create a backup plan; for example, students should send an email to the school student office with their submission attached, and ensure your students and school are aware of the plan you agree.


Guidance for students


    

  • ALL questions about the assessment, including technical queries, should be directed to a named contact in the faculty*, such as an instructor of the course.

      

    • This contact will then answer questions about the assessment and, in the case of technical questions they are unable to answer, they should use the “Raise Serviceline Ticket” link in the course’s Control Panel.
      • 

    

    • 







*this is based on on this section of Section IV: General Information and Regulations of the University Calendar that states that “The nominee(s) of the Associate Dean (Education) must be available during the whole of the examination period in case of queries”, to keep the Short Duration Online Assessment process aligned with this exams process that it replaces, the nominee (usually a course instructor) should be available for student queries during their working hours.







Assessment Recipes


If you:


    

  1. Have obtained the necessary approvals from your DHoSE and AQSC, 
    2. 

  2. Have taken steps to ensure risks are mitigated (including, but not limited to, the information above and in the assessment recipes),
    3. 

  3. Have satisfied yourself that the necessary functionality can be achieved using the tools above.
    4. 



Then review the following recipes which have been designed to support the following assessment types:



Checking a short duration online assessment


Certain staff will be responsible for checking and verifying assessment set up.




    


		
		


		
	

	
	



							
						
											



	
	
	


					

		
					 
Assessment Recipe: Combined formats (may use multiples of other practices) 
		
					


				
				Bb assignments, Bb Tests and Surveys, Blackboard, TurnItIn
											

		
		


		
			
Back to Short Duration Online Assessments page | Other Assessment Recipes


	 
If you prefer to use alternative methods to Blackboard such as eassignment or another tool, check with your Deputy Head of School for Education for guidance.
 

In person arrangements


When this type of exam is sat in person students sit spaced apart in a large hall. Students have a printed question paper with rubric on the front. The question book contains numerous sections, one section may be bubble sheet type assessments (MRQs), another section may be a set of short answer questions, and then a long form essay.  Students write their answers into either one or several answer booklet/s.


How to transfer this online


For each type of section use the appropriate Blackboard tool as set out in the other recipes.  Explain very clearly to the students what should be submitted to which area of your assessment folder. Try to make your various questions and submissions organised in a way that encourages students to undertake the assessment one section at a time submitting as they go along to reduce confusion or risks associated with technical issues.


	 
Exams will be scheduled by the Exams Office so you should match start and end dates/times for the assessment to match the exam schedule.

 

Online considerations


Other than the considerations in the other assessment recipes note that the more complexity you create – the more likely students will experience technical or procedural issues.


Risks and possible mitigation


In addition to the general risks outlined on the main Online Short Duration Assessments web page, review the relevant section in each of the lesson recipes.


Back to Short Duration Online Assessments page | Other Assessment Recipes



    


		
		


		
	

	
	



							
						
											



	
	
	


					

		
					 
Assessment Recipe: Maths questions (including workings) which need to be hand written 
		
					


				
				Bb assignments, Bb Tests and Surveys
											

		
		


		
			
Back to Short Duration Online Assessments page | Other Assessment Recipes


	 
If you prefer to use alternative methods to Blackboard such as eassignment or another tool, check with your Deputy Head of School for Education for guidance.
 

In person arrangements


When this type of exam is sat in person students sit spaced apart in a large hall. Students have a printed question paper with rubric on the front. The question book contains numerous questions to which students may have to draw graphs, write calculations, or otherwise fulfil requirements that are less straightforward where mathematical notation is needed.  Students write their answers into the answer booklet.


How to transfer this online






You have two approaches; one is to create Blackboard test that assess the same knowledge in a method suitable for delivery using the question types in Blackboard.  If so, follow the guidance in the assessment recipe for Computer based tests.


The other approach is to create an assessment paper and answer booklet and ask the students to either print out, complete (for example by drawing on graphs, writing calculations), or by completing a word document and adding notation with a stylus, or to create an exam paper and ask students to write the answers on blank paper in pen. If done on paper the students scanning the page using Office Lens or OneDrive on a mobile device and submitting it to a Blackboard assignment, or submitting the word document to a Blackboard assignment.


Blackboard assignment is recommended because it has a marking interface that allows marking with stylus (pen like tool) which colleagues report is more desirable for marking such assessments.







Blackboard assignment process


A new Final Assessment menu item has been added to most Blackboard courses. Please use this content area to add all content relating to your final assessment, or move your final assessments here if they are elsewhere. If you do not see a Final Assessment menu item, you can create one by following this guide:  How to add a Blackboard menu item.


Within this content area, add a content item explaining that this is where the assessment will be found on the day. This should include the rubric for the assessment (usually found on the front of the physical paper) so that students have a clear understanding of what the format will be. Include information such as who in the school students should contact with questions about the assessment or the paper, including technical questions*. Also, include a link to this Student Guide: Preparing for short duration online assessment (https://elearn.southampton.ac.uk/assessment/preparing-for-short-duration-online-assessment/)






*this is based on on this section of Section IV: General Information and Regulations of the University Calendar that states that “The nominee(s) of the Associate Dean (Education) must be available during the whole of the examination period in case of queries”, to keep the Short Duration Online Assessment process aligned with this exams process that it replaces, the nominee (usually a course instructor) should be available for student queries during their working hours.






 


If you wish for your students to include certain information such as their student number but not their name, or if you wish for students  to use a template for their submission make this clear and provide the students with the resources you require of them.  The file format recommended for students to use to submit is Microsoft Word (doc or docx) in the case of typed responses or PDF in terms of scanned responses.  Ensure that students have the equipment necessary such as a mobile phone running IOS or Android and the Office Lens (android, IOS) or OneDrive (android, IOS) apps in the case of scanned responses.  Include download links for the apps and recommend students practice with the apps in advance.


Create an item in the same content area and attach this PDF, use the text box to add any further information about the exam paper, and state that when the students open the PDF they will find the exam question/s within it.  Set this item to become available when you want students to have access to the exam paper.


Create a Blackboard assignment where the submission will be made by students.  Set the due date to the date and time after which students should no longer be able to submit.  Set the Display After to be the earliest time from which you wish students to make submissions.


	 
Short Duration Online Assessments will be scheduled by the Exams Office so you should match start and end dates/times for the assessment to match the exam schedule.

 

If you allow only one attempt, then if a student submits an incorrect file or makes a mistake they will not be able to resubmit without your assistance.  We recommend allowing three attempts and stating that you will mark the most recent attempt.


Students should use Microsoft Word or Office Lens / OneDrive as appropriate to create the document which should contain all the answers clearly labelled in one single document.  Using a personal computer (e.g. tablet, phone or smart tv) they should use Google Chrome to access Blackboard and submit their assignment whilst ensuring that third party cookies are allowed.  They should ensure that the file they are submitting is not open in another application when submitting to Blackboard.


Online considerations


This assessment will not be proctored (invigilated) which means that it is effectively open book. Students have access to all of Google and the wider web. There is no way to enforce that they complete the assessment with a set time other than the time between the start date and the due date.   This means potentially students could spend all 24 hours (for example) working on the exam. Your questions should be structured to take this into consideration. Questions which hinge on them to remember facts are unlikely to be useful identifying strong students.


Students may experience difficulties if they are unfamiliar with scanning handwritten submissions using their phones, transferring the file to their computer, and then uploading to Blackboard.  Students may forget to scan some pages within their submission due to the stress of the process.


Risks and possible mitigation


In addition to the general risks outlined on the main Online Short Duration Assessments web page, the table below covers risks specific to this scenario along with suggested mitigations.






What could go wrong?
Possible Mitigation




Error found in question/answers after assessment made available.
Ensure that the questions, rubric and all settings have been thoroughly proof read and reviewed for errors by yourself and relevant colleagues in good time before the assessment is due to take place.


Where a mistake is found after students have started the test please follow the appropriate AQSC Policy.





A student does not have a suitable device (smartphone/scanner etc) to create a PDF of their written submission.
AQSC approval for running such assessments is conditional on the proposing School providing assurance that it has made or assured adequate provision for the support of all students taking the assessment including, but not limited to: those taking the assessment outside the UK, those with inadequate connectivity or IT equipment and those experience IT failure during an assessment, provision or assurance of suitable space for all students to take the assessment, accessibility for disabled students and students with specific learning disabilities.




A student is not able to transfer a file they created in Office Lens (or similar app) on their phone to their computer for submission to Blackboard.
In all circumstances it is very beneficial to run a short formative assessment using the same methods and options that would be used in the summative assessment. This allows students to get used to the interface and process and allows staff to ensure that the results they receive are what they expect.




Student loses their internet connection and cannot participate within the time frame.
Agree within your School in advance how you will deal with this.




An assessment has been scheduled to be available for the wrong time, or for too little time.
You can change the settings at any time, but we recommended that you agree within your school or faculty how you will deal with this if it happened. If this is reported after the  start time has passed it may be a question of an appeal to go through the School’s standard process.  See also “Error found in question/answers after assessment made available” at the top of this table.




Blackboard becomes unavailable during the assessment.
Agree a back up plan, for example students should send an email to the school student office with their submission attached, and ensure your students and school are aware of this.




A student sees a score or correct answer feedback when they should not.
Set the ‘hide from students’ setting for the Grade Centre column and make sure the score is not included in the running total.




A student submitted an assignment just as the deadline passed, and their receipt shows it was late, even though they started submitting before the deadline.
Make it clear to students that it is their responsibility to have successfully completed their submission before the deadline, not to have started the submission process before the deadline.  Within your school or faculty you may agree to adapt this, but it is important to have made your approach to this clear to students from the outset.






Marking and giving feedback


To replicate the exams experience it is expected that no feedback will be given to students for the submissions other than a final score which would be delivered via Banner Self Service and the “Marks Release” process.  More information will be placed here shortly.  In the meantime it is important to ensure that marks are hidden from students.


Back to Short Duration Online Assessments page | Other Assessment Recipes



    


		
		


		
	

	
	



							
						
											



	
	
	


					

		
					 
Assessment Recipe: Multiple Choice / Defined Response / Computer based tests 
		
					


				
				Bb Tests and Surveys, Blackboard
											

		
		


		
			
Back to Short Duration Online Assessments page | Other Assessment Recipes


	 
This page is advice for running Short Duration Online Assessment, if that is not the type of assessment you are running then you should not follow this guidance.
 

In person arrangements


When this type of exam is sat in person students sit spaced apart in a large hall. Students have a printed question paper with rubric on the front. The question book contains a multitude of multiple response questions sometimes known as “bubble sheets”.  Questions are set out for each of which there may be one or more correct answers and additional distracters.  Students select out of the possible answer which answer/s they believe to be correct.


A further example is the computer based exams that have taken place in workstation rooms on the University campus.  These were run by the Exams Office and iSolutions.  Students used a dedicated exam launcher that was custom built for our workstation rooms, from where they could launch into a secure environment where they had only access to the test taking software.  Frequent backups were taken and invigilators patrolled the workstation room communicating with the Exams office and iSolutions using two way radios.  If a computer failed the invigilator would move the student to another computer where they could continue their exam.  This practice was the result of more than ten years of development and lesson learning between the Exams Office and iSolutions, you can read more about this topic here.


ExamStart is a tool used to secure computer-based, invigilated exams. It only works on University workstations within the University network. It can only be used by students who are physically attending exams on campus.


How to transfer this online


A new Final Assessment menu item has been added to most Blackboard courses. Please use this content area to add all content relating to your final assessment, or move your final assessments here if they are elsewhere. If you do not see a Final Assessment menu item, you can create one by following this guide:  How to add a Blackboard menu item.


Within this content area, add a content item explaining that this is where the assessment will be found on the day. This should include the rubric for the assessment (usually found on the front of the physical paper) so that students have a clear understanding of what the format will be. Include information such as who in the school students should contact with questions about the assessment or the paper, including technical questions*. Also, include a link to this Student Guide: Preparing for short duration online assessment (https://elearn.southampton.ac.uk/assessment/preparing-for-short-duration-online-assessment/)






*this is based on on this section of Section IV: General Information and Regulations of the University Calendar that states that “The nominee(s) of the Associate Dean (Education) must be available during the whole of the examination period in case of queries”, to keep the Short Duration Online Assessment process aligned with this exams process that it replaces, the nominee (usually a course instructor) should be available for student queries during their working hours.






 


Create a Blackboard test, set the creation options, then start building the test.  Once ready deploy the test to a content area using our recommended options.


	 
Exams will be scheduled by the Exams Office so you should match start and end dates/times for the assessment to match the exam schedule.

 


These steps assume that the test has already been created but not yet deployed to a content area for students to access.


	 
Select the ▶ symbol to expand each section.
 



1. Create the test

Create a Blackboard test, set the creation options, then start building the test.


When creating test questions and answers, use the Accessibility checker for each and follow its guidance to resolve any accessibility issues.







2. Deploy the test

Once ready deploy the test to a content area using the below settings that are particularly relevant for a Short Duration Online Assessment.


Open test in new window: No – this reduces the possibility of students accidentally logging out of Blackboard in a different window and being locked out of the test


Make available to students: Yes – students will not be able to access the test unless this option is set.


Multiple Attempts: tick and set a two attempts.  If a student is locked out of their test they can start at least one more attempt before requiring technical support or raising an appeal.


Score attempts using:  set this to set to First Marked Attempt if more than one attempt is set as an option (above). Students should be told that the second attempt is only to be used if they had technical issues with the first attempt (not an opportunity to improve their score). They should contact the school office, or the course instructor, if they want the second attempt counted.


Force Completion: Do not tick.  If you tick this box, then once participants have begun the Test, they may not exit and continue working on it at a later date.  This means if students lose their internet connection or have a power outage while taking the test they will not be able to complete it.


Set timer: Do not set a timer.  Students who lose their connection will find that the timer continues even though they have lost the ability to continue their attempt.


Display after/until: tick the boxes and set the dates and times to correspond with dates and times that the assessment should be accessible to students.


Password: to avoid increased support requests, do not set a password.


Due Date: do not set a due date, use the display until settings to set when the test is no longer accessible.  We recommend this because if you allow exceptions for AER students they will not affect the due date, and the due date is not a setting which can be added to exceptions.


Include this Test in Grade Centre Score Calculations: Do not tick – this will prevent the score of the test affecting the running total in the Grade Centre, potentially giving students an indication of their score even if results are hidden.


Hide results for this test completely from the instructor and the Grade Centre:  Do not tick: ticking this box will mean the results will never be recorded, once students start taking this assessment this option can never be undone.


Show test results and feedback to students: untick everything.  In most circumstances for summative assessment, the marks should not be shown to students after they have completed the assessment.  This can be adjusted later to reveal feedback at a set date if desired.


Test presentation: One at a Time – this keeps the screen space small, and keeps a connection with Blackboard when students move to the next question.


Prohibit Backtracking: Do not tick.  If this is ticked students will not be able return to an earlier question.


Randomise Questions: Tick if you wish for the order of questions in the test to be presented randomly.  This affects the question order.  Note that other areas of randomisation are:


    

  • When building a test question you may specify if possible answers (where applicable) should be presented in a random order.
    • 

  • When building a test you can state that there should X number of questions chosen at random from a question pool. Multiple pools can be used.
    • 



After submitting these options, make sure to follow the steps in this guide to ensure that the results will not be visible to students.





3. Important- Hide Blackboard GradeCentre column to ensure that results are not visible to students 

It’s really important to ensure that students do not see scores before you want them to.  Follow the steps in this guide to ensure that the results will not be visible to students.




Online considerations


This assessment will not be proctored (invigilated) which means that it is effectively open book. Students have access to all of Google and the wider web. While you can set a timer and enforce the attempts are made only within a limited time this invites an array of issues if the student experiences a fault and is not able to continue within the time limit, this is why we do not recommend setting a time limit in online tests. This leaves only the time between the start date and the display until date.   This potentially means students spend the whole of the availability period working on the exam. Your questions should be structured to take this into consideration. Questions which hinge on them to remember facts are unlikely to be useful identifying strong students.


Risks and possible mitigation


In addition to the general risks outlined on the main Online Short Duration Assessments web page, the table below covers risks specific to this scenario along with suggested mitigations.






What could go wrong?
Possible Mitigation




Error found in question/answers after assessment made available.
Ensure that the questions, rubric and all settings have been thoroughly proof read and reviewed for errors by yourself and relevant colleagues in good time before the assessment is due to take place.


Where a mistake is found after students have started the test please follow the appropriate AQSC Policy.





Student is disconnected and cannot continue assessment.
Note: This advice may change subject to AQSC policy


Allow at least two attempts on the assessment when deploying the assessment.


Do not select Force Completion. For details of these and other settings view this page.





An assessment has been scheduled to be available for the wrong time, or for too little time.
Scheduling can be adjusted when the test is deployed. If this needs to be done after the test you must refer to the AQSC Policy which will explain how respond to this should it arise.




Blackboard becomes unavailable during an assessment.
Allow at least two attempts on the assessment when deploying the assessment.


Do not select Force Completion. For details of these and other settings view this page.

If Blackboard does not become available again within the time window available for the student to complete the assessment it may be a question of an appeal to go via the School’s standard process.





A student sees a score or correct answer feedback when they should not.
Leave the “show test results and feedback to students” options empty in the Test Options. Set the ‘hide from students’ setting for the Grade Centre column and make sure the score is not included in the running total.




A student was still doing a test when the deadline was reached and did not submit on time, leaving their attempt marked “in progress”
You can submit a test on behalf of a student if required.






Marking and giving feedback


To replicate the exams experience it is expected that no feedback will be given to students for the submissions other than a final score which would be delivered via Banner Self Service and the “Marks Release” process.  More information will be placed here shortly.  In the meantime it is important to ensure that marks and feedback are hidden from students and ensure when deploying the test that no feedback options are ticked.


Back to Short Duration Online Assessments page | Other Assessment Recipes



    


		
		


		
	

	
	



							
						
											



	
	
	


					

		
					 
Assessment Recipes 
		
					


				
				Bb Assessment, Bb assignments, Bb Question Types, Bb Tests and Surveys, TurnItIn
											

		
		


		
			
Back to Short Duration Online Assessments page


Assessment Recipes


If you:


    

  1. Have obtained the necessary approvals from your DHoSE and AQSC, 
    2. 

  2. Have taken steps to ensure risks are mitigated (including, but not limited to, the information above and in the assessment recipes),
    3. 

  3. Have satisfied yourself that the necessary functionality can be achieved using the tools above.
    4. 



Then review the following recipes which have been designed to answer the following questions:



Back to Short Duration Online Assessments page



    


		
		


		
	

	
	



							
						
											



	
	
	


					

		
					 
Short Duration Online Assessment Considerations 
		
					


				
				Bb Assessment, Bb Grade Centre, Bb Question Types, Bb Tests and Surveys, Bb Top tips, eLearn Review
											

		
		


		
			
What should you be aware of when considering using a Short Duration Online Assessment in your module?



	 
Select the ▶ symbol to reveal answers to questions.
 

What to consider first



Is there any other way of assessing your learning outcomes?

Schools that wish to offer one or more on-line traditional exams and/or one or more on-line open book exams in AY 2020/21 are asked to request approval from AQSC. Approval is conditional on the proposing School providing assurance that it has made or assured adequate provision for the support of all students taking the assessment. Therefore, it is worth considering whether your learning outcomes can be assessed by a means other than a Short Duration Online Assessment. It’s worth seeking advice and guidance from CHEP, the Faculty’s Associate Dean for Education, and your Deputy Head of School for Education.


It’s also worth keeping in mind that the University has little experience with Short Duration Online Assessments delivered remotely to students’ own devices; computer-based exams run centrally in the past have focussed on the use of controlled environments such as Workstation computers and Faculty laptops.


In terms of support, while ServiceLine is available to resolve technical queries there are no current plans to provide dedicated and scalable support to “in flight” Short Duration Online Assessments.





Has your request to run a Short Duration Online Assessment been approved by AQSC?

Contact your Deputy Head of School Education to find out more about this process.





Have you considered students in different time zones?

Be mindful of those students outside of the UK time zone when selecting certain times for tests to be available .  Allowing the assessment to be taken at any time of day will be more inclusive for those outside of the UK.




Back to top of page.


Frequently Asked Questions


Available Tools



What is ExamStart and can I use it?

ExamStart is a tool used to secure computer-based, invigilated exams. It only works on University workstations within the University network. It can only be used by students who are physically attending exams on campus.





What is OnDemand and can I use it?

OnDemand is the new name for Questionmark Perception. It is a multiple choice / defined response system that integrates with our ExamStart system.


A limited number of licences for OnDemand are available to the University. To manage this demand those who have already used OnDemand in the past may continue to do so. New authors should only be recommended OnDemand if no other tool can serve the specific requirement needed (such as Review and Record if purchased).





What is the lead in time for additional features like Review and Record?

At least four weeks from a University mandate and the provision of a charge code for purchase.





What are Blackboard tests and can I use them?

Blackboard allows for tests to be built within a Blackboard course. A number of question types are available and tests can be deployed with a variety of options. Tests reside within their own course and can be delivered only to students on that same course. Tests may be copied from one course to another by instructors. Blackboard tests had been recommended only for formative use previously and OnDemand for summative use. As such iSolutions does not have experience supporting the use of Blackboard tests for summative use.


When making a Blackboard test, instructors are completely in control of all aspects of the setup and deployment of the test.  Conversely with OnDemand there has been a technical checking process because the scheduling of the assessment is done by iSolutions.




Back to top of page.


Question types



What’s the word limit for text entries?

In Blackboard there are two main question types for receiving text entry.


Essay: Blackboard state that there is no word limit.


Short Answer: Blackboard state that there is no word limit.  You can set how many lines the answer box should have, up to 6 lines.  This set an expectation of how long the answer is expected to be.


In OnDemand there is no word limit.


Our strong recommendation is to avoid using short answer and essay question types in Blackboard or OnDemand assessments.  It is much more efficient to use Blackboard Assignments, eassignment, or Turnitin if you require students to write short answers or essays.  There is also a current issue when writing feedback for short answer or essay questions in Blackboard tests: the text editor does not show the buttons for formatting text, so only plain text can be entered when writing feedback for these question types.


Refer to the comparison table below for further information on question types.




Back to top of page.


Automatic marking



What’s possible in the way of automated marking?

In Blackboard and OnDemand tests all question types except for Short Answer, Essay, and File Response are marked automatically.  It is much more efficient to use Blackboard Assignments, eassignment or Turnitin if you require students to write short answers, essays or upload files.


Refer to the comparison table below for further information on automatic marking of different question types.




Back to top of page.


Presentation / set-up



I have a PDF file of an exam paper, can I upload it to Blackboard and it automatically become a test that will be marked automatically?

No. When creating a test using a file import process, only a spreadsheet based file can be used. You may find it more practical to use the PDF exam paper as the basis for a Blackboard assignment, where students could upload a scan of their answer which could then be marked on line using Bb Annotate, which includes pen based annotation. The use of Blackboard tests tends to require a bit of thinking first about which question type is most appropriate, and whether the question should be changed in any way to better suit the potential of using a Blackboard test. Creating computer-based assessments like these tends to be front loaded in terms of effort, with the return on investment coming over time from less work required for marking.





Can tests be divided into sections, with several questions in one section?

Not in Blackboard tests.


In OnDemand you can divide assessments into sections, using question blocks, but students cannot move between those question blocks once a block has been submitted.





Can questions include images?

In Blackboard and OnDemand tests, yes.





Can options include images?

In Blackboard and OnDemand tests, yes.





Can questions include video?

In Blackboard tests, yes.  In Blackboard you can upload a video file up to 2500 MB into the question text box, but without using Panopto students may not be able to play that video.  However due to the recurrent issues that students report of problems where they have to “click to play” videos hosted on Panopto, it is not recommended to use videos in questions without thorough testing before hand.  The use of video increases the internet bandwidth required to complete a test.


Videos are not recommended in OnDemand. Videos are normally hosted externally and we do not recommend embedding external content.





Can I randomise the display of questions?

In Blackboard tests both the questions and the order of answers to questions (where applicable) may be randomised.





Can I build pools of questions, from which random questions can be automatically selected in an assessment?

Blackboard: You can build pools of questions, these are contained within the course in which you are building them.  You can then build tests that take for example 5 questions at random from pool A, 10 questions from pool B, and 2 questions from pool C.


The use of question pools is for confident advanced users because diagnosing issues with assessments based on question pools is by its nature more difficult.


OnDemand: Question banks are supported, but normal advice is to use the pool to make a static exam per sitting so that if a question is incorrect none or all of the students are disadvantaged.





Is there support available to get the assessment set up?

Step by step guides are available on elearn.southampton.ac.uk




Back to top of page.


Feedback



How do I hide results from students?

Blackboard: Leave the “show test results and feedback to students” options empty in the Test Options. Set the ‘hide from students’ setting for column in Grade Centre and make sure the score is not included in the running total.  See https://elearn.southampton.ac.uk/blackboard/assessment/hiding-or-showing-marks-to-students/

OnDemand: Yes.





Can I import questions in a spreadsheet?

Blackboard: You can create questions in excel, using a specific format specified by Blackboard, save it as a tab separated text file and then upload to Blackboard.


OnDemand: Yes.





How do I export results and analyse data?

Blackboard: See https://elearn.southampton.ac.uk/blackboard/testandsurveyresults/ and https://elearn.southampton.ac.uk/blackboard/itemanalysis/


OnDemand: view this guide




Back to top of page.


Security



How can I prevent cheating?

General approach:


Typically tests tend to focus on the ability to recall and organise information.  Design test questions to focus more on higher order and critical thinking that requires the application, analysis, and evaluation of learning, rather than simple knowledge recall.


Blackboard: with the current functionality and features available to us it is very difficult to prevent cheating.


Measures that would require additional cost and are not currently available at UoS:


Lockdown the browser:  A commercial add on is available which will lock down the test taker’s computer.  This still does not prevent test takers from photographing the screen during the test, or working with others or accessing resources using another device.  The use of lockdown browsers tends to introduce increases in support calls from test takers and tends to restrict the type of device they may use to take the test.


Online proctoring: While there are many commercial online proctoring services there are just as many guides on how to bypass them.  The use of online proctoring may also have privacy implications and would not necessarily have been part of what test takers believe they “signed up for”.


Mitigation techniques for Blackboard tests:


1. Use question pools and randomisation.


While building question pools is much more effective in OnDemand, you can build them in Blackboard.  Question pools can be used to make it more likely that test takers will receive different questions when taking the test.


When building the questions, you can select for the answers to be displayed in a random order, for example in multiple choice and multiple answer questions.


When deploying the test you can select that the questions are shown in a randomised order.


2. Ensure no feedback is shown once the test is complete.


When setting the deployment options set not to show any feedback to students, such as correct answers or what the possible answers were.


3. Make the test available during a tighter time period.


The less time that the test is available, either through using the availability criteria or by setting a timer, the less time there is for the questions to be shared between test takers.   The other side of this coin is that the use of timers or limited availability restrictions can have a negative impact, see the risk minimisation table.


4. Set questions to appear one at time.


If questions are shown all at once it is much simpler for a test taker to take a photo of the whole test and share it quickly onwards.


5. Create a “pre-test” that has to be completed before the real test becomes available.


You could create a “pre-test” with questions such as “I will follow the University’s academic integrity policy” and “I will ensure I have read all the guidance before taking the test” where using adaptive release the real test will not become available until this pre-test has been satisfactorily completed.


6. Use the Blackboard test access log


After a test has been taken you may use the Blackboard test access log to review a detailed list of all the interactions a student had with the test.





How can I protect my questions so I can use them again next year?

You can’t. Students can take a picture on their phone, or write down the questions, or video themselves taking the assessment.





Can questions be screenshotted?

See above.





Can I observe/record students while they’re taking the test?

To our knowledge, online proctoring is not available at UoS.





Can I send students messages while they’re doing the test, without them having access to other people/info through the messaging tool?

No




Back to top of page.


Accessibility



Has web accessibility testing been performed on Blackboard / OnDemand Tests.

Blackboard: not specifically for tests, but you can read our Blackboard accessibility statement for information about Blackboard accessibility.





Can individual students be allocated longer to complete the test?

Blackboard: Yes, go to https://elearn.southampton.ac.uk/blackboard/deploytest/ and scroll to the “Exceptions” section.


OnDemand: Yes, although all wording and guidance is to not have time limits as the system is designed for invigilators to time students




Back to top of page.


When Assessments Go Wrong



How can I guarantee it won’t go wrong?

In uncontrolled environments such as where students are completing computer-based assessments outside of an invigilated University computer based exam location there is no way to assure that problems will not occur.  However, a  number of mitigation strategies are suggested below.





Can students do it offline?

No.





Does it save as they are going along?

Blackboard: In most circumstances yes, although there are Blackboard Knowledge Base issues that state that sometimes this does not happen. We recommend students click the Save Answer button manually as they answer questions.


OnDemand: Yes.





Will ServiceLine offer dedicated support while the assessment is live?

To our knowledge this is not planned.




Back to top of page.


Minimising risk



General guidance

In all circumstances it is very beneficial to run a short formative assessment using the same question types and options that would be used in a summative assessment.


This allows students to get used to the interface and process and allows staff to ensure that the results they receive are what they expect.





What to tell students

Further recommendations to make to students are:


    

  • Ensure you are using a computer (not mobile or tablet) on a supported operating system with a supported web browser.
    • 

  • Ensure your computer is plugged in to a power source.
    • 

  • Ensure your computer is connected to the internet using a wired (Ethernet) connection.
    • 

  • Using an incognito window, access Blackboard via https://blackboard.soton.ac.uk direct, not via Sussed.  Use only this one tab to access Blackboard.  Do not open Blackboard in multiple tabs.
    • 

  • Select the Save Answer boxes, if available, as you progress through the test.
    • 

  • When using a mouse, do not double click on any buttons or boxes.
    • 

  • Do not use the browser “back button”, navigate the test using Blackboard’s own navigation interface.
    • 

  • When the test has been completed, take a full screen screenshot of the page that states your assignment has been submitted, and ensure the data and time is shown.
    • 



Example of what a students sees when they submit.





Expand to reveal the Scenario-based mitigations table





What could go wrong?
Blackboard Mitigation




Error found in question/answers after assessment made available.
If students have already started the assessment you may not be able to change a question that students have already attempted or if they are currently attempting the assessment.


It may be best to wait for the assessment to have been completed by all students, then delete the question with the issue and regrade the answers.





Student is disconnected and cannot continue assessment.
Allow at least two attempts on the assessment when deploying the assessment.


Do not select Force Completion. For details of these and other settings view this page.





A student requires extra time to complete the assessment but has not been given this extra time.
Exceptions can be added when the test is deployed, but if this is reported after a test start time has passed it may be a question of an appeal to go via the School’s standard process.




An assessment has been scheduled to be available for the wrong time, or for too little time.
Scheduling can be adjusted when the test is deployed, but if this is reported after a test start time has passed it may be a question of an appeal to go via the School’s standard process.




Blackboard becomes unavailable during an assessment.
Allow at least two attempts on the assessment when deploying the assessment.


Do not select Force Completion. For details of these and other settings view this page.

If Blackboard does not become available again within the time window available for the student to complete the assessment it may be a question of an appeal to go via the School’s standard process.





A student does not have the appropriate equipment or connectivity to complete an assessment.
Part of the approval process for Short Duration Online Assessments is conditional on this risk being mitigated.




A student inadvertently changes their answer by scrolling the mouse wheel.
With ordering, matching, and jumbled sentence questions, answers are selected with a drop down menu.  When the drop down is selected, using the mouse wheel will cycle through the options within the drop down.  It is possible for students to be unaware that a drop down is still selected as they try to scroll down a page using a mouse wheel.


Mitigations for this are:


    

  • Make students aware of this risk.
    • 

  • Avoid using question types where a drop down menu is used to select an answer.
    • 

  • Use “question at a time” mode to reduce the need to scroll.
    • 







A student sees a score or correct answer feedback when they should not.
Leave the “show test results and feedback to students” options empty in the Test Options. Set the ‘hide from students’ setting for the Grade Centre column and make sure the score is not included in the running total.




A student was still doing a test when the deadline was reached and did not submit on time, leaving their attempt marked “in progress”.
You can submit a test on behalf of a student if required.








Back to top of page.


Blackboard / OnDemand Comparison table



Expand to reveal the table





Feature
Bb Test
OnDemand




Question type






Multiple Choice
Yes
Yes




Multiple Answer
Yes
Yes




Ordering
Yes
Yes




Matching
Yes
Yes




Fill in the blank
Yes
Yes




Fill in multiple blanks
Yes
Yes




Short answer
Yes not recommended, better to use Bb assignment, eassignment or Turnitin .  There is a current issue when writing feedback for short answer or essay questions in Blackboard tests: the text editor does not show the buttons for formatting text, so only plain text can be entered when writing feedback for these question types.
Yes, but they are not marked, so better to use an assignment tool




Essay
Yes not recommended, it’s better to use Bb assignment, eassignment or Turnitin.  There is a current issue when writing feedback for short answer or essay questions in Blackboard tests: the text editor does not show the buttons for formatting text, so only plain text can be entered when writing feedback for these question types.
Yes, but not recommended




Calculated Formula
Yes 


Note: Specify answer should use decimal points rather than significant figures to avoid confusion.

Yes




Calculated Numeric
Yes
No




Either / Or
Yes
Yes




File Response
Yes not recommended, it’s better to use Bb assignment, eassignment or Turnitin .
Not Recommended




Hot Spot
Yes (there have been issues with this in the past but they appear to have been resolved.)
Yes




Jumbled Sentence
Yes
No




Quiz Bowl
Yes
No




True / False
Yes
Yes











Automatic marking






Multiple Choice
Yes
Yes




Multiple Answer
Yes but it is recommended to state how many answers should be selected in the question and not allow partial credit.
Yes




Ordering
Yes
Yes




Matching
Yes
Yes




Fill in the blank
Yes
Yes




Fill in multiple blanks
Yes
Yes




Short answer
No
No




Essay
No
No




Calculated Formula
Yes
Yes




Calculated Numeric
Yes
N/A




Either / Or
Yes
Yes




File Response
No
No




Hot Spot
Yes
Yes




Jumbled Sentence
Yes
N/A




Quiz Bowl
Yes
N/A




True / False
Yes
Yes











Presentation/set up






Can it be divided into sections, with several questions in one section?
No
Yes




Can questions include images?
Yes
Yes




Can options include images?
Yes
Yes




Can questions include video?
Yes?
Not recommended




Can random questions be allocated to each student?
Yes
Yes




Can questions be imported in a spreadsheet?
Yes, if in specific format
Yes if just text and no multiline questions.











Feedback






Can marks be hidden from students?
Yes
Yes




How do I export results and analyse data?
See this guide, and this guide.
See this guide




Does it send results to Banner?
No
No











Security






Can questions be screenshotted?
Yes
Yes




Can I observe / record students while they’re taking the test?
No
Only once the university commits to purchasing Review and Record




Can I send students messages while they’re doing the test, without them having access to other people/info through the messaging tool?
No
No











Accessibility






Can individual students be allocated longer time to complete the test?
Yes, follow this guide and scroll to the “Exceptions” section.
Yes, although all wording and guidance is to not have time limits as the system is designed for invigilators to time students











When it all goes wrong






Can students do it offline?
No
No




Does it save as they are going along?
Automatically, but recommended that students use the Save Answer button as they go to be certain.
Yes




Will ServiceLine offer dedicated support while the assessment is live?
No
No








Back to top of page.



These steps assume that the test has already been created but not yet deployed to a content area for students to access.



1. Prepare groups to organise AER students

Identify any students with additional exam requirements (AER) are expected to take the test.  There may be some who require more time than others.


Using the Blackboard groups tool, create a manual enrol group for each AER group.  Name each group so it will be clear which AER group it refers to, e.g. “AER + 30 mins extra”. Set “Group is visible to students” to  No and tick “Create Smart View for this group” add the corresponding students to each group.





2. Deploy the test

Follow the steps in the guide to deploying the test, settings that are particularly relevant for a Short Duration Online Assessment are listed below.


Open test in new window: No – this reduces the possibility of students accidentally logging out of Blackboard in a different window and being locked out of the test


Make available to students: Yes – students will not be able to access the test unless this option is set.


Multiple Attempts: tick and set a least two attempts.  If a student is locked out of their test they can start at least one more attempt before requiring technical support or raising an appeal.


Score attempts using: highest mark.  If more than one attempt is taken, the mark of the highest attempt is used.


Force Completion: Do not tick.  If you tick this box, then once participants have begun the Test, they may not exit and continue working on it at a later date.  This means if students lose their internet connection or have a power outage while taking the test they will not be able to complete it.


Set timer: If you do use a timer be aware that students who lose their connection to the quiz (e.g. network or power issue, closing tab accidentally.


Display after/until: tick the boxes and set the dates and times to correspond with dates and times that the assessment should be accessible to students.


Password: to avoid increased support requests, do not set a password.


Exceptions: add exceptions for AER student groups, you can allow more attempts, extra time if using a timer, and allow for extended display until dates.


Due Date: do not set a due date, use the display until settings to set when the test is no longer accessible.  We recommend this because if you allow exceptions for AER students they will not affect the due date, and the due date is not a setting which can be added to exceptions.


Include this Test in Grade Centre Score Calculations: Do not tick – this will prevent the score of the test affecting the running total in the Grade Centre, potentially giving students an indication of their score even if results are hidden.


Hide results for this test completely from the instructor and the Grade Centre:  Do not tick: ticking this box will mean the results will never be recorded, once students start taking this assessment this option can never be undone.


Show test results and feedback to students: untick everything.  In most circumstances for summative assessment the marks should not be shown to students after they have completed the assessment.  This can be adjusted later to reveal feedback at a set date if desired



Test presentation: One at a Time – this keeps the screen space small, and keeps a connection with Blackboard when students move to the next question.


Prohibit Backtracking: Do not tick.  If this is ticked students will not be able return to an earlier question.


Randomise Questions: Tick if you wish for the order of questions in the test to be presented randomly.  This affects the question order.  Note that other areas of randomisation are:


    

  • When building a test question you may specify if possible answers (where applicable) should be presented in a random order.
    • 

  • When building a test you can state that there should X number of questions chosen at random from a question pool. Multiple pools can be used.
    • 






3. Ensure results are not visible to students

Follow the steps in this guide to ensure that the results will not be visible to students.




Case Studies


Case studies will appear here once available.


Back to top of page.



    


		
		


		
	

	
	



							
						
											



	
	
	


					

		
					 
How to take a Blackboard Test 
		
					


				
				Bb Assessment, Bb Tests and Surveys, Blackboard, How to
 Bb Guides for students
													0 Comments
							

		
		


		
			
This guide explains how to take a test or survey in Blackboard.


What are Blackboard Tests?


Blackboard tests are usually used for short answer or multiple choice type questions. A link to a Test will look like this:




Tips for taking a test


    

  • Check your internet connection. A wired connection is more reliable than wifi. If you are using wifi, sit as close to the router as possible.
    • 

  • If you are using a laptop don’t rely on the battery, but plug into power if you can.
    • 

  • Don’t use the Blackboard mobile app when taking a test – some of the question types do not work correctly.
    • 

  • Read any instructions from your instructor carefully before starting the test. If you have any questions contact your instructor.
    • 

  • Avoid refreshing your page, closing your browser or using the Browser back button (use the navigation buttons in the test to move from question to question).
    • 

  • If you do lose connection during the test don’t panic. You can usually resume the test and your work will auto-save as you type or answer questions. If you have any issues during the test – see the Getting help section.
    • 



Starting and resuming a test


    

  • Select the link to the test to view information about the test settings. This will NOT start the test, but from here you can Begin the test when you are ready or resume an attempt if you have one in progress.
    • 



Test Settings


Before starting the test you will be able to view information about the test settings.


    

  • You may see a description and instructions provided by your instructor.
    • 

  • Timed Test – If this is a timed test a count down timer will display at the top of the screen. The test may be set to auto-submit when the time is up or you may be able to continue but your submission will be marked as late.
    • 

  • Multiple attempts – This shows how many times you can attempt the test. If this is a final assessment, the second attempt is only to be used if you have a technical issue with the first attempt.
    • 

  • Force Completion – This is not usually set so you will be able to close the test and resume it later or get back to the test if you lose connection. If this is a timed test, the timer will continue to run.
    • 



Submitting a Test


    

  • Select Save and Submit when you are sure you have finished the test or if you are prompted to do so in a timed test.
    • 

  • If you have missed any questions you will be warned about this before you agree to Submit.
    • 

  • You will not receive an email receipt when you submit a test. You will see a submission screen with details of your submission directly after you submit the test. Take a photo or screenshot of this screen. You can use this as proof that you completed the test.  You will not be sent an email confirmation. If the test is still available on Blackboard, you can select the link to the test to view the details of your attempts.
    • 



Watch a video about taking a test




Getting Help


You should usually contact your course instructor if you have any issues while taking a Test, but you may be asked to contact the School Office for support if the test is towards a final assessment. Please follow any specific guidance that is given with the test.






 


Example Question Types: Multiple Choice






Example Question Types: Multiple Choice


Multiple Choice questions require you to select one correct answer response from a choice of possible answers. Choose the correct answer by clicking on the radio button to the left of the correct answer.






Example Question Types: Multiple Answer


Example Question Types: Multiple Answer


A Multiple Answer question is like a Multiple Choice question, except that there are one or more correct answers. You will not usually be told how many of the answers are correct. Select the answers by placing ticks in the boxes beside the correct answers. Note that all (and only) the correct answers must be selected in order to score points.






Example Question Types: True or False


Example Question Types: True or False


True or False questions require you to decide whether a statement is true or false. Select the correct answer by clicking into the radio button to the left of the correct answer.






Example Question Types: Ordering


Example Question Types: Ordering


An ordering question requires you to put several answer values into the correct order. The order could be, for example, procedural (e.g. step one, step two) or chronological as in the example above. To put the answers into the correct order use the drop down menus to the left to select the correct order.






Example Question Types: Fill in the Blank


Example Question Types: Fill in the Blank


A fill in the blank question requires you to type a word into the text box in answer to the question above it. It may require you to complete a missing word from a sentence or answer a question. The answers are not case-sensitive, but must be spelt correctly. The test author may have allowed for common spelling mistakes or variations.






Example Question Types: Matching


Example Question Types: Matching


A matching question requires you to match one set of values to another. You may be required to match text to text, picture to text, picture to sound and so on. Use the drop down menus beside each item on the left to determine with whichever item they match on the right. Note that a selection from the drop-down menu (e.g. A) may be the correct match for more than one part of the question.






Example Question Types: Quiz Bowl


Example Question Types: Quiz Bowl


A quiz bowl question presents you with an answer.  You have to type in the question.






Example Question Types: Jumbled Sentence


Example Question Types: Jumbled Sentence


A jumbled sentence question requires you to complete a sentence, choosing words from a drop down list.






Example Question Types: Hot Spot


Example Question Types: Hot Spot


A hot spot question requires you to click on a specific part of a picture in order to answer the question.






Example Question Types: Fill in Multiple Blanks


Example Question Types: Fill in Multiple Blanks


A fill in multiple blanks question requires you to complete a sentence or paragraph by filling in several blanks.






Example Question Types: Calculated Formula


Example Question Types: Calculated Formula


A calculated formula question requires you to perform a calculation.






 




 




 








 







    


		
		


		
	

	
	



							
											
			

			

			
 1  2  3  4