Tags

, ,

This tutorial can be embedded in Blackboard via an LTI tool.  It will appear under Tools area in courses.  As an LTI tool it detects the course id and course role to show the specific content and allow instructors start creating Blackboard tests.  The tutorial is designed to help new instructors consider their options in remote assessment and examinations.

These are the tutorial steps:

Step 1: Welcome to Remote Assessment and Examinations

Assessment is a very important part of teaching. It helps learners and instructors to measure how well objectives are met. When coupled with meaningful and quality feedback, assessment can be a valuable learning experience.

The goal of any educational opportunity is for the learner to acquire knowledge and skills as a direct result of the learning experience. Without assessment tools, it is impossible to determine whether learners have acquired the requisite knowledge and skills, thereby meeting the given objectives. Classroom assessments can include a wide range of options – from recording anecdotal notes while observing a student to administering standardized tests. The options can be roughly divided into two categories – formative assessments (low-stakes, ongoing feedback) and summative assessments (examinations, final projects).

Classroom assessment and online assessment must be valid and reliable:

Reliability refers to the extent to which assessments are consistent. For example, instruments such as classroom tests and national standardized exams should show consistent results whether a student takes the assessment in the morning or afternoon; one day or the next.

Validity refers to the accuracy of an assessment — whether or not it measures what it is supposed to measure. Even if a test is reliable, it may not provide a valid measure.

Blackboard Learn system can help in accomplishing all these objectives remotely.

 

Step 2: Classroom Assessment Alternatives

While this tutorial will focus on online tests and exams, question-style assessments can be substituted with other assessment techniques that lend themselves well to remote learning:

  1. Course projects
  2. Recorded video presentations
  3. Written paper or research articles
  4. WIKI, Blogs, or web page projects
  5. Poster sessions with peer critique
  6. Annotated portfolios
  7. Op-Ed pieces sent to local newspaper

A summative online, question-style exam may be used in combination to any of the alternatives.

Step 3: To Proctor or not to Proctor

A major decision in designing online, question-style examinations is whether the learner must be monitored during the exam, or can the learner take the exam home and complete it on their own. Without at-home monitoring, most solutions should be considered a take-home exam. The following considerations should be given when making the decision to monitor students at home:

Exam monitoring

  1. Hardware requirement: Computer camera and microphone is often required on the learner’s home computer.
  2. Added cost: A hired person or AI looks at the learner, and often their room, while the exam is being completed. This means there is typically an added cost per exam or per person.
  3. The learner is asked to show personal ID to the camera.
  4. All activity on the computer is recorded and monitored. This monitoring and required software installation can be considered privacy and security concerns.
  5. The learner’s eyes are often tracked for attention on the screen versus other areas of the environment.
  6. This technique focuses on prevention of lapses to academic integrity by deterrence. Complex and expensive forms of hacking can sometimes overcome even the best-in-class monitoring services. However, the idea of monitoring works for majority of issues in academic integrity.

Take-home online exams

  1. Student completes an online test on most connected devices from phones to laptops.
  2. The exam can often be delivered as a final learning activity, not just as assessment.
  3. Students do not have to install any added software on their devices.

Step 4. Prepare for Online Testing

As with other assignments, it is important to develop clear and detailed instructions when creating tests for your course. The instructions should include the due date, number of questions, an estimation of how long it should take to complete the exam, and if there are any imposed time limits. For example, if you were giving a pop-quiz the length of time required from the learner would be quite different from the time it would take to complete an exam. Provide this information to your learners at the start, so that they are prepared to complete it.

When you set up a test, determine how many times learners will be able to take it (some systems allow users to take a quiz multiple times), whether there are penalties for incorrectly answered questions or attempts, and how multiple attempts are aggregated (e.g., will the grade be based on the average score, highest, lowest).

Some systems even allow instructors to implement security blockers to mitigate cheating with or without active monitoring of learners. Be sure to configure all of the exam options before making it available to learners. For simplicity, consider using identical settings for all exams in your courses.

 

Step 5. Academic Honesty Considerations

Instructors are often concerned about academic honesty when considering online assessment. While it may not be possible to eliminate cheating completely, you can take steps to minimize it:

  1. Have learners sign an academic integrity policy that defines cheating and outlines expectations for honesty.
  2. Use email or chat to verify the level of a learner’s test content knowledge after an unexpectedly good performance.
  3. Limit both duration of the test’s availability and time limit for taking it.
  4. Randomize the order in which the questions appear and the order of each question’s answers.
  5. Provide a different test for those taking the test late.
  6. Use a test as a learning tool, rather than as a measure of fact recall. Allow learners more than one attempt with the intent that they will return to the course content to learn what they missed the first time. Create a follow-up assignment requiring them to demonstrate knowledge in areas not mastered at the time of the test.
  7. Pair online objective tests with subjective measures such as writing assignments, projects, applied problems, and group work.
  8. Administer tests in a proctored location, if possible.
  9. If you suspect two learners of collaborating on tests, compare the times the tests were taken, and their answers and grades.

 

Step 6. Online Tests vs In-Classroom Tests

Blackboard online tests provide high level of accessibility letting learners access tests on most Internet-connected devices and on their own schedule. Blackboard tests also provide important statistics for each question after a test is completed:

  1. Average score: Sum of all scores divided by the number of participants.
  2. Average time spent: The average completion time for all submitted attempts.
  3. Discrimination: Indicates how well questions differentiate between learners who know the subject matter and those who don’t. Shows the number of questions that fall into these categories: good (greater than 0.3), fair (0.1 – 0.3), poor (less than 0.1 categories, or can’t calculate (i.e., a question’s difficulty is 100% or all learners received the same score on a question).
  4. Difficulty: Categorizes the questions based on how many learners answered the question correctly: easy (greater than 80%), medium (between 30% and 80%), or hard (less than 30%).

 

Step 7. Online Testing: Next Steps

Consider the following options for online testing:

  • Blackboard Tests: You create questions of various types including true/false, multiple-choice, multiple-answer, essay, and others. Questions make up tests, which can then be placed, or deployed, into content area. Learners are not monitored when taking the tests.
  • Respondus LockDown Browser and Monitoring: Once you create a standard Blackboard Test, you can require the Respondus LockDown Browser software to limit what can be done on the computer during the test. You can also use Respondus LockDown Monitoring, which will proctor the student during the exam.
  • Your department may have access to other online testing solutions that integrate with Blackboard Learn. Some examples include ExamSoft, ProctorU, or others.

 

End of the tutorial.

LTI Setup.

To configure the tutorial as an LTI tool:

1. Register the domain if you have not already done so:

domain: apps.dataii.com

Register for the global key, secret registered.

2. Create a placement for a course tool including students.

URL:  https://apps.dataii.com/bb/examcontinuity/

3. Icon: exams3

 

Tutorial Previews.

Student preview: https://apps.dataii.com/bb/examcontinuity/

Instructor preview: https://apps.dataii.com/bb/examcontinuity/?role=instructor