Showing posts with label Assessments. Show all posts
Showing posts with label Assessments. Show all posts

Wednesday, January 15, 2025

Cheating the Benefits of Cheating with AI


With the popularity and extraordinary, and dubious, claims of the ability of AI (Narayanan & Kapoor, 2024), a heightened concern about students cheating has occurred. Naturally, proctoring and anti-plagiarism companies benefit from this fear and the sales of their ‘solutions’ increase.  The problem with this policing solution is that beyond a point, teachers get into an ‘arms race’ between AI and AI-detection software where the real loser is the institution footing the bill.  One might think that the companies are happy to ‘stir the pot’ to increase anxiety for the benefit of their shareholders. Meanwhile, research has indicated that the level of academic dishonestly has not changed with the prevalence of AI (Lee, et al, 2024) Yet the focus on student policing places instructors in an adversarial relationship with the students, instead of an instructive one. 

 

A Better Solution

 

Low-stakes assignments have the added benefit of deterring academic dishonesty.  Because there is less risk involved in the assignment, there is less incentive to cheat, which could bring about sever consequences.  Why take the risk, when there is so little to gain?

 

Low-stakes assignments, such as threaded assignments, thwart using AI that quickly generates content.  The low-stakes assignments act as scaffold that requires further reflection and meta-cognitive skills that are not easily replicated by the stochastic language models modern AI uses. This will force the would-be culprit to reflect and expend so much energy producing something to submit that it isn’t worth the effort to get whatever the AI model can produce.

 

When assignments:

  •  offer less stress, 
  • supply tools for the students to succeed, 
  • clearly express expectations, and 
  • encourage the student to take control of their learning, 
the students begin to see the value in what they are learning.  This dissuades the learner from cutting corners instead of easily going through the process.  Not only will this lower academic dishonesty in your classes, low stakes assignments will encourage more of your students to become engaged active learners.

 

Low-stakes assignments engage the students in the learning process.  The best way to eliminate academic dishonestly is to remove the incentive to cheat and be up-front about the rules.  Students are often ‘cheating’ because often they have not received adequate instruction and expectations (Waltzer, Bareket-Shavit & Dahl, 2023).  To solve this, explicitly state the acceptable level of AI usage and you will curb the level of unintended violations to your academic expectations. A group of scaffolded low stakes assignments, often scaffolded to create a large assignment, then undermines the pay-off from cheating.

References

Futterman, K (2024) Zeitgeist 6.0: Results of the Campus sixth-annual student-body survey. The Middlebury Campus. Dec 13.

Lee, V., Pope, D., Miles, S, and R. Zarate (2024) Cheating in the age of generative AI: A high school survey study of cheating behaviors before and after the release of ChatGPT. Computers and Education: Artificial Intelligence. Vol 7, December.

Losey, R. (2024) Stochastic Parrots: How to tell if something was written by an AI or a human? 

Mintz, S (2023) 10 Ways to Prevent Cheating. Inside Higher Ed. February 16.

Mollick, E (2023) Centaurs and Cyborgs on the Jagged Frontier: I think we have an answer on whether AIs will reshape work. One Useful Thing. Sep. 16.

Narayanan, A & S. Kapoor (2024) AI Snake Oil: What artificial Intelligence can do, what it can’t, and how to tell the difference. Princeton University Press: Princeton, NJ.

Waltzer, T., Bareket-Shavit, C., & Dahl, A. (2023). Teaching the What, Why, and How of Academic Integrity: Naturalistic Evidence from College Classrooms. Journal of College and Character, 24(3), 261–284.

Wehlburg, K (2021) Assessment design that supports authentic learning (and discourages cheating) Times Higher Education. Nov 24


Wednesday, November 27, 2024

Low-Stakes Assignments for Grading


Did you ever have a dream where you were back in school, you enter a class, and you realize you have a final exam on a topic that you have no idea what it is?  The pressure of exams is so great that it unconsciously affects us decades later.  In fact, high-stakes assignments and testing have been linking to increasing suicidal ideations (Wang, 2016), and higher suicide rates (Kapur, 2021; Singer, 2017). They have been connected to undermining educational goals, perpetrating inequalities, crating unequitable learning environments, and encouraging cheating both from students and educational actors, such as teachers, administrators, and even state officials (Nicols & Berliner, 2007). So then, why do we use high-stakes testing and assignments?  Tradition?

 

Low-stakes assignments when taken individually do not significantly impact a students’ grades. Their purpose is primarily to provide students with a performance indicator.  Students can then reflect on the areas in need of improvement and how to improve. The low-stakes assignment also provides assistive scaffolding by providing regular formative feedback that is frequent and timely (Kuh, Kinzie, Schuh, & Whitt, 2010). They work best when providing formative feedback, starting and continuing throughout the course.

 

Benefits of Low-Stakes Assignments

A few of the benefits of low-stakes assignments include:

  • They provide feedback for instructors about how successful students are learning. This can be particularly effective in environments where it is hard to pick up on subtle clues of students struggling, such as in online or hybrid classes.
  • Allowing instructors to direct students to resources if they need further assistance or support
  • Early feedback opens up communication between students and their instructors, possibly increasing their likeliness to seek help when needed
  • Allowing students to be active participants in the evaluation of their own learning
  • Encouraging students and increasing the likelihood of their engagement and attendance

Many of these will rise your retention rates and help students succeed.

 

Examples of Low-Stakes Assignments

But what would a low-stakes assignment look like? Some examples of low-stakes assignments include:

  •  Self-tests. (ungraded or low-points). These can even be automated with online testing so that it does not take any time in the classroom.  These can also be anonymized to give students comfort. Self-tests are particularly effective when combined with having…
  • Multiple attempts (on questions or whole exam). This feature reduces test-anxiety and allows students to learn from their errors.  When feedback is given for each question, you will notice the best results. The knowledge of that they can take the exam another time also reduces the pressure to cheat (Wehlburg, 2021).
  • Discussion/Collaboration: Students sharing their writing or thoughts with others and get feedback will assist their learning and meeting learning outcomes
  • Multiple submissions of a paper. Feedback from a first submission with time to reflect and rewrite their paper allows students to hone their writing skills.  
  • Reflective journaling. Writing self-reflective content both increases one’s meta-cognitive skills used for learning as well as better develops writing skills.  An added perk is that AI tools have a hard time replicating this type of writing as well.
  •  A Threaded Assignment i.e., breaking down the assignment into several parts.  Individually, the grades or low, but collaboratively the project aggregates to a large assignment, such as a term paper. This technique often proves the scaffolds that help disenfranchised, or otherwise struggling, students succeed. A sample of deconstructing a large assignment into components would be making a thesis paper into the following smaller assignments:

    • Thesis/Abstract
    • Outline
    • Annotated bibliography
    •  1st draft
    • Final draft

These are just a few examples; however, they offer an excellent opportunity for both you and your students to get needed feedback to help improve your course’s student success rate. These also help develop a grading system that can clearly show the steps necessary for mastering the meeting the learning outcomes of the course. 


References

Bayraktar, B (2021) Tip: Many Low Stakes Assignments. Tips for Teaching Professors, Apr 6.

Drabick D. A. G., Weisberg R., Paul L., Bubier J. L. (2007). Keeping it short and sweet: Brief, ungraded writing assignments facilitate learning. Teaching of Psychology, 34, 172–176.

Greenhalgh, S. (2016) The Hidden Cost of Asia’s High Test Scores. The Diplomat. Dec 9

Hale, M. (2018) Thwarting Plagiarism in the Humanities Classroom: Storyboards, Scaffolding, and a Death Fair. Journal of the Scholarship of Teaching and Learning, v18 n4 p86-110 Dec.

Kapur, M (2021) Student suicides put a spotlight on high-pressure exams during India’s pandemic. Quartz. Sept 17.

Kuh, G. D., Kinzie, J., Schuh, J.S., and Whitt, E.J. (2010). Student success in college: Creating conditions that matter. San Francisco, CA: Jossey-Bass.

Nicols, S. and D. Berliner (2007) Collateral Damage: how High-States Testing Corrupts America’s Schools. Harvard Education Press, Cambridge, MA.

Mintz, S (2023) 10 Ways to Prevent Cheating. Inside Higher Ed. February 16.

Singer, S (2017) Middle School Suicides Double as Common Core Testing Intensifies. Huffington Post. Aug 2.

Wang, Liang. (2016). The effect of high-stakes testing on suicidal ideation of teenagers with reference-dependent preferences. Journal of Population Economics. 29.

Warnock, Scott. (2013). Frequent, low-stakes grading: Assessment for communication, confidence. Faculty Focus. Madison, WI: Magna Publications. 

Wehlburg, K (2021) Assessment design that supports authentic learning (and discourages cheating) Times Higher Education. Nov 24.

Friday, July 10, 2020

Do Not Use "Force Completion" and Stranger Things


You are putting together a test for your online course in Blackboard.  As you scroll down to look at the different options after the timing options, you see “Force Completion”.  If a student navigates away from a Test, and the Force Completion option is turned on, the Test is automatically submitted for them by the system.

This test you are creating is online, and we cannot be too careful.  This tool will ensure that students cannot cheat, like a proctor, right?

No.

While this feature is often wrongly attributed to making online examinations more effective by dissuading academic dishonesty, Forced Completion creates more harm than whatever good it is intended to do.  Problems with Internet providers or browsers are more often the culprit than students attempting to abuse the testing time. On countless occasions help desks get calls from distraught students because their system lost the connection and their test was submitted – incomplete. Worse still, there is nothing they can do to help. This has motivated multiple universities to recommend not using, ”Force Completion”

If your goal is to punish students who have unreliable internet, browser, then "Force Completion" is a great tool.  However there is no pedagogical value in this and it creates serious issues of equity in your course.  The unreliability of Force Completion and the technical issues it creates makes it ineffective at what it was designed for at best, and discriminatory at worst.


If an assessment requires forced completion to be effective, it wasn’t much of an assessment to begin with.  Forced Completion offers arbitrary criteria in evaluating students and burdens support services. Effective online assessments allow students to demonstrate their mastery of the subject.

This can be done without forcing student to recite answers in a short period of time with arbitrary punishments divvied out.  The key to online learning is to focus on constructivist techniques for learning, and not passive learning activities.  This is not to say that online tests are not effective, but to suggest how they are used, such as a tool to help students self-assess their understanding, is key to using test successfully.  While this can be difficult to navigate, minimally it means that you should not use Force Completion.


Thursday, March 29, 2018

Gamification: 2 Learning as Improving


Shedding a Traditional Approach to Assessments


It is the first day of class and students enter the course with a dread of failure.  Usually homework and learning opportunities offer only one chance to define their grade which will be more damning and permanent than Hester’s scarlet letter. 

This is often not a fault of faculty being lazy. They simply do not have the time to make the work better and afford students multiple attempts - effectively offering the students a chance to improve. Nevertheless, this results in students dreading assignments and fearing failure.

As an alternative, we should view failure as an opportunity for students to assess the situation and progress. One is reminded of John Stuart Mill’s line in chapter one of his autobiography:

‘A pupil from whom nothing is ever demanded which he cannot do,
never does all he can.’

John Stuart Mill Image with Quote ‘A pupil from whom nothing is ever demanded which he cannot do, never does all he can.’

Learning should be thought of as a way to improve and one does this through multiple attempts.  If you do not succeed at first, you try again – often using different strategies to learn how to solve a problem.

Discouraging students from trying new solutions, even though some may fail, is doing them, and ourselves, a great disservice. If we adopt the mindset that students should have the opportunity to experiment and explore solutions, we may not only find that they will be more open to try harder problems but that we are fostering creative problem solvers.  This requires that we consider creating learning activities that allow for multiple attempts.  These attempts will offer opportunities to assess and improve.

This is one of the keys to the success of the gaming industry.  Gamers recognize that they will fail in their attempts.  They also know that if they keep focused, try multiple times, and often have to think innovatively, they can overcome the challenge. The ability to save progress or use a reset button is not a coincidence. It allows for multiple attempts to test various strategies.


By allowing students to participate in learning activities multiple times so that they can improve their skills, we are fostering a non-defeatist mentality that will benefit our students. Students can explore options, review feed back and see how the lessons they choose to adopt directly affects their success.

We are also mirroring actual learning environments where people must experiment with strategies to determine which will succeed and to learning from their mistake. 

Another important feature of viewing Learning as improving, is that we are encouraging a sense of agency in the students.  Their decisions matter and what learning strategies they employ will directly affect their success.  This will contribute to their success both as a student and as a person.

So when designing your learning activities and assessments, consider offer more time and allowing multiple attempts.  The results may surprise you.

Thursday, March 15, 2018

Extending Time for Tests

You are finally getting around to implementing a test in Blackboard for your course.  You discovered great options, such as:
  • Randomizing the questions and the ordering of their answers,
  • Pools of questions that allow each student to get a random set of questions, ensuring each test is unique
  • Feedback for each question, to re-enforce learning and to guide the student to assess where they need to spend more time improving.
  • Students can take the tests multiple times, to work to improve their knowledge
  • Adaptive Release and automated announcements about the test
  • Tests can be timed.

Everything is going great and then you get a call from the Office of ADA Transition Services. You have a student that requires 90 minutes of time to take your 60 minute test.

At first, this seems scary.  Is there an easy way to do this without having to make another test or having to learn how to develop Groups and merge their grades in the Grade Center.  You already developed a Baroque grading schematic that requires calculus to understand, and you have no idea how this new development will play out in the Grade Center?



Yes there is!  You can take a deep breath and rest assured.  Blackboard made it easy and can solve all these problems.

You simply select Edit Test Options.
Scroll down to section titled “Test Availability Exceptions” and select the “Add User or Group” button.
This will open a window where you can select the student who needs more time.
You will notice you can:
  • allow more attempts,
  • change the time, and even
  • change the availability of the test for the student.
Now hit “submit” and you are done

Blackboard will take care of the rest and the testing will take place seamlessly.

One advantage over regular face-to-face instruction, is that using this medium allows those with ADA requirements to not even notice their exam is any different or feel singled out from the class.
Coincidentally, this is one of the reason a large number of people with disabilities prefer the anonymity of online environments.

So, when you need to extend the time, or add special exceptions to a test, know that you can easily do with the tools we have in Blackboard.  In fact, once you use them, you may wonder why you would ever go back to tradition pen and paper.

Blackboard Test Option Crib Notes


Do you want to investigate the slides from the above video for a tutorial on how to make the most from your online quizzes?  The slide show below will offer you tips on how to take advantage of these features in Blackboard.

Traditional Courses and Online Tests



There are several reasons for adopting online tests for a traditional (face-to-face) course.  The technology employs randomization to ensure each student receives a unique test and that the answer selections vary to prevent cheating.  The online nature also allows computers to increase accessibility for students, and accommodating ADA concerns is easier as well as more discrete.  Further, offering the test outside of class both can remove environmental factors that could impede student performance while freeing up more time for student-teacher interaction.  Finally, in developing successful online exams, we need to rethink exams to promote self-assessment which can be accomplished by treating the exam as an open-book exam.    Over all, this re-evaluation of evaluations assists us in developing a more pedagogically sound course.

 It is important to rethink the actual purpose of an assessment.  Are we trying to get a grade for the student or is the goal to teach the student?  If our goal is the latter you should definitely design the exam with feedback.  Having feedback to correct errors is critical in early stages.  Traditionally students have to wait days, if not weeks, to get feedback on their mistakes. This feedback is sometime not informative as well.  The benefit of online exams is that they can be created such that errors can been identified and students can be supplied with why they are incorrect and where they can go to discover the correct answers.  By empowering students with this knowledge, the test can motivate the student to review material that they have the weakest understanding. Consequently, the students will be working on learning the material better.

By developing tests that afford multiple attempts, students have an incentive to reflect on areas they need improvement and the opportunity to improve.  Swift feedback can encourage students to review areas that they have not yet mastered.  The exam becomes more of an active skill building tool, instead of a quick means to assign a grade.  Coincidentally, since an online environment may have several pitfalls, such as spotty WiFi, having multiple attempts to master the material also become best practices for both practical and pedagogical reasons.

If you haven’t considered adopting online exams instead of having an exam within your class, you should consider trying it once.  There is more work up-front in developing the question sets and the feedback. However, you will find that the adoption is very rewarding.  Besides not having to spend much energy on grading after the exams are taken, your students will better understand the material and succeed.

Principle of Engagement - Guideline 2 C5 Sustaining Effort & Persistence (version 3.0)

  Universal Design for Learning Principle of Engagement - Guideline 2 - Sustaining Effort and Persistence The third principle of UDL, "...