Make grading anonymous

There are plenty of good reasons to anonymize student work prior to evaluation.

• Anonymous grading => easier to deal with grader conflicts of interest. If work is anonymous, graders don't have to report conflicts of interest prior to grading because they can grade any submission.
• Anonymous grading => prevents implicit bias from impacting evaluation.

0. Agenda

To fully anonymize grading, we need to do two things.

1. Enable codePost's Anonymous Grading Mode, which hides student emails from graders. To learn more about the nitty-gritty details, check out this doc.
2. Remove any identifying information from student code prior to upload. codePost won't remove this info automatically.

1. Enable Anonymous Grading Mode

Let's start by creating a new Assignment from the Admin Console.

Then we'll enable Anonymous Grading Mode for this new Assignment.

That's it! Now let's move on to step 2.

2. Remove identifying info from student code

For this tutorial, we'll assume that the only identifying information present in student code takes the form of emails. We need to find any emails within code and replace them. Let's write a quick Python script to do this.

We'll start by writing a function to take in a file and output a version without any emails embedded in the file.

def removeEmails(code):
  replacement_string = "<REDACTED>"

  # from https://emailregex.com/
  # Will remove emails from any domain
  email_regex = r'([a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+)'

  return code.replaceAll(email_regex, replacement_string)

Nice. Now we need to iterate over all of our submissions, calling this function on each file.

We'll assume we have your submission files arranged in the format that's required for uploading to codePost via the GUI.

submissions/ # this can be called whatever you want
   [email protected]/ # the sub-folder names correspond to students in your course
      file1.py # files you want to upload go here
      file2.txt
   [email protected]/
      file1.py
      file2.txt
   [email protected],[email protected] # create partners by adding multiple students to the same directory, comma-separated

Now, we'll write the some code to crawl this file structure.

#!/usr/bin/env python3

def removeEmails(code):
  replacement_string = "<REDACTED>"

  # from https://emailregex.com/
  # Will remove emails from any domain
  email_regex = r'([a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+)'

  return code.replaceAll(email_regex, replacement_string)

# Loop over file structure and anonymize student code
for root, dirs, _ in os.walk(config['folder_name']):
  for subdir in dirs:
  	for _, _, files in os.walk(root+'/'+subdir):
    	for file in files:
      	toUpdate = file.open(file, "w")
      	newContents = removeEmails(toUpdate.read())
      	toUpdate.write(newContents)
      	toUpdate.close()

That's it! You can run this file in the folder containing your submissions folder to anonymize the submissions prior to upload.

cd /folder_name
./anonymizeSubmissions

Nice job. You just set up an anonymous grading workflow in 5 minutes!