Upload submissions in bulk

One of the most common operations you'll likely want to perform once you've created your course is upload student work to codePost for review. The codePost 101 tutorial explains how to upload single submissions, but this method is impractical for large courses.

In this tutorial, we'll use the codePost API to build a simple script to upload student work to codePost. We'll add in layers of complexity (partners, updating old submissions) in successive sections. If you want to use the codePost GUI to upload submissions (no code required), check out Upload submissions in bulk (GUI).

Single-Student Submissions

We'll start out by building a script to upload submissions corresponding to individual students (i.e. no partners or groups).

First, let's create a simple file structure. We'll use the following.

assignment1/
    student_1/
    	file_1.py	
    	readme.txt
    student_2/
    	file_1.py	
    	readme.txt
    ...
    student_n/
    	file_1.py	
    	readme.txt

Here, we've created one main folder called assignment1/ for our submissions. In this file, each sub-folder corresponds to a student (e.g. student_n/). Within each sub-folders are the files corresponding to the student's submission.

Although we can use the codePost API directly to upload submissions, we've created some handy, official utility functions that you can use to programmatically perform common tasks at a layer of abstraction above the codePost API. In this tutorial, we'll use the upload-to-codePost script.

For a full explanation of how you can tune upload-to-codePost, check out the readme in our repo. For now, all we need to know is the following. Each call of upload-to-codePost will attempt to upload one submission. To do so, we need to supply upload-to-codePost with the following arguments:

api_key : Your API key, which you'll use to authenticate yourself whenever you're using the API. You can retrieve (or set) your API key from https://codepost.io/settings.
course_name : The name of your course represented as a string (e.g. CS101)
course_period : The period of your course represented as a string (e.g. S19)
assignment_name : The name of your assignment represented as a string (e.g. assignment1)
students : The email address of the student for whom we are trying to upload a submission (e.g. [email protected])
files : A comma-separated list of file paths corresponding to the files we want to upload (e.g. assignment1/student_1/file1.py, assignment1/student_1/readme.txt)

A complete call to upload-to-codePost looks like this:

> ./upload-to-codePost 
 -api_key=<my_API_key>
    -course_name="CS101"
    -course_period="S19"
    -assignment_name="assignment1"
    -students="[email protected]"
    -files="assignment1/student_1/file1.py, assignment1/student_1/readme.txt"

However, executing a call manually for each submission is very tedious. Let's write a quick script to iterate over our file structure and make the right call to upload-to-codePost. We'll use Python, but you can use any language you want.

#!/usr/bin/env python3

import os
import subprocess

config = {
  'api_key' : '<YOUR_API_KEY>',
  'course_name' : "CS101",
  "course_period" : "Spring\ 2019",
  "assignment_name" : "assignment1"
}

# Assume file structure as above
for root, dirs, _ in os.walk(config['assignment_name']):
  for subdir in dirs:
    studentEmail = subdir + "@myschool.edu"
    for _, _, files in os.walk(root+'/'+subdir):
      for file in files:
        cmd = ["./upload-to-codePost",
              "-api_key", config["api_key"],
              "-course_name", config["course_name"],
              "-course_period", config["course_period"],
              "-assignment_name", config["assignment_name"],
              "-students", studentEmail,
              "-files"] + list(map(lambda x: root+'/'+subdir+'/'+x, files))

        # Execute upload-to-codePost call
        (errcode, output) = subprocess.getstatusoutput(" ".join(cmd))

This lacks features we'd want in a production-ready script (like error checking) but it implements the basic logic we need: walk through a pre-defined file structure while passing the right arguments to push-to-codePost.

Handling Partners

The script above assumes a one-to-one mapping between submissions and students. Now, let's extend our script to handle partner- and group-based assumes, where students will map to 1 or more students.

We'll need to modify our file structure a bit, since we can no longer embed the email of a student in a folder name (since we might have multiple). Instead, we'll use the following

assignment1/
    1/
    	file_1.py	
    	readme.txt  
        partners.txt
    2/
    	file_1.py	
    	readme.txt  
        partners.txt
    ...
    n/
    	file_1.py	
    	readme.txt  
        partners.txt

We added a new file to each folder called partners.txt. We'll use this file to specify the students corresponding to the submission. Each instance of the file will contain a comma-separated list of students belonging to the submission.

[email protected],[email protected]

Next, we need to modify codePost_upload_script.py to read this file, parse it, and use its contents to set the -students parameter of upload-to-codePost.

#!/usr/bin/env python3

import os
import subprocess

config = {
  'api_key' : '<YOUR_API_KEY>',
  'course_name' : "CS101",
  "course_period" : "Spring\ 2019",
  "assignment_name" : "assignment1"
}

# Assume file structure as above
for root, dirs, _ in os.walk(config['assignment_name']):
  for subdir in dirs:
    for _, _, files in os.walk(root+'/'+subdir):

        # we don't want to upload partners.txt
        toUpload = list(filter(lambda x: x != 'partners.txt', files))

        # get students from partners.txt
        partnersStr = open(root+'/'+subdir+'/'+'partners.txt').read()

        cmd = ["./upload-to-codePost",
              "-api_key", config["api_key"],
              "-course_name", config["course_name"],
              "-course_period", config["course_period"],
              "-assignment_name", config["assignment_name"],
              "-students", partnersStr,
              "-files"] + list(map(lambda x: root+'/'+subdir+'/'+x, toUpload))

        # Execute upload-to-codePost call
        (errcode, output) = subprocess.getstatusoutput(" ".join(cmd))

Voila! You can use the above script as a starting point for your upload submission upload workflow. If you want to build something more customized, don't forget to check out the utility functions repo and our API docs.