diff --git a/docs/feature-progress.md b/docs/feature-progress.md deleted file mode 100644 index 66ae29fa..00000000 --- a/docs/feature-progress.md +++ /dev/null @@ -1,81 +0,0 @@ -# Feature Progress - -The following is a breakdown of the features left to implement to get Orca up and running. Bottlenose is included here as changes to that project are necessary for Orca to funciton as intended. - -
- -## Orca - -### API/HTTP Server [ ] - -- Validate incoming JSON [ ] - - Create modules for Validation Schemas [ ] -- Create/Enqueue Grading Job [ ] - - Create Job in Orca (Postrgres) DB [X] - - Enqueue Job in Redis DB [ ] -- GET Grading Job(s) [ ] - - Paginated List of Enqueued Jobs [ ] - - Getting details on a specific job(?) [ ] -- Queue Management [ ] - - Deleting a Grading Job [ ] -- Authentication [ ] -- POST Grading Job Results [ ] - - Restart/Replace Grading Container [ ] - - Remove Matching Job(s) in PostgreSQL DB [ ] - - Send to Bottlenose (PUT request) [ ] - -### Web Client [ ] - -- Set Up Typescript [ ] -- Status Page [ ] -- Queue List [ ] - - Pagination [ ] - - Details on Specific Job [ ] - - Remove Job from Queue [ ] -- Authentication [ ] - -### Docker [ ] - -- Create Dockerfile [ ] -- Set Up Docker Container Management [ ] - -
- -## Grading Containers - -### Redis/Grading Queue - -- Retrieval of `GradingJobConfig` from the Queue [ ] - - Lock/Semaphore on a job such that only "this" container has access [ ] - -### Grading Script [ ] - -- Execution of Scripts [ ]s - - Retries [ ] - - Saving to Log [ ] - - Producing TAP Output [ ] - - Error Handling [ ] - - Assets (e.g., Javalib and Tester library) [ ] -- Retrieval of Student/Other Files [ ] -- Send Results to Orca [ ] - -### Docker - -- Dockerfile [ ] -- Cutting off/Reinstating Network Access On Demand [ ] - -
- -## Bottlenose - -### API [ ] - -- PUT Grading Job Results [ ] - - Parse JSON and TAP [ ] - - Update Grade in DB [ ] - -### Full Stack - -- Generate Job Config based on Grader [ ] - - Build commands [ ] - - Test commands [ ] diff --git a/docs/grading_queue_spec.md b/docs/grading_queue_spec.md deleted file mode 100644 index 725f730d..00000000 --- a/docs/grading_queue_spec.md +++ /dev/null @@ -1,313 +0,0 @@ -# Grading Queue: A Redis-based Architecture - -The Orca Grading Queue is ephemeral in nature: all jobs are created with the promise of eventual removal. Redis was the chosen tool to implement the Grading Queue, as an in-memory storage system with quick access to its elements. - -## Data Definitions - -A full implementation of the grading queue requires three pieces of functionality: - -1. Knowing when a job is ready to be extracted and graded. -2. Retrieving the next job to be graded for a given team or user. -3. Obtaining the correct `GradingJob` object. - -### Reservations: ZSet - -The data definiton for **(1)** must support a priority-based ordering of jobs and contain information to be used in **(2)**. - -Redis' `ZSet` data structure is an ordered set with keys sorted by a given _score_ value (in practice, this is more usefully typed as an `OrderedMap`). This can be utilized to create a priority queue of IDs to be popped off when a grading VM is ready to extract a new job. - -``` -ZSet Reservations => { - <"team" | "user">..nonce: release_time - <"immediate">..nonce: release_time -} -``` - -The set used under the key `Reservations`, and each `ZSet` member is a unique identifier needed for retrieving the associated grading job -- either directly or through an ordered list of jobs for team/user submissions. Members receive a score equivalent to the timestamp of when they should be released for grading. - -Each `ZSet` key also has a _nonce_, which is used to ensure students spamming the queue with multiple jobs reserve a corresponding spot. - -`GradingJob`s are enqueued to be processed based on a given priority OR to be processed immediately. In the latter case, the `ZSet` member string is composed of **the grading job's key, which is a string unique to that job generated by the Orca web server**. - -### SubmitterInfo: List - -Orca stores the keys for a user/team's jobs in a Stack. Orca implements "Last In First Out" behavior so that the most recent job, which is expected to yield the most useful feedback on a submission, will be graded first. - -This can be implemented with a Redis `List`, where the list is mapped to a key containing the given user/team ID. - -``` -List SubmitterInfo.<"team" | "user">. => [ job_key_0, job_key_1, ..., job_key_n ] -``` - -The stack order is maintained by only ever prepending job keys and popping the first one off the list. - -### GradingJobKey -> GradingJob - -`GradingJob` objects are individually mapped to a unique key utilizing the STRING data structure in Redis. - -``` -String (Key) => GradingJob -``` - -Only one `GradingJob` exists per `GradingJobKey` so that the most recent grading script specifications provided by a professor are used for grading. For instance, if a student submits and a professor updates an assignment's test files and clicks "regrade" for the student, this will update the grading job under that `GradingJobKey` such that when the job gets graded, it will run with the updated specs. - -### [BONUS] Nonces: Set - -For optimization purposes, users may need to know what nonces are in use by a collation type/id in the Reservations `ZSet`. - -These nonces are stored under the key `Nonces..`. - -### Invariants - -Given the definitions above, the following invariants define a valid state of the queue: - -1. All grading jobs correspond to a single key, and vice versa. -2. Job keys are unique within _and_ across `SubmitterInfo` lists. -3. Each reservation belongs to a single, unique job key. -4. Each reservation has a single, unique nonce. -5. The number of nonces in the `Nonces` set is equal to the number of reservations for that user or team. - -## Queue Operations - -All operations are performed by either the web server or a worker. - -### Commandments of Queue Operations: - -A couple of rules must be followed in order to ensure that the state of the queue is never corrupted: - -1. **Thou shall not run any queue operations without a lock.** While not ACID compliant, Redis offers up the Redlock algorithm as a method to implement locking resources to ensure that two clients do not interfere with each other. Failure to use this could result in undefined behavior -- such as two workers trying to pop off the same job. -2. **Thou shall not update, delete or create anything in the queue without a transaction**. Redis does not implement transactions, but it is possible to implement them through a combination of (a) utilizing the Redis `MULTI` command -- allowing an atomic execution of some series of operations _and_ (b) building some symblance of rollback steps -- also achieved by grouping them onto an `MULTI` command. Failure to rollback faulty operations could result in breaking of the previously defined invariants. - -### [PREREQ] Web Server: Generating a Unique Job Key - -As mentioned in the [Data Definitions](./orca_data_definitions.md) document, in order to create a `GradingJob` Orca must create an identifier unique to each given job prior to enqueuing. - -```typescript -import { createHash } from 'crypto'; - -funciton generateOrcaKey({ key, response_url }: GradingJobConfig): string { - const hash = createHash("someHashAlgorithm"); - hash.update(key + response_url); - return hash.digest('base64'); -} -``` - -This is accomplished by hashing the concatenation of a `GradingJobConfig`'s `key` property with its `response_url` property. - -The hash is then encoded using base64 to ensure that no `'.'` character becomes part of the key. This is necessary to preclude undefined behavior when splitting keys using `'.'` as a delimiter. - -### Web Server: Adding a Job to the Queue - -Adding a job to the queue is a create-or-update function (or a PUT in HTTP terms). If a job is already seen to be in the queue, then its contents will be updated without adding new data (e.g., reservation, nonce, etc) to the queue. - -In the create case, Orca will queue up grading jobs based on their user ID or team ID. With this value, the generated orca key will be pushed onto the `SubmitterInfo` stack to ensure it will be the first job to get graded. - -```typescript -function createOrUpdateGradingJob(jobConfig: GradingJobConfig) { - const orcaKey = generateOrcaKey(jobConfig); - const arrivalTime = time.now(); - if (!EXISTS(orcaKey)) enqueueJob(jobConfig, orcaKey, arrivalTime); - SET(orcaKey, jobConfig); -} - -function enqueueJob( - { key, collation, priority }: GradingJobConfig, - orcaKey: string, - arrivalTime: number -) { - const nextTask = `${collation.type}.${collation.id}`; - const releaseTime = priority + arrivalTime; - const nonce = generateNonce(releaseTime, collation); - RPUSH(`SubmitterInfo.${nextTask}`, orcaKey); - ZADD("Reservations", `${nextTask}.${nonce}`, releaseTime); - SADD(`Nonces.${collation.type}.${collation.id}`, nonce); -} - -function generateNonce(releaseTime: number, collation: Collation): number { - let nonce = releaseTime; - while (SISMEMBER(nonce.toString())) { - nonce++; - } - return nonce; -} -``` - -`GradingJobConfig`s sent to Orca will contain a _priority_, which is a delay to be placed on a job. For now, assume `priority = (# of subs in last 15 mins) * 1 min`. - -The _release time_ of a job is defined by the addition of the job's arrival time and its priority; this value is used as the score of the reservation in the `Reservations` ZSET. - -When generating the member string for this reservation, the release time is used to create a unique nonce -- starting with this value and then incrementing it until the value is found not to exist in the ZSET. - -The nonce of the reservation is then cached for other update functionality. - -### Web Server: Adding a Job for Immediate Grading - -A professor may want to submit a job for immediate grading in the event they change the original test criteria. - -While it's possible to place a job at the front of the line for a student/team by giving it a priority of 0, jobs may be added afterwards and thus a regrade attempt may be waiting for an arbitrarily long time. For this reason, if a non-immediate job already exists, it is replaced with a new immediate job. - -Jobs added to the queue for immediate grading are added to the `Reservations` `ZSet` using its server-generated key instead of the team or user ID. This allows the job to bypass the `SubmitterInfo` list, ensuring that it cannot be cut in line. - -```typescript -function createOrUpdateImmediateJob(jobConfig: GradingJobConfig) { - const orcaKey = generateOrcaKey(jobConfig); - if (!jobInQueue(jobConfig)) createImmediateJob(jobConfig, orcaKey); - if (nonImmediateJobExists(jobConfig)) upgradeJob(jobConfig, orcaKey); - SET(job.key, job); -} - -function nonImmediateJobExists({ key, collation }: GradingJob) { - for (jobKey in LRANGE(`SubmitterInfo.${collation.type}.${collation.id}`)) { - if (jobKey === key) return true; - } - return false; -} - -function upgradeJob(job: GradingJob) { - removeNonImmediateJob(job); - createImmediateJob(job); -} - -function removeNonImmediateJob( - { collation }: GradingJobConfig, - orcaKey: string -) { - LREM(`SubmitterInfo.${collation.type}.${collation.id}`, 0, orcaKey); - const nonce = SPOP(`Nonces.${collation.type}.${collation.id}`); - ZREM("Reservations", `${collation.type}.${collation.id}.${nonce}`); -} - -function createImmediateJob(job: GradingJob, orcaKey: string) { - arrival_time = time.now(); - ZSET(Reservations, `immediate.${orcaKey}`, arrival_time); -} -``` - -Its reservation score/release time is set directly to the arrival time to ensure there is no delay. - -### Web Server: Moving a Job in the Queue - -Jobs in the queue may either be: - -- Moved to the back of the queue. -- Moved to the front of the queue. - -```typescript -enum MoveJobAction { - RELEASE = "release", - DELAY = "delay", -} - -interface MoveJobRequest { - nonce: number; - jobKey: string; - moveAction: MoveJobAction; - collation: Collation; -} -``` - -Moving a job requires its unique key, the nonce used in the job's corresponding `Reservations` `ZSet` key, the type of action to be taken, and collation data. Jobs submitted for immediate grading cannot be moved. - -```typescript -function moveJob(req: JobMoveRequest) { - const { nonce, jobKey, moveAction, collation } = req; - switch (move_action) { - case MoveJobAction.RELEASE: - const job = GET(jobKey); - upgradeJob(job); - break; - case MoveJobAction.DELAY: - delayJob(req); - break; - default: - throw Error(); - } -} - -const MOVE_TO_BACK_BUFFER = 10; // seconds - -function delayJob(req: JobMoveReuquest) { - [last_job, last_priority] = ZRANGE("Reservations", -1, -1, WITHSCORES); // Gets job at back of queue - const new_priority = last_priority + MOVE_TO_BACK_BUFFER; - ZADD( - "Reservations", - `${req.collation.type}.${req.collation.id}.${req.nonce}`, - new_priority - ); - const subInfoKey = `SubmitterInfo.${req.collation.type}.${req.collation.id}`; - LREM(subInfoKey, req.jobKey); - RPUSH(subInfoKey, req.jobKey); - return new_priority; -} -``` - -A job is moved to the back of the queue by assigning it a new priority of `priorityOf(lastJobInQueue) + buffer` as its score in `Reservations`. The buffer is used to guarantee that the given job will be placed at the back. The job is also placed at the back of `SubmitterInfo`. - -Jobs are moved to the front of the queue by being replaced with an immediate job, ensuring they cannot be skipped. - -#### Examples - -- Release a job -> job at front of queue -- Delay a job -> job at back of queue -- Release job_1, release job_2 -> job_2 in front of job_1, job_1 in front of job_3...job_n -- Delay job_1, delay job_2 -> job_2 is behind job_1, job_1 is behind job_3...job_n - -### Web Server: Deleting a Job from the Queue - -A job can be deleted given its key, as well as its collation and nonce for nonimmediate jobs, in the `Reservations` `ZSet`. - -```typescript -function deleteJob( - jobKey: string, - nonce: number, - collation: Collation? = undefined -) { - if (collation) { - LREM(`SubmitterInfo.${collation.type}.${collation.id}`, 1, job_key); - ZREM("Reservations", [collation.type, collation.id, nonce].join(".")); - SREM(`Nonces.${collation.type}.${collation.id}`, nonce); - } else { - ZREM("Reservations", `immediate.${nonce}`); - } -} -``` - -All jobs have their data ejected from `Reservations`. Non-immediate jobs also have data removed from both the `Nonces` and `SubmitterInfo` set and list, respectively. - -### Worker: Extracting Job From Queue - -Grading jobs are popped from the queue using their associated `GradingJobKey`. - -```typescript -function getNextJob() { - const nextTask = ZPOPMIN("Reservations"); - const nextTaskInfo = next_task.split("."); - - let jobKey, collationType, collationID, nonce; - if (nextInfo[0] === "immediate") { - [_, jobKey] = nextTaskInfo; - } else { - [collationType, collationID, nonce] = nextTaskInfo; - SREM(`Nonces.${collationType}.${collationID}`, nonce); - jobKey = LPOP(`SubmitterInfo.${collationType}.${collationID}`); - } - reutrn JSON.parse(GETDEL(jobKey)); -} -``` - -This is either obtained directly by popping off the first item from `Reservations`, or by using the collation data from the reservation to pop the first job key off of `SubmitterInfo`. - -All data pointing to the job is deleted, either through a "pop" or deletion function. - -### Worker: Re-enqueuing a Job - -If a worker is to be shut down prior to finishing a job, that job should be re-enqueued to ensure it is not completely lost. - -```typescript -function reenqueueJob(job: GradingJob) { - ZSET("Reservations", `immediate.${job.orca_key}`, job.created_at); - SET(job.orca_key, JSON.stringify(job)); -} -``` - -To ensure the job will be picked up next by whatever worker is ready, it is enqueued with an immediate reservation with a timestamp equal to its original arrival time. diff --git a/docs/orca-workflow.md b/docs/orca-workflow.md new file mode 100644 index 00000000..89d849d1 --- /dev/null +++ b/docs/orca-workflow.md @@ -0,0 +1,119 @@ +# End-to-End Workflows +This document covers the full end-to-end workflow of all Orca processes, including: +* Creating a job +* Updating a job +* Building a grader image +* Deleting a job + +## Some Quick Notes +A couple items to cover for both the clarity and brevity of this document: +1. Assume all endpoint names are preceded by `/api/v1`, as all operations fall under this sub-route. +2. All data definitions referenced in the below text and diagrams can be found in the [data definitions document](./data_definitions.md) file. +3. As [Bottlenose]() is Orca's main "customer," it will be considered the caller of Orca's API. However, this could easily be replaced by _any_ client who implements the functionality needed to connect with Orca's APIs and have results returned to it. + +## The Base Workflow + +When a student submits an assignment to a client utilizing Orca, the client will submit a `GradingJobConfig` to the Orchestrator. The Orchestrator will enqueue the job and inform the client. + +```mermaid +sequenceDiagram + actor S as Student + participant B as Bottlenose + participant O as Orchestrator + participant Q as Postgres (Queue) + participant W as Worker + S->>+B: PUT /submissions/create + B-->>-S: HTTPS 200 OK + B->>+O: PUT /gradingJob GradingJobConfig + O->>+Q: createGradingJob(GradingJobConfig) + Q-->>-O: void + O-->>-B: HTTPS 200 OK + W->>+Q: getNextJob() + Q-->>-W: GradingJob + create participant C as Container + W->>C: runGradingJob(GradingJob) + C->>+B: getGraderAndSubmissionFiles() + B-->>-C: List[File] + C->>+B: pushResults(GradingJobResult) + B-->>-C: HTTPS 200 OK + destroy C + C-->>W: void + S->>+B: getGradeForSubmission(Submission) + B-->>-S: Grade +``` + +Eventually, the job will be picked up by a single worker out of a pool of VMs actively grading jobs. + +**TODO: Add image download step from worker** + +The worker then spins up a Docker container to grade the job, and given that the grading script is able to be compiled and executed, the container will send grading results back to the client before being shut down. + +**TODO: Add line about getting files. Use loop for GET reqs for files.** + +## Building a Grader Image + +Before clients can create a grading job, Orca mandates that there is either an existing Docker image for grading in its file system or one in the process of being built. + +```mermaid +sequenceDiagram + participant B as Bottlenose + participant O as Orchestrator + B->>+O: PUT /gradingJob (GradingJobConfig) + O-->>-B: HTTPS 400 Bad Request +``` + +Attempting to submit a job to the queue without an image will return a `400 Bad Request` error back to the client. + +Clients can submit an `ImageBuildRequest` to Orca via its `/buildImage` endpoint. + +```mermaid +sequenceDiagram + participant B as Bottlenose + participant O as Orchestrator + participant P as Postgres + participant I as Image Builder + B->>+O: POST /buildImage (ImageBuildRequest) + O->>+P: createImageBuildRequest(ImageBuildRequest) + P-->>-O: void + O-->>-B: HTTPS 200 OK + I->>+P: getNextBuildRequest() + P-->>-I: ImageBuildRequest + create participant D as Docker + I->>D: docker build ... + B->>+O: PUT /gradingJob GradingJobConfig + O->>+P: placeJobInHoldingPen(GradingJobConfig) + P-->>-O: void + O-->>-B: HTTPS 200 OK + destroy D + D-->>I: OK + I->>+P: releaseAllJobsInHolding() + P-->>-I: void +``` + +While an image is being built, it is possible to hit the `/gradingJob` endpoint with a `GradingJobConfig`. Orca will handle this request by placing the job in a "holding pen," where that job and any other jobs in the same holding pen will eventually be enqueued _after_ the image has been successfully built. Like jobs that have been enqueued, jobs in the holding pen may be updated. + +```mermaid +sequenceDiagram + participant B as Bottlenose + participant O as Orchestrator + participant P as Postgres + participant I as Image Builder + I->>+P: getNextBuildRequest() + P-->>-I: ImageBuildRequest + create participant D as Docker + I->>D: docker build ... + B->>+O: PUT /gradingJob GradingJobConfig + O->>+P: placeJobInHoldingPen(GradingJobConfig) + P-->>-O: void + O-->>-B: HTTPS 200 OK + destroy D + D-->>I: ERROR + I->>+P: removeJobsInHoldingPen + P-->>-I: List + loop For config in list + I->>+B: POST GraderConfigCreationResult + B->>-I: HTTPS 200 OK + end +``` + +In the event that the image fails to build, any jobs in the holding pen will be removed from Postgres and a notification will be sent back to the URL specified by each `GradingJobConfig`'s `response_url` prop. diff --git a/docs/orca_data_definitions.md b/docs/orca_data_definitions.md index d9921f4c..8f3195b6 100644 --- a/docs/orca_data_definitions.md +++ b/docs/orca_data_definitions.md @@ -1,11 +1,11 @@ -# Data Definitions - Orca/Grading VMs +# Data Definitions This document explores the shape of data exchanged between various points in the Orca workflow, where JSON objects are passed: -1. From grading job source to the Orca Web Server -2. From the Orca Web Server to the Redis Grading Queue -3. From the Redis Grading Queue to the Orca Grading VM -4. From the Orca VM to grading job source +1. From grading job source to the Orchestrator Web Server +2. From the Orca Web Server to the Grading Queue +3. From the Grading Queue to the Worker +4. From the Worker back to grading job source While Orca is mainly suited to integrate with Bottlenose, in theory it could be used with any source that can submit a grading job and accept its result. @@ -39,10 +39,7 @@ Orca needs to know if a job belongs to a single user or a team to maintain an _o ```typescript -enum CollationType { - User = "user" - Team = "team" -} +type CollationType = "user" | "team"; interface Collation { type: CollationType; @@ -56,7 +53,7 @@ interface Collation { The Orca Web Client is filterable by useful metadata for a given grading job. `GradingJob`s specify a dictionary of field names that are mapped to a given identifier. -For example, JUnit Grader for Assignment 1 might have a metadata field of: +For example, the JUnit Grader for Assignment 1 might have a metadata field of: ```json { @@ -72,7 +69,7 @@ For example, JUnit Grader for Assignment 1 might have a metadata field of: ### `FileInfo` -A `FileInfo` contains a URL to files necessary to grade this submission. A MIME type is also included so that the grading VM can download and extract (as necessary) files correctly. +A `FileInfo` contains a URL to files necessary to grade this submission. A MIME type is also included so that the worker can download and extract (as necessary) files correctly. ```typescript interface CodeFileInfo { @@ -90,7 +87,7 @@ Grading jobs have a numeric priority determined by their sources, which is inter Jobs are run inside Docker containers to provide a level of isolation from the local machine. When an assignment is generated, professors will provide a Dockerfile to build their grader image. Orca's web server will build this image and save it to a .tgz file with the name `.tgz`, where `` is the SHA sum generated from the image's Dockerfile. -The grading VM will use the `grader_image_sha`'s value to determine if the image already exists on its local machine or if it should be downloaded from the URL `https:///images/.tgz`. +The worker will use the `grader_image_sha`'s value to determine if the image already exists on its local machine or if it should be downloaded from the URL `https:///images/.tgz`. The script defines the actual grading process, as a state machine specified below. @@ -116,7 +113,11 @@ A `BashGradingScriptCommand` describes a step in the grading script that require Each command could either succeed or fail. If successful and `on_complete` says to _output_, or if failed and `on_fail` says to _abort_, then the script exits and sends the results back to the job's source. -If `on_complete` is not specified, then the state machine goes to the next command in the list. If `on_fail` is not specified, then it will automatically abort the script. +These terminating cases are indicated by the following: +* `on_fail` is either excluded or points to the reserved keyword `"abort"`. +* `on_complete` points to the reserved keywork `"output"`. + +If `on_complete` is not specified, then the state machine goes to the next command in the list. If either key points to the reserved keyword `"next"`, then the state machine will go to the next command. @@ -144,7 +145,7 @@ A `ConditionalGradingScriptCommand` allows the control flow of a script to branc The `GradingScriptCondition` defines how to check for the existence of an object in the file path: either as a file (`'file'`), a directory (`'dir'`), or as either one (`'exists'`). -Simlar to the `BashGradingScriptCommand`, the optional `on_true` and `on_false` keys specify which command to run next. +Similar to the `BashGradingScriptCommand`, the optional `on_true` and `on_false` keys specify which command to run next. These pointers are almost exactly congruent to the `BashGradingScriptCommand`'s `on_complete` and `on_fail` properties, except that a script **cannot exit** from a `ConfitionalGradingScriptCommand`. @@ -153,33 +154,36 @@ These pointers are almost exactly congruent to the `BashGradingScriptCommand`'s ## `GradingJob` ```typescript -interface QueuedJobInformation { - created_at: number; - release_at: number; - orca_key: string; +interface AdditionalJobInformation { + created_at: Date; + release_at: Date; + queue_id: number; } type GradingJob = GradingJobConfig & QueuedJobInformation; ``` -Once a `GradingJobConfig` has been received and validated by the server, Orca generates the requisite infromation from its contents to enqueue the job. This data is represented as `QueuedJobInformation`, and the resulting `GradingJob` is what the server adds to Redis. +Once a `GradingJobConfig` has been received and validated by the server, it is enqueued by creating records in the Postgres database. -While a `GradingJobConfig`'s `key` is unique to its source, that does not mean that it is unique to the queue -- i.e., multiple sources could have configurations with the same key. The `orca_key` is an identifier unique to a job in the Redis queue; it's value is the result of hashing the concatenation of the `GradingJobConfig`'s `key` and `response_url` properties. +All references to the job's information will now be in the form of a `GradingJob`: a combination of the original `GradingJobConfig` and new `AdditionalJobData`. The latter's properties are interpreted as the following: -The `created_at` property is the config's time of arrival at the server and the `released_at` property is that arrival time added with the config's priority. +* `created_at` - the date and time of the job's creation. +* `release_at` - the date and time at which a job is guaranteed to be in the front of the queue. +* `queue_id` - the primary key of the `Job` record in the database.
## `GradingJobResult` -Execution of a grading job will result in a `GradingJobResult` object to send back data to a given job's `response_url`. _Optionally_, a `container_response_url` may be included for local development as if the grading container needs to contact an test echo server also running in a container, then there will be a need for separate `http://localhost` and `http://` URLs. +Execution of a grading job will result in a `GradingJobResult` object to be sent back to a job's `response_url` with its `key`. + +_Optionally_, a `container_response_url` may be included for local development as if the grading container needs to contact an test echo server also running in a container, then there will be a need for separate `http://localhost` and `http://` URLs. ```typescript interface GradingJobResult { output?: string; shell_responses: [GradingScriptCommandResponse]; errors?: [string]; - key: JSONString; } interface GradingScriptCommandResponse { @@ -195,6 +199,6 @@ The output includes the key given in the original job for use by its originator. A successful `GradingJobResult` will _always_ contain `output`. -An unsuccessful `GradingJobResult` will still contain any responses of commands executed by the script. ' +An unsuccessful `GradingJobResult` will still contain any responses of commands executed by the script. If the worker fails (e.g., due to resource limits) or an operation on the server removes the job (e.g., cancelling a job in the queue), the `errors` property will contain a non-empty array.