Testing Java Spring applications

In brief

In our Java testing library there is a possibility to test web backend applications based on Spring library. Testing web backend applications written in Python (for example, using Django) is not yet fully supported.

Note: At the moment, for checking Spring applications on web interface, Hyperskill runs hs-test v7.1 but some features decsribed below may require newer version of hs-test, in this case it will be mentioned specifically. If you are developing tests for problems that meant to be run on Hyperskill web interface it is required to use v7.1 instead of release-SNAPSHOT version of hs-test in the root build.gradle. This version will be updated here as soon as it will be updated on Hyperskill.

Spring tests

If you want to develop code problems that are meant to be tested via Hyperskill's web interface then you should treat every stage as a separate individual code problem.

The most convenient way to write Spring tests is to use dynamic tests. However, you're not going to start any programs, Spring server will be running all the time while all tests are executed. Dynamic tests will allow you to execute requests in a way that each request will be treated as a separate test.

Setup Spring Boot template

We'll start from the end of the regular Java project setup.

Your setup so far should look like this:

Every Spring application should contain its own dependencies in a separate build.gradle file at the root of every stage.

Here its possible content (Note: currently, this is all the dependencies installed for web testing Spring applications on Hyperskill. You may use all of them or part of them, but no more if you are writing tests to be graded in web interface. Contact Hyperksill so we can add other useful dependencies)

plugins {
    id 'java'
    id 'org.springframework.boot' version '2.3.1.RELEASE'
    id 'io.spring.dependency-management' version '1.0.9.RELEASE'
}

sourceCompatibility = 11

repositories {
    mavenCentral()
}

sourceSets.main.resources.srcDirs = ["resources"]

dependencies {
    implementation 'org.springframework.boot:spring-boot-starter'
    implementation 'org.springframework.boot:spring-boot-starter-actuator'
    implementation 'org.springframework.boot:spring-boot-starter-web'
}

Also, you should create a resources/application.properties file and set up a port 8889 and open external shutdown possibility via actuator, so tests can shut down Spring application externally. Use the following lines (you can add other lines, but these ones should be here):

server.port=8889
management.endpoints.web.exposure.include=*
management.endpoint.shutdown.enabled=true

Now your setup may look like follows:

If you see the addition (excluded) near your files, you should right-click on the file and select Course Creator -> Include into Task

Note: On Hyperskill's web interface, only a single file can be shown, so you should mark files that users shouldn't change as hidden. Normally, in a regular project, only tests are hidden but keeping in mind that users should solve this problem via web interface you should leave only one file as visible. So, tests, build.gradle, application.properties and all except one Java sources should be hidden. You can hide files right-clicking and selecting Course Creator -> Hide from Learner

After all, to publish problems on Stepik do a right-click on the icon with 4 empty squares and select Course Creator -> Upload Hyperskill Lesson on Stepik

Example

Let's test some class named SpringDemoApplication.

First of all, you need to extend SpringTest class and pass this class to the constructor along with the port number on which you are going to run and test this Spring application.

import org.hyperskill.hstest.stage.SpringTest;
import springdemo.SpringDemoApplication;

public class DemoTest extends SpringTest {
    public DemoTest() {
        super(SpringDemoApplication.class, 8889);
    }
}

You can omit the port, this way the library will scan the properties file for the port number.

super(SpringDemoApplication.class);

If you want to use a database, you can pass the name of the database as the second parameter.

super(SpringDemoApplication.class, "../demodb.mv.db");

You can pass all 3 parameters:

super(SpringDemoApplication.class, 8889, "../demodb.mv.db");

Provided the following lines in the properties:

spring.datasource.url=jdbc:h2:file:../demodb
spring.datasource.driverClassName=org.h2.Driver
spring.datasource.username=sa
spring.datasource.password=password

spring.jpa.database-platform=org.hibernate.dialect.H2Dialect
spring.jpa.hibernate.ddl-auto=update

spring.h2.console.enabled=true
spring.h2.console.settings.trace=false
spring.h2.console.settings.web-allow-others=false

And in the build.gradle

dependencies {
    // ...
    runtimeOnly 'com.h2database:h2'
    implementation 'org.springframework.boot:spring-boot-starter-data-jpa'
    // ...
}

Now, let's imagine that on URL /api/user/ the Spring application should return a list of all users in the JSON format (in this case, there should be 2 users in the output). The test will look like this:

import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import org.hyperskill.hstest.dynamic.input.DynamicTestingMethod;
import org.hyperskill.hstest.mocks.web.response.HttpResponse;
import org.hyperskill.hstest.stage.SpringTest;
import org.hyperskill.hstest.testcase.CheckResult;
import springdemo.SpringDemoApplication;

...

CheckResult testGet() {
    HttpResponse response = get("/api/user/").send();

    if (response.getStatusCode() != 200) {
        return CheckResult.wrong("GET /api/user/ should respond with " +
            "status code 200, responded: " + response.getStatusCode() + "\n\n" +
            "Response body:\n" + response.getContent());
    }

    JsonElement json;
    try {
        json = response.getJson();
    } catch (Exception ex) {
        return CheckResult.wrong("GET /api/user/ should return a valid JSON");
    }

    if (!json.isJsonArray()) {
        return CheckResult.wrong("GET /api/user/ should return an array of objects");
    }

    JsonArray array = json.getAsJsonArray();

    if (array.size() != 2) {
        return CheckResult.wrong("GET /api/user/ should initially return 2 objects");
    }

    return CheckResult.correct();
}

Note: for complex JSON responses you may want to use advanced JSON checking.

Notice, that you shouldn't add annotation @DynamicTestingMethod, you will add it later.

Notice the method get - you can use it to send a GET request to the tested application. You also can use post, put, and delete methods - they all declared in the SpringTest class. They are:

1. HttpRequest get(String address) constructs a GET request to the specified address. Address should not contain host or port - they are added automatically. The address should already contain GET parameters in case you want to add them to the request.
2. HttpRequest post(String address, String content) constructs a POST request with JSON content and also adds header Content: application/json
3. HttpRequest post(String address, Map<String, String> params) constructs a POST request with params content and also adds header Content: application/x-www-form-urlencoded
4. HttpRequest put(String address, String content) constructs a PUT request with JSON content and also adds header Content: application/json
5. HttpRequest put(String address, Map<String, String> params) constructs a PUT request with params content and also adds header Content: application/x-www-form-urlencoded
6. HttpRequest delete(String address) constructs a DELETE request to the specified address

You can see an example with the post method below (for example, an application should create a new user using POST /api/user/ and return a total number of users):

private String newUser = "{\n" +
    "    \"id\": 8,\n" +
    "    \"name\": \"NEW-USER\",\n" +
    "    \"email\": \"superuser@hyperskill.org\",\n" +
    "    \"skills\": [\n" +
    "        {\n" +
    "            \"name\": \"JAVA\"\n" +
    "        },\n" +
    "        {\n" +
    "            \"name\": \"KOTLIN\"\n" +
    "        }\n" +
    "    ]\n" +
    "}";

...

CheckResult testPost(int requiredUsers) {
    HttpResponse response = post("/api/user/", newUser).send();

    if (response.getStatusCode() != 200) {
        return CheckResult.wrong("POST /api/user/ should respond with " +
            "status code 200, responded: " + response.getStatusCode() + "\n\n" +
            "Response body:\n" + response.getContent());
    }

    int totalUsers;
    try {
        totalUsers = Integer.parseInt(response.getContent());
    } catch (NumberFormatException ex) {
        return CheckResult.wrong("POST /api/user/ " +
            "should create a user and return a number of users. No number was in the response.");
    }

    if (totalUsers != requiredUsers) {
        return CheckResult.wrong("POST /api/user/ " +
            "should create a user and return a number of users. " +
            "Expected to receive \"" + requiredUsers + "\", received: \"" + totalUsers + "\"");
    }

    return CheckResult.correct();
}

You can define a series of requests as your test. This way, every request would be treated as an individual test. Mark with @DynamicTestingMethod annotation a public List or public array of DynamicTesting objects. See the real example below form the WebQuizEngine project:

@DynamicTestingMethod
DynamicTesting[] dt = new DynamicTesting[] {
    () -> testAllQuizzes(0),

    () -> testCreateQuiz(1),
    () -> testQuizExists(1),
    () -> testQuizNotExists(1),

    () -> testAllQuizzes(2),
    this::reloadServer,
    () -> testAllQuizzes(2),
    () -> checkQuizSuccess(quizIds[0], "[2]", true),
    () -> checkQuizSuccess(quizIds[0], "[3]", false),
    () -> checkQuizSuccess(quizIds[1], "[0]", false),
    () -> checkQuizSuccess(quizIds[1], "[1]", true),

    () -> addIncorrectQuiz(error400noTitle),
    () -> addIncorrectQuiz(error400emptyTitle),
    () -> addIncorrectQuiz(error400noText),
    () -> addIncorrectQuiz(error400emptyText),
    () -> addIncorrectQuiz(error400noOptions),
    () -> addIncorrectQuiz(error400emptyOptions),
    () -> addIncorrectQuiz(error400oneOption),

    () -> testCreateQuiz(3),
    () -> testQuizExists(3),
    () -> testQuizNotExists(3),
    () -> checkQuizSuccess(quizIds[3], "[]", false),
    () -> checkQuizSuccess(quizIds[3], "[0]", false),
    () -> checkQuizSuccess(quizIds[3], "[1]", false),
    () -> checkQuizSuccess(quizIds[3], "[2]", false),
    () -> checkQuizSuccess(quizIds[3], "[3]", false),

    () -> testAllQuizzes(7),
    this::reloadServer,
    () -> testAllQuizzes(7),
    () -> checkQuizSuccess(quizIds[5], "[]", true),
    () -> checkQuizSuccess(quizIds[5], "[0]", false),
    () -> checkQuizSuccess(quizIds[6], "[0,1,2]", false),
    () -> checkQuizSuccess(quizIds[6], "[0,1,3]", true),
};

You can see the this::reloadServer line, it's used to check if the user used the database to store data and not in-memory objects (in case it is mentioned in the description that the database should be used, otherwise no need to reload the server). Typical relaodServer method is presented below (it didn't implemented in SpringTest class):

private CheckResult reloadServer() {
    try {
        reloadSpring();
    } catch (Exception ex) {
        throw new FatalError(ex.getMessage());
    }
    return CheckResult.correct();
}