Similarity checking tests for the new version of FirstInstance (with JavaSimilarityChecker) #14
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from SimilarityChecker and related classes
Implement requests and handlers for certain similarity checking operations used in Java similarity switches Helps centralise the said operations Helps encapsulates some parameters that would otherwise unnecessarily clutter similarity checker and similarity switches
Builds upon the ...similarity.base package, extends and complements its elements with EMFtext Provides exemplary similarity checking requests and handlers (using them is not mandatory)
For Java-related similarity switches Currently empty Can help separate Java-related similarity switches from the rest in the future
to keep the logging logic for Java-related inner switches in one place
For Java-related inner switches to not implement some mutual operations multiple times
Created for Java-related similarity checking
For PCM-related similarity switches
For PCM-related inner similarity switches
For PCM-related similarity checker Use the extracted structure
For ID-based similarity checking of PCMs A separate similarity checker implementation will no longer be necessary To be integrated in the following future commits
For Java-related similarity checking
For logging Java-related inner switches
To their own files without modifying them Non-functional commit
Rename SimilaritySwitch to JavaSimilaritySwitch to indicate its intent better Use the extracted inner switches in the JavaSimilaritySwitch Non-functional commit
To create new Java-related similarity switches Non-functional commit
For Java-related inner switches that create further switches Non-functional commit
Use extracted structure in the Java-related inner switches Use implemented interfaces in the Java-related inner switches Non-functional commit
Non-functional commit
For Java-related similarity checking Non-functional commit
Rename SimilarityChecker to JavaSimilarityChecker Use the extracted structure in the JavaSimilarityChecker Delegate similarity switch creation by using a new similarity switch request Non-functional commit
Adapt classes affected by the changes to the Java-related part of the similarity checking Non-functional commit
To their own files without modifying them Non-functional commit
Use the extracted structure in PCM-related similarity switch Use extracted inner switches in PCM-related similarity switch Non-similarity switch
For creating new PCM-related similarity switches Non-functional commit
For PCM-related inner switches that create further switches Non-functional commit
Change to string comparing makes it so that null strings and empty strings ("") are considered to be different.
Improve test displays Rename local variables
Refactor Improve test displays
that care about statement positioning Centralises the logic behind determining whether statement positioning matters for a given type
Some tests are disabled, due to similarity checking currently not validating EObjects. They should be enabled once similarity checking can detect and handle such faulty EObjects.
They dynamically explore and set each attribute A of each EObject EO of type EOT, which accepts a value of type EOT (or an array thereof), to that EObject (EO.A = EO). This makes sure that similarity checking does not run into an endless cycle and cause exceptions. Currently disabled since similarity checking does not validate the EObjects it is comparing. Enable the test class once similarity checking can handle this.
It ensures that changes regarding the scope of local variables are accounted for by the similarity checking. This test has shown that there are issues regarding the definition of similarity within CIPM. Currently disabled for the sake of not causing issues regarding Git or the building process. Note: This test reproduces an issue, which emerged while using CIPM on TeaStore (propagation from v1.0.0 to v1.1.0). The said issue is about the comparison of the container of LocalVariableStatement instances, which are not directly contained by their intended container (example: TryBlock (intended container) -> Block (allows TryBlock to contain statements) -> LocalVariableStatement).
Add other dependencies necessary for Mockito
Add methods for generating test displays based on initialisers
These tests make sure that each sub-type of Commentable used by JaMoPP has a corresponding case method. Currently, some Commentable sub-types (interfaces), without any direct implementing concrete classes, have no such corresponding case methods. hence the null checks in some test methods. Once similarity checking is adapted to cover said types, those null checks should be removed.
These tests introduce mock objects with invalid namespaces and assert that similarity checking can handle them, without causing exceptions.
These tests introduce mock objects, whose statement retrieval method breaks down at some point in time. They assert that similarity checking can handle such cases without throwing exceptions.
This test ensures that similarity checking can handle faulty, in this case asymmetric, equals overrides accordingly. Currently only with BinaryIntegerLiteral, since this version of Mockito does not allow mocking .equals()
Use methods from IIdentifierReferenceTest instead Use Mockito instead of dummy class
This reduces the likelihood of the tests yielding misleading results
AlpTorac
force-pushed
the
fitests-newsc-pull-req
branch
from
60201ef to
95d99b3
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Introduction
Using the initialisers introduced in #12, as well as the FirstInstance changes from #10; this pull request introduces a large amount of unit tests focusing on verifying the similarity checking process within the FirstInstance. The aim is to supplement the tests within "cipm.consistency.vsum.test" with relatively lightweight tests that can help pinpoint issues regarding the similarity checking.
The contribution of this pull request is a plug-in called "fitests", which contains the said tests.
Note 1: Some of the tests are (partially) disabled, in order to make merging possible. Once those cases are handled in FirstInstance, said tests should be re-enabled. Do not forget to look and account for the "FIXME" comments.
Note 2: #13 (see 30b9723) and #10 (see 14d284c) are merged into this pull request, along with further branches that are merged into them. As such, this pull request contains all changes from #9, #10, #12 and #13.
Note 3: This pull requests introduces Mockito dependencies to the target platform.
About the structure of fitests
The "fitests" plug-in is structured in a way that minimises dependencies to outside plug-ins. To this end, there are several abstract test classes that extend one another. This way, many different parts of this plug-in can be re-used in the future to implement further tests.
Currently, all test classes extend the abstract test class for JaMoPP, as it is currently the only technology under test within this plug-in.
About the unit tests
The fitests plug-in introduces many unit tests, in order to cover as much of the similarity checking process as possible. Inside the said tests, various JaMoPP objects are mostly constructed programmatically, which means that no actual Java project is parsed. The reason behind this is to keep project parsing of JaMoPP separated from the similarity checking introduced in CIPM. That way, problems with parsing do not affect the outcome of the tests.
Different kinds of unit tests are introduced below, in their corresponding section. Each test (sub-)package, as well as "cipm.consistency.fitests.similarity.jamopp.unittests" contains a "package-info.java" file, which has more information on the respective test group.
Interface tests
The "cipm.consistency.fitests.similarity.jamopp.unittests.interfacetests" folder contains parameterised unit tests for JaMoPP objects' mutual aspects (i.e. capabilities they get from mutually implemented interfaces). These tests are parameterised over initialisers (see #12 for more information).
Future note: If certain aspects in certain types of JaMoPP objects behave in such differing ways that they have to be tested separately, they should be separated from interface tests and be implemented as implementation tests.
Implementation tests
The "cipm.consistency.fitests.similarity.jamopp.unittests.impltests" folder contains unit tests for JaMoPP objects' unique aspects, such as their unique attributes, which only singular JaMoPP objects have. Implementation tests are currently not parameterised over initialisers, as they only consider one type of JaMoPP object each.
Complex tests
While interface tests and implementation tests only consider certain aspects of certain JaMoPP objects in an isolated fashion, the tests under "cipm.consistency.fitests.similarity.jamopp.unittests.complextests" construct more advanced JaMoPP objects. This includes JaMoPP objects, which contain other JaMoPP objects that have to be set-up in certain ways.
Complex tests are separated from the rest of the tests, because the construction of individual JaMoPP objects becomes important and using initialiser adaptation strategies makes understanding the construction process harder.
Mock tests
While the other tests cover realistic cases, i.e. similarity checking scenarios that are producible without modifying the implementation of Java-model elements, "cipm.consistency.fitests.similarity.jamopp.unittests.mocktests" introduces tests, which use Mockito with the intention of pushing the robustness of similarity checking to its limits.
Their main purpose is to cover and test (currently) unreachable cases / safety mechanisms within similarity checking. In doing so, they also ensure the extendibility of similarity checking with newer Java model elements.
Other tests
The "fitests" plug-in additionally introduces other unit tests:
GeneralJaMoPPSimilarityTest: Tests the (general) control flow of the similarity checking process, while abstracting away from the concrete JaMoPP object type as much as possible.
SPLevoModelsSimilarityTest: Tests the similarity checking process by parsing some minimal Java projects under "splevo-testmodels". This is the only test class introduced here, where JaMoPP is used to parse Java projects.
UtilityTests: Serves as a "meta-test" for the tests within "fitests". Ensures that initialisers of all JaMoPP objects are accessible from tests and that each JaMoPP object has a corresponding test file.