Package org.eclipse.epsilon.eol.engine.test.acceptance.util

Class EolEquivalenceTests<C extends EolRunConfiguration>

  • Direct Known Subclasses:
    EclModuleEquivalenceTests, EgxModuleEquivalenceTests, EolModuleEquivalenceTests, EplModuleEquivalenceTests, EvlModuleEquivalenceTests
    public abstract class EolEquivalenceTests<C extends EolRunConfiguration>
extends Object
    A series of tests which use the standard EolModule as an oracle and test the concurrent implementations in various configurations (different number of threads) against it to ensure identical results and behavioural equivalence. The tests are carried out in the context of scenarios. A scenario is a given combination of script (EOL file) and model to execute the script on. Since each scenario is independent, it requires its own IEolModule. For this reason, each EolRunConfiguration has an identifier so that each scenario can be uniquely identified and different modules under the same scenario can then be compared.

    Regarding test ordering, only the testModuleCanExecute() method is required to be run before the others (for obvious reasons). Note that since the expected configurations are our oracles, they are assumed to pass and are exempt from testing; hence being executed in setUpBeforeClass().

    This test class is intended to be extended by tests for extensions of EOL. For a reference implementation/example, please see EvlModuleEquivalenceTests. A basic implementation would need to provide the following:
    • A constructor which calls super(C configUnderTest)
    • A setUpBeforeClass static method (annotated with @BeforeClass) which assigns expectedConfigs and subsequently calls setUpEquivalenceTest()
    • A static method returning an Iterable annotated with @Parameters
    • An implementation of _0test0() which simply calls the #beforeTests() method
    • The class should be annotated with @FixMethodOrder(MethodSorters.NAME_ASCENDING)
    The last two requirements are a workaround for JUnit's inadequate @Before semantics.
    Author:
    Sina Madani
    See Also:
    EolAcceptanceTestUtil

    • Field Detail

      • expectedConfigIDs

        protected static Map<Integer,​EolRunConfiguration> expectedConfigIDs
        Used to identify which scenario to compare our results with.

      • expectedConfig

        protected final C extends EolRunConfiguration expectedConfig
        The scenario and module combination under test. This is the parameterised test variable.

      • testConfig

        protected final C extends EolRunConfiguration testConfig
        The scenario and module combination under test. This is the parameterised test variable.

    • Constructor Detail

      • EolEquivalenceTests

        public EolEquivalenceTests​(C configUnderTest)

    • Method Detail

      • setUpEquivalenceTest

        protected static void setUpEquivalenceTest​(Collection<? extends EolRunConfiguration> expectedConfs)
        This should be called in setUpBeforeClass().
        Parameters:
        expectedConfs - The oracle configurations.

      • _0test0

        public abstract void _0test0()
                      throws Exception
        Subclasses should simply call beforeTests() in this method. Additional setup functionality may also be provided here.
        Throws:
        Exception

      • beforeTests

        protected final void beforeTests()
                          throws Exception
        Pre-requisite for testing.
        Throws:
        Exception

      • onFail

        protected boolean onFail​(String message)
        Actions to perform if the message is null. Generally useful for printing more detailed diagnostics than is provided by the testing framework.
        Parameters:
        message - The message to fail with. If null, no action is taken.
        Returns:
        whether the message is null, for convenience.

      • testModuleCanExecute

        protected void testModuleCanExecute()
                             throws Exception
        Throws:
        Exception

      • testScenariosMatch

        protected void testScenariosMatch()
                           throws Exception
        Throws:
        Exception

      • testExecutorFactories

        public void testExecutorFactories()
                           throws Exception
        Throws:
        Exception

      • testOperationContributorRegistries

        public void testOperationContributorRegistries()
                                        throws Exception
        Throws:
        Exception