Uploaded image for project: 'ReactOS Test Suite'
  1. ReactOS Test Suite
  2. ROSTESTS-127

[WIP] Rosautotest 2.0: New enhancements

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Incomplete
    • Major
    • None
    • ReactOS Tests
    • None

    Description

      Current Rosautotest is aimed towards automation tests.
      But when using manually I'm lacking several "setting features".

      Current Rosautotest (aside of some flags) can just be called in thee following ways:

      Rosautotest [module] [test]: Ie "Rosautotest user32 msg"
      – This just run that test in such specific module

      Rosautotest [module]: Ie "Rosautotest user32"
      – This just run all the tests in user32

      Rosautotest: ie "Rosautotest"
      – This runs the whole set of modules and tests

      I have added several new features for Rosautotest without breaking current behavior.
      Now, with this Rosautotest you can perform the following actions:

      :: Banning Module tests ::

      Rosautotest – [module_name]: "Rosautotest – user32_winetests"
      Result: All the tests, included user32_apitests, will be run but user32_winetests.

      Rosautotest – [module]: "Rosautotest – user32"
      Result: This command runs the whole test suite but the user32_winetests and user32_apitests modules.

      Rosautotest – [string]: "Rosautotest – u "
      Result: All the tests which begins with "u" are banned. The rest are tested.

      Advance usage #2: You can ban a set inside a range of modules:
      Result: Just the User32 tests are run but user32_winetests.

      :: Starting from a given position ::

      Rosautotest -+ [module] [test]: "Rosautotest -+ user32 msg"
      Result: It runs all the tests since user32 msg, but jumps all the previous ones.

      :: Advanced usage::

      Rosautotest – [module_name/module/string] [module]: "Rosautotest – user32_winetest user32"
      Result: It runs JUST the user32 tests but user32_winetest

      Rosautotest -+ [module] [test][module_name/module/string] : "Rosautotest -+ advpack install – browseui"
      Result: It runs all the tests beginning with the Advpack install and jumping all the browseui tests.

      :: Bugs and cleanup ::

      BUG: Rosautotest -+ [module] : "Rosautotest -+ user32"
      Result: It runs all the tests beginning with the first User32 test

      :: Future enhancements ::

      1) Define the – tests as: – user32:msg,advapi:install to include a list of banned tests.
      2) Define the – tests as: – blacklist.txt , to read the list from a text file.
      3) Define a new flag: ++ which will be a whitelist. Just the tests included in such list will be tested.

      :: Suggestions and comments ::

      :: Reason behind all this rework ::

      Right now I am moving towards FranRostein testings (ReactOS + a Windows DLL), so I am forced to run the tests manually. Some tests go into endless loops or crashes due to the changes, so being able to "Start" the Tests in a defined point (instead testing the undesired ones) and playing with the "Banning", I am able to create a command which tests the whole suite without a reboot.

      Hope it is also useful for others too.

      Soon I will upload the .patch for it, right now it is on cleaning state, and began to fix the known bugs and improve with the enhancements.

      Attached the rosautotest.exe for playing purposes.

      Attachments

        Activity

          People

            bug zilla Bug Zilla
            vicmarcal vicmarcal
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: