Load Test Automation

Getting Started with Automation

User Rating: / 13  |  Rate this article: PoorBest 

{ArticleIndex}

The first step to automating a load test, is creating the actual test. For details on doing so, please see the Getting Started and Working with LoadUI sections of this web site.

Once the test has been created and saved, the next step is to successfully run it using the command line. To do so, open a command prompt and navigate to the LoadUI installation directory. It should contain the loadUI-cmd.bat (loadUI-cmd.sh for *nix, loadUI-cmd.command for mac) executable.

Running from the Command Line

To invoke the command line runner with your project, use the -p flag to specify the location of your saved project file. This will load and start your saved project. If it has any limits set, then the test should stop once one of these limits is reached. If not, it will continue to run until manually stopped. In either case, the test execution can be stopped by pressing CTRL + C. For example:

loadui-cmd.bat -p c:\Documents\my-loadui-project.xml

This starts the project itself. If we have any VU Scenarios inside that should be run, then they must either be linked to the Project execution, or started by a scheduler.

Specifying Limits and Scenarios

Instead of starting the project, we may with to target a specific Scenario in the project for execution. If so, this can be achieved by using the -t flag. We can also customize the limits for the test directly from the command line, using the -L flag. For example:

loadui-cmd.bat -p c:\Documents\my-loadui-project.xml -t MyScenario -L 3600:0:0

The above command would again run the same project as before, but instead of starting the project execution, it would start running the Scenario named MyScenario. It would also run the test for one hour (3600 seconds).

Specifying Agents

VU Scenarios generally run in a distributed mode, on remote LoadUI Agents. Which Agents to use and which one does what is specified in the project, but this can also be specified using the command line. The -l flag tells LoadUI to run in local mode, where all Scenarios are run only by the local machine. Instead of running locally, we can use the -a flag to specify Agents to run on. The following example shows how to run the previous Scenario on two Agents specified by IP address:

loadui-cmd.bat -p c:\Documents\my-loadui-project.xml -t MyTScenario -a 10.0.0.5 10.0.0.6

Creating Reports

Finally lets add the creation of reports to our execution; use the -r parameter to specify where to create your report and the -F parameter to specify the desired format (see the Reference for supported formats). For example

loadui-cmd.bat -p c:\Documents\my-loadui-project.xml -r c:\dev\reports -F PDF

will run our project and generate the summary in both PDF and XML format (the XML version is always created).

Parameterizing tests

You might want to run the same test multiple times, but change a few parameters for each run. This can be done by passing command line options to LoadUI. As an example, let's have a look at how we could run the same LoadUI project multiple times with different endpoints/URL's.

  • Start LoadUI with the command line option "-DmyCustomEndpoint=http://www.myWebSite.com". For example:
    loadUI.bat "-DmyCustomEndpoint=http://www.myWebSite.com".
  • Use the property expansion ${=System.getProperty("myCustomEndpoint")} wherever you want to use your custom endpoint/URL in soapUI, as in the image below:
    enpointProperyExpansion

Now you can start LoadUI with a custom endpoint specified from a Bat-script, Maven, Hudson or whatever, like this:

REM testAllEndpoints.bat

call loadUI-cmd.bat "-DmyCustomEndpoint=http://myWebSite.com" -p my-loadui-project.xml
call loadUI-cmd.bat "-DmyCustomEndpoint=http://myWebSite.com/admin.php" -p my-loadui-project.xml
call loadUI-cmd.bat "-DmyCustomEndpoint=http://foo.bar"-p project1.xml -r c:\reports -F PDF
call loadUI-cmd.bat "-DmyCustomEndpoint=http://twitter.com/eviware" -p testSLA1.xml

REM Add more endpoints/configurations as needed...

Note that myCustomEndpoint is an arbitrary named variable that we happen to set to an endpoint/URL. You can actually call variables anything and set them to anything, e.g. -DnumberOfCustomers=4711.

 

.

Quick tip:

You can also set parameters by editing the file called system.properties.

You might also be interested in Parameterizing Scenarios and Data Driven Testing.

 

What is test automation?

User Rating: / 11  |  Rate this article: PoorBest 

CommandLineLogo

What is Load Test Automation?

Load test automation is the act of automating the execution of a load test.

Why would I want to automate load testing?

Running load tests should be done early and often during development. However, it can be a time consuming task, and thus having it automated can save a lot of time.

How does LoadUI support automating load tests?

LoadUI allows you to take your saved load tests, and run them from the command line. Using the LoadUI command line runner, you can run LoadUI tests from just about any task scheduler, or as an integrated part of your build process. You can even customize the test execution by using command line switches to override the test limits, which Agents to distribute the test to, and several other options.

 

Automation Reference

User Rating: / 6  |  Rate this article: PoorBest 

Command line runner usage

There are several command line switches you can use to control the behavior of the command. They are as follows:

-a, --agent
Setting this switch, once or more, allows you to specify which agents to run the test on. Multiple agents may be specified, and each agent may specify which Scenarios it should be assigned to. The syntax for doing this is:

-a <ip>[:<port>][=<scenario>[,<scenario>] ...]

With default values:
protocol
https://
port
8443
For instance:

-a 192.168.0.5 192.168.0.1=MyScenario,MyOtherScenario

would run the test with two agents, all Scenarios assigned to the 192.168.0.5 agent, and Scenarios MyScenario and MyOtherScenario would run on 192.168.100.1. When specifying agents on the command line, any agents assigned to the Scenarios in the Project file will be disregarded.
-A, --abort
Overrides "Abort ongoing requests on finish" project property. If set to true ongoing requests will be canceled, if false test will finish when all ongoing requests complete. If not set, property value from project will be used to determine what to do with ongoing requests.
-c, --compare <arg>
Specify a saved execution to use as a base for comparison in the generated statistics report.
-D <arg>
Sets system property with name=value.
-F, --format
Sets the output format for the generated reports. Supported formats are: PDF, XLS, HTML, RTF, CSV, TXT and XML. This parameter is not case sensitive. In case -r (--reports) parameter is not specified, this parameter is ignored.

-h, --help
Displays a short summary of the available command line switches.
-l, --local
Sets the run mode to local mode, meaning no agents will be used, and Scenarios will be executed by the local computer. The command line runner always defaults to distributed mode.

-nofx
Do not include or require the JavaFX runtime.

-L, --limits
Allows you to override the saved limits for the test with your own. The syntax is:

-L ::

For instance:

-L 60:0:5

would run the test for 60 seconds, or until 5 failures have occurred. Setting a limit to 0 means no limit.
-nolock
Disable the lock file that prevents several copies of LoadUI to run at the same time.

-p, --project
Specifies the path to the Project file to run.

-r, --reports
Sets the directory where the generated Summary Report will be stored. If specified directory does not exist it will be created automatically. If specified path represents the path to an existing file than reports will be saved in the parent folder of that file. If not specified, reports won't be generated at all.

-s, --summary
Set to include summary report in statistics report.
-S, --statistics
Sets which Statistics pages to add to the generated report (leave blank save all pages).
-t, --testcase
Specifies which TestCase in the project to execute. If omitted, the entire Project is executed. Deprecated since LoadUI 2.0.
-v
Specifies which Virtual User Scenario in the project to execute. If omitted, the entire Project is executed.
-w, --workspace
Specifies which Workspace file to use when executing. The Workspace holds global settings. If omitted, the default Workspace is used.
-z
Use the saved zoom levels for charts from the project.

Don't forget! If you have a space in your path or a Scenario name, you will have to wrap the argument in quotation (") marks.