Performance Center : Architecture overview

Loadrunner has the following high level conceptual components:

1. Injector(s) – Used to replicate virtual users
2. Controller – Used to coordinate the actions of the simulated users
3. Realtime monitors – gives information on hardware and software metric’s during simulation
4. Analysis Database – stores information for post analysis

Any commercial load-testing tool should consist of these.

Communication between LR components

Performance Center Components

• Administration Site: The Administration Site is used for administering the Performance Center system. From the Administration Site, you can perform tasks such as assigning user roles and privileges, creating and administering projects, and managing resource usage
• User Site: The User Site is used for creating, managing, and running load tests. From the User Site, you can perform tasks such as designing and configuring load tests, configuring monitors, reserving resources, running and monitoring test runs, and analyzing test results
• Utility Server: The Utility Server is the Performance Center License Manager and is also used for configuring monitor profiles and other maintenance services
• Database and File Servers: The Database and File Servers form the infrastructure for persistent data within Performance Center. The Database Server stores information on users, projects, host information, and load tests. The file server stores the test scripts and test results
• Performance Center Host: Performance Center hosts are used to control load tests, generate load, and analyze data. Performance Center hosts can be configured as Controllers, Load Generators, or Data Processors
• Controller: The Controller is the manager of a load test. The Controller receives the scripts, their run-time settings, and a list of the load generators to use. The Controller issues instructions to the load generators including which scripts to run, how many Vusers to run per script, and scheduler settings. At the conclusion of the test run, the Controller collates the data. There is only one Controller per load test
• Load Generator: Load generators generate load by running Vusers. The Controller dictates the manner in which they start and stop running. There can be any number of load generators for a given load test.
• Data Processor: The Data Processor host is used for publishing information to the Performance Center Dashboard.
• MI Listener: The MI Listener is one of the components needed to run Vusers and monitor applications over a firewall.
• Monitors Over Firewall: Used to monitor servers that are located over a firewall.

The following standalone applications, which are included on the Performance Center installation disk, integrate with your Performance Center System:

• HP Virtual User Generator: Virtual User Generator (VuGen) generates virtual users, or Vusers, by recording actions that typical end-users would perform on your application. VuGen records your actions into automated Vuser scripts which form the foundation of your load tests
• HP Analysis: HP Analysis provides graphs and reports with in-depth performance analysis information. Using these graphs and reports, you can pinpoint and identify the bottlenecks in your application and determine what changes need to be made to your system in order to improve its performance.
• SiteScope:  SiteScope is a data collector used for collecting performance data from network, application, database, and Web servers.
• ERP and CRM Mediator: The ERP and CRM Mediator gathers and correlates offline transaction data for the ERP/CRM diagnostics modules.

Using Loadrunner Pacing to Hit a Specific Transaction Rate

It is often necessary to apply load at a specific transaction rate per second (TPS).  For example, it may be necessary to test every build or patch of an application by running a fixed benchmark of 20 queries per second against the application, measuring the response time and server utilization at that fixed load.  By then examining the trend over time of response time and server resource utilization under uniform load, performance regressions can be easily identified.

One way to set a specific transaction rate using Loadrunner is by using the pacing feature.

Pacing specified how often an iteration starts.  For example, if an iteration runs in 300 - 500 milliseconds, setting pacing to 1 second for that script will case a user to run the iteration once every second as illustrated below:

Each second the iteration starts and then ends after 400 ms or so.  At the next second interval, the next iteration starts.  An exact transaction rate of 1 iteration per second is reached in this way.  By increasing the user count in this case to 10, a transaction rate of 10 TPS can be achieved.  

An iteration can have multiple transactions per iteration.  If the above example was an iteration that included three transactions (query1, query2, and query3), a single user would run three transactions per second and 10 users would give 30 TPS.

The following formula provides a calculator for determining what the pacing in seconds should be set to given three input parameters, transactions per iteration, user count, and target transaction rate:

• transactionsPerIteration  (the number of transactions included in an iteration)
• users (the number of users to be run)
• tps (the target transaction rate per second)

For example, suppose transactionsPerIteration = 3, users = 50, and target transaction rate = 10 TPS.  The equation gives a pacing in seconds of (3 * 50)/10 = 1.5 seconds.

One thing that can prevent the target transaction rate from being reached is when the iteration slows down when run under multiple users such that the time an iteration runs is longer than the pacing time.  In this case a warning is generated and the transaction rate is not reached.  This can be sometimes be avoided by increasing the user count somewhat.

However, the problem could also be due to limitations in the application's vertical scalability.  The application may have concurrency problems preventing concurrent users from executing the transaction without blocking other users.  In a worst case, the application's concurrency problems may make it impossible to reach the target transaction rate regardless of the number of users.  In any case, it is worthwhile to investigate what the bottlenecks are that are limiting scalability and resolving those issues.

How Many Load Generators Do We Need For Performance Testing

As a general rule, the more complex a protocol is to script, the more the protocol relies on monitoring the network stream and the less that it needs to understand the actual application under test’s user interface. It also tends to be true that with this type of protocol, the more virtual users of that protocol type will run on any specific load generator. 

This has led to the development of the Protocol Complexity Matrix and can be visualized by the graphic below.

This is a representation of common results and you may find that your individual use of a protocol may provide more or less virtual users than indicated.  This diagram shows you that if you want to make the job of scripting easier, it comes at the cost of memory.  Using more memory makes the protocol less scalable up to the point that you reach the GUI protocol, which out of the box has a 1:1 ratio between virtual user and load generator.

The goal of any organization is to find the right balance.  Nearly any application can be scripted with the Winsock protocol, regardless of technology used by the application under test. However, working with raw socket commutations can be difficult to learn and requires a great deal more time to develop and test; thus making the solution impractical for many projects. This is why higher-level protocols have been developed, to save time.

What is not easily represented in the matrix is another fact; the more reliant you are to the network stream, the more susceptible your scripts are to changes in that stream.  So the application that is scripted with QTP can undergo massive changes in the background and the scripts for that application can still function without modification.  But these same changes to an application recorded with the Winsock protocol could render the entire script useless and a throw away. This fact brings me to my next question:

How many users can my Load Generator run?

In previous versions of LoadRunner and Performance Center, a protocol footprint spreadsheet was distributed by HP to provide a guideline for how much memory a script of a particular protocol would consume.  The problem was recipients of the footprint spreadsheet thought of it as definitive rather than prescriptive. It is much better to use a repeatable process and a formula to mathematically estimate the total number of users a load generator can run on a per-script basis. The process and formula outlined below should be used for every script that you intend to use in load testing, and every load generator that will participate in the test as well.

Here are the steps you should take these scripts that you intend to use in a load test:

1)      Develop your script and ensure that it will run with multiple users, multiple iterations.  This is the normal development of the script of any protocol.

2)      Create a load test using the script that you want to examine with one virtual user and execute the test. Use a delay of a few minutes in starting the script and monitor the load generators available MB of RAM and note any decrease when the virtual user starts.  This is the “First VUser Memory”.

3)      Modify the load test to use five – ten virtual users and execute the test again.  Keep the same delay for the start and have each user start one minute apart.  Monitor the load generators available MB of RAM and note the drop in RAM as each user starts.  Average the amount of RAM that the second virtual user on up uses and this is the “Each Additional VUser Memory”

4)      Determine the total RAM available on the load generator This is the “TOTAL LG RAM”

5)      Subtract 700 – 750 MB of RAM for the OS

6)      Determine what is 75 percent of the remaining available RAM

7)      Subtract the “First Vuser Memory”

8)      Divide what is left by the “Each Additional Vuser Memory”. 

9)      Add one and you have the “Total Virtual Users that will run on that load generator”

10)  Repeat this process and formula for each script and every load generator that has a different level of memory.

The equation will look something like this:

This equation will work for all but a handful of protocols.  These are the protocols that this equation may not work: Citrix, RDP, any protocol with the term GUI in it, and QTP\UFT scripts.  That is because these scripts can further be limited due to GDI resources.  These are graphical resources that are an attribute of the video card installed on the computer.

GDI resources are not able to be monitored and they cannot be adjusted without replacing the video card.  When GDI resources are consumed any virtual user that attempts to run on that LG will fail.

To learn more about LoadRunner and Performance Center and how to estimate the total number of users a load generator can run on a per-script basis visit the product homepages here.

As an example.  If you have a 4GB LG machine the TOTAL LG RAM is 4,000

If your First Virtual User Memory is 10MB and Each Additional Virutual User Memory is 3MB then the result would be:

4,000 - 750 = 3,250

3,250 * .75 = 2,437

2,437 - 10 = 2,427 ( 1 virtual user running)

2,427 / 3= 809

809 + 1 = 810 (Total Virtual Users)

Hope this makes the process a little clearer.

Checking text in a PDF file with LoadRunner

My solution uses the popen() function to call a command-line program from LoadRunner which converts the PDF to text. The text can then be verified using simple string comparison functions. The command line program I used is pdftotext, which is part of the open-source Xpdf project. I have mirrored the file, so you can download pdftotext (from Xpdf v3.03) directly from my website.

Here is the example code:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71

char *strstr( const char *string1, const char *string2); // Explicit declaration required for C functions that do not return integers. #define BUFFER_SIZE 16384 // 16 KB. If this gets too big, use malloc() instead to avoid "too many local variables" compilation error.   Action() { int pdf_size; // size of the downloaded PDF (determined at runtime) int fp_save; // file pointer for saving the PDF int fp_popen; // file/stream pointer for the output from popen() int count; // number of characters that have been read from the popen() stream char buffer[BUFFER_SIZE]; // allocate memory for the output of popen()   lr_start_transaction("download_pdf");   // Save entire HTTP response body // Note that the PDF download must be in a step by itself in order to save the contents. It cannot be an EXTRARES. web_reg_save_param_ex( "ParamName=PdfContents", "LB/BIN=", "RB/BIN=", SEARCH_FILTERS, "Scope=Body", LAST);   web_reg_find("Text=PDF", LAST); // Basic check web_url("Download PDF", "URL=http://www.myloadtest.com/resources/hp-discover-2012-tb2337-moncrieff.pdf", "Resource=0", // Not a resource "RecContentType=application/pdf", LAST);   pdf_size = web_get_int_property(HTTP_INFO_DOWNLOAD_SIZE); if (pdf_size < 100000){ lr_error_message("Downloaded PDF is smaller than expected."); // By itself, this is not a very good check. lr_exit(LR_EXIT_ACTION_AND_CONTINUE, LR_FAIL); }   lr_end_transaction("download_pdf", LR_AUTO); lr_think_time(10); lr_start_transaction("check_pdf");   // Save file. lr_save_string(lr_eval_string("C:\\LoadRunner\\Data\\{TimeStamp}{VuserId}.pdf"), "FileName"); // Ensure that the file name is unique to avoid overwriting the contents fp_save = fopen(lr_eval_string("{FileName}"), "wb"); // open file in "write, binary" mode. fwrite(lr_eval_string("{PdfContents}"), pdf_size, 1, fp_save); fclose(fp_save);   // Run "C:\LoadRunner\Lib\pdftotext {FileName} -" and save the output to a parameter. fp_popen = popen(lr_eval_string("C:\\LoadRunner\\Lib\\pdftotext {FileName} -"), "r"); count = fread(buffer, sizeof(char), BUFFER_SIZE, fp_popen); if (feof(fp_popen) == 0) { lr_error_message("Did not reach the end of the input stream when reading. Try increasing BUFFER_SIZE."); return -1; } buffer[count] = NULL; // Make sure the string in the buffer is null-terminated. lr_save_string(buffer, "PdfText"); lr_output_message("The Windows version is: %s", buffer); pclose(fp_popen);   // Check that the PDF parameter contains the expected text ("LoadRunner"). if (strstr(lr_eval_string("{PdfText}"), "LoadRunner") == NULL) { lr_error_message("PDF file does not contain the expected string."); lr_exit(LR_EXIT_ACTION_AND_CONTINUE, LR_FAIL); }   // Cleanup. Delete the file. system(lr_eval_string("del /q {FileName}"));   lr_end_transaction("check_pdf", LR_AUTO);   return 0; }