How to do correlation when the boundaries are dynamically changing?

Loadrunner

Consider a scenario when the Left and Right Boundaries are dynamically changing, using which we will have to capture the correlation value.

1. If the LB and RB has dynamically changing numbers:

Then your LB and RB becomes LB/DIG & RB/DIG and replace the dynamically changing numbers in the boundary with #.

Example:

Iteration 1: City001"Chennai"001City
Iteration 2: City002"Bangalore"002City
Iteration 3: City003"Delhi"003City

Here we can see that LB and RB are getting incremented from 001 to 003. So, our boundary becomes

LB/DIG = City00#\"
RB/DIG = \"00#City

2. If the LB and RB has dynamically changing alphanumeric values:

Then your LB and RB becomes LB/ALNUM & RB/ALNUM and replace the dynamically changing alphanumeric values in the boundary with ^.

Example:

Iteration 1: Citya01"Chennai"a01City
Iteration 2: Cityb02"Bangalore"b02City
Iteration 3: Cityc03"Delhi"c03City

Here we can see that LB and RB are getting incremented from a01 to c03. So, our boundary becomes

LB/ALNUM = City^^^\"
RB/ALNUM = \"^^^City

SaveOffset attribute in web_reg_save_param

Loadrunner

SaveOffstet:

SaveOffset attribute defines how many places after the Left Boundary, dynamic value should be captured.

Example:

radiobutton1="Chennai"

Dynamic value to be captured is Chennai

web_reg_save_param("City",
"LB=radiobutton",
"RB=\"",
"SaveOffset=3",
LAST);

Result:

City = Chennai

Here, the LB is radiobutton and we have mentioned the SaveOffset as 3 . So it will capture Chennai leaving 1=" (3 places after the LB) and save it in the Correlation Parameter City.

Standard deviation and its importance in performance testing

What is Standard Deviation?

Mathematical Definition : It is the measure of how spread out the numbers are around the mean.
Performance Definition : It is the measure of how response time is spread out around the mean.

It is denoted by Sigma (σ).


Formula: 



Where

N = Count of the response time
x = Individual response time value
μ = Mean or Average response time

Importance of Standard Deviation in Performance Testing

• Standard Deviation tells whether a transaction is consistent throughout the test or not.
• A small standard deviation means that the response time of all the iterations of the same transaction are close to each other. 
• So smaller the value more close the transaction response times are and more consistent the transaction is.

Example:

Lets calculate the Std. Deviation for the above transactions.

Login Transaction:

Step 1 - Average Response Time or Mean value =  5

Step 2 - 

(5-5)²  = 0

(5-5)²  = 0

(6-5)²  = 1

(5-5)²  = 0

(4-5)²  = 1

Step 3 - (0+0+1+0+1)/5 = 0.4

Step 4  - √ 0.4 = 0.632

Inference :

In the Login transaction we can observe that response time of each iteration are close enough so the Std. Deviation is small. It means Login transaction is more consistent throughout the test.

Logout Transaction:

Step 1 - Average Response Time or Mean value =  5

Step 2 - 

(5-5)²  = 0

(4-5)²  = 1

(12-5)²  = 49

(3-5)²  = 4

(1-5)²  = 16

Step 3 - (0+1+49+4+16)/5 = 14

Step 4  - √ 14 = 3.741

Inference :

In the Logout transaction we can observe that response time of each iteration are having large deviations so the Std. Deviation is high. It means Logout transaction is less consistent throughout the test.

Cache and its impact on performance testing

What is Cache?

Cache is a temporary storage area where an applications non html resources are downloaded and stored when the user access the application for the first time. When the user again access the application, the request will not be sent to the server instead all the resources will be downloaded from the cache memory, making the user feel a faster page loading experience.

How does Cache impact performance testing?

Caching has a negative impact on the performance testing. When caching is enabled during the performance test the request will not be sent to the server and the resources will be downloaded from the cache memory thus showing an improved response time, providing improper test results. So cache has to be disabled during the performance tests.

Different Cache settings in loadrunner: (Run-time Settings --> Browser Emulation--> Browser Cache)

Simulate browser cache  - This option instructs the loadrunner to download the resources from the cache and it will not ask the server for the resources.

Simulate a new user on each iteration - This option instructs loadrunner to simulate a new user on each iteration and so a new cache. When this option is not selected same cache will be used for all the iterations.

With Simulate browser cache and Simulate a new user on each iteration - Loadrunner will use the same resources in the cache for each iteration but will load them only once per iteration.

With Simulate browser cache and without Simulate a new user on each iteration - Loadrunner will download the resources and save them in the cache during the first iteration and the same cache will be used from the second iteration.

web_cache_cleanup

• web_cache_cleanup loadrunner function can be used in the script to clear the cache which is similar to the Simulate a new user on each iteration.
• When web_cache_cleanup is used in the script, it will override all the cache Run-time settings.

Significance of 90 Percentile response time in performance testing

Loadrunner

We all would have seen the 90 percentile response time in our analysis summary report and this percentile value can be customized according to our NFR's. In this post we will take close look at what this 90 percentile response time is and its significance in performance testing.

What is 90 Percentile response time?

It is the value for which 90% of data points are smaller (or) The permissible response time within which 90% percentage of the users fall.

Here data points refer to the transaction response time.

Lets take an example to understand this.

Consider 10 instance of Login transaction with values 2,1,5,8,3,7,10,9,4,6 in seconds

1. Sort the Response Time values in descending order 1,2,3,4,5,6,7,8,9,10 seconds respectively.
2. Remove the top 10% of the Response Time values. Here the top 10% is 10 seconds. So remove it.
3. Now the higher value left if the 90 percentile Response time. So 9 seconds is our 90 percentile Response time for Login transaction.

Significance of 90 Percentile response time:

Consider you have the NFR's for 90 percentile response time as 9 seconds and from our above example 90 percentile response time is also 9 seconds, which means our NFR is met.

• It means that 90% of the transactions or users are within the permissible limit of the 90 percentile NFR value.
• It is more accurate and dependable than the Average response time or some times called as 50 percentile value because average response time means that only 50% of the transactions or users are within the permissible limit of the Average response time NFR. 
• The average response time will always be less than the 90 percentile response time. Using the below formula find out the 90 percentile to Average response time ratio.

(90 Percentile RT - Average RT) /  Average RT

• If the ratio is small it means that 90 Percentile RT & Average RT are close enough and the application is having good and stabilized performance.
• If the ratio is high it means that applications performance is poor.

Reducing the vugen scripts folder size by deleting the unwanted replay files

Loadrunner

If we are using Performance center for performance test execution, then we will be creating a test scenario using zipped vugen scripts which will hold only the runtime files. But this is not the case when we create a scenario using standalone controller. We will have to delete the files that are created during the script replay to reduce the folder size.

Every scenario will have number of scripts and deleting the replay files manually from each scripts folder is time consuming. Hence this has been automated using command prompt (.cmd) file.

Using Command prompt:

Place the Delete.cmd file in the folder where all the vugen scripts are saved and execute it. This will delete the replay files from all the vugen script folders in one go.

Below are the list of files that will be deleted from the vugen scripts folders.

*.idx
mdrv*.log
mdrv.txt
options.txt
*.ci
combined_*.c
output.txt
debug.inf
*.bak
\result1

Note : Make sure none of the vugen's scripts whose replay files are being deleted is open while running the Delete.cmd file.

Download the Delete.cmd file here

How to validate file download scenarios in loadrunner?

Loadrunner

Scenario:

Lets say we have a script in which we need to do file download operation and we need to find out time taken for a file download and based on the file download operation we need to validate the transaction.

Solution:

web_get_int_property - This function provides specific information about the previous HTTP Request.

1. This can be done using web_get_int_property loadrunner function.
2. Place the web_get_int_property  function below the file download web_request.
3. Time taken to download the file can be obtained using HTTP_INFO_DOWNLOAD_TIME attribute of web_get_int_property.
4. Size of the downloaded file can be obtained using HTTP_INFO_DOWNLOAD_SIZE attribute of web_get_int_property.

Sample code below:

long fsize;
int ftime;

lr_start_transaction("File_Download");

web_url("File download HTTP Request");

ftime = web_get_int_property (HTTP_INFO_DOWNLOAD_TIME);

lr_output_message("File download time : %d", ftime);

fsize = web_get_int_property (HTTP_INFO_DOWNLOAD_SIZE);

if (fsize == 0)
{
lr_end_transaction("File_Download", LR_FAIL);
}
else
{
lr_end_transaction("File_Download", LR_PASS);
}

Automating the creation of input files for load test using VB Script

Loadrunner

We will be needing to create thousands of dummy input text files for our load test. Creating them manually is a long and time consuming process. This can be automated using the below VB script.

Here 

1. We are appending the date and time to the file name to produce unique text files.

2. We have used FOR LOOP to iterate the file creation to the desired count.

3. Also we are writing content to the files to avoid creating empty files, which we can customize with whatever we want to write.

VB Script:

Dim strSafeDate, strSafeTime, strDateTime

Dim i

For i = 0 to 3

strSafeDate = DatePart("yyyy",Date) & Right("0" & DatePart("m",Date), 2) & Right("0" & DatePart("d",Date), 2)

strSafeTime = Right("0" & Hour(Now), 2) & Right("0" & Minute(Now), 2) & Right("0" & Second(Now), 2)

strDateTime = strSafeDate & "_" & strSafeTime

Set fso = CreateObject("Scripting.FileSystemObject")

FilePath = "C:\Users\vinoth.srinivasan\Desktop\VB_Scritping\MyText" & "_" & strDateTime & ".txt"

'FilePath= "C:\Users\vinoth.srinivasan\newtextfile" & counter & ".txt"

Set myfile = fso.CreateTextFile(FilePath,2)

myfile.WriteLine "Line1"

myfile.WriteLine "Line2"

myfile.close

set myfile =Nothing

set fso=Nothing

Next

How to create dependent file parameters in loadrunner?

Loadrunner

Scenario:

We have username and password to be parameterized in which password selected should be dependent on the username selected, means username and password should match.

Solution:

This can be achieved by creating dependent file parameters in loadrunner.

Step 1: Create 2 file parameters with names UserID & Password.
Step 2: Add 2 columns in each parameter and name them as UserID & Password.
Step 3: Add the input userid's and passwords to the respective columns in both the parameters.
Step 4: In the UserID parameter select column by name option and select the column UserID.
Step 5: In the UserID parameter select the option Sequential/Unique/Random for select next row.
Step 6: In the Password parameter select column by name option and select the column Password.
Step 7: In the Password parameter select the option Same line as UserID for select next row.

Now username and the corresponding password will be passed to the scripts.

Login once - execute business process multiple times - Logout once scenario Jmeter

Scenario : 
We need to execute the below scenario in Jmeter for a duration of 3600 seconds.
1)Login only once
2)Business Process need to run multiple times
3)Logout Only once

Limitation with loop controller:

Some might suggest to use the Loop Controller (i.e) we need to add all the Business Process HTTP  Samplers into loop controller, set the loop count to forever and set the duration to 3600 seconds in Scheduler Configuration (Thread Group setting).

Login HTTP Sampler
Loop controller - Business Process HTTP Sampler
Logout HTTP Sampler

But the drawback in this approach is, Loop counter will execute infinitely till the completion of the duration of the test and logout will not be executed. Hence the sessions will not get terminated and we have to wait till the session timeout. So running back to back tests will be impossible.

Solution:

This can be achieved using Simple and Runtime Controllers

Login - Simple controller 
Runtime controller with business logic set to desired run-time (3600 Seconds)
Logout - Simple controller

In this approach the user will login once and execute the business process repeatedly for the configured duration in the runtime controller and logout once.

This is something similar to Vuser Init (once) - Action (iterate) - Vuser End (once) in loadrunner.

How to automate reading the file names in a directory and give them as input to loadrunner parameter? (VB Script)

Loadrunner

Scenario:

When we are load testing the file upload scenario in loadrunner, we need to give unique file names as input to the file parameter. The amount of files to be given as input may vary from thousands to lakhs based on our load.

Now the question is how to give those thousands and lakhs of files names as input to the parameter file?

The below VB Script comes handy to read all the input file names in a folder and write to a text file which can be given as input to the load runner parameter.

VB Script:

Set fs = CreateObject("Scripting.FileSystemObject")
'File in which file names to be witten
Set logFile = fs.OpenTextFile("C:\Users\vinoth.srinivasan\Desktop\VB_Scritping\Vinoth.txt", 2, True)
'Directory from which file names to be read
Set folder = fs.GetFolder("C:\Users\vinoth.srinivasan\Desktop\VB_Scritping")

Set files = folder.Files
  For Each file in files
    logFile.writeline(file.name)

  Next
logFile.close

How to paramterize the file path & name which is declared as a C variable in loadrunner?

Loadrunner

We all would be familiar with C file operations in loadrunner, like to open a text file, do some operations on it and then close the file. Now lets see how to handle the below scenario in loadrunner.

Scenario: Open a text file, write some content to it and upload it into a web application. Please note that each text file should be unique.

So this scenario during load test can be explained as, each user should upload a unique text file during each iteration after writing some content on it.

This can be achieved in loadrunner using C file operations. We can declare the file path & name and using fopen function open the text file and do some operation, save it and using fclose function we can close the file.

Code will be like

char * filename ="C:\\Text Files\\Sample.txt";

long file;

fopen(filename, "r");
if ((file = fopen(filename, "r")) == NULL) {
lr_error_message ("Cannot open %s", file);
return -1;
}
fclose(file);

But in our case each text file should be unique. So we need to parameterize the file path & name in the below line, so each time the file operations will be done on a unique text file.

char * filename ="C:\\Text Files\\Sample.txt"; (parameterize the file path and name)

and the code will be like

char filename[500];
long file;

strcpy((char *)filename,lr_eval_string("{P_FileName}")); // P_FileName is a loadrunner file parameter

fopen(filename, "r");
if ((file = fopen(filename, "r")) == NULL) {
lr_error_message ("Cannot open %s", file);
return -1;
}
fclose(file);

• Now the loadrunner file parameter P_FileName will hold the file path & name C:\\Text Files\\Sample.txt.
• lr_eval_string will extract the value from the parameter P_FileName and copy it to the character pointer variable filename.

Significance of Think time in performance testing

Client Side Definition:
It is the time taken by the users to think or to navigate to different pages in the application.

Server Side Definition:
It is the time between the completion of one request and the start of the other request.

Significance of think time:

1. As per the client side definition think time is needed to simulate real time user behavior during the performance tests.
2. It is uncertain that all the users will wait or think for the same time across the pages in the application. Hence the think time need to be randomized during the performance tests.
3. Think time should not be placed in between the start and end transactions, if kept it will add to the response time of the page, resulting in higher or improper results.
4. Think time is factor which controls the rate at which requests reach the server (hits per second). Think time is inversely proportional to hits per second (load on the server).
5. Ignoring or removing the think time during the performance tests will flood the server with lots of request and will eventually overload & break it.
6. Think time ensures that concurrency (avoiding the same transactions) is maintained between the users.

Reading the full content of a text file using dynamic memory allocation (malloc) - Loadrunner

Loadrunner

Overview:

When we are trying to perform uploading files of different size scenario in loadrunner the below C Program comes handy.

C Program:

#define SEEK_SET 0 /* beginning of file. */
#define SEEK_CUR 1 /* current position. */
#define SEEK_END 2   /* end of file */


Action()
{
 long infile; // file pointer
 char *buffer; // buffer to read file contents into
 char *filename = "test.txt"; // file to read
 int fileLen; // file size
 int bytesRead; // bytes read from file
 //
 // open the file
 infile = fopen(filename, "rb");
 if (!infile) {
    lr_error_message("Unable to open file %s", filename);
    return;
 }


 // get the file length
 fseek(infile, 0, SEEK_END);
 fileLen=ftell(infile);
 fseek(infile, 0, SEEK_SET);
 lr_log_message("File length is: %9d bytes.", fileLen);


 // Allocate memory for buffer to read file
 buffer=(char *)malloc(fileLen+1);
 if (!buffer) {
    lr_error_message("Could not malloc %10d bytes", fileLen+1);
    fclose(infile);
    return;
 }


 // Read file contents into buffer
 bytesRead = fread(buffer, 1, fileLen, infile);
 if (bytesRead != fileLen)
 {
    lr_error_message("File length is %10d bytes but only read %10d bytes", fileLen, bytesRead);
 }
 else
 {
    lr_log_message("Successfully read %9d bytes from file: ", bytesRead);
 }
 fclose(infile);


 // Save the buffer to a loadrunner parameter
 lr_save_var( buffer, bytesRead, 0, "fileDataParameter");
 free(buffer);
 lr_log_message("File contents: %s", lr_eval_string("{fileDataParameter}"));
}

lr_read_file Loadrunner Function to read the content of a text file

Loadrunner

Overview:

1. When we record a file import in a web app using load runner, we might some times find that the files content will also be passed in the web_custom_request as a JSON request.
2. When we iterate the script we need to pass unique files as inputs and the files content also change.
3. So we need to read the files content using C Program in loadrunner and convert it into JSON format and save it in a variable and pass the variable to the web_custom_request.
4. lr_read_file loadrunner function reads the files content and saves it in a loadrunner variable.
5. As part of the JSON request CR(Carriage Return) & LF(Line Feed) at the end of each line in the text file needs to be removed. This can be achieved using strtok (StringToken) C function.

C Program:

long file;
int flength;
int fcontent;
char *mbuffer;
char *delimiter = "\r\n";
char *token;
char  *temp;
char  *tempall;
char *finalfile;
int res;

temp = (char *)calloc(8000000,sizeof(char));
tempall = (char *)calloc(8000000,sizeof(char));
finalfile = (char *)calloc(8000000,sizeof(char));
 
res = lr_read_file("C:\\ACH EFT Files\\ACH_10K_50003918_0.txt", "P_fileContent", 0);
lr_message("res = %d\n data = %s\n", res, lr_eval_string("{P_fileContent}"));
 
token = (char *)strtok(lr_eval_string("{P_fileContent}"), delimiter); // capture 1st sub string based on defined delimiter

while (token != NULL ) // While valid tokens are returned

{
        lr_output_message ("%s", token );
         strcpy(temp, token);
         strcat(temp, "\\r\\n");
        lr_output_message ("%s", temp );
       
token = (char *)strtok(NULL, delimiter); // Get the next token
strcat(tempall, temp);
 }
 
memcpy(finalfile, tempall, strlen(tempall) -8);

lr_save_string(finalfile,"P_fileContent1");
lr_output_message("%s", lr_eval_string("{P_fileContent1}"));

We can also pass parameter values to the files path and name in lr_read_file function using lr_eval_string function

Example:

res = lr_read_file("C:\\ACH EFT Files\\ACH_10K_50003918_0.txt", "P_fileContent", 0);

In the above code replace C:\\ACH EFT Files\\ACH_10K_50003918_0.txt with the parameter P_FileName.

Now the code becomes,

res = lr_read_file(lr_eval_string("{P_FileName}"), "P_fileContent", 0);