Using analytical data for measuring the concurrency

The most important thing as part of requirement gathering for a performance testing engagement is to measure the concurrency that has to be simulated/generated. A performance tester needs to get access to the server logs / Analytical data to perform this action.

Let us take a small example of a restaurant which says they server around 1500 customers a day for lunch, where the lunch hours being 12:00 – 3:00 PM. This translates to 500 customers every hour during their lunch time. Now the key factor here is the average duration of a customer.

If people spend an hour for their lunch, it means that the restaurant has to accommodate 500 people in one slot. Assuming that the average time spent at lunch is 30 minutes, now the restaurant has to accommodate only 250 users. 

Now the number either 500 or 250 is the number of customers having their lunch at the same time, which is also known as concurrency.

This is exactly the same way that we calculate the number of concurrent users on the system. The important factors to be considered while calculating the concurrency are:

• The average duration of a visitor on the application
• Total number of visits on a peak day / peak hour

Now to get the above said statistics, one can either configure Google analytics on the application or refer to Web server logs that have the information about the user session start time and end time. The third alternative is to get these statistics from the Business analyst in case if the target load is a forecast or prediction for the future (like the coming thanks giving day / Christmas Eve / Memorial Day/ Super bowl day etc.)

The scientific way of calculating the concurrency is to use little’s law. The little’s law says  “The long-term average number of customers in a stable system L is equal to the long-term average effective arrival rate, λ, multiplied by the average time a customer spends in the system, W; or expressed algebraically: L = λW.” For more details on little’s law refer wiki (http://en.wikipedia.org/wiki/Little's_law )

The total number of users on a web application can be determined by the average arrival rate (100 users / hour and average time spent in the application 1/10 hours (i.e. 6 minutes). Now the concurrency at any point of time L = 100*1/10 = 10 users.

Hope this information is helpful in determining the target user load for an application.

Being / Becoming a Performance Tester

If you wanted to become a performance tester or make your career in performance testing, below are the important characteristics that you need to have:

Common Sense and observation

There are several jobs which do not require applying common sense or minute observation skills. But Performance testing is not definitely one of those. You should have lot of observation as well as common sense.

Jump into the shoes of others:

The performance testing job expects you think from the various perspectives like from the customer / end user point of view, from the business point of view / from the application / technology point of view etc.  While automating the use cases you should think from the end user perspective. When providing a performance test report, you should ensure that the report can be understood for the business analysts as well as the one who owns the entire business. While analyzing the root causes for the performance issues, you should imagine yourself as a technologist.

Willingness to Learn:

Learning performance testing is not one time activity, though the basics remain the same. With the new technologies emerging pretty quickly, as a performance tester you should upgrade yourself to the new technologies and understand the core of them. Continuous learning is the one vital factor that makes a performance tester career prolonged.

The above said are the basic things needed. However, to start your career as a performance tester below is the technical stuff that you should have at least basic understanding about

Operating systems: How operating systems work, the CPU scheduling, memory management, disk management etc

Databases: The tables, queries, joins and indexes and how a query is executed

Computer networks: How the communication between the client and server happens, the TCP, UDP communications, HTTP

Web Application architecture and browser properties: Web, App and DB Servers, the way they work, some important features of web applications like session ID, cookies, request methods and response

Any programming language – You should be good in writing the basic programs with any one programming language

So, if you are considering a career in performance testing, ensure you have all the above said requirements met.

Understanding Randomization in LoadRunner

Randomization is a common practice with majority of the performance test scripts, where dropdown / Radio buttons / Links has to be chosen by the user randomly.

For example the application requires choosing a value from the drop down randomly, then the important thing is to know how many values are there in the drop down and what their values.

We all know the web_reg_save_param() or web_reg_save_param_ex() function has to be used to capture the values. Now, because we need all the values we'll specify the attribute "ORD=ALL" / "Ordinal=all". Assume that the function is written as below:

web_reg_save_param("DropdownValues", "Lb=<option>","rb=</option>", "Ord=all", LAST);

When the above function is executed, LoadRunner creates N+1 LR variables if there are N values in the drop down.

The first drop down value will be saved to an LR variable named "DropdownValues_1" and the second value from the drop down will be save to an LR variable named "DropdownValues_2" and so on. The total number of matches / count would be saved to another parameter called "DropdownValues_count".

The function used to choose a random value is lr_paramarr_random(). This function returns an address location of the random parameter.

In our example the function should be used as:

char * randVal;


randVal = lr_paramarr_random("DropdownValues");

As the variable randVal is C string variable, cannot be used in protocol functions,  this has to be converted to an LR parameter using below statement (for details on why, please refer to variable conversion in LoadRunner).
lr_save_string(randVal, "RandDropdownVal");

When lr_paramarr_random("DropdownValues") is seen the script, the script immediately checks if there is a parameter named "DropdownValues_count" and tries to retrieve its value. If this parameter is not available, an error would be thrown.

Assuming that the count is 10, now it generates a random value between 1 -  10 and refers to the corresponding address location. i.e. if the random number is 6, it searches if there is an LR parameter called "DropdownValues_6", if yes,  its address location is returned to the string variable.

If there is no such parameter, it would throw a warning saying that "The string with parameter delimiters is not a parameter"

Hence it is very important to capture all the values using the web_reg_save_param() function with the ORD attribute set to ALL if at all the parameter has to be used for randomization.

In the next article, we will see how we can utilize this behavior in building a custom C function that works similar to web_reg_save_param() with ORD being set to ALL.

in LoadRunner Did you Know?

• Save the current times tamp in Milli seconds to an LR parameter can be achieved with the function web_save_timestamp_param()
• web_url() and web_submit_data() are capable of the encoding the form data automatically. But web_custom_request() cannot encode the form data automatically. The encoding can be done by the function web_convert_param()
• Automation of the windows based web applications (that uses NTLM authentication) uses the web_set_user() function for sending the username and passwords. While replaying the script, it is recommended to use the run time setting Preferences --> Advanced -->WinInet Replay instead of Sockets
• Reading and writing from the same file can be achieved by Virtual Table Server
• The function lr_paramarr_random("x") first validates if there is an LR parameter called x_count, if there is no such parameter, it returns an error
•  The function web_global_verification() works same as the run time setting Content check, except that the global verification can be paused and resumed with the functions web_global_verification_pause() and web_global_verification_resume()

Variable conversion in LoadRunner

In the last post, I have given an overview on what types of variables are seen in an LR Script and where they should be used.

This article covers on conversion of LR Variables to C Variables and viceversa.  There are three important LR functions used for the conversion. 
We'll  see one at a time.

lr_eval_string()
This function evaluates a load runner variable and returns its value if placed in flower braces. Let us try to understand how it really works.

lr_eval_string("test") returns a string called "test"
lr_eval_string("{test}") - Now this function checks if there is LoadRunner variable called test or not. 
If there is a LoadRunner variable named test and its value is "Welcome". Now the function lr_eval_string("{test}") returns a string "Welcome"

If no LoadRunner variable exists with the name test, then the function lr_eval_string("{test}") returns a string "{test}"

Now let us see two more last example of lr_eval_string() function:

Assumption: There is an LR variable called test whose value is "{Welcome}". There is another LR variable called Welcome whose value is "Performance Testing Online".

lr_eval_string("{test}") - This returns a string "{Welcome}"
lr_eval_string() function does not do nested evaluation

lr_eval_string(lr_eval_string("{test}"))
 = lr_eval_string("{Welcome}");
 = "Performance Testing Online"

LR to C String
Now that you understand the usage of lr_eval_string() function let us see how we can convert an LR Variable to C string variable.

char CStr[100];

to save "Performance Testing Online" into the string variable CStr, the syntax is:

strcpy(CStr, lr_eval_string(lr_eval_string("{test}")));

Please note that an equal symbol (=) cannot be used to assign a value to a string variable. We will have a separate article on why.

So, the conclusion is, to save an LR value to a C string variable, the lr_eval_string() function is used.

Now let us see, how we can convert an LR Parameter to C integer variable
 
let us assume that the LR variable test stores a value of "10".

LR to C Integers
This value should be converted to a string format first using lr_eval_string() function. This function returns a string value. To convert a string value into an integer, we have C function called atoi() read it as A to I (meaning alphabets to Integers)

int Count;

Count = atoi(lr_eval_string("{test}"));

This is how a value in LR variable is saved to a C variable. Though the heading says converting an LR variable to C variable, in fact we are storing the value of an LR variable into a C variable.

Now let us see the otherway i.e. converting a C integer / String to an LR parameter.

we have two functions named lr_save_string() and lr_save_int() to save a string and integer values to an LR parameter respectivily.

C to LR
int i = 256;

lr_save_int(i, "Total"); 

Now Total is an LR variable which stored a value of 256 in it. Inside the web_url() or web_submit_data or web_custom_request it can be used as {Total}

char Text[100] = "Welcome to Performance Testing Online";

lr_save_string(text, "Status");
Now Status is an LR variable which stored the entire string in it. In a web request it should be used as {Status}


I hope this article is informative and helpful.  If you have any questions, please feel free to comment.

Thanks,
Ram N

Variables in a LoadRunner script

We have already seen the various types of functions in a Web (Http/HTML) script. The language used for the Web based script is "C", which means all the C data types can be used in a LoadRunner script.

In a Web protocol script, the variables are classified into two categories.

• C variables
• LR variables

C Variables - Being a procedural language, C expects all the variables to be declared at the beginning of the function/action.
For Example:

int randNum;
char Strbuff[100];
float thinktime;

However, any LR variable does not need any declaration as such. But, the script expects the variable name to be specified in the function.

For example

web_reg_save_param("SessID", "LB=............","RB=", LAST);
in the above statement, we are explicitly saying that SessID is an LR variable to which the dynamic data is to be saved.

Let us look at few other examples on how the LR variables specified in the script.

web_reg_find("Text=SampleText", "SaveCount=txt_Count", LAST);
in the above example, the LR variable that is created is txt_Count.

lr_save_string("SampleData", "x"); /lr_save_int(10, "y")

In the above cases, the string "Sample Data" or the integer 10 is stored to  LR variable called x /y ( created first and then value is stored).

So, the conclusion is C variables are to be declared at the beginning of the script and LR variables need not be.

Now the most important aspect is where to use what type of variables? Where can C variables be used, where can LR variables be used.

Putting it in simple terms, the protocol based functions like web_url()/web_submit_data(), LR variables are to be used with in flower braces for example {x} or {SessID} or {txt_Count}

In the Language specific functions, its obvious that only C variables are to be used. For example

randNum = rand()%Count + 1;
sprintf(buff, "Test_%d", i);
strcpy("Welcome", StrBuff);

LR functions accepts either types of parameters. For example, 

lr_save_int(i, "y") - where i is C variable and Y will become an LR variable.

lr_eval_string("{x}"); - here x is an LR parameter.

I hope this article provides you the basic information on types of variables of an LR Script. The next article explains how to convert these parameters from type to the other.

Understanding various log files in LoadRunner

Once a script is recorded using LoadRunner tool, one can notice 4 different tabs as part of the output window. This article is all about these tabs and their usage. The 4 tabs that you notice are: 

•   Replay Log
•  Recording Log
• Correlation Results
• Generation Log

Let us start the way these logs are generated.

Recording Log:

When a script is being recorded, the Virtual User Generator records all the communication that has happened between the client and the server into a log called “Recording Log.” Though this is not in a much readable format, the recording log will be the base for the generation log.

The option Regenerate script (Navigate to tools à Regenerate script) works purely using the recording log. If the recording log is missing, the script cannot be regenerated with different recording options

Generation Log:

Generation contains the information about the recording options used for the script, the request header, request body, response header and response body and the function that simulates the request. This function may be varied based on the recording options used.

The generation log content may be changed based on the recording options that are used and for the generation Log, recording log is input file

Once generated, the contents of the recording and generation logs are not altered

Replay Log

This log displays the output when the script is replayed. This log is helpful to debug the script and  customize the script. The contents of this log file can be controlled based on the run time settings (Vuser à Run Time settings à Log à either standard log or extended log)

The output functions like lr_output_message, lr_log_message() and lr_error_message() would write their content to the Replay Log.

Correlation Results

Once a script is recorded and replayed,  the script can be verified for the dynamic data.  The option “Scan for correlations” is used to compare the recording  and replay data and to highlight the dynamic data if any. The script can be correlated directly from this tab and this is one form of auto correlating the script.

As it compares the recording time data and the replay data, it is always necessary to have the “data” folder inside the script folder

I believe this article is informative and helpful.

Understanding a LoadRunner Web (HTTP/HTML) script

Once a script is recorded using the Virtual User Generator with the best recording options (please refer to the link ), the next important thing is to understand the complete script and the functions recorded in the script.

At a very high level, the entire script can be classified into three groups i.e.

• Protocol Specific functions
• LoadRunner functions
• Language specific functions

Protocol Specific functions: 
These functions can be used with a specific protocol only and they cannot be used with any other protocol. For a Web (Http/HTML) protocol, the commonly seen functions are:

• web_url(), web_image(), web_link() - All these functions are used to simulate a GET request
• Web_submit_form() is used to simulate a POST request
• web_submit_data() is used to simulate both GET and POST requests. The attribute "Method" tells if that request is a GET or POST.

web_reg_find() and web_reg_save_param() are the service functions used for page verification and correlation respectively.
All the above functions are starting with the word "web" indicating that they are web protocol specific functions and cannot be used outside Web protocol.
Few other web protocol functions are:

web_set_user(), web_set_max_html_param_len(), web_cache_cleanup(), web_cleanup_cookies() etc

LoadRunner functions:
All these functions are loadRunner specific and can be used across any protocol. All these functions start with lr_. Few examples for LoadRunner functions are:

lr_start_transaction() - To start a transaction to measure the response time
lr_end_transaction() - To stop a transaction to measure the response time
lr_think_time() - to wait for a specified duration
lr_eval_string() - To evaluate the value of a LoadRunner parameter placed in {}
lr_save_string() - To save a string to a LoadRunner parameter
lr_save_int() - To save an integer value to a LoadRunner parameter
lr_exit() - exit a loop/iteration/user from execution
lr_set_debug_message() - To control the log settings of the replay log
lr_output_message() - To write to the output log with an information level setting
lr_error_message() - To write to the output message with error level
etc

Language specific functions:
These functions are not a part of the tool, all the functions that can be used in a language(C or Java) can be directly used in the script provided that the protocol is supported in a language.
For example, The Web(HTTP/HTML) protocol is supported by the C language. Hence all the C functions can be used directly in Web script.
The commonly used functions are:

rand() - to generate a random number
atoi()  - converting a string into an integer
sprintf - Saving a formatted output to a string buffer
strcat - string concatenation
strcpy - Copying into a string
strtok - String tokenizer function

The JAVA protocol cannot support the C language and hence these functions cannot be used by the Java based script.

Hope the information provided about the classification a LoadRunner script is helpful in understanding how the scripts work.

Context based Recording Vs Context less Recording and HTML Vs URL based recording

To understand the "context based recording", if some one asks you a question "How is he doing?", you would definitely ask "Whom are you referring to?". But if the same question is asked during a discussion about one of your friends whose name is  Karthik, you would not ask the question because in the current context HE refers to Karthik. So you can understand who "HE" is. 


The context less question would be like "How is Karthik doing?". There won't be any more questions. Because you are explicitly pointing to a person and not using any generic terms like He, it. This is called context less mode.

In the above example of "HE", "HE" refers to Karthik only for that discussion and for a different discussion "HE" may refer to someone else.  The context is only until the discussion.
 
There are two recording modes available in LoadRunner to record the user actions of a Web Application. They are

·         HTML mode

·         URL Mode

HTML mode – In this mode of recording, each USER ACTION is recorded as a separate request.  To put it in simple terms, each HTML content (usually a page except in case of HTML Frames) is recorded as a request

If all the user actions are recorded into a Single ACTION of the script, then the HTML mode of recording acts as "Context Based Recording".

The Virtual User Generator understands the Context by looking at the previous request's response. Hence it identifies the Forms, Images etc and you would notice the below functions in a context based recording of the script.
web_submit_form() - to simulate a POST Request
web_image(), web_link() - to simulate a GET request



In VUGen, the context is applicable only till the Action. If you record a user action into a different action, the context would reset and again the context has to be created.

In a case where the tool has to enter some data in a form and if the form is not found in its previous response, the tool will halt the execution of the script.  Every request is directly dependant on the previous request’s response and has high chances of failure with the UI changes to the web application

The advantages of using the HTML recording mode is that the size is very compact and the customization efforts would be very less. The other side of the coin is that, with the UI changes to the web applications, these scripts would require very high maintenance costs.

URL Mode: In this mode of recording each resource requested by the user is recorded as a separate request.  in other words, whatever the content (like images, CSS, JS, HTML) that makes the HTML page is recorded as a separate request. When a web site is launched apart from the HTML content, there would be lot of images, java script files, CSS files downloaded. All these are called resources and each resource is recorded as a separate request.

URL mode is always context less recording because this mode refers to the data and the URL of a request directly instead of depending on previous response. This mode does not depend on the UI of the application, rather the actions associated with the user actions performed on UI. As each resource is recorded, the size of the script would be very high and this also involves in lot of customization. The benefit of having scripts with URL recording mode is that the maintenance cost associated with these are very less and can be used across various releases of the product despite lot of UI changes.

Usually URL mode of recording is used with Non –browser applications in other words any thick client activity is recorded using the URL mode.

Trade off – with the HTML recording mode, another option is available under “Advanced options” of HTML mode. The option available is Explicit URLs only mode.

The benefits of the HTML mode and URL mode is clubbed together in HTML à Explicit URLs only. With this mode, the size of the script would be compact (as only the user action is recorded as request, not at the UI level but at the request level) and requires a bit of more customization efforts but has the advance of high maintainability for longer durations. This is the most recommended recording mode for web applications where the scripts have to be maintained for longer durations

Tip: Have you forgotten to record the script using HTML à Explicit URLs mode? No problem....

Change the recording options to HTML à Explicit URLs only and now navigate to Toolsà Regenerate script. The regenerated script is as fresh as a recorded script using HTML Explicit URL’s only. But do remember that whatever the changes that are made to the script would be gone if the script is regenerated.

Next post: Understanding various logs in LoadRunner

Performance testing from tool perspective

The objective of any performance test is to assess the capabilities and limitations of the system when it is subjected to realistic user load. The keyword here is realistic user load. A performance test would be more realistic if it has the below items implemented.

• Simulation of real user behavior
• Simulation of real user load patterns

Simulation of real user behavior is to understand the way the application is accessed by an end user. There are four important aspects of Real user behavior simulation. They are as below:

Real user navigation are different - If there are 100 users accessing a web application,  its not mandatory that all of them using the application for the same purpose. For example a Banking application can be used by the users either to transfer funds, check their balances, Add beneficiaries, download statements and many more. Each of these navigation should be recorded as a separate script.

Real user inputs are different - The inputs provided by the end users to the product are also different. Different people log in with different user name and passwords, search for different products, enter different shipping and permanent addresses etc. To Simulate this aspect, each script has to be parameterized

Real user Selections are different - Once the users enter the application, their selections also can be different (selections from dropdown, radio buttons, check boxes etc). For example in case of booking flights, even if two users are searching for flights from Hyderabad to New York , there can be several flights available and there are chances that both the users may choose different flights. To simulate this aspect of real user behavior, each script has to be correlated

Real users wait between the requests - There would be some waiting time between two successive requests made by a real user. For example the real user launches gmail.com (request1)  and then he enters the username and password and click on Log in(request2). So between request1 and request2, the user waits for some time (this wait time is called think time, and the user waits between requests either to read content and fill in some details or to think and act upon). The same has to be implemented in every script by including think times in the script.

All the above are achieved using the Scripting part of the performance tool (in case of LR, it is VUGen).

The other important aspect of a realistic performance test is to simulate the realistic load patterns
which includes

• Simulating  User Mix (Various types of users for ex: Registered users vs non registered users, Admin users etc)
• Simulating Transaction Mix (The ratio at which each identified use case has to be executed or the load distribution across various use cases)
• Simulating Users' Geographical Locations - Either by having the Load Generators from the locations where the users are coming from or Having the WAN Simulator to simulate the users' geographical locations
• Simulating User's bandwidth - Though few users are from same location, there are possibilities that their Bandwidths are different. This point has to be kept in mind while generating the user load on to the system
• Simulating traffic rate - The rate at which the users come on to the system, the duration the peak user load is maintained, the rate at which users drop off from the system

Most of the above can be achieved in controller.

If the above two points are considered, a performance test that is executed will be more realistic and the results are more accurate.