Easiest way to test SCEP with Afaria is to make use of the delivered ServerSCEPtest application. This application comes with Afaria`s PackageServer component. It can be found in the bin directory of the package server.
The test application is a Windows executable that executes the SCEP process through Afaria. You have two options available for the test:
I am going to execute the SCEP/NDES test using the package server. This is the Afaria component used by all clients to receive a client certificate for apps.
To run the test, at least the common name value must be filled in. This is the CN= part of the certificate. Normally, this is your user id. Unfortunately, the test tool is limited to 2048 bit key (Afaria SP8) and does not select you higher or custom values. To run the test, just select perform test button. The additional CSR informations like city, org, etc are taken from the package server configuration. These values are given by the Afaria admin.
The status of the SCEP process is shown in the log area. You can see that the CSR is created and send to the package server CA. After the test ran without errors, the returned certificate is saved to: C:\ProgramData\SAP\Afaria.
The see and validate the value of the new certificate, you can use the Crypto Shell Extensions of Windows Server.
The certificate was issued by the CA: CA. Lifetime is one year. And the template is AfariaUser. This matches exactly how the NDES template was configured.
To be 100% sure, the CA can be consulted. Normally, all issued certificates are stored there and can be consulted. Taking a look into the issued certificate list, I can see that a new certificate by the NDES user was issued using as a template AfariaUser. Therefore, the new NDES configuration is validated and working.
Note: this blog is also published at SAP Community Network: http://scn.sap.com/community/mobile/blog/2014/10/13/controlling-android-network-state-with-jmeter
Testing a mobile web scenario includes testing the app end to end for different usage scenarios. This includes testing the state of the mobile device: online and offline. Consider the following scenarios: a HTTP POST request is send to SMP 3 which in turn will send out a push notification, or the UI5 app opens a HTTP request while the device is offline. Will the test pass as expected?
The Android emulator allows controlling the state of network connectivity by pressing F8. This turns gsm data on or off. Manually turning the data network on or off is possible while executing a test, but does not really reflect a real scenario where this happens randomly (entering elevator, subway, bad connection, etc) and is out of your control. To simulate a suddenly not available data connection means to be able to randomly deactivate and activate a data connection on the emulator.
First thing of course is to know how to turn off network in an automated way. To control the network state of the Android emulator, the command: gsm data off/on needs to be issued. To execute the command it is necessary to connect via telnet to the emulator.
Command: telnet localhost 5554
Note: 5554 is the number of the emulator. Connecting via adb shell won’t work.
Telnet opens the command shell and allows executing gsm command to turn off or on the network.
In case you followed some of my other blogs about testing, you`ll have noticed that I use jMeter. JMeter comes with an impressive list of functionality allowing you to cover almost everything in your test scenario. Therefore, my first thought was to use jMeter to control the network state of the Android emulator. This way, I can include this behavior in other test cases executed by jMeter.
While the commands and execution order is now known, a problem last: Windows telnet client cannot read a list of commands form command line and there is no jMeter telnet client plugin. There is no easy way to send the commands from jMeter to Android. These are two problems to solve:
Send scripted telnet commands
Execute these from jMeter
Send scripted telnet commands
Let’s solve problem 1: One possibility is to use a VB script to send the commands by emulating keyboard strikes. This implies that you should not use the keyboard at the same time – and writing a VBscript. And it does not solve the problem that depending on your Windows version you first will have to install telnet or even won`t be allowed doing so. The solution is a tool known in the Linux world, a tool created in a time WWW was only in its infancy and known only to a few: nc. The last released stable version of nc was released 10 years ago. Fortunately, there is a newer (better) implementation available from nmap: ncat. Ncat is included in the nmap download and available therefore for Windows. No compilation needed, just download and run ncat.exe. To open a telnet session to Android with ncat, the command is: ncat -4 –t localhost 5554
The commands to deactivate gsm data are:
gsm data off
Save these to a input file (e.g. gms_off.txt) and let ncat read the file. Command: ncat -4 –t localhost 5554 < gsm_off.txt
This solves issue #1. Now telnet can be used to send a list of commands to Android. Next is to solve issue #2 so jMeter can execute the ncat command.
Execute ncat from jMeter
jMeter offers a OS Process sampler that allows executing a command line program. I wasn`t able to start ncat.exe directly; starting CMD and pass ncat as a parameter however works.
The command to be run is CMD and the input parameters are:
/C <path to nmap>\ncat.exe -4 –t localhost 5554 < <path>\gms_off.txt
Including another sampler for turning data connectivity on again lets jMeter control the network state. Adding a timer element adds the random part to the test. Of course the procedure outlined here is not not applicable to network data, but to all commands available by the Android command shell.
To make it easier a jMeter test plan is attached to this blog. This plan can be used in conjunction with a WDJ application. Both were used for the creation of this blog.
JMeter is a great tool for testing your web applications. And you can even use it to test Web Dynpro Java applications. To facilitate the work of creating the tests, jMeter comes with a proxy to intercept browser requests and add them to the test. This allows for easy and fast recording of tests.
jMeter test for sample WDJ application:
As by default a WDJ application is accessible as anonymous – or Guest – user, authentication is not needed and a test can be executed several times against the WDJ application. Pretty easy but in real life you’ll hit real soon a hard limit. By default ,the HTTP Cookie Manager controller does not delete cookies with each iteration, or: JSESSIONID is not deleted.
Running the test and checking the user sessions in NWA shows that the same Guest session is used.
With that you may be tempted to run the test and to see the load generated by just 1 user. But try to run the test 50 times. After some calls, jMeter shows an error when executing the open WDJ app step.
The error message returned from the server is:
HTTP 403? Yes, 403. Re-using the same session will ensure that you`ll hit a well known limitation. You can now opt to implement SAP note 1012065, but it is better to end the session after each iteration. While only one Guest session is open, the maximum number of application sessions is exceeded: 23
Now let jMeter create a new session with each iteration.
The new sessions are created because the HTTP Cookie Manager deletes the JSESSIONID cookie with each iteration. Run the test 5 times, the open sessions in NWA will show
To access the report, go to: Availability & Performance -> Resource Monitoring -> Session Monitoring
If now a test with 500 iterations is run, 500 Guest sessions will be created. This may not a problem, but won’t reflect your reality where sessions are ended after the user navigates away. Logging in and off creates additional load, something worth to test too. Besides this, each session occupies some memory. Depending on your test case, this amount of memory can consume quite some substantial part of the free memory, compromising your test results.
The correct way to test WDJ applications is to end the user session after the test is executed, independently if Guest or named users are used. To end the session, all it takes is to delete the cookies each iteration and trigger a logout action:/irj/portal?logout_submit=1 [you have the portal installed in the same instance, right? If not, you’ll have to add yourown logoff functionalityto the application]
Add this step at the end of the test.
With a HTTP sampler as the last step, each iteration can be started with a new session without overloading the NetWeaver Java server with Guest or user sessions.
In my blog about performance load testing with ab SAP Mentor Anton Wenzelhuemer raised a question if and how you can use ab to test a specific action/event in WD applications. That’s a good question as these events besides input validation normally trigger a connection to the backend and change the context node and attributes. And that is the part where you can improve the performance of the application.
Now, is it possible to test a user action with ab? To remind you, ab is used to test a single resource. The intent is to see how fast your web server can serve a HTML page and test how different configuration parameters affect the performance. You cannot test the flow of an action (load page, press button, get result). To find out if ab can be used to test a WD application, you have to find out what happens when you trigger the event.
Sample WDJ application
I build an example WDJ application using the steps outlined here: it’s and Web Dynpro Java application containing a table that shows the data of the get flight BAPI. For testing forms that require an input you cannot use ab (meaning: setting the input and call the form action), but you can use ab to simulate actually what happens after the input is set and the send button is clicked: the POST action of the browser.
[For testing the form from command line: to insert the airline Id and press the button, use curl, as curl allows interacting with forms. But curl is not a load testing tool (you may look at curl-loader).]
To make things easy for me I implemented a button (named: Generic) that sets the input of the airline to LH, triggers an event that causes the controller to call the BAPI and return the table content. Clicking this button I can record the POST action with Firebug:
Replaying the POST in the browser gives me the content WDJ returns to the browser:
Response in the browser
Response in firebug
The returned content is an XML file containing the data for the table. As you can see, WDJ return the table content as HTML. [Note: Whoever at SAP was or is or will be responsible for this: why? And please stop doing this. This should be done via AJAX and JSON / XML, and definitely you should not transfer HTML. That makes me wonder if the HTML transferred back changes between browsers.]
Conclusion: That POST triggers the server side event and returns the right content. This POST is what ab should send to the server.
How to get ab to send the exact same POST? For this ab offers 2 parameters that have to be used together: -p and –T. The parameter –p defines a file that contains the data to post and –T the content type. Saving the POST data in a file named postwdj.txt and set –T to application/x-www-form-urlencoded; charset=UTF-8.
Using only these two parameters won’t work with WD applications as WD checks the header for the user agent. The browser has to send a user agent that is supported by WD. To set the user agent for ab to IE (I use IE to be on the safe side as support for other browsers heavily depends on your SPS):
-H “User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)”.
The next step is to authenticate the session if the application needs an authenticated user. How exactly you do authentication depends on your server configuration, you can enable basic authentication with ab or send the JSESSION (or MYSAPSSO2) cookie to the WD app: -C JSESSION=<value>
Setting the parameters:
ab -v 4 -n 1 -p postwdj.txt -T “application/x-www-form-urlencoded; charset=UTF-8”
-H “User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)”
There are several WD specific sessions involved. And these expire too. Make sure that the POST data you get from firebug is valid when you execute ab. With updated POST data:
Now, is that the result expected? Sometimes ab outputs the first lines of the response, sometimes not. To really know what the server returned I use Wireshark. Wireshark allows you to trace the TCP packages and to view the transmitted HTML. The image below shows the result of a not working POST, as the web page of the WD app is returned:
HTTP 200, but the POST was not triggered correctly, so the WD application returns the default view:
A Wireshark trace of a successful ab POST:
And the result at the command line when ab prints the returned message:
Running the test with 1 thread 100 times:
Running the test with 30 thread 100 times:
The problem is that too many connections to the application were made, and the session was never terminated, thus causing more and more memory consumption until my server crashed. There is a SAP note explaining this behavior and that in the end of the test run you should terminate the session. With ab this won’t work.
While ab allows you to test web and portal applications quite easily, it is not flexible enough for testing WD applications. Instead of being able to just run the test, you have to get the POST data and run the test shortly after. What ab is missing is to submit forms dynamically (like curl) or to build the POST data based on what the server expects. But for that you need to consider the flow of the application, something ab is not aware of. For testing complex web applications there is a handy tool also available from Apache: jMeter.
Load testing an application is more than a task to prepare for the go-live: it has to cover all phases of the lifecycle. It allows tracing the behavior of the application over time, allowing identifying when performance degeneration happened. It’s part of a toolset to avoid “yesterday it worked fine …”.
From the many load testing tools the ones for testing the end user perspective are the ones most useful. For applications that have a web interface this means to test the load from the browser perspective. This way, not only a specific part of the application like the DB is tested: all components involved to generate the HTML output are tested: the web server, application server, network connection, backend applications, DB, and so on. There are several load testing tools available, some are endorsed by SAP and are made available by SAP partners. They have one thing in common (beside the price): they are not easy to use. Because of this they normally get added at the quality test phase, and that is already too late. I will focus here on tools that can be freely downloaded, installed, are simply to set up and use and have a community so finding solutions on the internet is not a big deal.
The Apache web server comes with a simple tool: ab (http://httpd.apache.org/docs/2.0/programs/ab.html). Basically, that tool does everything you need to test the performance of a web page. You can use ab to upload a file, set cookies to find out HTTP server limitations (like 413), emulate authentication, and so on. The intended use case of ab is to test the Apache web server and the impact of configuration parameters on its performance, but as it simulates quite well the load of several browsers and a flexible number of requests, it can be used to test every web page. You can try out parameter changes and see how they impact the performance, how your code handles several requests, find out the impact of the network, and many more.
To run a test that includes 1000 requests, distributed over 10 threads to a URL, the syntax is:
Ab gives already some important information like requests/sec, number of failed requests and transfer rate. It’s an easy to use tool but pretty low level. Running ab against a local installation of SAP Portal:
This example makes also clear that one important aspect of load testing is the network connection. Instead of making the requests over Wifi to the URL of the 1st example somewhere in the internet and thus slowing down the number of requests, here the requests are done locally and I get an impressive 575 requests per seconds. The key takeaway here is: do your load testing not only at one location, do it at several locations inside your network to rule out or find network related bottlenecks: start near your server for optimizing the performance of your application and then move slowly away. Of course, you should define for each of the locations a threshold.
To test your single application, you need
A direct link
User authentication (if implemented)
The direct link is easy to get. The user authentication with ab is tricky, but the –C parameter does the trick. This will not give real world results as ab will do the tests as the same user, but it gives an overview of the response time of the application. The application is reading some data from the request object and some KM properties.
I ran the test several time and what you cannot see here is that the number of request went up from 49 to 92 and then stayed there. That’s an easy application that only reads some request parameters. Where serving a simple html page came back with 575 hits/sec, the application running inside the portal is not – as expected – as fast.
Now I can add and remove some code to find out if the application runs faster or slower and this way find out where the performance bottleneck is.
A tool that can help you creating test cases for web applications is Selenium. Selenium offers a Firefox plugin called Selenium IDE that records what the user is clicking and translates these actions into HTML code. To test an application with Selenium the following steps are needed:
Record the user action
Select a test framework and save the recorded actions in Java code
Import the generated code into your Java project
For the 1st step, Selenium offers a Firefox plugin called Selenium IDE. This means while the user actions can be replayed by several browsers (IE, Opera, FF, mobile, etc.), with the IDE only the user actions from a Firefox user can be recorded.
In the generated HTML code of Selenium, these values are shown as HTML:
It is easy to see a possible problem here: how Selenium will find the actual HTML UI to test. In the above example the absolute path is used. When the portlet is not run as a single application but inside the portal the path will change. So make sure that the HTML code of the application is surrounded by a unique HTML that can serve as a starting point of the path. As the HTML isn’t really useful, you’ll have to export the actions to a framework of your choice (I created my own export template called SAPHtmlbTestNG).
For those who already know Selenium: the IDE does not come with an export to TestNG and WebDriver, so you are bound to use Selenium RC. After exporting the test case you get Java code that actually can be used to run the tests:
What does that actually mean? In the Selenium IDE I added a test that checks if a HTML element (e.g. a UI element like a button, checkbox or any other HTML element) is in the HTML code retrieved from the application:
For this Selenium identified the actual location of the element in question in the HTML source code:
As for Selenium and testing: the actual Java code for executing the tests is Selenium. TestNG only serves as the framework running the Selenium Java code. That’s why there are testing annotations like @BeforeClass. TestNG will control the flow of the tests (starting selenium, running the tests, stopping selenium).
Running the Selenium test with TestNG
To run the tests a few things have to be considered:
A Selenium server needs to be up and running
TestNG can be controlled over a XML file
Starting the Selenium server can be done via a (ant/maven) script or manually. What is more interesting is the XML file that controls TestNG. There the actual tests to be performed together with parameters are defined:
<?xml version=“1.0” encoding=“UTF-8”?>
<!DOCTYPE suite SYSTEM “http://testng.org/testng-1.0.dtd”>
The parameters are read by the annotation @Parameters and are used to define the selenium server. I added a listener that allows me to take a screenshot when a test fails and in the <test> section the actual tests are listened. There is only one test listened; to add more tests just add them by <class> definitions.
Run the test
After the tests are prepared and the tests and parameters are defined in the XML, the tests can run: either by invoking them directly in Eclipse or by running a script. TestNG will create HTML reports of the tests results.
Adding new test cases isn`t really hard when using the Selenium IDE. Where the IDE doesn`t help creating test cases, the developer can create them directly in Java. A benefit of Selenium is that you`ll end up having functional tests, as a real browser will make the calls to the application and validate the result.
From the 21 tests in my test suite, 2 failed.
Because of my screenshot class added to my test suite, TestNG made a screenshot when the test failed.
Selenium with a test framework like TestNG allows for functional testing of SAP web application. This is not bound to SAP Portal or WDJ apps, everything that has a HTML based UI can be tested. You have to keep in mind as UI testing is important, you should not focus solely on it. Unit and service tests should make the largest part in your test efforts.
Let the world know