Install SAP Web Dispatcher on Docker using SWMP

SAP Web Dispatcher is an important component in a SAP landscape. While have been treated as optional for many years and found mainly in SAP Portal scenarios, with the increase adoption of Fiori, having a reverse proxy in the landscape is becoming pre-requisite. While it’s possible to choose from a wide range of alternatives of servers for a reverse proxy, SAP`s Web Dispatcher is normally always the best fit in a SAP landscape. A question that sometimes arises is how to install Web Dispatcher.

First you settle on what version of Web Dispatcher (WD) to install. SAP Note 908097 states that you should go for the latest version. “Version 7.49 is the recommended SAP Web Dispatcher version for all backend systems.”

The actual installation gives you two options:

  • easy and
  • recommended.

The easy alternative is to simply un-sapcar the WD SAR file downloaded from Service Marketplace into a directory. To run WD, it`s then just to bootstrap it or run it with a given profile file. This installation method gives you a up and running WD in just one minute. The problem is that the files are all in one directory and not in the “official” directory structure of a normal SAP installation. But you get something like a portable WD installation: zip the directory and you can copy it to another server and can run WD from there.

The recommended alternative ensures that the WD is installed like a normal SAP product: all files follow the normal directory structure, etc. Installation is done using SWPM. Important when you are going to do some advanced configuration like PSE encryption, CryptoLib installation, etc. I`ll try to show how to install SAP Web Dispatcher the recommended way.

Download software

Download the needed software. It`s SWPM, Web Dispatcher, SAPCAR and HOSTAGENT.

SWPM

SAP Web Dispatcher 7.49 PL 112

SAPCAR 7.21

SAP Host Agent 7.21

Result

After you have downloaded all software, you have four files, summing up to almost 900 MB.

In a Unix environment, your WD system won`t have a graphical user interface and access to the system is given by SSH. This kind of environment can perfectly be emulated using Docker. Note: the SAR files need to be copied over to the target host.

Docker

There are several Linux images available for Docker. Let`s use Debian for this.

docker pull debian

For the actual image, see my Docker setup for Web Dispatcher: Dockerfile and docker-compose.yml: https://github.com/tobiashofmann/docker_webdispatcher_swpm

After running the Docker image, you have the files on the Linux system up and running, the Web Dispatcher and sapinst files available. Web Dispatcher is not yet installed. This is done by using sapinst. To run the installation, you`ll have to connect to sapinst using a different computer (most cases: your laptop). Let`s call the Docker container the target, and your computer the client.

Logon

I use Kitematic and to log on to my docker container, I just click on the EXEC button.

Logon to Docker container:

The log on from shell, the command is something like this:

bash -c "clear && docker exec -it dockerwebdispatcher_web_1 sh"

Target

To work properly, sapinst must be started as root. You then connect to it and log on. The logon is done by default with the user id running sapinst. Problem is that with the Docker images you do not know the root password. Same for environments where root access is only provided to a few or via sudo. You need to enable sapinst to run as root, but allow a different user (like <sid>adm) to log on. You achieve this by providing a parameter to sapinst informing the OS user allowed to log on remotely. The process is then:

  • Run sapinst as root (or sudo)
  • Connect to it informing a OS user (wddadm)

Sapinst Parameters

The needed parameter can be retrieved by letting sapinst show all available parameters. More information available in SAP Note 1745524 and at SAPinst central note.

./sapinst –p

Run sapinst in Docker

Provide the <sid>adm user as a property to sapinst.

./sapinst --properties SAPINST_REMOTE_ACCESS_USER=wddadm -nogui

Confirm that you know what you are doing.

Client

Start sapinstgui and connect to the target server on port 21212.

./sapinstgui

Inform the host name or IP address. In my case, it is 192.168.0.16. The port is the default sapinst port 21212.

Accept the fingerprint. You can check the fingerprint with the one printed by sapinst on the target server to be extra sure you are connecting to the right server.

Authenticate. You`ll need to provide the user id and password of the user running sapinst on the target host. In my case, the user is wddadm with password whatever. This is defined in the Dockerfile when the user is created.

sapinst output in Docker:

Logon on using wddadm / whatever

After a successful logon, sapinst will start. Current setup is not supported by SAP. For a production case this is a no-go, for my personal use case this is totally acceptable.

Sapinst shows the list of installable software options available. Web Dispatcher can be found at the end of the list.

Selecting SAP Web Dispatcher will start the installation.

Inform the path on the target server where the SAR files for SAP Web Dispatcher and SAP Host Agent can be found.

The files were copied into the container during the execution of the Dockerfile. All files are located at /home/wddadm.

If all packages are found, validated and added as considered valid for the installation.

Debian in Docker for sure won`t pass all the pre-requirements check build into SWMP. You`ll get a warning message, but SWMP won`t stop the installation. Select No. Seems that inside Docker, checking for the available free space is not working correctly.

Web Dispatcher configuration

Bootstrapping

Don’t worry, the system must not be accessible, yet exist. It’s just informing the bootstrap parameters. In my case, I am using a system that is not available, and it worked. Just be aware that in case the backend system changes, or isn’t even a ABAP system, like SMP3, you need to configure the Web Dispatcher profile manually.

Confirmation

Installation

The last step is to start Web Dispatcher. You can follow this on the console log of sapinst on the target server

If all worked, you get a confirmation message and the installation finishes.

SAPinst on the client host ends and so does it on the target host.

Validate installation

This gives you the time to validate the installation and check if all files are correctly installed.

Users

A new user sapadm was created

Folders

Web Dispatcher is installed under /sapmnt and instance is found in folder /usr/sap

This is perfectly aligned with the default locations of a SAP instance, and way better than simply putting all files into the same folder when unzipping the SAR. Especially when you consider that you may have to open a CSS ticket to SAP in your production environment or have new consultants arriving that expect the files to be located at the default location.

SAP Host Agent

The host agent was started and is running.

Start and stop Web Dispatcher

Starting and stopping Web Dispatcher via stopsap and startsap is working

stopsap

startsap

Admin web interface

The admin port of Web Dispatcher is listening by default on port 44300.

To access the service, the URL is

https://localhost:44300/sap/wdisp/admin/public/default.html

Log on to Web Dispatcher using the user webadm and password informed during installation.

Enable certificate based logon – 5 Configure SAP Web Dispatcher

For SAP Web Dispatcher be able to forward the received client certificate received by the browser, it must

  1. Re-encrypt the connection
  2. Add the client certificate as a header in the request

To ensure the connection is forwarded encrypted via TLS, use the parameter wdisp/ssl_enrypt=2. Value 2 means that WD will always forward using TLS, independently if the request received was HTTP or HTTPS. To inform WD which client certificate to use, configure parameter wdisp/ssl_auth. Value 2 means that the certificate specified by parameter wdisp/ssl_cred is used. In the below sample profile, ssl_cred points to the client PSE (SAPSSLC.pse). Icm/HTTPS/verify_client controls whether or not WD is asking or demanding for a client certificate. 1 means that WD will ask for one, but continue if not presented, while 2 means that a certificate must be presented.

With this information, WD can be used to connect securely the backend and forward the client certificate as a header, while using its own client certificate to authenticate against the backend NetWeaver system.

Extract of a sample WD profile

# unique instance identifier

SAPSYSTEMNAME = TOB

# unique instance number

SAPSYSTEM = 00

wdisp/ssl_encrypt=2

wdisp/ssl_auth=2

#

# Configuration for handling certificates

#

wdisp/ssl_cred=$(DIR_PROFILE)/sec/SAPSSLC.pse

icm/HTTPS/forward_ccert_as_header = true

icm/HTTPS/verify_client = 1

icm/HTTPS/client_certificate_header_name = SSL_CLIENT_CERT

icm/HTTPS/client_key_size_header_name = SSL_CIPHER_USEKEYSIZE

icm/HTTPS/client_cipher_suite_header_name = SSL_CIPHER_SUITE

icm/HTTPS/client_certificate_chain_header_prefix = SSL_CLIENT_CERT_CHAIN_1

#

# Backend System

#

wdisp/system_0 = SID=GWD, MSHOST=nwgw74.tobias.de, MSPORT=8101, SRCURL=/, SRCSRV=*:*, SSL_ENCRYPT=1

# SAP Web Dispatcher Ports

icm/server_port_1 = PROT=HTTPS,PORT=443

Enable certificate based logon – 2 Maintain Client PSE of Web Dispatcher

OK, now it will get complicated. Certificate based logons do not really like reverse proxies. First step is to ensure that the client has a certificate that is accepted by the SAP NetWeaver ABAP PSE. For this, the certificate must be signed by a CA that the ABAP PSE trusts.

Log on the WD admin and select PSE Management

Select Recreate PSE.

In the DN field, use the name of the WD: CN=wd.tobias.de. The other information is optional, but should be added.

Right now, the certificate is created and the PSE has a private key, but the certificate is self-signed. To have an official certificate, it must be signed by a CA. To do this, select: “Create a CA Request” to create a CSR that will be send to the CA.

Create CA Request

Save the output in a TXT file, send it to your CA, let it get signed. Then, go back to WD admin and import the CA response.

The client PSE updates the issuer information. Now there must be shown the DN of the CA. In my case: C=BR, O=EJBCA, CN=ca.tobias.de

SAP Web Dispatcher as reverse proxy for SMP3

As of SMP3 SP07 you can use SAP Web Dispatcher as a reverse proxy for SMP3. Depending on your landscape, this simplifies A LOT your architecture. And you can reuse your WD knowledge and gain support from SAP. Installing the WD is done as usual, with one caveat: you have to inform the commonlib which TLS to use:

ssl/ciphersuites = 896:HIGH

ssl/client_ciphersuites =896:HIGH

With this, WD can connect to SMP3 using TLS. While this may look strange, it actually is necessary as SMP3 uses some high TLS security.

To understand better what these two parameters do, take a look at the Commonlib + WD SAP Note: 510007


A complete sample profile from a WD running on Windows

SAPSYSTEMNAME = WDP

SAPSYSTEM = 00

DIR_INSTANCE = C:\<dir>\SAPWDSMP3

DIR_EXECUTABLE = $(DIR_INSTANCE)

DIR_PROFILE = $(DIR_INSTANCE)

DIR_HOME = $(DIR_INSTANCE)

Autostart = 1

Restart_Program_00 = local $(DIR_EXECUTABLE)/sapwebdisp$(FT_EXE) pf=$(DIR_PROFILE)/sapwebdisp.pfl

wdisp/ssl_auth=0

wdisp/system_0 = SID=SMP, SSL_ENCRYPT=0, EXTSRV=http://smp3.tobias.de:8080, SRCSRV=*:9080, SRCURL=/, STICKY=true

wdisp/system_1 = SID=SEC, SSL_ENCRYPT=1, EXTSRV=https://smp3.tobias.de:8081, SRCSRV=*:9081, SRCURL=/, STICKY=true

wdisp/system_1 = SID=SEC, SSL_ENCRYPT=1, EXTSRV=http://smp3.tobias.de:8082, SRCSRV=*:9082, SRCURL=/, STICKY=true

icm/server_port_0 = PROT=HTTP,PORT=9080

icm/server_port_1 = PROT=HTTPS,PORT=9081

icm/server_port_2 = PROT=HTTPS,PORT=9082,VCLIENT=2

ssl/ciphersuites = 896:HIGH

ssl/client_ciphersuites =896:HIGH

icm/max_conn = 2000

icm/max_sockets = ($(icm/max_conn) * 2)

icm/req_queue_len = 6000

icm/min_threads = 10

icm/max_threads = 500

mpi/total_size_MB = (min(0.06 * $(icm/max_conn) + 50, 2000))

mpi/max_pipes = ($(icm/max_conn))

wdisp/HTTP/max_pooled_con = ($(icm/max_conn))

wdisp/HTTPS/max_pooled_con = ($(icm/max_conn))

icm/server_port_3 = PROT=HTTPS,PORT=4300

icm/HTTP/admin_0 = PREFIX=/sap/wdisp/admin,PORT=4300,DOCROOT=./admin,AUTHFILE=icmauth.txt

Bind ICM to port 443

To run SAP Portal on the standard web ports 80 and 443 you should use Web Dispatcher. In that case, WD runs on the privileged ports and SAP Portal / NetWeaver Java / ICM continue to run on their usual 5nnXX ports. Changing the ports directly on ICM of NetWeaver is something I cannot recommend, and you should not do it.

Configuration of ICM

To run NetWeaver on low ports, follow the procedure outlined in SAP Note 421359. ICMBND is the executable that will run at port 443. This file does not exists. To create it, follow the steps outlined in the SAP Note as user root:

  • cd /usr/sap/<SID>/J00/exe
  • cp icmbnd.new icmbnd
  • chown root:sapsys icmbnd
  • chmod 4750 icmbnd
  • ls –al icmbnd

The super user bit is now set. With this, the executable can now “act” as being root and listen on port 443. The instance profile must now be changed to include the new ICM parameters to bind to port 443 for HTTPS and to use the external program icmbnd for doing that.

Currently the port configuration may look like this:

After the change

Note: The parameter exe/icmbnd should not be needed as long as the binary resides in the normal place. I added it here to show how the parameter looks when configured.

Restart SAP system: stopsap; startsap.

Result

NetWeaver is now listening on port 443.

 

Default configuration is that NetWeaver first asks the client to provide a certificate and if none is given, proceeds with the normal authentication defined in logon profile.

This can be disabled be setting the parameter VCLIENT=0 in the instance profile:

icm/server_port_4=HTTPS,PORT=443,SSLCONFIG=ssl_config_4,EXTBIND=1,VCLIENT=0

Links

SAP WebDispatcher and Logstash – installation and configuration

This document explains how to install and configure an environment for analyzing SAP Web Dispatcher (WD) logs with logstash, elasticsearch and Kibana under Linux. Kibana 3 needs a running web server. The example shown here is using nginx, but won’t detail how to set up nginx.

Components referred to in this document:

SAP WebDispatcher

“The SAP Web dispatcher lies between the Internet and your SAP system. It is the entry point for HTTP(s) requests into your system, which consists of one or more SAP NetWeaver application servers.” http://help.sap.com/saphelp_nw73ehp1/helpdata/en/48/8fe37933114e6fe10000000a421937/frameset.htm

Logstash

“logstash is a tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use (like, for searching). Speaking of searching, logstash comes with a web interface for searching and drilling into all of your logs.” http://logstash.net/

Elasticsearch

“Elasticsearch is a powerful open source search and analytics engine that makes data easy to explore.” http://www.elasticsearch.org/

Kibana

“Kibana is an open source, browser based analytics and search dashboard for ElasticSearch.” http://www.elasticsearch.org/overview/kibana/

Nginx

“nginx (pronounced engine-x) is a free, open-source, high-performance HTTP server and reverse proxy, as well as an IMAP/POP3 proxy server.” http://wiki.nginx.org/Main

 

Install Elasticsearch

Installation in 3 steps: http://www.elasticsearch.org/overview/elkdownloads/

  1. Command: wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.4.2.tar.gz

  2. Extract archive

    Command: tar –zxvf elasticsearch-1.4.2.tar.gz

  3. Start Elasticsearch

    Command

    cd ealsticsearch-1.4.2

    cd bin

    ./elasticsearch

Install Logstash

Installation in 3 steps: http://www.elasticsearch.org/overview/elkdownloads/

  1. Command:

    wget https://download.elasticsearch.org/logstash/logstash/logstash-contrib-1.4.2.tar.gz

  2. Extract

    Command: tar –zxvf logstash-contrib-1.4.2.tar.gz

  3. Run logstsash logstash. Before logstash can be run, it must be configured. Configuration is done in a config file.

Logstash configuration

The configuration of logstash depends on the log configuration of WD. Logstash comes out of the box with everything it takes to read Apache logs. In case WD is configured to write logs in Apache format, no additional configuration is needed. WD also offers the option to write additional information to the log.

Logformats http://help.sap.com/saphelp_nw73ehp1/helpdata/en/48/442541e0804bb8e10000000a42189b/content.htm?frameset=/en/48/8fe37933114e6fe10000000a421937/frameset.htm&current_toc=/en/ed/2429371ec14c23a7508affa1280d07/plain.htm&node_id=46&show_children=false

  • CLF. This is how Apache is logging. It contains most information needed.
  • CLFMOD. Same format as CLF, but without form fields and parameters for security reason.
  • SAP: writes basic information and no client IP, but contains processing time on SAP Application Server. This is a field you really will need.
  • SMD: For SolMan Diagnostics and same as SAP, but contains the correlation ID.

As mentioned before, for CLF logstash comes with everything already configured. A log level that makes sense is SMD because of the response time. In that case, logstash must be configured to parse correctly the WD log. Logstash uses regular expressions to extract information. To make logstash understand SMD log format, the correct regular expression must be made available. Grok uses the pattern file to extract the information from the log http://logstash.net/docs/1.4.2/filters/grok The standard pattern file can be found here: https://github.com/elasticsearch/logstash/tree/v1.4.2/patterns

For instance, to extract the value of the correlation id when log format is set to SMD, the regular is:

CORRELATIONID [a-zA-Z]\[\-\]

For WD with SMD log the complete regular expression is

TEST2 \|

WEBDISPATCHER \[%{HTTPDATE:timestamp}\] %{USER:ident} “(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})” %{NUMBER:response} (?:%{NUMBER:bytes}|-) \[%{NUMBER:duration}\] %{CORRELATIONID:correlationid} %{TEST2:num1}

 

When the IP is added to the WD log with SMD, the regular expression is

TEST2 \|

CORRELATIONID [a-zA-Z]\[\-\]

WEBDISPATCHERTPP %{IP:ip} \[%{HTTPDATE:timestamp}\] %{USER:ident} “(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})” %{NUMBER:response} (?:%{NUMBER:bytes}|-)\[%{NUMBER:duration}\] %{CORRELATIONID:correlationid} %{TEST2:num1}

 

You can find an example pattern file here: https://github.com/tobiashofmann/wd_logstash. The standard grok pattern file defines regular expressions for user id, IPv4/6, data, etc.

The actual configuration file consists of three sections: input, filter and output. The input part defines the logs to read, the filter part defines the filter to be applied to the input and the output part specifies where to write the result to. Let’s take a look at each of the sections:

Input

input {

file {

type => “wd”

path => [“/usr/sap/webdispatcher/access*”]

start_position => “beginning”

codec => plain {

charset => “ISO-8859-1”

}

}

}

All files starting with access at directory /usr/sap/webdispatcher are being read by logstash. The codec parameter ensures URLs with special characters are read correctly. To all lines read a type named wd is added.

Filter

filter {

if [type] == “wd” {

grok {

patterns_dir => “./patterns”

match => { “message” => “%{WEBDISPATCHER}” }

}

date {

match => [“timestamp”, “dd/MMM/yyyy:HH:mm:ss Z” ]

}

mutate {

convert => [ “bytes”, “integer” ]

convert => [ “duration”, “integer” ]

}

}

}

The filter is applied to all lines with type wd (see input). Grok is doing the regular expressions and to find the customized patterns for WD, the patterns_dir parameter is used. The date value is given by the timestamp. If this is not set, logstash takes the timestamp when the line is read. What you want is the timestamp of the logged access time of the HTTP request. To facilitate later analysis, the values bytes and duration are transformed to integer values.

Output

output {

elasticsearch {

host => localhost

index => “wd”

index_type => “logs”

protocol => “http”

}

}

As output a local elasticsearch server is defined. The logs are written to the index wd to index type logs. This stores the log lines as a value to elasticsearch and makes it accessible for further processing.

A sample configuration file can be found here https://github.com/tobiashofmann/wd_logstash

 

Run logstash

To run logstash and let it read the WD logs, use the following command:

./logstash –f logstash.conf

This will start logstash. It takes a few seconds for the JVM to come up and read the first log file. Afterwards the log files are parsed and send over to elastic search.

Kibana

Installation in 3 steps: http://www.elasticsearch.org/overview/elkdownloads/

  1. Go to the HTML directory configured for NGinx, like /var/www/html

    Command: cd /var/www/html

  2. Command: wget https://download.elasticsearch.org/kibana/kibana/kibana-3.1.2.tar.gz

  3. Extract archive

    Command: tar –zxvf kibana-3.1.2.tar.gz

  4. Configure nginx

    Add a location in nginx configuration file to make the kibana application available under /kibana

    Location /kibana {

    alias /var/www/html/<dir of kibana>

    }

  5. Access Kibana on web browser: http://webserver:port/kibana