Install SAP Web Dispatcher on Docker using SWMP

Posted on May 25, 2017 by Tobias Hofmann

Let the world know ...

SAP Web Dispatcher is an important component in a SAP landscape. While have been treated as optional for many years and found mainly in SAP Portal scenarios, with the increase adoption of Fiori, having a reverse proxy in the landscape is becoming pre-requisite. While it’s possible to choose from a wide range of alternatives of servers for a reverse proxy, SAP`s Web Dispatcher is normally always the best fit in a SAP landscape. A question that sometimes arises is how to install Web Dispatcher.

First you settle on what version of Web Dispatcher (WD) to install. SAP Note 908097 states that you should go for the latest version. “Version 7.49 is the recommended SAP Web Dispatcher version for all backend systems.”

The actual installation gives you two options:

easy and
recommended.

The easy alternative is to simply un-sapcar the WD SAR file downloaded from Service Marketplace into a directory. To run WD, it`s then just to bootstrap it or run it with a given profile file. This installation method gives you a up and running WD in just one minute. The problem is that the files are all in one directory and not in the “official” directory structure of a normal SAP installation. But you get something like a portable WD installation: zip the directory and you can copy it to another server and can run WD from there.

The recommended alternative ensures that the WD is installed like a normal SAP product: all files follow the normal directory structure, etc. Installation is done using SWPM. Important when you are going to do some advanced configuration like PSE encryption, CryptoLib installation, etc. I`ll try to show how to install SAP Web Dispatcher the recommended way.

Download software

Download the needed software. It`s SWPM, Web Dispatcher, SAPCAR and HOSTAGENT.

SWPM

SAP Web Dispatcher 7.49 PL 112

SAPCAR 7.21

SAP Host Agent 7.21

Result

After you have downloaded all software, you have four files, summing up to almost 900 MB.

In a Unix environment, your WD system won`t have a graphical user interface and access to the system is given by SSH. This kind of environment can perfectly be emulated using Docker. Note: the SAR files need to be copied over to the target host.

Docker

There are several Linux images available for Docker. Let`s use Debian for this.

docker pull debian

For the actual image, see my Docker setup for Web Dispatcher: Dockerfile and docker-compose.yml: https://github.com/tobiashofmann/docker_webdispatcher_swpm

After running the Docker image, you have the files on the Linux system up and running, the Web Dispatcher and sapinst files available. Web Dispatcher is not yet installed. This is done by using sapinst. To run the installation, you`ll have to connect to sapinst using a different computer (most cases: your laptop). Let`s call the Docker container the target, and your computer the client.

Logon

I use Kitematic and to log on to my docker container, I just click on the EXEC button.

Logon to Docker container:

The log on from shell, the command is something like this:

bash -c "clear && docker exec -it dockerwebdispatcher_web_1 sh"

Target

To work properly, sapinst must be started as root. You then connect to it and log on. The logon is done by default with the user id running sapinst. Problem is that with the Docker images you do not know the root password. Same for environments where root access is only provided to a few or via sudo. You need to enable sapinst to run as root, but allow a different user (like <sid>adm) to log on. You achieve this by providing a parameter to sapinst informing the OS user allowed to log on remotely. The process is then:

Run sapinst as root (or sudo)
Connect to it informing a OS user (wddadm)

Sapinst Parameters

The needed parameter can be retrieved by letting sapinst show all available parameters. More information available in SAP Note 1745524 and at SAPinst central note.

./sapinst –p

Run sapinst in Docker

Provide the <sid>adm user as a property to sapinst.

./sapinst --properties SAPINST_REMOTE_ACCESS_USER=wddadm -nogui

Confirm that you know what you are doing.

Client

Start sapinstgui and connect to the target server on port 21212.

./sapinstgui

Inform the host name or IP address. In my case, it is 192.168.0.16. The port is the default sapinst port 21212.

Accept the fingerprint. You can check the fingerprint with the one printed by sapinst on the target server to be extra sure you are connecting to the right server.

Authenticate. You`ll need to provide the user id and password of the user running sapinst on the target host. In my case, the user is wddadm with password whatever. This is defined in the Dockerfile when the user is created.

sapinst output in Docker:

Logon on using wddadm / whatever

After a successful logon, sapinst will start. Current setup is not supported by SAP. For a production case this is a no-go, for my personal use case this is totally acceptable.

Sapinst shows the list of installable software options available. Web Dispatcher can be found at the end of the list.

Selecting SAP Web Dispatcher will start the installation.

Inform the path on the target server where the SAR files for SAP Web Dispatcher and SAP Host Agent can be found.

The files were copied into the container during the execution of the Dockerfile. All files are located at /home/wddadm.

If all packages are found, validated and added as considered valid for the installation.

Debian in Docker for sure won`t pass all the pre-requirements check build into SWMP. You`ll get a warning message, but SWMP won`t stop the installation. Select No. Seems that inside Docker, checking for the available free space is not working correctly.

Web Dispatcher configuration

Bootstrapping

Don’t worry, the system must not be accessible, yet exist. It’s just informing the bootstrap parameters. In my case, I am using a system that is not available, and it worked. Just be aware that in case the backend system changes, or isn’t even a ABAP system, like SMP3, you need to configure the Web Dispatcher profile manually.

Confirmation

Installation

The last step is to start Web Dispatcher. You can follow this on the console log of sapinst on the target server

If all worked, you get a confirmation message and the installation finishes.

SAPinst on the client host ends and so does it on the target host.

Validate installation

This gives you the time to validate the installation and check if all files are correctly installed.

Users

A new user sapadm was created

Folders

Web Dispatcher is installed under /sapmnt and instance is found in folder /usr/sap

This is perfectly aligned with the default locations of a SAP instance, and way better than simply putting all files into the same folder when unzipping the SAR. Especially when you consider that you may have to open a CSS ticket to SAP in your production environment or have new consultants arriving that expect the files to be located at the default location.

SAP Host Agent

The host agent was started and is running.

Start and stop Web Dispatcher

Starting and stopping Web Dispatcher via stopsap and startsap is working

stopsap

startsap

Admin web interface

The admin port of Web Dispatcher is listening by default on port 44300.

To access the service, the URL is

https://localhost:44300/sap/wdisp/admin/public/default.html

Log on to Web Dispatcher using the user webadm and password informed during installation.

Let the world know ...

Parallel download of files using curl

Posted on May 22, 2017 by Tobias Hofmann

Let the world know ...

In a previous blog, I showed how to download files using wget. The interesting part of this blog was to pass the authentication cookies to the server as well as using the file name given by the Content-Disposition directive when saving the file. The example of the previous blog was to download a single file. What if you want to download several files from a server? Maybe hundreds or even thousands of files? wget is not able to read the location from a file and download these in parallel, neither is curl capable of doing so. You can start the download as a sequence, letting wget/curl download the files one by one, as shown in my other blog. Just use a FOR loop until you reach the end.

Commands

For downloading a large amount of files in parallel, you`ll have to start the download command several times in parallel. To achieve this, several programs in bash must be combined.

Create the list of files to download. This is the same as shown in my previous blog.

for i in 1 {1..100}; do `printf "echo https://server.fqdn/path/to/files/%0*d/E" 7 $i` >> urls.txt; done

Start the parallel download of files. Start 10 threads of curl in background. This is an enhanced version of the curl download command of my previous blog. Xargs is used to run several instances of curl.

nohup cat urls.txt | xargs -P 10 -n 1 curl -O -J -H "$(cat headers.txt)" >nohup.out 2>&1 &

Explanation

The first command is creating a list of files to download and stores them in the file urls.txt.

The second command is more complex. First, cat is printing the content of urls.txt to standard-out. Then, xargs is reading from standard-in and uses it as input for the curl command. For authentication and other headers, the content of the file headers.txt is used. The input for curl for the first line is then:

curl -O -J -H "$(cat headers.txt)" https://server.fqdn/path/to/files/0000001/E

The parameter –P 10 informs xargs to run the command 10 times in parallel. It takes the first 10 lines of input and starts for each input a new curl process. Therefore, 10 processes of curl are running in parallel. To run more downloads in parallel, give a higher value for –P, like 20, or 40.

To run the download in background, nohup is used. All output is redirected to nohup.out: >nohup.out 2>&1

SSH

To have the download running while being logged on via SSH, the tool screen should be used. After logon via ssh, call screen, run the above command, and hit CTRL + A + D to exit screen.

ssh user@server.fqdn
screen
nohup cat urls.txt | xargs -P 10 -n 1 curl -O -J -H "$(cat headers.txt)" >nohup.out 2>&1 &
CTRL+A+D
exit

Let the world know ...

Download files with leading zero in name using wget

Posted on May 4, 2017 by Tobias Hofmann

Let the world know ...

In my previous blog I showed how wget can be used to download a file from a server using HTTP headers for authentication and how to use Content-Disposition directive send by the server to determine the correct file name. With the information of the blog it`s possible to download a single file from a server. But what if you must download several files? Maybe hundreds or thousands of files? Files whose file name is created using a mask, adding leading zeros?

Add leading zeros

What you need is a list of files to download. I`ll follow my example from the previous post and my files follow a specific patter: number. All files are numbered from 1 to n. To make it more special / complicated, it`s not only 1 to n. A mask is applied: 7 digits in total, with leading 0. 123 is 0000123, or 5301 is 0005301. In recent versions of Bash, you can use a FOR loop to loop through the numbers and printf for formatting the output and add the leading zeros. To get the numbers correctly formatted, the command is:

for i in 140000 {140001..140005}; 
  do echo `printf "%0*d" 7 $i`; 
done

This prints (echo) the numbers 140000 to 140005 with leading zero.

Start download

Adding the wget command in the printf directive allows to download the files. The execution flow is to let the FOR loop together with printf create the right number with mask, and wget downloads the file. After the file is download, the next iteration of the FOR loop starts, and the next file is downloaded. Assuming that I have PDF documents named 0140000.pdf to 0140005.pdf on server http://localhost:9080, the FOR loop with wget is:

for i in 140000 {140001..140005}; 
  do `printf "wget -nc --content-disposition http://localhost:9080/%0*d.pdf\n" 7 $i`; 
done

Result

Alternative

The above example is using wget. Of course, you can do the same using curl.

Let the world know ...

SAPInst error java.net.InetAddress.getHostAddress

Posted on May 2, 2017 by Tobias Hofmann

Let the world know ...

SAPinst is used to install SAP software. You can run the installation with its graphical user interface (GUI) directly on the target host, or use a remote installation. Remote installation is normally used when the target host offers no GUI and all work is done via remote logon, like SSH. In such a setup, on the target host sapinst is started in server mode, and on the admin laptop, sapinst is started in with a GUI. The admin then connects to the target and executes the installation of SAP software remotely.

Error: java.net.InetAddress.getHostAddress

The setup to reproduce the error is to have sapinst running on a target host and listening on the default port 21212 and connecting to this instance using sapinstgui.

Setup target host

Start sapinst on the target host:

./sapinst -nogui

Setup client host

Start sapinst in GUI mode: ./sapinstgui Inform the target host (computername or IP)

Accept the fingerprint.

Error

Sapinstgui hangs and is not advancing to the next screen. The status bar shows that it`s connected:

On the target server, sapinst gives this error message:

java.lang.NullPointerException: while trying to invoke the method java.net.InetAddress.getHostAddress() of a null object loaded from field com.sap.sdt.util.misc.Uid.address of an object loaded from local variable 'this'

Solution

To solve this error, a configuration on the client computer is needed. While not obvious, the error is caused by a misconfiguration of the client DNS resolution. Easiest solution is to manually change the hosts file and to ensure that the client computer name is listed. To find out the computers host name, run hostname.

hostname

In my case, the host name is linux-01sn.

Example: wrong hosts file

The host name is not listed in the hosts file. To correct this, insert the hostname for instance to localhost.

Example: working hosts file

Test

To see if sapinstgui can now connect to the target server without error, restart sapinstgui. After connecting to sapinst server, you`ll get the authentication screen.

sapinst output on target host:

Let the world know ...