xcrun: error: invalid active developer path

Mac is a nice computer for developing, MacOS and Apple can make your developer life a challange. After updating XCode – after all, why have a Mac when you do not develop iOS apps – it may happen that git stops working.

Running git gives you:

xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun

Usual situation: it worked yesterday, today it is broken and you did nothing. Besides updating XCode. The problem occurs easily. When you update XCode, normally you also update the command line tools.

In case the Apple App Store isn’t giving you the option to update the command line tools, run the command

xcode-select –install

Output

xcode-select: note: install requested for command line developer tools

This should either install the command line tools and give you back a working git tool, or let you install the tools manually via the App Store.

After this, git should be working again. Happy coding.

PowerLAN in real life

I bought a PowerLan adaptor to be able to connect to my private “server” (aka Raspberry PI) from the other side of the house via WLAN. Connecting the RP via WiFi wasn’t an option: two far away from the signal, I’d need to install at least one additional repeater and then two RP are without wireless adaptor. The PowerLan adaptor supports up to 1.200 MB/s and let’s me connect 3 devices directly. I won’t even need to buy an additional switch. And 1.200 MB/s. In theory, this is FAST. In reality the setup is complicated. The two adaptors are in the opposite location of a not-so-small house, the electrical stuff and cables were installed some 40 years ago. Some things you can only test after investing, and when thinking about how to connect to my servers, PowerLan was the only viable choice.

The setup was easy: one PoweLan adaptor connects to the Internet router, the other connects the RPs. Both find them and establish a secure connection. The connection signal is green, indicating a good connection. Connection tests showed that it is working: ping, ssh, all fine. It was time to check the speed. The RP isn’t the fastes when it comes to file server duties, but back in Rio I had 11 MB/s for copying files in Windows. That was fast enough. How will it be now? Laptop connected to WiFi (that is already not so good, given the layout of the rooms) and PowerLan that needs to communicate through several meters. It’s a worst case setup. I do not expect much.

A way to measure the speed is using netcat, or nc. Available in Linux and Mac. Basically the idea is to send data from the laptop to the RP and measure how long it does take to send 1MB, 10MB or more. The RP will serve as the receiving part, the laptop will send the data.

RP / Linux

Start nc, let it listen on port 2222 and send all incoming data to /dev/null

nc -v -l 2222 > /dev/null

Laptop

Read data from /dev/null, pipe it to nc and let it send to the nc instance running on the RP

dd if=/dev/zero bs=1024k count=512 | nc -v 192.168.2.112 2222

bs tells dd how much to read (1MB here) and count how many times. In the above example 512 MB will be send. To send 1 MB only, use count=1, for 10MB use count=10.

Results

1 MB

dd if=/dev/zero bs=1024k count=1 | nc -v 192.168.2.112 2222
found 0 associations
found 1 connections:
     1: flags=82<CONNECTED,PREFERRED>
 outif en0
 src 192.168.2.108 port 55619
 dst 192.168.2.112 port 2222
 rank info not available
 TCP aux info available
Connection to 192.168.2.112 port 2222 [tcp/rockwell-csp2] succeeded!
1+0 records in
1+0 records out
1048576 bytes transferred in 1.423923 secs (736399 bytes/sec)

Result: 736399 bytes/sec => 0.7 MB/sec.

10 MB

dd if=/dev/zero bs=1024k count=10 | nc -v 192.168.2.112 2222
found 0 associations
found 1 connections:
     1: flags=82<CONNECTED,PREFERRED>
 outif en0
 src 192.168.2.108 port 55624
 dst 192.168.2.112 port 2222
 rank info not available TCP aux info available
Connection to 192.168.2.112 port 2222 [tcp/rockwell-csp2] succeeded!
10+0 records in
10+0 records out
10485760 bytes transferred in 9.997975 secs (1048788 bytes/sec)

 Result: 1048788 bytes/sec => 1 MB/sec.

100 MB

dd if=/dev/zero bs=1024k count=100 | nc -v 192.168.2.112 2222
found 0 associations
found 1 connections:
     1: flags=82<CONNECTED,PREFERRED>
 outif en0
 src 192.168.2.108 port 55781
 dst 192.168.2.112 port 2222
 rank info not available
 TCP aux info available
Connection to 192.168.2.112 port 2222 [tcp/rockwell-csp2] succeeded!
100+0 records in
100+0 records out
104857600 bytes transferred in 103.159005 secs (1016466 bytes/sec)

Result: 1016466 bytes/sec => 1 MB/sec

 

Overall, the result is slower than the 11 MB/sec what I had before in Rio, but both connected  to the same WLAN. Now, with and unfortunate setup, PowerLAN gives me 1MB, and a working connection. It’s fast enough for storing documents. For copying a 100GB VM image it’s too slow.

UUIDD problem when running SAP NW ABAP inside Docker

UUID is a good old problem when it comes to running SAP NetWeaver on Linux / SuSE. You have a problem when you log in to your SAP system and get an error message. The error message shows the root cause and solution: “The UUID daemon (uuidd) is not active (code 59999). Check SAP note 1391070.”

Solution

SAP NetWeaver isn’t meant to run in Docker. When the software was designed, Docker or event containerization wasn’t around (maybe SUN). NetWeaver assumes that it is executed inside a real Linux. And the Docker version of OpenSuSE isn’t 100% a real Linux. A lot of services you get “automatically” when installing OpenSuSE are not available. One of those is that the init.d system is not starting services. Because of this, there is no UUID daemon running.

Make sure that the UUIDD service is running. For a normal Linux distribution, I blogged about this at a previous blog of mine. In case you are using Docker with OpenSuSE, make sure that uuidd is installed and executed during the image creation:

Installation

RUN zypper --non-interactive install --replacefiles  uuidd

Execution

RUN mkdir /run/uuidd && chown uuidd /var/run/uuidd && /usr/sbin/uuidd

Result

With the UUIDD running, the logon to SAP NetWEeaver ABAP is working. No restart of NetWeaver is needed.

Connect to NetWeaver ABAP instance running inside Docker

This blog will help you to connect to your SAP NetWeaver ABAP instance running inside a Docker container. For how to get NetWeaver running inside a Docker container, please see my blog Docker for SAP NetWeaver ABAP 7.5x Developer Edition.

SAPGui

Open SAPGui and create a new connection.

Give a name for the connection and click on tab Advanced. I use NPL Docker. Activate expert mode and give the correct connection String. Check to which port the message server port is mapped to by Docker. Inside the container, the port is 3200, and in my case, the external port is 32771. Therefore, the connection String is:

Connection String: conn=/H/localhost/S/32771

Note: the port information is specified when you start the container. As an alternative, you can use Kitematic to see the port mapping.

Save and connect to NetWeaver.

The users and passwords can be found in the readme.html of the extracted SAP NW ABAP 751 download. Standard users are SAP* and Developer.

HTTP Access

You can test if access to your new SAP system is working via HTTP by calling the ping service: http://localhost:32769/sap/public/ping

For this to work, first activate the ping service in SICF.

When you get the response “Server reached.” you can start using the HTTP access.

SAP WebGui

For general WebGui activation, you can see my previous blog “Activation of SAP WebGui”. Here is a short version of this guide. As in the previous HTTP service access, the same procedure must be followed to have access to NPL via WebGui.

Activate the service webgui

To activate the SAP WebGui service, activate the node:

/sap/bc/gui/sap/its/webgui

Activation of public resources

You also need to activate the public service that contains the HTML files (JS, etc):

/sap/public/bc/its

Note

It is not sufficient to only activate the webgui node. The app is using additional resources that are available under /sap/public/bc/its. If this node is not activated, you’ll get an error message when logging in to webgui.

Therefore, for SAP WebGui to load the node /sap/public/bc/its must be activated too.

Activate the node its and its subnodes. Select Activate Service.

Activate with all sub nodes nodes (second Yes).

Result

After activating these two nodes, access to WebGui should work. To test this, call the URL http://localhost:32769/sap/bc/gui/sap/its/webgui After logging in, you should see the SAP Menu.

Dockerfile for SAP NetWeaver ABAP 7.5x Developer Edition

This blog will help you to run your own SAP NetWeaver 7.51 ABAP instance inside a Docker container. This work was inspired by the Dockerfile created by Gregor Wolf and hosted at bitbucket.

The difference is that in Gregor’s version you download the NW ABAP installation files and when the container is build, you go manually through the installation. My Dockerfile assumes that you have downloaded the NW 7.51 ABAP installation files already and will automate the installation. Once you have downloaded the installation files from SAP you can make them locally available and create new Docker images / containers based on these, without having to download almost 16 GB again. And the installation script will run without prompting for user input.

Another differentiation is that you can “easily” change the Dockerfile to install NetWeaver 7.50 of the developer edition.

Pre-requisites

To be able to run the Dockerfile, you need

  • Docker installed
  • Downloaded and extracted installation files of SAP NW ABAP Developer Edition
  • Internet connection

Installation

1 Get the Dockerfile

From my GitHub repository, you can find a Dockerfile that helps you to create a Docker image and container that will install your downloaded NetWeaver version. All you need is the Dockerfile, so a simple download is sufficient. You can also download the file by cloning the GitHub repository: https://github.com/tobiashofmann/sap-nw-abap-docker

2. Download SAP NetWeaver DE installation files

Download your version of SAP NetWeaver ABAP 7.5x Developer Edition from SAP. The files are compressed (RAR).

  1. Un-compress them into a folder named NW751. The folder must be at the same location where your Dockerfile is.
  2. Build the Docker image

Build the Docker image

Command:

docker build -t nwabap .

Sample output

After the build is finished, the last line you should see is

Successfully tagged nwabap:latest

To see the ID and name of the newly created image, run the following command:

docker images

Sample output

The command lists the ID, tag and size of the image. As you can see, it’s a 15 GB Docker image. Using this image, you can start a container and install NW ABAP 7.51 DE inside the container.

Create container from image

You can now create a container from the image. You’ll have to connect to the container and run the installation script run.sh. The file was created during docker build. It will run SAP’s install.sh and fill in the input automatically.

docker run -P -h vhcalnplci --name nwabap751 -it nwabap:latest /bin/bash

This will start the container and log you in. What you’ll get is the bash shell.

bash-4.3#

In case you have Kitematic installed, you can see the running container listed.

The container configuration for the ports is also visible there. The ports are automatically mapped by Docker. The message server port 3200 is accessible through localhost:32771, and the HTTP port 8000 through localhost:32769. This mapping can be changed either inside Kitematic or when the container is started on the command shell.

Run ls to see the content of the current directory. You can see the install.sh file from SAP (feel free to start the installation manually) and the run.sh script that will automate the installation.

Start installation

Run the script run.sh to install SAP NetWeaver ABAP 7.51. The script will enter all information requested by install.sh automatically. The installation will take some time, +/- 20 minutes.

./run.sh

Sample output

[…]

[…]

The installation worked when the script ends and you can see the output:

Installation of NPL successful

 

 

Enable Wake on LAN on Windows 10

To be able to wake up your computer via wake-up-on-lan (WOL), you need to enable this feature in the BIOS and in the Windows 10 LAN adaptor settings.

Configuration: BIOS

Configuration depends on the BIOS of your computer. In my case, wake up on LAN is in the power on section and disabled by default. To use this feature, just enable it.

Do not forget to save the change.

Configuration: Windows 10

After activating WOL in the BIOS, you need to configure Windows 10 to allow the device to wake Windows. My test computer is a Lenovo Q180 running Windows 10 German. More information on how to activate WOL for this device can be found here.

Go to Network and adaptor properties. Select the LAN adaptor and open its properties. In the property screen, select Configuration.

Go the Entergy settings. Check all check boxes.

Now go to the next tab: Advanced. Ensure that Wake on Magic Packet is set to Enabled.

Windows Firewall configuration

To know that a computer is running, you can use ping. If the computer responses to a ping, it’s up and running. To allow ping requests through the standard Windows firewall, ensure that the rule for file and print service is activated for your network.

There are two network types: private and public. Activating ping for the private network should be sufficient. If you are unsure if your LAN is part of private or public, you can activate ping requests for both. At least, when your network is still secured by a router with firewall.

Block access from country by IP in Apache

In this blog I will show how you can block access to your Apache hosted internet services, forbidding access to a whole country. The access is blocked based on the IP address of a client. In case of a VPN where the user connects to a VPN server in another country, the user will still be able to access the site.

The internet is a great to ensure freedom of speech. Anyone can raise his/her voice; use the information to be informed on what is happening in the world, let others know about something, share knowledge. You can do so by using a social site or by hosting your own site. The ease of access to information; be able to search it instantly; have huge amount of information able to be discovered by a large number of the world population. This is one of the true great contributions to really make the world a better place. Some countries don’t like this, applying censorship, access restriction, or worse. And basically, if you decide to block a country to access your site, it’s one step to the wrong direction.

Why would you block a whole country? Isn’t a great thing about the internet that it’s accessible from anywhere in the world, just using a browser? It’s not as simple. A few reasons to block a country can be:

  • Legal requirements. Your site is not in compliance with the countries law. For instance, maybe you are logging too much personal information?
  • The functionality is not meant for that country. You have a commercial service, and are not offering a payment option or a localized version.
  • You are popular in a country and flooded with a lot of requests, but these are just operational overhead for you as your site is not targeted for these users.
  • If you think hard enough, you can come up with a good reason.

After finding yourself in the situation to block a specific country, the question is: HOW? You can use a blocker in your web platform (WordPress plugin), or use Apache to do so. Using a .htpasswd file for this is not optimal due to performance. Better is to use a module. A quick Google search reveals that a good option is to use the GeoLite DB from MaxMind. And they also offer an Apache 2.4 module. The module works with Apache 2 and the HTTPD server available on Amazon AMI images.

Some references to projects used to set the country blocking up.

Steps

Steps for using GeoLite2 DB for blocking countries in Apache

  1. Download GeoLite 2 DB
  2. Install dependencies
  3. Install Apache module
  4. Configuration
  5. Activation

1. Download GeoLite2 database

The GeoLite2 DB is available as a free and commercial license. The free version should be good enough for a private blog. You can get the free version from MaxMind site.

Select GeoLite 2 Country and binary format. Download the file using wget.

wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-Country.tar.gz

Unzip the file.

tar zxvf GeoLite2-Country.tar.gz

The actual DB file is close to 3 MB in size.

Copy it to a directory were the apache users can find it. A good default location is /usr/local/share in a new directory named GeoIP.

sudo mkdir /usr/local/share/GeoIP
sudo cp /home/ec2-user/geolite2db/GeoLite2-Country_20170704/GeoLite2-Country.mmdb /usr/local/share/GeoIP/

2. Install dependencies

Install libmaxmind

For the Apache module to work, the C library libmaxmind must be installed. This can be done by using yum.

sudo yum install libmaxminddb.x86_64 libmaxminddb-devel.x86_64

HTTPD devel files

Another dependency is the HTTP development files. These can also easily installed using yum.

sudo yum install httpd24-devel.x86_64

3. Install Apache module

The Apache module is available as source code from GitHub. For installation, download the latest release from GitHub. In my case, the latest release was version 1.1.0. Download the tar file.

Download the release to Linux using wgetand unzip it.

wget https://github.com/maxmind/mod_maxminddb/releases/download/1.1.0/mod_maxminddb-1.1.0.tar.gz
tar zxvf mod_maxminddb-1.1.0.tar.gz

Now you can compile and install the module. To do so, run

./configure
sudo make install

This should compile and put the files correctly into the right directory of HTTPD. If an error occurs during configuration, compilation or installation, look at the error message and good luck.

The directive to load the new module was automatically added to the file /etc/httpd/conf/httpd.conf

To test that the module can be loaded, restart HTTPD.

sudo service httpd restart

The service needs to start without error. This indicates that the module was successfully loaded. To validate this, check if the new module is actually loaded by HTTPD. To do so, list all loaded modules.

sudo httpd –M

Search for the maxmind module:

maxminddb_module (shared)

The new module is correctly loaded by HTTPD. Now we can configure Apache to make use of the module.

4. Configuration

Edit the HTTP config file and add the directive to block a specific country. The GitHub site of MaxMind contains an example that serves as a very good starting point.

MaxMindDBEnable On
MaxMindDBFile DB /usr/local/share/GeoIP/GeoLite2-Country.mmdb
MaxMindDBEnv MM_COUNTRY_CODE DB/country/iso_code
SetEnvIf MM_COUNTRY_CODE ^(RU|DE|FR) BlockCountry
Deny from env=BlockCountry

Using the above example, let’s adjust it to block Brazil. No worry, I won’t block Brazil, this is just a test as my IP currently is from Brazil, making it easier for me to test the setup. To block Brazil, check if MM_COUNTRY_CODE starts with BR: SetEnvIf MM_COUNTRY_CODE ^(BR) BlockCountry

MaxMindDBEnable On
MaxMindDBFile DB /usr/local/share/GeoIP/GeoLite2-Country.mmdb
MaxMindDBEnv MM_COUNTRY_CODE DB/country/iso_code
SetEnvIf MM_COUNTRY_CODE ^(BR) BlockCountry
Deny from env=BlockCountry

Add the above configuration snippet into a Location or Directory directive. This is because of the Deny command. This cannot be added directly under a virtual host.

<VirtualHost _default_:443>
  <Location />
    MaxMindDBEnable On
    MaxMindDBFile DB /usr/local/share/GeoIP/GeoLite2-Country.mmdb
    MaxMindDBEnv MM_COUNTRY_CODE DB/country/iso_code
    SetEnvIf MM_COUNTRY_CODE ^(BR) BlockCountry
    Order deny,allow
    Allow from al1
    Deny from env=BlockCountry
  </Location>
</VirtualHost>

5. Activation

To activate the configuration and to block Brazil, a restart of HTTPD is needed.

sudo service httpd restart

After HTTPD is successfully restarted, the new configuration is activated. To see if it is working, a basic test is to just access the site from an IP address that is blocked.

Test

My IP is from Brazil, accessing my site now should give me an access denied message.

It works!

Convert SVG to PDF

Some time ago I registered for a virtual training. The training material was made available in a web app. No PDF or a downloadable version of the material. Not a big issue, as long as I had internet access and only wanted to read the material. Problems started when I tried to access the material without a stable Internet connection. Depending where I am online is not an option. Plus point for having training material in PDF format. I can print PDF, making it easier to learn: offline, annotate pages, mark words, easier to read for the eyes. While a cloud solution is good for the vendor, it’s not always a very good option for the consumer.

Taking a closer look I found out that the material was loaded by the web app as SVG files. All pages where available as SVG: page1.svg to pageX.svg. So I could save the SVG files to my computer. Not something feasible for all content, but for selected chapters or pages this can make sense. Now I wanted to have these SVG files as PDF. It’s possible to transform SVG to PDF, especially on Linux. The most used tools available are

  • convert
  • Inkscape
  • rsvg-convert

The challenge was to convert the files to PDF in an acceptable quality. All tools transform SVG to PDF, but the quality differs, and most important: how the images are included. The only tool that rendered the embedded images correctly in the PDF was rsvg-convert. A sample of a page that shows perfectly the problem with the image is this one. The SVG as shown in the browser:

Font is clear and easy to read, the image in correctly embedded and shows all content. Now I wanted to have this page in PDF, with the exact same information included and in a good quality. For this, I tried all three tools and compared the result. All tools were installed and tested in Raspbian.

Convert

Convert is part of Image Magick. To get the tool under Debian you must install imagemagick.

Installation

apt-get install imagemagick

Run

convert page1.svg page1.pdf

Result

After the conversion from SVG to PDF, the result was far away from being useful. The font is not sharp, low quality, and the image is only showing half of the content. I think it’s possible to increase the quality of the font, but that the image isn’t correctly shown is a no-go.

Inkscape

Inkscape is an image editor, and comes with a GUI for end users. But there is a command line option available that can be used to transform a SVG to PDF.

Installation

sudo apt-get install inkscape

Run

inkscape page1.svg --export-area-page --without-gui --export-pdf=page1.pdf

Result

The quality of the font is better. Easy to read, sharp and clear. The image however is still not complete. This disqualifies Inkscape also as an acceptable solution.

rsvg-convert

This tool is part of librsvg. It comes from Gnome and has some “heavy” dependencies. The tool can convert SVG files to images like PNG or PDF.

Installation

sudo apt-get install librsvg2-bin

Run

rsvg-convert -f pdf -o page1.pdf page1.svg

Result

The quality of the font is good, easy to read and clear. The image is correctly shown. Definitely the best solution.

Conclusion

A lot of tools are available to convert SVG files to PDF. I did not even mention libraries available for Java or Javascript to do the job. The difference between the tools is how more complex SVG files are converted into PDF and if all information is included. Especially images can be a challenge for the tools. Based on my tests, I can recommend rsvg-convert. It is fast and gives a very good result.

Install SAP Web Dispatcher on Docker using SWMP

SAP Web Dispatcher is an important component in a SAP landscape. While have been treated as optional for many years and found mainly in SAP Portal scenarios, with the increase adoption of Fiori, having a reverse proxy in the landscape is becoming pre-requisite. While it’s possible to choose from a wide range of alternatives of servers for a reverse proxy, SAP`s Web Dispatcher is normally always the best fit in a SAP landscape. A question that sometimes arises is how to install Web Dispatcher.

First you settle on what version of Web Dispatcher (WD) to install. SAP Note 908097 states that you should go for the latest version. “Version 7.49 is the recommended SAP Web Dispatcher version for all backend systems.”

The actual installation gives you two options:

  • easy and
  • recommended.

The easy alternative is to simply un-sapcar the WD SAR file downloaded from Service Marketplace into a directory. To run WD, it`s then just to bootstrap it or run it with a given profile file. This installation method gives you a up and running WD in just one minute. The problem is that the files are all in one directory and not in the “official” directory structure of a normal SAP installation. But you get something like a portable WD installation: zip the directory and you can copy it to another server and can run WD from there.

The recommended alternative ensures that the WD is installed like a normal SAP product: all files follow the normal directory structure, etc. Installation is done using SWPM. Important when you are going to do some advanced configuration like PSE encryption, CryptoLib installation, etc. I`ll try to show how to install SAP Web Dispatcher the recommended way.

Download software

Download the needed software. It`s SWPM, Web Dispatcher, SAPCAR and HOSTAGENT.

SWPM

SAP Web Dispatcher 7.49 PL 112

SAPCAR 7.21

SAP Host Agent 7.21

Result

After you have downloaded all software, you have four files, summing up to almost 900 MB.

In a Unix environment, your WD system won`t have a graphical user interface and access to the system is given by SSH. This kind of environment can perfectly be emulated using Docker. Note: the SAR files need to be copied over to the target host.

Docker

There are several Linux images available for Docker. Let`s use Debian for this.

docker pull debian

For the actual image, see my Docker setup for Web Dispatcher: Dockerfile and docker-compose.yml: https://github.com/tobiashofmann/docker_webdispatcher_swpm

After running the Docker image, you have the files on the Linux system up and running, the Web Dispatcher and sapinst files available. Web Dispatcher is not yet installed. This is done by using sapinst. To run the installation, you`ll have to connect to sapinst using a different computer (most cases: your laptop). Let`s call the Docker container the target, and your computer the client.

Logon

I use Kitematic and to log on to my docker container, I just click on the EXEC button.

Logon to Docker container:

The log on from shell, the command is something like this:

bash -c "clear && docker exec -it dockerwebdispatcher_web_1 sh"

Target

To work properly, sapinst must be started as root. You then connect to it and log on. The logon is done by default with the user id running sapinst. Problem is that with the Docker images you do not know the root password. Same for environments where root access is only provided to a few or via sudo. You need to enable sapinst to run as root, but allow a different user (like <sid>adm) to log on. You achieve this by providing a parameter to sapinst informing the OS user allowed to log on remotely. The process is then:

  • Run sapinst as root (or sudo)
  • Connect to it informing a OS user (wddadm)

Sapinst Parameters

The needed parameter can be retrieved by letting sapinst show all available parameters. More information available in SAP Note 1745524 and at SAPinst central note.

./sapinst –p

Run sapinst in Docker

Provide the <sid>adm user as a property to sapinst.

./sapinst --properties SAPINST_REMOTE_ACCESS_USER=wddadm -nogui

Confirm that you know what you are doing.

Client

Start sapinstgui and connect to the target server on port 21212.

./sapinstgui

Inform the host name or IP address. In my case, it is 192.168.0.16. The port is the default sapinst port 21212.

Accept the fingerprint. You can check the fingerprint with the one printed by sapinst on the target server to be extra sure you are connecting to the right server.

Authenticate. You`ll need to provide the user id and password of the user running sapinst on the target host. In my case, the user is wddadm with password whatever. This is defined in the Dockerfile when the user is created.

sapinst output in Docker:

Logon on using wddadm / whatever

After a successful logon, sapinst will start. Current setup is not supported by SAP. For a production case this is a no-go, for my personal use case this is totally acceptable.

Sapinst shows the list of installable software options available. Web Dispatcher can be found at the end of the list.

Selecting SAP Web Dispatcher will start the installation.

Inform the path on the target server where the SAR files for SAP Web Dispatcher and SAP Host Agent can be found.

The files were copied into the container during the execution of the Dockerfile. All files are located at /home/wddadm.

If all packages are found, validated and added as considered valid for the installation.

Debian in Docker for sure won`t pass all the pre-requirements check build into SWMP. You`ll get a warning message, but SWMP won`t stop the installation. Select No. Seems that inside Docker, checking for the available free space is not working correctly.

Web Dispatcher configuration

Bootstrapping

Don’t worry, the system must not be accessible, yet exist. It’s just informing the bootstrap parameters. In my case, I am using a system that is not available, and it worked. Just be aware that in case the backend system changes, or isn’t even a ABAP system, like SMP3, you need to configure the Web Dispatcher profile manually.

Confirmation

Installation

The last step is to start Web Dispatcher. You can follow this on the console log of sapinst on the target server

If all worked, you get a confirmation message and the installation finishes.

SAPinst on the client host ends and so does it on the target host.

Validate installation

This gives you the time to validate the installation and check if all files are correctly installed.

Users

A new user sapadm was created

Folders

Web Dispatcher is installed under /sapmnt and instance is found in folder /usr/sap

This is perfectly aligned with the default locations of a SAP instance, and way better than simply putting all files into the same folder when unzipping the SAR. Especially when you consider that you may have to open a CSS ticket to SAP in your production environment or have new consultants arriving that expect the files to be located at the default location.

SAP Host Agent

The host agent was started and is running.

Start and stop Web Dispatcher

Starting and stopping Web Dispatcher via stopsap and startsap is working

stopsap

startsap

Admin web interface

The admin port of Web Dispatcher is listening by default on port 44300.

To access the service, the URL is

https://localhost:44300/sap/wdisp/admin/public/default.html

Log on to Web Dispatcher using the user webadm and password informed during installation.

Parallel download of files using curl

In a previous blog, I showed how to download files using wget. The interesting part of this blog was to pass the authentication cookies to the server as well as using the file name given by the Content-Disposition directive when saving the file. The example of the previous blog was to download a single file. What if you want to download several files from a server? Maybe hundreds or even thousands of files? wget is not able to read the location from a file and download these in parallel, neither is curl capable of doing so. You can start the download as a sequence, letting wget/curl download the files one by one, as shown in my other blog. Just use a FOR loop until you reach the end.

Commands

For downloading a large amount of files in parallel, you`ll have to start the download command several times in parallel. To achieve this, several programs in bash must be combined.

Create the list of files to download. This is the same as shown in my previous blog.

for i in 1 {1..100}; do `printf "echo https://server.fqdn/path/to/files/%0*d/E" 7 $i` >> urls.txt; done

Start the parallel download of files. Start 10 threads of curl in background. This is an enhanced version of the curl download command of my previous blog. Xargs is used to run several instances of curl.

nohup cat urls.txt | xargs -P 10 -n 1 curl -O -J -H "$(cat headers.txt)" >nohup.out 2>&1 &

Explanation

The first command is creating a list of files to download and stores them in the file urls.txt.

The second command is more complex. First, cat is printing the content of urls.txt to standard-out. Then, xargs is reading from standard-in and uses it as input for the curl command. For authentication and other headers, the content of the file headers.txt is used. The input for curl for the first line is then:

curl -O -J -H "$(cat headers.txt)" https://server.fqdn/path/to/files/0000001/E

The parameter –P 10 informs xargs to run the command 10 times in parallel. It takes the first 10 lines of input and starts for each input a new curl process. Therefore, 10 processes of curl are running in parallel. To run more downloads in parallel, give a higher value for –P, like 20, or 40.

To run the download in background, nohup is used. All output is redirected to nohup.out: >nohup.out 2>&1

SSH

To have the download running while being logged on via SSH, the tool screen should be used. After logon via ssh, call screen, run the above command, and hit CTRL + A + D to exit screen.

ssh user@server.fqdn
screen
nohup cat urls.txt | xargs -P 10 -n 1 curl -O -J -H "$(cat headers.txt)" >nohup.out 2>&1 &
CTRL+A+D
exit