Adjust image size of Docker qcow2 file

Posted on December 22, 2017 by Tobias Hofmann

Twitter

Google+

Facebook

Short version

Increase image size by 100GB:

qemu-img resize ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2 +100G

Resize partition:

qemu-system-x86_64 -drive file=~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2  -m 512 -cdrom ~/Downloads/gparted-live-0.30.0-1-amd64.iso -boot d -device usb-mouse -usb

Get an empty Docker.qcow2 image from my GitHub page and make your Docker use it:

https://github.com/tobiashofmann/sap-nw-abap-docker

How to adjust the Docker image size for using large containers like SAP NetWeaver ABAP

Docker uses an image file to store Docker containers. The file is named Docker.qcow2 and is located (on Mac) at:

~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2

By default, the file can grow to a size of 64 GB.

When you first start Docker, the size of this image is around 1.4GB. Adding containers, image, etc and it will grow to 64GB.

The 64GB default size can be seen when using qemu-img info:

qemu-img info ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2

When this limit is reached, Docker should automatically increase the size of the image, but this isn’t working always. As a result, when the image is at 64 GB, you can get an error message stating that the device is full:

no space left on device

At least with my Dockerfile for SAP NetWeaver ABAP Developer Edition Docker is not increasing the image file dynamically. Because of this I had to split the automatic installation process in two parts: base image setup and installation. I guess that right now the SAP Installation is filling up space faster than Docker can react.

The Docker.qcow2 file is a VM disk. Therefore, it is possible to manipulate it like any other virtual disk: you can increase the disk size and access files within the VM disk when you mount the image in a VM. An easy solution to change the disk size Docker has available to store images and containers is to increase the disk size. This can be done by using Qemu and GParted.

Preparations

Locate qcow2 on your Computer

Click on open in finder. Finder opens at the specified location.

Shut down Docker.

Make a backup of the Docker.qcow2 file.

Install QEMU

To install qemu, use brew on Mac.

brew install qemu

Now Qemu should be installed.

Download GParted

Download the x64 gparted ISO image from their web site:

https://downloads.sourceforge.net/gparted/gparted-live-0.30.0-1-amd64.iso

Resize Docker.qcow2

Resizing the Docker.qcow2 file to a new size consists of two steps.

Make the disk larger
Adjust the partition

Increase disk size

First, let’s make the disk larger. SAP can occupy some space, make sure you add enough GB to the image. An additional 100 GB should do it.

qemu-img resize ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2 +100G

Output is a simple status message.

Image resized.

Adjust partition table

To resize the image, start Qemu, use the GParted ISO image as boot file and mount the Docker.qcow2 disk.

qemu-system-x86_64 -drive file=~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2  -m 512 -cdrom ~/Downloads/gparted-live-0.30.0-1-amd64.iso -boot d -device usb-mouse -usb

I got some error messages, but Qemu started.

Starting the virtual machine will take some time. Be patient. Next you’ll have to configure the GParted ISO image.

The default values should be enough. This gives you a keyboard, mouse, English and X. After that, Gparted is started and you should see the Docker.qcow2 disk in the Gparted app.

Select the disk and click on Resize / Move. In the new size (MiB) field, enter the new size of the disk you need. The disk size is allocated dynamically and won’t occupy immediately space on your physical disk. So don’t be shy. Assign all free space to the partition.

Click on Resize/Move and on the Apply button

Last chance to stop. But as you need the new free space for Docker, click again on Apply.

The partition will be resized. In case something goes wrong, please restore the backup of the Docker.qcow2 file you made previously.

After the operation finishes, you can see that the partition is now offering 164GB.

Shutdown the VM. As the Docker.cqow2 file changed was the original one used by Docker, you have only to restart Docker to benefit from the new image size. Now you can use Docker to run SAP NetWeaver ABAP with just one command. As the Docker.qcow2 file is empty, even when the image size is reported as 4 GB, compressed (zipped) it’s just a few MB.

With the new Docker disk file you can even start SAP NetWeaver ABAP without getting the “no space left on device” message.

Image creation works. The space occupied by just the SAP NetWeaver ABAP image is already at 65 GB.

Start a container

docker run -P -h vhcalnplci --name nwabap751 -it nwabap:latest /bin/bash

In Kitematic

Start UUIDD

/usr/sbin/uuidd

Change to user npladm

su - npladm
startsap

Problem with starting SAP

When you log in to your container and run startsap, the program will fail. It will report that no instance profiles were found.

startsap

Take a look at the available profiles.

ls -1 /sapmnt/NPL/profile/

During the installation, the installation script installed the profile files for the container with the dummy name 4f65[…], after starting the container, we specified a specific host name: vhcalnplci. Of course, these do not match and make sapstart fail.

Let’s adjust the instance profile configuration.

Rename files
Substitute references to old hostname to correct one vhcalnplci

mv NPL_ASCS01_4f6e4ee4de40 NPL_ACS01_vhcalnplci
mv NPL_D00_4f6e4ee4de40 NPL_D00_vhcalnplci
sed -i -- 's/4f6e4ee4de40/vhcalnplci /g' *

Now run again sapstart and it should work. If not, stop and start the container and try again.

Twitter

Google+

Facebook

UUIDD problem when running SAP NW ABAP inside Docker

Posted on October 12, 2017 by Tobias Hofmann

Twitter

Google+

Facebook

UUID is a good old problem when it comes to running SAP NetWeaver on Linux / SuSE. You have a problem when you log in to your SAP system and get an error message. The error message shows the root cause and solution: “The UUID daemon (uuidd) is not active (code 59999). Check SAP note 1391070.”

Solution

SAP NetWeaver isn’t meant to run in Docker. When the software was designed, Docker or event containerization wasn’t around (maybe SUN). NetWeaver assumes that it is executed inside a real Linux. And the Docker version of OpenSuSE isn’t 100% a real Linux. A lot of services you get “automatically” when installing OpenSuSE are not available. One of those is that the init.d system is not starting services. Because of this, there is no UUID daemon running.

Make sure that the UUIDD service is running. For a normal Linux distribution, I blogged about this at a previous blog of mine. In case you are using Docker with OpenSuSE, make sure that uuidd is installed and executed during the image creation:

Installation

RUN zypper --non-interactive install --replacefiles  uuidd

Execution

RUN mkdir /run/uuidd && chown uuidd /var/run/uuidd && /usr/sbin/uuidd

Result

With the UUIDD running, the logon to SAP NetWEeaver ABAP is working. No restart of NetWeaver is needed.

Twitter

Google+

Facebook

Connect to NetWeaver ABAP instance running inside Docker

Posted on October 9, 2017 by Tobias Hofmann

Twitter

Google+

Facebook

This blog will help you to connect to your SAP NetWeaver ABAP instance running inside a Docker container. For how to get NetWeaver running inside a Docker container, please see my blog Docker for SAP NetWeaver ABAP 7.5x Developer Edition.

SAPGui

Open SAPGui and create a new connection.

Give a name for the connection and click on tab Advanced. I use NPL Docker. Activate expert mode and give the correct connection String. Check to which port the message server port is mapped to by Docker. Inside the container, the port is 3200, and in my case, the external port is 32771. Therefore, the connection String is:

Connection String: conn=/H/localhost/S/32771

Note: the port information is specified when you start the container. As an alternative, you can use Kitematic to see the port mapping.

Save and connect to NetWeaver.

The users and passwords can be found in the readme.html of the extracted SAP NW ABAP 751 download. Standard users are SAP* and Developer.

HTTP Access

You can test if access to your new SAP system is working via HTTP by calling the ping service: http://localhost:32769/sap/public/ping

For this to work, first activate the ping service in SICF.

When you get the response “Server reached.” you can start using the HTTP access.

SAP WebGui

For general WebGui activation, you can see my previous blog “Activation of SAP WebGui”. Here is a short version of this guide. As in the previous HTTP service access, the same procedure must be followed to have access to NPL via WebGui.

Activate the service webgui

To activate the SAP WebGui service, activate the node:

/sap/bc/gui/sap/its/webgui

Activation of public resources

You also need to activate the public service that contains the HTML files (JS, etc):

/sap/public/bc/its

Note

It is not sufficient to only activate the webgui node. The app is using additional resources that are available under /sap/public/bc/its. If this node is not activated, you’ll get an error message when logging in to webgui.

Therefore, for SAP WebGui to load the node /sap/public/bc/its must be activated too.

Activate the node its and its subnodes. Select Activate Service.

Activate with all sub nodes nodes (second Yes).

Result

After activating these two nodes, access to WebGui should work. To test this, call the URL http://localhost:32769/sap/bc/gui/sap/its/webgui After logging in, you should see the SAP Menu.

Twitter

Google+

Facebook

Dockerfile for SAP NetWeaver ABAP 7.5x Developer Edition

Posted on September 25, 2017 by Tobias Hofmann

Twitter

Google+

Facebook

This blog will help you to run your own SAP NetWeaver 7.51 ABAP instance inside a Docker container. This work was inspired by the Dockerfile created by Gregor Wolf and hosted at bitbucket.

The difference is that in Gregor’s version you download the NW ABAP installation files and when the container is build, you go manually through the installation. My Dockerfile assumes that you have downloaded the NW 7.51 ABAP installation files already and will automate the installation. Once you have downloaded the installation files from SAP you can make them locally available and create new Docker images / containers based on these, without having to download almost 16 GB again. And the installation script will run without prompting for user input.

Another differentiation is that you can “easily” change the Dockerfile to install NetWeaver 7.50 of the developer edition.

Pre-requisites

To be able to run the Dockerfile, you need

Docker installed
Downloaded and extracted installation files of SAP NW ABAP Developer Edition
Internet connection

Installation

1 Get the Dockerfile

From my GitHub repository, you can find a Dockerfile that helps you to create a Docker image and container that will install your downloaded NetWeaver version. All you need is the Dockerfile, so a simple download is sufficient. You can also download the file by cloning the GitHub repository: https://github.com/tobiashofmann/sap-nw-abap-docker

2. Download SAP NetWeaver DE installation files

Download your version of SAP NetWeaver ABAP 7.5x Developer Edition from SAP. The files are compressed (RAR).

Un-compress them into a folder named NW751. The folder must be at the same location where your Dockerfile is.
Build the Docker image

Build the Docker image

Command:

docker build -t nwabap .

Sample output

After the build is finished, the last line you should see is

Successfully tagged nwabap:latest

To see the ID and name of the newly created image, run the following command:

docker images

Sample output

The command lists the ID, tag and size of the image. As you can see, it’s a 15 GB Docker image. Using this image, you can start a container and install NW ABAP 7.51 DE inside the container.

Create container from image

You can now create a container from the image. You’ll have to connect to the container and run the installation script run.sh. The file was created during docker build. It will run SAP’s install.sh and fill in the input automatically.

docker run -P -h vhcalnplci --name nwabap751 -it nwabap:latest /bin/bash

This will start the container and log you in. What you’ll get is the bash shell.

bash-4.3#

In case you have Kitematic installed, you can see the running container listed.

The container configuration for the ports is also visible there. The ports are automatically mapped by Docker. The message server port 3200 is accessible through localhost:32771, and the HTTP port 8000 through localhost:32769. This mapping can be changed either inside Kitematic or when the container is started on the command shell.

Run ls to see the content of the current directory. You can see the install.sh file from SAP (feel free to start the installation manually) and the run.sh script that will automate the installation.

Start installation

Run the script run.sh to install SAP NetWeaver ABAP 7.51. The script will enter all information requested by install.sh automatically. The installation will take some time, +/- 20 minutes.

./run.sh

Sample output

[…]

The installation worked when the script ends and you can see the output:

Installation of NPL successful

Twitter

Google+

Facebook

Install SAP Web Dispatcher on Docker using SWMP

Posted on May 25, 2017 by Tobias Hofmann

Twitter

Google+

Facebook

SAP Web Dispatcher is an important component in a SAP landscape. While have been treated as optional for many years and found mainly in SAP Portal scenarios, with the increase adoption of Fiori, having a reverse proxy in the landscape is becoming pre-requisite. While it’s possible to choose from a wide range of alternatives of servers for a reverse proxy, SAP`s Web Dispatcher is normally always the best fit in a SAP landscape. A question that sometimes arises is how to install Web Dispatcher.

First you settle on what version of Web Dispatcher (WD) to install. SAP Note 908097 states that you should go for the latest version. “Version 7.49 is the recommended SAP Web Dispatcher version for all backend systems.”

The actual installation gives you two options:

easy and
recommended.

The easy alternative is to simply un-sapcar the WD SAR file downloaded from Service Marketplace into a directory. To run WD, it`s then just to bootstrap it or run it with a given profile file. This installation method gives you a up and running WD in just one minute. The problem is that the files are all in one directory and not in the “official” directory structure of a normal SAP installation. But you get something like a portable WD installation: zip the directory and you can copy it to another server and can run WD from there.

The recommended alternative ensures that the WD is installed like a normal SAP product: all files follow the normal directory structure, etc. Installation is done using SWPM. Important when you are going to do some advanced configuration like PSE encryption, CryptoLib installation, etc. I`ll try to show how to install SAP Web Dispatcher the recommended way.

Download software

Download the needed software. It`s SWPM, Web Dispatcher, SAPCAR and HOSTAGENT.

SWPM

SAP Web Dispatcher 7.49 PL 112

SAPCAR 7.21

SAP Host Agent 7.21

Result

After you have downloaded all software, you have four files, summing up to almost 900 MB.

In a Unix environment, your WD system won`t have a graphical user interface and access to the system is given by SSH. This kind of environment can perfectly be emulated using Docker. Note: the SAR files need to be copied over to the target host.

Docker

There are several Linux images available for Docker. Let`s use Debian for this.

docker pull debian

For the actual image, see my Docker setup for Web Dispatcher: Dockerfile and docker-compose.yml: https://github.com/tobiashofmann/docker_webdispatcher_swpm

After running the Docker image, you have the files on the Linux system up and running, the Web Dispatcher and sapinst files available. Web Dispatcher is not yet installed. This is done by using sapinst. To run the installation, you`ll have to connect to sapinst using a different computer (most cases: your laptop). Let`s call the Docker container the target, and your computer the client.

Logon

I use Kitematic and to log on to my docker container, I just click on the EXEC button.

Logon to Docker container:

The log on from shell, the command is something like this:

bash -c "clear && docker exec -it dockerwebdispatcher_web_1 sh"

Target

To work properly, sapinst must be started as root. You then connect to it and log on. The logon is done by default with the user id running sapinst. Problem is that with the Docker images you do not know the root password. Same for environments where root access is only provided to a few or via sudo. You need to enable sapinst to run as root, but allow a different user (like <sid>adm) to log on. You achieve this by providing a parameter to sapinst informing the OS user allowed to log on remotely. The process is then:

Run sapinst as root (or sudo)
Connect to it informing a OS user (wddadm)

Sapinst Parameters

The needed parameter can be retrieved by letting sapinst show all available parameters. More information available in SAP Note 1745524 and at SAPinst central note.

./sapinst –p

Run sapinst in Docker

Provide the <sid>adm user as a property to sapinst.

./sapinst --properties SAPINST_REMOTE_ACCESS_USER=wddadm -nogui

Confirm that you know what you are doing.

Client

Start sapinstgui and connect to the target server on port 21212.

./sapinstgui

Inform the host name or IP address. In my case, it is 192.168.0.16. The port is the default sapinst port 21212.

Accept the fingerprint. You can check the fingerprint with the one printed by sapinst on the target server to be extra sure you are connecting to the right server.

Authenticate. You`ll need to provide the user id and password of the user running sapinst on the target host. In my case, the user is wddadm with password whatever. This is defined in the Dockerfile when the user is created.

sapinst output in Docker:

Logon on using wddadm / whatever

After a successful logon, sapinst will start. Current setup is not supported by SAP. For a production case this is a no-go, for my personal use case this is totally acceptable.

Sapinst shows the list of installable software options available. Web Dispatcher can be found at the end of the list.

Selecting SAP Web Dispatcher will start the installation.

Inform the path on the target server where the SAR files for SAP Web Dispatcher and SAP Host Agent can be found.

The files were copied into the container during the execution of the Dockerfile. All files are located at /home/wddadm.

If all packages are found, validated and added as considered valid for the installation.

Debian in Docker for sure won`t pass all the pre-requirements check build into SWMP. You`ll get a warning message, but SWMP won`t stop the installation. Select No. Seems that inside Docker, checking for the available free space is not working correctly.

Web Dispatcher configuration

Bootstrapping

Don’t worry, the system must not be accessible, yet exist. It’s just informing the bootstrap parameters. In my case, I am using a system that is not available, and it worked. Just be aware that in case the backend system changes, or isn’t even a ABAP system, like SMP3, you need to configure the Web Dispatcher profile manually.

Confirmation

Installation

The last step is to start Web Dispatcher. You can follow this on the console log of sapinst on the target server

If all worked, you get a confirmation message and the installation finishes.

SAPinst on the client host ends and so does it on the target host.

Validate installation

This gives you the time to validate the installation and check if all files are correctly installed.

Users

A new user sapadm was created

Folders

Web Dispatcher is installed under /sapmnt and instance is found in folder /usr/sap

This is perfectly aligned with the default locations of a SAP instance, and way better than simply putting all files into the same folder when unzipping the SAR. Especially when you consider that you may have to open a CSS ticket to SAP in your production environment or have new consultants arriving that expect the files to be located at the default location.

SAP Host Agent

The host agent was started and is running.

Start and stop Web Dispatcher

Starting and stopping Web Dispatcher via stopsap and startsap is working

stopsap

startsap

Admin web interface

The admin port of Web Dispatcher is listening by default on port 44300.

To access the service, the URL is

https://localhost:44300/sap/wdisp/admin/public/default.html

Log on to Web Dispatcher using the user webadm and password informed during installation.

Twitter

Google+

Facebook

SAPInst error java.net.InetAddress.getHostAddress

Posted on May 2, 2017 by Tobias Hofmann

Twitter

Google+

Facebook

SAPinst is used to install SAP software. You can run the installation with its graphical user interface (GUI) directly on the target host, or use a remote installation. Remote installation is normally used when the target host offers no GUI and all work is done via remote logon, like SSH. In such a setup, on the target host sapinst is started in server mode, and on the admin laptop, sapinst is started in with a GUI. The admin then connects to the target and executes the installation of SAP software remotely.

Error: java.net.InetAddress.getHostAddress

The setup to reproduce the error is to have sapinst running on a target host and listening on the default port 21212 and connecting to this instance using sapinstgui.

Setup target host

Start sapinst on the target host:

./sapinst -nogui

Setup client host

Start sapinst in GUI mode: ./sapinstgui Inform the target host (computername or IP)

Accept the fingerprint.

Error

Sapinstgui hangs and is not advancing to the next screen. The status bar shows that it`s connected:

On the target server, sapinst gives this error message:

java.lang.NullPointerException: while trying to invoke the method java.net.InetAddress.getHostAddress() of a null object loaded from field com.sap.sdt.util.misc.Uid.address of an object loaded from local variable 'this'

Solution

To solve this error, a configuration on the client computer is needed. While not obvious, the error is caused by a misconfiguration of the client DNS resolution. Easiest solution is to manually change the hosts file and to ensure that the client computer name is listed. To find out the computers host name, run hostname.

hostname

In my case, the host name is linux-01sn.

Example: wrong hosts file

The host name is not listed in the hosts file. To correct this, insert the hostname for instance to localhost.

Example: working hosts file

Test

To see if sapinstgui can now connect to the target server without error, restart sapinstgui. After connecting to sapinst server, you`ll get the authentication screen.

sapinst output on target host:

Twitter

Google+

Facebook

Personas 3.0 SP4 post-installation tasks

Posted on December 8, 2016 by Tobias Hofmann

Twitter

Google+

Facebook

After installing Personas 3.0 SP4, you have to do some additional tasks in your system to really enjoy the power of Personas 3.0. These steps are outlined in the release note 2376426, as well in the note 2383296 and 2376427. Both SAP Note are going to be uploaded manually to my NetWeaver system, as I do not have a connection to SAP. You can follow the steps my other blog on how to do this.

Required post-installation corrections

First apply note SAP Note 2383295: SAP Screen Personas 3.0 SP04: Required post-installation corrections. Download the SAP Note and upload it to your system via SNOTE.

The status of the note: can be implemented. The note will implement four corrections.

Lot of changes going to happen.

After executing this, the SAP Note is implemented.

Update of client source files

Next note to apply is 2376427 – SAP Screen Personas 3.0 SP04: Update of Client Sources Files. Download the SAP Note and upload it to your system via SNOTE.

The status of the note: can be implemented. Confirm the changes the note will apply.

The note was successfully implemented.

Test

Access https://<server>:<port>/sap/bc/personas

Pressing ALT or Option (Mac), I can see now the keyboard shortcuts.

Instant script feature.

Twitter

Google+

Facebook

Install Personas 3.0 SP4

Posted on December 5, 2016 by Tobias Hofmann

Twitter

Google+

Facebook

Pre-requisites

Check that Personas 3 add-on and is already installed. SP04 is a support package, therefore SPAM is used to install it. SP4 needs SPAM with patch 63 in NetWeaver 7.5. Ensure to update first your SPAM to the right version. See my other blog on how to update SPAM. Before you can update, check that there are not objects locked in a transport request.

Update

Transaction: SPAM
Client: 000
User: user with right permissions

Check that SPAM status is green and no queue is defined.

Upload SAR file: Support Package -> Load packages -> From Front End

Decompress.

Check the uploaded package. Select new support package under directory and then display.

The status needs to be yellow (not yet imported) and the perquisite set 01 all must be green and ok.

Select the package and click on Queue to define a new queue.

This SP04 is not protected with a password.

Select No. This brings you back to SPAM main screen. The status is now yellow and next action is given as import queue.

Import queue. Go to: Support Package -> Import queue.

You can now go get a coffee or follow the status messages in the status bar.

At the end of the import, a dialog is shown. It should be a nice “success!!” dialog, I got this:

What happened is that I applied a Personas 3 note that changed some standard objects. The dialog is SAP’s polite reminder to check if I want to keep these changes or discard them and let Personas 3 SP03 overwrite them. I want to not keep them and let Personas 3 use its own repository objects, so I selected continue. At the end you’ll see a short walk through what you should do (analyse the stuff, etc), although this isn’t possible in my demo system, as I do not have a SAP Note connection – something needed to run the validation.