Adjust image size of Docker qcow2 file

Short version

Increase image size by 100GB:

qemu-img resize ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2 +100G

Resize partition:

qemu-system-x86_64 -drive file=~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2  -m 512 -cdrom ~/Downloads/gparted-live-0.30.0-1-amd64.iso -boot d -device usb-mouse -usb

Get an empty Docker.qcow2 image from my GitHub page and make your Docker use it:

https://github.com/tobiashofmann/sap-nw-abap-docker

How to adjust the Docker image size for using large containers like SAP NetWeaver ABAP

Docker uses an image file to store Docker containers. The file is named Docker.qcow2 and is located (on Mac) at:

~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2

By default, the file can grow to a size of 64 GB.

When you first start Docker, the size of this image is around 1.4GB. Adding containers, image, etc and it will grow to 64GB.

The 64GB default size can be seen when using qemu-img info:

qemu-img info ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2

When this limit is reached, Docker should automatically increase the size of the image, but this isn’t working always. As a result, when the image is at 64 GB, you can get an error message stating that the device is full:

no space left on device

At least with my Dockerfile for SAP NetWeaver ABAP Developer Edition Docker is not increasing the image file dynamically. Because of this I had to split the automatic installation process in two parts: base image setup and installation. I guess that right now the SAP Installation is filling up space faster than Docker can react.

The Docker.qcow2 file is a VM disk. Therefore, it is possible to manipulate it like any other virtual disk: you can increase the disk size and access files within the VM disk when you mount the image in a VM. An easy solution to change the disk size Docker has available to store images and containers is to increase the disk size. This can be done by using Qemu and GParted.

Preparations

Locate qcow2 on your Computer

Click on open in finder. Finder opens at the specified location.

Shut down Docker.

Make a backup of the Docker.qcow2 file.

Install QEMU

To install qemu, use brew on Mac.

brew install qemu

Now Qemu should be installed.

Download GParted

Download the x64 gparted ISO image from their web site: 

https://downloads.sourceforge.net/gparted/gparted-live-0.30.0-1-amd64.iso

Resize Docker.qcow2

Resizing the Docker.qcow2 file to a new size consists of two steps.

  1. Make the disk larger
  2. Adjust the partition

Increase disk size

First, let’s make the disk larger. SAP can occupy some space, make sure you add enough GB to the image. An additional 100 GB should do it.

qemu-img resize ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2 +100G

Output is a simple status message.

Image resized.

Adjust partition table

To resize the image, start Qemu, use the GParted ISO image as boot file and mount the Docker.qcow2 disk.

qemu-system-x86_64 -drive file=~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2  -m 512 -cdrom ~/Downloads/gparted-live-0.30.0-1-amd64.iso -boot d -device usb-mouse -usb

I got some error messages, but Qemu started.

Starting the virtual machine will take some time. Be patient. Next you’ll have to configure the GParted ISO image.

The default values should be enough. This gives you a keyboard, mouse, English and X. After that, Gparted is started and you should see the Docker.qcow2 disk in the Gparted app.

Select the disk and click on Resize / Move. In the new size (MiB) field, enter the new size of the disk you need. The disk size is allocated dynamically and won’t occupy immediately space on your physical disk. So don’t be shy. Assign all free space to the partition.

Click on Resize/Move and on the Apply button

Last chance to stop. But as you need the new free space for Docker, click again on Apply.

The partition will be resized. In case something goes wrong, please restore the backup of the Docker.qcow2 file you made previously.

After the operation finishes, you can see that the partition is now offering 164GB.

Shutdown the VM. As the Docker.cqow2 file changed was the original one used by Docker, you have only to restart Docker to benefit from the new image size. Now you can use Docker to run SAP NetWeaver ABAP with just one command. As the Docker.qcow2 file is empty, even when the image size is reported as 4 GB, compressed (zipped) it’s just a few MB.

With the new Docker disk file you can even start SAP NetWeaver ABAP without getting the “no space left on device” message.

Image creation works. The space occupied by just the SAP NetWeaver ABAP image is already at 65 GB.

Start a container

docker run -P -h vhcalnplci --name nwabap751 -it nwabap:latest /bin/bash

In Kitematic

Start UUIDD

/usr/sbin/uuidd

Change to user npladm

su - npladm
startsap

Problem with starting SAP

When you log in to your container and run startsap, the program will fail. It will report that no instance profiles were found.

startsap

Take a look at the available profiles.

ls -1 /sapmnt/NPL/profile/

During the installation, the installation script installed the profile files for the container with the dummy name 4f65[…], after starting the container, we specified a specific host name: vhcalnplci. Of course, these do not match and make sapstart fail.

Let’s adjust the instance profile configuration.

  1. Rename files
  2. Substitute references to old hostname to correct one vhcalnplci
mv NPL_ASCS01_4f6e4ee4de40 NPL_ACS01_vhcalnplci
mv NPL_D00_4f6e4ee4de40 NPL_D00_vhcalnplci
sed -i -- 's/4f6e4ee4de40/vhcalnplci /g' *

Now run again sapstart and it should work. If not, stop and start the container and try again.

Subsonic on Raspberry Pi

About Subsonic

“Subsonic is an open source, web-based media server. It is written in Java, so it can run on any operating system with Java support. Subsonic supports streaming to multiple clients simultaneously, and supports any streamable media.” (Source: Wikipedia)

My first contact with Subsonic was several years ago. If memory serves me right, it was around 2008 when I was looking for a media software that can be accessed from remote. At that time, Subsonic and the internet didn’t serve me well enough in Rio de Janeiro to continue my endeavor with Subsonic. Only in 2015 I came back to it, thanks to Raspberry Pi. This combination gave me a new look at media access. Up to now the experience I have is good enough to make me want to share it with others. If you want to stream your private music collection without spending money on a cloud based server / service, this blog may be for you.

Pre-Requisites

Install Java 8

Subsonic wants Java 8, and Java 8 is available for Raspberry Pi. You can also download it form the Oracle Java website. The version you need is the one compatible with the Raspberry Pi processor: jdk-8-oracle-arm-vfp-hflt. Or you install it using aptitude.

Command: sudo apt-get install oracle-java8-jdk

This downloads the required packages

Afterwards, Java 8 is configured.

To test if Java 8 is available and correctly installed, just call Java.

Command: java –version

The output shows that Java 8 is installed. Congratulations!

Set JAVA_HOME

Java is installed, but for applications to know where to find it, an environment variable is used: JAVA_HOME. This variable points to the install dir of Java. To not have to configure this for each user, the configuration can be made global to all. The above command installed Java 8 at this location: /usr/lib/jvm/jdk-8-oracle-arm-vfp-hflt

Command: sudo vim /etc/environment

Insert JAVA_HOME=/usr/lib/jvm/jdk-8-oracle-arm-vfp-hflt

Installation

Download SubSonic

Subsonic can be downloaded from the Project homepage: http://www.subsonic.org/pages/download.jsp

Click the link to go to the download page and copy from there the actual download link and use wget to download it from Raspberry Pi.

Command: wget –O http://downloads.sourceforge.net/project/subsonic/subsonic/5.2.1/subsonic-5.2.1.deb

In case the file wasn’t saved as subsonic-5.2.1.deb, rename it. You do not have to, but it makes things easier.

Install Subsonic

The file downloaded above is a deb file. These files are meant to be used by the debian package manager and contain the actual file to be installed and dependencies.

Command: sudo dpkg -i subsonic-5.2.1.deb

This installs and already starts subsonic. To see the output log:

Command: sudo tail /var/subsonic/subsonic_sh.log

Not exactly what we want, as now sSubsonic is already running, but not configured. To stop subsonic:

Command: sudo /etc/init.d/subsonic stop

Subsonic stores its data in default folders. By default, for Debian it is /var/subsonic. Because subsonic was already started, this folder is created and filled with content, using the default subsonic user: root (yep, BAD, very BAD!).

Configuration

Subsonic will be run in the background at start as a service. For this to work, a subsonic user needs to be configured.

Create user

Command: sudo adduser subsonic

Add the user to the audio group, in caes you want subsonic to output audio.

Command: sudo adduser subsonic audio

How to make subsonic use that user and run under that user id and not as root? The user information is stored in the default subsonic configuration file: /etc/default/subsonic.

Command: more /etc/default/subsonic

The last line must be changed to: SUBSONIC_USER=subsonic

Permissions

Make user subsonic owner of /var/subsonic

Command: sudo chown subsonic:subsonic /var/subsonic –Rv

Reverse Proxy

Subsonic can now be accessed, but I want to be able to access it through my standard web site (this one). I want to do that without having to do much port forwarding or virtual hosts. The easiest solution is to make use of Apache as a reverse proxy.

Change URL

As subsonic will be run from behind a reverse proxy, the standard URL will be different: the URL used will be /subsonic. Therefore, the configuration of subsonic must be made aware of that. To find out the correct parameter, take a look at which parameters Subsonic supports.

Command: subsonic –help

The parameter is context-path. This parameter must be added to the config file.

Configure Apache Reverse Proxy

Add the following RP rules to the config file of the virtual server:

In my case, it is default-ssl

For reverse proxy to work, the module must be enabled.

Command: a2enmod proxy_http

Restart Apache

Command: sudo apache2ctl restart

That’s it from the Apache as reverse proxy part. Subsonic is already configured to use the new URL and Apache is ready.

Start subsonic

To be able to use Subsonic from the internet, just start it and check that everything is working correctly. Start subsonic:

Command: sudo /etc/init.d/subsonic start

Check pid:

Command: ls -alh /run/subsonic.pid

  • Created as user subsonic

Check process:

Command: ps -ef | grep subsonic

Use Subsonic

Log on to Subsonic.

Advanced features

Transcoding

It may be useful to transcode some music files on the fly. For instance, when the consumed bandwidth is too high, FLAC is used or when the user is accessing Subsonic over a low bandwidth network like 4G in Brazil. Subsonic allows for automatic transcoding of files. This feature can be activated for each user and the sampling limit can also be specified. It is therefore possible to define a user for mobile client usage and specify a max bitrate of 128 Kbps for him. The max bandwidth is defined in the user section of the configuration settings.

User settings

Transcoding settings

The programs ffmpeg and lame are installed automatically when Subsonic was installed via Debian package manager.