ERR_CONTENT_DECODING_FAILED

Configuring a reverse proxy is not an easy task. It involves some trial and error and dealing with unexpected errors. One of those errors is ERR_CONTENT_DECODING_FAILED. The web site won’t load in your browser will show this error message:

Error ERR_CONTENT_DECODING_FAILED may show up in your browser when a resource is configured on your reverse proxy, and the backend communication is working. That is: the backend is returning data, but not in a form the browser expects.

Example: browser expects a GZIP response, but receives plain text. Therefore the hint from your browser about content decoding failed. The content is received, but the browser is not able to decode / understand the data. If a plain text response is expected, but the received response from the backend is zipped, the browser cannot read the content.

To solve this error, reset the Accept-Encoding request header in your Reverse Proxy configuration.

Apache

Documentation

RequestHeader unset Accept-Encoding

Example

<Location /test>
    RequestHeader unset Accept-Encoding
    ProxyPass https://0.0.0.0:443
    ProxyPassReverse https://0.0.0.0:443/
    Order allow,deny
    Allow from all
</Location>

NGINX

Documentation

proxy_set_header Accept-Encoding "";

How to access UI5 model data

Old habits are not easily dying and replaced by best practices and general recommendations. In the early days, when UI5 started to gain traction, people discovered it, tried it out, wrote apps, made them somehow work. Everybody was learning, and things we can do today were not possible or known then. Changes to the documentation, API and recommendations are simply the result of lessons learned.

As UI5 apps are based around a model, its data, representation and manipulation, a lot of questions around UI5 development are about the model: how to access data, change, update or delete it. A good thing of UI5 is that it is following semantic versioning, and code written for UI5 1.1x or 1.2x will still work with latest versions like 1.5x. It doesn’t mean that you, as a developer, should simply copy & paste example code found somewhere.

Example

After loading the model, be it JSON or OData, you may have to access a specific property in you code. What you can find in the Internet is code like:

oData

var oData = this.getView().getModel().oData;
var firstname oModel.getModel().oData[entity].firstname;
var name = oData[entity].name;

JSON

var oData = this.getView().getModel(“device”).oData;
var osname = oModel.oData.os.name;
var osname = this.getView().getModel().getData().os.name;

First line will give you the model data object, 2nd and 3rd line the property osname of the model. The 3rd line is partly correct, as the access to the data is done through a method, but access to property name is done directly. You can work with the data object now directly, but you shouldn’t. What you want is not to work with the raw data that defines your model, but with properties.

Use access methods

Data binding is important when developing an UI5 app. You update properties, and you want to have the UI elements automatically be updated too. When accessing the properties directly, you have to check if the change is correctly propagated. Using access methods, UI5 will for sure take care of this. The method to access properties of your data model are getData, getProperty or getObject. Applying this to above code:

JSON

var oData = this.getView().getModel().getData();
var osname = this.getView().getModel(“device”).getProperty(“/os/name”);

OData

var entity = this.getView().getModel().getObject("/"+key);
var property = this.getView().getModel().getProperty("/"+key+"/Depth");

This is now only using the methods to access data. To alter the property, use the setProperty method.

OData

this.getView().getModel().setProperty("/"+key+"/Depth", 11.111)

JSON

this.getView().getModel("device").setProperty("/os/name", "windows")

The above examples may not be perfect. They show that UI5 offers methods to access model data. Using these methods your code will continue to work in the future and is guaranteed to work in future releases of UI5. Direct access to the model data is done at your own risk. In case you work on a UI5 app, use the access methods. If you work on an older app that uses direct access to the model, try to refactor the app. The change from oModel.oData to oModel.getData() is as simple as executing a find and replace.

 

How to use find to sort files across folders

Short version

You have files named File0.txt to File100.txt in different folders and want to move the first 30 files in a separate directory (command for Mac users. Linux users can use find and mv):

For sorting FileNN.txt (character + number)

gfind -type f -printf "%f %p\n" | sort -n -k 1.5 | sed 's/.* //' | head -30 | xargs gmv -t ./A/

For sorting NN.txt (numeric filename)

gfind -type f -printf "%f %p\n" | sort -n | sed 's/.* //' | head -30 | xargs gmv -t ./A/

Preparation

For the below commands to work, you’ll need to use GNU find. If you are using a Mac, you’ll need to install the GNU version of find and mv via homebrew.

brew install findutils coreutils

Create a test folder structure. There will be 3 folders and several files in them.

mkdir 1
mkdir 2
mkdir 3

Create 100 files with name TestNN.txt with sample content and place them in one of the three directories randomly.

for i in {000..100}
  do
    Num=$((1 + RANDOM % 3))
    echo hello > "$Num/File${i}.txt"
done

After running the above script, the folder will look like this (running ls -R)

Also create the target directory A:

mkdir A

Commands

After the initial setup is done, we have several files in 3 directories. If you use find to get a list of all files, you’ll see that the output is not sorted.

gfind ./ -type f

A Unix command to sort files is sort. Applying sort in this scenario won’t help, as the files are sorted by the folder name:

gfind ./ -type f | sort -n

The output is now sorted by folder name and then by file name, but not only by file name. Copying the first 50 elements won’t result in the File1 – File 50. The files are not distributed across the directories as needed.

It is possible to see a solution to the problem: sort only on the filename, while still having the complete path in the output for piping the parth to the copy command. Find includes exactly this possibility: print a specific field. To control the output, parameter -printf is available, and %f prints the filename, while %p includes the folder.

gfind -type f -printf "%f\n"

The output of the command only prints the filename.

To output the file with path, use %p. In both cases \n is used to have each file in a new line.

gfind -type f -printf "%p\n"

Both output parameters can be combined. %f %p\n will first print the filename, then space, then the path.

gfind -type f -printf "%f %p\n"

Applying sort on this output will sort on the file name only.

gfind -type f -printf "%f %p\n" | sort -n

Close, but not exactly how it should be. In case your filename consists only of numbers, this will already work. In the example however, the filename contains characters. Therefore, sorting is not working correctly. It starts with File0.txt, then File1.txt, but then comes File10.txt and not File2.txt. To sort by the number, add to sort an additional parameter: -k 1.5. As the filename contains a fixed value (File), the parameter will instruct sort to ignore this part when sorting and focus only on the number.

Note: you may apply the same sort parameter without using find, just ls. As long as your path has the same size, it will work. For folders named 1..9 it’s ok, but when your folder has two or more chars (like 10, or 213, or test), the parameter needs to be adjusted.

List all files with directory name using ls:

ls -d1 */*

Sort by number in filename:

ls -d1 */* | sort -n -k 1.7

gfind -type f -printf "%f %p\n" | sort -n -k 1.5

With the last command, the output is correctly sorted based on the filename. Now, how to use this output to move the files to the target directory? Just piping the output to mv won’t work. The first part with the filename is not needed, only the second part. Both parts are separated by blank, and using sed, it’s possible to eliminate the part before the blank from the output.

gfind -type f -printf "%f %p\n" | sort -n -k 1.5 | sed 's/.* //'

The last step is now to use mv to move the files to the target directory. To not have to move all files, let’s take only the first 30 files. Gnu mv is needed to move the files, as the default MacOS BSD mv does not include the -t parameter. To pass the files line by line, xargs is used together with gmv.

gfind -type f -printf "%f %p\n" | sort -n -k 1.5 | sed 's/.* //' | head -30 | xargs gmv -t ./A/

Result

Now there are the first 30 files in folder A.

gls -1v ./A

 

SAP Web IDE: Invalid backend response received by SCC

Connectivity between SAP Cloud Platform and an on premise SAP NetWeaver system is normally achieved via SAP Cloud Connector. A nice feature depending on this is the remote connection of SAP Web IDE to an on premise ABAP system. The feature allows to easily load apps from the ABAP system and change or extend them from everywhere.

For this feature to work, some ICF services must be active on the ABAP system and remote access enabled on SCC. If not, Web IDE cannot “talk” to NW ABAP. Some possible errors and solutions regarding the setup are shown in this blog.

Scenario

A NetWeaver ABAP system with Fiori apps is available and the SAP Cloud Connector is configured to expose the system to SAP Cloud. I am using the SAP NetWeaver ABAP 7.51 Developer Edition for the scenario.

In the destination section of SCP, the SCC is shown as connected and the destination NPL is configured and working. A connection tests gives back a successful message: SCP <–> SCC <–> NW works.

Problem

A developer tries to extend a Fiori app. In Web IDE, the project wizard for an extension project is used.

After selecting the on premise system destination, an error message is displayed. The actual error message can differ. Sometimes you see an informative error message or just some red text or maybe nothing.

Error messages

In all cases, you can check the log of SCC and see a detailed information on the error.

The error message is:

Access denied to /sap/bc/adt/discovery for virtual host npl:443

Solution

The ICF service /sap/bc/adt/discovery is not accessible. This can be because the user does not have the right permissions, or the service is not active in the NW system, or SCC is not exposing the service.

Alternative A: SCC not exposing service

Adding a service in SCC will only expose the exact path, not the sub path. Either you add all paths exactly in the resource list, or change the access policy to accept sub-paths too.

Root cause: Path only, excluding sub-paths.

Solution: Change this to will allow Web IDE to access the resource.

Alternative B: ICF service not active

In the NW ABAP system, got to transaction SICF and check node /sap/bc/adt. This node must be activated. By default, this node is deactivated and must be activated by Basis.

Root cause: Service deactivated

Solution: Activate node adt. Right click and select Activate Service.

Alternative C: Missing authorization

Check with SU53 and SAP Help what is missing and assign the right permissions to your user.

Result

After applying the correct solution, the developer can use the extension project wizard in SAP Web IDE to load available applications.

 

Download resources from SAP Cloud for your CI job

When running a CI job you may need to use some SAP tools. For instance, the MTA builder or Neo tools. Many CI servers include integration to build tools or plugins are provided by the community or vender. Jenkins offers plugins for Maven, Ant or Node that let you easily integrate these into a CI jobs. If you have a CI job for SAP, it is your task to make the necessary tools available. There are not many plugins for SAP available for Jenkins.

Some tools you may need can be found on SAP’s tool site. For instance, the MTA builder. A simple JAR file that is available for download and needed in case you are working with MTA apps.

Before you can download the JAR file, you need to agree to the EUL.

This means that you cannot download the JAR using cli:

wget https://tools.hana.ondemand.com/additional/mta_archive_builder-1.1.0.jar

Solution

Running the above wget command will not download the tool, but a web site. Some may know that this is very close to how Oracle protected it’s Java download. And the “solution” here is the same: send the right cookie via wget.

wget --header "Cookie: eula_3_1_agreed=tools.hana.ondemand.com/developer-license-3_1.txt" https://tools.hana.ondemand.com/additional/mta_archive_builder-1.1.0.jar

Works for downloading other tools from the download page like the Neo SDK too:

wget --header "Cookie: eula_3_1_agreed=tools.hana.ondemand.com/developer-license-3_1.txt" https://tools.hana.ondemand.com/sdk/neo-javaee6-wp-sdk-2.137.0.1.zip

Let’s hope SAP provides some Jenkins plugins that take care of downloading these automatically.

Clone a SCP git repository from command line

I have a git repository on SCP that I want to clone using git on my laptop. I thought this should be easy to do. The source code of my project is available in the git repo at SCP. Cloning the repo using git clone from this URL should work.

git clone https://git.hanatrial.ondemand.com/p539123trial/cisample

The clone fails with “service not enabled.” Looking at SAP’s documentation, this should not have happened. Here SAP Cloud Platform documentation for the git service differs from reality.

SAP Help

I did a), and b) did not apply, as I wasn’t asked for my SCN user ID nor password. SAP’s git troubleshooting guide contains a section about the error message. Good to know that there is a possible solution, but I already did already what the proposed solution to the error is:

Ensure that you have the correct repository URL. Copy it from the Source Location section of the repository’s details page in the SAP Cloud Platform cockpit.

As it is possible to access the repository in SAP Web IDE, it should also be possible to access it from outside SCP. I know that the git repository is protected. Maybe the requests from git cli is blocked by SCP? After all, I was not asked to authenticate. Maybe I can force SCP to ask me for my password? Changing the URL to include my SCN user ID did just that: I was asked to provide my password.

git clone https://p539123@git.hanatrial.ondemand.com/p539123trial/cisample

SCP is now asking for my password and – magic happening – the git service is now accessible and the repo can be cloned. Would be nice if the git service would ask me to authenticate instead of failing directly.

Solving reverse proxy error ERR_CONTENT_DECODING_FAILED

Configuring a reverse proxy is not an easy task. It involves some trial and error and dealing with unexpected errors. One of those errors is ERR_CONTENT_DECODING_FAILED. The web site won’t load in your browser and Chrome will show this error message:

Error ERR_CONTENT_DECODING_FAILED may show up in your browser when a resource is configured on your reverse proxy, and the backend communication is working. That is: the backend is returning data, but not in a form the browser expects. Like: browser expects a GZIP response, but receives plain text. Therefore the hint content decoding failed. Content received, but the browser is not able to decode / understand the data.

To solve this error, reset the Accept-Encoding request header in your Reverse Proxy configuration.

Apache

RequestHeader unset Accept-Encoding

http://httpd.apache.org/docs/current/mod/mod_headers.html 

Example Apache configuration section for a location named test.

<Location /test>
  RequestHeader unset Accept-Encoding
  ProxyPass https://0.0.0.0:443
  ProxyPassReverse https://0.0.0.0:443/
  Order allow,deny
  Allow from all
</Location>

NGINX

proxy_set_header Accept-Encoding "";

http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header

Setup OpenVPN troubleshooting

While setting up OpenVPN I came accross some common errors or workarounds that make life easier. To make it easier to remember these I have documented them in this blog. Maybe they are useful for others as well.

Remove pass phrase

In case you want to remove the pass phrase from the server key to make it easiert to start the OpenVPN server part, use the following command:

mv server.key server.key.orig
openssl rsa -in server.key.orig -out server.key

You’ll have to enter one more time the pass phrase of the key, and then a new server.key file is written without the pass phrase. You can see this when looking into the key files.

With pass phrase:

Note: file starts with: BEGIN ENCRYPTED PRIVATE KEY

Without pass phrase:

Note: file starts with: BEGIN RSA PRIVATE KEY

Run OpenVPN as a service on Linux

After installing openvpn via yum on AWS AMI Linux, a service script is also installed. How the file works and can be activated is written in the file itself:

more /etc/init.d/openvpn

The file should already be copied by yum to /etc/rc.d/init.d/openvpn

Activate the service

chkconfig

Check whether or not openvpn is already configured to run as a service. For each run level, the status is either on or off. In case of on, openvpn is already configured to run as a service. In this example, opevpn is not configured to run as a service in any runlevel.

sudo chkconfig --add openvpn

sudo chkconfig openvpn on

OpenVPN will now be started as a service in the run levels 2, 3, 4 and 5. Output of openvpn is then written to /var/log/messages

sudo tail -f /var/log/messages

Systemd

To start and control openvpn via systemd. Check status of openvpn.

sudo systemctl status openvpn

Edit service configuration

sudo vim /etc/default/openvpn

Insert the client configuration to start automatically. Here, I am going to start client1.conf:

AUTOSTART=”client1”

Start service

sudo systemctl start openvpn
sudo systemctl status openvpn

Solving common OpenVPN connection error message

Some information on how to solve common OpenVPN error message on the server and client. Most occur when trying to start OpenVPN for the first time.

TA.KEY

Client starts connecting but no connection is established.

Error message

TLS Error: cannot locate HMAC in incoming packet from [AF_INET]

Cause

Server is configured to use ta.key.

Solution

Copy the ta.key into the openvpn configuration directory and specify its location in the conf file.

Cipher final failed

OpenVPN server accepts a client connection, but communication fails.

Error message

Authenticate/Decrypt packet error: cipher final failed

Cause

Server and client are using different algorithms for encryption and decryption. On the server, the log gives more information:

WARNING: 'cipher' is used inconsistently, local='cipher AES-256-CBC', remote='cipher BF-CBC'

Solution

Server uses AES-256-CBC, while the client is using BF-CBC. Adjust the client configuration in client.conf. Insert cipher AES-256-CBC in client.conf

Other parameters to adjust

During first startup, some warning message may be written on the server log. Most common they refer to link-mtu, cipher, keysize or comp-lzo.

WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1557', remote='link-mtu 1542'
WARNING: 'keysize' is used inconsistently, local='keysize 256', remote='keysize 128'
WARNING: 'comp-lzo' is present in remote config but missing in local config, remote='comp-lzo'

Solution

Adjust the parameters in the client.conf file so that they match the server configuration. Also good to check this way if a not controlled/configured client is connecting to your server.

Link-mtu

Configure the client to use the same mtu size as the server. Insert parameter link-mtu into client.conf.

link-mtu 1557

Keysize

Keysize used by client and server should be the same. Insert parameter keysize into client.conf.

keysize 256

Comp-lzo

Uncomment the parameter in server.conf.

OpenVPN connection test

After configuring and running both the OpenVPN server and client, it’s a good idea to test if the VPN is working. This involves some tests on both the server and client.

OpenVPN Server

Network Device

After the server is started, a new interface should be created. Run ifconfig to get a list of all available interfaces. In case tun is configured in the conf file as device type, a new interface with name tun0 is created.

ifconfig

Check server log for client connection

In case OpenVPN is started as a service, the log can be found at /var/log/messages. If you start it directly on the command line, the log will be shown on the shell. When a client connects, the log of the server shows the connection information.

tail -f /var/log/messages

The last lines show client1, meaning that the client not only connected, but is also correctly identified as client1. The connection is working.

OpenVPN client

Start OpenVPN and the client will try to connect to the server specified in the client.conf file. Client connecting and receiving IP.

openvpn /etc/openvpn/client.conf
tail -f /var/log/messages

After the connection was established, the client is also creating a new interface named tun0. Here a client named client1 connects and receives the IP 10.8.0.6.

ifconfig

Connection test

Easiest way to test that client and server can talk to each other is to ping both. Just run a ping from the server to the client IP, and from the client to the server IP. For this, the VPN IP address must be used (e.g. 10.8.0.x).

OpenVPN server

Ping client1 from server.

ping 10.8.0.6

OpenVPN client

Ping server from client.

ping 10.8.0.1

Setup OpenVPN client on Raspberry Pi

OpenVPN uses certificates to authenticate the server and clients. Therefore, the client needs to have a valid client certificate. This certificate needs to be issued by the CA server that also issued the certificate of the OpenVPN server. In my case, this server is installed together with the OpenVPN server on the AWS EC2 instance. The process to create the client certificate is the same as with the server certificate, only the certificate type must be client, or: TLS Web Client Authentication. This is done by specifying the client parameter in the generate certificate request command.

Depending whether or not easy-rsa or any other tool to generate a certificate request is available on the client, the request can be generated directly on the client. The vantage by creating the request on the client is that the private key will stay on the client. In my example, I’ll make use of the already available infrastructure on the OpenVPN server and generate the client request and certificate on the server and copy later the generated artifacts over to the client.

Create client certificate

Log in to the CA (OpenVPN) server and issue a client certificate request. The name of the client will be client1. Note that you can use a different name, like the FQDN of the client.

cd /etc/openvpn/easyrsa
sudo ./easyrsa gen-req client1

As with the server certificate, give a passphrase and common name.

Next: sign the client1 certificate by the CA.

sudo ./easyrsa sign-req client client1

You need to confirm the signing request by entering yes and informing the pass phrase of the CA certificate.

The client certificate is now issued.

  • Private key: easy-rsa/pki/private/client1.key
  • Public certificate: easy-rsa/pki/issued/client1.crt

Move these files to the OpenVPN client.

OpenVPN client Installation

The client going to connect to the OpenVPN server running on AWS EC2 is a Raspberry Pi. The RP uses a Debian based Linux, therefore apt is used to install software. On the RP, install OpenVPN. Easy-rsa is not needed, as the CA is running on the EC2 instance.

sudo apt-get update
sudo apt-get install openvpn

Client Certificates

Create a openvpn directory. Can be in /etc/ or in your user’s home. Put the client’s public certificate and privte key there. To use HMCA for additional security, copy the ta.key file from the server there too.

Configuration

Copy the OpenVPN sample client configuration to your openvpn directory and edit the file client.conf.

cd openvpn
cp /usr/share/doc/openvpn/examples/sample-config-files/client.conf .

Adjust the following lines to point to the correct server (AWS EC2) and local certificates and key. Example:

  • remote server.domain.com 1194
  • ca /home/tobias/openvpn/ca.crt
  • cert /home/tzobias/openvpn/client.crt
  • key /home/tobias/openvpn/client.key
  • tls-auth /home/tobias/openvpn/ta.key 1

The tls-auth parameter is needed in case the server is configured to use HCMA. The shared key ta.key from the server is needed for this to work.

Start OpenVPN client

To start the OpenVPN as client, run the executable and pass the path to the configuration file as parameter.

openvpn ./client.conf

You need to provide the pass phrase of the client1 private key.

The client will automatically connect to the OpenVPN server defined in the client.conf file (remote parameter) and the given port (1194). Make sure that on AWS EC2, this port is accessible for the client.

Result

If all works, the client connects to the server and gets an internal IP assigned.