Setup OpenVPN server on Amazon EC2

Recently I got some new hardware that I will use to run some useful software. To use the software from anywhere, I’ll need to have remote access. As I cannot do DMZ or port forwarding with my new internet provider, I decided to connect my home server using VPN to a access machine running on AWS.

The AWS EC2 Linux computer will serve as my entry point. Services running on the RP at home connected via VPN can be accessed from EC2. Other computers at my home cannot be accessed, as the IP is different and no route is configured.

This setup comes with several architectural questions to solve:

  • How to ensure the communication is secure?
  • How to guarantee the tunnel is up?
  • How to enable access from EC2 to the services running on the client?
  • The client must be assigned the same IP for the services be accessible from EC2
  • How to give access to the services from the internet?

The three top question will be answered in my next blogs about how to set up OpenVPN server and client. The first question is the easiest to answer: by using a VPN solution. I am going to use OpenVPN and this blog is about how to setup OpenVPN. I’ll cover the installation on the EC2 instance and on the Raspberry Pi, as well as the initial setup with the certificates, server and client configuration and how to connect. Starting the client and server as service keeps them running and in case the connection fails, an automatic reconnect is attempted. The EC2 instance can access the services running on the client automatically. The last two questions will be answered sometimes later.

OpenVPN Server

Install OpenVPN on EC2

The OpenVPN software is available in yum on EC2 Linux AMI. You may need to enable the REPL repository. I assume you did this already. The packages to install a openvpn and easy-rsa.

sudo yum update
sudo yum install openvpn easy-rsa

This will also install a public key to install a package and ask for your permission to do so.

The easy-rsa package is needed to set up a certificate authority. In case you do have a CA available, you can use your CA to generate the certificates used by OpenVPN. For those that do not have a CA available, take the easy-rsa functionality.

Generate CA

The command above installs easy-rsa 3.x. With 3.x, the way how to use easy-rsa and to set up a CA and issue the certificates changed. You can see in detail how to use easy-rsa 3.x at the documentation available at the GitHub project site.

OpenVPN uses certificates, and easy-rsa issues those certificates. Basically, you have two components of easy-rsa to deal with:

  • CA software
  • Certificates

Configuration of OpenVPN is put and read from /etc/openvpn. Easy-rsa software should be in a separate folder, like /home/ec2-user/easy-rsa, but to keep all in one place I’ll put easy-rsa inside the /etc/openvpn directory.

Note: for real productive usage, don’t do this. Separate easy-rsa executables and config files.

Copy easy-rsa

Copy easy-rsa to your selection location. For this, first find out where easy-rsa is installed.

repoquery -l easy-rsa

Location is /usr/share/easy-rsa/3.0.3. I’ll copy these files to /etc/openvpn/easy-rsa.

sudo mkdir /etc/openvpn/easy-rsa
sudo cp -Rv /usr/share/easy-rsa/3.0.3/* .

Start easy-rsa

Follow the steps outlined at the easy-rsa git site. For the following steps, go into the directory where easy-rsa is installed.

cd /etc/openvpn/easy-rsa

Init PKI

sudo ./easyrsa init-pki

Build CA

This will create the CA certificate to sign certificate requests. In other words: whoever gets access to the private key of the CA created in this step, can create new valid OpenVPN clients for your setup. Take care of the CA certificate and key.

sudo ./easyrsa build-ca

You’ll need to enter:

  • PEM pass phrase
  • Common Name

The passphrase is used to unlock the private key and is an additional level of security. Even when someone gets a copy of the private key of your CA, without the pass phrase the key is not usable. The common name is used to identify the CA. I used the FQDN of my web server. After execution these two commands, the CA is initialized and can be used to issue certificates.

Diffie-Hellman

Generate Diffie-Hellman parameters.

sudo ./easyrsa gen-dh

Generate OpenVPN server certificate

The OpenVPN server needs a certificate issued by the CA to identify itself against the clients. This is a nice “feature” when using PKI. Server and client can validate the other side. Both need just to trust the CA certificate for this. The difference between the two certificates (client and server) is the included type. This is done by including an additional value in the certificate specifying the type of certificate:

  • TLS Web Server Authentication for the server and
  • TLS Web Client Authentication for the client

Which kind of certificate is going to be issued is specified by the easy-rsa command when creating the certificate request.

Generate certificate request

Create a certificate request containing the identity information of the server and let this request be signed by the CA. By specifying the server parameter, the request is for a server and the CA will include the value TLS Web Server Authentication in the extension.

sudo ./easyrsa gen-req server

Inform:

  • Pass phrase
  • Common Name

As with the CA certificate, inform a pass phrase that adds additional security to the private key and a common name to uniquely identify the server. I used server as CN. Of course, it could also have been openvpn.mydomain.com or something else.

Sign request

Send the request to the CA and sign it to issue a valid certificate. With that, the CA information is added to the CA, making it official and clients that connect to OpenVPN server will know if they can trust the server. Only when trust is verified, a connection will be established between the server and client.

sudo ./easyrsa sign-req server server

You’ll need to confirm the request by typing yes and the pass phrase.

TLS-AUTH

The following certificate is needed to harden the overall security of OpenVPN. As OpenVPN is using TLS, it makes sense to add HMAC to validate integrity of the packages received. For this to work, a shared secret key is needed. This key will be written to a file named ta.key.

Generate ta.key

cd /etc/openvpn
sudo openvpn --genkey --secret ta.key
sudo mv /etc/openvpn/ta.key /etc/openvpn/easy-rsa/private

OpenVPN server configuration

Take a sample configuration file as a template. Can be found in the doc folder of openvpn. The sample configuration file for the server is server.conf, and for the client, client.conf.

ls -1 /usr/share/doc/openvpn-2.4.4/sample/sample-config-files/

Copy server.conf to /etc/openvpn and edit the file.

sudo cp /usr/share/doc/openvpn-2.4.4/sample/sample-config-files/server.conf /etc/openvpn/
sudo vim /etc/openvpn/server.conf

Adjust the path to the ca, cert, key and dh files

These parameters inform OpenVPN where the certificates and Keys are stored. The CA cert ca.crt is used to validate the client certificates. They must be issued by this CA. The server.crt and server.key are used by the OpenVPN server to encrypt traffic and authenticate itselfs against clients. Diffie hellman dh.pem is used to provide Perfect Forward Secrecy.

Start OpenVPN server

To start the OpenVPN server and to test the current setup, run the following command:

sudo openvpn /etc/openvpn/server.conf

During startup, you need to provide the passphrase of the server certificate.

If all works, OpenVPN starts without erros: Initialization Sequence Completed. After this, the server is waiting for clients to connect.

 

 

Note:

If someone is reading my blogs for the last years you may remember that I have once written about setting up OpenVPN for accessing SUP on AWS. That blog was all about Windows and is outdated. I wrote it in 2012. But, as I published it once at SAP Community Network, it is not available anymore. SAP lost it during their last migration.

Uncompressing a multi-part 7zip file in Debian

7zip is a popular compression program for Windows. It allows to effectively compress files, split them into several archives and to add protection by using a password. This all works fine if you are a Windows user. In case you now want to extract such a multi part password protected file in Linux, you’ll find out that this isn’t a standard use case. Uncompressing these files involves some work. 7zip is not made available for Linux by the developer. Gzip or zip won’t work with 7zip compressed files. But: an unofficial version is available and it is possible to extract 7zip files in Debian/Linux.

You have some options available for installing 7zip for Debian, like apt or by compilation. The version you get with apt is quite old: 9.2. In case the version of 7zip used to compress the file on Windows is higher than the one available for Debian, uncompressing may not work. An algorithm may be used that is not available on the lower version. In that case, 7zr will exit with an error and showing Unsupported Method.

Compilation from source

This option will give you the latest available version of 7zip for Linux. Especially useful when you try to unzip a file and get the message: Unsupported Method. To solve this, try to install a higher version of p7zip by downloading the source and compile p7zip.

Get the latest version of p7zip from SourceForge. Unzip it and then run make. After the compilation is done, you’ll have the executable 7za in the bin folder. This version should be able to work with files compressed by 7zip for Windows. Make sure to read the README.

Copy the correct makefile. 7zip provides several makefiles, for each target platform / architecture. In case of Linux, the default one should work. To start compilation, a simple make is sufficient.

make

This gives you the binary ./bin/7za

Unzip a file multi-part password protected file.

7za x h1.7z

APT

Install the 7zip program for Debian. This installs version 9.2.

sudo apt-get install p7zip

Let’s say we have 1 file that was zipped to file h1.7z using 7zip and splitter into 650 MB. 7zip produces 2 archives:

  • h1.7z.001
  • h1.7z.002

To list the archive:

7zr l h1.7z.001 -tsplit

We can see that the split archives contain one file named h1.7z. That is the zip file created by 7zip under Windows.

To unzip the file, use

7zr x h1.7z.001 -tsplit

A start job is running for dev-disk-by

Recently I restarted one of my Linux servers and the computer did not start up as expected. No external accessible service was running, like apache or SSH. This made the computer inaccessible from remote and left me in the dark. After a while, the server responded to ping, but nothing more.

After I connected the server to a display and keyboard, I could see the error message: “a start job is running for dev-disk-by […]”. After that, Linux gave me only the option to log on in rescue mode or to restart the system. A restart didn`t help. I checked the internet and found out that the message can be caused by a fstab entry.

Looking at the content of my /etc/fstab file I could see an old entry I once created for a test and never maintained (aka deleted). The system is trying to mount a partition that was not available / broken and the system stopped.

UUID=a6674495 -4249-9696-0d9c83 /mnt/disk btrfs defaults,noatime,auto 0 0

I commented out this line in fstab and restarted the system. Now the system was restarted correctly and all the services came up again.

Install SMP3 with Oracle DB

The following procedure for installing SMP3 with an Oracle DB is for Linux. For tests, you can use Oracle Express. Check your environment/company if you can use that version.

Prerequisites

Ensure that Oracle XE is up and running. It is important that the tnslistener is working! Run the listener and check the status:

/u01/app/oracle/product/11.2.0/xe/bin/lsnrctl status

Configure installation parameters

The steps are documented at SAP Help. You’ll have to edit the SilentInstall_Linux.txt file and adjust the installation parameters.

vim SilentInstall_Linux.txt

For Oracle, you’ll need to change these parameters (at the end, you’ll find a complete example file):

Activate that SMP3 uses an external DB

-V developerInstall="false"
-V productionInstall="true"
-V sqlaEmbeddedDB="false"
-V existDB="true"

Inform the Oracle XE connection parameters

-V existDBType="oracle-sid"
-V dbHostName="localhost"
-V dbPortNumber="1521"
-V dbLogin="gomobile"
-V dbPassword="secret"
-V dbDBName="XE"

Inform the JDBC driver location

-V jdbcDriver="/u01/app/oracle/product/11.2.0/xe/jdbc/lib/ojdbc6.jar"

Prepare Oracle DB

Form the above connection parameters you can see that SMP3 is going to use the user gomobile with the password secret to connect itself to Oracle XE. This means that the user with the password and a schema must be created in the DB. SMP3 comes with a SQL script for Oracle that does exactly that. The script is located at /db_tools/db/oracle/smp3/sql. The file is 001_SMP3_drop_and_create_user.DDL The file contains the SQL statements to create the user with the right permissions:

CREATE ROLE SY365_OBJOWNER;
GRANT CREATE SEQUENCE TO SY365_OBJOWNER;
GRANT CREATE SESSION TO SY365_OBJOWNER;
GRANT CREATE SYNONYM to SY365_OBJOWNER;
GRANT CREATE TABLE TO SY365_OBJOWNER;
GRANT CREATE VIEW TO SY365_OBJOWNER;
GRANT CREATE PROCEDURE TO SY365_OBJOWNER;
GRANT CREATE SEQUENCE TO SY365_OBJOWNER;
GRANT CREATE TRIGGER TO SY365_OBJOWNER;
GRANT CREATE INDEXTYPE TO SY365_OBJOWNER;
DROP USER GOMOBILE CASCADE;
CREATE USER GOMOBILE
IDENTIFIED BY secret
DEFAULT TABLESPACE USERS
TEMPORARY TABLESPACE TEMP
PROFILE DEFAULT
ACCOUNT UNLOCK;
-- 2 Roles for GOMOBILE
GRANT SY365_OBJOWNER TO GOMOBILE;
GRANT CREATE SESSION TO GOMOBILE;
GRANT CONNECT TO GOMOBILE;
ALTER USER GOMOBILE DEFAULT ROLE ALL;
-- 1 Tablespace Quota for GOMOBILE
ALTER USER GOMOBILE QUOTA UNLIMITED ON USERS;

You’ll have to add the command EXIT; at the end of the file

To run the SQL script, run:

sqlplus system/Sap123 @001_SMP3_drop_and_create_user.DDL > smp3.log
  • Note: Sap123 is the password for the user system.

Output is written to smp3.log

SQL*Plus: Release 11.2.0.2.0 Production on Wed Aug 24 21:37:08 2016
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production
Role created.
Grant succeeded.
[…]
DROP USER GOMOBILE CASCADE
ERROR at line 1:
ORA-01918: user 'GOMOBILE' does not exist
User created.
Grant succeeded.
[…]
User altered.
User altered.

The error regarding DROP user is normal, as the user gomobile hasn’t been created before, so there is no user to drop.

Run installer

With the above steps done, SMP3 installer is ready to be run.

./SilentInstall_Linux.sh

The output will contain information regarding the Oracle DB:

dbg, existDBType:oracle-sid
WARNING: Selecting this option confirms SMP database is already created
dbg, jdbcDriver: /u01/app/oracle/product/11.2.0/xe/jdbc/lib/ojdbc6.jar
dbg, jdbcDriver fullFileName: /u01/app/oracle/product/11.2.0/xe/jdbc/lib/ojdbc6.jar
dbg, jdbcDriverFile: /sap/SAP/MobilePlatform3/Util/ojdbc6.jar
dbg, ojdbc6.jar will be renamed to ojdbc.jar in the installation
dbg, queryExit:oracle-sid localhost gomobile [pwd entered] 1521 XE
dbg, Ping succcesful: 0
dbg, smpDataExists:false
dbg, New node install

If everything works fine, you’ll get a confirmation message at the end of the installation.

Installation Successful

Validation

SAP Help contains some information on how to validate the installation. You can search for error message in the installation log, but when an error occurs, normally the installer stops. My preferred way to check SMP3 is to start the server and see if I can log on, create apps, etc. Base test is therefore to start SMP3 and to log on.

Install Oracle Express 11G R2 on CentOS 7

Preparations

Install some additional packages via yum to ensure that the installation and execution of the database will work. The list may differ, depending on the actual version of CentOS you are using, but the internet gave me back the following packages and you should be on the safe side.

yum update
yum install unzip libaio bc flex

Download

Before using the express edition, make yourself familiar with the license and usage restriction this edition is shipped with. If it still fits your needs, be aware that CentOS is not on the list of officially supported Linux distributions. You are on your own. Download Oracle Express 11G R2 from Oracle. It`s a 308MB file.

The downloaded file is a zipped RPM package; first step is to unzip the file.

unzip oracle-xe-11.2.0-1.0.x86_64.rpm.zip

This gives you a new directory called Disk1. This folder contains the installation RPM.

Installation

Go to the folder and install the RPM via rpm tool.

cd Disk1
rpm -ivh oracle-xe-11.2.0-1.0.x86_64.rpm

After the installation, you`ll be prompted to configure the database.

Configuration

Run the tool /etc/init.d/oracle-xe to configure the database.

/etc/init.d/oracle-xe configure

Configure the port

Specify the port of the listener

Inform the system user password. Be sure to note this down somewhere or to really remember it!!!

Specify if you want the database to be started at boot time.

The configuration should now start automatically and only take a few minutes to complete.

The database is install at /u01/app/oracle/product/11.2.0/xe/

The oracle_env script is in the folder bin.

Oracle Express 11G R2 is also started and using ps –ef you can see the processes running.

That`s it, Oracle Express 11G R2 is now installed on CentOS.

Test available internet bandwidth under Linux

It’s 2015 and somehow I think that is a reason to simply have enough bandwidth for upload and download to help me with my digital life style. Cloud is in, and accessing files from everywhere is tempting. As cloud only means to be able to access data from any internet connection, I can host the data also at home. Only thing needed for this is a upload bandwidth that is not too slow. But what is the upload and download rate available in my data plan? I am a NET customer and after a few upgrades over the years I am not at 15 Mbps. This should give me 2 Mbps upload. How to measure this under Linux?

http://www.netcombo.com.br/internet-2

I found an easy tool to measure upload and download bandwidth: speedtest. It is not part of my Raspberry PI Debian Linux OS, so I had to install it first. After this I am able to check the upload rate and it comes close to the announced 2 Mbps.

Download and run

First step is to download speedtest.

Command:
wget -O speedtest-cli https://raw.github.com/sivel/speedtest-cli/master/speedtest_cli.py

Next is to make the file executable.

Command:
chmod +x speedtest-cli

Final step is to run the tool. It will automatically test the upload and download bandwidth.

Command: ./speedtest-cli

Result:

Download bandwidth is reprted at 14.5 Mbit and upload at 1.86 Mbit.

Is this enough to stream for instance audio? Command bit rate for MP3 is 128, 192 or 256 Kbps. That is Kilobits per second. 2 Mbps is 2 Mbit == 2000 Kilobits. This is way above 192 Kbps. Streaming audio should not be a problem.

Freeing up space

I like VMs. Sure, everything should be cloud, and AWS, Azure, etc are great (if you can afford them). But nothing beats having a local VM running with 6GB RAM even when you are not online. Cost? 0 $. I am just electively using the laptop I already use. Put a VM on an external USB drive is not a problem. With USB 3.0 the performance is quite good, even with USB 2.0 you can run a VM; starting and stopping takes a while, but once the services are started, they work quite nicely.

To not occupy all the disk space at once, my VMs are configured to dynamically allocate space. This allows me to run VMs from the SSD of the laptop, speeding things up even more. A normal VM takes only a few GB: the OS + software.

A problem is the ext4 file system used by my Linux VMs. Once a space was allocated, it stays allocated. The effect is that a VM is occupying internally 30 GB consumes 100 GB on my hard drive. Why? Adding and deleting files, caches, etc. The DB may take only 20 GB, but adding and deleting 10GB results that the VM occupies 30 GB. Do that for some time and suddenly the VM eats up 100 GB of your hard drive instead of 30GB.

How to regain hard drive space?

VMWare comes with a compact tool to recover space, but that only works ootb with Windows VMs. For Linux VMs with a journaling file system like ext4 some preparations are needed. In short, the preparation is to create huge empty file that only contains 0.

First, stop the running services. The file going to be created will consume the free space of the virtual disk. Running services will get into trouble when they cannot write anymore to the disk. Like log files or real data. To not let your database or Tomcat server crash uncontrolled, stop them before creating the file.

Command to create the file:

dd if=/dev/zero of=/dummy bs=4096

This command lets dd create a file named dummy in /. The file size will be the entire free space of the virtual disk. If you configured your virtual disk to be 500GB and you have 400GB free, the resulting file size will be 400GB. You do not need to have 400GB free on your physical disk where the VM is stored on. As you write zeros, the real space needed will be 0 MB.

After dd filled you file system, it will exit. Delete the dummy file

rm /dummy

Shut down your VM. Open VM Player, open the properties of the VM, select the hard disk you want to compact and start the Compact tool from the Utilities drop down.

This will start the compact tool. After a while you’ll see a success message, informing you that the disk was compacted.