I have a RAID5 consisting of three 10TB HDDs. This RAID5 has a total capacity of 20 TB.
I bought a new 10 TB HDD that I want to use to extend the RAID5: 4 HDDs with a total capacity of 30 TB. The file system on md0 is ext4. Currently, the RAID5 disks are sdc1, sdf1 and sde1. The additional disk is sdd1.
The RAID5 is formatted with ext4 and available as md0.
Prepare new disk
Add disk to RAID
Extend ext4 files system.
Prepare new disk
First start with the preparation of the new disk. The disk is /dev/sdd and needs to have a partition. I use parted for this. First, create a label of type gpt.
parted -s -a optimal /dev/sdd mklabel gpt
Next is to create the partition using parted. This time, I am using the interface.
Add disk to RAID
The RAID is a software RAID on Linux, therefore mdadm is used to control the raid. To add a new disk, option –add is used and the raid and new disk are passed as parameters.
mdadm --add /dev/md0 /dev/sdd1
The result of the operation can be seen in mdstat.
The new disk is added as a spare device. The (S) behind sdd1 means spare device. In case a device would fail, the spare device will take over automatically and a RAID rebuild will be triggered. This gives me less trouble in case a device fails, as I won’t have to do anything, but it won’t give me more space. The RAID5 is still at 20 TB.
To make the RAID5 aware of the new disk and that it should be used for data storage, the RAID must be informed to use the new HDD using the grow command.
mdadm --grow --raid-devices=4 /dev/md0
The command informs the RAID that there are now 4 HDDs to be used, instead of 3. This command will trigger a RAID rebuild, as the information must be distributed to the HDDs.
The RAID5 consists now of 4 HDD, all working [UUUU]. The size of the RAID is still 20 TB. This is because the md0 has capacity of 30 TB, but the ext4 filesystem is still configured to make use of 20 TB.
Resize ext4 filesystem
To be able to use the 30TB available on the RAID5, you need to resize the file system. First, run an integrity check.
e2fsck -f /dev/md0
After the e2fsck ended without errors, the file system can be extended. This is done by using the tool resize2fs.
After resize2fs completes (can take a while), the size available is now 30TB:
Running a server allows you to do a lot of stuff from remote. Copying files is one of those tasks you can do from anywhere on the world while being logged on via SSH. For this task it is good to know the speed of read/write to get an idea if it’s working s expected. When sitting in front of your computer, you can see if a HDD is working, in Windows you see a MB/s indication, and in Linux? Not all copy commands show you the transfer rate by standard. Some disk intensive tasks won’t at all (RAID sync).
To monitor disk activities in Linux, several tools are available. One is iostat.
To install iostat in Debian, you must install the package sysstat
apt-get install sysstat
To run iostat, just enter iostat in the shell.
The output will list the captured read / write speed of the available devices. To get a continuous output of the disk activites, run iostat -y 1. This will update the output every second until you end the program.
iostat -y 1
Several options are available to control the output. To get the disk read / write in Mb and not in kB, add the -m flag
iostat -y 1 -m
Using iostat you can see the throughput oft he disks, even when you are running “hidden” tasks like a RAID sync or copy process in another session (screen).
When you add a new HDD to an existing RAID, a sync is started. In my case I added a 10TB disk to a RAID5. The sync started and as estimated time I got something in the range of days. The estimated time is listed in finish=5384 min.
This number goes up and down a little bit, but overall result is that the sync will need days. After checking the status again after a while, it still showed days: finish=3437min.
The main problem here Is the rate at which mdadm can sync the data. The value is between 30000K and 43000K. That’s not much given the size of the RAID. There are several tips available on the internet. What help me was to set the stripe_cache_size.
You set the size of stripe_cache_size for each RAID device (mdX). In case your RAID is md0:
echo 32768 > /sys/block/md0/md/stripe_cache_size
The speed increased to 100000K/sec. That’s close to 3x faster than before. Time went down drastically.
My internet provider is Unitymedia. Their default router comes with a DHCP server. Honestly, it’s one of the worst products I ever had to work with. My private network is 192.168.0.x. The DHCP server of the Unitymedia box is distributing from time to time leases for 192.168.192.x. Changing my private network to 192.168.192.x one is not working, as then the DHCP server picks another address range. Advise from Unitymedia help desk was to reboot the box, which, of course, won’t solve the problem. Because of this error, some of my devices are in a different network: Chromecast won’t work, broken internet connection on smartphones, etc.
I do have a Raspberry Pi (RP) in 24/7 use. My idea is to run my own DHCP server on the RP. This not only solves the DHCP problem, but also gives me more control over the DHCP configuration.
This installs ISC DHCP server. As you can see in the output, starting the DHCP server failed.
sudo systemctl status isc-dhcp-server.service
The error is simply caused because the DHCP server is not configured. Let’s change that.
Several parameters must be activated and configured.
sudo vim /etc/dhcp/dhcpd.conf
Activate DHCP server
# If this DHCP server is the official DHCP server for the local
# network, the authoritative directive should be uncommented.
This configures what IP address are going to be distributed. My private network is 192.168.0.x with the router on 192.168.0.1. As DNS you can use whatever you want, as an example I am using Google DNS servers.
This will give DHCP clients an IP address between .150 and .240, with router .1, Google DNS and sets the domain name to my own.
Deactivate old DHCP server
To not have the DHCP server provided by Unitymedia box still issuing wrong IP address, I am going to deactivate the service via the web interface.
Start DHCP server
After installing and configuring the new DHCP server on RP and deactivating the one from the router box, it’s time to start the new DHCP server.
To see if a IP address is assigned, use this command:
sudo systemctl status isc-dhcp-server.service
Putting my Android device into flight mode and back makes it connect to Wifi again and obtain a new IP address via DHCP. In the DHCP status log, I can see the DHCPDISCOVER from the Android device and that it got the IP address 192.168.0.150 assigned.
As my Mac always got the wrong IP assigned, I changed it to manual configuration. Change the mode to DHCP, apply and deactivate / activating Wifi.
And my soundbar that got a strange IP address assigned by the Unitymedia router box? Works too!
Chromcast streaming shows the SoundBar is now in the same network.
Recently I had the problem that running apt-get update stalled while trying to connect to an IPv6 address. For instance, on a Raspberry Pi, the update process stalls while trying to connect to archive.raspberrypi.org. All other connections worked fine. Looking at the console output, a difference was that apt was trying to connect to an IPv6 address.
The problem was caused by:
100% [Connecting to archive.raspberrypi.org (2a00:1098:0:80:1000:13:0:8)]
A quick internet search showed that you can force apt to not use IPv6 and only IPv4. As the download worked for IPv4, this seems like a reasonable workaround.
You can pass a parameter to disable IPv4 to apt-get, or write it to apt config file to make it persistent.
Create a new configuration file. This makes it easy for you to keep the change during updates and to know that you configured this.
sudo vim /etc/apt/apt.conf.d/99disable-ipv6
Insert Acquire::ForceIPv4 "true";
To disable IPv6 just once while calling apt, the parameter is Acquire::ForceIPv4=true.
sudo apt-get -o Acquire::ForceIPv4=true update
Loading the package data from archive raspberrypi.org is now ignored and apt-get update works again.
Recently I got some new hardware that I will use to run some useful software. To use the software from anywhere, I’ll need to have remote access. As I cannot do DMZ or port forwarding with my new internet provider, I decided to connect my home server using VPN to a access machine running on AWS.
The AWS EC2 Linux computer will serve as my entry point. Services running on the RP at home connected via VPN can be accessed from EC2. Other computers at my home cannot be accessed, as the IP is different and no route is configured.
This setup comes with several architectural questions to solve:
How to ensure the communication is secure?
How to guarantee the tunnel is up?
How to enable access from EC2 to the services running on the client?
The client must be assigned the same IP for the services be accessible from EC2
How to give access to the services from the internet?
The three top question will be answered in my next blogs about how to set up OpenVPN server and client. The first question is the easiest to answer: by using a VPN solution. I am going to use OpenVPN and this blog is about how to setup OpenVPN. I’ll cover the installation on the EC2 instance and on the Raspberry Pi, as well as the initial setup with the certificates, server and client configuration and how to connect. Starting the client and server as service keeps them running and in case the connection fails, an automatic reconnect is attempted. The EC2 instance can access the services running on the client automatically. The last two questions will be answered sometimes later.
Install OpenVPN on EC2
The OpenVPN software is available in yum on EC2 Linux AMI. You may need to enable the REPL repository. I assume you did this already. The packages to install a openvpn and easy-rsa.
sudo yum update
sudo yum install openvpn easy-rsa
This will also install a public key to install a package and ask for your permission to do so.
The easy-rsa package is needed to set up a certificate authority. In case you do have a CA available, you can use your CA to generate the certificates used by OpenVPN. For those that do not have a CA available, take the easy-rsa functionality.
The command above installs easy-rsa 3.x. With 3.x, the way how to use easy-rsa and to set up a CA and issue the certificates changed. You can see in detail how to use easy-rsa 3.x at the documentation available at the GitHub project site.
OpenVPN uses certificates, and easy-rsa issues those certificates. Basically, you have two components of easy-rsa to deal with:
Configuration of OpenVPN is put and read from /etc/openvpn. Easy-rsa software should be in a separate folder, like /home/ec2-user/easy-rsa, but to keep all in one place I’ll put easy-rsa inside the /etc/openvpn directory.
Note: for real productive usage, don’t do this. Separate easy-rsa executables and config files.
Copy easy-rsa to your selection location. For this, first find out where easy-rsa is installed.
repoquery -l easy-rsa
Location is /usr/share/easy-rsa/3.0.3. I’ll copy these files to /etc/openvpn/easy-rsa.
Follow the steps outlined at the easy-rsa git site. For the following steps, go into the directory where easy-rsa is installed.
sudo ./easyrsa init-pki
This will create the CA certificate to sign certificate requests. In other words: whoever gets access to the private key of the CA created in this step, can create new valid OpenVPN clients for your setup. Take care of the CA certificate and key.
sudo ./easyrsa build-ca
You’ll need to enter:
PEM pass phrase
The passphrase is used to unlock the private key and is an additional level of security. Even when someone gets a copy of the private key of your CA, without the pass phrase the key is not usable. The common name is used to identify the CA. I used the FQDN of my web server. After execution these two commands, the CA is initialized and can be used to issue certificates.
Generate Diffie-Hellman parameters.
sudo ./easyrsa gen-dh
Generate OpenVPN server certificate
The OpenVPN server needs a certificate issued by the CA to identify itself against the clients. This is a nice “feature” when using PKI. Server and client can validate the other side. Both need just to trust the CA certificate for this. The difference between the two certificates (client and server) is the included type. This is done by including an additional value in the certificate specifying the type of certificate:
TLS Web Server Authentication for the server and
TLS Web Client Authentication for the client
Which kind of certificate is going to be issued is specified by the easy-rsa command when creating the certificate request.
Generate certificate request
Create a certificate request containing the identity information of the server and let this request be signed by the CA. By specifying the server parameter, the request is for a server and the CA will include the value TLS Web Server Authentication in the extension.
sudo ./easyrsa gen-req server
As with the CA certificate, inform a pass phrase that adds additional security to the private key and a common name to uniquely identify the server. I used server as CN. Of course, it could also have been openvpn.mydomain.com or something else.
Send the request to the CA and sign it to issue a valid certificate. With that, the CA information is added to the CA, making it official and clients that connect to OpenVPN server will know if they can trust the server. Only when trust is verified, a connection will be established between the server and client.
sudo ./easyrsa sign-req server server
You’ll need to confirm the request by typing yes and the pass phrase.
The following certificate is needed to harden the overall security of OpenVPN. As OpenVPN is using TLS, it makes sense to add HMAC to validate integrity of the packages received. For this to work, a shared secret key is needed. This key will be written to a file named ta.key.
Take a sample configuration file as a template. Can be found in the doc folder of openvpn. The sample configuration file for the server is server.conf, and for the client, client.conf.
ls -1 /usr/share/doc/openvpn-2.4.4/sample/sample-config-files/
Copy server.conf to /etc/openvpn and edit the file.
sudo cp /usr/share/doc/openvpn-2.4.4/sample/sample-config-files/server.conf /etc/openvpn/
sudo vim /etc/openvpn/server.conf
Adjust the path to the ca, cert, key and dh files
These parameters inform OpenVPN where the certificates and Keys are stored. The CA cert ca.crt is used to validate the client certificates. They must be issued by this CA. The server.crt and server.key are used by the OpenVPN server to encrypt traffic and authenticate itselfs against clients. Diffie hellman dh.pem is used to provide Perfect Forward Secrecy.
Start OpenVPN server
To start the OpenVPN server and to test the current setup, run the following command:
sudo openvpn /etc/openvpn/server.conf
During startup, you need to provide the passphrase of the server certificate.
If all works, OpenVPN starts without erros: Initialization Sequence Completed. After this, the server is waiting for clients to connect.
If someone is reading my blogs for the last years you may remember that I have once written about setting up OpenVPN for accessing SUP on AWS. That blog was all about Windows and is outdated. I wrote it in 2012. But, as I published it once at SAP Community Network, it is not available anymore. SAP lost it during their last migration.
7zip is a popular compression program for Windows. It allows to effectively compress files, split them into several archives and to add protection by using a password. This all works fine if you are a Windows user. In case you now want to extract such a multi part password protected file in Linux, you’ll find out that this isn’t a standard use case. Uncompressing these files involves some work. 7zip is not made available for Linux by the developer. Gzip or zip won’t work with 7zip compressed files. But: an unofficial version is available and it is possible to extract 7zip files in Debian/Linux.
You have some options available for installing 7zip for Debian, like apt or by compilation. The version you get with apt is quite old: 9.2. In case the version of 7zip used to compress the file on Windows is higher than the one available for Debian, uncompressing may not work. An algorithm may be used that is not available on the lower version. In that case, 7zr will exit with an error and showing Unsupported Method.
Compilation from source
This option will give you the latest available version of 7zip for Linux. Especially useful when you try to unzip a file and get the message: Unsupported Method. To solve this, try to install a higher version of p7zip by downloading the source and compile p7zip.
Get the latest version of p7zip from SourceForge. Unzip it and then run make. After the compilation is done, you’ll have the executable 7za in the bin folder. This version should be able to work with files compressed by 7zip for Windows. Make sure to read the README.
Copy the correct makefile. 7zip provides several makefiles, for each target platform / architecture. In case of Linux, the default one should work. To start compilation, a simple make is sufficient.
This gives you the binary ./bin/7za
Unzip a file multi-part password protected file.
7za x h1.7z
Install the 7zip program for Debian. This installs version 9.2.
sudo apt-get install p7zip
Let’s say we have 1 file that was zipped to file h1.7z using 7zip and splitter into 650 MB. 7zip produces 2 archives:
To list the archive:
7zr l h1.7z.001 -tsplit
We can see that the split archives contain one file named h1.7z. That is the zip file created by 7zip under Windows.
Recently I restarted one of my Linux servers and the computer did not start up as expected. No external accessible service was running, like apache or SSH. This made the computer inaccessible from remote and left me in the dark. After a while, the server responded to ping, but nothing more.
After I connected the server to a display and keyboard, I could see the error message: “a start job is running for dev-disk-by […]”. After that, Linux gave me only the option to log on in rescue mode or to restart the system. A restart didn`t help. I checked the internet and found out that the message can be caused by a fstab entry.
Looking at the content of my /etc/fstab file I could see an old entry I once created for a test and never maintained (aka deleted). The system is trying to mount a partition that was not available / broken and the system stopped.
Form the above connection parameters you can see that SMP3 is going to use the user gomobile with the password secret to connect itself to Oracle XE. This means that the user with the password and a schema must be created in the DB. SMP3 comes with a SQL script for Oracle that does exactly that. The script is located at /db_tools/db/oracle/smp3/sql. The file is 001_SMP3_drop_and_create_user.DDL The file contains the SQL statements to create the user with the right permissions:
CREATE ROLE SY365_OBJOWNER;
GRANT CREATE SEQUENCE TO SY365_OBJOWNER;
GRANT CREATE SESSION TO SY365_OBJOWNER;
GRANT CREATE SYNONYM to SY365_OBJOWNER;
GRANT CREATE TABLE TO SY365_OBJOWNER;
GRANT CREATE VIEW TO SY365_OBJOWNER;
GRANT CREATE PROCEDURE TO SY365_OBJOWNER;
GRANT CREATE SEQUENCE TO SY365_OBJOWNER;
GRANT CREATE TRIGGER TO SY365_OBJOWNER;
GRANT CREATE INDEXTYPE TO SY365_OBJOWNER;
DROP USER GOMOBILE CASCADE;
CREATE USER GOMOBILE
IDENTIFIED BY secret
DEFAULT TABLESPACE USERS
TEMPORARY TABLESPACE TEMP
-- 2 Roles for GOMOBILE
GRANT SY365_OBJOWNER TO GOMOBILE;
GRANT CREATE SESSION TO GOMOBILE;
GRANT CONNECT TO GOMOBILE;
ALTER USER GOMOBILE DEFAULT ROLE ALL;
-- 1 Tablespace Quota for GOMOBILE
ALTER USER GOMOBILE QUOTA UNLIMITED ON USERS;
You’ll have to add the command EXIT; at the end of the file
SQL*Plus: Release 220.127.116.11.0 Production on Wed Aug 24 21:37:08 2016
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Oracle Database 11g Express Edition Release 18.104.22.168.0 - 64bit Production
DROP USER GOMOBILE CASCADE
ERROR at line 1:
ORA-01918: user 'GOMOBILE' does not exist
The error regarding DROP user is normal, as the user gomobile hasn’t been created before, so there is no user to drop.
Install some additional packages via yum to ensure that the installation and execution of the database will work. The list may differ, depending on the actual version of CentOS you are using, but the internet gave me back the following packages and you should be on the safe side.