Running a server allows you to do a lot of stuff from remote. Copying files is one of those tasks you can do from anywhere on the world while being logged on via SSH. For this task it is good to know the speed of read/write to get an idea if it’s working s expected. When sitting in front of your computer, you can see if a HDD is working, in Windows you see a MB/s indication, and in Linux? Not all copy commands show you the transfer rate by standard. Some disk intensive tasks won’t at all (RAID sync).
To monitor disk activities in Linux, several tools are available. One is iostat.
To install iostat in Debian, you must install the package sysstat
apt-get install sysstat
To run iostat, just enter iostat in the shell.
The output will list the captured read / write speed of the available devices. To get a continuous output of the disk activites, run iostat -y 1. This will update the output every second until you end the program.
iostat -y 1
Several options are available to control the output. To get the disk read / write in Mb and not in kB, add the -m flag
iostat -y 1 -m
Using iostat you can see the throughput oft he disks, even when you are running “hidden” tasks like a RAID sync or copy process in another session (screen).
When you add a new HDD to an existing RAID, a sync is started. In my case I added a 10TB disk to a RAID5. The sync started and as estimated time I got something in the range of days. The estimated time is listed in finish=5384 min.
This number goes up and down a little bit, but overall result is that the sync will need days. After checking the status again after a while, it still showed days: finish=3437min.
The main problem here Is the rate at which mdadm can sync the data. The value is between 30000K and 43000K. That’s not much given the size of the RAID. There are several tips available on the internet. What help me was to set the stripe_cache_size.
You set the size of stripe_cache_size for each RAID device (mdX). In case your RAID is md0:
echo 32768 > /sys/block/md0/md/stripe_cache_size
The speed increased to 100000K/sec. That’s close to 3x faster than before. Time went down drastically.
My internet provider is Unitymedia. Their default router comes with a DHCP server. Honestly, it’s one of the worst products I ever had to work with. My private network is 192.168.0.x. The DHCP server of the Unitymedia box is distributing from time to time leases for 192.168.192.x. Changing my private network to 192.168.192.x one is not working, as then the DHCP server picks another address range. Advise from Unitymedia help desk was to reboot the box, which, of course, won’t solve the problem. Because of this error, some of my devices are in a different network: Chromecast won’t work, broken internet connection on smartphones, etc.
I do have a Raspberry Pi (RP) in 24/7 use. My idea is to run my own DHCP server on the RP. This not only solves the DHCP problem, but also gives me more control over the DHCP configuration.
This installs ISC DHCP server. As you can see in the output, starting the DHCP server failed.
sudo systemctl status isc-dhcp-server.service
The error is simply caused because the DHCP server is not configured. Let’s change that.
Several parameters must be activated and configured.
sudo vim /etc/dhcp/dhcpd.conf
Activate DHCP server
# If this DHCP server is the official DHCP server for the local
# network, the authoritative directive should be uncommented.
This configures what IP address are going to be distributed. My private network is 192.168.0.x with the router on 192.168.0.1. As DNS you can use whatever you want, as an example I am using Google DNS servers.
This will give DHCP clients an IP address between .150 and .240, with router .1, Google DNS and sets the domain name to my own.
Deactivate old DHCP server
To not have the DHCP server provided by Unitymedia box still issuing wrong IP address, I am going to deactivate the service via the web interface.
Start DHCP server
After installing and configuring the new DHCP server on RP and deactivating the one from the router box, it’s time to start the new DHCP server.
To see if a IP address is assigned, use this command:
sudo systemctl status isc-dhcp-server.service
Putting my Android device into flight mode and back makes it connect to Wifi again and obtain a new IP address via DHCP. In the DHCP status log, I can see the DHCPDISCOVER from the Android device and that it got the IP address 192.168.0.150 assigned.
As my Mac always got the wrong IP assigned, I changed it to manual configuration. Change the mode to DHCP, apply and deactivate / activating Wifi.
And my soundbar that got a strange IP address assigned by the Unitymedia router box? Works too!
Chromcast streaming shows the SoundBar is now in the same network.
Recently I had the problem that running apt-get update stalled while trying to connect to an IPv6 address. For instance, on a Raspberry Pi, the update process stalls while trying to connect to archive.raspberrypi.org. All other connections worked fine. Looking at the console output, a difference was that apt was trying to connect to an IPv6 address.
The problem was caused by:
100% [Connecting to archive.raspberrypi.org (2a00:1098:0:80:1000:13:0:8)]
A quick internet search showed that you can force apt to not use IPv6 and only IPv4. As the download worked for IPv4, this seems like a reasonable workaround.
You can pass a parameter to disable IPv4 to apt-get, or write it to apt config file to make it persistent.
Create a new configuration file. This makes it easy for you to keep the change during updates and to know that you configured this.
sudo vim /etc/apt/apt.conf.d/99disable-ipv6
Insert Acquire::ForceIPv4 "true";
To disable IPv6 just once while calling apt, the parameter is Acquire::ForceIPv4=true.
sudo apt-get -o Acquire::ForceIPv4=true update
Loading the package data from archive raspberrypi.org is now ignored and apt-get update works again.
The example used in this blog is a CIFS share from a Samba server running on a Raspberry Pi mounted on demand by a client running Debian.
The goal is to have a share on a client that is dynamically mounted. The share should only be mounted when an app needs to access.
In my case I do have a server with a data storage share connected. The storage is made available to clients in the network. To not have the share being mounted by the clients all the time, the share should be mounted only when real demand exists. For instance, an app needs to read data from the share. If the share is not needed by the client, it should not be mounted.
To understand the scenario better, take a look at the picture below. The process can be separated into 4 steps.
Step 1: the client is configured, but the share is inactive.
Step 2: An app is accessing the share. This can be an app as simple as ls /mnt/share
Step 3: The client is connecting to the server and mounting the share to the local mount point /mnt/share. The data is now available to the app.
Step 4: The app is not using the data from the share any longer. After a timeout, the client disconnects the share.
The example is using for the server a Raspberry Pi with Raspbian and for the client a Debian based system (Proxmox). As share type, CIFS is used. On the client, Samba is running and configured to give access to a named user to the data storage.
Install Samba and configure access for a named user. This is not part of this blog.
Autofs is the package and tool taking of mounting shares automatically. Install it using apt-get.
apt-get install autofs
Configuration of autofs
Autofs needs to be configured. To make it easier, the packages comes with templates. I am going to use the autofs master template as my starting point. Take a look at the master template as it contains an explanation of what is needed.
To add an auto mount share a new line must be added to the file. The format is: mount-point file options.
Before adding the line, you first must understand how the template and autofs works and what you want to achieve. First parameter is for the local mount point. The directory informed here is the parent. The actual shares mounted are created as sub-folders in that directory. For instance, if you choose /mnt/server and the remote share is data, the final mount point will be /mnt/server/data. I am going to mount the remote share to /mnt/server.
The seconds parameter is for the configuration file for that mount point. The third parameter specifies how autofs is treating the mount point. To unmount the share after 1 minute of inactivity, use option –timeout=60. The ghost option will create the subfolder even in case the server is not reachable.
Edit master template
Add a new configuration line for mounting the server share
7zip is a popular compression program for Windows. It allows to effectively compress files, split them into several archives and to add protection by using a password. This all works fine if you are a Windows user. In case you now want to extract such a multi part password protected file in Linux, you’ll find out that this isn’t a standard use case. Uncompressing these files involves some work. 7zip is not made available for Linux by the developer. Gzip or zip won’t work with 7zip compressed files. But: an unofficial version is available and it is possible to extract 7zip files in Debian/Linux.
You have some options available for installing 7zip for Debian, like apt or by compilation. The version you get with apt is quite old: 9.2. In case the version of 7zip used to compress the file on Windows is higher than the one available for Debian, uncompressing may not work. An algorithm may be used that is not available on the lower version. In that case, 7zr will exit with an error and showing Unsupported Method.
Compilation from source
This option will give you the latest available version of 7zip for Linux. Especially useful when you try to unzip a file and get the message: Unsupported Method. To solve this, try to install a higher version of p7zip by downloading the source and compile p7zip.
Get the latest version of p7zip from SourceForge. Unzip it and then run make. After the compilation is done, you’ll have the executable 7za in the bin folder. This version should be able to work with files compressed by 7zip for Windows. Make sure to read the README.
Copy the correct makefile. 7zip provides several makefiles, for each target platform / architecture. In case of Linux, the default one should work. To start compilation, a simple make is sufficient.
This gives you the binary ./bin/7za
Unzip a file multi-part password protected file.
7za x h1.7z
Install the 7zip program for Debian. This installs version 9.2.
sudo apt-get install p7zip
Let’s say we have 1 file that was zipped to file h1.7z using 7zip and splitter into 650 MB. 7zip produces 2 archives:
To list the archive:
7zr l h1.7z.001 -tsplit
We can see that the split archives contain one file named h1.7z. That is the zip file created by 7zip under Windows.
Some while ago I posted a blog detailing how to install Subsonic on a Raspberry Pi 1. If you google for “subsonic raspberry pi” my blog shows up as a top search result (normally 3rd place, but even second is possible). In addition, each day I get more than one access from somewhere in the world to that blog. There is definitely interest in running Subsonic on a RP.
Subsonic now closed source
Since the last time some things have changed. I updated to a RP2 and Subsonic was updated up to 5.3. I won`t update to version 6 and beyond. The developer of Subsonic changed the license of 6.x and made it closed source. Not a big problem for you as an end user. You can still download and install Subsonic 6 as a binary without problems. The license change however makes it unclear what the future of Subsonic will be. Is it going to be premium only, forcing you to buy a license? Currently the premium features are of no interest to me. Although not having Ads in the UI would be nice. Either way, I do not want to change software (got used to it), and staying in 5.3 for the rest of my life isn`t an option too (yes, I DO update).
Moving to Libresonic
Good news: one person is offering his own fork of Subsonic since a while: Eugene E. Kashpureff Jr. Started originally to take away the license check feature of Subsonic, he started a new project based on Subsonic 5.3: Libresonic. Available on Github, the source code is freely available and continues to offer the functionality of Subsonic. Community already started to give feedback on this new software. The name is no surprise, considering LibreOffice, Libreelec and now Libresonic.
No surprises here. It`s the same procedure as with 5.x. Ensure you have the pre-requisites installed like maven, Java, Debian tools.
git clone https://github.com/Libresonic/libresonic.git
mvn -P full -pl libresonic-booter -am install
mvn -P full -pl libresonic-installer-debian/ -am install
sudo dpkg -i ./libresonic-installer-debian/target/libresonic-*.deb
The name of the project changed to Libresonic, and so did the location of the configuration files and executables. The conf file is now located at: /etc/default/libresonic. The name of the parameters changed too, while the values are the same. So copy over the values of the old Subsonic conf file.
To start Libresonic, it`s now /etc/init.d/libresonic and the program files are at /var/libresonic. After you have done this, you can access Libresonic again via web interface.
I upgraded to EugeneKay fork some time ago when it was still Subsonic without license check. Therefore, I installed it using dpkg. The package is still installed and it won`t work nicely together with Libresonic because of sharing the same configuration (port, etc.). To uninstall your officially downloaded Subsonic DEB file, just follow the same steps.
dpkg –l subsonic
To uninstall that package:
sudo dpkg –r subsonic
This removes the package, but leaves the config files (can be seen when running dpkg –l subsonic again):
To remove also the config files:
sudo dpkg –P subsonic
The directory /var/subsonic containing some files wasn`t removed, you`ll have to do this manually.
Let the world know