Remove last n characters of file in MacOs

With MacOS and finder you can easily substitute characters of files using the rename functionality. Just select 2 or more files, right click, and inform the character you want to substitute, like _ with space.

To remove the last N characters from a file that looks like Text-2018221112.mp4 to Text is more complicated. The rename dialog does not understand regex. What you can use is the shell and rename

Install rename

brew install rename

Go to the directory with the files and run

rename -n 's/.{11}.mp4/\.mp4/' *

Rename uses the well known sed syntax s/char/replace/.

  • -n runs the replace in simulation mode. It will print the result, without renaming the files yet. Perfect for testing.
  • {11} number of characters to replace
  • \.mp4 is to insert mp4 again, as .{}11.mp4 will replace also the file suffix

In case the output matches your goal, run rename without -n and the files will be renamed.

rename 's/.{11}.mp4/\.mp4/' *

PowerLAN in real life

I bought a PowerLan adaptor to be able to connect to my private “server” (aka Raspberry PI) from the other side of the house via WLAN. Connecting the RP via WiFi wasn’t an option: two far away from the signal, I’d need to install at least one additional repeater and then two RP are without wireless adaptor. The PowerLan adaptor supports up to 1.200 MB/s and let’s me connect 3 devices directly. I won’t even need to buy an additional switch. And 1.200 MB/s. In theory, this is FAST. In reality the setup is complicated. The two adaptors are in the opposite location of a not-so-small house, the electrical stuff and cables were installed some 40 years ago. Some things you can only test after investing, and when thinking about how to connect to my servers, PowerLan was the only viable choice.

The setup was easy: one PoweLan adaptor connects to the Internet router, the other connects the RPs. Both find them and establish a secure connection. The connection signal is green, indicating a good connection. Connection tests showed that it is working: ping, ssh, all fine. It was time to check the speed. The RP isn’t the fastes when it comes to file server duties, but back in Rio I had 11 MB/s for copying files in Windows. That was fast enough. How will it be now? Laptop connected to WiFi (that is already not so good, given the layout of the rooms) and PowerLan that needs to communicate through several meters. It’s a worst case setup. I do not expect much.

A way to measure the speed is using netcat, or nc. Available in Linux and Mac. Basically the idea is to send data from the laptop to the RP and measure how long it does take to send 1MB, 10MB or more. The RP will serve as the receiving part, the laptop will send the data.

RP / Linux

Start nc, let it listen on port 2222 and send all incoming data to /dev/null

nc -v -l 2222 > /dev/null

Laptop

Read data from /dev/null, pipe it to nc and let it send to the nc instance running on the RP

dd if=/dev/zero bs=1024k count=512 | nc -v 192.168.2.112 2222

bs tells dd how much to read (1MB here) and count how many times. In the above example 512 MB will be send. To send 1 MB only, use count=1, for 10MB use count=10.

Results

1 MB

dd if=/dev/zero bs=1024k count=1 | nc -v 192.168.2.112 2222
found 0 associations
found 1 connections:
     1: flags=82<CONNECTED,PREFERRED>
 outif en0
 src 192.168.2.108 port 55619
 dst 192.168.2.112 port 2222
 rank info not available
 TCP aux info available
Connection to 192.168.2.112 port 2222 [tcp/rockwell-csp2] succeeded!
1+0 records in
1+0 records out
1048576 bytes transferred in 1.423923 secs (736399 bytes/sec)

Result: 736399 bytes/sec => 0.7 MB/sec.

10 MB

dd if=/dev/zero bs=1024k count=10 | nc -v 192.168.2.112 2222
found 0 associations
found 1 connections:
     1: flags=82<CONNECTED,PREFERRED>
 outif en0
 src 192.168.2.108 port 55624
 dst 192.168.2.112 port 2222
 rank info not available TCP aux info available
Connection to 192.168.2.112 port 2222 [tcp/rockwell-csp2] succeeded!
10+0 records in
10+0 records out
10485760 bytes transferred in 9.997975 secs (1048788 bytes/sec)

 Result: 1048788 bytes/sec => 1 MB/sec.

100 MB

dd if=/dev/zero bs=1024k count=100 | nc -v 192.168.2.112 2222
found 0 associations
found 1 connections:
     1: flags=82<CONNECTED,PREFERRED>
 outif en0
 src 192.168.2.108 port 55781
 dst 192.168.2.112 port 2222
 rank info not available
 TCP aux info available
Connection to 192.168.2.112 port 2222 [tcp/rockwell-csp2] succeeded!
100+0 records in
100+0 records out
104857600 bytes transferred in 103.159005 secs (1016466 bytes/sec)

Result: 1016466 bytes/sec => 1 MB/sec

 

Overall, the result is slower than the 11 MB/sec what I had before in Rio, but both connected  to the same WLAN. Now, with and unfortunate setup, PowerLAN gives me 1MB, and a working connection. It’s fast enough for storing documents. For copying a 100GB VM image it’s too slow.

Last preparations before the games begin

Yeah, let the games begin, I am prepared. Last week’s Rio held some test games, used for athletes to qualify for the Olympic Games as well as to test the infrastructure, stadiums, processes, etc. I live close to the main Olympic Games revenue location, and those tests showed me that I am not prepared. Each time a sports location was prepared to host an event, I ran out of electric power. When they booted up a location, I went down. After a few hours, energy came back. Then the event stopped and a few hours later when they shut down the main power switch, and this meant again for me: Stone Age.

What was the impact of the fact that the only and therefore monopolistic energy provider in Rio didn`t invest correctly in the last 6 years into a power grid that simply works? Event started in the morning, so I had no power starting 4:00am, or 5:00am. Power came back after a few hours (like 11:00am). Event ended in the evening, so they turned off the power at around 10pm, and guess what, no power from 10pm to early morning (2am). Run, rinse, repeat. I guess the power grid isn`t capable for handling the spikes that occur when you power on a whole stadium planned for thousands of people. Little bit more than 2 months until the games start, there is little hope this will be fixed.

The test events showed me clearly the priority going to be applied throughout the Olympic Games. Sport venues first, f@%# the rest.

I have some computers and entertainment equipment at home, like a TV, computer, a RAID5 holding 20TB/15TB. Especially the computer and RAID is something I do not want to see crash simply because a ping-pong game between Azerbaijan and Salomon Islands is starting. My solution? UPS.

My choice fell on APC, 1500VA. This model comes with an USB port, allowing connecting to it and shutting down a computer in a controlled matter. 8 power connectors let me connect enough of my equipment. Even considering that some vendors think their tiny router is so important that it needs a power connector double size of a Raspberry Pi (looking at you, Apple!). For worst-case scenarios, this APC model lets me connect an additional battery, extending further the autonomy.

Here are some pictures of my new UPS.

A look back at the first 4 month in 2016

At the beginning of this year, I published a review on my blog activities in 2015. In 2016, I do not want to wait 12 month to give my readers an overview on how my private blog is doing. I thought on sharing each quarter a short review, but as now it`s already end of Abril, I guess you can see that plans are there to be adjusted.

My site started in March 2015 and in 9 months I managed to publish 116 blogs, gaining 4.234 unique visitors and 10.017 page views. I went from Raspberry Pi 1 to Raspberry Pi 2, increasing overall speed of my site. Now to 2016:

Blogs

In 2016, I have 37 blogs published (on average, this makes almost 10 per month), I have 34 drafts in the pipeline, and attracted so far 2500 unique visitors, contributing to 3.300 visits, totaling 6.575 page views. I get visits from around the world.

Analytics

In January I installed elasticsearch and logstash to be able to run some analytical reports on the apache 2 logs. Amazing what you can run on a RP2 with just 1 GB RAM.

Being popular

When I started my site, I decided to use a local server, running on Raspberry Pi. Not only because it’s cool, its cost efficient, I can control it / have absolute power and can run whatever other software on top of it (Subsonic, GateOne, git, etc). My site gained enough popularity that my internet provider decided that I run or am affected by a bot net, and stopped blocking port 443 last month. While this is not nice, I accept that having people from around the world accessing my site may look strange. Nevertheless, contract says they can, so I decided to put an AWS nano instance as a reverse proxy in front of my site. Yes, www.itsfullofstars.de is not anymore pointing directly to my Raspberry Pi 2, it`s going to AWS. In the end, you still end up in my living room.

Problems

Thanks to Olympic Games, I am suffering on constant power outages, as the city thinks that the population can be affected without problems, so they can host some test games for the IOC. This means that I am not always online, and my site is offline. Not sure what are the benefits of hosting Olympic Games, but if you think that in 2016 a stable power of internet connection is one of them, it`s not. At least a few million dollar a spent for sport stadiums.

Expose a BAPI using JSON and REST

Note: 1st published on SCN at 22.5.2012

 

REST.

JSON.

These are the technologies you need when writing modern web applications. And that is what makes it so hard to use SAP together with modern web applications: Currently you only get REST from SAP with Gateway, but not JSON. OData comes from Microsoft and support is not as wide spread as someone expects. How OData does looks like? You can try it out by downloading the Gateway Trial from SCN or use the online demo system. To experiment with Gateway the usual flight example from SAP can be used. To get the details of a specific flight: sap/opu/sdata/iwfnd/RMTSAMPLEFLIGHT/FlightCollection(carrid=’AA’,connid=’0017′,fldate=’20110601′)

Data returned from Gateway looks like this:

Instead of being able to use the data exposed by Gateway by the widely used Javascript frameworks like jQuery you need to find an OData plugin. Of course you can still use SAP UI5, but what when you are forced to use jQuery, Sencha, Dojo or something else?

That’s a problem with SAP and ABAP in general: you have to wait until SAP or developer implements functionality, and because of the limit resources available in the ABAP ecosystem this can take a while, costs some serious amount of money or will never happen. That’s where we can be happy that SAP also embraces the use of Java. As Java was made for the internet there is a huge Java community out there that most probably already developed a tool that fits your needs.

For exposing data as JSON in a REST-full way the tool available is: Jersey.

After Jersey transformed the data, the response looks like:

{“EXTENSION_IN”:{“item”:[]},”EXTENSION_OUT”:{“item”:[]},”RETURN”:{“item”:[{“TYPE”:”S”,”ID”:”BC_IBF”,”NUMBER”:”000″,”MESSAGE”:”Method was executed successfully”,”LOG_NO”:””,”LOG_MSG_NO”:”000000″,”MESSAGE_V1″:””,”MESSAGE_V2″:””,”MESSAGE_V3″:””,”MESSAGE_V4″:””,”PARAMETER”:””,”ROW”:0,”FIELD”:””,”SYSTEM”:”NPLCLNT001″}]},”ADDITIONAL_INFO”:{“FLIGHTTIME”:361,”DISTANCE”:2572.0000,”UNIT”:”MI”,”UNIT_ISO”:”SMI”,”PLANETYPE”:”747-400″,”FLIGHTTYPE”:””},”AVAILIBILITY”:{“ECONOMAX”:385,”ECONOFREE”:20,”BUSINMAX”:31,”BUSINFREE”:1,”FIRSTMAX”:21,”FIRSTFREE”:3},”FLIGHT_DATA”:{“AIRLINEID”:”AA”,”AIRLINE”:”American Airlines”,”CONNECTID”:”0017″,”FLIGHTDATE”:1306897200000,”AIRPORTFR”:”JFK”,”CITYFROM”:”NEW YORK”,”AIRPORTTO”:”SFO”,”CITYTO”:”SAN FRANCISCO”,”DEPTIME”:50400000,”ARRTIME”:61260000,”ARRDATE”:1306897200000,”PRICE”:422.9400,”CURR”:”USD”,”CURR_ISO”:”USD”}}

Jersey needs Java 6 and runs in a servlet container. As NetWeaver CE >= 7.2 fulfills these requirements, Jersey can be used to transform POJOs into JSON and expose them using REST. NW CE comes with a framework for creating composite applications (CAF) that can consume BAPIs. CAF uses standard J2EE technology like JCA and Beans. This bean can be used by Jersey to automatically extract the data, transform it into JSON and make it accessible using REST.

Consuming a BAPI using CAF can be done with no coding involved at all as CAF comes with some nice wizards. Just map the input and output parameters and the code will be generated including the bean that can be used to interact with the BAPI. In the Jersey web application the URL and protocol get defined:

@GET

@Path(“getFlight/carrid/{carrid}/connid/{connid}/fldate/{fldate}”)

@Produces(“application/json”)

The CAF bean gets called using the parameters retrieved from the URL:

out = e.bapiFLIGHTGETDETAIL(null, null, null, carrid, connid, flightDate);

That’s it. Jersey will do the rest:

How the URL gets constructed is up to you, the parameters can be part of the URL as above or a query. You can also define if it is GET, POST, PUT, etc. If you want to do some coding you can adjust the output, or combine the result of several BAPIs into one Java object that JSON will expose.

Now that the BAPI can be accessed in a RESTful way and the data retrieved is in the JSON format, it’s easy to use the data in a Javascript framework like jQuery with Datatables:

The actual coding for making the CAF bean accessible to Jersey and expose the data is less than 10 lines of code. From CAF to Jersey to a running HTML application it does not even take 30 minutes.

The framework I used for interacting with the ABAP system is CAF, a framework available for NetWeaver Java since a while (7.0): well documented enterprise ready and supported. If you want or need to expose your BAPI by REST and JSON or XML and you have a NetWeaver CE >= 7.2 (like Portal 7.3) available, you can start today. As your NW CE system is a separate one, upgrades won’t affect your backend system and as Jersey is also a separate application; changes to Jersey don’t affect your other CE server and applications. Of course you don’t need NW CE for using Jersey and CAF for interacting with BAPI. Every platform that Jersey supports and where you can use JCo from SAP to call a BAPI can be used like tomcat. It just means that some extra configuration and coding will be necessary. This also implies that your backend ABAP system can be used as long as a JCo connection to it is possible.