Fish with OData

Rui Nogueira published a while back a blog series on SCN on how to implement an IoT scenario using a Raspberry Pi and HCP. I think the example shows very well how what the main use case of IoT is. When the blog was published, there was no SAP HCP IoT service available; if you want to implement the same example in a more correct way, you should use HCP IoT. Nevertheless, Rui`s example is easy to implement and shows how the different parts play together: client, server, user.

When I first came across Rui`s blog I noticed that he uses REST and goes through some effort to persist the data. I thought that it would be nice to adopt this to make use of OData. Took me some while to publish this blog J In the end, I did not adjusted his code, it merely served as an orientation. I wrote my own IoT server and client app. The result is a simple, clean and easy to read JEE app that uses JPA and Olingo for exposing the JPA entities and a Java client that does not need to be run on an IoT device. My user dashboard is very simple, implemented in D3.js, and only shows one sensor`s measurement data.

The client is a Java app that reads current weather data from openweathermap.org. To make this work, you`ll need an API key (free). In case you do not want this, I added a jMeter test that creates random temperature data (as seen in above picture). JMeter test file is located here: fish-with-odata\iotserver\test\jmeter\LoadData.jmx. The test is pre-configured to use localhost and port 7080. The test will run for 3 minutes as the 100 measurements are not created at once, but with a fixed time interval of 3 seconds.

The app

The source code can be found on GitHub: https://github.com/tobiashofmann/fish-with-odata

You will find two folders:

  • iotclient, containing the client app
  • iotserver, containing the server and user dashboard

Both are maven projects. It should not be a problem to transform them into Eclipse projects via mvn eclipse:eclipse, but while I developed both in Eclipse, I did not test transforming to an Eclipse project from maven. Sensor and Measurements are implemented using JPA. The relationship between both is that one sensor can have many measurement assigned, but a measurement can only be assigned to one sensor. In the Snesor class, this is done via @OneToMany

Sensor class

@Entity(name = "Sensor")
public
 class Sensor implements Serializable {
    @Id
    @GeneratedValue(strategy = GenerationType.TABLE)
     @Column(name = "ID")
     private long id;
     private String device;
     private String type;
     private String description;
     @OneToMany(mappedBy = "sensor", cascade = CascadeType.ALL)
     private List<Measurement> employees = new ArrayList<Measurement>();

Measurement class

@Entity(name = "Measurement")
public
 class Measurement implements Serializable {
     @Id
     @GeneratedValue(strategy = GenerationType.TABLE)
     @Column(name = "ID")
     private Long id;
     private String unit;
     @Temporal(TemporalType.TIMESTAMP)
     @Column(insertable = true, updatable = false)
     private Date createdAt;
     @Temporal(TemporalType.TIMESTAMP)
     @Column(insertable = false, updatable = true)
     private Date updatedAt;
     private Double value;
     @ManyToOne
     @JoinColumn(name = "SID", referencedColumnName = "ID")
     private Sensor sensor;

I am lazy so I let JPA decide when a measurement is created or updated. This may not be acceptable in most scenarios, especially when you depend on the exact time when the data was captured by the device and not when it was persisted in the DB. I implemented it that way to not have to take care of capturing the date in my client app and to keep the payload low.

Run server

To run the server:

mvn clean pre-integration-test

This will download the HCP SDK, install the server, run it on port 7080 and deploy the WAR file. After some while, the IoT server is ready.

A benefit of OData can be seen when comparing how Rui is consulting the latest added measurement for a sensor: he adds the latest measurement as an object to the sensor.

private Measurement lastMeasurement;

With OData, the latest added measurement for a sensor can be retrieved by simply adding some parameters to the URL:

$top parameter controls how many data points are returned. Beware that with OData, there is a page size defined that limits the max number of requests returned. This parameter is configurable in the class de.tobias.service.ODataSampleJPAServiceFactory

private static final int PAGE_SIZE = 50;
Assign any value to PAGE_SIZE you consider useful.

Run client

To run the client, you first must add your API key. This is done in the class de.itsfullofstars.iot. WeatherData. Add your API to APPID.

private static final String APPID = “YOUR API KEY”;

To run the client, create the jar:

mvn package
java –jar target\fishodataclient-1.0.0.jar

As an alternative, a jMeter test is included in the server: fish-with-odata\iotserver\test\jmeter\ LoadData.jmx

The final chart can be seen by accessing: http://localhost:7080/iotserver/. Depending on what data source you use, the chart will look like a flat line or like a heart attack.

Real data (Rio de Janeiro)

Fake data

My view on the new on demand portal

Note: first published at SCN on 31.1.2012

What does on demand portal offer and do you need it? Cannot the SAP Portal also be an on demand portal?

On demand is the “next big thing”: every product, every solution has to be available as an on premise and an on demand version. Simplified, on demand means that you can access your server and solution via the internet, from everywhere you are. For a normal user there is no difference in how to access a new on demand solution and how Yahoo Mail or Google Mail is accessed and used: enter the URL in the browser and start using it. For some solutions on demand is more a cultural shock than for others. Basically the main benefits for on demand are access, costs and maintenance.

SAP Portal users are familiar with web enabled access. Most of the time they are bound to the corporate network; sometimes they can access the services from outside the corporate network, by VPN or even by a “normal” URL. So where are the benefits of an on-demand portal http://wiki.sdn.sap.com/wiki/display/EP/SAP+Portal+On+Demand? Configure your infrastructure right and you can have an on-demand version.

The tricky part is the “your infrastructure”. Not every company does know how to do it right or even has the skills to do that in a secure way. The technology stack needed to run the SAP Portal is NetWeaver Java. There are stacks out there that are easier to maintain and that need fewer resources to run. You need a full J2EE stack for you application? Most portal applications only need a servlets container (like tomcat). The framework and standard UI of the SAP Portal are too heavy for Internet usage. Even with the External Facing Portal (EFP) framework, light weighted is defined differently. Licenses for the SAP Portal are cheap when your users are Business Suite users. As licenses are already covered, costs like bandwidth (if your company doesn’t have a flat rate or the money for enterprise grade backbone connection) and maintenance remain.

But still: problems that can be solved, so why an on demand portal?

Maintenance is where Basis surely will be relieved as the task for applying service packs and notes will be delegated and end-users will be happy too as a good on demand solution offers a higher availability than the infrastructure of a normal company can. Setup time and costs are inexistent compared to the on premise portal.

The ODP will be – naturally – an external facing portal (EFP). Considering the problems the on premise portal has when it comes to make it an EFP in regards to:

  • Browser support
  • Mobile support
  • Security
  • Speed
  • Access

How will the ODP treat and solve these problems? And when you are an EP user, what kind of options will you get to use the ODP as your EFP? And will the ODP be the starting of the end of the EFP of the SAP Portal?

Looks like SAP is going to use the on demand portal to introduce a new stack to run the portal on. Open source based, OSGI support, something more like tomcat. The connectivity won’t be able to compete with what the SAP Portal offers, but as long as your backend exposes the data using HTTP/S it can be integrated; implying that you still have to be able to expose your backend data in a secure manner. If you know how to do that you can still opt for opening your corporate SAP Portal. But you won’t get the new SAP UI5. And that new interface alone justifies the on demand portal. Compared to the “old” SAP UI, UI5 was designed to be used over the internet in mind.

For the developer ODP is portlet development (WAR). It will be interesting to see if portlets developed for ODP also run on a native tomcat or on JBoss or on other competing products or what the effort is to make them compatible.

How will the access to information handled? A portal with portlets is just the visible interface to the user, but what about portal services? Will ODP come with a predefined architecture for accessing portal services and data?

What do I expect from ODP?

A new software stack, cleaner, easier, more open source and support of more and newer standards. The new SAP UI5. If everything works out well SAP will be forced to merge the two code lines of on-demand and on-premise portal. Refreshing thus to “real” SAP Portal too. What can be wrong about that? Mobile access is crucial. Of what help is a portal accessible from everywhere and you need a desktop browser? This should also drive the adoption of mobile access to SAP and the Portal on device http://wiki.sdn.sap.com/wiki/display/EP/SAP+NetWeaver+Portal+on+Device for the on premise SAP Portal.

As ODP gives us a revitalized portal running on new technology it should attract more developers. Done right developers have the freedom to choose how and with what they want to code: GWT, jRuby, PHP for Java, JSF, Java 5, 6 or 7, etc.

Open access to the information available at ODP. Everyone that already had to integrate the on premise portal – or the information stored and made accessible there – into another portal or product know that the SAP Portal is meant to be the last point of access. The SAP Portal’s primary design is to integrate content, but not to share it. Especially an ODP cannot be designed that way. As it is available 24/7 to everybody, so has to be the information.

So one problem remains: access. SAP has shown us more than once that this is a topic where SAP continues to deliver below the expectations. Currently, developing for and learning SAP on your own private environment comes with some constrains: downloading, installing, renewing the license every 90 days, and you cannot create your environment as you wish, you have to use what SAP gives you. (ex: CE 7.2). Not everybody can download several GB of data and install it; the hardware requirements are even today still a challenge for laptops – not everybody has more than 2 GB memory installed. Contrary to this, tomcat is downloaded and running in minutes. No wonder that tomcat is a popular servlets container.

It lies in the nature of on-demand that access isn’t a real problem anymore. The question is: will developers get free and no time limited access to ODP? To evaluate, learn and code the access does not need to be unlimited in all aspects: 1 or 2 users, limited bandwidth, CPU and memory usage, performance also does not count much, data base can be SAPDB. What counts is: give access to developers, from the very beginning.