The missing KPI

One dream, one soul
One prize, one goal
One golden glance of what should be
It’s a kind of magic

It’s a kind of magic – Queen

A key performance indicator is a nice tool to evaluate the success or progress towards a given goal. SAP has many KPIs to measure success and channels to communicate these. There are partner prices, awards for teams and even individuals are rewarded. Follow SAP on Twitter, on their press news site and you see that there are several awards and recognitions available for SAP to show that a wide range of performance are measured and rewarded.

The wrong incentives?

These awards have in common that they can be measured via KPIs. In the most basic interpretation, it goes down to: you sell, you get an award. Your product is a success, enough clients have bought it in the first years? Partners and customers are implementing it and paying license fees? You team sold enough licenses, met the selected sales criteria? High level customer satisfaction, adoption rate, successful go-lives, this is easily measurable. If you have been on both sides, you know the emails and forms to send and fill out to be evaluated, the deadlines to report your sales and go-live numbers, the questionnaires to answer. These inputs are crunched and compressed to a number that shows: KPI reached or nor. In case they are really good, you win a prize. The beauty here is: all this is easy to understand. The rules are simple, and you can repeat this year after year.

Knowing that there are many obvious and tangible KPIs that result in awards, what is helping and important, however not winning an award? It makes sense to look at the KPIs that are not measured and the awards that are not given. As we can see above, the selling, adoption, x over y numbers is easy to measure. Having an award for these numbers is a no brainer. But what about an award for something that is hard(er) to measure? Where is the award for implemented feature requests? The product with less bugs shipped, less (very) high tickets, escalations, downtimes, fastest resolution time across all products? The most improved product, gained most active users in the community?

The joke about it is: these numbers are easily measurable too. But are hard to collect.

Delayed damages

There are not really SAP awards celebrating the team/product that implemented most feature requests. That improved most the last year. The product that customers love to work with (there is one for go-live, but …). A product with an engaging community. For these, you cannot simply send out a questionnaire or look at some numbers of your CRM system. Here you have to work and also collect manually insights to get those numbers. I think the missing direct connection between effort and revenue is a reason why there are no recognitions. It doesn’t really matter if a team listens to customer feedback and implements the complex features asked for. It helps in the long run, as the product keeps relevant, but for a few years timeframe? There is no real reward for the effort. Imagine yourself now in the role of an SAP employee that is responsible for a product. Improving it is part of your job. But would you go the extra mile to make it a great product if your personal outcome is the same? Going to community events to talk with your users, out of your normal secure closed setting like a private customer meeting with AE, support, sales at your side? It is possible and some do it. A handful compared to thousands that could.

Having in mind that searching proactively for feedback is not having a great impact on the daily live, what are the implications? One is that the official feedback channels are not really made with user interaction in mind, nor made to offer visibility. Another important one is that if you report a bug or a possible new feature somewhere else, like e.g., on Twitter: forget it. No one from SAP is going to put it on their own into a request. SAP responsibles are not doing this on their own, for you. If you have a meeting with SAP people that have a certain public role like product managers, dev advocates, product owners, VP and you point to a missing feature: don’t bet on it that they are going to forwarding it to the right people or even put it in the backlog; and keep you updated on the status of your feature request. You might get a nice message and that they will check internally. If you hear the phrase: “good point, I’ll take this with me and check it internally” I really hope you know that person or someone in the team personally. Because you will have to check if it was discussed internally.

Recently I got several automated e-mails from a customer engagement initiative informing me about the status of feature requests. Sending a “no” after months or even more than a year, that’s the same as saying: we don’t care. Specifically, as you do not know why a decision was taken. I’d like to say that this is the exception. Unfortunately, at SAP, the exception is the rule. I cannot even remember how many times at DSAG, ASUG, TechEd, community events the participants brought ideas to a discussion, and while during the session SAP promised to get back, nothing ever happened. At least noticeable from external. This hidden, black box acting on feedback is a real problem. Nothing keeps motivated people better away from participating than communicating close to nothing and having to wait if or not a feature at some point in time makes it on the roadmap.

That is months after SAP acquired Qualtrics and is promoting the new Customer Experience. Well, SAP, deliver on that promise.

Better is possible

Of course, there are notable exception from the rule. Remember the status of SAP Portal from 2017? No? Let me help you: it was a mess. Listening to customer and partner feedback, the product was improved drastically. In 2011, the product was not comparable with what SAP delivered 4 years earlier: speed, features, administration, integrations, it was like a new product. Same with SAP Screen Personas. The first two version were nice (ok, we laughed at it and dissed the SAP team heavily because of some technical decisions), but what the Personas team delivered with version 3 was a massive step forward. Why? They listened and implemented features requested by their users. They managed to establish a direct communication channel with their users through regular meetings. There, users showed how they use Personas, concerns were discussed, ideas raised and, super important, a status of the latest changes and how the feedback is influencing the product was provided. SAPUI5 started nice, and luckily for us, the team behind it listened and acted on feedback. They provided a wide range of UI controls, made sure that you can easily upgrade your UI5 version without having to think too much about legacy code (it works). With the Fiori Launchpad team, performance was acknowledged publicly as a problem and they worked on making it better. Without this underlying and mostly hidden work, I cannot imagine where UI5 and Fiori would be today. When SAP treats their users as equal, listens, acts and is transparent about the feedback, implementation and releases, great products emerge.

Dear SAP, where are the KPIs and public awards & celebrations for teams that act on feedback? That implement feature requests? That pro-actively work together with users? That ensure that a product is rock solid? That OSS tickets are solved as expected, that developers take a look at issues? That users feedback is treated as a valuable asset? That have an active and engaged community? I have an idea on how to make measure this and make it happen. Hint: take a look at teams like Kyma, Gardener or UI5 and you get an idea how this could work.