Defining the ROI of Social Is Still Difficult

We all inherently understand the value of person-to-person interactions. For some tasks, solitude has its benefits — but more often than not, it is collaboration that unlocks road blocks in our minds, and blends solitary ideas into collective innovation. Looking back at the modern history of social collaboration — beginning with early portal and knowledge management platforms, the addition of project or activity-based systems that attempted to track and then quantify innovation processes, and more recently with the separation of structured unstructured collaboration models — executives teams always tried to understand, and put a number to, the benefits of these platforms.

It’s understandable: collaboration platforms are expensive, they tend to require tremendous overhead, and they also require expertise to build, customize, and maintain them. Add to all this the fact that so many collaboration initiatives fail (Gartner), and its no wonder that executives are asking for empirical data as to the benefits that social collaboration will provide to the company’s bottom line.

The problem is — how do you measure the return on investment (ROI) of social collaboration?

Last year, before Metalogix acquired my previous company, Axceler, I had a great conversation with my then-peer Donna Shaw (@Donnasueshaw) on this topic, and thought I’d again share some of the insights from our discussion. Donna has an extensive background in social collaboration, beginning with the Lotus Notes platform, and then working as a product manager for Ray Ozzie’s next venture, Groove. With the acquisition of Groove by Microsoft, Donna helped create the Groove Workspace functionality within SharePoint, and later joined the ‘social experiences’ team to develop social features for the SharePoint platform, including SharePoint 2013. After helping with the 2013 product launch, she looked around for her next opportunity, and found her way to Axceler where we ended up working together.

We all inherently understand the need for strong collaboration within the enterprise, but our leadership teams also need to be able to justify the expense of these solutions. I asked Donna the question that everyone wants to understand: What is the return on investment (ROI) of social? It’s not a new question, but what I discovered as I looked for answers is that most data captured by our customers, or shared online through some of the leading vendors, relied almost entirely on anecdotal data and qualitative impressions, rather than hard numbers and quantitative data.

So I positioned this problem to Donna, and asked her to explore the topic and share some of her experiences. Hopefully these insights will help as you construct your own measurements and value propositions for social within your organization.

Christian: How are organizations measuring the success (ROI) of social platforms and tools today?

Donna: From the customers I have talked to, they are struggling with this a bit.  Meaning, without a way to monitor usage and activity, it’s tough to make the good ROI story.  There are definitely products that provide some measurements and analysis, but not always as deep as what people would like, especially as it pertains to ROI.  One customer specifically said to me that they’ve gone out and purchased licenses for Yammer and they really want validation that all of those licenses are being used.

Christian: But showing activated licenses, or even metrics like Page Views or Unique Visitors do not give you a real picture of what is happening, and whether that activity is driving business value. What is the gap?

Donna: My impression is the gap is in the lack of depth of what the out of the box tools offer.  I think this gets to the heart of the issue, so let me elaborate here.  It’s helpful to measure adoption.  The idea that you can see the number of users who have joined the network and the number of groups that have been created is just a piece of that.  The additional value is seeing the growth trends over a period of time. One customer saw a spike in growth for both users and groups within his company and was able to point that growth to an internal marketing campaign the company had run to get more users up on Yammer.

But the adoption numbers alone don’t tell the whole story.  Sure, it’s helpful to see that 50 users joined the network and 15 groups were created, but without seeing the activity and usage of those groups, you really can’t validate the ROI.  Add to that measurements on user activity and you have a more complete picture.

There is one other aspect that a customer brought up to me around ROI and that is the idea of measuring what topics are trending and popular.  She referenced the tag cloud as one concept.  Her example was that in her company, the R&D team had requested Yammer for their group because they had a distributed team and needed a way to collaborate asynchronously since the team was spread across different locations.  The team felt it was invaluable using Yammer as a way to share information, ask questions, review content, etc.  She mentioned that seeing topics like “research”, “product”, and “development” as tending topics confirmed that the R&D group was indeed using their Yammer licenses, thus validating the ROI in purchasing those licenses.

Christian: Showing trending topics and even activity leaderboards are useful metrics, but organizations most definitely have a difficult time in translating these data points into ROI. It’s the problem of ‘qualitative’ versus ‘quantitative.’ Collaboration improves the quality of our communication, but it takes more effort to translate those things into trackable, measurable, quantifiable data. What visibility into social do you believe is missing from existing web analytics tools, or from SharePoint (and Yammer) analytics? And how important is it to measure ‘influencers’ within the enterprise?

Donna: I don’t think SharePoint provides much out of the box for this, though I should say I’m not familiar with everything offered in 2013.  I do know what Yammer offers, as well as products like Chatter.  Both provide some really good metrics, especially since they are included out of the box.  There are also 3rd party companies like GoodData partnering with Kanjoya, who provide sentiment analysis.  I haven’t heard much, but what little I have heard is that sentiment analysis is hard to get right and in some cases that level of analysis isn’t what users are looking for.

In my honest opinion, the concept of “Social Influencers” is one that can be tapped into further.  Sure, seeing which users have the most followers is interesting, but doesn’t always speak to the true influencers.  Likely, everyone will follow the CEO of their company, but that doesn’t mean that CEO is a key influencer.  Aggregating other measurements like who makes the most posts or more importantly, whose posts get the most replies, stirring on further interaction?   Whose content gets liked and shared the most?  Who gets mentioned most often?  When you see these measurements altogether, you can really get a picture about who are a company’s influencers.  Why is this important?  Cust
omers I have spoken to have said these people are key to evangelizing the use of Social within their organization.  One customer used the phrase “internal ambassadors of social”, which I thought was great.  Another large tech company talked about regional champions of social.  As a global organization, they need local champions within the regions to ensure the successful rollout and usage of Yammer. In fact, he mentioned that these champions have it in their commitments to ensure there is adoption and usage of Yammer in their respective regions.

Finally, in terms of gaps, I would have to say that since organizations often have multiple social and collaboration products being used across the company, the idea of seeing measurements for adoption, activity and influencers federated across multiple platforms is not something we’ve really seen yet.

Christian: How do you explain to people the differences between adoption and engagement, and why is it important to measure both?

Donna: I believe I got to that above – – adoption tells you how many users joined the network and how many groups or sites were created and yes, over a period of time, but those numbers don’t tell you if the product is actually being used.  Adoption coupled with engagement, which shows the level of activity within those groups and sites gives a more complete picture.  This combination is what really validates the ROI.

Christian: I’ve often stated that one of the primary reasons CIOs restrict the use of social within the enterprise is a general lack of governance across these platforms. What does governance look like in social today, and what should it look like?

Donna: This is a tough one.  IT and business leaders definitely want some sort of control, but not at the risk of stunting adoption and usage.  Still the things I’ve heard have mostly been around protecting the company’s IP.  Is someone talking about or sharing something either internally or externally that they shouldn’t be?  How can those who need to be concerned about this protect the company?  You can’t necessarily prevent this from happening, but you can be notified when it does happen and then take the appropriate action.

There is also other less risky, yet still important needs in terms of governance.  Customers have talked about some of the challenges with the “sprawl” of sites (SharePoint) and groups (Yammer).  Some groups just sit out there idle with no activity for months at a time.  So, sometimes there is the need to just be able to “clean things up” and consolidate – – being able to remove multiple groups at once from the network.

Christian: I agree. What is needed is something similar to the governance tools available across more structured collaboration platforms, such as SharePoint. Obviously not to the same degree, but there is a need for managers to set basic policies, provide some degree of oversight. I think there is a partner opportunity here.

—

For those interested in this topic, I will continue to do some research into the areas of governance, risk and compliance of social collaboration, because I believe its still a greenfield opportunity for partners. While the competitive landscape for metrics and analytics vendors is getting crowded, solutions that take action on that data and provide some degree of control and management are slim to none. It’ll be interesting to see the space mature over the next few years, especially as Microsoft further integrates Yammer and Office Graph capabilities into its Office365 platform.