Shubho Ghosal's Blog on BI Strategy

Just another weblog

Formulating a BI strategy

leave a comment »

Formulating a BI strategy for your organization can be a daunting task. Often you have folks on Linked In groups asking the question, “How do I create a BI strategy for my organization?” This can be a tricky thing to undertake. Consulting organizations have processes that evaluate the current state and desired future state or BI and propose a roadmap to get them to the future state. This doesn’t provide a complete strategy, however, though it does provide elements of it.

In order to get a strategy going, I believe you have to set the right context for it. What are you creating the strategy for? What are your goals? You might say, very generally, that your goal is to make BI successful in the organization. Or to make BI pervasive. Or something very non-specific like that. While it is possible to create a strategy to “make BI successful” in general, having more specific goals can mean a more meaningful strategy whose success is measurable. For example, you might come up with a few specific goals, such as below -

- My organization is looking to streamline cost, and so one of my goals is to minimize the cost burden of BI.

- We are a lean IT organization that cannot afford to service every BI request, so I would like to figure out how to create a strategy around self service.

- Our senior managers need quick access to critical information, and they are frustrated by the lack of it. I need a means to provide them ready access to KPIs.

- BI hasn’t really taken off in our organization. No one is excited about it. It is not considered a strategic asset. I would like to change the perception of BI and generate more excitement about it.

(Okay, this was not so specific but this is a very common problem in organizations).

So, formulate your strategy around specific goals. Goals are likely to change as an organization evolves, and therefore your BI strategy will be an evolving thing. Below are some commonly used strategies to deal with situations such as above. These are also tips to “make BI successful” in your organization, if you will.

Aligning your BI deliverables with organizational pain points and needs

This is something that my former employer, a consulting organization, taught me and this makes sense. In order to do this, you need to interview your users. While this can provide very valuable information, where it falls apart is in organizations that don’t know what they don’t know and are not very articulate about their needs. For that, a separate or complementary strategy should be adopted, such as -

Filling in gaps with industry best practices

This was an idea my current manager gave me and I loved it. If your organization cannot articulate its needs, you can propose solutions that are considered industry standard. Most organizations should have a large portion of their needs in common. While functions in the organization differ in their needs, a particular function such as Finance should have broadly similar needs across organizations. You can start with that, put it in front of your users in a “How about this” mode and see if they bite.

Making performance metrics or KPIs your cornerstone

One of the key reasons for BI’s existence is to measure the performance of organizations, often referred to as Corporate Performance Management (CPM) or Business Performance Management (BPM). Often, this gets overlooked and focus is put on very specific and mundane reports for discrete tactical needs, making BI essentially tactical in nature. Returning the focus to performance metrics will elevate the status of BI and make it more strategic. Find out if your organization uses a formal performance management approach such as the Balanced Scorecard and what metrics it is using. Find out how these metrics are delivered currently and how to deliver them better, using your data warehouse foundation. If there are gaps, propose industry standard metrics to fill them in as a starting point, and refine those.

Using Dashboards as the primary means of disseminating the KPIs

Dashboards are essentially compact visuals providing users ready access to KPIs. End users, especially senior managers, take very well to the concept of dashboards, as they tend to be very visual, making the consumption of the metrics easy. Thus, building effective dashboards should be part of every organization’s BI strategy.  Focusing on both strategic and operational needs is critical, otherwise too much focus is often paid to one side, usually the operational one, resulting in great support for the running of day-to-day operations but not enough for the making of strategic decisions. Ensuring dashboards are well designed and provide compact, visually rich views of the most critical metrics catered to each individual’s needs is critical. Enterprise wide coverage is also critical. One exercise that can help determine the set of dashboards needed for your organization is to create a matrix of needs, with the groups or functions in your organization as columns and Need Type (Operational or Strategic) as rows. Well orchestrated interviews that dig into your users’ analytic needs on both sides is required to fill in this matrix.

Creating an effective analytics portal to attract and retain users

This aspect is often overlooked. How is BI dished out? Is it a set of folders with links to reports? That is not very appealing (though it does get the job done). In a very recent project, I was exploring the idea of an analytics portal with a group who was very intent at exploring new ideas. We settled on a portal that a user would go to and directly see their dashboards, standard reports, ad-hoc analyses, BI related documents, notifications, etc. It was a customized view of each person’s analytics offered through a standard look-and-feel interface. End users loved it. You don’t want the experience to be boring and functional. A BI portal should be like a well designed web page, making it easier to get your job done.

Choosing the right technologies

If you handle large amounts of data, choosing a weak database solution will kill your BI. You need to look at Big Data platforms. Enterprise reporting platforms provided by the big vendors usually have enough functionality to meet everyone’s needs but if your organization is cost focused that may not be a very effective strategy to pick one of those. Do you need tools for self service? For OLAP style exploration? All these are considerations for coming up with a technology strategy. The biggest thing about this is that it must be aligned with the business aspect of the BI strategy. Both must occur or one or the other may fail.

Identifying the product / application roadmap

This is the other good thing that my previous consulting employer taught me. This involves coming up with a product or application roadmap to satisfy needs. After gathering needs, you need to prioritize them in importance and scope, and separate the low hanging fruit (quick wins) from the rest. I once came up with a Gartner style bubble chart that had Ease of Access to Data as one axis, and Enterprise Coverage (or something like that) as the other, with bubble size being determined on scope. I will have to dig this up and show this here. This can be a great way of representing the applications and showing which ones the quick wins are. I’ll do that shortly.

And the list goes on. This post feels incomplete still. I would like to keep this post open for a while and keep adding to it as more thoughts come up.

Written by Shubho Ghosal

August 30, 2012 at 3:02 am

Big Data Analytics shakes up EDW

leave a comment »

Having been on a break from consulting since the year started, I decided to catch up on my reading and started with TDWI’s very well summarized article on Big Data Analytics, written by Phillip Russom (see link elsewhere on this blog). As the article explains, Big Data Analytics is the coming together of Big Data and Advanced Analytics. Big Data is not just about large volumes, but about diversity in data types (structured, semi-structured and unstructured), and variety in data refresh speeds as well – from real-time to delayed, including traditional transactional data which presumably occurs in discrete, varied time intervals, to sensor-type data which is a continuous stream. Advanced Analytics is a revival of everything that was considered too esoteric for most EDW groups to aspire to, from recognizable techniques such as Predictive Analysis, Data Mining, Complex SQL and Statistical Analysis to some less recognizable ones such as Natural Language Processing (used to understand written text) and artificial intelligence (not sure how this is used yet). There is also content analysis, or the analysis of video and audio, that seems even more advanced.

All this is to be supported on a diverse set of fairly new platform choices ranging from hadoop-based implementations to DW appliances to columnar, analytic, or in-memory databases to in-database analytic functions.

It feels as if under the single banner of Big Data Analytics, all that was exciting albeit challenging to consider and certainly not your everyday topic in DW circles is being revived and made mainstream. Now, a distinction can be made between “old-school” EDW-BI solutions which offer a very structured and fairly predictable model of storage and consumption through well-defined data warehouses, dashboards and cubes, and this new “exploratory” or “discovery-oriented” way of looking at data. So, does the thinking around EDW need to change? And how?

Do we need to throw away our EDWs and move all our data to a brand new platform such as above? One point that was made in the article is that the data that is fed into such technologies needs to be as raw as possible, and that traditional ETL processing should not b applied. So, the data can clearly bypass the EDW entirely and be fed directly into one of the above-mentioned technologies to achieve results. Is the role of the EDW diminished by this and will it become simply a historical source to this new, all-encompassing world?

Not so soon. First of all, a little data cleansing and staging doesn’t hurt. Picture raw data with mis-spelled customer names. A little massaging can only add to the quality of the analysis. Thus, the EDW can be used to stage an optimally-cleansed data set that is used as input towards the analytics. If your EDW architecture includes an Operational (ODS) layer to it that already houses cleansed data from the transactional systems, that can be used as a source for the analytics as well.

Beyond that, the traditional database platform that houses the EDW seems a poor choice for being the main platform for supporting Big Data Analytics. For one, it does not support well the storage of the diverse data types that seem to be desired. Nor will it respond well to the exploratory nature of the analysis that seems to be heart and soul of Big Data, with its fixed indexing and partitioning schemes.

It looks like EDW implementations would have to coexist with Big Data Analytics systems, acting as a source of structured data towards it. Either Big Data Analytics would have to be supported on a separate platform, or both it and the EDW would have to be moved to the new platform (This works if the implementation is a hybrid platform supporting the range of structured to unstructured data). This makes sense as traditional EDW platforms have been notoriously ill-suited to the exploratory analysis that some users desire. This coexistence of technologies that support what is well-defined and what is fuzzy seems apt. Also, the traditional EDW approach with its proven way of handling the clearly understood analytics through dashboards and cubes does not have to be thrown away.

Another aspect to think about is that Big Data Analytics is not just for Big Organizations, or even just for implementations literally with immense volumes of data. The Advanced Analytics aspect of it can be applied to any situation. For example, text analytics can be used to extract nuggets of information from the wealth of textual data collected in any organization. I was at a recent engagement with a telecom infrastructure services provider where it occurred to me that the descriptions being collected for tower development projects could be mined for reasons why projects were being killed. Unfortunately, such topics are still considered too far-fetched. One good thing that might happen with the recent upsurge of Big Data is to bring Advanced Analytics more to the forefront.

In conclusion, Big Data Analytics seems disruptive to traditional EDW approaches at first, but in the end it appears a symbiotic union.

Written by Shubho Ghosal

January 5, 2012 at 3:17 am

TDWI Big Data Analytics User Perspectives

leave a comment »

TDWI Big Data Analytics User Perspectives

A TDWI report on Big Data Analytics

Written by Shubho Ghosal

January 5, 2012 at 2:05 am

Posted in Uncategorized

The Interplay of BI and Social Media

leave a comment »

In order to address the role that Social Media plays in BI and vice versa, first let’s look at the identity that social media will take within the organization. I don’t believe it will look anything like Twitter or Facebook, but it will take ideas from it.

Social media within the corporate firewall will be secured, not only from the public, but within groups within the organization. The key focus will be sharing of information across the enterprise, between groups. Currently, the ways information is shared electronically in organizations is via email, chat, and collaborative portals such as Sharepoint. However, few would argue that any of these avenues currently has the appeal of a Facebook, Twitter, or Linked In. So, these avenues have to grow a little and borrow some of the ideas from popular social media channels, or alternatively, we should ditch those altogether and start afresh.

Social Media within an enterprise has to have a focus. For example, I raised the notion of a Decision Tracking System in an earlier post (distinct from the old notion of a Decision Support System, by the way, and simpler). This can be an application of social media within the enterprise. Decisions will be exposed within select groups and secured from others, but visible to all within that group and open to feedback from members within the group. Analyses put forward in support of the decision could also be added to discussion threads (see my earlier post on telling stories about the data). This could be one of the critical applications of social media within the enterprise.

Sharing knowledge across analysts could be another important application. I got this idea from talking to the head of BI of a well known movie rental and streaming service. This ties in with the story-telling notion as well. The idea is to allow analysts to share their analyses with each other and present their own thoughts along with it and get feedback. This would let folks learn from each other and not repeat each other’s mistakes. 

The more global an organization is, the more social media within the enterprise could have an impact. Think about how Facebook or Linked In have brought together people from across the globe. I have always felt a thrill at being able to connect with a like-minded thinker in Spain, Poland, Russia or elsewhere. It demonstrated to me how ideas can come from anywhere and that some people essentially think similar regardless of culture or geography. Now apply this notion to sharing ideas within the global organization’s many locations. My experience is that more people are likely to open up to a social media outlet than in conference calls or face-to-face meetings. And ideas often come in the middle of the night to creative thinkers.

If the social media application had a data or information focus, that is where BI can play a role, as a provider of the information and a basis for decisions made or conclusions derived. Snippets of BI would go into the analysis that supports the discussion thread. Or, the discussion thread itself may drive the need for new BI assets as well, driving the direction that BI should take.

If we had to take BI to the next level and really connect it to decision making, social media within the enterprise could play a big role in that. The need of the hour is to develop these thoughts into product ideas that would merge these concepts and free up the BI assets within the enterprise.

Written by Shubho Ghosal

July 16, 2010 at 8:07 pm

Connecting BI to Decision Making

with one comment

Being able to connect BI to the critical, strategic decisions being made within the organization is BI’s holy grail. How often does this really happen?

BI is well established in the world of course monitoring and correction – the operational dashboard and scorecard takes good care of that. While the need and ability to put together a BI system, along with a DW, a MDM system, and other information assets is well understood, and the world is full of people who can do this and are excited and passionate about this, there are precious few people who can testify to how often these large and expensive systems were actually used in strategic and/or critical decision making in organizations.

Let us briefly speculate on the nature of decision making in organizations. While this may differ from place to place, one can safely assume that a germ of an idea (or thought or opinion) occurs in the decision maker’s mind. This may be triggered by a market event, a desire to get to the next level, or to survive in the current economy. This germ of an idea grows in the decision maker’s mind, until it is well formulated, at which point it can be shared with the group of influencers. The influencers bring their own opinions to the table, the idea gets threshed out and bandied about, and with time a decision is made.

Information can play a significant role in various points in this process. Graphs, charts, and numbers can get thrown about. One must have confidence in the numbers (data quality). Everyone must see the same results (integration). A widget should be a widget (MDM). The right visualization should be used (Visualization tools). Powerful stories must be told (see my earlier post about telling stories with your data).

The unfortunate truth is, even if this happens, there is no way to track it. Imagine if we had systems where we were able to track our decisions. Wherever we used a BI artifact, that link is preserved. If a story influenced a decision, that analysis is included. The ability to tell a story is provided – this itself is lacking in BI technology right now.

If we were able to do this, imagine the sense of importance that BI would gain within the organization. The ROI would be clearly recognizable. At the very least, people would recognize it as being a strategic asset.

There are some gaps we would have to fill technology-wise, and process-wise as well. This snippet which I wrote as part of a TDWI discussion I started captures the thought -

1. The ability to track decisions and capture the basis of them – call this a decision tracking system, if you will. By the way, what happened to decision support systems at least in the world of business? By “basis”, I mean the link to the piece of BI that supported it, if it did.

2. The ability to tell stories about your data. These are the powerful analyses that sway decisions. BI is not very good at doing this – it seems to be missing from vendor’s foci.

3. The ability to present scenarios (again, this correlates with one of Ron’s thoughts). We had seen some examples of tools that enable this, but they went away. Scenario building is critical as it can simulate the potential outcome of making a decision.

Having these capabilities would bring BI closer to decision making and make it more strategic. Vendors need to recognize this though, and provide the capability for it. Organizations would also have to adopt this within their processes.

More on this later.

Written by Shubho Ghosal

July 9, 2010 at 2:55 am

Does your data tell a story?

with 8 comments

I attended a very thought provoking presentation recently by Jonathan Koomey on turning numbers into knowledge. One aspect of the many things he was proposing really caught my attention – that having a good story is as important as having good data. This opened up a floodgate of ideas in my mind. It reminded me of two examples of very compelling analyses I have seen – both told good stories about their data.

One was by a researcher analyzing the progress of nations across time and making the point that, over the years, some nations have progressed faster in terms of learning. This was attributed to certain factors, and a simple bubble chart that was animated to show movement of the bubbles across the chart with time showed how these nations had moved to towards a better position. All along, the researcher made a compelling case around the data, bringing in the factors that were tied to the movement of data across the chart with time. He achieved this by having a narrative around the presentation of the data.

Another was a New York Times analysis on census data presented on the web. The presentation was unique in that the reader could interact with the graph and see how the trends were for a specific gender, age group or ethnicity. However, the reason why it stood out was that a story was drawn alongside the data and the graphical presentation of it – a story that clicked with the data and helped make sense of it, making us draw certain conclusions that the facts supported.

(When discussing this with my colleague at work, I was pointed to yet another example – this one was sketched in 1869! The link is here: This describes Napoleon’s army’s progress to Moscow and back, tracking the way the army dwindled due to casualties – and relating this to climatic conditions. A visionary analysis indeed and far ahead of its time).

In all of the above cases, what made the data meaningful was the story. This wasn’t a case of putting a clickable analysis interface in front of us and leaving us wondering what the heck we are supposed to do with it. The same is true for most BI solutions. We are so caught up with the technology that is being offered to us – in-memory analytics, columnar databases, cloud – that we are missing out on this extremely critical aspect of better BI – the need for technologies or simply the discipline to create a meaningful analysis around the data that we are presenting such that it leads to effective decisionmaking. This is completely opposed to the traditional notion in BI that you need to present clickable interfaces that let you analyze (the vendor classic) “which of my regions are responsible for my drop in sales in August”. It is not that simple.

How about presenting the data as a storyline such that the decisionmaker does not have to click anything? How about presenting each RELEVANT click as another piece of the analysis in a storyline, and providing enough context around the graphics such that the point of each piece of data presented is clear? Providing a headline and then leading us to some conclusions about the data?

I’m sure there are great analysts in organizations that are already doing this – but how? Is the current spate of BI technologies really helping them or hindering the process? Is this why excel is predominant in organizations- because it helps great analysts tell a story? How effective is it in doing that? I once saw a weak attempt at telling such a story in excel and it involved putting hints along with the data – small comments here and there that help you draw certain conclusions from the mess of numbers thrown at you – a first attempt at least but nonetheless a weak one.

Is “Office Integration” all we could come up with? What about putting serious thought into putting the ability to tell the story into the technology? Giving the great analyst the ability to tell their story right alongside the fancy charts and graphs? And NOT asking the users to click?

My guess is that this would transform BI – and allow decisionmakers to actually benefit from having those expensive DWBI solutions around.

I have decided to dedicate my free time to pursuing research on how the information is best presented to make decisionmaking easier – and how technology can aid it as opposed to hindering it with sheer complexity. If you are reading this and are aware of any such research being conducted now or having been in the past, please contact me at

And the next time you present data - don’t forget to tell a story  about it.

Written by Shubho Ghosal

June 26, 2010 at 2:55 am

Sell me the whole car! Not the pieces of it!

with one comment

I wrote an earlier post on requirements for the next generation of DWBI technology directed towards product vendors. This is a follow up on that and an analogy that popped into my mind later about the direction that should be taken by DWBI vendors in general.

Currently, vendors have provided us the components needed to build a car (DWBI solution), but the expectation is that we build it ourselves.

Some have provided us an engine (database platform) – and some have given us a vastly improved and powerful one (columnar dbs and dw appliances). Others have given us a dashboard for the vehicle (BI), and an entertainment system (in-memory analytics). Others wheels (ETL).

Now, every organization just needs to assemble the team of automotive engineers and designers to put it all together. Simple, right? Not.

Why not sell the whole car? There are at least six major vendors that are currently producing all the components required. There are others (open source vendors) that can easily collaborate. How difficult is it to take the next step to start producing cars that we can actually start driving in, that are actually good and can be customized easily?

Easier said than done. But critically important. This will change the ballgame, in my opinion.

Written by Shubho Ghosal

December 12, 2009 at 5:43 pm


Get every new post delivered to your Inbox.