Archive for Business Intelligence

Using Real Time Data Analytics and Visualization Tools to Drive Your Business Forward

Businesswoman work with documents and tablet pc

Business leaders need timely information about the operations and profitability of the businesses they manage to help make informed decisions. But when information delivery is delayed, decision makers lose precious time to adjust and respond to changing market conditions, customer preferences, supplier issues or all three. When thinking about any business analytics solution, a critical question to ask is: how frequently can we (or should we) update the underlying data? Often, the first answer from the business stakeholders is “as frequently as possible.” The concept of “real time analytics,” with data being provided up-to-the minute, is usually quite attractive. But there may be some confusion about what this really means.

While the term real time analytics does refer to data which is frequently changing, it is not the same as simply refreshing data frequently. Traditional analytics packages which take advantage of data marts, data warehouses and data cubes are often collectively referred to as a Decision Support System (DSS). A DSS helps business analysts, management and ownership understand historical trends in their business, perform root cause analysis and enable strategic decisions. Whereas a DSS system aggregates and analyzes sales, costs and other transactions, a real time analytics system ingests and processes events. One can imagine a $25 million business recording 10,000 transactions a day. One can imagine that same business recording events on their website: login, searches, shopping cart adds, shopping card deletes, product image zoom events. If the business is 100% online, how many events would that be? The answer may astonish you.

Why Real Time Analytics?

DSS solutions answer questions such as “What was our net income last month?”, “What was our net income compared to the same month last year?” or “Which customers were most profitable last month?” Real time analytics answers questions such as “Is the customer experience positive right now?” or “How can we optimize this transaction right now?” In the retail industry, listening to social media channels to hear what customers are saying about their experience in your stores, can drive service level adjustments or pricing promotions. When that analysis is real-time, store managers can adjust that day for optimized profitability. Some examples:

  1. Social media sentiment analysis – addressing customer satisfaction concerns
  2. Eliminating business disruption costs with equipment maintenance analytics
  3. Promotion and marketing optimization with web and mobile analytics
  4. Product recommendations throughout the shopping experience, online or “brick and mortar”
  5. Improved health care services with real time patient health metrics from wearable technology

In today’s world, customers expect world class service. Implicit in that expectation is the assumption that companies with whom they do business “know them”, anticipate their needs and respond to them. That’s easy to say, but harder to execute. Companies who must meet that expectation need technology leaders to be aware of three concepts critical to making real time analytics a real thing.

The first is Internet of Things or IoT. The velocity and volume of data generated by mobile devices, social media, factory floor sensors, etc. is the basis for real time analytics. “Internet of Things” refers to devices or sensors which are connected to the internet, providing data about usage or simply their physical environment (where the device is powered on). Like social media and mobile devices, IoT sensors can generate enormous volumes of data very, very quickly – this is the “big data” phenomenon.

The second is Cloud Computing. The massive scale of IoT and big data can only be achieved with cloud scale data storage and cloud scale data processing. Unless your company’s name is Google, Amazon or Microsoft, you probably cannot keep up. So, to achieve real-time analytics, you must embrace cloud computing.

The third is Intelligent Systems. IBM’s “Watson” computer achieved a significant milestone by out-performing humans on Jeopardy. Since then, companies have been integrating artificial intelligence (AI) into large scale systems. AI in this sense is simply a mathematical model which calculates the probability that data represents something a human would recognize: a supplier disruption, a dissatisfied customer about to cancel their order, an equipment breakdown. Using real time data, machine learning models can recognize events which are about to occur. From there, they can automate a response, or raise an alert to the humans involved in the process. Intelligent systems help humans make nimble adjustments to improve the bottom line.

What technologies will my company need to make this happen?

From a technology perspective, a clear understanding of cloud computing is essential. When evaluating a cloud platform, CIO’s should look for breadth of capability and support for multiple frameworks. As a Microsoft Partner, BlumShapiro Consulting works with Microsoft Azure and its Cortana Intelligence platform. This gives our clients cloud scale, low cost and a wide variety of real time and big data processing options.

CIO Article 1

This diagram describes the Azure resources which comprise Cortana Intelligence. The most relevant resources for real time analytics are:

  1. Event Hubs ingest high velocity streaming data being sent by Event Providers (i.e. Sensors and Devices)
  2. Data Lake Store provide low cost cloud storage which no practical limits
  3. Stream Analytics perform in-flight processing of streaming data
  4. Machine Learning, or AzureML, supports the design, evaluation and integration of predictive models into the real-time pipeline
  5. Cognitive Services are out-of-the-box Artificial Intelligence services, addressing a broad range of common machine intelligence scenarios
  6. Power BI supports streaming datasets made visible in a dashboard context

Four Steps to Get Started with Real Time Analytics

Start with the Eye Candy – If you do not have a dashboard tool which supports real-time data streaming, consider solutions such as Power BI (link to our web page). Even if you are not ready to implement an IoT solution, Power BI makes any social media or customer marketing campaigns much more feasible. Power BI can be used to connect databases, data marts, data warehouses and data cubes, and is valuable as a dashboard and visualization tool for existing DSS systems. Without visualization, it will be very difficult to provide human insights and actions for any kind of data, slow or fast.

Get to the Cloud – Cloud storage costs and cloud processing scale are the only mechanisms by which real time analytics is economically feasible (for most companies). Learn how investing in technologies like Cloud Computing can really help move your business forward.

Embrace Machine Intelligence – To make intelligent systems a reality, you will need to understand machine learning technologies, if only at a high level. Historically, this has meant developing a team of data scientists, many of whom have PhD’s in Mathematics or Statistics, and open source tools like R or Python. Today, machine learning is much more accessible then it has ever been. AzureML helps to fast track both the evaluation and operationalization of predictive models.

Find the Real-Time Opportunity – As the technology leader in the organization, CIO’s will need to work closely with other business leaders to understand where real-time information can increase revenue, decrease costs or both. This may require imagination. Start with the question – what would we like to know faster? If we knew our customer was going to do this sooner, how would we respond? If we knew our equipment was going to fail sooner, how would we respond? If we knew there was an opportunity to sell more, how would we respond?

Berry_Brian-240About Brian: Brian Berry leads the Microsoft Business Intelligence and Data Analytics practice at BlumShapiro. He has over 15 years of experience with information technology (IT), software design and consulting. Brian specializes in identifying business intelligence (BI) and data management solutions for upper mid-market manufacturing, distribution and retail firms in New England. He focuses on technologies which drive value in analytics: data integration, self-service BI, cloud computing and predictive analytics

Power BI Demo CTA


Applied Machine Learning: Optimizing Patient Care in Hospitals, Profitably

Health Care

Why are so many industries exploring Machine Learning as a means of delivering innovation and value?  In my view, the technology speaks to a primitive urge – machine learning is like having a crystal ball, telling you what will happen next.  For a business, it can convey information about a customer before they introduce themselves.  On a personal level, when I consider what I would like to have information about in advance, the first thing that springs to mind is obvious: my health.  Am I about to get sick?  How can I improve my wellness and overall health?  If you are wearing a Fit-Bit right now, then you probably agree with me.

In my last blog, I shared some real-world examples for how the Hospitality Industry applies Machine Learning.  What about Health Care and Hospitals?  While hospitals have similar challenges, in that they accommodate guests who stay overnight, the objectives in health care are quite different, and changing rapidly.   The Affordable Care Act is driving new business models, incentivizing outcome based reimbursement as opposed to volume based reimbursement.  Unlike hotels, today’s hospitals are interested in ensuring their guests do not have to return, at least not in the short term.   They also need to manage costs in a way they have not been incentivized to do in the past.  Hospitals across the country are considering how predictive analytics can have a  meaningful impact on operations, leading to improved patient health and improving the bottom line.

The cost savings opportunity for health care providers is startling. Here are just three examples:

Reducing Hospital Readmissions – in 2014, Medicare fined 2,610 hospitals $428 million for having high hospital readmission rates. Leaving actual fines aside, industry analysts estimate that the overall cost of preventable readmissions approaches $25 Billion annually. As a result, hospital systems all over the nation are mobilizing to intervene, using ML to identify risk factors which are highly predictive of readmission. Carolinas Healthcare System, partnering with Microsoft, did just that. Using data from 200,000 patient-discharge records, they created a predictive model deliver customized discharge planning, saving the hospital system hundreds of thousands of dollars annually. Read the article in Healthcare IT News.

Clinical Variation Management – Mercy Hospital is partnering with Ayasdi to find the optimal care path for common surgical procedures. Using knee replacement as an example, the Clinical Variation Management software helps hospital administrators find clusters of patient outcomes, then enables the exploration of those clusters in order to correlate a metric (i.e. Length of Stay) with a certain regiment or activity. Watch this video to learn how Mercy Hospital saved $50 million by applying Machine Learning to an extremely common procedure.

Improving Population Health – Dartmouth Hitchcock, a healthcare network affiliated with Dartmouth University, is piloting a remote monitoring system for patients requiring chronic care. 6,000+ patients are permitting the hospital to collect biometric data (i.e. blood pressure, temperature, etc.) in order that nurses and health coaches can monitor their vital signs, and machines can predict good days and bad days. Quite the opposite of hospitality: Dartmouth Hitchcock is trying to keep the guests from needing to checking in! Read more about the Case Study from Microsoft.

Are machines taking over for physicians? No. The Patient – Physician relationship remains (and I think will always remain) central to the delivery of personal health care. However, it seems clear that ACA is providing significant rewards to health care providers who manage population risk better. Machines can help here: through data, machine learning can find risk “hiding in the data”.

Contact Blum Shapiro Consulting to learn more about how Azure Machine Learning can curtail hospital readmissions, identify variations in common clinical procedures and improve patient population health.

Drive Your Business with a Dashboard, not Intuition

Dashboard on a Tablet

Business Owners and Senior Management should be constantly monitoring the performance of their business.  Some managers may accomplish this by talking to the people who work for them.  Ultimately, however, management is called upon to make decisions, and they do so with the information they have available to them.   Often the biggest challenge is bringing all of the data together into something which can be understood.

Most business leaders want to base their decisions upon clear, reliable information.  But, this can be a challenge.  Information can be difficult to obtain in a timely manner.  The accuracy of the data contained in reports is sometimes questionable.  Management reports may be extremely detailed without providing critical metrics which are easy to locate.  So what do you do?

If you are a manager with some of these issues, I would urge you to clear your head for a moment and go for a spin in your car.  That’s right – turn on the ignition, be careful in the parking lot, put on your favorite music and go for a 30 minute drive.  While you are there, take a moment to appreciate the amazing information solution right in front of you – your car dashboard.

A car dashboard exhibits three powerful concepts.

Consolidation– my dashboard prominently shows me my speed, my engine temperature, the gasoline remaining in my tank, among other things.  Where does this information come from? Generally, I know – but not specifically, and frankly, I’m not that interested.  But to answer the question, the data feeding the three dials I mention above each comes from three distinct components, undoubtedly manufactured by three separate vendors.   Can you imagine if your car started behaving the way most IT departments do? “I’m sorry, but we can’t tell you how much gas you have left, because that information is in a different system.” I know what I would do. I’d get a new car.

Context – I’m not an auto mechanic. I have no idea what the correct oil pressure should be in my car, or at what temperature my engine overheats. Fortunately, my dashboard is color coded. When the dial goes Red, that is bad. I need to take corrective action to move the dial out of the Red, into the Yellow or Green.

Relevance – for some pieces of information, any kind of numeric measurement is not useful. I simply want to know if I should be concerned or take action. For example, the alert that tells me that I don’t have my seatbelt on (and the car is moving). Or the one that tells me that I need to check my engine.  Fortunately, the dashboard does not provide me with a diagnostic code – because I would not know how to interpret it.  But I do know how to respond to a Check Engine light.

Operating a business is not very different from operating an automobile.  In both cases, you are working with a sophisticated, complex system which has a lot of moving parts.  Further, you need to keep your eyes on the road in front of you.  Driving or operating your vehicle is not the time to be looking for information, and getting that information can mean having to take your car out of service, costing money.

Unlike automobiles, it’s up to you as the manager of the business to decide which metrics are Key Performance Indicators – the ones you need to keep your eye on to ensure optimal performance.  Therefore, recognize that you will need to address your organizational readiness to begin monitoring your business in the same manner.

  1. Define your strategy – have a clear understanding of what the critical metrics are, and remove data and information which is not important to monitor.
  2. Embrace Data Driven Decision making – organizations which have committed to data driven decision making take good care of their operational data, attending to data quality issues as they arise.
  3. Keep it Actionable – Your envisioned dashboards will need right-time access to operational data – depending upon what you are looking to accomplish, you may or may not need “real-time information”. What is important is that you have information in time to make a correction, and don’t spend time and money investing on technology capabilities which your organization does not need.
  4. Keep it simple – You’ll need easy to use tools, both for IT and Management – and there are plenty of them available today!

By implementing a dashboard, or set of dashboards for your business, you can expect to be able to apply better focus to the critical processes that drive your business, because “What gets measured gets done”.  You’ll make better decisions, and also make these good decisions in time to take impactful action.  Finally, dashboards can help you communicate your strategy and performance to your management team.




4 Reasons Power BI is Better than Tableau, Qlik Sense


I enjoyed reading this article by Martin Heller in which the author analyzed three top Data Visualization products: Tableau, Qlik Sense and Power BI.  Heller does a nice job explaining how these products represent an evolution in BI, making real data insights attainable for non-IT business users.  Mr. Heller says that each of these make self-service BI “remarkably easy” for users throughout the organization, but that in his opinion Tableau stood out as the best of the three.

But if Heller’s analysis is correct, then his conclusion makes no sense.  He first cautions that none of these products is well suited for Enterprise Reporting;: an important point – if you are looking for financial reporting, you are looking in the wrong place.  He then details each product’s features at length, with a focus on ease of use and the breadth of visualizations available in each.  He notes that there is very little difference in mobile capabilities (the Android app for Power BI is an exception, slated for September release).  Finally, he concedes that Power BI offers the best value as compared to Qlik Sense, and significantly better value than Tableau.  My question to Martin is this:  if you cannot afford Tableau or Qlik Sense licenses, and therefore cannot truly democratize business insights from data in your organization – what difference does the rest make?

Here are four reasons why Power BI is the smart choice:

1. Better Value – I’m not talking about a few dollars here and there, I’m talking 4x – Power BI Professional is $10 per month per user, Tableau Online is $500 per year per user.  I’ll let you do the math.  Mr. Heller does state that Qlik Sense is less expensive than Tableau, but does not go into specifics.

2. SaaS model – who wants another server on-premises to publish BI reports and dashboards?  Business users love cloud services: they pay a simple monthly subscription for the insights and visualizations they need, and  they don’t need to ask IT for anything!  Further, they get accelerated product updates from the cloud – much faster than traditional IT shops can maintain.  The net result is that vendors inevitably achieve parity, and the question “which product has the best visualizations?” quickly becomes a zero sum game.

3. Q&A – Power BI is the only product with Natural Language Query capabilities.  You simply type a question (i.e. “What was our customer churn rate in the past 3 months?”) and Power BI selects a visualization for you to explore.  The visualization chosen may or may not have been created by the dashboard’s author.

4. Power Query – all of these tools offer simple connectors to databases, Hadoop, CSV files, cloud data providers, but neither Tableau nor Qlik Sense provides a data shaping tool with the capability of Power Query.  For real power data users, Power Query’s M Language has all the capability of an IT Pro’s data transformation package, with none of the IT headache.  Power Query can be used in Excel, and it can also be used in the Power BI Desktop application. In either case, analysts consume and shape data on the desktop, build reports and publish to the cloud.

But is Power BI easy to use?  Yes it is – I invite you to come see for yourself! Dashboard in a Day is a Power BI workshop which Blum Shapiro Consulting is hosting at various locations in New England through the remainder of 2015.   We are offering Dashboard in a Day sessions at No Cost to participants.

Our first Dashboard in a Day workshop will be hosted by Microsoft at the Hartford, CT office on Thursday August 27th.  We’ll be teaching and working with 12 participants starting promptly at 9:00 AM.  Those of you who would like to “Bring Your Own Data” are invited stay for a working lunch where we’ll help you get started with your data (in Excel or CSV format, or organizational data stored in SalesForce, Google Analytics, Marketo, Dynamics CRM).  Register to attend here