Tag Archive for Machine Learning

Applied Machine Learning – Hospitality and Hotel Management

Machine Learning, or Predictive Analytics, is in common use in the travel and hospitality industry.  Online travel sites such as Expedia, Priceline and Trip Advisor have been putting data to use for over a decade, and they are very good at selling us stuff with Machine Learning.  They start by figuring out what someone will pay (relative to other customers) for a travel service, classifying preferences for all of the travel services they have available to sell, and recommending the next thing a website visitor need.  Flying to Vegas you say?  It’s very likely a hotel or car rental is in your future.  But which hotel offers will keep you on their site, buying from Expedia and not Expedia’s competitors?  Do you expect a premium hotel experience?  Do you need a Fiat or a GMC Acadia?  Are you a fan of Blue Man Group, Celine Dion or Penn and Teller?

And they usually get it right! They are using what is called a Recommendation Engine in order to guide customers through a series of buying decisions, based upon patterns and preferences which are similar to other customers.  Underneath, it’s just math, but the impact is undeniable.  It enables companies to predict what their customers want, and then (this is the critical piece) anticipate and deliver a response to that want, at a time when a customer truly needs it and is willing to pay for it.  Right thing, right place, right time.

Recommender is one of four core Machine Learning problems.  These four problems represent what ML can solve.  Now let’s apply Machine Learning to another business model within the Hospitality vertical: Hotel Management.  Is machine learning relevant to hotels?   If an algorithm can anticipate a guest’s need, what can hotel staff do with that information? The answer is: create a customer for life.

One common misconception about Machine Learning is that, in order to produce any useful insights, you need tons and tons of data (or, Big Data).  Property Management Systems (PMS) used by hotels and hotel groups are proving this wrong: they are using ML on a small scale to manage revenues, improve profitability and delight their customers.  Here are three examples of how hotel and property management groups are using Machine Learning to manage revenue today.

Market Segmentation – Customer segmentation is not a new thing.    We understand the difference between family vacationers and business travelers – those are big segments. But Machine Learning goes further, helping companies discover segments they may not realize existed. Which customers want to be near the pool, and which ones need three morning papers before they can even get dressed in the morning? Armed with this knowledge, hotels understand what matters to guests, at the individual level, enabling them to anticipate their needs immediately. Even more, hotels can understand key characteristics of their most profitable customers, and recognize the next one when they login to the online reservation site.

Demand Forecasting – what kinds of services will be in demand throughout the year? Can a hotel optimize room offerings to suit changing demand patterns, and perform renovations or scheduled maintenance in a way that has no net impact on profitability? While hotels may have limited ability to grow or shrink their inventory of rooms, management can optimize resource needs in order to respond to predicted demand levels.

Price Optimization – airlines understand very well that different customers are happy to pay different prices for the same airplane seat.  The same is true for hotel rooms.  Often, profitable customers are ones who are least price sensitive, but some are quite price sensitive – choosing to spend their money on other hotel services offered.

Machine Learning may seem intimidating, especially if you are assuming you’ll need Big Data and massive computing power in order to yield an investment.  Actually, the key ingredient is knowledge of your industry, and what you, as a services provider, can offer customers.  Most hotels and hotel groups have enough core guest information to begin deriving valuable insights and begin competing for customers in the new world of data.

Contact Blum Shapiro Consulting to learn more about how Azure Machine Learning can deliver these kind of predictive capabilities to your business, leading to sustained customer loyalty and profitable business outcomes.

3 Tips to Jump Start your Data Science Plan

Are you looking to form a Data Science capability at your company?

If you answered, yes, then you probably already get the Machine Learning concept (The 4 Machine Learning Problems).  Maybe you are coming from either a Statistics or Computer Science background.  Either way, you see the potential of Data Science and Predictive Analytics and you’re ready to demonstrate some tangible benefit to management.

How are you getting started?  I’m hearing about two core hurdles:

  1. We’re looking for a great business problem to solve, one which could reasonably be solved with data the business already collects
  2. Our internal resources have very little practical experience working on a formal data science team, and don’t understand how it aligns to more traditional project teams

Time to Value is critical, but you need to do it in a way that has a formal process for managing risk, one which can be communicated inside and outside the team.  Here are the things you want to have in place, in order to launch your first project.

Establish Your Data Science Methodology – every project has a project plan and data science projects are no different.  What should the Data Science one look like?  Several teams of very smart people have already asked this question and independently arrived at the same conclusion.  My favorite is the “Cross Industry Standard Process for Data Mining”  (CRISP-DM) because it calls out the need for basic Business Understanding of the problem first.  Basically there are 6 phases of the process

  1. Set the Business Objectives
  2. Find the Data
  3. Prep and Cleanse the Data
  4. Do the Machine Learning Work
  5. Evaluate the Model You Created – does it meet the Business Objectives?
  6. Deploy the Model

Need a picture?  Note the backwards arrows – Data Science is an iterative process.

Assess your Data Capabilities – Data Science needs Data.  Teams that try to predict outcomes without relevant data are setup for failure.  An example: let’s say that you would like to forecast demand for your products, in order to reduce your inventory.  You might start with basic sales data and find that you are not getting the  level of prediction accuracy you expected.  What other factors might be driving demand?  Customer Satisfaction might be one you decide to include.  But what if your company is not measuring customer satisfaction in any quantifiable way?  Data Science leaders need to understand the capabilities of their company (in effect, the Data Science customer) with respect to data assets, in order to effectively determine which business problems are ripe for prediction.

Outsource the Team – Data Science requires a very specialized set of skills.  You probably have some of those skills yourself: Computer Science, Statistics and an understanding of the principles behind Machine Learning.  These three are important, but equally important is Business and Domain Knowledge.  Do you have a team of resources which possess all four?  If you are working with a technology provider who already understands your business and who also has demonstrated capability in  delivering data science value – then outsourcing the work to that team becomes very attractive.  If you don’t have such a resource, consider a business and technology consulting partner such as Blum Shapiro Consulting.  Provided you already understand the CRISP-DM process, you’ll be able to effectively manage a seasoned team of business and data science pros.

Can Data Science increase your bottom line?  Improve Customer Loyalty?  Drive down costs?  Yes it can, provided you have a methodology to manage the work as a project, data to support it and a capable team.  If you’re convinced the opportunity is there, follow these tips and Data Science will have a strategic role within your company after your first big win!



The 4 Machine Learning Problems, Explained

Machine Learning and Predictive Analytics have been receiving a lot of attention lately!  Without question, this is an exciting technology with extremely broad applicability.  After all, who wouldn’t want to be able to predict the future?  Still, with hype comes confusion, and there is a lot of confusion today about what exactly Machine Learning is and how to use it.

I have good news!  There are really only 4 (yes, four) Machine Learning problems.  For anyone who wants to explore the value of Machine Learning, it’s important to understand them, because the first step in any Machine Learning process is to figure out which of these problems you are trying to solve.  Data Science teams address this question before they begin designing a Machine Learning model.  If your problem does not fit into one of these buckets, forget the hype! You’re better off taking a simpler approach.

Classification – in this machine learning problem, we’re trying  to figure out if some bit of data (an observation) represents something simple which we already understand (a Label).  This label can either be a Yes or No decision, (Two Class) or it can be one of a set of possible answers (Multi Class).  In order for this to work well, you need to provide the Machine Learning model with examples first.  Applications include:

  1. Facial Recognition – is this picture an image of my customer?
  2. Voice Recognition – what word is represented by this sound?
  3. Handwriting Recognition – which letter in the alphabet does this image represent?
  4. Fraud Detection – is this transaction fraudulent?
  5. Medical Outcomes – will this person have a stroke in the next year?
  6. Proactive Maintenance – will this piece of machinery fail in the next 72 hours?
  7. Credit Default Risk – will this borrower default on his/her loan?

Regression – in this machine learning problem, a Yes or No answer is not going to be enough.  In order to solve this problem, the machine needs to predict a value (i.e. a price, a temperature, a measurement) by understanding the numeric relationship of that value to other values (or Factors).  If you took Calculus, this might sound like a simple “Rate of Change” function: you’re on the right track.  Just as with Classification, Regression problems need some examples in order to work well.  Applications include:

  1. Cost Analysis – when will be the best time to buy something?
  2. Demand Prediction – how many widget’s will we sell next year?

Clustering – this is where things get complicated (!!)  With the first two problems, we have examples we can use to “train” our machines to predict a label AND we can test them with labeled observations (known to Data Scientists as “Ground Truth”).  But what if we don’t have a ground truth?  The best we can do is identify clusters of observations.   Fair warning: without ground truth, evaluating the results will be a challenge.  Still, some applications include:

  1. Grouping of Content – Grouping Today’s News into Categories, or Documents into Topics
  2. Materials Classification – take a Raw Materials Master File and organize it into a taxonomy
  3. Customer Segmentation – identify similar customers based upon purchase behavior

Recommender – have you ever been on a website which presented a recommendation of something you might “Like”?  Movie recommendations on Netflix, product recommendations on Amazon, or advertisements on your apps – if you are familiar with the internet, you probably understand the premise here.

That’s it.  Now you know how to recognize a problem which Machine Learning can help you with.   If your business problem does not fall into one of these four, you don’t need a machine learning model to solve it.  More importantly, if you know the factors which drive a business outcome, just build a model in Excel – you don’t need a Data Science team for that.

Good luck!

6 Critical Technologies for the Internet of Things

Picture7If you and your company prefer Microsoft solutions and technologies, you may be fearing that the Internet of Things is an opportunity which will pass you by.

Have no fear: Microsoft’s transformation from “Windows and Office” company to “Cloud and Services” company continues to accelerate.  Nowhere is this trend more evident than in the range of services supporting Internet of Things scenarios.

So – What are the Microsoft technologies that would comprise an Internet of Things solution architecture?

And – How do Cloud Computing and Microsoft Azure enable Internet of Things scenarios?

Here are the key Microsoft technologies which architects and developers need to understand.

Software for Intelligent Devices

First, let’s understand the Things.  The community of device makers and entrepreneurs continues to flourish, enabled by the emergence of simple intelligent devices.  These devices have a simplified lightweight computing model capable of connecting machine-to-machine or machine-to-cloud. Windows 10 for IoT, released in July 2015, will enable secure connectivity for a broad range of devices on the Windows Platform.

Scalable Event Ingestion

The Velocity of Big Data demands a solution capable of receiving telemetry data at cloud scale with low latency and high availability.  This component of the architecture is the “front-end” of an event pipeline which will sit between the Things sending data and the consumers of the data.  Microsoft’s Azure platform delivers this capability with Azure Event Hubs – extremely easy  to setup and connect to over HTTPS.

Still – Volume + Velocity lead to major complexity when Big Data is consumed; the data may not be ready for human consumption. Microsoft provides options to analyze this massive stream of “Fast data”.  Option 1 is to process the events “in-flight” with Azure Stream Analytics.  ASA allows developers to combine streaming data with Reference Data (e.g. Master Data) to analyze events, defects, “likes” and summarize the data for human consumption.  Option 2 is to stream the data to a massive storage repository for analysis later (see The Data Lake and Hadoop).  Regardless of whether you analyze in flight or at rest, a third option can help you learn about what is happening behind the data (see Machine Learning).

Machine Learning

We’ve learned a lot about “Artificial Intelligence” over the past 10 years.  Indeed, we’ve learned that machines “think” very differently than humans.  Machines use principles of statistics to assess which features (“columns”) of a dataset provide the most “information” about a given observation (“row”).  For example, which variable(s) are most predictive (or closely correlated) with the final feature of the dataset?  Having learned how the data is related to one another, a machine can be “trained” to predict the outcome of the next record in the dataset; given an algorithm and enough data – a machine can learn about the real world.

If the IoT solution you envision includes predictions or “intelligence”, you’ll want to look at Azure Machine Learning.  Azure ML provides a development studio for data science professionals to design, test and deploy Machine Learning services to the Microsoft Azure Cloud.

Finally, you’ll also want to understand how to organize a data science project within the structure of your company’s overall project management processes.  The term “Data Science” is telling – it indicates an experimental aspect to the process.  Data scientists prepare datasets, conduct experiments, and test their algorithms (written in statistical processing languages like “R” and “Python”) until the algorithm accurately predicts correct answers to questions posed by the business, using data.  Data Science requires a balance between experimentation and business value.

The Data Lake and Hadoop

A Data Lake is a term used to describe a single place where the huge variety of data produced by your big data initiatives is stored for future analysis.  A Data Lake is not a Data Warehouse.  A Data Warehouse has One Single Structure; data from a variety of formats must be transformed into that structure.  A Data Lake has no predefined structure.  Instead, the structure is determined when the data is analyzed.  New structures can be created over and over again on the same data.

Businesses have the choice of simply storing Big Data in Azure Storage.  If the data velocity and volume exceed certain limits of Azure Storage, Azure Data Lake is a specialized storage service optimized for Hadoop, with no fixed limits on file size.  Azure Data Lake is a service announced in May 2015, and you can sign up for the Public Preview.

The ability to define a structure as the data is read is the magic of Hadoop.   The premise is simple – Big Data is too massive to move from one structure to another, as you would in a Data Warehouse/ETL solution.  Instead, keep all the data in its native format, wait to apply structure until analysis time, and perform as many reads over the same data as needed.  There is no need to buy tons of hardware for Hadoop: Azure HDInsight provides Hadoop-as-a-Service, which can be enabled/disabled as needed to keep your costs low.

Real Time Analytics

The human consumption part of this equation is represented by Power BI.  Power BI is the “single pane of glass” for all of your Data Analysis needs, including Big Data.  Power Bi is a dashboard tool capable of transforming company data into rich visuals. It can connect to data sources on premises, consume data from HDInsight or Storage, and receive real-time updates from data “in-flight”.  If you are located in New England, attend one of our Dashboard in a Day workshops happening throughout the Northeast in 2015.


IoT solutions are feasible because of the robust cloud offerings currently available.  The cloud is an integral part of your solution, and you need resources capable of managing your cloud assets as though they were on premise.  Your operations team should be comfortable turning on and off services in your cloud, just as they are comfortable enabling services and capabilities on a  server. Azure PowerShell provides the operations environment for managing Azure cloud services and automating maintenance and management of those services.


Enterprises ready to meet their customers in the digital world will be rewarded.  First, they must grasp Big Data technologies.  Microsoft customers can take advantage of the Azure cloud to create Microsoft Big Data solutions.  They are designed first by connecting Things to the cloud, then creating and connecting Azure services to receive, analyze, learn from, and visualize the data.  Finally, be ready to treat those cloud assets as part of your production infrastructure, by training your operations team in cloud management tools from Microsoft.