Archive for Internet of Things

Using Real Time Data Analytics and Visualization Tools to Drive Your Business Forward

Businesswoman work with documents and tablet pc

Business leaders need timely information about the operations and profitability of the businesses they manage to help make informed decisions. But when information delivery is delayed, decision makers lose precious time to adjust and respond to changing market conditions, customer preferences, supplier issues or all three. When thinking about any business analytics solution, a critical question to ask is: how frequently can we (or should we) update the underlying data? Often, the first answer from the business stakeholders is “as frequently as possible.” The concept of “real time analytics,” with data being provided up-to-the minute, is usually quite attractive. But there may be some confusion about what this really means.

While the term real time analytics does refer to data which is frequently changing, it is not the same as simply refreshing data frequently. Traditional analytics packages which take advantage of data marts, data warehouses and data cubes are often collectively referred to as a Decision Support System (DSS). A DSS helps business analysts, management and ownership understand historical trends in their business, perform root cause analysis and enable strategic decisions. Whereas a DSS system aggregates and analyzes sales, costs and other transactions, a real time analytics system ingests and processes events. One can imagine a $25 million business recording 10,000 transactions a day. One can imagine that same business recording events on their website: login, searches, shopping cart adds, shopping card deletes, product image zoom events. If the business is 100% online, how many events would that be? The answer may astonish you.

Why Real Time Analytics?

DSS solutions answer questions such as “What was our net income last month?”, “What was our net income compared to the same month last year?” or “Which customers were most profitable last month?” Real time analytics answers questions such as “Is the customer experience positive right now?” or “How can we optimize this transaction right now?” In the retail industry, listening to social media channels to hear what customers are saying about their experience in your stores, can drive service level adjustments or pricing promotions. When that analysis is real-time, store managers can adjust that day for optimized profitability. Some examples:

  1. Social media sentiment analysis – addressing customer satisfaction concerns
  2. Eliminating business disruption costs with equipment maintenance analytics
  3. Promotion and marketing optimization with web and mobile analytics
  4. Product recommendations throughout the shopping experience, online or “brick and mortar”
  5. Improved health care services with real time patient health metrics from wearable technology

In today’s world, customers expect world class service. Implicit in that expectation is the assumption that companies with whom they do business “know them”, anticipate their needs and respond to them. That’s easy to say, but harder to execute. Companies who must meet that expectation need technology leaders to be aware of three concepts critical to making real time analytics a real thing.

The first is Internet of Things or IoT. The velocity and volume of data generated by mobile devices, social media, factory floor sensors, etc. is the basis for real time analytics. “Internet of Things” refers to devices or sensors which are connected to the internet, providing data about usage or simply their physical environment (where the device is powered on). Like social media and mobile devices, IoT sensors can generate enormous volumes of data very, very quickly – this is the “big data” phenomenon.

The second is Cloud Computing. The massive scale of IoT and big data can only be achieved with cloud scale data storage and cloud scale data processing. Unless your company’s name is Google, Amazon or Microsoft, you probably cannot keep up. So, to achieve real-time analytics, you must embrace cloud computing.

The third is Intelligent Systems. IBM’s “Watson” computer achieved a significant milestone by out-performing humans on Jeopardy. Since then, companies have been integrating artificial intelligence (AI) into large scale systems. AI in this sense is simply a mathematical model which calculates the probability that data represents something a human would recognize: a supplier disruption, a dissatisfied customer about to cancel their order, an equipment breakdown. Using real time data, machine learning models can recognize events which are about to occur. From there, they can automate a response, or raise an alert to the humans involved in the process. Intelligent systems help humans make nimble adjustments to improve the bottom line.

What technologies will my company need to make this happen?

From a technology perspective, a clear understanding of cloud computing is essential. When evaluating a cloud platform, CIO’s should look for breadth of capability and support for multiple frameworks. As a Microsoft Partner, BlumShapiro Consulting works with Microsoft Azure and its Cortana Intelligence platform. This gives our clients cloud scale, low cost and a wide variety of real time and big data processing options.

CIO Article 1

This diagram describes the Azure resources which comprise Cortana Intelligence. The most relevant resources for real time analytics are:

  1. Event Hubs ingest high velocity streaming data being sent by Event Providers (i.e. Sensors and Devices)
  2. Data Lake Store provide low cost cloud storage which no practical limits
  3. Stream Analytics perform in-flight processing of streaming data
  4. Machine Learning, or AzureML, supports the design, evaluation and integration of predictive models into the real-time pipeline
  5. Cognitive Services are out-of-the-box Artificial Intelligence services, addressing a broad range of common machine intelligence scenarios
  6. Power BI supports streaming datasets made visible in a dashboard context

Four Steps to Get Started with Real Time Analytics

Start with the Eye Candy – If you do not have a dashboard tool which supports real-time data streaming, consider solutions such as Power BI (link to our web page). Even if you are not ready to implement an IoT solution, Power BI makes any social media or customer marketing campaigns much more feasible. Power BI can be used to connect databases, data marts, data warehouses and data cubes, and is valuable as a dashboard and visualization tool for existing DSS systems. Without visualization, it will be very difficult to provide human insights and actions for any kind of data, slow or fast.

Get to the Cloud – Cloud storage costs and cloud processing scale are the only mechanisms by which real time analytics is economically feasible (for most companies). Learn how investing in technologies like Cloud Computing can really help move your business forward.

Embrace Machine Intelligence – To make intelligent systems a reality, you will need to understand machine learning technologies, if only at a high level. Historically, this has meant developing a team of data scientists, many of whom have PhD’s in Mathematics or Statistics, and open source tools like R or Python. Today, machine learning is much more accessible then it has ever been. AzureML helps to fast track both the evaluation and operationalization of predictive models.

Find the Real-Time Opportunity – As the technology leader in the organization, CIO’s will need to work closely with other business leaders to understand where real-time information can increase revenue, decrease costs or both. This may require imagination. Start with the question – what would we like to know faster? If we knew our customer was going to do this sooner, how would we respond? If we knew our equipment was going to fail sooner, how would we respond? If we knew there was an opportunity to sell more, how would we respond?

Berry_Brian-240About Brian: Brian Berry leads the Microsoft Business Intelligence and Data Analytics practice at BlumShapiro. He has over 15 years of experience with information technology (IT), software design and consulting. Brian specializes in identifying business intelligence (BI) and data management solutions for upper mid-market manufacturing, distribution and retail firms in New England. He focuses on technologies which drive value in analytics: data integration, self-service BI, cloud computing and predictive analytics

Power BI Demo CTA

 

Building Meaningful Relationships Through IoT Driven Customer Engagement

iStock_000042834508_Full

We all know the importance of building meaningful relationships with our customers. Consumers want to know the products they buy are reliable. They want to see value from the services they pay for. They want to know the companies they’re doing business with are trustworthy, helpful and honest. Whether it be through social media, sending out newsletters and surveys, hosting webinars and seminars, or other activities, most (if not all) companies employ some sort of process or platform for engaging with their customers. With the emergence of new technology and the rise of smart devices companies are now positioned to take customer engagement to the next level.

How Can the Internet of Things (IoT) Help Your Customers?

What if you could help solve your customers’ problems before they have them? What if you could make meaningful recommendations and provide targeted guidance to your customers based on how they’re using your products? What if you could always be connected to your customers? You can, with the Internet of Things.

Based on data from the State of the Market: Internet of Things 2016 report from Verizon, IoT is having a large impact on how businesses are engaging their customers. Here are a few key points taken from the report.

  • 72% of organizations feel IoT is critical to their competitive advantage.
  • 76% of early movers in manufacturing say IoT is increasing insight into customer preferences and behavior.
  • 54% of early movers in healthcare are using IoT to enrich products and services with information
  • 81% of early movers in the public sector believe their citizens increasingly expect them to offer enhanced services using data from IoT.
  • 77% of retailers are seeing IoT change the customer experience.
  • 84% of retail early movers say their customers value exchanging information to improve their experience.
  • 83% of early movers in financial services say customer relationships are increasingly driven by ongoing service agreements rather than transactional product sales.

For me, one of the more interesting statistics was that in the public sector 81% of early movers believe citizens increasingly expect them to offer enhanced services using data from IoT. Using smart devices and sensors, data can be processed practically in real-time so that decisions can be made on the fly to enhance services. This information can then be relayed back to cloud servers to provide insight and drive policies to further enhance services while cutting costs. Soon customers will come to expect these types of enhancements, and companies that don’t follow suit may lose customers to competitors that are early movers. With more and more private sector companies across an array of industries leveraging IoT to engage their customers, are consumers going to begin expecting services enhanced by technology from all businesses? I believe many already do and that the trend is on the rise.

About Matt:

As a senior in BlumShapiro’s Technology Consulting Group, Matt has over 7 years of experience with Microsoft .NET software application development, including solutions for web, client/server and mobile platforms.

 

Matt Rankin IoT CTA

Internet of Things Modern Application Development

Technology in the hands of businessmen

Over the past decade modern application development has shifted from mainframe computing to personal PCs, and now to smartphones and cloud services. These shifts required new software languages, new hardware and new application development solutions. The best illustration of this shift came with the 90’s “Internet Boom.”  This shift resulted in application development on mainframe and personal PCs to applications that can run on a web browser. The shift also included new tools such as Visual Studio, new languages like HTML/JavaScript, new Architecture Patterns such as MVC and new application life cycle processes like Agile/Scrum.

Then came the smart phone. This shift from personal PCs to small mobile devices such as iPhones forced modern application development to support multiple screen resolutions, and a need to operate off-line while remaining connected to cloud services. Our next shift is to the Internet of Things (IoT), once again giving a new meaning to modern application development. Now, applications need to be developed to run on different types of devices like thermostats, doorbells and small Bluetooth sensors. The application must be secure, cloud ready and able to perform predictive analysis using machine learning. Below are my thoughts on this latest shift in modern application development:

Devices

The IoT modern application development shift includes a multitude of devices that range from televisions to cameras, to refrigerators, to pretty much any device that is powered into an outlet. One of the more notable products in this space is the Amazon Echo which uses voice recognition as its main interface, and can provide control over your light switches, thermostats and even your music collection. Amazon Echo is an example of an IoT device which breaks away from the previous modern application development, as it uses voice as its interface, is always connected to the cloud, and can connect with other IoT devices. This changes everything about how we think of modern application development. No longer is it about supporting multiple device resolutions, but rather about what data can be captured via the latest IoT devices and how that data can be used to improve our lives. This means we need new software tools, new cloud services, new analysis software and new machine learning algorithms.

These applications do not always include fancy user interfaces, as they are often function specific. For example, an IoT device could capture changes in temperature on a farm, take soil sample readings or even capture images and video of the fields. This data can then be sent to cloud services where it can be analyzed and run through machine learning to produce an easy to understand update on the farm. The data from the disparate “things” needs to be collected in a common format for actionable insights. Of note, most of the “big data” being processed and collected today is machine-to-machine. Cloud services help to aggregate and display this data in ways humans can understand, analyze and take action on the insights delivered.

Cloud Services

Cloud services are at the heart of IoT. Devices are built to perform a simple purpose and leave all complex user interfacing, analysis and thinking to the cloud. Cloud services such as the Azure IoT hub provide both the software tooling and service for a device to talk to the cloud and the device to connect to other devices. For example, in the manufacturing industry, IoT devices using the Azure IoT hub can be developed to monitor the production line and equipment use, which is then submitted to cloud service which then can be interpreted by human intelligence to predict equipment maintenance.

With this shift to IoT modern application development software is developed to capture data from a range of sensors, submit that data to cloud services and then process that data using analytics services such as Business Intelligence dashboards for timely and relevant role based information.

Machine Learning

So what is the point of these IoT devices in our homes, cars and at work, capturing data and sending it to the cloud? Well that’s what machine learning is all about. We now need to develop algorithms that can learn based on data from the IoT. For example: home IoT devices using machine learning will learn the normal patterns in your house and only notify you when there is a disruption such as the lights staying on past a normal pattern or when you leave your windows open while you are away. Machine learning is one of the most important aspects of IoT and without it, all we would have is raw data in a cloud service with no meaningful way to utilize it.

BlumShapiro Consulting is a Microsoft Advanced Analytics partner, with experience building modern IoT apps. 

Technology Talks Newsletter CTA

About Hector: 

hectorHector Luciano, Jr. is a Consulting Manager at BlumShapiro, a Microsoft Gold Partner focusing on SharePoint, Office 365, mobile technologies and custom development solutions. Hector is an active member of the SharePoint community. His experience reflects a breadth of Microsoft .Net Technologies experience. With a focus on Software Application development, Hector has worked on various projects including architected and designed solutions for web, client/server and mobile platforms. He has worked closely with business owners to understand the business process, then design and build custom solution. Hector currently holds Microsoft Certified Solution Developer (MCSD), Microsoft Certified Professional Developer (MCPD).

 

 

6 Critical Technologies for the Internet of Things

Picture7If you and your company prefer Microsoft solutions and technologies, you may be fearing that the Internet of Things is an opportunity which will pass you by.

Have no fear: Microsoft’s transformation from “Windows and Office” company to “Cloud and Services” company continues to accelerate.  Nowhere is this trend more evident than in the range of services supporting Internet of Things scenarios.

So – What are the Microsoft technologies that would comprise an Internet of Things solution architecture?

And – How do Cloud Computing and Microsoft Azure enable Internet of Things scenarios?

Here are the key Microsoft technologies which architects and developers need to understand.

Software for Intelligent Devices

First, let’s understand the Things.  The community of device makers and entrepreneurs continues to flourish, enabled by the emergence of simple intelligent devices.  These devices have a simplified lightweight computing model capable of connecting machine-to-machine or machine-to-cloud. Windows 10 for IoT, released in July 2015, will enable secure connectivity for a broad range of devices on the Windows Platform.

Scalable Event Ingestion

The Velocity of Big Data demands a solution capable of receiving telemetry data at cloud scale with low latency and high availability.  This component of the architecture is the “front-end” of an event pipeline which will sit between the Things sending data and the consumers of the data.  Microsoft’s Azure platform delivers this capability with Azure Event Hubs – extremely easy  to setup and connect to over HTTPS.

Still – Volume + Velocity lead to major complexity when Big Data is consumed; the data may not be ready for human consumption. Microsoft provides options to analyze this massive stream of “Fast data”.  Option 1 is to process the events “in-flight” with Azure Stream Analytics.  ASA allows developers to combine streaming data with Reference Data (e.g. Master Data) to analyze events, defects, “likes” and summarize the data for human consumption.  Option 2 is to stream the data to a massive storage repository for analysis later (see The Data Lake and Hadoop).  Regardless of whether you analyze in flight or at rest, a third option can help you learn about what is happening behind the data (see Machine Learning).

Machine Learning

We’ve learned a lot about “Artificial Intelligence” over the past 10 years.  Indeed, we’ve learned that machines “think” very differently than humans.  Machines use principles of statistics to assess which features (“columns”) of a dataset provide the most “information” about a given observation (“row”).  For example, which variable(s) are most predictive (or closely correlated) with the final feature of the dataset?  Having learned how the data is related to one another, a machine can be “trained” to predict the outcome of the next record in the dataset; given an algorithm and enough data – a machine can learn about the real world.

If the IoT solution you envision includes predictions or “intelligence”, you’ll want to look at Azure Machine Learning.  Azure ML provides a development studio for data science professionals to design, test and deploy Machine Learning services to the Microsoft Azure Cloud.

Finally, you’ll also want to understand how to organize a data science project within the structure of your company’s overall project management processes.  The term “Data Science” is telling – it indicates an experimental aspect to the process.  Data scientists prepare datasets, conduct experiments, and test their algorithms (written in statistical processing languages like “R” and “Python”) until the algorithm accurately predicts correct answers to questions posed by the business, using data.  Data Science requires a balance between experimentation and business value.

The Data Lake and Hadoop

A Data Lake is a term used to describe a single place where the huge variety of data produced by your big data initiatives is stored for future analysis.  A Data Lake is not a Data Warehouse.  A Data Warehouse has One Single Structure; data from a variety of formats must be transformed into that structure.  A Data Lake has no predefined structure.  Instead, the structure is determined when the data is analyzed.  New structures can be created over and over again on the same data.

Businesses have the choice of simply storing Big Data in Azure Storage.  If the data velocity and volume exceed certain limits of Azure Storage, Azure Data Lake is a specialized storage service optimized for Hadoop, with no fixed limits on file size.  Azure Data Lake is a service announced in May 2015, and you can sign up for the Public Preview.

The ability to define a structure as the data is read is the magic of Hadoop.   The premise is simple – Big Data is too massive to move from one structure to another, as you would in a Data Warehouse/ETL solution.  Instead, keep all the data in its native format, wait to apply structure until analysis time, and perform as many reads over the same data as needed.  There is no need to buy tons of hardware for Hadoop: Azure HDInsight provides Hadoop-as-a-Service, which can be enabled/disabled as needed to keep your costs low.

Real Time Analytics

The human consumption part of this equation is represented by Power BI.  Power BI is the “single pane of glass” for all of your Data Analysis needs, including Big Data.  Power Bi is a dashboard tool capable of transforming company data into rich visuals. It can connect to data sources on premises, consume data from HDInsight or Storage, and receive real-time updates from data “in-flight”.  If you are located in New England, attend one of our Dashboard in a Day workshops happening throughout the Northeast in 2015.

Management

IoT solutions are feasible because of the robust cloud offerings currently available.  The cloud is an integral part of your solution, and you need resources capable of managing your cloud assets as though they were on premise.  Your operations team should be comfortable turning on and off services in your cloud, just as they are comfortable enabling services and capabilities on a  server. Azure PowerShell provides the operations environment for managing Azure cloud services and automating maintenance and management of those services.

Conclusion

Enterprises ready to meet their customers in the digital world will be rewarded.  First, they must grasp Big Data technologies.  Microsoft customers can take advantage of the Azure cloud to create Microsoft Big Data solutions.  They are designed first by connecting Things to the cloud, then creating and connecting Azure services to receive, analyze, learn from, and visualize the data.  Finally, be ready to treat those cloud assets as part of your production infrastructure, by training your operations team in cloud management tools from Microsoft.