Send Custom Emails Using Azure Functions as Scheduled Tasks in SharePoint Online

Recently, a client of ours was looking to have a daily email sent to each user that has an overdue task, or a task that is set to expire. The email had a specific format. Each task needed direct links and had to be grouped by the date that it was due. Since it was an on-premises Project Server and SharePoint 2013 farm, it was not too difficult to build and embed the solution into the farm. I took care of this by building a SharePoint timer job, which leveraged a SharePoint search to retrieve all tasks where the due date was before the next business day. Once deployed, and activated, this timer job was automatically scheduled to run every morning, and the SharePoint admins could trigger it manually.  Everything worked great.

Another client of ours was looking for a solution exactly like this, except they were strictly SharePoint Online / Project Online. They had no on-premises farm, there were no real servers to even speak of. One option would have been to create a PowerShell script or .NET executable to run the code, and have that process run as a Scheduled Task on some server. However, there were no servers. Even if they did, what was the point of being in the cloud, if you are still stuck with a foot (or process) on the ground?

So, I turned to Microsoft Azure, and that’s where Azure Functions came into play.

Azure Functions

Azure Functions are programs or code snippets that run in the cloud. They can run on schedules or can be triggered by different types of events (HTTP request, item added to Azure Blob Storage, Excel file saved to OneDrive, etc.). Think of this as a Windows Scheduled Task that can be triggered by modern events and activities.

The programs or code snippets can be created and edited within the Azure Portal, making it easy to get up and running with an Azure Function. The languages supported by Azure Functions are more than adequate: PowerShell, C#, JavaScript, F#, Python, PHP, Bash, and Batch.

Note that I could have also used Azure WebJobs to accomplish this, but I felt that Azure Functions had many positives. Azure Functions are easy for the client to maintain, it has automatic scaling, they only pay for how long the code executes, it supports WebHooks and can be easily triggered via an HTTP request.

Send Custom Emails from SharePoint Online

For this solution, I created the Azure Function in Visual Studio pre-compiled with the SharePoint Online client-side object model (CSOM) DLLs. The solution was straightforward, as it would use CSOM to query SharePoint Online’s search service for all overdue tasks and tasks due within the next business day. It would then do some logic to build the email content based on the assigned users, and then send out emails using SendGrid. SendGrid is built into Microsoft Azure, so configuring it was a breeze, and you get 25,000 free emails a month!

Once deployed, I configured the Azure Function to run on schedule (like before), and it can even be triggered by an HTTP request, so putting an HTTP request in a SharePoint site workflow or Microsoft Flow means that any site user would be able to trigger this function as needed.

Long gone are the days where there are integration servers laying around in the data center waiting to get more processes to help them consume more of their over-allocated resources. Most servers, virtual machines, really, are now dedicated to a specific application, and shouldn’t share their resources with one-off processes.

Azure Functions is a great server-less architecture solution to these problems. Whether you need to send emails, calculate metrics, or analyze big data, Azure Functions can be a solution for you. Learn more about how BlumShapiro can help your organization with issues like this.

About Brent:

Brent

Brent Harvey has over 10 years of software development experience with a specific focus on SharePoint, Project Server, and C #and web development. Brent is an Architect at BlumShapiro Consulting. Brent is a Microsoft Certified Solutions Expert in SharePoint 2013, Solutions Associate in Windows Server 2012, Specialist in Developing Azure Solutions and Professional Developer in SharePoint 2010.

Our 5 Rules of Data Science

In manufacturing, the better the raw materials, the better the product. The same goes for data science, where a team cannot be effective unless the raw materials of data science are available to them. In this realm, data is the raw material which produces a prediction. However, raw materials alone are not sufficient. Business people who oversee machine learning teams must demand that best practices be applied, otherwise investments in machine learning will produce dubious business results. These best practices can be summarized into our five rules of data science.

For the purpose of illustration, let’s assume the data science problem our team is working on is related to the predictive maintenance of equipment on a manufacturing floor. Our team is working on helping the firm predict equipment failure, so that operations can replace the equipment before it impacts the manufacturing process.

Our 5 Rules of Data Science

1. Have a Sharp Question

A sharp question is specific and unambiguous. Computers do not appreciate nuance. They are not able to classify events into yes/no buckets if the question is: “Is Component X ready to fail?” Nor does the question need to concern itself with causes. Computers do not ask why – they calculate probability based upon correlation. “Will component X overheat?” is a question posed by a human who believes that heat contributes to equipment failure. A better question is: “Will component X fail in the next 30 minutes?”

2. Measure at the Right Level

Supervised learning requires real examples from which a computer can learn. The data you use to produce a successful machine learning model must demonstrate cases where failure has occurred. It must also demonstrate examples where equipment continues to operate smoothly. We must be able to unambiguously identify events that were failure events, otherwise, we will not be able to train the machine learning model to classify data correctly.

3. Make Sure Your Data is Accurate

Did a failure really occur? If not, the machine learning model will not produce accurate results. Computers are naïve – they believe what we tell them. Data science teams should be more skeptical, particularly when they believe they have made a breakthrough discovery after months of false starts. Data science leaders should avoid getting caught up in the irrational exuberance of a model that appears to provide new insight. Like any scientific endeavor, test your assumptions, beginning with the accuracy and reliability of the observations you started with to create the model.

4. Make Sure Your Data is Connected

The data used to train your model may be anonymized, because factors that correlate closely to machine failure are measurements, not identifiers. However, once the model is ready to be used, the new data must be connected to the real world – otherwise, you will not be able to take action. If you have no central authoritative record of “things”, you may need to develop a master data management solution before your Internet of Things with predictive maintenance machine learning can yield value. Also, your response to a prediction should be connected. Once a prediction of failure has been obtained, management should already know what needs to happen – use insights to take swift action.

5. Make Sure You Have Enough Data

The accuracy of predictions improve with more data. Make sure you have sufficient examples of both positive and negative outcomes, otherwise it will be difficult to be certain that you are truly gaining information from the exercise.

The benefits of predictive maintenance, and other applications of machine learning, are being embraced by businesses everywhere. For some, the process may appear a bit mysterious, but it needn’t be. The goal is to create a model which, when fed real-life data, improves the decision making of the humans involved in the process. To achieve this, data science teams need the right data and the right business problem to solve. Management should work to ensure that these five questions are answered to their satisfaction before investing in data science activities.

Not sure if you have the right raw materials? Talk to BlumShapiro Consulting about your machine learning ambitions. Our technology team is building next generation predictive analytics solutions that connect to the Internet of Things. We are helping our clients along each step of their digital transformation journey.

Berry_Brian-240About Brian: Brian Berry leads the Microsoft Business Intelligence and Data Analytics practice at BlumShapiro. He has over 15 years of experience with information technology (IT), software design and consulting. Brian specializes in identifying business intelligence (BI) and data management solutions for upper mid-market manufacturing, distribution and retail firms in New England. He focuses on technologies which drive value in analytics: data integration, self-service BI, cloud computing and predictive analytics

Technology Talks Episode 4: Cybersecurity

Listen to our new podcast, Technology Talks, hosted by Hector Luciano, Consulting Manager at BlumShapiro Consulting. Each month, Hector will talk about the latest news and trends in technology with different leaders in the field.

In this episode, Hector speaks with Jeff Ziplow, Partner at BlumShapiro Consulting about cybersecurity and the tips your organization can take to keep themselves protected.

Listen to our previous episodes on our SoundCloud page >>

Microsoft Announces Power BI Premium: Removes Functionality on Free Version

Many of our clients come to us looking for solutions to help them achieve “Business Intelligence for Everyone” in their organization while avoiding the pitfalls of reporting in Excel. Our response is simple: Microsoft Power BI is an easy-to-use, non-technical business intelligence tool which is far more robust than Microsoft Excel for reporting. End users who rely upon Excel for reporting often view Power BI as a logical step up. With Power BI, users can automate mundane data transformation steps, connect to a broad range of data sources and securely collaborate with colleagues  —all within an environment that looks and feels just like Excel. Our clients have reported that Power BI’s free edition includes enough functionality to get started on any reporting initiative, automate data extraction and transformation activities and share the results with a team of executives, analysts, managers and colleagues. However, as Power BI data and report volumes grow, organizations may choose to step up to Power BI Pro, which upgrades users from 1GB to 10GB of data and enables complex analytics sharing capabilities, even outside the organization.

Finding a Solution for Larger Organizations

The current Power BI service does present some challenges to larger, more sophisticated organizations. Some of the issues include:

  •   Sharing and collaboration features would often become complex and difficult to manage
  • Compute resources are shared, not dedicated, and there is no ability to provision additional compute resources
  • Structured reporting capabilities are not well suited for interactive reports and “single pane of glass” dashboards delivered in Power Bi

These issues begged for a simpler, more manageable model for large organizations.

Introducing Power BI Premium

In early May 2017, Microsoft announced its intention to introduce a new licensing level for Power BI, Power BI Premium. Power BI Premium is designed to address the shortcomings of Power BI Pro. Here are three things to know about Power BI Premium:

  1. Power BI Premium Edition will support Power BI Apps. Power BI Apps replace Content Packs and Power BI Embedded. Organizations that currently share Power BI content externally with Power BI Embedded should plan to migrate to Power BI Premium Edition.
  1. Power BI Premium Edition offers dedicated capacity for organizations that need more control. Instead of paying strictly per user, Power BI Premium is licensed on a combined capacity and usage model. This enables organizations who struggle with the per user data limits enforced on Free and Pro Edition users (1 GB and 10GB maximums, respectively) to load data models that are much larger. As with other Azure services, organizations can scale up and scale down capacity as their needs change.
  1. Power BI Premium Edition includes a license for Power BI Report Server—a full featured on-premises solution supporting both Power BI (interactive) reports and Reporting Services (paginated, structured) reports.

Important Note for Power BI Free Edition Users

Power BI Free Edition became quite attractive because many users within the same organization could share content without paying any fee. Unfortunately, Power BI Free Edition functionality will be changing soon. Users on the Free Edition will no longer be able to share dashboards with colleagues, other than by printing them out, or showing others their “personal dashboard” in a browser. As of June 1, users enjoying dashboard sharing will no longer be able to do so under the Free Edition.

June 1st is right around the corner, and some organizations have built fully functional company dashboards using Free Edition licenses. These organizations now face the prospect of having to either upgrade to Power BI Pro Edition ($10/user/month) or lose vital collaboration features. This is why Microsoft is offering a 1-year trial of Power BI Pro licenses to users who have previously signed up for Power BI Free Edition. This allows organizations to carefully consider which users need Power BI Pro for data model, report and dashboard creation and collaboration and which do not. Some organizations will stay on the Free Edition, and simply share their BI content via PowerPoint. Others will look at Power BI Pro or Premium licensing and continue to see value.

Next Steps

Microsoft has stated that general availability of Power BI Premium is on the horizon, but no specific release date has been communicated. If your organization has many users creating reports and dashboards with the Free Edition, here are some things you can do to get ready for the change.

  1. Take advantage of the 1-Year Power BI Pro trial – encourage users to respond to any email communication from Microsoft and take advantage of the grace period
  1. Download the Power BI Report Server and take it for a spin
  1. Review the Power BI Premium Calculator to understand what your costs would look like under the Power BI Premium model

For more information on how to achieve high performance analytics and reporting with Power BI, contact Brian Berry and our Data Analytics team at bberry@blumshapiro.com, or by phone at 860.570.6368.

Berry_Brian-240About Brian: Brian Berry leads the Microsoft Business Intelligence and Data Analytics practice at BlumShapiro. He has over 15 years of experience with information technology (IT), software design and consulting. Brian specializes in identifying business intelligence (BI) and data management solutions for upper mid-market manufacturing, distribution and retail firms in New England. He focuses on technologies which drive value in analytics: data integration, self-service BI, cloud computing and predictive analytics