Tag Archive for Master Data Management

6 Steps For Creating Golden Records

If you are an organization seeking to improve the quality of the data in your business systems, begin by automating the creation of Golden Records. What is a Golden Record? A Golden Record is the most accurate, complete and comprehensive representation of a master data asset (i.e. Customer, Product, Vendor). Golden Records are created by pulling together incomplete data about some “thing” from the systems in which they were entered. The System of Entry for a customer record may be a Customer Relationship Management (CRM) or Enterprise Resource Planning (ERP) system. Having multiple systems of entry for customer data can lead to poor quality of customer master data – even giving your employees bad information to work off of.

But why not simply integrate the CRM and ERP systems, so that each system has the same information about each customer? In theory, this is a perfect solution; in practice, it can be difficult to achieve. Consider these problems:

  1. What if there are duplicate records in the CRM? Should two records be entered into each ERP? Or the reverse: what if one CRM customer should generate two customer in the ERP (each with different pricing terms, for example)?
  2. What if one or more ERP systems require data to create a record, but that data is not typically (or ever) collected in the CRM? Should the integration process fail, what will be the remediation process?
  3. What if one of your ERP systems cannot accommodate the data entered in CRM or other systems? For example, what if one of your ERP systems cannot support international postal codes? Are you prepared to customize or upgrade that system?

There are many more compatibility issues that can occur. The more Systems of Entry you must integrate, the more likely you are to have many obstacles standing between you and full integration. If your business process assumptions change over time, the automated nature of systems integration itself can become a source of data corruption, as mistakes in one system are automatically mirrored in others.

Golden Record Management, by contrast, offers a significantly less risky approach. Golden Records are created in the Master Data Management (MDM) system, not in the business systems. This means that corrections and enhancements to the master data can be made without impacting your current operations.

6 Steps For Creating Golden Records

At a high level, the process of creating Golden Records looks like this:

  1. Create a model for your master data in the master data management system. This model should include all the key attributes MDM can pull from Systems of Entry that could be useful to creating a Golden Record.
  2. Load data into the model from the variety of SOE’s available. These can be business systems, spreadsheets, or external data sources. Maintain the identity of each record, so that you know where the data came from and how the SOE identifies it (for example, the System ID for the record).
  3. Standardize the attributes that will be used to create clusters of records. For Customers and Vendors, location and address information should be standardized.
  4. If possible, verify attributes that will be used to create clusters of records.
  5. Create clusters of records, by Matching key attributes, to create groups of master data records. The cluster identifier will be the Golden Record identifier. You can also think of this in terms of a hierarchy. The Golden Record is the Parent and the source records are the Children.
  6. Populate the Golden Record, created in MDM, with attributes from the records in its cluster (the source data). This final step, called Survivorship, requires a deeper understanding of how the source data was entered than the previous five steps. We want to create a Golden Record that contains all the best data. Therefore, we need to make some judgements about which of the SOE’s is also the best System of Record for a given attribute (or set of attributes).

Great! We’ve consolidated our master data, entered from a variety of systems, into one system which also contains a reference to a parent record, called the Golden Record. This Golden Record is our best representation of the “thing” we need to understand better.

But wait! The systems of entry, the systems your business USES to operate, have not been updated. Can you still take advantage of these Golden Records?

The answer is “yes” – you can take advantage of the Golden Records in two ways:

  1. As the basis for reporting, because each Golden Record is also a “roll-up” of real system records that are referenced by orders, returns, commissions, etc. Golden Records provide a foundation for consistent Enterprise Reporting.
  2. As the basis for data quality improvements in each system of entry, assuming these systems can import a batch of data and update existing records that match a system ID.

These benefits of Golden Records are gained without the high risk and high costs that come with systems integration. Further, if you have modeled your master data correctly, it is possible to automate the data quality benefits of Golden Records Management, by updating these systems in real-time. See how BlumShapiro can help with your master data needs and golden record creation.

Berry_Brian-240About Brian: Brian Berry leads the Microsoft Business Intelligence and Data Analytics practice at BlumShapiro. He has over 15 years of experience with information technology (IT), software design and consulting. Brian specializes in identifying business intelligence (BI) and data management solutions for upper mid-market manufacturing, distribution and retail firms in New England. He focuses on technologies which drive value in analytics: data integration, self-service BI, cloud computing and predictive analytics

The Value of Golden Records

Running multiple ERP systems simultaneously can be quite painful for any mid-size organization. Since each ERP maintains their own chart of accounts, financial consolidation and reporting can become all-consuming for the finance teams. When each ERP has its own Customer Master, sales team visibility into strategic accounts is limited, while smaller accounts receive terms that can become big problems for AR. These separate ERP systems lead to issues for other departments—marketing wants a single comprehensive product master; supply chain managers want a single comprehensive vendor master.

Obviously, there is hyperbole involved in my description. However, these are some of the many reasons executive management would like all business units working from a single ERP, with integrated financial reporting, consistent business processes for the whole company and lowered costs of operations.

So, you initiated a multi-year ERP implementation / migration / consolidation project.

At the outset, each ERP specialist is skeptical of the consolidation strategy. “Our ERP is tailored to our business unit” is a common argument for keeping each ERP running. When asked, “How’s the quality of the data?” the same ERP specialists may complain that the data quality is poor. Unfortunately, data problems don’t get better by maintaining the status quo.

Severe master data quality problems present an obstacle to an efficient ERP transition. Let’s think about the customer: if you were to bring all customer master records into a new system wholesale, you’d have many duplicated accounts. You’d have diverse naming convention issues. You’d have some accounts that refer to distribution centers, some to end users, some to drop ship locations. You’d have a wide variety of payment terms.

Get your ERP ambitions moving again, and focus on data quality in a way that enables the final goal—centralized and integrated business processes. Here’s how:

  1. Build Golden Records for Customer. A Golden Record is a representation of your master data, which is the fullest, cleanest and most accurate information available. They are created from consolidating master data from multiple Systems of Record (ERP’s and other systems), standardizing that data, verifying the accuracy where possible, and then building clusters of similar records. This process of matching facilitates the creation of Golden Records, which contain the best information from all the master data in the cluster.
  2. Do the same for Product
  3. Do the same for Vendor

Are you sensing a pattern? Provided your systems of record have a reasonable amount of data characterizing each row of data, similarity clusters can be built. Inaccurate, non-standard data makes the process a little harder, but feasible. Accounting Master Data (i.e., GL Accounts) further benefit from a Uniform Chart of Accounts, to which all other systems may be mapped.

Golden Records Management is a non-intrusive, low-risk tool for accelerating the ERP migration process. Building Golden Records is repeatable for many types of master data and provides a means for preparing the best possible data for import into any new system. In Part 2, I’ll talk about how Golden Records and Master Data Management deliver a perpetual framework for Data Quality, extending the lifetime of legacy systems.

Want to learn more about the impact of master data on your organization? Join us on December 6 in Hartford, CT for our half-day workshop Discovering the Value in Your Data. Hear from data governance experts from BlumShapiro Consulting and Profisee as they address key topics for business, finance and technology leaders on data and master data management.

Berry_Brian-240About Brian: Brian Berry leads the Microsoft Business Intelligence and Data Analytics practice at BlumShapiro. He has over 15 years of experience with information technology (IT), software design and consulting. Brian specializes in identifying business intelligence (BI) and data management solutions for upper mid-market manufacturing, distribution and retail firms in New England. He focuses on technologies which drive value in analytics: data integration, self-service BI, cloud computing and predictive analytics

Three Steps to High Quality Master Data

Data quality is critical to business, because poor business data leads to poor operations and poor management decisions. For any business to succeed, especially now in this digital-first era, data is “the air your business needs to breathe”.  If leadership at your organization is starting to consider what digital transformation means to your business or industry – and how your business needs to evolve to thrive in these changing times, they will likely assess the current business and technology state. One of the most common outcomes management may observe is that the business systems are “outdated” and “need to be replaced”. As a result, many businesses resolve to replace legacy systems with modern business systems as part of their digital transformation strategy.

Digital Transformation Starts with Data

More than likely, those legacy systems did a terrible job with your business data. They often permitted numerous, incomplete master data records to be entered into the system. Now, you have customer records which aren’t really customers. The “Bill To’s” are “Sold-To’s”, the “Sold-To’s” are “Ship-To’s”, and the data won’t tell you which is which. You might even have international customers with all of their pertinent information in the NOTES section. Each system which shares customer master data with other systems contains just a small piece of the customer, not the complete record.

This may have been the way things were “always done” or departments made due with the systems available, but now it’s a much larger problem, because in order to transform itself, a business must leverage its data assets. It’s a significant problem when you consider all the data your legacy systems maintain. Parts, assets, locations, vendors, material, GL accounts: each suffer from different, slightly nuanced data quality problems. Now it hits you: your legacy systems have resulted in legacy data.  And as the old saying goes – “garbage in, garbage out.” In order to modernize your systems, you must first get a handle on data and your data practices.

Data Quality Processes

The data modernization process should begin with Master Data Management (MDM), because MDM can be an effective data quality improvement tool to launch your business’ Digital Transformation journey. Here’s how a data quality process works in MDM.

Data Validation – Master Data Management systems provide the ability to define data quality rules for the master data. You’ll want these rules to be robust — checking for completeness and accuracy. Once defined and applied, these rules highlight the gaps you have in your source data and anticipate problems which will present themselves when that master data is loaded into your shiny new modern business applications.

Data Standardization – Master Data thrives in a standardized world. Whether it is address standardization, ISO standardization, UPC standardization, DUNS standardization, standards assist greatly with the final step in the process.

Matching and Survivorship – If you have master data residing in more than one system, then your data quality process must consider the creation of a “golden record”. The golden record is the best, single representation of the master data, and it must be arrived at by matching similar records from heterogeneous systems and grouping them into clusters. Once these clusters are formed, a golden record emerges which contains the “survivors” from the source data. For example, the data from a CRM system may be the most authoritative source for location information, because service personnel are working in CRM regularly, but the AR system may have the best DUNS credit rating information.

Modernize Your Data and Modernize Your Business

BB Art

These three data quality processes result in a radical transformation in the quality of master data, laying the foundation for critical steps which follow. Whether or not your digital transformation involves system modernization, your journey requires clean, usable data. Digital transformation can improve your ability to engage with customers, but only if you have a complete view of who your customers are. Digital transformation can empower your employees, but only if your employees have accurate information about the core assets of the business. Digital transformation can help optimize operations, but only if management has can make informed data driven decisions. Finally, digital transformation can drive product innovation, but only if you know what your products can and cannot currently do.

Berry_Brian-240About Brian: Brian Berry leads the Microsoft Business Intelligence and Data Analytics practice at BlumShapiro. He has over 15 years of experience with information technology (IT), software design and consulting. Brian specializes in identifying business intelligence (BI) and data management solutions for upper mid-market manufacturing, distribution and retail firms in New England. He focuses on technologies which drive value in analytics: data integration, self-service BI, cloud computing and predictive analytics. 

Technology Talks Newsletter CTA

Save Operational MDM for Phase 2

In my final installment on the 5 Critical Success Factors for Initiating Master Data Management, I want to discuss why tackling Operational MDM is valuable, and when to do it. 

A major contributor to disruptions in MDM projects is lack of stakeholder agreement with respect to what the team is trying to accomplish with MDM.  It’s important to get everyone on the team clear on the two purposes of MDM.

The first purpose is to facilitate better reporting (Reporting MDM).  The goal with Reporting MDM is to gather and aggregate data (sales, invoices, purchase orders, etc.) such that in an Enterprise Reporting Context, all of the data is included from a set of source systems.  A Reporting MDM system does this by Matching Master Data records from each of these systems.  Then, it provides Views of these matches (groups of master records) to subscribing systems and users to consume and use for their own purposes.  It sounds simple, and in fact, it is pretty simple.

The second purpose is to improve the overall data quality in each operational system (Operational MDM).  The goal with Operational MDM is to ensure that each representation of the same “thing” (e.g. Vendor) is the same in all systems which house master data for that “thing”.  An Operational MDM system does this by Matching Master Data records from each source and then Harmonizing the records (i.e. makes all of the master records in a group “line up”).  Finally, it Distributes the harmonized data back to the source systems.  Imagine knowing that all representations of your most valued customers, are verifiably represented in a logically consistent way in all of your AR systems.

Visually, an Operational MDM synchronization process might look like this.

Operational MDM in Action

Once, we have those two concepts solidly understood, the question becomes: can we have both?  Yes, you can have both.  However, if delivering value to the business quickly is a consideration (it should be), I recommend that you tackle Reporting MDM first.  Reporting MDM has fewer technology hurdles, initiates a Data Governance program, and delivers real value quickly.

Here is what Operational MDM will take:

  1. A Data Bus – you’ll need a integration solution which can handle connections to all of the LOB systems which you want to synchronize.  My team uses Microsoft BizTalk Server for this.
  2. Subject Matter Expertise – you’ll need access to the people who understand the target systems extremely well.  Often they will need to expose API’s to the MDM team, so that synchronization can be “real-time” (a change is made to MDM and the change event propagates to all of the affected systems)
  3. Business Process Review – your Data Governance team will likely need to consider the full lifecycle of the master data- creation, maintenance  and archive.

In summary, Operational MDM is achievable and yields tremendous value.    But first, build the foundation and “put some points on the board”.  If you build a  Federated Data Model, Keep MDM Separate, Flip the Script and Formulate your Governance Plan, Phase 1 will be successful, and you’ll get funding for Operational MDM in Phase 2.

Good luck!