“But We Have the Data” - The Importance of Data Architecture

Ever know you have the data, but you can’t seem to get it out of the database?

We come across situations when an association is collecting a piece of information from a customer on a membership application or in a member profile, but they cannot produce a meaningful report using that data. 

The problem is not that the data doesn’t exist; the problem is in how that data is structured, a.k.a. “data architecture.” 

What is Data Architecture?

“Data architecture” is the set of rules, policies, standards, and models that govern and define the type of data collected and how it is used, stored, managed, and integrated within the organization and its database systems. 

In general, the primary objective of data architecture is the standardization of data for the benefit of the organization.

Info-Tech’s five-tier data architecture model summarizes an organization’s data environment at a logical level. Data flows from left to right but can also flow from the presentation layer (how the data is presented in the real world, i.e., reports, dashboards, etc.) back to the warehousing layer for repatriation of data.

1. Sources - Where all the data enters the organization.
2. Integration and Translation – Where integration, transformation and aggregation occur.
3. Data Warehouse – Where the data rests in long-term storage.
4. Analytics – Where data is used for a purpose.
5. Presentation – Where data is presented in a knowledge form.

The Role of Data Architect

The data architect must develop a vision for the enterprise’s data architecture. They must be able to create processes for governing the identification, collection, and use of accurate and valid metadata, as well as for tracking data quality, completeness, and redundancy. They need to create strategies for data security, backup, disaster recovery, business continuity, and archiving, and ensure regulatory compliance.

Because associations have limited staff and IT resources, the role of data architect is sometimes filled by the database manager, the director of IT or the membership director.  Below is a list of tasks to help you determine who in your association may best fit this role.

• Acts as a “translator” between the business and data workers to communicate data and technology requirements.
• Facilitates the creation of the data strategy.
• Manages the enterprise data model.
• Has a greater knowledge of operational and analytical data use cases.
• Recommends data management policies and standards and maintains data management artifacts.
• Reviews project solution architectures and identifies cross impacts across the data lifecycle.
• Is a hands-on expert in data management and warehousing technologies.
• Is not necessarily its own designated position, but a role that can be completed by a variety of IT professionals.

Key Reasons to Optimize Your Data Architecture

Info-Tech has identified these four common drivers that lead to the need to optimize your data architecture.

1. Becoming More Data Driven - The business is looking to leverage data and information by processing data through BI tools and self-service.
2. Regulations and Compliance – Some requirements can be data-element driven. For example, requirements around data elements that are associated with Personally Identifiable Information (PII) and PHI (Personal Health Information (PHI).  Alternatively, some requirements can be process driven. For example, like those that restrict data read/write to certain groups.
3. Mergers and Acquisitions - The organization acquires/merges with another organization (like a chapter or affiliate group) and wants to integrate the data.
4. New Functionality or Business Rule - An application is changed through an application version upgrade, migration to cloud, or application customization, or as a result of application rationalization or changes in the way that application data is generated.

Now that we better understand what data architecture is and the drivers for optimization, let's revisit that scenario from above. If the association is collecting the data, why can’t they report on it? 

Let’s say, for example’s sake, that the data collected is on a user’s interest area.  Reviewing a basic overall data aggregation or sliced with a second data point, such as member type, should be a simple reporting exercise. Right? 

It depends on how the data is stored in the database (the data architecture). 

Multiple interest areas may be presented in a web form using check boxes. And you may allow a user to select multiple interests. 

One of our clients had this situation. The AMS was storing all of the interest data in one field, as text, using a comma to separate the multiple values selected.  Since they didn’t have a data warehouse, they were exporting the data to Excel and manually manipulating the data to calculate the number of users who reported interest in a certain topic area. This process could take days for staff to compile. 

This offers a perfect opportunity to reoptimize the data source for easier processing and reporting.  If they invest in a data warehouse, the translation stage could perform that data separation, preparing the data for easier storage, analytics in context of other data points and eventual presentation in a report. 

It is critical to think through the entire data architecture and desired outcome before creating new data fields and collecting new data points.  A savvy data architect will find ways to work with the systems on hand to enable the organization to use the information in a meaningful way. Understanding your data architecture is a critical task for organizations that are on the path to being data-driven.