The Association Integration Problem: Where We Are & Where We’re Headed
Your association is unique, right? Really, every association says they’re very different from the rest. What tends to happen is organizations assemble a bunch of different “best-of-breed” applications to meet their unique needs rather than an all-in-one, integrated system.
You may have one system for events, one for association management, one for learning, and so on. These so-called best-of-breed tools provide very specific features for a niche of functionality within your broad association ecosystem.
It’s as if these niche systems have guard rails around the specific functionality that they provide and anything outside of those barriers has to be fulfilled by and integrated with another system or combination of systems. In turn, these systems and their associated data become silos that function independently.
The problem is that the best-of-breed applications in the association industry are natively not well bonded ie. non-integrated, at least at this present time. That’s where the heart of the integration problem lies; whether you're integrating systems, cultures, or people for that matter, it's about bonding.
So how do you get very diverse, best-of-breed technology to bond really well together?
Where We’re At
I've never been to a conference where the integration problem wasn't one of the central topics. There are always discussions around why our different systems work great independently for what they’re designed to do but don’t work well together.
Associations tend to be slower to change when it comes to digital transformation. Today, we’re still selecting our applications around our Association Management Systems (AMS) and websites.
In fact, the most common, basic integration for associations is between the Content Management System (CMS, i.e. your website) and the AMS (i.e. the CRM for tracking member data across an association). These two core systems predominantly influence how most associations integrate with everything else they need to integrate.
We've had a client with as many as 172 of these best-of-breed systems. Generally speaking though, an association may have anywhere from 12 to 20+ applications that all need to talk to one another in some way. Pretty much every application needs to communicate with the accounting system and the AMS.
But this level of system integration is easier said than done for associations.
Top 5 System Integration Challenges
Do these technology frustrations sound familiar?
How do I get my technology to give me the answers I need? Why can’t my systems just work together? I need information to make this decision and I can’t seem to get it right.
These questions all stem from fundamental integration issues that impact every level of an association—from the user experience, to the folks running your events and your marketing departments, to your membership management teams, all the way up to the C-suite.
Let’s unravel the most common sources of frustration:
Challenge #1: Recognizing Users
The first, relatively small obstacle you have to navigate is figuring out how to validate members from your website against your AMS/CRM. You also want your staff to be able to sign in to your website using the same log-in credentials for the AMS.
This is the first connection that’s often needed: the single sign-on integration. Most AMS systems are the “identity providers” in which every other system pulls this information to authenticate users.
Once you’ve integrated these two core systems (your website and AMS), then all “best-of-breed” systems you have on top of that will need to be integrated as well.
Challenge #2: Getting the Data to Flow
The second layer of integration challenges is that these best-of-breed systems often have specific architectures for how they store and process data, which need to be relayed to each other.
For example, a member may order and pay for books or content on your websites through your e-commerce payment processing platform. Then that transaction needs to be recorded in your AMS/CRM and then flow through to your accounting system. To make this work, you have to:
- Allow members to single sign-on to your AMS/CRM system.
- Get the information to flow from one system to the next.
The more systems you bolt on to your application stack, the more complicated it gets. Whether it’s getting a certification, buying a book, or signing up for an event, every transaction involves two or three or more best-of-breed systems.
Challenge #3: Webs of One-to-One Integrations
To get these systems to work together, we’ve traditionally built a bunch of point-to-point integrations between systems, instead of having data flow through one fluid process. For example, you may find a way to get your Learning Management System (LMS) to send data straight to your accounting system so that a journal is posted for a purchase, which then gets recorded back in the CRM.
If you have 12 to 20 systems, and they're all these one-to-one integrations, you end up with a spider web of integrations, from system to system, which tends to break down pretty quickly and require heavy maintenance.
This is why data often gets locked in one system without being relayed to another system where you need the information. Plus, people don’t want to mess with the point-to-point integrations of live systems. As a result, these disjointed integrations multiply and eventually become overpowering technology problems that prevent you from undergoing digital transformation initiatives.
For example, if you want to pull out your LMS vendor and plug in a new system, you risk the whole house crumbling from that single system replacement.
That's why it’s so frustrating hearing how long it takes or how complicated it is to change vendors or get integrations done between systems.
You can see how everything happens in the margins:
MOST PROBLEMS OCCUR IN THE SPACE BETWEEN SYSTEMS.
Challenge #4: Analytics Problem
You need information and analytics to run your business. You run into problems when this information isn’t accessible or all in one place. On our iPhones and mobile devices, we buy apps that work together all the time: Slack, Zapier, Trello, Asana, and all of these other platforms have ways of connecting to each other to pass data. It's not so easy with the association application stack.
Some of this data may flow through the transactional interfaces that associations use, but there are not very many data interfaces that make your job easier. So, integration needs span far beyond just single sign-on and transactional processes.
You also need to know how many members perform some action under given circumstances so you can make decisions. For example, you may want to know which folks were certified in a given area (which may be recorded in your LMS) who also attended a conference (which might be in your events portal).
The problem is that your LMS and events tool may not be properly integrated to correlate that data. So, you find yourself frustrated because you know the data is there; your systems just don’t talk to each other organically. So, essentially, you struggle to make your data work for your association.
Challenge #5: Lack of Standardization
What programmers tend to do is pick technologies that they think are cool. They’re not necessarily the ones that are the most standardized or the ones that everybody else uses.
If a new technology comes along that they think is sexier, they may start using it. And it may or may not work with everything else in your ecosystem of 12 to 20 applications or however many you use.
It's like the human fingerprint; no two associations have the same 12 to 20 applications. Just think of all the different AMS options alone!
There are 10 or 12 dominant AMS systems in the association world. Then there are at least half a dozen LMS systems, and three or four different data analytics platforms. Plus, everyone has a different website approach, whether it’s WordPress, Sitefinity, Sitecore, or another content management solution.
If you threw all of that spaghetti on a wall and looked at it, you’d think, “no wonder it’s so difficult to make my systems work together.”
The technologies that associations use today all have different standards and come from different places, different programmers, and different architectures. There's nothing that bonds them together.
What's the solution then?
Solutions to the System Integration Problem
The first place where people tend to start is what's called application programming interfaces or web service APIs. For the most part, almost all the vendors publish an API of some sort, but they’re difficult to maintain because they often don’t work so well.
Here are options that do work:
Solution #1: Integration Platforms
There are some integration platforms called microservices that have potential, but we've only seen minimal traction for these toolsets in the association world so far. Progress is being made but these integration platforms are generally for early adopters at this stage in terms of what they do.
Fundamentally, it’s challenging to get these solutions to work until vendors who build applications in the association space define frameworks for using microservices that are used commercially today.
But we’re seeing some promising traction with this effort and expect this technology to become more available to associations in the near future.
The predecessor to microservices, middleware is relatively more commonly used to integrate the web of systems in the best-of-breed environment. Middleware is the “software glue” that bridges gaps between vertical applications, databases, and devices to provide a unified experience.
Instead of siloed, system-to-system integrations with third-party vendors, middleware is the single piece of software that works with all other vendors. It’s like the central hub for all the spokes in the ecosystem.
In theory, middleware makes it easier to change a system, if and when needed. Since you’re just dealing with one piece of middleware, none of your other systems should be affected by that one change. There’s only “one throat to choke,” if you will.
If there is a problem, then the new vendor you’re transitioning to and your middleware vendor can work together to solve it; rather than your entire ecosystem of vendors making necessary adjustments and risking a chain of interactions.
Plus, with both microservices and middleware technology, you don’t need to rely on in-house tech staff to make these one-off system replacements and keep everything up and running.
Solution #2: Data Lakes
One of the other solutions that we've used in the past is at the data level. The best practice is to ensure you have a good data governance plan in place so that your information gets recorded in a Data Warehouse or a more modern Data Lake.
Tools like Acumen (Association Analytics) and Gravitate (Nucleus), for example, are data lakes that are built to consolidate data from all systems and then extract the information into analytics reports for decision-making.
Shortcomings of Data Level Integration
Associations tend to run into problems when you try to “make stuff work” at the data level because there are fundamental differences between transactional data and analytics data.
For example, let’s say I'm a member of the XYZ Association. That association would have my profile record that says: this member is Rick Bawcum, and his renewal date is on December 31st. That way, the organization can start sending me renewal notices ahead of time. Once I renew, my next renewal date is updated in that member record.
But many of these systems don’t keep easily accessible information on my past renewal dates or how many times I’ve renewed my membership. That system probably also doesn’t know how many times I’ve attended a certain event or purchased a learning resource.
You lose because transactional systems are built for speed to create as little friction as possible for the member. They're not built for analytics. Nobody wants to sit around, clicking through buttons to renew their membership. Instead, we may have systems like data lakes on the analytics side that store past transactional data that is optimized for reporting and comparisons.
Beyond the analytics platform, you also need a data governance plan in place. In other words, you need structured processes to feed your analytics platform so you can access the information you need when you need it. Ultimately, there's a very different structure for analytics than there is for transactions.
This is why we currently have best-of-breed transaction processing systems with our data analytics systems living behind the scenes. Kind of like the back office sweeping up all the transactional dust that falls so we can go back later to analyze everything.
The Upsides
On the plus side, data lakes solve the all-too-common problem of not being able to get the data you need to make a decision. You don’t have to gather spreadsheets from different systems, jam them together, and then use pivot tables and a series of acrobatics to get the information.
As you know, once you’ve massaged that data two or three times, the validity becomes questionable. This gymnastics approach leaves a lot of room for error because you don’t know the assumptions that were made during the process.
In short, data lakes are one solution to the system integration problem for associations. But are they the best option? Not always.
The Best Solution Today?
The best solution today is ultimately better integrations.
Why?
Bottom line: our best-of-breed systems aren't natively built to work together on the data level. Some of them work together at the single sign-on level to navigate between systems with one universal login. But when you get down to the data level and have to share transactional data between systems, this is where better integration is a must.
In short, integration platforms like microservices are currently the best option for solving the integration problem, at least out of the solutions we have today.
We always tell our clients to avoid customizations whenever possible because of the technical debt it causes. But system integration is a customization that you almost certainly can’t avoid, at least at this time.
Where We’re Headed
Change is coming. As we’ve seen, there are some up-and-coming green shoots in this association space that show promise in solving the integration problem. But many of them are either not quite mature yet or wildly expensive.
There are also some customer data platforms that are making waves in the “built-to-integrate” space. These platforms don't do all the little things that many AMS systems try to do today. Instead, they provide a very stripped-down core of functions.
Yet, these built-to-integrate systems play nicely with all applications that plug into it. Kind of like these non-association-centric tools people use daily like Slack, Trello, and Asana. These applications are built to onboard different kinds of data so they work well together. So, the two budding solutions to the association integration problem are:
- Microservices and integration platforms.
- Making applications that are built to integrate accessible for associations.
This second option though is the ultimate answer: our applications fundamentally need to work better together.
Don’t get me wrong: there are a few all-in-one association technology solutions available that, in some cases, are far more powerful than most AMS systems on the market today. But this technology is five times more expensive. So, what we get are tiers of adoption based on what people can afford.
If an organization can’t afford such robust, all-encompassing solutions, then you buy something you can afford now. But then you’re forced to bolt stuff on and customize your system. Sooner or later, you may find yourself with information silos and a ton of technical debt, likely in a worse position than if you’d just invested in a better solution to begin with.
Related Posts
