Use metric stores as the primary data service layer
In a single generation, we have witnessed the internet revolution, the cloud revolution and we can say that we are in the midst of a data revolution. Data has always been essential, but today its size, speed and usefulness are reaching dizzying new heights.
Now that data applications and analytics are a permanent, essential, and growing part of our professional lives, we can’t get enough of them. And that’s a problem.
A low-coding or no-coding data platform will be an essential component for data-driven decision-making and day-to-day business operations.
By using a metrics store as the core service layer, data platforms enabled by Natural Language Processing (NLP) and AI algorithms can reduce or even eliminate dependency on SQL for business users.
Although huge improvements have been made in the area of data engineering, we still need to make the information available to regular business users, not just power users. Power users understand the nuances of data well enough to successfully extract insights from it and typically achieve this goal with SQL.
But this model does not fit most other businesses. Can your end users (store managers, sales reps, marketers, clerks) have this level of SQL skills? Can they download any dataset, launch a database, create table joins, and run SQL queries to get insights?
Data platforms should enable regular business users who do not have extensive knowledge of SQL. This is the only way to enlighten everyone.
What does LC/NC mean for data platforms?
Analytics data is usually stored in a data lake/house. End users need to understand how to query these data platforms with some sort of query language like SQL or via Python scripts.
To enable citizen analysts and Citizen Data Scientists, we need to reduce or eliminate this coding step and understand that they:
- Don’t worry about tables and columns
- Only care about business information (sales volume, shipping cost, etc.)
- Focus on business performance rather than obscure technical skills
- Want information readily available when a question is asked
How to Build an LC/NC Data Platform
Metrics Store as the main data service layer
Instead of managing data from a data platform, companies should create metrics stores where business metrics are defined and organized. End users can simply drag and drop these metrics into their tools such as Excel, BI dashboards, web applications, etc.
In a metrics store, business metrics are defined, calculated, and stored in a central location overlaid with relevant governance processes. End users can then define and derive important metrics for their day-to-day tasks (eg, sales figures by product, year-over-year sales growth, profit margin). These are the data points that users want to know. So why not define them once, calculate them correctly and give access to everyone in the company?
Natural language to help users ask questions
Instead of asking users to learn SQL, why not ask them to ask questions in plain language? With the advent of NLP technologies, we should expect today’s data platforms to understand the everyday language of everyday users.
We should also expect the platform to push NLP capabilities a bit further with context awareness. Knowing the context is essential for an interactive analytics experience. The “machine” can have a conversation with users in which follow-up questions can be asked and answered.
With AI integrated into today’s data platforms, it will become much faster and easier to analyze data and metadata, cross-reference user behaviors and optimize how users get answers. to their questions. Today’s artificial intelligence algorithms can even predict the questions users might ask and prepare the answers. AI can dramatically improve the user experience when they want to interact with their data. At the same time, AI in the data platform can also automatically optimize data storage to eliminate waste and reduce costs.
The magic happens when today’s most exciting technologies converge. In the world of data, moving from raw data to insight and intelligence has been a massive undertaking. But like the data itself, the creation of intelligence, advantage and insight must be seen from a distance. From this distance, we can now see that the problem can be solved by leveraging metric stores, NLP, and AI to enable a Low Code/No Code platform.