Everyone has moved their data to the cloud — now what?

Companies of all shapes and sizes increasingly understand that there is a need to continually improve competitive differentiation and avoid falling behind the digital-native FAANGs of the world — data-first companies like Google and Amazon have leveraged data to dominate their markets. Additionally, the global pandemic has galvanized digital agendas, data and agile decision-making for strategic priorities spread across remote workspaces. In fact, a Gartner Board of Directors study found 69% of respondents said COVID-19 has led their organization to accelerate data and digital business initiatives.

Migrating data to the cloud isn’t a new thing, but many will find that cloud migration alone won’t magically transform their business into the next Google or Amazon. 

And most companies discover that once they migrate, the latest cloud data warehouse, lakehouse, fabric or mesh doesn’t help harness the power of their data. A recent TDWI Research study of 244 companies using a cloud data warehouse/lake revealed that an astounding 76% experienced most or all of the same on-premises challenges.

The cloud lake or warehouse only solves one problem — providing access to data — which, albeit necessary, doesn’t solve for data usability and definitely not at absolute scale (which is what gives FAANGs their ‘byte’)! 

Data usability is key to enabling truly digital businesses — ones that can draw on and use data to hyper-personalize every product and service and create unique user experiences for each customer.

The path to data usability

Using data is hard. You have raw bits of information filled with errors, duplicate information, inconsistent formats and variability and siloed disparate systems. 

Moving data to the cloud simply relocates these issues. TDWI reported that 76% of companies confirmed the same on-premise challenges. They may have moved their data to one place, but it’s still imbued with the same problems. Same wine, new bottle.

The ever-increasing bits of data ultimately need to be standardized, cleansed, linked and organized to be usable. And in order to ensure scalability and accuracy, it must be done in an automated manner.

Only then can companies begin to uncover the hidden gems, new business ideas and interesting relationships in the data. Doing so allows companies to gain a deeper, clearer and richer understanding of their customers, supply chains, processes and convert them into monetizable opportunities. 

The objective is to establish a unit of central intelligence, at the heart of which are data assets—monetizable and readily usable layers of data from which the enterprise can extract value, on-demand.

That is easier said than done given current impediments: Highly manual, acronym soupy and complex data preparation implementations — namely because there isn’t enough talent, time, or (the right) tools to handle the scale necessary to make data ready for digital.  

When a business doesn’t run in ‘batch mode’ and data scientists‘ algorithms are predicated on constant access to data, how can current data preparation solutions that run on once-a-month routines cut it? Isn’t the very promise of digital to make every company anytime, anywhere, all in?

Furthermore, few organizations have enough data scientists to do that. Research by QuantHub shows there are three times as many data scientist job postings versus job searches, leaving a current gap of 250,000 unfilled positions.

Faced with the dual challenges of data scale and talent scarcity, companies require a radical new approach to achieve data usability. To use an analogy from the auto industry, just as BEVs have revolutionized how we get from point A to B, advanced data usability systems will revolutionize the ability for every business to create usable data to become truly digital. 

Solving the usability puzzle with automation

Most see AI as a solution for the decisioning side of analytics, however the FAANGs’ biggest discovery was using AI to automate data preparation, organization and monetization.  

AI must be applied to the essential tasks to solve for data usability — to simplify, streamline and supercharge the many functions necessary to build, operate and maintain usable data.  

The best approaches simplify this process into three steps: ingest, enrich and distribute. For ingest, algorithms corral data from all sources and systems at speed and scale. Second, these many floating bits are linked, assigned and fused to allow for instant use. This usable data must then be organized to allow for flow and distribution across customer, business and enterprise systems and processes. 

Such an automated, scaled and all-in data usability system liberates data scientists, business experts and technology developers from tedious, manual and fragile data preparation while offering flexibility and speed as business needs change.

Most importantly, this system lets you understand, use and monetize every last bit of data at absolute scale, enabling a digital business that can rival (or even beat) the FAANGs.

Ultimately, this isn’t to say cloud data warehouses, lakes, fabrics, or whatever will be the next hot trend are bad. They solve for a much-needed purpose — easy access to data. But the journey to digital doesn’t end in the cloud. Data usability at scale will put an organization on the path to becoming a truly data-first digital business.

Abhishek Mehta is the chairman and CEO of Tresata