This is part 3 of a 4 part series on DataOps
What is DataOps?
DataOps is an emerging idea, with many perspectives. For this post, I am using the following definition is from Hitachi-Vantara:
DataOps is enterprise data management for the AI era. Now you can seamlessly connect your data consumers and creators to rapidly find and use all the value in your data. Data operations is not a product, service or solution. It’s a methodology, a technological and cultural change to improve your organization’s use of data through better data quality, shorter cycle time and superior data management.
The most significant challenge to implement DataOps is going to be cultural, not technology. The following protocols are suggested to address some of the main cultural barriers that exist.
The Center Cannot Hold
Centralized control always looks great on paper, but it never works in reality. It has two main assumptions:
- The future will operate like the past.
- All variables are known.
Both of these assumptions are wrong. Processes are out of date as soon as they are documented. And no one can foresee all variables (like the coronavirus).
Centralizing Data Integration (the act of bringing data together) and Data Governance (the oversight of the use of data) may seem the way to streamline data management. Data is an asset like money and it is natural to put similar management mechanisms in place.
Keep in mind, there is one significant difference between data and money – data can be continuously shared and reused. Money cannot.
So why have the same control mechanisms on data that you have on money?
Data Integration and Data Governance is not a department’s function but a group activity for a corporation. Many studies have shown looser (people) networks of information sharing always outperforms centralized control. Adam Smith’s invisible hand works because people are social creatures and will self-organize and self-synchronize continuously. Even if it means working around leadership’s specific directives.
This does not mean data should remain in silos – quite the opposite. Integrating data is an essential element of DataOps. Why? Integrating data increases its value which makes it a unique type of asset. Quick, easy and broad access to integrated data is an essential foundation for successful DataOps.
- Integrating data is not the same thing as centralizing control on data. The data can be anywhere, created by anyone, for any purpose, and still integrated without centralized control.
- Data Governance is not the same thing as centralizing control on data. Data Governance provides the acceptable boundaries for data. It is guardrails and guidance, not rules and punishments.
- There will be new sources of data. On boarding and integrating new data cannot be an afterthought but part of the plan. Slow, clumsy and expensive data integration processes and tools will be rejected.
Interested in learning more about DataOps? We recommend the DataOps.NEXT virtual conference by our premier partner Hitachi-Vantara. There are 4 tracks:
- Optimize the Data Fabric
- Build and Manage Data Pipelines
- Improve Data Governance and Agility
- Expand Analytics and Machine Learning
There is no cost to join live or to watch replays for 90 days.
Date: Thu 14 May, 2020
Time: 9.00am EST / 1.00pm UTC
Where: Online
