A book by a Yale University econometrician, Ian Ayers, looks at a trend in organizations that is changing the decision-making process from one based on expertise and intuition to a data-based effort. This change is possible due to an almost inexhaustible supply of data on every topic, gathered from many sources and made available by the development of huge databases and the tools to manipulate the data in a variety of ways.
The book, Super Crunchers: Why Thinking-by-Numbers Is the New Way to Be Smart, calls the data set analysts “super crunchers” and discusses the changes they are making to industries as varied as medical diagnostics, airline ticket pricing, screenwriting and online dating services. These super crunchers are responsible for managing data effectively to drive decision-making. Although the author presents both sides of this debate (intuition versus use of data), clearly he is convinced that the use of large amounts of data for “objective” decision-making is the better approach.
What is Data Governance?
Data governance is an approach that encompasses principles, practices, and tools to manage an organization’s data assets throughout its lifecycle. It aligns data-related requirements with business strategy to provide superior data management, quality, visibility, security, and compliance capabilities. Effective data governance ensures that data is easily accessible for data-driven decision-making while safeguarding it from unauthorized access and ensuring compliance with regulatory requirements. By implementing a robust data governance framework, organizations can optimize their data assets, enhance data quality, and maintain data security, ultimately driving business success.
Difference in Approaches
The difference in the two approaches is not just a matter of managerial preference according to the author: “We are in an historic moment of horse vs. locomotive competition where intuitive and experiential expertise is losing out time and time again to number crunching.” Ayers shows that some older industries, such as wine-making, still rely more on feeling and experience than on the quantitative method. He believes that the data-based approach is needed to improve performance in every operation, using the incredible volumes of data accumulated in every organization, regardless of field.
This trend, which started with the development of data warehouses and other large databases for decision support in the late 1980’s and early 1990’s, is increasing due to the availability of enormous amounts of raw data, the relatively inexpensive data storage mechanisms and the creation of many sophisticated data mining and artificial intelligence software systems. As these factors continue their inexorable progression, the use of very large data sets to make “objective” decisions will increase.
Ensuring Data Quality Through Analytics-Driven Data Governance
This trend shows the need for improved and consistently applied data governance, so that the decisions are made with accurate, timely and valid data. Humans can overcome data anomalies with experience and intuition (“that data just doesn’t look right”, “I don’t think those values are accurate”, etc.) but software is programmed to accept the data as it is presented and is expected to use it according to rules and routines instantiated in the code. Without well-governed data for super-crunching applications to use, the decisions made by the “machines” will be flawed, and could result in loss of revenue, loss of market share, loss of lives. Without well-governed processes that represent accurately the business activities and rules, the analysis software will not perform as Dr. Ayers expects, and will provide inaccurate or false or misleading results. The governance of data and process becomes increasingly important as the trend toward data-based decision-making permeates organizations from every field.
Need for Human Interaction with Data
The author still wants both human and machine to be in a mutually supportive relationship, with more weight given to machine predictions as time proceeds. Dr. Ayers answers the fundamental question of what place humans are to have in this “new world order” by identifying the need for humans to lay the foundations that enable super-crunching to occur. Humans must still “hypothesize,” he states; they must make the decisions about the variables to be used, while the computers actually perform the statistical analysis.
Humans govern data and process, humans act as data stewards, humans make the decisions about the data to be used in a data set or with an analytical application; all of these actions can fall under the role that Dr. Ayers describes for people: “laying the foundations” that enable super-crunching. This foundation must be solid, knowing the current state of a data governance effort from assessment, using accepted best practices for developing data governance and executing the role of stewardship, using data quality approaches and relevant software to ensure the accuracy and validity of the data. These foundations are also important for processes and analysis methods, since it is essential to use good data with good analysis methods to ensure good results.
Data Governance: Enhancing Compliance, Security, and Strategic Value
To maximize the benefits of data-driven decision-making, organizations must implement a strong data governance strategy. Such a strategy outlines clear goals and ensures compliance with regulatory frameworks like GDPR or HIPAA. By creating a centralized metadata repository, such as a data catalog, businesses enhance data discoverability, enabling stakeholders to better understand their data assets. Furthermore, data governance efforts are critical for managing sensitive data. Applying appropriate data security measures, such as access controls and data masking, protects against breaches and misuse, ensuring regulatory compliance while safeguarding customer trust.
A robust governance framework not only mitigates risks but also fosters team collaboration. Shared access to accurate data streamlines operations, aligning data usage with business objectives. Ultimately, businesses with effective governance can optimize operations, derive valuable insights, and gain a competitive advantage in today’s data-driven world.
The Role of Data Analytics in Driving Business Performance
Data analytics is revolutionizing how organizations utilize their data assets to make informed decisions and enhance business performance. By examining and interpreting large datasets, analytics allows organizations to extract valuable insights, uncover customer behavior patterns, and predict market trends. For instance, integrating machine learning algorithms into the analytics process can identify new opportunities and optimize resource allocation.
Secure and compliant data, governed by robust data governance practices, ensures data accuracy and integrity, creating a strong foundation for analytics initiatives. Effective data governance frameworks enable data scientists and analysts to trust the data, improving operational efficiency and customer experiences. Additionally, accurate data supports data visualization and sharing, empowering stakeholders with actionable insights.
When data governance focuses on maintaining data quality standards and preventing data misuse, organizations can maximize the value of their analytics efforts. By aligning analytics governance with organizational goals, businesses can transform raw data into a strategic enterprise asset, gaining a competitive edge in today’s dynamic market environment.
Developing an Effective Data Governance Strategy
A well-defined data governance strategy serves as a roadmap, outlining goals and priorities that guide decision-making and resource allocation within an organization. By creating a single source of truth, organizations can reduce data duplication, prevent data sprawl, and enhance efficiency. This unified approach also enforces data quality standards, ensuring accuracy and consistency, which are vital for driving reliable analytics and compliance efforts.
Robust data governance policies promote a culture of accountability by clearly defining roles and responsibilities within data management. For instance, data owners and stewards oversee critical aspects like data lineage, security, and accessibility, ensuring that organizational data assets remain trustworthy and secure. Adopting advanced tools for metadata management and data lifecycle management provides visibility into data sources and usage, further supporting regulatory compliance and minimizing risks like data breaches.
Organizations that prioritize effective data governance not only meet regulatory requirements but also position themselves for competitive success. Leveraging governance frameworks to secure data assets and integrate data across departments builds a strong foundation for data democratization, empowering employees to access and utilize accurate, trusted data for better decision-making.
Conclusion
Any organization that uses data-based decision-making or is contemplating it, should institute a data governance program to ensure their business processes provide the right data for the “super-crunchers” to load into their very large databases for their statistical packages to operate against. Since Dr. Ayers’ research shows that most types of organizations are using or planning to develop the data-based decision-making capabilities, the book Super Crunchers can be viewed as a testament for the need to develop a data governance program in all organizations.
