Big data is a major topic of discussion in all organizations, of every size and industry. Technical authorities talk about petabytes of data and the potential value of information that may be found in unstructured data from in social media, multimedia, electronic messages and other sources.
So, most organizations are aware of the topic of big data, many have attempted a “big data effort" but many of those initiatives have not been successful. In reality, big data/analytics success can be elusive for a variety of reasons, including but not limited to the cost of these programs’ technical requirements, need for specialized skills to interpret the data and analysis gained from the extraction of aggregated data and the complexity of the confluence of the variety of sources of data.
However, one major reason that so many big data management initiatives fail, or do not succeed to their expected level, is the fact that few of these programs have had enterprise data management best practices applied to them. For some reason, the proponents of big data forgot the need to use the very effective practices of data governance, metadata management, enterprise data architecture, enterprise data integration, data quality management, and master data/reference data management when designing and implementing these big data initiatives.
To derive benefit from big data, organizations should understand and implement the following best practices:
i. Start with an enterprise data / information
To benefit from the high potential of big data analytics, organizations of every size and industry must adopt an approach that is based on a framework for enterprise data / information management. Following such a framework, from an organization (e.g., DAMA International - www.dama.org) allows the enterprise to use industry validated best practices. This includes the foundational concepts of business requirements, data governance, metadata management, data and process
architecture and modeling, data quality management and master / reference data management competencies.
ii. Begin with a pilot project
A successful start to big data is with a pilot project in a department or business subject area that has some significant value to the organization. The pilot should not be a test to be discarded; rather it should be a basic solution that can be used as a starting point for the rest of the organization’s efforts in big data, remembering to include the enterprise data management framework components in the pilot and all subsequent big data efforts.
iii. Focus on business value
Select the relevant data for each analysis, knowing what business value will be gained from the use of the analytics applied to that data. Carefully select the data, define it according to metadata management standards, ensure it is of high quality using metrics and that it has been integrated according to data integration best practices.
Additionally, the organization must develop and manage the metrics it uses to measure the business value for the big data initiative, to demonstrate success of the effort.
iv. Select the proper analytics tools
Many organizations are facing challenges with tool selections for big data and analytics. Some vendors insist that a tool can be used for big data analytics when it was not designed for that purpose, and other vendors are working to offer clever big data "templates" that may not answer an organization’s business needs. Choose wisely. Although companies can have custom solutions developed, most organizations will purchase products for big data analytics.
v. Enterprise data management is an essential part of the
big data program
Without sustained alignment between an organization’s enterprise data/information management components and its big data initiative, analytics will fail to be as successful as it could be. Additionally, it is essential to view enterprise data / information management and big data as continuing programs, not projects.
In the final analysis, including enterprise data/information management as part of the foundation of a big data management initiative is a main factor for success. Following these best practices can give organizations a major advantage in bringing value to big data analytics [1-3].
Anne Marie holds the degrees Bachelor of Arts and a Master's of Business Administration in Management Information Systems from La Salle University; she earned a PhD in MIS at Northcentral University. Anne Marie has earned the designations of Certified Data Management Professional (CDMP) and Certified Business Intelligence Professional (CBIP) and holds project management and facilitation certification. Additionally, Anne Marie received the DAMA International Professional Achievement award in 2015 for excellence in enterprise information management and data governance leadership and education. Anne Marie can be reached at firstname.lastname@example.org