Managing Data in Organizations

Harvesting data has become a top priority for many businesses around the world. According to a recent survey by the accounting firm KPMG, 70% of business leaders report that their companies have ramped up their collection of consumer data in the last year. The same survey, however, found that 86% of Americans feel concerned about data privacy, and 68% worry about the amount of data being collected by businesses.¹

Companies can help alleviate these fears by ensuring data security and managing data ethically and responsibly. They can also use data to improve decision-making and streamline their business processes. However, successful data management strategy involves numerous ethical and technical considerations.

Read on to explore data management best practices and technologies available to help organizations handle vast amounts of data.

The National Institutes of Health defines data management as, “the process of validating, organizing, protecting, maintaining, and processing scientific data to ensure the accessibility, reliability, and quality of the data for its users.”² Clearly, good data management is a complex, multi-step process.

Companies use a broad range of data collection methods, including interviews, surveys, and analysis of data sources such as emails and customer purchasing histories.³ Data analysts also use quality checks to ensure they’ve gathered trustworthy data. This process involves verifying that a given data set is accurate, complete, secure, and readily available.⁴

Qualitative data enables analysts to explore problems without obvious solutions, while quantitative data can reveal patterns and trends.³ For instance, business users may rely on qualitative focus groups to investigate why customers aren’t buying a new product and analyze quantitative sales data to review purchasing trends.

Companies often use data management solutions to store the information they’ve gathered in secure locations. There are several types of data storage options:⁵

• Databases use online transaction processing and allow users to insert, delete, and update data easily
• Data warehouses utilize online analytical processing to evaluate large amounts of data faster
• Data marts are enterprise data warehouses typically used to store finance and marketing data

Companies can build custom data management infrastructure from the ground up or use data management platforms, such as SAS Data Management or Snowflake.⁶

Not all data assets are usable. Data sets often contain errors, inconsistencies, and other flaws that may lead those conducting data analysis to reach incorrect conclusions. Analysts can ensure they get accurate results by cleansing and preprocessing data. This process involves:⁷

• Arranging all data in the same format
• Correcting mistakes
• Deleting anomalous data
• Filling in missing data

Data governance refers to organizational policies and standards that ensure businesses use high-quality data, handle it ethically, and comply with all relevant laws. Best data governance practices include:⁸

• Making data-related decisions transparent
• Establishing policies for the full data lifecycle
• Promoting data accessibility
• Choosing a strong leader to guide the data governance team
• Providing adequate resources for data governance

Regulations for data governance vary by location and industry. Organizations that operate in the European Union must comply with the General Data Protection Regulation (GDPR). This law regulates how businesses can use personal consumer data, such as email addresses and tax identification numbers.⁹

Several national and state laws govern data use in the United States. For example, the Colorado Privacy Act gives consumers the right to opt out of having their personal information sold or used for targeted marketing.¹⁰ The Children’s Online Privacy Protection Rule addresses data security for young people, restricting businesses from gathering personal data online from children younger than 13 years old.¹¹

Data integration and data consolidation are closely related concepts. Data consolidation refers to the initial movement of data from different sources to a new location.¹² Data integration refers to combining data from different sources into a central data warehouse to gain new insights. Analysts often use the Extract, Transform, and Load (ETL) process to clean data and migrate it to a new location. They may also use tools such as the Talend Open Studio to streamline this process.¹³

Business analysts use software and statistical formulas to help them derive meaning from data. Data visualization—”the graphical representation of information and data”¹⁴—allows them to deepen their evaluation of data and observe patterns and trends that might otherwise go unnoticed. Because humans readily perceive visual information, the use of charts, graphs, maps, infographics, and other data visualization tools makes it easier for analysts to communicate their findings effectively with diverse audiences.¹⁵

There are two main sub-types of data visualization:¹⁵

• Information visualization represents abstract data, such as business data
• Scientific visualization represents scientific data, such as the human body, the environment, or the atmosphere

Analysts face many ethical considerations when they manage and use personal data. According to the Data Science Association Code of Professional Conduct, ethical data management strategies include:¹⁶

• Protecting confidential information throughout the data lifecycle
• Informing authorities if a client misuses data science to deceive others
• Ensuring that data isn't cherry-picked
• Taking steps to avoid harm when creating and implementing algorithms to analyze data

Rather than simply collecting vast amounts of raw data, most companies want to use their data to guide decision-making. Businesses can follow these steps to foster a data-driven culture that embraces information:¹⁷

• Ensure that leaders model informed decision-making
• Encourage business analysts to collaborate with colleagues in other departments
• Offer specialized training when staff need it, but don’t overwhelm them with advanced tools
• Encourage teams to explain their processes for using data to make decisions

Learn to navigate a data-driven world with confidence. Santa Clara University’s Online Master of Science in Business Analytics (MSBA) program builds the expertise you need for the career you want. Whether you’re a seasoned professional looking to acquire new skills or an aspiring business leader seeking a strong career start, the Online MSBA curriculum will empower you to analyze complex data sets, quickly identify trends and opportunities, make informed decisions, and improve performance.

Learn more about the Leavey School of Business: the expert faculty, networking opportunities, and admissions process. Schedule a call with an admissions outreach advisor today.