How to Write an Effective Data Acquisition Strategy: An In-Depth Guide

Within the public sector, there is an ever-increasing need for data to support decision making. Despite this fact, many public officials are unfamiliar with the processes involved in evaluating, purchasing, and storing new data assets in formats that can be easily accessed. The sheer quantity of data sources on the market, combined with the number of evaluative factors to consider can make the process feel quite daunting. However, when guided by a tactful strategy, data acquisition can be efficient and productive.

We designed this guide to help public officials build effective strategies for acquiring new data and to integrate these methods into existing governance policies. Whether your agency is just starting to experiment with emerging data sources, or simply looking to improve the overall management of existing data assets, here are some tested tips and pieces of advice that will help you to craft an effective data acquisition strategy.

Data acquisition strategy 101

A data acquisition strategy lays out a clear plan for gathering new data in a way that meets your organization’s strategic needs. It’s the roadmap that guides staff in how to evaluate, acquire, store, and leverage that new information efficiently and effectively. A well-designed strategy should provide clarity on the specific roles and responsibilities of each person involved throughout the process.

An effective data acquisition strategy should address the following:

• Needs Assessment: Identifying the information required to satisfy the objectives of your project or use case, as well as your wider organizational goals.

• Auditing Existing Data: Auditing your existing data to prevent duplicate purchases and find areas that require improvement.

• Data Evaluation: Evaluating the costs, benefits, and quality of different data sources on the market and determining whether they meet your objectives.

• Data Privacy: Planning how to protect personally identifiable information (PII), and address any data privacy issues throughout the data evaluation and acquisition process.

• Storage and Security: Determining the location where raw and processed data will be stored (locally within your organization or in the cloud in the appropriate region) to ensure the security and accessibility of your data.

• ETL and Data Integration: Evaluating the steps required to engineer raw data into useful information, as well as any tools needed in this process.

Components of an effective data acquisition strategy

Avoid costly mistakes in your data acquisition strategy

An effective data acquisition strategy is a valuable tool that will help your organization avoid costly mistakes. For example, not adequately auditing your existing data before new purchases could lead to ineffective or duplicate investments. Not being realistic about the internal resources you have at your organization could lead to data sitting idly on servers with no one to process it. 

Common pitfalls of data acquisition

The importance of data governance

An effective data acquisition strategy should intersect with your agency’s overarching data governance framework. Data governance refers to the overall processes and methodology for managing an organization’s data resources at each stage of their lifecycle. By optimizing management at each stage, organizations can increase effectiveness, efficiency, and ultimately generate greater value from the data.

The government data value cycle

Source: van Ooijen, C., B. Ubaldi and B. Welby (2019[3]), “A data-driven public sector: Enabling the strategic use of data for productive, inclusive and trustworthy governance”, https://doi.org/10.1787/09ab162c-en.

The best way to think about data governance (and how it differs from data acquisition) is that data governance is the framework under which all data efforts take place – from capture to utilization, to protection across the enterprise. Data acquisition is just one piece (box #1 in the figure above) that guides capturing and entering new data into your systems.

Evaluating your needs

Before evaluating or purchasing any data, it is important to determine your objectives. There are many different types of data on the market, so it is important to thoughtfully consider what you want to measure and the outputs you want to get from the analysis, as this will influence the kinds of data that you should procure. This ultimately comes back to understanding your use case and selecting the most conducive and relevant data.

Consider including the following prompts in the needs assessment section of your data acquisition strategy:

• What are the goals and objectives for this project, use case, or analysis?
• What are your personal, team-wide, and agency-wide priorities?
• How will you measure success? What are the key performance indicators?
• What type of audience will consume this analysis? Is it the public?
• What is the level of transformation required to present this data to that audience?

Once you’ve identified what you want to measure, and how you plan to measure success, you can consider what data you need to make it happen.

Auditing your data

The next step after the needs assessment is to evaluate the current state of your data. Do you own or have access to existing data that will satisfy your analysis, or will you need to explore new sources of information? Evaluating what you have will help you avoid duplicate purchases and identify gaps or areas for improvement. For example, if your agency has a list of business data from 2010, you may need a new dataset to supplement that list with businesses that have since opened or closed.

This is where the importance of data governance comes back into play. A data catalogue is an extremely useful resource that serves as the ‘data state of the union’ at your organization. It makes for easy search and navigation and without one, especially in larger and more siloed organizations, it can be very difficult to know exactly what data you have without spending a lot of time digging through folders. 

Making a business case

After identifying the gaps in your existing data, consider the following questions:

• Why is your existing data insufficient for this use case?
• Why is it important to fill in the identified data gaps?
• What are different ways to fill in this data gap? I.e hiring more staff to perform surveys, installing additional sensors, purchasing additional data.
• Will the potential new data be reusable for other use cases?

The answers to these questions will form the foundation of your business case for new data acquisition. If you are unable to articulate why certain data collection activities are important to your organization, the implementation will most likely fail.

Once you’ve determined your objectives, assessed your existing data, and built your case for filling in the gaps, it’s time to decide where, specifically, you’re going to get the new data. For that, it’s important to understand a bit of background on what data is currently available on the market.

The data collection revolution

The world of data collection has progressed dramatically over the past few decades. For a long time, sourcing new information was solely reliant on surveys, questionnaires, manual counting, and other human-driven methods. Since then, technological advancements have enabled faster, more accurate, and more convenient methods to enter the mainstream.

For example, the 1900s ushered in a new wave of data collection technology with the notable proliferation of the pneumatic tube and the induction loop. Cutting edge for their time, these technologies are still widely used today to count traffic volumes.

Pneumatic tube road sensor

Today, the Internet of Things (IoT) and the sophistication of sensor technology has taken us into the next frontier of data collection. This technology, coupled with the proliferation of 5G, enables data and insights that would otherwise be impossible to obtain (as sensors can be placed virtually anywhere). This means that we can collect more accurate data faster than ever before, with little latency or human intervention. All this data is extremely valuable if we know how to collect and use it ethically and effectively.

Rise in Google searches for “Internet of Things” over the years
Source: Google Trends

Modern strategies for modern data sources

The diversity of modern data collection methods and sources presents compelling avenues to use data in strategic and operational decision making. Government agencies now have the opportunity to capitalize on emerging data sources to supplement traditional ones, or in some cases, replace them altogether.

This diversity also emphasizes the importance of data sharing standards both embedded within connected devices themselves and between the public and private sector to ensure that data can talk to each other and be fully leveraged.

Futureproofing your data acquisition strategy

Even though today’s methods are more advanced, data collection techniques are still evolving. Data acquisition strategies should take into account likely changes in technology, such as advances in machine learning or artificial intelligence, which may allow new types of information to be gathered more easily. For example, cameras with edge technology process captured images and send only insights back i.e traffic count metrics. This type of technology helps mitigate potential data privacy concerns since data consumers only see aggregated data.

Data evaluation

There are many important factors to consider when evaluating new data. Is it representative? Cost-effective? Ethical? Does your organization have sufficient internal resources to process and use it effectively?  To help with these questions, we’ve put together a data evaluation checklist to run through when weighing the cost, benefits, and quality of new potential data sources.

Download the free guide

In the guide, we detail each stage of the evaluation process so that public officials can assess new data sources effectively, securely, and with confidence.

Data privacy

Baking data privacy considerations into your data acquisition strategy is vital to mitigating risk and fostering a culture of critical thinking. Below are a few ideas for public officials considering data privacy in their data acquisition strategies:

• Form a Data Governance Board to plan out policies for handling privacy issues (or take lessons from others).
• Write clear instructions surrounding access control of all data
• Write a clear policy for handling PII

As UrbanLogiq’s Chief Technology Officer (CTO) Max Burke puts it, “security, privacy, and insights are not mutually exclusive and can be done with technology that is developed responsibly and with safety and protection as first principles.”

Data security and storage

Requiring all IT vendors to provide proof of third-party certification of their security and privacy procedures and practices is a good place to start regarding the data security section of your strategy. According to UrbanLogiq’s CTO Max Burke, “tangible proof of compliance with a well-known standard, such as ISO or NIST, is one concrete way governments can protect themselves against potential risk. It also creates a network effect of verified, reliable, trusted providers to compare equally against the same standard.”

As a purveyor of data, UrbanLogiq is certified in the world’s leading standards in cybersecurity and data privacy compliance. Namely, ISO 27001 and 27701, and NIST 800. Our extensive work with emerging data sets has led us to conduct and pass many third-party privacy and bias impact assessments. As a result, our team is able to help agencies navigate the application of big data responsibly, transparently, and ethically.

Below are some actionable ideas for public officials considering data security in their data acquisition strategies:

• Know your ISO 2700 series and NIST 800 series 
• Join an ISACS group to share threat information and best practices 
• Form alliances with neighboring municipalities to cooperatively procure cybersecurity services
• Schedule regular security conversations with your procurement partners (both existing and new) 
• Bake security terms into contracts with IT providers. For example, specify that you must be notified of any security breaches
• Proactively communicate your data management privacy policies to data providers

Storing data

The quantity of raw data generated from IoT and the resulting file sizes (i.e in the terabytes of data) are often too large for government agencies to store on-premise. Many require a data platform or partner to store and analyze the data for them, due to resource constraints.

For example, UrbanLogiq makes it easy for public officials to integrate unconventional or big data sources to serve a wide variety of use cases in the UrbanLogiq Platform.  We store, normalize, visualize, and deliver the data in our geospatial data platform alongside an agency’s existing data. We have established partnerships with leading market providers that ensure the highest data quality, value, and privacy standards.

ETL and data integration

Raw data must be transformed into useful information before it can be used effectively. As such, this section of your strategy should outline the steps required to clean, standardize, and transform the raw data. You should also outline any tools and technology required for this process.

As mentioned above, some data formats will be easier to transform into useful information than others. Therefore, consider the data science and data engineering resources you have available at your organization.

Procuring new information safely and effectively

In this ever-evolving data landscape, a thoughtful strategy can help your agency procure new information safely and effectively. Planning and preparation are essential to building a comprehensive approach that both reduces risk and streamlines the data evaluation process. Your strategy should be a living document that evolves as you develop new procedures and make important shifts.

How UrbanLogiq helps

With deep expertise in data spanning many disciplines, we at UrbanLogiq believe in the universal value of data for everyone. UrbanLogiq helps agencies determine where their data gaps are and formulate strategies for new data acquisition. 

UrbanLogiq collects no margin on any third-party data provider. Further, we do have any commercial incentive to advocate for one supplier over another. We recommend data sources based on the needs of our clients, who will always make the final decision.

Want to learn more? 

Contact us today to book a 30-minute demo!