Written by Ebbe Kernel, data gathering analyst and consultant

Some say data is now more valuable than oil; when you see how data has become a commodity in today’s modern world, it is easy to see why. In fact, more businesses collect data about their customers and the market in general today than ever before.

There are several reasons why data is now considered to be very important and valuable. The push for data-driven business operations is one of the most prominent reasons. At the same time, technologies such as artificial intelligence allow for faster and more accurate data processing.

Data collection as a business process

The process of utilising data starts with collection. There are a lot of ways to gather data that businesses – and even individuals – now use. Scraping the World Wide Web for personal details, for example, is a common practice. A leading proxy provider Smartproxy writes about the most efficient way to scrape data. Since the details appear in public sources (i.e. social media posts) , collecting them using scraping methods is considered legal.

Another common method of data collection is through forms on websites. By explicitly asking users to provide certain details, usually in exchange for web services or specific benefits, companies can learn more about their users than ever before. It is not uncommon for web services and apps to ask for more details to unlock certain features.

Of course, there are also ways to collect data without actual user inputs. Tracking using a scripted pixel or an analytics tool is a good example. When you log into your Facebook account, a cookie is added to your device. Whenever you access sites that incorporate Facebook Pixel, your online activities are tracked and recorded. The more Facebook knows about the sites that you visit, the more accurate it can be at predicting your behaviour.

An opportunity for entrepreneurs

Data presents many opportunities for businesses. It is not the exclusive domain of large corporations with big budgets; far from it. Data is now something that even home businesses can collect and process. Your small business can become more competitive in a saturated market when you know exactly the kind of data you need to collect to form a better understanding of the customer.

When you have data and insights about your customers, for instance, you can begin sending targeted emails for cross-selling and other business purposes. The insights generated by your data framework can be fed to automation tools, digital campaign resources, or to deliver a more personalised user experience.

Data can also be used to push for operational excellence. By knowing at which stage potential customers drop their carts, you can refine your checkout process and make it friendlier to users. Lowering the dropped carts rate is easy when you have a clear view of how users interact with your UI. Even better, you can take things further and customise the checkout process for certain users to maximise sales.

Challenges of data hoarding

The use of data is not without its challenges. At the top of the list, we have privacy concerns surrounding data collection and the methods businesses use to run their data collection routines. Regulations such as GDPR are put in place to limit the ability to collect consumer data, especially private details that are not meant to be used publicly or for tracking purposes.

Methods such as scraping are under specific scrutiny. A recent case faced by LinkedIn has the potential to set a far-reaching precedent. In a lawsuit filed by an automated scraping company, hiQ, LinkedIn’s claim of property and users’ expectation of privacy is being challenged. The 9th circuit court – as part of its preliminary decision – doubts LinkedIn’s claim that hiQ can collect user data through other means.

The case puts forward a risky notion that users have no expectation of privacy when they post their personal details online. This means companies like hiQ can collect publicly available data from LinkedIn as long as they only use legal means to scrape it.

In another case, Ryanair and Expedia decided to settle their differences following a screen-scraping dispute. Ryanair claimed that Expedia used screen-scraping to get information about airfare and deals. Despite the settlement, this method of scraping for information is still deemed legal under most circumstances.

The future of data collection

The two cases showed just how fragile the legality of data collection through scraping and other common means really is. New regulations are being put in place to limit businesses’ ability to scrape data from public sites. They are also adding extra security measures for users (i.e. the ability to ask for collected data to be removed from databases) along the way.

Will data collection remain legal and possible? It remains to be seen whether businesses will be able to collect and hoard data for long.

[Image – Photo by Curtis MacNewton on Unsplash]