Big Data for Big Good

With all of this talk of “big data,” it’s sometimes hard to sift through the potential functions of using big data to find a way to apply it for the greater good. Tim Barker from DataSift sat down with Daniel Pedraza from UN Global Pulse to dig deeper into their altruistic mission.


So what exactly is Global Pulse? Pedraza explained that they are the “innovation initiative” startup of the United Nations that provides research and technological innovation to help achieve the “2030 Agenda.” The goals are massive in scope: reduce poverty, decrease global hunger, provide better healthcare, and ensure clean water. To do this, Global Pulse partners with businesses that collect big data sets, and work to better understand human behavior.

Quantify Tweets to protect vulnerable populations

By using social data, Global Pulse can also observe trends in real-time and essentially stay ahead of the curve in helping others make important decisions. In our modern society, humans are acting as “sensor networks,” which allows organizations like Global Pulse to look at their collective behavior to help control human reactions to different factors.

When prices of food, gasoline, and other consumer commodities increased in Jakarta, Indonesia, Global Pulse used Twitter Data to understand changes in trends and behavior. One example: when changes in consumer goods prices in Indonesia increased, so did Tweets—in fact, there was a 0.9 correlation between the two. By monitoring “complainanomics” in this way, Global Pulse and other governments can observe spikes in conversation regarding food security, and understand how at-risk populations are impacted. And Twitter is there to provide information from their rich data source, an aggregate of human thought and emotion, all in real-time.

Observe financial transactions to provide better disaster relief

Although social data exists as some of the most available sets of big data, Global Pulse looks to multiple types of information to understand how global populations react to events. And in the same way that social data relies on private companies (like Twitter) to donate data, other types of business can help, too.

After Hurricane Odile, BBVA—one of the world’s most influential banks—provided anonymized point of sale and ATM data from Mexico to Global Pulse. The aim? Understand how the population coped: by understanding the types of goods bought before and after the storm, the UN can help governments provide better disaster relief in the future.

This work is also hard though

But with how rewarding the work that Global Pulse accomplishes, it does come with its fair share of risks and regulation. With everything they do, Global Pulse has to be cognisant of the risk involved, especially when analyzing behavior in more tumultuous regions or vulnerable communities. By working with global regulators to foster a worldwide conversation, Global Pulse can better aggregate data to gain a clearer picture before making decisions, especially those that could potentially damage society, if executed poorly.

Partnerships with the Private sector is key to better serving global populations (and there’s upside for them, too)

The UN and its member governments generally don’t have the ability or funds to collect data at scale—even though large data sets are key to achieving the “2030 Agenda” items like reduced poverty and hunger. Thanks to its startup mentality and technological savvy, Global Pulse is able to bridge the gap between the UN, government, and businesses for the greater good.

One recent example is that in February 2017, GSMA (the trade association composed of 1200 mobile carriers worldwide) announced the Big Data for Social Good Initiative. Though the data is still new and use cases are being developed, one of the first projects will be to analyze how people travel through regions facing epidemics. By understanding anonymized, aggregated movement patterns tracked on personal mobile devices—and correlating it to hospital intake data and other public signals—relief providers and governments can better predict when and how to deploy aid to affected regions.

According to Pedraza, there’s a common idiom at the UN: “The UN was created not to take mankind to heaven but to save humanity from hell.” By connecting the data from the private sectors to the needs of at-risk populations, Global Pulse plays a modern, crucial role in that mission.

The Role of Alternative Data in Finance

“What is alternative data in finance, what is its role, and how does it relate to data driven investing?” This was just one of the discussion questions posed to the three panelists during “The Role of Alternative Data in Finance” session. The moderator Megan Kelley from Fidelity Labs probed around the themes of proving the value of a given data set, what an ideal data set actually looks like, and the role of compliance as it relates to data for the finance world. This panel, including Shaivali Shah of Morgan Stanley Research, started by defining and discussing exactly what “data driven investing” is all about, which focuses on using data to produce “Alpha” (generating returns higher than a benchmark index, such as the S&P 500 for example).


One of the big themes that emerged from the panelists was regarding trust and the need for clean and accurate data. Matei Zatreanu from System2 noted that “At the end of the day you’re really solving for trust.” Pierce Crosby from StockTwits added that the whole concept of social data is inherently tricky – you need to trust the contributors to the data set, not solely the end provider you are working with.

Another large theme discussed was not only about the raw data itself, but the associated value that data ultimately provides. Matei went on to note that “At the end of the day these folks don’t care about the data, they just want the insights.” Pierce expanded further that if you can summarize statistics early on, it can go a long way towards getting people more comfortable.

The panel concluded with a discussion about key use cases and areas of opportunity: one that was unanimous was around the immense potential for collecting more and larger data sets around emerging markets.

Social Data Analytics in China

Social Data Analytics in China

CIC conducts social media research and monitoring in China for global brands that have a presence in China, such as L’Oreal. Social Touch is an end-to-end marketing solutions provider in China for more than 50 internationally recognized brands, including P&G and Airbnb, their newest customer.

Yu and Zhang began by describing the current landscape of social data as an evolution unique to the preferences of Chinese consumers. Not surprisingly, China’s social landscape did not always mirror changes in the US or global social communities. One reason may be the different cultural context. In general, Chinese customers have been less focused on privacy issues, although both later mentioned that Chinese consumers privacy has recently begin to change with privacy possibly playing a more prominent role. Also, the large population requires social companies to ‘tier’ different regions and cities. This, of course, introduces different adoption and usage and therefore, challenges to collecting and measuring the data.

The dialogue shifted to potential challenges and opportunities in China, at least as understood by Yu and Zhang. One potential challenge is data integration across multiple media. For example, understanding relationships (and possibly correlation) between television viewership and social data. This will likely to be a potential challenge and an ongoing dialogue in the years to come that extends beyond China. As for opportunities, tapping into China’s older generation was suggested as a new potential market. Here is why. Improved technology has introduced tech gadgets and software that is more intuitive to use and this alone has removed a significant barrier to adoption.

This conversation touches the tip of the iceberg when considering opportunities within social data analytics in China. Hopefully, Big Boulder will facilitate more in depth conversations among its attendees and within the larger, global social data community.