News & Blogs
August 31, 2012 – You probably learned in grade school that most of the Earth is covered in water; the ocean alone covers 71% of the planet's surface. The ocean contains fathoms of data, and with over 90% of it still to be explored, its processing and analysis is the very model of a Big Data problem. Marinexplore is a new open data collaboration platform and community containing 463,447,500 oceanographic measurements collected from 23,422 sensors.
Fast Co. Exist
August 31, 2012 – Hurricane season is in full swing. Here’s what it would look like if every storm since 1851 was circling at the same time. As Isaac barrels down on the Gulf Coast, take a look at this map of every hurricane on the planet since 1851 to give some sense of how these forces of nature have been battering us for the last century and a half. A pulled back view of all the storms comes close to resembling a giant hurricane; a bit of visual wordplay from nature itself.
The 451 Group
August 30, 2012 – Today is the last chance for a few months for football clubs to outsmart their rivals by buying the players that they hope will give them a competitive advantage for the rest of the season. How will data play a part? Whereas the 2002 Oakland Athletics provided a clear example of how statistics can be used to gain competitive advantage in baseball player recruitment, evidence of similar success in football is harder to find.
August 29, 2012 – How much data is too much? Depending on who’s answering, the answer may be “there's no such amount.” Many don’t share that perspective, however, and are instead overwhelmed by the amount of data available at their fingertips. It’s a growing concern for consumers of online media, engorging themselves on the endless buffet of information served through social media, smartphones, and news aggregators.
August 28, 2012 – When attempting to understand the concept of Big Data, the words MapReduce and Hadoop cannot be avoided. Although you might know the basic idea of each, digging a little deeper might help create a better understanding of how it works in relation to Big Data. MapReduce is a processing function, which consists of two parts – mapping and reducing. When I explain MapReduce, I paint a visualization of a funnel (although others use an organizational chart, but to me, a funnel is a better representation).
August 27, 2012 – Data is Big, the predictive enterprise is the way of the future, and data scientists are in high demand: you can’t glance at technology news sites in 2012 without being aware of these developments. But there’s another challenge facing organizations as they deal with the influx of data, one which receives less attention: a lack of the custom applications, skills, and development methodologies necessary to tap into its value. William Davis, Greenplum’s Senior Product Marketing Manager, characterizes this as Big Data’s “last mile problem.” With the acquisition of Pivotal Labs, Greenplum is now well-equipped to address their customer’s ever-changing needs.
August 27, 2012 – Can social media sites go through large scale emotional trends? Like, can Twitter or Facebook get depressed or have a manic episode? Can Tumblr as a whole wake up on the wrong side of the bed? We recently tried to answer that question, at least for BuzzFeed, by using numbers pulled from our reaction bar. BuzzFeed's team of data scientists were given time periods of major news stories — controversial ones from the last year that caused an uproar of some kind — and they measured positive and negative reaction use for each story.
August 27, 2012 – Someone who can bridge the raw data and the analysis – and make it accessible. A data scientist works across IT and business departments. They must be able to take a large data set, model it, and ultimately tell stories from data – usually the hardest part.
Wall Street Journal
August 24, 2012 – Mitt Romney's success in raising hundreds of millions of dollars in the costliest presidential race ever can be traced in part to a secretive data-mining project that sifts through Americans' personal information—including their purchasing history and church attendance—to identify new and likely, wealthy donors, the Associated Press has learned.
August 24, 2012 – Online pundits and media critics warn that as social media increasingly becomes a dominant source of news, and aggregators like Google News develop algorithms to surface stories that are presumably more interesting to users, we're participating in an echo chamber where self-selected social groups and online habits reinforce existing beliefs. Online activist Eli Pariser named the phenomenon "The Filter Bubble" in his book of the same name, arguing that "left to their own devices, personalization filters serve up a kind of invisible autopropaganda, indoctrinating us with our own ideas." The idea makes a fair amount of intuitive sense, and similar concerns have been raised by web luminaries such as Clay Shirky and Tim Berners-Lee. But a recently-released internal study of the behavior of 250 million Facebook users suggests such fears may be overblown.