Modern Metrix

The Rise of Data Scientists? Apparently, Not Fast Enough

As data become more abundant (and messy), there is increasing demand for more specialists who could not only separate noise from proverbial "golden nuggets" of information, but also make those "nuggets" useful and actionable:

"Hilary Mason, chief scientist for the URL shortening service bit.ly, says a data scientist must have three key skills. "They can take a data set and model it mathematically and understand the math required to build those models; they can actually do that, which means they have the engineering skills…and finally they are someone who can find insights and tell stories from their data. That means asking the right questions, and that is usually the hardest piece."

It is this ability to turn data into information into action that presents the most challenges. It requires a deep understanding of the business to know the questions to ask. The problem that a lot of companies face is that they don't know what they don't know, as former U.S. Defense Secretary Donald Rumsfeld would say. The job of the data scientist isn't simply to uncover lost nuggets, but discover new ones and more importantly, turn them into actions. Providing ever-larger screeds of information doesn't help anyone."

Gmail Meter Helps You Analyze Your Gmail Usage

Just as I started looking for better ways to measure my Gmail usage, Gmail labs rolled out the Gmail Meter script! Among the statistics, provided on a monthly basis and summarized via nice charts, are volume, traffic patterns and time analysis, length of messages and conversations, and even how long does it take for you to respond to e-mails. Check it out!

Where Facebook Ads Work Better: States with the Highest and Lowest Click-Through Rates

This article in Mashable shares interesting statistics on geographical difference in click-through rates for Facebook ads,  provided by digital marketing agency:

"Facebook users in South Dakota, Tennessee and Colorado click through ads on Facebook at record rates. In South Dakota, Facebook users click on ads at a rate of 174% the national average.

Meanwhile, residents of Vermont, Alabama and Indiana are the least likely to take the bait. In Vermont, the click-through rate (CTR) is 57% the national average."

The differences are not explained. If you ask me, the first what comes to mind is demographics, most likely, education (states with less educated population have higher click-through rates). I also suspect that states with the higher click-through rates are are ealier stages of Facebook adoption (the novelty factor comes in play, hence more clicks). But that is just IMHO.

Free Twitter Data!

I am always on the look out for free data to play with, and today stumbled upon awesome depository of Twitter data on Infochimps website. Check it out!

 

Neighborhood Lifestyle Metrics using Foursquare Data!

In this post on their blog, Foursquare engineering team describes how they used Foursquare activity data to compute lifestyle metrics of city neighborhods, and then used these metrics to compare neighborhoods across different cities:

"First, to collect data about neighborhoods, we built some Hive queries to access our large collection of check-ins (stored in S3) and count the number of check-ins per category for every neighborhood in the world. For example, the East Village of New York has 230k check-ins at bars, 57k check-ins at pizza places, 18k check-ins at yoga studios, and 34k check-ins at karaoke places (hipsters like to sing!).

We then used MATLAB to represent each neighborhood by a 400 dimensional vector which specifies the normalized probability distribution of checking in to a place in each category relative to the baseline distribution of the city. This approach allows us to compare neighborhoods with each other using a similarity metric such as cosine similarity, or KL divergence."

What an awesome hackday project!

The Rise of Self-Tracking Reports

It looks like life-logging is becoming more popular . I myself have been looking lately into various tools for collecting lifestyle, biometric, psychographic, and sociometric data points on a daily basis, and soon will start posting reviews of some of these tools on my blog. Not sure if I will ever be ready to release the summary reports, though...

 

Using Consumer Data to Change their Shopping Habits

Fascinating article in NYT magazine on how companies use consumers' data to change and shape their shopping habits. 

"As Pole’s computers crawled through the data, he was able to identify about 25 products that, when analyzed together, allowed him to assign each shopper a “pregnancy prediction” score. More important, he could also estimate her due date to within a small window, so Target could send coupons timed to very specific stages of her pregnancy."

Unfortunately, author makes it sound a bit too creepy and invasive. But yep, that's what we do.

OkCupid's Data and Their Analysts - A Match Made in Heaven (Valentine's Day Post)

Interesting article in AdAge about OkCupid dating service and how their use their treasure trove of psychographic and behavioral data in new innovative ways:

"OKCupid also mines its data sets to conduct market research for brands, such as big-box retailers. By examining the way women in certain target ZIP codes describe themselves on their profiles and answer questions on the site, Mr. Yagan believes brands can uncover findings that can help them market better to current and prospective customers alike."

 That includes including sharing insights on their awesome blog OkTrends.

Google Study of Web Browsing Habits

Wow, it looks like Google has partnered with Knowlede Networks to conduct a large-scale study of web browsing habits:

"A new Google project named Screenwise looks set to offer Amazon vouchers in return for information about your browsing habits.

While it doesn’t appear to have launched officially yet, a landing page highlighted bySearch Engine Land notes that “Google is building a new panel to learn more about how everyday people use the Internet.”

Screenwise will work via a Chrome browser extension which Google will use to track the sites you visit and how often you visit them. “What we learn from you, and others like you, will help us improve Google products and services and make a better online experience for everyone,” the landing page says. You can enter your email address on the page if you’d like to learn more."

I tried to sign up but got an error message. Perhaps, Google Spreadsheets can't handle too many requests?

Visualization of Annual Energy Consumption by NYC Buildings

Check out this awesome map visualizing annual energy consumption in NYC at the tax-lot (essentially, an apartment building) level. Interestingly, it's based not on data from Con Edison or other utility companies, but using estimates obtained from the model created by researchers from Columbia University (link to full article here). It would be interesting to validate these numbers against the ConEd data.