Modern Metrix

New York City Council Frees NYC Public Data

This is great: NY City Council passed yesterday the Intro29A measure that requires all city agencies to make their public data available on NYCOpenData portal:

"Intro 29 would phase in over the next six years, starting with a technical manual to be released by DoITT this summer guiding city agencies on how to format and upload their data. Agencies would have until early next year to upload all public datasets currently on the internet to the new portal. By 2018, all public data would have to be posted. For the time being, DoITT is planning to power the new data portal with Socrata, the service that currently hosts NYC OpenData."

The Life of the Number-Cruncher, in Charts (Humor)

Check out these awesome charts, depicting a daily life of a number-crunching analyst. The Mindset (below) is my favorite one:

 

 

 

The Rise of Self-Tracking Reports

It looks like life-logging is becoming more popular . I myself have been looking lately into various tools for collecting lifestyle, biometric, psychographic, and sociometric data points on a daily basis, and soon will start posting reviews of some of these tools on my blog. Not sure if I will ever be ready to release the summary reports, though...

 

Using Consumer Data to Change their Shopping Habits

Fascinating article in NYT magazine on how companies use consumers' data to change and shape their shopping habits. 

"As Pole’s computers crawled through the data, he was able to identify about 25 products that, when analyzed together, allowed him to assign each shopper a “pregnancy prediction” score. More important, he could also estimate her due date to within a small window, so Target could send coupons timed to very specific stages of her pregnancy."

Unfortunately, author makes it sound a bit too creepy and invasive. But yep, that's what we do.

Datapointed

Check out DataPointed: bits of interesting data insights, visualized. 

OkCupid's Data and Their Analysts - A Match Made in Heaven (Valentine's Day Post)

Interesting article in AdAge about OkCupid dating service and how their use their treasure trove of psychographic and behavioral data in new innovative ways:

"OKCupid also mines its data sets to conduct market research for brands, such as big-box retailers. By examining the way women in certain target ZIP codes describe themselves on their profiles and answer questions on the site, Mr. Yagan believes brands can uncover findings that can help them market better to current and prospective customers alike."

 That includes including sharing insights on their awesome blog OkTrends.

Annual Income as an Antecedent of Life Expectancy?

This study suggests that life expectancy is tied to income, and thus economic equality often leads to health disparities across different areas of the same city/town.

Infogr.am - Web App for Infographics

Check out http://infogr.am/ - new web app for creating and sharing infographics. Looks promising!

Google Study of Web Browsing Habits

Wow, it looks like Google has partnered with Knowlede Networks to conduct a large-scale study of web browsing habits:

"A new Google project named Screenwise looks set to offer Amazon vouchers in return for information about your browsing habits.

While it doesn’t appear to have launched officially yet, a landing page highlighted bySearch Engine Land notes that “Google is building a new panel to learn more about how everyday people use the Internet.”

Screenwise will work via a Chrome browser extension which Google will use to track the sites you visit and how often you visit them. “What we learn from you, and others like you, will help us improve Google products and services and make a better online experience for everyone,” the landing page says. You can enter your email address on the page if you’d like to learn more."

I tried to sign up but got an error message. Perhaps, Google Spreadsheets can't handle too many requests?

"Allocation Flags" in ACS and Census Data

I am currently working with just released ACS 2006-2010 data, and was puzzled by so called "allocation flags". Turns out that Census Bureau uses these flags to mark the records for which original responses have been altered. My immediate question was why would Census alter the responses, and how would they do that. So naturally, I googled it, and the best explanation I found was on Minnesota Population Center web site. In summary, if record has a variable  that is missing or inconsistent with other information, its value is obtained or replaced by means of logical induction (e.g., if marital status is missing for a person of 12 years old, it is automatically filled as "single"), ascription ("hot deck allocation", finding a similar respondent with full record, and using his response to replace the missing value), or imputation ("cold deck allocation", assigning a random value from a pre-determined distribution, or replacing it with mode).