Just another WordPress site

Blog

  • \”Cash for Clunkers\” – Clunker by Country Vizualization

    I\’m currently working on a chapter for the upcoming O\’Reilly book \”Beautiful Visualization\"\"\” (a new book in the \”Beautiful\” series) and one of the things that I do is walk readers step by step through gathering data and sifting through it in order to create a visualization from the Cash for Clunkers data.

    As I was looking through the Cash for Clunkers data, I was fascinated by the extent to which it seemed that the clunkers being turned in were disproportionally from companies based in the US. So I dug into the data and found out that it didn\’t just seem that way… 85% of the cars \”clunked\” came from US based manufacturers.

    So I decided to create a visualization to identify which countries gained market share due to the Cash for Clunkers program. So… here it is. Click for a larger view. (caveats below).

    \"\"

    You can access the raw data here.

    Caveats:

    1. Yes, nearly all Toyota and Honda and Hyundai vehicles are built in the US. I used the \”where is the parent company headquartered\” as my way of determining country size. That made for a more compelling image.
    2. It makes a certain kind of sense that people would dump a lot of old US-made vehicles because US manufacturers were at the forefront of the SUV boom in the early-mid 2000\’s (aughts? oughts? naughts? This next decade will be so much easier), so it seems to make sense that people who bought SUV\’s would be most eligible for a Cash for Clunkers rebate. If you bought a fuel efficient Toyota Camry in 2002, you\’re not going to be eligible to trade your vehicle in, so it seem unlikely that you would do so.

    With all that being said, I think it\’s obvious that US manufacturers have lost market share on these transactions. I\’d need to do a shade more research, but my understanding is that Ford (which didn\’t take any bailout cash) didn\’t do too badly while Chrysler and GM saw a large number of their vehicles turned in and comparatively very little purchasing.

    What does this mean for the future? I don\’t know. This was more for fun and for my book chapter than for anything else. And if you want to learn how to do something like this, just buy \”Beautiful Visualization\"\"\” when it comes out.

  • \”Cash for Clunkers\” – Clunker by Country Vizualization

    I\’m currently working on a chapter for the upcoming O\’Reilly book \”Beautiful Visualization\"\"\” (a new book in the \”Beautiful\” series) and one of the things that I do is walk readers step by step through gathering data and sifting through it in order to create a visualization from the Cash for Clunkers data.

    As I was looking through the Cash for Clunkers data, I was fascinated by the extent to which it seemed that the clunkers being turned in were disproportionally from companies based in the US. So I dug into the data and found out that it didn\’t just seem that way… 85% of the cars \”clunked\” came from US based manufacturers.

    So I decided to create a visualization to identify which countries gained market share due to the Cash for Clunkers program. So… here it is. Click for a larger view. (caveats below).

    \"\"

    You can access the raw data here.

    Caveats:

    1. Yes, nearly all Toyota and Honda and Hyundai vehicles are built in the US. I used the \”where is the parent company headquartered\” as my way of determining country size. That made for a more compelling image.
    2. It makes a certain kind of sense that people would dump a lot of old US-made vehicles because US manufacturers were at the forefront of the SUV boom in the early-mid 2000\’s (aughts? oughts? naughts? This next decade will be so much easier), so it seems to make sense that people who bought SUV\’s would be most eligible for a Cash for Clunkers rebate. If you bought a fuel efficient Toyota Camry in 2002, you\’re not going to be eligible to trade your vehicle in, so it seem unlikely that you would do so.

    With all that being said, I think it\’s obvious that US manufacturers have lost market share on these transactions. I\’d need to do a shade more research, but my understanding is that Ford (which didn\’t take any bailout cash) didn\’t do too badly while Chrysler and GM saw a large number of their vehicles turned in and comparatively very little purchasing.

    What does this mean for the future? I don\’t know. This was more for fun and for my book chapter than for anything else. And if you want to learn how to do something like this, just buy \”Beautiful Visualization\"\"\” when it comes out.

  • "Cash for Clunkers" – Clunker by Country Vizualization

    I\’m currently working on a chapter for the upcoming O\’Reilly book \”Beautiful Visualization\"\"\” (a new book in the \”Beautiful\” series) and one of the things that I do is walk readers step by step through gathering data and sifting through it in order to create a visualization from the Cash for Clunkers data.

    As I was looking through the Cash for Clunkers data, I was fascinated by the extent to which it seemed that the clunkers being turned in were disproportionally from companies based in the US. So I dug into the data and found out that it didn\’t just seem that way… 85% of the cars \”clunked\” came from US based manufacturers.

    So I decided to create a visualization to identify which countries gained market share due to the Cash for Clunkers program. So… here it is. Click for a larger view. (caveats below).

    \"\"

    You can access the raw data here.

    Caveats:

    1. Yes, nearly all Toyota and Honda and Hyundai vehicles are built in the US. I used the \”where is the parent company headquartered\” as my way of determining country size. That made for a more compelling image.
    2. It makes a certain kind of sense that people would dump a lot of old US-made vehicles because US manufacturers were at the forefront of the SUV boom in the early-mid 2000\’s (aughts? oughts? naughts? This next decade will be so much easier), so it seems to make sense that people who bought SUV\’s would be most eligible for a Cash for Clunkers rebate. If you bought a fuel efficient Toyota Camry in 2002, you\’re not going to be eligible to trade your vehicle in, so it seem unlikely that you would do so.

    With all that being said, I think it\’s obvious that US manufacturers have lost market share on these transactions. I\’d need to do a shade more research, but my understanding is that Ford (which didn\’t take any bailout cash) didn\’t do too badly while Chrysler and GM saw a large number of their vehicles turned in and comparatively very little purchasing.

    What does this mean for the future? I don\’t know. This was more for fun and for my book chapter than for anything else. And if you want to learn how to do something like this, just buy \”Beautiful Visualization\"\"\” when it comes out.

  • Bad Visualizations – David McCandless Lets Politics Get In The Way

    Well, this makes me sad.

    David McCandless runs the fantastic information visualization blog Information is Beautiful. Nearly all of his work is fantastic information visualization (his piece on drug deaths in England is really cool.

    He recently created a visual about troops and troop deaths in Afghanistan. One part of the visualization got picked up by Andrew Sullivan. It was a graph on troop levels in Afghanistan and who has contributed the most troops. Mr. McCandless accompanies the chart with the words \”That\’s a huge amount of hired guns.\”

    \"\"

    The problem is that McCandless doesn\’t source that number. I said to myself \”71,700 hired guns? That seems high.\” It didn\’t pass the smell test.

    I looked into the number. Near as I can tell, its basically a huge mistake on McCandless\’ part. He didn\’t source where he got the \”71,700 private security contractors\” stat and he didn\’t say anything when I tweeted to him to ask where he got it. And he didn\’t respond in the comments section of his blog when I asked. So I had to go searching for it.

    It looks like the number comes from this Washington Times piece which mentions that there are 71,700 contractors, not all of whom are private security contractors. And yet McCandless not only changes this important data point (HUGE no-no in my book), he goes on to push the point with his \”hired guns\” comment.

    Why would he do such a thing? My guess is that he doesn\’t like the war in Afghanistan, so that kind of makes it OK to push a \”mercenaries\” point of view by lumping all contractors into the \”private security\” category.

    Are you teaching in Kabul? You\’re a \”hired gun\”.

    Building a bridge? You\’re a \”hired gun\”.

    Flying supplies in? \”Hired gun.\”

    Maintaining a network for the government? \”Hired gun.\”

    Working as a translator? \”Hired gun.\”

    It\’s basically a data labeling mistake made worse by an wildly inaccurate (and, frankly, quite stupid) comment.

    The reason it\’s taken me so long to get to this is because I didn\’t want to say anything bad about Mr. McCandless without giving him a chance to explain. It\’s obvious to me that he\’s not going to. If he does, I\’ll post his explanation at the top of this post. But it\’s given me new insight into the old saw that a lie is half-way around the world before the truth can get its pants on. Being right and being generous to others is something that takes caution and time.

    (By the way, I proceeded to contact Andrew Sullivan and tell him that I thought the information was bogus. He hasn\’t responded, but I figure that\’s because he\’s completely consumed with other correspondence. I optimistically maintain he would correct his post if he had read my note.)

  • Visualizing the CRU E-Mails

    Very cool visualization of the Climategate e-mails over here. For more information see the Computational Legal Studies blog post.

    Additionally, they have hub and authority scores for the authors of the e-mails. I like.

    Thanks to Pankaj Gupta and Drew Conway for pointing me to this.

  • ClimateGate: Free The Data

    I wanted to get this out because I\’m quickly becoming consumed with other things. But I\’ve been following the ClimateGate scandal for coming up on a week now. And every time I turn around it looks worse for anthropogenic global warming.

    For those of you who don\’t know what I\’m talking about, here\’s a quick summary:

    Someone stole (or possibly leaked) a ton of files and e-mails from the Climate Research Unit

    My position on climate change has heretofore been: \”I\’m not a climate scientist, but there seems to be a pretty significant agreement among those who are that the main points of climate change are solid. The earth is warming and humans are causing it to some degree. The extent to which humans are causing it (do we account for 90% of the change? 50%? 30%?) and what to do about it seems to still be a matter of debate. \”

    I\’ve read a number of the journal articles on the matter just because I\’m interested enough in what is going on and my inclination is to get as close to the data as I can.

    Because that\’s my thing. Data.

    Everything about data is vital to the scientific process. How we collect it, how we analyze it, how we compare different sets… these things are desperately important to good scientific work. When data gets too big, we use statistical analysis to understand it and models to predict what will happen next.

    Most importantly, for science to work we need people to check our work. The next scientist down the line should be able to work his way to the same conclusion in order to be able to rely on moving toward the next conclusion. Verification is the heart and soul of the scientific process.

    And the process is more important than the result. If you don\’t believe me, go read up on Fermat\’s last theorem. Pierre de Fermat made a conjecture in 1637 that turned out to be true, but mathematicians couldn\’t prove it for over 300 years. That the conjecture was true is important, but how we know it is true is the key part.

    That is why I am so pissed off at the scientists at CRU. If you read their e-mails (a good collection of what they say has been collected by Bishop Hill), they spend a ton of energy making sure other people can\’t do independent verification of their data. They attack people who disagree with them, not because those people have bad data or use poor process, but because the results are not consistent with the message the CRU scientists are trying to propagate.

    Add to that the fact that the CRU e-mails reveal an almost violent disregard for proper scientific peer review in favor of bullying journals into accepting only appropriate papers. And they make no bones about it: Appropriate is defined in relation to the desired result. If the result is different from what they want to hear, they worked tirelessly to politically punish people who found those results.

    And we haven\’t even started talking about the code.

    I have a solution to this, one that I believe is non-partisan and vital to future work:

    • If a paper is going to be referenced in an IPCC report, they need to post their all the data, an explanation of the process and the code for the paper where anyone can look at it and verify it.
    • Any grants that are offered with federal money should require public access to the data, the process and the modeling code. If \”the people\” bought the research, we should be able to look at it, not just at some 10 page summary report.
    • Any paper used for public policy purposes should hold the same requirement.

    In short, this is a call to free the data. We can\’t make decisions in the dark. If these guys have done good science, anyone with an appropriate expertise will be able to verify it.

    Is this unfair to climate scientists? A violation of intellectual property?

    Forgive me if I don\’t give a sh**. These guys have crapped all over the scientific method and made a mockery of objective science. This kind of bad PR will take years, possibly decades, to overcome. If they want to keep their data to themselves, they can get a private firm to support their research and stop using their findings to push public policy.

    Take note: This does not mean that the conclusions the CRU scientists have come to are wrong. They could be 100% right and still be huge assholes who want to hide their data from everyone else. But we have no reason to believe that they are 100% right because we can\’t see the data and we don\’t know their process. Just because you cheer the deaths of your opponents doesn\’t make you wrong. In the future it\’s going to take more to convince me than \”But the scientists SAID SO!\”

    Also, given the blatant and horrific way in which these people have manipulated the peer review process, the \”But the skeptics aren\’t published in peer reviewed journals\” argument is a pretty sh***y line of attack from here on out. Just from reading the e-mails, we can see that:

    1. That isn\’t even remotely true
    2. Manipulation of the peer review process has been a top priority for these scientists, to the point of intentionally ruining careers and lives.

    From here on out, they can have my confidence in their results when I see their data.

  • Why Take Math? So Your Ignorance Isn\’t Broadcast Nationwide on the AP Wire

    This is pretty funny. Or horrifying. Depends on how you want to look at it.

    Several days ago, I noted on Twitter that there were a lot of \”saved\” jobs that weren\’t saved at all but actually cost of living increases. About 24 hours after I noted this, there was an Associated Press article about that very phenomena.

    Coincidence? Almost certainly. But I\’ll flatter myself anyway.

    But the laugh riot comes several paragraphs into the article as they look into why Southwest Georgia Community Action Council was able to save 935 jobs with a cost of living increase for only 508 people. The director of the action council said:

    \”she followed the guidelines the Obama administration provided. She said she multiplied the 508 employees by 1.84 — the percentage pay raise they received — and came up with 935 jobs saved.

    \”I would say it\’s confusing at best,\” she said. \”But we followed the instructions we were given.\”

    \”Confusing at best\”? The multiplication of percentages is \”confusing at best\”? It seems obvious to me she should have multiplied 508 people by the amount the increase (.0184) and gotten 9.3. But she forgot that you have to divide the percentage by 100 before you multiply.

    The fact that she had \”saved\” more jobs than there were people in the organization should have been a tip-off. But this is a pretty common problem with people who don\’t have a very good grasp on mathematics… they don\’t recognize obvious mathematical errors, they just plug in the numbers and go with whatever comes out.

    And this, children, is why you pay attention at school. So you don\’t get in the national news for doing something really stupid and then blame it on the instruction manual.

  • Why Take Math? So Your Ignorance Isn\’t Broadcast Nationwide on the AP Wire

    This is pretty funny. Or horrifying. Depends on how you want to look at it.

    Several days ago, I noted on Twitter that there were a lot of \”saved\” jobs that weren\’t saved at all but actually cost of living increases. About 24 hours after I noted this, there was an Associated Press article about that very phenomena.

    Coincidence? Almost certainly. But I\’ll flatter myself anyway.

    But the laugh riot comes several paragraphs into the article as they look into why Southwest Georgia Community Action Council was able to save 935 jobs with a cost of living increase for only 508 people. The director of the action council said:

    \”she followed the guidelines the Obama administration provided. She said she multiplied the 508 employees by 1.84 — the percentage pay raise they received — and came up with 935 jobs saved.

    \”I would say it\’s confusing at best,\” she said. \”But we followed the instructions we were given.\”

    \”Confusing at best\”? The multiplication of percentages is \”confusing at best\”? It seems obvious to me she should have multiplied 508 people by the amount the increase (.0184) and gotten 9.3. But she forgot that you have to divide the percentage by 100 before you multiply.

    The fact that she had \”saved\” more jobs than there were people in the organization should have been a tip-off. But this is a pretty common problem with people who don\’t have a very good grasp on mathematics… they don\’t recognize obvious mathematical errors, they just plug in the numbers and go with whatever comes out.

    And this, children, is why you pay attention at school. So you don\’t get in the national news for doing something really stupid and then blame it on the instruction manual.

  • Why Take Math? So Your Ignorance Isn\’t Broadcast Nationwide on the AP Wire

    This is pretty funny. Or horrifying. Depends on how you want to look at it.

    Several days ago, I noted on Twitter that there were a lot of \”saved\” jobs that weren\’t saved at all but actually cost of living increases. About 24 hours after I noted this, there was an Associated Press article about that very phenomena.

    Coincidence? Almost certainly. But I\’ll flatter myself anyway.

    But the laugh riot comes several paragraphs into the article as they look into why Southwest Georgia Community Action Council was able to save 935 jobs with a cost of living increase for only 508 people. The director of the action council said:

    \”she followed the guidelines the Obama administration provided. She said she multiplied the 508 employees by 1.84 — the percentage pay raise they received — and came up with 935 jobs saved.

    \”I would say it\’s confusing at best,\” she said. \”But we followed the instructions we were given.\”

    \”Confusing at best\”? The multiplication of percentages is \”confusing at best\”? It seems obvious to me she should have multiplied 508 people by the amount the increase (.0184) and gotten 9.3. But she forgot that you have to divide the percentage by 100 before you multiply.

    The fact that she had \”saved\” more jobs than there were people in the organization should have been a tip-off. But this is a pretty common problem with people who don\’t have a very good grasp on mathematics… they don\’t recognize obvious mathematical errors, they just plug in the numbers and go with whatever comes out.

    And this, children, is why you pay attention at school. So you don\’t get in the national news for doing something really stupid and then blame it on the instruction manual.

  • Why Take Math? So Your Ignorance Isn't Broadcast Nationwide on the AP Wire

    This is pretty funny. Or horrifying. Depends on how you want to look at it.

    Several days ago, I noted on Twitter that there were a lot of \”saved\” jobs that weren\’t saved at all but actually cost of living increases. About 24 hours after I noted this, there was an Associated Press article about that very phenomena.

    Coincidence? Almost certainly. But I\’ll flatter myself anyway.

    But the laugh riot comes several paragraphs into the article as they look into why Southwest Georgia Community Action Council was able to save 935 jobs with a cost of living increase for only 508 people. The director of the action council said:

    \”she followed the guidelines the Obama administration provided. She said she multiplied the 508 employees by 1.84 — the percentage pay raise they received — and came up with 935 jobs saved.

    \”I would say it\’s confusing at best,\” she said. \”But we followed the instructions we were given.\”

    \”Confusing at best\”? The multiplication of percentages is \”confusing at best\”? It seems obvious to me she should have multiplied 508 people by the amount the increase (.0184) and gotten 9.3. But she forgot that you have to divide the percentage by 100 before you multiply.

    The fact that she had \”saved\” more jobs than there were people in the organization should have been a tip-off. But this is a pretty common problem with people who don\’t have a very good grasp on mathematics… they don\’t recognize obvious mathematical errors, they just plug in the numbers and go with whatever comes out.

    And this, children, is why you pay attention at school. So you don\’t get in the national news for doing something really stupid and then blame it on the instruction manual.