you're reading...

What are degrees of freedom?

I remember getting frustrated as an undergraduate trying to find straight answer to this question.

The standard text book answer is something like this:

"In statistics, the number of degrees of freedom is the number of values in the final calculation of a statistic that are free to vary"

That’s from Wikipedia but it’s fairly typical.

I could just about make sense of this for something like a chi-squared statistic but why, could someone explain to me, are the degrees of freedom for linear regression n-k-1?

I realise this doesn’t keep many people awake but it did me so I was pleased to find the following quote:

"The person who is unfamiliar with N-dimensional geometry or who knows the contributions to modern sampling theory 
only from secondhand sources such as textbooks, this concept often seems almost mystical, with no practical meaning."
Walker, 1940

Many years later I’m nasty enough to use it as an interview question. In a kinder frame of mind I thought I’d post my slightly XKCD inspired notes for explaining, as simply as I can, the concept in terms of N-dimensional geometry.

I hope you can read my writing and apologies to mathematicians if the language is a bit loose!







About Simon Raper

I am an RSS accredited statistician with over 15 years’ experience working in data mining and analytics and many more in coding and software development. My specialities include machine learning, time series forecasting, Bayesian modelling, market simulation and data visualisation. I am the founder of Coppelia an analytics startup that uses agile methods to bring machine learning and other cutting edge statistical techniques to businesses that are looking to extract value from their data. My current interests are in scalable machine learning (Mahout, spark, Hadoop), interactive visualisatons (D3 and similar) and applying the methods of agile software development to analytics. I have worked for Channel 4, Mindshare, News International, Credit Suisse and AOL. I am co-author with Mark Bulling of Drunks and Lampposts - a blog on computational statistics, machine learning, data visualisation, R, python and cloud computing. It has had over 310 K visits and appeared in the online editions of The New York Times and The New Yorker. I am a regular speaker at conferences and events.


3 thoughts on “What are degrees of freedom?

  1. A mystical response…
    A degree of freedom could be the remainder left over from a degree of “unfreedom”,..
    In N dimensionally geometric terms the convex angle at the axis of an X / Y graph
    In human ones, a last resort.

    Posted by mark lewcock | July 3, 2013, 10:03 pm
  2. I still don’t get it!

    Posted by jasmine | July 12, 2013, 6:44 am

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog Stats

  • 336,388 hits

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 528 other followers

%d bloggers like this: