Author Archives: kevin

About kevin

I write the posts

How to find a Pearson’s correlation and ordinary least squares in Python

Posted on
I'm going to try to blog more technical stuff, as well as simple recipes, here more. I'm working out the best way to present it - whether to bundle everything together or separate out the technical posts. Today we're going to use Python to find a simple correlation, and then fit a straight line to the curve. First you want to install the SciPy and NumPy libraries - they have a lot of cool functions for Python. On Mac, if you have MacPorts installed, this is trivial: 
$ sudo port install py27-numpy py27-scipy
Then you find a Pearson's correlation as follows: 
# The dependent variable
>>> x = [1, -2, 2, 3, 1]

# The independent variable
>>> y = [7.5, -3.5, 14.5, 19, 6.6]

#Find a correlation
>>> from scipy.stats.stats import pearsonr

#First value is the r-value, 2nd is the p-value
>>> pearsonr(x,y)
(0.98139984935586166, 0.0030366388199721478)

# To find the best-fit line, use the numpy directory
>>> from numpy.linalg import lstsq

# Put the x variable in the correct format.
>>> A = numpy.vstack([x, numpy.ones(len(x))]).T

>>> A
array([[ 1.,  1.],
       [-2.,  1.],
       [ 2.,  1.],
       [ 3.,  1.],
       [ 1.,  1.]])

>>> lstsq(A, y)
(array([ 4.5 ,  4.32]), array([ 10.848]), 2, array([ 4.53897844,  1.84327826]))
# 4.5 is the slope, 4.32 is the y-intercept. 10.848 is the sum of the residuals. 
# 2 is the rank. The last array are the singular values.
For more details, including how to plot the correlation, see the NumPy documentation here.

Liked what you read? I am available for hire.

Kevin goes to Coachella

Posted on

I went to the Coachella music festival last week. It was a wild weekend and I will surely go again in the future.

  • I was surprised at how little thought bands put into their onstage appearance and mannerisms. It’s natural for artists to want to focus on their music, and expect others to judge them based on their music, but the fact is the show is more than just the music, especially at a festival where you are going to have more casual fans than a regular concert. The three artists who did the best job of putting on a show were Empire of the Sun, Kanye and Sleigh Bells.
  • Kanye’s show had the clear marks of an egotistical maniac: forty dancers dressed in white, several interludes where he proclaimed his own greatness, and a fifty-foot-tall Greek tragic relief behind him. He let the music play and the dancers dance for about three minutes before appearing. He performed the first song while circling thirty feet above the crowd on a riser. I thought this was brilliant cause it let people take camera phone pictures of Kanye alone against a dark background.
  • Phones were really unreliable for most of the event, so we had to make really concrete plans like “meet at this spot at exactly 6pm” and so on. I don’t understand how people did things before cellphones. People must have just lost each other a lot.
  • At times it seemed like the only thing everyone was doing was just walking around. For a while I flipped out that everyone walking the other way had better information. That theory did not hold weight.
  • As my college career is coming to a close I’m getting really nervous about whether I’ll continue meeting people and making friends. So I’m making more of an effort to talk to strangers. This generally went well. However it’s not a good sign that at a place like Coachella, with 150,000 attractive people in my own demographic, with something really obvious to talk about (the music), and virtually no downside to having a conversation go badly, I only maybe talked to 5 strangers each day.
  • Cell phones aside I was amazed at how smoothly everything ran. I was particularly impressed by these trash cans:

    They are a great example of form following function. The round hole was for round recyclables (bottles, cans). The square hole was for compost items (plates, forks, paper). The other hole was for trash. The one critique I’d have is that the holes were not very large so a drunk or lazy person might just throw their recyclable in the trash. This error may be better than throwing trash in the recycling, however.

  • My favorite acts were Sleigh Bells, Kanye, the Strokes, Cut Copy, Two Door Cinema Club, and Wiz Khalifa. If you have a choice between two acts, choose the louder, more upbeat, or more electronic sounding one.

Liked what you read? I am available for hire.

links for 2011-04-27

Posted on

Liked what you read? I am available for hire.