Programming

Over the Christmas holidays, I started to dive more into Python. I have been trying to learn more about NumPy, SciPy, and PyLab. So far, I love these libraries. However, there doesn’t seem to be a good way to create movies/animations.

The best way to create a movie using Python is to generate plots using PyLab or PyPlot and save them as pictures. You can then use another program to stitch them together (ie. mencoder or ffmpeg). This process takes substantial amount of time and seems a little bit hacky at best. Please leave comments if you know of a better way.

It took some time to figure out how to get everything configured on MacOS X. My first problem was trying to get a copy of mencoder or ffmpeg. We need one of these programs in order to stitch all the pictures together. I found an interesting post which allowed me to easily install ffmpeg onto my mac. You can download my code and a copy of ffmpeg here.

Once you downloaded the zip file, you will need to

1. Copy ffmpeg to /usr/local/bin
2. Execute the following commands from your terminal:
   sudo chown root:wheel /usr/local/bin/ffmpeg
   sudo chmod 755 /usr/local/bin/ffmpeg

Once you have installed ffmpeg, we can look at how to create a movie. I created a function called CreateMovie which will easily create a movie

def CreateMovie(plotter, numberOfFrames, fps=10):
	import os, sys
	import matplotlib.pyplot as plt
 
	for i in range(numberOfFrames):
		plotter(i)
		fname = '_tmp%05d.png'%i
 
		plt.savefig(fname)
		plt.clf()
 
	os.system("rm movie.mp4")
	os.system("ffmpeg -r "+str(fps)+" -b 1800 -i _tmp%05d.png movie.mp4")
	os.system("rm _tmp*.png")

This function creates a movie called movie.mp4 in the current working directory which has a default frame rate of 10 frames per second. Make sure that you don’t have any files called movie.mp4 and _tmp*.png in the current working directory because they will be deleted.

The function takes the following parameters:

In order to illustrate the use of this function, check out this code for a moving gaussian.

import CreateMovie as movie
import matplotlib.pyplot as plt
import numpy as np
 
x = np.linspace(-.5,4,500)
 
# Plots a given frame
def plotFunction( frame ):
	plt.plot(x, np.exp( -10*(x - frame/10.0)**2) )
	plt.axis((-.5,4,0,1.1))
 
movie.CreateMovie(plotFunction, 50)

This code will generate the following movie.

People who are really good at their jobs obsess about their tools- artists obsess over their paint, carpenters obsess over their power tools, cooks obsess over their knives.

PHP is like buying a chef’s knife at Wal-Mart. You can create a 5 star dish with it, it cuts perfectly fine, but it doesn’t inspire the same passion that a Shun knife does.

-Comment on Hacker News

I am finally learning Python and I am absolutely loving it. I never bothered learning the language before because I thought it was to similar to Perl. However, I am finding that it is not the case. The code seems so much cleaner and contains much better mathematical libraries. Right now, I am playing around with SymPy which is a computer algebra system for Python. If it wasn’t for Rod, I never would have known about it.

Anyways, I wanted to print out my first Python script. Please remember that I’m still learning and it is probably complete garbage. I ended up coding the first algorithm that I could find in my Numerical Analysis textbook which was the Bisection Method.

from sympy import *
 
(a, b, i) = (1.0, 2.0, 1)
x = symbols('x')
 
# f(x) = x^3 + 4^2 - 10
f = x**3 + 4*x**2 - 10
FA = f.subs(x,a).evalf()
 
while i<=1000:
	p = a + (b-a)/2
	FP = f.subs(x,p).evalf()
 
	if FP == 0 or (b-a)/2 < .0001:
		print "%f %f %d" % (p, FP, i) 
		break
	i = i+1
 
	if FA*FP >0:
		a = p
		FA = FP
	else:
		b = p

I could have easily solved the problem by just calling the solve function. However, I wouldn’t have learned much about Python by doing that.

from sympy import *
 
x = symbols('x')
print [elem.evalf() for elem in solve(x**3 + 4*x**2 - 10, x)]

Yesterday I read a great article by Joel Spolesky on “How to be a Program Manager”. One thing Joel pointed out is that Program Managers should be on a equal playing field with programmers. It just doesn’t work when your Program Manager is also your boss.

To make sure that the debate happens respectfully and on a rational basis of facts, it’s absolutely critical that the program managers and developers be peers. If developers report to the program manager, at some point during the debate the program manager is going to get sick of the whole thing and just say, “OK, enough talking, now we do it my way.” When they’re peers, this can never happen. It’s a little bit like courts of law: we don’t allow a lawyer for one side to be the judge, and we work on the theory that the truth is most likely to be uncovered through a process of debate between equals. The debate can only be a fair one if neither side has an unfair advantage.

From personal experience, I think this is really a good idea. Unfortunately, a lot of programmers (including myself) don’t really have the luxury of not having their Program Manager as a boss. In spite of this fact, if you have a good boss, they will appreciate it when their ideas are challenged. This got me thinking about the nature of software quality and how great software is so often born out of conflict. Since I have a background in physics, I can’t help to see the resemblance between good science and good software design principals.

In the book “The Logic of Scientific Discovery”, Karl Popper introduces the idea of falsification. In science, a theorem is considered falsifiable when there exists an empirical test that would disprove a theorem. For example, Newton’s Law of Gravitation would be a good example of a falsifiable theorem. For centuries, this theory held up to empirical tests (at least in comparison to its rivals). It wasn’t until General Relativity was developed in 1916 that this theory got superseded. General Relativity was able to predict things like Mercury’s orbit and the cosmic expansion of the universe which Newton’s Law of Gravitation failed to predict. By using the idea of falsification, we can judge the merit of a theorem by how many falsifiable statements exist. If a theorem has a lot of falsifiable statements, it is more likely to be disproven and hopefully corrected. Theorems with high degrees of falsifiability can undergo Darwinian Evolution where only the fittest theorems survive critical review.

I speculate that we can apply a similar methodology to software. If we expose software to as much critique as possible, the software project will evolve in such a way that only the fittest elements will survive. Although we can’t use the idea of falsification directly in software design, we can use another idea which I will call “Correctabilty”. For example, let us look at an arbitrary software “defect”. This defect can be a physical bug, a usability issue, or an incorrect design. The defect is considered “correctable” if there exists an opportunity for this defect to be fixed. If this defect was a physical bug, a peer review would be an opportunity for this bug to be corrected. If we expand this thought further, we can say that there is a positive correlation between software quality and the degree of correctability. Potential “defects” would be exposed to more opportunities to be corrected and should therefore have less defects which would increase the quality of the software. For example, this would imply that large open-source software should have low defect rates. If a body of code is exposed to countless programmers, it would be unlikely that a physical bug would exist. Likewise, this same project could suffer from poor usability testing and would have a low degree of correctability in this area.

Using the principal of correctability, we can judge the merit of different software engineering principals. When given two different decisions, we can ask our self if it would increase the correctability of the software project. If the answer is yes and the decision is feasible, you should do it. This would mean software engineering practices like peer review, unit testing, and usability testing would be obvious ways to increase quality. Although the benefits of these practices are obvious, the principal of correctability gives us a single metric to judge the quality of software.