The scientific method works in programming too!

Briefly speaking, the scientific method is the idea that you should formulate a hypothesis about the expected results of an experiment before looking at its actual results. Then compare the actual results to the expected results. In computer science – and software engineering, its applied field – the scientific method seems to be widely neglected though. I read a lot of computer science papers that don’t follow this method and it’s practically absent in software programming (notable exception: TDD).

This doesn’t have to be case. The scientific method works very well even in the lowliest places of programming. The first time I remember consciously thinking about this topic was in college when I – then a TA – watched students step through code in their debugger. Students single-stepped over lines of code, observing what happens. That’s not how debugging works best. Rather, before you step over a line of code, form a hypothesis about what’s going to happen, and check the expected result against the actual result once you stepped over the line.

Outside single-stepping through lines of code, the scientific method works pretty much during all parts of software developments. Making a change to speed up a part of the program? Estimate how much faster the program is going to run; maybe 10%. If the actual result is far off, you need to backtrack to understand why your understanding of the program didn’t align with reality. Same with A/B testing of GUIs. Is a round button or a rectangular going to lead to more click-throughs? Don’t just test it and see what happens. First, formulate a hypothesis how many more click-throughs you expect from the button change. If the observed user behavior is far off, you don’t understand your users and need to backtrack.

Further reading:

The scientific method works in programming too!

3 thoughts on “The scientific method works in programming too!

  1. Joel says:

    You should formulate an hypothesis on what “THE SCIENTIFIC METHOD WORKS IN PROGRAMMING!” means.
    Then have people program with the scientific method and check that you got the expected result.
    Otherwise this post is not very scientific 🙂
    I mean, what you write here is that it is possible to code with a scientific method. This is a non-information.
    The interesting question is not (IMHO):
    Is it possible?
    but
    has it advantages?

    You seem to think that an advantage is in debugging faster. Conduct an experience, I’ll be happy to read if it works. I am not so sure.This is dueto the fact that expressing the hypothesis of what is going to happen to make it a real scientific method is tedious. For scientific papers I tend to agree more.

    Now in terms of computer science, I think that these notions:
    http://en.wikipedia.org/wiki/Precondition
    http://en.wikipedia.org/wiki/Postcondition
    are used and match the “scientific method”. But these are used either in specific theory fields or in specific development fields, were the time investment needed to write these conditions is worth it (nuclear power, airplanes, micro processors…)

    Best,

    Joel

  2. gr1m4c3 says:

    Dear programmers,
    Here’s a cool trick (which is directly related to this blog entry) to a.) save yourself from writing a bunch of code which is quickly thrown away or immediately rewritten, b.) not have to write nearly as much documentation, and c.) ensure that the documentation is never outdated. When your manager / project lead gives you a task, require it to be in the form of documentation. This way, the documentation always matches the code, and if it doesn’t it’s a coding error, not a matter of outdated documentation. Managers and projects leads should not fear the extra work, as it’s a way to ensure that they get what you ask for the first time around. It is not a difficult change, but it can have a profound impact.

    Hypothesis: If you implement this workflow, you will have more accurate documentation, tasks will get done faster, and programmers will understand the requirements better, and thus be able to spot issues in the request before spending any time writing code. The end result: the code matches the intent more closely. If the design is being done properly, this will result in less bugs and a cheaper maintenance phase of the product life-cycle. Whether or not your product will be easy to extend will be a function of how well it was designed, which is what this workflow is dependent on. If you implement this workflow and find that this isn’t the case, then you’ve probably uncovered the real problem, which is a crummy design.

    Sebastian,
    Feel free to re-post this in your blog. Attribute it to me, or not, I don’t really care. It’s more important to spread the ideas than to feed my ego.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s