Testing is not a waste of time, I don't know that your code works

I've been in the land of formal mathematics where all the equations are good and correct.

My venture into computing however is not so pure as I would have liked. I tried to be virtuous in this land by bring the gospel of proof and correctness, but I have failed. I have given into temptation of the wild out laws. I have abandoned static types in favor of dynamic types. I have abandoned super-planning waterfall methodologies to agile methodologies. I have abandoned my safe IDE for the raw ninja powers of nano/gedit. I have abandon formal techniques and am working on developing testing based techniques (relevant to my domains and projects).

Here are my thoughts on testing.

Interaction between Proof and Testing

If you can prove it, then you can test it. You know where it should work and where it shouldn't work. Thinking about testing first will give you hypothesis on what to prove. Once you get it working, do you need to prove it? Usually no; but you should have unit tests that enable you or your team to reasonably do the engineering needed to take it to production.

For instance, if you are writing PHP, then the academic portion of the brain can bang out the parser and interpreter in a couple of hours. You can bang out the proof in a day. The problem is that real engineering has to come in and optimize it for production. This means going down into the bowels of code and optimizing loops, changing data structures, finding code equivalences. Unit tests help engineering efforts undergo regression testing as the entire code base is optimized. As the code grows, the feasibility of a proof diminishes. Ideally thou, the proofs enabled the tests to be constructed such that "If all these tests pass, then this proof reasonable asserts the code is good"

Stupid Shit

When I dropped static typing, we hired a QA guy whose job was to use the product every day. So far, we don't having typing issues. We have buttons that don't work due to typos or overlapping divs. Sometimes, your code can be proved correct if it just works. Usually humans need to test this.

I think I can solve some of the issues with selenium IDE, but the time for me to solve it using that methodology combined with the opportunity cost of doing something else means I should I just hire some scrubs off the street to check buttons and check for other faults as well.

You and I are not in control

Let me introduce you to google maps and how they did versioning. I picked a URL for google maps that represents their bleeding edge version. One day, half of our clients didn't work any more. WTF? They updated to a new version and broke our shit. Fortunately, they also provided an archive of old versions and a little hacking into production server and it was working again. I didn't know this at the time, but fortunately Google knew enough about it and started archiving.

We live in a time of interdependence. I trust you to develop service XYZ, and you trust me to develop ABC. Either you or myself can fuck up, and those bridges need to be tested and measured. For instance, I have a server to proxy geo coding requests. I do this so I can enable workarounds when google fucks up (and they do about 0.01% of time which is why I have a table of 18 addresses that google couldn't geo code correctly).

When google started promoting version 3 of their api, I found out that a quick URL change didn't work for me. My tests however told me where it didn't work and I could treat my tests as a simple todo list and grind through the version change. As long as the tests measured every use case that I needed, then I was fine.

You need to develop tests for things that you don't control. You will have many black boxes and you need to test how you use them and your assumptions.

In summary

No matter how smart you are, I guarantee that you will eventually need testing because testing will find you and the business will depend on it. Refusing to test is just ego because a test will fail and that will compromise the integrity of your ego's self image of you as a perfect and heroic being. Once you accept this, allow your ego to rest and stay hidden during real engineering.