Test-driven development

[Rick]

Does anyone follow this method? Reading up on it (saw someone post about it on gamedev in an article) and thought it was interesting but it seemed like it would add way more dev time, at least on the front end.

[Pixel Perfect]

So be honest Rick I think most developers inherently follow this methodology in as much that it is rare for any reasonably accomplished programmer to develop code without a good idea of the functional requirements they are trying to achieve and to not test these at appropriate steps along the development to check their code is conforming to the functional specification. In most cases once the developer has finished with the code module a Quality Assurance team would take over and independently check the conformance of the software too.

 

Sure it may not be as rigid and formalized but it's essentially the same thing.

[Brent Taylor]

@Pixel Perfect

Not necessarily. There's a big difference between simply compiling and running your code to make sure it works and TDD. TDD relies heavily on unit tests. Small tests that check the input, output and result of every function, object and method in your code are behaving as intended. It's time consuming, but you produce far more stable software.

 

Now you'll find this isn't often done in game dev houses for a number of reasons. It however IS very common in web dev fields and any number of other development fields. I'd say the Ruby on Rails community in general would be one of the larger and best examples. It's integrated directly into the Rails solution.

 

@Rick

I follow it sporadically. It depends entirely on the work I'm doing. I do a lot of Rails work these days, so I've been doing quite a bit of it as of late. Not so much when I'm doing gamedev work, though I should seriously consider it.

[Pixel Perfect]

Not necessarily. There's a big difference between simply compiling and running your code to make sure it works and TDD. TDD relies heavily on unit tests. Small tests that check the input, output and result of every function, object and method in your code are behaving as intended. It's time consuming, but you produce far more stable software.

 

Whilst I accept that testing at a function/object level represents in many ways the ideal it's very costly in time and in a competitive world a balance often needs to be struck.

 

I guess it all boils down to how complex the functionality of the software is and hence your ability to fully test it. As its gets more complex it requires testing at increasingly lower levels to ensure correct operational behaviour. But even testing at each function or object level is still effectively testing to a functional specification because how else do you test anything?

 

You can argue that if a piece of software passes the tests against its high level functional specification then it's fit for purpose as it meets its design criteria. Granted an internal function may have a bug that a lack of function level testing failed to identify but if that bug does not effect the overall functional operation of the software then that is generally not of operational concern.

 

I'm not arguing against the fact that methodologies like this might well catch and identify potential software bugs at an earlier stage, they undoubtedly would but you are also paying a high cost of testing most of your good code. It does smack of overkill especially if the functionality is fairly straight forward. I guess I'm saying you apply the level of testing that is required by the complexity of the software being produced unless cost is of little concern. If it’s not easy to test all functional outcomes at a modular level then it’s probably better to test at this level. If it's mission critical software then it would go without saying.

[Brent Taylor]

First, I'd just like to point out, you'll notice I didn't say I agree with TDD, I only specified what I believed to be a misconception on your part. I was defining what TDD is, versus your explanation.

 

----

Now that said...

Put this in the context of most applications are either web enabled or cloud based to begin with. Where security is of the highest concern. TDD can and often will close off a number of attack vectors. The problem isn't just the public facing API's and views, it's anything down the call stack that happens to see that data that is at risk. TDD certainly lowers (but does not eliminate) most attack vectors. That's a big deal.

[Rick]

But even testing at each function or object level is still effectively testing to a functional specification because how else do you test anything?

 

My understanding of this, which I've never formally done this but only read about it, is that you are to create test cases that will fail your code AND you are to do these test cases BEFORE you even write the functional code in question. This seems very odd to me to even try and think in those terms, so was just wondering what people who have used this method think about it.

[Brent Taylor]

My understanding of this, which I've never formally done this but only read about it, is that you are to create test cases that will fail your code AND you are to do these test cases BEFORE you even write the functional code in question. This seems very odd to me to even try and think in those terms, so was just wondering what people who have used this method think about it.

 

It's a good mindset to have, even if you don't actually write unit tests for everything. As you become more experienced as a programmer you tend to code by the seat of your pants less and less. That time is taken up with proper planning. When you're doing it right, unit testing takes little work and time but produces far more stable code.

[Rick]

I've been programming professionally for 10 years and I've never done proper unit testing. Believe it or not both jobs I've had (they weren't technical companies (retail & trading)) didn't do any formal unit testing. I think the fields programmers are in might have a big role in ones experience with this.

 

As you become more experienced as a programmer you tend to code by the seat of your pants less and less. That time is taken up with proper planning. When you're doing it right, unit testing takes little work and time but produces far more stable code.

 

I guess I'm doomed then, although I seem to make a pretty good living by coding "off the seat of my pants" and not doing proper unit testing (this is how the team I'm on works). What industries are you guys working in to give such luxury? The only thing my bosses care about is getting the programs done as fast as possible and it "basically" does what the verbal spec the business person gave us. Traders want their programs in hours not days/weeks/months.

[Brent Taylor]

I've been programming professionally for 10 years and I've never done proper unit testing. Believe it or not both jobs I've had (they weren't technical companies (retail & trading)) didn't do any formal unit testing. I think the fields programmers are in might have a big role in ones experience with this.

 

I guess I'm doomed then, although I seem to make a pretty good living by coding "off the seat of my pants" and not doing proper unit testing (this is how the team I'm on works). What industries are you guys working in to give such luxury? The only thing my bosses care about is getting the programs done as fast as possible and it "basically" does what the verbal spec the business person gave us. Traders want their programs in hours not days/weeks/months.

 

Unit testing isn't done very often in most dev houses (web dev tends to be the notable exception). You'll see it a lot in Java dev houses and of course with Ruby on Rails devs. I've seen it occasionally elsewhere, but it's fairly sporadic.

 

In my experience there are really three types of programmers:

1. The most common are those who are fresh out of college and their acedemic learning stops there. They tend to work for VB or VB equivalent Java shops. They may have some grasp of the basics, but they tend to do a lot of copy and pasting of code found on google. You don't want to work with these people if you can avoid it. Unfortunately most of the non tech biz hire these almost exclusively, usually out of ignorance or the cheaper cost.
   
2. Your general programmers are a little better. General programmers tend to stick to one area of expertise and rarely learn anything outside of their usual problem domain. They do know the basics and maybe a bit more, but new paradigms or frameworks aren't much of a concern as what they already know and use works just fine. General programmers will do the occasional personal project but it's rare it's outside their usual problem domain. When it is they tend not to learn the new concepts or paradigms that go along with it and produce cobbled together projects as a result.
   
3. And you have my favorite group. The fabled "Rock Star" programmer. They are constantly working on personal projects, always learning new frameworks, concepts and paradigms. These guys actually know the difference between OOP, procedural and functional languages. Usually they actually know a functional language and their code is better for it. These guys don't have a primary problem domain, they are simply problem solvers at heart. They love a challenge and they love teaching even more. They hate working with the first group and love working with the second group provided they are willing to learn and experiment.
    
   To be clear, "Rock Star" programmers are not know it alls. They have a lot of experience, but they never look down on you for not knowing something. If they do, they aren't a Rock Star programmer. They're just a jerk and a nuisance to a good team.
   

You'll run into all three and some of them just rub you the wrong way.

 

As for me, I've done a lot of things. I do mostly freelancing at this point in any number of areas. I taught both programming and game development at the university level for two years. I worked IT for a school district for three years (experience with both Windows Domain Controllers and Linux server/client rollouts). I've been reverse engineering programs, file formats and protocols for as long as I can remember. I've built VPN solutions in perl, inventory applications for store chains, software for automating different types of reversing and analysis, quake 2 mods and UT2k4 mods. I've a lot of experience in developing game engines and of course games themselves. Heck, I've even done a little OS dev. Oddly enough I originally went to school for art. I've got a pretty decent amount of experience doing 3D modeling, texturing and animation. My projects folder is littered with experiments and personal projects and I've got a pretty decent portfolio as a result. At the moment I'm working on two different startups (Quest Companions, check us out on Google!), one being a web service, the other being a game dev house.

 

I'm 25 (26 in May). You're making me feel old.

[diedir]

Wow Brent what a work story line behind you !

 

I was reading you and thinking with all that background this guy is 60's old, but what no is 25 (near 26) !

All my respects guy for your work experience, you seem so convinced and full of certainty, hope the road still remain so straight for you.

 

Having not a so far background --15 years (despite my age) i am coming from what you call first "google" group, but since

i think with experiences and time, i crossed some of group membranes.

 

I made VBA procedures for my work, started in computer with a quizz game for fun and learning 15 years ago.

not debugging much as : is it working ? ok it's good=> run

The only great reward of my work now is knowing that what i did 12 years ago is still working and useful at this time.

Now i am focusing on VBS for my company but still happy to see where i am and from where i came.

 

My point is, don't try to put anyone in a cell of a chart of a group, people can grow up by themselves,

as not anybody got the chance to make long studies, got great diplomas (some just need money to get)

sure they will take a lot more time than you to achieve something.

But i can't imagine a person doomed in a position that he don't overtake a day, or he is not interested in that domain.

 

So whatever work you done and proud of it,

don't forget that only the time will tell you, you was not so wrong (don't ever think you are right)

 

sorry if i misunderstood your words (not native english), i never had a 25 years old teacher

[Brent Taylor]

Thanks for the kind words.

 

My point is, don't try to put anyone in a cell of a chart of a group, people can grow up by themselves,

 

Oh don't get me wrong, just because you fit in one group on my chart doesn't mean you won't fit in another the next day. The only difference between the three groups I listed is the willingness to learn and how far you want to go. There are only so many hours in the day and we need to prioritize what is actually import to us. Priorities what they are, not everyone has the time to continue broadening their knowledge set in a given subject. There's nothing wrong with that. I only get grumpy when someone actively refuses to learn something.

 

----

 

Just one last thing I want to clarify. When I say "academic" learning, I do not necessarily mean schooling. In fact I'd say it's rather rare to actually learn how to program in college. At most, people learn syntax and some of the basics. I consider "academic" learning to simply be active studying. Whether you get your information from school, books, online forums or google....really doesn't matter.

[Pixel Perfect]

I find it is fascinating getting insights into other people's experiences and working practices so I'll illustrate mine.

 

Ours is maybe not a typical software development house in as much that we have a team of 8 developers which have been together for over 15 years and a couple of mature products; so be it they are constantly evolving.

 

In my company 'Product Specialists' produce change requests and requirement (functional) specifications which are passed to the development team to implement. There is a review stage that allows for interaction between the development manager\developers and the Product Specialists in order to where ever possible ensure that the proposed requirements are unambiguous and achievable but effectively all additional design and modification of existing products is driven by these.

 

The Quality department will have already written the test specifications for the proposed additions/modifications based on the requirements spec by the time the programmer gets to start the job and any code is produced.

 

Depending on the complexity of the task the programmer may be asked to produce a design spec with time scales at this stage for submission which is then reviewed by the development manager and possibly other programmers before being rubber stamped.

 

It is each individual programmer's responsibility at that point to then code the software and test it to a sufficient level to ensure conformance to the design/functional specification which is left in our organisation to the discretion of the individual programmer.

 

Once the programmer has completed the task it's released to the QA department for testing to their specifications before being either passed or failed.

 

Maybe we are a lot more formalised than many software houses but I am used to a high level of professionalism in the programmer's ability to not only design but test their own code to an acceptable level.

 

Sure we have bugs, we have over 1.2 million lines of code in our main product and it's almost inevitable, but it's managed to an acceptable level.

 

I just find that a lot of these 'methodologies' are cloaked in an air of mysticism and actually just boil down to good common sense application of functional testing in the strictest sense of ... if you design anything, test it to a sufficient level to reasonably satisfy yourself that it's fit for purpose and this involves the definition of functionality at many different levels and the testing of that. This is especially true of component software designed to be reusable where it certainly pays to test that thoroughly at the point you create it so it can be reused with confidence. Whilst this is not strictly formalised in our company, but it is something that our programmers do as a matter of course.

 

Having your software repeatedly failed by the QA department is incentive enough for increasing your level of testing whilst having your customers report failures and bugs reflects upon your organisations ability to control the definition and compliance of software and acts as feedback for corrective and preventative action to try to prevent this in the future.

 

There are no perfect systems or simply 'follow this and all will be well' solutions as, per most things in life, reality is a little more complicated than that. But could we benefit from reviewing our own and looking at other systems ... I'm sure we could. It's been an interesting topic and its certainly something I'll consider looking at in more detail.