Crunch an inevitable part of software engineering?

Every company I’ve worked at has crunched to varying degrees.

Every company I’ve worked at has crunched to varying degrees.[/quote:b20ab28a03]

All I’m trying to say is that it’s not an inevitable part of software development – other areas of the software industry deliver large projects, under similar pressures, without anywhere as much crunch time.
[/quote:b20ab28a03]

I don’t think many other sectors of the industry have the same ‘feature creep’ and polishing that games need. There’s no such thing as a finished game, it can always be better. I think in other sectors you’ve got a specific set of requirements, you meet those requirements and you’re done. But games can just be endlessly improved the more time you throw at them.

Not that it’s a healthy practice or that it should be accepted, but I don’t think it’s as simple as saying ‘look how they do it’. I do think there needs to be better planning and management in place to keep it at a bare minimum but it will always happen to some extent when it comes to games.

We have law in Ireland that limits the working week to an average 48 hours… Out of interest do your crunches exceed that? Do you get sufficient comp time off afterwards?[/quote:107969744c]

Companies tend to ask if you will sign an opt out of the working time directive.

A few game companies in the UK pay overtime (Bizarre Creations for one) but not many. Free Radical do too, but I think it affects other bonuses…

We have law in Ireland that limits the working week to an average 48 hours… Out of interest do your crunches exceed that? Do you get sufficient comp time off afterwards?[/quote:2066cd0ef0]

Companies tend to ask if you will sign an opt out of the working time directive.

A few game companies in the UK pay overtime (Bizarre Creations for one) but not many. Free Radical do too, but I think it affects other bonuses…[/quote:2066cd0ef0]
Every game company contract i’ve seen has the opt out clause in the uk. Yep some pay overtime (SRS did also). Yep free rad take it outta your bonus, but bonuses are rarely mega big these days (unless your gta). God the money those guys will get next week. :(

I see I might have opened up a can of worms here with my previous comments.. :P

I don’t think many other sectors of the industry have the same ‘feature creep’ and polishing that games need. There’s no such thing as a finished game, it can always be better. I think in other sectors you’ve got a specific set of requirements, you meet those requirements and you’re done. But games can just be endlessly improved the more time you throw at them.[/quote:09c6a81465]

I don’t think that is neccesarily 100% true. I agree completely with the ‘no such thing as a finished game’ statement but I wouldn’t go as far as saying that other fields in software development are completely immune from this problem either.

I’m no systems programmer, I’ve never worked on a web server, a database frontend or a networked / distributed computing system; but I’m pretty sure that the same temptations are there for systems programming that are there for games too. Once you’ve fulfilled your initial requirements there’s always more improvements that can be made, whether it’s beefing up security, performance, fixing outstanding bugs, usability tweaks, or taking preventative measures to increase the robustness of the system.

The only difference I think is that in systems programming there is more discipline from everyone involved and they know where to draw the line in the sand. Also, I doubt system developers are a passionate about their jobs (in general) as games developers are, which makes them much less likely to feature creep.

This all comes down to bad Leads, bad Project Managers, bad Producers and Bad Game Designers.

1. The Game should be reasonably speced out by the Game Designer. If this is not the case how can any schedule be put in place.

2. The Producer should be aware of what it takes to create a game, what is required to create the game according to the game designers spec and being able to adequately cost and account the timing of the project. If in doubt double or triple the timing of aspects to give adequate development time.

3. The lead is responsible for their department under no circumstances should they be giving into the whim of a designer, an artist, a programmer or a tester. There needs to be a strong process in place that does not allow people to make change requests simply because they want them done. If your lead doesn’t have the balls to back his team he shouldn’t be a lead.

Yes I am aware it’s game development but simply accepting every new feature, bug or change request because it’s a "Must" for the game is bulllshit. You have a spec, you have a design document, a content bible, and all these other documents so use them.

Granted if something unforseen arises via playtesting or analysis then make a change request, have all leads abide by it and adjust your timing. If the Producer was anyway good they would have accounted for such issues and there would be timing available for it.

Automated Builds, Unit Testing, Code Analysis, Peer Reviews should all be common practice and not seen as something that simply gets in the way of making games.

4. Project Management, this is where it all falls on it’s ass, while inner structures of companies differ, Project Management across the Games Industry as a whole seems to be in shite. You have people who cannot accurately cost how long it takes to development content or code aspects of the game. How can you manage a project without knowing how long X, Y or Z takes to do within your current team. Not your previous team, not your great team back in company B, your current team. You need to know what your team can do, how long it takes them and their availability / schedule.

Project Management and Producers are in a business relationship, that does not mean the Baddie Publisher gets to call the shots, it means that a good Project Management / Producer will sit down with the Publisher discuss the various documentation, make their pitches and neogotiate what the game will be, how it will be developed and what the schedule. It does not mean the Project Management will promise the Sun the Moon and the Stars and be surprised when the publisher is pissed off because all they got was a B-Rate Game. Nor does it mean the publisher should be let away by reducing the budget by 50%, cutting the schedule by 6 months and still expect a AAA game.

This is business, it’s a professional industry with massive profit margins if done right, but it takes good managers to do this right.

As a result the main problem with the games industry is it’s immaturity and lack of procedure. I have worked on many games where the engines have been hacked together on the whim of artists or designers. Where good practices were not followed and as a result long delays and crunches kicked in. We somehow believe we are special and that the hard lessons learnt by others do not apply to us, that because we make games the software development rules that apply to web development, application development or any other forms of development do not apply.

We are not special, we are not unique, we are making a product just like every other software / technology company out there and unless we can accurately Manage these projects, these budgets, these teams we are pissing in the wind and killing off what innovation and productivity we may have.

At the end of the day it all comes down to the lack of procedures, project managers not following them, producers not knowing them, and the leads not having the balls to stand up to these procedures.

There is a reason why IBM, Microsoft, Ericsson’s, Vodafone, Oracle and all the other large Multinational companies are in business and developing their technologies. That is simply because they have a set of procedures and Managers who can follow those procedures to the letter. Sure they have issues with their technology, they have delays but their business structure and procedure is mature enough to account for these. Something the games industry has yet failed to realise.

If you worked on the latest Ironman or Batman movie and your camera crew failed to turn up. Would you wait for them to show up or bring in more hands to get your shoot that is costing 2 million per day done? Would you allow the script to be changed midway through or towards the end of the shoot? Would you allow an extra 30 minutes of movie to be added because you came in under budget or ontime?

So why would you do it in the games industry? What benefit is there in crunching the hell out of your staff? We don’t get paid for excessive overtime, we don’t get time in lieu, if we are lucky we get 3 weeks holidays after 12 – 18 months of development… That’s a bullshit racket and the industry needs a serious look at how the more mature industries are working and it needs to adapt those solutions.

And for god sake can we get some Leads with a pair of balls and the ability to say NO.

im not going to write a long winded response. All i will say is that everyone knows best practises, so dont bother preaching them. What i will say is in the field it doesnt always work like that, yes it would be great if it did but it doesnt, due to numerous reasons which i wouldnt state but they are obvious if you work in sw long enough..

Clearly not everyone does, if every games company is facing crunches then there is something wrong with internal development and the business management.

Have you ever wondered why crunches come about? You said in your previous posts that it’s as a result of the game being behind schedule or ahead of schedule (and packing in more content).

If you are behind schedule what was the reasoning? Did every member of staff sit on their hands for 6 months, playing WOW or trawling Kotkau when they should be working or was it really bad project management?

If you were ahead of schedule and the features were complete why on earth did your Leads, Producers or otherwise not have the balls to simply say No. Clearly for that milestone or game all the features in the agreed upon design doc was complete. So why add un-needed crunch?

If good practices that everyone knows were enforced and implemented very few bugs would be introduced. Nothing worse than some guy checking in code at 11pm breaking the build and going home only to turn up at 10am next morning.

You have clearly accepted that crunch is part of the industry, but why should it be when so many other industries expect Over-Time, On Call or Time In Lieu?

Surely Peter you can at least see there are some huge issues with doing business this way? Staff Burnout is high, Morale can get very low, errors are made….

I think a lot of it has to do with developer/publisher relationships. Regardless of how well specced out the design is (and yes I concede that it frequently isn’t very well specced out at all) it all comes to nothing if the publisher shifts the goal posts.

Things like including new tech that has become popular, insisiting on a simultaneous release on multiple platforms, changes to story because of some real world event.

But really what it comes down to is the dev staff saying no to unreasonable requests no matter who they come from

Another issue is that games need to be fun. With applications, it needs to work, and maybe be somewhat userfriendly, but most of this can be designed beforehand. If you’re in the middle of games development, and you find the game isn’t fun to play, you’re in trouble, and changes HAVE to be made.

I don’t believe there is a lack of software engineering knowledge, I believe it’s the level of software engineering knowledge being implemented in the games industry.

The previous projects I have worked on the company as a whole was reluctant to make the move to automated builds. I personally went ahead and implemented in my own time and gave it to the studio manager who instantly saw the benefits of QA being able to test the tech / games / tools on a per bug by bug basis.

So instead of the previous mantra where QA would pick up the bugs after a 1 or 2 month milestone they would be catching bugs between checkins of developers.

The point is the studio as a whole were against automated builds because they were not willing to look at the practices used in other industries. They simply thought that because they were Senior developers in the games industry that their old habits carried far more weight than tried and tested practices of other industries.

They were perfectly aware of automated builds, they could have easily done it themselves but were not willing to do so (perhaps it was ego, perhaps they couldn’t be arsed..)

I have seen the same rejection to things like code analysis, unit testing, documentation, procedures etc.

When your lead is accepting every single task as a critical bug and does not have the balls to say : No that’s not a bug that’s new feature

Where does it stop?

I will say that the studio managers were business men and for that reason they could control the publishers and ensure there was no additional features being added late into the project. They had the balls to say no to the guys paying the bills, it was only internally that Producers & Leads fail to say no to each other.

The greatest trick I have every seen for effective management of tasks and bugs was implemented by my systems manager in ercissons. She took the time to read every bug and rejected all the bugs that were not bugs in the software (When in doubt she rejected them anyway). This not only reduced the workload but gave the team freedom to expand the platform. Where she really nailed this process was by introducing a system whereby any change request (everything rejected as not being a bug), needed to be formally submitted as a change request and was forced into a formal peer review.

Now when a developer can just submit a new game feature as a bug they will do so without a seconds thought.

But when they have to write up the documentation and follow an actual process they will think twice about whether or not it’s worth their time to follow through on what could potentially be a whim.

Regarding the feature creep in games, this would come down to having a bad spec and not prototyping the game enough to see if it were fun. If these process were done correctly then all major issues would have been caught before full production began, and any additional features added to a game during development should be easily added in. A friend of mine who worked on Skate said that they took the approach of prototyping an entire proof of concept for the game to determine what would and would not be fun. Over 1/3 of the games schedule was spent on this prototype and he is 100% confident that because the resources were made available at the start of the project did not see any major issues or feature creep.

I am under no illusions that the games industry has some of the most talented developers out there, but what I am trying to simply put across is that there is very little use formal discipline within the games industry. Whether this is simply because developers choose to ignore them, don’t see any benefit in them or as a result of where the Games industry started with those bedroom coders / developers setting the foundation for the industry.

At the end of the day the games industry is not a special club, it’s a business, a very very big business. Every company needs to develop their own business process, their own business procedures and their own HR policies. If the industry insists on plodding along patting each other on the back because they make games and ignore the fact they are in a competitive industry we will see alot more closures.

Surely Peter you can at least see there are some huge issues with doing business this way? Staff Burnout is high, Morale can get very low, errors are made….[/quote:41ea48781d]

the initial topic wasent about best practises, which ppl keep going on about!! it was about reality, what really happens within the industry!! i do see a problem, yes, but when asked about crunch i laid down the cold reality of the games industry.. ivan backed me up as he too works in this industry (the rest of the posts are an outsiders view in no??).

I’m bowing out of further comment because this topic lost the point and its consisting of mostly comment of how stuff should be and not facing the reality of how it is. majority of games PM have no software eng training, and this topic has turned into sw management 101. ;)

as for opting out clause, all the banks in the uk have this policy too. so its not very unusual.. we’d probably have it in ireland only its illegal.

the initial topic wasent about best practises, which ppl keep going on about!! it was about reality, what really happens within the industry!! i do see a problem, yes, but when asked about crunch i laid down the cold reality of the games industry..[/quote:1a364d3bf3]

Good point, but surely the whole point of this forum is to discuss these problems and somehow come to a resolution. Sure, it’s not going to make an immediate difference, but the idea that people should not have a frank and open discussion on the topic is crazy. Otherwise, how would anything get done!

ivan backed me up as he too works in this industry (the rest of the posts are an outsiders view in no??).
[/quote:1a364d3bf3]

No one is disagreeing with you, they are disagreeing with the subject matter. And some of these "outsiders" may soon be "insiders", so it would be preferable to not have these people just accept defeat before they get off the starting line. Viva la revolution! :)

I work in the industry and have my name on several titles : Hellboy (360,PS3), Viva Pinata : Party Animals, StarWars : Force Unleashed. Some other un-announced games and a few XBLA Games / Prototypes. Then of course there is the various mods, games and applications I am working on.

So I certainly wouldn’t consider myself an outsider picking apart the industry because I think I know how to do it better than those in the industry.

From my standpoint the reality of the games industry is that best practices are often shoved aside for the sake of development. This is what I feel leads to crunch. So it is a relevant point of view while discussing the "Reality" of the games industry. We all know crunch is part of the industry perhaps it’s best to try and figure out what creates crunch and what methods or steps can be taken to reduce or address the issue of crunch.

[Edit]

ivan backed me up as he too works in this industry (the rest of the posts are an outsiders view in no??).[/quote:e161fbdc0b]

The more I think of this comment the more it just pisses me off. This is a huge problem I have with the games industry where everyone seems to think just because we make games we are special and people outside the industry don’t understand it.

This is a forum for Gamedevelopers, whether they are in the industry, wishful onlookers, students or simply random visitors. Surely they are allowed to express their opinion and views on Crunch and what can be done to fix it or accept it. Just because you are in the industry and another person within the industry also accepts crunch as being the norm for the industry doesn’t mean others peoples opinions are void because they don’t work in the industry.

If we want to expand the industry, the communities, the innovation we have to be willing and open to accept opinions and views from external sources. Not ignore them just because they are not in the industry.

Again this is a Gamedevelopers forums, so there are a lot of people here who are in the industry and even more who are not in the industry. Everyones point is valid.

Not an insider myself. But Relentless here in Brighton say they never work past 5 and never work on the weekends. They said that it was about working properly all the time. Dont know the truth of it, but they might be implementing some of these things already.

I’m bowing out of further comment because this topic lost the point and its consisting of mostly comment of how stuff should be and not facing the reality of how it is. [/quote:8060f382bc]

Nobody here is questioning the reality of how it is; I think you pretty accurately summed up the way the ‘cookie crumbles’ already and I doubt anyone would here would say otherwise.

What people are trying to discuss (and in keeping with the thread title I might add) is whether crunch is utterly inevitable, and if not then how could it be avoided. I think it’s pretty clear from the discussion to date that there are ways and means to get around it, but again you as (rightly) pointed out the culture within the industry is not yet geared towards those means. Thats just the way it is.

While it might seem like a pointless discussion to some, I think it is a very valuable discussion. The only way the situation will ever change in the industry is if the mindset of the industry changes also. That’s not something that’s going to happen overnight; and that’s assuming it’s possible at all in the first place. But if it does happen it will be because awareness of the issue was raised, and that only happens slowly but surely when people get together and talk about it- like what we are doing here.

Just blindly accepting things as they are is not enough- and that goes for life in general too. The games industry is a highly creative industry and people are chosen to be in it not only because of their excellent technical and team working skills but also because of their creativity and ability to think out of the box. Therefore it is only right that you should question the processes being followed and ask is there a better way.. In fact, you should be expected to do this, because ultimately it is not only to your benefit but to the benefit of the company and industry at large also.

Furthermore, I would like ask those within the industry who may be looking at the ‘outsiders’ comments here with disdain to remember the purpose of this website and why many people are here. Some of the ‘outsiders’ here who might seem irrelevant now will ultimately end up becoming a part of this industry, and consequently play their own part (whether big or small) in guiding its future direction. The industry is not stuck in a vacuum and the people outside of it do have an influence on it; please remember this before dismissing their views completely.

I personally have a lot of hope for the industry and it’s future direction. Considering where it was 10 years I think it has come a long way even since then. There is definitely much more maturity in the way it goes about it’s business now, and I think that trend will continue in the future. Also, not all so called ‘crunch’ periods are that bad in reality; for some companies ‘crunch’ means just an extra hours work every day- which is hardly the panic induced marathon that some people make it out to be. Other companies are now even going as far as banning ‘crunch’ completely:

http://www.gamasutra.com/php-bin/news_index.php?story=16159

So it’s not all bad; the situation is improving on a yearly basis. At the same time everyone involved needs to play their part to foster a culture of development where not only crunch will be minimized, but the need for it in the first place will be removed.

I like their counter (bottom left);

http://www.relentless.co.uk/games/

I wont pretend to know how to eliminate crunch so I wont try and come up with some solutions.

Especially since I’ve had a few pints ;)

Try me 2moro

In my himble opinion the main key to successful games dev is planning. Work smart and you don’t have to work hard. If you find yourself working silly hours in a crunch it’s either becasue there’s been a sudden unexpected development that nobody could have forseen or else someone has failed to do their job correctly. A good project manager or producer is essential. I’ve worked in many games companies over the years and I’ve found that the ones who find themselves in crunch scenarios seem to have them continuously. All down to poor planning/ project management etc. Staff in these companies end up one way. Burned out.

Or humble opinion even. :)

Okay, these posts have been quiet long and I’ve been on holiday and am currently on crunch, I’m finding it catch up sufficiently.

One of the points I’m not sure whether its been mentioned as a reas on for crunch…is the Publisher. While the Developer designs the game, as the end of the day, unless you’re in a really strong position, its the publisher who can scupper best laid plans for planning as they want a new feature, or like a different feature and want more work put into it. At the end of the day, its their money and they can get a lot of decisions passed, some you can circumvent, but in doing so, you need to give way in some cases to keep a good relationship…so best laid plans usually only last for so long.

And another point on crunch…as I said I’m currently on crunch. However we’re no longer doing the painstaking crunch that everyone speaks of, we’re doing managed crunch…2 weeks on, 2 weeks off ( sometimes not even that ). Everyone is out of the building at 9pm too. Its a much better system as youy get the extra hours in AND you don’t kill the team. There is also payment for time spent.
So, as things progress, companies are looking to improve crunch, I don’t think it will ever go away (submission never go to plan do they?) but I think a much more managed way of crunch is coming into a lot of companies. And for those that don’t…well I spent 3 months at one of those companies and got out before my probation ended as I couldn’t see my working life being worth the hassel…