Marek Blotny's Blog

In one of the previous post I was writing about estimates and commitments. I felt that this is an important topic because life shows that very often deadlines are beyond our control. I don’t know if it was only me, but I was living in a world where deadlines were always a matter of negotiations between a client and a supplier. Apparently this is not true for web development industry. This surprising (at least for me) truth was uncovered for me very recently when talking with the COO at Cognifide – Stuart Dean.

But if you think about it for a while, it actually makes sense. You have to realize that our clients are typically marketers. And for them, new website is most often only a part of a bigger venture. It can be a launch of a new product, new marketing campaign or some important for given industry conference. So from our client’s point of view all subprojects should end in more or less the same time. If one subproject has a delay then entire venture has a delay. In consequence new product or new campaign will be launched a few weeks later. And a few weeks delay can be converted into money not earned because of it.

Those are the market conditions we are living in. That’s why quite often we don’t have much room to negotiate deadlines. And that’s why the way to win and sign new contract is to come up with a realistic plan to meet those deadlines. Then simply we have to deliver them on time. I know – it’s easy to say, no so easy to do. But also that’s all what it takes to keep clients happy.

One thing which I see happening quite often on number of different projects is that developers have very limited access to the production servers. Actually in some cases they don’t have access at all. That causes number of problems, especially when there are integration points with other systems.

Of course as long as application is up and running lack of access is not a problem. But when something is wrong, developers are supposed to support the application and fix the ongoing problems. Usually in scenarios like this, when system on production is not fully functional, there is a lot of pressure to nail down the problem and fix it quickly. Developers without access are literally blind hence emails like ‘can we get remote desktop to production servers?’ between them, management and server admins are normal. If in the end developers will get a remote desktop access they can consider themself being lucky. Typically server admins and security policies are an impassable wall.

So what can development team do to put themselves in a better position?

Here are a few ideas:

1. There should be a diagnostics tool for each integration point. Especially if an application relay on 3rd party systems. Developers should have a way to check if 3rd party system is responding and if it’s responding in reasonable time. Another useful feature is to have ability to invoke calls on 3rd party system with some test data and verify if results are correct.
2. Each call to external systems should be logged and developers should have an option to generate a report with all the calls. It’s a great help when there is a need to investigate which system doesn’t work as expected. As long as developers can show evidence that for instance the applications asked 3rd party system to send x emails and only subset of them got delivered then it is clear where the problem is.
3. There should be a way to gather data regarding server’s performance. With time application most likely will be more and more popular which means increased load. Also amount of data stored within application will grow. It’s essential to monitor if servers are dealing with current level of traffic and data.
4. All unexpected errors should be logged and easily accessible. It again means that there should be an easy way to generate report with all unexpected errors. (like 500 error pages) I have seen a few times that information about unexpected errors were logged but they were buried somewhere deep in 1GB log files among other completely irrelevant information. In such cases developers can say that this information is logged but can they easily find it? Can they easily get all log files? Can they easily access all the servers?

Having access to all those data, developers can perform investigation and in reasonable time identify a few potential culprits. Moreover, on top of that layer they can build automatic monitoring system which can do most of the job for them and in case of unexpected errors for instance send emails to relevant people.

But to have all of that in place developers and also management have to realize that having diagnostics tools in place is not a strange requirement; it’s their duty as a professionals to foresee the problems and in future be able to tell what is going on with their application in any environment.

There are three simple ways to check if a team is really self-organizing:

1. There are always daily scrums, it doesn’t matter if scrummaster is present or on sick leave, self-organizing teams understand that daily scrums are for them, not for scrummaster
2. Team members voluntarily take tasks and work on them until they are finished. They don’t wait for scrummaster to come and assign something to them. Team knows what are the strong sides of all members, they know who should work on what to be efficient
3. Team members help each other to finish all stories. If each member is working on their own and don’t really care if others are blocked by something, this is not exactly the team work which self-organizing teams should have

Do you know any other quick tests showing if team is self-organizing or not?

Every day we are facing some deadlines. There might be a need to finish a report, answer important emails, estimate something, fix some bugs or test next release. Those are all urgent and usually important things. We have to deal with them as a matter of priority.

This is obvious part; big question which I have is what next? What will you do next when all urgent things are done, the pressure is gone and you still have a bit of time?

Basically you have two options. First one is easy; you can switch to things which are not so challenging to relax and rest a bit. Problem however is that those not-so-challenging tasks are most likely almost completely not important. I can tell you without further ado that this is not the way to go.

Alternatively, you can focus on things which can have significant impact on your effectiveness and work quality. First step on that way will be to stop for a while and ask a question: what can I do to make my work more effective? Try to come up with at least one change that will have a significant impact.

If you start thinking in such categories you may be able to identify things like:

• tools which are not as good and useful as they should be – you can replace them with better ones,
• scarcities in your development process – you can work to eliminate them, visible problem is a killable problem,
• manual tasks which can be automated

You can also spend that time on personal development, and for instance:

• master tools which you use every day – learn some advanced features or new keyboard shortcuts,
• do some research,
• learn new design pattern, framework or programming language

Trick is to minimize time which you spend on urgent things. Urgent problems usually have some underlying cause. When you are close to a deadline there is no time for fixing the core issue. Typically it takes too much time so the only viable option is to find a workaround. However without fixing the core issue you can be sure that sooner or later that problem will come back and again it will be something urgent. That’s why it makes sense to spend your time on identifying and fixing core problems. That will reduce number of things which can go wrong and make you more effective. As a consequence it will also allow you to spend even more time on important things like personal development.

Effective people know how to distinguish tasks which are important and really matter, they know how to “put first things first”. They are proactively looking for opportunities to to get better. Are you one of them? What are your ways to have more time for important things?

If you are a member of development team, sooner or later someone will ask you to estimate something. It can be something small like a bug or task but it can be also something bigger like entire project. Regardless what that is, try to figure out what kind of answer you are expected to give. I don’t mean here that you should be trying to give the “right” estimate. I’m thinking about something different — usually there are two possible types of answers:

1. We need 12 to 18 weeks for this project.
2. We need 4 developers, 1 interface developer and 2 testers to have this project done before …. (some fixed date)

In the first case … you were really asked to give unbiased estimates, deadlines don’t exist for you. Project plan and deadlines will be planned based on estimates and negotiations with a client.

Second type is more interesting. It indicates that estimates are not the final objective, there is something more. Target (deadline) for this project is already set up front. So the real question here is “Can you come up with a plan to meet the deadline?” To answer it you need estimates, to provide a plan it’s good to know how big the project is. However keep in mind that it is only an intermediate step. It’s better to be very careful with this type of questions because even though you were asked for estimates, in fact it was a question about a commitment. By saying that you need given number of people to get something done, you are in fact giving your word that it can be done. If you don’t believe me then let me rephrase the second answer without changing the meaning into something like this “Sure, we can do it before … (some fixed date), we just need a team of 4 devs, 1 interface dev and 2 testers”

That’s why I think whenever someone asks you for estimates, it’s always good to check if the question is really about estimates or maybe it’s about a plan to meet the target.

What’s the relation between estimates and commitment?

Commitment is a promise to finish agreed functionality in given time. It doesn’t have to be the same as estimate. When you truly need to win a contract with a high-profile client then commitment can be more aggressive than estimates. You may assume that this contract will be a first step to a longer relationship and accept higher risk. On the other hand when you know about number of integration points and expect problems with them, you may want to go with more conservative commitment.

Conventionally role of developers in a project was fairly narrow. Project Manager was assigning tasks to developers; they were working on it and when done whole process was repeated for another task. Usually developers where left alone with their task, not bothered by the outer world.

Now, consider how different expectations are for developers in agile teams. Agile methodologies are tailored for cross-functional and self-organizing teams. That means a set of completely new responsibilities:

• Requirements are usually not very detailed so it’s up to a developer to refine them by asking questions and talking with business people.
• There is no PM who is assigning tasks, team should self-organize and take tasks which are available on their own.
• There is no PM who is estimating everything and committing on behalf of team. Developers are involved in estimation process and are asked to make a commitment.
• In cross-functional teams there are no specializations in some narrow aspects of software development or one specific technology. Everyone can pick up any task.
• Developers are not only responsible for coding … they are responsible for getting story/task done.
• Team should be able to reflect and adapt to the situation they are in. Finding the most effective techniques/processes is part of being agile.

That list can be much longer but I think you already see my point here. Developers in agile teams have to take much more responsibility on their shoulders. They are not only focused on coding, focus has been shifted towards communication, interactions with clients and other team members. Developers have to be now effective communicators. Understanding and using business terms is a must. Remaining with technical jargon is simply not an option.

Transition to agile developer can be shocking. Not everyone will be able to cope with this challenge. Some developers simply may lack interpersonal skills necessary for it. Others may not be keen to change their focus and attitude. So what’s the reward for the efforts?

I think agile methodologies give developers a chance to influence the development process, change it into more effective and this way improve quality of their own work. Their ideas matter, their actions can make a difference. Agile make developers empowered to change things and self-organize. I think many people will find such environment stimulating and motivating.

Generally speaking there are known problems and unknown problems. Some people say that unknown problems can be also divided into known-unknown problems (the one which we know that exist but we don’t know anything about them yet) and unknown-unknown problems (we don’t know anything about them and we don’t even expect that they exist)

Good thing about known problems is that you can solve them, workaround them or at least minimize impact which they have. In short – visible problem is a killable problem. Unknown problems on the other hand are hard to predict (especially unknown-unknown category is nasty) and you can be sure of one thing … Murphy’s law will work … they will show up and kick your ass in the worst possible moment.

So whatever you do, you want to expose as many problems as quickly as you can. Agile methodologies place emphasis on interactions and frequent communication with a client. Check how that can help you with uncovering problems …

1. Daily Scrums – prevent people from going dark. Everyday all team members are reporting what they are working on. It’s easily noticeable if someone has stuck with something. Daily scrums make hiding and struggling secretly with a problem very hard.
2. Short iterations and close cooperation with client – if for some reason you are building something different than client wants it’s better to know about it after a first few iterations than after entire development phase. At the beginning of a project you still have time to do something about it. But when you are after development phase, somewhere in the middle of UAT then your options are significantly limited.
3. Retrospective meetings – team is not as efficient as it can be? There are some bottlenecks? There are tools which are slowing team down? Or maybe you are experiencing communication problems? Retrospective meetings is a chance for entire team to meet and for half an hour stop thinking about deadlines, it’s time to focus on what went well and what went not so well and think what team members can do to make following iterations better.
4. Burndown chart and story board – visual representation of progress makes it easier to identify certain problems. For instance if you see that all of user stories are in ‘QA’ state (check attached picture) then you instantly known that something is wrong – there are not enough testers in the team. And finally burndowns … here you can see if your progress is good enough to meet the deadlines. From the very first iteration you can verify if your estimates where more or less correct. If not you still have a time to manage the situation.

I see daily stand-up meetings (“daily scrums”) as a one of the most beneficial practices in SCRUM’s repertory. Theory says that during the daily scrum each team member should answer three well-known questions:

• What did you do yesterday?
• What will you do today?
• Are there any impediments in your way?

Daily scrum is a great example of small things which make great difference. Three simple answers from each team member ensure that entire team is on the same page. It’s clear who is working on what, what will be achieved by the end of day, what problems we have etc.

What I find strange is that theory doesn’t leave much room for any discussion during the daily scrums. Discussions should take place during the follow-up meetings, immediately after the daily scrum. In general I agree with this rule. Bigger issues which don’t require entire team can be talked through later.

But I think it’s much more beneficial for a team if all members are allowed to jump in anytime during the daily scrum to add anything they find helpful. I have seen many scenarios where such impulsive hints work very well. Although scrummaster has a tough task with keeping the discussion under control. On the other hand the worst thing which can happen is when team members are in ‘on hold’ state until it’s their time to speak. That kills spontaneous knowledge sharing and potentially turns daily scrum into a progress report with limited profit for a team.

It’s vital however to keep the meeting under 15 minutes and stay high level. There is not enough time for the team to go into each and every detail. That should happen during the follow-up meetings. Scrummaster responsibility is to ensure that discussions are on reasonable level.

So if you ask me if enforcing so rigid structure of daily scrum make sense. I will answer … definitely not! On one hand you have effective and useful daily scrums which encourage teamwork and on the other quite rigid structure and doing everything by the book. I’m far from saying that traditional approach doesn’t work, but it can be improved, so for me it’s a simple choice.

What’s your view on this? Do you think it makes more sense to follow traditional rigid structure of daily scrums?