Episode 60: Allan Kelly on Agile Development and what’s wrong with it

Meet our next guest, Allan Kelly. Allan began his career as a software developer but quickly moved into consulting. He has authored several books on software development and shares his love of tech and how to be more efficient in software development with others in the industry. Listen in to episode 60 and hear his story.

Say hello to Allan on Twitter!

Allan’s Bio:

Allan makes digital development teams more effective and improves delivery with continuous agile approaches to reduce delay and risk while increasing value delivered. He helps teams and smaller companies including start-ups and scale-ups with advice, coaching, and training. Managers, product, and technical staff are all involved in his improvements.

Allan is the originator of Retrospective Dialogue Sheets and Value Poker. He is also the author of four books, including “Xanpan -team-centric Agile Software Development” and “Business Patterns for Software Developers” and more recently “Continuous Digital: An agile alternative to project”. On Twitter he is @allankellynet.

Episode Highlights and Show Notes:

Arsalan: Hi everyone. Welcome to another episode of mentoring developers. Today my guest is Allan Kelly. Allan, how are you?

Allan: I’m very well. Thank you for asking me, Arsalan.

Arsalan: I’m so happy to have you here. You are not in the US. You are in London. Is that correct?

Allan: Yes. I’m sitting here in sunny London. No rain today.

Arsalan: That is incredible. I am so sorry that I have to hold you in your home office right now instead of out there having fun on those rare sunny days in London. So the question is, who Allan is and you told me that you are an Agile coach and you coach people and teams about how to do better. You mentor people. You have so much that you have done in your life, and we really want to go into that. There is one story about your first proper programming job that you got. That was five years after you graduated, and it was supposed to be this amazing new opportunity where you do things and it didn’t go according to plan. What happened?

Allan: I had already been earning money for programming before college. When I went to college I learned how to do it properly. You know that thing when you ask users what they want and you write it all down, and you design it, and you build it, and everything’s cool? I learned all that in college and then I graduated. When I got to the real world, nobody does it like that. It’s all over the place. I do these jobs because you have to earn money, but I spent five years feeling guilty doing projects after projects. I was thinking that we don’t do these things the way we’re supposed to. I was thinking that it was all wrong because we don’t do it how I was taught in college.

Allan: I started working this job for the British railway system, and they were doing it properly. Everything was written down, and they have an ISO approval on all the rest of it. I was so happy to get this job. It was a nightmare. The documentation was as high as a five-year-old kid. I spent half my week writing documents. We got audited and we were coached on how to answer the auditor’s questions. We had these architects who came from a Cobalt background and we were writing in C++. They had no idea about the technology we were using. So, the code and the architecture were two completely different things.

Allan: One day at lunchtime, I wandered into a bookshop that was close to the office. I found this book by a guy called Jim McCarthy. He’s still around and he’s brilliant. He opened my eyes and I suddenly realized that the way I was taught in school was a bit wrong. It never works out like that. It can’t work out like that. Jim laid out how his team at Microsoft does things differently. I suddenly no longer felt alone or the odd one out. That was in the late 90s. That was the start of my journey to the thing we now call Agile. So, I got to do it properly and it just didn’t work.

Arsalan: I have a lot of listeners who are just starting out and who may not know about all these processes were talking about. There is this idea that exists in academia and a lot of other circles where you talk about with the best way is to build software reliably that will be maintainable and where the process that you follow is repeatable for high-quality software, just like you can do with other pieces of engineering. You can say do this and you will result in having a very high-quality piece of software with fewer bugs, fewer defects, and it will work as expected and everything will be great. Then we have this idealized notion of how to build software. We go into the real world and we see that has flaws. Tell me a little about the issues you faced.

Allan: We were doing software about trains and how trains move around the network. One problem we had was with a source control system, but when you check something and there’s a load of paperwork. So, you had to fill in these forms before you could check something in and you had to get a line manager to check in your code. It slowed everything down. Once your code was checked in, the form goes to the testing department who builds your code and they test it. So, one day we do this and it gets sent back to me with the book report that said that my train was doing this and it shouldn’t be doing that. I asked if they were sure. I went and talked to the testers.

Allan: We realized that what it said in the requirements department of what the train should do and what the trains actually do in real life are two different things. There’s this one station where the trains go in and come out in an odd way. According to the manual, it never happens. So I mentioned that we know that trains do this. So I was told to speak to one of the customs people. But I’m also told that I must find out what the trains do in real life, but not let on that the document says one thing when we think something else, because that’s a contract issue, and they have to escalate it.

Allan: This is a project where we are working six days a week. On the seventh day, you carry a pager back in the pager days because the testers are working seven days a week. So, were trying to do as much as we can and as fast as we can. Yet, we have all these paper forms flying around and all these conversations that don’t exist because you’re not allowed to talk honestly to your customers. There are all these barriers because we’re doing it properly. It was utterly crazy. The way you can write something down, and the way things happen in real life will never meet. The more you write it down, the more complicated it gets. The more you write it down, the lesser the people will remember if they do read it.

Arsalan: These things happen because as we discover that people don’t just turn up one day and they are enlightened software developers. They have a history. They have some baggage. In your case, there were some programmers who were perhaps coached and trained in a completely different coding paradigm. They are experienced, but they are not necessarily well-versed in this methodology and programming languages. Also, people have their quirks and software development is not an exact science, unfortunately. I believe it’s more like Ninja, Kung Fu stuff.

Allan: Yes. It’s a bit of everything, isn’t it? I believe software development is engineering. I also believe engineering requires a bit of inventiveness and a bit of art. I know there are people who argue about whether or not it’s engineering. I say were not good enough to be engineers. We had to get one haven’t quite mastered it. The thing is, most people who talk about software ring engineering don’t know much about engineering. There are lots of different types of engineering. I’m a 3^rd generation engineer. My father was an engineer. My grandfather was an engineer. They really had oil under their fingernails. They were ship engineers. At times, they built ships. They spent most of their lives making ships work.

Allan: As a kid of seven or eight, I’d been down to ships’ engine rooms. I saw ship engines over hulls. That informed me. I know that when you’re engineering things, that’s how the engine should work, but it doesn’t always work like that when there’s a bit of dirty oil in there. Or, the ship has to sail but the guys haven’t finished the maintenance. The ship has to sail…there’s a commercial imperative there. The idea that many people have about engineering and why they think software is a form of engineering is because they aren’t really informed about actual engineering. Engineering is the art of the possible. Engineering is making things work within constraints. That is absolutely what we do in software and it does require some innovation and some imagination, blue sky thinking, call it what you will. There are a lot of different things in there. I think that’s all part of the engineering mix.

 Arsalan: Right. So, it’s not clearly defined and it’s somewhere in the middle. Let’s talk about you, Allan. We want to understand who you are and where you’re coming from. Tell me a little about yourself. Let’s let our listeners understand who you are.

Allan: Who I am? I am Allan Kelly. I live here in London. I used to be a programmer. I used to put my fingers on the keyboard and write code. I still think of myself as that kind of person, although, no one has paid me to code for many years. I still have my pet projects at home, but when I work with teams and developers I see in them many of the frustrations that I had as a programmer. I reached a point a few years after I turned 30 and realized that I could make the code fly. I could make the code do what I wanted it to do. The code wasn’t a challenge for me anymore. The challenge was the way we are asked do stuff and the way we are set on teams.

Allan: Once upon a time a guy in his bedroom could write a program and he could make a lot of money. As a teenager, I was that guy. I didn’t make a lot of money. I made a little bit of money. I never had a Saturday job. I never delivered papers or anything like that. I never worked in a shop. I used to write programs in my bedroom and I used to them to TV, magazines, BBC, and people. That was how I had money as a kid.

Allan: Code is still a problem for a lot of people. What makes it so much more difficult is the way they are expected to perform, the structures that are put in, the way they are managed, the way their teams are sorted out, and the way the needs and requirements reach them. So, when I work with teams and managers, I see the problems they are having and I know those problems. I have lived that life. I want to help them with that problem. Before we can fix the problem with their code, we have to rethink how they’re going about structuring the code. It’s the structures, the organization, the culture and everything around it. When you’re setting up a code, what are you thinking about? Can you think clearly about your code? Or, are you worried about something else? It’s all about the environment that you’re in.

Allan: I still think of myself as a software engineer, but I don’t write the code directly. I influence how the code is written by helping teams and organizations do a better job. So, that’s what I’d say about myself if I’d had a few drinks at a party.

Arsalan: What was your first encounter with programming? How did you even know that was the thing to do?

Allan: I grew up in Liverpool in the 80’s. It was when home computers were just coming in. One of our neighbors bought a Tandy TSR80 from Radio Shack. I’d watch him and his dad typing in programs. I wanted some of that. So, I persuaded my mom and dad to buy me a machine. It was a ZED X81. That was what we called them here. It was a ZED80 machine with 1K of RAM, which, after a few months we got a PAC with 16K, not megabytes, K.

Arsalan: That’s kilobytes, people.

Allan: That’s kilobytes, yes.

Arsalan: And people are wondering what a kilobyte is. I haven’t heard that in a long time.

Allan: That’s like a thousandth of an MB, not a thousandth of a GB. So with this machine, I had found my thing. I was just hitting my teenage years. The machine and I used to commune. I moved up from this one K machine to a 2K machine with a 6502 processor. I spent most of my money for it. I used a little of the money and bought a book on machine code. So, I used to write 6502 assembly stuff. When I got a bit more money I could afford a Pascal compiler. As a self-taught programmer, I had go-to’s all over the place, but there were no go-to’s in Pascal. So I had to relearn how to program in a proper fashion the way you’re supposed to. That has still stuck with me.

Allan: I’ll tell you a funny story. When I was 16, like most 16-year-olds here in England, you do your first proper school exams where they get marked and recorded forevermore. I was one of those kids who did computer studies back then. We had to do a programming project as part of our assignment. One of my teachers had it in as one of the examiners. He told me a year or two later that I scored 98% and the programming assessment. Do you know what I lost 2% for? I didn’t use a go-to.

Allan: The exam board had this list of things that you had to do to demonstrate that you could program. One of the things that you had to demonstrate was that you could use the go to. Had I known that, I could’ve put a go to in there somewhere.

Arsalan: That’s a problem with computer science education because the teachers themselves are not equipped or trained. We follow these checklists of things. It happens now as well. You have to do this because it’s in the curriculum even though the rest of the world has realized that it is bad practice to do it.

Allan: Yes, so this is a really key point. It was one of my first big moments when you realized that those experts aren’t necessarily right. It’s like where I’ve mentioned before that, on a project where we were doing it properly. Doing it properly was hell. What is happening is with our education system and the people who are coming out of our education system of certain models. But because our technology is racing ahead, our technology is already invalidating some of those models. So if you go back to the traditional model that we used to develop software, we call it the waterfall because you have to get out the requirements and do the analysis and do the design and on and on. That makes perfect sense if it’s 1970. That model comes from 1970. We’ve spent the 70s and 80s perfecting that model.

Allan: Let me remind you of something. I’ve read up on this and most of your readers could read up on this as well as they want. In 1970, we had teletypes to communicate with our computers. If you had a screen, it was a green screen. We programmed our computers in Cobalt on an IBM 360 OS mainframe. The IBM state-of-the-art mainframe had 4 GB of RAM and 10 MIPS of processing power. My eight-year-old has a Raspberry Pie and it has 4500 MIPS of processing power and 1 GB of RAM. It is 2017.

Allan: That traditional software process that we’ve been taught makes sense if you’re programming IBM mainframes in 1970. In 1970, CPU cycles were really expensive. Paper and desk time were cheap. In 2017, when you’re programming your Lenox or Android in Ruby or Closure or one of these other modern languages, CPU cycles are now so amazingly cheap. That machine in your pocket has spent more cycles than Neil Armstrong going to the moon. Now that CPU cycles are so cheap, paper and dust time are expensive. It’s far cheaper and we can try out new ideas and get a quicker feedback loop. But, many of the things that are still being taught stem from an earlier age wall we’re racing ahead. The trick is to spot what those earlier lessons we keep and which we move on from.

Arsalan: Okay. That’s very wise because we are constantly going to always be learning and evaluating our successes and failures. Over time, the software industry tends to get better. We tend to understand how this works better. I think you will agree that context is also very important. What works in a very rigid scientific environment or an environment where the requirements don’t change, your requirements as a software developer per or your methodology may be very different from something else where you don’t understand the problem that you are iterating.

Arsalan: We’ve talked about Agile. You’ve referred to it a few times and mentioned that we didn’t have this term Agile before. You’d also don’t necessarily like this term because it doesn’t encompass everything. So, tell us a little about what you mean by all this.

Allan: The only thing that you can really do wrong. If you claim to be Agile, is to do things the same way that you did them three months ago. You should constantly be experimenting, innovating, and trying new stuff. Some of it is going to work and some of it is going to fail. I say that in and Agile context, but really that applies to all software development. That’s because software is always changing. By definition, when we create software, we are creating something new and innovative. If it already exists, you’re going to reuse it, take it and download it from source Forge or somewhere. If you have some code that already does it, you aren’t going to write it again. Instead, you’re going to do control C and control V. You’re going to copy it.

Allan: When we’re writing code, we are always doing something new. We are always innovative. All software development is about learning. We are learning about our tools, our languages, and our options. That’s the most obvious thing. We are also learning about the problems we are solving. Whether you are creating medical equipment or train software or iPhones, you’re learning about the problem you’re trying to solve because we don’t really understand the problem that we’re trying to solve. We think we do. But the truth is that as we are creating the solution we view the problem in a different way. The next time you’re having trouble getting email on your phone, that’s a new problem. How many people 10 or 15 years ago knew that we would get email on a phone? How many people 10 years ago knew that they needed to send 140-character message to the world?

Allan: We are constantly doing new stuff. As we are doing this new stuff, we are thinking of new ways of approaching it. We redefine the problems we are approaching and addressing. We create new problems to solve. So, the second thing that we’re learning is about the thing that we’re creating. The third way that we are learning is about the processes and actions that we are going to use to go about doing this. To some degree, we are learning about learning. For me, all of our technology and software is a multi-layered learning process and we are constantly layering.

Allan: I think that the traditional model, the thing that we call waterfall, tried to freeze learning and tell us that we’ve learned all about the problem. We have written it down. We have now studied the architecture and we have written it all down. We will now give it to a simple programmer who will code something up and when he is done, that will pass the test. We try to put in boxes and freeze it, but what we see is that those boxes feedback to one another. Over time, as we’ve gotten faster, our cycles are getting faster and faster. Our chain cycles are getting faster and faster. So those boxes are interfering with each other at a faster rate. The big thing about Agile is that Agile suggest that we have a model that incorporates learning. It’s the kind of model where we do something and see what happens, we think about it, we reflect on it, and maybe that changes what happens.

Important Links

• AllanKelly.net

Thanks for Listening!

Do you have some feedback or some advice for us or our audience? Please give us a review on iTunes, Spotify, Google Podcasts, or Stitcher and share your thoughts.

If you found this episode useful, please go ahead and share it with your friends and family. You can also listen directly and give your feedback on the website.

You can subscribe to Mentoring Developers via iTunes, Stitcher Radio, Spotify, or Google Podcasts. 

Podcast: Play in new window | Download