Regular expressions – those mysterious creatures we love to hate – now cower in a corner as you stand victorious! Here is your tale of regular expression mastery. Listen in!
Master regex genie Reuven Lerner shows you how it’s done. Some magic reg-dust and POOF!
Reuven’s Bio:
Reuven teaches Python and data science to companies around the world. He has written two eBooks, “Practice Makes Python” and “Practice Makes Regexp,” which contain 50 exercises to improve your fluency with Python and regular expressions, respectively. He also publishes “Weekly Python Exercise,” a subscription service that sends you a new Python exercise every week.
Listeners to the “Mentoring Developers” podcast can take 10% off any of these products with the “mentoring” coupon code. Just enter the coupon code at checkout!
Reuven also publishes three free weekly newsletters:
- Better developers (http://lerner.co.il/newsletter), with new programming tips every Monday
- Trainer Weekly (http://trainerweekly.com/), with new tips for professional trainers every Monday
- Mandarin Weekly (http://mandarinweekly.com/), for students of Mandarin Chinese — not related to computers, but a lot of fun!
Reuven lives in Modi’in, Israel (halfway between Tel Aviv and Jerusalem) with his wife and three children.
You can contact and read more about Reuven at:
- Main site: http://lerner.co.il/
- Blog: http://blog.lerner.co.il/
- E-mail: reuven@lerner.co.il
- Twitter: @reuvenmlerner
Episode Highlights and Show Notes:
Arsalan: Hi, everyone. Today, my guest is Reuven Lerner. Reuven, how are you?
Reuven: I am doing great, Arsalan. It’s great to be back on the show.
Arsalan: It’s great to have you back. You were on the show in episode 13 and 36. So, if anyone wants to know who Reuven is and you want to learn more about what you have done and all the wonderful things because we went into so much detail, then listen to episodes 13, 14, and 36. Yet, I still want to know a little more. Tell us a little about your background, who you are, and what you’ve been doing.
Reuven: So, my background is a developer. I graduated with a degree in computer science and I started off programming for big companies. For more than 20 years, I’ve been a self-employed freelance developer with my own small company. It’s had its ups and downs in terms of the number of people. Right now it’s just me plus one other programmer who works with me on projects. Really, it’s mostly me doing a lot of, especially nowadays, training. Over the last 5 to 10 years, I’ve transferred from doing some development, some consulting, and some training. So, now it’s about 90% or more training. I go every day to a different company and teach their developers or non-developers how to program, usually in Python, or using some other related technologies.
Reuven: In addition to that, I do a few other things. I produce some newsletters that I put out. I have some books that I’ve published. I have a service that I recently launched. I’m a busy guy.
Arsalan: Clearly, you don’t have a family.
Reuven: I do have a family. I have a wife and three children who occasionally recognize who I am. No, they’re great. I’m actually speaking to you from a hotel, not because I run away from my family, but because I happen to be teaching a bit farther away from home in two places in northern Israel. So, rather than taking the train all the way back here, I’m just staying here for the evening. So, I have two newsletters that I put out every week. One is for people doing training to give them some tips on how to do training, whether it’s the pedagogic aspect to it like how to teach or have to learn, the business aspect, the logistic aspect like do you eat lunch with your students and what things should you bring in your bag, especially when you travel internationally.
Reuven: I also have another weekly newsletter that is called Better Developers, which is, again mostly for Python people, but not only. It talks about how you can improve your skills as a software engineer. I dive deep into all things having to do with Python. So I’m currently on week s 8, 9, or 10. I’m talking about threading and processing in Python and how that works.
Reuven: Separately, I’ve got my e-books on Python and Regular Expressions and what I like to call Weekly Python Exercise, which is a subscription service for pay that people get every Tuesday. People get a Python exercise emailed to them and every Friday they get the answer. Someone just recently said to me that that was just like subscribing to a crossword puzzle and I thought that was a great analogy. I’m not sure if there are any professional crossword puzzle solvers out there, but the analogy holds up completely and my idea and a lot of my work is that you need to be fluent in a programming language as much as you’re fluent in any human language. The only way that you get fluent in any language is just by using it and practicing making mistakes and figuring out the where those mistakes are and improving them. It’s a lifelong process. It never ends. Yet, there does come some point when you cross that threshold from looking everything up to looking things up only occasionally. That is what I try to do to help push people across.
Arsalan: I think that’s the right approach. In any skill that you want to learn and in anything that you want to learn in life, you have to keep learning. There is no end to learning. You do reach a point where you are more confident and you are situated. Basically, you are on your own two feet and you know where to go instead of looking around randomly. If I dropped you onto a new planet, like Mars, in order for you to survive the environment of Mars you would know where to go. Yet, if you had signposts or mile markers and there was a destination and you knew where the good restaurants were, then I think you would know which path to chart. I think this is how learning happens, which is something that we all need to remind ourselves.
Reuven: So, you mentioned that you’ve been learning guitar and you see this is a lifelong thing. That’s something that you can continue keep going on and learning. I am convinced that anyone who wants to teach or mentor should learn some things because it puts them into a perspective of being a learner. It means that they are experiencing that confusion and that wonder, and putting the pieces together, and it’s important to have that perspective if you’re going to teach. Otherwise, it becomes too easy. One of the rules that I’ve come up with for myself is any time that I think that something is obvious; it means that I need to go into depth and explain. If it’s obvious to me, there is no way on God’s green earth that it’s going to be obvious to my students. It’s only become obvious to me, as it were, because when I learn things people say that it’s just sort of clear. No, it’s not. Then, I have to probe and asked lots of questions.
Arsalan: Learning is hard. So, anyone who is listening to this podcast right now is thinking that all these people say to just go practice or go on GitHub and pick up a project and contribute. What’s the big deal? If you do that, then everything will be easy. But, everything is not easy for everybody and we empathize with that, especially people who are in the mentoring business or in the teaching business. It doesn’t mean that you’re dumb. It doesn’t mean that you’re not smart enough or good enough to do this. Maybe, you learn a different way. That’s okay. We should all be able to bring our own differences into the mix and find the path to success. I believe that most people who get into development can be really successful. You don’t have to be the Eddie Van Halen of programming, right?
Reuven: Right.
Arsalan: You could be the guy who holds the song together. You could be somebody who gets the job done. You could be somebody who has a rewarding career and everyone’s happy with. We don’t all need to be superstars or rock stars, as they say. That’s a term that I don’t like when they use it for programmers. I like it when they use it for actual rock stars, but I don’t like it when they use it for programmers because what they mean is somebody who is going to come in and solve all the problems and everyone else has to work around them and that creates a problem with that person being elevated above everyone else. I don’t think that’s a good team environment, personally. Yet, I see a lot of companies looking for rock stars. If you have half a dozen rock stars working for you, good luck.
Reuven: I guess I never thought about the rock star skill set coming with the rock star ego, but they do come hand-in-hand. I guess your point was that someone else has to manage them and that might not be so easy.
Arsalan: I think that we overemphasize and we put too much importance on skill set. The raw skill set or ‘brain power’ is when you calculate things and come up with solutions or to solve complex problems. That is a valuable skill, but it is far less valuable than a cohesive team that works well together because most of the problems that we solve are not really rocket science. They are not.
Reuven: That is absolutely right. I’m reading this book now about the history of the iPhone and how it was put together. One of the points of the book is that everyone thinks that Steve Jobs sort of woke up one day and said “I got a make an iPhone,” and he waved his magic wand and a few people worked on it. Hundreds of thousands of people who each contributed for 50 years to the technologies associated with it. Part of the point of the book is to show that it wasn’t just a team effort, but a massive team effort. It’s unfair to say that it branched out from the efforts of one person or even a few people. Those people needed to work together and unlike Steve Jobs, who obviously did not work well with other people, unless they were on the receiving end of his yelling. A lot of these people just needed to work together and grind and grind and grind as a good team to get good stuff out. They did, which is very impressive.
Arsalan: Most of the times when you have a good product, a good, successful product, it comes together not because of the brilliance of a single mind. It’s never like that. You might have the beginning of an idea. Some brilliant scientist thinks of an idea. But the product that comes out is always a collaboration, always.
Arsalan: Even if a brilliant person could come up with a prototype, you still need to productize it and then you need to support it. So, if you are in the business of making money through software, then you really don’t need a lot of rock stars. You need people who can get the job done. You can have people with different skill levels, someone who is really good at solving complex problems and someone who is good at thinking about the overall design and thinking about the user, and someone who is good dealing with the customers and understanding their problems and internalizing them and making sure that the products that are being built are also being supported and are in line with what the customers want. So everyone has a role.
Arsalan: When a new person comes into a job, I’ve seen and I’ve worked with some programmers, some new kids, some college kids, even some high school kids that are really smart. They are smarter than me. It is obvious to me that at their age I couldn’t do the things that they did. They come with this attitude of “I know everything.” I see that all the time. They’re not quite ‘rock stars’ if you look at their resume, but they have that attitude. Sometimes that’s good because they’re very stubborn. They’re going to go in and stick to it. If you give them a problem and they don’t know anything about it, they’re going to pretend that they know it and they’re going to keep banging their head until they learn.
Arsalan: I’ve been in situations where I was mentoring a group. I used to run a mentorship program for a company in Chicago and I would have a batch of people who just came out of college and I would train them for a month and then they would start working on real projects. There was this one kid who was clearly brilliant and he wouldn’t accept anything that I was saying. So, I would ask a question just to see how much everyone knows and he would give me the wrong answer but insist that he was right. You don’t know what you’re talking about. I did know how to deal with that, but after a few years of working, he was a standout programmer. He got really good and he learned the things that he didn’t know, but from a technical point, he was very good. There is room for people like that. I don’t want to dismiss those people to say that they can’t work in software, but there are also people who constantly struggle and I would talk to them and they would tell me how much pressure and stress they are always under. That happens. Those people eventually left the company or they had a really bad taste in their mouth and some of them would’ve switched careers. Some of them went into management. Some of them did something else. Maybe that’s okay. Maybe that’s the right thing to do, but they don’t have to be. Most of them could have made it. If we just created an environment where different types of people thrived.
Arsalan: That’s what I believe in and on this podcast that’s what we talk about. Today, I really want to talk to you about Regular Expressions because you love talking about it and you are the Regular Expressions expert and you wrote a book on it. You literally wrote a book and gave me a copy of it before the interview and I was going through it. I went through it and was expecting a 30 or 40-page little reference book with just a couple of points, but this was like learning a language. It was deep. It was very detailed and the examples were amazing. If I could do the exercises in this book, I would feel very confident that I was one of the best among my peers. I’ve always been interested in Regular Expressions, but I always keep forgetting how the grouping goes, how the syntax works, or backtracking. I always keep forgetting, but it doesn’t bother me because I know I’m just going to look it up and figure it out. If I have a good editor where I can try a few things and see the answer, I’m going to figure it out. So it doesn’t bother me.
Arsalan: So, you did write a whole book on it. I wonder this information that you’re giving out in the book and maybe you’re also teaching. You’re not also teaching a course on it, are you?
Reuven: I do have a Regular Expressions course that I give at companies. It’s not super often but like once every two or three months. It’s actually a two-day course, believe it or not. People scoff at it. I was teaching at this one company and this woman said that all you need to know is .* and you know everything. I sort of slapped my forehead and said: “of course.” Regular Expressions are these things where you look at a manual page once in a while and/or you search on Stack Overflow for the answers that you need to get and then you move on. You can live your life that way.
Arsalan: The question is “why?” Why write a book and why should I read a whole book? Here’s the thing, I can go through the book and learn the syntax and learn the different conditions and do the exercises and feel pretty good. I know Regular Expressions and I’m pretty good at it. Then, a year later, I don’t remember any of it. Is that really possible or do you think there’s a way to avoid that?
Reuven: First of all, it’s possible. The world is a big place there are some people who will read the book and not just read it but go through it and do all the exercises and then not use things for a year and then they won’t remember very much. But, the book isn’t aimed at people who are not going to use it for a year. The book is aimed at people, and I think most programmers with some quantity of text, even the people who I work with who deal with log files. They search for things and log files and all kinds of different fields. They search for things from inputs they get from users on web applications. In those sorts of situations, Regular Expressions can be invaluable. I am super not handy, but I do like to cook and we just got a new set of kitchen knives. So I’ve been seeing how the different knives work. Now that I have these tools I can use them. I can use this knife and this way. In a way, it opens my eyes to solving problems that I’ve never solved before. My belief is that once people learn Regular Expressions, they will have this tool in their toolbox that they will use and want to use every week, every other week, every month. If they do that, they might not remember 100% of the syntax, but they are going to remember a ton of the syntax. Certainly, they will remember much more after having done the exercises rather than having read the manual page and promptly forgetting that. I’ll definitely forget that, I’m sure.
Arsalan: So, I’m glad that you gave that answer because that’s what I’m looking for. That was kind of tongue-in-cheek when I said that. I’m trying to get into the mind of my listener because this is what they’re thinking. This is one of those things where you have to use it in certain situations, but you try to avoid them. A lot of people think like that, but that’s one more tool in your tool belt. If you have that tool, a lot of your problems will go away. A lot of your parsing problems will go away in every programming language where you could create an elegant solution with Regular Expressions. In one of my jobs, I was parsing these really large text files. They were random formats, not really random but different text files that needed to be parsed in a certain way and normalized and you needed to create a word dictionary to figure out where every word was to be able to filter out the common words and to keep a list of important words and where they were. That was the project. How do you do that without Regular Expressions? It’s impossible.
Reuven: It’s theoretically possible, but come on; no one wants to do it. When you see the Regular Expressions solution you think to yourself “I can’t believe I thought about doing it the other way. This is so much shorter.”
Arsalan: Right. I ended up with a Regular Expression that was six lines long, but that Regular Expression parsed all of the different formats of all those legal transcripts. So, when you go in and you are in a lawsuit and someone is taking notes for a legal deposition, it produces a transcript. So, they are all different. Yet, that sixth line Regular Expression was able to parse all of them and create a single, normalized result across all of them. It wasn’t just six lines of text that just curled around, it was six substrings. So, what I would do was I would think about how I would break the problem down. I would break it down into six different parts and each part was a substring which represented a Regular Expression. With them all together, you can create the final thing. So, you name it. That’s one of the problems with Regular Expressions because they are terse. That also means that they are hard to decipher. If you’re looking at someone else’s Regular Expression, and they have not named it, if it’s just an inline Regular Expression, the chances are that the new person will not know the intention.
Thanks for Listening!
Do you have some feedback or some advice for us or our audience? Please give us a review on iTunes, Spotify, Google Podcasts, or Stitcher and share your thoughts.
If you found this episode useful, please go ahead and share it with your friends and family. You can also listen directly and give your feedback on the website.
You can subscribe to Mentoring Developers via iTunes, Stitcher Radio, Spotify, or Google Podcasts.
Podcast: Play in new window | Download
