ASF 031: Paul Andrew interview

Introduction

Audio version

Don’t you have time to read? You can listen to this as a podcast! Wherever you are, whatever you use. Just use the player directly from this site (above), find it on Spreaker, Apple Podcasts, Spotify (new!) or simply download MP3. Enjoy!

Transcript

KN: Today is a special guest with me, very well experienced data engineer and architect and also my company mate now, which is…

PA: Paul Andrew. Hello everybody.

KN: Paul, could you introduce yourself from the name but also where do you live?

PA: My name is Paul Andrew. I’m a married, non-smoker husband, two children. The personal things but, I currently live just North of Birmingham in the Midlands. A lot of people from London so I’m a northerner but no actually I live in the Midlands. There is many other places further north than I am.

KN: Come on, only the Britons knows those parts of UK.

PA: In terms of professional wise. Yes, as Kamil says principal consultant at the moment since solution architects working with Altius. Probably about 10 to 15 years of experience now in the industry. Starting very much as on-premises SQL Server BI Developer, maybe originally, and then there’s the technology is involved, my role has evolved and coming into various consultants is now and things are obviously progressed and got a lot more crazy since then since we started working with Azure and what are job titles are and what we do.

KN: Exactly. What actually do you do during your business day?

PA: Interesting one. At the moment I’m the solution lead for delivery team, for Altius’s retail customers. So I will support the team with the design of the solution, getting the next bits of work lined up. You know, attending the Sprint Ceremony is to do the refinement of the store is doing a lot of the design documentation working with the customer requirements. So, really. As far as supporting customers goes, that’s solution lead role is where I’m at the moment doing the architecture, supporting that delivery team. Which at the moment that delivery team is split between India between the Netherlands, partly in the UK in London and then the customer is based in Belgium. So, it’s quite an interesting set up at the moment with the remote teams.

KN: So mainly working from the UK? How often do you fly to the Netherlands?

PA: Sure, I’ll predominantly based at home, so working from home a lot of the time during a two-week sprint and then probably go over to Belgium once a week. Maybe to see the team in Amsterdam as well probably every fortnight so. That’s the current set-up, but sometimes yeah, it does feel like I am a visiting Birmingham Airport far too much.

KN: But working from home most of the time. Lucky Man.

PA: Yes. I mean, my youngest daughter, she’s just turned one so having her get back from nursery come into the office and start pulling on the chair and bashing the keyboard. And so it was a nice way to be told: “It’s time to finish work”.

KN: I still remember when you were speaking at SQLBits with Chris Testa O’Neil. It was most probably not your first time when you were speaking but that time or around the time you’ve been awarded with MVP title. It was probably a few months after.

PA:. I think I was awarded my MVP in sort of June/July time of 2017, I think. It was just before they actually changed the renewal cadence. I think, I just kind of got in there before. They then started the new March sort of renewal cycle. But, it was very much a Microsoft session that Chris was leading around Data Factory. I think you know just talking about the I think it was probably the V1 version of Data Factory…

KN: Yeah, yes, definitely, it was V1. I remember the complex diagram that you were showing.

PA: So, I mean I talked about Data Factory in terms of the good, the bad and the ugly at the time because it was version one of the products and Chris introduced the tool and the resource. And then I just started, kind of I think 10 minutes at the end from just real-world experience having implemented it.

KN: The speaking what does commitment in the community mean for you?

PA: Lot’s of things I guess. I can safely say that I’m definitely passionate about what I do, and I love sharing that knowledge. Sharing that passion with other people. So for me, the community is the best way to do that. I really do love speaking. I get nervous before every single talk.

KN: Like we all, I think. Which is probably good.

PA: I think it’s good. I think that a little bit of nervousness keeps you sharp. It doesn’t matter how many times you spoken yet. I still get nervous. But for me the community is that way to share that passion. To share that knowledge. And that’s why I like doing.

KN: Good to know that it’s not only me, being nervous before speaking. Only the native English-speakers as the same thing.

PA: It could be an audience of 10 people, it could be an audience of 400; for me, the nervousness is the same as the time.

KN: I know that you record and report all your activities in what kind of form. Could you tell us a bit more about that? Who is maybe not familiar with or not having an account on Twitter or something?

PA: Sure, so when it comes to the MVP recognition is that what you mean?

KN: It’s a bit related, but I’m talking about your Power BI reports.

PA: Sure. I guess really for my own sanity, I started keeping a log of where I had spoken around the UK at the time, and then as I attended more events around the world. So I think I kept this log this database and then playing around I put the Power BI dashboard on the front of that and I think, given where right now we’re saying that this is my 100th talk.

KN: 100th exactly? This one?

PA: Yeah. Last week I was over at SQL Conference in Germany and then SQL Saturday Belgium. So I think if my database is correct then this will be my 100th community talk in the last six or three or four years so.

KN: Amazing, I’m so now very proud.

PA: Thank you, thank you for having me.

KN: Fantastic, 100, good. And yeah, if some people would like to or interested in see your report they can check. That’s probably all. Find it out on some on your Twitter. Probably in some timeline.

PA: There’s a code GitHub repo for it. I think I just called community speaking log and the database scripts and the latest version the Power BI dashboard is in there, if you want to see where I’ve been.

KN: I put the link to that as well.

KN: So, MVP since 2017, so we both know that there’s a big sacrifice behind the being an MVP member. But also there’s are some benefits to it. How did you use it? What does it mean for you?

PA: Good question. So I think I’ve said this to Microsoft as well, but I mean there all the benefits that you get from being MVP. The free clothes, the Azure credits, all those things, for me I would trade all of those benefits for the interaction you get with the Microsoft product team. They listen to you about what changes should be coming in the tack, and they also share what they’re doing with it. So, that’s the NDA, that non-disclosure agreement that you have with Microsoft for me that is the biggest benefits of being an MVP. More so than the free clothes and things I joke about…

KN: The free clothes are actually not free. I mean you need to pay for a delivery, which is in most cases much more expensive than the clothes itself.

PA: Yeah, if you’re not in the US, you have to pay to get a jacket.

KN: Which is funny enough.

PA: Did I answer your question?

KN: Ahh, yes, thank you.

KN: So under NDA you have the information, the updates from the first hand.

PA: So it’s good to know what’s Microsoft working on, what’s coming next, and you know having that insight, it’s definitely helped me shape my career a little bit in some cases and then help my customers in terms of what we deliver for them. It’s a really nice relationship that you get with Microsoft, which gives you that future look.

KN: Even having art of updates from MVP members, I mean we as a community and also I’m being and feeling part of this data platform things. I am talking about that we couldn’t avoid, you know, having a Data Factory in version 1, now, we have version two. Somethings happens anyway.

KN: MVP Global Summit supposed to happen next week, but due to the corona-virus it’s will be held online only. Do you regret?

PA: Sure, I think it’s a great event. It’s a great networking opportunity for everybody that goes to Redmond for the MVP Summit. And you know, as I’ve already suggested, that that interaction you get with the product teams, that’s definitely something that’s better face to face. But, I mean we are a nomadic people in the IT industry, so it being an online event it’s not great, but I guess here with my young family there are some benefits to not spending nearly 24 hours in an airport to go there and back. But, I will miss the MVP Summit being in Redmond this year.

KN: And place, and other people and a need to experience talking and etc.

PA: Yeah, the dates for the 2021 once Summit have just been announced, so looking forward to allow to spin the next chance to go to the Microsoft HQ.

KN: But it’s one year… we need to wait.

KN: I love the numbers and accidentally we have changed our job pretty much in the same time. Like about six months ago.

PA: Yeah, you did copy me?

KN: Ha ha, no, I haven’t known that time that you are changing your job. And after being working 18 months in the previous company, we both. we’re working in 18 months.

PA: Yeah, I changed a couple of times. With Purple Frog, I think two years and then with Adatis, I think about a year and I changed to Altius.

KN: So we both changed to Altius company right now, another consultancy company. You have more experience than me in consultancy company. What do you think about that kind of changes? Do you see any differences working for just different company in consultancy market for customers?

PA: Absolutely. Something that I was maybe a little naïve to once upon a time was how calculating, I think, Microsoft or when it comes to aligning customer requirements with one of their partners and certainly having been with a few different consultants is now of different sizes just how calculating that relationship is and being with Altius now engaging with some of the larger enterprise customers, it’s really interesting to see some of the challenges that they have. None of which most of the time, our technical challenges. More of which are challenges around policy and bureaucracy and all those other sorts of things. And you know, the team I’m currently working with now we are all remote, so it’s those new sorts of challenges, which is what I’ve certainly found in this latest move.

KN: That’s interesting. So you mentioned that you were working remotely most of the time because you are neither not living in London and nor around. So how much it takes you to come to London?

PA: How much does it take to come here? Not too long actually, so there’s for me it’s quite nice so I get a direct train from Stafford that comes straight into Houston and that only takes an hour and 15 minutes…

KN: From door to door?

PA: From door to door, probably 2 hours. So maybe you know a 30-minute drive home to the train station, then maybe 20 minutes on the Northern line and Circle Line ground to Blackfriars. So, probably 2 hours door-to-door, which I think given how far North I am in the country. I know I have family just outside the M25 in St. Albans and for them to commute into London can take an hour and a half on it…

KN: compared to where you living is like almost nothing but it’s still 2 hours in one direction here.

PA: So I think, it’s relative. I don’t mind it. So I come into the office probably every week.

KN: How about your work-life balance then? How you can cope with that?

PA: I wouldn’t call it a “balance”. It’s a hard one. I mean, I have the advantage of we’ve talked about of working from home, which is nice a lot of the time and I think if anything that probably makes up for some of the travellings that we do for customers for community events or just come into the office. But I think this is the life of any consultant really. There’s two parts to it I guess: there’s a work-life balance when it comes to travelling and then there’s also a balance when it maybe comes to some of the community commitments that we have around blogging and speaking and all these other things. And it’s hard. It’s a sacrifice but safe for me it’s that passion, that kind of fuels that and the rewards that we’ve spoken about the MVP program is what kind of makes some of that worthwhile. It’s a trade-off, I guess. It’s hard to call it a balance, I don’t think there’s a balance.

KN: Yeah, because you mentioned travelling, commitment to the conferences. Where is the time for life or private life for the family?

PA: I think if I had a hobby I might struggle, but I think I realized probably quite early on and probably this is where the passion comes from, that to some extent my job is my hobby.

KN: That’s my next question: how do you know? So what’s your hobby data?

PA: I mean well… wouldn’t so upon a time, I’m I used to swim a lot, I used to running half marathons. I used to really enjoy biking, I used to be on the darts team, but all of these things have kind of slipped away from here in the pursuit of a career maybe and also because of a young family takes that time so…

KN: The other things disappeared.

PA: Yeah, yeah, I don’t know if I have any hobbies anymore. Maybe, just… sleeping is my hobby…

KN: I like this also, but there’s not too many time to having that hobby, to practice that hobby.

PA: Absolutely.

KN: So conferences and event is also great opportunity to visit new places. So how often do you take your family to conferences like SQL Saturday, like past weekend for example?

PA: As often as I can, so these events they are a little bit for me in the networking and you know for the sharing but also trying to make them miniature Holidays for the family. My wife Christine, She’s been to quite a few events now. She’s been with me to SQL Saturday Iceland. That was the main one because she really enjoyed that. She’s a geography teacher, so Iceland is a great place to visit for lots of reasons. But I mean our daughter was born in December of last year, so sorry, the year before, she’s 15 months old now. So yeah, we’ve not been to as many because we’ve had a baby. But I mean, in theory, our daughter, she came to SQLBits, when it was the Speakeasy theme, I’m losing track of what year that was, whether that was last year, or the year before. But the family always gets it comes to community events if logistics and things work out, ’cause I like to bring them along.

KN: So in that way also you can spend, you know, a little bit time and weekend with your family in other places, interesting places.

PA: Yeah, it’s definitely a life goal of mine to try and visit every single country in the world, but that’s obviously very ambitious, but I’d like to. And if the family can come as well then great.

KN: OK, how are you preparing yourself for a speech?

PA: How do I prepare? Lots of ways. I will always go through my slides multiple times. For me, I don’t know if this is just me or not, but when I build my slides and the way I build them in quite a visual way. I know the act of creating the slides for me… it helps me think about what I’m going to say. So that’s kind of one thing, just building the slides for me teaches me what I’m going to talk about. But then I will go through the slides multiple times and practice, practice, practice because you have to. I will also run through them, I will present them to the wife, to the family sometimes. I will record myself and listen to myself in the car or on the train, which I hate the sound of my own voice, so this I will be in the car and I’ll probably be cringing and somebody next to me will probably look at me. But listening to myself sometimes helps. There’s lots of techniques. Depends on the talk.

KN: If you’re working from home you don’t have too many opportunities to travel by car. So, when do you do that?

PA: And sometimes small train or plane or just recording myself going through the talk is one way of doing it. Rehearse in the demos and doing screen recordings of the demos and things and then watching them back is a good one.

KN: So are you recording only the audio, or also the video? How the body works and how you are presenting with your body? Or just in that part of preparing it doesn’t matter?

PA: I think just the audio really, when comes to actually being on stage and kind of the problems of the slides. It’s maybe a little different. There’s no preparation there, that’s just all adrenaline.

KN: All those are at least maybe since I remember, probably since Data Rally, it was called at that time SQL Relay, like two years ago, you started your icon game at the beginning of each session. So what’s your favourite service in Azure then?

PA: My favourite service. I don’t know if I have a favourite. I mean, let’s go the easy one then. Let’s say Data Factory is now.

KN: Exactly. I know that you’re speaking and speaking, maybe not a lot, but, many times at least, and also recently writing a blog posts about it many times, so started also the serious about ADF, etc. And I remember that you started working with version one, so you have experience with that, which I haven’t. And that service would be obfuscated by Microsoft right now, yes?

PA: Yeah, I don’t think you can create a new Data Factory V1 instances anymore or in certain if you have a V1 instance, I think you need special permissions now to do these sort of authoring by the portal. So but you’re right, I started working with Data Factory, I think in 2016 or sort of late 2015 when V1 was still in preview. So, it’s been a long time with Data Factory.

KN: So now you have experience with version 2. How you could compare both of those versions?

PA: Comparing V1 and V2? I mean they’re very different services. The goal of course being the same, but I don’t know. There is things about version one, I think that I probably still miss. There are things that in version two that, of course, a lot better. The number of activities we can now use, the dynamic content that we can build into pipelines is great, and the UI, I think in version two makes things really nice for a developer to put things together.

KN: But there’s still lack of UI, like in terms of having UI in Visual Studio, for example, not in the browser that many of users complain about it.

PA: Yeah, having that Data Factory projects that you got in V1 in Visual Studio 2015 is something that I do still miss, because ultimately it means that you can’t work on Data Factory when you are on a plane or a train and travelling quite a lot as I do. It means I do need a good, reasonable Internet connection to actually work in the browser to develop the tool. So, I mean that’s for me the biggest thing I would miss about V1 and you know, there’s a new answers around the deployment process, which I think were slightly better when you publish the solution in V1 of Data Factory versus how you do the ARM Template deployments now in V2. But yes, it’s a good tool. It’s just these differences.

KN: It’s a different way how to use it, yeah?

PA: Yeah.

KN: I think it’s worth mentioning that the ADF V2 is very well integrated with Git repository in general. So it’s also helps you to manage the code, the first thing. But also it helps you somehow do the deployment through the “ADF_publish” branch which basically is created when and you publish things. But don’t you think that it’s overcomplicated a little bit?

PA: I don’t know about overly complicated, but I think certainly for somebody that is used to interacting with source code repository via Visual Studio where they have that you know that offline copy that local version of the code and they do the changes that commits the pushes and things. I think if you come from that environment to then working with the online tool is quite interesting and probably different because you know you can be working in a feature branch in Data Factory and every time that user interface changes or becomes dirty, you click save behind the scenes it is doing a commit every time to the GIT, so it can make the version history of a particular file looking very chatty, when somebody is looking at the source code repo. So, I’m not sure about complicated, but certainly different to probably have other tools and how other programmers might interact now with source control.

KN: Have you had that experience, you know, when you have quite so well complicated or complex, branching strategy, having more than one or even more than two (branches), like “master” and “developer” and at the end, there’s only one “adf_publish” branch.

PA: Yeah. So that I mean I we used to get flow branching strategy now as probably a concept. But, Data Factory doesn’t really align to that. So I think the common way we get round it is by not using the content from the “adf_publish” branch, but actually using the component parts of Data Factory in the JSON files.

KN: So, actually the codes which is saved and holds in the branch. So iteratively going through all the objects and deploying them one by one.

PA: Yeah, so like very much what Data Factory V1 did with Visual Studio 2015 project when you hit publish. You select as what parts of the Data Factory wants to deploy and it did it. We kind of take a very similar approach now with V2 to target, which bits of the JSON we want to deploy.

KN: OK, so in general – do you have feelings which version is better one to deploy things?

PA: That’s why it “depends” questions. I think it’s got to come down to the complexity of your environment. You know, if you have dedicated: DEV, TEST, PRODUCTION, UAT, PRE-PRODUCTION, however many environments you have and I think that complexity, but then also the complexity of your Data Factory. Do you have 100 pipelines or do you have three? I think if you’ve got two environments and three pipelines, then maybe just using the publish option and arm templates is easier. If you have 16 environments and 100 pipelines and many, many activities then yeah, probably using the underlying JSON files might be a better option for you.

KN: Azure SQL Data Warehouse has been renamed to the Azure Synapse.

PA: Yes.

KN: OK, and do you have access to that?

PA: No.

[We both have access to private preview version of Azure Synapse in shared pool for MVPs]

KN: Yes, me neither. Also, SQL Server 2019 Big Data Cluster – how do you think: what’s opportunity those technologies gives to the customers these days?

PA: Good question. I think, or my perception in both cases, it’s where Microsoft is now moving forward and they’re fusing their tech with the open-source world in a lot of cases. Bob Ward got a great podcast they did for the Azure Podcast around Big Data Clusters and he talks about all the good bits of those tools coming together and what they’ve kind of put into that new product. So, I think for me, the Big Data Cluster offering and what we hope is the Synapse offering, it is that a fusion of Microsoft SQL bits with the open-source good bits and bringing together a suite of tools that could probably deliver 90% of what the customer needs. It’s interesting times to see how those two things will affect the data engineering landscape or certainly my job.

KN: So yeah, working with Azure right now, might be quite challenging to back to on-prem when some customers have more servers on-prem and, you know, to install SQL Server 2019, but yeah, maybe it also quite a challenge and goods benefit of that.

PA: I mean if they have hardware than I can just deploy Kubernetes and they can use it on-premises until that hardware becomes obsolete and then they could always migrate that Kubernetes instance elsewhere if they need to.

KN: Oh yeah, definitely. A much easier way to migrate the things to the cloud.

PA: Sure.

KN: What do you think, what is the biggest challenge in IT those days?

PA: The biggest challenge in IT? That’s of a large question. I don’t know if I have time to answer such a big question.

KN: Or maybe, you know, the biggest challenge in the clouds in general or specifically in Azure.

PA: So, maybe I’ll draw on the talk I gave at SQL Saturday Belgium, so maybe help answer that. My talk was about an architect sky to delivering data insight. So I think to answer the question what is the biggest challenge… I guess for most people it’s what service do I use? What tool is right for the situation that I’m working with now, because there’s so many different ways you can build and deliver things now within the within Azure and all the capabilities that you have there.

KN: You can deliver the same software, the same service with using different services.

PA: Yeah, it’s really about knowing what not to use, when and what is the right tool. I always say to my customers that don’t be distracted by the new and shiny technology and make your requirements fit the technology. It’s about the other way around. It’s about taking your requirements and making sure that you choose the tech that fits what you need to do rather than you have been distracted, maybe with new things.

KN: I’ve been on your session two days ago and you were talking also about constantly changing requirements. So how to be prepared for that?

PA: It’s a good one. So this is a fault of customers, I think this is just always the case that you deliver one thing and then it means that you end up delivering more and more because requirements change. But I guess it’s about trying to have a more of a plug and play architecture having that flexibility to bolt on those new requirements and not being too rigid with you delivery approach, which means you can add more two things as you need. I don’t think this is a one good methodology that will allow that, but certainly some of the tools that you got in Azure now they do give you that flexibility just to add more and support those changing requirements, but it’s a challenge either way.

KN: Yes, it’s always a challenge. But do you think it’s maybe is a way to prepare for that or maybe avoid that always changing requirement? Maybe, if the first requirement comes, maybe we can do something better to avoid the second iteration of set of ideas.

PA: Sure, I mean, maybe a crystal ball is the thing that we do better. Somebody that can look into the future and say “we know every single requirement that’s coming”. In some cases, I think, the now you can plan ahead, but it’s I don’t have a good answer, really, I think. In some cases, changing requirements the technology allows it, it’s very easy. You know the case is changing requirements means that you almost need to start again. It would be great to be able to predict all of those things. I don’t know if we can.

KN: What do you think what is the best way to be up-to-date with technology?

PA: The best way to be up-to-date with technology… the technology we use or just any technology?

KN: Yes, we use and we are working with.

PA: I would say just immerse yourself in. It’s just every day. Try and try and keep up with all the latest news and announcements where you can. I listen to quite a lot of podcasts in the car to try and keep up with things. But just I think almost you could say, except that you’re never going to know everything, just keep trying and just keep running along to try and keep up with it. But I feel like in the IT industry now we’re going to end up in a situation. Probably a lot like professional footballers, where you know they retire at the age of 35, because they’re not young people anymore in their industry and they can’t keep up. I think probably will have the same problem in the IT industry that it’s only young people that they can keep up with the pace of change.

KN: That’s right, but with a small difference. We will not be as rich as much as those people. [laugh]

PA: No, we definitely won’t be worth as many as footballers, no. But I think I can see by the time I get there maybe 40, 50, I won’t be able to keep up with the IT industry anymore and I’ll have to rethink my life.

KN: That’s true.

KN: The last question: what hints would you give to young people who wanted to start working on the market, on the IT market specifically. At the beginning of their journey?

PA: I would say: don’t be afraid to get it wrong. Go and figure out what you like doing, what you don’t like doing, and start eliminating your options because the IT industry now is so broad in terms of the job roles that you have. Would not so upon a time you could have a system admin that looked after all parts of an IT network and now the breath of the industry it’s so massive. And then the depth in which you can specialize in so many different areas. So I think it’s a case of going make some wrong decisions, figure those things out, and then find out what is the right decision. A little bit of trial and error almost and say not being afraid of that trial and error.

KN: Which might be tricky for young people, especially if you’re starting working.

PA: I mean you know somebody that’s just completed a University degree in computer science, and you know that breath of that and the scope what you do next, it’s a hard one. I started out in the village where I was living, repairing friends and family’s computers and setting up printers and putting RAM in computer towers and things.

KN: I can see that we all have such history…

PA: Yeah, unfortunately, I’m still go-to guy within the family when it comes to Technical Support of all things from “my Word document isn’t landscape or portrait” to “my website is broken and my printer needs setting”. So, I’m still that guy, unfortunately. But I think that for me helps me figure out what parts of the IT industry I wanted to then focus on.

KN: And at the end of our conversation, could you tell us where we can find you. Were people can find you?

PA: Where the people can find me?

KN: I’m not talking about your private address. (laugh)

PA: No, haha… this is “my dress”… My blog is MrPaulAndrew.com and then from there yeah you could probably get my email and my various profiles and things. But my blog is probably the best landing point and reach out to me.

KN: Cool. Thank you. But, do not your blog ends with “.tech”?

PA: My blog ends with all things. I say that I play Internet wars with another Paul Andrew that’s out there. A chaper, I think is far more famous to me actually, but I think he designed shoes. So this shoe designer has something “paulandrew.com” and then what I’ve tried to do is I’ve tried to buy up all the other “PaulAndrew” domains and they all do I think redirect to my blog. mrpaulandrew.com was the 1st and I’ve registered a few more just to play Internet wars with the other more famous Paul Andrew that’s out there.

KN: But now, you know what you can do, what you can change in terms of your job when you get retirement?

PA: Yeah, I will design shoes. Just, you know, impersonate him and start doing that job instead.

KN: OK, cool. Thank you very much. So it was your 100th activity for the community – our podcast yeah?

PA: Yes, SQL Player 100 community talk.

KN: Great, great. We will advertise that in that way as well.

PA: I should have, I should go and celebrate personally, shouldn’t I?

KN: We can do, haha. Thank you very much, Paul, again for that conversation.

PA: Thanks Kamil, thanks for having me.

Useful links

Paul’s profiles: Twitter | LinkedIn | Blog | GitHub
Related events: Data Relay
Paul’s second home: Altius Data