ASF 006: Bob Ward interview

Introduction

Bob Ward is a Principal Architect for the Microsoft Data Group (Tiger Team) which owns the development and servicing for all SQL Server versions. Bob has worked for Microsoft for 24 years supporting and speaking on every version of SQL Server shipped from OS/2 1.1 to SQL Server 2016. He has worked in customer support as a principal escalation engineer and Chief Technology Officer (CTO) interacting with some of the largest SQL Server deployments in the world. Bob is a well-known speaker on SQL Server often presenting talks on internals and troubleshooting at events such as SQL PASS Summit, SQLBits, SQLIntersection, and Microsoft Ignite.

This talk has taken place during PASS Summit in Seattle, WA, on 2nd November 2017 (Thursday).

Do you want to find out what is the next goal of Tiger Team for SQL Server?
What was his first database he worked with and how SQL’s world looked like before DMV appeared? What does Bob have to do with the city of Łódź and why he does like to return to Poland?
Listen to this episode to find out all answers and more.

Transcript

Kamil Nowinski: Hi, Bob!

Bob Ward: Hello.

KN: Thank you for talking the invitation for the podcast SQL Player.

BW: Happy to be here.

KN: It is good to see you again. We first met on the SQLDay about three years ago…?

BW: Yes, I think it was quite a while ago. I enjoyed it very much. It was a very good conference.

KN: However, PASS Summit conference is probably the best in the world.

It is certainly the largest in the world. I am not sure that I would say it is the best because one thing that I loved about smaller conferences like SQLDay is that they give you opportunity to meet people one on one and talk to them. I get to do that here at past. But it is like four thousand people, there is a lot of people here. It is a great conference. One of the thing that makes it really good is just that there are so many different verities of topics and so many experts. The top experts in the world of SQL Server topics are here.

KN: There are many of paths here and a big opportunity to meet people and talk to them face to face.

BW: Absolutely.

KN: Could you please introduce yourself? I know that all the attendee knows you very well.

BW: Sure, sure. My name is Bob Ward. I work for Microsoft and my title is Principal Architect. And now I am working for SQL engineering team. I have been in Microsoft for 24 years. I just turned twenty-fourth anniversary at Microsoft this last week. Most of my career up until last year was working at support. When I was in SQLDay I was still at support team. As an escalation engineer, I did a lot of deep troubleshooting on hard customer problems. Then I got to join the engineering team here and I am here a couple of years. I spend a lot of time still working with customers. I spend a lot of time as well doing evangelism, some of the work that we do. I do a lot of performance work like today. I did a talk on Linux yesterday. I was part of the keynote, as you probably saw.

KN: Very internal things.

BW: There is no question I have a reputation for doing that, right? And I love doing that. But I also have expanded it out to make sure that I do talks that are more intermediate level talks, explaning the value. The talk that I am doing today on performance is about SQL server 2017 and performance. I would not consider it deep internal stuff. I’ve kind of tried to broaden my expansion of talking about things that are not just internal.

KN: Currently I heard that SQL server 2017 is the fastest in the world. What is the next goal?

BW: I think so.

We will obviously continue to make it fast. Any time we release a new version of SQL server, we are always going to look for better performance. One thing to keep in mind about SQL server that I take pride in is price performance. It is not just about being the fastest database in the world, but it is about being the most affordable fastest database in the world. Maybe you saw in the keynote here in PASS, I did the demonstration using HP’s gen X server technology in the technology called persistent scalable memory. And this concept is where HP’s device has the ability to combine standard RAM modules – memory modules with SSD drives as storage. So, you can access memory as storage in very, very fast time. The cool thing about this is that it is 50% of a cost of a normal system for that the same amount space using older SSD’s drives. And if you think about that it is really groundbreaking. Think about a world where SQL server could potentially access memory as our disk in a way that bypasses of software storage layer. What if we could get rid of Page IO latch waits or something like that. I mean, think of those kinds of things, like get rid of all that IO waiting that you used to see in SQL. Those are the kind of advances that we still look out in the future.

KN: It could not have been even imaginable ten year ago?

BW: Yes, nobody would have thought we would be able to do such things. Think about drive and storage technology. That is one of the things that I love about our company that we are always going to be partnering with other companies, saying: “Let’s take advantage of the greatest innovation that is out there.” Though we are always thinking of price performance. It is not just about “fast”. It is like: “Can we find speed for people that is affordable for them instead of just spending millions of dollars on amazing hardware?” That is one thing I am pretty excited to look for.

KN: Yes, so your job is really exciting.

BW: I think so. I think it is an exciting product. If you think about the strategy of what you have seen us doing right now in Mircosoft SQL server, the SQL server engine still ends up being the most popular database engines in the planet but we are pretty serious about our Cloud Technology. Maybe you have heard about something called managed database instance which is us taking SQL Azure database and exposing that in the way that it looks like SQL server. You get the SQL Agent jobs, and link survey queries – things that you could not do before the Azure database were more constrained – now we have opened that up. But we still give you the power of the Cloud. So, I am actually excited about the ability to give our customers the choice. You want to run in your Cloud, we make SQL great for you. We will do it on Linux or Windows, we will do in containers. But if you want to choose our Public Cloud Technology, it is actually becoming pretty good.

Damian Widera: Bob, do you think that at some point of time we are going to not used anything on premises. We go to the cloud only? Does it the only option for us?

BW: Yeah, I do not see that happening anytime soon. We are pretty committed to giving you a choice. When we talk about Cloud, we often use the term Cloud first but not Cloud only for SQL server. What we mean by that is that we will do innovation in our Cloud products, iterate on it and make it successful and then bring it down to on-premise at the right time. You think about some of the work we are doing with automatic tuning or query stores or some of these features, those were first born in the Cloud and Usher database, and then we use them with some of our customers and got really good at it and then we push it is to our on-premise product. Damian, in the keynote you ask the people to show their hands of to see how many are ready to move to the Cloud. And very few people really raised their hand. It does not mean that people are not thinking about it, but we realized that a vast majority of customers still loves SQL server and wanna run it in their Cloud. So, you are not going to see anytime soon us abandoning that. We are going to continue to do that, but what we are going to do it and to give people the choice for those that do want to move to the Cloud and make that innovation and successful. One thing you are going to see us doing it to continue to expend on the scalability what is in the Cloud. We are going to start pushing for larger database sizes, larger memory footprints, faster storage. So that people who already made that transition, can do it. Some companies I talked to are thinking of the split model. With their current applications they are going to work on-premise but if they are building a brand-new application, they are taking a serious look at the Cloud as a possibility.

KN: Yes, I think that people are still thinking about it and some DBAs are afraid of that.

BW: I think DBAs are afraid of it only because they think like: “If this sink into the Cloud, I do not have to do anything more.” I think if you look at Azure database, that is false. Yes, we tried to put automation and things there to help developers and customers. But at the end of the day, query performance, query tuning in that design, some of the things are still important to make things run well. You are still going to have to need to do that even if you are running in the Cloud-type situation. Like I said, I think in foreseeable future customers are running in both worlds. Frankly, I think it is an exciting time for the DBA or data profession, because how about trying to learn both worlds? Having that skill set? That is actually a very good skill to have.

DW: I would like to ask about SQL Server on Linux, because I can remember our discussion 3 years ago in Wroclaw during SQLDay and I know that we asked you: how big effort would it be to move from SQL Server to Linux and you said to us, well, it would be really big effort due to how SQL Server is built and now, 3 years later, we can see like a new-star: SQL Server on Linux works, it is very fast and people are really looking for an adoption. So, they try to do this. My question would be: do you think that you can go to any other operation systems? Can you see something like this apart from Windows and Linux?

BW: That is a good question. I guess I was not a very good predictor three years ago, was I? [laugh] What I did not know back then but what I know now – quite frankly which happened not to long after I left your Poland conference – there was a Microsoft research project called Drawbridge. Its goal was to find the way to use SQL server in a Windows environment almost like a container. It was a really brilliant design and it provided the ability for teams like us to take the work they did and say: “I think we can find a way to make the core engine SQL server run on Linux but makes it thinking it is running on Windows.” But doing it in a way that is light-weight and fast. And you still get the performance. That is one of the things that we are able to make possible. When we have got SQL server on Linux we have pieces of code that are recompiled on Linux and these are those once that talked directly to the Linux kernel. But SQL server itself runs actually in the process based on Linux and that code is unchanged. It is still the same code that runs it on Windows. And when the engine runs, it does not really know. The reason we achieve that is this is still all Intel assembly code – no matter what happens, whether it is running on the Linux operating system or running on the Windows operating system it is still Intel assembly code. We have just done it in a clever way to run that Assembly code and not worry about the fact whether it is on Linux or Windows. Because – using this Drawbridge concept – we have this light-weight layer that knows how to talk to the Linux kernel and knows how to take all the calls that SQL makes that need the kernel whether it is IO or thread or memory make that call. So, your question about: “Can we do it on any other operating system?” We surely set ourselves up to that. It does not necessarily mean that is something that we have got on our plate, like we are looking at others OS footprints. But the architectural that we have built, what we called SQLPAL, which you should think of a new SQL OS that certainly has set ourselves up for it. If we want to that in the future, we probably could.

KN: Bob, this podcast is not only about the technical stuff. Many people would like to know some of your habits, hobbies, etc. What is your hobby?

BW: If you talk to me or see me here in the USA and when you hear me speak, you know that I am a big sports fan. Maybe you see me before I did demos, I used sports teams names. Absolutely sport is a huge hobby of mine. I play sports. I like playing golf. I have two sons: Ryan and Troy. My younger son Ryan is still in college and he plays college baseball in America. And I love watching him playing baseball. My older son Troy also played baseball in college. He is now graduated. I am a big sports fan. This is still something that I enjoy doing. I love all sports by the way. You might find it interesting, because you are from Poland, I was a soccer player in college. That was my sport. I was actually really good at it. So, I played soccer in college and I enjoyed it very much.

The other thing that I am really into is my Christian faith. I spend a lot of time at my Church – volunteering and helping out. I helped with our Youth program for many years, we called it the high-schoolage. I still do that, so that’s another big thing, that’s a huge part of my life

KN: You told me yesterday that your wife has been in Łódź in Poland.

BW: Can you believe that? You came up to me yesterday and it was so funny that Poland was on my mind. My wife had an opportunity to go to Łódź as a missionary to spend two weeks there in the program called Lets start talking where she taught English. We use the Christian Bible to do this with. She loved that city. She had so much fun in Łódź. She just said that everybody loved beets there and everything that she ate there had a lot of it. She also got to do some fun. She went to Warsaw for a few days. I am a big fan of history, especially the World War II history. And she got to go to some places in Łódź which I did not realize was one of the largest population of Jewish people during that time. And then she went to Warsaw to visit some historical places. I was kind of jealous of her. But she was in Poland for about two and a half week. She loved the Polish people. She loved them so much that one of the people that she met, who works as a missionary in Łódź, came to our home last week. He talked more about Poland and the history. I told him that I have been to Wrocław.

KN: What about the work-life balance?

BW: It is a struggle. Quite frankly I think it is a struggle for many people in technology. I have been in Microsoft for 24 years and it has always been a struggle. But I will tell you that my family is very important to me. So my family comes first. I do some creative things at Microsoft which is such an amazing company to work for. They allow flexible hours, flexible work schedules. Sometimes I work from home. So I tried to use creative ways to make sure that I dedicate the time to those hobbies I talked about but also to my family. It is a little easier for me today because both of my boys are grown and out of my house. But I still want to connect with them. And not only them, but also my mother, and my family from Texas. I just try to make sure that I am conscious about it every week. Sometimes I slip like everyone else and I am working at home. But maybe after a couple of days I will shut my computer off and I won’t turn it back again when I am at home. Or sometimes I turn my phone off for a couple of days. It is hard. But I tried to force myself to do it. One thing that I do on my personal vacation is shutting my email off for the whole time. I could be on vacation for a week with my family and I will not read any emails from my company for week. It does not matter. And I tell my boss: “I guess you can call me if there is an emergency like the world is coming to an end.” But I try to make sure I do not do that. And that is really helpful. That is a really good tip for everybody. Do not read your company email when you are on your vacation. It forces you to dedicate your time for vacation.

KN: Yes, and you started to be addicted to checking emails.

BW: Of, course. No question.

DW: When did you start interesting in SQL Server?

KN: This is an interesting story. I came out of college and develop the skill in C programming when that was a new thing back in the nineties and late eighties. I came out of college in 1986 and nobody now C programming back then. And I quickly learned to write C programming on UNIX systems. I joined the job when somebody said: “We’d like you to be our developer and database administrator on Ingres which was a database system many years ago and eventually Oracle. I just develop to learning for the SQL language and database systems because I was kind of force to do it in the job as a programmer. So in the early nineties, I was looking for a new thing to do. A friend of my wife was working for a Microsoft and said they were looking for people who new database products, especially Oracle skills. Because SQL server back then was just a very small product. Nobody really used or new SQL a lot. It was mostly Sybase so it would have been a plus. But I knew Oracle very well so one of the reasons I was hired at Microsoft was because I had this Oracle skills but I also had some of the development skills. The people who hired me said: “You could be very valuable in the support space because you know how to program and write code. You could probably debug easily customers problems.” So, I actually learnt very early on how to read the SQL server source code. That was fun for me. I was reading the SQL server engines code back in 1994 how we wrote it and thought these years one of the things I love about my job is that I have the ability to still look at our source code and kind of translate it for everybody. When you think of the internal talks that I do, that is really me translating our source code.

KN: Yes, I still remember your workshop. I think that most of the people do not understand the whole session – including me!

BW: It is hard. I have a fun in doing it though. I enjoy transferring knowledge to people. That is one of the thing I love about my job. I get a great pleasure of translating very difficult topics, not that everybody else are not smart, but you do not have an access to the source code either you have spent all your time on it. But that is kind of my history on SQL, I was kind of forced to do it at my job as a young college graduate. I loved it. This kind of database is fun. I had also some friends that said it was a great skill to have. Ingres, Oracle that was my background, now I’m a SQL Server guy

DW: I just wanted to say that, because I’ve attended a lot of your sessions and I can see that you can explain everything, even if those things are very complicated. Because you have an enormous passion – so you can talk about complicated things in a very easy way that most people can understand once we have a teacher like you. That’s my opinion.

That’s so kind of you to say. Thank you.

DW: I know that Kamil asked a lot of questions, but I would like to know if you have any animals at home?

BW: Yes, I have two dogs. I have a Border collie and a Labrador. These dogs are now quite old. We have those dogs since my kids were young. The Border collie is like fourteen years old, and the Lab is like twelve years old. I still love them. I love dogs. Man’s best friend. Very loyal animals. I have some property – everything is bigger in Texas – I have three acres of land, so my dogs loved to run around my house. That is actually one of my hobbies – maintaining my property.

KN: Damian maintained also that you can explain very easily the very complicated stuff. How are you preparing yourself for a speech?

BW: That is a great question. Maybe you have heard me speaking and it seems that it comes easily. But it does not. I spend a very long time in a very hard preparation for every session I do. The way I usually build my presentation it is like anything you will do in life to build a paper or a book. So I outline my idea – what I am going to do. I use PowerPoint to do that. Then I start filling it in with what I need to do. And I always overbuild my presentation. Sometimes in my presentation, you can see me having some bonus material section – at the very back there will be extra slides that did not go over. I build more slides than I need and then based on the time I am allowed for my talk I take slides that I am not gonna cover and put them in the back.

And another thing I do, this is gonna sound crazy, I like to exercise – running or lifting weights – and usually when I do that and I am having presentation coming up, I am thinking and rehearsing during those sessions. Like, I run for three miles, and during this run I will be rehearing in my mind what I am going to say and typically every time I run I have some new ideas. “Oh, I need to change this slide. I need to make sure I cover this.” Sometime when I am on my run I will take my phone out and I am recording on my phone: “Hey, Bob, don’t forget to change something.” I found out that exercising is a great thing to clear your mind and it helps you think about things in a new way. A lot of my preparation goes into these talks. I also am a great believer in making sure that demos are done early. Always do your demos early on, because you do not want to put you on your slides things that you cannot backup with a demo.

DW: How do you teach something new. What about teaching new stuff, when you know that you go, for example, we have to learn a lot of Linux right now to be an expert. How do you do this, how do you manage that?

BW: Sure. That is not easy. First of all, you mentioned passion. So I have a passion for the technology. That makes it a lot easier. If you are passionate about something, it is a lot easier to learn something new. You also got to love learning. If you are afraid to learn new things, transition your career, you cannot teach people stuff. Just having a passion is the first place.

One of the biggest things I do when I try to take on something brand new is to learn how to use it from the perspective of a user. For example, about a year ago I was challenged by my vice president to start teaching people about machine learning services. This is R and Python so far. I did not know a thing about that stuff. So, I started from a perspective of a user. I learned how to use the software first. Why would somebody want to use this? I looked for key studies, value propositions and business cases. So even though you see my talks that are very technical, I actually start to learn something new from the perspective of the question “Why? Why would you want to do this?” Because if you do not have that on your mind who cares about the technical part? So, I always start from this kind of high-level perspective and then I start to figure out: “How does it work?” Because as you have seen in my talks, I love to explain how it works. But if you cannot explain the value why would someone want to use that new technology, it does not really matter if you can explain how it works. So I kind of combine those two together. And that is how I teach something new, to start from that perspective: “How to do it? How to use it and why does it matter?” And sometimes I have been asked to look at the new topic that I did not really think was all that great. I did not have a lot of passion for doing this. Let’s take example on Linux. I used to be a UNIX developer, so I loved Linux. It was very easy to start building training on Linux because I like it. And I think it is a value.

KN: Now, I can see. It was your idea. [laugh]

BW: No. It was not my idea That’s why I brought up Tobias, Slava, and Travis, all those are the founders of it. But I think Linux is very valuable for customers that want to be on Linux. Three are people in the technology world that think that Linux is a better operating system than Windows. I am gonna stay neutral because I am supporting SQL server both on Windows and Linux. I think Windows has many great capabilities and that what I have used for years but there are people out there who are Linux people. That is what they want to do. Maybe you have used Postgres on Linux for years and that is what you liked to do, but Postgres does not meet your performance needs anymore because you scaled up. But you want to stay on Linux. That is why we did this. Put SQL in this platform so that people who love that operating system have an opportunity. We had a customer DV01 which we were showing out into the world that he is a great customer and that is exactly what happened to him: big Linux shop, big Postgres developers shop, the Postgres would not meet their performance needs so they did not know what to do. They saw us starting to go for the Linux proposition and they become big fans very quickly. That is how I believe about the teaching new things. You have to have passion for it and you have to be able to explain people why do they even want to use the technology.

KN: Yeah, the business needs first and the user experience as well.

BW: Yes, and then you start talking how it works. Because I think that if you talk about something how it works it gives people an appreciation. Machine learning services – one of the reasons I started presenting on how that works – is to give DBAs a trust factor that they can trust running this software. Because the DBA is like: “I’m not gonna let R script run in my SQL Server! I don’t trust that.” But if you understand the architecture and what we built then you become more comfortable with that.

DW: About achievements – what achievement are you satisfied the most?

BW: I tell you what. As for as SQL server goes. I think the biggest achievement I am most proud of is that there are features today in the product that I felt directly responsible for. Especially when I was back in support. You look at extended events, maybe you have seen this feature, I was one of the people that first proposed that technology along the Slava Oks to be in the product. Slava came to me and said: “Hey, we have this new technology. What do you think?” I was like: “Yeah!” I remember being in the meeting with the development team and the development leaders pushing the say: “We should put this in the product as a set good.” You look at dynamic management views that was also another one back in the days before we had DMVs, myself, Robert Dorr, we would do these really long debugging sessions to solve problems with Windows debuggers, which was great but it would take hours. So, Slava came to us and say to work together on this DMVs concept.

KN: It was before the DMV happened?

BW: We would use debuggers and now there is the DMV. Slava came back in 2005 when was the first release and she said: “Hey, let’s put the DMVs in here, they can really help the debugging using SQL queries.” It was not only for me but also for everybody else. I think also about the work I did with Paul Randall. He used to be in the development team. His big passion about corruption scenarios, check DB, things in the storage engine – all these different types of technology. We would work directly with Paul to add these features into the products. Those are huge accomplishment for SQL I am proud of.

From my personal perspective, I am just excited that I have two great wonderful young boys. That now I consider to be great young men. They made of high character. It just swells me with great pride when I have heard other people telling me that they have asked them what they like about their dad the most. And they say: “My dad is a person to have high ethics, high integrity and high responsibility.” And when I hear that, that makes me feel very special. And it makes me feel special that I have been able to instil that same sense of character into them.

KN: Do you think Microsoft Ignite that recently has happened is bigger or better conference than PASS Summit?

BW: I have been in all three of them because I have been in PASS Summit since 2003. These are really two different conferences in a way. And I actually treat them a little bit differently. Microsoft Ignite is now the largest IT conference in the world. There are like thirty thousand people coming to that conference. And certainly, SQL server and our data technology is our part of that conference but it is much larger than that. It is a showcase in a way for people across the world that come to talk to Microsoft and our partners and the community about the wide range of Microsoft products, whether it is Cloud, SQL or Windows. The PASS Summit is all about data. This is a data conference. If you like data you come there and I think they complement each other. I think that you will see some content at Ignite presenting also at PASS like I am doing that because some people did not get to go on Ignite. But you will never see me do insight Linux at Ignite because the audience that usually goes there does not want to see that. A little too deep. Ignite will never be a deep internal conference ever, it will be about the value proposition or performance. Whereas PASS provides us unique opportunity. Right now, we are sitting in the data Clinic. We do not have such a thing in Ignite.

KN: Yes, this is an amazing place. You can come here and ask about anything.

BW: If you look at this right now, you can see all these people are asking these questions to these experts. Funny story about the Clinic, you will find interesting, maybe you have heard about this gentleman Ken Henderson – that was an expert on the SQL community many years ago. He passed away around 2008. But Ken was the reason that I got involved in all those speaking arrangements and in PASS. In 2003 at the first PASS Summit that I attended Ken and I started this Clinic concept by sitting in a room with a couple computers, a desk and a couple of chairs and we announced: “Anybody wants to ask us questions, come in.” It was not organized with all those desks and chairs and boards. It was just Ken and I – just two people. And the first day we had like two people. The next day they made an announcement at the keynote: Ken Henderson and Bob Ward are answering questions. The next day so many people came in, the fire marshal had to move people out of the room because there were too many people in the room causing a possible fire hazard. We called it something else back then. But now it is born into this IC of the Microsoft data Clinic which is amazing. Anybody in this conference, for no charge, can walk in here and talk to the top experts of SQL server at Microsoft in the world. Where else can you get that? You will get it at Ignite and here.

KN: How can we adopt at least this small part of that Microsoft Clinic in SQLDay in Poland?

BW: I think it is not hard at all. I remember being at SQLDay and there were several people from Microsoft there. Or even just the experts. If you think about your speakers in SQLDay – those are you experts. All you need for a clinic is a couple of tables and some laptops and a whiteboard.

KN: I am talking about the people from Microsoft.

BW: I understand. We organize these smaller Clinic events in other SQL Saturday events. We have got some of the local Microsoft resources – the PFE engineers, the architects of Windows all over the world. And we just say: “Let’s just get some small table set together and we will advertise a clinic and we will just help people come and ask us questions.” It is really easy to put this together as long as you are able to get people who are willing to staff it and answer the questions. You just need laptops to show something, couple of tables and you got a clinic. That is how we started that and look what has it become?

DW: Are you a perfectionist?

BW: Oh my Gosh! If you ask my wife about this, she would probably say yes. If you ask people who see my present and talk to me, they would probably say: “Yes”. I do not consider myself that. I do consider myself to be thorough. I am a big believer in accuracy and making sure that I do the best I possibly can to present things which are correct. I would rather do things that are correct and that you will understand them than them being polished. Course I have spoken now for 20+ years so I have gotten all of those kind of combined together now. But I do not think that I am a perfectionist. Maybe people think that I am, but I try not to focus too much on perfection. Yesterday I had a small incident at my Linux talk where I had a problem in my demos. And I was not happy about it but mostly people said that they could not see that I was angry. I am still not happy but I am not gonna let it affect the rest of life because of it. I would say I am close to the perfectionist but I am not that.

KN: Bob, thank you very much.

BW: Thank you for having me. I hope you find it helpful.

KN: Yes, absolutely.

BW: Make sure you send it to me so I can hear it.

KN: Yeah, I will. Absolutely.

DM: By the way, we would like to invite you, if you have time, if you would like to come, you’re always welcomed in SQLDay in Wroclaw.

BW: That’s so nice of you. Thank you, you’re so kind. Thank you for an invitation, I appreciate it. Thank you for your time.

DM: Thank you

BW: Thank you Damian.

Find Bob Ward:

Bob’s blog posts: SQL Server According to Bob
Bob Ward’s Twitter: @bobwardms