ASF 014: Itzik Ben-Gan interview

Introduction

Itzik Ben-Gan is a Mentor and Co-Founder of SolidQ. A Microsoft Data Platform MVP (Most Valuable Professional) since 1999, Itzik has delivered numerous training events around the world focused on T-SQL Querying, Query Tuning and Programming. Itzik is the author of several books including Microsoft SQL Server High-Performance T-SQL Using Window Functions, T-SQL Fundamentals and T-SQL Querying. He has written articles for SQL Server Pro, SolidQ Journal and MSDN. Itzik’s speaking activities include SQLPASS, SQLBits, SQLTeach and various user groups around the world. Itzik is the author of SolidQ’s Advanced T-SQL Querying, Programming and Tuning and T-SQL Fundamentals courses along with being a primary resource within the company for their T-SQL related activities.

This talk has taken place during SQLDay conference in Wroclaw (Poland), on 16th May 2018 (Wednesday).
Interviewers: Kamil Nowinski & Damian Widera.

Do you know what Itzik does like doing the most? What kind of thing is important when writing a book? Why worth to be at conferences and also when he has started working with Microsoft SQL Server databases? Find out more in this talk.

Transcript

Damian Widera: We did already the introduction so you know what it will be all about. I mean, in a nutshell. Would you like to introduce yourself to our listeners? So, who are you? Where are you from? What are you doing?

Itzik Ben-Gan: Sure. So. Well, first of all, I have a hard time answering where I’m from because I was born in Georgia. When I was one year old, my parents took me to Israel. And then about eight years ago my wife and I and our two dogs we moved to the US.

DW: So you are very international right now.

IBG: So if the question is where do you live? It’s in Redmond Washington in the Seattle area. And my name is Itzik Ben-Gan but I guess you said it’s in the introduction.

DW: Yes.

IBG: I’m with SolidQ, I’m one of the founders of the company. We started back in 2002, at the time we were six people. And the thinking was first of all people who kept meeting each other in conferences and events like this and like each other and like working with each other, and we all like SQL. And we thought that together we could join forces and try to cover more than each of us could do alone. It worked quite well. So, I’m still part of the company and my main focus is T-SQL. I did a lot of things related to databases. I started with them around the early 90s, 91 or so. But at some point, I started focusing more on T-SQL and Query Tuning. So pretty much anything that has to do with the performance of the code. So internal structures, query plans, writing efficient code. It’s not just about the range of the physical structure. You often have different ways to write the code that under different conditions each will tend to work better. So that’s pretty much what I focus on, and I still like it very much. I think I’m fortunate in the way that I can do what I like and not everyone can do this. So mostly teaching. So it’s T-SQL and mostly teaching, that’s very much what I focus on.

DW: And do you like teaching, spending time with people? Because teaching can be pretty exhaustive because you have to spend like two, three, sometimes five days with a group, depending on how long the training is.

IBG: Yes, so teaching is what I really love doing. So all the rest I do just to actually enable the teaching. And I know that for different people there’s different things that they like doing that make them happy. For me that’s this part. Actually for me it’s much easier than anything else. I talk about what I like, I have people that like listening to it, which is also nice.

Kamil Nowinski: And then it’s much easier to do that.

IBG: And it’s also the area that I like focusing on. So yes, I really love it.

DW: And how many books have you written because I have a lot of them. And we still have a bookstore in the venue and there are a lot of your books about T-SQL, windowing functions. The windowing functions is really my favourite one. It’s not that big but it’s very condensed.

KN: And from my perspective, you’re also focusing on this windows functions. Very often I’ve seen that you talk about that subject.

IBG: Yes. First of all the number, I believe it’s around 10-11-12, something in that area. The window functions specifically is also my favourite I have to say. And the reason that topic is so interesting and that I focus on it so much is that the original intention behind getting it was kind of a specific type of calculation which is data analysis. Doing things like aggregates and so on. But in a more, I would say sophisticated way. Just a very different way than using group queries using subqueries, those kinds of things. It has some kind of ingenious ideas about it. You group data, the grouping causes you to lose the detail. And often you need to mix, when we do calculations like percent of the current value out of a group total and stuff like this. You don’t want to lose the detail information. And with subqueries, you get an independent view of the data, whereas with windowing you apply the logic after all of the query elements have already been processed, not an independent view of the data. And that’s usually what you want. So it’s a more sophisticated tool but I think what’s interesting about it is that you can use it not just for the original types of problems it was designed to solve. That with time that you work with it more, you discover you can do much more generalized things. And it’s as if nowadays you have these two kinds of problems in SQL. Not problems, I would say for almost any kind of problem I should say. There are two kinds of solutions. As if you have this fork: one is the classic predicate-based solutions that use subqueries, that use JOINs with predicates and the ON clause / WHERE clause. But often it’s more complicated for people to think in those terms and it turns out to be far more expensive in many cases in terms of how the SQL Server engine ends up optimizing. And you can find all kinds of creative ways to use window functions to solve those problems way more efficiently. So areas you would not expect to see them, you know temporal data. When you need to deal with intervals, doing techniques like pecking intervals, finding the maximum number of concurrent intervals. You wouldn’t think initially there’s anything window functions have to do with it. So nowadays, I think of problems typically as: do we solve it with a predicate approach or do a window function approach. So it seems to people like a narrow area is much more general than people realize before they start using them. And that specific book, I wrote it back for 2012, when the major editions like the window frame was added or the leg and lead and statistical functions. They make it much more powerful and I’m actually waiting for the next SQL Server version, what they call vNext now, and we don’t know when it will be released to update. That’s the next book I want to update. But it is a very specific feature I’m waiting for in order to update the book.

KN: I wanted to ask if Microsoft is extending those functions?

IBG: So that they didn’t extend the functionality. So since 2012 they didn’t add new kinds, even though they are still a whole bunch of very, very interesting features in the SQL standard and that you do see in some other platforms that were not yet implemented in SQL Server. But the 2016 version is when they added this new optimisation of window function using this batch mode, window aggregate operator.

KN: Exactly. It’s a huge difference.

IBG: It’s an unbelievable difference in performance for the cases that satisfy the requirements for using it. But anyway, it’s amazingly fast. It’s just that it uses a batch mode kind of processing, where instead of processing one row at a time, it will process one chunk of rows at a time, a chunk of 900 rows. But currently, there is a condition in the engine that the batch processing can only be used if this columnstore index is present. Even if it could be very relevant if the data is pulled from row store index, and in fact if you’re able to create a supporting index, with a B-tree-based row store index you could avoid the sorting that is necessary to use the batch mode. Sorry, if you use columnstore, that will then enable a batch mode.

KN: A very expensive operation.

IBG: Yeah. So you have a regular columnstore index, you either need to pull it more compressed but then sort it and then have the window calculation done in batch mode or pull it from row store, already presorted but then use the row mode. Well, apparently if you create some kind of a columnstore index, even if it’s not used, the optimizer can already enable the batch processing despite the fact that it’s actually pulling the data from row store, as long as a columnstore index is present.

DW: It’s just like a trick or workaround.

IBG: Yes it’s like a trick but you can get to so many people and explain to them: look, now you can actually get it, but you have to create this index even if it’s a meaningless filtered index. But you need to do it and many people will not hear about this or not know about this. Obviously it’s better if in the engine they remove the connection between the presence of columnstore and the use of batch mode. And in the last press conference, Microsoft announced already a number of… they publicly announced a few things that they are planning, what they call vNext and one of the things they were talking about was the removal of the dependency. So for all those systems that will not need to use any tricks or create any special indexes, it will be kind of available out of the box. And I think it will be a huge performance difference, without doing any code changes or creation of any objects.

DW: We don’t change the code but they have to change. And that might be complicated, I believe. It might not be a simple disconnection in code but…

IBG: Well, I think the reason they didn’t make it available already, they knew obviously that ultimately the plan is to have batch processing not tied only to columnstore. They obviously had this in mind. Because inherently it’s not supposed to be this strong connection. Yeah, the arrangement of columnstore data lends itself to building batches but no one says that pulling it from row store building batches and getting all the benefits shouldn’t really be an option. So I’m certain they had in mind this scenario for the long term but they wanted to do regression tests and make sure that there are no all kinds of cases where the caustic formulas are such that when it shouldn’t really use the batch processing, it does use it. So I’m guessing they wanted a much more stable environment. And currently when we use this trick, it could be that there are here and there cases where it didn’t make sense for the optimizing to use it but it uses it, I don’t know. But yeah, I’m very much looking forward to the next version because I think it will be kind of a big performance boost with no need to create objects and no need to change the code for those queries using window functions and many others. I mean it’s just a basic ‘group by an aggregate’, where currently most of your workload doesn’t really benefit from columnstore. But now those kinds of queries that do, just basic aggregates that do sorting, they do a lot of kinds of activities that just run much, much faster in batch mode. And many other improvements that they make. Almost anything that they do in the optimizer, they invest mainly in batch processing. And either no investment at all in the row mode or they do very little. So if you want to enable those new features, you need the removal of this connection. You probably heard of the adaptive query processing.

DW: Yes, of course.

IBG: It is another good example that if you take that the adaptive join, where you have a situation where in some conditions the optimizer needs to use a loop join, typically when you have a small number of rows involved in another small table. If you have a larger number, it needs to use a hash join. But then you have those parameter sensitive queries where you use a parameter in the store procedure and the first execution determines the kind of strategy it chooses and caches the plan and now we’re stuck with this one. Even if there are many executions with values that would need the other plan.

DW: And it seems to be a typical problem still.

KN: Yeah, a very popular problem with parameter sniffing.

IBG: What people end up doing is they end up actually using a recompile option to force the system to create a new plan every time. And this very smart group of operators did it in 2017, called adaptive query processing. One of them is this kind of smart join where the outer branch is identical for both loop and hash but there is one inner branch for the hash and another inner branch for the loop and then it compiles this threshold. And if the number of rows involved is below the threshold, it activates the hash branch, actually it’s the loop branch, below, and then on or above it activates the hash branch. And then you don’t have to force a recompile so it is able to cache the plan with this more dynamic operator and activate the right branch during execution. The tricky part is what does it need in order to activate it? Batch processing. This is an operator that can only walk on the batch more so if you didn’t have the columnstore index, it’s not going to consider it.

KN: Yes.

IBG: So you realize that this can only make sense obviously to remove the connection.

DW: But it looks like maybe for people who just write a query, it’s like “oh, it’s a simple query” but as we discussed, they might not know how complicated the query optimizer is and how many decisions it has to take every single query to optimize even a very simple statement.

KN: Yeah, at the beginning it looks like a very simple process but internally it’s very long let’s say and very complicated.

DW: And you can still be fast. Good enough, not the best one, but very, very fast so the compilation is no longer than the query actually. It could be looking for the very, very best plan forever.

IBG: This is interesting the point you mentioned, because they could a lot more rules to the optimizer that don’t require too sophisticated additions. It’s just that they would make the optimization process even longer and counterproductive. So a good example where people often fall into this kind of trap is with the search arguments. Where if you do some kind of manipulation, like there’s this common one where the input for a procedure is a parameter but where the values can be either non-nulls or sometimes the input could be null, like give me the orders with the ship date is the specific one but all orders were not shipped yet will have a null ship date. And you want to support a null prompter. So in order for the filter to work, people often apply calculation like isnull on the parameter and the column, and replace with something else that cannot appear in the data, but the same in both sides, and then you get the correct treatment but you lose the searchability of the filter, because you manipulate the column. So this example or when you say “give me the rows where the year of the date column is equal to some year”, you lose searchability and then it does a full scan. And people often don’t realize this and then when you explain to them, “well you know, if you change it to not doing manipulation but with the same meaning, like column equals parameter or column is null and parameter is null, it’s a searchable kind of a filter and it can use the index. And then they say “well, couldn’t Microsoft add this to the optimizer and not ask to do the rewrite?” And they don’t always realize that yes, they could but then if you overwhelm the optimizer with too many rules, the optimization process like you said could become longer and even longer than it takes for the query to run.

KN: Longer and more generic and that could lead to much more problems.

IBG: And Microsoft I’m sure intentionally didn’t put it in the optimizer. Rather they said: “if we can give it as a best practice to people then we can pull all this logic out of the optimizer and then people will just follow: here is the pattern that is recommended to use when you write the filter”. That is true. But what I think is interesting in this kind of whole world is that both the optimization part is very kind of deep and there are all kinds of levels you could go into. You have people like Paul White, who knows the optimizer to such levels that very few people in the world manage to reach. But then there are many other people that do get into some level but decide at this point we stop. So there are different levels and the same I can say about the SQL itself. That you start working with it, you get kind of a false impression that it’s an easy, simple kind of language, probably a week into the process of learning SQL you already have queries with joins and where and group by, and you feel very confident, and it’s after 10 years that you start realizing… well, maybe I’m exaggerating with 10 years but the more you work with it, you realize there’s a lot of depth there and you can dedicate your life to this topic.

DW: The more queries you write, the more you’ll see that it’s not that easy. And when I start training with people in T-SQL, they are always like: “oh, it will be simple. What are we going to do for the next three days? It’s just a select”. And then we go deeper and deeper and deeper, and it’s not a very deep level and they see, “oh it’s not that easy”.

IBG: And more questions after three days, I guess.

KN: So we are going in the direction that the world generally works with more and more data, that’s why we have columnstore indexes, that’s why we should at the beginning focus from the more analytical point of view when we are looking at the data. So do you think that in the nearest future we’ll be talking more about the columnstore index because it’s connected like we said?

IBG: Well, first of all for sure columnstore index will play a big role in analytics, batch processing, specifically the optimization part, not just with columnstore, but this will be I think just a small part. But what you’re completely right is that it’s more and more data, it’s obvious to everyone. And by the way there is this author called Yuval Noah Harari, he’s a historian, teaches history at a university and he wrote two books, one about Sapiens and the other one he calls Homo Deos. So the first is just the way homo sapiens kind of evolved. But the second one, he talks about where he sees the future going, very kind of original type of thinking. So strongly recommended, but he talks about data as becoming kind of… I know for some people it would sound a bit painful but it kind of a religion in a way. That everything is around data more and more, more of our decisions are around data. And take data together with AI algorithms and so on, at some point those algorithms will know humans better than we know ourselves, because they can capture a lot more data and process it than we can. Pretty fascinating but yes, certainly that’s what you see, it’s more and more critical part of decision making. So we are in the right place.

DW: Somewhere in the middle maybe, when we see a lot of data but not that much of data. But if you think that maybe right now… I’ve seen a presentation about big data in Twitter, in a minute there are like 600 millions of tweets every minute. This is impressive. Every minute, every day.

KN: If you want to follow some special tags, hashtags and if you put something like machine learning but especially big data, it’s a lot of messages every single second. We can see a lot of your activities not only in the USA but also across the world. You’re travelling a lot. I remember that I met you…

IBG: I think it was in London.

KN: I was so surprised that I can see you on that kind of meeting, that kind of local meeting. But then we had an occasion to chat to each other, very quick chat. So I know that you have some customers over there as well, because it’s a big city, so could you tell us how often you travel to Europe or maybe to other parts of the world?

IBG: I moved to the US eight years ago. The main reason was because my work was mainly in the US, but ironically as soon as I moved, I started getting a lot more work in Europe. But like I told you, I grew up in Israel. Now my base is in the US, I travel a lot. I think something like 70-80% of my time I’m travelling. Probably half of it is Europe and half of it is the US. But yeah. So for instance next week, I’m in Sweden but I have a flight home on Thursday, I will spend two days at home, I will be back in Sweden. Then I will have a week in the US and then I will have a week in Nurnberg in Germany, it’s like this. I don’t like the flying part, but if you told me you have to do something else that you like less but you will stay always home, I’d prefer the way it is now. So at least I know that I’m doing what I like doing.

DW: So when do you find time to write another book?

IBG: Well the writing. First of all the writing is not just the point of physically writing. You do it in your head all the time. So I work always with new ideas related to SQL, related to optimization and I process them more in my head. The writing part is more like kind of opening a drawer and pulling one of those things and maybe more polishing it. The writing part does it because you force yourself to make sure that you have all the angles kind of covered and so on. So there is an important part of the writing process but I usually do it every two hours a day. So usually before the working day or after the working day, I will have something like half an hour in the morning and maybe an hour and a half or two hours in the evening, and it sounds like a little bit but every day you do it for half a year and there’s a book. That’s my process.

KN: That’s interesting that it works for you because for most people it’s hard to start something new or focus on something new. It takes time, at least a couple of months for example.

IBG: For me, it’s not a new thing, that’s the thing. There are things I’m working on all the time but it’s just a matter of putting them into words.

KN: Maybe it’s not new. I mean different activities that during the day you just used previously.

IBG: I don’t take vacation because I would normally take vacation if I wanted to just sit and write and do nothing else.

DW: That’s what we do when we wrote one of our books. We had two weeks and during these two weeks, we just wrote it and it was a very intense moment.

IBG: Yes, for some people I imagine that’s a process, for me it’s every day a little bit.

DW: But we were in a group, there were four people and we had to do it then, because it was very hard to synchronize all four people during the entire year, so we spent two weeks just writing a book. That’s how it worked.

KN: The next question must be about your work-life balance. Because I’m interested in what your approach to work-life balance is, how you can manage that.

IBG: My wife and I are very good friends and we don’t have kids, we have two dogs. So obviously it’s easier.

KN: So we already have the answer to our next question.

IBG: But yes, we are very, very good friends and the fact that nowadays you have Skype and things like this, it makes it much easier. She used to travel with me years ago. It was fantastic but now because our dogs are very, very old, there are 15 years old, it’s impossible to take them in a car or close to impossible to put them in a can. So lately she stays with them, but every day we have Skype, we have WhatsApp and stuff like this. I sent messages to my wife this morning. There are no special kind of challenges other than the fact that I’m away. And I would like to be with her and with the dogs but we understand that this is part of life. That’s why for instance if I have two days between two projects in Europe, I will go back home and say “OK, I don’t want to suffer the jet lag and so on” and then I will just stay in Europe.

DW: It would be easier just to move from here to Sweden as you said, because it’s like a two-hour flight from Wrocław.

IBG: Yeah, but you see, if I was a lot at home, then of course, it would have made sense. But since I’m little time at home, then I will go back, spend a couple of days and go back. Maybe I will not change my watch. I will keep my watch on Europe time and be in the US but stay in Europe time. And then I would not need to go through a jetlag, we’ll see.

KN: Maybe that will be the trick. So if you could describe your general activities, what would it be? I mean we already know that there is a lot of training but I think you’re writing some white papers for Microsoft?

IBG: Yeah, there were a few papers I wrote in the past but I maintain a column for close to 20 years now and recently, by the way, I switched the column to sqlperformance.com.

DW: Like last week you posted I think the last article, second part.

IBG: Yes. I just started working with them a couple of months ago. So just my monthly column is now going to be in the SentryOne website. I really like both the company and the group, so being part of people like Aaron Bertrand (T), Paul White (T) and so on who also regularly write there is really amazing. But yeah, I write a monthly column. And by the way, for many people who kind of look for ways to enhance their knowledge and so on, sometimes you pick a topic that you’re working on and you’re writing on it. But sometimes when you know that you have every month a certain column that you need to deliver, you keep processing in your mind all kinds of ideas, like even if it could be a customer, could be a student that presents some kind of question or an idea and in your mind you already know you need to research this further and it becomes an article. So during this process you get to learn a lot. So for people who look for ways to expand their kind of knowledge and keep always fresh, maintaining a monthly column is a fantastic way. So you just don’t want to burn yourself so you don’t want to try and write two blog entries every week. And then OK, you do it for maybe a couple of months and then it’s just impossible for you to continue to produce good quality and deep enough topics.

KN: Let’s start from something frequent but like one month…

IBG: I think it’s better to just ahead to decide on something more realistic that you can keep in the long run. So I’m running it since ninety-nine this column. Looking back, what are the things that for me contributed to the knowledge, one is the column, another is participating in forums, all kinds of back then was newsgroups and so on, where you are you have exposure to lots of different questions on one hand from different kinds of areas and also other people’s ideas. So there is a thread someone asked how do I handle this situation and then you first of all want to work on the task yourself without looking at other people’s solutions but then you come up with one idea, then you look at other people’s ideas and you get exposed to a lot of information and techniques and ideas.

KN: So in which forum can we find you?

IBG: Nowadays, actually I do less forums. I’m saying that in the beginning how I started getting a lot of ideas, it used to be newsgroups and NNTP newsgroups if you remember them, they were so nice. Currently there are MSDN forums, Stack Overflow… Nowadays I do less and I really wish I did more of this even now. But thinking back where I gained a lot of the knowledge it was those things. The column I still maintain of course, like I said every month.

KN: So apart of white papers, books and trainings, do you work for some specific customers, specific projects?

IBG: First of all as a company, I said I’m with SolidQ. The company does a lot more customer work than the training part, is more my personal favorite things. So as much as I can control it, when they ask me what I want to do, I tell them I prefer much more training. As a company, a lot more customer-facing, and field work, some small projects like there’s a query tuning problem or some performance problem, and some very big projects, designing the whole systems and so on. But yeah, we have customers all over the globe. And nowadays it’s a big company with about 200 people. We have some regular customers, some new customers that are coming all the time. So it’s in all fields. I don’t want to name just one, but finances, health, government-related, pretty much everywhere.

DW: And what do you like to do apart from T-SQL and working? Like hiking, swimming?

IBG: I love hiking. I love friends, so this is one of the reasons why it’s so nice to have those conferences, even though I don’t do many conferences. But when people walk in a specific place, they meet their colleagues every day. For me I’m travelling all the time, I mainly see the students but it’s every week a different group. By those conferences I can meet people I know and catch up and so on. But hiking — I love hiking so every year we usually have this kind of long hike, usually in the summer, something like a whole week in the backcountry. And it’s a group somehow that got connected through SQL. You probably know Lubor Kollar, he’s now retired but he was responsible for many very big, important features in SQL Server like partitioning, etc.

DW: I was fortunate to meet him about 10 years ago. And I think it was a PASS conference in Dusseldorf. It was one of his last appearances. Now I see on Facebook he’s retired for some years. He’s with his kids but he’s still very, very active, many peaks taken from the mountains.

IBG: So he started the idea. I think it was 2006. He started this idea of “let’s go do some interesting long hike” and then we did a segment of what’s called the Pacific Crest Trail. This is a very long trail that starts in Mexico and goes to Canada. We did a small piece of it. It was something like 70 miles I think, something like this. But since then every year we have this group that does that hike sometime in Europe, sometimes in the US. So one year we did… in Ireland there is a beautiful hike called the Carry Way. It has this kind of shape like 8, but 110 kilometers. And so I love this and we call it SQL Hike because it’s mostly… I mean, here and there we have people not from the SQL but it just happened because of our work. We meet each other, it’s how we make our friends. Yeah, Boris Baryshnikov (L), maybe you know him. He also did a number of important things in SQL like Resource Governor, he was responsible for the feature and so on. Dejan Sarka (T), Herbert Albert (L)…

KN: Do you need to specially prepare yourself for the speeches for example, if you’re speaking at some conferences?

IBG: Usually you just want to have the first couple of sentences, you want them ready. But I don’t do this a lot of time ahead. So like before we start, maybe a couple of minutes before the session is to start, you want to make sure that you have like a couple of sentences ready. But then nothing special, it kind of flows. And for me, because I’m much more interested in the code part than slides and those things, and I do this regularly all the time, it’s kind of a very natural process. It’s easier for me, for instance, to start showing queries than to speak in English.

DW: So the query language is easier.

IBG: The query language is much easier for me. Because my brain is thinking in such patterns. The English part is still very hard for me because it’s not my native language but the SQL part is because my brain is all day all night thinking in SQL terms, it’s kind of natural. And also, for people I think it’s much better when they see the code developing as opposed to if you just show a piece of code on a slide or just highlight a piece of code in a script and run it, because then it’s very hard for them to see how it evolved. So if I write the queries and explain the thinking during the writing part, then I think it gets absorbed much better. And also, when they will work on the technique themselves, I hope they will think in such terms. Start thinking about the FROM clause, what table is involved, what do you join, and so on.

DW: So how did you start with conferences, when was your first conference?

IBG: First conference I think it was PASS, maybe 2000, maybe 2002. This was like the first big conference. But there were before, through the MVP, what used to be called SQL Server MVPs and now is the Microsoft Data Platform MVPs, there were Summits, and we still have those, with Microsoft. But those are internal conferences, where Microsoft mainly provides information about future plans and stuff like this. But as a speaker, I think the beginning was maybe 2002.

KN: Do you remember your first conference?

IBG: It was PASS Summit. I will never forget it, that’s for sure because I had a very big stage fright at the beginning. The first conference I was supposed to give a lecture, I didn’t know originally, but when I checked the agenda and looked at the organization of the rooms, my lecture was supposed to be in what they call the ballroom 6C. And compared to all the other rooms that looked like small squares in the picture, this took almost the whole page and I realized there’s going to be something like 500 people there. And you know that the fear of public speaking is a big fear for many people and for me it was like this. And I came from Israel, so I was first jetlagged. And with the excitement around I’m going to speak to so many people. I didn’t sleep for three days and I tried all kinds of things, I just literally didn’t sleep for three days. And then when I was supposed to give the lecture, I just went to the hotel I asked for a bucket, and I went to fill it with ice and I filled the sink with ice and water and I put my head inside. Then I took a quadruple coffee from Starbucks and I started the lecture. It went quite well, amazingly but…

KN: At least for the one hour that you needed.

IBG: I was not concerned at all about the topic or being able to show interesting stuff or the demos will not work. My concern was more to be able to start to speak, to be able to breath and not get excited. But since then, it became easier and easier and I still get very excited when I speak, but it’s now kind of a healthy type where it keeps your adrenaline up.

KN: So do you still feel some stress?

IBG: Yes, certainly, I still feel, but I’m saying, I feel that it’s now a healthy kind of stress. If you have less of it and you start becoming it different or something, you would have a different type of lecture and the quality would be lower. So I think now it’s at a good level. But it’s 20 years after.

DW: So now you don’t need to bring the ice and put into the sink.

IBG: Exactly, now I don’t need ice. Still coffee…

DW: But the ice part is no longer needed.

IBG: It was funny, though.

KN: With which version of SQL Server did you start working?

IBG: I started with 6 5. This was my first.

DW: So it’s 90s?

IBG: This was the very early 90s. Wait a minute, early 90s it was still other database platforms. So it’s more mid-90s towards the late 90s with the 6.5, you’re right. So back then I was working in this big beer factory and the database was a DB2 database and AS/400. And also I pulled off data from AS/400 into an access database and did a lot of financial analysis, costing, sales analysis, those kinds of things. And by the way, back then I used to do a lot more kind of all-up work. And I used the original product that Microsoft then acquired to create what nowadays is analysis services in SQL. So back then it’s a company called Panorama, an Israeli company that developed this cube engine and also the viewer that they had, and in that company,  we used their products. So I created cubes. My first cube was beer. Beer-related cube. Then Microsoft did an acquisition of the engine, so that company kept developing the viewer. They still own the viewer but the engine became first OLAP Services and then later they made it Analysis Services.

KN: That’s an interesting part.

IBG: Yeah. And the funny thing is when I left the job, there was a drink party where you say goodbye to everyone, bring your presents and so and, and then my boss told me “but I need another cube”. Just before I left, after we already drank and said hello. “But I really need this one more cube”. So I sat down and had to create another cube before I left.

KN: Interesting. 

DW: I have one more question about… Do you have any advice for people who would like to start, I don’t know, at the conferences and they just fear about how to start talking, how I should prepare?

IBG: It’s hard to say because I guess it’s a bit different for different people, but what I would say is that it’s an important part if you want to get the exposure… For me, for instance, I prefer the classroom setup than the conference but in order to get people in my classes, the way I create the exposure is the public speaking and writing and so on. And they’re all healthy parts of the whole kind of situation that you have. So I would say just get over your fear, pick a topic that you are very comfortable with, and just do it. Maybe to try and do things like what I mentioned, to try and at least tell yourself the first couple of sentences before you start. Picking a topic that you really like because after you get over the first couple of minutes of excitement, then it becomes more natural. So do it. I would say if you’re planning to become part of this community and you want to be able to, especially if you’re independent. If you’re working in a certain job, you don’t need the exposure so much but if you’re independent, you’re consulting or teaching and so on, you need to get your customers and that’s a great way to do it.

DW: And how important is the community? Do you think it’s important?

IBG: Yeah I think first of all the SQL community is a very lively community. And it’s a big part of your knowledge. So first of all you meet each other, you make friendships, you make connections. But then a lot of times when you sit and chat, some of it is just jokes but some of it is ideas. So you don’t always get your SQL ideas from a customer or from students. Sometimes it’s just the colleagues that discuss “hey, I work on this thing and I face this challenge”. Definitely, it’s a way to get to know other people, to get exposed to other ideas, and our reality is that work is such a central part of our lives for many of us, that we make our friends through this kind of environment. So I think the community… both the community and you know, you look at the teams that develop the product and it’s a very open kind of group.

KN: For new ideas?

IBG: No, I’m saying that SQL Server developers, the development team, the Tiger team, the customer advisor team, they’re very, very open. They see the importance of the community and the interaction with them, they’re very accessible. We meet them at conferences, we chat with them. Obviously, they will not tell you every single thing they are planning because they simply can’t, but still, what they can’t talk about, they’re very open, very friendly. So I see it as a very positive kind of community and part of the world.

KN: Currently I think it’s going to Azure basically and in the next couple, maybe several years there will be more and more systems and data pushing to the cloud. So how do you think it’s from your perspective or from the perspective of people who are working with performance, is it an important part? I mean, all those people lose their job or maybe lose the contract or customers?

IBG: Not at all. The way I see it is just like nowadays, do you worry about the electricity that you’re using for your laptop or something like this? Historically, people used their own devices to create energy but at some point, it just doesn’t make any more sense. So you just plug into a socket in the wall and you don’t think too much about electricity. So in a similar way I guess the cloud will be where things will be running and where the data will be stored but then again you will connect to a database. And nowadays it’s already like this. If you connect with your SQL Management Studio, does it feel any different when you connect to a cloud or when you connect to the box product? Not really. And the way I write queries is the same, the way I tune them is the same, the indexing, the query plans… You just don’t need to worry about patching, you don’t worry about many of the things that are related to the hardware itself…

DW: A lot more possibilities because you can connect, let’s say, the bricks together. You have a block like SQL, you have an analytics block, you have reporting blocks, whatever, machine learning blocks, and you can build very good solutions.

IBG: Of course, the people that were very, very focused on hardware for instance, they have less relevance when the product is in the cloud. But for people who deal with the actual code…

KN: From this perspective it would be the same. But in the cloud, there are more automated things like performance, query store, and AI would be engaged in this, so probably there would be less work from the performance tuning.

IBG: Well, from one aspect of performance tuning. So yes, the hardware tuning is not relevant anymore but index tuning for instance, doesn’t really change.

DW: It’s a different environment so different patterns maybe and different things to look at.

IBG: So for me at least, being focused on T-SQL and query tuning, I don’t see any difference. What I like actually about the cloud, even though many companies still don’t use the cloud. And maybe it will happen quickly or not, how they will transition, if they will transition. But for me, the interesting part about the cloud is that now Microsoft… First of all the code base for SQL Server and SQL Database is the same codebase. It’s just that every so often they freeze a build and release it as a box product. But they get a lot more feedback compared to when there was no cloud involved. They have literally millions of databases, they get a lot of telemetry information. The feedback cycle, both in terms of seeing what is used more, what is useless and what are the problems, they manage to deal with those in a much more efficient way. And this results in much better box product because all of the experience there from the cloud, we end up with a better box product as well. And also you start seeing features being released much more frequently, and they usually appear first in the cloud and later in the box products. So even for the purposes of being able to play with a feature and to learn about it, it’s convenient to have access to the cloud. So I see it as a positive result, even for people who don’t use it. And I keep serving. Like when I teach, I check with people how many are using the cloud, and I get out of maybe 12 people, one is using the cloud. It could change. It very likely will change but I’m saying, even if the majority now don’t use it, they enjoy from it, whether they know it or realize it or not. They do gain benefits from it. And eventually, I see it as a service that you won’t need to think about too much. The problem I see nowadays is that there’s a lot of focus that the companies who provide the technology put on the cloud obviously because they want customers to move to the cloud. But the way I see it, it’s kind of inevitable. And the point where we will be completely there is when we will stop talking. It’s when we will be in the cloud but we won’t need to think about it. Just like we don’t keep talking about electricity today, even though we use it all the time.

DW: It will be so natural. The cloud is just there.

IBG: And you will just focus on the technology, not on the infrastructure behind it. That’s the day when we will really be there. As long as you keep hearing companies talking about the cloud, probably we’re not there yet, because they’re trying to get customers to move to the cloud. That’s why they are talking about it so much.

DW: We are in transition somewhere at some point. Some companies still are on the premises but more companies are now trying to think about what and how we can benefit.

KN: This process is ongoing. OK, Itzik. Thank you very much for accepting our invitation.

IBG: Thank you for having me here at the conference, I’m really enjoying it.

DW: Yes, people are really amazed about your sessions. My colleague, who I worked with at the university, told me that he’s literally hypnotized. And he had a positive push and kick to learn something new. He was not aware that this is so fascinating. That a query can be so fascinating. So it was really amazing. So people are really, really thankful for this workshop and the lecture. And they still have one left today.

KN: At the end, because I forgot. Tell us where we can find you, in the Internet for example.

IBG: So, first of all, like I said, I’m with SolidQ. So if you go to solidq.com you can see our company. And my personal stuff like the book related content and T-SQL-related resources that I maintain, I have a personal website that uses the same domain. Just starts with tsql.solidq.com instead of the www.solidq.com. And on Twitter @ItzikBenGan. Every time there’s a new article or a book, I will usually send a tweet.

DW: And of course, SQLPerformance right now?

IBG: SQLPerformance like I said, is the blog. So now it’s a monthly column basically. I don’t think of it as a blog. I started writing it before the term “blog” really existed. Nowadays you call them blogs. So it’s a monthly column. Every month there’s a new idea, usually related to advanced SQL or query tuning topics. And if anyone really wants deep knowledge about T-SQL, then I run regular 5-day classes, so they can just look up in the tsql.solidq.com website and in the courses section, there’s a schedule where I am in the upcoming weeks, and 5 days of just T-SQL and query tuning content.

KN: They can join the class. Great, perfect.

DW: OK, so thank you very much again.

IBG: Thank you.

Useful links:

Itzik’s twitter: @ItzikBenGan
Itzik Ben-Gan on: SolidQ
Itzik’s posts: SQLPerformance & books