I’ve been producing Tech Outbound SQL Cruise events since 2010. These are week-long, immersive training events covering the complete Microsoft Data Platform: Azure, Machine Learning, AI, Power BI and, of course, enterpriseSQL Server. The model is simple: instructor-led training while at sea; exploration and fun while in port. We sprinkle in a healthy dose of free consulting by those joining us as Technical Leads during“Office Hours.” We structure the week so that there are no lines between “attendee” and “trainer.” That extends to class time. The sessions are instructor-led, but they are treated as a discussion because one thing I’ve learned over my 20 years as a data professional (and 20 years as a parent) is that no one has all the answers, and everyone brings something to the table.
We had such a great group of Technical Leads aboard the last SQL Cruise that I wanted to invite them back together to catch up and also get their opinions on the state ofcloud adoption, T-SQL enhancements, the evolution of enterprise SQL Server performance tuning and mentorship, among other topics. Joining me in this discussion were Jes Borland, Premier Field Engineer at Microsoft; Itzik Ben-Gan, T-SQL Trainer at SolidQ; Kevin Kline, Principal Program Manager at SentryOne; and Jason Hall, Vice President of Product at SentryOne. (Buck Woody, Applied Data Scientist (among other responsibilities) at Microsoft was unavailable for our discussion, but I will be sitting down with him soon to discuss data science, AI and the importance of mentoring for a separate article.)
Tim Ford:Thank you for joining me today for this talk. Can you each take a second to tell readers a bit about yourself?
Itzik Ben-Gan:I’m with SolidQ. I spend most of my time traveling around the world teaching about T-SQL. I’ve been working with SQL and databases since the early '90s.
Jes Borland:I've been working with SQL Server for over 10 years. I started as a report writer, working with SSRS in SQL Server 2005. I transitioned to DBA work, then got into consulting, and now I'm a Premier Field Engineer for Microsoft. I love working with a variety of customers in different industries, on different environments, with different challenges. My main focus area is performance tuning, in both on-premises and Azure SQL Database. I've enjoyed watching the SQL database mature over the last five years, from v11 with limited functionality to the recently introduced Managed Instances.
Kevin Kline:I started my IT career back in the ‘80s working on Oracle running on VAX/VMS and mainframe hardware using character-mode front-ends. In the early ‘90s I wrote my first book on Oracle development products for the first generation of OSes that had GUIs. I was part of the IT team that did beta testing on SQL Server when it was being ported to windows NT. Since NT wasn’t GA, we had to do our beta testing on servers using OS/2 Warp, an IBM operating system. I’ve been intensely focused on SQL Server ever since then.
Jason Hall:I have a long history with SQL Server. It started with SQL Server 6.5, when I was working as a systems admin and I inherited a financial application using SQL Server as a data store. It did not take long to reach the conclusion that the database was the backbone of the system. If the experience was poor for end users, it could usually be traced back to database performance. That was in 1998, and I've been on just about every side of the database server in some capacity ever since, until I landed at SentryOne (then SQL Sentry) in 2006.
Ford:Let’s kick this off with talking about performance monitoring. Are you seeing differences between the problems that used to be most prevalent versus what the landscape looks like today? Would you agree that solid-state drives, virtualization, hardware costs and cloud have made a considerable impact on alleviating some problems but surfacing others?
Kline:In the old pre-cloud days, the first emphasis of tuning in enterprise-class estates was heaviest on tuning server hardware and the I/O subsystem. Once hardware and I/O was ruled out, we’d then move on to diving deeper on database design, indexing and application code, including the SQL code. Hardware and I/O tuning have diminished in the last several years as virtualization, SANs and SSDs have grown increasingly common, and that doesn’t even touch on cloud providers. But a consistent set of bottlenecks in any scenario is the quality of database design, indexing and application code, like SQL tuning. In fact, with Azure and AWS, poorly performing SQL code and poor design cost you more money, since you pay for CPU, storage and data ingress/egress.
Hall:“I've seen this in three primary ways, and in this order:
Virtualization: Nearly everything is virtual now, and lots of data pros do not have the window they need to see into performance at the virtual layer. Many are getting there, but we have some distance to travel in this area still. Flash storage: Flash arrays are cheaper than ever before and getting cheaper every day. Storage performance and everything related to it are less of a concern than they were 10 years ago. Unfortunately, this is so true in many cases that some forget to still keep an eye on it. While the overall landscape is better, storage performance is still something we need to consider. Cloud and PaaS: Many firms are moving toward cloud and PaaS. This abstracts away many things we're used to dealing with. The things we monitor, and the levers we have to improve performance, have shifted more toward the code we write and the strategies we can use inside the database rather than tuning at the service and hardware level.Ford:That ties into my next question: What tends to be the differences you’re hearing about from clients or peers when it comes to performance tuning issues in cloud environments versus those on-premise?
Hall:In the cloud, we have less control over available resources and how they are used. The focus shifts largely to queries and indexing. In addition, with migration being such a big factor, for the next few years, we'll see an increasing need to understand what to migrate to and how to migrate and manage the spend once we're there.
Borland:The biggest differences I see are in automation and monitoring. The tools for automation at many enterprises are very ingrained and automate across a wide variety of products. In Azure. There is a new set of tools, and there can be a learning curve. There also can be challenges getting existing monitoring tools to work or finding new ones to provide the same functionality. Both of these items involve change, which is difficult, but can be overcome.
Kline:There is definitely a paradigm shift occurring. However, the thing I always encounter is that the basics of our business as data professionals are still hard. So many new people come into our industry with little or no practical training or experiences. As a consequence, we never run out of poor database designs to amend, bad indexing to adjust and oh-so-much SQL code to tune. These aspects of performance tuning will always be true in the cloud, as well as in on-premise applications.
Ford:Let’s talk cloud adoption since the conversation has moved that way. Do you think the shift toward cloud is finally accelerating? Have we hit the tipping point?
Borland:I think more companies are seeing the benefits of cloud services and understanding that the infrastructure and security are solid. I don't see companies going all-in to cloud services, but many aspects of it can solve business problems. A hybrid approach is best for most organizations. There are more tools to transit