Securing SQL Server in EC2: certificates, ports and gotchas

One thing you know you should never, ever do is expose a database server to the internet, right? It’s scary what willhappen to that machine. You’re sureit’ll be pwned in seconds. So, you resist until the client demands you do it.

“OK, ok…” you say, exhaustedlydeferring to the client, “If we mustexposea DBMS to the internet, let’s add some protection by moving it from its default listening ports and by forcing encryption between clients andthe DBMS.”

This post detailsabout a few thingsI learned whiledoing just thatwith Microsoft SQL Server in an AWS EC2 instance. The objective was simple: createa SQL Server “data mart”fed production data via SQL Server replicationfrom DBMS instances in private VPC subnets. The replicated data, exposed on the internet, is used for creating reports viaSQL Server Reporting Services (SSRS). Allof what I talk about here is, of course, documented and Google-able. But I hope by collecting some pointersin one place with a few representativescreenshots, it’ll help you get to success faster and save you a little of the head-scratching I experienced.

One of the first things you do when you have a child is name her. Thinking about names ― in this case DNS names ― is the first thing you should do when setting up any EC2 instance. Create a naming convention ― and stick with it.

I’m a fan ofRoute 53 private DNSwhich, while imperfect, allows you to have both an “internal” name for an EC2 instance and apublic DNSname on different IP addresses. My convention is that a Route 53 privatehosted zoneresolvesrequests in the .aws domain to the instance’s private address in the subnet and the same instance’s Route 53 .com entry resolves to the Elastic IP (EIP) address of the instance in a public VPC subnet. As you can see in the screen shot below from Route 53, I havetwo zones ― one private and the other public ― that arereferring to the same instance with an internal address and an external address. I won’t go into the VPC routing and/or firewall implications here except to say that you have to set up VPC route tables to permit traffic between the public and private subnets and you must also permit inbound traffic in the windows firewall in the EC2 instance.

Route 53 private zones make it convenient to have a public and a private DNS entry (click to enlarge)

(Images in this post are taken from a client’s AWS account so I’ve obscured identifyinginformation. You should be able to figure out what’s going on from what’s not been blurred.)

The most important takeaway about DNS for SQL Server in EC2is thatsince you want toencrypt allcommunications between the EC2 instance and clients on the internet, you are going to have to install a cert directly into SQL Server . And since clients will includebrowsers using a variety of connections to the DBMS, a self-signed cert is off the table. A self-signed cert is just too much work and too error-prone.

That meansyou need a high-quality cert from a CA whose root certificates are already configured into commonbrowsers (Firefox) and OSs (Windows, macOS for Chrome, Edge, IE and Safari). Save yourself a ton of work ― just go get a cert from DigiCert . These guys areexpert, helpful and responsive. And their certificates are competitively priced.

Selecting names and deciding where to get a certificate are just the beginning of what I found to be a very complex certificateinstallation process. The documentation isaccurate but unhelpful without screenshots or examples. There are some nuances not discussed in the documentationthat will help you set up youcert correctly in SQL Server. Here are some tips:

Forget about ― and I mean don’t even think about― usinga wildcard cert for SQL Server. That is, unless you have a DigitCert wildcard cert. DigiCert wildcardcerts come with unlimited reissues ― and you can reissue your wildcard for a specific FQDN (fully qualified domain name) for free. You have to ask DigiCert todo it for you, however. You need a high-qualitycert with the correct FQDN because… The cert must match the FQDN of the SQL Server instance exactly .That is, the FQDN must be in the “Subject” field of the cert. If you tryto install a cert with the FQDN in the “SAN” (Subject Alternative Name) field of the cert, you will fail. SQL Server will never recognize it.Full stop.

Cert for SQL Server must match FQDN in subject field (click to enlarge) Make certain that you create a certificate signing request (CSR) on the target EC2 instance. That’s because Windows does its best to make the private key inaccessible (just try looking for it in the registry or the file system!). DigiCert makes this easy with a utility you can download and run on Windows. Once you obtain your cert and install it in the machine store, you can export the installed certas a .pfx file with the private key . The .pfx file can be used on a replacement EC2 instance, should you need to do so. Here’s a screen shot showing where to select exporting the private key in the cert export wizard:

Export cert on Windows with private key (click to enlarge) Here’s a really obscure requirement: you must give the SQL Server process ( mssqlSERVER

) permission to access the cert’s private key. I’ve never needed to do this for any other app, IIS included. But the requirement is documentedand if you fail to grant access, SQL Server won’t be able to usethe cert. This setting is buried in the Windows Server UI and I couldn’t find an easy way to do this in PowerShell. Here’s where to change the permissions on the cert’s private key:

Manage permissions for private keys in Windows certificates (click to enlarge)

And here’s the service to grant permission to:

Grant SQL Server permission to access a private key (click to enlarge) Finally, after all this work, turn on force encryption in SQL Server Configuration Manager so that all connections are secure:

Turn on SQL Server encryption (click to enlarge)

So, with all these certificate caveats and gotchas, what does success look like? Two things. First, you’ll be able to select your cert in the SQL Server Configuration Manager drop-down on the certificates tab. Second, if you take a look at the SQL Server startup log, you’ll see that the cert’s hash (thumbprint) matches the cert in the computer’s certificate store:

Configure certificate in SQL Server Configuration Manger (click to enlarge)

Check hash of certificate in SQL Server log (click to enlarge) By comparison, moving SQL Server off of its default TCP listening