I have this query that's running too slow. I'm not sure what all info I should provide in order to make it easy for you to help me, but I'll take a stab at it and then add more when you big-brains inevitably ask for stuff that I either didn't think to include or don't know what is.
I want to identify customers (but using only part of their address -- accommodating households and businesses) who first registered a purchase in 2006.
My first attempt was:
select distinct a.line1 + '|' + substring(a.zip,1,5) from registrations r join customers c on r.custID = c.id join addresses a on c.addressID = a.id where year(r.purchaseDate) = 2006 and a.line1 + '|' + substring(a.zip,1,5) not in ( select distinct a.line1 + '|' + substring(a.zip,1,5) from registrations r join customers c on r.custID = c.id join addresses a on c.addressID = a.id where year(r.purchaseDate) < 2006 )and when it was running too long, I switched a NOT EXISTS (with which I am less comfortable, but willing to experiment) like
select distinct a.line1 + '|' + substring(a.zip,1,5) from registrations r join customers c on r.custID = c.id join addresses a on c.addressID = a.id where year(r.purchaseDate) = 2006 and not exists ( select 1 from registrations r join customers c on r.custID = c.id join addresses ia on c.addressID = ia.id where ia.line1 + '|' + substring(ia.zip,1,5) = a.line1 + '|' + substring(a.zip,1,5) and year(r.purchaseDate) < 2006 ) group by a.line1 + '|' + substring(a.zip,1,5)But it runs too long too. Like no results in 17 hours kind of too long. I think the first thing to consider is where my SQL might be wrong or sub-optimal but in case it's not that, I want to also give you enough info to consider the environment.
So, diagnostic info. You probably don't care, but just in case: it's running on a G6 server with four quads and 20 GB RAM; each query is limited to occupying four processors to keep performance up for requests from the web server; when I'm running this query, we're clearing off other big imports and reports because of deadlocks but the web server is customer-facing and can't be stopped.) There are roughly: 15 million registrations, 11 million customers and 8.6 million addresses. I rebuilt all of the indexes just to be sure that fragmentation wasn't the problem. However, I'm not really sure how to index properly, so I'm totally open to this being a problem -- some of these indexes were a result of me futzing around and some were scripts that one of the MS analysis tools gave me to improve performance. I'm also not exactly sure how to convey index info to you, so I'll just give the create scripts:
ALTER TABLE [dbo].[registrations] ADD CONSTRAINT [PK_flatRegistrations_1] PRIMARY KEY CLUSTERED ( [Id] ASC )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY].
ALTER TABLE [dbo].[customers] ADD CONSTRAINT [PK_flatCustomers_1] PRIMARY KEY CLUSTERED ( [Id] ASC )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY].
ALTER TABLE [dbo].[addresses] ADD CONSTRAINT [PK_addresses] PRIMARY KEY CLUSTERED ( [ID] ASC )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY] CREATE NONCLUSTERED INDEX [addresses] ON [dbo].[addresses] ( [line1] ASC, [line2] ASC, [city] ASC, [state] ASC, [zip] ASC )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY] CREATE NONCLUSTERED INDEX [deliverable] ON [dbo].[addresses] ( [addressDeliverable] ASC )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY] CREATE NONCLUSTERED INDEX [_dta_index_addresses_5_1543676547__K9_K1_6] ON [dbo].[addresses] ( [addressDeliverable] ASC, [ID] ASC ) INCLUDE ( [zip]) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY] CREATE NONCLUSTERED INDEX [_dta_index_addresses_5_1543676547__K1_K9_6] ON [dbo].[addresses] ( [ID] ASC, [addressDeliverable] ASC ) INCLUDE ( [zip]) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY] CREATE NONCLUSTERED INDEX [_dta_index_addresses_5_1543676547__K1_6] ON [dbo].[addresses] ( [ID] ASC ) INCLUDE ( [zip]) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]Thanks a bunch for your time!
I think your Not Exists subquery's table alias is wrong. Try this:
select r.custID, a.line1 + '|' + substring(a.zip,1,5) from registrations r join customers c on r.custID = c.id join addresses a on c.addressID = a.id where r.purchaseDate between '2006-01-01' and '2006-12-31' and not exists ( select 1 from registrations ir join customers ic on ir.custID = ic.id join addresses ia on ic.addressID = ia.id where ia.line1 = a.line1 and substring(ia.zip,1,5) = substring(a.zip,1,5) and ir.purchaseDate < '2006-12-31' )