I’m glad you asked. Before I answer, I have a question for you: Have you ever been told by a developer, “It works on my machine.”? Do you know why it works on the developer’s machine?
Defaults.
Developers build software that contains configurations information. In the first iteration of building the software, most developers do no externalize or parameterize all configuration values. Instead, they hard-code these values into the application, usually as variable defaults.
I can hear you thinking…
“But This Is SSIS, Andy”And? Oh, you ask because SSIS stands for “SQL Server Integration Services.” I get that. It’s important you understand this:
SSIS development is software development.SQL Server Integration Services suffers somewhat from its name which contains the name of a popular relational database engine (SQL Server). But SSIS is not a database, and SSIS development is not database development (and most definitely not database administration).
SSIS development is software development.
Software Best Practices ApplyBecause SSIS development is softwaredevelopment, software development best practices apply. For example, SSIS developers should source-control and test their SSIS code.
Perhaps testing is that thing you do in Production right after deployment.I have this pithy saying:
While I freely admit this statement is pithy, it is also true. The remainder of this quote is even pithier: “Some software is tested by your soon-to-be-former largest customer.”
I don’t want you to lose customers especially your largest customer. That’s why I advocate you test your software by developing it on one Data Integration Lifecycle Management or DILM tier and then promoting the code to at least two other DILM tiers before deploying to the Production tier.
Tiers. Or Tears. Your Call.So which tiers do I recommend? I’m glad you asked! i recommend:
Development Test QA Production DevelopmentYou must build the software somewhere. Wherever you build the software is your development environment.
Now let’s take a step back and describe the development environment for an SSIS developer. An SSIS development DILM tier shares some characteristics. In Dev, SSIS developers can:
Deploy SSIS projects and packages Overwrite existing SSIS projects and packages Delete existing SSIS projects and packages Execute SSIS packages Manage (deploy, overwrite, delete) SSIS configurations Manage (deploy, overwrite, delete) SSIS metadata Create databases, tables, and filesThe most important part of the SSIS Development DILM tier?SSIS developers can perform each action listed above without help from anyone else in the enterprise.
I can hear you thinking, “What if the SSIS developers blow something up, Andy? Huh? HUH?!?”Let me ask another question: What happens when things go awry or amiss in Production? Isn’t the answer to that question, “I fix it.”? If that’s true in Production…
Then you fix it.
In Dev.
In fact (and I know this is a crazy thought) you could actually use a more-open Development tier to test and perhaps, even improve break-fix procedures for Production. After all, developers are notoriously excellent testers (I write this as a developer: I can break things like nobody’s business!).
What if the Dev tier becomes a place for you to test your recovery procedures for Production? If your HA/DR procedures work in Development where developers have too much access, then they should work in Production where the environment has been locked down. Am I correct?
Over the years I have found myself making this statement to enterprises: “You have an environment named Development but it is not a Development environment.”How do you know it’s a Development environment? If the developers can destroy it, it’s Dev.
One last thing before I kick this soapbox back under my desk. Consider that developers will be inconvenienced by the destruction of their Dev environment. It may slow them down. Heck, it may cost them a deadline. If that happens, you can take the opportunity to educate them, sharing what happened and why you believe it happened. Maybe they won’t do the dumb thing that burned down Dev again… Maybe.
Would having developers that better understand databases make your job a little easier in the long run?
TestOnce the software works on the SSIS developer’s machine, they need another environment to which they can deploy the SSIS project. Why? Remember those defaults? SSIS packages will always execute in Dev because default values are aimed at local resources for which the SSIS developer has authorization and access.
Which resources? Required access (CRUD create, read, update, delete) to files and directories, databases, servers, etc. No SSIS developer would claim an SSIS package is ready unless and until she or he successfully executes the package in the SQL Server Data Tools (SSDT) debugger. But a successful SSDT debugger execution does not a completed SSIS package make.
To suss out missing configurations (externalization, parameterization), one needs to move the software to another lifecycle management tier and execute it there. Only then can the developer be sure they’ve externalized required defaults.
In Test, SSIS developers can:
Deploy SSIS projects and packages Overwrite existing SSIS projects and packages Delete existing SSIS projects and packages Execute SSIS packages Manage (deploy, overwrite, delete) SSIS configurations Manage (deploy, overwrite, delete) SSIS metadata Create databases, tables, and filesI can hear you thinking, “But Andy, this is the same list of permissions as Dev!”
Yes.
This part of the work must be completed. As professionals, we get to decide when and where the work gets done. Remember, it can always be done in Production. And it will be done in Production unless we are intentional about doing this work on another DILM tier.
Someone needs to do this part of the work. Do you want to do it? Or would you rather have the person most familiar with the code the SSIS developer do it? They are not going to catch everything that needs to be externalized, either; not on this pass. But the Pareto Principle informs us that they will catch 80% of the missing externalization.
Deployment to the DILM Test tier serves as the first deployment test. I want the SSIS developers bundling up SSIS deployments. Why? It’s good practice and practice makes perfect.
On Estimation…One last point regarding software testing: When estimating a software project I begin by estimating how much time it will take to build the software. After this step the calculation for how much time is required to test the software is really easy: It’s the same amount of time required to develop the software.
Andy, are you telling me it takes just as long to test software as to develop it?
Yep.
“Cutting Testing to Save Time” You can change this relationship by “cutting testing to save time.” But you can never shorten testing, only lengthen it. Decades of software development experience inform me that the costs of testing software increase by an order of magnitude for each tier of testing. The most expensive environment i