I’m not good at naming things. “Biml Express Metadata Framework” is descriptive but the acronym BEMF is difficult to pronounce (for me, at least). Kent Bradshaw wants to name it George . We’ll keep working on a name…
What is the Biml Express Metadata Framework ?
The Biml Express Metadata Framework uses Business Intelligence Markup Language (Biml) to:
Read metadata stored in a SQL Server database. Build SSIS projects that create the target database, schemas, and tables. Build an SSIS project that contains one SSIS package per table, plus a Controller package that executes each table-package.
I demonstrated this solution in the webinar Save Time and Improve SSIS Quality with Biml . It’s free; both the webinar recording and the framework (although registration is required to view the webinar recording). It’s the latest addition to the DILM Suite . The zip file contains a backup of the AdventureWorks2014 database, BimlMetadata database, and the SQL Server Data Tools (SSDT) solution BimlExpressMetadataFramework. You may download the zip file here .
Once open, the SSDT solution appears as shown here:
To begin, please open the text file named README_BimlExpressMetadataFramework.txt found in the Miscellaneous virtual folder:
This file will contain the latest execution notes for the solution:
Generating the Build Destination Database SSIS Package
To generate the SSIS Package named 10_Build_Destination_Databases.dtsx multi-select the Biml files 0_00_BuildConnections.biml and 0_10_BuildDestinationDatabaseMain.biml. Right-click the selected files and click Generate SSIS Packages:
If there are no errors, the Project Connection Manager named __master__AdventureWorks2014_Stage.conmgr and the SSIS Package named 10_Build_Destination_Databases.dtsx are generated:
Generating the Build Destination Schemas SSIS Package
To generate the SSIS Package named 20_Build_Destination_Schemas.dtsx multi-select the Biml files 0_00_BuildConnections.biml and 0_20_BuildDestinationSchemasMain.biml. Right-click the selected files and click Generate SSIS Packages:
If there are no errors, the Project Connection Manager named AdventureWorks2014_Stage.conmgr and the SSIS Package named 20_Build_Destination_Schemas.dtsx are generated:
At any time during this process, you may see a dialog similar to the one shown here:
If you see this dialog, always Select All and click the Commit button.
You may also see this dialog:
If you see this dialog, always click the Yes to All button.
Why? The Biml Express Metadata Framework was not originally developed for Biml Express, the free SSIS package code-generation tool from Varigence . Originally, this framework was built in Mist ( Biml Studio ) and ported to Biml Express. One unfortunate result of the port was some SSIS artifacts are recreated during the process (note: this does not happen in the Commercial version of the Biml Framework, currently sold as an implemented solution available from Enterprise Data & Analytics ).
Generating the Build Destination Tables SSIS PackageTo generate the SSIS Package named 30_Build_Destination_Tables.dtsx multi-select the Biml files 0_00_BuildConnections.biml and 0_30_BuildDestinationTablesMain.biml. Right-click the selected files and click Generate SSIS Packages:
If there are no errors, the SSIS Package named 30_Build_Destination_Tables.dtsx is generated:
Executing the Build SSIS Packages
Open and execute the SSIS Package named 10_Build_Destination_Databases.dtsx:
If the database already exists, the SSIS Package named 10_Build_Destination_Databases.dtsx will succeed and do nothing.
Open and execute the SSIS Package named 20_Build_Destination_Schemas.dtsx:
If the schemas already exist, the SSIS Package named 20_Build_Destination_Schemas.dtsx will succeed and do nothing.
Open and execute the SSIS Package named 30_Build_Destination_Tables.dtsx:
There’s an Execute SQL Task for each table. The T-SQL statement in each Execute SQL Task drops the table if it exists and then creates the table. Once created, the database, schemas, and tables appear in SQL Server Management Studio (SSMS) Object Explorer:
Delete the existing SSIS artifacts. This is another side-effect of porting this framework from Biml Studio to Biml Express . Before proceeding, we need to delete the existing Project Connection Managers and SSIS Packages:
You will need to confirm your intention to delete these artifacts:
Generating the SSIS Project That Contains the Loader SSIS Packages
To generate the SSIS packages that load the AdventureWorks2014_Staging database, multi-select the following Biml files:
0_00_BuildConnections.biml 0_10_BuildDestinationDatabaseMain.biml 0_20_BuildDestinationSchemasMain.biml 0_30_BuildDestinationTablesMain.biml 1_99_Main.bimlRight-click the selected files and click Generate SSIS Packages:
The SSIS loader packages are generated.
A Controller SSIS package is also generated, named 99_Execute_AdventureWorks2014_Stage_Loader_Staging_Packages.dtsx: