Six different methods to copy tables between databases in SQL Server

In this article, you’ll learn the key skills that you need to copy tables between SQL Server instances including both on-premises and cloud SQL databases. In this article, I’ll walk-through several ways of copying a table(s) between SQL databases, helping you to see the benefits and trade-offs of each option.

Introduction

Before we begin the article, though, let’s go over the objectives of the article. We then move on to the overview of each module or methods. In this guide, we briefly discuss several aspects of SQL Server’s available built-in options, as well as show you a few PowerShell and 3 rd party tools can be used to copy SQL tables between the databases and between the instances as well. At the beginning of each method, I’ve given you enough information that the following modules. We follow this module up with several modules, each of which is dedicated to specific methods.

Objectives: Introduction Discuss various methods to copy tables Using .Net class library to copy tables with PowerShell Using Import-and-Export Wizard Using sqlpackage.exe Extract and Publish method Using Generate Scripts wizard in SSMS ( SQL Server Management Studio) Using INSERT INTO SQL statement And more… Get started

In SQL Server, copying tables between the databases of the same SQL instances are relatively easier than copying the data between the remote servers. To minimize the work-load on the production database, it is always recommended to restore the database from the backup to the new database and then use the best methods to copy the data to the target database. Again, this depends on the number of tables, size, and available space. If the size of the table(s) is more than 50% of the total size of the database than the backup-and-restore method is a recommended option.

In some cases, you might have to copy a few very large table(s), and then you may probably end-up in moving the table(s) to separate file-groups and perform a partial backup-and-restore method to copy the data. You can refer to the article Database Filegroup(s) and Piecemeal restores in SQL Server for more information.

You can also use third-party tools to perform an object level restore from a backup file .

SqlBulkCopy object class for Data copy with PowerShell

PowerShell is always my first choice for any administrative task. Net provides a SqlBulkCopy class library to bulk load the table(s) into the database.

You can refer to the article 6 methods to write PowerShell output to a SQL Server table to get more information about .Net class libraries.

PowerShell script

The following PoSH script creates a function named Get-SQLTable. The function has several mandatory parameters.

function Get-SQLTable { [CmdletBinding()] param( [Parameter(Mandatory=$true)] [string] $SourceSQLInstance, [Parameter(Mandatory=$true)] [string] $SourceDatabase, [Parameter(Mandatory=$true)] [string] $TargetSQLInstance, [Parameter(Mandatory=$true)] [string] $TargetDatabase, [Parameter(Mandatory=$true)] [string[]] $Tables, [Parameter(Mandatory=$false)] [int] $BulkCopyBatchSize = 10000, [Parameter(Mandatory=$false)] [int] $BulkCopyTimeout = 600 ) $sourceConnStr = "Data Source=$SourceSQLInstance;Initial Catalog=$SourceDatabase;Integrated Security=True;" $TargetConnStr = "Data Source=$TargetSQLInstance;Initial Catalog=$TargetDatabase;Integrated Security=True;" try { Import-Module -Name SQLServer write-host 'module loaded' $sourceSQLServer = New-Object Microsoft.SqlServer.Management.Smo.Server $SourceSQLInstance $sourceDB = $sourceSQLServer.Databases[$SourceDatabase] $sourceConn= New-Object System.Data.SqlClient.SQLConnection($sourceConnStr) $sourceConn.Open() foreach($table in $sourceDB.Tables) { $tableName = $table.Name $schemaName = $table.Schema $tableAndSchema = "$schemaName.$tableName" if ($Tables.Contains($tableAndSchema)) { $Tablescript = ($table.Script() | Out-String) $Tablescript Invoke-Sqlcmd ` -ServerInstance $TargetSQLInstance ` -Database $TargetDatabase ` -Query $Tablescript $sql = "SELECT * FROM $tableAndSchema" $sqlCommand = New-Object system.Data.SqlClient.SqlCommand($sql, $sourceConn) [System.Data.SqlClient.SqlDataReader] $sqlReader = $sqlCommand.ExecuteReader() $bulkCopy = New-Object Data.SqlClient.SqlBulkCopy($TargetConnStr, [System.Data.SqlClient.SqlBulkCopyOptions]::KeepIdentity) $bulkCopy.DestinationTableName = $tableAndSchema $bulkCopy.BulkCopyTimeOut = $BulkCopyTimeout $bulkCopy.BatchSize = $BulkCopyBatchSize $bulkCopy.WriteToServer($sqlReader) $sqlReader.Close() $bulkCopy.Close() } } $sourceConn.Close() } catch { [Exception]$ex = $_.Exception write-host $ex.Message } finally { #Return value if any } }

The $tables array variable is used to assign the list of the table(s) to be copied to the target database

[string[]] $tables = @('dbo.OPERATION','dbo.OPERATION_DETAIL')

Let us invoke the Get-SQLTable function with the below mentioned parameters to copy the tables from Adventureworks2016 database on ‘HQDBT01’ to Adentureworks2012 database on hqdbt01/sql2017’ instance.

Get-SQLTable -SourceSQLInstance hqdbt01 -SourceDatabase AdventureWorks2016 -TargetSQLInstance hqdbt01\sql2017 -TargetDatabase AdventureWorks2012 -Tables $tables -BulkCopyBatchSize 5000

The output shows the tables OPERATION and OPERATION_DETAIL copied to the target instance.

SSMS Import-and-Export Wizard

Let’s take a look at the Import-and-Export Wizard. The interface is very similar to all other wizards, allow you to easily step through a process, and to execute the data copy process with writing very little or no code. To do that, we have very few options that we can do within the wizard. However, for this, for importing and exporting data from one source into another, this is really an excellent tool. If you want to do almost any kind of transformations, then you don’t want use this tool, you may need to use Visual Studio Data Tools (VSDT), and do a data flow.

So let’s get started. The first thing is to open Microsoft SQL Server Management Studio (SSMS). And we’re going use to AdventureWorks2016 database, and we’re going to move it over to another instance of SQL.