NumberOfBCPThreads setting not available in 2 command utilities?

Feb 29, 2012 at 7:02 PM

Any reason why this was dropped from the backup/restore utilities? 

Is the operation single threaded now? 

This has a drastic impact on import/export performance: 

http://sqlcat.com/sqlcat/b/msdnmirror/archive/2010/07/30/loading-data-to-sql-azure-the-fast-way.aspx 

Coordinator
Feb 29, 2012 at 10:35 PM


Hi,

 

No, this is another of those things that is on my list of things to do.  I have not put parallel upload into the batch utilities yet.  The framework is there, I just need to put the pieces together.  I will bump this up on my list of things to do.  Putting parallel download is another thing that I want to do.

 

Regards,

George

Mar 1, 2012 at 2:04 AM

Thanks, George. It would be much appreciated!

The current process is rather painful even for a meager 5GB database (2 hours to backup, at least 7+ hours to restore... still running). 

And this is on a 10 Mbit connection (Canada -> US). 

Coordinator
Mar 4, 2012 at 2:22 AM

Hi Auspicious,

I added parallel uploading to SQLAzureMWBatchUpload. I added a new key to SQLAzureMWBatchUpload.exe.config file that will allow you to specify the number of parallel upload threads (see:
NumberOfBCPThreads). Also, since doing parallel threads and writing out to a single console windows, all of the status will be intermixed. For this, I created “<schema>.<table>.tmp” files. Each file will contain the upload status for the respective tables. At the end of the process, I take all tmp files and append them to the “Results.txt” file, but I leave the tmp files just as an FYI.

Give SQLAzureMWBatchUpload v3.8.5 a test and let me know if anything goes wrong.

Regards,
George