The proper usage of BCP is bcp { dbtable query } {in out queryout format} datafile followed by one or more switches. In this syntax, dbtable is the database_name + owner + table_name view_name (for example, Northwind.dbo.customers or "Northwind.dbo.customers"): database_name is the name of the database in which the specified table or view resides. If not specified, this is the default database for the user . owner is the name of the owner of the table or view. table_name view_name is the name of the destination table or view when copying data in to SQL Server ( in ), and the source table when copying data from SQL Server ( out ). query is a Transact-SQL query that returns a resultset. queryout must also be specified when bulk-copying data from a query. in out queryout format Specifies the direction of the bulk copy ( in copies from a file in to the database table or view, out copies from the database table or view to a file). queryout must be specified when bulk-copying data from a query. format creates a format file based on the switch specified ( -n , -c , -w , -V , or -N ) and the table or view delimiters. If format is used, the -f option must be specified as well. data_file is the data file used when bulk-copying a table or view in to or out of SQL Server. All available Bulk-Copy Program switches are listed in Table 20.1. Table 20.1. BCP Switches
Fundamentals of Exporting and Importing DataOne of the great things about BCP is its ease of use. In this section, the full export and import of data using BCP will be done for Company X's Sales Force Automation requirements specified earlier. The minimum number of options and switches needed to fulfill these requirements will be described and execution examples illustrated . All tables being used here can be found in the "Northwind" sample database supplied by Microsoft in SQL Server 2000. Let's first look at the exporting requirement. As you might recall, the customers and products tables need to be exported once per week and sent to each salesperson (via e-mail). In turn , these files will be pulled in to the salesperson's Access application. The flat files will also need to be comma delimited. From BCP's point of view, the following things must be specified:
That's it. So at the command prompt you would see this: C:> BCP northwind..customers OUT customers.dat S C814\DBARCH01 U SFA-Admin P t "," c Starting copy... 91 rows copied. Network packet size (bytes): 4096 Clock Time (ms.): total 30 Here's a sample of the data in the customers.dat file: ALFKI,Alfreds Futterkiste,Maria Anders,Sales Representative,Obere Str. 57, Berlin,,12209,Germany,030-0074321,030-0076545 ANATR,Ana Trujillo Emparedados y helados,Ana Trujillo,Owner,Avda. de la Constitucion 2222,Mexico D.F.,,05021,Mexico,(5) 555-4729,(5) 555-3745 ANTON,Antonio Moreno Taqueria,Antonio Moreno,Owner,Mataderos 2312, Mexico D.F.,,05023,Mexico,(5) 555-3932 . . . This would be followed by the export of products: C:> BCP northwind..products OUT products.dat S C814\DBARCH01 U SFA-Admin P t "," c Starting copy... 77 rows copied. Network packet size (bytes): 4096 Clock Time (ms.): total 21 The data in products.dat file would be this: 1,Chai,1,1,10 boxes x 20 bags,18.0000,39,0,10,0 2,Chang,1,1,24 - 12 oz bottles,19.0000,17,40,25,0 3,Aniseed Syrup,1,2,12 - 550 ml bottles,10.0000,13,70,25,0 . . . Now, let's look at importing data into SQL Server 2000. Each salesperson is providing two flat files containing new orders and order details to be imported in to SQL Server 2000 every week. As part of the weekend batch processing, all of these flat files will be merged ( concatenated ) into consolidated files ( new_orders.dat and new_orddtl.dat ). This is what will be imported into the orders and order details tables in the Northwind database. Remember, the new orders keys are assigned by the Access DB application and must be the keys that get loaded in to the orders table (the OrderID in the orders table is an identity column key). The flat files will also be comma delimited. From BCP's point of view, the following things must be specified:
That's it. So at the command prompt you would see this: C:> BCP northwind..orders IN new_orders.dat S C814\DBARCH01 U SFAAdmin P t "," c -E Starting copy... 989 rows copied. Network packet size (bytes): 4096 Clock Time (ms.): total 13550 A quick SELECT * from the orders table shows the success of this operation: 12018 SIMOB 7 1998-05-06 00:00:00.000 1998-06-03 00:00:00.000 NULL 2 18.4400 Simons bistro Vinb[ae]ltet 34 Kobenhavn NULL 1734 Denmark 12019 RICSU 8 1998-05-06 00:00:00.000 1998-06-03 00:00:00.000 NULL 2 6.1900 Richter Supermarkt Starenweg 5 Genve NULL 1204 Switzerland 12020 RATTC 1 1998-05-06 00:00:00.000 1998-06-03 00:00:00.000 NULL 2 8.5300 Rattlesnake Canyon Grocery 2817 Milton Dr. Albuquerque NM 87110 USA . . . In addition, a quick check of the current identity value of this table reassures you that all is well (using OSQL/ISQL/SQL Analyzer): Use northwind go DBCC CHECKIDENT ('orders', NORESEED) Go This yields the following: Checking identity information: current identity value '12020', current column value '12020'. DBCC execution completed. If DBCC printed error messages, contact your system administrator. You can now do the import into the order details table. Note: The -E switch is not needed for this table because no columns in the order details table are defined as identity columns. In addition, the order details table contains a space in its name and must be enclosed in brackets ( [] ) for BCP. At the command prompt you would have the following: C:> BCP northwind..[order details] IN new_orddtl.dat S C814\DBARCH01 U SFAAdmin P t "," c Starting copy... 1000 rows sent to SQL Server. Total sent: 1000 1000 rows sent to SQL Server. Total sent: 2000 1000 rows sent to SQL Server. Total sent: 3000 3243 rows copied. Network packet size (bytes): 4096 Clock Time (ms.): total 5163 Again, a quick SELECT * from the order details table shows the successful imports: 12020 64 33.2500 2 2.9999999E-2 12020 66 17.0000 1 0.0 12020 73 15.0000 2 9.9999998E-3 12020 75 7.7500 4 0.0 12020 77 13.0000 2 0.0 . . . Great! The Sales Force Automation requirements have been successfully implemented quickly and accurately. That's the beauty and power of using BCP. The next sections take a look at how BCP can work with basic data representations (character, native, or unicode), the use of format file, and a few other extended BCP capabilities. File DatatypesBCP can handle data in one of three forms: character (ASCII), native, or Unicode. You have the choice of which character format is used, depending on the source or destination of the data file.
The Format FileBy using a format file, you can customize the data file created by BCP or specify complex field layouts for data loads. There are two ways to create this format file: interactive BCP and the format switch. Customizing a Format File Using Interactive BCPIf you do not specify one of the -n , -c , or -w datatype format switches, BCP (in or out) prompts you for the following information for each column in the data set:
BCP offers a default for each of these prompts that you can accept. If you accept all the defaults, you wind up with the same format file you would have by specifying the native format (with the -n switch). The prompts look like this: Enter the file storage type of field au_id [char]: Enter prefix length of field au_id [0]: Enter length of field au_id [11]: Enter field terminator [none]: or Enter the file storage type of field OrderID [int]: Enter prefix length of field OrderID [0]: Enter field terminator [none]: By pressing the Enter key at the prompt, you take the default. Alternatively, you can type your own value at the prompt if you know the new value and it is different from the default. Creating a Format File Using a SwitchBy using the format option, you can create a format file without actually transferring any data. Here is an example of creating a format file for the orders table in the Northwind database: C:> BCP "northwind..orders" format orders.dat “S C814\DBARCH01 U SFAAdmin P f orders.fmt c The format file created looks like this: 8.0 14 1 SQLCHAR 0 12 "\t" 1 OrderID "" 2 SQLCHAR 0 10 "\t" 2 CustomerID SQL_Latin1_General_ CP1_CI_AS 3 SQLCHAR 0 12 "\t" 3 EmployeeID "" 4 SQLCHAR 0 24 "\t" 4 OrderDate "" 5 SQLCHAR 0 24 "\t" 5 RequiredDate "" 6 SQLCHAR 0 24 "\t" 6 ShippedDate "" 7 SQLCHAR 0 12 "\t" 7 ShipVia "" 8 SQLCHAR 0 30 "\t" 8 Freight "" 9 SQLCHAR 0 80 "\t" 9 ShipName SQL_Latin1_General_ CP1_CI_AS 10 SQLCHAR 0 120 "\t" 10 ShipAddress SQL_Latin1_General_ CP1_CI_AS 11 SQLCHAR 0 30 "\t" 11 ShipCity SQL_Latin1_General_ CP1_CI_AS 12 SQLCHAR 0 30 "\t" 12 ShipRegion SQL_Latin1_General_ CP1_CI_AS 13 SQLCHAR 0 20 "\t" 13 ShipPostalCode SQL_Latin1_General_ CP1_CI_AS 14 SQLCHAR 0 30 "\r\n" 14 ShipCountry SQL_Latin1_General_ CP1_CI_AS Table 20.2 provides a description of the lines and columns in the preceding format file example. Table 20.2. The Contents of the Format File
You get different format files depending on your table and whether you chose character, native, or Unicode as your data type. As you can see in the preceding example, only the last two columns in the format file relate to the actual table; the remaining columns specify properties of the data file. File Storage TypeThe storage type is the description of how the data is stored in the data file. Table 20.3 lists the definitions used during Interactive BCP and what appears in the format file. The storage type allows data to be copied as its base type (native format), as implicitly converted between types ( tinyint to smallint ), or as a string (in character or Unicode format). Table 20.3. Storage Datatypes
Note: If the table makes use of user-defined data types, these customized data types appear in the format file as their base data type. If you are having problems loading certain fields in to your table, you can try the following tricks:
Prefix LengthFor reasons of compactness in native data files, BCP precedes each field with a prefix length that indicates the length of the data stored. The space for storing this information is specified in characters and is called the prefix length. Table 20.4 indicates the value to specify for prefix length for each of the datatypes. Table 20.4. Prefix Length Values
Prefix lengths are likely to exist only within data files created using BCP. It is unlikely that you will encounter a reason to change the defaults BCP has chosen for you. Field LengthWhen using either the native or the character data format, you must specify the maximum length of each field. When converting data types to strings, BCP suggests lengths large enough to store the entire range of values for each particular data type. Table 20.5 lists the default values for each of the data formats. Table 20.5. Default Field Lengths for Data Formats
The field length value is used only when the prefix length is and you have specified no terminators. In essence, you are doing a fixed-length data copy. BCP uses the exact amount of space stated by the field length for each field; unused space within the field is padded out.
Field TerminatorIf you are not making use of fixed-width fields or length prefixes, you must use a field terminator to indicate the character(s) that separate fields; for the last field in the data row, you must also indicate which character(s) ends the line. BCP recognizes the following indicators for special characters:
You cannot use spaces as terminators, but you can use any other printable character. Choose field and row terminators that make sense for your data. Obviously, you should not use any character you are trying to load. You must combine the \r and \n characters to get your data into an ASCII data file with each row on its own line.
The prefix length, field length, and terminator values interact. In the following examples, T indicates the terminator character(s), P indicates the prefix length, and S indicates space padding. For data of type char , the data file has the following repeating pattern:
For data of other types converted to char , the data file has the following repeating pattern:
The next few sections examine how to load data into tables when there are differences in column number and layout. Different Numbers of Columns in File and TableIf you have fewer fields in the data file than exist in the table, you have to " dummy up" an extra line in your format file. Suppose that you want to load a data file that is missing most of the address information for each customer. By using the format file you created in the section "The Format File," you can still load the data file. Suppose that the data file looks like this: WELLI Wellington Importadora Jane Graham Sales (14)555-8122 (14)555-8111 WHITC White Clover Markets Donald Bertucci Owner (206)555-4112 (206)555-4113 To introduce a dummy value for the missing ones, you must make the following changes to the format file: Make the prefix and data lengths and set the field terminator to nothing ( "" ). The modified format file will look like this ( custwoaddr.fmt ): 8.0 11 1 SQLCHAR 0 10 "\t" 1 CustomerID SQL_Latin1_General_ CP1_CI_AS 2 SQLCHAR 0 80 "\t" 2 CompanyName SQL_Latin1_General_ CP1_CI_AS 3 SQLCHAR 0 60 "\t" 3 ContactName SQL_Latin1_General_ CP1_CI_AS 4 SQLCHAR 0 60 "\t" 4 ContactTitle SQL_Latin1_General_ CP1_CI_AS 5 SQLCHAR 0 0 "" 5 Address SQL_Latin1_General_ CP1_CI_AS 6 SQLCHAR 0 0 "" 6 City SQL_Latin1_General_ CP1_CI_AS 7 SQLCHAR 0 0 "" 7 Region SQL_Latin1_General_ CP1_CI_AS 8 SQLCHAR 0 0 "" 8 PostalCode SQL_Latin1_General_ CP1_CI_AS 9 SQLCHAR 0 0 "" 9 Country SQL_Latin1_General_ CP1_CI_AS 10 SQLCHAR 0 48 "\t" 10 Phone SQL_Latin1_General_ CP1_CI_AS 11 SQLCHAR 0 48 "\r\n" 11 Fax SQL_Latin1_General_ CP1_CI_AS Now BCP can load the data file by using this new format file, with the Address, City, Region, PostalCode, and Country columns containing NULL s for the new rows. For data files that have more fields than the table has columns, you change the format file to add additional lines of information. Suppose that the Customer data file contains an additional CreditStatus value at the end: WELLI Wellington Importadora Jack McElreath Sales Manager Rua do Mercado, 12 Resende SP 08737-363 Brazil (14) 555-8122 NULL 1 WHITC White Clover Markets Scott Smith Owner 305 - 14th Ave. S. Suite 3B Seattle WA 98128 USA (206) 555-4112 (206) 555- 4115 2 Starting with the same format file as before, you modify it in two important areas: Change the second line to reflect the actual number of values, and add new lines for the extra column in the file that is not in the table. Notice that the column position has a value of to indicate the absence of a column in the table. Thus the modified format file will look like this ( custwcrdt.fmt ): 8.0 12 1 SQLCHAR 0 10 "\t" 1 CustomerID SQL_Latin1_General_ CP1_CI_AS 2 SQLCHAR 0 80 "\t" 2 CompanyName SQL_Latin1_General_ CP1_CI_AS 3 SQLCHAR 0 60 "\t" 3 ContactName SQL_Latin1_General_ CP1_CI_AS 4 SQLCHAR 0 60 "\t" 4 ContactTitle SQL_Latin1_General_ CP1_CI_AS 5 SQLCHAR 0 120 "\t" 5 Address SQL_Latin1_General_ CP1_CI_AS 6 SQLCHAR 0 30 "\t" 6 City SQL_Latin1_General_ CP1_CI_AS 7 SQLCHAR 0 30 "\t" 7 Region SQL_Latin1_General_ CP1_CI_AS 8 SQLCHAR 0 20 "\t" 8 PostalCode SQL_Latin1_General_ CP1_CI_AS 9 SQLCHAR 0 30 "\t" 9 Country SQL_Latin1_General_ CP1_CI_AS 10 SQLCHAR 0 48 "\t" 10 Phone SQL_Latin1_General_ CP1_CI_AS 11 SQLCHAR 0 48 "\t" 11 Fax SQL_Latin1_General_ CP1_CI_AS 12 SQLCHAR 0 1 "\r\n" 0 CreditStatus SQL_Latin1_General_ CP1_CI_AS The bold italic in the preceding format file indicates the changes made. These two examples show you the possibilities that the format file offers for customizing the loading and unloading of data. Renumbering ColumnsUsing the techniques described previously, you can also handle data file fields that are in different orders than the target tables. All that needs to be done is to change the column order number to reflect the desired sequence of the columns in the table. The fields will then be automatically mapped to the corresponding columns in the table. For example, suppose that a customer data file that you got from another source system came with the following layout (the fields are in this order):
The SQL Server table itself has columns in a different order. To load your data file into this table, you modify the format file to look like this ( custreord.fmt ): 8.0 11 1 SQLCHAR 0 10 "\t" 11 CustomerID SQL_Latin1_General_ CP1_CI_AS 2 SQLCHAR 0 80 "\t" 6 CompanyName SQL_Latin1_General_ CP1_CI_AS 3 SQLCHAR 0 60 "\t" 7 ContactName SQL_Latin1_General_ CP1_CI_AS 4 SQLCHAR 0 60 "\t" 8 ContactTitle SQL_Latin1_General_ CP1_CI_AS 5 SQLCHAR 0 120 "\t" 1 Address SQL_Latin1_General_ CP1_CI_AS 6 SQLCHAR 0 30 "\t" 2 City SQL_Latin1_General_ CP1_CI_AS 7 SQLCHAR 0 30 "\t" 5 Region SQL_Latin1_General_ CP1_CI_AS 8 SQLCHAR 0 20 "\t" 4 PostalCode SQL_Latin1_General_ CP1_CI_AS 9 SQLCHAR 0 30 "\t" 3 Country SQL_Latin1_General_ CP1_CI_AS 10 SQLCHAR 0 48 "\t" 10 Phone SQL_Latin1_General_ CP1_CI_AS 11 SQLCHAR 0 48 "\r\n" 9 Fax SQL_Latin1_General_ CP1_CI_AS The bold italic in the preceding format file indicates the changes made. The principal thing to remember with the format file is that all but the last three columns deal with the data file. The last three columns deal with the database table. Using ViewsBCP can also use views to export data from the database. What this means is that an export of data can be a resultset of data from multiple tables (and with distributed queries, even multiple servers). You can also use a view with BCP to load data back in to tables. However, as is the case with normal Transact-SQL inserts , you can load into only one of the underlying tables at a time. Loading Image DataIt is actually fairly easy for BCP to load image data into SQL Server. For example, the command to load the data file custlogo.doc (a word document) into the pub_info table in the pubs database using the bcp utility is this: C:> bcp pubs..pub_info in c:\temp\custlogo.doc -Usa -Ppassword Sservername bcp prompts: Enter the file storage type of field pub_id [char]: Enter the prefix length of field pub_id [0]: Enter length of field pub_id [4]: Enter the field terminator [none]: Enter the file storage type of field logo [image]: Enter the prefix length of field logo [4]: 0 Enter length of field logo [4096]: 5578 Enter the field terminator [none]: In this example, the data file custlogo.doc will be loaded into column logo, and 5578 is the length of the data file. Using the BULK INSERT statement, a format file needs to be created first and then used to provide the format information. To create the format file, use the bcp utility format option: C:> bcp "pubs..pub_info" format xxx.dat S servername Usa P f pub_info.fmt The bcp utility prompts for the file storage type, prefix length, field length, and field terminator of each column of pub_info . The values for the logo column are the same as previously. This yields the following format file ( pub_info.fmt ): 8.0 3 1 SQLCHAR 0 4 "" 1 pub_id SQL_Latin1_General_ CP1_CI_AS 2 SQLIMAGE 0 5578 "" 2 logo "" 3 SQLCHAR 4 0 "" 3 pr_info SQL_Latin1_General_ CP1_CI_AS Now the BULK INSERT can be executed: BULK INSERT "pubs..pub_info" FROM 'c:\temp\custlogo.doc' WITH (FORMATFILE = 'c:\pub_info.fmt') More information on BULK INSERT will be given later.
|