通过Powerhsell将大型XML文件导入MSSQLExpress

时间:2015-08-31 14:02:05

标签: sql-server xml powershell sqlbulkcopy

作为标题sugests我必须将大(6GB)XML文件导入SQLExpress服务器。由于这必须每天在VServer本身上发生,并且我不允许安装更多软件,仅限于PowerShell(尽管升级到版本4)。 我的灵感来源是Use PowerShell to Load XML Data into SQL Server

我当前的脚本如下所示:

#database variables
$serverInstance="QLIKVIEWSQL\QLIKV_SQLEXPRESS";
$database = "Xml-Import_TEST"; 
$table = "dbo.vkjournal";

#opening the sql connection
$cn = new-object System.Data.SqlClient.SqlConnection;
$user="user";
$password="password"
$cn.ConnectionString="Server=$serverInstance;Database=$database;user   Id=$user;password=$password;Integrated Security=SSPI;";
$cn.Open();


#Semi-automatic creation of the XmlSchema
$cmd = New-Object System.Data.SqlClient.SqlCommand
$cmd.CommandText = "SELECT * FROM "+$table+" WHERE 1 = 2";
$cmd.Connection = $cn
$da = New-Object System.Data.SqlClient.SqlDataAdapter;
$da.SelectCommand = $cmd
$ds = New-Object System.Data.DataSet
$da.Fill($ds, $table) | Out-Null
$ds.WriteXmlSchema('C:\Users\qlikview\Desktop\SQLTEST\'+$table+'.xsd');


#Reading the xml-schema and loading bw1.xml into an DataSet object. Then bulkcopying to sql.
$ds = new-object "System.Data.DataSet"
$ds.ReadXmlSchema("C:\Users\qlikview\Desktop\SQLTEST\dbo.vkjournal.xsd");
$ds.ReadXml("C:\Users\qlikview\Desktop\SQLTEST\bw1.xml")
$dtProd = $ds.Tables[0]
$bc = new-object ("System.Data.SqlClient.SqlBulkCopy") $cn
$bc.DestinationTableName = $table;
$bc.WriteToServer($dtProd)

$cn.Close()

像往常一样,powershell不会抛出任何错误,只有IgnoreSchema(如果我注释掉ReadXmlSchema行)或InferSchema(如果我没有),并且SQLServer端没有任何反应。

创建的XMLSchema(缩写)如下所示:

<?xml version="1.0" standalone="yes"?>
<xs:schema id="NewDataSet" xmlns="" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
  <xs:element name="NewDataSet" msdata:IsDataSet="true" msdata:UseCurrentLocale="true">
    <xs:complexType>
      <xs:choice minOccurs="0" maxOccurs="unbounded">
        <xs:element name="dbo.vkjournal">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="KASS_NR" type="xs:short" minOccurs="0" />
              <xs:element name="VKJO_DATUM" type="xs:dateTime" minOccurs="0" />
              <xs:element name="VKJO_ZEIT" type="xs:duration" minOccurs="0" />
              <xs:element name="VKJO_BESTE" type="xs:string" minOccurs="0" />
            </xs:sequence>
          </xs:complexType>
        </xs:element>
      </xs:choice>
    </xs:complexType>
  </xs:element>
</xs:schema>

虽然我试图按1中的建议更改某些元素名称,但结果并没有改变。

我会非常感谢一些提示,谢谢。

1 个答案:

答案 0 :(得分:0)

即使问题在某种程度上不受欢迎,我也找到了使用this microsoft link的工作方法。完整的脚本会抛出一些错误,因此我放弃了验证并仅复制了批量加载。我必须安装msxml 6.0,如果这很重要,我确实安装了SQLXML 4.0。 对于像我这样有大表定义的人,上面的脚本(参见#半自动创建XmlSchema一节)有助于大大减少所需的手写。

set objBL = CreateObject("SQLXMLBulkLoad.SQLXMLBulkload.4.0")
objBL.ConnectionString = "provider=SQLOLEDB;data source=SERVER\SQLEXPRESS_INSTANCE;database=YOURDB;integrated security=SSPI"
objBL.ErrorLogFile = "C:\error.log"

' Check constraints and initiate transaction (if needed)
   ' objBL.CheckConstraints = True
   ' objBL.Transaction=True
  'Execute XML bulkload using file.
  objBL.Execute "C:\XSDfile.xml", "C:\File.xml"
  set objBL=Nothing