以编程方式编写脚本SSIS包 - 动态XML到OLE DB

时间:2018-02-15 01:55:51

标签: c# sql-server xml ssis etl

我一直在努力拼凑其他用户如何完成他们的项目,但我的理解仍然有限。

我想获取任何给定的XML源,制作数据流任务,并将其数据传递到与XML文件的表名匹配的OLE DB目标。使用可视化工具运行它意味着我无法执行动态数据流任务,因为元数据不会刷新。

我创建了一个创建包的脚本,但是当我在Visual Studio中打开包时,它有一个red-x表示不能有零输入列。当我向下钻取并查看OLE DB Destination的映射时,然后单击OK - 它会为我更正它。我无法弄清楚如何以编程方式做到这一点。

我见过其他人通过使用foreach循环和输入列来解决它,但我似乎无法弄明白。

我还有一个单独的脚本,我试图模仿几个人的脚本,它有不同的问题。不确定如何将其作为附件发布

提前感谢您的帮助:)

修改

我一直在尝试BIML得到积极的反馈,我会...但我想知道在短期内是否有人可以帮我弄清楚为什么这不会填写ExternalMetaDataColumnId作为我的输入。我在下面发布了我的更新代码,其中foreach循环没有按照我的预期进行。

谢谢         #区域     使用系统;     使用System.Collections.Generic;     使用System.Linq;     使用System.Text;     使用System.Threading.Tasks;     使用Microsoft.SqlServer.Dts.Runtime;     使用Microsoft.SqlServer.Dts.Pipeline.Wrapper;     使用System.Xml;     #endregion

namespace ConsoleApplication3
{
    class Program
    {
        static void Main(string[] args)
        {
            #region Initial Setup
            Application a = new Application();
            Package p = new Package();
            TaskHost t = p.Executables.Add("DTS.Pipeline") as TaskHost;
            t.Name = "DataFlow Task";
            t.Description = "Flat File to Database";
            MainPipe mp = t.InnerObject as MainPipe;
            #endregion

            #region Flat File Source in Dataflow Task
            IDTSComponentMetaData100 md = mp.ComponentMetaDataCollection.New(); 
            md.ComponentClassID = "Microsoft.XmlSourceAdapter"; 
            md.Name = "XML Source"; 
            CManagedComponentWrapper wrp = md.Instantiate(); 
            wrp.ProvideComponentProperties(); 

            #endregion

            #region Add connection manager to OLE DB
            ConnectionManager conn = p.Connections.Add("OLEDB");
            conn.Name = "westcoastuserDBO";
            conn.ConnectionString = "Data Source=SERVER;Initial Catalog=DBO;Provider=SQLNCLI11.1;Integrated Security=SSPI;Auto Translate=False;";
            #endregion


            #region XML Source Properties
            wrp.SetComponentProperty("XMLData", @"C:\Users\file.xml");
            wrp.SetComponentProperty("XMLSchemaDefinition", @"C:\Users\file.xsd");
            wrp.SetComponentProperty("AccessMode", 0);
            wrp.SetComponentProperty("UseInlineSchema", false);
            //below does not work
            //wrp.SetComponentProperty("XMLIntegerMapping", 0).TypeConverter = "Microsoft.SqlServer.Dts.Pipeline.XmlSourceAdapter + XMLIntegerMappingConverter";

            wrp.ReinitializeMetaData();
            wrp.ReleaseConnections();
            IDTSComponentMetaData100 md2 = mp.ComponentMetaDataCollection.New();
            md2.ComponentClassID = "Microsoft.OLEDBDestination";
            CManagedComponentWrapper wrp2 = md2.Instantiate();
            wrp2.ProvideComponentProperties();
            md2.Name = "OLE DB Connection";
            md2.UsesDispositions = true;
            md2.Version = 4;
            wrp2.SetComponentProperty("OpenRowset", "dbo.authorizations");
            #endregion

            IDTSPath100 path = mp.PathCollection.New();
            path.AttachPathAndPropagateNotifications(md.OutputCollection[0], md2.InputCollection[0]);

            IDTSInput100 input = md2.InputCollection[0];
            IDTSVirtualInput100 vInput = input.GetVirtualInput();

            //below taken from https://stackoverflow.com/questions/12587709/c-sharp-ssis-data-flow-component-creating-custom-input-columns
            IDTSExternalMetadataColumnCollection100 externalColumnCollection = input.ExternalMetadataColumnCollection;


            // Iterate through the virtual input column collection.
            foreach (IDTSVirtualInputColumn100 vColumn in vInput.VirtualInputColumnCollection)
            {
                // Call the SetUsageType method of the destination
                // to add each available virtual input column as an input column.
                wrp2.SetUsageType(
                   input.ID, vInput, vColumn.LineageID, DTSUsageType.UT_READONLY);
            }

            // Get the destination's default output collection
            IDTSOutputCollection100 outColl = md2.OutputCollection;

            // Iterate through the outputs in default output collection
            foreach (IDTSOutput100 output in outColl)
            {
                // Iterate through the default output columns in the output 
                int count = output.OutputColumnCollection.Count;
                foreach (IDTSOutputColumn100 outputColumn in output.OutputColumnCollection)
                {
                    // Get the output's external metadata column collection
                    IDTSExternalMetadataColumnCollection100 extMetadataColumnColl = output.ExternalMetadataColumnCollection;

                    // Iterate through the external metadata column collection's external metadata columns
                    foreach (IDTSExternalMetadataColumn100 extMetadataColumn in extMetadataColumnColl)
                    {
                        // Call the MapOutPutColumn method of the destination to map
                        // each available output column to an external metadata column 
                        wrp2.MapOutputColumn(
                            output.ID, outputColumn.ID, extMetadataColumn.ID, true);
                    }
                }
            }

            md2.RuntimeConnectionCollection[0].ConnectionManager = DtsConvert.GetExtendedInterface(conn);
            md2.RuntimeConnectionCollection[0].ConnectionManagerID = conn.ID;
            conn.AcquireConnection(null);




            #region Save Package to FileSystem
            string packageXml = @"C:\Users\test.dtsx";
            XmlDocument myPkgDocument = new XmlDocument();
            p.SaveToXML(ref myPkgDocument, null, null);
            a.SaveToXml(packageXml, p, null);
            #endregion
        }
    }
}

1 个答案:

答案 0 :(得分:2)

我认为您没有将输入列映射到OLEDB Destination的问题,打开包后,如果单击OLEDB Destination并转到Mapping部分,它将根据其名称自动映射列。其他人使用的Foreach循环是循环遍历列并将它们映射到相关的目标列。

有许多文章讨论动态创建SSIS包,您可以参考它们获取更多信息: