所以我很快就学会了C#的方式(继承了这个问题的完整noob);我编写了以下代码,该代码调用返回JSON的Web服务,该服务并不总是格式正确。这里的任务是获取JSON字符串并将其分解为数组段,这些数组段将插入到SQL表中以进行进一步的解析和测试。即如果返回字符串类似于
{1234:{5678:{1:{"key":"val","key":"val"},{2:{"key":"val","key":"val"}}}}
然后行将是:
{1234}
{5678}
{1:{"key":"val","key":"val"}
{2:{"key":"val","key":"val"}
这是.NET 3.0和SQL Server 2008 R2(遗留的东西)。 这是我的工作代码:
public partial class UserDefinedFunctions
{
[Microsoft.SqlServer.Server.SqlFunction(DataAccess =
DataAccessKind.Read)]
public static SqlString TestParse(SqlString uri, SqlString username, SqlString passwd, SqlString postdata)
{
//-----
// The SqlPipe is how we send data back to the caller
SqlPipe pipe = SqlContext.Pipe;
SqlString document;
try
{
// Set up the request, including authentication
WebRequest req = WebRequest.Create(Convert.ToString(uri));
if (Convert.ToString(username) != null & Convert.ToString(username) != "")
{
req.Credentials = new NetworkCredential(
Convert.ToString(username),
Convert.ToString(passwd));
}
((HttpWebRequest)req).UserAgent = "CLR web client on SQL Server";
// Fire off the request and retrieve the response.
using (WebResponse resp = req.GetResponse())
{
using (Stream dataStream = resp.GetResponseStream())
{
//SqlContext.Pipe.Send("...get the data");
using (StreamReader rdr = new StreamReader(dataStream))
{
document = (SqlString)rdr.ReadToEnd();
rdr.Close();
//-----
string connectionString = null;
string sql = null;
connectionString = "Data source= 192.168.0.5; Database=Administration;User Id=Foo;Password=Blah; Trusted_Connection=True;";
using (SqlConnection cnn = new SqlConnection(connectionString))
{
sql = "INSERT INTO JSON_DATA (JSONROW) VALUES(@data)";
cnn.Open();
using (SqlCommand cmd = new SqlCommand(sql, cnn))
{
String payload = "";
String nestpayload = "";
int nests = 0;
String json = document.ToString();
/*first lets do some housekeeping on our payload; double closing curly braces need to be escaped (with curly braces!) in order to keep them in the string.*/
json = json.Replace("\\", "");
int i = json.Length;
//return new SqlString(json);
while (i > 1)
{
/*find the first closing "}" in the string and then check to see if there are more than one.
We need to read the data up to each closing brace, pull off that substring and process it for each iteration until the string is gone.*/
int closingbrace = json.IndexOf("}"); //First closing brace
int nextbrace = Math.Max(0, json.IndexOf("{", closingbrace)); //Next opening brace
String ChkVal = json.Substring(closingbrace + 1, Math.Max(1, nextbrace - closingbrace)); //+1 to ignore the 1st closing brace
int checks = Math.Max(0, ChkVal.Length) - Math.Max(0, ChkVal.Replace("}", "").Length);
payload = json.Substring(0, Math.Max(0, (json.IndexOf("}") + 1)));
/*Remove the payload from the string*/
json = json.Substring(payload.Length + 1);
/*"nests" is how many nested levels excluding the opening brace for the closing brace we found.*/
nests = (payload.Length - payload.Replace("{", "").Length);
/*If we have more then one nest level check to see if any of them go with the payload*/
if (nests > 1)
{
/*Break out the nested section and remove it from the payload.*/
nestpayload = payload.Substring(0, payload.LastIndexOf("{"));
payload = payload.Substring(payload.LastIndexOf("{"), payload.Length - payload.LastIndexOf("{"));
while (nests > 1)
{
if (checks > 0) //# of right braces in payload equals number of left-side nests go with the payload
{
// payload = nestpayload.Substring(Math.Max(0, nestpayload.LastIndexOf("{")), Math.Max(0, nestpayload.Length) - Math.Max(0, (nestpayload.LastIndexOf("{")))) + payload;//The second Math.Max defaults to 1; if we got here there is at minimum one "{" character in the substring
payload = nestpayload.Substring(nestpayload.LastIndexOf("{")) + payload;
nestpayload = nestpayload.Substring(0, Math.Max(0, Math.Max(0, nestpayload.LastIndexOf("{"))));
checks--;
nests--;
}
else
{
/*If we got here there are no more pieces of the nested data to append to the payload.
We use an array and string.split to keep the nest ordering correct.*/
string[] OrderedNest = nestpayload.Split('{');
for (int s = 0; s < OrderedNest.Length; s++)
{
if (OrderedNest[s] != "")
{
cmd.Parameters.AddWithValue("@data", "{" + OrderedNest[s].Replace(":", "}"));
cmd.ExecuteNonQuery();
cmd.Parameters.Clear();
}
}
//cmd.Parameters.AddWithValue("@data", nestpayload.Substring(Math.Max(0,nestpayload.LastIndexOf("{"))).Replace(":","}"));
//cmd.Parameters.AddWithValue("@data", OrderedNest[1].Replace(":","}")+OrderedNest[2]);
// cmd.ExecuteNonQuery();
//cmd.Parameters.Clear();
//nests = Math.Max(0, nests - 1);
nests = 0;
//nestpayload = nestpayload.Substring(0, Math.Max(0, Math.Max(0,nestpayload.LastIndexOf("{"))));
}
}
}
/*At the very end payload will be a single "}"; check for this and discard the last row*/
if (payload != "}")
{
cmd.Parameters.AddWithValue("@data", new SqlChars(payload));
cmd.ExecuteNonQuery();
cmd.Parameters.Clear();
}
/*Get the new string length*/
i = json.Length;
payload = "";
}
}
}
//-----
/* }
catch (Exception e)
{
return e.ToString();
}*/
}
// Close up everything...
dataStream.Close();
}
resp.Close();
// .. and return the output to the caller.
}//end using
return ("Finished");
}
catch (WebException e)
{
throw e;
}
}
}
虽然它有效,但它很慢; 4分钟以上写入1500行到服务器。每天一次这需要写入约60,000条记录;其余的时间可能会有100条记录张贴并返回(我还没有完成POST部分)。我确定有很多事情我在这里做得不那么正确导致问题,但我完全不知道从哪里开始。我很兴奋,我可以得到正确的答案!任何想法/想法/帮助/同情将不胜感激。
答案 0 :(得分:3)
这里有几个问题,其中最重要的是,你似乎已经在这些公共互联网上发布了你的“sa”密码。以下是我看到的代码问题:
SqlPipe
行及其上方的注释行。函数不会通过SqlPipe
将数据传递回调用者;这是存储过程。WebRequest
document
应该是string
,而不是SqlString
。您永远不会返回document
,只会将其转换回string
,所以应该就是这样。HttpWebRequest
代替WebRequest
。这样您就不必偶尔将其转换为HttpWebRequest
。SqlString
输入参数转换为string
(例如Convert.ToString(uri)
)。所有Sql*
类型都具有Value
属性,该属性返回本机.NET类型中的值。因此,只需使用uri.Value
,依此类推。NULL
检查Convert.ToString(username) != null
输入。所有Sql*
类型都有IsNull
属性,您可以检查。因此,请使用!username.IsNull
。HttpWebRequest
连接打开的同时,不要进行所有文本处理(尤其是与另一个系统联系以进行逐行插入的处理)。您应该在using (WebResponse resp = req.GetResponse())
内执行的仅事件填充document
变量。在您位于最外层document
之外时,请勿对using()
的内容进行任何处理。while (i > 1)
循环)。他们甚至不在交易中。如果在文档中间出现错误,则会加载部分数据(除非此过程没有问题)。JSON_DATA
应为dbo.JSON_DATA
(或者如果不是dbo
则使用任何架构。)connectionString
中,您同时拥有身份/密码和Trusted_Connection
。不要同时使用它们,因为它们是互斥的选项(如果两者都有,则忽略Id /密码并仅使用Trusted_Connection
)。sa
登录或让您的应用程序以sa
登录。那只是乞求灾难。SqlProcedure
,以便可以使用Context_Connection=True;
作为连接字符串。这是连接到正在调用它的会话的进程内连接。Parameters.AddWithValue()
。馊主意。使用特定且适当的数据类型创建SqlParameter。然后通过Parameters
添加到Add()
集合。可能还有其他问题,但这些是显而易见的问题。正如我在第1点所说,你可能会在这里过头。不要试图消极,只是试图避免另一个糟糕的SQLCLR实现,这往往导致对这个非常有用的功能的负面看法。如果你想追求这个,那么请首先研究一下SQLCLR的工作原理,最佳实践等。一个好的起点是我在SQL Server Central上就这个主题撰写的一系列文章:Stairway to SQLCLR。 / p>
或者,另一种选择是使用完整版SQL# SQLCLR库(我写的)中提供的 INET_GetWebPages SQLCLR TVF。此选项不是免费的,但它允许您简单地安装Web请求片,然后您只需要在SQLCLR标量UDF中分别解析返回的文档(这可能是最好的方法,即使您执行Web请求自己的功能/存储过程)。实际上,如果要插入到同一SQL Server实例中的表中,则可以为文档解析器创建SQLCLR TVF,并使用OrderedNest
传回每个yield return
值(以将结果传回)并使用如下:
DECLARE @JSON NVARCHAR(MAX);
SELECT @JSON = [content]
FROM SQL#.INET_GetWebPages(@uri, .....);
INSERT INTO dbo.JSON_DATA (JSONROW)
SELECT [column_name]
FROM dbo.MyBrokenJsonFixerUpper(@JSON);
祝你好运!
答案 1 :(得分:1)
我正在回答这个问题,因为很明显需要的是重写和重新思考我的原始剧本。 @Solomon Rutzky赞成提供有用的信息,这些信息使我得出了这个结论。对于那些 感兴趣的是重写:
import React, { Component } from "react";
import Select from "react-select";
import "react-select/dist/react-select.css";
export default class SelectValid extends Component {
render() {
this.required = !this.props.required
? false
: this.state && this.state.value ? false : true;
let inputProps = undefined;
let onInputChange = undefined;
if (this.props.required) {
inputProps = {
onInvalid: e => e.target.setCustomValidity(this.required ? "foo" : "")
};
onInputChange = value => {
this.selectComponent.input.input.setCustomValidity(
value
? ""
: this.required
? "foo"
: this.selectComponent.props.value ? "" : "foo"
);
return value;
};
}
return (
<Select
onChange={value => {
this.required = !this.props.required ? false : value ? false : true;
let state = this && this.state ? this.state : { value: null };
state.value = value;
this.setState(state);
if (this.props.onChange) {
this.props.onChange();
}
}}
value={this && this.state ? this.state.value : null}
options={[{ label: "yes", value: 1 }, { label: "no", value: 0 }]}
placeholder={this.props.placeholder}
required={this.required}
clearable
searchable
inputProps={inputProps}
ref={input => (this.selectComponent = input)}
onInputChange={onInputChange}
/>
);
}
}
}