I need to submit some data to a web service from our SQL Server then capture the results into a SQL Server table.
Up to this point I have been looking at creating a SQL CLR (C#) stored procedure to accept a/some parameter(s) and then call the web service and capture the results into a SQL Server table; But I am open to any avenues that might work.
My questions are these:
Am I barking up the right tree so-to-speak, or is there a better way?
Assuming a SQL CLR stored procedure is the best way; I am relatively new to C# in this capacity and was hoping someone could help point me in the right direction. Does anyone have some example code or pointers? I've tried doing some Google searches and could find a few things
Any help would be greatly appreciated.
Here's a few technical details.
- The web service is hosted on an IIS 8.5 Web Server
- I do have access to a WSDL file the vendor provided (if that helps)
- I could run the SQL processes on any of the following: SQL Server 2008 R2, SQL Server 2012, SQL Server 2014
- I am using Visual Studio version 2015
- The web service both expects an XML for submitting, and returns its results in XML
Edit: for a small bit of added clarification the reason I was thinking of a SQL CLR stored procedure is that we already have a T-SQL stored procedure running that import, cleans up data in batch each night, we are adding a new system to the existing process and the vendor only supports using web service to gather the information we need . We are trying to keep our import processes together. Though as mentioned I am open to better alternatives, some direction would be exceptionally helpful.
Edit 2: Here's a quick overview of how the current process works.
A user performs a search on a website
System searches the tables of the database according to the parameters the user input (the search is a stored procedure). For current systems a nightly feed is imported using a scheduled SSIS package and stored procedures. The new additional system we are being required to add into the search only supports web-services for querying and extracting data, and we are being required to keep more real-time results from this system. (It is this step where we need to add the search capability to submit info to the web-service and get the results to merge in with the other results.)
System logs the results of the search, and resulting values from various business logic calculations. (the logging & retention of data is required by gov't regulators, so it's not good enough to just get the results from the web-service, we need it logged in with our other search results. )
System shows user the newly logged information.
I've decided I should have broken my question out into smaller steps and focus on each step.
1) Call a CLR Stored Proc from a Stored Proc, feeding the CLR some input paramaters.
2.1) in the CLR stored Proc build the WebRequest XML needed. 2.2) in the CLR stored proc capture the WebResponse XML
3) Post the (parsed) data elements we need from the response XML into SQL (or post the whole XML into SQL and parse it out from there. ?
For Step 1) I'm unsure how to make a CLR Stored Proc accept input parameters - but I haven't searched this yet so that's on me still.
Step 2.x) I believe I have this figured out (code below)
Step 3) I could use help with how to parse the response XML (example response below)
STEP 2 CODE (Posting to WebService and capturing Response):
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml;
using System.Threading.Tasks;
using System.Web;
using System.Net;
using System.IO;
namespace WebServiceParseResultsTest
{
class Program
{
public static void Main(string[] args)
{
string api_URL = "https://MySubDom.MyDomain.com/Services/SearchUtil/Search.aspx";
string api_usr = "abcdef0123456789";
string api_pas = "0123456789abcdef";
string rec_typ = "C3";
string rec_sta = "PEN";
string searchParam = "a0123456789z";
string XMLPostString = "<?xml version=\"1.0\"?> <REQUEST> <LOGIN api_password = \"" + api_pas + "\" api_user_id = \"" + api_usr + "\" />";
XMLPostString += "<SEARCH_QUERY record_status = \" "+rec_sta+" \" record_type = \" "+rec_typ+" \" SearchSysNum = \""+searchParam+"\" />";
XMLPostString += "</REQUEST>";
Console.WriteLine("-----------------");
Console.WriteLine(XMLPostString);
Console.WriteLine("-----------------");
WaitForEnterKey();
Console.WriteLine(DateTime.Now + " <NN_Test_XML_POST>");
string postResponse = PostXMLString(XMLPostString, api_URL);
Console.WriteLine(DateTime.Now + " SERVER RESPONSE: " + postResponse);
Console.WriteLine(DateTime.Now + " </NN_Test_XML_POST>");
Console.WriteLine(DateTime.Now + " Press Enter to Exit");
WaitForEnterKey();
}
public static void WaitForEnterKey()
{
Console.WriteLine(DateTime.Now + " Press Enter to Continue");
while (Console.ReadKey().Key != ConsoleKey.Enter) { }
}
public static string PostXMLFile(string fileName, string uri)
{
// Create a request and pass the URL to receive the post.
WebRequest request = WebRequest.Create(uri);
// We set the Method property of the request to POST.
request.Method = "POST";
// We create what is being sent by the POST method and convert it to byte array.
string postData = GetTextFromXMLFile(fileName);
//this.GetTextFromXMLFile(fileName);
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
// We set the ContentType of the WebRequest to xml.
request.ContentType = "text/xml";
// We set the ContentLength of the WebRequest.
request.ContentLength = byteArray.Length;
// We get the request stream.
Stream dataStream = request.GetRequestStream();
// write the data to the request stream.
dataStream.Write(byteArray, 0, byteArray.Length);
// create the Stream object.
dataStream.Close();
//-----
HttpWebResponse response;
response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
Stream responseStream = response.GetResponseStream();
string responseStr = new StreamReader(responseStream).ReadToEnd();
return responseStr;
}
return null;
//------
} // END: PostXMLFile
public static string PostXMLString(string XMLtext, string uri)
{
// Create a request and pass the URL to receive the post.
WebRequest request = WebRequest.Create(uri);
// We set the Method property of the request to POST.
request.Method = "POST";
// We create what is being sent by the POST method and convert it to byte array.
string postData = XMLtext;
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
// We set the ContentType of the WebRequest to xml.
request.ContentType = "text/xml";
// We set the ContentLength of the WebRequest.
request.ContentLength = byteArray.Length;
// We get the request stream.
Stream dataStream = request.GetRequestStream();
// write the data to the request stream.
dataStream.Write(byteArray, 0, byteArray.Length);
// create the Stream object.
dataStream.Close();
//-----
HttpWebResponse response;
response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
Stream responseStream = response.GetResponseStream();
string responseStr = new StreamReader(responseStream).ReadToEnd();
return responseStr;
}
return null;
//------
} //END: PostXMLString
private static string GetTextFromXMLFile(string file)
{
StreamReader reader = new StreamReader(file);
string ret = reader.ReadToEnd();
reader.Close();
return ret;
}
}
}
EXAMPLE RESPONSE XML
<RESPONSE version="1.3">
<SEARCH_RESULTS>
<RECORD record_type="CC"
record_id="0123456789abcdef0123456789abcdef"
record_num="987"
record_status="PEN"
record_date="8/11/2017 9:22:57 PM"
u_name="TEST20"
create_date="2/1/2017 6:15:49 AM"
/>
</SEARCH_RESULTS>
</RESPONSE>
Step 3 (Parsing XML RESPONSE FROM WEBSERVICE)
I decided to try and do this in SQL (I'm much stronger in SQL than C#) Here's what I came up with. But if anyone has good information on how to do this in C# I'd love to learn!
DECLARE @vMyXML XML
SET @vMyXML = '<RESPONSE version="1.3">
<SEARCH_RESULTS>
<RECORD record_type="CC"
record_id="0123456789abcdef0123456789abcdef"
record_num="987"
record_status="PEN"
record_date="8/11/2017 9:22:57 PM"
u_name="TEST20"
create_date="2/1/2017 6:15:49 AM"
/>
<RECORD record_type="BC"
record_id="1234567890bcdefa1234567890bcdefa"
record_num="879"
record_status="APR"
record_date="8/12/2017 10:23:58 PM"
u_name="TEST21"
create_date="3/2/2017 7:16:50 AM"
/>
</SEARCH_RESULTS>
</RESPONSE>'
SELECT
Attribs.value('@record_type' , 'nvarchar(10)') as [record_type]
, Attribs.value('@record_id' , 'nvarchar(50)') as [record_id]
, Attribs.value('@record_num' , 'int') as [record_num]
, Attribs.value('@record_status' , 'nvarchar(25)') as [record_status]
, Attribs.value('@record_date' , 'DateTime') as [record_date]
, Attribs.value('@u_name' , 'nvarchar(75)') as [u_name]
, Attribs.value('@create_date' , 'DateTime') as [create_date]
INTO AA_TMP_MyTestXMLResults
FROM @vMyXML.nodes('/RESPONSE/SEARCH_RESULTS/RECORD') as myXML(Attribs)
Select * from AA_TMP_MyTestXMLResults
DROP TABLE AA_TMP_MyTestXMLResults
SQLCLR should be ok in this scenario, as long as you are careful and aware of the various nuances of working in the highly restricted environment that is SQL Server's CLR host (i.e. SQLCLR), namely:
- You cannot use native web service code. Instead, you need to use
HttpWebRequestandHttpWebResponse. And, that means that you need to generate the request XML by hand, and then parse the XML response yourself (as opposed to getting back a .NET object). - In order to avoid the performance bottle neck that most people refer to when dealing with networking calls, you need to increase the URI Connection Limit via
ServicePointManagerfor the URI you are connecting to. The default limit is 2 and any calls above that will wait until one of those 2 connections completes. I am working on a blog post that will explain this in detail, with examples, and I will post the link here once it is published. - The Assembly will need to be set to
EXTERNAL_ACCESS. In order to do that, do not set the Database toTRUSTWORTHY ON. Instead, sign the Assembly, create an Asymmetric Key inmasterfrom that Assembly, create a Login from that Asymmetric Key, and finally grant that Login theEXTERNAL ACCESS ASSEMBLYpermission.
For more information on working with SQLCLR in general, please see the series I am writing on this topic on SQL Server Central (free registration is required to read their content): Stairway to SQLCLR.
Also, while not a free option, if you want the ability to make these web calls without having to deal with the coding, figuring out best practices, scripting, etc, then take a look at the Full version of SQL# (which I wrote). The Full version has INET_GetWebPages which allows for making web requests. And, it has INET_GetConnectionLimitForURI, INET_GetCurrentConnectionCountForURI, and INET_SetConnectionLimitForURI which allow you to manage that URI connection limit and reduce/avoid that performance bottleneck.
来源：https://stackoverflow.com/questions/45536835/how-to-create-a-sql-clr-stored-procedure-to-get-data-from-a-web-service-and-inse