Tek-Tips is the largest IT community on the Internet today!
Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!
-
Congratulations Mike Lewis on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!
How To Programmatically Fill In a Form (HTTP GET/POST)
Internet
(please note: the tek-tips parser has changed some of the simple ampersand characters in the following to being the encoded & ... so anywhere you see & imagine it just has & 
Forms on the web can be submitted in two different ways: GET and POST. (HTTP describes the protocol 'suite' for the web, "HyperText Transfer Protocol" )
HTTP GET -- Simply includes the fields in the form as part of the request URL. Basically takes the URL from the "ACTION" parameter of the "FORM" tag, adds "?" to seperate the URL from the field values, then adds all the field values as "Name=value" pairs, pairs seperated by "&". ie: the Google search form using url http://www.google.com/search then adds "?" followed by the fields of the form: hl, ie, oe, q and btnG, like this:
"hl=en&ie=UTF-8&oe=UTF-8&q=search+terms+here&btnG=Google+Search"
Of course, the characters "?", "=" and "&" would be confusing if they appeared in the field values, so the field values should be URL-Encoded. The final google GET url is: http://www.google.com/search?hl=en&ie=UTF-8&oe=UTF-8&q=search+terms+here&btnG=Google+Search
HTTP POST -- POST, on the other hand, does not put the parameters in the request URL. Instead, after connecting to the server and issuing a "POST" command to the specified FORM ACTION URL, the browser sends the form values in the request header. Since there is a limitation on the length of a URL, POST is a way to send larger amounts of information (like entire files) to a form processor on a web server. Many web forms use POST even when only accepting a couple form fields: POST prevents the user from bookmarking the result of a form submission.
The code here is primarily concerned with POSTing form data, since GET is so easy.
There are many third party utilities to help with POSTing data, and some tools from Microsoft, too, that may or may not be installed in your users' Windows installs, so I'll start with a very basic, direct to TCP/IP WinSockets illustration of what goes on with a POST:
( The GET and SendMail support here is a bonus: I keep them together in this utility class )
If you need to support authentication, or more features than the above code has, either write them yourself (and, I'd appreciate if you'd allow me to include your improvements here), or (particularly if you don't want to be responsible for complex features working) use an external tool. Here are some you may explore:
WinHttp control: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winhttp/http/about_winhttp.asp
( MS control included in:
WinHTTP 5.1 is available only with Windows Server 2003, Windows XP SP1, and Windows 2000 SP3. A redistributable merge module (.msm) file is not available for WinHTTP 5.1.
The WinHTTP 5.0 redistributable, winhttp5.dll, runs on Windows 2000 or later, or Microsoft Windows NT 4.0 with Microsoft Internet Explorer 5.01 or later.
Important With the release of WinHTTP Version 5.1, the WinHTTP 5.0 download is no longer available.
Reference: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winhttp/http/what_s_new_in_winhttp_5_1.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winhttp/http/winhttp_versions.asp )
( Thanks to Ralph/rkolva in thread184-664165 for this code )
MS XML HTTP request object:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/xmlsdk30/htm/xmobjxmldomserverxmlhttp.asp
Code excerpted from: http://fox.wikis.com/wc.dll?Wiki~SendHotmailMessageFromFoxPro
3rd party tool from West Wind:
http://www.west-wind.com/wwIPStuff.asp
Forms on the web can be submitted in two different ways: GET and POST. (HTTP describes the protocol 'suite' for the web, "HyperText Transfer Protocol" )
HTTP GET -- Simply includes the fields in the form as part of the request URL. Basically takes the URL from the "ACTION" parameter of the "FORM" tag, adds "?" to seperate the URL from the field values, then adds all the field values as "Name=value" pairs, pairs seperated by "&". ie: the Google search form using url http://www.google.com/search then adds "?" followed by the fields of the form: hl, ie, oe, q and btnG, like this:
"hl=en&ie=UTF-8&oe=UTF-8&q=search+terms+here&btnG=Google+Search"
Of course, the characters "?", "=" and "&" would be confusing if they appeared in the field values, so the field values should be URL-Encoded. The final google GET url is: http://www.google.com/search?hl=en&ie=UTF-8&oe=UTF-8&q=search+terms+here&btnG=Google+Search
HTTP POST -- POST, on the other hand, does not put the parameters in the request URL. Instead, after connecting to the server and issuing a "POST" command to the specified FORM ACTION URL, the browser sends the form values in the request header. Since there is a limitation on the length of a URL, POST is a way to send larger amounts of information (like entire files) to a form processor on a web server. Many web forms use POST even when only accepting a couple form fields: POST prevents the user from bookmarking the result of a form submission.
The code here is primarily concerned with POSTing form data, since GET is so easy.
There are many third party utilities to help with POSTing data, and some tools from Microsoft, too, that may or may not be installed in your users' Windows installs, so I'll start with a very basic, direct to TCP/IP WinSockets illustration of what goes on with a POST:
( The GET and SendMail support here is a bonus: I keep them together in this utility class )
Code:
DEFINE CLASS SocketWrapper As Custom
* Author: William GC Steinford
* Based on Anatoliy Mogylevets SMTP code at http://fox.wikis.com/wc.dll?Wiki~SendSmtpEmail
#DEFINE SMTP_PORT 25
#DEFINE HTTP_PORT 80
#DEFINE AF_INET 2
#DEFINE SOCK_STREAM 1
#DEFINE IPPROTO_TCP 6
#DEFINE SOCKET_ERROR -1
#DEFINE FD_READ 1
#DEFINE CRLF chr(13)+chr(10)
host = ""
IP = ""
Port = 80
hSocket = 0
cIn = ""
WaitForRead = 0
UserAgent = 'Mozilla/4.0'
AcceptLanguage = 'en-us'
PROCEDURE Init()
THIS.decl
IF WSAStartup(0x202, Repli(Chr(0),512)) <> 0
* unable to initialize Winsock on this computer
RETURN .F.
ENDIF
RETURN .T.
ENDPROC
PROCEDURE Destroy
= WSACleanup()
ENDPROC
PROCEDURE Host_Assign( vNewVal )
if empty(vNewVal)
THIS.IP = ""
else
THIS.IP = THIS.GetIP(vNewVal)
endif
if not empty(THIS.IP)
THIS.Host = vNewVal
else
THIS.Host = ""
endif
ENDPROC
PROTECTED FUNCTION GetIP( pcHost )
#DEFINE HOSTENT_SIZE 16
LOCAL nStruct, nSize, cBuffer, nAddr, cIP
nStruct = gethostbyname(pcHost)
IF nStruct = 0
RETURN ""
ENDIF
cBuffer = Repli(Chr(0), HOSTENT_SIZE)
cIP = Repli(Chr(0), 4)
= CopyMemory(@cBuffer, nStruct, HOSTENT_SIZE)
= CopyMemory(@cIP, THIS.buf2dword(SUBS(cBuffer,13,4)),4)
= CopyMemory(@cIP, THIS.buf2dword(cIP),4)
RETURN inet_ntoa(THIS.buf2dword(cIP))
ENDFUNC
PROTECTED FUNCTION Connect
LOCAL cBuffer, cPort, cHost, lResult
THIS.hSocket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP)
IF THIS.hSocket = SOCKET_ERROR
RETURN .F.
ENDIF
cPort = THIS.num2word(htons(THIS.Port))
nHost = inet_addr(THIS.IP)
cHost = THIS.num2dword(nHost)
cBuffer = THIS.num2word(AF_INET) + cPort + cHost + Repli(Chr(0),8)
lResult = (ws_connect(THIS.hSocket, @cBuffer, Len(cBuffer))=0)
RETURN lResult
FUNCTION httpGet( pcServer, pcUrl )
LOCAL lResult
THIS.Host = pcServer
IF THIS.Connect()
THIS.snd('GET '+pcURL+' http/1.0'+crlf)
THIS.snd('Accept: */*'+crlf)
THIS.snd('Accept-Language: '+THIS.AcceptLanguage+crlf)
THIS.snd('Accept-Encoding: gzip, deflate'+crlf)
THIS.snd('User-Agent: '+THIS.UserAgent+crlf)
THIS.snd('Host: '+pcServer+crlf)
THIS.snd('Pragma: no-cache'+crlf)
THIS.snd(crlf,.t.) && End of headers
*info=url encoded string
lResult = .T.
ELSE
lResult = .F.
ENDIF
THIS.Disconnect()
ENDFUNC
FUNCTION httpPost( pcServer, pcUrl, pcData, poFdbk )
The parameters are:
httpPost( pcServer, pcUrl, pcData, poFdbk )
* Meaning of parameters:
* pcServer: IP or hostname of the server to POST to
*
* pcURL: the URL of the page (CGI/Servlet/PHP, whatever)
* on the server that the POST data will be sent to.
*
* pcData: the url-formatted string comprised of
* name=value&name2=value2 data,
* where all "value" data is UrlEncoded
*
* poFdBk: (optional) A reference to an object that has
* a method called "Feedback" which accepts
* a single decimal value, between 0 and 1,
* which indicates the portion of the transfer
* that is complete.
* I usually add a "Feedback" method to a form,
* and pass THISFORM as poFdbk, and in
* THISFORM.Feedback, just update a progressbar
* or something.
LOCAL lResult, lnLen, lnComplete, lcRemain, lcSnd
THIS.Host = pcServer
IF THIS.Connect()
THIS.snd('POST '+pcURL+' http/1.0'+crlf)
THIS.snd('Content-Type:application/x-www-form-urlencoded'+crlf)
THIS.snd('Accept: */*'+crlf)
THIS.snd('Accept-Language: '+THIS.AcceptLanguage+crlf)
THIS.snd('Accept-Encoding: gzip, deflate'+crlf)
THIS.snd('User-Agent: '+THIS.UserAgent+crlf)
THIS.snd('Host: '+pcServer+crlf)
lnLen = len(pcData)
THIS.snd('Content-Length: '+tran(lnLen)+crlf)
THIS.snd('Pragma: no-cache'+crlf)
THIS.snd(crlf) && End of headers
* If we have a valid feedback object, provide feedback
if vartype(poFdbk)='O' and PEMStatus(poFdbk,'Feedback',5) ;
and upper(PEMStatus(poFdbk,'Feedback',3))='METHOD'
lcRemain = pcData
lnComplete = 0
poFdbk.Feedback( 0 )
do while len(lcRemain)>0
lcSnd = LEFT( lcRemain, 100 )
lcRemain = SUBSTR( lcRemain, 101 )
THIS.snd(lcSnd)
lnComplete = lnComplete+len(lcSnd)
poFdbk.Feedback( lnComplete/lnLen*100 )
enddo
THIS.snd("",.t.) && get a response, too.
else && no feedback object, just send one big chunk.
THIS.snd(pcData,.t.) && get a response, too.
endif
lResult = .T.
ELSE
lResult = .F.
ENDIF
THIS.Disconnect()
ENDFUNC
FUNCTION SendMail( pcSender, pcRecipient, pcSubject, pcBody )
LOCAL lResult
IF THIS.Connect()
THIS.snd('HELO', .T.)
THIS.snd('MAIL FROM:<' + pcSender + '>', .T.)
THIS.snd('RCPT TO:<' + pcRecipient + '>', .T.)
THIS.snd('DATA', .T.)
THIS.snd('From: ' + pcSender)
THIS.snd('To: ' + pcRecipient)
THIS.snd('Subject: ' + pcSubject)
THIS.snd("")
THIS.snd(pcBody)
THIS.snd('.', .T.)
THIS.snd('QUIT', .T.)
lResult = .T.
ELSE
= MessageB('Unable to connect to [' + THIS.Host +;
'] on port ' + LTRIM(STR(SMTP_PORT)) + '. ',;
48, ' Connection error')
lResult = .F.
ENDIF
THIS.Disconnect()
RETURN lResult
ENDFUNC
Function URLencode
LPARAMETER pcInStr
* ' encode Percent signs
* ' Double Quotes
* ' CarriageReturn / LineFeeds
LOCAL lcOut, lnI
* StrTran is WAY faster than building the string in memory
lcOut = StrTran(pcInStr, [%], '%25' )
lcOut = StrTran(lcOut, [+], '%2B' )
lcOut = StrTran(lcOut, [ ], '+' )
for lnI = 0 to 31
lcOut = StrTran( lcOut, chr(lnI), '%' + Right( Transform(lnI,'@0'), 2 ) )
endfor
for lnI = 127 to 255
lcOut = StrTran( lcOut, chr(lnI), '%' + Right( Transform(lnI,'@0'), 2 ) )
endfor
RETURN lcOut
ENDFUNC && UrlEncode
FUNCTION Disconnect
if THIS.hSocket<>SOCKET_ERROR
= closesocket(THIS.hSocket)
endif
THIS.hSocket = SOCKET_ERROR
ENDFUNC
FUNCTION snd(cData, lResponse)
LOCAL cBuffer, nResult, cResponse
cBuffer = cData && + CrLf
nResult = send(THIS.hSocket, @cBuffer, Len(cBuffer), 0)
IF nResult = SOCKET_ERROR
RETURN .F.
ENDIF
IF Not lResponse
RETURN .T.
ENDIF
LOCAL hEventRead, nWait, cRead
DO WHILE .T.
* creating event, linking it to the socket and wait
hEventRead = WSACreateEvent()
= WSAEventSelect(THIS.hSocket, hEventRead, FD_READ)
* 1000 milliseconds can be not enough
THIS.WaitForRead = WSAWaitForMultipleEvents(1, @hEventRead, 0, 2000, 0)
= WSACloseEvent(hEventRead)
IF THIS.WaitForRead <> 0 && error or timeout
EXIT
ENDIF
* reading data from connected socket
THIS.cIn = THIS.cIn+THIS.Rd()
ENDDO
RETURN .T.
ENDFUNC
PROTECTED FUNCTION Rd
#DEFINE READ_SIZE 16384
LOCAL cRecv, nRecv, nFlags
cRecv = Repli(Chr(0), READ_SIZE)
nFlags = 0
nRecv = recv(THIS.hSocket, @cRecv, READ_SIZE, nFlags)
RETURN Iif(nRecv<=0, "", LEFT(cRecv, nRecv))
ENDFUNC
PROCEDURE decl
DECLARE INTEGER gethostbyname IN ws2_32 STRING host
DECLARE STRING inet_ntoa IN ws2_32 INTEGER in_addr
DECLARE INTEGER socket IN ws2_32 INTEGER af, INTEGER tp, INTEGER pt
DECLARE INTEGER closesocket IN ws2_32 INTEGER s
DECLARE INTEGER WSACreateEvent IN ws2_32
DECLARE INTEGER WSACloseEvent IN ws2_32 INTEGER hEvent
DECLARE GetSystemTime IN kernel32 STRING @lpSystemTime
DECLARE INTEGER inet_addr IN ws2_32 STRING cp
DECLARE INTEGER htons IN ws2_32 INTEGER hostshort
DECLARE INTEGER WSAStartup IN ws2_32 INTEGER wVerRq, STRING lpWSAData
DECLARE INTEGER WSACleanup IN ws2_32
DECLARE INTEGER connect IN ws2_32 AS ws_connect ;
INTEGER s, STRING @sname, INTEGER namelen
DECLARE INTEGER send IN ws2_32;
INTEGER s, STRING @buf, INTEGER buflen, INTEGER flags
DECLARE INTEGER recv IN ws2_32;
INTEGER s, STRING @buf, INTEGER buflen, INTEGER flags
DECLARE INTEGER WSAEventSelect IN ws2_32;
INTEGER s, INTEGER hEventObject, INTEGER lNetworkEvents
DECLARE INTEGER WSAWaitForMultipleEvents IN ws2_32;
INTEGER cEvents, INTEGER @lphEvents, INTEGER fWaitAll,;
INTEGER dwTimeout, INTEGER fAlertable
DECLARE RtlMoveMemory IN kernel32 As CopyMemory;
STRING @Dest, INTEGER Src, INTEGER nLength
ENDPROC
FUNCTION buf2dword(lcBuffer)
RETURN Asc(SUBSTR(lcBuffer, 1,1)) + ;
BitLShift(Asc(SUBSTR(lcBuffer, 2,1)), 8) +;
BitLShift(Asc(SUBSTR(lcBuffer, 3,1)), 16) +;
BitLShift(Asc(SUBSTR(lcBuffer, 4,1)), 24)
ENDFUNC
FUNCTION num2dword(lnValue)
#DEFINE m0 256
#DEFINE m1 65536
#DEFINE m2 16777216
IF lnValue < 0
lnValue = 0x100000000 + lnValue
ENDIF
LOCAL b0, b1, b2, b3
b3 = Int(lnValue/m2)
b2 = Int((lnValue - b3*m2)/m1)
b1 = Int((lnValue - b3*m2 - b2*m1)/m0)
b0 = Mod(lnValue, m0)
RETURN Chr(b0)+Chr(b1)+Chr(b2)+Chr(b3)
ENDFUNC
FUNCTION num2word(lnValue)
RETURN Chr(MOD(m.lnValue,256)) + CHR(INT(m.lnValue/256))
ENDFUNC
ENDDEFINEIf you need to support authentication, or more features than the above code has, either write them yourself (and, I'd appreciate if you'd allow me to include your improvements here), or (particularly if you don't want to be responsible for complex features working) use an external tool. Here are some you may explore:
WinHttp control: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winhttp/http/about_winhttp.asp
( MS control included in:
WinHTTP 5.1 is available only with Windows Server 2003, Windows XP SP1, and Windows 2000 SP3. A redistributable merge module (.msm) file is not available for WinHTTP 5.1.
The WinHTTP 5.0 redistributable, winhttp5.dll, runs on Windows 2000 or later, or Microsoft Windows NT 4.0 with Microsoft Internet Explorer 5.01 or later.
Important With the release of WinHTTP Version 5.1, the WinHTTP 5.0 download is no longer available.
Reference: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winhttp/http/what_s_new_in_winhttp_5_1.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winhttp/http/winhttp_versions.asp )
Code:
loHTTP = CREATEOBJECT('WinHttp.WinHttpRequest.5.1')
* OR:
loHTTP = CREATEOBJECT('WinHttp.WinHttpRequest.5.0') && or is that just '... .5'???
loHTTP.Open('POST', 'http://www.SomeSite.cgi'; , .F.)
loHTTP.SetCredentials('UserName', 'Password', 0)
loHTTP.SetClientCertificate('CertficateName')
loHTTP.SetRequestHeader('content-type', 'application/x-www-form-urlencoded')
loHTTP.Send('SendThis')MS XML HTTP request object:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/xmlsdk30/htm/xmobjxmldomserverxmlhttp.asp
Code excerpted from: http://fox.wikis.com/wc.dll?Wiki~SendHotmailMessageFromFoxPro
Code:
oHttp = CreateObject('MSXML2.ServerXMLHTTP')
WITH oHttp
.Open('POST', cSendmsgHref, 0)
.SetRequestHeader('Content-Type', 'message/rfc821')
.SetRequestHeader('SAVEINSENT', 'f') && set field properties
.Send(cMessage)
nRespCode = VAL(SUBSTR(.GetResponseHeader('X-Dav-Error'), 1,3))
ENDWITH3rd party tool from West Wind:
http://www.west-wind.com/wwIPStuff.asp