Building a Web Proxy Server using URL and URLConnection classes ???

[transtec]

Hi ,

Im desparately in need of a way to build an application that would function as a Proxy Server that would intervene HTTP Requests and deal with requests to particular sites ( especially Search Engine sites defined in the config file ). I browsed this on the net for such possible tutorials/samples and found that many have advised to use the URL and URLConnection classes to build the Proxy Server. But Im clueless how to do this.

I got hold of some source code for building a Web Proxy Server from

http://www.jps.at/java/tools/proxyserver.html

, but that doesnt use the URL and URLConnection classes . Worst of all the source code provided wouldnt compile properly. I even tried decompililng the classfiles and got it to work, but that wouldnt show me the functionality prpoerly.

So could you guys kindly giv me sum pointers on how I should accomplish this task using URL or URLConnection ? Source code would be ideally helpful to gain an insight to see how the app shud function. I would greatly appreciate this favour.

 

[sedj]

Where/How would you consider running this code ?
Inside a servlet container, or standalone ?

--------------------------------------------------
Free Java/J2EE Database Connection Pooling Software

http://www.primrose.org.uk

 

[Diancecht]

Maybe this helps:

http://www.manageability.org/blog/stuff/open-source-personal-proxy-servers-written-in-java/view

And for URLConnection examples ...

http://java.sun.com/docs/books/tutorial/networking/urls/readingWriting.html

Cheers,
Dian

 

[sedj]

Dian, I want the OP to answer my question, because if they are running the code in a servlet container, then they can use filters rather than a proxy, which will be a lot quicker, and more efficient ...

--------------------------------------------------
Free Java/J2EE Database Connection Pooling Software

http://www.primrose.org.uk

 

[Diancecht]

And do you want me to threaten the OP to answer you?

I'm just giving some resources to look at. I have no idea of what the OP is looking for and I suspect neither he/she has. Maybe reading some stuff gives some light to the issue.

Cheers,
Dian

 

[sedj]

>>>> And do you want me to threaten the OP to answer you? :

Would you ?

--------------------------------------------------
Free Java/J2EE Database Connection Pooling Software

http://www.primrose.org.uk

 

[Diancecht]

Sure.

transtec: from this point you won't get any answer in any Java forum within tek-tips until you answer sedj`s question.

Furthermore, you will get annoying popups when surfing and spamming emails.

Cheers,
Dian

 

[transtec]

sorry guys... was busy with this project of mine.
Actually the web proxy is for a Java desktop app. The functionality is such that it resides in the user's PC and monitors http traffic as it passes by. When requests to certain sites ( to web search engines, specifically ) are made the app should obstruct the requests and generate a custom web page, if not it should get the pages asked for.

Actually I have done some work on this. I hav come so far enough as to getting the app to connect to a web resource and get the HTML for the requested page. But thats all i get. When I use this app, I can see ONLY the text in a page. The images dont appear at all. Here is part of the code that does the main job :

            URL url=new URL(this.httpURI);
                URLConnection connection=url.openConnection();
                connection.setDoInput(true);
                
                BufferedReader buff=new BufferedReader(new                         InputStreamReader(connection.getInputStream()));
                String line;
                StringBuffer send=new StringBuffer();
                while((line=buff.readLine())!=null)
                {
                     send.append(line);                        
                }   
                 this.output_c.write(send.toString().getBytes());
                System.out.println("responded to request");

My question now is, is it possible to get the images of a webpage when I use the InputStream returned by the URLConnection class's getInputStream() method. OR do i have to query for them and fetch them from the server, in a different process ?

thanx.

 

[sedj]

So are you setting a proxy in the browser as locahost ?

--------------------------------------------------
Free Java/J2EE Database Connection Pooling Software

http://www.primrose.org.uk

 

[transtec]

yes yes... my browser is pointing to this proxy. I know it sounds weird to be running a proxy on a typical workstation, but it actually aint a fully fledged proxy as u mite think. Only there to monitor http traffic and respond to certain web sites only, and with a custom generated HTML page.

 

[sedj]

Yes, but you have to understand that when you tell a browser to run all traffic through a proxy, you have to fully implement proxy functionality.

You will have to fully implement the HTTP protocol, including the following features :

Rewriting ALL header information (including cookies, content types etc)
Supporting HTTP POST data in both directions

This means that when the browser connects to you, you will have to fully parse the HTTP request, and resend to the target server.

This is not a trivial application !


--------------------------------------------------
Free Java/J2EE Database Connection Pooling Software

http://www.primrose.org.uk

 

[thumpkidd9]

I've done something similar to what you have and I was getting the same problem with images. If they still aren't showing up after you rewrite all the headers and support HTTP POST as sedj suggests. It might have to do with data conversion between String and Byte. At least that is what my problem was.

~Matt

 

[transtec]

yo thumpkidd9,

I tried writing the reponse header to the browser, but prob is that when i do that, the response header details get written onto the web page as HTML ( browser interprets them as HTML and writes the content onto the page that is shown ).
Where do u think Im going wrong ? as for using byte streams vs. text based string streams, I tried that too, but my knowledge of Byte Streams aint that good enuf. ( Im not at all good in Java I/O ) If its not a prob, could you show me some code on how to handle the response from the server and writing it back to the client ( the browser ) ?

thanx for the help.

 

[sedj]

As I said before, you need to fully implement the HTTP protocol - this in effect means writing an HTTP server.
Have a read of this article, which explains the very basics of an HTTP compliant server :

http://www.onjava.com/pub/a/onjava/2003/04/23/java_webserver.html

--------------------------------------------------
Free Java/J2EE Database Connection Pooling Software

http://www.primrose.org.uk