Need to write a browser

[venky79]

Hi,

I need to write a rudimentary browser that will implement http, ftp and pop3.

For implementing HTTP, I know how to connect to the server using the Socket Class, but I dont know how to get the HTML and display it. Where do I start?

A little detailed explanation would be appreciated as I am still a novice in Java.

Thanks in advance,
Regards,
Venkatesh Kanchan.

 

[meadandale]

Why do you "need" to implement a browser?

Don't take this the wrong way but I get the feeling that you are biting off more than you can chew. Writing a browser is not something for a novice. How familiar are you with network protocols and the RFC's for HTTP, POP3 and FTP? Have you read up on them? Familiarizing yourself with this information should be the first thing you do before you try to write any code.

And actually, you are wrong. You usually WOULDN'T use the socket class to connect to the HTTP server. You would open a URLConnection and read from the input stream.

I'd suggest that you point yourself over at the java docs and spend some time browsing the java.net package.

 

[Blaxo]

meadandale is right, you don't need to use a socket for http.

To get an HTML file, you can create a new URL object and use its openStream() method the read its contents from the web.

An example:
import java.io.*;
import java.net.*;

public class GetURL {
    public static void main(String[] args){
        try {
            URL htmlFile = new URL("[URL unfurl="true"]http://www.host.com/index.html");[/URL]
            InputStreamReader isr = new InputStreamReader(htmlFile.openStream());
            char[] buffer = new char[1024];
            int charsRead = isr.read(buffer, 0, buffer.length);
            while(charsRead > 0){
                System.out.println(buffer);
                charsRead = isr.read(buffer, 0, buffer.length);
            }
            isr.close();
        }
        catch(Exception e){
            System.out.println("There was an exception: " + e.getMessage());
        }
    }
}

To display the HTML data you need to parse the HTML data using an HTML parser, maybe using the package javax.swing.text.html.parser (

http://java.sun.com/j2se/1.3/docs/api/javax/swing/text/html/parser/package-summary.html).

I have never used it, so I don't really know. regards,
Blaxo

 

[venky79]

Hi Meadandale,

Thanks for your suggestion. Well, I should have added that I have read the RFC's in question and that I do know the URLConnection Class and its methods. I am just wondering where to take from there as I have never done a project (its my final year project) on this scale before though I certainly am familiar with the concepts involved. Its just that I have not got my hands dirty writing code for such kinda work. Thanks anyways.

Regards,
Venkatesh Kanchan.

 

[meadandale]

Ahh, a class project. Ok then 8^)

Your biggest design task is to determine the scope of what you are going to try to do! You won't be able to implement all of the things that a modern browser does so you'll have to clearly define what it is that you want to support.

I'd check the web for preexisting solutions, at least so you can model them (e.g. creating an ftp client module).

POP3 support is fairly easy. Use the javamail API.

 

[venky79]

Hi and thanks again,

The specification is as follows :

I need to parse only a minimal subset of HTML, specifically the img and a tags (i.e. images and links). For each downloaded page, I need only display the images you have downloaded and a button for each link in the document.

I have to implement the following features as well :
1.) caching
2.) authentication
3.) persistent \ non-persistent connections depending on the option set in the settings menu.

Besides that, the following line in the specification is the source of my confusion :

"We expect you to implement the 3 protocols yourself on top of the socket layer provided by the operating system (or the JVM), i.e. socket() and connect()."

Thanks again in advance.

Regards,
Venkatesh Kanchan.

 

[venky79]

And yes, for the POP3 Support, I have been asked not to make use of the JavaMail API and that all the network portions of the assignment are to use TCP Sockets only.

 

[meadandale]

Well, that really sucks. That means that you can't take advantage of the URLConnection class or any of the other classes that take care of socket level handling for you. It also means that, when you send a request to the webserver, that you have to handle all of the header information yourself.

What this means is that you are definately going to have to be familiar with the protocols as you will have to send the commands directly through the sockets via the outputstreams, e.g.

InetAddress ia = InetAddress.getByName("mysite.com"
Socket s = new Socket(ia, 80);
OutputStream os = s.getOutputStream();
PrintStream ps = new PrintStream(os);
String reqStr = "GET /index.html\n";
ps.print(reqStr);
InputStream in = s.getInputStream();
// now read the HTML response from the server
// and/or handle any response codes in the header

This will be alot of work but it will teach you an enormous amount about network application protocols. You will have a lot more respect for what a complicated piece of software that your browser is 8^)

Good luck!!

C--

 

[MissouriTiger]

Hey Meadandale,

I've been sporadically visiting this forum for about a year and a half, and I've always been intrigued by your "name" Meadandale. It just occured to me me where you apparently got it. It's mead and ale, right?

So, what's with the C--?

And thanks, you've always been a big help! _______________________________________
Constructed from 100% recycled electrons

 

[venky79]

Hi,

I have been able to get the HTML from the server after sending it the request message. Now, how do I display it? I have been trying to use the DocumentParser Class the constructor of which requires a DTD. But the DTD Constructor is protected due to which I cant use it. Am I going the right way?

Thanks again in advance for putting up with me.

Regards,
Venkatesh Kanchan.

 

[meadandale]

MissouriTiger,

Right you are!! I used to work as a brewmaster at a local brewpub before my computer days. My name reflects my favorite beverages.

C-- ????

No, it has nothing to do with C++, just my first initial and a couple of dashes.

See you around 8^)

 

[MissouriTiger]

Okay Meadandale, one more...

I notice you participate in a huge variety of forums. You've never once started a thread. You always have answers, but never asked questions. Are you God?

But seriously, I'm curious, what exactly do you do, I mean besides the 12 oz. curls?

(Am I out of line posting this here?)

_______________________________________
Constructed from 100% recycled electrons

 

[meadandale]

Missouri,

Hehe, yeah, well...

I suppose that it is the former teaching assistant in me--always wanting to help others with their programming problems.

Sadly, I am not God :. I do come across programming problems and system issues from time to time but, as a former tech support engineer, am fairly resourceful when it comes to finding answers. Many of the problems that I have are fairly specific and are of a technical nature that aren't likely to be answerable in these forums (no offense to the likes of wushutwist). I tend to find more focused answers for these types of issues over at the Sun forums.

What do I do? Well, I am finishing my MS in CS and work as the manager of software engineering at a smallish company.

BTW, I am looking for a solid server side java programmer with some VB skills down here in San Diego so if you know of anyone...

What about you? ;-)

 

[sedj]

I'm feeling a lot of love here.

 

[gny]

All you guys who haven't seen it yet and don't have an evil teacher making you work with sockets could check out the javax.swing.JEditorPane if you want to quickly make an html 3.2 browser.

/gny

 

[MissouriTiger]

Mead,

"Manager of software engineering"- what a title!

I'm working on a BA in CS at Mizzou. Seems like it's taking forever.

What kind of software do you engineer?


_______________________________________
Constructed from 100% recycled electrons

 

[meadandale]

I develop ecommerce software for an insurance company.

Keep up the work on the degree--it'll pay off eventually.

 

[venky79]

Hi,
Well...I managed to implement HTTP (I left out caching though) and POP3. The only problem that I am facing now is that how do i get the servername, username, password and tcp port number from the Settings class that I have written, to the Mail class (which also I have written). The Settings class takes care of the settings window and the Mail class displays the mail window, and retrieves mail from the server, etc.

Thanks again in advance.

 

[venky79]

I think I've got it. I made the required variables "static" in the settings class and referenced them directly in the mail class. Is that the correct way?