Java -> PHP communication problem

[developerinlondon]

I am having a strange problem trying to invoke a PHP script from within a java servlet.
What I want to do is the following -

I have a java servlet that is capable of fetching multiple URLs simultaneously using threads. I am using this servlet to execute a PHP script. What I want to happen is
a) the servlet invokes the php script and opens a connection. Writes all the header information/parameters to it and waits for a reply.
b) the php script does its own work and returns 1 or nothing.
c) control flows on in the java servlet and it reports the work has been done.

What is happening however is the following-
a) the servlet calls the php script and opens a connection. Writes all the header information/parameters to it and waits for a reply.
b) the php script does NOT do its work, instead it waits...
c) the java servlet waits until the 'timeout' time to read something from the php script and reports a timeout and finishes.
d) the php script resumes what it was suppose to do and finishes its work.

I am totally dumbfound as to why the php is acting in this regard... (I know this may be php related too but I thought people with java knowledge would have a better idea whats going on here).

thanks in advance.

nayeem

 

[timw]

... maybe if you posted some representative code snippets.

Tim
---------------------------
"Your morbid fear of losing,
destroys the lives you're using." - Ozzy

 

[developerinlondon]

it seems to be happening when php tries to set a sessionID and then starts the session. It seems the input and output pipes that java opens dont quite work well when php sends headers back as well as contents...

heres an extract of the code in the java class that calls the PHP: (it is a bit more complicated then plain vanilla as its used to call various types of URLs/web services) -

           URL url = new URL(this.url);
     
           /**************************************************************************************
            * now open a url connection, it automatically turns into httpsurlconnection if needed
            **************************************************************************************/
             HttpURLConnection conn = (HttpURLConnection) url.openConnection();
            conn = null;
            //if (this.url.startsWith("https")) {
            if ((this.url.startsWith("https")) && (this.url.lastIndexOf(new String("bcp"))<1)) {
                conn = (HttpsURLConnection) url.openConnection();
                logger.info("Using custom security manager for secure connection.");
            } else {
                conn = (HttpURLConnection) url.openConnection();
            }

            /********************************
            * configure timeouts + method   *
            *********************************/
            conn.setDoOutput(true);
            conn.setConnectTimeout(connection_timeout); // maximum 30 seconds connection timeout.
            conn.setReadTimeout(read_timeout); // maximum 30 seconds read timeout.
            conn.setRequestMethod(this.request_method); // set the request method

            /********************************
            * send output (if passed)       *
            *********************************/
            // send output if there is any.
            OutputStreamWriter wr = null;
            if (xml_out != null) {
                wr = new OutputStreamWriter(conn.getOutputStream());
               // wr.write (this.header_out);
                wr.write (this.xml_out);
                wr.flush();
            }



            conn.connect();
            /***********************************
            * read the reply and put in xml_in *
            ************************************/
          //if (this.no_read == false) {
            logger.info("Reading Reply from "+this.url+".");
            InputStream in = conn.getInputStream();
            // Get the response
            DataInputStream rd = new DataInputStream(in);
            byte[] buffer = new byte[4096];
            ByteArrayOutputStream baos = new ByteArrayOutputStream();
            int i = -1;
            while (true) {
                i = rd.read(buffer, 0, buffer.length);
                if (i == -1) break;
                baos.write(buffer, 0, i);
            }

            byte[] byteData = baos.toByteArray();

            // utf16 strings needs to be read differently.
            if (this.is_utf16(byteData)) {
                logger.info ("utf-16 detected.");
                visible_xml_in = new String(byteData, "utf-16");
            } else {
                visible_xml_in = new String(byteData);
            }
            xml_in = new String(byteData);

            baos.close();


            //logger.debug(xml_in);

            if (wr != null) {
                wr.close();
            }
            rd.close();
          //}

            // Get elapsed time in milliseconds
            long elapsedTimeMillis = System.currentTimeMillis()-start;

            logger.info("Finished Fetching from "+this.url+" ["+elapsedTimeMillis+"ms].");
            return elapsedTimeMillis;

        } catch (IOException e) {
            // report error
            String error = e.getMessage();
            xml_in = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<JURL><ERROR type=\"IOException\" thread=\"" + Thread.currentThread().getName() + "\">"+error+" [ca
use:"+"]</ERROR></JURL>";
            visible_xml_in = xml_in;
            logger.error(error);
            if (error.startsWith("Read")) {
                return -1;
            } else if (error.startsWith("connect")) {
                return -2;
            } else {
                return 0;
            }
        } catch (Exception e) {
            // report error
            xml_in = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<JURL><ERROR type=\"Exception\" thread=\"" + Thread.currentThread().getName() + "\">"+e.getMessage(
)+"</ERROR></JURL>";
            visible_xml_in = xml_in;
            logger.error(e.getMessage());
            return 0;

 

[timw]

Have you tried closing the OutputStreamWriter after you've written the header/data to it?

Tim
---------------------------
"Your morbid fear of losing,
destroys the lives you're using." - Ozzy

 

[developerinlondon]

ok just tried the following -

/********************************
            * send output (if passed)       *
            *********************************/
            // send output if there is any.
            OutputStreamWriter wr = null;
            if (xml_out != null) {
                wr = new OutputStreamWriter(conn.getOutputStream());
               // wr.write (this.header_out);
                wr.write (this.xml_out);
                wr.flush();
            }

            if (wr != null) {
                wr.close();
            }

ie closed the outputstreamwriter straight after the data is written instead of later. But still seems to be having the same behaviour. I think it has to do with when PHP tries to send a cookie header php expects the browser to return something? Is there more then one conversation between a browser and php in general for a script invocation? With Java it might be waiting for a long time and Java is only waiting for php to write something to it and it ends up in a deadlock until Java times out... ?

 

[sedj]

Why are you doing :

conn.connect();

after you get the output stream ?

--------------------------------------------------
Free Java/J2EE Database Connection Pooling Software

http://www.primrose.org.uk

 

[developerinlondon]

I have no idea but I am sure I had a good reason for it when I put it there earlier... but in any case I just tried it with removing it, I am still getting the same problem running php scripts however...

 

[sedj]

well I wouldn't remove it - I would place it above any calls to retrieve streams

--------------------------------------------------
Free Java/J2EE Database Connection Pooling Software

http://www.primrose.org.uk

 

[developerinlondon]

ah there we go. thanks for that. I just didnt remember why I had it there!

the problem remains however... php is being so dumb and sitting down doing nothing until java hangs up!

 

[developerinlondon]

ok tried putting it above the 'send output' section but all seems same... I dont think its here the problem.

 

[timw]

Not done much of this type of thing myself. Is your xml_out one big line of text? Does it have an end-of-line marker (CR/LF) at the end?

Tim
---------------------------
"Your morbid fear of losing,
destroys the lives you're using." - Ozzy

 

[developerinlondon]

for this case the xml_out is just an empty string as all the GET information is in the URL already.
I really dont think the error is in the Java code as its being used to fetch all sorts of scripts from various different sites without ever causing problems.
Also it doesnt seem to cause problems fetching php scripts in general. The script in question however has the following code within the php -

session_id($_REQUEST[sessionid]);
session_start;

according to php documentation the above code sets a session cookie in the browser. If I have just 'session_start() without the first part the java fetcher manages to retrieve it without much problem. So the problem lies there, but I cant see whats a fix for this...

 

[Diancecht]

At first sight, you're opening two connectios at a time, since openConnection opens one new connection in each call.

Cheers,
Dian

 

[timw]

I thought cookies were returned to the client with the http response.

Tim
---------------------------
"Your morbid fear of losing,
destroys the lives you're using." - Ozzy

 

[developerinlondon]

me too... does the input buffer above read header information also? Or do they get discarded somehow/causes a deadlock in the connection... I cant see why else php would 'wait' for this connection to break before continuing with the rest of its script.

 

[sedj]

All header information (sending and recieving) is handled internally in the URLConnection classes.

You should only use the output stream if the URLConnection method is set to POST.

If you are writing and reading off the socket from the connection, you should close the output stream before getting input stream.

As Dian pointed out too, you are opening two connections - bad idea - but would not affect things in this case.

I would put together a small test app (hardcoded for ease of use), and pare things down until you have a working example. Then, build it back up to what you want in a dynamic version.

Don't just test it against your PHP page - try it against a plain HTML page first, or a JSP or something to check it is not actually your PHP script.

--------------------------------------------------
Free Java/J2EE Database Connection Pooling Software

http://www.primrose.org.uk

 

[developerinlondon]

ok got it working now at last. Aparantly it seems PHPs default session mechanism doesnt allow you to override the sessionid to an existing sessionid. So I had to override the default session mechanism first and now it all works quite well.

thanks for your help everyone.