DevX Home    Today's Headlines   Articles Archive   Tip Bank   Forums   

Results 1 to 2 of 2
  1. #1
    Join Date
    Dec 2005

    Session Tracking and HTTP Data Extraction using java.net

    Hi, I have a question for those of you who have experience with java.net, I would really appreciate your help.

    I'm trying to extract data from web pages using Java, I understand that if a web page is static then we can use HttpURLConnection to get inputstream and therefore read its contents.

    However the page I need to extract is dynamicly generated by a HTTP request query (using either GET or POST method), let's say the query is a HTTP GET request: http://www.myhost.com/search.jsp?par...is&param2=that , if I paste this query into a brower it would work just fine and return the results I want, but when I call this query through a URLConnection object it fails to retrieve the data because it complains the session is invalid, this doesn't seem to work even when I set the cookie into my request... :-(

    So has anyone done anything similar to this before? Can you give me any leads on this? Or do you know how to monitor the HTTP traffic goes in and out of your local machine so I can somehow send the request and intercept the response?

    Many thanks for your help!

  2. #2
    Join Date
    Dec 2005
    Just got the problem sorted. The reason it didn't work before is because I did not handle the page redirection, a http request send to the server gets redirected 6 times and everytime I need to pass back all the cookies obtained from previous redirection responses. Found an excellent article on cookie management and web query, http://www.javaworld.com/javaworld/j...26-cookie.html , this guy really has done it all.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
HTML5 Development Center
Latest Articles
Questions? Contact us.
Web Development
Latest Tips
Open Source

   Development Centers

   -- Android Development Center
   -- Cloud Development Project Center
   -- HTML5 Development Center
   -- Windows Mobile Development Center

We have made updates to our Privacy Policy to reflect the implementation of the General Data Protection Regulation.