Http

[freesoft_2000]

Hi everyone,

I am currently trying to do an application that does the upload as well the download of files from a http server that needs authentication.
The thing is i am not very sure where to begin and i am open to any suggestions.

Basically what i am trying to upload is a html file onto the http server and also have the capability to download back that file back to my computer if i need it. The server i am using does not support ftp so using the sun's ftp classes are out but i am not sure if there are classes that exist for the uploading and downloading of files from a http server.

I am not using servlets but trying to upload or download a simple html file to http server from a java application

Sample codings as well as informative links would be helpfull

Any help is greatly appreciated

Thank You

Yours Sincerely

Richard West

[sjalle]

This class does the job of downloading:

import java.net.*;
import java.io.*;

/**
 * Gets the contents of an url as a stringBuffer.
 *
 */

class SiteConn {
  URL url = null;
  public SiteConn(URL url) {
    this.url = url;
  }

  public StringBuffer getContents() throws Exception {
    StringBuffer buffer;
    String line;
    int responseCode;
    HttpURLConnection connection;
    InputStream input;
    BufferedReader dataInput;
    connection = (HttpURLConnection) url.openConnection();
    responseCode = connection.getResponseCode();
    if (responseCode != HttpURLConnection.HTTP_OK) {
      throw new Exception("HTTP response code: " +
                          String.valueOf(responseCode));
    }
    try {
      buffer = new StringBuffer();
      input = connection.getInputStream();
      dataInput = new BufferedReader(new InputStreamReader(input));
      while ( (line = dataInput.readLine()) != null) {
        buffer.append(line);
        buffer.append("\r\n");
      }
      input.close();
    }
    catch (Exception ex) {
      ex.printStackTrace(System.err);
      return null;
    }
    return buffer;
  }
}

[freesoft_2000]

Hi everyone,

Thanks Sjalle your above code is exactly what i need but the thing is that the http site that to am going to download the file from requires authentication(ie. a username and a password). How do i modify your code that does authentication before i can excess the stream

Any help is greatly appreciated

Thank You

Yours Sincerely

Richard West

[sjalle]

If the server wants the username, password and a file-ref (that is returned as a
stream to the client) you could just use a SiteConn instance, load it with a properly
formatted URL that contains these parameters and do a getContents(), or ...

[freesoft_2000]

Hi everyone,

One more thing i can use the InputStream class to download the content from a http source. Say for example if the URL of that website is "http://www.pol.com/tell.zip"

By using the URLConnection class and connecting to that above address, can it actually download that particular zip file down back to my disk by using streams??

Another thing is if i were needed to connect to that URL via a proxy how could that be achieved??

Some example codings would really be helpfull for the proxy question

Any help is greatly appreciated

Thank You

Yours Sincerely

Richard West

[sjalle]

zip download:

If the url points to a zip file and you just read it as a byte stream into a buffer (using
FileInputStream and not a BufferedReader) then you could just write that buffer to a
file, and the zip file will arrive. I haven't done this myself buf I cannot see why this
should not work.

[freesoft_2000]

Hi everyone,

If i were to set the http proxy for a particular url i did this

Properties systemProperties = System.getProperties();
systemProperties.setProperty("http.proxyHost", proxy);
systemProperties.setProperty("http.proxyPort", port);

Is the way that i did the best way to do it and did i do it correctly??

Richard West

[freesoft_2000]

Hi everyone,

Sorry to you Sjalle about this topic but i have one more question.

You see your program that you listed in the second reply you get the server replies as well as the html content of the website. But first please assume i have a zip file at this address http://www.yahoo.com/my_life_story.zip

if i do this

URL url1 = new URL("http://www.yahoo.com/my_life_story.zip");
URLConnection urlc = url1.openConnection();
urlc.setOutput(false);

FileOutputStream out = new FileOutputStream("C:/my_life_story.zip");
InputStream in = urlc.getInputStream();

//other codes to read that file to disk

Now basically my question is when i read that file as stream bytes and save it to disk will all the server commands also be saved because that's not what i want. I only want the zip file.

Any help is greatly appreciated

Thank You

Yours Sincerely

Richard West

[sjalle]

Check this piece of (crappy) code I hacked up..

import javax.swing.*;
import java.awt.*;
import java.awt.event.*;
import java.net.*;
import java.io.*;

public class FileGet extends JFrame implements ActionListener
{
  JLabel jLabel1 = new JLabel();
  JTextField urlTF = new JTextField();
  FlowLayout flowLayout1 = new FlowLayout();
  JButton getBtn = new JButton();
  JLabel repLbl = new JLabel();

  public FileGet() {
    try {
      jbInit();
      getBtn.addActionListener(this);
    }
    catch(Exception e) {
      e.printStackTrace();
    }
  }

  public static void main(String[] args) {
    FileGet fg = new FileGet();
    fg.addWindowListener(new WindowAdapter() {
      public void windowClosing(WindowEvent e) {
        System.exit(0);
      }
    });
    fg.pack();
    fg.setVisible(true);
  }
  public void actionPerformed(ActionEvent ae) {
    try {
      URL url = new URL(urlTF.getText().trim());
      SiteConnect sc = new SiteConnect(url);
      byte [] b =sc.getByteContents();
      repLbl.setText("Got "+b.length+" bytes");
      FileOutputStream out=new FileOutputStream("c:\\tmp\\data.zip");
      out.write(b);
      out.close();
    }
    catch (Exception ex) {
      ex.printStackTrace();
    }

  }
  private void jbInit() throws Exception {
    jLabel1.setRequestFocusEnabled(true);
    jLabel1.setText("Enter URL");
    urlTF.setText("http://some.kinda.host/test.zip");
    urlTF.setColumns(22);

    this.getContentPane().setLayout(flowLayout1);
    getBtn.setText("Get");
    repLbl.setBorder(BorderFactory.createLineBorder(Color.black));
    repLbl.setPreferredSize(new Dimension(350, 22));
    repLbl.setVerifyInputWhenFocusTarget(true);
    repLbl.setText("No process yet");
    this.getContentPane().add(jLabel1, null);
    this.getContentPane().add(urlTF, null);
    this.getContentPane().add(getBtn, null);
    this.getContentPane().add(repLbl, null);
  }
}
class SiteConnect {
  URL url = null;
  public SiteConnect(URL url) {
    this.url = url;
  }

  public byte[] getByteContents() throws Exception {
    int responseCode;
    HttpURLConnection connection;
    InputStream input;

    connection = (HttpURLConnection)url.openConnection();

    responseCode = connection.getResponseCode();
    System.out.println("HTTP response code:"+String.valueOf(responseCode));
    if (responseCode != HttpURLConnection.HTTP_OK) {
      throw new Exception("HTTP response code: " +
         String.valueOf(responseCode));
    }
    byte [] b=null;
    try {
      InputStream in = connection.getInputStream();
      int avail = in.available();
      while(avail==0) { // dirty loop...
        avail = in.available();
        Thread.currentThread().sleep(1000);
      }
      System.out.println("available in: "+this.url.getFile()+" is "+avail);
      b=new byte[avail];
      in.read(b);
      in.close();
    } catch (Exception ex) {
       ex.printStackTrace(System.err);
       return null;
    }
    return b;
  }
}

[freesoft_2000]

Hi everyone,

Sjalle i need to ask you something about you above code. Could you expalin in detail if its of no inconvinience what is the purpose of your below while loop in the above code

while(avail==0) 
{ 
// dirty loop...
avail = in.available();
Thread.currentThread().sleep(1000);
}

The avail is always zero and why do you pause the thread This is what it says in the inputstream api

"The available method for class InputStream always returns 0.
This method should be overridden by subclasses. "

Would that not cause the entire loop to run forever

I did this instead

URL url1 = new URL("http://www.yahoo.com/my_life_story.zip");
URLConnection urlc = url1.openConnection();
urlc.setOutput(false);

FileOutputStream out = new FileOutputStream("C:/my_life_story.zip");
InputStream in = urlc.getInputStream();

int len = 0;
byte[] buffer1 = new byte[1024]; 

while ((len = in.read(buffer1)) > 0)
{ 
out.write(buffer1, 0, len);
}
 
in.close(); 
out.close();

Now my question is when i read that file as stream bytes and save it to disk will all the server commands also be saved because that's not what i want. I only want the zip file.

Did you experience such thing before when you tried to download any file from a url using the URL class with java while getting its stream??

If you did is there a way to overcome it??

Any help is greatly appreciated

Thank You

Yours Sincerely

Richard West

[sjalle]

Well, I ran exactly that code from my pc and retrieved a zip file from my webhotel.
Since seeing is beleiving I say that the statement "The available method for class
InputStream always returns 0" is a lie. The loop is a hack, of course it has
the risk of running forever, and that is not clean at all I have it coded like that
because the avail may return 0 at (the) first attempt(s).

Your method looks much cleaner, and it works. I think my hack would fail for big zip files

btw, the urlc.setDoOutput(false) is redundant as this is the default.

Anyway, when the URL points to a zip file, and you load it like that, you only get the
zip file, nothing else. However, line trouble may cause the zip to get corrupted.

[freesoft_2000]

Hi everyone,

However, line trouble may cause the zip to get corrupted.

When you say line trouble you mean if the program or thread gets interrupted right or something is wrong to the connection to that website right??

If that is what you meant is there a way to overcome that??

Any help is greatly appreciated

Thank You

Yours Sincerely

Richard West

[sjalle]

Yes, but its rare, and the only remedy I know of is to try again .
Its hard to tell wether the zip is ok or not until the download is complete.