Reading large text files ( about 10 MB )


DevX Home    Today's Headlines   Articles Archive   Tip Bank   Forums   

Results 1 to 3 of 3

Thread: Reading large text files ( about 10 MB )

Hybrid View

  1. #1
    Anand Guest

    Reading large text files ( about 10 MB )


    Friends,

    Iam trying to read text files and display them inside a scroll pane. The
    following code works absolutely fine for smaller file sizes upto 5-6 MB.
    But for larger files, an OutOfMemory error is thrown. Though I can understand
    its meaning - i just want to know if there is any better way of handling
    large files.

    NB: The same file loads perfect even using NotePad !!

    Thanks,
    Anand

    //code

    // build the scroll pane and the text area
    public void addScrollText() {

    // create the editor pane
    jePane = new JEditorPane();
    jePane.setEditable(false);

    // create the scroll pane
    jsPane = new JScrollPane(jePane);
    jsPane.setVerticalScrollBarPolicy(JScrollPane.VERTICAL_SCROLLBAR_ALWAYS);
    getContentPane().add(jsPane);
    }

    public void displayFile(File jfile) {
    try {
    FileInputStream fis = new FileInputStream(jfile);
    jePane.read(fis,jfile);
    } catch (IOException e) {
    System.out.println("IO Exception in display file");
    }
    }


  2. #2
    Ramsin Guest

    Re: Reading large text files ( about 10 MB )


    To make this more efficient try reading in bytes of data instead. Eg.

    InputStream file_reader = new BufferedInputStream(new FileInputStream
    (filename));

    now construct an array of bytes
    int BUFFER_SIZE = 4096;
    byte[] buffer = new byte[BUFFER_SIZE];

    and then to read 4096 bytes of data
    int bytesRead = file_reader.read(buffer, 0, BUFFER_SIZE);

    now if u want to turn buffer into a string
    String line = new String(buffer, 0, bytesRead);

    of course your displayFile(File jfile) function will need to be changed since
    in the file_reader above u need filename to be a string rather than of type
    File. I tested this with files of size greater than 26MB and it worked fine.
    Hope it helps.
    If you still have trouble just send me an email.


    "Anand" <theerthan@hotmail.com> wrote:
    >
    >Friends,
    >
    >Iam trying to read text files and display them inside a scroll pane. The
    >following code works absolutely fine for smaller file sizes upto 5-6 MB.
    >But for larger files, an OutOfMemory error is thrown. Though I can understand
    >its meaning - i just want to know if there is any better way of handling
    >large files.
    >
    >NB: The same file loads perfect even using NotePad !!
    >
    >Thanks,
    >Anand
    >
    >//code
    >
    > // build the scroll pane and the text area
    > public void addScrollText() {
    >
    > // create the editor pane
    > jePane = new JEditorPane();
    > jePane.setEditable(false);
    >
    > // create the scroll pane
    > jsPane = new JScrollPane(jePane);
    > jsPane.setVerticalScrollBarPolicy(JScrollPane.VERTICAL_SCROLLBAR_ALWAYS);
    > getContentPane().add(jsPane);
    > }
    >
    > public void displayFile(File jfile) {
    > try {
    > FileInputStream fis = new FileInputStream(jfile);
    > jePane.read(fis,jfile);
    > } catch (IOException e) {
    > System.out.println("IO Exception in display file");
    > }
    > }
    >



  3. #3
    Anand Guest

    Re: Reading large text files ( about 10 MB )


    Hi Ramsin,

    Thank you very much for your reply .

    I managed to it work by allocating more memory to the JVM, instead of changing
    the code, as my intention was just to display the file.

    java -Xms512m -Xmx512m <class-name>

    Thanks,
    Anand


    "Ramsin" <rbetyousef@hotmail.com> wrote:
    >
    >To make this more efficient try reading in bytes of data instead. Eg.
    >
    > InputStream file_reader = new BufferedInputStream(new FileInputStream
    > (filename));
    >
    >now construct an array of bytes
    > int BUFFER_SIZE = 4096;
    > byte[] buffer = new byte[BUFFER_SIZE];
    >
    >and then to read 4096 bytes of data
    > int bytesRead = file_reader.read(buffer, 0, BUFFER_SIZE);
    >
    >now if u want to turn buffer into a string
    > String line = new String(buffer, 0, bytesRead);
    >
    >of course your displayFile(File jfile) function will need to be changed

    since
    >in the file_reader above u need filename to be a string rather than of type
    >File. I tested this with files of size greater than 26MB and it worked fine.
    >Hope it helps.
    >If you still have trouble just send me an email.
    >
    >
    >"Anand" <theerthan@hotmail.com> wrote:
    >>
    >>Friends,
    >>
    >>Iam trying to read text files and display them inside a scroll pane. The
    >>following code works absolutely fine for smaller file sizes upto 5-6 MB.
    >>But for larger files, an OutOfMemory error is thrown. Though I can understand
    >>its meaning - i just want to know if there is any better way of handling
    >>large files.
    >>
    >>NB: The same file loads perfect even using NotePad !!
    >>
    >>Thanks,
    >>Anand
    >>
    >>//code
    >>
    >> // build the scroll pane and the text area
    >> public void addScrollText() {
    >>
    >> // create the editor pane
    >> jePane = new JEditorPane();
    >> jePane.setEditable(false);
    >>
    >> // create the scroll pane
    >> jsPane = new JScrollPane(jePane);
    >> jsPane.setVerticalScrollBarPolicy(JScrollPane.VERTICAL_SCROLLBAR_ALWAYS);
    >> getContentPane().add(jsPane);
    >> }
    >>
    >> public void displayFile(File jfile) {
    >> try {
    >> FileInputStream fis = new FileInputStream(jfile);
    >> jePane.read(fis,jfile);
    >> } catch (IOException e) {
    >> System.out.println("IO Exception in display file");
    >> }
    >> }
    >>

    >



Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
HTML5 Development Center
 
 
FAQ
Latest Articles
Java
.NET
XML
Database
Enterprise
Questions? Contact us.
C++
Web Development
Wireless
Latest Tips
Open Source


   Development Centers

   -- Android Development Center
   -- Cloud Development Project Center
   -- HTML5 Development Center
   -- Windows Mobile Development Center