Reading large text files ( about 10 MB )

[Anand]

Friends,

Iam trying to read text files and display them inside a scroll pane. The
following code works absolutely fine for smaller file sizes upto 5-6 MB.
But for larger files, an OutOfMemory error is thrown. Though I can understand
its meaning - i just want to know if there is any better way of handling
large files.

NB: The same file loads perfect even using NotePad !!

Thanks,
Anand

//code

// build the scroll pane and the text area
public void addScrollText() {

// create the editor pane
jePane = new JEditorPane();
jePane.setEditable(false);

// create the scroll pane
jsPane = new JScrollPane(jePane);
jsPane.setVerticalScrollBarPolicy(JScrollPane.VERTICAL_SCROLLBAR_ALWAYS);
getContentPane().add(jsPane);
}

public void displayFile(File jfile) {
try {
FileInputStream fis = new FileInputStream(jfile);
jePane.read(fis,jfile);
} catch (IOException e) {
System.out.println("IO Exception in display file");
}
}

[Ramsin]

To make this more efficient try reading in bytes of data instead. Eg.

InputStream file_reader = new BufferedInputStream(new FileInputStream
(filename));

now construct an array of bytes
int BUFFER_SIZE = 4096;
byte[] buffer = new byte[BUFFER_SIZE];

and then to read 4096 bytes of data
int bytesRead = file_reader.read(buffer, 0, BUFFER_SIZE);

now if u want to turn buffer into a string
String line = new String(buffer, 0, bytesRead);

of course your displayFile(File jfile) function will need to be changed since
in the file_reader above u need filename to be a string rather than of type
File. I tested this with files of size greater than 26MB and it worked fine.
Hope it helps.
If you still have trouble just send me an email.


"Anand" <theerthan@hotmail.com> wrote:
>
>Friends,
>
>Iam trying to read text files and display them inside a scroll pane. The
>following code works absolutely fine for smaller file sizes upto 5-6 MB.
>But for larger files, an OutOfMemory error is thrown. Though I can understand
>its meaning - i just want to know if there is any better way of handling
>large files.
>
>NB: The same file loads perfect even using NotePad !!
>
>Thanks,
>Anand
>
>//code
>
> // build the scroll pane and the text area
> public void addScrollText() {
>
> // create the editor pane
> jePane = new JEditorPane();
> jePane.setEditable(false);
>
> // create the scroll pane
> jsPane = new JScrollPane(jePane);
> jsPane.setVerticalScrollBarPolicy(JScrollPane.VERTICAL_SCROLLBAR_ALWAYS);
> getContentPane().add(jsPane);
> }
>
> public void displayFile(File jfile) {
> try {
> FileInputStream fis = new FileInputStream(jfile);
> jePane.read(fis,jfile);
> } catch (IOException e) {
> System.out.println("IO Exception in display file");
> }
> }
>

[Anand]

Hi Ramsin,

Thank you very much for your reply .

I managed to it work by allocating more memory to the JVM, instead of changing
the code, as my intention was just to display the file.

java -Xms512m -Xmx512m <class-name>

Thanks,
Anand


"Ramsin" <rbetyousef@hotmail.com> wrote:
>
>To make this more efficient try reading in bytes of data instead. Eg.
>
> InputStream file_reader = new BufferedInputStream(new FileInputStream
> (filename));
>
>now construct an array of bytes
> int BUFFER_SIZE = 4096;
> byte[] buffer = new byte[BUFFER_SIZE];
>
>and then to read 4096 bytes of data
> int bytesRead = file_reader.read(buffer, 0, BUFFER_SIZE);
>
>now if u want to turn buffer into a string
> String line = new String(buffer, 0, bytesRead);
>
>of course your displayFile(File jfile) function will need to be changed
since
>in the file_reader above u need filename to be a string rather than of type
>File. I tested this with files of size greater than 26MB and it worked fine.
>Hope it helps.
>If you still have trouble just send me an email.
>
>
>"Anand" <theerthan@hotmail.com> wrote:
>>
>>Friends,
>>
>>Iam trying to read text files and display them inside a scroll pane. The
>>following code works absolutely fine for smaller file sizes upto 5-6 MB.
>>But for larger files, an OutOfMemory error is thrown. Though I can understand
>>its meaning - i just want to know if there is any better way of handling
>>large files.
>>
>>NB: The same file loads perfect even using NotePad !!
>>
>>Thanks,
>>Anand
>>
>>//code
>>
>> // build the scroll pane and the text area
>> public void addScrollText() {
>>
>> // create the editor pane
>> jePane = new JEditorPane();
>> jePane.setEditable(false);
>>
>> // create the scroll pane
>> jsPane = new JScrollPane(jePane);
>> jsPane.setVerticalScrollBarPolicy(JScrollPane.VERTICAL_SCROLLBAR_ALWAYS);
>> getContentPane().add(jsPane);
>> }
>>
>> public void displayFile(File jfile) {
>> try {
>> FileInputStream fis = new FileInputStream(jfile);
>> jePane.read(fis,jfile);
>> } catch (IOException e) {
>> System.out.println("IO Exception in display file");
>> }
>> }
>>
>