Operation/Interpolation on very big size test file
Good evening everybody,
I have some issues handling very large text files (from 500MB to 2GB) containing data that i need to use to perform calculations.
I made several tries but i always ended running of CPU memory...
I have a text file called S1_data.txt:
12344 99997 -75.5
12344 99998 -75.7
12344 99999 -75.3
12345 99997 -78.4
12345 99998 -75.8
...etc with about 1 or 2 millions lines like that. Basically the first column is X coordinate, second is Y coordinate and third is a value of water depth.
i want to create a VB program to determine the water depth at a new location X1, Y1. In our example, let say X1=12344.7 ; Y1=99998.2
My initial idea was to calculate the distance of this point to all the points in the file S1_data.txt, select the 4 smallest points the closest to my new location X1, Y1 and perform an interpolation between those 4 points to determine the new water depth.
If the list of points is short, lets say it works but as soon as the size of my file becomes much bigger...i face this CPU memory issue.
Thanks for your advices...
Last Post: 02-08-2010, 07:05 PM
Last Post: 04-19-2009, 05:01 AM
By Arjuna in forum Database
Last Post: 07-25-2007, 03:22 AM
Last Post: 11-06-2001, 09:43 AM
Last Post: 08-30-2001, 11:45 AM
Top DevX Stories
Easy Web Services with SQL Server 2005 HTTP Endpoints
JavaOne 2005: Java Platform Roadmap Focuses on Ease of Development, Sun Focuses on the "Free" in F.O.S.S.
Wed Yourself to UML with the Power of Associations
Microsoft to Add AJAX Capabilities to ASP.NET
IBM's Cloudscape Versus MySQL