Reading big .csv files

[Robert42]

I'm trying (in Mathcad 11) to read a file in .csv format. File consists of 6100 3-character words, separated by commas. There are no carriage returns. (To be precise, it's an array of 3-digit hex numbers separated by commas).

The File-read component fails - error messsge suggests "check file is not in use by another programme". Works ok if file length is reduced to 4000 3-charcter words, or 6100 2-character words) so i suspect it's a file length problem.

Any ideas?

 

[GregLocock]

Split the file into two halves?



Cheers

Greg Locock

 

[IRstuff]

There may be something corrupted in your installation. The file read component can easily handle 64K elements.

You might try to use the repair option in the install disk or reinstall.

TTFN

 

[Robert42]

Thanks.
Not sure what tool I'd use to split the file, though I suppose I could do a manual split using a text editor. Might be a bit error-prone doing it manually.

Yes, I agree that the file read component can read big files. I can happily read a file with 300,000 numerical entries. Problem seems to be associated with the file being alphanumeric - ie I'm tring to get a vector of strings - or maybe being comma separated.

I get idential results with Mathcad 11 at work and Mathcad 200 at home, so problem seems to be with me, not with the installation.

Any thoughts?

 

[IRstuff]

My posting was based on reading a CSV file with 64K 3 character elements separated by commas.

TTFN

 

[Robert42]

Right, thanks ... could you post the file (or the relevant snippet of it) so's I can try it out? Mathcad 2000 would be good.

 

[IRstuff]

There's not much to it and it doesn't paste well:

ARR:---file input from i:\test.csv----
length(ARR)=65536
ARR[0="abc"
ARR[65535="abc"

I didn't try to get anything more complicated than that



TTFN

 

[Robert42]

Thanks.
Right, I've had another look at this. The problem is only there if the file is a single line, example:
cat,cat,cat,....
If it's:
cat
cat
cat

there's no problem.
Problem file can be created by
n:=0 .. 6000/Vn:="cat"/T:=transpose of V/save V, an then try to read it.
The failure occurs at 20,000 characters, including the commas. There's a work-around by using READBIN.

 

[IRstuff]

The issue as indicated by answers in the Mathcad collab to the same question is that that Excel, itself, cannot read more than 256 columns.

TTFN

 

[Robert42]

... err .... could you elucidate the bit about Excel? I puzzled as to why an Excel limitation might affect Mathcad.

I did search (but obviusly not well enough) the Collabatory. Could you point me to the relevant bit?