Text file analyser 3

[GregLocock]

I have an ongoing need for a program that will read in a text file and deconstruct it so that the important numerical results can be extracted and dumped into a spreadhseet.

For example

Some header stuff
Run 17
Number of widgets 16
Volume 8.3, 4.7
lots of stuff of variable length that I don't need that may include numbers
Run 18
Number of widgets 19
Volume 8.7, 5.6
followed by lots of other stuff I don't need that may include numbers

Would produce the output

17
16
8.3
4.7
18
19
8.7
5.6

only. The file to drive this would just contain "Run" and 4, ie give me the next 4 numbers after you see the phrase Run.

Obviously I can sort of do this by pulling it into excel, but it tends to be fragile and need a lot of customising.

Is there a general purpose solution to this - I'm guessing a database? I should point out that the files are very variable in length and the order in which things occure, but each segment that I am interested in is of a regular structure.




Cheers

Greg Locock

SIGlease see FAQ731-376 for tips on how to make the best use of Eng-Tips.

 

[katmar]

I suspect ParseRat could do this for you, but it might take a bit of setting up

http://www.guysoftware.com/parserat.htm

 

[SomptingGuy]

This sounds like a job for a really trivial C program. If you are using a Linux machine, it doesn't get any easier to knock one up (a few minutes max). Less easy if using Windoze though.

- Steve

 

[SomptingGuy]

... if you have a compiler, I'm happy to post source here. It will have to wait until Tuesday though.

- Steve

 

[GregLocock]

Thanks dudes.

katmar- good try. see also Depeche View. I'll have a play tomorrow.

SG - Well, no I haven't got a C compiler (a nasty language) but that is not a restriction I placed, so go for it. I do have a C to Fortran converter (grins).

For what its worth I've got a matlab prototype, but it is long on horrible and short on flexibility



Cheers

Greg Locock

SIGlease see FAQ731-376 for tips on how to make the best use of Eng-Tips.

 

[Onda]

Hi Greg,

On my point of view You should use VisualBasic inside an Excel Spreadsheet.

Cheers

Onda

 

[IRstuff]

I've used FileParse, but it hasn't ever been updated:

http://sbutton.tripod.com/deadpig/

TTFN

FAQ731-376

 

[CajunCenturion]

Hey Greg, this would be a rather trivial task for any program or language that supports regular expressions.

Good Luck
--------------
As a circle of light increases so does the circumference of darkness around it. - Albert Einstein

 

[SomptingGuy]

Something like this should do the trick. It does assume that there is always a throwaway token at the start of each line, possibly a bad thing?


/******************************/
#include <stdio.h>
#include <stdlib.h>
#include <strings.h>

int main(int argc, char *argv[])
{
char *fname=argv[1];
char *key=argv[2];
int n=atoi(argv[3]);
char line[1024], *p;

int searching, count;
float value;

FILE *fp=fopen(fname,"r");

if(argc<4)
{
printf("Usage %s file key count\n", argv[0]);
return 1;
}

searching=1;
while(fgets(line, 1024, fp))
{
p=line;
if(searching && !(p=strstr(p,key))) continue;
searching=0;

strtok(p," ,"); /* Assume there is always a throwaway leading token */

while(p=strtok(NULL, " ,"))
{
if(sscanf(p, "%f", &value)==1)
{
printf("%g\n", value);
if(++count==n)
{
searching=1;
count=0;
break;
}
}
}
}
fclose(fp);
return 0;
}
/******************************/


- Steve

 

[GregLocock]

No soup for you SG, uncommented C is just cryptic crosswords for krazy kids.

I'll run it through the scruncherator/compiler tomorrow and see what it does. He says with his heart in his mouth.



Cheers

Greg Locock

SIGlease see FAQ731-376 for tips on how to make the best use of Eng-Tips.

 

[SomptingGuy]

Greg,

If you have Matlab for Windoze, then you have LCC. A really rubbish C compiler, but it does actually work.

Give me my dues though, there is one comment in that program ;-)

- Steve

 

[truckcab]

I think this can be solved quite easily with awk or even Perl. Very good for pattern matching which is really the task here. You can find a free Perl suite on the net that works great, dont remember the name though. Have it at home.



Live Long and Prosper !

 

[truckcab]

Hm... Active Perl, thats the suite I mean.



Live Long and Prosper !

 

[MikeHalloran]

There is a free awk for the PC, called, oddly, pc-awk or something like that. I knew how to use it once upon a time.



Mike Halloran
Pembroke Pines, FL, USA

 

[truckcab]

And of course, you can download Cygwin, including awk and perl, for free.



Live Long and Prosper !

 

[joerd]

And of course, gawk works fine on Windows (GNU-Awk)

Cheers,
Joerd

Please see FAQ731-376 for tips on how to make the best use of Eng-Tips.

 

[brengine]

How about this?

Ken

 

[GregLocock]

Thanks Ken, that works on a well formed text file.

Actually I screwed up, the output in the example I gave should only have been
17
16
8.3
4.7






Cheers

Greg Locock

SIGlease see FAQ731-376 for tips on how to make the best use of Eng-Tips.

 

[DanTex]

That should be relatively straight forward in most languages. You hint that you have a fortran compiler?

Open the file.

Read each line (1 at a time) into a long character string.

Use the INDEX() function to search for "RUN" until it is found.

After finding "RUN" search the remainder of that line and each additional line for numeric data (ie. just seach form digits and decimal pts or whatever ). When found, Do a READ on the numeric characters to get the data into a numeric data type.

Continue doing this with each line until the desired # of data items are found or the file comes to an end.

Dan

 

[yakpol]

Greg,
Here'a VBA code to read and analyze the text files. Executable sub: ReadFile.

Sub ReadFile()
Dim TextLine As String
Dim strWords(1 To 50) As String
Dim Nwords%, Nlines%, iLine%, iword%
Dim OutputNumber(1 To 1000)

Open Application.GetOpenFilename() For Input As #1    ' Open file.
Do While Not EOF(1)    ' Loop until end of file.
    Line Input #1, TextLine    ' Read line into variable.
    Nwords = SeparateWords(TextLine, strWords())
    iLine = iLine + 1
    For iword = 1 To Nwords
        If IsNumeric(strWords(iword)) Then
            OutputNumber(iLine) = Val(strWords(iword))
            Exit For
        Else
            OutputNumber(iLine) = ""
        End If
    Next i
    Range("A1")(iLine) = OutputNumber(iLine)
Loop
Close #1    ' Close file.


End Sub

Function SeparateWords(RightText As String, strWords() As String) As Integer

Dim iwords As Integer
'''
iwords = 0
'''
Do Until RightText = ""
    iwords = iwords + 1
    strWords(iwords) = NextWord(RightText)
Loop
SeparateWords = iwords

End Function


Private Function NextWord(ByRef tempText As String) As String
'''function returns string between the spaces and trims input string
Dim CharCounter As Long
'''
CharCounter = 1
tempText = LTrim(tempText)
Do While Mid(tempText, CharCounter, 1) <> " " And CharCounter <= Len(tempText)
    CharCounter = CharCounter + 1
Loop
NextWord = Mid(tempText, 1, CharCounter - 1)
tempText = Mid(tempText, CharCounter)
'''
End Function