PDA

View Full Version : mining data based on defined field values



AwsedreswA
Jan 1st, 2014, 10:46 AM
Please help, I know it can be done.

I have 60,075 .txt files(TAB delimited) that are created from ImageJ that contains the XY coordinates and RGB tuples for each frame(60075 frames) of a video. (Macro took me days to make, 16 hours to run, and ate up 360GB)

I need the RGB tuple information from one coordinate x=384, y=377 for each frame. (this information is not always found on the same line#, which is why I need a piece of logic to find the data)

I need this data in a new file that can be loaded into a spreadsheet. So it needs to have either TAB or comma separated values. The file being created should have 60076 lines of data.

Each RGB tuple file is named 00000.txt - 60075.txt
(G:\Frames as bitmap\00000.txt) <-where it sits for me

RGB tuple file example line;
0 (TAB) 0 (TAB) 113 (TAB) 0 (TAB) 255 {X Y R G B}

Desired new file example line
00000,255,255,255 or 00000 (TAB) 255 (TAB) 255 (TAB) 255

I would like for it to be a script rather than a compiled program so that it can be adapted to later uses easier.

This is not a joke, I need this to defend the Constitution of the USA.

If you can't move this ahead, could you please not respond to this thread, I'll get the email and go to my computer. It is not that fun to rush to my computer to see some one trying to boost their forum stats with a meaningless post.

After completing this step, I still have to do the statistical analysis.

Thank you for your time in advance.

*I tried to attach a real tuple file but its brokenish

rnd me
Jan 2nd, 2014, 07:41 PM
it's easily do-able, but we need more info.

why is the output 255 when the input is 113 or 0?

also show us a complete example file where the info is not on line 1.

AwsedreswA
Jan 5th, 2014, 02:00 PM
This is where I stand atm.

This code works but it is lacking a few things.

It would be nice to have some progress indicator

It would also be nice to crunch more data than just 1 set of coordinates. (assuming a lot harder) thinking maybe an array.

Used PERL. (activeperl for windows)


$out_file="C:\\perl\\scripts\\bluetime.txt"; # out_file is the output file name
#$out_file="C:\\perl\\scripts\\bluetime.txt"; # uncomment and put your source here!!!!
unlink $out_file; # erased (deletes) the output file FIRST,
# otherwise it will continue to append to it!

open ($output_file,">>",$out_file) or die "Could not open output_file!!!"; #open file for append

for ($i=1;$i<=60075;$i++) # files 00000.txt thru 60075.txt
{
$output_name = sprintf("%05d",$i); # first field in bluetime, 5 digits

$input_file = sprintf("G:\\Frames_as_RGB_Tuples\\%05d.txt",$i); # FQ input filename
# $input_file = sprintf("C:\\perl\\scripts\\%05d.txt",$i);

open ($infile, "<",$input_file) or die "Could not open input file $input_file!!!";

## "<" means read from, ">>" means append to, ">" means write to!

# printf("G:\\Frames_as_RGB_Tuples\\%s.txt\n", $result);

while(<$infile>) # read one line from infile, put in $_
{
if($_ =~ /^390\t379/)
{
# print $_;
# print sprintf("%s\t-->%s<--\n",$output_name,$_);
print $output_file sprintf("%s\t%s",$output_name,$_); #5 digits plus tuple
}
}

close $infile;

}

close $output_file;