Empty Files

[brett7481]

Hey, I need some help. I have several Perl programs that access the same files text files. Many users use these programs, so when two people happen to access the same file at the same time with two different programs, the file becomes empty and all data is lost. I've tried flocking the files, and it is still occuring. I don't quite know what else to try, so I thought I would see what you guys might have to say. Thanks.
Brett

[mlseim]

Not that this is your problem, but make sure this is not happening....

Try to update the file from each program, one at a time, and make
sure one of your scripts isn't wiping out the file. It could be just a
syntax error in one of the scripts that is causing it.

... and are you using: flock(FILEHANDLE, 2) ;
example: flock("myfile.txt", 2);

There are 4 flock operators 1,2,4,8.
Using operator 2 locks the file until it's closed.
No need to unlock it, just close the file.

Lastly, it's also possible that your hosts version of Perl might not
support flock ... you would have to check with your webhost.

Here's a good tutorial on flock:
http://perlmonks.thepen.com/7058.html

[brett7481]

My problem is is that two instances of the programs will run at once and the two will get out of phase. One will open the file before the other has had a chance to lock it, and then the file gets reset back to an empty file.

I've read something about Semaphore files being able to fix the problem at: http://interglacial.com/~sburke/tpj/as_html/tpj23.html, but I don't quite understand it. Can you offer any help on this subject?

[mlseim]

I see what they are doing here ...

You need to create a new file and call it something like: empty.txt
This file will be empty.

Then, you first open the empty file and flock it:

open SEM, ">empty.txt"
or die "Can't write-open empty.txt: $!";
flock (SEM, 2);

Now, you can go ahead and do whatever you want to your
REAL file. After you update your REAL file, close SEM to
remove your flock. The key here is the code "2" which is
Exclusive-Lock. Opening a file to WRITE (not read) will
guarantee an Exclusive-Lock. If a program can successfully
lock the "empty" file (meaning nobody else has it locked),
it can then write to ANY file. If it can't lock the "empty"
file, it cannot write to ANY file. (that's what Exclusive means).

In a nutshell, all of the simultaneous messing-up will happen
with the "empty" file, and you don't care about that one.

--------------------------------------------------------

Then, they talk about not ONE program being run simultaneous,
but more than one program accessing the same data file.

Every program must first try to open the SAME "empty" file
for writing. If they can't open it (because another program
has exclusive lock), the program can't write to ANY file.

I found another example that is a little bit more clear:
http://perlguy.com/articles/locking.html

All the examples I found use flock classes: use Fcntl qw(:flock);
You may or may not be able to use these depending on your server.
I never tried it, but I think: flock (FILEHANDLE, 2); is the same thing.

[brett7481]

Hey, thanks for all your help thus far, I've tried implementing the semaphore files, but it is still not working. I'm including one of the programs below, the one that is giving me the most trouble. It is a sub program within a larger program. Maybe you can take a look at it and see what I am doing wrong. The program itself does what it should, it is just as I mentioned before that if two instances of it begin running at the same time the two call on the same file and corrupt it, resulting in an empty file.


sub arrive
{







## This Sub Program Causes Player's Forces To Arrive At Their Destinations
## After The Appropriate Amount Of Time


#### Open All The Player's Files

open (ALL, "+<$bpdata/players.txt");
flock(ALL, 2);
@account = <ALL>;
close (ALL);
@accounts = sort(@account);
chomp (@accounts);





foreach $accounts (@accounts)
{
#chop $accounts if ($accounts =~ /\n$/);
open (ACCOUNT, "+<$bpdata/$accounts.txt");
flock(ACCOUNT,2);
@userlines = <ACCOUNT>;

close (ACCOUNT);

$designation = @userlines[3];
chomp ($designation);




#### Open Each Player's Troops File

open (ALLTROOP, "+<$bpdata/players/$designation/troops.txt");
flock(ALLTROOP, 2);
@troop = <ALLTROOP>;

close (ALLTROOP);

@troops = sort(@troop);






foreach $troops (@troops)


{

chomp ($troops);

my $file = 'test_lock.txt';
my $SEMAPHORE = $file . '.lck';
open(S, ">$bpdata/$SEMAPHORE") or die "$SEMAPHORE: $!";
flock(S, 2) or die "flock() failed for $SEMAPHORE: $!";

open (USERFILE, "+<$bpdata/players/$designation/troops/$troops.txt");

@userfile = <USERFILE>;


$arrivaltime = @userfile[19];
chomp ($arrivaltime);

$currenttime = time;

#$currenttime = $currenttime1 + 16900;
#chomp ($currenttime);




if ($currenttime > $arrivaltime){$location = @userfile[20];}
else {$location = @userfile[1];}
chomp ($location);

if ($currenttime > $arrivaltime){$gridh = @userfile[21];}
else {$gridh = @userfile[2];}
chomp ($gridh);

if ($currenttime > $arrivaltime){$gridv = @userfile[22];}
else {$gridv = @userfile[3];}
chomp ($gridv);

if ($currenttime > $arrivaltime){$squareh = @userfile[23];}
else {$squareh = @userfile[4];}
chomp ($squareh);

if ($currenttime > $arrivaltime){$squarev = @userfile[24];}
else {$squarev = @userfile[5];}
chomp ($squarev);

if ($currenttime > $arrivaltime){$newstatus = @userfile[26];}
else {$newstatus = @userfile[27];}
chomp ($newstatus);


if ($currenttime > $arrivaltime){$newtime = 0;}
else {$newtime = $arrivaltime;}



seek (USERFILE,0,0);
truncate (USERFILE,0);
foreach $userfile (@userfile)
{
chop $userfile if ($userfile =~ /\n$/);
print USERFILE "$userfile\n";
}
close (USERFILE);
close S;


if (($arrivaltime ne "0") && ($currenttime > $arrivaltime)){



$sectormove = "7853-$gridh-$gridv";


$lineid = "$squareh-$squarev";
chomp ($lineid);

if ($lineid eq '1-1'){$line = 26;}
elsif ($lineid eq '1-2'){$line = 27;}
elsif ($lineid eq '1-3'){$line = 28;}
elsif ($lineid eq '1-4'){$line = 29;}
elsif ($lineid eq '1-5'){$line = 30;}
elsif ($lineid eq '2-1'){$line = 31;}
elsif ($lineid eq '2-2'){$line = 32;}
elsif ($lineid eq '2-3'){$line = 33;}
elsif ($lineid eq '2-4'){$line = 34;}
elsif ($lineid eq '2-5'){$line = 35;}
elsif ($lineid eq '3-1'){$line = 36;}
elsif ($lineid eq '3-2'){$line = 37;}
elsif ($lineid eq '3-3'){$line = 38;}
elsif ($lineid eq '3-4'){$line = 39;}
elsif ($lineid eq '3-5'){$line = 40;}
elsif ($lineid eq '4-1'){$line = 41;}
elsif ($lineid eq '4-2'){$line = 42;}
elsif ($lineid eq '4-3'){$line = 43;}
elsif ($lineid eq '4-4'){$line = 44;}
elsif ($lineid eq '4-5'){$line = 45;}
elsif ($lineid eq '5-1'){$line = 46;}
elsif ($lineid eq '5-2'){$line = 47;}
elsif ($lineid eq '5-3'){$line = 48;}
elsif ($lineid eq '5-4'){$line = 49;}
elsif ($lineid eq '5-5'){$line = 50;}
chomp ($line);

open (IMAGES, "+<$bpdata/lands/images_$sectormove.txt");
flock(IMAGES, LOCK_EX);
@images = <IMAGES>;

$newcount = @images[$line] + 1;


$images[$line] = "$newcount\n";
seek (IMAGES,0,0);
truncate (IMAGES,0);
foreach $images (@images)
{
chop $images if ($images =~ /\n$/);
print IMAGES "$images\n";
}
close (IMAGES);


}

my $file = 'test_lock.txt';
my $SEMAPHORE = $file . '.lck';
open(S, ">$bpdata/$SEMAPHORE") or die "$SEMAPHORE: $!";
flock(S, 2) or die "flock() failed for $SEMAPHORE: $!";

open (USERFILE, "+<$bpdata/players/$designation/troops/$troops.txt");

@userfile = <USERFILE>;

$userfile[1] = "$location\n";
$userfile[2] = "$gridh\n";
$userfile[3] = "$gridv\n";
$userfile[4] = "$squareh\n";
$userfile[5] = "$squarev\n";
$userfile[19] = "$newtime\n";
$userfile[27] = "$newstatus\n";

seek (USERFILE,0,0);
truncate (USERFILE,0);
foreach $userfile (@userfile)
{
chop $userfile if ($userfile =~ /\n$/);
print USERFILE "$userfile\n";
}
close (USERFILE);
close S;
$update = 1;

}




}



}

[mlseim]

Did you try something like this:
First, create a text file called "empty.txt" and upload it into your directory, CHMOD to 777.

If multiple programs use the same data files, you need to have the
same open SEM lines in those programs also.

I just noticed though that all of your file stuff here are "READS",
not "WRITES" ... you need to really do the SEM stuff with your
FILE WRITES. flock does not care about READING files.


## This Sub Program Causes Player's Forces To Arrive At Their Destinations
## After The Appropriate Amount Of Time

#### Open the empty file to create the flock ####

open SEM, ">empty.txt"
or die "Can't write-open empty.txt: $!";
flock (SEM, 2);

#### Open All The Player's Files

open (ALL, "+<$bpdata/players.txt");
#flock(ALL, 2); #this one is commented out (you don't need this line)
@account = <ALL>;
close (ALL);
@accounts = sort(@account);
chomp (@accounts);





foreach $accounts (@accounts)
{
#chop $accounts if ($accounts =~ /\n$/);
open (ACCOUNT, "+<$bpdata/$accounts.txt");
#flock(ACCOUNT,2); #this one is commented out (you don't need this line)

@userlines = <ACCOUNT>;

close (ACCOUNT);

$designation = @userlines[3];
chomp ($designation);




#### Open Each Player's Troops File

open (ALLTROOP, "+<$bpdata/players/$designation/troops.txt");
#flock(ALLTROOP, 2); #commented out (you don't need this line)
@troop = <ALLTROOP>;

close (ALLTROOP);

@troops = sort(@troop);

close (SEM); # close the SEM handle, removes the flock.

[brett7481]

I'll try that... I think I may have been making things more complicated than they needed to be. Thanks.

[mlseim]

Concentrate on the parts that WRITE to your data files ... not the READS.