View Full Version : Securing files on web server
12-23-2003, 05:03 PM
We want to make a number of custom reports available to clients to log in, and view according to their login. This is the health care industry, so they're rather touchy about allowing someone to view someone else's information, due to recent privacy laws going into effect.
These will be primarily pdf files, maybe some word, excel, or txt.
Anyway, obviously, I don't want to just drop them all in a directory, and only provide links to the ones that apply--because the rest are still able to be accessed by changing a url. A user could probably browse around if he/she wanted to and find someone else's information that way.
12-23-2003, 07:06 PM
You basically need to set up a password system on the site and give each user name a unique id.
You then in turn can assign that unique id to documents to show when the page loads.
I would like to help youout a bit more, but I am in a crunch with projects.
12-23-2003, 07:30 PM
My site is a username/password authenticated site. After logging in, we set a session variable. To view any pages, I just include a file at the top of every page checking for the value of this session variable. That's not a problem.
I then have a database table with their id #, file name, description, etc...and a path to the file.
My question is about displaying a list of the files that a user has access to. I have no problem with querying the database and seeing the list of files, and paths that they are authorized to view. If I display a page of 20 links, allowing the user to click and download any of the 20 files, wouldn't the user be able to view the path of the file, then be able to play with the URL a bit to try to download someone else's files? Am I making sense?
If they browse directly to the location of the file, there really is no way to authenticate if they're logged in. If they have the path to the file, they don't have to go to the page with the links first.
What you can do depends of your actual situation.
How is the relation between the files and viewers? a hierarchical permissionstructure ?
In any case, this will envolve a quite elaborate permissionsystem, with different lookup-paths depending on the users profile.
Using the querystring is not necessarely unsafe. --> encode the fileID using a sessionkey + set real low timout-times for their input + LOGGING!!!
Encodig will be easier then maintaining a matrix of the permissions for all users for all files
12-23-2003, 08:46 PM
raf....I hadn't really hashed out the actual directory structure. I do know that we've got probably 2000 users on the site. To create a seperate subfolder for each would not be feasible.
We will likely have a community-type folder, with all the applicable reports in that.
Why not make the user go to another page after selecting a file, then after verifying their credentials, redirect to the file? They would not be able to see the actual path to the file, and there would not be a page to let them bookmark.
If the files are stored on a different server, can I still redirect to it? I need to set it up as a virtual directory...right?
12-23-2003, 08:52 PM
Are these static reports generated say once a week or once a month and then uploaded to the server? or are they dynamic and will change as information in the database is added?
If the reports are driven by information stored in a database then you most likely have a unique identifier for each client that is associated with each record in the database. What I have done in similar situations is in the links to the various reports instead of writing all of the info into the querystring, I omit part of it, and instead will get that info from the session variable.
For example let's say that user foo has the unique identifier of 1267 and bar has the unique identifier of 3658.
The url string will look like href = "some_report.asp?name=foo" .
Then when the user clicks the link and goes to some_report.asp, I not only look for the name value, in this case foo, but also compare it to the session variable. If the variable is equal to 1267 they are given the report, if not they are sent a notice that says they are unauthorized to view the record. (you could also redirect to another page).
I will also do something similar in reports where the client will have users with different access levels. The access level is stored as a session variable. If they don't have the value of 1267 plus then have the access level associated or higher than that of the file they are requesting they are not allowed access to it.
Now if they are static then I would use multiple directories and restrict access to the directory that the file is stored in using ntfs permissions.
12-23-2003, 09:01 PM
They will be reports that will be generated, and then dropped into the file for viewing until a certain date, which is stored in a database table. So...they might need to be available for a few weeks, or even a month or two.
They will be fairly standard, in that the same reports will be generated for different users--different content, of course.
If I'm redirecting to the file, would I need to set up the directory as a virtual directory of the site?
12-23-2003, 10:19 PM
Put the files into the database as blobs and then the page which reads them from the database and sends them to the user can also check and make sure that exact user is supposed to be able to request that exact report, otherwise you can log an invalid access request.
That saves you from having to develop an elaborate directory struture and the files are not laying around in a directory where someone can "fish" for reports they aren't supposed to see.
Originally posted by BigDaddy
raf....I hadn't really hashed out the actual directory structure. I do know that we've got probably 2000 users on the site. To create a seperate subfolder for each would not be feasible.
:confused: :confused: Where do you get the subfolder idea?
What I have done in similar situations is in the links to the various reports instead of writing all of the info into the querystring, I omit part of it, and instead will get that info from the session variable.
I don't see the point in that. Just storing the userID in a sessionvariable will give you the same.
If it needs to be safer, then you better have a sessiontable inside your db, that holds the userID and sessionID. You then run a select against this sessiontable (joined with the usertabel) to find out if this client has sufficient permission.
But thus far, you didn't give any info on how you define which users can see which file. If this is not the problem, and you can look up in the db if that user is allowed to request that file, then i don't see your problem?
From your initial posts, i deduct you are concerned that the users may change the querystringvalue. Well, then just encode (with a sessionkey or a session-specific salt yhat you store inside the db or sessionvariable) and you set the timeoutvalues low enough to evoyd bruteforcing-attacks?
So your link to file 12u452.doc
will look like
<a href="loadpage.asp?pid=zesdfsdt5et6ertdf54dgdfg57g">Your file</a>
which can then be decoded inside loadpage.asp to 12u452.doc and you then load the page.
If they want to fix that, then they need to guess your encoding algoritme + guess you session-specific salt + a valid fielname they would have access to.
You can make it even more secure by first 'clearing' the pages --> when you send a link (to the files) to the client, you register these inside the sessiontable (in a bar delimited list for instance, like 12u451.doc|12u452.doc|12u453.doc that you store inside column 'cleared'). Then, inside loadpage.asp, you select the value from the 'cleared' of the record with ASPsessionID = the asp-sessionID.
Then create an array with explode on the | and you check if the decoded fileadress in an element of the array. If so, well, load the page. If not, then they are trying to change the querystring, or they jumped back to the linkpage.
Then you delete the value of 'cleared'.
Maybe this last mechanisme alone is already sufficient for your situation.
(Oh yeah, put the files above the webroot and mail them to the user instead of loading them, would also improve security, and puts the main risks at their end (securing their mailbox)
12-24-2003, 03:49 PM
Thanks for the help. To display the file, don't I need to link to or redirect to the file itself?
How would I display a pdf or whatever w/out actually redirecting to the page? I'm concerned that if I do that, that the user will just be able to bookmark the location, or change the url--since they will have the path already to the folder where I'm keeping the files.
Maybe I'm just being extremely clueless and hard-headed, here, and there's a really simple answer.
To display the file, don't I need to link to or redirect to the file itself?
To the webserver, there is no difference between a redirect and a regular request from the client. So if you first decode the querystring and then redirect; it will be just the same as if a client hit a link to the files url.
The big problem for your situation, is that you can't have a check inside the file and you cant wrap something around it to make it secure enough...
The easiest ways out are:
- passwordprotect all files (if possible in your situation)
- choose realy long filenames ! like 2 encoded values sticked together. If you have filenames that are 52 positions long, then it becomes very unlikely that someone can get another valid filename.
(you compose the filenames be encoding 2 values that you store inside the filestable. Say your file normally is named test.asp, with file-specific salt = dqsd41qsd and an extra random value for the second hash= sdfs4772df. well, then you store test.asp etc inside the db, alongside with result of a hashfunction on that filename (say sdfgsdfs2d4fsdf45sfsdf3sd438sd4f8s7d8082sdf72sd2f877dsfsdfd7f4sdf22sdf24sd2fsdf). You then name the file sdfgsdfs2d4fsdf45sfsdf3sd438sd4f8s7d8082sdf72sd2f877dsfsdfd7f4sdf22sdf24sd2fsdf.pdf.
To get another file, you can either try to get another valid filename with bruteforcing (good luck) or you need to be able to break the hashfunction (quit impossible) and then geuss the normal filenames structure.
- or, you could name the files the same as the users password (but this will only work if each user has his own files, that should not be accesible to others) So only the 'owner' can now the filenames (since he is the only one that knows his password
- mail the file (or require downloads (if you can hide the source-adress, don't know if that is possible + only an idiot would download files from an unknown source)). This way, they can see the filescontent withou being able to request them on the server (the files should then be placed above the webroot so that they are not accesible from the web)
- else you need to work with some sort of tickets (like in a Kerberos environment) where a client gets a ticket to request that specific file on that server, within a specified timelimit.
But i never worked with it and i don't know how to implement in in a webenvironment. But it's probably doable.
12-24-2003, 06:46 PM
Thanks for the ideas. I'm thinking I'll probably use a random type name for the files.
Email is out--that's the reason we wtd to make them available on the web--because we need to have them secure. Since we don't want to make people have secure email, and we have an SSL protected site, we figured we'd go with that.
or require downloads (if you can hide the source-adress, don't know if that is possible + only an idiot would download files from an unknown source)). This way, they can see the filescontent withou being able to request them on the server (the files should then be placed above the webroot so that they are not accesible from the web)
How do I make them download w/out viewing it in browser? That confused me a bit. I thought you just sent them to the file's location.
12-25-2003, 04:18 AM
Out of all of these ideas, I must say raf's have merit and will probably work, but Roy Sinclair's sounds the most secure to me since the file is stored as a blob file in a database and probably rendered on the fly.
After giving that some thought, I still like Roy Sinclair's idea the best... at least that way there IS NO FILE to actually link to if I understand what he's getting at. It can be sent as a content stream that way, and not actually be a physical file... right Roy?
12-25-2003, 04:32 AM
After giving this additional thought, that HAS to be the most secure way to go... unless I missed something.
Random file names might be satisfactory to the client but there's still the possibility of someone randomly guessing a file name, or brute forcing it- although that's highly unlikely.
If you can just send the file itself as streaming content from the database, then it shouldn't work again, especially if you COMBINE that idea with some of raf's ideas, like session variables, salt, or whatever. I think it's totally doable... I can see it in my head, anyway. And that's usually the first clue that something will work (for me, anyway)... if I can envision it, it can most likely be done. :p
This still has the possibility of falling short on the client's side, as they might not have very good passwords, or whatever... but in that case it's not really your responsibility if you can make these ideas work. With the files stored in a database as blobs, if any issues DID come up, you should already be tracking each download by user, by date, etc. and even ip address... that way you have some CYA... :)
Anyone here agree (or disagree) that this is the right track?
I wish I had the time at my company to focus on security issues like this, even to just brainstorm them. :eek:
I doubt if blobs can be realy secured.
I don't know the exact reference anymore, but one of the reasons not to use blobs (compaired to seperate files) was that you did not needed any accesrights to the db to request them. Just the resource indicator (the ID) to the blob was enough to request and get it. So since blobs can be requested without having select-permissions ....
But it could indeed be probably be the best way, if you encode the content and generate the file on the fly, it is without any doubt the safest way. But that is kind of a different issue then what you asked. I'm not sure if all your files could be encoded and stored as blob --> depends a bit on where they are generated.
The only settup where blogs will provide extra security, is if you encode all content (with a file specific salt) AND if you have a permissionstable inside your db, with 1 record for each user and file he has permission to.
combinID | userID | fileID
1 | 2 | 125
2 | 2 |189
3 | 3 | 568
To the client, you can then send links with the combinID in, like
In getfile.asp, you then first look up who the user is (indide your sessiontable) and then you check if the record with combinID=1 has that userID, and if so, you look up the filespecific salt (with the fileID) and then the blob etc.
So you have an intermediate table and you don't use a direct reference to the blod, inside your links (or forms cause thats basically the same) but the user-file combination ID
Next problem then is: how are you going to build and maintain that table?
The SSL doesn't do much more then encoding-decoding your content for transport to and from the client + adds some authentication. It wount give you extra security in that you can block a logged in client to request other pages then the ones you provide links to, but it allows you to identifythe client where a request came from
12-26-2003, 09:16 PM
"I can't believe that there's no way to prevent users from browsing to a file....other companies do it..."
Man. That sucks to hear your boss say other companies can do something that we can't do...and imply that it's a failing of the web developer if we can't do it. How do other companies do it? How does a download site allow some folks to download something, but not others?
How does buymusic.com keep someone from bookmarking the download site for a song after paying for it--then distributing it to others--or fishing for other titles?
I've given the boss the options:
1. Put them in a database.
2. Redirect to the file itself, using really long, hard-to-guess, dynamic file names based on a person's personal id.
He seems to think that there's a way that we can set up a secure way using IIS that would allow some folks access to certain directories, but not others. The guy doesn't understand that all users browsing a web site have access to the same stuff--and it doesn't matter where they're coming from or who they are.
Suppose you set up an ftp-site or a regulard directory under the webroot (mysite.com/tempfiles/) which is initialy empty.
Your files are all stored above the webroot in a non-accesible location.
Now the user logs in and wants to browse his files.
You run a select on your db to see which files he has acces to and supply links for these files (all files point to one file (proces.asp with the file-id in the querystring).
In proces.asp, you run a control-select against the db to see if that user has permission to request that file.
If so, then you copy the file to this ftp-site or the empty directory, and you change the name into a completely meaningles code (qrdgqfgdf.doc). After the copy-rename, you redirect the user to ./tempfiles/qrdgqfgdf.doc .
And then you delete tempfiles/qrdgqfgdf.doc (or you delete it after 2 minutes or so.)
This should be completely safe, because the files are only available for a very short period (if you make the filesnames 20 character long, then that is probably enough to counter any bruteforce attack.)
And it can be made completely safe, because the only way to get the completely maeningless filename is through an asp page, and that you can secure with your server side security.
Bookmarking the files is then pointles since they are just removed. If you want to wrap it up completely, then you can probably check somehow to see if the file is completely sent, but just removing it after 1 or 2 minutes will probably do (you need to check the loadingtime under 56k modems to set your limit.)
12-26-2003, 11:38 PM
You need some sort of serverside cleanupproces that is trigered by a useraction.
For instance, the next pagerequest.
Say you record the filename (the new, random one) in your db (a variable like "fname" inside the usertable or a sessiontable). Then on each pagerequest you check if there is a value in "fname", and if so, delte the file with that name.
If the user doesn't request a new page, then you can delete the file inside your global.asa's sessiononend-sub. (a bit unsafe though cause with a bit a knowledge you can keep the session alive for quite a while
Or you can have a sort of garbage-action each 5 minutes, triggered by a CRON job or scheduled task, or by another users request (for instance inside your global.asa or in any other file realy --> store the last time you did a filecleanup inside an application-variable, and when a new user connects (if you use the global.asa) or there is a pagereuest (check on your main page or so), check if that value is more then 5 minutes old. If so, redirect to another page that selects all files that were requested since time-2minutes and deletes them, and then redirect the user to his requested page.
It will have a smal performance impact for the pagerequest of that one user...
For audit-reasons, it might be a good idea to store all cleared files inside a table (with the normal filename, the new filename, the sessionID ror userID) that requested it, the time it was cleared, the time it was removed again) and to work on this table to clean up 'expired files'.
12-27-2003, 08:52 PM
Blobs can be secured as long as you can verify that the user is who the user says they are.
Since the original thesis was that you knew who the users were already and already had the information on what each user was supposed to be able to see contained in a database I suggested using blobs as a way of keeping the content unavailable except for the people who should be able to see it.
12-29-2003, 08:31 PM
How about getting the path of the file or filename from the database, reading in the binary, then writing it out?
Sort of like taking the file, storing it as a blob file, then pulling it out, and writing it--but without the blob file part. Think that'd work?
Make sense? Any chance you can help me figure out how to read a file to binary, then write it out?
01-05-2004, 09:43 PM
See this (http://www.codingforums.com/showthread.php?s=&threadid=23667) thread for more information on handling blobs from ASP.
01-05-2004, 09:51 PM
Actually, Roy, I'm using a page based on your code to take the file, read it into a stream, then write out from the stream (skipping the insert part). It seems to work fairly well.
I haven't decided yet if I want to do that, or do a copy and redirect. With the copy, I need to find a way to do some trash cleanup later. It might be something as simple as a scheduled task that runs every night -- or me manually clearing out the folder every morning when I get to work (someday if I'm not here... :cool: ). With the loading it from the stream, it gets a little more complicated to make sure I get the response.contenttype set correctly.
I don't know...further testing will help determine that. The important thing for the time being is just getting it to work. Our file server won't let me copy off of it right now. I can copy off of either of 2 other servers...just not our file server for some reason--including reading from the file system into a stream.