View Full Version : data storage online and off

01-24-2013, 04:11 AM
I hope this is the right place to ask.

I am trying to implement (or pay someone else to start implementing) a storage solution for data entered into a medium size web of pages.

The pages can either be stored on a local network or hosted on the internet. Either way, some sort of data collection process is required. I understand mySQL is the way to do it on a website online. (I use several mySQLs already on my own website - just a basic user). I am not sure about the intranet. Would mySQL still work?

The scenarios are these:

CASE A: say, 10 users on a network accessing a local bunch of pages (intranet).

They login and their individual state persists each time they login because of a local storage system.

CASE B: users from anywhere access an internet website.

They login and their individual state persists each time they login because of a *local* storage system.

CASE C: users from anywhere access an internet website.

They login and their individual state persists each time they login because of a server-side storage system.

Can anybody advise where I should be heading? I don't really want two sets of the pages (there are 100s of them) one for online and one for the intranet.

I don't really know where to begin. I have been dabbling with localStorage and jstorage (lawnchair etc.) but it seems there are security issues with these methods.

Any advice is welcome.

Old Pedant
01-24-2013, 08:09 AM
Your case B is almost surely a really bad idea:

They login and their individual state persists each time they login because of a *local* storage system.

You have just put your entire system--your entire database--at risk because you have NO CONTROL AT ALL over the security of a user's local storage system.

Unless your understanding of "local storage system" and mine are widely different, I would never allow this option.

Oh...and for an INTRANET scenario, if you trust your network security, you really wouldn't even need users to login: You could just inspect their IP address and connect them accordingly. Or use some other kind of integrated authentication, depending on the kind of network and the operating system(s) in use.

Old Pedant
01-24-2013, 08:15 AM
This phrase is...confusing...

their individual state persists each time they login

I'm not sure if you are saying their state persists *BETWEEN* logins or only so long as they are logged in.

And if you only mean so long as they are logged in, then why do you *need* to distinguish between inTRAnet and inTERnet clients? Why not simply always maintain session state on the server? That's how one of the sites I help maintain works. It doesn't care how you got to the login page; it just remembers you with standard session state. (It happens to be an ASP.NET site, and ASP.NET allows both cookie-based and cookieless session state, but because this is not truly a public site, we insist on cookie-based sessions. Just because it's simpler, not better.)

01-24-2013, 08:35 AM
Thanks OP. I am still considering things you see.

CASE B seems to me no different to many a website that remembers your answers to on-site questions, say, because of cookies. But cookies have severe limitations for me.

In all cases, the idea is to save any work that an individual user has done between logins, not just as long as they are logged on.

01-24-2013, 08:47 AM
OP, wouldn't the use of cookies on an intranet (where the site resides on the, say, C Drive) mean that individual sessions could not exist?

One user's answers would wipe out another's, wouldn't they?

Old Pedant
01-24-2013, 08:27 PM
OP, wouldn't the use of cookies on an intranet (where the site resides on the, say, C Drive) mean that individual sessions could not exist?

One user's answers would wipe out another's, wouldn't they?

Cookies are not saved on the *SITE*. They are saved on each user's personal computer.

You and I must have a very different view of a typical intranet. The one I work with, as an example, has two central servers (one is just for the database, one is for the web server as well as various other applications) and about 60 individual workstations. At this company, each person is assigned his/her own workstation and, except for supervisors, no one is allowed to use another person's workstation. So, for all practical purposes, it is no different than an internet (just that nobody can connect from the outside world except through a tightly controlled and ip-address-managed firewall). An employee turns on his/her workstation, brings up a browser, and hits the web site using an internal URL (that is, something like http://mainserver, rather than specifying a www. or .com or whatever external URL...the local DNS server of course knows about all the internal "sites" and so never goes outside the local network to connect). And when the connection is made, the "mainserver" puts a cookie on the individual user's computer that identifies his/her session.

All in all, no different at all than an inTERnet connection.

You say you need to avoid cookies: I would hope and presume you mean that applies to external users. Clearly, for inTRAnet users, you should be able to dictate what they will and must do, which would include keeping cookies turned on at all times.

Cookieless session state isn't too hard to do, but it is even more intrusive than cookies, in my opinion. There are really only two ways to do it: (1) *ALL* movement between pages *MUST* be done via <form> submittal, and usually is done with <form method="post">, and the encrypted session identifier is stored in an <input type="hidden"> field in the <form>. (2) The encrypted session identifier is passed as part of a query string in every moverment from page to page. This is a common mechanism in some JSP frameworks, where you will see "xxx.do?sessionid=818X!GMaa9Zaa" or worse.

Remember, in either of these cookieless scenarios, *ALL* page to page movement must be done this way. So even a navigation menu would have to be something like

<a href="home.php?sessionid=818X!GMaa9Zaa"> Home </a>
<a href="catalog.php?sessionid=818X!GMaa9Zaa"> Catalog </a>
<a href="contact.php?sessionid=818X!GMaa9Zaa"> Contact Us </a>

or, using forms:

<form action="transfer.php">
<input type="hidden" name="sessionid" value="818X!GMaa9Zaa"/>
<input type="submit" name="gotoPage" value="Home"/>
<input type="submit" name="gotoPage" value="Catalog"/>
<input type="submit" name="gotoPage" value="Contact Us"/>
(And then "transfer.php" examines the value of $_POST["gotoPage"] and indeed transfers to the appropriate page.)

And so on. If you ever make a movement to another page *without* passing the sessionid, then the entire session connection is lost and can only be restored by logging in again.

Are you *SURE* you want to try to do this without cookies?

Old Pedant
01-24-2013, 10:33 PM
Ahhh...I just saw your query about jStorage in the JS forum.

I think that is barking up an entirely wrong tree.

What happens if a user logs in at the office, does some work, logs out.
Then he goes home and logs in from home, from a different computer.
Or he wants to check something with his iPad or iPhone.

*ANY* kind of local storage is *USELESS* for this kind of thing, in my never overly humble opinion. I simply can *NOT* get excited about storing anything important to any kind of real-world business situation on the user's local computer.

If you want to store the current progress/state of a user's work and you want it to be associated with his/her *LOGIN* and not just a specific single computer, then you *MUST* store it on the server. Period.

The local storage option might have been okay 10 years ago, when almost nobody had more than one computer or device that they logged in from, but it's an utterly CRAP solution into today's world and will only get to be a worse answer as time goes on.

Heck, even here at home, I occasionally login from my wife's Android pad rather than walk downstairs to my computer. I would be *ENORMOUSLY* annoyed at any system that depended on local storage to remember where I left off.

01-25-2013, 01:43 AM
Very succint explanations - thank you.

Cookies, as far as I can tell, will quickly run out of space for the amount of saved data I am envisaging. And while I perfectly understand your need to log in from a number of different devices, it won't really affect the intranet scenario - in which, in my case, no computer would be assigned to a particular person. The discreet part of it would need to be done with login passwords.

What about a MySQL database? Forgive my ignorance, but how would I go about exploring whether answers to questions could be saved to such a database? Doesn't it require non-html pages? I really am in the dark here, I'm afraid.

Old Pedant
01-25-2013, 02:44 AM
My head hurts.

it won't really affect the intranet scenario - in which, in my case, no computer would be assigned to a particular person. The discreet part of it would need to be done with login passwords.

So you will have to store the internet-based data on the server, accessed via the login password.

And if you are going to have to build that infrastructure, *ANYWAY*, why would you want to do it any differently for inTRAnet users????

I will grant that you *COULD* do it differently, but why build two completely different systems for the same purpose?


Your question about MySQL I don't understand at all. Are you asking if you could (a) install MySQL onto each and ever inTRAnet user's machine and then (b) from the browser, store data into the MySQL DB that is local to the user's machine????

The answer to (a) is "Of course." Just are you sure you really want to be responsible for maintaining all those databases. I *guarantee* that some users will find out that they have MySQL installed on their machines and start using the DB for their own purposes. You could make it really difficult for them to do this, but it just makes for an incredible systems administration nightmare.

The answer to (b) is "Maybe". You certainly could do it if you required the users to use only MSIE browsers. Doing it with other browsers is more problematical. I *think* you could do it by using JAVA (not JavaScript) applets assuming that you can set the security levels low enough to allow such interaction between the applet and the DB.

But I will ask again: WHY? It certainly doesn't help the problem of one person logging in from multiple locations and/or machines one tiny iota.

Can you possibly explain why you find the idea of storing data on the local machines so attractive? Perhaps the correct solution, if you are adamant about this, is to *NOT* use a browser-based answer, at all. Simply create a stand-alone executable, with an embededded database engine, and distribute the ".exe" package to all users.

01-25-2013, 03:26 AM
I have absolutely no preconceptions about this! If cookies will do it, I am happy to bake the project. Naturally, I don't want to set up two solutions if one will suffice for both the intra and internet situations. (The intranet would have the same pages but I am told that the internet will not be available from that network.)

In short, this is the problem seeking a solution: I currently have about 150 html pages each with multiple choice and/or text input options - the inputs to which I would like to save from session to session for both intra and internet scenarios. Ideally, each user should be available to return to his/her session state simply by logging in, but not from the same machine.

I understand how to write and retrieve cookie information but I am certain that the limits would quickly be reached.

Couldn't just one MySQL database for each of both situations handle the job server-side? If so, where I do I start to learn how to write the code to save the answers to the db?

I'm sorry to be such a noob to this. But once I get going on something I am reasonably capable.

01-25-2013, 03:40 AM
I ought to add that by returning to a session state I don't mean at the page they left off. Just with all the answers and text entered remembered.

Old Pedant
01-25-2013, 04:03 AM
Oooshhh... I had no idea you didn't know how to save the data in a DB. Explains a lot.

Okay...an important question: After the user has completed all 150 pages, do you need to continue to remember all the answers from all pages? (I would kind of assume so, else why collect the answers in the first place, but...)

There are several simplistic approaches, but whether they match your needs I can't tell.

Simplest of all:

CREATE TABLE questions(
pageNumber INT,
questionNumber INT,
question VARCHAR(2000),
PRIMARY KEY( pageNumber, questionNumber )

username VARCHAR(50),
password VARCHAR(50)

CREATE TABLE userAnswers(
userid INT,
pageNumber INT,
questionNumber INT,
answer VARCHAR(1000),
CONSTRAINT FOREIGN KEY (pageNumber, questionNumber)
REFERENCES questions(pageNumber, questionNumber)

That says: Every page gets a page number (could of course be a page name if you preferred). Every question on each page gets a question number.

Every user has an id. When they login, you lookup that id by username and password. (If not found, they can't log in.) When found, you store the userid in a session variable. THAT IS THE ONLY DEPENDENCE on cookies you have: The userid stored in a session variable, on the server.

When they answer the questions on a page, you add all their answers to the answer table. Each answer is identified by the userid along with, of course, its page number and question number.

Simple as that.

At *ANY* time, you can go look in the database to see what answer *IF ANY* the current user has given to any question.

So if they were in the middle of answering questions on page 73 and needed to stop for the day, they hit the SAVE button. You save all their answers on that page. (You might also save their current page number in a separate table, so that they don't have to remember it.)

Tomorrow, they login. They ask for page 73 (or you remember that's where they left off), you bring it up, but in the process you fill in all the answers they had already made yesterday.

What could be simpler?

It's not terribly sophisticated, because it doesn't allow for questions with multiple answers, for example. And there are many other improvements possible. But it's an easy starting point.

Old Pedant
01-25-2013, 04:07 AM
Oh, and one database could easily handle BOTH intranet and internet, but if there is some reason you think they need to be separate databases, that's trivial. A single MySQL database server can create and remember hundreds if not thousands of databases. Though I would hope nobody would actually do that. I consider maybe a dozen databases per server to be a reasonable number. One server I work with has 8 at the current time and at least 3 of those databases get maybe 5 hits per second, each, and MySQL isn't even breathing hard.

01-25-2013, 04:52 AM
<After the user has completed all 150 pages, do you need to continue to remember all the answers from all pages?> Yes, that's it!

I have pm'ed you, OP. Can't thank you enough.