1) I would use notepad+ to edit the text file ... cleaning it up.
Example,
change all / characters to spaces
change all , to spaces
change all | to spaces
change all \ to spaces
etc.
2) Get it cleaned up so that you only have the special characters you allow, like & _ . -
3) So now you'll have a text file, each line has the characters that what you want.
Some lines will have multiple spaces in between some words, but that's OK.
4) Now you'll use PHP to clean up the extra spaces.
PHP Code:
<?php
// Put your file into an array ... each line is one array element.
$data=file("lexicon.txt");
// Connect to your MySQL database here.
// Loop through the array of lines, clean up extra spaces, and insert into table ...
foreach($data as $line){
// Remove extra spaces so there is only one space.
$line = preg_replace('/\s+/', ' ', $line);
// Explode the line into separate elements
$parts=explode(" ",$line);
// Now you have some parts to write into your database table
// Example: Average NNP JJ NN
// $parts[0] = Average
// $parts[1] = NNP
// $parts[2] = JJ
// $parts[3] = NN
// insert $parts[0] for sure, and then the other columns if they exist.
}
?>
That's sort of what I would do. Using Notepad+ to edit files, PHP, HTML, CSS is the best text editor.
.