large arraylist running out of memory, options?

[edzillion]

Hi

I have a program that needs to load a very large csv file (2meg file, 192718 items) - I have a loop going through adding each item to as a new arraylist item.

while ((line = bufRdr.readLine()) != null) {
				StringTokenizer st = new StringTokenizer(line, "\n");
				while (st.hasMoreTokens()) {
					// get next token and store it in the array
					String word = st.nextToken().toUpperCase();
					boolean containsApostrophe = false;
					if(word.indexOf('\'') != -1)
						containsApostrophe = true;
					if (word.length() > 3 && !containsApostrophe)
					{
						char[] wordCh = word.toCharArray();
						Word newWord = new Word(wordCh);
						words.add(newWord);
						System.out.println(words.size());
					}
				}
			}

After about 17 thousand the console (in eclipse) starts displaying memory addresses and nothing else, eg:

com.package.example.Word@1e41969, com.package.example.Word@407d11

etc.

I would like to take all these in once, and then save them (as an arraylist) to a file so I don't have to do it again. What are my options?

[cs_student]

If you already know the amount of items you are going to be storing, it would be wise to set the initial capacity of the ArrayList to keep from having to dynamically resize the array.

Why not just use an String instead of a Word class?
You should post the code for the word class so we can inspect that, as we don't know how it's implemented.

Also, the StringTokenizer class is deprecated. You should look into using the Scanner class instead.