New Tokenizer and DrJava Properties with Attributes

As described in an earlier post, I am looking at all the variables an external process could need. The problem I saw was a large growth of variables that express essentially the same idea, e.g. all the open files, but that are represented differently, e.g. by using ‘:’ as separator instead of ‘ ‘.

I have now solved the problem by using attributes. Each variable can register any number of attributes and set defaults, but the user can override them. To do that, I had to write a new stream tokenizer, as the old one wasn’t easy to use when quote/brace balancing was necessary. The new one, BalancingStreamTokenizer is fairly simple; arguably simpler than the original one. There is no distinction between ordinary characters, word characters, numbers, comments, and quotes. The tokenizer always returns a string.

However, it is possible to register keywords, which are a major replacement over the single-length “ordinary character” concept of Java’s StreamTokenizer. It is furthermore possible to register multi-character quotation symbols where the beginning and the end symbol do not have to match, e.g. /* and */ or ${ and }. Inside quotes, quotation marks are balanced, so a quotation will not cut off at the first matching ending quotation mark, but rather permit correct nesting.

Here are just a few examples of what can be done now:

  • ${drjava.current.file}
  • ${drjava.current.file; dir="/Users/mgricken"}
  • ${drjava.current.file; dir="${user.home}"} — variables can be used again inside attribute values.
  • ${drjava.all.files}
  • ${drjava.all.files; sep=" "}
  • ${drjava.all.files; dir="${drjava.working.dir}"}
  • ${drjava.all.files; dir="${drjava.working.dir}"; sep=" "}
  • ${drjava.current.time.millis}
  • ${drjava.current.time.millis; fmt="full"}
  • ${drjava.current.time.millis; fmt="long"}
  • ${drjava.current.time.millis; fmt="medium"}
  • ${drjava.current.time.millis; fmt="short"}

When the user tries to set an attribute that isn’t valid for a property, an error is emitted. The syntax highlighting works fairly well.

Now I just have to add in all the properties I mentioned earlier…


About Mathias

Software development engineer. Principal developer of DrJava. Recent Ph.D. graduate from the Department of Computer Science at Rice University.
This entry was posted in DrJava, Ramblings. Bookmark the permalink.

Leave a Reply