Monday, November 08, 2010 at 12:04 AM.
system.verbs.builtins.mainResponder.search.utilities.compactText
on compactText (s) {
<<Remove punctuation (but not spaces) from text.
<<Changes:
<<07/19/00; 11:29:59 AM by PBS
<<Don't do the char-by-char loop, it's redundant, as searchEngine.cleanText does most of the text clean-up.
s = string.lower (s);
bundle { //a few punctuation characters aren't handled by searchEngine.cleanText
s = string.replaceAll (s, "\"", " ");
s = string.replaceAll (s, "'", " ");
s = string.replaceAll (s, ",", " ");
s = string.replaceAll (s, "~", " ");
s = string.replaceAll (s, " ", " ", false)};
s = searchEngine.cleanText (s);
s = string.replaceAll (s, ">", " ");
s = string.replaceAll (s, "<", " ");
local (i = sizeOf (s));
<<loop
<<if i == 0
<<break
<<
<<if string.isPunctuation (s [i])
<<delete (@s [i])
<<
<<if mod (I, 500) == 0
<<sys.systemTask ()
<<
<<i--
s = string.trimWhiteSpace (s);
return (s)}
This listing is for code that runs in the OPML Editor environment. I created these listings because I wanted the search engines to index it, so that when I want to look up something in my codebase I don't have to use the much slower search functionality in my object database. Dave Winer.