Monday, November 08, 2010 at 12:05 AM.
system.verbs.builtins.searchEngine.indexLocalPage
on indexLocalPage (adrPage, siteName, adrIndex=nil, adrPreviews=nil, adrStopWords=nil) {
<<Index a page in a Frontier website.
local (url, title, pageText);
local (fileExtension, pageTable);
local (flDynamic = false);
bundle { //ensure index and previews addresses
if adrIndex == nil {
adrIndex = searchEngine.getIndexAddress (siteName)};
if adrPreviews == nil {
adrPreviews = searchEngine.getPreviewsAddress (siteName)}};
bundle { //should this page get indexed?
case typeOf (adrPage^) { //index certain types of objects only
wpTextType;
outlineType;
stringType {
true};
scriptType {
flDynamic = true}} //always build script-based pages before indexing
else {
return (false)}; //don't index this page
<<Set the indexThisPage directive to false to skip a page.
if not (html.getPagePref ("indexThisPage", adrPage)) {
<<Remove this page from the index, in case it was previously indexed.
searchEngine.deIndexPage (adrPage, adrIndex);
return (false)}}; //don't index this page
fileExtension = html.getPagePref ("fileExtension", adrPage);
new (tableType, @pageTable);
if adrStopWords == nil {
adrStopWords = @searchEngine.data.stopWords};
if flDynamic == false { //find out if we need to build the page first
<<Set the #regularIndex directive to false to force
<<largely dynamic pages to get built before being indexed.
if not (html.getPagePref ("regularIndex", adrPage)) {
flDynamic = true}};
if flDynamic { //build the page
pageTable.indexThisPage = false; //avoid loop when finalFilter wants to index page
pageText = html.buildObject (adrPage, @pageTable)}
else { //just build the page table
html.buildPageTable (adrPage, @pageTable);
pageText = string (adrPage^)};
bundle { //get the URL and title of the page
url = pageTable.url;
if not (url endsWith fileExtension) {
url = string.popSuffix (url, '.');
url = url + fileExtension};
title = pageTable.title;
if not flDynamic {
title = html.getPagePref ("title", adrPage)}};
msg ("Search Engine: Indexing: " + adrPage);
<<Create a preview for this page.
<<This should always be done before adding a page to the index.
searchEngine.createPreview (pageText, title, url, adrPage, adrPreviews);
<<Add the page to the index.
<<It's okay to call this verb if the page has been indexed before.
searchEngine.indexPage (adrPage, url, title, pageText, adrIndex, adrStopWords);
return (true)}
This listing is for code that runs in the OPML Editor environment. I created these listings because I wanted the search engines to index it, so that when I want to look up something in my codebase I don't have to use the much slower search functionality in my object database. Dave Winer.