Monday, November 08, 2010 at 12:05 AM.
system.verbs.builtins.radio.utilities.synchronizeFolder
on synchronizeFolder (baseFolder, directoryOpmlUrl, flDeleteFiles=false, flTolerateDownloadErrors=true) {
<<Changes
<<8/31/02; 2:11:04 AM by JES
<<Fixed a bonehead bug in the last change.
<<8/31/02; 1:58:05 AM by JES
<<Small modification to prevent files which should not be downloaded from being downloaded: Lwarence found that sometimes the file modification dates were off by a second in the filesystem, as compared to the date in the OPML file. Instead of an absolute timestamp comparison, we now provide a couple of seconds of leeway in the modification date, since we can't be 100% certain that the operating system will keep up when setting the mod date of files we've downloaded.
<<3/20/02; 11:23:37 PM by JES
<<Fixed a bug which would cause files which should be downloaded, not to be.
<<3/12/02; 1:28:56 AM by JES
<<Download the file if its mod-date is different than, rather than greater than, the file that exists on disk.
<<3/12/02; 1:17:54 AM by JES
<<If the <outline> element has a url attribute, load the file from the url specified by the url attribute, instead of from the relative path as determined by the outline structure.
<<3/11/02; 11:13:34 PM by JES
<<Call tcp.httpClient instead of tcp.httpReadUrl, passing flAcceptOpml as true, so that if requesting the OPML from a Radio server, we get back OPML, instead of rendered HTML.
<<12/30/01; 11:44:13 PM by JES
<<Created.
<<Synchronize the contents of a folder on disk, with a folder accessible over the Internet, based on the contents of the directory.opml file specified at directoryOpmlUrl.
<<flDeleteFiles specifies whether to delete files from disk, which don't appear in the directory-opml.
<<flTolerateDownloadErrors specifies whether an error downloading a file is a scriptError, or is ignored.
<<Cribbed from workspace.userlandSamples.clonePublicRadioFiles.
local (baseUrl = string.popSuffix (directoryOpmlUrl, "/") + "/");
<<local (xmltext = tcp.httpReadUrl (directoryOpmlUrl))
local (xmltext);
bundle { //read xmltext from the URL
local (urlparts = string.urlSplit (directoryOpmlUrl));
local (server = urlparts[2], port = 80);
if server contains ":" {
port = string.nthField (server, ":", 2);
server = string.nthField (server, ":", 1)};
xmltext = string.httpResultSplit (tcp.httpClient (server:server, port:port, path:urlparts[3], flAcceptOpml:true))};
xml.compile (xmltext, @xstruct);
on dolevel (adrtable, folder) {
local (adrsub, text, type);
for adrsub in adrtable {
if nameof (adrsub^) endswith "\toutline" {
text = adrsub^.["/atts"].text;
type = adrsub^.["/atts"].type;
if type == "folder" {
local (innerfolder = folder + text + "/");
dolevel (adrsub, innerfolder)}
else {
local (modified = date (adrsub^.["/atts"].whenLastUploaded));
local (created = date (adrsub^.["/atts"].whenCreated));
local (url = baseurl + folder + text); //default
if defined (adrsub^.["/atts"].url) {
url = adrsub^.["/atts"].url};
local (f = basefolder + string.replaceall (folder, "/", file.getpathchar ()) + text);
local (flread = false);
if file.exists (f) {
if abs (number (file.modified (f) - modified)) > 3 { //give +/- 3 sec leeway (explanation below)
<<8/31/02; 1:54:58 AM by JES
<<Lawrence found that sometimes local mod-dates were off by a second. Adding a +/- 3 second tolerance will prevent, for example, theme files from being repeatedly downloaded. Almost nothing will ever be modified more frequently than every 5 seconds, and even if it were, it's very unlikely that there were significant changes.
flread = true}}
else { //file doesn't exist -- we must download it
flread = true};
if flread {
try {
local (filetext = tcp.httpReadUrl (url));
file.surefilepath (f);
file.writewholefile (f, filetext, creationdate:created);
file.setmodified (f, modified)}
else {
if not flTolerateDownloadErrors {
scriptError (tryError)}}}}}}};
local (adrdoc = xml.getaddress (@xstruct, "opml"));
dolevel (xml.getaddress (adrdoc, "body"), "")}
This listing is for code that runs in the OPML Editor environment. I created these listings because I wanted the search engines to index it, so that when I want to look up something in my codebase I don't have to use the much slower search functionality in my object database. Dave Winer.