Re: Search Engine questions ...
This WebDNA talk-list message is from 2002
It keeps the original formatting.
numero = 44784
interpreted = N
texte = >> > >HOW do they read the DNA stuff from the server? If the pages are preprocessed by Webcat, all of the DNA is stripped out before being sent to the SE, right??? >Here's what I found in the archives. I think I misquoted JP.>2002/08/03 03:01:34 John Peacock Re: OT: Site Downloaders >Lester Emo wrote: >> Is there a way to prevent a program like Offline Explorer >> by MetaProducts.com to suck up every page linked in a >> site and download it to a persons computer??? >> >If your site is set up properly, any page with webdna will not be served up >except through WebCatalog itself (hence processed). Any actively generated site can >be spidered as long as the spider can generate the appropriate links. There is >nothing you can do about it, since the spiders can ignore the ROBOTS file and >pretend they are an ordinary browser. >John I still say the spiders can recognize dynamically generated links. That's based on my experience of examining log files and searching the se's for my stuff. I know it's so for AllTheWeb. I'm still waiting for Google Bot to come back around so I can see how it spiders the same stuff again.Glenn------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list
. To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://search.smithmicro.com/
Associated Messages, from the most recent to the oldest:
>> > >HOW do they read the DNA stuff from the server? If the pages are preprocessed by Webcat, all of the DNA is stripped out before being sent to the SE, right??? >Here's what I found in the archives. I think I misquoted JP.>2002/08/03 03:01:34 John Peacock Re: OT: Site Downloaders >Lester Emo wrote: >> Is there a way to prevent a program like Offline Explorer >> by MetaProducts.com to suck up every page linked in a >> site and download it to a persons computer??? >> >If your site is set up properly, any page with webdna will not be served up >except through WebCatalog itself (hence processed). Any actively generated site can >be spidered as long as the spider can generate the appropriate links. There is >nothing you can do about it, since the spiders can ignore the ROBOTS file and >pretend they are an ordinary browser. >John I still say the spiders can recognize dynamically generated links. That's based on my experience of examining log files and searching the se's for my stuff. I know it's so for AllTheWeb. I'm still waiting for Google Bot to come back around so I can see how it spiders the same stuff again.Glenn------------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list . To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://search.smithmicro.com/
Glenn Busbin
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
WebDNA Developer Resource Center (2002)
Search Context Strips URL chrs.? (1999)
[WebDNA] Mondy morn... [date] question (2009)
More questions about serial number dishing (1997)
Was 5.0 Pricing, now Sandbox versus Website and ruminating (2003)
DNS Lookup 2 (2000)
Refering page (1998)
how many users (2000)
HELP WITH DATES (1997)
6.0 upgrade issue? (2005)
WCS Newbie question (1997)
Car Database - Imporant (2002)
Frames and WebCat (1997)
[GROUPS] followup (1997)
Shipping charges depending on tax rate? (1997)
What file? (1997)
HomePage Caution (1997)
Major Security Hole (1998)
Using Cookie for client specific info? (1997)
Getting the protocol FINAL (2004)