Re: [WebDNA] Hard-coded db write delay when running certain code?

This WebDNA talk-list message is from

2011


It keeps the original formatting.
numero = 106224
interpreted = N
texte = WebDNA 7.0 will append to RAM and automatically flush to disk every 1000 = records; otherwise, it writes to RAM and the db file should not show = anything on disk, since this is the RAM copy which is updated. May be = your script adds 1000 records every 2min20s? Maybe you are appending to = a huge database or to a slow disk (if you checked the option flush to = disk after each write) and it takes 2min20s to be written to disk? Could you provide your code and the coordinates the server you = tcpconnect to and i will try this on my side. - chris On Jan 31, 2011, at 11:37, Kenneth Grome wrote: > I'm running a tcpconnect script to extract data from an online = database on a non-webdna server, and I'm seeing EXACTLY 2 minutes and 20 = seconds in between db writes. > I'm using a 10 second delay in a meta-refresh tag so the page is only = supposed to delay 10 seconds before reloading itself and sending the = next tcpconnect. This meta-refresh delay works fine, but something is = up with my db writes ... > The results that get written to my results.db each time a new = tcpconnect is processed is always exactly 2 minutes and 20 seconds later = than the last db write. Just take a look at these timestamps and you'll = see what I mean: > 01/31/2011@06:30:37 > 01/31/2011@06:32:57 > 01/31/2011@06:35:17 > 01/31/2011@06:37:37 > 01/31/2011@06:39:57 > 01/31/2011@06:42:17 > 01/31/2011@06:44:37 > 01/31/2011@06:46:57 > 01/31/2011@06:49:17 > 01/31/2011@06:51:37 > 01/31/2011@06:53:57 > 01/31/2011@06:56:17 > 01/31/2011@06:58:37 > 01/31/2011@07:00:57 > 01/31/2011@07:03:17 > 01/31/2011@07:05:37 > 01/31/2011@07:07:57 > 01/31/2011@07:10:17 > 01/31/2011@07:12:37 > As you can see, every db write is exactly 2 minutes and 20 seconds = later than the previous one. To me this suggests that WebDNA's has some = kind of a "time delay" built into it, and that's why every one of these = records is written to the db on exactly the same schedule. > Does WebDNA 7.0 have some internal code that's preventing db writes = more frequently than 2:20? > I think there must be some code inside the WebDNA 7.0 software that is = delaying each write to the db until exactly 2:20 after the last write, = or at the very least I believe that WebDNA has some internal code that = is causing the results seen above. =20 > Because it seems virtually impossible that these db writes would occur = on such a precise and predictable schedule otherwise, especially when = the data being written is obtained via tcpconnect which naturally = includes unpredictable response times from the remote server. > Instead of seeing these 2:20 delays for every db write I SHOULD be = seeing a variety of different times inbetween db writes -- unless = something internal to WebDNA is causing this pattern. > Here's my code structure: > = --------------------------------------------------------------------------= ------- > [search db=3Dsearch.db ... &max=3D1] > [founditems] > [text]content=3D > [middle startafter=3Dxxx] > [tcpconnect] > [tcpsend] > [/tcpsend] > [/tcpconnect] > [/middle] > [/text] > [append = db=3Dresults.db]stamp=3D[date]@[time]&content=3D[content]...[/append] > > [/founditems] > [/search] > [flushdatabases] > = --------------------------------------------------------------------------= ------- > Something internal to WebDNA *MUST* be controlling the timing of db = writes. That's my conclusion until someone tells me otherwise. > By the way, when I use Firefox to visit the page specified in my = tcpconnect I get a response in a couple of seconds, so I know for a fact = that the same pages I'm retrieving via tcpconnect should not take = anywhere near 2:20 to be returned. =20 > But that's how long it's taking until thedata is written to the db, = and of course the response page cannot be returned until my WebDNA code = has been fully processed. This is why I think the delay is the result = of some internal WebDNA code that's preventing db writes until the "next = interval" -- which appears to be 2:20 since the last db write, or = perhaps on the next x seconds of time that ends in 17, 37 or 57? > Please if you know something about this, let me know what's going on = here. My script is working fine except for these frustrating and = unexplained db write delays, and it seems to me that given the precise = time interval of these db writes there must be a WebDNA explanation for = this behavior. > Thanks. > Sincerely, > Kenneth Grome > --------------------------------------------------------- This message = is sent to you because you are subscribed to the mailing list . To = unsubscribe, E-mail to: archives: = http://mail.webdna.us/list/talk@webdna.us Bug = Reporting:support@webdna.us Associated Messages, from the most recent to the oldest:

    
  1. Re: [WebDNA] Hard-coded db write delay when running certain code? (christophe.billiottet@webdna.us 2011)
  2. [WebDNA] Hard-coded db write delay when running certain code? (Kenneth Grome 2011)
WebDNA 7.0 will append to RAM and automatically flush to disk every 1000 = records; otherwise, it writes to RAM and the db file should not show = anything on disk, since this is the RAM copy which is updated. May be = your script adds 1000 records every 2min20s? Maybe you are appending to = a huge database or to a slow disk (if you checked the option flush to = disk after each write) and it takes 2min20s to be written to disk? Could you provide your code and the coordinates the server you = tcpconnect to and i will try this on my side. - chris On Jan 31, 2011, at 11:37, Kenneth Grome wrote: > I'm running a tcpconnect script to extract data from an online = database on a non-webdna server, and I'm seeing EXACTLY 2 minutes and 20 = seconds in between db writes. > I'm using a 10 second delay in a meta-refresh tag so the page is only = supposed to delay 10 seconds before reloading itself and sending the = next tcpconnect. This meta-refresh delay works fine, but something is = up with my db writes ... > The results that get written to my results.db each time a new = tcpconnect is processed is always exactly 2 minutes and 20 seconds later = than the last db write. Just take a look at these timestamps and you'll = see what I mean: > 01/31/2011@06:30:37 > 01/31/2011@06:32:57 > 01/31/2011@06:35:17 > 01/31/2011@06:37:37 > 01/31/2011@06:39:57 > 01/31/2011@06:42:17 > 01/31/2011@06:44:37 > 01/31/2011@06:46:57 > 01/31/2011@06:49:17 > 01/31/2011@06:51:37 > 01/31/2011@06:53:57 > 01/31/2011@06:56:17 > 01/31/2011@06:58:37 > 01/31/2011@07:00:57 > 01/31/2011@07:03:17 > 01/31/2011@07:05:37 > 01/31/2011@07:07:57 > 01/31/2011@07:10:17 > 01/31/2011@07:12:37 > As you can see, every db write is exactly 2 minutes and 20 seconds = later than the previous one. To me this suggests that WebDNA's has some = kind of a "time delay" built into it, and that's why every one of these = records is written to the db on exactly the same schedule. > Does WebDNA 7.0 have some internal code that's preventing db writes = more frequently than 2:20? > I think there must be some code inside the WebDNA 7.0 software that is = delaying each write to the db until exactly 2:20 after the last write, = or at the very least I believe that WebDNA has some internal code that = is causing the results seen above. =20 > Because it seems virtually impossible that these db writes would occur = on such a precise and predictable schedule otherwise, especially when = the data being written is obtained via tcpconnect which naturally = includes unpredictable response times from the remote server. > Instead of seeing these 2:20 delays for every db write I SHOULD be = seeing a variety of different times inbetween db writes -- unless = something internal to WebDNA is causing this pattern. > Here's my code structure: > = --------------------------------------------------------------------------= ------- > [search db=3Dsearch.db ... &max=3D1] > [founditems] > [text]content=3D > [middle startafter=3Dxxx] > [tcpconnect] > [tcpsend] > [/tcpsend] > [/tcpconnect] > [/middle] > [/text] > [append = db=3Dresults.db]stamp=3D[date]@[time]&content=3D[content]...[/append] > [thisurl]"> > [/founditems] > [/search] > [flushdatabases] > = --------------------------------------------------------------------------= ------- > Something internal to WebDNA *MUST* be controlling the timing of db = writes. That's my conclusion until someone tells me otherwise. > By the way, when I use Firefox to visit the page specified in my = tcpconnect I get a response in a couple of seconds, so I know for a fact = that the same pages I'm retrieving via tcpconnect should not take = anywhere near 2:20 to be returned. =20 > But that's how long it's taking until thedata is written to the db, = and of course the response page cannot be returned until my WebDNA code = has been fully processed. This is why I think the delay is the result = of some internal WebDNA code that's preventing db writes until the "next = interval" -- which appears to be 2:20 since the last db write, or = perhaps on the next x seconds of time that ends in 17, 37 or 57? > Please if you know something about this, let me know what's going on = here. My script is working fine except for these frustrating and = unexplained db write delays, and it seems to me that given the precise = time interval of these db writes there must be a WebDNA explanation for = this behavior. > Thanks. > Sincerely, > Kenneth Grome > --------------------------------------------------------- This message = is sent to you because you are subscribed to the mailing list . To = unsubscribe, E-mail to: archives: = http://mail.webdna.us/list/talk@webdna.us Bug = Reporting:support@webdna.us christophe.billiottet@webdna.us

DOWNLOAD WEBDNA NOW!

Top Articles:

Talk List

The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...

Related Readings:

NY taxRates.db? (1997) A few questions. . . (1997) Bug alert! (1997) Norton Internet Security filtering out WebDNA processsed (2004) Re1000001: Setting up shop (1997) Re:Item options w/ price adjustment (1997) WebCatalog-NT?'s (1996) [WebDNA] Digest from talk@webdna.us (2018) OT JavaScript question (1999) Many $WebCat.exe processes (1998) Occasional crashes with $remove (1997) process SSI (1998) (2000) Error Log.db --however (1997) Order not created error (1997) sort problems....bug or brain fart? (1997) RE: Questions To Answer (1997) RE: [WebDNA] ubuntu can't append db (2019) International Orders (1998) sorting by date (1999)