Re: [WebDNA] two ideas for running a cluster of WebDNA servers
This WebDNA talk-list message is from 2019
It keeps the original formatting.
numero = 114880
interpreted = N
texte = 2508 I=E2=80=99m very interested in this. At times we have hundreds of devices a minute hitting our server.=20 The only thing that has made performance better is investing in new = hardware that can deal with the load. (Fast SSDs made a huge difference) A singular store for databases would be the only way I can think of to = achieve this across endpoints. - Greg > On Oct 5, 2019, at 10:42 AM, WebDNA Solutions =
wrote: >=20 >> One front-end server that will redirect the requests >> through POST or GET to a cluster of back-end servers. When >> the request is for writing, then only one of the back-end >> servers is solicited (always the same), when is is for >> reading, it could be any of the back-end servers. >>=20 >> The "writing" server is the master, the others are slaves. >> Every 30 sec, through rsync, the master is copied to a >> slave, and the WebDNA slave reloads the databases in RAM. >=20 > Actually in this situation the WebDNA dbs on the slave servers > must be flushed immediately BEFORE each rsync overwrites > them. If this happens every 30 seconds it will effectively > defeat one of the main advantages of WebDNA -- its typically > superfast "read from RAM" performance -- because every updated > db would have to be read into RAM agai every 30 seconds. >=20 >=20 >=20 >> Another idea would be the same front-end server with the >> same backend servers, and instead of using rsync, the front >> end server would POST the request to each one of the >> back-end servers: no more master and slave. >=20 > I did something similar many years ago, but instead of > frontend and backend servers each server was equally > available via a sequential round-robin load distribution > system. This eliminated the need to flush the dbs every time > one server posted an update to another server, because they > would all be posted to at virtually the same time (within a > fraction of a second from each other given a fast enough > network connection). >=20 > But when the internet connection between the servers goes > down it creates SERIOUS out-of-sync problems such as failure > to login, failure to see the updates the visitors just made, > etc. ... and this would absolutely require your next idea in > order to rebuild the dbs correctly when necessary: >=20 >> Another idea would be to keep a log database of all the >> writing requests so, by reading this log file, all the >> "slaves" would get the same information. Databases could >> even be rebuilt in case of necessity. >=20 > A system like this would become very complex and error-prone > in my opinion, thus unlikely worth the effort when non-WebDNA > solutions are readily available and have been tested and > proven in terms of reliability for many years already. >=20 >=20 >> I am sure there are other solutions. Any other idea? >=20 > WebDNA's design is ideal for use on a single frontend server. > To try and turn it into a backend server would probably suck, > not just because of the amount of work required in an attempt > to turn it into something it is not, but mostly because if it > loses the inherent advantages it offers as a RAM-resident > frontend db server you'll end up with a system that > underperforms the competition. >=20 > I'm wondering why you're thinking about this? >=20 > Are you trying to come up with a plan to "add value" to > WebDNA and perhaps broaden its market by attempting to make > WebDNA a suitable system for websites that need more than one > server in order to handle extreme loads? >=20 >=20 > Regards, > Kenneth Grome > WebDNA Solutions > http://www.webdnasolutions.com > Urgent/Emergency Phone: (228) 222-2917 > Website, Database, Network, and Communication Systems >=20 >=20 >=20 > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list talk@webdna.us > To unsubscribe, E-mail to: talk-leave@webdna.us > archives: http://www.webdna.us/page.dna?numero=3D55 > Bug Reporting: support@webdna.us --------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list talk@webdna.us To unsubscribe, E-mail to: talk-leave@webdna.us archives: http://www.webdna.us/page.dna?numero=3D55 Bug Reporting: support@webdna.us .
Associated Messages, from the most recent to the oldest:
2508 I=E2=80=99m very interested in this. At times we have hundreds of devices a minute hitting our server.=20 The only thing that has made performance better is investing in new = hardware that can deal with the load. (Fast SSDs made a huge difference) A singular store for databases would be the only way I can think of to = achieve this across endpoints. - Greg > On Oct 5, 2019, at 10:42 AM, WebDNA Solutions = wrote: >=20 >> One front-end server that will redirect the requests >> through POST or GET to a cluster of back-end servers. When >> the request is for writing, then only one of the back-end >> servers is solicited (always the same), when is is for >> reading, it could be any of the back-end servers. >>=20 >> The "writing" server is the master, the others are slaves. >> Every 30 sec, through rsync, the master is copied to a >> slave, and the WebDNA slave reloads the databases in RAM. >=20 > Actually in this situation the WebDNA dbs on the slave servers > must be flushed immediately BEFORE each rsync overwrites > them. If this happens every 30 seconds it will effectively > defeat one of the main advantages of WebDNA -- its typically > superfast "read from RAM" performance -- because every updated > db would have to be read into RAM agai every 30 seconds. >=20 >=20 >=20 >> Another idea would be the same front-end server with the >> same backend servers, and instead of using rsync, the front >> end server would POST the request to each one of the >> back-end servers: no more master and slave. >=20 > I did something similar many years ago, but instead of > frontend and backend servers each server was equally > available via a sequential round-robin load distribution > system. This eliminated the need to flush the dbs every time > one server posted an update to another server, because they > would all be posted to at virtually the same time (within a > fraction of a second from each other given a fast enough > network connection). >=20 > But when the internet connection between the servers goes > down it creates SERIOUS out-of-sync problems such as failure > to login, failure to see the updates the visitors just made, > etc. ... and this would absolutely require your next idea in > order to rebuild the dbs correctly when necessary: >=20 >> Another idea would be to keep a log database of all the >> writing requests so, by reading this log file, all the >> "slaves" would get the same information. Databases could >> even be rebuilt in case of necessity. >=20 > A system like this would become very complex and error-prone > in my opinion, thus unlikely worth the effort when non-WebDNA > solutions are readily available and have been tested and > proven in terms of reliability for many years already. >=20 >=20 >> I am sure there are other solutions. Any other idea? >=20 > WebDNA's design is ideal for use on a single frontend server. > To try and turn it into a backend server would probably suck, > not just because of the amount of work required in an attempt > to turn it into something it is not, but mostly because if it > loses the inherent advantages it offers as a RAM-resident > frontend db server you'll end up with a system that > underperforms the competition. >=20 > I'm wondering why you're thinking about this? >=20 > Are you trying to come up with a plan to "add value" to > WebDNA and perhaps broaden its market by attempting to make > WebDNA a suitable system for websites that need more than one > server in order to handle extreme loads? >=20 >=20 > Regards, > Kenneth Grome > WebDNA Solutions > http://www.webdnasolutions.com > Urgent/Emergency Phone: (228) 222-2917 > Website, Database, Network, and Communication Systems >=20 >=20 >=20 > --------------------------------------------------------- > This message is sent to you because you are subscribed to > the mailing list talk@webdna.us > To unsubscribe, E-mail to: talk-leave@webdna.us > archives: http://www.webdna.us/page.dna?numero=3D55 > Bug Reporting: support@webdna.us --------------------------------------------------------- This message is sent to you because you are subscribed to the mailing list talk@webdna.us To unsubscribe, E-mail to: talk-leave@webdna.us archives: http://www.webdna.us/page.dna?numero=3D55 Bug Reporting: support@webdna.us .
Greg Nelson
DOWNLOAD WEBDNA NOW!
Top Articles:
Talk List
The WebDNA community talk-list is the best place to get some help: several hundred extremely proficient programmers with an excellent knowledge of WebDNA and an excellent spirit will deliver all the tips and tricks you can imagine...
Related Readings:
Emailer (1997)
Make sure I understand this??? (1997)
Long/Lat (2002)
Can't load tmpl files (1997)
strange IE 4.0 (1998)
What file? (1997)
Secure Server (1997)
WebCat2.0 acgi vs plugin (1997)
[TaxableTotal] - not working with AOL and IE (1997)
WebCatalog stalls (1998)
What am I missing (1997)
RE: [WebDNA] Clean URLS job - will pay (2010)
[Announce]: Web server security and password protection (1997)
Who needs Yoda! (2002)
Please please help (2001)
Javascript and Pull Down menus (1999)
Newbie questions (1999)
[referrer] -truncates at & (1998)
OT: SMSI stock (2003)
Reserved words - Enhancement request (2000)