Re: A sendmail warning

This WebDNA talk-list message is from

2005

It keeps the original formatting. numero = 63686
interpreted = N
texte = At 1:04 AM -0500 12/27/05, William DeVaul wrote:>...These emails need to be tailored to the customer, i.e. I can't send out one email and bcc: everyone. Does anyone have some thoughts on how to better use WebDNA for customized mass emailings?We have a solution.... Being a 100% mac server shop for so many years there really was no other software to do the individual e-mails that we wanted to send to our customers. Now of course with Mac OS-X we use solutions such as phplists and others. But we still have one website running on mac OS-9 and that site needs to send thousands of e-mails to customers every day. (26,000 at the peak)Early on we found the same problems you did. Specifically, doing 26,000 [sendmail...]s in one WebCat template crashed the system. There was no way the mac OS could keep up with WebCat creating 26,000 files in a single pass. Cranking the number of e-mails down to say... 1,000 at a clip... solved the system crash, but then caused the mail server to become strained, overloaded and sometimes crash.Basically what we do is set-up a [spawn]ed task to do the emailing in small batches. The main template reads the data base to determine how many e-mail need to be sent. It then does some calculations to determine how many loops we need to make through the database and sets-up a [spawn] to dribble out the e-mails. We've determined that doing batches of 200 e-mails in 3 minute intervals works for us. There's no danger of the server crashing or becomeing overloaded and we have to leave some processing time available for the rest of the webserving tasks that the server is doing so we keep the e-mail task on a low impact level at 200 every 3 minutes. It does however take ~ 7 hours to mail the 26,000 messages. Your mileage will vary.I've posted the code below. I've hacked out some stuff specific to the site... Hopefully you can make sense out of it.Two caveats:1. We know for certain that this database will not change during the mailing. If there is the possibility that the database will change (edits, additions or deletions)... your first step should be to duplicate the database and do the mailing from the database copy so you don't screw-up the "startat" counters.2. If the server crashes -- for whatever reason -- during the spawn, there is no way (in our system) to know with 100% accuracy where the spawned task is and therefore no way to start again without the possibility of duplicate messages. The system does send the administrator a progress e-mail at the start of each batch. Using the info in that e-mail and changing the starting point in the template should theoretically all you to restart the mailing task.Notes:1. [themessage] and [mailsubject] wereset-up in a previous step.2. [myReply], [theFrom], [siteemail] and [adminemail] are all e-mail address values that come from the included config file.3. use "&debug=yes" on the call to the template to see some extra stuff.4. The spawn depends on WebCat's [waitforfile] context to do the delay. Reasoning, if you wait a specific amount of time for a file that will never exist , you have effectively created a pause in the execution of your template.Here goes...[include file=/db/mdb/-config.inc][protect [siteprefix]-xxxxx]

Sending E-mail - step 3 (revised) - Email is now being sent

[mailMessage]

[!] set up some starting places. edit these three values to tweak the batch sizes and timing [/!][math show=f]mymax=200[/math][math show=f]myDelay=180[/math][math show=f]myStart=1[/math][text]myReply=theReply[/text][text]myFrom=theFrom[/text][showif [debug]=yes]--==debug==--
[formvariables name=_&exact=F][getchars start=2][name][/getchars]=[value]
[/formvariables]--==debug==--
[/showif][spawn][text]myReply=[theReply][/text][text]myFrom=[theFrom][/text][search db=/db/mdb/addresslist.db&netheAddressdata=find-all][math show=f]loops=ceil([numfound]/[mymax])[/math][sendmail to=[siteemail],[adminemail]&from=[adminemail]&subject=I will do [loops] loops]A new mailing is beginning with the subject line of:[mailSubject]It is being sent to [numfound] members[/sendmail][/search][loop start=1&end=[loops]&advance=1][sendmail to=[siteemail],[adminemail]&from=[adminemail]&subject=[time] Loop# [index] has begun. mymax=[mymax] mystart=[mystart]][/sendmail][search db=/db/mdb/addresslist.db&neTHEADDRESSdata=find_all&asMAILFLAGsort=1&MAILFLAGtype=num&startat=[mystart]&max=[mymax]][founditems][!]HTML MAIL ---------------------------------------[/!][sendmail to=[theAddress]&from=[myFrom]&subject=[mailSubject]&Reply-To=[myReply]&Errors-To=[adminemail]&X-OPID=[MEMID]&MIME-Version=1.0&Content-Type=multipart/alternative; charset=us-ascii; boundary="[cart]"&Content-Transfer-Encoding=7bit]--[cart]Content-Type: text/plain; charset=us-ascii[unurl]%0D%0A[/unurl][unurl]%0D%0A[/unurl]HTML mail enclosed[unurl]%0D%0A[/unurl]--[cart]Content-Type: text/html; charset=us-ascii[unurl]%0D%0A[/unurl][unurl]%0D%0A[/unurl] Out Professionals - Newsletter Mailing[convertchars db=hmailconversions.db][mailMessage][/convertchars][unurl]%0D%0A[/unurl]--[cart]--[unurl]%0D%0A[/unurl][/sendmail][/founditems][/search][math show=f]myStart=myStart+mymax[/math][WaitForFile file=noway.txt&timeout=[mydelay]][/loop][sendmail to=[siteemail],[adminemail]&from=[adminemail]&subject=I am done][/sendmail][WaitForFile file=noway.txt&timeout=[mydelay]][/spawn]The server will send [siteemail] a message when all the mail has been sent.

-------------------------------------------------------------This message is sent to you because you are subscribed to the mailing list .To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to Web Archive of this list is at: http://webdna.smithmicro.com/ Associated Messages, from the most recent to the oldest:

Re: A sendmail warning ( Patrick McCormick 2005)
Re: A sendmail warning ( William DeVaul 2005)
Re: A sendmail warning ( William DeVaul 2005)
Re: A sendmail warning ( Peter Ostry 2005)
Re: A sendmail warning ( Peter Ostry 2005)
Re: A sendmail warning ( William DeVaul 2005)
Re: A sendmail warning ( William DeVaul 2005)
Re: A sendmail warning ( Joe D'Andrea 2005)
Re: A sendmail warning ( William DeVaul 2005)
Re: A sendmail warning ( William DeVaul 2005)
Re: A sendmail warning ( Terry Wilson 2005)
Re: A sendmail warning ( Terry Wilson 2005)
Re: A sendmail warning ( Peter Ostry 2005)
Re: A sendmail warning ( Kenneth Grome 2005)
A sendmail warning ( William DeVaul 2005)

At 1:04 AM -0500 12/27/05, William DeVaul wrote:>...These emails need to be tailored to the customer, i.e. I can't send out one email and bcc: everyone. Does anyone have some thoughts on how to better use WebDNA for customized mass emailings?We have a solution.... Being a 100% mac server shop for so many years there really was no other software to do the individual e-mails that we wanted to send to our customers. Now of course with Mac OS-X we use solutions such as phplists and others. But we still have one website running on mac OS-9 and that site needs to send thousands of e-mails to customers every day. (26,000 at the peak)Early on we found the same problems you did. Specifically, doing 26,000 [sendmail...]s in one WebCat template crashed the system. There was no way the mac OS could keep up with WebCat creating 26,000 files in a single pass. Cranking the number of e-mails down to say... 1,000 at a clip... solved the system crash, but then caused the mail server to become strained, overloaded and sometimes crash.Basically what we do is set-up a [spawn]ed task to do the emailing in small batches. The main template reads the data base to determine how many e-mail need to be sent. It then does some calculations to determine how many loops we need to make through the database and sets-up a [spawn] to dribble out the e-mails. We've determined that doing batches of 200 e-mails in 3 minute intervals works for us. There's no danger of the server crashing or becomeing overloaded and we have to leave some processing time available for the rest of the webserving tasks that the server is doing so we keep the e-mail task on a low impact level at 200 every 3 minutes. It does however take ~ 7 hours to mail the 26,000 messages. Your mileage will vary.I've posted the code below. I've hacked out some stuff specific to the site... Hopefully you can make sense out of it.Two caveats:1. We know for certain that this database will not change during the mailing. If there is the possibility that the database will change (edits, additions or deletions)... your first step should be to duplicate the database and do the mailing from the database copy so you don't screw-up the "startat" counters.2. If the server crashes -- for whatever reason -- during the spawn, there is no way (in our system) to know with 100% accuracy where the spawned task is and therefore no way to start again without the possibility of duplicate messages. The system does send the administrator a progress e-mail at the start of each batch. Using the info in that e-mail and changing the starting point in the template should theoretically all you to restart the mailing task.Notes:1. [themessage] and [mailsubject] wereset-up in a previous step.2. [myReply], [theFrom], [siteemail] and [adminemail] are all e-mail address values that come from the included config file.3. use "&debug=yes" on the call to the template to see some extra stuff.4. The spawn depends on WebCat's [waitforfile] context to do the delay. Reasoning, if you wait a specific amount of time for a file that will never exist , you have effectively created a pause in the execution of your template.Here goes...[include file=/db/mdb/-config.inc][protect [siteprefix]-xxxxx]

Sending E-mail - step 3 (revised) - Email is now being sent

[mailMessage]

DOWNLOAD WEBDNA NOW!

Re: A sendmail warning

2005

Sending E-mail - step 3 (revised) - Email is now being sent

Sending E-mail - step 3 (revised) - Email is now being sent

Top Articles:

Related Readings: