[opendmarc-dev] Delayed DB writes

Murray S. Kucherawy msk at blackops.org
Wed Jun 20 06:18:47 PDT 2012


I'm planning to have the opendmarc filter write records to an SQL database 
for each message it receives, to allow report generation.

In the event the database is unavailable, we have a few options:

1) Temp-fail messages when the DB is unavailable

2) Write things to a flat text file which then gets batch-imported when 
the database is available (maybe by cron, maybe by a separate filter 
thread); if the flat file can't be written (disk full or such), fall back 
to 1

3) Queue some number of events internally and then replay the queue into 
the DB when it's available; when the queue gets too large, fall back to 1

I'm leaning toward having a separate thread that does all of the DB 
insertions, accepting and queueing requests from the other thread(s).  It 
would otherwise implement option 3 above.  Any comments?

-MSK


More information about the opendmarc-dev mailing list