/admin/spam system
part of the ArsDigita Community System
by Philip Greenspun
The Big Picture
This is a module to be used only by the publisher. Moreover, it has
very little user interface of its own: URLs underneath /admin/spam are
typically only invoked from within /admin/users. The data model is in
/doc/sql/spam.sql.
However, there is a facility for posting scheduled, periodic alerts or mailings,
which has an interface under /admin/spam.
Under the Hood
- we keep a history of all the spam we've sent to users in the
spam_history
table
- forms that allow a publisher to spam (e.g.,
/admin/users/action-choose) generate a new spam_id for the blank
form; this way a double click does not result in a spam being sent twice
(spam is bad enough!)
- The spam sending system tries to send email using each users email_type preference
(from the users_preferences table). You can enter copies of the spam mail message
formatted as plain-text, HTML, and AOL-HTML (some HTML subset). The spam sending routine
will attempt to send the proper format to each user based on their preference. The default
is plain text.
Sending spam from the browser
The /admin/spam page has a link
to send a plain spam message to a class of users, or to send a
combined plain and HTML message. In both cases you will be given a
form which asks for a subject line, a target class of users, and a
date to send the message. The from-address will probably be overridden
with a machine-generated email address in order to make the automated bounce
handling scripts work.
General Tcl Substitution in Message Body and Subject Line
The spam entry forms all have a checkbox labeled Template?.
If checked, then Tcl evaluator will be run at the time the message
is actually sent on message subject and body, substituting
variables or procedure calls wherever \$ and \[\] chars are
found. This is specially useful for creating automatically generated
templates, such as a daily report which runs a tcl procedure to query
the database or create some other summary message.
Note: if you have the Template? option selected, make sure you
use backslash to quote any $ or '[]' characters in your message, if you do
not want them to undergo evaluation by the Tcl parser.
The following variables are guaranteed to be set in the environment,
for use by your template.
db2
a database handle for any subqueries you want to make
user_id
the user_id of the current target user
email
the email address of the current target user
first_names
last_name
Guessing user's email preferences
In the site .ini file, you can set up a list of patterns to be used to
guess the email type preference of new users, based on their email address.
The param is an association-list, with SQL patterns on the left, and
pseudo-mime types on the right. Supported types right now are text/html,
text/html, and text/aol-html
.
EmailTypes={%@hotmail.com text/html} {%@aol.com text/aol-html}
Manually Sending Spam From a File on the Server
You can schedule a spam which gets its content from files in the file system,
using the "Post spam from file" link on the admin spam page. It will look
n the "Drop Zone" directory (described below) for files with the specified names, and
send the spam using them as the content to the target user class of your choice.
Spam Which Is Sent Automatically: Newsletter Groups
The spam system will periodically check a "drop zone" directory for
uploaded files. You can associate a filename prefix with a scheduled
spam message, so that every time a new file is uploaded, it is
scheduled to be sent out to a particular target class of users. The
system can check either for a file containing static content, or for a
template file which contains Tcl code that is executed to
generate the message body. The template option is useful for
generating periodic status reports or alerts.
Below is an example of the periodic spam admin form, showing the options for
configuring periodic messages which get their content from named files the drop-zone directory:
Daily Spam File Locations
Your Workspace : Admin Home : Spam : List Daily Spam Files
Spam files to look for in drop-zone directory "/web/arsdigita/spam/".
To delete an entry, just enter an empty string for the filename and subject, and press the Modify button.
'From address' is optional; if left blank, the default spam system from-address will be used.
|
You can enter the following information for an automatic spam daily message:
- User Class
- (pulldown menu)
- Subject
- Note that you can include the current date in the subject line of
the spam, by including the string "%%DATE%%" in the subject.
- File Prefix
- The filename prefix where you will deposit the new content for periodic mailings.
- Period
- How often the message should be sent. This only makes sense for
template files, i.e., where the message content is generated automatically on demand.
If the
template?
option is
selected, then the named file will be used as a Tcl template.
If you are supplying
"static" content by uploading new files to the dropzone with the date conventions described
in the next section, then leave the default setting of Daily
for the period.
Drop Zone Directory
The files should be placed in the "Drop Zone" directory specified by the .ini
parameter DailySpamDirectory
:
Example:
DailySpamDirectory=/web/eplay/spam
For each spam defined in the Daily Spam list, the system will look for
the following files:
file_prefix-MM-DD-YYYY
file_prefix
Note: Be sure to always use two-digits for Month and Day fields, i.e.,
03-06-1999. Don't forget the leading zeros.
If a file whose name matches with the specified prefix and the current
day's date is found, the contents are queued up to be sent as spam to
the designated target user class.
The spam system will only send a dated spam once. It keeps a history of
all spams sent, and will be robust across server restarts. If the server
is restarted in the middle of sending a spam, the spam daemon will resume
sending where it left off in the list of users.
Be very careful with filenames that have no date suffix!
If you use a filename with no date suffix, the spam will be sent once a day
from the file. This behavior is designed to support a special case spam
for new users, where the user class contains a magic query like
select user_id from users where trunc(registration_date)=trunc(sysdate-1)
which is carefully constructed to always select a mutually exclusive set of users each day, and never repeat the same user twice.
HTML and AOL content types
Some users will have designated preferred MIME types for email via the
users_preferences table. Currently we support HTML and AOL types in addition
to plain text. If you create auxiliary files with the names
file_prefix-html-MM-DD-YYYY
file_prefix-aol-MM-DD-YYYY
Then content from these files will be sent preferentially to users who have
their email types preferences set in the database.
Internal API for other modules to post scheduled spam messages to
a user or group of users
If another module in the ACS wishes to post a spam message, they should use the
following interface function:
# insert a new spam message into the spam queue, returns the spam_id of the new message
spam_post_new_spam_message {{
-db 0
-template_p "f"
-from_address ""
-title ""
-body_plain ""
-body_html ""
-body_aol ""
-target_users_description ""
-target_users_query ""
-send_date "sysdate"
-creation_user ""
}}
The target_users_description
should be an English description of
who the designated target users are, for administrative documentation purposes.
The target_users_query
should be a SQL query which returns the user_id
for each user you want to send email to. Example
select user_id from users where last_name = 'Kennedy'
Example SQL Queries for spam messages
You supply a SQL query which will be used to generate the target list of
recipients for your spam message.
Your query should return a single column, user_id
.
The spam system will automatically join this with the users_spammable view,
which is a view that filters out users who have elected not to receive
email from the system.
So don't worry about pre-filtering your query via the users_spammable view. Just
supply SQL which coughs up a list of user_ids.
A simple plain text spam to members of a specific user_group:
select u.user_id
from users u, user_group_map ugm
where ugm.group_id = 12345
and ugm.user_id = u.user_id
A query which gets all users with an email address at aol.com.
select user_id
from users
where email like '%@aol.com'
Setting the .ini Parameters
Here is a summary of the .ini params you will need
[ns/server/yourserver/acs/spam]
; Pairs of {email_addr_pattern pseudo-mime-type}
EmailTypes={%@hotmail.com text/html} {%@aol.com text/aol-html}
DailySpamDirectory=/web/yourserver/spam
SpamRobotFromAddress=email-robot@yourdomain.com
BulkMail
By default the spam system uses the built-in
AOLserver ns_sendmail routine. This is adequate for low volume
mailings, but if you need to send mail to more than about 500 users,
then it has serious limitations. A high-performance module called
bulkmail which has a multi-threaded mail client which can talk
to multiple mail servers concurrently, and can generate the QMAIL
style VERP return addresseses to make automatic bounce handling
possible. This will soon be available as part of the core ACS
distribution, and the spam module will have an option to use this as
the mail transport rather than sendmail.
For more info on the configuration of bulkmail and qmail, see bulkmail and qmail configuration
Utilities
There are some useful utilities defined by the spam and email handling modules:
send_email_attachment_from_file
allows you to send a file via ns_sendmail as a MIME encoded email attachment. The file
can have binary content such as a JPEG image or PDF document.
philg@mit.edu
hqm@arsdigita.com