ArsDigita Archives
 
 
   
 
spacer

Calendar Package Design Documentation

I. Essentials

II. Introduction

At a personal level, the ArsDigita calendar allows one to enter and keep track of anything that could reasonably be written on a paper calendar. In addition, the calendar can be accessed from any web browser. Various types of additional information related to an item can be added. This information is integrated with other components of the ACS. The calendar also provides different ways of viewing the calendar information. Eventually the calendar will integrate with other systems such as PDAs and email.

At a group level, the calendar can be tied to particular ACS groups where all group members can see items entered for that group. Items from the various groups one is a member of can be shown on one's personal calendar.

The calendar is not intended as a replacement for a PDA but should eventually work in conjunction with one's PDA. The calendar is not a meeting scheduler, but could work in conjunction with a scheduler or free time finder. The calendar is also not designed for conference agendas and management, though it could be used to list conference scheduling at a high level. It's also not a project management or workflow tool, but again, it could be used in conjunction with such tools.

This package could be used with any application where tracking items in time is important. This would include most business settings for meetings and appointments, educational settings, clubs, and other community organizations. For example, the calendar package is a key component of the education solution that has been implemented for the Sloan School of Management, ArsDigita University, and MIT's 6.916 course calendar. There are a number of features which any application must bring to the table to be reasonably considered a complete calendaring application. The ArsDigita calendar package currently supports all but one of these features:

  • Multiple different ways of viewing the calendar. At a minimum this must include day, week, month, and year views. The ArsDigita calendar also includes two list views.
  • A rich set of navigation features for easily selecting and changing both the views and the time spans viewed.
  • The ability to easily add, edit and delete items.
  • The ability to easily set up recurring items.
The one key missing feature is the ability to set up reminders or alarms. While this is an important feature, it wasn't a critical client feature for the first release. However, this feature will be in the next version.

There are several features that add value to the ArsDigita calendar over and above the "table stakes". These include:

  • A web interface. This provides one form of portability; increasingly people need to have convenient access to their calendar information in multiple locations.
  • Group calendars. This is the key feature that distinguishes the ArsDigita calendar package. Not only do you have a personal calendar, but potentially every group you belong to has a separate calendar which you can easily access or, optionally, you can have items from any group calendar also appear on your personal calendar.
  • Attachments. The package supports easily attaching multiple uploaded files, blocks of arbitrary text, or URLs to a single item.
  • User-definable types. Each user and each group can define their own set of types which can be attached to items.
  • API. The package supports a public API which can be used by other packages and modules to manipulate calendaring information in personal and group calendars.
There are a number of other systems which, at least superficially, might appear to address the same requirements. I will go into more detail below in the competitive analysis section, but briefly here are the key differences between these systems and the ArsDigita calendar package.
  • Ical. This is a standalone Tcl based personal calendar application. Since it is a standalone application, it's not portable. There is no support for groups, attachments, or user-definable types, though it does have a public API.
  • Palm DateBook. This does provide one form of portability. However, there is no support for groups, user-definable types or, attachments. It does have a public API which can be used by other Palm applications.
  • The vCalendar/iCalendar proposed standards. While some future version of the ArsDigita calendar should probably support these standards, they are primarily intended to "define a transport and platform-independent format for exchanging calendaring and scheduling information in an easy, automated, and consistent manner."
  • Yahoo Calendar and Excite Planner. These are the key competitors for the calendar package. However, once reminders/alarms are added, the package will have a clear advantage because of group calendars, attachments, user-definable types, and a public API.

III. Historical Considerations

Recurring items

The choice of design of the data model and implementation for recurring items was critical because it impacted the design of the key tables and the implementation of the most important queries and data manipulation statements.

Briefly, one choice is to insert only a single row for each recurring item, regardless of the number of times it will recur. Then to get the items for any particular day, it is necessary to compute for each item whether it would recur on that particular day. The alternative is to insert a row for each recurrence (possibly on an as needed basis.) Then to get the items for a particular day, one can query directly for items whose date is the same as the day in question.

Fundamentally this presents a time vs. space trade-off. Given that computation is relatively expensive, and that in most cases, only a small additional amount of information needs to be stored for recurrences, this seemed a reasonable trade-off, at least for the first version of the package. Additionally this approach made it much easier to handle individual exceptions and to individualize instances by adding attachments and changing titles. However it does make it more difficult to handle items that recur indefinitely; in fact this is not yet supported in the calendar package.

Templates vs. Tcl

It would be ideal if the calendar package were simple to customize for clients. Initially the calendar package was implemented using the version of the templating system and form manager available. However, as time went on, less and less of the package used the templating system. First, the form manager was dropped. Then, various views became so complex that they couldn't be done using templates. Finally, the performance liabilities proved to be too great and all use of the templating system was dropped.

It is possible that the new Dynamic Publishing System (DPS) may solve the various problems with the old templating system. This will be investigated for a future version of the calendar package.

Old Calendar

Finally, a nod must be given to the old calendar module. This module had a very simple data model. The key missing element were times (and not just dates) for items. However, pieces of the existing calendar widgets were the basis of some of the existing display code, and the need to emulate the monthly calendar view of employee absences drove some design decisions. Finally, much of the calendar categories code was re-used in the implementation of the current calendar item types feature, with the addition of multiple types per item.

IV. Competitive Analysis

These are the key features missing from competing software:
  • Group calendars. (Though Time Guides in Yahoo allow some minor integration with other Yahoo components.)
  • User-definable types.
  • Recurring item model. The model captures all the flexibility available in our competitors, but the UI presents it in an easier to use and understand manner than our other web-based competitors.
  • File, text, and URL attachments.
  • Related link. The data model supports making the link from an item's title optionally link to an arbitrary URL rather than to the default item view. This is useful, for example, in adding items that are directly related to other systems. For example, the employee absence items link directly to the intranet/absences system. The UI does not currently support setting this feature, but the API does so that other modules and packages can make use of it.
  • List view. The list view of upcoming and new items are presented in tables which support sorting on the different columns. This is a powerful way of looking at calendar items which is not supported by any of our competitors.
  • API. Only Ical claims to have an API for other applications to use. The calendar package API is a powerful and easy mechanism for adding calendar functionality to other packages and modules. It is already in use by the reservations and intranet modules. In the future it may be used by the address book to add birthdays and by the bootcamp module to add bootcamp items such as TA hours.
There are a number of features lacking in the calendar package which are present in competing software. Unfortunately, given the limited resources available for the calendar package, not all possible features could be implemented. However, some of these features are clearly very important and I will note those I expect to be implemented within the next version or two.
  • Reminders/alarms. With the possible exception of Palm synchronization, this is the most requested missing feature. This is the one feature that really keeps the calendar package from being complete, and will be addressed ASAP.
  • Palm synchronization. This is a very desirable feature. With synchronization and reminders/alarms, the calendar package will be obviously superior to both Yahoo and Excite. However, this is also a difficult feature to implement. Both Yahoo and Excite went to third-parties for their implementation. We are currently attempting to build this support in-house.
  • Pre-defined public types. There is no standard default set of item types supported in the calendar package. Providing this will mean that users and groups will not have to be recreating the same types over and over again.
  • User preferences. The data model supports a wide range of user preferences, but only a small fraction have been implemented.
  • Sharing. Currently it is not possible to easily share my personal calendar with anyone else.
  • Find free times. With Yahoo, it is possible to compare calendars with another person to find mutually free times. The calendar package should eventually support finding free times amongst the members of a particular group, or users that have shared their calendars.
  • Search. It is not currently possible to search for items. Clinsights is adding this feature to an earlier version of the calendar package, so it may be possible to adapt this to the current version without too much difficulty.
  • Holidays. There is no built in administrative support for adding standard holidays to calendars. It can be done with the current package by creating a special holidays group, but that solution seems unsatisfactory.
  • Invitations. This is the ability to send out email to a group to invite them to participate in some event. The email includes URLs for accepting or declining the invitation. It should also be possible to accept or decline by replying to the email.
  • ToDos. List of items that need to be done are a natural extension of the calendar data model. They are merely items that are not associated with a particular date. The calendar package would need to be extended to support adding, displaying, and checking-off these items.
  • Update future items only. Currently, when editing a recurring item, the user has the choice of updating only the current item or all instances of the recurring item. A useful addition would be updating all instances that occur after the current instance.
  • Changing recurring frequency. The Palm DateBook supports changing the frequency of a recurring item after it has been added. For example, a twice-weekly meeting can be easily changed to a weekly meeting. (Note: I have found bugs in the Palm implementation.)

V. Design Tradeoffs

Recurring items

Two choices were considered for supporting recurring items. One choice is to insert only a single row for each recurring item, regardless of the number of times it will recur. This row contains all the information necessary to compute whether or not that item would recur on a particular day. So to get all the items for any particular day, it would be necessary to compute for each item whether it would recur on that particular day.

The alternative is to insert a row for each recurrence. To get the items for a particular day, one can query directly for items whose date is the same as the day in question, and selecting for items across a range can be done with a single query

In choosing the second approach, there were really two tradeoffs being made. The obvious tradeoff is time vs. space. Computation, particularly if it might need to be done in Tcl and not solely in the database, is relatively expensive compared to storing additional information in the database. In most cases, the only information that needs to be stored for recurrences is the date and time of the recurrence.

The second tradeoff is in software complexity. In looking at the two choices, the first approach, to insert only a single row, seemed to require a significantly more complex design. Thus the design, implementation and eventual maintenance time would be greater. Particularly given that there were hard deadlines in the use of the calendar package in the education solution, it made sense to go with the simpler approach.

There were additional reasons that made the multiple rows approach more compelling. First, this approach made it much easier to handle exceptions to recurrences. Second, it was a requirement of the education solution to handle individualizing instances by adding attachments and changing titles. This proved relatively simple to add.

However, there are significant drawbacks to this approach that still need to be addressed. First, it is more difficult to handle items that recur indefinitely, and in fact this is not yet supported in the calendar package. Second (but related) is that additional safeguards need to be put in place to prevent pathological (accidental or intentional) cases from swamping the database.

User preferences

The data model was designed to support a wide range of user preferences. However, adding support for every one of these preferences would have taken time away from implementing other higher priority features. So there is currently no single user preferences form. Only a few of the most important user preferences (hiding/showing group items, compressing day view, and remembering last view) were implemented by adding links specifically to set those preferences.

Templates vs. Tcl

The calendar package is likely to be used by a number of clients. Many of these clients will need to customize it for their look and feel. While the public API functions for generating various pieces of the UI provides some support for this customization, it would have been even better if the calendar package used some form of templating.

Initially the package was implemented with the version of the templating system and form manager available at the time. However, first the form manager was dropped because it interfered with the fancy date/time widgets needed and because it couldn't handle the complex inserts and updates that were sometimes needed. Then, various views became so complex that they couldn't be done using templates. Often this was because we needed to display an ordered list of dates and times but efficient queries would produce results with gaps, and thus the multiple tag couldn't handle the complexity of the layout needed. Finally, the performance liabilities proved to be too great and all use of the templating system was dropped.

It appears that the new Dynamic Publishing System (DPS) may solve some of the problems with the old templating system. This will be investigated for future versions of the calendar package. Rather than attempting to redo the package completely using templating, it will probably be best to get a feel for the portions of the package that clients will most want to customize and then to use appropriate tools to make those portions more customizable.

Security and general permissions

The security model in the current package is fairly simple. It's clear that the package should move to using general permissions. However, when the initial implementation was being done, general permissions was relatively new and seemed like overkill for what was needed. There were also rumors of performance problems. I believe these issues have been addressed in ACS 4.0.

VI. Data Model Discussion

Items

The cal_items table is the heart of the calendar package. This table has columns for all the basic item information as well as columns for recurring items. The parent_id column is used to connect instances of recurring items to the parent item which contains default information. Initially, in a child item, only the start_date and possibly end_date columns are filled in. When the item appears on the calendar, all the other information is pulled from the parent item.

Mappings

There are a number of mappings made between calendar items and other tables in both the calendar data model and in the rest of the ACS. These mappings are:
  • cal_item_type_map which maps between cal_items and cal_item_types.
  • cal_user_group_item_map which maps between cal_items and users and groups. This is the table that controls ownership of items.
  • cal_item_db_row_map which maps between cal_items and other arbitrary rows in the database. This is how attachments are implemented, and this table is also used by the public API to associate calendar items with other packages and modules.

Types

Types are equivalent to what were called categories in the old calendar data model. In essence, types are pre-defined labels which can be associated with arbitrary calendar items via the cal_item_types and cal_item_type_map tables.

Preferences

User preferences are stored primarily in the cal_user_prefs table. The columns that are used in the current version of the package are: default_view, compress_day_view_p, and update_type Several other columns are defined for future use. The cal_user_update_prefs table is used to store how items from a calendar for a group that a particular user is a member of, are to be displayed on the user's personal calendar.

Unusual columns

There are a few tables/columns whose use is not immediately obvious:
  • The cal_groups table is used to store a list of groups that have calendars. This is needed for cases where a group calendar has been newly created. It has no items, but we want it to show up in various lists.
  • The cal_group_types table stores list of group types that have calendars. All groups of this type will automatically have calendars.
  • The hours_span column of the cal_user_prefs table is intended to control the span of hours displayed in the day view. It is specified as HH24HH24, e.g. 0717 represents 7am-5pm and 0023 represents 24 hours. It's not settable in the current version and defaults to 0023.
  • The compress_day_view_p column is a toggle controlling whether or not the day view is limited to just those hours containing items or if the full range in hours_span is shown.
  • The update_type in cal_user_update_prefs controls how items for a particular group calendar show up on the personal calendar of a member of that group. It defaults to the corresponding value in cal_user_prefs. The two values supported by the current interface are 'automatically' and 'ask online'. 'automatically' indicates that groups items for that group automatically appear on the user's personal calendar. 'ask online' indicates that there should be an option on the personal calendar to switch to 'automatically'. The current UI supports switching between these two values with a link on the personal calendar next to the link to corresponding group calendar. The 'none' value is to be used to control whether or not the link to the group calendar shows up on the personal calendar. This value is not currently settable but will be a future user preference. The 'email' preference is also not currently used, but the intent is that the user would receive an email for each item added to the group calendar. This email would provide links or an auto-response facility to allow the user to indicate whether or not the particular item should appear on the personal calendar.

Extensions

There are a few extensions that allow the data model to tie into other systems.
  • The users table is used for creation, ownership and preferences information.
  • The groups table is used for ownership and preferences information.
  • There is a cal_alerts table which will eventually be used to tie into the general_alerts (or similar) module.

Conversion of old data

There is a significant amount of data stored in the old calendar model and in the user_vacations table which needs to be displayed in the new calendar package. Thus, as part of the package there is a script, calendar-cvt-intranet.sql which converts this data into corresponding data in the new calendar data model.

VII. Legal Transactions

Integrity Constraints

  • Primary key and foreign key constraints are used appropriately throughout the data model.
  • Check constraints are used to maintain boolean columns.
  • The following columns use check constraints to limit the possible values specified: cal_items.interval_type, cal_alerts.type, cal_user_prefs.size_type, cal_user_prefs.update_type, cal_user_update_prefs.update_type,
  • cal_item_types uses a check constraint to insure that the appropriate id is set for the specified scope.
  • Also for cal_item_types, because group_id and user_id can be null, a unique constraint is specified for scope, type, group_id and user_id rather than a composite primary key.
  • Similarly for cal_user_group_item_map.
  • Ditto for cal_user_update_prefs.

Triggers

  • cal_group_delete_tr
    Removes calendar when a group with a calendar is removed.
  • cal_groups_insert_tr
    When a user is added to a group with a calendar, this trigger adds a row for the default value of showing the group's events on the individual's calendar.
  • cal_groups_add_tr
    When a calendar is added for a group, this trigger adds a row for the default value of showing the group's events on the individual's calendar.
  • cal_groups_remove_tr
    When a user is removed for a group, this trigger removes the corresponding row in cal_user_update_prefs.
  • cal_user_add_tr
    When a user is added, this trigger adds a row in cal_user_prefs.
  • cal_user_remove_tr
    When a user is removed, this trigger removes the corresponding row in cal_user_prefs.

API transactions

  • cal_insert_item (Tcl function)
    Depending on the exact arguments given, this function will perform one of the following transactions:
    • Call the cal_insert_item_with_ids PL/SQL function followed by an insert into cal_item_db_row_map.
    • Call the cal_ins_item_on_id_table PL/SQL function.
    • Call the cal_insert_item_with_ids function alone.
    • Call the cal_insert_item PL/SQL function.
  • cal_insert_item_with_ids
    This function will perform an insert of the parent item into cal_items followed by an insert of the child item followed by an insert into the cal_user_group_item_map.
  • cal_ins_item_on_id_table
    This function calls the cal_insert_item PL/SQL function followed by an insert into cal_item_db_row_map.
  • cal_insert_item (PL/SQL function)
    This function calls cal_insert_item_with_ids.
  • cal_delete_mapped_instances
    This function calls cal_delete_parent_item for each mapped instance.
  • cal_delete_item
    This function deletes from cal_item_type_map, then from cal_item_db_row_map, then cal_items.
  • cal_delete_parent_item
    This function deletes from cal_user_group_item_map, then from cal_item_type_map, then cal_item_db_row_map, cal_alerts, and finally cal_items.
  • cal_insert_repeating_item
    This function repeatedly calls cal_insert_item.

User transactions

These transactions occur because of user actions on the calendar UI.
  • Add item
    When an item is added, cal_insert_item is called, and the resulting transaction is an insert of the parent item into cal_items followed by an insert of the child item followed by an insert into the cal_user_group_item_map. This is followed by a delete and then insert into cal_item_type_map.

    If a recurring item is added, an additional transaction takes place. This is an update of cal_items to set the recurrence information on the parent item followed by a call to the cal_insert_recurrences PL/SQL procedure, which will insert zero or more items in cal_items.

  • Edit item
    • All instances or non-recurring item
      This does an update of the parent item in cal_items followed by an update of the child items in cal_items followed by an update of cal_items to set times followed by a delete and then insert into cal_item_type_map.
    • Current instance
      This conditionally does an update of cal_items to update the time, followed conditionally by another update to set other changed columns followed by a delete and then insert into cal_item_type_map.
  • Delete item
    • All instances
      Delete from cal_user_group_item_map, then cal_item_type_map, then cal_item_db_row_map, then cal_alerts, then all the child items from cal_items, and finally the parent item.
    • Current instance
      Delete from cal_item_type_map, then cal_item_db_row_map, then cal_items.
  • Add attachment
    This does an insert into fs_files, followed by an insert into fs_versions followed by four calls to ad_general_permissions.grant_permission_to_user to set read, write, comment, and administer permissions, followed by a call to ad_general_permissions.grant_permission_to_all_users to set read permission, possibly followed by two calls to grant_permission_to_group to set read and comment permissions, followed by four calls to grant_permission_to_role to set read, comment, write and administer privileges, possibly followed by additional calls to grant_permission_to_role.
  • Change view: this does an update of cal_user_prefs.
  • Toggle compression: this does an update of cal_user_prefs.
  • Toggle group calendar: this does an update of cal_user_update_prefs.
  • Manage group calendars
    Creating a group calendar does an insert into cal_groups. Deleting a group calendar does a delete from cal_alerts, followed by a delete from cal_user_update_prefs, then cal_user_item_filter, then cal_user_group_item_map, and finally from cal_groups.

calendar/admin transactions

  • Add type: insert into cal_item_types.
  • Edit type: update cal_item_types.
  • Delete type
    If no items with that type, delete from cal_item_types, otherwise update cal_item_type_map.

admin/calendar transactions

  • Update group types
    Delete from cal_groups_types, possibly followed by insert into cal_group_types, followed by insert into cal_groups.
  • Create group calendar: insert into cal_groups.
  • Delete group calendar: delete from cal_alerts, followed by delete from cal_user_update_prefs, then cal_user_item_filter, then cal_user_group_item_map, and finally from cal_groups.

VIII. API

Public

These are the functions that are available for other packages and modules to use to manipulate calendar items.

Displays and Portals

These are the functions for displaying the various calendar views. They are used internally by the calendar package and also by the portals package.

Internal

These are the key functions used internally by the calendar package.

IX. User Interface

The user's UI

Views
The calendar package provides the same views that other calendar applications such as Yahoo and Excite do plus some additional features. Considerable effort has been put into optimizing the performance of the views because of the potentially large number of items that are being dealt with.

The day view handles overlapping items. In addition, it supports "compressing" the view so that only hours with items are shown. This is particularly useful when the view is inside a portal.

The list view provides a way of viewing items that the other calendar applications don't have. The upcoming and new items views are both useful for embedding in portals. The sortable columns for these views provide considerable flexibility.

Navigation
The mini calendar makes it easy to get to any particular view of any date quickly. Because the configuration changes based on the view, it's very flexible.
Items
Items appear slightly differently depending on the view. This allows a trade-off between the space available and the amount of information presented.

The form for setting the parameters of recurring events has gone through several revisions to improve usability. I believe the current form is significantly better than what either Yahoo or Excite provide. Color is used to distinguish the different types of intervals that can be set.

Group calendars
On the personal calendar, under the heading Your Calendars: there are links to the group calendars for the groups that the user is a member of. Next to these links are toggles to show or hide the group items for that group. Below the group calendars, Personal is listed.

Group calendars are distinguished from personal calendars in three minor ways but are otherwise identical:

  1. The page heading reads "ArsDigita Calendar for <group_name>" rather than "...for <user_name>".
  2. The link under Your Calendars: for the group calendar is turned off while Personal becomes a link to the personal calendar.
  3. The column for hiding/showing items is removed.
Types for Your Calendar
There is a separate subsystem for adding new types for your calendar. It provides a series of forms allowing the user to add new types, edit the names of existing types, and delete types.

The sub-site administrator's UI

Sub-admins have access to controls for administering calendars for the groups for which they are members.
Manage group calendars
Under this link are two subsections: "Your Groups' Calendars" and "Create Group Calendar". The first lists user groups you manage that have calendars. Links are provided to the admin page for the groups, to delete the group calendar, and to add types for the group.

The second lists those groups you manage which don't yet have calendars, with checkboxes to select those groups you want to create calendars for.

Types for <group_name>
This link goes to the same subsystem as described above, but sets types for this particular group.

The site-wide administrator's UI

Administrators have access to site-wide controls for manipulating group calendars.
Group Types With Calendars
This is a multiple select box. Selecting one or more types and clicking update will cause group calendars to be added for all groups of the selected types. This dialog needs to be made clearer on the exact behavior implemented. For example, deselecting a group type does not remove the calendars for all groups of this type but this is not obvious from the interface.
Groups With Calendars
This section lists all groups that have calendars, with a link to delete the corresponding calendar. It also provides a link to a page where multiple group calendars can be added at once. However, this secondary page is very cumbersome to use when there are many groups and should probably be redesigned.

X. Configuration/Parameters

FileAttachmentsFolder
Specifies the default name of the folder used to store calendar item attachments. If this parameter is not set, it defaults to "CalendarAttachments".

XI. Acceptance Tests

Acceptance Tests: acceptance.txt

The tests are split into two main sections. The first section tests the functioning of the standalone calendar. The second section tests those modules and other packages which have been integrated with the calendar.

XII. Future Improvements/Areas of Likely Change

There are a number of enhancements that should be made to the calendar package. Most of these have already been detailed in the competitive analysis section and I will just list those here. More detail will be given for those enhancements not already covered.
  • Reminders/alarms.
  • Synchronization.
  • Safeguards against excessive recurrences. Because rows are added for each instance of a recurring item, additional code needs to be added to prevent adding excessive recurrences which could seriously impact performance. A complete solution to this problem will almost certainly be related to supporting indefinitely recurring items.
  • Indefinitely recurring items.
  • User preferences.
  • Sharing.
  • Search.
  • Links. The UI needs to be completed to support user specification of links and setting whether or not the item's title redirects to the specified link.
  • Pre-defined public types.
  • Filtering. Given that every item can have any number of types specified, it would be useful to allow the user to set up calendar views that filter on the types of items.
  • Further integration. There are several other modules in the ACS that could benefit from integration with the calendar package, such the address book (for birthdays) and the bootcamp module.
  • Location information. It would be useful to be able to link items with location information that is either entered by the user or stored in other parts of the ACS.
  • Holidays.
  • Find free times.
  • Invitations.
  • ToDos.
  • DPS. Someone needs to investigate the possibility of rewriting the calendar package to use the Dynamic Publishing System.
  • General permissions. The current ad hoc security measures should be replaced with a complete solution based on general permissions.

XIII. Authors

System owner: W. Scott Meeks

System creators:

Documentation author: W. Scott Meeks

smeeks@arsdigita.com
spacer