Integrating Google Apps: Provisioning and Management of Email

Preamble

The Open Campus uses Google Apps to provide email service for all students. There are over 4500 active email accounts. For the month of November 2008, an average of 1200 accounts are accessed per day.

History: The DEC MIS Implementation

The DEC course delivery model shifted to more online interaction and the need to provide an email solution for students grew. The Google Apps for Hosted Domains was one of the solutions considered. It is free for educational institutions and at the time offered 2GB mailboxes. The hardware and software of service would be handled by Google. The DEC had to handle the user management.

Creating Accounts

In the testing phase accounts were created individually using the Google Apps admin web interface. The admin interface also provided a bulk account creation tool. Account information would be uploaded in a csv file and the user accounts would be created. Any issues with the data caused the account creation to stop. The successfully created account removed from the file, the error was corrected and the file was re-uploaded.

The required information was extracted from the MIS database and saved as an spreadsheet. This was then broken up into csv files to make the error management easier.

This was a tedious task that required fortitude and patience. Clearly a better way was required.

First Steps in Provisioning

Sada Systems released an open source bulk provisioning toolkit. We first tested it with the Excel file generated for our original method. The toolkit used python to create accounts at a rate of almost 10 per second. It was amazingly fast. The actual magic was in the error detection before it connected to google. The toolkit highlighted all errors allowing correction to be made and account creation to be a much nicer affair.

Eventually we dropped the need for the spreadsheet and connected the toolkit to the MIS, using a mySQL view to provide the data. All errors would then be corrected at the source and thus the potential for data disparity issues removed.

Creating many accounts now required a single click in most instances. Individual accounts was still done manually using the admin interface. All management tasks, including password resets, checking if accounts were created and active, etc were also done in the admin interface.

Current: The Open Campus OCMS Implementation

Bulk Account Creation

We still required the provisioning toolkit to do the original bulk account creation as the gdata zend library could never match it in terms of speed. The OCMS was connected much like the MIS to the toolkit and one click later 8000 accounts were created.

The No Click Way

In the OCMS we aimed for a (almost) zero management solution leveraging the Zend Framework and Google provisioning API. The one click method was still one click too much. And so the student email model was written.

When a student logs in for the first time the student email model goes to work. It looks in the database and generates an email address for the student based on the first and last name. It also checks for and avoids duplicates. The account is then provisioned and the student advised that they should activate the new account. When the account is activated the email model notes the activation.

Individual accounts are now handled by the OCMS. All new students will get email accounts in this way. Separate password resets are also a thing of the past as the students access their email using SSO from the OCMS.

The experience of doing things the hard way helped to mold effective design choices that integrated the provisioning into the OCMS.

Open Campus Management System: The Choice to Build

In today’s educational environment, there is a greater reliance on Information Technology to improve the production, flow and control of information. This is particularly true for the UWI Open Campus since it’s nature as a virtual campus makes it’s reliance on Information Technology extremely important with software becoming one of it’s driving forces to providing it’s services.

What is of particular interest is the decision to build it’s core application, the Open Campus Management System (OCMS),  in house. Here we will give some background on why this decision was made.

The Open Campus Distributed Environment

The organisation of the Open Campus is different from a traditional campus in a few respects. The Open Campus is an amalgamation of the previous Office of the Board for Non-Campus Countries & Distance Education (BNNCDE), the School of Continuing Studies (SCS), the UWI Distance Education Centre (UWIDEC), and the Tertiary Level Institutions Unit (TLIU).

The Open Campus is a virtual campus providing many of it’s programmes online through distance education. This is supported by a number of software tools including Learning Management Systems (LMS) for course delivery and Student Management Systems for managing student data.  In addition to the virtual campus, there are over 50 site locations of the Open Campus currently serving over 20 countries in the English-speaking Caribbean.

This combination of online and face-to-face learning modes across a wide geographical region makes the Open Campus’s approach to education unique. When it comes to Student Management, this poses a problem since the current proprietary tools that were in use at the other UWI campuses were not designed for such an environment. The choices were to either heavily modify the proprietary tools to fit the needs of the Open Campus or develop custom software.

Build vs Buy

It has been the custom of educational organisations in the Caribbean to buy proprietary packaged applications. However, there are compelling reasons for building software in house, especially in the case of Open Campus. The distributed nature of Open Campus and it’s mix of virtual and on-site learning means that there are a number of unique needs to be met both in course delivery and student management. Combined with the wide rage of programmes, from vocational to post graduate, that are offered by the campus and it was clear that there was no existing solution that would completely fill the needs of the organisation.

In addition to this, the software packages that are available require a long implementation time which the Open Campus could not afford.  The Open Campus is a quickly evolving organisation which is still in it’s initial stages so many of the processes are not fully implemented. As the organisation evolves, the processes change quickly and sometimes, radically. Any software implementation must be flexible enough to not only adapt to these changes, but do so quickly. This level of flexibility was not available in any of the software packages that were currently used by the other campuses.

The existence of these challenges and decision to build are not new and actually pre-date the formation of the Open Campus. UWIDEC initially saw the issues that come about when trying to implement a large proprietary student management system into it’s operations and had made the decision to implement it’s own Student Management System. The addition of the other units have added a magnitude of complexity to the operations and so have served to reinforced the need for a custom solution. This saw the evolution of UWIDEC’s simple student management system to what is now the Open Campus Management System.

Now that we know why the choice was made to build a custom solution, the question remained as to how this was to be achieved. In the next article, we will explore the advantages of setting up an in house development team and the challenges that come along with it.

ACL resource reflection and role inheritance

The new ACL now uses reflection to build its page resources. Previous versions used the page table to build these resources. Why the change? Because the previous method was error prone and did not reflect (sorry) any changes in the web application quickly enough.

Using reflection to build resources

The new method of updating page resources involves building a list of all actions found using reflection, then creating page resources for each of these actions. The acl_resource table is then updated with any new page resources, so that it mirrors the new web application structure. Optionally, the acl_resource table can also be cleared of all page resources before being updated.

Deploying the ACL

Since the new ACL no longer uses the pages table, the database must be primed for it to work. This is done by using the dev.forceAclReload config option, and only has to be done once to insert page resources into the database. Using dev.forceAclReload removes all page resource rows from the database exclusively, so it will not remove other resource types (such as calendar), and uses reflection to insert new page resource rows. Once primed, new page resource rows can be added to the existing database by using the ‘update page resource tree’ button in the access page.

Introducing multiple role inheritance

While the OCMS ACL had always used the very powerful Zend_Acl class from the Zend Framework, the support for multiple role inheritance was not available due to our ACL’s inability to load the necessary data from database. However, the ACL in R440 now has support for multiple role inheritance, and the new analysis page can determine and display inherited rules both via resource rule inheritance and role parent/ancestor rule inheritance (yes, I know - amazing!).

So how does this work? Well the ACL allows roles to have zero, one or multiple parents, so the role inherits the rules applied to the roles parents, if any.

To show how this works, lets suppose we have the following four roles:

1. staff, which has no parent
2. technician, which has no parent
3. exam-staff, which has one parent (staff)
4. support, which has two parents (staff and technician)

And this resource tree:

• event
  • event/teleconference
  • event/class
  • event/exam

The default rule does not allow any role to access the resource tree root. Due to resource rule inheritance, this rule will also apply to every resource in the resource tree. With this in mind we define these rules (note that we do not define any rule for the ’support’ role):

1. allow staff access to event/class
2. allow technician access to event/teleconference
3. allow exam-staff access to event

When we load these rules into the ACL and use the analysis tool, we get the following results:

Fig. 1 - Access analysis for staff role

The results in Fig. 1 are not surprising; We explicitly set a rule for the staff role and that resource. This demonstrates the most basic form rule definition in access control.

Fig. 2 - Access analysis for technician role

The results in Fig. 2 are equally boring; We explicitly set a rule for the technician role and that resource. Again, a vanilla rule.

Fig. 3 - Access analysis for support role

Now, according to the results in Fig. 3, the support-staff role has effective rules applied to it. However, no rules were explicitly defined for this role at all. The results show that the permissions were inherited from the staff and technician roles. Very tasty, but what if we define a rule in a child role as well? This is what we did for the exam-staff role.

Fig. 4 - Access analysis for exam-staff role

So, in Fig. 4 we see that the rule we defined for the exam-staff role was effective, and was inherited down the resource tree. We also see that the rule inherited from the staff role was also effected, and overrides the exam-staff rule. This is because the staff role was defined with a more localized resource target. If we had also created a rule for the exam-staff which targeted the event/class resource, it would have taken precedence over the staff rule.

Implementation

Now, to implement the roles in the example above, in code, we would define them as shown below:

// these roles have no parent
$acl->addRole( new Zend_Acl_Role( 'staff' ) );
$acl->addRole( new Zend_Acl_Role( 'developer' ) );
 
// role with single parent
$acl->addRole( new Zend_Acl_Role( 'exam-staff' ), 'staff' );
 
// role with multiple parents
$acl->addRole( 
    new Zend_Acl_Role( 'support' ), 
    array( 'staff', 'developer' )
);

Note: I think that roles with parents must be added after the parent roles have been added, or an error will be thrown.

Now, this is all very nice and pretty, but our super-advanced mirror wielding ACL loads its rules from the database. So to define these rules in the database, we need to have the following rows in acl_role and acl_role_parent tables:

TABLE `acl_role` (
   `id` SMALLINT(5) UNSIGNED NOT NULL AUTO_INCREMENT,
   `name` VARCHAR(32) NOT NULL,
   `description` VARCHAR(32) NOT NULL,
)
 
INSERT INTO `acl_role` 
    VALUES (1, 'technician', 'Teleconference Technician');
INSERT INTO `acl_role` 
    VALUES (2, 'staff', 'General Staff');
INSERT INTO `acl_role` 
    VALUES (3, 'exam-staff', 'Examinations Staff');
INSERT INTO `acl_role` 
    VALUES (4, 'support', 'Help Desk');
 
TABLE `acl_role_parent` (
    `id` SMALLINT(5) UNSIGNED NOT NULL AUTO_INCREMENT,
    `role_id` SMALLINT(5) UNSIGNED NOT NULL,
    `parent_role_id` SMALLINT(5) UNSIGNED NOT NULL,
)
 
INSERT INTO `acl_role_parent` VALUES (1, 3, 2);
INSERT INTO `acl_role_parent` VALUES (2, 4, 1);
INSERT INTO `acl_role_parent` VALUES (3, 4, 2);

Note: If caching is enabled, the ACL cache must be cleared to force it to reload its data from the database.

That’s all for the role definition. Now all you have to do is assign rules to the roles for your resource tree and use the analysis tool to see how it affects the permissions for the roles and their child roles. However, since we already know how to implement regular roles and rules, I will stop here.

So, the ACL the system now auto-builds page resources via reflection, and we now have an idea of how role inheritance works. We’ve only touched the possibilities of multiple role inheritance, as there does not seem to be any limit to the number of parents or how many ancestors a role can have.

In addition, ACL still has at least two more dimensions of complexity: Assertions and Permissions. Although our ACL currently has limited support for these features, the existing features supported by our ACL make it capable of handling almost every access control situation in the OCMS.