Difference between revisions of "Moving off Google Apps"

From Wiki | LUG@UCLA
Jump to: navigation, search
 
(14 intermediate revisions by 3 users not shown)
Line 13: Line 13:
 
* lists: Mailman
 
* lists: Mailman
 
* storage format: Maildir
 
* storage format: Maildir
* storage redundancy: Tahoe-LAFS
+
* storage backup: cron + duplicity to LUG Lounge, cron + duplicity to VTLUUG, etc.
* storage backup: duplicity to NFS share, duplicity to VTLUUG, etc.
+
* user directory: LDAP
+
* user authentication: Kerberos
+
  
 
=== Mail transfer ===
 
=== Mail transfer ===
  
 
* set up two MTA servers (''at most'' one in the lounge), each with its own MDA and MSS.
 
* set up two MTA servers (''at most'' one in the lounge), each with its own MDA and MSS.
* configure BIND to broadcast two mx records (multihomed A record?)
+
* configure BIND to broadcast two mx records (one primary, one secondary)
 +
* use an SPF record in DNS configuration
 +
* tutorial - exim on debian: https://wiki.debian.org/Exim
 +
* tutorial - dkim with exim: http://atmail.com/kb/2008/installing-dkim-for-outbound-messages/
  
 
=== Mail delivery ===
 
=== Mail delivery ===
  
* configure one MDA for each MTA server.
+
* configure one MDA for each MTA server
* MDA shall deliver to a Maildir located under a mounted Tahoe-LAFS share.
+
* MDA shall deliver to a Maildir (to be backed up)
  
 
==== Online storage ====
 
==== Online storage ====
  
* Make the MDA store the Maildir under the Tahoe-LAFS $BASEDIR directory on the mail server so it can be accessed from multiple different MSSs and MUAs.
+
* Make the MDA store the Maildir on:
* Set up Tahoe-LAFS "dumb" storage servers at the mail site and the lounge. Configure additional storage servers wherever possible (e.g. members can volunteer their server). Tahoe-LAFS storage servers contain only encrypted data, so it doesn't matter who volunteers their space. Only the Tahoe-LAFS '''gateways''' (MSS and MUA servers) can decrypt the mails and securely hand them to authenticated/authorized users.
+
*# the bare filesystem, or
 +
*# under a mounted clustered directory (e.g. glusterfs)
  
 
==== Offline storage ====
 
==== Offline storage ====
  
Occasionally copy the Maildir directory out of the Tahoe-LAFS share since we don't actually trust Tahoe-LAFS. We respect people's privacy, so don't just rsync it out to a 3rd party. Easiest solution would be to use Duplicity to automatically perform encrypted, incremental backups to the 3rd party.
+
Occasionally copy the Maildir directory out of the clustered share since we don't actually trust online solutions. We respect people's privacy, so don't just rsync it out to a 3rd party. Easiest solution would be to use Duplicity to automatically perform encrypted, incremental backups to the 3rd party.
  
 
=== Access ===
 
=== Access ===
  
 
* MSS will provide POP3 and IMAPS access
 
* MSS will provide POP3 and IMAPS access
* MSS uses LDAP for authentication/authorization ([http://wiki2.dovecot.org/HowTo/DovecotOpenLdap Dovecot + LDAP tutorial]).
+
* MSS authentication/authorization:
* each MSS should utilize separate replicas of the LDAP directory, so configure periodic synchronization of LDAP servers.
+
** use LDAP+Kerberos for lookup/authentication/authorization '''(for linux.ucla.edu emails only)'''
 +
*** [http://wiki2.dovecot.org/HowTo/DovecotOpenLdap Dovecot + LDAP tutorial]
 +
** manually configure MSS credentials in /etc/dovecot/users.conf '''(for other domains (e.g. acm.ucla.edu))'''
 +
* configure multiple LDAP+Kerberos instances for each MSS.
 +
* periodically and automatically replicate LDAP directory and Kerberos principals across all servers.
 +
** [http://www.openldap.org/doc/admin24/replication.html LDAP replication tutorial]
 +
** [http://www.tldp.org/HOWTO/Kerberos-Infrastructure-HOWTO/server-replication.html Kerberos replication tutorial]
  
 
ports:
 
ports:
Line 53: Line 60:
 
** fetch all mails using fetchmail, dump into Maildir.
 
** fetch all mails using fetchmail, dump into Maildir.
 
** delete all my personal mails that got pulled in.
 
** delete all my personal mails that got pulled in.
 +
**
 
* How to migrate users of @linux.ucla.edu emails to the internal system (e.g. login access to POP3/IMAP/Roundcube)?  
 
* How to migrate users of @linux.ucla.edu emails to the internal system (e.g. login access to POP3/IMAP/Roundcube)?  
 
** look for a way to export a list of users from Google Apps.
 
** look for a way to export a list of users from Google Apps.

Latest revision as of 23:27, 25 February 2014

LUG@UCLA plans to move all mail services (including lists) off Google apps. This is a long term project, but the ETA is before Summer 2014.

Design[edit]

KISS. Try to use the least amount of components, and don't overcomplicate the configuration. For example, don't use Maildrop if Dovecot already has an MDA/LDA. Don't use the high-performance sdbox format if Maildir is well supported and tested.

Overview[edit]

  • MTA: Exim
  • MDA/LDA: Dovecot LDA
  • MSS: Dovecot
  • MUA: Roundcube
  • lists: Mailman
  • storage format: Maildir
  • storage backup: cron + duplicity to LUG Lounge, cron + duplicity to VTLUUG, etc.

Mail transfer[edit]

Mail delivery[edit]

  • configure one MDA for each MTA server
  • MDA shall deliver to a Maildir (to be backed up)

Online storage[edit]

  • Make the MDA store the Maildir on:
    1. the bare filesystem, or
    2. under a mounted clustered directory (e.g. glusterfs)

Offline storage[edit]

Occasionally copy the Maildir directory out of the clustered share since we don't actually trust online solutions. We respect people's privacy, so don't just rsync it out to a 3rd party. Easiest solution would be to use Duplicity to automatically perform encrypted, incremental backups to the 3rd party.

Access[edit]

  • MSS will provide POP3 and IMAPS access
  • MSS authentication/authorization:
    • use LDAP+Kerberos for lookup/authentication/authorization (for linux.ucla.edu emails only)
    • manually configure MSS credentials in /etc/dovecot/users.conf (for other domains (e.g. acm.ucla.edu))
  • configure multiple LDAP+Kerberos instances for each MSS.
  • periodically and automatically replicate LDAP directory and Kerberos principals across all servers.

ports:

  • POP3 over SSL: 995 tcp/udp
  • IMAP over SSL: 993 tcp/udp
  • HTTP over SSL: 443 tcp

Transitional details[edit]

  • How to migrate emails from Google Groups to Maildir readable by Mailman?
    • fetch all mails using fetchmail, dump into Maildir.
    • delete all my personal mails that got pulled in.
  • How to migrate users of @linux.ucla.edu emails to the internal system (e.g. login access to POP3/IMAP/Roundcube)?
    • look for a way to export a list of users from Google Apps.
    • make use of LDAP/Kerberos to authenticate.
  • How to migrate subscribers to the GNU Mailman mailing list?
    • export a CSV list of users from the Google Groups members page.
    • grep/sed the list for the following information: Full Name, subscribed email,

For users with LUG emails[edit]

For subscribers to the mailing lists[edit]

External links[edit]