Difference between revisions of "Moving off Google Apps"

From Wiki | LUG@UCLA
Jump to: navigation, search
 
(23 intermediate revisions by 5 users not shown)
Line 5: Line 5:
 
KISS. Try to use the least amount of components, and don't overcomplicate the configuration. For example, don't use Maildrop if Dovecot already has an MDA/LDA. Don't use the high-performance sdbox format if Maildir is well supported and tested.
 
KISS. Try to use the least amount of components, and don't overcomplicate the configuration. For example, don't use Maildrop if Dovecot already has an MDA/LDA. Don't use the high-performance sdbox format if Maildir is well supported and tested.
  
* MTA: Postfix
+
=== Overview ===
 +
 
 +
* MTA: Exim
 
* MDA/LDA: Dovecot LDA
 
* MDA/LDA: Dovecot LDA
 
* MSS: Dovecot
 
* MSS: Dovecot
Line 11: Line 13:
 
* lists: Mailman
 
* lists: Mailman
 
* storage format: Maildir
 
* storage format: Maildir
* storage redundancy: Tahoe-LAFS
+
* storage backup: cron + duplicity to LUG Lounge, cron + duplicity to VTLUUG, etc.
* storage backup: duplicity to NFS share, duplicity to VTLUUG, etc.
+
  
 
=== Mail transfer ===
 
=== Mail transfer ===
  
* Set up two MTA servers (''at most'' one in the lounge), each with its own MDA and MSS.
+
* set up two MTA servers (''at most'' one in the lounge), each with its own MDA and MSS.
* configure BIND to broadcast two mx records (multihomed A record?)
+
* configure BIND to broadcast two mx records (one primary, one secondary)
*  
+
* use an SPF record in DNS configuration
 +
* tutorial - exim on debian: https://wiki.debian.org/Exim
 +
* tutorial - dkim with exim: http://atmail.com/kb/2008/installing-dkim-for-outbound-messages/
  
ports:
+
=== Mail delivery ===
* POP3 over SSL: 995 tcp/udp
+
* IMAP over SSL: 993 tcp/udp
+
  
=== Online storage ===
+
* configure one MDA for each MTA server
 +
* MDA shall deliver to a Maildir (to be backed up)
  
Use the traditional Maildir format for storing mail. Make the MDA/LDA store it under the Tahoe-LAFS $BASEDIR directory on the mail server so it can be accessed from multiple different MSSs and MUAs. Set up Tahoe-LAFS "dumb" storage servers at the mail site and the lounge. Configure additional storage servers wherever possible (e.g. members can volunteer their server). Tahoe-LAFS storage servers contain only encrypted data, so it doesn't matter who volunteers their space. Only the Tahoe-LAFS gateways (MSS and MUA servers) can decrypt the mail.
+
==== Online storage ====
  
=== Offline storage ===
+
* Make the MDA store the Maildir on:
 +
*# the bare filesystem, or
 +
*# under a mounted clustered directory (e.g. glusterfs)
  
Occasionally copy the Maildir directory out of the Tahoe-LAFS share since we don't actually trust Tahoe-LAFS. We respect people's privacy, so don't just rsync it out to a 3rd party. Easiest solution would be to use Duplicity to automatically perform encrypted, incremental backups to the 3rd party.
+
==== Offline storage ====
 +
 
 +
Occasionally copy the Maildir directory out of the clustered share since we don't actually trust online solutions. We respect people's privacy, so don't just rsync it out to a 3rd party. Easiest solution would be to use Duplicity to automatically perform encrypted, incremental backups to the 3rd party.
 +
 
 +
=== Access ===
 +
 
 +
* MSS will provide POP3 and IMAPS access
 +
* MSS authentication/authorization:
 +
** use LDAP+Kerberos for lookup/authentication/authorization '''(for linux.ucla.edu emails only)'''
 +
*** [http://wiki2.dovecot.org/HowTo/DovecotOpenLdap Dovecot + LDAP tutorial]
 +
** manually configure MSS credentials in /etc/dovecot/users.conf '''(for other domains (e.g. acm.ucla.edu))'''
 +
* configure multiple LDAP+Kerberos instances for each MSS.
 +
* periodically and automatically replicate LDAP directory and Kerberos principals across all servers.
 +
** [http://www.openldap.org/doc/admin24/replication.html LDAP replication tutorial]
 +
** [http://www.tldp.org/HOWTO/Kerberos-Infrastructure-HOWTO/server-replication.html Kerberos replication tutorial]
 +
 
 +
ports:
 +
* POP3 over SSL: 995 tcp/udp
 +
* IMAP over SSL: 993 tcp/udp
 +
* HTTP over SSL: 443 tcp
  
 
=== Transitional details ===
 
=== Transitional details ===
Line 37: Line 60:
 
** fetch all mails using fetchmail, dump into Maildir.
 
** fetch all mails using fetchmail, dump into Maildir.
 
** delete all my personal mails that got pulled in.
 
** delete all my personal mails that got pulled in.
 +
**
 
* How to migrate users of @linux.ucla.edu emails to the internal system (e.g. login access to POP3/IMAP/Roundcube)?  
 
* How to migrate users of @linux.ucla.edu emails to the internal system (e.g. login access to POP3/IMAP/Roundcube)?  
 
** look for a way to export a list of users from Google Apps.
 
** look for a way to export a list of users from Google Apps.
Line 49: Line 73:
  
 
== For subscribers to the mailing lists ==
 
== For subscribers to the mailing lists ==
 +
 +
== External links ==
 +
 +
* http://shearer.org/MTA_Comparison

Latest revision as of 00:27, 26 February 2014

LUG@UCLA plans to move all mail services (including lists) off Google apps. This is a long term project, but the ETA is before Summer 2014.

Design[edit]

KISS. Try to use the least amount of components, and don't overcomplicate the configuration. For example, don't use Maildrop if Dovecot already has an MDA/LDA. Don't use the high-performance sdbox format if Maildir is well supported and tested.

Overview[edit]

  • MTA: Exim
  • MDA/LDA: Dovecot LDA
  • MSS: Dovecot
  • MUA: Roundcube
  • lists: Mailman
  • storage format: Maildir
  • storage backup: cron + duplicity to LUG Lounge, cron + duplicity to VTLUUG, etc.

Mail transfer[edit]

Mail delivery[edit]

  • configure one MDA for each MTA server
  • MDA shall deliver to a Maildir (to be backed up)

Online storage[edit]

  • Make the MDA store the Maildir on:
    1. the bare filesystem, or
    2. under a mounted clustered directory (e.g. glusterfs)

Offline storage[edit]

Occasionally copy the Maildir directory out of the clustered share since we don't actually trust online solutions. We respect people's privacy, so don't just rsync it out to a 3rd party. Easiest solution would be to use Duplicity to automatically perform encrypted, incremental backups to the 3rd party.

Access[edit]

  • MSS will provide POP3 and IMAPS access
  • MSS authentication/authorization:
    • use LDAP+Kerberos for lookup/authentication/authorization (for linux.ucla.edu emails only)
    • manually configure MSS credentials in /etc/dovecot/users.conf (for other domains (e.g. acm.ucla.edu))
  • configure multiple LDAP+Kerberos instances for each MSS.
  • periodically and automatically replicate LDAP directory and Kerberos principals across all servers.

ports:

  • POP3 over SSL: 995 tcp/udp
  • IMAP over SSL: 993 tcp/udp
  • HTTP over SSL: 443 tcp

Transitional details[edit]

  • How to migrate emails from Google Groups to Maildir readable by Mailman?
    • fetch all mails using fetchmail, dump into Maildir.
    • delete all my personal mails that got pulled in.
  • How to migrate users of @linux.ucla.edu emails to the internal system (e.g. login access to POP3/IMAP/Roundcube)?
    • look for a way to export a list of users from Google Apps.
    • make use of LDAP/Kerberos to authenticate.
  • How to migrate subscribers to the GNU Mailman mailing list?
    • export a CSV list of users from the Google Groups members page.
    • grep/sed the list for the following information: Full Name, subscribed email,

For users with LUG emails[edit]

For subscribers to the mailing lists[edit]

External links[edit]