At 6.00am on Wednesday 25th we began the upgrade to our email server. The upgrade involved the developers of the software in Australia, our hosting company in Carlow and ourselves in Dublin. The email system on dublin.ie has been in place since 2003 so we’ve amassed quite a few accounts in that time. The upgrade in question is also significant as the ‘Simple’ and ‘Advanced’ versions have been replaced with a unified Gmail-esque interface. At approx 8am the developers indicated they were having difficulty with the migration script and would need to adapt it to our specific installation. The changes required would take additional time and it was those changes that pushed us beyond the original estimated recovery time of 9am. We finished the majority of the upgrade at 11.15am with the mail system going live and calendar entries taking a bit longer (they finished at approx 2pm).
This morning (26th), we received a call from a user about older emails which were missing from his account. We contacted our hosting company and they discovered that the mail storage locations were out of sync. To reduce the possibility of additional problems, it was decided to close off access temporarily. Once the two mail stores were synchronized, which took approximately an hour, we turned it on for access once again.
Following on from the issue this morning, at approx 3.45pm, the server experienced a spike in web traffic which brought it to a standstill. The new software is more demanding of our server so we’ll be upgrading it’s capacity out of hours very soon.
We understand how frustrated and disappointed a lot of you will have been with the service over the past few days. The upgrade has been a huge shift in how we do things in the email system but we think it will be beneficial in the long run. Rest assured that we are doing the best we can and are monitoring it 24/7.